Validation of a questionnaire about the use of interactive response controls on Higher Education

How o ci e his a icle Mingo ance-Es ada AC, G anda-Ve a J, Rojas-Ruiz G, Alemany-A ebola I. Valida ion o a ques ionnai e on he use o In e ac i e Response Sys em in Highe Educa ion. Re . La ino-Am. En e magem. 2021;29:e3418. [Access daymon h yea ]; A ailable in: URL . DOI: h p://dx.doi.o g/10.1590/1518-8345.3374.3418. 1 Uni e si y o G anada, Depa men o Didac ics and School O ganiza ion, Melilla, ES, Spain. 2 Uni e si y o G anada, Depa men o Didac ics o Co po al Exp ession, Melilla, ES, Spain. 3 Uni e si y o G anada, Depa men o De elopmen al and Educa ional Psychology, Melilla, ES, Spain. Valida ion o a ques ionnai e on he use o In e ac i e Response Sys em in Highe Educa ion Objec i e: his s udy aims o design and alida e a ques ionnai e o measu e he s uden s’ pe cep ion o he use o IRS as a echnopedagogical esou ce in he class oom. Me hod: a 24 i ems ques ionnai e (In e ac i e Response Sys em o he Imp o emen o he Teaching-Lea ning P ocess) was designed ad hoc o his esea ch and applied o 142 uni e si y s uden s. Resul s: bo h he explo a o y and con i ma o y ac o ial analysis yielded 3 dimensions: class oom en i onmen , eaching-lea ning p ocesses and lea ning assessmen . The esul s ob ained bo h in eliabili y (C onbach’s alpha= 0.955) and in he Con i ma o y Fac o Analysis (χ2/d =1.944, CFI=0.97; GFI=0.78; RMR=0.077; RMSEA=0.08) e eal highly sa is ac o y indices. Conclusion: s a is ical analyses con i m ha his ins umen is a alid, eliable, and easy- o-apply ool o p o esso s o e alua e he s uden pe cep ion o s uden - cen ed lea ning. Desc ip o s: Educa ion; Highe Educa ion; Lea ning; Valida ion S udy; Su eys and Ques ionnai es; Technology Educa ion. O iginal A icle Re . La ino-Am. En e magem 2021;29:e3418 DOI: 10.1590/1518-8345.3374.3418 www.ee p.usp.b / lae Ángel Cus odio Mingo ance-Es ada1 h ps://o cid.o g/0000-0003-4478-3011 Juan G anda-Ve a2 h ps://o cid.o g/0000-0001-6888-7785 Glo ia Rojas-Ruiz1 h ps://o cid.o g/0000-0001-8541-383X Inmaculada Alemany-A ebola3 h ps://o cid.o g/0000-0002-4127-3502 www.ee p.usp.b / lae 2 Re . La ino-Am. En e magem 2021;29:e3418. In oduc ion Highe Educa ion mus es ablish a space o he exchange o p ac ical expe iences ha p omo es knowledge and esea ch wi hin he common amewo k o he Eu opean and Ibe o-Ame ican Highe Educa ion A ea (1-2) . The implemen a ion o s uden -cen e ed lea ning is o ien ed o es ablish a model ha e ec i ely in eg a es echnology wi h knowledge o didac ic media ion, e ol ing owa ds he Technological Pedagogical and Con en Knowledge Model (TPACK). This sys em combines disciplina y, pedagogical and echnological knowledge, bu always conside ing he con ex in which i in e enes (3-4) , and inc easing he in e ac ion be ween p o esso s and s uden s wi hin a c i ical dialogical app oach(2). Some au ho s conside necessa y o es ablish didac ic knowledge h ough he ela ionship be ween di e en ypes o knowledge (coming om he own discipline, gene al pedagogy and s uden s) and he p o esso ’s biog aphy (5) . Thus, du ing he nu sing ini ial aining, bo h he pedagogical aspec s, including he implemen a ion o In e ac i e Response Sys em (he eina e IRS) wi h emo e answe ing de ices o moni o s uden s’ p og ess, and he in ol emen o expe p o esso s lead o high- quali y eaching, among o he issues(6). In his ega d, encou aging p o esso s o in eg a e echnology in o he class oom is c ucial, as highligh ed in he Ho izon Repo (7), since i will signi ican ly impac on educa ion in he coming yea s. To do his, uni e si y p o esso s mus use he echnological ools hey a e amilia wi h, as well as access new echnological esou ces o imp o e eaching p ocesses(8). Simila ly, echnological changes in uni e si y p o esso s ollow a endency and a e no adical; hey in oduce hose ha a e consis en wi h hei eaching p ac ices in o he lea ning ac i i ies hey no mally ca y ou (9) . In o de o ace hese echnological challenges, an in e sion o lea ning p ocess is equi ed; s uden s should be p o ided wi h ma e ials in a ious o ma s, so ha hey can ca y ou p elimina y wo k be o e a i ing a he class oom, inco po a ing IRS o e i y he imp o emen s in he s uden cen e ed-lea ning p ocess (10-12) . IRS has been al eady in eg a es in some uni e si y classes (13) . Thus, he a ailable li e a u e abou Highe Educa ion on he use o his echnology in ecen yea s ocuses on he ields o Science, Technology, Enginee ing and Ma hema ics, Sociology, Humani ies, Heal h (Medicine and Nu sing), Business Adminis a ion and English language(13-14). The e iewed s udies indica e ha in eg a ing IRS in uni e si y class ooms imp o es h ee main a eas: he class oom en i onmen , he eaching-lea ning p ocesses and lea ning assessmen (15-20) . Thus, de ining he possibili ies and limi a ions o his ool is inc easingly impo an o he imp o emen o he quali y o Highe Educa ion(6). Conce ning he class oom en i onmen ac o , IRS inc eases a endance(21) and s uden pa icipa ion(22-25), esul ing in a highe le el o in ol emen du ing classes in compa ison o he adi ional me hodology (23,26-27) . Wi hin he lea ning ac o , some s udies es ablish ha equen and posi i e in e ac ion makes classes mo e dynamic when using IRS (13,28-29) , p omo ing ac i e lea ning (30-32) . In addi ion, a en ion (33-34) , concen a ion (35) and memo y (36-37) a e encou aged du ing he lea ning p ocess. Ex ensi e esea ch sugges s ha a be e pe o mance is he esul o he use o IRS, as some s udies indica e (31-32,38-41) , al hough o he s udies do no ind such an e ec (42) . In ela ion o he assessmen ac o , he indings o he li e a u e suppo he capaci y o IRS as a ool o assessmen and eedback (43-44) . I is conside ed ha bo h s uden s and p o esso s bene i om he eedback hey ecei e wi h he use o his educa ional echnology (20,27) . All o his leads o a key lea ning p ocess o he in e ac ion o knowledge and know-how (45) . The aim o his s udy is o design and alida e his s udy aims o design and alida e a ques ionnai e o measu e he s uden s’ pe cep ion o he use o IRS as a echnopedagogical esou ce in he class oom. Me hod The design was ansec ional and desc ip i e, as he da a we e collec ed in a single ime in o de o desc ibe he phenomenon and analyze i a a ce ain ime. The esea ch was ca ied ou a Melilla Campus o he Uni e si y o G anada (Spain), loca ed in No h A ica, whose s uden s a end o he Heal h Sciences, Social and Legal Sciences, and Educa ion and Spo s Sciences schools. Fo his pu pose, an in en ional non-p obabilis ic sample was ca ied ou . The selec ion c i e ia ha e been: i s ly, p o esso s who use echnology in hei class ooms, speci ically in e ac i e esponse de ices. And secondly, he willingness o he s uden s o pa icipa e in his s udy. The e o e, he sample comp ises 142 s uden s: 110 women (77.5%) and 32 men (22.5%). In ela ion o he academic yea , 17 s uden s a e in i s yea (12%), 95 in second yea (66.9%) and 30 in hi d yea (21.1%). To ca y ou his s udy, an ad hoc ques ionnai e was designed o his esea ch, “In e ac i e Response Sys em o he Imp o emen o he Teaching-Lea ning P ocess (IRS-ITLP)”. In ela ion o he i ems o he IRS-ITLP, hey we e w i en a e an ex ensi e bibliog aphic e iew on he h ee ac o s highligh ed abo e (21,46-47) . Despi e o he absence o expe s on his ield, his p ocess p o ides alidi y o he ques ionnai e i ems. www.ee p.usp.b / lae 3 Mingo ance-Es ada AC, G anda-Ve a J, Rojas-Ruiz G, Alemany-A ebola I. We s a ed wi h 65 i ems g ouped in o ca ego ies (48) . A e he analysis o he i ems, we selec ed 35 i ems ha we e g ouped in o he h ee dimensions: lea ning en i onmen , p ocess and assessmen . Rega ding he ques ionnai e esponse o ma , a Like ype scale was used, wi h 5 esponse al e na i es, anging om 1, o ally disag ee, o 5, o ally ag ee. The esea ch was o ien ed o di e en deg ees o he Uni e si y o G anada (Spain), all o hem being subjec s conce ning he basic o ma ion o he s uden s. Fo his pu pose, he collabo a ion o he eaching s a was eques ed o pa icipa e as olun ee in his p ojec and o in eg a e IRS in hei classes. The use o he IRS in hese basic aining subjec s was ca ied ou h oughou he semes e o he 2016-17 academic yea , be o e, du ing and a he end o he classes. A he end o he semes e , he IRS-ITLP ques ionnai e was applied he las week o he semes e , wi h a du a ion o app oxima ely 15 minu es, o ind ou he pe cep ion o he expe ience. S uden s we e asked o ag ee o pa icipa e in his expe ience olun a ily and anonymously, ollowing he ules o he Commi ee on Publica ion E hics (COPE). The s a is ical so wa e SPSS e sion 20.0 has been used o he s a is ical p ocessing o he da a. To know he eliabili y o each g oup o i ems, C onbach’s alpha was used, and o he alidi y o he ques ionnai e, an Explo a o y Fac o Analysis was ca ied. Fo he Con i ma o y Fac o Analysis, he p og am LISREL 8.8 was used. Resul s Fi s ly, he eliabili y o he IRS-ITLP ques ionnai e consis ing o 35 elemen s was analyzed using C onbach’s alpha in e nal consis ency coe icien , which was 0.965. Al hough his index was high, we p oceeded o elimina e hose i ems whose i em- o al co ela ion was in e io o 0.20. Finally, he ques ionnai e was made up o 24 i ems wi h a α=0.955, showing homogenei y indexes anging om 0.42 o 0.85. Subsequen ly, he means, s anda d de ia ions, asymme y, and i em- o al co ela ions o each o he i ems we e ob ained. As can be seen in Table 1, he asymme y is nega i e in all i ems, which shows a g ea e concen a ion o esponses co esponding o he high sco es in hose i ems. Table 1 - Desc ip i e alues o he i ems in he IRS-ITLP ques ionnai e. G anada, Spain, 2017 Nº I ems M*SD† Asymme y Co el. i em- o al 1 I am mo e ocused du ing he classes since he implemen a ion o IRS 3.40 1.15 -0.477 0.689 2Thanks o IRS, I measu e i I am ollowing co ec ly he con en s o he subjec du ing he classes 3.81 1.01 -0.658 0.619 5 Du ing my expe ience wi h he IRS I ha e a good ime lea ning 3.52 1.18 -0.559 0.566 9IRS is used o ind ou he ini ial knowledge o he s uden s 3.54 1.258 -0.595 0.488 10 The use o IRS is ca ied ou by expe ienced p o esso s o p o ide good eedback 3.97 0.891 -0.616 0.610 14 The use o IRS helps me o de elop my comp ehension on he con en s I am wo king on 3.59 1.162 -0.666 0.639 15 The use o he IRSs makes he classes enjoyable and dynamic 3.82 1.119 -0.877 0.656 18 The use o IRS imp o es my lea ning pe o mance 3.76 1.129 -0.653 0.762 19 The con inuous use o he IRS inc eases my class a endance 3.70 1.266 -0.793 0.579 20 The use o IRS allows you o know and compa e you colleagues’ answe s wi h you own answe s 3.42 1.234 -0.646 0.441 21 The use o he IRS allows o co ec mis akes o misunde s andings abou he subjec con en s du ing he classes 3.71 1.121 -1.032 0.558 22 I am mo e in e es ed in classes when using IRS 3.65 1.066 -0.698 0.768 24 I like he use o IRS as an a endance con ol 3.65 1.005 -0.605 0.699 29 The use o IRS imp o es mo i a ion du ing classes 3.83 1.111 -0.825 0.753 30 The use o IRS allows ac i e discussion o misconcep ions o build knowledge 3.80 1.168 -0.771 0.792 35 The use o he IRS e alua es my comp ehensi e knowledge o he con en s in each o he opics co e ed du ing he classes 3.99 0.971 -1.211 0.788 36 The use o IRS p omo es egula s udy o he subjec o be be e p epa ed o classes 3.64 1.094 -0.762 0.715 42 The use o IRS allows you o be mo e con iden when asking ques ions du ing classes 3.76 1.254 -0.739 0.787 46 The use o he IRS is done a he end o he classes o e iew he con en s explained du ing he session 3.80 1.100 -0.736 0.772 47 The use o IRS makes he classes mo e pleasan and in e ac i e compa ed o adi ional classes 3.86 1.082 -0.804 0.769 48 The use o IRS imp o es you pa icipa ion in classes behind anonymi y 3.87 1.104 -0.773 0.712 53 The answe s p o ided h ough he IRS inc ease my con idence in he classes a e e i ying ha I answe ed co ec ly 3.44 1.164 -0.420 0.700 59 IRS p o ides aluable in o ma ion o imp o e you lea ning p ocess 4.09 1.017 -1.007 0.687 65 The use o IRS imp o es he unde s anding o he con en s explained in class 3.87 1.160 -0.911 0.727 *M = Mean; †SD = S anda d de ia ion www.ee p.usp.b / lae 4 Re . La ino-Am. En e magem 2021;29:e3418. Since any s udy abou he ac o analysis o he IRS- ITLP ques ionnai e had p e iously been published, be o e pe o ming a Con i ma o y Fac o Analysis i was con enien o ca y ou an Explo a o y Fac o Analysis (EFA) o explo e how he i ems a e g ouped in o ac o s. To ensu e ha he da a i a ac o analysis model, he da a we e subjec ed o he Kaise -Meye -Olkin es (KMO= 0.941) and o he Ba le ’s Tes o Sphe ici y (c2= 2446.206; d = 300; p<0.001). The alues indica e ha a ac o analysis is a sui able echnique o s uc u e he in o ma ion con ained in he ma ix. The EFA e eals he exis ence o 3 ac o s ha explain 61.61% o he o al a iance, being his p opo ion accep able. In addi ion, he i em communali ies a e abo e h 2 =0.40, anging om 0.421 “The con inuous use o IRS inc eases my class a endance” o 0.791 “The use o IRS allows you o know and compa e you colleagues’ answe s wi h you own answe s”. Table 2 shows he ac o s, i ems, ac o loadings and eliabili y o each dimension, as well as he in e p e a ion o hese ac o s. To de e mine he dimensions, he ac o ial loadings c i e ion has been ollowed, being he cu o alue 0.30(49). Table 2 - Fac o s, i ems and loadings ob ained in he Explo a o y Fac o Analysis o he IRS-ITLP. G anada, Spain, 2017 Alpha, Fac o loadings I ems, ac o s and a iance explained 1 2 3 h2* α† FACTOR 1: Lea ning en i onmen F1 51.07% 15.The use o IRS makes he classes enjoyable and dynamic 0.759 0.625 0.926 30. The use o IRS allows ac i e discussion o misconcep ions o build knowledge 0.748 0.303 0.736 48. The use o IRS imp o es you pa icipa ion in classes behind anonymi y 0.731 0.650 42. The use o IRS allows you o be mo e con iden when asking ques ions du ing classes 0.685 0.403 0.673 53. The answe s p o ided h ough IRS inc ease my con idence in he classes a e e i ying ha I answe ed co ec ly 0.640 0.537 5.Du ing my expe ience wi h IRS I ha e a good ime lea ning 0.602 0.347 0.484 22. I am mo e in e es ed in classes when using IRS 0.577 0.497 0.641 47. The use o IRS makes he classes mo e pleasan and in e ac i e compa ed o adi ional classes 0.562 0.491 0.644 24. I like he use o IRS as an a endance con ol 0.538 0.480 0.590 1. I am mo e ocused du ing he classes since he implemen a ion o IRS 0.483 0.560 0.564 FACTOR 2: Teaching-lea ning p ocess F2 5.47% 18. The use o IRS imp o es my lea ning pe o mance 0.404 0.726 0.740 0.869 9. The IRS is used o ind ou he ini ial knowledge o he s uden s 0.725 0.550 2. Thanks o he IRS I measu e i I am ollowing co ec ly he con en s o he subjec du ing he classes 0.702 0.599 59. IRS p o ides aluable in o ma ion o imp o e you lea ning p ocess 0.628 0.512 10. The use o IRS is ca ied ou by expe ienced p o esso s o p o ide good eedback 0.324 0.576 0.483 46.The use o he IRS is done a he end o he classes o e iew he con en s explained du ing he session 0.402 0.687 19. The con inuous use o he IRS inc eases my class a endance 0.516 0.422 FACTOR 3: Lea ning Assessmen F3 5.07% 20. The use o IRS allows you o know and compa e you colleagues’ answe s wi h you own answe s 0.864 0.755 0.871 36.The use o IRS p omo es egula s udy o he subjec o be be e p epa ed o classes 0.403 0.678 0.686 21. The use o he IRS allows o co ec mis akes o misunde s andings abou he subjec con en s du ing he classes 0.361 0.641 0.576 35. The use o he IRS e alua es my comp ehensi e knowledge o he con en s in each o he opics co e ed du ing he classes 0.604 0.523 0.715 29. The use o IRS imp o es mo i a ion du ing classes 0.391 0.491 0.680 65. The use o IRS imp o es he unde s anding o he con en s explained in class 0.332 0.486 0.602 *h2 = Communali y; †α = C onbach’s alpha www.ee p.usp.b / lae 5 Mingo ance-Es ada AC, G anda-Ve a J, Rojas-Ruiz G, Alemany-A ebola I. Subsequen ly, he Con i ma o y Fac o Analysis was pe o med, and he 3- ac o s model was es ed. The maximum likelihood es ima ion me hod was used o analyze he co ela ion ma ix. The Goodness-o -Fi o he p oposed model was e alua ed using a ious indica o s. The χ 2 /d (484.13/249) sco es 1,944, a alue ha is wi hin he accep able s anda ds. Mo eo e , he Roo Mean Squa e Residual (RMR) is 0.077 and he Roo Mean Squa e E o o App oxima ion (RMSEA) is 0.080, being bo h indexes conside ed accep able since hey a e be ween 0.5 and .08(49). The Goodness-o -Fi Index (GFI) and he Compa a i e Fi Index (CFI) (wi h alues o 0.78 and 0.97 espec i ely) a e wi hin he ole ance limi s. These esul s con i m ha he 3- ac o model i s he da a, so he model can be main ained as a plausible explana ion o he p oposed dimensional s uc u e. Figu e 1 - Con i ma o y ac o analysis o he ques ionnai e “In e ac i e Response Sys em o he Imp o emen o he Teaching-Lea ning P ocess” To es he eliabili y o he ins umen , a C onbach’s alpha es is ca ied ou , ob aining a o al alue wi h a α= 0.955 and he dimensions ha make i up, ob aining alues ha ange om α=0.922 o ac o 1, “En i onmen ”, o α= 0.869 in ac o 2 “Teaching-lea ning p ocess”. These da a show ha he eliabili y o he ques ionnai e is good in all he ac o s, being lowe in ac o 3, “Assessmen ”. Al hough his s a is ic has been widely used in social esea ch, i should be complemen ed wi h o he analysis, such as he Composi e Reliabili y Index (CF) and he Mean Ex ac ed Va iance (MEV). The esul s ob ained a e shown in Table 3, being in all cases accep able. www.ee p.usp.b / lae 6 Re . La ino-Am. En e magem 2021;29:e3418. Table 3 - Composi e Reliabili y and Mean Ex ac ed Va iance om he ac o s in he IRS-ITLP ques ionnai e. G anada, Spain, 2017 Fac o s CF*MEV† F1: Lea ning En i onmen 0.955 0.544 F2: Teaching-lea ning p ocess 0.930 0.508 F3: Lea ning assessmen 0.863 0.540 *CF = Composi e Reliabili y; †MEV = Mean Ex ac ed Va iance Discussion The aim o his s udy is he cons uc ion o a alid and eliable ques ionnai e o measu e he use o IRS in uni e si y s uden -cen ed lea ning. The esul s p o ide empi ical e idence o he alidi y o his in e en ion model using IRS, which allows uni e si y p o esso s in gene al, and heal h sciences p o esso s in pa icula , o ans o m he eaching-lea ning p ocess. This model encou ages and in ol es s uden s in his p ocess h ough a mo e ac i e app oach o 21 s cen u y p o esso s, who can measu e s uden s’ pe cep ion on he use o IRSs, an ad ance ha eally makes a di e ence. Mo eo e , apa om a echnopedagogical esou ce, i u ns in o a play ul ac i i y o s uden s in a non- game con ex , being his model included in an eme ging educa ional me hodology called Gami ica ion. In summa y, his model p o ides he oppo uni y o d as ically ans o m adi ional class ooms so ha i imp o es he class oom en i onmen , he lea ning p ocess and hei academic pe o mance in a play ul and enjoyable way. The IRS-ITLP ques ionnai e consis s o 24 i ems ha a e g ouped in o 3 ac o s: Lea ning En i onmen , Teaching-Lea ning P ocess and Assessmen . In ela ion o he quali y o he i ems, which was measu ed h ough i em- o- o al co ela ion, he da a indica es high a es anging om 0.488 o 0.787. These alues show a high in e nal consis ency suppo ing he ideas ha he i ems a e co ela ed, and he scale is accu a e. Fu he mo e, a desc ip i e analysis on he i ems show ha he e is nega i e asymme y, which e eals ha uni e si y s uden s end o ag ee wi h he ques ionnai e s a emen s. As o he eliabili y o he scale, he alue o C onbach ‘s alpha is 0.965, indica ing a high eliabili y. These da a a e in line wi h he composi e eliabili y indexes o he 3 ac o s comp ising IRS-ITLP, which each op imum le els: 0.955, 0.930 and 0.863 o En i onmen , Teaching-Lea ning P ocess and Assessmen , espec i ely, being he minimum accep able alue 0.70. Fu he mo e, he alidi y o he cons uc was es ed h ough an Explo a o y Fac o ial Analysis (EFA) and a Con i ma o y Fac o ial Analysis (CFA). Fi s ly, he EFA was ca ied ou since he e we e no simila alida ed ins umen s ha measu e he pe cep ion o uni e si y s uden s owa ds lea ning; we only knew he dimensions ha make up he cons uc acco ding o he li e a u e consul ed. To do so, i s ly he da a we e assessed ob aining signi ican alues bo h in he Kaise -Meye -Olkin es , (KMO= 0.941) and he Ba le ’s sphe ici y es (c 2 = 2446.206; d = 300; p < 0.001). These alues indica e ha a ac o ial analysis was an adequa e echnique o in e p e ing he in o ma ion con ained in his ma ix. This analysis yielded h ee clea ly de ined ac o s ha explain 61.61% o he o al a iance. The esul s o he CFA con i m he h ee- ac o model. The indexes o goodness used we e χ2/d =1.944 (which is wi hin he es ablished s anda ds), he RMR is 0.077 and he RMSEA is 0.080, which a e conside ed accep able as hey a e be ween 0.5 and 0.08(49). In addi ion, he GFI ep esen ing he join adjus men is 0.78 and CFI is 0.97, so bo h alues a e wi hin he ole ance limi s. These esul s con i m ha he da a i o he 3- ac o model, suppo ing he p oposed dimensional s uc u e. Fo all hese easons, hese esul s allow us o be con iden in he eliabili y and alidi y o his ins umen . The e o e, he cons uc ion and alida ion o his ques ionnai e make possible he applica ion o his ins umen o measu e he s uden ’s pe cep ion in he use o IRS in he lea ning p ocess. Rega ding he ac o “Lea ning en i onmen ”, eliabili y was measu ed using he C onbach’s alpha coe icien , which sco es 0.926 and sugges s a high in e nal consis ency. The EFA explains 51.7% o he o al a iance, being he ac o wi h he highes punc ua ion in his ques ionnai e. Among he i ems comp ising his dimension a e: i em 15: he use o IRS makes he classes enjoyable and dynamic, i em 48: he use o IRS imp o es my pa icipa ion in he classes behind anonymi y, i em 47: he use o IRS makes he classes mo e enjoyable and in e ac i e compa ed o adi ional classes, among o he s. This ac o is decisi e since uni e si y s uden s wi h an adequa e class en i onmen inc ease class a endance (22) and imp o es hei pa icipa ion(13,17,44). I also comes on in e ac ion be ween p o esso s and s uden s(13), and has a posi i e in luence on a en ion (33-34) and concen a ion (35) , as he s udies analyzed show. The second dimension, which co esponds o he “ eaching-lea ning p ocess” ac o , shows an in e nal consis ency index o 0.869 and explains 5.4% o he o al a iance, signi ican ly lowe ing he weigh o he ac o . This dimension e e s o hose elemen s ha a e basic o acqui e knowledge, such as deba es and in e ac ion be ween p o esso s and s uden s, which posi i ely a ec s hei lea ning p ocess since i helps s uden s o e iew and unde s and he con en s. Among he i ems ha make up his ac o a e: i em 47 he use o IRS makes www.ee p.usp.b / lae 7 Mingo ance-Es ada AC, G anda-Ve a J, Rojas-Ruiz G, Alemany-A ebola I. he classes mo e pleasan and in e ac i e in compa ison wi h adi ional classes, i em 59 IRS p o ides aluable in o ma ion o imp o e my lea ning p ocess, i em 46 he use o IRS is done a he end o he classes o e iew he con en s explained du ing he session. These elemen s coincide wi h s udies suppo ing ha his wo king me hodology imp o es pe o mance(31-32,38-41) hanks o he pa adigm shi ha implies ac i e lea ning h ough he connec ion be ween knowledge and know- how(45) and he equen and posi i e in e ac ion, which esul ed in o mo e dynamic classes when IRS is used (13,28) . This me hod p omo es ma e ial comp ehension by acqui ing a deepe knowledge, helping o e iew and unde s and he con en s and imp o ing long- e m e en ion(36-37); he inal esul is an imp o emen in he lea ning p ocess. In ela ion o he hi d dimension, “Lea ning Assessmen ” p esen s a C onbach’s alpha o 0.871, and he EFA explains 5.07% o he o al a iance, showing a simila pe cen age o he p e ious ac o . This dimension is ela ed o eedback and o ma i e e alua ion, which help o co ec e o s o misunde s andings abou he con en s o he subjec wo ked on (4,20,40) . The i ems included in his ac o a e: i em 20 he use o IRS allows you o know and compa e you colleagues’ answe s wi h you own answe s; i em 36 he use o IRS p omo es egula s udy o he subjec o be be e p epa ed o classes; i em 35, The use o IRS e alua es my comp ehensi e knowledge o he con en s in each o he opics co e ed du ing he classes. The immedia e eedback has a posi i e impac on bo h he s uden s in hei lea ning p ocess and he p o esso in hei eaching p ocess, d i ing he s uden s’ o ma i e e alua ion. In his way, in ou global con ex wi h no academic on ie s, he Eu opean and La in Ame ican Highe Educa ion a eas mee he p emise ha uni e si y sys em mus be in a con inuous ans o ma ion owa ds ac i e s uden lea ning and li elong lea ning. In addi ion, i is necessa y o es ablish e alua ion and acc edi a ion o he wo k ca ied ou in uni e si ies, p omo ing he ans e ence o knowledge be ween successi e esea ches among di e en ields o knowledge. This is he way an in e disciplina y unde s anding can be de eloped o assess he eali y and o ques ion he adi ional conside a ion o he ields o knowledge as isola ed compa men s sepa a ed by disciplina y bounda ies. Among he limi a ions o his esea ch, we lis he ollowing: i s ly, i is necessa y o con inue inc easing he numbe o pa icipan s, since he sample is no excessi ely la ge, bu i does p o ide a s a ing poin o ans e ing i o o he ields o knowledge. The eason o his is he limi ed numbe o p o esso s a he Melilla Campus o he Uni e si y o G anada who use echnopedagogical esou ces such as IRS in hei class ooms and in eg a e hem o p ocesses o ac i e gami ica ion me hodology, which is why i is necessa y o p omo e and explo e hese esou ces. I is he e o e essen ial o aise awa eness and ain uni e si y p o esso s in he TPACK (Technological Pedagogical and Con en Knowledge) model as a concep ual amewo k ha can guide p o esso s when in eg a ing echnology in o s uden s’ lea ning p ocesses, which will p omo e eme ging me hodologies such as gami ica ion. Secondly, imp o ing he ins umen is c ucial so ha i can be used by p o esso s in any ield o knowledge and in any si ua ion (as has been done du ing he COVID-19 con inemen ). The objec i e is launching an ins umen o u u e pe iods, acco ding o he Go e nmen and uni e si y app oaches, ha can be used in ace- o- ace aining, B-Lea ning o E-Lea ning. The e o e, he ins umen needs o be de eloped by including new i ems ha assess his new i ual educa ion models. In addi ion, we mus deepen he in luence o his ins umen depending on he di e en disciplines o he Heal h Sciences, a esea ch ha we in end o ca y ou in he u u e. The applica ion o echnological ad ances in he uni e si y ield comes on he ac i e lea ning p ocess o s uden s, and i is necessa y o know hei opinion o achie e p og ess and imp o emen in eaching. Thi dly, i is necessa y u he quasi-expe imen al esea ch in he se e al ields o uni e si y eaching o analyze he in luence o gami ied ac i e me hodology h ough hese echnopedagogical esou ces and adi ional me hodology, bea ing in mind i s scope o ela ionship wi h academic pe o mance. Conclusion The con ibu ion o his s udy o knowledge is he design, de elopmen and dissemina ion o a new ins umen ha allows he measu emen and assessmen o s uden s’ pe cep ion on he use o IRS du ing uni e si y aining by using a echnopedagogical esou ce in any ield o knowledge (whe he Heal h Sciences, Social and Legal Sciences, A s and Humani ies, o Enginee ing and A chi ec u e). The ins umen conside s h ee undamen al ac o s: lea ning en i onmen , eaching-lea ning p ocess and assessmen . This ques ionnai e can be applied in he class oom du ing he aining, helping o imp o e he eaching-lea ning p ocesses. In his sense, Heal h Sciences p o esso s mus es ablish and explo e inno a i e ways o in ol e s uden s and s imula e ac i e lea ning. I is impo an o inco po a e ac i e me hods by using in e ac i e esponse commands in disciplines such as nu sing, medicine, pha macy, pa amedical educa ion, psychology, den is y, physio he apy, speech he apy, bio echnology, epidemiology, gene ics, biochemis y, occupa ional he apy, human nu i ion and die e ics, www.ee p.usp.b / lae 8 Re . La ino-Am. En e magem 2021;29:e3418. among o he s. This ins umen p o ides a posi i e pedagogical app oach in he eaching-lea ning p ocess o Heal h Sciences p o esso s and s uden s, imp o ing he academic pe o mance wi h he pu pose o acqui ing a deep knowledge o he subjec s. Re e ences 1. So illo JA, Rod ígue I, Echa E, Ojeda T. El Espacio Ibe oame icano de Educación Supe io . Diagnós ico y p opues as ins i ucionales. Mad id: Fundación Ca olina, CeALCI; 2009. 2. Mingo ance AC, G anda J, Rojas G, Alemany I. Flipped Class oom o Imp o e Uni e si y S uden Cen e ed Lea ning and Academic Pe o mance. Soc Sci. 2019;8(11):315-28. doi: h ps://doi.o g/10.3390/ socsci8110315 3. Ha is J, Mish a P, Koehle M. Teache s’ echnological pedagogical con en knowledge and lea ning ac i i y ypes: cu iculum-based echnology in eg a ion e ained. J Res Technol Educ. [In e ne ]. 2009 [ci ed Oc 21, 2018];41(4):393-416. A ailable om: h ps:// iles.e ic. ed.go / ull ex /EJ844273.pd 4. Cheung G, Wan K, Chan K. E icien Use o Clicke s: A Mixed-Me hod Inqui y wi h Uni e si y Teache s. Educ. Sci. 2018;8(31):1-15. doi: h ps://doi.o g/10.3390/ educsci8010031 5. Schube V, Medina J, Lenise M. P oceso de cons ucción del conocimien o pedagógico del docen e uni e si a io de en e me ía1. Re . La ino-Am. En e magem. [In e ne ]. 2011 [ci ed Oc 22, 2018];19(2). A ailable om: h p:// www.scielo.b /pd / lae/ 19n2/es_26.pd 6. Cheung G, Wan K, Chan K. E icien Use o Clicke s: A Mixed-Me hod Inqui y wi h Uni e si y Teache s. Educ. Sci. 2018;8(31):1-15. doi: h ps://doi.o g/10.3390/ educsci8010031 7. Johnson L, Adams S, Es ada V, F eeman A. NMC Ho izon Repo : 2016 Highe Educa ion Edi ion. [In e ne ]. Aus in: The New Media Conso ium; 2016 [ci ed Sep 15, 2018]. A ailable om: h p://www.ap ende i ual.o g/ cen o-documen acion-pd /2016-nmc-ho izon- epo - HE-ES.pd 8. Kapsalis G, Fe a i A, Punie Y, Con ads J, Collado A, Ho ulainen R, e al. E idence o Inno a i e Assessmen : Li e a u e Re iew and Case S udies, EUR 29882. Luxembou g: Eu opean Commission; 2019. doi: h ps://doi.o g/10.2760/552774 9. Ma celo C, Yo C, Mayo C. Enseña con ecnologías digi ales en la Uni e sidad. Comunica . 2015;45:117-24. doi: h ps://doi.o g/10.3916/C45-2015-12 10. Hinojo FJ, Mingo ance AC, T ujillo JM, Azna I, Cáce es MP. Incidence o he Flipped Class oom in he Physical Educa ion S uden s’ Academic Pe o mance in Uni e si y Con ex s. Sus ainabili y. 2018;10(5):1334-46. doi: h ps://doi.o g/10.3390/su10051334 11. Njie-Ca VP, Ludeman E, Lee MC, Do dunoo D, T ocky NM, Jenkins LS. An In eg a i e Re iew o Flipped Class oom Teaching Models in Nu sing Educa ion. J P o Nu s. 2017;33(2):133-44. doi: 10.1016/j. p o nu s.2016.07.001 12. Wilson J, Walke S. Tu ning a c isis in o an in e ac i e d ama: he in oduc ion o a ‘clicke s hea e’ in nu se educa ion. Nu se Educ Today. 2017;51:109-11. doi: 10.1016 / j.ned .2016.11.023 13. Ca ino SS. Digi al connec ion in a physical class oom: Clicke s and he s uden - eache ela ionship [ hesis]. G eensbo o: Uni e si y o No h Ca olina; 2015 [ci ed Dec 12, 2019]. A ailable om: h ps://lib es.uncg.edu/ i /uncg/ /Ca ino_uncg_0154D_11791.pd 14. Zainuddin Z, Halili SH. Flipped class oom esea ch and ends om di e en ields o s udy. In Re Res Open Dis ance Lea n. [In e ne ]. 2016 [ci ed Jul 18, 2018];17(3):313-40. A ailable om: h ps:// iles.e ic. ed.go / ull ex /EJ1102721.pd 15. Ali RA, Alna ou A, Alnuaimi K, Alzoubi F, Almomani M, O hman A. E ec s o in e ac i e eaching on uni e si y s uden s’ knowledge and a i ude owa d ep oduc i e heal h: a pilo s udy in Jo dan. J Mul idiscip Heal hc. 2018;11:211-21. doi: h ps://doi.o g/10.2147/JMDH. S160135 16. Ba celo JM. Medical labo a o y science and nu sing s uden s’ pe cep ion o he academic lea ning en i onmen a a Philippine uni e si y using he Dundee Ready Educa ion En i onmen Measu e. J Educ E al Heal h P o . 2016;13(33):1-7. doi: h ps://doi.o g/10.3352/ jeehp.2016.13.33 17. B ady M, Seli H, Rosen hal J. “Clicke s” and me acogni ion: a quasi-expe imen al compa a i e s udy abou me acogni i e sel - egula ion and use o elec onic eedback de ices. Compu Educ. 2013;65:56-63. doi: h ps://doi.o g/10.1016/j.compedu.2013.02.001 18. Cas o MJ, López M, Cao MJ, Fe nández-Cas o M, Ga cía S, F u os M, e al. Impac o educa ional games on academic ou comes o s uden s in he Deg ee in Nu sing. PLoS One. 2019;14(7):e0220388. doi: h ps:// doi.o g/10.1371/jou nal.pone.0220388 19. Nishimu a A. E ec s o di e en me hods o e lec ion on nu ses’ gaze and judgemen in a ask using a ouch panel. J Clin Nu s. 2018;27:569-77. doi: h ps://doi. o g/10.1111/jocn.14096 20. Too hake R. Millennial’s pe spec i e o clicke echnology in a nu sing class oom: A mixed me hods esea ch s udy. Nu se Educ Today. 2018;62:80-4. doi: h ps://doi.o g/10.1016/j.ned .2017.12.027 21. Chen TL, Lan YL. Using a pe sonal esponse sys em as an in-class assessmen ool in he eaching o basic college www.ee p.usp.b / lae 9 Mingo ance-Es ada AC, G anda-Ve a J, Rojas-Ruiz G, Alemany-A ebola I. chemis y. Aus alas J Educ Technol. 2013;29(1):32-40. doi: h ps://doi.o g/10.14742/aje .95 22. Ak ekin NÇ, Çelebi H, Ak ekin M. Le ’s kahoo ! Ana omy. In . J. Mo phol. 2018;36(2):716-21. doi: h p:// dx.doi.o g/10.4067/S0717-95022018000200716 23. Ba ML. Encou aging college s uden ac i e engagemen in lea ning: The in luence o esponse me hods. Inno High Educ. 2013;39(4):307-19. doi: h ps://doi.o g/10.1111/jcal.12205 24. Iskande M. Sys ema ic e iew o he implemen a ion o audience esponse sys ems and hei impac on pa icipa ion and engagemen in he educa ion o heal hca e p o essionals. BMJ Simula ion & Technology Enhanced Lea ning. 2018;4(2):47-50. doi: h p://dx.doi. o g/10.1136/bmjs el-2017-000245 25. Lico ish SA, Owen HE, Daniel, B, Geo ge, JL. S uden s’ pe cep ion o Kahoo !’s in uence on eaching and lea ning. RPTEL. 2018;13(9):1-23. doi: h ps://doi.o g/10.1186/ s41039-018-0078-8 26. Oli ei a C, Ti apelli C, Rod igues CT, Domaneschi C, Caldei a SA. In e ac i e audience esponse sys ems in o al and maxillo acial adiology unde g adua e lec u es. Eu J Den Educ. [In e ne ]. 2017;22:63-9. doi: h ps:// doi.o g/10.1111/eje.12258 27. Mohan R. Enhancing s uden engagemen and immedia e eedback wi h clicke s and esponse ca ds. In J Inno Lea n. 2018;24(1):81-97. doi: h ps://doi. o g/10.1504/IJIL.2018.092924 28. Banks D. Re lec ions on he use o ARS wi h small g oups. In: ______. Audience Response Sys ems in Highe Educa ion: Applica ions and Cases. Pennsyl ania: IGI Global; 2006. p. 373-86. 29. Wood A. Nu se Pe cep ions o In e ac i i y du ing Thei Onboa ding O ien a ion: E ec o an Audience Response Sys em [ hesis]. Boiling Sp ings: Hun School o Nu sing; 2017 [ci ed Dec 12, 2019]. A ailable om: h ps:// digi alcommons.ga dne -webb.edu/cgi/ iewcon en .cgi ?a icle=1268&con ex =nu sing_e d 30. Ismaile S, Alhosban F, Hawamdeh S. Making lea ning un o inc ease nu sing s uden s’ success: Fo ma i e eedback in communica ion lea ning. Aus alas Med J. 2017;10(12):1014-21. doi: h ps://doi.o g/10.21767/ AMJ.2017.3228 31. Cas o MJ, López M, Cao MJ, Fe nández-Cas o M, Ga cía S, F u os M, e al. Impac o educa ional games on academic ou comes o s uden s in he Deg ee in Nu sing. PLoS One 2019;14(7):e0220388. doi: h ps:// doi.o g/10.1371/jou nal.pone.0220388 32. Co ell A, Regue as LM, Ve dú E, Ve dú MJ, Cas o JP. E ec s o compe i i e lea ning ools on medi- cal s uden s: A case s udy. PLoS One. 2018;13(3):e0194096. doi: h ps://doi.o g/10.1371/jou nal.pone.0194158 33. Oswald KM, Rho en SE. Imp o ing class oom clicke p ac ices: E ec s o incen i es and eedback on e en ion. N Am J Psychol. [In e ne ]. 2014 [ci ed Dec 6, 2017];16(1):79-88. A ailable om: h ps://www. esea chga e.ne /publica ion/285958781_Imp o ing_ class oom_clicke _p ac ices_E ec s_o _incen i es_and_ eedback_on_ e en ion 34. Ma shall LL, Va non AW. An Empi ical In es iga ion o Clicke Technology in Financial Accoun ing P inciples. J Lea n High Educ. [In e ne ]. 2012 [ci ed Jun 3, 2018];8(1):7-18. A ailable om: h ps:// iles.e ic.ed.go / ull ex /EJ1144930.pd 35. Bojino a ED, Oiga a JN. Teaching and lea ning wi h Clicke s: A e Clicke s good o s uden s? In e discip J E-Lea ning Lea n Objec s. 2011;7:169-83. doi: h ps:// doi.o g/10.28945/1506 36. Benson JD, Szucs KA, DeIuliis E, Le i A. Impac o S uden Response Sys ems on Ini ial Lea ning and Re en ion o Cou se Con en in Heal h Sciences S uden s. J Allied Heal h. [In e ne ]. 2017 [ci ed Feb 12, 2018];46(3):158- 63. A ailable om: h ps://www.ingen aconnec .com/ con en /asahp/jah/2017/00000046/00000003/a 00008 37. Njie-Ca VP, Ludeman E, Lee MC, Do dunoo D, T ocky NM, Jenkins LS. An In eg a i e Re iew o Flipped Class oom Teaching Models in Nu sing Educa ion. J P o Nu s. 2017;33(2):133-44. doi: 10.1016/j. p o nu s.2016.07.001 38. Rana NP, Dwi edi YK. Can clicking p omo e lea ning? measu ing s uden lea ning pe o mance using clicke s in he unde g adua e in o ma ion sys ems class. J In Educ Bus. 2017;10(2):201-15. doi: h p://doi.o g/10.1108/ JIEB-06-2016-0010 39. Iwamo o D, Ha gis J, Tai ano E, Vuong K. Analyzing he e icacy o he es ing e ec using kahoo m on s uden pe o mance. Tu kish Online J Dis ance Educ. [In e ne ]. 2017;18(2):93-80. doi: h ps://doi.o g/10.17718/ ojde.306561 40. Geo ge C, Gallegos C, Tesa AJ, Conno K, Ma z K. The use o a game-based lea ning pla o m o engage nu s- ing s uden s: A desc ip i e, quali a i e s udy. Nu se Educ P ac . 2017;27:101-6. doi: h ps://doi.o g/10.1016/j. nep .2017.08.019. 41. Kalaian SA, Kasim RM. E ec i eness o a ious inno a i e lea ning me hods in heal h science class- ooms: a me a-analysis. Ad Heal h Sci Educ Theo y P ac . [In e ne ]. 2017;22(5):1151-67. doi: h ps://doi. o g/10.1007/s10459-017-9753-6 42. Maloney LM, Dilge JP, We el PA, Cimino LM. A e Audience Response Sys ems Wo h he Cos ? Compa ing Ques ion-D i en Teaching S a egies o Eme gency Medical Technician Educa ion. In e ne J Allied Heal h Sci P ac . [In e ne ]. 2017 [ci ed May 7, 2018];16(1):1-