How o ci e his a icle
Mingo ance-Es ada AC, G anda-Ve a J, Rojas-Ruiz G, Alemany-A ebola I. Valida ion o a ques ionnai e
on he use o In e ac i e Response Sys em in Highe Educa ion. Re . La ino-Am. En e magem. 2021;29:e3418.
[Access
daymon h yea
]; A ailable in:
URL
. DOI: h p://dx.doi.o g/10.1590/1518-8345.3374.3418.
1
Uni e si y o G anada, Depa men o Didac ics and School
O ganiza ion, Melilla, ES, Spain.
2
Uni e si y o G anada, Depa men o Didac ics o Co po al
Exp ession, Melilla, ES, Spain.
3 Uni e si y o G anada, Depa men o De elopmen al and
Educa ional Psychology, Melilla, ES, Spain.
Valida ion o a ques ionnai e on he use o In e ac i e Response Sys em
in Highe Educa ion
Objec i e: his s udy aims o design and alida e a
ques ionnai e o measu e he s uden s’ pe cep ion o he use o
IRS as a echnopedagogical esou ce in he class oom. Me hod:
a 24 i ems ques ionnai e (In e ac i e Response Sys em o
he Imp o emen o he Teaching-Lea ning P ocess) was
designed ad hoc o his esea ch and applied o 142 uni e si y
s uden s. Resul s: bo h he explo a o y and con i ma o y
ac o ial analysis yielded 3 dimensions: class oom en i onmen ,
eaching-lea ning p ocesses and lea ning assessmen . The
esul s ob ained bo h in eliabili y (C onbach’s alpha= 0.955)
and in he Con i ma o y Fac o Analysis (χ2/d =1.944,
CFI=0.97; GFI=0.78; RMR=0.077; RMSEA=0.08) e eal highly
sa is ac o y indices. Conclusion: s a is ical analyses con i m
ha his ins umen is a alid, eliable, and easy- o-apply ool
o p o esso s o e alua e he s uden pe cep ion o s uden -
cen ed lea ning.
Desc ip o s: Educa ion; Highe Educa ion; Lea ning; Valida ion
S udy; Su eys and Ques ionnai es; Technology Educa ion.
O iginal A icle
Re . La ino-Am. En e magem
2021;29:e3418
DOI: 10.1590/1518-8345.3374.3418
www.ee p.usp.b / lae
Ángel Cus odio Mingo ance-Es ada1
h ps://o cid.o g/0000-0003-4478-3011
Juan G anda-Ve a2
h ps://o cid.o g/0000-0001-6888-7785
Glo ia Rojas-Ruiz1
h ps://o cid.o g/0000-0001-8541-383X
Inmaculada Alemany-A ebola3
h ps://o cid.o g/0000-0002-4127-3502
www.ee p.usp.b / lae
2
Re . La ino-Am. En e magem 2021;29:e3418.
In oduc ion
Highe Educa ion mus es ablish a space o he
exchange o p ac ical expe iences ha p omo es knowledge
and esea ch wi hin he common amewo k o he Eu opean
and Ibe o-Ame ican Highe Educa ion A ea
(1-2)
.
The implemen a ion o s uden -cen e ed lea ning is
o ien ed o es ablish a model ha e ec i ely in eg a es
echnology wi h knowledge o didac ic media ion, e ol ing
owa ds he Technological Pedagogical and Con en
Knowledge Model (TPACK). This sys em combines
disciplina y, pedagogical and echnological knowledge, bu
always conside ing he con ex in which i in e enes
(3-4)
,
and inc easing he in e ac ion be ween p o esso s and
s uden s wi hin a c i ical dialogical app oach(2).
Some au ho s conside necessa y o es ablish didac ic
knowledge h ough he ela ionship be ween di e en
ypes o knowledge (coming om he own discipline,
gene al pedagogy and s uden s) and he p o esso ’s
biog aphy
(5)
. Thus, du ing he nu sing ini ial aining, bo h
he pedagogical aspec s, including he implemen a ion
o In e ac i e Response Sys em (he eina e IRS) wi h
emo e answe ing de ices o moni o s uden s’ p og ess,
and he in ol emen o expe p o esso s lead o high-
quali y eaching, among o he issues(6).
In his ega d, encou aging p o esso s o in eg a e
echnology in o he class oom is c ucial, as highligh ed
in he Ho izon Repo (7), since i will signi ican ly impac
on educa ion in he coming yea s. To do his, uni e si y
p o esso s mus use he echnological ools hey a e
amilia wi h, as well as access new echnological
esou ces o imp o e eaching p ocesses(8).
Simila ly, echnological changes in uni e si y p o esso s
ollow a endency and a e no adical; hey in oduce hose
ha a e consis en wi h hei eaching p ac ices in o he
lea ning ac i i ies hey no mally ca y ou
(9)
.
In o de o ace hese echnological challenges, an
in e sion o lea ning p ocess is equi ed; s uden s should
be p o ided wi h ma e ials in a ious o ma s, so ha
hey can ca y ou p elimina y wo k be o e a i ing a he
class oom, inco po a ing IRS o e i y he imp o emen s in
he s uden cen e ed-lea ning p ocess
(10-12)
. IRS has been
al eady in eg a es in some uni e si y classes
(13)
. Thus,
he a ailable li e a u e abou Highe Educa ion on he use
o his echnology in ecen yea s ocuses on he ields
o Science, Technology, Enginee ing and Ma hema ics,
Sociology, Humani ies, Heal h (Medicine and Nu sing),
Business Adminis a ion and English language(13-14).
The e iewed s udies indica e ha in eg a ing IRS
in uni e si y class ooms imp o es h ee main a eas:
he class oom en i onmen , he eaching-lea ning
p ocesses and lea ning assessmen
(15-20)
. Thus, de ining
he possibili ies and limi a ions o his ool is inc easingly
impo an o he imp o emen o he quali y o Highe
Educa ion(6).
Conce ning he class oom en i onmen ac o , IRS
inc eases a endance(21) and s uden pa icipa ion(22-25),
esul ing in a highe le el o in ol emen du ing classes in
compa ison o he adi ional me hodology
(23,26-27)
. Wi hin
he lea ning ac o , some s udies es ablish ha equen
and posi i e in e ac ion makes classes mo e dynamic
when using IRS
(13,28-29)
, p omo ing ac i e lea ning
(30-32)
. In
addi ion, a en ion
(33-34)
, concen a ion
(35)
and memo y
(36-37)
a e encou aged du ing he lea ning p ocess. Ex ensi e
esea ch sugges s ha a be e pe o mance is he esul
o he use o IRS, as some s udies indica e
(31-32,38-41)
, al hough
o he s udies do no ind such an e ec
(42)
. In ela ion o he
assessmen ac o , he indings o he li e a u e suppo he
capaci y o IRS as a ool o assessmen and eedback
(43-44)
. I
is conside ed ha bo h s uden s and p o esso s bene i om
he eedback hey ecei e wi h he use o his educa ional
echnology
(20,27)
. All o his leads o a key lea ning p ocess
o he in e ac ion o knowledge and know-how
(45)
.
The aim o his s udy is o design and alida e his
s udy aims o design and alida e a ques ionnai e o
measu e he s uden s’ pe cep ion o he use o IRS as a
echnopedagogical esou ce in he class oom.
Me hod
The design was ansec ional and desc ip i e, as he
da a we e collec ed in a single ime in o de o desc ibe
he phenomenon and analyze i a a ce ain ime.
The esea ch was ca ied ou a Melilla Campus o
he Uni e si y o G anada (Spain), loca ed in No h A ica,
whose s uden s a end o he Heal h Sciences, Social
and Legal Sciences, and Educa ion and Spo s Sciences
schools. Fo his pu pose, an in en ional non-p obabilis ic
sample was ca ied ou . The selec ion c i e ia ha e been:
i s ly, p o esso s who use echnology in hei class ooms,
speci ically in e ac i e esponse de ices. And secondly,
he willingness o he s uden s o pa icipa e in his
s udy. The e o e, he sample comp ises 142 s uden s:
110 women (77.5%) and 32 men (22.5%). In ela ion o
he academic yea , 17 s uden s a e in i s yea (12%),
95 in second yea (66.9%) and 30 in hi d yea (21.1%).
To ca y ou his s udy, an ad hoc ques ionnai e was
designed o his esea ch, “In e ac i e Response Sys em
o he Imp o emen o he Teaching-Lea ning P ocess
(IRS-ITLP)”.
In ela ion o he i ems o he IRS-ITLP, hey we e
w i en a e an ex ensi e bibliog aphic e iew on he
h ee ac o s highligh ed abo e
(21,46-47)
. Despi e o he
absence o expe s on his ield, his p ocess p o ides
alidi y o he ques ionnai e i ems.
www.ee p.usp.b / lae
3
Mingo ance-Es ada AC, G anda-Ve a J, Rojas-Ruiz G, Alemany-A ebola I.
We s a ed wi h 65 i ems g ouped in o ca ego ies
(48)
.
A e he analysis o he i ems, we selec ed 35 i ems
ha we e g ouped in o he h ee dimensions: lea ning
en i onmen , p ocess and assessmen . Rega ding he
ques ionnai e esponse o ma , a Like ype scale was
used, wi h 5 esponse al e na i es, anging om 1, o ally
disag ee, o 5, o ally ag ee.
The esea ch was o ien ed o di e en deg ees o
he Uni e si y o G anada (Spain), all o hem being
subjec s conce ning he basic o ma ion o he s uden s.
Fo his pu pose, he collabo a ion o he eaching s a
was eques ed o pa icipa e as olun ee in his p ojec
and o in eg a e IRS in hei classes.
The use o he IRS in hese basic aining subjec s
was ca ied ou h oughou he semes e o he 2016-17
academic yea , be o e, du ing and a he end o he classes.
A he end o he semes e , he IRS-ITLP ques ionnai e was
applied he las week o he semes e , wi h a du a ion o
app oxima ely 15 minu es, o ind ou he pe cep ion o he
expe ience. S uden s we e asked o ag ee o pa icipa e
in his expe ience olun a ily and anonymously, ollowing
he ules o he Commi ee on Publica ion E hics (COPE).
The s a is ical so wa e SPSS e sion 20.0 has been
used o he s a is ical p ocessing o he da a. To know he
eliabili y o each g oup o i ems, C onbach’s alpha was used,
and o he alidi y o he ques ionnai e, an Explo a o y
Fac o Analysis was ca ied. Fo he Con i ma o y Fac o
Analysis, he p og am LISREL 8.8 was used.
Resul s
Fi s ly, he eliabili y o he IRS-ITLP ques ionnai e
consis ing o 35 elemen s was analyzed using C onbach’s
alpha in e nal consis ency coe icien , which was 0.965.
Al hough his index was high, we p oceeded o elimina e
hose i ems whose i em- o al co ela ion was in e io o
0.20. Finally, he ques ionnai e was made up o 24 i ems
wi h a α=0.955, showing homogenei y indexes anging
om 0.42 o 0.85.
Subsequen ly, he means, s anda d de ia ions,
asymme y, and i em- o al co ela ions o each o he i ems
we e ob ained. As can be seen in Table 1, he asymme y is
nega i e in all i ems, which shows a g ea e concen a ion o
esponses co esponding o he high sco es in hose i ems.
Table 1 - Desc ip i e alues o he i ems in he IRS-ITLP ques ionnai e. G anada, Spain, 2017
Nº I ems M*SD†
Asymme y
Co el. i em- o al
1 I am mo e ocused du ing he classes since he implemen a ion o IRS 3.40 1.15 -0.477 0.689
2Thanks o IRS, I measu e i I am ollowing co ec ly he con en s o he subjec du ing
he classes 3.81 1.01 -0.658 0.619
5 Du ing my expe ience wi h he IRS I ha e a good ime lea ning 3.52 1.18 -0.559 0.566
9IRS is used o ind ou he ini ial knowledge o he s uden s 3.54 1.258 -0.595 0.488
10 The use o IRS is ca ied ou by expe ienced p o esso s o p o ide good eedback 3.97 0.891 -0.616 0.610
14 The use o IRS helps me o de elop my comp ehension on he con en s I am
wo king on 3.59 1.162 -0.666 0.639
15 The use o he IRSs makes he classes enjoyable and dynamic 3.82 1.119 -0.877 0.656
18 The use o IRS imp o es my lea ning pe o mance 3.76 1.129 -0.653 0.762
19 The con inuous use o he IRS inc eases my class a endance 3.70 1.266 -0.793 0.579
20 The use o IRS allows you o know and compa e you colleagues’ answe s wi h you
own answe s 3.42 1.234 -0.646 0.441
21 The use o he IRS allows o co ec mis akes o misunde s andings abou he
subjec con en s du ing he classes 3.71 1.121 -1.032 0.558
22 I am mo e in e es ed in classes when using IRS 3.65 1.066 -0.698 0.768
24 I like he use o IRS as an a endance con ol 3.65 1.005 -0.605 0.699
29 The use o IRS imp o es mo i a ion du ing classes 3.83 1.111 -0.825 0.753
30 The use o IRS allows ac i e discussion o misconcep ions o build knowledge 3.80 1.168 -0.771 0.792
35 The use o he IRS e alua es my comp ehensi e knowledge o he con en s in each
o he opics co e ed du ing he classes 3.99 0.971 -1.211 0.788
36 The use o IRS p omo es egula s udy o he subjec o be be e p epa ed o
classes 3.64 1.094 -0.762 0.715
42 The use o IRS allows you o be mo e con iden when asking ques ions du ing
classes 3.76 1.254 -0.739 0.787
46 The use o he IRS is done a he end o he classes o e iew he con en s explained
du ing he session 3.80 1.100 -0.736 0.772
47 The use o IRS makes he classes mo e pleasan and in e ac i e compa ed o
adi ional classes 3.86 1.082 -0.804 0.769
48 The use o IRS imp o es you pa icipa ion in classes behind anonymi y 3.87 1.104 -0.773 0.712
53 The answe s p o ided h ough he IRS inc ease my con idence in he classes a e
e i ying ha I answe ed co ec ly 3.44 1.164 -0.420 0.700
59 IRS p o ides aluable in o ma ion o imp o e you lea ning p ocess 4.09 1.017 -1.007 0.687
65 The use o IRS imp o es he unde s anding o he con en s explained in class 3.87 1.160 -0.911 0.727
*M = Mean; †SD = S anda d de ia ion
www.ee p.usp.b / lae
4
Re . La ino-Am. En e magem 2021;29:e3418.
Since any s udy abou he ac o analysis o he IRS-
ITLP ques ionnai e had p e iously been published, be o e
pe o ming a Con i ma o y Fac o Analysis i was con enien
o ca y ou an Explo a o y Fac o Analysis (EFA) o explo e
how he i ems a e g ouped in o ac o s. To ensu e ha he
da a i a ac o analysis model, he da a we e subjec ed
o he Kaise -Meye -Olkin es (KMO= 0.941) and o he
Ba le ’s Tes o Sphe ici y (c2= 2446.206; d = 300;
p<0.001). The alues indica e ha a ac o analysis is a
sui able echnique o s uc u e he in o ma ion con ained in
he ma ix. The EFA e eals he exis ence o 3 ac o s ha
explain 61.61% o he o al a iance, being his p opo ion
accep able. In addi ion, he i em communali ies a e abo e
h
2
=0.40, anging om 0.421 “The con inuous use o IRS
inc eases my class a endance” o 0.791 “The use o IRS
allows you o know and compa e you colleagues’ answe s
wi h you own answe s”.
Table 2 shows he ac o s, i ems, ac o loadings and
eliabili y o each dimension, as well as he in e p e a ion
o hese ac o s. To de e mine he dimensions, he
ac o ial loadings c i e ion has been ollowed, being he
cu o alue 0.30(49).
Table 2 - Fac o s, i ems and loadings ob ained in he Explo a o y Fac o Analysis o he IRS-ITLP. G anada, Spain, 2017
Alpha,
Fac o
loadings
I ems, ac o s and a iance explained 1 2 3 h2* α†
FACTOR 1: Lea ning en i onmen
F1
51.07%
15.The use o IRS makes he classes enjoyable and dynamic 0.759 0.625
0.926
30. The use o IRS allows ac i e discussion o misconcep ions o
build knowledge 0.748 0.303 0.736
48. The use o IRS imp o es you pa icipa ion in classes behind
anonymi y 0.731 0.650
42. The use o IRS allows you o be mo e con iden when asking
ques ions du ing classes 0.685 0.403 0.673
53. The answe s p o ided h ough IRS inc ease my con idence in
he classes a e e i ying ha I answe ed co ec ly 0.640 0.537
5.Du ing my expe ience wi h IRS I ha e a good ime lea ning 0.602 0.347 0.484
22. I am mo e in e es ed in classes when using IRS 0.577 0.497 0.641
47. The use o IRS makes he classes mo e pleasan and
in e ac i e compa ed o adi ional classes 0.562 0.491 0.644
24. I like he use o IRS as an a endance con ol 0.538 0.480 0.590
1. I am mo e ocused du ing he classes since he implemen a ion
o IRS 0.483 0.560 0.564
FACTOR 2: Teaching-lea ning p ocess
F2
5.47%
18. The use o IRS imp o es my lea ning pe o mance 0.404 0.726 0.740
0.869
9. The IRS is used o ind ou he ini ial knowledge o he s uden s 0.725 0.550
2. Thanks o he IRS I measu e i I am ollowing co ec ly he
con en s o he subjec du ing he classes 0.702 0.599
59. IRS p o ides aluable in o ma ion o imp o e you lea ning
p ocess 0.628 0.512
10. The use o IRS is ca ied ou by expe ienced p o esso s o
p o ide good eedback 0.324 0.576 0.483
46.The use o he IRS is done a he end o he classes o e iew
he con en s explained du ing he session 0.402 0.687
19. The con inuous use o he IRS inc eases my class a endance 0.516 0.422
FACTOR 3: Lea ning Assessmen
F3
5.07%
20. The use o IRS allows you o know and compa e you
colleagues’ answe s wi h you own answe s 0.864 0.755
0.871
36.The use o IRS p omo es egula s udy o he subjec o be
be e p epa ed o classes 0.403 0.678 0.686
21. The use o he IRS allows o co ec mis akes o
misunde s andings abou he subjec con en s du ing he classes 0.361 0.641 0.576
35. The use o he IRS e alua es my comp ehensi e knowledge o
he con en s in each o he opics co e ed du ing he classes 0.604 0.523 0.715
29. The use o IRS imp o es mo i a ion du ing classes 0.391 0.491 0.680
65. The use o IRS imp o es he unde s anding o he con en s
explained in class 0.332 0.486 0.602
*h2 = Communali y; †α = C onbach’s alpha
www.ee p.usp.b / lae
5
Mingo ance-Es ada AC, G anda-Ve a J, Rojas-Ruiz G, Alemany-A ebola I.
Subsequen ly, he Con i ma o y Fac o Analysis
was pe o med, and he 3- ac o s model was es ed.
The maximum likelihood es ima ion me hod was used o
analyze he co ela ion ma ix. The Goodness-o -Fi o he
p oposed model was e alua ed using a ious indica o s.
The χ
2
/d (484.13/249) sco es 1,944, a alue ha is
wi hin he accep able s anda ds. Mo eo e , he Roo Mean
Squa e Residual (RMR) is 0.077 and he Roo Mean Squa e
E o o App oxima ion (RMSEA) is 0.080, being bo h
indexes conside ed accep able since hey a e be ween
0.5 and .08(49). The Goodness-o -Fi Index (GFI) and he
Compa a i e Fi Index (CFI) (wi h alues o 0.78 and
0.97 espec i ely) a e wi hin he ole ance limi s. These
esul s con i m ha he 3- ac o model i s he da a, so
he model can be main ained as a plausible explana ion
o he p oposed dimensional s uc u e.
Figu e 1 - Con i ma o y ac o analysis o he ques ionnai e “In e ac i e Response Sys em o he Imp o emen o
he Teaching-Lea ning P ocess”
To es he eliabili y o he ins umen , a C onbach’s
alpha es is ca ied ou , ob aining a o al alue wi h a α=
0.955 and he dimensions ha make i up, ob aining alues
ha ange om α=0.922 o ac o 1, “En i onmen ”, o
α= 0.869 in ac o 2 “Teaching-lea ning p ocess”. These
da a show ha he eliabili y o he ques ionnai e is good
in all he ac o s, being lowe in ac o 3, “Assessmen ”.
Al hough his s a is ic has been widely used in social
esea ch, i should be complemen ed wi h o he analysis,
such as he Composi e Reliabili y Index (CF) and he Mean
Ex ac ed Va iance (MEV). The esul s ob ained a e shown
in Table 3, being in all cases accep able.

www.ee p.usp.b / lae
6
Re . La ino-Am. En e magem 2021;29:e3418.
Table 3 - Composi e Reliabili y and Mean Ex ac ed
Va iance om he ac o s in he IRS-ITLP ques ionnai e.
G anada, Spain, 2017
Fac o s CF*MEV†
F1: Lea ning En i onmen 0.955 0.544
F2: Teaching-lea ning p ocess 0.930 0.508
F3: Lea ning assessmen 0.863 0.540
*CF = Composi e Reliabili y; †MEV = Mean Ex ac ed Va iance
Discussion
The aim o his s udy is he cons uc ion o a alid
and eliable ques ionnai e o measu e he use o IRS in
uni e si y s uden -cen ed lea ning. The esul s p o ide
empi ical e idence o he alidi y o his in e en ion model
using IRS, which allows uni e si y p o esso s in gene al,
and heal h sciences p o esso s in pa icula , o ans o m
he eaching-lea ning p ocess. This model encou ages and
in ol es s uden s in his p ocess h ough a mo e ac i e
app oach o 21
s
cen u y p o esso s, who can measu e
s uden s’ pe cep ion on he use o IRSs, an ad ance ha
eally makes a di e ence.
Mo eo e , apa om a echnopedagogical esou ce,
i u ns in o a play ul ac i i y o s uden s in a non-
game con ex , being his model included in an eme ging
educa ional me hodology called Gami ica ion. In summa y,
his model p o ides he oppo uni y o d as ically
ans o m adi ional class ooms so ha i imp o es he
class oom en i onmen , he lea ning p ocess and hei
academic pe o mance in a play ul and enjoyable way.
The IRS-ITLP ques ionnai e consis s o 24 i ems
ha a e g ouped in o 3 ac o s: Lea ning En i onmen ,
Teaching-Lea ning P ocess and Assessmen . In ela ion
o he quali y o he i ems, which was measu ed h ough
i em- o- o al co ela ion, he da a indica es high a es
anging om 0.488 o 0.787. These alues show a high
in e nal consis ency suppo ing he ideas ha he i ems
a e co ela ed, and he scale is accu a e. Fu he mo e,
a desc ip i e analysis on he i ems show ha he e is
nega i e asymme y, which e eals ha uni e si y
s uden s end o ag ee wi h he ques ionnai e s a emen s.
As o he eliabili y o he scale, he alue o
C onbach ‘s alpha is 0.965, indica ing a high eliabili y.
These da a a e in line wi h he composi e eliabili y
indexes o he 3 ac o s comp ising IRS-ITLP, which each
op imum le els: 0.955, 0.930 and 0.863 o En i onmen ,
Teaching-Lea ning P ocess and Assessmen , espec i ely,
being he minimum accep able alue 0.70.
Fu he mo e, he alidi y o he cons uc was es ed
h ough an Explo a o y Fac o ial Analysis (EFA) and a
Con i ma o y Fac o ial Analysis (CFA). Fi s ly, he EFA was
ca ied ou since he e we e no simila alida ed ins umen s
ha measu e he pe cep ion o uni e si y s uden s owa ds
lea ning; we only knew he dimensions ha make up he
cons uc acco ding o he li e a u e consul ed.
To do so, i s ly he da a we e assessed ob aining
signi ican alues bo h in he Kaise -Meye -Olkin es ,
(KMO= 0.941) and he Ba le ’s sphe ici y es (c
2
=
2446.206; d = 300; p < 0.001). These alues indica e
ha a ac o ial analysis was an adequa e echnique o
in e p e ing he in o ma ion con ained in his ma ix. This
analysis yielded h ee clea ly de ined ac o s ha explain
61.61% o he o al a iance. The esul s o he CFA
con i m he h ee- ac o model. The indexes o goodness
used we e χ2/d =1.944 (which is wi hin he es ablished
s anda ds), he RMR is 0.077 and he RMSEA is 0.080,
which a e conside ed accep able as hey a e be ween 0.5
and 0.08(49). In addi ion, he GFI ep esen ing he join
adjus men is 0.78 and CFI is 0.97, so bo h alues a e
wi hin he ole ance limi s. These esul s con i m ha he
da a i o he 3- ac o model, suppo ing he p oposed
dimensional s uc u e.
Fo all hese easons, hese esul s allow us o be
con iden in he eliabili y and alidi y o his ins umen .
The e o e, he cons uc ion and alida ion o his
ques ionnai e make possible he applica ion o his
ins umen o measu e he s uden ’s pe cep ion in he
use o IRS in he lea ning p ocess.
Rega ding he ac o “Lea ning en i onmen ”,
eliabili y was measu ed using he C onbach’s alpha
coe icien , which sco es 0.926 and sugges s a high
in e nal consis ency. The EFA explains 51.7% o he o al
a iance, being he ac o wi h he highes punc ua ion
in his ques ionnai e. Among he i ems comp ising his
dimension a e: i em 15: he use o IRS makes he classes
enjoyable and dynamic, i em 48: he use o IRS imp o es
my pa icipa ion in he classes behind anonymi y, i em
47: he use o IRS makes he classes mo e enjoyable and
in e ac i e compa ed o adi ional classes, among o he s.
This ac o is decisi e since uni e si y s uden s wi h an
adequa e class en i onmen inc ease class a endance
(22)
and imp o es hei pa icipa ion(13,17,44). I also comes on
in e ac ion be ween p o esso s and s uden s(13), and has a
posi i e in luence on a en ion
(33-34)
and concen a ion
(35)
,
as he s udies analyzed show.
The second dimension, which co esponds o he
“ eaching-lea ning p ocess” ac o , shows an in e nal
consis ency index o 0.869 and explains 5.4% o he o al
a iance, signi ican ly lowe ing he weigh o he ac o .
This dimension e e s o hose elemen s ha a e basic
o acqui e knowledge, such as deba es and in e ac ion
be ween p o esso s and s uden s, which posi i ely a ec s
hei lea ning p ocess since i helps s uden s o e iew
and unde s and he con en s. Among he i ems ha
make up his ac o a e: i em 47 he use o IRS makes
www.ee p.usp.b / lae
7
Mingo ance-Es ada AC, G anda-Ve a J, Rojas-Ruiz G, Alemany-A ebola I.
he classes mo e pleasan and in e ac i e in compa ison
wi h adi ional classes, i em 59 IRS p o ides aluable
in o ma ion o imp o e my lea ning p ocess, i em 46 he
use o IRS is done a he end o he classes o e iew he
con en s explained du ing he session.
These elemen s coincide wi h s udies suppo ing ha
his wo king me hodology imp o es pe o mance(31-32,38-41)
hanks o he pa adigm shi ha implies ac i e lea ning
h ough he connec ion be ween knowledge and know-
how(45) and he equen and posi i e in e ac ion, which
esul ed in o mo e dynamic classes when IRS is used
(13,28)
.
This me hod p omo es ma e ial comp ehension by
acqui ing a deepe knowledge, helping o e iew and
unde s and he con en s and imp o ing long- e m
e en ion(36-37); he inal esul is an imp o emen in he
lea ning p ocess.
In ela ion o he hi d dimension, “Lea ning
Assessmen ” p esen s a C onbach’s alpha o 0.871, and
he EFA explains 5.07% o he o al a iance, showing a
simila pe cen age o he p e ious ac o . This dimension is
ela ed o eedback and o ma i e e alua ion, which help
o co ec e o s o misunde s andings abou he con en s
o he subjec wo ked on
(4,20,40)
. The i ems included in his
ac o a e: i em 20 he use o IRS allows you o know
and compa e you colleagues’ answe s wi h you own
answe s; i em 36 he use o IRS p omo es egula s udy
o he subjec o be be e p epa ed o classes; i em 35,
The use o IRS e alua es my comp ehensi e knowledge
o he con en s in each o he opics co e ed du ing he
classes. The immedia e eedback has a posi i e impac
on bo h he s uden s in hei lea ning p ocess and he
p o esso in hei eaching p ocess, d i ing he s uden s’
o ma i e e alua ion.
In his way, in ou global con ex wi h no academic
on ie s, he Eu opean and La in Ame ican Highe
Educa ion a eas mee he p emise ha uni e si y sys em
mus be in a con inuous ans o ma ion owa ds ac i e
s uden lea ning and li elong lea ning. In addi ion, i
is necessa y o es ablish e alua ion and acc edi a ion
o he wo k ca ied ou in uni e si ies, p omo ing he
ans e ence o knowledge be ween successi e esea ches
among di e en ields o knowledge. This is he way an
in e disciplina y unde s anding can be de eloped o assess
he eali y and o ques ion he adi ional conside a ion
o he ields o knowledge as isola ed compa men s
sepa a ed by disciplina y bounda ies.
Among he limi a ions o his esea ch, we lis he
ollowing: i s ly, i is necessa y o con inue inc easing he
numbe o pa icipan s, since he sample is no excessi ely
la ge, bu i does p o ide a s a ing poin o ans e ing
i o o he ields o knowledge. The eason o his is he
limi ed numbe o p o esso s a he Melilla Campus o he
Uni e si y o G anada who use echnopedagogical esou ces
such as IRS in hei class ooms and in eg a e hem o
p ocesses o ac i e gami ica ion me hodology, which is why
i is necessa y o p omo e and explo e hese esou ces. I is
he e o e essen ial o aise awa eness and ain uni e si y
p o esso s in he TPACK (Technological Pedagogical and
Con en Knowledge) model as a concep ual amewo k
ha can guide p o esso s when in eg a ing echnology in o
s uden s’ lea ning p ocesses, which will p omo e eme ging
me hodologies such as gami ica ion.
Secondly, imp o ing he ins umen is c ucial so ha
i can be used by p o esso s in any ield o knowledge and
in any si ua ion (as has been done du ing he COVID-19
con inemen ). The objec i e is launching an ins umen o
u u e pe iods, acco ding o he Go e nmen and uni e si y
app oaches, ha can be used in ace- o- ace aining,
B-Lea ning o E-Lea ning. The e o e, he ins umen
needs o be de eloped by including new i ems ha assess
his new i ual educa ion models. In addi ion, we mus
deepen he in luence o his ins umen depending on he
di e en disciplines o he Heal h Sciences, a esea ch
ha we in end o ca y ou in he u u e. The applica ion
o echnological ad ances in he uni e si y ield comes
on he ac i e lea ning p ocess o s uden s, and i is
necessa y o know hei opinion o achie e p og ess and
imp o emen in eaching. Thi dly, i is necessa y u he
quasi-expe imen al esea ch in he se e al ields o
uni e si y eaching o analyze he in luence o gami ied
ac i e me hodology h ough hese echnopedagogical
esou ces and adi ional me hodology, bea ing in mind
i s scope o ela ionship wi h academic pe o mance.
Conclusion
The con ibu ion o his s udy o knowledge is
he design, de elopmen and dissemina ion o a new
ins umen ha allows he measu emen and assessmen
o s uden s’ pe cep ion on he use o IRS du ing uni e si y
aining by using a echnopedagogical esou ce in any
ield o knowledge (whe he Heal h Sciences, Social and
Legal Sciences, A s and Humani ies, o Enginee ing and
A chi ec u e). The ins umen conside s h ee undamen al
ac o s: lea ning en i onmen , eaching-lea ning p ocess
and assessmen . This ques ionnai e can be applied in
he class oom du ing he aining, helping o imp o e
he eaching-lea ning p ocesses. In his sense, Heal h
Sciences p o esso s mus es ablish and explo e inno a i e
ways o in ol e s uden s and s imula e ac i e lea ning.
I is impo an o inco po a e ac i e me hods by using
in e ac i e esponse commands in disciplines such as
nu sing, medicine, pha macy, pa amedical educa ion,
psychology, den is y, physio he apy, speech he apy,
bio echnology, epidemiology, gene ics, biochemis y,
occupa ional he apy, human nu i ion and die e ics,
www.ee p.usp.b / lae
8
Re . La ino-Am. En e magem 2021;29:e3418.
among o he s. This ins umen p o ides a posi i e
pedagogical app oach in he eaching-lea ning p ocess
o Heal h Sciences p o esso s and s uden s, imp o ing
he academic pe o mance wi h he pu pose o acqui ing
a deep knowledge o he subjec s.
Re e ences
1. So illo JA, Rod ígue I, Echa E, Ojeda T. El Espacio
Ibe oame icano de Educación Supe io . Diagnós ico y
p opues as ins i ucionales. Mad id: Fundación Ca olina,
CeALCI; 2009.
2. Mingo ance AC, G anda J, Rojas G, Alemany I.
Flipped Class oom o Imp o e Uni e si y S uden
Cen e ed Lea ning and Academic Pe o mance. Soc
Sci. 2019;8(11):315-28. doi: h ps://doi.o g/10.3390/
socsci8110315
3. Ha is J, Mish a P, Koehle M. Teache s’ echnological
pedagogical con en knowledge and lea ning ac i i y
ypes: cu iculum-based echnology in eg a ion e ained.
J Res Technol Educ. [In e ne ]. 2009 [ci ed Oc 21,
2018];41(4):393-416. A ailable om: h ps:// iles.e ic.
ed.go / ull ex /EJ844273.pd
4. Cheung G, Wan K, Chan K. E icien Use o Clicke s:
A Mixed-Me hod Inqui y wi h Uni e si y Teache s. Educ.
Sci. 2018;8(31):1-15. doi: h ps://doi.o g/10.3390/
educsci8010031
5. Schube V, Medina J, Lenise M. P oceso de cons ucción
del conocimien o pedagógico del docen e uni e si a io de
en e me ía1. Re . La ino-Am. En e magem. [In e ne ].
2011 [ci ed Oc 22, 2018];19(2). A ailable om: h p://
www.scielo.b /pd / lae/ 19n2/es_26.pd
6. Cheung G, Wan K, Chan K. E icien Use o Clicke s:
A Mixed-Me hod Inqui y wi h Uni e si y Teache s. Educ.
Sci. 2018;8(31):1-15. doi: h ps://doi.o g/10.3390/
educsci8010031
7. Johnson L, Adams S, Es ada V, F eeman A. NMC
Ho izon Repo : 2016 Highe Educa ion Edi ion. [In e ne ].
Aus in: The New Media Conso ium; 2016 [ci ed Sep 15,
2018]. A ailable om: h p://www.ap ende i ual.o g/
cen o-documen acion-pd /2016-nmc-ho izon- epo -
HE-ES.pd
8. Kapsalis G, Fe a i A, Punie Y, Con ads J, Collado A,
Ho ulainen R, e al. E idence o Inno a i e Assessmen :
Li e a u e Re iew and Case S udies, EUR 29882.
Luxembou g: Eu opean Commission; 2019. doi:
h ps://doi.o g/10.2760/552774
9. Ma celo C, Yo C, Mayo C. Enseña con ecnologías
digi ales en la Uni e sidad. Comunica . 2015;45:117-24.
doi: h ps://doi.o g/10.3916/C45-2015-12
10. Hinojo FJ, Mingo ance AC, T ujillo JM, Azna I, Cáce es
MP. Incidence o he Flipped Class oom in he Physical
Educa ion S uden s’ Academic Pe o mance in Uni e si y
Con ex s. Sus ainabili y. 2018;10(5):1334-46. doi:
h ps://doi.o g/10.3390/su10051334
11. Njie-Ca VP, Ludeman E, Lee MC, Do dunoo D,
T ocky NM, Jenkins LS. An In eg a i e Re iew o Flipped
Class oom Teaching Models in Nu sing Educa ion.
J P o Nu s. 2017;33(2):133-44. doi: 10.1016/j.
p o nu s.2016.07.001
12. Wilson J, Walke S. Tu ning a c isis in o an in e ac i e
d ama: he in oduc ion o a ‘clicke s hea e’ in nu se
educa ion. Nu se Educ Today. 2017;51:109-11. doi:
10.1016 / j.ned .2016.11.023
13. Ca ino SS. Digi al connec ion in a physical class oom:
Clicke s and he s uden - eache ela ionship [ hesis].
G eensbo o: Uni e si y o No h Ca olina; 2015 [ci ed
Dec 12, 2019]. A ailable om: h ps://lib es.uncg.edu/
i /uncg/ /Ca ino_uncg_0154D_11791.pd
14. Zainuddin Z, Halili SH. Flipped class oom esea ch
and ends om di e en ields o s udy. In Re Res
Open Dis ance Lea n. [In e ne ]. 2016 [ci ed Jul 18,
2018];17(3):313-40. A ailable om: h ps:// iles.e ic.
ed.go / ull ex /EJ1102721.pd
15. Ali RA, Alna ou A, Alnuaimi K, Alzoubi F, Almomani
M, O hman A. E ec s o in e ac i e eaching on uni e si y
s uden s’ knowledge and a i ude owa d ep oduc i e
heal h: a pilo s udy in Jo dan. J Mul idiscip Heal hc.
2018;11:211-21. doi: h ps://doi.o g/10.2147/JMDH.
S160135
16. Ba celo JM. Medical labo a o y science and nu sing
s uden s’ pe cep ion o he academic lea ning en i onmen
a a Philippine uni e si y using he Dundee Ready
Educa ion En i onmen Measu e. J Educ E al Heal h
P o . 2016;13(33):1-7. doi: h ps://doi.o g/10.3352/
jeehp.2016.13.33
17. B ady M, Seli H, Rosen hal J. “Clicke s” and
me acogni ion: a quasi-expe imen al compa a i e s udy
abou me acogni i e sel - egula ion and use o elec onic
eedback de ices. Compu Educ. 2013;65:56-63. doi:
h ps://doi.o g/10.1016/j.compedu.2013.02.001
18. Cas o MJ, López M, Cao MJ, Fe nández-Cas o M,
Ga cía S, F u os M, e al. Impac o educa ional games
on academic ou comes o s uden s in he Deg ee in
Nu sing. PLoS One. 2019;14(7):e0220388. doi: h ps://
doi.o g/10.1371/jou nal.pone.0220388
19. Nishimu a A. E ec s o di e en me hods o e lec ion
on nu ses’ gaze and judgemen in a ask using a ouch
panel. J Clin Nu s. 2018;27:569-77. doi: h ps://doi.
o g/10.1111/jocn.14096
20. Too hake R. Millennial’s pe spec i e o clicke
echnology in a nu sing class oom: A mixed me hods
esea ch s udy. Nu se Educ Today. 2018;62:80-4. doi:
h ps://doi.o g/10.1016/j.ned .2017.12.027
21. Chen TL, Lan YL. Using a pe sonal esponse sys em as
an in-class assessmen ool in he eaching o basic college
www.ee p.usp.b / lae
9
Mingo ance-Es ada AC, G anda-Ve a J, Rojas-Ruiz G, Alemany-A ebola I.
chemis y. Aus alas J Educ Technol. 2013;29(1):32-40.
doi: h ps://doi.o g/10.14742/aje .95
22. Ak ekin NÇ, Çelebi H, Ak ekin M. Le ’s kahoo !
Ana omy. In . J. Mo phol. 2018;36(2):716-21. doi: h p://
dx.doi.o g/10.4067/S0717-95022018000200716
23. Ba ML. Encou aging college s uden ac i e
engagemen in lea ning: The in luence o esponse
me hods. Inno High Educ. 2013;39(4):307-19. doi:
h ps://doi.o g/10.1111/jcal.12205
24. Iskande M. Sys ema ic e iew o he implemen a ion
o audience esponse sys ems and hei impac on
pa icipa ion and engagemen in he educa ion o
heal hca e p o essionals. BMJ Simula ion & Technology
Enhanced Lea ning. 2018;4(2):47-50. doi: h p://dx.doi.
o g/10.1136/bmjs el-2017-000245
25. Lico ish SA, Owen HE, Daniel, B, Geo ge, JL. S uden s’
pe cep ion o Kahoo !’s in uence on eaching and lea ning.
RPTEL. 2018;13(9):1-23. doi: h ps://doi.o g/10.1186/
s41039-018-0078-8
26. Oli ei a C, Ti apelli C, Rod igues CT, Domaneschi C,
Caldei a SA. In e ac i e audience esponse sys ems in
o al and maxillo acial adiology unde g adua e lec u es.
Eu J Den Educ. [In e ne ]. 2017;22:63-9. doi: h ps://
doi.o g/10.1111/eje.12258
27. Mohan R. Enhancing s uden engagemen and
immedia e eedback wi h clicke s and esponse ca ds.
In J Inno Lea n. 2018;24(1):81-97. doi: h ps://doi.
o g/10.1504/IJIL.2018.092924
28. Banks D. Re lec ions on he use o ARS wi h small
g oups. In: ______. Audience Response Sys ems in Highe
Educa ion: Applica ions and Cases. Pennsyl ania: IGI
Global; 2006. p. 373-86.
29. Wood A. Nu se Pe cep ions o In e ac i i y du ing Thei
Onboa ding O ien a ion: E ec o an Audience Response
Sys em [ hesis]. Boiling Sp ings: Hun School o Nu sing;
2017 [ci ed Dec 12, 2019]. A ailable om: h ps://
digi alcommons.ga dne -webb.edu/cgi/ iewcon en .cgi
?a icle=1268&con ex =nu sing_e d
30. Ismaile S, Alhosban F, Hawamdeh S. Making lea ning
un o inc ease nu sing s uden s’ success: Fo ma i e
eedback in communica ion lea ning. Aus alas Med J.
2017;10(12):1014-21. doi: h ps://doi.o g/10.21767/
AMJ.2017.3228
31. Cas o MJ, López M, Cao MJ, Fe nández-Cas o M,
Ga cía S, F u os M, e al. Impac o educa ional games
on academic ou comes o s uden s in he Deg ee in
Nu sing. PLoS One 2019;14(7):e0220388. doi: h ps://
doi.o g/10.1371/jou nal.pone.0220388
32. Co ell A, Regue as LM, Ve dú E, Ve dú MJ, Cas o JP.
E ec s o compe i i e lea ning ools on medi- cal s uden s:
A case s udy. PLoS One. 2018;13(3):e0194096. doi:
h ps://doi.o g/10.1371/jou nal.pone.0194158
33. Oswald KM, Rho en SE. Imp o ing class oom
clicke p ac ices: E ec s o incen i es and eedback on
e en ion. N Am J Psychol. [In e ne ]. 2014 [ci ed Dec
6, 2017];16(1):79-88. A ailable om: h ps://www.
esea chga e.ne /publica ion/285958781_Imp o ing_
class oom_clicke _p ac ices_E ec s_o _incen i es_and_
eedback_on_ e en ion
34. Ma shall LL, Va non AW. An Empi ical In es iga ion
o Clicke Technology in Financial Accoun ing P inciples.
J Lea n High Educ. [In e ne ]. 2012 [ci ed Jun 3,
2018];8(1):7-18. A ailable om: h ps:// iles.e ic.ed.go /
ull ex /EJ1144930.pd
35. Bojino a ED, Oiga a JN. Teaching and lea ning wi h
Clicke s: A e Clicke s good o s uden s? In e discip J
E-Lea ning Lea n Objec s. 2011;7:169-83. doi: h ps://
doi.o g/10.28945/1506
36. Benson JD, Szucs KA, DeIuliis E, Le i A. Impac o
S uden Response Sys ems on Ini ial Lea ning and Re en ion
o Cou se Con en in Heal h Sciences S uden s. J Allied
Heal h. [In e ne ]. 2017 [ci ed Feb 12, 2018];46(3):158-
63. A ailable om: h ps://www.ingen aconnec .com/
con en /asahp/jah/2017/00000046/00000003/a 00008
37. Njie-Ca VP, Ludeman E, Lee MC, Do dunoo D,
T ocky NM, Jenkins LS. An In eg a i e Re iew o Flipped
Class oom Teaching Models in Nu sing Educa ion.
J P o Nu s. 2017;33(2):133-44. doi: 10.1016/j.
p o nu s.2016.07.001
38. Rana NP, Dwi edi YK. Can clicking p omo e lea ning?
measu ing s uden lea ning pe o mance using clicke s in
he unde g adua e in o ma ion sys ems class. J In Educ
Bus. 2017;10(2):201-15. doi: h p://doi.o g/10.1108/
JIEB-06-2016-0010
39. Iwamo o D, Ha gis J, Tai ano E, Vuong K. Analyzing he
e icacy o he es ing e ec using kahoo m on s uden
pe o mance. Tu kish Online J Dis ance Educ. [In e ne ].
2017;18(2):93-80. doi: h ps://doi.o g/10.17718/
ojde.306561
40. Geo ge C, Gallegos C, Tesa AJ, Conno K, Ma z K. The
use o a game-based lea ning pla o m o engage nu s-
ing s uden s: A desc ip i e, quali a i e s udy. Nu se Educ
P ac . 2017;27:101-6. doi: h ps://doi.o g/10.1016/j.
nep .2017.08.019.
41. Kalaian SA, Kasim RM. E ec i eness o a ious
inno a i e lea ning me hods in heal h science class-
ooms: a me a-analysis. Ad Heal h Sci Educ Theo y
P ac . [In e ne ]. 2017;22(5):1151-67. doi: h ps://doi.
o g/10.1007/s10459-017-9753-6
42. Maloney LM, Dilge JP, We el PA, Cimino LM. A e
Audience Response Sys ems Wo h he Cos ? Compa ing
Ques ion-D i en Teaching S a egies o Eme gency
Medical Technician Educa ion. In e ne J Allied Heal h
Sci P ac . [In e ne ]. 2017 [ci ed May 7, 2018];16(1):1-