P oo s
Impac o a Gamifica ion Lea ning Sys em on he Academic
Pe o mance o Mechanical Enginee ing S uden s*
ROSA PA
`MIES-VILA
`
Depa men o Mechanical Enginee ing, Uni e si a Poli e
`cnica de Ca alunya, A . Diagonal, 647, 08028, Ba celona, Ca alonia, Spain.
E-mail: [email p o ec ed]
ALBERT FABREGAT-SANJUAN
Depa men o Mechanical Enginee ing, Uni e si a Ro i a i Vi gili, A . Paı¨sos Ca alans, 26, 43007, Ta agona, Ca alonia, Spain.
E-mail: [email p o ec ed]
JOAN PUIG-ORTIZ and LLUI
¨SA JORDI NEBOT
Depa men o Mechanical Enginee ing, Uni e si a Poli e
`cnica de Ca alunya, A . Diagonal, 647, 08028, Ba celona, Ca alonia, Spain.
E-mail: [email p o ec ed], [email p o ec ed]
ANTONI HERNA
´NDEZ FERNA
´NDEZ
Ins i u e o Educa ion Sciences, Uni e si a Poli e
`cnica de Ca alunya, Plac¸a Eusebi Gu
¨ell, 6, 08034, Ba celona, Ca alonia, Spain.
E-mail: [email p o ec ed]
This s udy examines he effec s o using a gamifica ion ool as a eaching s a egy. Specifically, Kahoo ! is e alua ed as a
ool o enhancing s uden lea ning. The ac i i ies we e pa o he labo a o y sessions o he subjec Mechanism and
Machine Theo y du ing wo consecu i e academic yea s. We analyze he effec o a gamifica ion lea ning sys em on bo h,
s uden s’ g ades and mo i a ion, in a cou se wi h a la ge numbe o s uden s (n
1
= 283 s uden s, n
2
= 306 s uden s). The
s uden s we e di ided in o h ee diffe en g oups (con ol g oup, gamifica ion g oup and w i ing g oup) and hei esul s
we e e alua ed depending on he lea ning me hod applied du ing he class. In e ms o gamifica ion, his p ojec
in oduces eal- ime eedback o s imula e he in e es o s uden s and help hem use he ypical ools and me hodologies
o game-based lea ning. The analysis o hei pe o mance in he labo a o y exam shows significan diffe ences be ween
he g oup ha used gamifica ion and he g oups ha did no . The esul s sugges ha gamifica ion in enginee ing lab
ac i i ies has a posi i e effec on s uden s’ mo i a ion and lea ning ou come. The s udy concludes ha game-based
elemen s and compe i i e ac i i ies enhanced s uden pe o mance.
Keywo ds: gamifica ion; game-based lea ning; highe educa ion; mechanical enginee ing
1. In oduc ion
The combina ion o eaching and games can be
aced back o he humanis ic app oach, bu in
ecen yea s game design elemen s ha e s a ed o
be used o non-play ul pu poses [1]. Al hough he
e m is s ill being e ised concep ually – see [2] o a
heo e ical e iew – gamifica ion can be defined as
using game-based mechanics, aes he ics and hink-
ing o engage people, mo i a e ac ion, p omo e
lea ning, and sol e p oblems [3]. In educa ion, he
idea is o mo i a e and s imula e s uden s by using
ac i i ies o he han adi ional ones, and acili a e
– almos wi hou hem being awa e – eaching-
lea ning i sel , especially in a social con ex in
which s uden engagemen needs o be inc eased [4].
The e is a b oad deba e among game designe s,
esea che s and educa o s, abou wha games a e,
how hey impac indi iduals and, in gene al, how
hey can be used in class ooms. Insufficien a en-
ion has been paid o gamifica ion g ounded in bo h
heo ies and e idence om empi ical s udies [5]. In
his ega d, [6] desc ibes he ad an ages and dis-
ad an ages o gamifica ion Among he ad an ages,
he says, a e ha games and gamifica ion can lead o
high le els o lea ne engagemen and mo i a ion
since hey connec wi h he skills o 21s -cen u y
s uden s [7]. On he o he hand, he e is a isk o
applying ep oduc ion wi hou p io design, esul -
ing in p oblems such as exploi a ion o he c ea ion
o hos ile and ense en i onmen s. Gamifica ion
models in educa ion domain could help gamifica-
ion p ac i ione s o make new s a egies in lea ning
ac i i ies o inc ease s uden s’ mo i a ion, achie e-
men and in ol emen [8]. Rigo ous s udies a e
equi ed o ully examine he effec s o gamifica ion
and de e mine how lea ning is bes achie ed [9].
In gene al, gamifica ion echniques ha e posi i e
effec s on he in ol emen and mo i a ion o s u-
den s [4, 10, 11], – see [12] o scoping e iew –.
S uden s alue i s compe i i e na u e, he immedi-
acy o eedback on hei knowledge and s uc u ed
oppo uni ies o u he discussion [13] and hey
also iden i y gamifica ion as a mul i ace ed ool o
a g ea lea ning expe ience [14]. Gamified lea ning
en i onmen s con ibu e o he lea ning and each-
* Accep ed 10 July 2022.1434
IJEE 4266 PROOFS
In e na ional Jou nal o Enginee ing Educa ion Vol. 38, No. 5(A), pp. 1434–1442, 2022 0949-149X/91 $3.00+0.00
P in ed in G ea B i ain #2022 TEMPUS Publica ions.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
ing p ocess by aising le els o engagemen , c ea ing
enjoyable lea ning en i onmen s and ensu ing
ac i e pa icipa ion [11, 15–20]. Howe e , some
s udies ha e no iden ified any significan effec s
on lea ning o ha e e en de ec ed wo se academic
esul s when s uden s a e o ced o use game ele-
men s [21–23]. Howe e , all e iews o da e ag ee
ha he e is insufficien e idence o suppo he
long- e m benefi s o gamifica ion in educa ional
con ex s [1, 13, 17, 24, 25], so mo e empi ical
e idence is needed o jus i y ha gamifica ion is
be e han o he pedagogical al e na i es [26, 27].
The e o e, he p esen s udy aims o p o ide new
e idence on he effec s o gamifica ion in he class-
oom.
1.1 Gamifica ion wi h Pe sonal Response De ices
Pe sonal esponse de ices (PRDs) – some imes
called class oom esponse sys ems, s uden
esponse sys ems, o audience esponse sys ems –
appea ed a he beginning o he 21s cen u y and
hey consis o an emi e and a ecei e ha ,
oge he wi h he co esponding so wa e, enable
eache s o ask hei s uden s a mul iple-choice
ques ion ( he ques ion is p ojec ed on a sc een)
and s uden s o send an answe using hei indi i-
dual con ol o clicke [28, 29]. Clicke s p o ide a
simple way o gene a e an a mosphe e o s uden
in e ac ion ha can enhance eache -s uden com-
munica ion [30].
Se e al PRDs can be used wi h iPads, And oid
able s, mobile phones and compu e s. These new
sys ems ha e he same u ili ies as o he as
esponse me hods such as clicke s, none o he
co esponding echnical-logis ical p oblems, and
new ea u es like gaming elemen s, music, modal-
i ies, and design [28, 31] Usually, he in eg a ion o
his de ices do no p esen echnical difficul ies and
gaming is success ul in enabling ac i e pa icipa ion
and in e ac i e lea ning [13]. Some o hese applica-
ions a e Men ime e , In use Lea ning, Soc a i e,
Quiz Socke , Kahoo !, Ve so, Poll E e ywhe e o
VoxVo e, which enable you o p epa e mul iple-
choice ques ionnai es, ue/ alse ques ionnai es
and, in some cases, ques ions wi h sho answe s
(i.e. Soc a i e). Wi h hese ools, s uden s answe all
ques ions simul aneously in class, da a is collec ed
and s a is ics on he esponses o he s uden s a e
gi en immedia ely. The iming is p og ammed by
he eache s, who can de ec common e o s, high-
ligh aspec s ha a e mos deficien o he s uden s
and p o ide immedia e eedback [32].
The indi idual esponse sys em c ea es an en i-
onmen o immedia e in e ac i e lea ning and
discussion in he class oom [33, 34]. I also p o ides
o ma i e eedback on lea ning ( o bo h eache s
and lea ne s) [35]. Howe e , he benefi s o using
s uden esponse sys ems a e also con o e sial
[36, 37]. While he e is conside able e idence o
sugges ha uni e si y s uden s ha e e y posi i e
opinions abou he use o hese sys ems [28, 30, 33],
some s udies conclude ha hese ools do no
gua an ee be e lea ning [35, 38]. I seems ha i
is he implemen a ion o pedagogical s a egies in
combina ion wi h he echnology ha ul ima ely
influences s uden success.
S udies on indi idual esponse sys ems o en
compa e he s a is ics and eedback gi en wi h he
s uden ’s final g ade. The co ela ions a e o en
posi i e bu weak, which shows ha hey can be
use ul o o ma i e assessmen bu no o sum-
ma i e [30, 35, 39].
Kahoo ! is a ee i ual ool ha has gained in
popula i y among eache s o i s use - iendly
na u e and i s abili y o es ablish wo king dynamics
in he class oom. I is highly app ecia ed by s u-
den s [28]. Kahoo ! allows eache s o c ea e su -
eys, ques ionnai es, puzzles and deba es, and
ob ain s uden s’ answe s in eal ime. Va ious
s udies on Kahoo ! ag ee ha his ool imp o es
pa icipa ion and he posi i e ela ionship be ween
class membe s [13, 39–41].
1.2 Case S udy in Mechanical Enginee ing
Mechanism and Machine Theo y is a co e subjec
augh in he ou h semes e o he Deg ee in
Indus ial Enginee ing a Uni e si a Poli e
`cnica
de Ca alunya. I is one o he fi s imes ha he
Indus ial Enginee ing s uden s ha e come in o
con ac wi h he wo ld o mechanical enginee ing.
The o ma i e assessmen s om p e ious semes e s
showed ha he s uden s did no acqui e he
equi ed skills a labo a o y classes: he pe cen age
o s uden s who passed he labo a o y exam was
e y low, and he eache s conside ed i a p oblem
since i sugges s ha s uden s we e no able o pu
in o p ac ice he knowledge hey had acqui ed in
he heo y classes. On a e age, he pe cen age o
s uden s passing he cou se is 70%, whe eas he
pe cen age passing he p ac ical examina ions is
40%, no ably lowe . The e o e, a new me hod was
needed in o de o imp o e he eaching/lea ning
p ocess.
We hypo hesize ha he in oduc ion o gamified
eedback will help o highligh he mos impo an
concep s a he end o each labo a o y session, and
he e o e, imp o e he lea ning p ocess. Ano he
impo an issue is which o he ac o s in ol ed in
gamifica ion is mos ela ed o he imp o emen in
lea ning (i indeed he e is an imp o emen ). To
add ess his issue using a mul i ac o ial app oach
[42], we diffe en ia ed wo ac o s: he ac ha
s uden s ecei e eedback ( his will be checked by
a con ol g oup doing w i en es s) and he ac
Impac o a Gamifica ion Lea ning Sys em on he Academic Pe o mance o Mechanical Enginee ing S uden s 1435
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
ha gamifica ion p omo es o he a iables (mo i-
a ion, engagemen , compe i i eness, e c.).
Fo his eason, he second hypo hesis o his
s udy is ha he fi s o hese wo ac o s (gamifica-
ion) is mo e impo an han he second ( he eed-
back i sel ). In o de o es his hypo hesis, he
ques ions ha he s uden s in he gamifica ion
g oup we e asked we e also p esen ed o ano he
expe imen al g oup in which s uden s did a w i en
es wi hou using Kahoo !. The solu ions o he es
( eedback) we e also p o ided a e he labo a o y
session.
2. Me hods
The p esen s udy uses an empi ical-analy ical
me hodology o s udy gamifica ion as a ool in
labo a o y sessions. The subjec Theo y o
Machines and Mechanisms has a la ge numbe o
s uden s each semes e (be ween 270 and 320) so he
s uden s we e andomly dis ibu ed in o 11 labo a-
o y g oups augh by 4 diffe en lec u e s. The aim
o ou in e en ion was o imp o e lea ning in he
labo a o y sessions.
The o e all cou se g ade is calcula ed acco ding
o he ollowing weigh ed a e age, ounded o one
decimal place:
Mcou se ¼Maxð0:6M e þ0:2Mpe;0:8M eÞ þ
0:10 Mlab1 þ0:10ðMlab2 MsimÞ1=2ð1Þ
whe e, Mcou se is he final g ade o he cou se, M e is
he ma k o he final exam, Mpe is he ma k o he
mid e m exam, Mlab1 is he ma k o he fi s
labo a o y exam (assessing sessions 1, 2 and 3),
and Mlab2 is he ma k o he second labo a o y
exam (assessing sessions 4 and 5). Finally, Msim is
he ma k o a simula ion exe cise.
The in e en ions ook place du ing he second
e m o he academic yea s 2016–17 and 2017–18
and aimed o imp o e he ma ks o 3 labo a o y
sessions which accoun o 10% o he final g ade.
A es ques ionnai e has been in oduced as a
eedback ool. Quick eedback helps s uden s
become awa e, and hey ha e g ea e pe cep ion
o wha has happened in he labo a o y. This eed-
back has been in oduced as a es ques ionnai e
ha has o be answe ed in he las 15–30 minu es o
each session.
Two diffe en eedbacks a e analyzed. The fi s
uses Kahoo ! ques ionnai es. Since Kahoo ! is a as
esponse sys em o he s uden , i is expec ed o be
effec i e a imp o ing knowledge e en ion and skill
acquisi ion. The second uses a adi ional ques ion-
nai e which, he e o e, in ol es no compe i ion o
coope a i e lea ning. To de e mine he effec o
in oducing no only a eedback ool bu a eedback
gamifica ion ool, he labo a o y g oups we e
di ided in o h ee g oups:
An expe imen al g oup gi en eedback h ough
he Kahoo ! ques ionnai es– (Gami ica ion
g oup, GG). These lea ne s use he mobile e -
sion o he app.
An expe imen al g oup gi en a w i en es a he
end o he session (wi h he same ques ions as in
Kahoo !), ac ing as ein o cemen and eedback,
bu wi hou he o he componen s ha Kahoo !
may ha e (W i ing g oup, WG).
A con ol g oup subjec o no in e en ion (Con-
ol g oup, CG).
The s uden s we e di ided up in his way o a oid
eache and ime able ac o s. Table 1 summa izes
he numbe o s uden s in each g oup. No e ha
some s uden s do no pa icipa e in he labo a o y
sessions.
Academic pe o mance was assessed by compa -
ing he ma ks o s uden s in each o he pedagogical
g oups. The mean ma k, s anda d de ia ion and
numbe o s uden s who passed he exam we e
calcula ed o each e alua ion (M
lab1
,M
lab2
,M
pe
,
M
sim
,M
e
). A S uden ’s T-Tes was also used o find
significan diffe ences be ween he expe imen al
(GG and WG) and he con ol (CG) g oups.
The e o e, o he GG he ela ion be ween he
Kahoo ! es sco e and he g ades in he o he
e alua ions was s udied. Likewise, o he WG,
he ela ion be ween he w i ing es sco e and he
g ades in o he e alua ions was examined. To his
end, linea co ela ions we e calcula ed and Pea -
son, Spea man and Kendall coefficien s de e -
mined.
Finally, whe he o no he e was a eache effec
was s udied ( ha is o say, whe he a pa icula
s uden ge s a be e o a wo se ma k depending on
he eache who has augh he subjec ). The e o e,
he s uden s we e g ouped acco ding o he lec u e
who augh he sessions and a S uden ’s T-Tes was
used o de e mine significan diffe ences be ween
he ou g oups.
Finally, i was no conside ed app op ia e o
measu e success only by compa ing he summa i e
ma ks, because his is no he only pu pose o he
gamifica ion ool. A 12-ques ion su ey was p e-
Rosa Pa
`mies-Vila
`e al.1436
Table 1. O e all numbe o s uden s o each g oup and academic
yea
Numbe o s uden s 2016–17 2017–18
Gamifica ion G oup – GG 37 41
W i ing G oup – WG 115 86
Con ol G oup – CG 113 100
No a ending 41 56
To al 306 283
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
pa ed o s uden s in he gamifica ion g oup (See
he Appendix).
3. Resul s
Because eedback is now a pa o labo a o y
sessions 1, 2 and 3, diffe ences in he labo a o y
exam 1 ma ks (M
lab1
) can be expec ed among he
h ee g oups. Du ing he academic yea 2016–17,
62.16% o he s uden s who ook pa in he
gamifica ion passed he exam while only 54.87%
o he con ol g oup and 58.26% o he w i ing
g oup did he same. Simila ly, du ing he academic
yea 2017–18, 87.80% o he s uden s who ook pa
in he gamifica ion passed he exam while in he
con ol g oup and he w i ing g oup he pe cen-
ages we e 74.74% and 77.91%, espec i ely.
Table 2 shows he mean and s anda d de ia ion
o he ma ks o each e alua ion (M
lab1
,M
lab2
,M
pe
,
M
sim
,M
e
) o bo h o he academic yea s analyzed.
I can be seen ha o labo a o y exam 1 (M
lab1
),
he mean g ade ob ained by he s uden s who ook
pa in he gamifica ion sessions (5.59 2.43,
academic yea 2016–17; and 6.90 1.68, academic
yea 2017–18) is mo e han one poin highe han
he con ol g oup (4.50 2.17, academic yea 2016–
17; and 5.75 2.30, academic yea 2017–18).
Howe e , his diffe ence is no so clea o he
w i ing g oup (4.71 2.37, academic yea 2016–
17; and 5.57 2.24, academic yea 2017–18). A
S uden ’s T- es be ween GG and CG demon-
s a ed ha he e is a significan diffe ence be ween
hese wo g oups (p- alue < 0.05). Mo eo e , he
diffe ences be ween he WG and he CG g oup a e
no s a is ically significan .
Fig. 1 shows he boxplo ob ained o M
lab1
o
bo h academic yea s and o each eaching me ho-
dology. The cen al block is delimi ed by he posi-
ion o Q1 and Q3 qua iles and he line ep esen ing
he median is d awn in he box. I can be seen ha
he median is also highe o he gamifica ion g oup
han o he w i ing and con ol g oups.
Diffe ences be ween GG, CG and WG a e no
p esen ed o he o he e alua ion ma ks (M
lab2
,
Impac o a Gamifica ion Lea ning Sys em on he Academic Pe o mance o Mechanical Enginee ing S uden s 1437
Table 2. Mean SD o he ma ks o each e alua ion
G oup M
lab1
M
lab2
M
pe
M
sim
M
e
Academic yea
2016–17
GG 5.59 2.43* 4.22 2.97 5.24 2.34 7.16 2.16 2.88 1.82
WG 4.71 2.37 3.69 2.86 5.57 2.24 7.13 1.76 3.12 1.62
CG 4.50 2.17 3.49 2.54 5.53 2.37 7.06 1.84 2.90 1.87
Academic yea
2017–18
GG 6.90 1.68* 4.94 2.36 5.90 2.76 7.06 2.23 4.94 2.30
WG 6.08 2.23 5.56 2.59 6.28 2.50 7.14 2.23 4.83 2.10
CG 5.75 2.30 4.95 2.78 5.92 2.11 7.26 2.27 4.32 2.16
*p- alue < 0.01.
Fig. 1. Boxplo o labo a o y exam 1 ma ks. (a) Academic yea 2016–17 (b) Academic yea 2017–18.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
M
pe
,M
sim
,M
e
) (Table 2): he p- alues a e g ea e
han 0.05 and he e o e s a is ical diffe ences
canno be assumed. This shows ha s uden s a e
andomly dis ibu ed among g oups. Diffe ences
only appea when a gamifica ion me hodology is
applied.
Th ee diffe en co ela ion coefficien s (Pea son,
Spea man and Kendall) and he p- alues o he
s a is ical es s we e calcula ed. Fo GG, he p-
alues we e much lowe han 0.05 (Table 3), so
he e is a significan posi i e co ela ion be ween
he ma ks ob ained in he Kahoo ! es and he ones
ob ained in he labo a o y 1 exam (co ela ions
be ween 0.6 and 0.77 depending on he indica o
used). Howe e , he co ela ions o he w i ing
g oup a e eally low and hey a e no significan
(see Table 3).
Fu he mo e, he ela ion be ween he eedback
es s and he g ades ob ained in he o he e alua-
ions we e s udied using he same coefficien s. The
co ela ions in hese cases we e poo (0.180-0.337)
and non significan .
Fig. 2 shows, he ela ionship be ween he calcu-
la ed Kahoo ! g ades (M
Kahoo !
) and he g ades
ob ained by s uden s on he labo a o y exam
1(M
lab1
) o he gamifica ion g oup (GG). The
g aphs also show he polynomial eg ession line
ha adjus s hese alues and he co esponding R
2
pa ame e .
The eache effec was also analysed. As
explained abo e, he sessions a e augh by ou
diffe en lec u e s. Fo his analysis, s uden s we e
g ouped acco ding o he lec u e who augh hei
labo a o y session. Howe e , he S uden ’s T- es
does no de ec any significan diffe ences be ween
he ou g oups s udied (p- alue > 0.05). The e o e,
i canno be affi med ha he eaching s aff has an
effec on he g ades o he s uden s.
Finally, he opinion poll shows whe he s uden s
see gamifica ion as an imp o emen in hei lea n-
ing p ocess and hei mo i a ion. The esul s o
bo h cou ses we e simila . They a e p esen ed in
Fig. 3 whe e he do ed blocks ep esen a posi i e
answe (s ongly ag ee o ag ee) and lined blocks
Rosa Pa
`mies-Vila
`e al.1438
Table 3. Co ela ion coefficien s be ween eedback es and labo a o y exam 1. M
Kahoo !
and M
lab1
o gamifica ion g oup and M
WT
and
M
lab1
o w i ing g oup.
Coefficien
Academic yea 2016–17 Academic yea 2017–18
Value (p- alue) Value (p- alue)
Gamifica ion G oup Pea son 0.726 (3.63 10
–7
) 0.774 (2.92 10
–9
)
Spea man 0.754 (7.34 10
–8
) 0.709 (2.13 10
–7
)
Kendall 0.603 (7.32 10
–7
) 0.615 (2.74 10
–7
)
W i ing G oup Pea son –0.010 (0.913) –0.101 (0.357)
Spea man 0.003 (0.970) –0.174 (0.110)
Kendall 0.003 (0.965) –0.136 (0.084)
Fig. 2. Sca e g aph o he lab exam 1 ma ks (M
lab1
) e sus he Kahoo ! ma ks (M
Kahoo !
). (a) Academic yea 2016–17, (b) Academic yea
2017–18.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

ep esen a nega i e answe (disag ee o s ongly
disag ee)..
In gene al, Kahoo ! has been e y well accep ed.
On he basis o he answe s o he fi s h ee
ques ions, i can be said ha mo e han 90% o
he s uden s hink ha gamifica ion has helped
hem o unde s and he subjec which, in u n,
makes hem mo e mo i a ed. Mo eo e , s uden s
confi m ha he ime spen on he ac i i y is offse
by imp o ed lea ning (Q3).
In e ms o wha he Kahoo ! esul s demons a e
(Q4), s uden s a e di ided: some ag ee ha hey a e
p oo o knowledge acqui ed, bu o he s do no .
Simila esul s we e ob ained o Q5 and Q7. F om
hese esul s, i is difficul o see i enough ime was
gi en o answe he Kahoo ! ques ions and i
Kahoo ! ques ionnai es make hem mo e a en i e
o he labo a o y session.
All s uden s affi m ha Kahoo ! ques ionnai es
will be posi i e in o he subjec s (Q6) and hey all
ag ee ha discussion a e hey ha e gi en hei
answe s allows hem o cla i y concep s (Q12).
Mos o hem posi i ely e alua e he eedback
hey ge h ough hese quizzes (Q11).
Acco ding o 80% o he s uden s, i Kahoo !
g ades we e added o he summa i e e alua ion
g ade hey would pay mo e a en ion in class
(Q8). A simila pe cen age pe cei es he compe i-
i eness c ea ed by Kahoo ! as a posi i e s imulus o
lea ning (Q10).
4. Discussion
The e is a lack o esea ch on he eal effec s o
gamifica ion on he lea ning p ocess and whe he
hese effec s a e be e han hose ob ained wi h
adi ional app oaches [43]. This pape p esen s an
expe imen al s udy wi h 589 s uden s en olled. As
has been men ioned, he e was a need o elucida e
whe he Kahoo ! is effec i e o no . The main
objec i e was o con as i gamifica ion h ough
game-based s uden esponse sys ems imp o es
ac i e s uden lea ning, pa icipa ion and e en ion
o concep s [30, 44], o on he con a y, i is no
gua an ee o be e lea ning [35, 38]. So, he esul s
p esen ed he e can be ega ded as a pilo es which
shows ha game-based s uden esponse sys ems
(Kahoo ! he e) can imp o e academic pe o mance.
In addi ion, his s udy also con ibu es o e alua e i
bo h s uden s and eache s hink ha gamifica ion
is s imula ing, e ealing, mo i a ing and, in essence,
un [28].
Resul s show ha he gamifica ion g oup (GG)
had a highe success a e in he labo a o y exam
(Labo a o y exam 1) han he con ol g oup (CG).
Mo eo e , on his e alua ion, he a e age g ade o
GG s uden s was s a is ically g ea e han he
a e age o CG s uden s. Fu he mo e, he g ades
o he o he e alua ions do no show hese diffe -
ences. I can be seen ha gamifica ion has a posi i e
effec on g ades as [30, 44] sugges ed. No e ha he
Impac o a Gamifica ion Lea ning Sys em on he Academic Pe o mance o Mechanical Enginee ing S uden s 1439
Fig. 3. Resul s o he opinion polls o he 12 ques ions. (a) Academic yea 2016–17, (b) Academic yea 2017–18.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
w i ing g oup shows no significan imp o emen
wi h espec o he con ol g oup. When he eed-
back is no gamified, i does no enhance academic
esul s. These esul s sugges ha gamifica ion is he
key o he imp o emen no he eedback i sel .
No significan diffe ences we e de ec ed in he
g ades o he a ious g oups who did labo a o y
exam 1. This e eals ha i was no he lec u e o
he labo a o y session who ma ked he diffe ence
bu he in e en ion i sel . Tha sugges ha he
gamifica ion is he key, no he lec u e .
The esul s o he opinion poll show ha he
s uden s alue he in e en ion posi i ely, as o he
s udies ha e poin ed ou [28]. All o he s uden s
s a ed ha he discussion a e hey had gi en hei
esponses cla ified concep s, and mos o hem el
ha gamifica ion helped hem o unde s and he
subjec be e and mo i a ed hem. These esul s a e
also in ag eemen wi h he li e a u e [15–18]: gami-
fica ion in ol es mo i a ion.
5. Conclusions
The main goal o his s udy was o analyse whe he
a gamifica ion ool could imp o e academic pe o -
mance and mo i a ion in he labo a o y sessions o
he subjec Mechanism and Machine Theo y. Fo
his pu pose, du ing wo consecu i e academic
yea s, we di ided he s uden s in o h ee g oups
and each g oup had diffe en me hodological in e -
en ions. A he end o he fi s h ee sessions, he
gamifica ion g oup (GG) answe ed a Kahoo ! ques-
ionnai e; he w i ing g oup (WG) answe ed he
same ques ionnai e bu on pape , and he con ol
g oup (CG) did no ake any ques ionnai e.
In he ligh o he esul s p esen ed, in gene al i
can be concluded ha gamifica ion has p o ided a
(modes ) inc ease in he eaching-lea ning p ocess
in he labo a o y sessions o he subjec Mechanism
and Machine Theo y wha i is so consis en ly wi h
he heo e ical easoning ha mo i a ed his p o-
posal.
Howe e , his s udy has se e al limi a ions and
u he esea ch will be equi ed. One limi a ion o
he s udy is ha we a e no co e ing how gende
diffe ences influenced he effec s o gamifica ion.
Gende and pe sonali y could affec s uden s’ pe -
cep ion owa d gamifica ion ac i i ies. In addi ion,
we ha e only use one in e ace in gamifica ion. In
u u e, simila applica ions should be es ed o
compa e he ob ained esul s. Addi ionally, he e
a e diffe en ways o gamified eedback mechanisms
such as poin s, badges, ewa d, le els, e c. ha a e
no co e ed in his s udy. Mo e design in es iga ion
is equi ed o u he gene alize he esul s.
The s udy has p esen ed a me hodology ha
allow us o alida e gamifica ion as a ool o
imp o e academic pe o mance o mechanical engi-
nee ing s uden s. Howe e , o gene alize ou esul s,
he pu posed me hodology should be applied o
o he mechanical enginee ing subjec s using
Kahoo ! o simila pe sonal esponse applica ions.
Acknowledgmen s – The au ho s would like o hank he s uden s
who pa icipa ed in he s udy. This in es iga ion has been
pa ially unded by he Ins i u e o Educa ion Sciences om
he Uni e si a Ro i a i Vi gili h ough he p ojec A10/18
(Analysis, b oadcas and in e na ionaliza ion o he imp o e-
men in he eaching/lea ning p ocess h ough gamifica ion).The
da ase s used and/o analyzed du ing he cu en s udy a e
a ailable om he co esponding au ho on easonable eques .
Re e ences
1. C. Diche and D. Diche a, Gami ying educa ion: wha is known, wha is belie ed and wha emains unce ain: a c i ical e iew, In . J.
Educ. Technol. High. Educ.,14(9), 2017.
2. K. Seabo n and D. I. Fels, Gamifica ion in heo y and ac ion: A su ey, In . J. Hum. Compu . S ud.,74, pp. 14–31, 2015.
3. K. M. Kapp, The Gamifica ion o Lea ning and Ins uc ion: Game-Based Me hods and S a egies o T aining and Educa ion, San
F ancisco, Cali o nia: P eiffe , 2012.
4. R. S. Alsawaie , The effec o gamifica ion on mo i a ion and engagemen , In . J. In . Lea n. Technol.,35(1), pp. 82–93, 2018.
5. L. Ding, C. Kim and M. O ey, Design o gamified asynch onous online discussions, Technol. Pedagog. Educ., pp. 1–17, Aug. 2020.
6. K. Kapp, The Gamifica ion o Lea ning and Ins uc ion Fieldbook: Ideas in o P ac ice, San F ancisco, Cali o nia: Wiley, 2013.
7. K. M. Kapp, The Gamifica ion o Lea ning and Ins uc ion Fieldbook: Ideas in o P ac ice, San F ancisco, Cali o nia: Wiley, 2013.
8. G. P. Kusuma, E. K. Wiga i, Y. U omo and L. K. Pu e a Su yap ana a, Analysis o Gamifica ion Models in Educa ion Using MDA
F amewo k, P ocedia Compu . Sci.,135, pp. 385–392, 2018.
9. D. Diche a, C. Diche , G. Ag e and G. Angelo a, Gamifica ion in Educa ion?: A Sys ema ic Mapping S udy, Educ. Technol. Soc.,
18(3), pp. 75–88, 2015.
10. G. Huesca Jua
´ ez and M. M. Reyes Ca ballo, Lea ning Gains , Mo i a ion and Lea ning S yles in a Gamified Class, In . J. Eng.
Educ.,32(1), pp. 438–447, 2016.
11. E. Kim, L. Ro h ock and A. F ei alds, An Empi ical S udy on he Impac o Lab Gamifica ion on Enginee ing S uden s ’ Sa is ac ion
and Lea ning, In . J. Eng. Educ.,34(1), pp. 201–216, 2018.
12. K. A. I. J. U. N. Chew, In e ac ion Be ween Lea ning Assessmen and S uden Mo i a ion?: A Scoping Re iew, In . J. Eng. Educ.,
37(5), pp. 1313–1329, 2021.
13. C. E. Holb ey, Kahoo ! Using a game-based app oach o blended lea ning o suppo effec i e lea ning en i onmen s and s uden
engagemen in adi ional lec u e hea es, Technol. Pedagog. Educ.,29(2), pp. 191–202, Ma . 2020.
14. B. Balak ishnan Nai , Endo sing gamifica ion pedagogy as a help ul s a egy o offse he COVID-19 induced dis up ions in ou ism
educa ion, J. Hosp. Leis. Spo Tou . Educ., 30 (Sep embe 2021), p. 100362, 2022.
Rosa Pa
`mies-Vila
`e al.1440
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
15. L. De-Ma cos, A. Domı
´nguez, J. Saenz-De-Na a e e and C. Page
´s, An empi ical s udy compa ing gamifica ion and social
ne wo king on e-lea ning, Compu . Educ.,75, pp. 82–91, 2014.
16. N. Go don, M. B ayshaw and S. G ey, Maximising gain o minimal pain: U ilising na u al game mechanics, I al. Inno . Teach.
Lea n. In . Compu . Sci.,12(1), pp. 27–38, 2013.
17. C.-H. Chen and H.-C. Yeh, Effec s o in eg a ing a ques ioning s a egy wi h game-based lea ning on s uden s’ language lea ning
pe o mances in flipped class ooms, Technol. Pedagog. Educ.,28(3), pp. 347–361, May 2019.
18. O. Baydas and M. Cicek, The examina ion o he gamifica ion p ocess in unde g adua e educa ion: a scale de elopmen s udy,
Technol. Pedagog. Educ.,28(3), pp. 269–285, May 2019.
19. M. Papas e giou, Digi al Game-Based Lea ning in high school Compu e Science educa ion: Impac on educa ional effec i eness and
s uden mo i a ion, Compu . Educ.,52(1), pp. 1–12, 2009.
20. C. Hosseini, O. Humlung, A. Fage s øm and M. Hadda a, An expe imen al s udy on he effec s o gamifica ion on ask
pe o mance, P ocedia Compu . Sci.,196(Janua y), pp. 999–1006, 2021.
21. M. D. Hanus and J. Fox, Assessing he effec s o gamifica ion in he class oom: A longi udinal s udy on in insic mo i a ion, social
compa ison, sa is ac ion, effo , and academic pe o mance, Compu . Educ.,80, pp. 152–161, 2015.
22. A. Domı
´nguez, J. Saenz-De-Na a e e, L. De-Ma cos, L. Fe na
´ndez-Sanz, C. Page
´s and J. J. Ma ı
´nez-He a
´iz, Gami ying lea ning
expe iences: P ac ical implica ions and ou comes, Compu . Educ.,63, pp. 380–392, 2013.
23. M. Bekk, R. Eppmann, K. Klein and F. Vo¨lckne , All ha gli e s is no gold: An in es iga ion in o he undesi ed effec s o
gamifica ion and how o mi iga e hem h ough gamifica ion design, In . J. Res. Ma k., no. xxxx, 2022.
24. C. Pe o a, G. Fea he s one, H. As on and E. Hough on, Game-based Lea ning: La es E idence and Fu u e Di ec ions (NFER
Resea ch P og amme: Inno a ion in Educa ion), Slough: NFER, 2013.
25. J. Hama i, J. Koi is o and H. Sa sa, Does Gamifica ion Wo k? – A Li e a u e Re iew o Empi ical S udies on Gamifica ion, in 47 h
Hawaii In e na ional Con e ence on Sys em Science, pp. 3025–3034, 2014.
26. Z. Zainuddin, S. K. W. Chu, M. Shujaha and C. J. Pe e a, The impac o gamifica ion on lea ning and ins uc ion: A sys ema ic
e iew o empi ical e idence, Educ. Res. Re .,30, p. 100326, 2020.
27. L. Rai skaya and E. Tikhono a, Gamifica ion as a field landma k in educa ional esea ch, J. Lang. Educ.,5(3), pp. 4–10, 2019.
28. M. Ranie i, J. E. Raffaghelli and I. B uni, Game-based s uden esponse sys em: Re isi ing i s po en ials and c i icali ies in la ge-size
classes, Ac . Lea n. High. Educ., pp. 1–14, 2018.
29. D. B uff, Teaching wi h Class oom Response Sys ems: C ea ing Ac i e Lea ning En i onmen s, San F ancisco: Jossey-Bass, 2009.
30. C. Fies and J. Ma shall, Class oom Response Sys ems: A Re iew o he Li e a u e, J. Sci. Educ. Technol.,15(1), pp. 101–109, 2006.
31. B. And e
´s, R. Sanchis and R. Pole , Quiz game applica ions o e iew he concep s lea n in class: An applica ion a he Uni e si y
con ex , in P oceedings o INTED2015 Con e ence, pp. 5654–5662, 2015.
32. V. J. Shu e, Focus on Fo ma i e Feedback, Re . Educ. Res.,78(1), pp. 153–189, 2008.
33. J. E. Caldwell, Clicke s in he La ge Class oom?: Cu en Resea ch and Bes -P ac ice Tips, CBE – Li e Sci. Educ.,6(1), pp. 9–20,
2007.
34. C. B. Paschal, Fo ma i e Assessmen in Physiology Teaching Using a Wi eless Class oom Communica ion Sys em, Ad . Physiol.
Educ.,26(4), pp. 299–308, 2015.
35. S. W. D ape and M. I. B own, Inc easing in e ac i i y in lec u es using an elec onic o ing sys em, J. Compu . Assis . Lea n.,20, pp.
81–94, 2004.
36. J. I. Cas illo-Manzano, M. Cas o-Nun˜o, L. Lo´pez-Valpues a, M. T. Sanz-Dı´az and R. Yn˜iguez, Measu ing he effec o ARS on
academic pe o mance: A global me a-analysis, Compu . Educ.,96, pp. 109–121, 2016.
37. N. J. Hunsu, O. Adesope and D. J. Bayly, A me a-analysis o he effec s o audience esponse sys ems (clicke -based echnologies) on
cogni ion and affec , Compu . Educ.,94, pp. 102–119, 2016.
38. L. A. Van Dijk, G. C. Van De Be g and H. Van Keulen, In e ac i e lec u es in enginee ing educa ion, Eu . J. Eng. Educ.,26(1), pp.
15–28, 2001.
39. M. Cu o, L. O cos, P. J. Bla
´zquez and F. J. Molina, S uden Assessmen o he Use o Kahoo in he Lea ning P ocess o Science and
Ma hema ics, Educ. Sci.,9(1), p. 55, 2019.
40. S. G. B yan , J. M. Co ell and B. M. Cla ke, Fun wi h pha macology: Winning s uden s o e wi h kahoo ! game-based lea ning, J.
Nu s. Educ.,57(5), p. 320, 2018.
41. E. Za zycka-Pisko z, Kahoo i o no ? can games be mo i a ing in lea ning g amma ?, Teach. English wi h Technol.,16(3), pp. 17–36,
2016.
42. D. A. Gen ile, The Mul iple Dimensions o Video Game Effec s, Child De . Pe spec .,5(2), pp. 75–81, 2011.
43. O. Bo as-gene and M. Ma inez-nun, New Challenges o he Mo i a ion and Lea ning in Enginee ing Educa ion Using
Gamifica ion in MOOC, In . J. Eng. Educ.,32(1(B)), pp. 501–512, 2016.
44. V. Simpson and M. Oli e , Elec onic o ing sys ems o lec u es hen and now: A compa ison o esea ch and p ac ice, Aus alas. J.
Educ. Technol.,23(2), pp. 187–208, 2007.
Impac o a Gamifica ion Lea ning Sys em on he Academic Pe o mance o Mechanical Enginee ing S uden s 1441
Appendix
The 12 pool ques ions we e:
Q1. Has Kahoo ! helped you o be e unde s and he subjec ?
Q2. Ha e you become mo e mo i a ed because o Kahoo !?
Q3. Is he ime you in es ed in Kahoo ! offse by how much you ha e lea ned?
Q4. Does he Kahoo ! sco e eflec you unde s anding?
Q5. Did you ha e enough ime o answe he Kahoo ! ques ions?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
Rosa Pa
`mies-Vila
`e al.1442
Q6. Would you welcome he use o Kahoo ! in o he subjec s?
Q7. Did Kahoo ! make you mo e a en i e o he class?
Q8. I he Kahoo ! ques ions had mo e weigh in he e alua ion, would you ha e been mo e ca e ul/a en i e?
Q9. Has Kahoo ! imp o ed you ela ionship wi h eache s?
Q10. Is he compe i i eness c ea ed by Kahoo ! posi i e?
Q11. Can Kahoo ! help you acqui e knowledge and cla i y concep s?
Q12. Does he discussion o cla ifica ion o you sco e cla i y some concep s?
Rosa Pa
`mies-Vila
` ecei ed he PhD. deg ee in Biomedical Enginee ing in 2012 om Uni e si a Poli e
`cnica de Ca alunya,
Spain. She is assis an p o esso a he Mechanical Enginee ing Depa men o UPC, whe e she has been since 2008. He
cu en esea ch a ea includes s udies on mechanisms and machine heo y, specifically in he field o biomechanics o
human mo ion. Mo eo e , she has expanded he esea ch a ea o include eme ging educa ional echnologies and how
gamifica ion can ans o m adi ional eaching me hods.
Albe Fab ega -Sanjuan is an Assis an P o esso in he Depa men o Mechanical Enginee ing a he Uni e si a Ro i a
i Vi gili (URV) since 2009. He has been wo king in he field o expe imen al and compu a ional mechanics esea ch and
has augh se e al subjec s ela ed o mechanical enginee ing. As a coo dina o o se e al educa ion inno a ion p ojec s,
his pa icula in e es ocuses on implemen ing inno a i e me hods o eaching, especially he use o gamifica ion and
se ice-lea ning. He has been awa ded o he quali y o his eaching me hods in he collec i e (2015) and indi idual (2018)
ca ego y om he URV Social Council.
Joan Puig O iz ecei ed his PhD deg ee in Physics Science a Uni e si a Poli e
`cnica de Ca alunya, Spain, in 2006. He is
Collabo a ing Lec u e o he Mechanical Enginee ing Depa men o UPC. His esea ch in e es s a e in he a ea o
mechanisms and machine heo y and he new me hods o eaching in STEM.
Lluı¨sa Jo di Nebo ecei ed he PhD deg ee in Physics Science in 1999 om Uni e si a Poli e
`cnica de Ca alunya, Spain.
She is uni e si y lec u e a he Mechanical Enginee ing Depa men o UPC, whe e she has been since 1988. He esea ch
in e es s a e in he a ea o mechanisms and machine heo y – specially, cons an -b ead h cam mechanism and ba
linkages. She has expanded he esea ch a ea o include new educa ional echnologies and how ans o m adi ional
eaching me hods.
An oni He na
´ndez Fe na
´ndez has been an assis an p o esso o Physics and Educa ional Science a he Ins i u e o
Educa ion Sciences (ICE–UPC) since 2008. He has also been a pa - ime eache in Seconda y School and Voca ional
T aining a Te assa A and Design School since 1997. He has deg ees in Physics (1997) and Linguis ics (2003) and
ecei ed his doc o a e in Linguis ics om he Uni e si a de Ba celona (UB) in 2014. His cu en esea ch in e es s include
educa ion in a b oad sense (gamifica ion, p ojec -based lea ning, echnology educa ion, e c.) and quan i a i e linguis ics,
i.e. he s udy o gene al pa e ns in communica ion sys ems (Zip ’s law, Menze a h-Al mann’s law, e c.).
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57