Impact of a Gamification Learning System on the Academic Performance of Mechanical Engineering Students

P oo s Impac o a Gamifica ion Lea ning Sys em on he Academic Pe o mance o Mechanical Enginee ing S uden s* ROSA PA `MIES-VILA ` Depa men o Mechanical Enginee ing, Uni e si a Poli e `cnica de Ca alunya, A . Diagonal, 647, 08028, Ba celona, Ca alonia, Spain. E-mail: [email p o ec ed] ALBERT FABREGAT-SANJUAN Depa men o Mechanical Enginee ing, Uni e si a Ro i a i Vi gili, A . Paı¨sos Ca alans, 26, 43007, Ta agona, Ca alonia, Spain. E-mail: [email p o ec ed] JOAN PUIG-ORTIZ and LLUI ¨SA JORDI NEBOT Depa men o Mechanical Enginee ing, Uni e si a Poli e `cnica de Ca alunya, A . Diagonal, 647, 08028, Ba celona, Ca alonia, Spain. E-mail: [email p o ec ed], [email p o ec ed] ANTONI HERNA ´NDEZ FERNA ´NDEZ Ins i u e o Educa ion Sciences, Uni e si a Poli e `cnica de Ca alunya, Plac¸a Eusebi Gu ¨ell, 6, 08034, Ba celona, Ca alonia, Spain. E-mail: [email p o ec ed] This s udy examines he effec s o using a gamifica ion ool as a eaching s a egy. Specifically, Kahoo ! is e alua ed as a ool o enhancing s uden lea ning. The ac i i ies we e pa o he labo a o y sessions o he subjec Mechanism and Machine Theo y du ing wo consecu i e academic yea s. We analyze he effec o a gamifica ion lea ning sys em on bo h, s uden s’ g ades and mo i a ion, in a cou se wi h a la ge numbe o s uden s (n 1 = 283 s uden s, n 2 = 306 s uden s). The s uden s we e di ided in o h ee diffe en g oups (con ol g oup, gamifica ion g oup and w i ing g oup) and hei esul s we e e alua ed depending on he lea ning me hod applied du ing he class. In e ms o gamifica ion, his p ojec in oduces eal- ime eedback o s imula e he in e es o s uden s and help hem use he ypical ools and me hodologies o game-based lea ning. The analysis o hei pe o mance in he labo a o y exam shows significan diffe ences be ween he g oup ha used gamifica ion and he g oups ha did no . The esul s sugges ha gamifica ion in enginee ing lab ac i i ies has a posi i e effec on s uden s’ mo i a ion and lea ning ou come. The s udy concludes ha game-based elemen s and compe i i e ac i i ies enhanced s uden pe o mance. Keywo ds: gamifica ion; game-based lea ning; highe educa ion; mechanical enginee ing 1. In oduc ion The combina ion o eaching and games can be aced back o he humanis ic app oach, bu in ecen yea s game design elemen s ha e s a ed o be used o non-play ul pu poses [1]. Al hough he e m is s ill being e ised concep ually – see [2] o a heo e ical e iew – gamifica ion can be defined as using game-based mechanics, aes he ics and hink- ing o engage people, mo i a e ac ion, p omo e lea ning, and sol e p oblems [3]. In educa ion, he idea is o mo i a e and s imula e s uden s by using ac i i ies o he han adi ional ones, and acili a e – almos wi hou hem being awa e – eaching- lea ning i sel , especially in a social con ex in which s uden engagemen needs o be inc eased [4]. The e is a b oad deba e among game designe s, esea che s and educa o s, abou wha games a e, how hey impac indi iduals and, in gene al, how hey can be used in class ooms. Insufficien a en- ion has been paid o gamifica ion g ounded in bo h heo ies and e idence om empi ical s udies [5]. In his ega d, [6] desc ibes he ad an ages and dis- ad an ages o gamifica ion Among he ad an ages, he says, a e ha games and gamifica ion can lead o high le els o lea ne engagemen and mo i a ion since hey connec wi h he skills o 21s -cen u y s uden s [7]. On he o he hand, he e is a isk o applying ep oduc ion wi hou p io design, esul - ing in p oblems such as exploi a ion o he c ea ion o hos ile and ense en i onmen s. Gamifica ion models in educa ion domain could help gamifica- ion p ac i ione s o make new s a egies in lea ning ac i i ies o inc ease s uden s’ mo i a ion, achie e- men and in ol emen [8]. Rigo ous s udies a e equi ed o ully examine he effec s o gamifica ion and de e mine how lea ning is bes achie ed [9]. In gene al, gamifica ion echniques ha e posi i e effec s on he in ol emen and mo i a ion o s u- den s [4, 10, 11], – see [12] o scoping e iew –. S uden s alue i s compe i i e na u e, he immedi- acy o eedback on hei knowledge and s uc u ed oppo uni ies o u he discussion [13] and hey also iden i y gamifica ion as a mul i ace ed ool o a g ea lea ning expe ience [14]. Gamified lea ning en i onmen s con ibu e o he lea ning and each- * Accep ed 10 July 2022.1434 IJEE 4266 PROOFS In e na ional Jou nal o Enginee ing Educa ion Vol. 38, No. 5(A), pp. 1434–1442, 2022 0949-149X/91 $3.00+0.00 P in ed in G ea B i ain #2022 TEMPUS Publica ions. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 ing p ocess by aising le els o engagemen , c ea ing enjoyable lea ning en i onmen s and ensu ing ac i e pa icipa ion [11, 15–20]. Howe e , some s udies ha e no iden ified any significan effec s on lea ning o ha e e en de ec ed wo se academic esul s when s uden s a e o ced o use game ele- men s [21–23]. Howe e , all e iews o da e ag ee ha he e is insufficien e idence o suppo he long- e m benefi s o gamifica ion in educa ional con ex s [1, 13, 17, 24, 25], so mo e empi ical e idence is needed o jus i y ha gamifica ion is be e han o he pedagogical al e na i es [26, 27]. The e o e, he p esen s udy aims o p o ide new e idence on he effec s o gamifica ion in he class- oom. 1.1 Gamifica ion wi h Pe sonal Response De ices Pe sonal esponse de ices (PRDs) – some imes called class oom esponse sys ems, s uden esponse sys ems, o audience esponse sys ems – appea ed a he beginning o he 21s cen u y and hey consis o an emi e and a ecei e ha , oge he wi h he co esponding so wa e, enable eache s o ask hei s uden s a mul iple-choice ques ion ( he ques ion is p ojec ed on a sc een) and s uden s o send an answe using hei indi i- dual con ol o clicke [28, 29]. Clicke s p o ide a simple way o gene a e an a mosphe e o s uden in e ac ion ha can enhance eache -s uden com- munica ion [30]. Se e al PRDs can be used wi h iPads, And oid able s, mobile phones and compu e s. These new sys ems ha e he same u ili ies as o he as esponse me hods such as clicke s, none o he co esponding echnical-logis ical p oblems, and new ea u es like gaming elemen s, music, modal- i ies, and design [28, 31] Usually, he in eg a ion o his de ices do no p esen echnical difficul ies and gaming is success ul in enabling ac i e pa icipa ion and in e ac i e lea ning [13]. Some o hese applica- ions a e Men ime e , In use Lea ning, Soc a i e, Quiz Socke , Kahoo !, Ve so, Poll E e ywhe e o VoxVo e, which enable you o p epa e mul iple- choice ques ionnai es, ue/ alse ques ionnai es and, in some cases, ques ions wi h sho answe s (i.e. Soc a i e). Wi h hese ools, s uden s answe all ques ions simul aneously in class, da a is collec ed and s a is ics on he esponses o he s uden s a e gi en immedia ely. The iming is p og ammed by he eache s, who can de ec common e o s, high- ligh aspec s ha a e mos deficien o he s uden s and p o ide immedia e eedback [32]. The indi idual esponse sys em c ea es an en i- onmen o immedia e in e ac i e lea ning and discussion in he class oom [33, 34]. I also p o ides o ma i e eedback on lea ning ( o bo h eache s and lea ne s) [35]. Howe e , he benefi s o using s uden esponse sys ems a e also con o e sial [36, 37]. While he e is conside able e idence o sugges ha uni e si y s uden s ha e e y posi i e opinions abou he use o hese sys ems [28, 30, 33], some s udies conclude ha hese ools do no gua an ee be e lea ning [35, 38]. I seems ha i is he implemen a ion o pedagogical s a egies in combina ion wi h he echnology ha ul ima ely influences s uden success. S udies on indi idual esponse sys ems o en compa e he s a is ics and eedback gi en wi h he s uden ’s final g ade. The co ela ions a e o en posi i e bu weak, which shows ha hey can be use ul o o ma i e assessmen bu no o sum- ma i e [30, 35, 39]. Kahoo ! is a ee i ual ool ha has gained in popula i y among eache s o i s use - iendly na u e and i s abili y o es ablish wo king dynamics in he class oom. I is highly app ecia ed by s u- den s [28]. Kahoo ! allows eache s o c ea e su - eys, ques ionnai es, puzzles and deba es, and ob ain s uden s’ answe s in eal ime. Va ious s udies on Kahoo ! ag ee ha his ool imp o es pa icipa ion and he posi i e ela ionship be ween class membe s [13, 39–41]. 1.2 Case S udy in Mechanical Enginee ing Mechanism and Machine Theo y is a co e subjec augh in he ou h semes e o he Deg ee in Indus ial Enginee ing a Uni e si a Poli e `cnica de Ca alunya. I is one o he fi s imes ha he Indus ial Enginee ing s uden s ha e come in o con ac wi h he wo ld o mechanical enginee ing. The o ma i e assessmen s om p e ious semes e s showed ha he s uden s did no acqui e he equi ed skills a labo a o y classes: he pe cen age o s uden s who passed he labo a o y exam was e y low, and he eache s conside ed i a p oblem since i sugges s ha s uden s we e no able o pu in o p ac ice he knowledge hey had acqui ed in he heo y classes. On a e age, he pe cen age o s uden s passing he cou se is 70%, whe eas he pe cen age passing he p ac ical examina ions is 40%, no ably lowe . The e o e, a new me hod was needed in o de o imp o e he eaching/lea ning p ocess. We hypo hesize ha he in oduc ion o gamified eedback will help o highligh he mos impo an concep s a he end o each labo a o y session, and he e o e, imp o e he lea ning p ocess. Ano he impo an issue is which o he ac o s in ol ed in gamifica ion is mos ela ed o he imp o emen in lea ning (i indeed he e is an imp o emen ). To add ess his issue using a mul i ac o ial app oach [42], we diffe en ia ed wo ac o s: he ac ha s uden s ecei e eedback ( his will be checked by a con ol g oup doing w i en es s) and he ac Impac o a Gamifica ion Lea ning Sys em on he Academic Pe o mance o Mechanical Enginee ing S uden s 1435 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 ha gamifica ion p omo es o he a iables (mo i- a ion, engagemen , compe i i eness, e c.). Fo his eason, he second hypo hesis o his s udy is ha he fi s o hese wo ac o s (gamifica- ion) is mo e impo an han he second ( he eed- back i sel ). In o de o es his hypo hesis, he ques ions ha he s uden s in he gamifica ion g oup we e asked we e also p esen ed o ano he expe imen al g oup in which s uden s did a w i en es wi hou using Kahoo !. The solu ions o he es ( eedback) we e also p o ided a e he labo a o y session. 2. Me hods The p esen s udy uses an empi ical-analy ical me hodology o s udy gamifica ion as a ool in labo a o y sessions. The subjec Theo y o Machines and Mechanisms has a la ge numbe o s uden s each semes e (be ween 270 and 320) so he s uden s we e andomly dis ibu ed in o 11 labo a- o y g oups augh by 4 diffe en lec u e s. The aim o ou in e en ion was o imp o e lea ning in he labo a o y sessions. The o e all cou se g ade is calcula ed acco ding o he ollowing weigh ed a e age, ounded o one decimal place: Mcou se ¼Maxð0:6M e þ0:2Mpe;0:8M eÞ þ 0:10 Mlab1 þ0:10ðMlab2 MsimÞ1=2ð1Þ whe e, Mcou se is he final g ade o he cou se, M e is he ma k o he final exam, Mpe is he ma k o he mid e m exam, Mlab1 is he ma k o he fi s labo a o y exam (assessing sessions 1, 2 and 3), and Mlab2 is he ma k o he second labo a o y exam (assessing sessions 4 and 5). Finally, Msim is he ma k o a simula ion exe cise. The in e en ions ook place du ing he second e m o he academic yea s 2016–17 and 2017–18 and aimed o imp o e he ma ks o 3 labo a o y sessions which accoun o 10% o he final g ade. A es ques ionnai e has been in oduced as a eedback ool. Quick eedback helps s uden s become awa e, and hey ha e g ea e pe cep ion o wha has happened in he labo a o y. This eed- back has been in oduced as a es ques ionnai e ha has o be answe ed in he las 15–30 minu es o each session. Two diffe en eedbacks a e analyzed. The fi s uses Kahoo ! ques ionnai es. Since Kahoo ! is a as esponse sys em o he s uden , i is expec ed o be effec i e a imp o ing knowledge e en ion and skill acquisi ion. The second uses a adi ional ques ion- nai e which, he e o e, in ol es no compe i ion o coope a i e lea ning. To de e mine he effec o in oducing no only a eedback ool bu a eedback gamifica ion ool, he labo a o y g oups we e di ided in o h ee g oups: An expe imen al g oup gi en eedback h ough he Kahoo ! ques ionnai es– (Gami ica ion g oup, GG). These lea ne s use he mobile e - sion o he app. An expe imen al g oup gi en a w i en es a he end o he session (wi h he same ques ions as in Kahoo !), ac ing as ein o cemen and eedback, bu wi hou he o he componen s ha Kahoo ! may ha e (W i ing g oup, WG). A con ol g oup subjec o no in e en ion (Con- ol g oup, CG). The s uden s we e di ided up in his way o a oid eache and ime able ac o s. Table 1 summa izes he numbe o s uden s in each g oup. No e ha some s uden s do no pa icipa e in he labo a o y sessions. Academic pe o mance was assessed by compa - ing he ma ks o s uden s in each o he pedagogical g oups. The mean ma k, s anda d de ia ion and numbe o s uden s who passed he exam we e calcula ed o each e alua ion (M lab1 ,M lab2 ,M pe , M sim ,M e ). A S uden ’s T-Tes was also used o find significan diffe ences be ween he expe imen al (GG and WG) and he con ol (CG) g oups. The e o e, o he GG he ela ion be ween he Kahoo ! es sco e and he g ades in he o he e alua ions was s udied. Likewise, o he WG, he ela ion be ween he w i ing es sco e and he g ades in o he e alua ions was examined. To his end, linea co ela ions we e calcula ed and Pea - son, Spea man and Kendall coefficien s de e - mined. Finally, whe he o no he e was a eache effec was s udied ( ha is o say, whe he a pa icula s uden ge s a be e o a wo se ma k depending on he eache who has augh he subjec ). The e o e, he s uden s we e g ouped acco ding o he lec u e who augh he sessions and a S uden ’s T-Tes was used o de e mine significan diffe ences be ween he ou g oups. Finally, i was no conside ed app op ia e o measu e success only by compa ing he summa i e ma ks, because his is no he only pu pose o he gamifica ion ool. A 12-ques ion su ey was p e- Rosa Pa `mies-Vila `e al.1436 Table 1. O e all numbe o s uden s o each g oup and academic yea Numbe o s uden s 2016–17 2017–18 Gamifica ion G oup – GG 37 41 W i ing G oup – WG 115 86 Con ol G oup – CG 113 100 No a ending 41 56 To al 306 283 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 pa ed o s uden s in he gamifica ion g oup (See he Appendix). 3. Resul s Because eedback is now a pa o labo a o y sessions 1, 2 and 3, diffe ences in he labo a o y exam 1 ma ks (M lab1 ) can be expec ed among he h ee g oups. Du ing he academic yea 2016–17, 62.16% o he s uden s who ook pa in he gamifica ion passed he exam while only 54.87% o he con ol g oup and 58.26% o he w i ing g oup did he same. Simila ly, du ing he academic yea 2017–18, 87.80% o he s uden s who ook pa in he gamifica ion passed he exam while in he con ol g oup and he w i ing g oup he pe cen- ages we e 74.74% and 77.91%, espec i ely. Table 2 shows he mean and s anda d de ia ion o he ma ks o each e alua ion (M lab1 ,M lab2 ,M pe , M sim ,M e ) o bo h o he academic yea s analyzed. I can be seen ha o labo a o y exam 1 (M lab1 ), he mean g ade ob ained by he s uden s who ook pa in he gamifica ion sessions (5.59 2.43, academic yea 2016–17; and 6.90 1.68, academic yea 2017–18) is mo e han one poin highe han he con ol g oup (4.50 2.17, academic yea 2016– 17; and 5.75 2.30, academic yea 2017–18). Howe e , his diffe ence is no so clea o he w i ing g oup (4.71 2.37, academic yea 2016– 17; and 5.57 2.24, academic yea 2017–18). A S uden ’s T- es be ween GG and CG demon- s a ed ha he e is a significan diffe ence be ween hese wo g oups (p- alue < 0.05). Mo eo e , he diffe ences be ween he WG and he CG g oup a e no s a is ically significan . Fig. 1 shows he boxplo ob ained o M lab1 o bo h academic yea s and o each eaching me ho- dology. The cen al block is delimi ed by he posi- ion o Q1 and Q3 qua iles and he line ep esen ing he median is d awn in he box. I can be seen ha he median is also highe o he gamifica ion g oup han o he w i ing and con ol g oups. Diffe ences be ween GG, CG and WG a e no p esen ed o he o he e alua ion ma ks (M lab2 , Impac o a Gamifica ion Lea ning Sys em on he Academic Pe o mance o Mechanical Enginee ing S uden s 1437 Table 2. Mean SD o he ma ks o each e alua ion G oup M lab1 M lab2 M pe M sim M e Academic yea 2016–17 GG 5.59 2.43* 4.22 2.97 5.24 2.34 7.16 2.16 2.88 1.82 WG 4.71 2.37 3.69 2.86 5.57 2.24 7.13 1.76 3.12 1.62 CG 4.50 2.17 3.49 2.54 5.53 2.37 7.06 1.84 2.90 1.87 Academic yea 2017–18 GG 6.90 1.68* 4.94 2.36 5.90 2.76 7.06 2.23 4.94 2.30 WG 6.08 2.23 5.56 2.59 6.28 2.50 7.14 2.23 4.83 2.10 CG 5.75 2.30 4.95 2.78 5.92 2.11 7.26 2.27 4.32 2.16 *p- alue < 0.01. Fig. 1. Boxplo o labo a o y exam 1 ma ks. (a) Academic yea 2016–17 (b) Academic yea 2017–18. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 M pe ,M sim ,M e ) (Table 2): he p- alues a e g ea e han 0.05 and he e o e s a is ical diffe ences canno be assumed. This shows ha s uden s a e andomly dis ibu ed among g oups. Diffe ences only appea when a gamifica ion me hodology is applied. Th ee diffe en co ela ion coefficien s (Pea son, Spea man and Kendall) and he p- alues o he s a is ical es s we e calcula ed. Fo GG, he p- alues we e much lowe han 0.05 (Table 3), so he e is a significan posi i e co ela ion be ween he ma ks ob ained in he Kahoo ! es and he ones ob ained in he labo a o y 1 exam (co ela ions be ween 0.6 and 0.77 depending on he indica o used). Howe e , he co ela ions o he w i ing g oup a e eally low and hey a e no significan (see Table 3). Fu he mo e, he ela ion be ween he eedback es s and he g ades ob ained in he o he e alua- ions we e s udied using he same coefficien s. The co ela ions in hese cases we e poo (0.180-0.337) and non significan . Fig. 2 shows, he ela ionship be ween he calcu- la ed Kahoo ! g ades (M Kahoo ! ) and he g ades ob ained by s uden s on he labo a o y exam 1(M lab1 ) o he gamifica ion g oup (GG). The g aphs also show he polynomial eg ession line ha adjus s hese alues and he co esponding R 2 pa ame e . The eache effec was also analysed. As explained abo e, he sessions a e augh by ou diffe en lec u e s. Fo his analysis, s uden s we e g ouped acco ding o he lec u e who augh hei labo a o y session. Howe e , he S uden ’s T- es does no de ec any significan diffe ences be ween he ou g oups s udied (p- alue > 0.05). The e o e, i canno be affi med ha he eaching s aff has an effec on he g ades o he s uden s. Finally, he opinion poll shows whe he s uden s see gamifica ion as an imp o emen in hei lea n- ing p ocess and hei mo i a ion. The esul s o bo h cou ses we e simila . They a e p esen ed in Fig. 3 whe e he do ed blocks ep esen a posi i e answe (s ongly ag ee o ag ee) and lined blocks Rosa Pa `mies-Vila `e al.1438 Table 3. Co ela ion coefficien s be ween eedback es and labo a o y exam 1. M Kahoo ! and M lab1 o gamifica ion g oup and M WT and M lab1 o w i ing g oup. Coefficien Academic yea 2016–17 Academic yea 2017–18 Value (p- alue) Value (p- alue) Gamifica ion G oup Pea son 0.726 (3.63 10 –7 ) 0.774 (2.92 10 –9 ) Spea man 0.754 (7.34 10 –8 ) 0.709 (2.13 10 –7 ) Kendall 0.603 (7.32 10 –7 ) 0.615 (2.74 10 –7 ) W i ing G oup Pea son –0.010 (0.913) –0.101 (0.357) Spea man 0.003 (0.970) –0.174 (0.110) Kendall 0.003 (0.965) –0.136 (0.084) Fig. 2. Sca e g aph o he lab exam 1 ma ks (M lab1 ) e sus he Kahoo ! ma ks (M Kahoo ! ). (a) Academic yea 2016–17, (b) Academic yea 2017–18. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 ep esen a nega i e answe (disag ee o s ongly disag ee).. In gene al, Kahoo ! has been e y well accep ed. On he basis o he answe s o he fi s h ee ques ions, i can be said ha mo e han 90% o he s uden s hink ha gamifica ion has helped hem o unde s and he subjec which, in u n, makes hem mo e mo i a ed. Mo eo e , s uden s confi m ha he ime spen on he ac i i y is offse by imp o ed lea ning (Q3). In e ms o wha he Kahoo ! esul s demons a e (Q4), s uden s a e di ided: some ag ee ha hey a e p oo o knowledge acqui ed, bu o he s do no . Simila esul s we e ob ained o Q5 and Q7. F om hese esul s, i is difficul o see i enough ime was gi en o answe he Kahoo ! ques ions and i Kahoo ! ques ionnai es make hem mo e a en i e o he labo a o y session. All s uden s affi m ha Kahoo ! ques ionnai es will be posi i e in o he subjec s (Q6) and hey all ag ee ha discussion a e hey ha e gi en hei answe s allows hem o cla i y concep s (Q12). Mos o hem posi i ely e alua e he eedback hey ge h ough hese quizzes (Q11). Acco ding o 80% o he s uden s, i Kahoo ! g ades we e added o he summa i e e alua ion g ade hey would pay mo e a en ion in class (Q8). A simila pe cen age pe cei es he compe i- i eness c ea ed by Kahoo ! as a posi i e s imulus o lea ning (Q10). 4. Discussion The e is a lack o esea ch on he eal effec s o gamifica ion on he lea ning p ocess and whe he hese effec s a e be e han hose ob ained wi h adi ional app oaches [43]. This pape p esen s an expe imen al s udy wi h 589 s uden s en olled. As has been men ioned, he e was a need o elucida e whe he Kahoo ! is effec i e o no . The main objec i e was o con as i gamifica ion h ough game-based s uden esponse sys ems imp o es ac i e s uden lea ning, pa icipa ion and e en ion o concep s [30, 44], o on he con a y, i is no gua an ee o be e lea ning [35, 38]. So, he esul s p esen ed he e can be ega ded as a pilo es which shows ha game-based s uden esponse sys ems (Kahoo ! he e) can imp o e academic pe o mance. In addi ion, his s udy also con ibu es o e alua e i bo h s uden s and eache s hink ha gamifica ion is s imula ing, e ealing, mo i a ing and, in essence, un [28]. Resul s show ha he gamifica ion g oup (GG) had a highe success a e in he labo a o y exam (Labo a o y exam 1) han he con ol g oup (CG). Mo eo e , on his e alua ion, he a e age g ade o GG s uden s was s a is ically g ea e han he a e age o CG s uden s. Fu he mo e, he g ades o he o he e alua ions do no show hese diffe - ences. I can be seen ha gamifica ion has a posi i e effec on g ades as [30, 44] sugges ed. No e ha he Impac o a Gamifica ion Lea ning Sys em on he Academic Pe o mance o Mechanical Enginee ing S uden s 1439 Fig. 3. Resul s o he opinion polls o he 12 ques ions. (a) Academic yea 2016–17, (b) Academic yea 2017–18. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 w i ing g oup shows no significan imp o emen wi h espec o he con ol g oup. When he eed- back is no gamified, i does no enhance academic esul s. These esul s sugges ha gamifica ion is he key o he imp o emen no he eedback i sel . No significan diffe ences we e de ec ed in he g ades o he a ious g oups who did labo a o y exam 1. This e eals ha i was no he lec u e o he labo a o y session who ma ked he diffe ence bu he in e en ion i sel . Tha sugges ha he gamifica ion is he key, no he lec u e . The esul s o he opinion poll show ha he s uden s alue he in e en ion posi i ely, as o he s udies ha e poin ed ou [28]. All o he s uden s s a ed ha he discussion a e hey had gi en hei esponses cla ified concep s, and mos o hem el ha gamifica ion helped hem o unde s and he subjec be e and mo i a ed hem. These esul s a e also in ag eemen wi h he li e a u e [15–18]: gami- fica ion in ol es mo i a ion. 5. Conclusions The main goal o his s udy was o analyse whe he a gamifica ion ool could imp o e academic pe o - mance and mo i a ion in he labo a o y sessions o he subjec Mechanism and Machine Theo y. Fo his pu pose, du ing wo consecu i e academic yea s, we di ided he s uden s in o h ee g oups and each g oup had diffe en me hodological in e - en ions. A he end o he fi s h ee sessions, he gamifica ion g oup (GG) answe ed a Kahoo ! ques- ionnai e; he w i ing g oup (WG) answe ed he same ques ionnai e bu on pape , and he con ol g oup (CG) did no ake any ques ionnai e. In he ligh o he esul s p esen ed, in gene al i can be concluded ha gamifica ion has p o ided a (modes ) inc ease in he eaching-lea ning p ocess in he labo a o y sessions o he subjec Mechanism and Machine Theo y wha i is so consis en ly wi h he heo e ical easoning ha mo i a ed his p o- posal. Howe e , his s udy has se e al limi a ions and u he esea ch will be equi ed. One limi a ion o he s udy is ha we a e no co e ing how gende diffe ences influenced he effec s o gamifica ion. Gende and pe sonali y could affec s uden s’ pe - cep ion owa d gamifica ion ac i i ies. In addi ion, we ha e only use one in e ace in gamifica ion. In u u e, simila applica ions should be es ed o compa e he ob ained esul s. Addi ionally, he e a e diffe en ways o gamified eedback mechanisms such as poin s, badges, ewa d, le els, e c. ha a e no co e ed in his s udy. Mo e design in es iga ion is equi ed o u he gene alize he esul s. The s udy has p esen ed a me hodology ha allow us o alida e gamifica ion as a ool o imp o e academic pe o mance o mechanical engi- nee ing s uden s. Howe e , o gene alize ou esul s, he pu posed me hodology should be applied o o he mechanical enginee ing subjec s using Kahoo ! o simila pe sonal esponse applica ions. Acknowledgmen s – The au ho s would like o hank he s uden s who pa icipa ed in he s udy. This in es iga ion has been pa ially unded by he Ins i u e o Educa ion Sciences om he Uni e si a Ro i a i Vi gili h ough he p ojec A10/18 (Analysis, b oadcas and in e na ionaliza ion o he imp o e- men in he eaching/lea ning p ocess h ough gamifica ion).The da ase s used and/o analyzed du ing he cu en s udy a e a ailable om he co esponding au ho on easonable eques . Re e ences 1. C. Diche and D. Diche a, Gami ying educa ion: wha is known, wha is belie ed and wha emains unce ain: a c i ical e iew, In . J. Educ. Technol. High. Educ.,14(9), 2017. 2. K. Seabo n and D. I. Fels, Gamifica ion in heo y and ac ion: A su ey, In . J. Hum. Compu . S ud.,74, pp. 14–31, 2015. 3. K. M. Kapp, The Gamifica ion o Lea ning and Ins uc ion: Game-Based Me hods and S a egies o T aining and Educa ion, San F ancisco, Cali o nia: P eiffe , 2012. 4. R. S. Alsawaie , The effec o gamifica ion on mo i a ion and engagemen , In . J. In . Lea n. Technol.,35(1), pp. 82–93, 2018. 5. L. Ding, C. Kim and M. O ey, Design o gamified asynch onous online discussions, Technol. Pedagog. Educ., pp. 1–17, Aug. 2020. 6. K. Kapp, The Gamifica ion o Lea ning and Ins uc ion Fieldbook: Ideas in o P ac ice, San F ancisco, Cali o nia: Wiley, 2013. 7. K. M. Kapp, The Gamifica ion o Lea ning and Ins uc ion Fieldbook: Ideas in o P ac ice, San F ancisco, Cali o nia: Wiley, 2013. 8. G. P. Kusuma, E. K. Wiga i, Y. U omo and L. K. Pu e a Su yap ana a, Analysis o Gamifica ion Models in Educa ion Using MDA F amewo k, P ocedia Compu . Sci.,135, pp. 385–392, 2018. 9. D. Diche a, C. Diche , G. Ag e and G. Angelo a, Gamifica ion in Educa ion?: A Sys ema ic Mapping S udy, Educ. Technol. Soc., 18(3), pp. 75–88, 2015. 10. G. Huesca Jua ´ ez and M. M. Reyes Ca ballo, Lea ning Gains , Mo i a ion and Lea ning S yles in a Gamified Class, In . J. Eng. Educ.,32(1), pp. 438–447, 2016. 11. E. Kim, L. Ro h ock and A. F ei alds, An Empi ical S udy on he Impac o Lab Gamifica ion on Enginee ing S uden s ’ Sa is ac ion and Lea ning, In . J. Eng. Educ.,34(1), pp. 201–216, 2018. 12. K. A. I. J. U. N. Chew, In e ac ion Be ween Lea ning Assessmen and S uden Mo i a ion?: A Scoping Re iew, In . J. Eng. Educ., 37(5), pp. 1313–1329, 2021. 13. C. E. Holb ey, Kahoo ! Using a game-based app oach o blended lea ning o suppo effec i e lea ning en i onmen s and s uden engagemen in adi ional lec u e hea es, Technol. Pedagog. Educ.,29(2), pp. 191–202, Ma . 2020. 14. B. Balak ishnan Nai , Endo sing gamifica ion pedagogy as a help ul s a egy o offse he COVID-19 induced dis up ions in ou ism educa ion, J. Hosp. Leis. Spo Tou . Educ., 30 (Sep embe 2021), p. 100362, 2022. Rosa Pa `mies-Vila `e al.1440 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 15. L. De-Ma cos, A. Domı ´nguez, J. Saenz-De-Na a e e and C. Page ´s, An empi ical s udy compa ing gamifica ion and social ne wo king on e-lea ning, Compu . Educ.,75, pp. 82–91, 2014. 16. N. Go don, M. B ayshaw and S. G ey, Maximising gain o minimal pain: U ilising na u al game mechanics, I al. Inno . Teach. Lea n. In . Compu . Sci.,12(1), pp. 27–38, 2013. 17. C.-H. Chen and H.-C. Yeh, Effec s o in eg a ing a ques ioning s a egy wi h game-based lea ning on s uden s’ language lea ning pe o mances in flipped class ooms, Technol. Pedagog. Educ.,28(3), pp. 347–361, May 2019. 18. O. Baydas and M. Cicek, The examina ion o he gamifica ion p ocess in unde g adua e educa ion: a scale de elopmen s udy, Technol. Pedagog. Educ.,28(3), pp. 269–285, May 2019. 19. M. Papas e giou, Digi al Game-Based Lea ning in high school Compu e Science educa ion: Impac on educa ional effec i eness and s uden mo i a ion, Compu . Educ.,52(1), pp. 1–12, 2009. 20. C. Hosseini, O. Humlung, A. Fage s øm and M. Hadda a, An expe imen al s udy on he effec s o gamifica ion on ask pe o mance, P ocedia Compu . Sci.,196(Janua y), pp. 999–1006, 2021. 21. M. D. Hanus and J. Fox, Assessing he effec s o gamifica ion in he class oom: A longi udinal s udy on in insic mo i a ion, social compa ison, sa is ac ion, effo , and academic pe o mance, Compu . Educ.,80, pp. 152–161, 2015. 22. A. Domı ´nguez, J. Saenz-De-Na a e e, L. De-Ma cos, L. Fe na ´ndez-Sanz, C. Page ´s and J. J. Ma ı ´nez-He a ´iz, Gami ying lea ning expe iences: P ac ical implica ions and ou comes, Compu . Educ.,63, pp. 380–392, 2013. 23. M. Bekk, R. Eppmann, K. Klein and F. Vo¨lckne , All ha gli e s is no gold: An in es iga ion in o he undesi ed effec s o gamifica ion and how o mi iga e hem h ough gamifica ion design, In . J. Res. Ma k., no. xxxx, 2022. 24. C. Pe o a, G. Fea he s one, H. As on and E. Hough on, Game-based Lea ning: La es E idence and Fu u e Di ec ions (NFER Resea ch P og amme: Inno a ion in Educa ion), Slough: NFER, 2013. 25. J. Hama i, J. Koi is o and H. Sa sa, Does Gamifica ion Wo k? – A Li e a u e Re iew o Empi ical S udies on Gamifica ion, in 47 h Hawaii In e na ional Con e ence on Sys em Science, pp. 3025–3034, 2014. 26. Z. Zainuddin, S. K. W. Chu, M. Shujaha and C. J. Pe e a, The impac o gamifica ion on lea ning and ins uc ion: A sys ema ic e iew o empi ical e idence, Educ. Res. Re .,30, p. 100326, 2020. 27. L. Rai skaya and E. Tikhono a, Gamifica ion as a field landma k in educa ional esea ch, J. Lang. Educ.,5(3), pp. 4–10, 2019. 28. M. Ranie i, J. E. Raffaghelli and I. B uni, Game-based s uden esponse sys em: Re isi ing i s po en ials and c i icali ies in la ge-size classes, Ac . Lea n. High. Educ., pp. 1–14, 2018. 29. D. B uff, Teaching wi h Class oom Response Sys ems: C ea ing Ac i e Lea ning En i onmen s, San F ancisco: Jossey-Bass, 2009. 30. C. Fies and J. Ma shall, Class oom Response Sys ems: A Re iew o he Li e a u e, J. Sci. Educ. Technol.,15(1), pp. 101–109, 2006. 31. B. And e ´s, R. Sanchis and R. Pole , Quiz game applica ions o e iew he concep s lea n in class: An applica ion a he Uni e si y con ex , in P oceedings o INTED2015 Con e ence, pp. 5654–5662, 2015. 32. V. J. Shu e, Focus on Fo ma i e Feedback, Re . Educ. Res.,78(1), pp. 153–189, 2008. 33. J. E. Caldwell, Clicke s in he La ge Class oom?: Cu en Resea ch and Bes -P ac ice Tips, CBE – Li e Sci. Educ.,6(1), pp. 9–20, 2007. 34. C. B. Paschal, Fo ma i e Assessmen in Physiology Teaching Using a Wi eless Class oom Communica ion Sys em, Ad . Physiol. Educ.,26(4), pp. 299–308, 2015. 35. S. W. D ape and M. I. B own, Inc easing in e ac i i y in lec u es using an elec onic o ing sys em, J. Compu . Assis . Lea n.,20, pp. 81–94, 2004. 36. J. I. Cas illo-Manzano, M. Cas o-Nun˜o, L. Lo´pez-Valpues a, M. T. Sanz-Dı´az and R. Yn˜iguez, Measu ing he effec o ARS on academic pe o mance: A global me a-analysis, Compu . Educ.,96, pp. 109–121, 2016. 37. N. J. Hunsu, O. Adesope and D. J. Bayly, A me a-analysis o he effec s o audience esponse sys ems (clicke -based echnologies) on cogni ion and affec , Compu . Educ.,94, pp. 102–119, 2016. 38. L. A. Van Dijk, G. C. Van De Be g and H. Van Keulen, In e ac i e lec u es in enginee ing educa ion, Eu . J. Eng. Educ.,26(1), pp. 15–28, 2001. 39. M. Cu o, L. O cos, P. J. Bla ´zquez and F. J. Molina, S uden Assessmen o he Use o Kahoo in he Lea ning P ocess o Science and Ma hema ics, Educ. Sci.,9(1), p. 55, 2019. 40. S. G. B yan , J. M. Co ell and B. M. Cla ke, Fun wi h pha macology: Winning s uden s o e wi h kahoo ! game-based lea ning, J. Nu s. Educ.,57(5), p. 320, 2018. 41. E. Za zycka-Pisko z, Kahoo i o no ? can games be mo i a ing in lea ning g amma ?, Teach. English wi h Technol.,16(3), pp. 17–36, 2016. 42. D. A. Gen ile, The Mul iple Dimensions o Video Game Effec s, Child De . Pe spec .,5(2), pp. 75–81, 2011. 43. O. Bo as-gene and M. Ma inez-nun, New Challenges o he Mo i a ion and Lea ning in Enginee ing Educa ion Using Gamifica ion in MOOC, In . J. Eng. Educ.,32(1(B)), pp. 501–512, 2016. 44. V. Simpson and M. Oli e , Elec onic o ing sys ems o lec u es hen and now: A compa ison o esea ch and p ac ice, Aus alas. J. Educ. Technol.,23(2), pp. 187–208, 2007. Impac o a Gamifica ion Lea ning Sys em on he Academic Pe o mance o Mechanical Enginee ing S uden s 1441 Appendix The 12 pool ques ions we e: Q1. Has Kahoo ! helped you o be e unde s and he subjec ? Q2. Ha e you become mo e mo i a ed because o Kahoo !? Q3. Is he ime you in es ed in Kahoo ! offse by how much you ha e lea ned? Q4. Does he Kahoo ! sco e eflec you unde s anding? Q5. Did you ha e enough ime o answe he Kahoo ! ques ions? 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 Rosa Pa `mies-Vila `e al.1442 Q6. Would you welcome he use o Kahoo ! in o he subjec s? Q7. Did Kahoo ! make you mo e a en i e o he class? Q8. I he Kahoo ! ques ions had mo e weigh in he e alua ion, would you ha e been mo e ca e ul/a en i e? Q9. Has Kahoo ! imp o ed you ela ionship wi h eache s? Q10. Is he compe i i eness c ea ed by Kahoo ! posi i e? Q11. Can Kahoo ! help you acqui e knowledge and cla i y concep s? Q12. Does he discussion o cla ifica ion o you sco e cla i y some concep s? Rosa Pa `mies-Vila ` ecei ed he PhD. deg ee in Biomedical Enginee ing in 2012 om Uni e si a Poli e `cnica de Ca alunya, Spain. She is assis an p o esso a he Mechanical Enginee ing Depa men o UPC, whe e she has been since 2008. He cu en esea ch a ea includes s udies on mechanisms and machine heo y, specifically in he field o biomechanics o human mo ion. Mo eo e , she has expanded he esea ch a ea o include eme ging educa ional echnologies and how gamifica ion can ans o m adi ional eaching me hods. Albe Fab ega -Sanjuan is an Assis an P o esso in he Depa men o Mechanical Enginee ing a he Uni e si a Ro i a i Vi gili (URV) since 2009. He has been wo king in he field o expe imen al and compu a ional mechanics esea ch and has augh se e al subjec s ela ed o mechanical enginee ing. As a coo dina o o se e al educa ion inno a ion p ojec s, his pa icula in e es ocuses on implemen ing inno a i e me hods o eaching, especially he use o gamifica ion and se ice-lea ning. He has been awa ded o he quali y o his eaching me hods in he collec i e (2015) and indi idual (2018) ca ego y om he URV Social Council. Joan Puig O iz ecei ed his PhD deg ee in Physics Science a Uni e si a Poli e `cnica de Ca alunya, Spain, in 2006. He is Collabo a ing Lec u e o he Mechanical Enginee ing Depa men o UPC. His esea ch in e es s a e in he a ea o mechanisms and machine heo y and he new me hods o eaching in STEM. Lluı¨sa Jo di Nebo ecei ed he PhD deg ee in Physics Science in 1999 om Uni e si a Poli e `cnica de Ca alunya, Spain. She is uni e si y lec u e a he Mechanical Enginee ing Depa men o UPC, whe e she has been since 1988. He esea ch in e es s a e in he a ea o mechanisms and machine heo y – specially, cons an -b ead h cam mechanism and ba linkages. She has expanded he esea ch a ea o include new educa ional echnologies and how ans o m adi ional eaching me hods. An oni He na ´ndez Fe na ´ndez has been an assis an p o esso o Physics and Educa ional Science a he Ins i u e o Educa ion Sciences (ICE–UPC) since 2008. He has also been a pa - ime eache in Seconda y School and Voca ional T aining a Te assa A and Design School since 1997. He has deg ees in Physics (1997) and Linguis ics (2003) and ecei ed his doc o a e in Linguis ics om he Uni e si a de Ba celona (UB) in 2014. His cu en esea ch in e es s include educa ion in a b oad sense (gamifica ion, p ojec -based lea ning, echnology educa ion, e c.) and quan i a i e linguis ics, i.e. he s udy o gene al pa e ns in communica ion sys ems (Zip ’s law, Menze a h-Al mann’s law, e c.). 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57