Engaging and Assessing Students through their Electronic Devices and Real Time Quizzes

Mul idisciplina y Jou nal o Educa ion, h p://dx.doi.o g/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Fe ándiz e al. (2016) h p://polipape s.up .es/index.php/MUSE/ Mul . J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 173 engaging and assessing s uden s h ough hei elec onic de ices and eal ime quizzes Fe ándiz, E.*, Puen es, C., Mo eno, P.J., Flo es, E. Depa men o Economics, Uni e sidad de Cádiz *es he [email p o ec ed]; Recei ed: 2016-01-23; Accep ed: 2016-03-13 Abs ac This pape desc ibes a eaching expe ience using Soc a i e, a hi d pa y elec onic ool, o eal- ime ques ioning in lec u es o Econome ics. Econome ics is a heo e ical-p ac ical subjec , bu adi ionally a la ge p opo ion o ou s uden s end o ocus on he p ac ical and disca d he heo y, o en skipping classes on heo y and a oiding s udying i s con en , p obably mo i a ed by i s complexi y. As a consequence, s uden s’ ma ks ob ained in he heo e ical pa o he exam a e usually low. In his con ex , we pu o wa d a change in ou eaching me hodology o include he use o Soc a i e, a eely a ailable app, ha allows s uden s o answe eache s’ sho , ue/ alse, o mul iple choice ques ions posed du ing each class using hei sma phones (o o he elec onic de ices wi h In e ne connec ion). The objec i es o his p ojec a e wo old: 1) o engage s uden s and inc ease a endance a lec u es; 2) o imp o e eedback on he lea ning p ocess. The esul s o a su ey o a sample o 186 s uden s e eal ha Soc a i e has been an e ec i e ool o achie ing hese objec i es. Keywo ds S uden s esponse sys em, clicke , quizzes, ac i e lea ning, s uden engagemen , Soc a i e, gami ica ion Mul idisciplina y Jou nal o Educa ion, h p://dx.doi.o g/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Fe ándiz e al. (2016) h p://polipape s.up .es/index.php/MUSE/ Mul . J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 174 1. In oduc ion In o ma ion echnology (IT) o e s a powe ul oppo uni y o inc easing s uden engagemen (Roblye & Wiencke 2003). P ominen examples o echnology include s uden esponse sys ems (SRSs), 1 which allow an ins uc o o pose ques ions o he class, allow s uden s o en e hei answe s in some kind o de ice (e.g., clicke o sma phone) and ins an ly summa ize and g aphically ep esen esul s o he ins uc o (Bea y 2004), who can decide whe he he esponses a e anonymous o no . Te ion and Ace i (2012) ha e highligh ed he impo ance o s uden engagemen because unengaged s uden s do no lis en, p ocess, o a end o he lea ning p ocess. Se e al pape s ha e analysed he ole o SRSs as a means o engaging s uden s and making hem ac i e in he class oom, among o he bene i s ( o a e iew see Aljaloud e al. 2015). Al hough much o he esea ch in SRS has adi ionally been ela ed o he use o clicke s, he de elopmen o cloud-based so wa e ha enables any de ice o become an SRS o e comes some o he c i icism o clicke s such as cos , and allows educa o s o exploi he po en ial o pe sonal elec onic de ices, mos ly sma phones, o educa ional pu poses. Al hough educa ion is no among he main uses o hese de ices ye , his use is expec ed o g ow in he coming yea s as new applica ions a e de eloped. Companies such as Apple ha e al eady o eseen his possibili y, de eloping iTunes U, an applica ion s o e especially ocused on educa ion. The highes a es o sma phone owne ship a e among he iche economies and he op coun ies in he ank include Sou h Ko ea (88%), 77% o Aus alia (77%), Is ael (74%), USA (71%) and Spain (71%), wi h a g ea e p esence among young people (Pew Resea ch Cen e , 2016). Gaming (65%), social ne wo king (54.3%), and ins an messaging (48.3%) a e among he mos common uses mino s and younge people ha e o hei sma phones’ ad anced ea u es (INTECO, 2011). These s a is ics a e in line wi h Yuso e al. (2012), who a gue ha sma phones a e mainly used o en e ainmen and social ne wo king. The e o e, he po en ial o sma phones o educa ion (m-lea ning) has ye o be unlocked. 1 SRSs a e also e e ed o as class oom pe o mance sys ems, audience esponse sys ems, pe sonal esponse sys ems, class oom communica ion sys ems, elec onic esponse sys ems, elec onic o ing sys ems, polling sys ems, o clicke sys ems (Aljaloud e al. 2015). Mul idisciplina y Jou nal o Educa ion, h p://dx.doi.o g/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Fe ándiz e al. (2016) h p://polipape s.up .es/index.php/MUSE/ Mul . J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 175 Addi ionally, he subjec o Econome ics augh in he Business and Adminis a ion Deg ee p og am elies on heo e ical easoning and compu e p ac ices ha s uden s o en ind di icul o unde s and. This, join ly wi h he limi ed ime a ailable in lec u es and he complexi y o he subjec , hinde he lea ning, comp ehension, and mo i a ion o s uden s when hey deal wi h Econome ics. Mo i a ed by all he abo e-men ioned easons, ins uc o s o Econome ics a he Uni e si y o Cadiz ha e p oposed an inno a ion p ojec based on Soc a i e, a cloud-based SRS. The main objec i es o he p ojec we e wo: 1) inc ease a endance and engagemen o s uden s in heo e ical classes and 2) imp o e he eedback p ocess o s uden s ega ding hei lea ning p ocesses. This pape is s uc u ed as ollows. Sec ion 2 summa izes he ad an ages and disad an ages o SRSs. Sec ion 3 desc ibes he me hodology and he implemen a ion o he s udy. Sec ion 4 desc ibes he esul s; and inally, in sec ion 5 we d aw some conclusions and commen on u u e wo k. 2. Bene i s and d awbacks o using s uden esponse sys ems Kay and LeSage (2009a) and, mo e ecen ly, Aljaloud e al. (2015) e iew he li e a u e on he bene i s o using SRSs, which a e ela ed no only o engagemen , bu also o in e ac i i y and academic pe o mance. Focusing on he o me , p e ious esea ch has shown ha SRSs enhance s uden engagemen in lea ning by c ea ing a un lea ning en i onmen . Fu he , he esea che s ound inc eased a endance and posi i e s uden a i udes and an inc ease in he s uden s’ desi e o imp o e hei pe o mance by iden i ying a eas o imp o emen . Because o inc eased engagemen , an SRS has he po en ial o imp o e academic pe o mance. Finally, SRSs imp o e he p ocess o ins uc o eedback since he ins uc o has mo e in o ma ion abou he lea ning p ocesses o hei s uden s. Howe e , he e a e also some d awbacks, such as ime equi ed o eache s o o mula e good ques ions and deal wi h echnical p oblems. O he o en-ci ed incon eniences o clicke s, such as cos o ime o deli e he clicke s o s uden s in each class, a e o e come ho ough he use o cloud-based so wa e such as Soc a i e, which enables any elec onic de ice (such as s uden s’ sma phone o able s) o be used as a clicke . Mul idisciplina y Jou nal o Educa ion, h p://dx.doi.o g/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Fe ándiz e al. (2016) h p://polipape s.up .es/index.php/MUSE/ Mul . J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 176 3. Me hodology This p ojec ocused on 385 s uden s en olled in Econome ics classes in he Business and Adminis a ion Deg ee p og am a he Uni e si y o Cadiz. Econome ics is augh in he second semes e o hi d-yea s uden s on h ee di e en campuses, in ol ing se en eache s. The a e age numbe o pa icipan s was 160, sp ead among i e classes (i.e., 32 s uden s pe class). The p ojec mainly consis ed o he egula use o Soc a i e quizzes du ing lec u es. The p ocedu e o de elop he ac i i y ollowed se e al s eps: 1. Design and aining o eache s. This phase includes he ollowing asks: a) Elabo a ion o ques ions in Soc a i e o each o he 10 lessons o he syllabus. Each quiz had 3-4 ques ions and was ocused on ele an con en s explained in class. We included mul iple choice and ue/ alse ques ions, bea ing in mind ha he mos e ec i e ques ions o be used in SRS a e hose (Kay and LeSage 2009b) ha allow s uden s o apply knowledge ecen ly acqui ed; a e highe le el; ocus on p ocess and easoning as opposed o ac ual con en ; iden i y and help esol e misconcep ions; and suppo a comp ehensi e e iew o a speci ic se o concep s. b) S a emen o he ules o he game. - The pa icipa ion wi h Soc a i e is indi idual unless he eache says he opposi e, and is limi ed o s uden s who ha e a ended all he classes. - The use o elec onic de ices is only allowed o esponding o eache s’ ques ions. - S uden s a e esponsible o b inging a wo king elec onic de ice wi h In e ne connec ion. The Uni e si y p o ides ee wo king Wi-Fi o s uden s in hei acili ies. c) Selec ion o he ewa d. S uden s who egula ly a end class and p o ide co ec answe s o a leas 75% o he ques ions would ge 0.5 ex a poin s added o hei inal ma k. d) T aining o eache s. The coo dina o o he p ojec de eloped a aining session o eache s and elabo a ed a isual guide on how o use Soc a i e, co e ing some o he ollowing aspec s: - How o c ea e a pe sonal accoun in Soc a i e.com - How o impo a quiz p e iously elabo a ed by he coo dina o and how o s a i . Rega ding he la e , we used he ollowing op ions:  S uden s pace he quiz, which allows hem o na iga e h ough i a hei own pace. Mul idisciplina y Jou nal o Educa ion, h p://dx.doi.o g/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Fe ándiz e al. (2016) h p://polipape s.up .es/index.php/MUSE/ Mul . J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 177  Disable immedia e eedback. We chose his op ion because we wan ed o keep he co ec answe concealed un il he en i e class had submi ed hei answe s o make chea ing less a ac i e.  Random ques ions and answe s. We used hese op ions o make chea ing mo e di icul . - How o e iew he class’ answe s, sa e epo s and ge ba g aphs o gi e eedback o he s uden s. - 2. In o ma ion session o s uden s and sys em es ing. In he i s class o he cou se, each eache in oduced he subjec o his/he class, including he me hodology and he e alua ion sys em. Rega ding he la e , he ins uc o in oduced Soc a i e and p esen ed he ules o he game. Subsequen ly, we made a ial es in class o check ha he In e ne connec ion wo ked well in he class oom and o ge s uden s amilia wi h he sys em. In o de o acili a e he link be ween s uden names and hei ma ks in Soc a i e, a oiding misspellings, s uden s we e asked o log in wi h hei iden i y ca d numbe ins ead o hei names. 3. Classes, quizzes and exi icke s. Once s uden s and eache s knew how o use Soc a i e, we began he cou se. In each session, a e he eache explained he heo e ical con en , he/she ga e he s uden s a quiz. By he end o he class, hey had o ob ain an “exi icke ” ha included he ollowing ques ions: How well did you unde s and oday’s ma e ial? Wha did you lea n in oday’s class? and, op ionally, a eache ’s open ques ion. The la e o en polled he s uden ’s desi e o explain a poin o poin s again o o de elop mo e deeply some speci ic ma e ials o concep s ha we e no clea enough o hem. A e comple ing each quiz, he ins uc o should e iew and p ojec he agg ega ed pe o mance o he class o each ques ion, commen on he co ec answe , and sa e he epo au oma ically gene a ed by Soc a i e. The o al es ima ed in-class ime de o ed o quizzes and exi icke s should be 10-12 minu es. Be o e he nex class, lec u e s had o e iew he exi icke s o check s uden s’ needs and hei eques s in he open ques ion. When all he quizzes had been un, eache s had o me ge he ma ks o each quiz wi h he s uden s’ names. 4. S uden s’ eedback on he p ojec . By he end o he cou se, we ga he ed s uden s’ opinion h ough a su ey o e alua e he con ibu ion o Soc a i e o hei lea ning p ocess. Mul idisciplina y Jou nal o Educa ion, h p://dx.doi.o g/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Fe ándiz e al. (2016) h p://polipape s.up .es/index.php/MUSE/ Mul . J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 178 4. Resul s In o de o ge in o ma ion abou s uden s’ sa is ac ion wi h he expe ience, hey we e asked o answe ques ions ela ed o he con ibu ion and limi a ions o Soc a i e. This su ey was answe ed h ough Google Fo ms. We use a Like scale om 1 o 5, whe e 1 is s ongly disag ee, 2 somewha ag ee, 3 neu al, 4 somewha ag ee, and 5 s ongly ag ee. The e we e 186 esponden s. Fo se e al ques ions, a ela i ely high pe cen age o answe s was concen a ed in he cen al alue, 3, bu his is no unusual because cen al endency is a common bias when using a Like scale. Figu e 1 shows ha 47.31% o s uden s s ongly/somewha ag ee ha Soc a i e enhances comp ehension o con en and acquisi ion o compe ence ela ed o Econome ics (20.43% s ongly/somewha disag ee). Figu e 2 shows ha 56.99% o s uden s ag ee ha Soc a i e has mo i a ed hem o a end o class e sus 23.12% who disag ee. Figu e 1. Soc a i e imp o es comp ehension o con en s and acquisi ion o compe ence ela ed o he subjec . Figu e 2. Soc a i e has mo i a ed me o a end class. I is no able ha 73.12% o he s uden s ag ee ha Soc a i e is a mo i a ion o lis ening o he eache (Figu e 3). Figu e 4 shows ha 55.91% o he s uden s ag ee ha Soc a i e has con ibu ed o helping hem memo ize some basic concep s, e sus 20.97% o he s uden s who disag ee. Acco ding o Figu e 5, 65.05% o he s uden s ag ee ha Soc a i e 3,23% 17,20% 32,26% 38,71% 8,60% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 9,14% 13,98% 19,89% 32,26% 24,73% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 Mul idisciplina y Jou nal o Educa ion, h p://dx.doi.o g/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Fe ándiz e al. (2016) h p://polipape s.up .es/index.php/MUSE/ Mul . J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 179 is help ul in es ing hei le el o unde s anding o he con en s explained by he eache (13.44% disag ee). Figu e 6 shows ha 50% o he s uden s ag ee ha Soc a i e has inc eased hei sel - con idence o answe ing eache s’ ques ions e sus 17.74% who disag ee. Figu e 7 demons a es ha 44.62% o he s uden s ag ee ha Soc a i e inc eased hei sel - con idence in hei lea ning p ocess in con as wi h 23.66% o he s uden s, who disag ee. Fu he , as ep esen ed in Figu e 8, 56.99% o he s uden s p e e Soc a i e o he adi ional sys em o aising hands (23.66% do no ). Figu e 3. Soc a i e has mo i a ed me o lis en o he eache . Figu e 4. Soc a i e has helped me o memo ize some basic concep s. 4,84% 4,84% 17,20% 45,70% 27,42% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 6,45% 14,52% 23,12% 34,95% 20,97% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 Mul idisciplina y Jou nal o Educa ion, h p://dx.doi.o g/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Fe ándiz e al. (2016) h p://polipape s.up .es/index.php/MUSE/ Mul . J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 180 Figu e 5. Soc a i e has con ibu ed o es ing my le el o unde s anding. Figu e 6. Soc a i e has inc eased my sel -con idence in answe ing eache s’ ques ions. Figu e 7. Soc a i e has inc eased my sel -con idence in my lea ning p ocess. Figu e 8. Soc a i e is be e han adi ional ques ioning sys ems (e.g., aising hands, di ec ques ions om he ins uc o ). 3,23% 10,22% 21,51% 43,01% 22,04% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 12345 8,06% 9,68% 32,26% 34,95% 15,05% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 7,53% 16,13% 31,72% 34,41% 10,22% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 12345 10,75% 12,90% 19,35% 25,27% 31,72% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 Mul idisciplina y Jou nal o Educa ion, h p://dx.doi.o g/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Fe ándiz e al. (2016) h p://polipape s.up .es/index.php/MUSE/ Mul . J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 181 Figu e 9 makes clea he pe cei ed con ibu ion o eedback in compa ison o o he heo e ical subjec s, wi h 67.2% o he s uden s ag eeing wi h his i em (only 9.14% disag ee). Figu e 10 shows ha 61.29% o he s uden s ag ee ha Soc a i e acili a es in e ac ion while only 15.05% o s uden s disag ee wi h his s a emen . Al hough we do no p esen a de ailed g aph he e, 68.81% o he s uden s s ongly/somewha ag ee ha Soc a i e con ibu es o making classes mo e pleasan and un (only 10.75% o he s uden s disag ee). Finally, Table 1 p esen s he esul s o ques ions abou he impo ance o some o he incon eniences s uden s could ace wi h he use o Soc a i e. Ou esul s show ha he mos ele an incon enience was ela ed o In e ne connec ions and p oblems wi h he de ice. Ou esul s sugges ha using Soc a i e does no hinde class dynamics and does no p e en s uden s om going back o he lesson and lis ening o he ins uc o . F om he ins uc o s’ iewpoin , he expe ience is also posi i ely e alua ed, and we a e mo i a ed by he esul s ob ained in he s uden s’ su ey. Al hough i ep esen ed an ex a e o ha equi ed eache commi men and sligh changes in he eaching me hodology, he sys em is easy, as , and con enien and can be used on a egula base. Figu e 9. I gi es mo e eedback o he eache han in o he heo e ical subjec s. Figu e 10. I acili a es in e ac ion. 4,30% 4,84% 23,66% 35,48% 31,72% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 12345 6,99% 8,06% 23,66% 37,10% 24,19% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5