Following-Up on Feedback through Repetition inAssessments (in the Science Disciplines)

AISHE-J Volume 1, Numbe 1 (Au umn 2009) Page 5.1 Following-Up on Feedback h ough Repe i ion in Assessmen s (in he Science Disciplines)* Séamus McLoone Na ional Uni e si y o I eland Maynoo h I eland Abs ac In p ac ice, he s uden lea ning cycle in ol es lea ning, assessmen and eedback. The s uden lea ns abou a pa icula opic h ough a combina ion o lec u es, esea ch and/o ac ion. They a e hen assessed on wha hey ha e lea ned and a e gi en o ma i e eedback on hei pe o mance. Un o una ely, he cycle seems o end a his poin . A new opic is p esen ed and he lea ning cycle epea s. This is ypical o a module in which se e al di e en opics a e co e ed o e he cou se o a semes e . Howe e , some ob ious ques ions eme ge. Wha do he s uden s do wi h he eedback? Do hey eally lea n om i ? How do we know? This pape p oposes o ollow-up on eedback by simply ensu ing ha a p opo ion o he nex assessmen co e s ma e ial ha s uden s had di icul y wi h on he p e ious one. S uden s a e in o med o his ac so as o encou age hem o ac i ely engage wi h he eedback. Tes esul s show a ma ked imp o emen in he s uden s’ pe o mance, pa icula ly in he case o he epea ed ma e ial, while s uden eedback esponds a ou ably o his me hod o ensu ing ha he s uden lea ning cycle is p ope ly comple ed. De ails o he assessmen s, es esul s and s uden eedback, as well as some pe sonal obse a ions and discussions, a e p esen ed wi hin. Keywo ds: assessmen , eedback, he s uden lea ning cycle * Resea ch A icle. URL: h p://ojs.aishe.o g/index.php/aishe-j/a icle/ iew/5 All I eland Jou nal o Teaching and Lea ning in Highe Educa ion (AISHE-J) C ea i e Commons A ibu ion-NonComme cial-Sha eAlike 3.0 AISHE-J Volume 1, Numbe 1 (Au umn 2009) Page 5.2 1. In oduc ion “Assessmen is a he hea o he s uden expe ience” acco ding o B own and Knigh (1994, p.1). As eloquen ly desc ibed by B own (2001, p.4), assessmen “de ines wha s uden s ega d as impo an , how hey spend hei ime and how hey come o see hemsel es as s uden s.” The li e a u e clea ly highligh s ha assessmen is indeed a key elemen in s uden lea ning – “i you wan o change s uden lea ning hen change he me hods o assessmen ” (G. B own e al. 1997, p.7). Shuell (1986, p.429) s a es ha “wha he s uden does is ac ually mo e impo an in de e mining wha is lea ned han wha he eache does.” Righ ly o w ongly, s uden s ocus mo e on wha is assessed a he han on wha is augh . As such, assessmen should be ega ded as an in insic pa o he lea ning p ocess and, along wi h he eaching me hod, should be ca e ully aligned wi h he lea ning ou comes in o de o suppo e ec i e s uden lea ning. This p ocess o cons uc i e alignmen is undamen al o e ec i e eaching (Biggs 1996, pp.347-364; Biggs 2002; Biggs 2004, pp.11-33). The issues associa ed wi h assessmen , anging om wha assessmen is o p o iding e ec i e assessmen , a e well documen ed in he esea ch li e a u e (Knigh 1994; Ha is & Bell 1994; Heywood 2000; Race e al. 2005; Angelo & C oss 1993; G. B own e al. 1997; Moon 2002). One impo an aspec ha is e iden h oughou he li e a u e is he dis inc ion be ween he wo ypes o assessmen , namely o ma i e and summa i e. The o me is mainly used o imp o e s uden lea ning while he la e is used o iden i y wha he s uden has lea n (Rus 2000). In gene al, o ma i e assessmen and he co esponding eedback is mos e ec i e when gi en h oughou a semes e , so ha s uden s can lea n om hei mis akes. This in ol es gi ing he s uden eedback se e al imes o e he cou se o a semes e so ha he s uden can imp o e upon u u e pe o mance. Summa i e eedback is gene ally associa ed wi h he inal exam a which poin he only eedback a s uden ecei es is ha o a inal g ade. In s ic e ms, o ma i e assessmen does no con ibu e o he inal module g ade. Howe e , in p ac ice, assessmen s a e o en designed o p o ide a mix u e o bo h o ma i e and summa i e eedback. As eache s, i is impo an ha we s i e o mo e o ma i e eedback in ou assessmen by making mo e use o con inuous assessmen and mo ing u he away om he inal exam scena io. Fo ma i e eedback is a i al ac o in suppo ing s uden lea ning. Race (1996) eels ha lec u e s can help lea ning o ake place by p o iding eedback o s uden s and helping hem make sense o wha hey a e lea ning. Gosling (2003, p.172) s a es ha p o iding eedback is one o “ he mos impo an aspec s o suppo ing s uden lea ning”. This sen imen is suppo ed by Rus (2002, p.149) when he s a es ha assessmen aids s uden lea ning h ough eedback. B own e al. (1995, p.81) s a e ha he key o using assessmen as he “engine o lea ning” is o make su e ha each assignmen p o ides plen y o oppo uni ies o s uden s o ecei e imely posi i e eedback. Ramsden (1992, p.99) lis s app op ia e assessmen and eedback as one o he “six key p inciples o e ec i e eaching”. Ha is and Bell (1994) belie e ha “assessing should ha e a s ong o ma i e ole” and, in addi ion, eel ha in o de o assessing o be uly o ma i e, i should in ol e discussions be ween he lec u e and he s uden s. AISHE-J Volume 1, Numbe 1 (Au umn 2009) Page 5.3 As al eady men ioned, assessmen s in p ac ice s i e o p o ide a blend o bo h o ma i e and summa i e eedback. Howe e , i is a guable ha mos assessmen s ocus oo much on he summa i e aspec and no enough on p o iding s uden s wi h o ma i e eedback. Mo eo e , e en in he cases, whe e o ma i e eedback is o he o e, ano he p oblem eme ges. B own (2001) a gues ha “un o una ely, he eedback p o ided is no always ead, le alone used by he s uden s”. He ollows his poin wi h a sugges ion ha s uden s need o be augh how o use he eedback and ha we, as lec u e s, need o check ha he s uden s ha e used eedback om hei p e ious assignmen s. Yo ke (2002, p.39) eels ha lec u e s need o spend ime helping s uden s o use he eedback hey ecei e. The cu en esea ch li e a u e con ains nume ous a icles ela ing o he opic o eedback, anging om collec ing eedback o using and ac ing on eedback o help s uden s lea n (Hoga h e al. 2003; G. B own 2001; Race 2005). The ‘S udy Guides and S a egies’ web si e1 p o ides a use ul guide o s uden s on how hey hemsel es can make e ec i e use o he eedback gi en o hem. Simila ly, Race (1999) o e s ad ice o s uden s on “using eedback o make ac ion plans”. Howe e , he e appea s o be, om wha I can asce ain, a dea h o in o ma ion ela ing o B own’s sugges ion on checking up on he s uden s’ use o he eedback gi en. In o he wo ds, ha e hey ully app ecia ed he eedback ecei ed? Will hey use i co ec ly in ele an u u e ci cums ances? How do we know? This p ocess o checking-up on he s uden ’s use o eedback can be iewed as he inal componen o he s uden lea ning cycle. In simple e ms, he ull cycle in ol es lea ning, assessmen , eedback and imp o emen . This i s closely wi h he p ocess model o expe ien ial lea ning desc ibed by Dennison and Ki k (1990). This model, based on he model by Kolb (1984), clea ly iden i ies ou key s eps. These can be iewed as he s uden doing a new piece o wo k, e iewing hei e o s h ough assessmen , lea ning om hei mis akes h ough eedback and applying hei newly acqui ed in o ma ion in a u u e con ex . I is his la e s ep ha is ypically missing in cu en eaching p ac ice, wi h he excep ion o he inal exam, in which a s uden has a chance o apply hei knowledge. Howe e , his occu s oo la e and is a guably o li le o no bene i o he s uden ’s lea ning. As eache s, i is ou du y o p o ide an en i onmen in which s uden s a e suppo ed and encou aged o lea n. As pa o his, i is impo an ha we ensu e ha he s uden lea ning cycle is comple e. In o he wo ds, we need o ollow he “do- e iew-lea n-apply” p ocess model o lea ning in i s en i e y. As such, i is impo an ha we check ha s uden s a e making use o he eedback we gi e hem and ha hey apply his eedback in a co ec ashion. I eel ha a use ul me hod o achie ing his is o ha e epe i ion o ma e ial ac oss se e al assessmen s. He e ma e ial can e e o bo h con en and skill. I is he aim o his pape o show ha epea ing ma e ial ac oss mul iple assessmen s will encou age s uden s o ollow-up on he eedback gi en o hem. The esul s ob ained sugges ha his is indeed he case. The es o his pape is s uc u ed as ollows. The nex sec ion ou lines he me hodology used o his esea ch. A b ie o e iew o he educa ional si ua ion is also p o ided. Tes esul s and analysis a e ou lined in sec ion 3. This sec ion also includes bo h s uden and pee eedback. The pape ends in sec ion 4 wi h some conclusions and sugges ions o u u e wo k. 1 “Since 1996 he S udy Guides and S a egies web si e has been esea ched, au ho ed, main ained and suppo ed by Joe Landsbe ge .” URL: h p://www.s udygs.ne / AISHE-J Volume 1, Numbe 1 (Au umn 2009) Page 5.4 2. Me hodology I each a ma hema ical module on op imisa ion, as pa o he Bachelo o Enginee ing deg ee p og amme in he Depa men o Elec onic Enginee ing, Na ional Uni e si y o I eland (NUI) Maynoo h. While his module akes place in he inal yea o he deg ee p og amme, i is ne e heless he i s one on he opic o op imisa ion. As a esul , a signi ican p opo ion o he con en can be ega ded as elemen a y in e ms o he discipline. This module is s uc u ed o ha e h ee lec u es pe week o an eigh week pe iod. The e a e h ee class es s ha ake place on he hi d, i h and se en h week espec i ely. I is hese class es s ha I used o ca y ou he esea ch wo k o his pape . In 2008, he class consis ed o eigh in e na ional s uden s and six I ish s uden s, wo o which we e ma u e. The o e all gende balance was 3 emales o 11 males. Such igu es ep esen a ypical b eakdown o mos Elec onic Enginee ing classes a NUI Maynoo h o da e. I use mul iple class es s (one o m o con inuous assessmen ) o se e al easons: hey allow s uden s o moni o hei own lea ning; hey p o ide s uden s wi h he oppo uni y o ecei e eedback and, hence, imp o e hei own lea ning; hey encou age s uden s o wo k consis en ly h oughou he semes e ; hey po en ially imp o e s uden mo i a ion, pa icula ly when he class es s con ibu e owa ds he inal module g ade (McLoone 2003, pp.64-67). The class es s con ain a mix u e o ques ions, anging om s aigh o wa d su ace lea ning ype ques ions o mo e complex ones which os e a deep lea ning app oach, whe e he concen a ion o such ques ions depends mainly on he ype o ma e ial being assessed. Wi h he la e ques ion s yle, i is hoped ha s uden s will de elop a “good unde s anding, good long- e m ecall and be e ma ks” (G. Gibbs 1995). In gene al he class es s a e designed o co e di e en sec ions o he no es in a gi en module. P e iously, he e was no o e lap om one es o he nex in e ms o he ma e ial being assessed. This was especially ue in he case whe e one sec ion o he no es is qui e di e en o ano he . The p oblem, he e o e, was ha when I ga e eedback o my s uden s, I did no ac ually ollow-up on i . Hence I did no know i he s uden s had gained any bene i om he eedback o , indeed, i I needed o gi e u he eedback o hem. In an e o o emedy his p oblem, I decided o ca y ou he ollowing p ocedu al s eps in ela ion o he h ee class es s o my Op imisa ion module: 1. Conduc class es 1 in Week 3. 2. Co ec class es 1 and p o ide indi idual and g oup eedback o s uden s, whe e g oup eedback ela es o common mis akes made by se e al o he s uden s. 3. Ga he eedback om s uden s ega ding he i s class es . 4. In o m s uden s ha class es 2 will con ain a p opo ion o ques ions ela ing o eedback om class es 1. 5. Conduc class es 2 in Week 5. 6. Co ec class es 2 and, once again, p o ide indi idual and g oup eedback o s uden s. 7. Ga he eedback om s uden s ega ding he second class es , wi h pa icula ele ance o he ques ions ela ing o ma e ial co e ed in class es 1. 8. Conduc class es 3 in Week 8. AISHE-J Volume 1, Numbe 1 (Au umn 2009) Page 5.5 This me hodology closely aligns wi h he cyclical na u e o he ac ion esea ch model p oposed by Ellio (1991). All class es s we e conduc ed online using he Moodle i ual lea ning en i onmen 1. The es s consis ed o eigh o nine ques ions wi h a ying leng h and di icul y. Each es las s exac ly one hou and con ibu es 5% o he inal module g ade. Sample ques ions om he class es s a e shown below. Ques ion X1 Conside he unc ion (x) = 2x2 + 8x + 1. Gi en he ini ial b acke ing in e al [-3, 1] and an e o ole ance o 0.02, answe each o he ollowing ques ions. Whe e applicable, gi e you answe o 3 decimal places. (i) applying one i e a ion o he Fibonacci sea ch me hod would gi e he educed in e al o : [ , ]T (ii) wi hou ac ually calcula ing he inal local minimum es ima e p oduced by he Fibonacci sea ch me hod, we can s ill say ha his alue would ha e a maximum local e o o (iii) had we, om he s a , used he Golden Sec ion sea ch me hod ins ead (wi h a golden a io o 0.618), he leng h o he i s educed in e al would be (i ) again, wi hou ac ually calcula ing he inal local minimum es ima e p oduced by he Golden Sec ion sea ch me hod, we can s ill say ha his alue would ha e a maximum local e o o Ques ion Y1 Conside he unc ion (x) = 3x3 + 1.5x2 + 3.5. We wan o es ima e he local minimum o his unc ion using he G id line sea ch me hod. Selec , om he ollowing lis , ALL he sui able s a ing b acke ing in e als: Choose a leas one answe . a. [-3, -1] b. [-0.33, 0.1] c. [-1, 1] d. [-2, -1] e. [-0.33, -0.1] . [-0.1, 0.1] 1 URL: h p://www.moodle.o g/ AISHE-J Volume 1, Numbe 1 (Au umn 2009) Page 5.6 Ques ion X2 Conside he unc ion (x) = 3x2 + 5x + 4. Gi en he ini ial b acke ing in e al [-3, 2] and an e o ole ance o 0.1, answe each o he ollowing ques ions. (i) applying one i e a ion o he Fibonacci sea ch me hod would gi e he educed in e al o [ , ]T (ii) applying one i e a ion o he Golden Sec ion sea ch me hod, using a golden a io o 0.6, would gi e he educed in e al o [ , ]T (iii) applying one i e a ion o he G id sea ch me hod, using 5 di isions, would gi e he educed in e al o [ , ]T (i ) ha ac ual minimum o (x) is Ques ion Y2 The Quad a ic sea ch me hod is o be applied o he unc ion (x) = 8x2 + 4x + 3. Selec om he ollowing lis ALL sui able se s o s a ing poin s o his app oach: Choose a leas one answe . a. -1, 0 and 1 b. -1, 0 and 2 c. 0, 1 and 2 d. -3, -2 and -1 e. 0, -2 and 2 . -10, -5 and 20 g. None o he abo e Ques ions X1 and Y1 a e aken om class es 1 while ques ions X2 and Y2 a e aken om class es 2. These pa icula ques ions we e chosen as hey ep esen ma e ial ha s uden s s uggled wi h on he i s class es . The emainde o he ques ions on class es 2 ela ed o new ma e ial. In he nex sec ion, he esul s om he class es s a e summa ised and analysed. AISHE-J Volume 1, Numbe 1 (Au umn 2009) Page 5.7 3. Resul s and Analysis 3.1 Tes esul s The a e age esul o each o he class es s a e gi en in Table 1 below. The i s class es had no p io eedback gi en and, hus, can a guably ac as he con ol pa ame e . The o he wo es s we e subjec o some p io eedback, ela i e o he class pe o mance on he p e ious es . As such, hese es s can be iewed as he es pa ame e s. A i s glance, he p ocess o gi ing eedback and pa ly e-assessing he same ma e ial seems o ha e wo ked e y well, wi h a signi ican inc ease in he class a e age om class es 1 o class es 2. Howe e , i is impo an o conside hese esul s in he app op ia e con ex . Class Tes 1 2 3 Class A e age (%) 41.4 70.4 69.0 Table 1: Class Tes Resul s Fi s ly, i should be no ed ha he s uden s we e ela i ely unp epa ed o he i s class es . They did no know wha o expec in e ms o ques ions bu , mo e impo an ly, hey did no know wha o expec om an online class es . I was he i s ime o all he s uden s o ake an online class es and no only did hey ha e o ace he challenge o he es i sel bu hey also had o de elop a new s a egy o coping wi h an online assessmen . In e es ingly, one s uden epo ed ha hey el ha he idea o looking a he compu e sc een o he ques ion and hen wo king ou he answe s on pape and hen yping he answe on he compu e was no he ideal es scena io o hem. Fo class es s 2 and 3, hey knew wha o expec in e ms o he es o ma and he e o e could concen a e comple ely on he es i sel . I eel ha his was one o he key easons o he as imp o emen om he i s class es o he o he s. In my p e ious yea o eaching his module, I had a simila inc ease in ma ks om class es 1 o 2. The class a he ime, consis ing o 13 s uden s, ob ained an a e age o 53% in he i s class es and 78% in he second one. In his case, he e was no epe i ion in he ma e ial assessed and ye he s uden s’ pe o mance imp o ed d ama ically om one es o he nex . Feedback om hose s uden s was simila in na u e o he cu en eedback, i.e., hey we en’ used o he o ma o he online class es . Some had eques ed a ial un o he online class es be o e he eal hing. This is some hing ha I in end implemen ing on a ial basis in he u u e. Secondly, i ’s impo an o no e ha he class size was 14 s uden s and he e o e, wi h such a ela i ely small sample, i is di icul o d aw any absolu e conclusions om he esul s gi en. Fu u e expe imen a ion on his opic should p o ide an inc ease in da a and be e e i ica ion o conclusions. A mo e in e es ing and ele an se o esul s can be ob ained by looking a a speci ic compa ison be ween wo ques ions in class es 1 (ques ions X1 and Y1) and a ela ed wo ques ions in class es 2 (ques ions X2 and Y2). These se s o ques ions, while no iden ical, co e ed simila ma e ial which was s udied p io o class es 1, i.e., he wo ques ions on class es 2 we e e-assessing he ma e ial which, on a e age, he s uden s s uggled wi h on AISHE-J Volume 1, Numbe 1 (Au umn 2009) Page 5.8 he i s class es . The s uden s we e gi en eedback ega ding hei mis akes on hese ques ions a e class es 1. The accumula i e ma ks ob ained o hese wo speci ic ques ions we e eco ded o each s uden o bo h class es s and a e gi en in Table 2 below. Fo he pu poses o cla i y, he di e ence be ween he wo se s o ma ks is plo ed o each o he 14 s uden s in Figu e 1. These esul s clea ly show ha all bu 2 o he s uden s imp o ed hei ma ks in ela ion o he e-assessed ma e ial. Thus, hese s uden s clea ly used he eedback gi en o hem in a posi i e manne . I is in e es ing o no e ha s uden 3 shows he bigges inc ease in ma ks ob ained. In his case, he s uden ob ained almos ze o ma ks in he a o emen ioned ques ions in he i s class es and almos ull ma ks on he second a emp . A he o he end o he spec um, 2 o he 14 s uden s ecei ed a wo se g ade a he second ime o asking. While he ques ions on class es 2 we e no iden ical o ha o class es 1, I s ill ind i su p ising ha some s uden s could do signi ican ly poo e on he second a emp . Pe haps i is possible ha he wo s uden s in ques ion concen a ed mo e on he new ma e ial o he de imen o he olde ma e ial. A e all, hei o e all indi idual g ade o class es 2 showed as imp o emen o e hei espec i e class es 1 g ades. This is an issue o u u e s udy. S uden No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 % Ma k o wo speci ic ques ions on Class Tes 1 70 0 10 20 30 10 30 20 70 27 30 70 70 60 % Ma k o wo speci ic ques ions on Class Tes 2 100 25 98 59 64 22 5 73 53 86 65 79 100 52 % Di e ence 30 25 88 39 34 12 23 53 -17 59 35 9 30 -8 Table 2: Compa ison o Tes Resul s o wo speci ic ques ions Figu e 1: Pe cen age di e ence in ma ks om class es 1 o class es 2 o wo speci ic ques ions S uden No. Pe cen age di e ence -40 -20 0 20 40 60 80 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 AISHE-J Volume 1, Numbe 1 (Au umn 2009) Page 5.9 3.2 S uden eedback A e class es 2, I asked he s uden s o p o ide eedback on he e ec i eness o my ac ion esea ch idea. In summa y, he s uden s ound ha epea ing examined ma e ial om class es 1 was qui e help ul (a e age a ing o 3.86 ou o 5) and ound ha he eedback om class es 1 was e y help ul (a e age a ing o 4.43 ou o 5). No s uden esponded nega i ely o ei he aspec . Due o ime cons ain s, I was unable o epea he ques ionnai e a e class es 3, bu ne e heless, one in e es ing e en did ake place. One o my s uden s came o me o seek cla i y on one o he ques ions om he p e ious class es on he mo ning ha class es 3 was aking place. He did no ask any ques ions abou he new ma e ial. The idea o ha ing ques ions on p e iously co e ed ma e ial clea ly had he desi ed e ec on his s uden . 3.3 Pee Feedback Se e al o my pee s showed posi i e in e es in my esea ch idea and concu ed wi h my assessmen ha , in p ac ice, we ypically ail o ollow-up on eedback. This was eloquen ly summed up by one membe who ema ked ha “… we do he assessmen , we p o ide he eedback and we ick he box – ou wo k is done … ”. Un o una ely, oo many eaching s a do hei job because hey ha e o and no because hey wan o and, as a esul , hei a i ude some imes becomes one o doing he ba e minimum. One c i ical iend also exp essed how simple my idea was and ye how e ec i e i could po en ially be. The ac ha he idea was so su p isingly simple made i e en mo e e ec i e and some o he eaching s a , pa icula ly hose who conduc mul iple assessmen s, el ha i was some hing ha hey could easily implemen since i was no an o e ly ime-consuming exe cise. An in e es ing commen was men ioned in ela ion o he epe i ion o assessed ma e ial, as one o my pee s wonde ed i I was excessi ely penalising a s uden o e ec i ely making he same mis ake wice. This would no be an issue i I was ca ying ou s ic ly o ma i e assessmen s, which did no con ibu e o he inal module g ade. Since my assessmen s a e bo h o ma i e and summa i e, whe e he assessed ma ks do con ibu e o he inal g ade, he aised conce n wa an ed some discussion. Howe e , we bo h el , on e lec ion, ha awa ding ma ks o he epea ed assessmen s ongly encou aged s uden s o ac i ely engage wi h he eedback. Fu he mo e, as he epea ed sec ion cons i u ed a ela i ely small pa o he o e all assessmen , we bo h el ha any penal ies encoun e ed by a s uden we e minimal. 4. Conclusions and Fu u e Wo k The wo k in his pape clea ly shows ha he idea o ollowing-up on eedback h ough epe i ion o assessed ma e ial has po en ial alue. My ellow eaching s a el ha i was a wo hwhile exe cise and my s uden s wholehea edly ag ee. Mo e impo an ly, he esul s om he class es s ce ainly showed a ma ked imp o emen om one es o he nex in ela ion o he epea ed ma e ial. Howe e , i should be no ed ha amilia isa ion o online es ing may also ha e impac ed on his imp o emen . Fu u e wo k will de e mine he deg ee o which his is indeed he case.