Experimenting with the structured assessment dialogue in physics and mathematics classes

This is an elec onic ep in o he o iginal a icle. This ep in may di e om he o iginal in pagina ion and ypog aphic de ail. Au ho (s): Ti le: Yea : Ve sion: Please ci e he o iginal e sion: All ma e ial supplied ia JYX is p o ec ed by copy igh and o he in ellec ual p ope y igh s, and duplica ion o sale o all o pa o any o he eposi o y collec ions is no pe mi ed, excep ha ma e ial may be duplica ed by you o you esea ch use o educa ional pu poses in elec onic o p in o m. You mus ob ain pe mission o any o he use. Elec onic o p in copies may no be o e ed, whe he o sale o o he wise o anyone who is no an au ho ised use . Expe imen ing wi h he s uc u ed assessmen dialogue in physics and ma hema ics classes Nieminen, Pasi; Hähkiöniemi, Ma kus; B uun, Jespe ; Vii i, Jouni Nieminen, P., Hähkiöniemi, M., B uun, J., & Vii i, J. (2017). Expe imen ing wi h he s uc u ed assessmen dialogue in physics and ma hema ics classes. In P oceedings o he annual FMSERA symposium 2016 (pp. 66-75). Finnish Ma hema ics and Science Educa ion Resea ch Associa ion (FMSERA). P oceedings o he FMSERA annual symposium. h ps://jou nal. i/ mse a/a icle/ iew/60944/27039 2017 Nieminen e al. P oceedings o he annual FMSERA symposium 2016 66 EXPERIMENTING WITH STRUCTURED ASSESSMENT DIALOGUE IN PHYSICS AND MATHEMATICS CLASSES Pasi Nieminen1, Ma kus Hähkiöniemi1, Jespe B uun2 and Jouni Vii i1 1Uni e si y o Jy äskylä, 2Uni e si y o Copenhagen ABSTRACT This s udy deals wi h a o mal, p e-planned eache –s uden discussion, called s uc u ed assessmen dialogue (SAD) ha assesses s uden s’ inqui y compe ences by combining o ma i e and summa i e assessmen s. Six SAD sessions ( ou lowe -seconda y physics and wo uppe -seconda y ma hema ics) we e implemen ed, ideo eco ded and analysed using heo y-based ca ego ies and ne wo ks. The esul s showed ha s uden s usually ga e lowe -o de answe s e en hough he eache asked highe -o de ques ions. In he lowe seconda y, he eache s uggled o engage s uden s by using wha s uden s hemsel es had said. In he uppe seconda y, he eache ocused on add essing a pa icula idea, which was di icul o he s uden . Despi e he challenges, SAD could be a use ul me hod in Finnish schools i eache s and s uden s acclima e o i . INTRODUCTION Class oom assessmen ends o e alua e s uden s’ achie emen s o g ading o o p omo e hei lea ning by means o con inual eedback abou p og ess and di icul ies conce ning he lea ning p ocess. In educa ional li e a u e (e.g., Ha len & Qual e , 2014), hese wo aims gene ally a e known as summa i e and o ma- i e assessmen s, espec i ely. In a good assessmen , he decen c i e ia o a ain he lea ning goals inc ease he alidi y o he summa i e assessmen and help communica e he s uden s’ lea ning p og ess o o ma i e pu poses. S uden lea ning can be assessed by using a ious da a sou ces abou hei lea ning, such as es s and di e en s uden p oduc s. Likewise, i can be assessed by means o class oom dialogue, which has been s udied widely and ecognised as cen al o success ul lea ning (Alexande , 2006; Mo ime & Sco , 2003). In class oom dia- logue, s uden s can exp ess and sha e hei hinking and ideas, and he eache has he oppo uni y o explo e hese. The e a e s udies abou eache s’ o ma i e p ac ices du ing unplanned spon aneous assessmen con e sa ions (e.g., Niemi- nen, Hähkiöniemi, Leskinen & Vii i, 2016; Ruiz-P imo & Fu ak, 2006). Du ing Nieminen e al. P oceedings o he annual FMSERA symposium 2016 67 hese con e sa ions, eache s need, in eal ime, o elici in o ma ion om s u- den s’ unde s anding and make decisions abou he amoun and manne o guid- ing ha will help s uden s in hei lea ning. He e, he eache ’s discu si e ac ions, such as asking ques ions and main aining discussion, play a cen al ole (Chin, 2007; Dys he, 1996; Sahin & Kulm, 2008). Al hough he e a e many s udies abou class oom dialogue, mos esea ch has been done in au hen ic class oom se ings du ing whole class o eache –s uden discussions. In his s udy, we ocused on a no el me hod called s uc u ed class- oom dialogue (SAD). He e, he eache –s uden discussions we e p e-planned in o de o assess how lea ning c i e ia had been a ained a e a ce ain inqui y ac i i y. Thus, he eache needed o plan main ques ions be o ehand acco ding o he c i e ia bu s ill was equi ed also o pe o m spon aneous discu si e ac- ions, such as ollow-up ques ions. SAD can be used o o ma i e and summa- i e pu poses and can be s udied om di e en pe spec i es, bu his pape o- cuses only on he discu si e ac ions o he eache and s uden s. Howe e , as SAD is a no el me hod, we p esen i gene ally. The SAD was conduc ed h ee imes in h ee classes by one physics and one ma hema ics eache . Video–audio da a was analysed in o de o answe he ollowing esea ch ques ion: How did he eache s and s uden s use di e en discu si e ac ions du ing he SAD in ela ion o he lea ning c i e ia? METHODS Desc ip ion o SAD SAD akes place a e s uden s ha e unde aken an inqui y ac i i y. Be o e s a - ing he inqui y ac i i y, he eache explains o he s uden s he ask and he ac- i i y-speci ied lea ning c i e ia. The c i e ia help s uden s unde s and wha hey a e expec ed o pe o m, and hey help he eache assess he s uden s’ wo k. Ta- ble 1 shows he lea ning c i e ia o a physics inqui y ac i i y abou “In e ac ion o a magne and a coil”. A e he inqui y, he SAD s a s as a i ualised dialogue be ween he eache and one s uden ( he ocus-s uden dialogue), hen mo es on o a pee - eedback phase, in which he ocus s uden holds a discussion wi h a g oup o eedback s uden s (5–7 pe sons). Du ing hese in e ac ions, he es o he class obse es and e lec s on hei own unde s anding. The du a ion o he ocus-s uden dialogue is abou 5 minu es. Fo his discus- sion, he eache has p epa ed ques ions ha help map he s uden s’ a ainmen o he lea ning c i e ia in he inqui y ac i i y (see example in Table 1). The eache should pose ques ions ha ollow up on s uden s’ esponses and u e ances, and Nieminen e al. P oceedings o he annual FMSERA symposium 2016 68 he eache should alue he s uden s’ con ibu ions as well as challenge he s u- den s’ hinking and easoning. The eache also may add new in o ma ion and cla i y con lic s in s uden s’ a gumen s and easoning. I is also impo an o make oom o and p omo e s uden e lec ion. Table 1. Lea ning c i e ia and he p e-planned ques ions o he inqui y ask “In e ac ion o a magne and a coil“ Le el C i e ion P e-planned ques ions I S uden is able o desc ibe how he in e ac ion be ween a magne and a coil is obse ed. Did you obse e ha he needle o he ampe e me e mo ed? Wha did you do hen? Did he poles o he pe manen magne ha e an e ec ono he magni ude o he elec ic cu en ? II S uden is able o desc ibe which a iables ha e an e ec on he magni ude o in e ac ion. How did he changing o he magne s (di e en s eng h) a ec he magni ude o he elec ic cu en ? How did he changing o he coils (di e en amoun s o loops o wi e) a ec he magni ude o he elec ic cu en ? III S uden is able o explain which a iables ha e an e ec on he gene a ion o elec ic cu en in a coil. Could you explain which a iables ha e an e ec on he gene a ion o an elec ic cu en in a coil? Du ing he ocus-s uden dialogue, eedback s uden s ake no es ( e ain o on pape ) o assess he le el o he dialogue wi h espec o he lea ning c i e ia gi en in he beginning o he inqui y. A e his dialogue, eedback s uden s hold discussions wi h he ocus s uden . The du a ion o his eedback dialogue is also abou 5 minu es. The eache ’s ole he e is o suppo s uden s when hey gi e eedback o ensu e ha he dialogue is ele an and p oduc i e. Ideally, he eache ’s in e en ions should diminish om session o session. The eedback session s a s wi h a discussion abou how he hings which we e said in he o- cus-s uden dialogue a e ela ed o he lea ning c i e ia. To help jus i y hei sug- ges ions, he c i e ia a e explici ly p esen ed du ing his pa by handing hem ou on pape , o example. A e his, s uden s a e asked o discuss wha should be added o he ocus-s uden dialogue in o de o be e mee he c i e ia o lea ning. The discussions a e ollowed by a 5-minu e e lec ion session, du ing which s uden s ill he s uden sel - e lec ion o m. Da a ga he ing The SAD was implemen ed in wo lowe -seconda y physics classes (n = 2 x 16) by he same eache and in one uppe -seconda y ma hema ics class (n = 27). Bo h Nieminen e al. P oceedings o he annual FMSERA symposium 2016 69 eache s we e expe ienced. In all he classes, SADs we e conduc ed h ee imes, bu he i s ime was a p ac ice wi hou he da a collec ion. Be o e a SAD session, s uden s wo ked abou 45 minu es in an inqui y ask (elec omagne ism o ge- ome y) ha had speci ic lea ning c i e ia. Fo he ocus s uden dialogue, he eache s ended o selec a s uden who had adequa e social and subjec compe- ences. Ou esea ch design also included ques ionnai es o s uden s and eache s (Dolin e al., in p ess), bu we do no deal wi h ha da a he e due o he limi ed leng h o he pape . Analysis o discu si e ac ions Dialogues we e coded by 5-second in e als. Coding conce ned he discu si e ac ions, ela ed c i e ia and ges u es (see Fig. 1). The codes a e based on Quis gaa d’s (2017) in e p e a ion o Dys he’s (1996) amewo k (Table 2). Figu e 1. Example o he sp eadshee used o coding o he ocus-s uden dia- logue Table 2. Codes o eache s’ discu si e ac ions (Dys he, 1996; Quis gaa d, 2017) Code Desc ip ion Up ake Inco po a ing s uden s’ esponses in o he nex ques ion, hus ge ing he s uden s o e lec u he abou wha hey said, and in eg a ing he answe in o he dialogue. Focus Mean as an opposi e o up ake. Focus can be seen as an emphasis on he se eaching goals, whe e up ake can go ou on a angen . P ecise aluing Analogous o high aluing, bu migh no be s ic ly posi i e. The poin is ha i is p ecise and pu s alue o wha is said. Lowe -o de ques ion Lowe o de co e s he knowledge and comp ehension s eps (Bloom's axonomy; K a hwohl, 2002). Highe -o de ques ion Highe o de co e s applica ion, analysis, syn hesis and e alua ion (Bloom's axonomy; K a hwohl, 2002). Nieminen e al. P oceedings o he annual FMSERA symposium 2016 70 This pape ocuses on discu si e ac ions, so he codes o eache ’s ac ions a e used in he esul s la e on (Fig. 2 and 3). Fou s uden s’ discu si e ac ions, lowe - o de o highe -o de answe s and lowe -o de o highe -o de s a emen s, also a e ela ed o Bloom’s axonomy (K a hwohl, 2002). The coded da a a e used o c ea e ne wo ks o in e ela ed codes ha could be used o in es iga e he s uc- u e and cha ac e is ics o he dialogue. RESULTS Focus s uden dialogue The ou ne wo ks depic ing he dialogue o he lowe seconda y a e simila o each o he in e ms o which ac ions a e p e alen . Likewise, he wo ne wo ks o he uppe seconda y a e also e y simila . Howe e , he lowe -seconda y and uppe -seconda y dialogues show di e ences. Dolin e al. (in p ess) pe o m a de- ailed analysis o he ne wo ks in his s udy in ela ion o o he dialogues. He e, we discuss ep esen a i e ne wo ks o he lowe seconda y and uppe second- a y in ela ion o he analyses pe o med abo e. The wo ne wo ks a e shown in Figu es 2 and 3. Bo h ne wo ks show a low le el o s uden engagemen ; he s uden is silen o he majo i y o he dialogue. Besides his, he wo dialogues show di e en pa - e ns in he lowe - and uppe -seconda y le els. The lowe -seconda y dialogue is domina ed by a lowe -o de answe o a lowe -o de ques ion pa e n. The lowe -o de ques ion/lowe -o de answe pa e n does no domina e he uppe - -seconda y dialogue. He e, he eache seems o ocus on making he s uden ad- d ess a pa icula poin , which in ol es asking highe -o de ques ions in e ms o Bloom’s axonomy. This is seen as he la ge blue ci cle and su ounding smalle ci cles in Figu e 3. Howe e , he eache ’s e o s do no lead o highe - o de answe s bu o s uden s’ con usion, silence o lowe -o de answe s. Fo example, in he ollowing exce p , he main ask is o exp ess and jus i y a claim abou he ela ion be ween wo angles. One o he angles was be ween wo lines angen o a ci cle. The o he angle was a co esponding cen al angle in he ci cle. The s uden had jus s a ed ha he angle be ween he adius and angen was 90°. Fi s , he eache asked a highe -o de ques ion, and he s uden ga e a lowe -o de answe . A las , he eache summa ised wha hey ha e jus said. Teache : You ha e d awn hese angen s so ha hey cu he e in ha poin (shows he in e sec ion poin ). I hey would cu he e, o example, (shows ano he poin u he om he ci cle) so ha hey would no go so s eep, would hese angles he e s ill be 90°? (Poin s o he angle be ween adius and angen ) S uden : I don’ know. P obably no . Teache : You a e no su e abou ha ? S uden : I am no qui e su e. Nieminen e al. P oceedings o he annual FMSERA symposium 2016 71 Teache : Okay, so you a e no qui e su e i his is ue gene ally. Like, whe he his is ue only in his si ua ion o ue in gene al. Figu e 2. The ne wo k o he ou h SAD in he lowe seconda y. The le pa o he ne wo k p esen s he s uden s’ con ibu ions. The igh pa o he ne - wo k p esen s he eache ’s con ibu ions. The s uden and eache nodes in he same posi ion indica e ha hey we e ela ed. The colou o he node indica es he discu si e ac ion. The ex on he nodes ep esen s he lea ning c i e ion ad- d essed in he discu si e ac ion, i he ac ion is no ela ed o any c i e ion (si- lence) o i i is ou side o any c i e ion ( alk conce ns some hing else); NoSC = no s uden c i e ion; NoTC = no eache c i e ion. The size o he nodes ep e- sen s he ime spen on a pa icula discu si e ac ion. Links indica e ha one ac- ion ollowed ano he a one ime du ing he dialogue. Nieminen e al. P oceedings o he annual FMSERA symposium 2016 72 Ano he key di e ence be ween he wo dialogues is a di e en ocus on he eache using wha he s uden has said p e iously in ying o shape he dia- logue. This ac ion is ep esen ed by he up ake code. The lowe -seconda y eache seems o ocus mo e on including he p e ious s a emen s o he s uden han does he uppe -seconda y eache . Fo example, in he ollowing exce p , he eache asks abou he s uden ’s obse a ions, he s uden gi es sho s aigh answe s, and he eache uses he sho answe s o ask ollow-up ques ions (up- ake). Teache : How abou i you changed he coil? Did he numbe o ounds a ec he ma e (magni ude o elec ic cu en )? S uden : Yes. Teache : How did i a ec i ? S uden : E m... when he coil was smalle he cu en was a li le smalle . Teache : Wha i he coil was bigge ? Mo e ounds? S uden : I was bigge . Finally, he uppe -seconda y eache mo e o en used summa ising, which means ha he eache epea s o sums up wha has been said by he s uden wi hou co ec ing i . In his dialogue, summa ising seems o be igh ly connec ed o p ecise aluing, which aims a assigning alue on some hing speci ic ha he s uden said. This may no be s ic ly posi i e (bu o en is), bu i is no an explici co - ec ion. In e es ingly, in he lowe -seconda y ne wo k, summa ising is no a all dominan and p ecise aluing is absen . In many ways, he lowe -seconda y ne wo k seems o show a eache who ies o in ol e he s uden by using wha he s uden is saying, bu his is no e lec ed in he s uden ’s pa icipa ion. Dolin e al. (in p ess) cha ac e ises hese kinds o dialogues as s uggles, because he eache ies o engage he s uden , bu he s uden esis s. The uppe -seconda y ne wo k seems o show a eache who e- ally wan s o engage wi h a pa icula issue. Howe e , he issue seems beyond he s uden s’ abili ies in he pa icula con ex . Dolin e al. (in p ess) cha ac e ises dialogues o his so as di icul con en . Nieminen e al. P oceedings o he annual FMSERA symposium 2016 73 Figu e 3. The ne wo k o he i s SAD in uppe seconda y is shown in he same way as in Figu e 2 Pee eedback dialogue Gene ally speaking, he dialogue be ween he ocus s uden and he pee s was poo in he lowe -seconda y school because s uden s did no alk wi hou he eache ’s ques ioning. The eedback g oup was no able o e alua e he ocus- s uden dialogue well, and hey could no add mo e de ails o unde s andings in ela ion o he ocus-s uden dialogue. Also, in he uppe -seconda y school, he eedback dialogue did no p og ess wi hou he eache ’s ques ioning, bu he eache did no need o p ess as much as in he lowe -seconda y school. The eed- back g oup was able o e alua e he dialogue be e and add new in o ma ion han in he lowe seconda y.