Pair interactions in online assessments of collaborative problem solving : case-based portraits

This is a sel -a chi ed e sion o an o iginal a icle. This e sion may di e om he o iginal in pagina ion and ypog aphic de ails. Au ho (s): Ti le: Yea : Ve sion: Copy igh : Righ s: Righ s u l: Please ci e he o iginal e sion: CC BY 4.0 h ps://c ea i ecommons.o g/licenses/by/4.0/ Pai in e ac ions in online assessmen s o collabo a i e p oblem sol ing : case-based po ai s © The Au ho (s), 2018 Published e sion Pöysä-Ta honen, Johanna; Ca e, Es he ; Awwal, Na isa; Häkkinen, Päi i Pöysä-Ta honen, J., Ca e, E., Awwal, N., & Häkkinen, P. (2018). Pai in e ac ions in online assessmen s o collabo a i e p oblem sol ing : case-based po ai s. Resea ch and P ac ice in Technology Enhanced Lea ning, 13, 12. h ps://doi.o g/10.1186/s41039-018-0079-7 2018 RESEARCH Open Access Pai in e ac ions in online assessmen s o collabo a i e p oblem sol ing: case-based po ai s Johanna Pöysä-Ta honen 1* , Es he Ca e 2,3 , Na isa Awwal 4 and Päi i Häkkinen 1 * Co espondence: johanna.poysa- [email p o ec ed] 1 Finnish Ins i u e o Educa ional Resea ch, Uni e si y o Jy askyla, P.O. Box 35, 40014 Jy askyla, Finland Full lis o au ho in o ma ion is a ailable a he end o he a icle Abs ac This explo a o y case s udy ocuses on how pai s o s uden s can build a sha ed unde s anding and acqui e collabo a i e p oblem-sol ing (CPS) p ac ices du ing an online assessmen o CPS skills, which is seen in he con ex o he CPS cons uc , in a symme ical and asymme ical ask ype. E en hough CPS is widely ecognised as a co e wen y- i s -cen u y compe ency, i s na u e is no ye well unde s ood. Also, un il ecen ly, mos o s udies ha e ocused on he indi idual’s solu ion o a p oblem o on he skills indi iduals b ing in o a p oblem-sol ing space. This s udy ex ends om an indi idual- o g oup-le el ocus in CPS, emphasising he ole and quali y o he social aspec s in CPS p ocesses and ou comes. Focusing on he g oup le el because i media es mul iple le els o lea ning, including indi idual cogni ion and socio-cul u al p ac ices, may p o ide us wi h a be e unde s anding o how pai s es ablish CPS p ac ices. Because o he complexi y o CPS and he gene al challenges o emo e collabo a ion in an online con ex , he s udy elies on he iangula ion o mul iple da a sou ces and phases o analysis. In his pape , he aim is o explo e and isualise h ough con as ing case-based po ai s o wo pai s how mic o-in e ac ion p ocesses e ol e a he pai le el. The esul s show ha despi e s uden s’simila CPS pe o mance ou come sco es and ask designs aimed o acili a e collabo a ion, a ia ions in mic o-in e ac ions occu ac oss pai s, o example as indi idual and join solu ion endea ou s and as balanced and unbalanced dynamics o g oup in e ac ions. S udying hese pa e ns a he pai le el may p o ide new insigh s in o CPS and suppo s a egies o acqui ing hese p ac ices. Keywo ds: Case s udies, Collabo a i e p oblem-sol ing, Compu e -suppo ed collabo a i e lea ning, Di ec ed con en analysis, Small-g oup p ocesses, Social aspec s o lea ning and eaching, Quali a i e esea ch In oduc ion The cu en a icle ocuses on he ole and quali y o he social aspec s in collabo a i e p oblem-sol ing (CPS) p ocesses and ou comes, which is explo ed du ing an online assessmen o CPS skills in pai s o uni e si y s uden s in a emo e collabo a ion con ex . Recen ly, CPS has ecei ed inc easing in e es as one o he mos impo an gene ic skills o be mas e ed, as illus a ed by la ge-scale na ional and in e na ional assessmen s (e.g. Assessmen and Teaching o 21s Cen u y Skills (ATC21S) p ojec , www.a c21s.o g,and OECD’s PISA 2015 s udy, h p://www.oecd.o g/pisa/; see also G aesse e al. 2017). CPS © The Au ho (s). 2018 Open Access This a icle is dis ibu ed unde he e ms o he C ea i e Commons A ibu ion 4.0 In e na ional License (h p://c ea i ecommons.o g/licenses/by/4.0/), which pe mi s un es ic ed use, dis ibu ion, and ep oduc ion in any medium, p o ided you gi e app op ia e c edi o he o iginal au ho (s) and he sou ce, p o ide a link o he C ea i e Commons license, and indica e i changes we e made. Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 h ps://doi.o g/10.1186/s41039-018-0079-7 and o he gene ic skills, o so-called wen y- i s -cen u y skills, such as communica ion, c ea i i y, c i ical hinking, digi al li e acies and mo e, a e no seen as pa icula ly new bu a e alued as c i ical o u u e lea ne s and wo ke s (e.g. Ca e e al. 2018;Ca ee al.2016; G aesse e al. 2017; G i in and Ca e 2015; G i in e al. 2012; Scoula e al. 2017). P oblem-sol ing skills, o example, ha e been assessed o se e al decades, o ins ance, in he domain o ma hema ics, bu in cu en s udies and policy pape s, hese skills a e consid- e ed discipline- ee skills ha should be mas e ed as he ac ual ocus o lea ning (Ha ding e al. 2017). Al hough sligh ly di e en amewo ks o CPS exis , he speci ic skills and expec a ions iewed necessa y o he new wo k o ce a e consis en ac oss he a ious amewo ks. In he cu en s udy, he de ini ion o CPS is consis en wi h ha o Hesse e al. (2015). In he p esen s udy, ou heo e ical unde s anding o CPS builds on he ATC21S p ojec and i s ex ensi e amewo k o echnology-enhanced o ma i e assessmen o CPS skills (Hesse e al. 2015; Scoula e al. 2017). In his ega d, Hesse e al. concep ualised CPS as a complex skill ha links c i ical hinking, p oblem sol ing, decision making and collabo - a ion ac oss bo h social and cogni i e domains. CPS is deemed mo e complex han me ely wo king oge he and equi es people wo king oge he on he same ask o pu sue a common goal whe e di ision o labou exis s bu whe e inpu is in e wo en. In CPS, he quali y o in e ac ions dis inguishes good collabo a ion (Dillenbou g 1999). In his con- ex , collabo a ion equi es ac i e pa icipa ion h ough sea ches o ele an in o ma ion, join use o esou ces, sha ed e alua ions and an ag eemen on s a egies and solu ion pa hs (Ca e and G i in 2014). In CPS, indi iduals may possess di e en skills and in o - ma ion, bu hey mus wo k collabo a i ely and exchange ideas o maximise hei join ou come owa d a sha ed goal. The skills de ined by CPS a e pe inen o sol ing p ob- lems ha a e, by de ini ion, complex, ill-s uc u ed and ambiguous. Acco dingly, CPS comp ises a se o subskills ha consis o i e s ands o indi id- ual- and g oup-le el capaci ies unde he b oad social and cogni i e skills (Hesse e al. 2015). The wo hypo hesised componen s o CPS a e no mu ually exclusi e. The social componen d aws on he li e a u e om social and o ganisa ional psychology while he cogni i e componen d aws hea ily on classical app oaches o indi idual p oblem sol - ing. Following Hesse e al. (2015), social skills (i.e. pa icipa ion, pe spec i e aking and social egula ion) a e abou managing pa icipan s (including onesel ), which e e s o he ‘collabo a i e’pa o his me hod. Cogni i e skills (i.e. ask egula ion and know- ledge building) a e abou managing he ask, which e e s o he ‘p oblem sol ing’pa o he me hod. F om he social s ands o he CPS cons uc , ‘pa icipa ion’ e e s o he eadiness o sha e in o ma ion and ex e nalise hough s, ‘pe spec i e aking’means he abili y o ake in o accoun o he s’pe spec i es and ‘social egula ion’poin s o he awa eness o he s eng hs and he weaknesses o he g oup membe s (i.e. g oup/ eam awa eness; see F ansen e al. 2011). F om he cogni i e s ands o he CPS cons uc , ‘ ask egula ion’is de ined as planning and moni o ing he skills o de eloping s a - egies o p oblem sol ing and sha ed p oblem ep esen a ion (i.e. ‘join p oblem space’, see Ba on 2003; Roschelle and Teasley 1995), whe eas ‘knowledge building’he e e e s o he abili y o lea n and build knowledge h ough g oup in e ac ion. Wi hin each o hese skills, he e a e subskills ( oge he , 19 subskills, including he ‘solu ion’), which a e demons a ed by obse able indica o s o ac ions o p ocesses ( o an o e iew o he ATC21S CPS amewo k, see Fig. 1). Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 2 o 29 In he ATC21S p ojec , a echnology-enhanced assessmen en i onmen was designed, comp ising a se ies o Web-based in e ac i e, game-like asks (see e.g. Ca e e al. 2015). The asks we e designed o measu e he skills, subskills and p ocesses cha ac e ised by he CPS cons uc du ing dyads pa icipa ing in he CPS ac i i ies. The ATC21S assess- men asks, discussed la e in he p esen pape , ely on and cap u e hese ac ions o p o- cesses o measu e and e lec he CPS cons uc , as de ined by Hesse e al. (2015). Al hough CPS is widely ecognised as a co e compe ency o wen y- i s -cen u y lea n- ing, i s na u e is no well unde s ood pa icula ly in e ms o i s de elopmen ajec o y om basic o mo e sophis ica ed le els (Scoula e al. 2017). Also, in he ATC21S assess- men en i onmen , he p ima y ocus has been on how o assess he skill o he indi idual s uden in collabo a i e pa ne ship (Ca e e al. 2016). In his explo a o y case s udy, he ocus is ex ended o g oup- ela ional aspec s o CPS (Pöysä-Ta honen e al. 2016; Pöysä-Ta honen e al. 2017). The cu en s udy emphasises he ole and quali y o he so- cial aspec s o CPS p ocesses and ou comes (e.g. Ba on 2003; Dillenbou g e al. 2016; Simpson e al. 2017; Wege i e al. 2017). Acco dingly, one pa o CPS is he ask, and he o he pa is he en i onmen wi hin which he pa icipan s c ea e and sha e know- ledge, moni o hei p og ess and de ec and epai he b eakdowns in communica i e ac s ha may hampe collabo a ion (Al e man and Ha sch 2017; Roschelle and Teasley 1995). The cu en s udy concen a es on he second pa o CPS, which is seen i al in compu e -suppo ed collabo a i e lea ning (CSCL) esea ch, as i is he con ex o he cu en s udy. In ea lie wo k, i has been ound, o example, ha in collabo a i e lea n- ing and p oblem sol ing, he p ocesses o building a sha ed unde s anding o he si ua ion a hand and he quali y o in e ac ions appea ed o be, o some poin , e en mo e impo - an han he ac ual abili ies o he indi idual pa icipan s when i came o sol ing he asks (Dillenbou g e al. 2016; see also Ba on 2003; Simpson e al. 2017). The cu en s udy akes a small-g oup ocus on CPS p ac ices. S udies on collabo a- i e lea ning, in gene al, ha e educed hei scope o s udy ei he indi idual cogni ion Fig. 1 The se o CPS skills and subskills om he ATC21S CPS amewo k (modi ied om Hesse e al. 2015; see also Ca e e al. 2016; Scoula e al. 2017) Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 3 o 29 o socio-cul u al p ac ices (S ahl 2017). As S ahl (2017) s a ed (in lea ning esea ch), ‘T adi ional me hods (poin ing o quan i a i e o quali a i e me hods ha analyse ne changes) o en p o ide e idence ha change has aken place, bu do no desc ibe how he change has aken place’(p. 114, o iginal emphasis). Following S ahl (2017), i is a - gued ha an emphasis on small-g oup p ac ices, because i media es he mul iple le els in ol ed in lea ning (e.g. indi idual cogni ion on he one hand and he socio-cul u al p ac ices o he class oom and ools used on he o he ), may p o ide us wi h a ounda- ion o be e unde s and how CPS akes place in his con ex . No ably in he cu en s udy, om heo e ical and me hodological poin s o iew, we ega d pai s as small g oups (Williams 2010). As Williams (2010) s a ed, despi e ce ain p ope ies ha do no lend hemsel es o small g oups (i.e. coali ion o ma ion o he di icul y o imagine newcome s and old- ime s in a g oup o wo), ‘[i]n mos ins ances dyads a e g oups o wo and ope a e unde he same p inciples and heo ies ha explain g oup p ocesses o g oups o h ee and la ge ’(p. 268). To explo e CPS as a dynamic, g oup-le el se o p ocesses in an online se ing, a case-based s udy was unde aken. As Yin (2009) a gued, he how and why ques ions a e be e answe ed h ough a case-based app oach because hey a e well sui ed o explo - ing complex social phenomena. In case s udy esea ch, an in ensi e s udy o a single uni can p o ide ways o be e unde s and a la ge class o simila uni s, especially when elying on he iangula ion o mul iple da a sou ces (Baška ada 2014; Ge ing 2004; Hmelo-Sil e e al. 2009; Yin 2009). The e o e, in he cu en s udy, he da a comp ise objec i e measu es and p ocess da a (i.e. au osco ing o CPS skills, ac i i y logs and sc een eco dings) combined wi h subjec i e p ocess da a (i.e. cued e ospec - i e epo ing (CRR) in e iew da a) o o m he basis o he desc ip i e analysis o pai -le el CPS p ocesses, which a e ac ualised as p ocess isualisa ions o mic o-in e ac ion p ocesses (see Da is e al. 2015). In he majo i y o he ATC21S s ud- ies o da e (see Ca e e al. 2018), he scope o analysing he social aspec s has been on he placemen (and occu ence) o cha ac ions in he CPS p ocess (Ca e e al. 2015) a he han he cha con en i sel . Focusing on he ‘how’o CPS equi es an analysis ha goes beyond pe o mance sco es o he solu ion o he p oblem. The ‘how’explo es whe he di e en se s o in- e ac ions can unde lie success ul comple ion and whe he he na u e o a ask s imu- la es di e en se s o in e ac ions. Fo example, wo s yles o asks ha e been used o assess CPS (Ca e and G i in 2017). One s yle p esen s and gi es access o he same s imulus se o bo h he s uden s in a pai , and he o he p esen s and gi es access o di e en s imulus se s o he wo s uden s. The i s s yle is ypically e e ed o as symme ic and he second as asymme ic. Acco ding o Ca e e al. (2015), he accessi- bili y he e e e s o ‘bo h o di ec e ie al as well as human capaci y o unde s and and manipula e he equi ed a e ac s- be hese objec , knowledge, o p ocesses’(p. 87). This di e ence be ween he ask ypes can easonably be hypo hesised as s imula ing di e en se s o skills ha can be applied o he ask. In a symme ic ask ype, collab- o a ion may be alued o i s social aspec bu may also be ega ded as a coun e o he s uden s’in e es s (Ca e e al. 2015). In u n, he si ua ion o accomplishing an asym- me ic ask, whe e esou ces a e no equally accessible o bo h s uden s in he pai , is expec ed o inc ease he need o collabo a ion o be e p omp collabo a ion and he collec i e ac i i y o de elop (Ca e e al. 2015). Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 4 o 29 Taken oge he , in he cu en s udy, we ha e b oadened he analysis beyond he measu ed CPS skill le els o he indi idual s uden s, ha ing hem sea ch o he di e - ences in he quali y o CPS p ocesses a he pai le el du ing he online assessmen o CPS skills, in ega d o he di e en ypes o he asks. Based on mul iple, ich da a and mul iple phases o analysis, we ask he ollowing: (1) A e he e di e ences in he quali y o he mic o-in e ac ions a he pai le el ha a e unde lying simila CPS measu es o indi idual s uden s? (2) Does he ask ype (i.e. symme ic/asymme ic) quali a i ely p o oke di e en mic o-in e ac ions in he s uden pai s? Me hods Pa icipan s and asks The esea ch pa icipan s (n= 20) we e s uden s en olled in a mas e -le el eache edu- ca ion p og amme a a Finnish uni e si y. S uden s we e andomly assigned o hei pai s. Each s uden pai comple ed one bundle o asks in he ATC21S assessmen en i onmen . The asks we e de eloped a he Assessmen Resea ch Cen e a he Uni e si y o Melbou ne om 2010 o 2013. The assessmen en i onmen is based on human- o-human app oaches o assessing CPS (Ca e e al. 2015) and comp ises a se o online in e ac i e and collabo a i e p oblem-sol ing asks in STEM domains. In he asks, he s uden pai s a e gi en a unique subse o esou ces ha a e equi ed o sol e he p oblem. To ully unde s and he p oblem space and iden i y all he necessa y esou ces, he s uden s need o ely on hei pa ne (Ca e e al. 2015; Ca e e al. 2016). The communica ion akes place ia a ee- o m cha in e ace. The se o asks used in he cu en s udy comp ised ou di e se asks, oge he las - ing app oxima ely 60–90 min. Du ing he asks, s uden pai s (s uden s A and B) we e sea ed in di e en class ooms o ensu e ha he only means o communica ion was he cha in e ace. Fo his s udy, he bundle comp ised ‘Laughing Clowns’and ‘Oli e Oil’, which a e con en - ee asks, and ‘Plan G ow h’and ‘Small Py amids’, which a e con en -dependen asks (see Ca e e al. 2015). Con en - ee asks enhance induc i e and deduc i e hinking whe eas con en -depended asks equi e s uden s o apply knowledge ela ed o cu iculum. Laughing Clowns is a symme ic ask, whe eas he o he h ee a e designed as asymme ic. Acco ding o Ca e e al. (2015), in a symme ic ask, he s uden pai is p esen ed wi h he same images, pe spec i es, ins uc ions and esou ces wi hin he online ask space. In an asymme ic ask, he s uden pai is p esen ed wi h asymme - ical pe spec i es, p o iding di e en in o ma ion and esou ces ( o simila and di - e en iews o he pai s in a symme ic and asymme ic ask, espec i ely, see Figs. 2 and 3). In he cu en pape , we explo e he mic o-in e ac ion p ocesses o pai s in comp ising (a) a symme ic ask (Laughing Clowns) and (b) an asymme ic ask (Oli e Oil). These asks a e con en - ee asks ha do no equi e any p e equisi e knowledge o he cu iculum and me ely ely on he applica ion o easoning (see Ca e e al. 2015). We also educed he scope on he con en - ee asks o he bundle used in his s udy. This cha ac e is ic may enhance compa abili y o he indi iduals, since di e en ial access o he p io knowledge is ende ed i ele an . Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 5 o 29 In he Laughing Clowns ask, wo pa icipan s a e p esen ed wi h a clown machine and 12 balls o be sha ed be ween hem. The goal is o he s uden s o de e mine whe he hei clown machines wo k in he same way. To do his, he wo s uden s need o sha e in o ma ion and discuss he ules, nego ia ing how many balls hey should each use. The s uden s mus place he balls in o hei clown’s mou h while he mou h is mo ing o de e mine he ule go e ning he di ec ion he balls will go (en y: le , middle and igh ; exi : posi ions 1, 2 and 3). Each s uden mus hen indica e whe he hey belie e he wo machines wo k in he same way. S uden s do no ha e access o each o he ’s sc een. In Fig. 2, he iews o s uden s A and B a e shown. In he Oli e Oil ask, pa icipan s a e p esen ed wi h di e en iews o he ask, includ- ing di e en esou ces. The goal is o ill a ja wi h 4 l o oli e oil wi h he a ailable e- sou ces. To achie e his objec i e, he s uden s mus wo k ou wha esou ces a e a ailable and how o sequence he use o hese esou ces. S uden A has a i ual 3-l ja , oli e oil dispense , ans e pipe and bucke . S uden B has a i ual 5-l ja , ans e pipe, accep bu on and bucke . None o he pa icipan s a e awa e be o e beginning communi- ca ion wha is a ailable o hei pa ne . The pai hus needs o ecognise ha s uden A mus ill his o he ja a he dispense and place i unde he ans e pipe so ha s uden B can accep he oil ans e ed by his o he pa ne om he pipe. Un il his poin , s u- den B canno comple e any meaning ul ac ions and is dependen on he ac ions and in- e ac ions on s uden A. The s uden s need o explo e and na iga e he ask space oge he un il hey manage o place 4 l o oil in s uden B’s ja . This ask ollows he ea- soning p ocesses equi ed in he Towe o Hanoi p oblem, which was popula ised by ma hema ician Edua d Lucas in 1883 (Newell and Simon 1972;Pe ko ić2009). In he Oli e Oil ask, bo h p oblem sol e s a e equi ed o wo k ou a sequence o mo emen s o achie e he goal o he ask. S uden s s ill do no ha e access o each o he ’ssc eens, Fig. 2 A sc een cap u e om he sc een ac i i y da a (pe spec i es o s uden s A and B, Laughing Clowns) (In Finnish) Fig. 3 A sc een cap u e om he sc een ac i i y da a (pe spec i es o s uden s A and B, Oli e Oil) (In Finnish) Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 6 o 29 and he ask is complica ed by he di ision o esou ces and dispe sed in o ma ion among dyads. Figu e 3shows he di e en pe spec i es o s uden s A and B. Da a collec ion Objec i e measu es and p ocess da a: au o-sco ing and sc een eco dings In he ATC21S assessmen en i onmen , s uden s a e assessed indi idually du ing dyad pa icipa ion in CPS ac i i ies. CPS asks a e designed o gene a e speci ic beha iou s, aligned wi h he skills and subskills o he CPS cons uc (see Fig. 1; Ca e e al. 2016; Hesse e al. 2015; Scoula e al. 2017). S uden s’comple ion o he asks gene a es log s eam da a (i.e. he ac i i y log), and he pa e ns in hese da a (i.e. he mo emen o a e ac s and he occu ence o cha ) a e cap u ed algo i hmically and a e coded by he sco ing engine and hen sco ed acco ding o he Rasch model, p oducing in o ma ion on s uden s’social and cogni i e skill le els (see Adams e al. 2015; Ca e e al. 2016). The ac i i y log shows he e en s be ween wo s uden s wo king on he ask. (Fo an example o ac i i y log, see Table 1). Acco dingly, o cap u e he CPS skills and he subskills shown in Fig. 1, indica i e be- ha iou s a e p oposed o sugges he e idence o he beha iou being obse ed, demon- s a ing he p esence o absence o a pa icula subskill and how much o ha subskill is p esen in an indi idual (see e.g. Adams e al. 2015; Ca e e al. 2016). Examples o he CPS skills and he indica i e beha iou s in he Laughing Clowns and Oli e Oil asks a e shown in Table 2. Beha iou s hypo hesised as indica i e o pa icula subskills o elemen s a e hus ex- ac ed om he log s eam h ough au oma ed coding p ocesses. Au o-sco ing cap- u es wo ypes o esponse ep esen ing common and unique e en s (Adams e al. 2015; Ca e e al. 2016). Common e en s include beginning and end o a ask, sys em messages con i ming ac ions and p o iding na iga ion, and cha be ween pa ne s. Unique, o local, e en s include hose ha a e speci ic o ask. Fo he Laughing Clowns ask, o example, hese include he d ag and d op o a ball and i s loca ion. A coun o he ‘d opShu e’ac ions (d opping he balls in o he clown’s mou h, see Table 1), o example, can be coded o indica e how well a s uden managed hei esou ces ( he balls). These coded indica o s a e classi ied as ei he di ec o in e ed. Indica o s ha Table 1 A sec ion om an ac i i y log ( ask Laughing Clowns) ( ansla ed om Finnish) Task Page Playe E en ype Con en s Times amps 23 1 A Cha Do you ha e any clue wha we’ e supposed o do he e? 06/10/2015 16.45 23 1 B Cha I don’ know I’m hinking :D 06/10/2015 16.45 23 1 A Cha Yeah, would you h ow i s ? 06/10/2015 16.45 23 1 B Cha Le ’s h ow a ball and see wha happens 06/10/2015 16.45 23 1 B Ac ion s a D ag:ball1:70:150 06/10/2015 16.45 23 1 B Ac ion s opD ag:ball1:70:150 06/10/2015 16.45 23 1 B Ac ion s a D ag:ball1:70:150 06/10/2015 16.45 23 1 B Ac ion s opD ag:ball1:509:135 06/10/2015 16.45 23 1 B Ac ion d opShu eL:ball1:509:135 06/10/2015 16.45 23 1 B Cha Whe e did i come ou o you? 06/10/2015 16.46 Table 1displays he ask and playe IDs ( ask 23, s uden A o B), he page o he ask (he e, page 1), ac ions (e.g. he e en ype), he con en s o he cha ( ex exchanged) and hei imes amps Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 7 o 29 a e classi ied as di ec a e hose ha desc ibe a pa icula ac ion, which can be in e p e ed in i s own igh . Indica o s classi ied as in e ed include hose which include a sequence o ac ions, om which sequence a pa icula hinking o exchange p ocess can be in e ed. Fo example, i one s uden akes a pa icula ac ion, he e is hen a cha exchange, ollowed by he s uden changing he ac ion, i can be in e ed ha he cha o he pa ne has in lu- enced he i s s uden . An example o he au o-sco ing app oach is p o ided in Table 3. As desc ibed in Adams e al. (2015), da a a e con e ed in o dicho omies o pa ial c edi , wi h each indica o being ea ed somewha simila ly o i ems on a adi ional es . In u n, hese a e agg ega ed wi h like indica o s ac oss se bundles o asks. Cu -o alues o de e mine le els o quali y ac oss subskills a e calcula ed o he pu - pose o scaling. Applying he Rasch measu emen model, i is possible o de e mine he pa e ns wi hin he CPS concep ual amewo k and o in e p e a s uden ’s CPS abili y Table 2 Example o social o cogni i e skills and indica i e beha iou s o exempla asks in Laughing Clowns and Oli e Oil (adap ed om Ca e e al. 2015) CPS elemen s/ subskills Indica i e beha iou E idence o da a Laughing Clowns Oli e Oil In e ac ion In e ac s wi h he pa ne P esence o cha be o e allowing he pa ne o make a mo e P esence o cha be o e allowing he pa ne o make a mo e Audience awa eness Adap s con ibu ions o inc ease unde s anding o he pa ne Numbe o ball mo es a emp ed be o e s opping and wai ing o he pa ne o mo e o espond P esence o in o exchange on indi idual con aine s a es Resou ce managemen Manages esou ces Realises ha balls a e mean o be sha ed and uses only allo ed hal (No obse ed) Rela ionships Iden i ies connec ions and pa e ns be ween elemen s o knowledge The wo s uden s come o an ag eemen on how hei machine wo ks P esence o cha exchanging in o ma ion when A o B ecognises he signi icance o his o he ja con aining only 1 l Re lec s and moni o s Adap s easoning o cou se o ac ion as in o ma ion o ci cums ances change (No obse ed) Lea ning om edundan ac i i ies, such as A mo ing ja o bucke Solu ion A i es a he co ec answe Iden i ies connec ions and pa e ns be ween elemen s o knowledge Las ac ion equi es B’s ja o con ain 4 l o oil Table 3 Use o algo i hm o ela e ac ions o he elemen sys ema ici y o he collabo a i e p oblem-sol ing cons uc (adap ed om Adams e al. 2015) Indica o name De ails Algo i hm Ou pu U2L004A U2L004B Sys ema ic app oach All posi ions ha e been co e ed. Sco ing ule: h eshold alue Task name: Laughing Clowns S ep 1: Find all d op ball occu ences cap u ed as d opShu e and hei co esponding posi ions as d opShu eL, d opShu eR and d opShu eM. S ep 2: Then coun all he occu ences o he ac ion eco ded unde ‘d opShu e’and hei unique posi ions om he log. S ep 3: Inc ease he alue o he indica o by 1 i 1 o mo e ‘d opShu e’occu s in he o m o d opShu eR, d opShu eL o d opShu eM. S ep 4: I he o al numbe o unique d opShu es (d opShu eR, d opShu eL and d opShu eM) om he log is less han 3, he alue o he indica o is de ined as −1 o indica e missing da a Coun alues In Table 3, he i s column ep esen s he name o he indica o , being unique o each indica o . In he example, ‘U2’ poin s o he Laughing Clowns ask, ‘L’ e e s o local and explains ha he indica o is unique o he ask, ‘004’means ha he indica o is ou h c ea ed o he ask and ‘A’means ha he indica o is applicable o s uden s A and ‘B’ ha i is applicable o s uden B Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 8 o 29 Fig. 6 Join solu ion endea ou (pai 24, Laughing Clowns). The ull episode as a gene al pa e n o in e ac ion is p esen ed on he le . On he igh , an exce p (indica ed as a g ey a ea in he ull episode) is p esen ed, including a ime-s amped, condensed ac i i y pa e n (A = ac ion, C = cha ). The dialogue is ex ac ed om he ac i i y log, and he speech bubbles con ain exce p s om he coded CRR da a ha a e ela ed o he con en o he pa icula exce p . (T ansla ed om Finnish). (The ull episode is a ailable also in Fig. 10 in Appendix 2) Fig. 7 Indi idual solu ion endea ou (pai 29, Laughing Clowns). The ull episode as a gene al pa e n o in e ac ion is p esen ed on he le . On he igh , an exce p (indica ed as a g ey a ea in he ull episode) is p esen ed, including a ime-s amped, condensed ac i i y pa e n (A = ac ion, C = cha ). The dialogue is ex ac ed om he ac i i y log, and he speech bubbles con ain exce p s om he coded CRR da a ha a e ela ed o he con en o he pa icula exce p . (T ansla ed om Finnish). (The ull episode is a ailable also in Fig. 11 in Appendix 3) Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 15 o 29 Pa icipan s needed o be awa e ha hei balls we e sha ed and ha he mos e ec i e way o inding he solu ion was o assign he balls so ha bo h s uden s would ha e an ad- equa e and equal oppo uni y o y hei machine and each a conclusion. In con as o pai 24, pai 29 did no discuss he alloca ion bu s a ed wo king independen ly, d opping and d agging he balls wi hou in e ac ing and coo dina ing hei e o s. Also, e idence o audience awa eness (i.e. how a pa icipan adap s con ibu ions o inc ease unde s anding o o he s) is missing wi h pai 29. S uden s who a e p o icien in his a ea would be mo e likely o in e ac wi h hei pa ne be ween ball d ops and adap hei beha iou o bes sui hei pa ne ’s needs (in con as , see pai 24). S uden 24B s a ed (see exce p 1): ‘[…] I unde s ood ha she [ he pa ne ] has some hing di e en ly he e han on my sc een and ha I don’ knowwha she’s go he e, so hen I el ha I ha e o explain wha I’m seeing […]’(exce p 1, s uden 24B, CRR in e iew). In e ms o he cogni i e aspec s o CPS, s uden s wi h a low p o iciency o e- sou ce managemen skills may only hink o hemsel es when checking how hei machine unc ions, he eby monopolising he use o esou ces, which seems o be he si ua ion wi h pai 29, while mo e skilled s uden s would mo e likely ecognise he need o sha ed use o he balls and hence would sha e hem equally (pai 24). Mo eo e , o each a solu ion, he s uden s needed o iden i y he ela ionship be- ween he en y and exi poin o he balls and de e mine i he e was consis ency in how he machines unc ion; he pai needed o cons uc a way o ep esen ing his in o ma ion and communica ing i o each pa ne , as well as being able o unde s and he o he o ms o ep esen a ion ha he pa ne uses. Fo example, o pai 29, s uden A p o ided a na a i e while s uden B lis ed pieces o in o ma ion. S uden 29B desc ibed hei pa allel wo king on he ask as ollows (exce p 2): […] Well my pa ne (didn’ ealise pe haps) ha I had checked i . Then he e a one poin I s ill es ed and hen I w o e down whe e hey go, so ha i I d op he e om L. And hen I ied o es he same ones like in hose i i goes always like in he same way. Bu hen a some poin when hose pa ne ’s messages appea ed he e, so hen I s ill ied wi h hose, and hen did i a he same ime (you know) and hen I w o e i down like ha [as le e s and numbe s on cha ] so ha whe e hey go om hose […] (exce p 2, s uden 29B, CRR in e iew). Skil ul s uden s will also challenge he pa e ns and es he assump ions ha unde pin hei obse a ions, which happened wi h s uden 29B (exce p 2). The inal s ep com- p ised he s uden s compa ing hei ep esen a ions so ha a decision conce ning he simila i y o he unc ioning o he clown machine could be made (solu ion). Pai 24 com- munica ed h oughou hei ac i i ies, whe eas pai 29 only discussed hei sha ed unde - s anding owa d he end. I he ask was no a o ced collabo a ion exe cise ( hanks o he las concluding ques ion), s uden 29A would ha e been able o sol e he ask qui e inde- penden ly because he also poin ed ou he ollowing in he CRR in e iew (exce p 3): […] Like one can see i om his, I ealised like a once wha he ask he e is. And hen I jus ied i ou a ew imes how he hing was unc ioning and epo ed [gi es a sho laugh] igh away. I don’ know whe he one could ha e like [a sho laugh] Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 16 o 29 buil some coope a ion he e like so ha one would i s ha e like old he ins uc ion in he way one unde s ood i […] (exce p 3, pai 29A, CRR in e iew). Case 2: balanced e sus unbalanced CPS p ocess in e ms o he dynamics o in e ac ion ob- se ed in he asymme ic ask Oli e Oil In con as o he Laughing Clowns ask, in he asymme ic ask Oli e Oil, he cumula- i e accoun s ob ained om he di ec ed con en analysis o pai s 24 and 29 (Table 8) we e in line wi h he ask design, e en emphasising mo e social aspec s han he o i- ginal design o he Oli e Oil ask did. Howe e , when compa ing hese cumula i e ac- coun s om he di ec ed con en analysis on he CRR da a o he mic o-le el in e ac ion p ocesses isible in he ac i i y logs, he e we e e iden di e ences ega d- ing he quali y o collabo a ion ac oss he wo pai s. Acco dingly, he mic o-in e ac ion con en s e ealed quali a i ely di e en CPS p ocesses as balanced e sus unbalanced dynamics o in e ac ion ( o mic o-in e ac ion pa e ns, see Figs. 8and 9). E en hough he asymme y o he ask s yle was designed o o ce in e ac ion in pai s, he quali y o Fig. 8 Balanced CPS p ocess in e ms o he dynamics o in e ac ion (pai 24, Oli e Oil). The exce p includes a ime-s amped, condensed ac i i y pa e n (A = ac ion, C = cha ); dialogue ex ac ed om he ac i i y log, and he speech bubbles con ain exce p s om he coded CRR da a. (T ansla ed om Finnish) Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 17 o 29 he ways in which he pai s nego ia ed he ask seems o a y. Pai 24 con inued o e- semble a join explo a ion p ocess. Pai 24 nego ia ed hei explo a ions and join ly de- eloped and discussed hei unde s andings o sol e he ask (see Fig. 8). In pai 29 (see Fig. 9), he dynamics o he in e ac ion in hei p oblem-sol ing p ocess we e unbal- anced in e ms o he powe ela ions being unequal posi ions o s a us (see e.g. Simp- son e al. 2017). S uden A ended o assign he ole o ‘leade ’o ‘d i e ’ o himsel in he sha ed ac i i y, assigning s uden B he ole o ‘ ollowe ’o ‘passenge ’(Schneide and Pea 2013; Shae e al. 2011). Howe e , s uden B seemed o accep his ole. In Figs. 8and 9, he quali a i e di e en dynamics o he in e ac ion a e exempli ied. Fig- u e 8depic s he pa e n o pai 24, which can be cha ac e ised as a balanced, join in e ac ion, wi h equal dis ibu ion o wo k and esponsibili ies. Figu e 9, in u n, shows an example o he unbalanced na u e o he dynamics o he in e ac ion in pai 29. Fig- u e 9gi es an example o an explici disag eemen (Richie 2002; Simpson e al. 2017) in which s uden 29A sho ly ques ions his pa ne ’s p oposal o he possible s eps o each he p oblem solu ion, a ou ing his own idea. Figu e 9displays he pi o al mo- men o pai 29 when he idea o how o sol e he ask is c ys allised o s uden B. Howe e , he idea is ejec ed by s uden A, who simply con inues o implemen his own idea o how o each he solu ion. When s uden B seemingly asked o con i m- a ion, ‘[…] o did we go some hing w ong […]’, s uden A me ely eplied using a deli- ca e impe a i e: ‘[…] No, bu we’ e s ill on he map when we do as ollows: […]’. Fig. 9 Unbalanced CPS p ocess in e ms o he dynamics o in e ac ion (pai 29, Oli e Oil). The exce p includes a ime-s amped, condensed ac i i y pa e n (A = ac ion, C = cha ). The dialogue is ex ac ed om he ac i i y log, and he speech bubbles con ain exce p s om he coded CRR da a. (T ansla ed om Finnish) Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 18 o 29 Based on he asymme ical ask design, a he beginning o he Oli e Oil ask, s uden B depended on he ac ions and in e ac ion o s uden A. S uden A, e en hough highly ac i e, did no uly in e ac wi h s uden B du ing he p oblem-sol ing e o s; he s uden also ailed o espond o he co-s uden ’s e o s o sol e he ask. Ins ead, s uden A a emp ed o di ec he p ocess, as is acknowledged by his pa ne in he CRR in e iew (exce p 4): ‘[…]Ijus hough asmypa ne oldme he elikequi ealo wha Iha e odo,soI hough hen ha he like igu es his ou […]’(exce p 4, s uden 29B, CRR in e iew). F om he cogni i e aspec s o CPS, p oblem analysis is he i s s ep in planning. P oblem analysis equi es pa icipan s o explo e he ask and de ine sub asks and con- s uc subgoals o success ully sol e he p oblem oge he . In pai 29, e en hough s u- den A seems ocused on un a elling he ask, he subgoals se we e no aligned a he pai le el. Also, be o e o ganising he necessa y s eps o sol e he p oblem, he s uden s needed o sha e in o ma ion and explain he esou ces a ailable, as was e icien ly dis- cussed by pai 24. Pai 29 did sha e in o ma ion bu did no sys ema ically discuss he elemen s each had on his o he sc een, as was he case wi h pai 24. This caused mis- unde s andings. S uden 29A e lec ed on hei wo king (exce p 5): ‘[…] We wen h ough wha we ha e he e. [Pause] Wha we can see on ou sc een, bu ac ually we wen h ough his only la e […]’(exce p 5, s uden 29A, CRR in e iew). The p oblem analysis phase is ollowed by o ganising he necessa y s eps o sol e he ask. Cause and e ec equi ed he s uden s o success ully hink o s eps ahead o hei cu en p oblem s a e and wo k ou sub asks be o e ac ing. In he Oli e Oil ask, he essen ial equi emen was, in ad ance, o wo k ou sequences o mo emen s o achie e he goal, ha is, o ill he ja wi h 4 l o oil. Pai 24B e lec ed on his (see exce p 6): ‘[…] I like his kind o hings whe e you mus hink like, hose ew mo es so ha you’ll ge hem, you each he inal esul […]’(exce p 6, s uden 24B, CRR in e iew). As he ci cums ances change, p o icien s uden s we e be e able o adap hei ea- soning and e-o ganise hei unde s anding o he ask a hand (i.e. e lec s and moni- o s; Hesse e al. 2015). Fo example, pai 24 ecalled he ollowing (exce p s 7 and 8): s uden A said, ‘[…] So hen I was s ill hinking ha i one could pou i om he bucke , so I ealised ( ha no hing is coming om he e). The e we no iced i , o now I ealised how we should do his you know […]’(exce p 7, s uden 24A, CRR in e iew). S uden B, in u n, o mula ed hei easoning as ollows: […] Well hen he e was he hing ha i would ha e gone he e in o he bucke he igh amoun , ha is ou li es, bu hen we s a ed o hink ha pe haps i ’s such a glass con aine a e all, whe e we should ha e i . [Pause, he ideo shows ha 24b has a ull glass con aine .] The e is now ha one li e he e in [pa ne ’s] bucke . Now, she only should h ee mo e, so hen he e would be he ou li es in he bucke . Bu hen we ealise ha i ough o be a glass con aine . (We ha e hough i ) w ong […] (exce p 8, s uden 24B, CRR in e iew). In he Oli e Oil ask, he cogni i e elemen , ela ionship, was isible in he cha because s uden A o B ecognised and communica ed he signi ican de ails o wha ollows i he ja con ains only 1 l o oil. In pai 29, al eady in he middle o he en i e p oblem-sol ing p ocess, s uden B came up wi h a co ec , de ailed plan o he sequences o mo emen s o be execu ed o achie e he desi ed goal s a e, bu s uden A did no ecognise his Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 19 o 29 e o , con inuing wi h his ongoing ac ions, so he idea was dismissed (Fig. 7). S uden 29B depic ed he ollowing (exce p 9): […] As I know ha i [own idea] is igh , and hen as he was ye ha ‘le ’s do like his’so hen I began o hink ha which one he con aine ac ually is. And as he seemed o be so ce ain he e in his explana ion. And I was ce ain as well, bu as he was like ha ‘le ’s do like his […] (exce p 9, s uden 29B, CRR in e iew). La e on, s uden 29A came up wi h he same idea, as s uden 29B s a ed (see exce p 10): […] Now he would ha e liked o do i like ha , hen [laugh e ]. Now he’s explaining he same wha I explained o him al eady ea lie [a sho laugh]. As now he ells ‘I’ll send you 1 and hen I’ll send 3’so he e i was. And I was like in ac I old you so! [a sho laugh]! I knew his all igh , ha hen i will be sol ed. Tha I knew i all he ime. I knew his be o e him […] (exce p 10, s uden 29B, CRR in e iew). Discussion and conclusions The aim o he cu en explo a o y case s udy was o examine he mic o-in e ac ions in pai s du ing he online assessmen o CPS skills in he ATC21S assessmen en i onmen . The p esen s udy shi ed om an indi idual and solu ion-o ien a ion ocus on he CPS p ocess o he g oup- ela ional aspec s o CPS. Ou g ounding assump ions we e based on he de in- i ion o he CPS cons uc as comp ising mul iple, in e ac ing subelemen s o skills a he han a linea se o p ocesses (see Ca e e al. 2016; Hesse e al. 2015; Scoula e al. 2017). In he cu en s udy, we asked how s uden pai s accomplished ambiguous CPS asks and how hei nego ia ions owa d sha ed unde s anding we e mani es ed in mic o-in e ac ions du ing he CPS p ocess. We we e in e es ed in whe he di e en se s o in e ac ions can unde lie success ul comple ion and whe he he na u e o a ask s imula es di e en se s o in e ac- ions. Ou case-based po ai s showed ha despi e he s uden s’compa able and ela i ely high o mode a e CPS pe o mance ou come sco es, especially in social skills (sco ed be- ween he le els o 5–6, wi h 6 being he highes skill le el), a ia ions in mic o-in e ac ions occu ed ac oss pai s, o example as indi idual and join solu ion endea ou s and as bal- anced and unbalanced CPS p ocesses in pai s. Also, he asymme ical ask ype, in compa i- son o he symme ical ask ype, appea ed o be mo e e ec i e in o cing in e ac ions in pai s. Howe e , his did no gua an ee he ac ual quali y o he mic o-in e ac ions. Acco dingly, in small-g oup collabo a ion, he e a e mul iple in e ac ing elemen s ha con ibu e o he s uc u e and low o collabo a ion, such as he a o emen ioned elemen s ha a e ela ed o he join p oblem space (Al e man and Ha sch 2017; Roschelle and Teasley 1995). As was also wi nessed in case 1 (Laughing Clowns), sig- ni ican challenges in collabo a ion may a ise i pa icipan s do no a emp o coo din- a e hei indi idual pe spec i es in he join p oblem space. Pa icipan s who sensed a co-p esence pe cei ed hei pa ne s and paid a en ion o each o he (Al e man and Ha sch 2017). As shown (e.g. Go man 1963), o achie e a sense o co-p esence is es- sen ial o high-quali y collabo a ion. This is, howe e , mo e demanding when s uden s a e ope a ing in online se ings, such as he con ex o he cu en s udy. Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 20 o 29 In CPS, he p ocess o cons uc ing and main aining a sha ed unde s anding o he ask a hand is needed o collabo a i e lea ning o e ol e (Dillenbou g e al. 2016; Roschelle and Teasley 1995). Building a sha ed unde s anding is highly con ex ual: he media may acili a e o hampe in e ac ions. When compa ed wi h co-loca ed collabo a ion, in an on- line se ing, he c ea ion o a sha ed unde s anding equi es addi ional e o om he pa - icipan s (Dillenbou g e al. 2016) because o egula e in e ac ions and achie e ag eemen is much mo e complex (Al e man and Ha sch 2017). He e, he e may be p essu e o achie e join goals wi h educed le els o sha ing. Bu as Da is e al. (2015)s a ed,no - mally, in join ac i i y, people b ing hei own, indi idual goals o he si ua ion, bu a he same ime, hey join ly wo k owa d a de ini ion o he si ua ion ha wo ks o gi e he in e ac ion cohe ence (p. 61). In case 2 (Oli e Oil), he oles aken and assigned (i.e. s u- den A as a leade , s uden B as a ollowe , see Schneide and Pea 2013) may e lec ( ol- lowing Simpson e al. 2017) he ways in which powe ela ions and in ellec ual au ho i y a e nego ia ed in g oup in e ac ions in CSCL se ings. Despi e he many posi i e p omises o CSCL, such as inc eased lea ning gains esul ing om join in e ac ion, Simpson e al. (2017) no ed he dange s o CSCL en i onmen s, such as he possibili y o hampe ing he lea ning oppo uni ies o hose s uden s who lack a oice and a e subjec ed o hei co-s uden s’ins uc ions (see also Lange -Osuna 2016). Consequen ly, as Dillenbou g e al. (2016) poin ed ou abou collabo a i e lea ning in gene al, he asks a e p ima ily designed o acili a e sha ed meaning and collabo a ion. In he CPS cons uc examined he e, he asks, e en hough being p ima ily designed o measu e indi idual s uden ’s CPS skills in a collabo a i e si ua ion, canno be suc- cess ully comple ed wi hou he pai (e.g. Ca e e al. 2015,2016; Hesse e al. 2015; Sco- ula e al. 2017). Howe e , collabo a i e ac i i y and lea ning should no be conside ed some hing ha would au oma ically happen: despi e he simila asks and equal suppo p o ided o he pa icipan s, he pedagogical designs can be pe cei ed and in e p e ed di e en ly by di e en pa icipan s (e.g. A aja and Pöysä-Ta honen 2013; Simpson e al. 2017; Sinha e al. 2015). Fo example, he deg ee o sha ed unde s anding ha he pai s needed o each was ela ed o he ype o ask hey needed o pe o m (Dillen- bou g e al. 2016). In case 1, he symme y o he esou ces a ailable o he pai s migh ha e dilu ed he e ec o in e dependence. In case 2 (Oli e Oil), in u n, he asymme y o he ask ac i a ed posi i e in e ac ion pa e ns, such as sha ed nego ia ions and ue ac s o collabo a ion, as expec ed om his s yle o ask o ensu e he emo al o , bu no gua an eeing, unbalanced in e ac ions. In line wi h he indings o Simpson e al. (2017), his seemingly in luenced he pe o mance and p oduc i i y o pai 29. Taken oge he , success ul collabo a i e p oblem sol ing is a complex combina ion o mul iple in e wined ac o s, such as cogni i e, social and emo ional ac o s (Lud igsen 2016). He e, o be e unde s and how pa icipan s p oduce lea ning in collabo a i e si u- a ions equi es a ocus no only on how hey build sha ed meaning bu also on how hey engage in his ac i i y (Dillenbou g e al. 2016;S ahl2007). Engagemen may appea a a ious le els, ha is, a he beha iou al le el (i.e. e o and con ibu ing o he ask), so- cial le el (i.e. he quali y o socio-emo ional in e ac ion and equi able pa icipa ion) and cogni i e le el (i.e. planning, moni o ing and e alua ion) (e.g. Sinha e al. 2015). In ela i ely limi ed collabo a i e si ua ions, such as in case 1 (Laughing Clowns), a a he shallow in e ac ional quali y migh be enough o comple e he ela i ely simple and sho ask o a symme ic s yle. Howe e , in he o he asks s udied, being iche and Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 21 o 29 asymme ical in hei designs, such as case 2 (Oli e Oil), he o he pa icipan ’s esou ces and unde s anding become mo e c i ical. Bu in ou da a, pa e ns o in e ac ional di i- cul ies o e en b eakdowns could be ound wi h he same pai ha had di icul ies in ask 1 (Laughing Clowns) bu because o he inc easing complexi y, no in such linea modes as p esen ed he e. In e ms o success ul ask comple ion, when oubles a ise in he s u- den s’in e ac ion, how pai s a e able and mo i a ed o epai hese encoun e s is c ucial. Fu he mo e, deepening ou unde s anding o he appea ance collabo a i e aspec s o CPS du ing an online assessmen o CPS skills, he cu en s udy con ibu es o he me hods o analysis o help isualise he in e ac ion pa e ns wi hin a pai o e eal hei collabo a i e app oach (o lack o ). We used a case s udy me hod (see e.g. Baška - ada 2014; Ge ing 2004) o ca e ully obse e s uden pai s building sha ed meaning and acqui ing CPS p ac ices in an online assessmen con ex . We applied mul iple da a sou ces and phases o analysis and, in his way, succeeded in main aining a chain o e idence o he audience o ollow (see Baška ada 2014; Yin 2009). Also, he me hods used we e ich in he sense ha hey combined di ec e idence due o an objec i e p ocess da a bu we e combined and seen in ela ion o he pa icipan s’subjec i e de- sc ip ions, as cued by he e ospec i e epo ing (CRR) da a. This can be seen o en- hance he alidi y o he cu en s udy and educe i s bias (see Baška ada 2014). Also, ou indings ein o ce he ecen unde s anding ha o be e g asp he quali y o social in e ac ion, one mus ex end he unde s anding om cumula i e accoun s o in e - ac ion owa d g asping he empo ali y in he da a (e.g. Kapu 2011; Kapu e al. 2008; Reimann 2009). Taken oge he , he dep h o he analysis is he p ima y s eng h o he case s udy me hod (Ge ing 2004). As a case s udy, he aim is no o gene alise o pop- ula ions bu a he o a i e a concep ualising he egula i ies o small-g oup p ocesses pe aining o he CPS cons uc (see also S ahl 2017) and o ans o m aci knowledge in o explici knowledge (Baška ada 2014). The Laughing Clowns case illus a es a qual- i y con as o mic o-in e ac ions in he mos condensed and con inuous o m, as op- posed o he mic o-in e ac ions in he o he asks o he bundle s uden s comp ised, such as Oli e Oil, which equi ed mo e comp ehensi e ex apola ion. The examples chosen he e, howe e , seem o be ep esen a i es o he gene al quali y o he da a co - pus p oduced by hese wo pai s ac oss all he asks o he bundle. Finally, because ou explo a o y case s udy was conduc ed in a highly s uc u ed as- sessmen en i onmen online, one cha ac e ised by a challenging communica ion chan- nel (cha ) and wi h pai s as he uni o analysis, o be able o gene alise he con as ing cases beyond his special esea ch design equi es eplica ion wi h mo e subjec s and addi ional case examples. Also, i should be no ed ha i is unclea whe he and how he assessmen si ua ion, e en wi h i being olun a y, impac ed he ways in which he s uden s collabo a ed. Fu he , because a CPS ac i i y is associa ed wi h bo h ask and social egula ion, one o he u u e lines o in es iga ion in his con ex could be whe he s uden s ope a ing in an online se ing may bene i om ools and suppo s uc u es o enhance egula ion ac i i ies and awa eness be ween hem. Fo example, o include no el me hods, such as dual-eye acking when wo king on online collabo a- i e asks analysed in espec o iden i ied key in e ac ional e en s o ocusing on he ‘mic o-moni o ing’o a pa ne ’s beha iou in dyads (Schneide and Pea 2013), could en ich he unde s anding o CPS. Mo eo e , in he cu en s udy, he challenges disco - e ed in e ms o collabo a ion we e no pa icula ly new and pe ained o he ela i ely Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 22 o 29 Appendix 1 Table 9 Examples o he coded p ima y da a om he cued e ospec i e epo ing (CRR) da a ( om he Laughing Clowns ask) Skill/elemen Beha iou indica o (as assessed in he ATC21S en i onmen ) An example o he CRR da a cap u ed o coding he ca ego y in he Laughing Clowns ask In e ac ion In e ac ing wi h pa ne (e.g. p esence o cha be o e allowing pa ne o make a mo e) ‘Bu hen she [ he pa ne ] was again so much sma e he e, ha she like s a ed o explain wha she was seeing.’ ‘In his ask i el su p isingly di icul jus o hink ha one should always explain o he o he pe son wha one is seeing.’ ‘I I ecall I ead he ins uc ion and hen, hen I s a ed o check i ou . and like wha , so i he e’s any idea as o wha we we’ e supposed o do he e, and hen we we e h owing he ball and hen discussed abou i [a sho laugh]’ ‘He e one somehow s a ed o be like, how would I pu i , you know like (one immedia ely s a ed o ell wha he e is and whe e) and desc ibe i like all he ime asking ha wha do you see, like o ell wha is happening on you own sc een’. Audience awa eness Adap s con ibu ions o inc ease unde s anding o pa ne ‘I unde s ood ha [ he pa ne ] has some hing di e en ly he e han on my sc een and ha I don’ know wha she’s go he e, so hen I el ha I ha e o explain wha I’m seeing. And I like wan ed o ask wha – hen we h ew [ he pa ne h ew] he ball again [a sho laugh] and I had his same ha I couldn’ see a all wha was happening, and I see ha he ball disappea s.’ ‘As I ha e now pu , (is he e) I hink I oo, [ he pa ne ] oo go . A e his I migh pu again o M3, no I mean, wha did I ge he e? So yes, I go R3. No, i was R2. Yes. Then he e I was eally hinking how i migh , o in ac I was hinking whe he i would go somehow like in a logical o de , so ha o me o ins ance some M3 is always epea ed, hen he e will be R2, some ce ain [pa e ns] like hese. And hen a e his, now soon he e, I hink, [ he pa ne ] will like ell how hese go, you know. Then I am like aha, yes, igh , yes and hen, bu she jus ge s always R, did she ge always R1 hen and o me R2 (o somehow like ha ).’ ‘I hink [ he pa ne ] did ell a some poin ha hey go. Because she’s [ old] me, so she old ha hey go, how he clown’s head goes. Then I jus hough ha well, should we s ill make su e, so ha le ’s use all hose balls.’ Responsibili y ini ia i e Takes esponsibili y o p og ess o he g oup ask ‘A his poin we we e ponde ing i we should mo e on o he nex one o s ill keep h owing hese balls, bu . Yes bu hen we decided o h ow s ill one ball each.’ ‘I ied as quickly as possible so ha we’ll ge his done.’ ‘And hen we ega ded ha we should mo e on. (This was indeed a di icul one.)’ Resou ce managemen Manages esou ces ‘A ha poin I was a li le a aid ha did she ealise ha he balls an ou .’ ‘Pe haps I was a bi in a hu y he e. [A sho laugh] and he e we no iced ha hey [ he balls] we e ac ually join ly o us. I hink we didn’ ead i qui e p ope ly he ins uc ion om he e. As I was pu ing his ball in o he [clown’s] mou h bu hen i suddenly disappea ed as my pa ne had pu i .’ ‘And hen we didn’ ealise [gi ing a laugh] ha hey a e he same balls. So in p inciple as I ha e wo balls le , hen she [ hough ] ha she, oo, has. [Pause] So hen we jus ealised ha hey a e join ly o us.’ Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 23 o 29 Table 9 Examples o he coded p ima y da a om he cued e ospec i e epo ing (CRR) da a ( om he Laughing Clowns ask) (Con inued) Skill/elemen Beha iou indica o (as assessed in he ATC21S en i onmen ) An example o he CRR da a cap u ed o coding he ca ego y in he Laughing Clowns ask Sys ema ici y Implemen s possible solu ions o a p oblem ‘I like in a way ied, so ha I’d been eady mysel he e now al eady o p ess ha ou compu e s a e wo king di e en ly, bu hen [ he pa ne ] s ill wan ed o y ou hose ows [L, M, R]. Uses hal o he balls o co e he posi ions in a sequen ial o de .’ ‘Well, we bo h had no iced ha i you d opped a ball when he mou h is he e a M, so i will hen go o numbe h ee. And hen we decided o es hose o he le e s as well. As we had o ind ou whe he he machines we e simila o no .’ ‘He e you could pe haps see how di e en I and [ he pa ne ] a e in a way, because I would ha e been a his s age [poin s o he cha sc een] so o eady in p inciple like o click ha we a e eady. And hen [ he pa ne ] like wan ed ha I would es i wi h wo mo e balls, al hough I had al eady no iced a his s age ha okay, i wo ks his way, om M o 2 and om R o 1, you know, so I don’ need o es i any u he . So [ he pa ne ] hough ha I should s ill wi h wo mo e balls, al hough I would ha e been like eady a his poin . [Pause, on he ideo, she is b owsing upwa ds on he cha sc een.] So he e I jus checked ha [ he pa ne ] had also ea lie , no I mean wha she had, ea lie some hing, I hink she es ed s ill one some hing. I appea ed he e like in he same way as wha she had go he e ea lie . So we could see ha i was like.’ Rela ionships Iden i ies connec ions and pa e ns be ween elemen s o knowledge ‘In a way so a his poin we bo h ha e ealised ha i wo ked di e en ly, when (-) like when we bo h had i . Because now again, o we’ e again ying o d op he balls. The wo s uden s come o an ag eemen on how hei machine wo ks.’ ‘So hen we s a ed o conside ha whe he hese compu e s a e hus wo king di e en ly, because when [ he pa ne ] h ows a ball so on my sc een (a ball disappea s), When I h ow a ball so hen – hen well, my compu e is wo king, (bu ). We conside ed ha he sc eens a e anyway qui e iden ical, he e a e hose le , middle and igh , (-- hey a e he same). [Pause] I can’ ecall wha I was hinking he e [gi ing a laugh] bu p obably ha whe e is i coming ou [ he ball]. and when i came ou om a di e en poin , so (I hough hey we e wo king) di e en ly, because i you h ow ha ball in he e and i comes a once, (O ou I mean). (Bu hen i could be nice you know also o look a bi ha which ball.)’ ‘So I s a ed o pay a en ion o wha e e is happening hen, launching hose balls. [Pause] And I mo e –I launched his one ball and hen I ealised, i made he R. I hink I ied again.’ Solu ion Co ec answe ‘Then jus as ha one didn’ go s aigh so I like ealised i a ha poin [ ha he compu e s we e unc ioning di e en ly]. Then I like con i med wha he ou come was Selec ion o he co ec op ion by S uden s A and B on how hei machines wo k.’ ‘(we no iced he e ha ) hey’ e eally wo king di e en ly, as [ he pa ne ] had go hose o he le e s.’ ‘I’d been eady mysel he e now al eady o p ess ha ou compu e s a e wo king di e en ly.’ The ca ego isa ion ma ix o he di ec ed con en analysis applied he same beha iou al indica o s ha we e used in he au oma ed coding p ocedu es in he ATC21S en i onmen (see Ca e e al. 2015) Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 24 o 29