This is a sel -a chi ed e sion o an o iginal a icle. This e sion
may di e om he o iginal in pagina ion and ypog aphic de ails.
Au ho (s):
Ti le:
Yea :
Ve sion:
Copy igh :
Righ s:
Righ s u l:
Please ci e he o iginal e sion:
CC BY 4.0
h ps://c ea i ecommons.o g/licenses/by/4.0/
Pai in e ac ions in online assessmen s o collabo a i e p oblem sol ing : case-based
po ai s
© The Au ho (s), 2018
Published e sion
Pöysä-Ta honen, Johanna; Ca e, Es he ; Awwal, Na isa; Häkkinen, Päi i
Pöysä-Ta honen, J., Ca e, E., Awwal, N., & Häkkinen, P. (2018). Pai in e ac ions in online
assessmen s o collabo a i e p oblem sol ing : case-based po ai s. Resea ch and P ac ice in
Technology Enhanced Lea ning, 13, 12. h ps://doi.o g/10.1186/s41039-018-0079-7
2018
RESEARCH Open Access
Pai in e ac ions in online assessmen s o
collabo a i e p oblem sol ing: case-based
po ai s
Johanna Pöysä-Ta honen
1*
, Es he Ca e
2,3
, Na isa Awwal
4
and Päi i Häkkinen
1
* Co espondence: johanna.poysa-
[email p o ec ed]
1
Finnish Ins i u e o Educa ional
Resea ch, Uni e si y o Jy askyla,
P.O. Box 35, 40014 Jy askyla, Finland
Full lis o au ho in o ma ion is
a ailable a he end o he a icle
Abs ac
This explo a o y case s udy ocuses on how pai s o s uden s can build a sha ed
unde s anding and acqui e collabo a i e p oblem-sol ing (CPS) p ac ices du ing an
online assessmen o CPS skills, which is seen in he con ex o he CPS cons uc , in
a symme ical and asymme ical ask ype. E en hough CPS is widely ecognised as
a co e wen y- i s -cen u y compe ency, i s na u e is no ye well unde s ood. Also,
un il ecen ly, mos o s udies ha e ocused on he indi idual’s solu ion o a p oblem
o on he skills indi iduals b ing in o a p oblem-sol ing space. This s udy ex ends
om an indi idual- o g oup-le el ocus in CPS, emphasising he ole and quali y o
he social aspec s in CPS p ocesses and ou comes. Focusing on he g oup le el
because i media es mul iple le els o lea ning, including indi idual cogni ion and
socio-cul u al p ac ices, may p o ide us wi h a be e unde s anding o how pai s
es ablish CPS p ac ices. Because o he complexi y o CPS and he gene al challenges
o emo e collabo a ion in an online con ex , he s udy elies on he iangula ion o
mul iple da a sou ces and phases o analysis. In his pape , he aim is o explo e and
isualise h ough con as ing case-based po ai s o wo pai s how mic o-in e ac ion
p ocesses e ol e a he pai le el. The esul s show ha despi e s uden s’simila CPS
pe o mance ou come sco es and ask designs aimed o acili a e collabo a ion,
a ia ions in mic o-in e ac ions occu ac oss pai s, o example as indi idual and
join solu ion endea ou s and as balanced and unbalanced dynamics o g oup
in e ac ions. S udying hese pa e ns a he pai le el may p o ide new insigh s
in o CPS and suppo s a egies o acqui ing hese p ac ices.
Keywo ds: Case s udies, Collabo a i e p oblem-sol ing, Compu e -suppo ed
collabo a i e lea ning, Di ec ed con en analysis, Small-g oup p ocesses, Social
aspec s o lea ning and eaching, Quali a i e esea ch
In oduc ion
The cu en a icle ocuses on he ole and quali y o he social aspec s in collabo a i e
p oblem-sol ing (CPS) p ocesses and ou comes, which is explo ed du ing an online
assessmen o CPS skills in pai s o uni e si y s uden s in a emo e collabo a ion con ex .
Recen ly, CPS has ecei ed inc easing in e es as one o he mos impo an gene ic skills
o be mas e ed, as illus a ed by la ge-scale na ional and in e na ional assessmen s (e.g.
Assessmen and Teaching o 21s Cen u y Skills (ATC21S) p ojec , www.a c21s.o g,and
OECD’s PISA 2015 s udy, h p://www.oecd.o g/pisa/; see also G aesse e al. 2017). CPS
© The Au ho (s). 2018 Open Access This a icle is dis ibu ed unde he e ms o he C ea i e Commons A ibu ion 4.0 In e na ional
License (h p://c ea i ecommons.o g/licenses/by/4.0/), which pe mi s un es ic ed use, dis ibu ion, and ep oduc ion in any medium,
p o ided you gi e app op ia e c edi o he o iginal au ho (s) and he sou ce, p o ide a link o he C ea i e Commons license, and
indica e i changes we e made.
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning
(2018) 13:12
h ps://doi.o g/10.1186/s41039-018-0079-7
and o he gene ic skills, o so-called wen y- i s -cen u y skills, such as communica ion,
c ea i i y, c i ical hinking, digi al li e acies and mo e, a e no seen as pa icula ly new bu
a e alued as c i ical o u u e lea ne s and wo ke s (e.g. Ca e e al. 2018;Ca ee al.2016;
G aesse e al. 2017; G i in and Ca e 2015; G i in e al. 2012; Scoula e al. 2017).
P oblem-sol ing skills, o example, ha e been assessed o se e al decades, o ins ance, in
he domain o ma hema ics, bu in cu en s udies and policy pape s, hese skills a e consid-
e ed discipline- ee skills ha should be mas e ed as he ac ual ocus o lea ning (Ha ding e
al. 2017).
Al hough sligh ly di e en amewo ks o CPS exis , he speci ic skills and expec a ions
iewed necessa y o he new wo k o ce a e consis en ac oss he a ious amewo ks. In
he cu en s udy, he de ini ion o CPS is consis en wi h ha o Hesse e al. (2015). In
he p esen s udy, ou heo e ical unde s anding o CPS builds on he ATC21S p ojec
and i s ex ensi e amewo k o echnology-enhanced o ma i e assessmen o CPS skills
(Hesse e al. 2015; Scoula e al. 2017). In his ega d, Hesse e al. concep ualised CPS as a
complex skill ha links c i ical hinking, p oblem sol ing, decision making and collabo -
a ion ac oss bo h social and cogni i e domains. CPS is deemed mo e complex han me ely
wo king oge he and equi es people wo king oge he on he same ask o pu sue a
common goal whe e di ision o labou exis s bu whe e inpu is in e wo en. In CPS, he
quali y o in e ac ions dis inguishes good collabo a ion (Dillenbou g 1999). In his con-
ex , collabo a ion equi es ac i e pa icipa ion h ough sea ches o ele an in o ma ion,
join use o esou ces, sha ed e alua ions and an ag eemen on s a egies and solu ion
pa hs (Ca e and G i in 2014). In CPS, indi iduals may possess di e en skills and in o -
ma ion, bu hey mus wo k collabo a i ely and exchange ideas o maximise hei join
ou come owa d a sha ed goal. The skills de ined by CPS a e pe inen o sol ing p ob-
lems ha a e, by de ini ion, complex, ill-s uc u ed and ambiguous.
Acco dingly, CPS comp ises a se o subskills ha consis o i e s ands o indi id-
ual- and g oup-le el capaci ies unde he b oad social and cogni i e skills (Hesse e al.
2015). The wo hypo hesised componen s o CPS a e no mu ually exclusi e. The social
componen d aws on he li e a u e om social and o ganisa ional psychology while he
cogni i e componen d aws hea ily on classical app oaches o indi idual p oblem sol -
ing. Following Hesse e al. (2015), social skills (i.e. pa icipa ion, pe spec i e aking and
social egula ion) a e abou managing pa icipan s (including onesel ), which e e s o
he ‘collabo a i e’pa o his me hod. Cogni i e skills (i.e. ask egula ion and know-
ledge building) a e abou managing he ask, which e e s o he ‘p oblem sol ing’pa
o he me hod. F om he social s ands o he CPS cons uc , ‘pa icipa ion’ e e s o he
eadiness o sha e in o ma ion and ex e nalise hough s, ‘pe spec i e aking’means he
abili y o ake in o accoun o he s’pe spec i es and ‘social egula ion’poin s o he
awa eness o he s eng hs and he weaknesses o he g oup membe s (i.e. g oup/ eam
awa eness; see F ansen e al. 2011). F om he cogni i e s ands o he CPS cons uc ,
‘ ask egula ion’is de ined as planning and moni o ing he skills o de eloping s a -
egies o p oblem sol ing and sha ed p oblem ep esen a ion (i.e. ‘join p oblem space’,
see Ba on 2003; Roschelle and Teasley 1995), whe eas ‘knowledge building’he e e e s
o he abili y o lea n and build knowledge h ough g oup in e ac ion. Wi hin each o
hese skills, he e a e subskills ( oge he , 19 subskills, including he ‘solu ion’), which
a e demons a ed by obse able indica o s o ac ions o p ocesses ( o an o e iew o
he ATC21S CPS amewo k, see Fig. 1).
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 2 o 29
In he ATC21S p ojec , a echnology-enhanced assessmen en i onmen was designed,
comp ising a se ies o Web-based in e ac i e, game-like asks (see e.g. Ca e e al. 2015).
The asks we e designed o measu e he skills, subskills and p ocesses cha ac e ised by
he CPS cons uc du ing dyads pa icipa ing in he CPS ac i i ies. The ATC21S assess-
men asks, discussed la e in he p esen pape , ely on and cap u e hese ac ions o p o-
cesses o measu e and e lec he CPS cons uc , as de ined by Hesse e al. (2015).
Al hough CPS is widely ecognised as a co e compe ency o wen y- i s -cen u y lea n-
ing, i s na u e is no well unde s ood pa icula ly in e ms o i s de elopmen ajec o y
om basic o mo e sophis ica ed le els (Scoula e al. 2017). Also, in he ATC21S assess-
men en i onmen , he p ima y ocus has been on how o assess he skill o he indi idual
s uden in collabo a i e pa ne ship (Ca e e al. 2016). In his explo a o y case s udy, he
ocus is ex ended o g oup- ela ional aspec s o CPS (Pöysä-Ta honen e al. 2016;
Pöysä-Ta honen e al. 2017). The cu en s udy emphasises he ole and quali y o he so-
cial aspec s o CPS p ocesses and ou comes (e.g. Ba on 2003; Dillenbou g e al. 2016;
Simpson e al. 2017; Wege i e al. 2017). Acco dingly, one pa o CPS is he ask, and
he o he pa is he en i onmen wi hin which he pa icipan s c ea e and sha e know-
ledge, moni o hei p og ess and de ec and epai he b eakdowns in communica i e
ac s ha may hampe collabo a ion (Al e man and Ha sch 2017; Roschelle and Teasley
1995). The cu en s udy concen a es on he second pa o CPS, which is seen i al in
compu e -suppo ed collabo a i e lea ning (CSCL) esea ch, as i is he con ex o he
cu en s udy. In ea lie wo k, i has been ound, o example, ha in collabo a i e lea n-
ing and p oblem sol ing, he p ocesses o building a sha ed unde s anding o he si ua ion
a hand and he quali y o in e ac ions appea ed o be, o some poin , e en mo e impo -
an han he ac ual abili ies o he indi idual pa icipan s when i came o sol ing he asks
(Dillenbou g e al. 2016; see also Ba on 2003; Simpson e al. 2017).
The cu en s udy akes a small-g oup ocus on CPS p ac ices. S udies on collabo a-
i e lea ning, in gene al, ha e educed hei scope o s udy ei he indi idual cogni ion
Fig. 1 The se o CPS skills and subskills om he ATC21S CPS amewo k (modi ied om Hesse e al. 2015;
see also Ca e e al. 2016; Scoula e al. 2017)
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 3 o 29
o socio-cul u al p ac ices (S ahl 2017). As S ahl (2017) s a ed (in lea ning esea ch),
‘T adi ional me hods (poin ing o quan i a i e o quali a i e me hods ha analyse ne
changes) o en p o ide e idence ha change has aken place, bu do no desc ibe how
he change has aken place’(p. 114, o iginal emphasis). Following S ahl (2017), i is a -
gued ha an emphasis on small-g oup p ac ices, because i media es he mul iple le els
in ol ed in lea ning (e.g. indi idual cogni ion on he one hand and he socio-cul u al
p ac ices o he class oom and ools used on he o he ), may p o ide us wi h a ounda-
ion o be e unde s and how CPS akes place in his con ex . No ably in he cu en
s udy, om heo e ical and me hodological poin s o iew, we ega d pai s as small
g oups (Williams 2010). As Williams (2010) s a ed, despi e ce ain p ope ies ha do
no lend hemsel es o small g oups (i.e. coali ion o ma ion o he di icul y o imagine
newcome s and old- ime s in a g oup o wo), ‘[i]n mos ins ances dyads a e g oups o
wo and ope a e unde he same p inciples and heo ies ha explain g oup p ocesses
o g oups o h ee and la ge ’(p. 268).
To explo e CPS as a dynamic, g oup-le el se o p ocesses in an online se ing, a
case-based s udy was unde aken. As Yin (2009) a gued, he how and why ques ions a e
be e answe ed h ough a case-based app oach because hey a e well sui ed o explo -
ing complex social phenomena. In case s udy esea ch, an in ensi e s udy o a single
uni can p o ide ways o be e unde s and a la ge class o simila uni s, especially
when elying on he iangula ion o mul iple da a sou ces (Baška ada 2014; Ge ing
2004; Hmelo-Sil e e al. 2009; Yin 2009). The e o e, in he cu en s udy, he da a
comp ise objec i e measu es and p ocess da a (i.e. au osco ing o CPS skills, ac i i y
logs and sc een eco dings) combined wi h subjec i e p ocess da a (i.e. cued e ospec -
i e epo ing (CRR) in e iew da a) o o m he basis o he desc ip i e analysis o
pai -le el CPS p ocesses, which a e ac ualised as p ocess isualisa ions o
mic o-in e ac ion p ocesses (see Da is e al. 2015). In he majo i y o he ATC21S s ud-
ies o da e (see Ca e e al. 2018), he scope o analysing he social aspec s has been on
he placemen (and occu ence) o cha ac ions in he CPS p ocess (Ca e e al. 2015)
a he han he cha con en i sel .
Focusing on he ‘how’o CPS equi es an analysis ha goes beyond pe o mance
sco es o he solu ion o he p oblem. The ‘how’explo es whe he di e en se s o in-
e ac ions can unde lie success ul comple ion and whe he he na u e o a ask s imu-
la es di e en se s o in e ac ions. Fo example, wo s yles o asks ha e been used o
assess CPS (Ca e and G i in 2017). One s yle p esen s and gi es access o he same
s imulus se o bo h he s uden s in a pai , and he o he p esen s and gi es access o
di e en s imulus se s o he wo s uden s. The i s s yle is ypically e e ed o as
symme ic and he second as asymme ic. Acco ding o Ca e e al. (2015), he accessi-
bili y he e e e s o ‘bo h o di ec e ie al as well as human capaci y o unde s and
and manipula e he equi ed a e ac s- be hese objec , knowledge, o p ocesses’(p. 87).
This di e ence be ween he ask ypes can easonably be hypo hesised as s imula ing
di e en se s o skills ha can be applied o he ask. In a symme ic ask ype, collab-
o a ion may be alued o i s social aspec bu may also be ega ded as a coun e o he
s uden s’in e es s (Ca e e al. 2015). In u n, he si ua ion o accomplishing an asym-
me ic ask, whe e esou ces a e no equally accessible o bo h s uden s in he pai , is
expec ed o inc ease he need o collabo a ion o be e p omp collabo a ion and he
collec i e ac i i y o de elop (Ca e e al. 2015).
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 4 o 29

Taken oge he , in he cu en s udy, we ha e b oadened he analysis beyond he
measu ed CPS skill le els o he indi idual s uden s, ha ing hem sea ch o he di e -
ences in he quali y o CPS p ocesses a he pai le el du ing he online assessmen o
CPS skills, in ega d o he di e en ypes o he asks. Based on mul iple, ich da a and
mul iple phases o analysis, we ask he ollowing:
(1) A e he e di e ences in he quali y o he mic o-in e ac ions a he pai le el ha
a e unde lying simila CPS measu es o indi idual s uden s?
(2) Does he ask ype (i.e. symme ic/asymme ic) quali a i ely p o oke di e en
mic o-in e ac ions in he s uden pai s?
Me hods
Pa icipan s and asks
The esea ch pa icipan s (n= 20) we e s uden s en olled in a mas e -le el eache edu-
ca ion p og amme a a Finnish uni e si y. S uden s we e andomly assigned o hei
pai s. Each s uden pai comple ed one bundle o asks in he ATC21S assessmen
en i onmen . The asks we e de eloped a he Assessmen Resea ch Cen e a he
Uni e si y o Melbou ne om 2010 o 2013. The assessmen en i onmen is based on
human- o-human app oaches o assessing CPS (Ca e e al. 2015) and comp ises a se
o online in e ac i e and collabo a i e p oblem-sol ing asks in STEM domains. In he
asks, he s uden pai s a e gi en a unique subse o esou ces ha a e equi ed o sol e
he p oblem. To ully unde s and he p oblem space and iden i y all he necessa y
esou ces, he s uden s need o ely on hei pa ne (Ca e e al. 2015; Ca e e al. 2016).
The communica ion akes place ia a ee- o m cha in e ace.
The se o asks used in he cu en s udy comp ised ou di e se asks, oge he las -
ing app oxima ely 60–90 min. Du ing he asks, s uden pai s (s uden s A and B) we e
sea ed in di e en class ooms o ensu e ha he only means o communica ion was he
cha in e ace. Fo his s udy, he bundle comp ised ‘Laughing Clowns’and ‘Oli e Oil’,
which a e con en - ee asks, and ‘Plan G ow h’and ‘Small Py amids’, which a e
con en -dependen asks (see Ca e e al. 2015). Con en - ee asks enhance induc i e
and deduc i e hinking whe eas con en -depended asks equi e s uden s o apply
knowledge ela ed o cu iculum.
Laughing Clowns is a symme ic ask, whe eas he o he h ee a e designed as
asymme ic. Acco ding o Ca e e al. (2015), in a symme ic ask, he s uden pai is
p esen ed wi h he same images, pe spec i es, ins uc ions and esou ces wi hin he
online ask space. In an asymme ic ask, he s uden pai is p esen ed wi h asymme -
ical pe spec i es, p o iding di e en in o ma ion and esou ces ( o simila and di -
e en iews o he pai s in a symme ic and asymme ic ask, espec i ely, see Figs. 2
and 3). In he cu en pape , we explo e he mic o-in e ac ion p ocesses o pai s in
comp ising (a) a symme ic ask (Laughing Clowns) and (b) an asymme ic ask
(Oli e Oil). These asks a e con en - ee asks ha do no equi e any p e equisi e
knowledge o he cu iculum and me ely ely on he applica ion o easoning (see
Ca e e al. 2015). We also educed he scope on he con en - ee asks o he bundle
used in his s udy. This cha ac e is ic may enhance compa abili y o he indi iduals,
since di e en ial access o he p io knowledge is ende ed i ele an .
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 5 o 29
In he Laughing Clowns ask, wo pa icipan s a e p esen ed wi h a clown machine
and 12 balls o be sha ed be ween hem. The goal is o he s uden s o de e mine
whe he hei clown machines wo k in he same way. To do his, he wo s uden s need
o sha e in o ma ion and discuss he ules, nego ia ing how many balls hey should
each use. The s uden s mus place he balls in o hei clown’s mou h while he mou h
is mo ing o de e mine he ule go e ning he di ec ion he balls will go (en y: le ,
middle and igh ; exi : posi ions 1, 2 and 3). Each s uden mus hen indica e whe he
hey belie e he wo machines wo k in he same way. S uden s do no ha e access o
each o he ’s sc een. In Fig. 2, he iews o s uden s A and B a e shown.
In he Oli e Oil ask, pa icipan s a e p esen ed wi h di e en iews o he ask, includ-
ing di e en esou ces. The goal is o ill a ja wi h 4 l o oli e oil wi h he a ailable e-
sou ces. To achie e his objec i e, he s uden s mus wo k ou wha esou ces a e
a ailable and how o sequence he use o hese esou ces. S uden A has a i ual 3-l ja ,
oli e oil dispense , ans e pipe and bucke . S uden B has a i ual 5-l ja , ans e pipe,
accep bu on and bucke . None o he pa icipan s a e awa e be o e beginning communi-
ca ion wha is a ailable o hei pa ne . The pai hus needs o ecognise ha s uden A
mus ill his o he ja a he dispense and place i unde he ans e pipe so ha s uden
B can accep he oil ans e ed by his o he pa ne om he pipe. Un il his poin , s u-
den B canno comple e any meaning ul ac ions and is dependen on he ac ions and in-
e ac ions on s uden A. The s uden s need o explo e and na iga e he ask space
oge he un il hey manage o place 4 l o oil in s uden B’s ja . This ask ollows he ea-
soning p ocesses equi ed in he Towe o Hanoi p oblem, which was popula ised by
ma hema ician Edua d Lucas in 1883 (Newell and Simon 1972;Pe ko ić2009). In he
Oli e Oil ask, bo h p oblem sol e s a e equi ed o wo k ou a sequence o mo emen s
o achie e he goal o he ask. S uden s s ill do no ha e access o each o he ’ssc eens,
Fig. 2 A sc een cap u e om he sc een ac i i y da a (pe spec i es o s uden s A and B, Laughing Clowns)
(In Finnish)
Fig. 3 A sc een cap u e om he sc een ac i i y da a (pe spec i es o s uden s A and B, Oli e Oil)
(In Finnish)
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 6 o 29
and he ask is complica ed by he di ision o esou ces and dispe sed in o ma ion among
dyads. Figu e 3shows he di e en pe spec i es o s uden s A and B.
Da a collec ion
Objec i e measu es and p ocess da a: au o-sco ing and sc een eco dings
In he ATC21S assessmen en i onmen , s uden s a e assessed indi idually du ing dyad
pa icipa ion in CPS ac i i ies. CPS asks a e designed o gene a e speci ic beha iou s,
aligned wi h he skills and subskills o he CPS cons uc (see Fig. 1; Ca e e al. 2016;
Hesse e al. 2015; Scoula e al. 2017). S uden s’comple ion o he asks gene a es log
s eam da a (i.e. he ac i i y log), and he pa e ns in hese da a (i.e. he mo emen o
a e ac s and he occu ence o cha ) a e cap u ed algo i hmically and a e coded by he
sco ing engine and hen sco ed acco ding o he Rasch model, p oducing in o ma ion
on s uden s’social and cogni i e skill le els (see Adams e al. 2015; Ca e e al. 2016).
The ac i i y log shows he e en s be ween wo s uden s wo king on he ask. (Fo an
example o ac i i y log, see Table 1).
Acco dingly, o cap u e he CPS skills and he subskills shown in Fig. 1, indica i e be-
ha iou s a e p oposed o sugges he e idence o he beha iou being obse ed, demon-
s a ing he p esence o absence o a pa icula subskill and how much o ha subskill
is p esen in an indi idual (see e.g. Adams e al. 2015; Ca e e al. 2016). Examples o
he CPS skills and he indica i e beha iou s in he Laughing Clowns and Oli e Oil
asks a e shown in Table 2.
Beha iou s hypo hesised as indica i e o pa icula subskills o elemen s a e hus ex-
ac ed om he log s eam h ough au oma ed coding p ocesses. Au o-sco ing cap-
u es wo ypes o esponse ep esen ing common and unique e en s (Adams e al.
2015; Ca e e al. 2016). Common e en s include beginning and end o a ask, sys em
messages con i ming ac ions and p o iding na iga ion, and cha be ween pa ne s.
Unique, o local, e en s include hose ha a e speci ic o ask. Fo he Laughing
Clowns ask, o example, hese include he d ag and d op o a ball and i s loca ion. A
coun o he ‘d opShu e’ac ions (d opping he balls in o he clown’s mou h, see Table 1),
o example, can be coded o indica e how well a s uden managed hei esou ces ( he
balls). These coded indica o s a e classi ied as ei he di ec o in e ed. Indica o s ha
Table 1 A sec ion om an ac i i y log ( ask Laughing Clowns) ( ansla ed om Finnish)
Task Page Playe E en ype Con en s Times amps
23 1 A Cha Do you ha e any clue wha we’ e supposed o do he e? 06/10/2015 16.45
23 1 B Cha I don’ know I’m hinking :D 06/10/2015 16.45
23 1 A Cha Yeah, would you h ow i s ? 06/10/2015 16.45
23 1 B Cha Le ’s h ow a ball and see wha happens 06/10/2015 16.45
23 1 B Ac ion s a D ag:ball1:70:150 06/10/2015 16.45
23 1 B Ac ion s opD ag:ball1:70:150 06/10/2015 16.45
23 1 B Ac ion s a D ag:ball1:70:150 06/10/2015 16.45
23 1 B Ac ion s opD ag:ball1:509:135 06/10/2015 16.45
23 1 B Ac ion d opShu eL:ball1:509:135 06/10/2015 16.45
23 1 B Cha Whe e did i come ou o you? 06/10/2015 16.46
Table 1displays he ask and playe IDs ( ask 23, s uden A o B), he page o he ask (he e, page 1), ac ions (e.g. he
e en ype), he con en s o he cha ( ex exchanged) and hei imes amps
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 7 o 29
a e classi ied as di ec a e hose ha desc ibe a pa icula ac ion, which can be in e p e ed
in i s own igh . Indica o s classi ied as in e ed include hose which include a sequence o
ac ions, om which sequence a pa icula hinking o exchange p ocess can be in e ed. Fo
example, i one s uden akes a pa icula ac ion, he e is hen a cha exchange, ollowed by
he s uden changing he ac ion, i can be in e ed ha he cha o he pa ne has in lu-
enced he i s s uden . An example o he au o-sco ing app oach is p o ided in Table 3.
As desc ibed in Adams e al. (2015), da a a e con e ed in o dicho omies o pa ial
c edi , wi h each indica o being ea ed somewha simila ly o i ems on a adi ional
es . In u n, hese a e agg ega ed wi h like indica o s ac oss se bundles o asks.
Cu -o alues o de e mine le els o quali y ac oss subskills a e calcula ed o he pu -
pose o scaling. Applying he Rasch measu emen model, i is possible o de e mine he
pa e ns wi hin he CPS concep ual amewo k and o in e p e a s uden ’s CPS abili y
Table 2 Example o social o cogni i e skills and indica i e beha iou s o exempla asks in
Laughing Clowns and Oli e Oil (adap ed om Ca e e al. 2015)
CPS elemen s/
subskills
Indica i e beha iou E idence o da a
Laughing Clowns Oli e Oil
In e ac ion In e ac s wi h he
pa ne
P esence o cha be o e
allowing he pa ne o
make a mo e
P esence o cha be o e allowing
he pa ne o make a mo e
Audience
awa eness
Adap s con ibu ions o
inc ease unde s anding
o he pa ne
Numbe o ball mo es
a emp ed be o e s opping
and wai ing o he pa ne
o mo e o espond
P esence o in o exchange on
indi idual con aine s a es
Resou ce
managemen
Manages esou ces Realises ha balls a e
mean o be sha ed and
uses only allo ed hal
(No obse ed)
Rela ionships Iden i ies connec ions and
pa e ns be ween elemen s
o knowledge
The wo s uden s come
o an ag eemen on how
hei machine wo ks
P esence o cha exchanging
in o ma ion when A o B ecognises
he signi icance o his o he ja
con aining only 1 l
Re lec s and
moni o s
Adap s easoning o
cou se o ac ion as
in o ma ion o
ci cums ances change
(No obse ed) Lea ning om edundan ac i i ies,
such as A mo ing ja o bucke
Solu ion A i es a he co ec
answe
Iden i ies connec ions and
pa e ns be ween elemen s
o knowledge
Las ac ion equi es B’s ja o con ain
4 l o oil
Table 3 Use o algo i hm o ela e ac ions o he elemen sys ema ici y o he collabo a i e
p oblem-sol ing cons uc (adap ed om Adams e al. 2015)
Indica o
name
De ails Algo i hm Ou pu
U2L004A
U2L004B
Sys ema ic
app oach
All posi ions ha e
been co e ed.
Sco ing ule:
h eshold alue
Task name:
Laughing Clowns
S ep 1: Find all d op ball occu ences cap u ed as d opShu e and
hei co esponding posi ions as d opShu eL, d opShu eR and
d opShu eM.
S ep 2: Then coun all he occu ences o he ac ion eco ded
unde ‘d opShu e’and hei unique posi ions om he log.
S ep 3: Inc ease he alue o he indica o by 1 i 1 o mo e
‘d opShu e’occu s in he o m o d opShu eR, d opShu eL
o d opShu eM.
S ep 4: I he o al numbe o unique d opShu es (d opShu eR,
d opShu eL and d opShu eM) om he log is less han 3, he
alue o he indica o is de ined as −1 o indica e missing da a
Coun alues
In Table 3, he i s column ep esen s he name o he indica o , being unique o each indica o . In he example, ‘U2’
poin s o he Laughing Clowns ask, ‘L’ e e s o local and explains ha he indica o is unique o he ask, ‘004’means
ha he indica o is ou h c ea ed o he ask and ‘A’means ha he indica o is applicable o s uden s A and ‘B’ ha i
is applicable o s uden B
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 8 o 29
Fig. 6 Join solu ion endea ou (pai 24, Laughing Clowns). The ull episode as a gene al pa e n o
in e ac ion is p esen ed on he le . On he igh , an exce p (indica ed as a g ey a ea in he ull episode) is
p esen ed, including a ime-s amped, condensed ac i i y pa e n (A = ac ion, C = cha ). The dialogue is
ex ac ed om he ac i i y log, and he speech bubbles con ain exce p s om he coded CRR da a ha a e
ela ed o he con en o he pa icula exce p . (T ansla ed om Finnish). (The ull episode is a ailable also
in Fig. 10 in Appendix 2)
Fig. 7 Indi idual solu ion endea ou (pai 29, Laughing Clowns). The ull episode as a gene al pa e n o
in e ac ion is p esen ed on he le . On he igh , an exce p (indica ed as a g ey a ea in he ull episode) is
p esen ed, including a ime-s amped, condensed ac i i y pa e n (A = ac ion, C = cha ). The dialogue is
ex ac ed om he ac i i y log, and he speech bubbles con ain exce p s om he coded CRR da a ha a e
ela ed o he con en o he pa icula exce p . (T ansla ed om Finnish). (The ull episode is a ailable also
in Fig. 11 in Appendix 3)
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 15 o 29

Pa icipan s needed o be awa e ha hei balls we e sha ed and ha he mos e ec i e
way o inding he solu ion was o assign he balls so ha bo h s uden s would ha e an ad-
equa e and equal oppo uni y o y hei machine and each a conclusion. In con as o
pai 24, pai 29 did no discuss he alloca ion bu s a ed wo king independen ly, d opping
and d agging he balls wi hou in e ac ing and coo dina ing hei e o s. Also, e idence o
audience awa eness (i.e. how a pa icipan adap s con ibu ions o inc ease unde s anding
o o he s) is missing wi h pai 29. S uden s who a e p o icien in his a ea would be mo e
likely o in e ac wi h hei pa ne be ween ball d ops and adap hei beha iou o bes
sui hei pa ne ’s needs (in con as , see pai 24). S uden 24B s a ed (see exce p 1): ‘[…]
I unde s ood ha she [ he pa ne ] has some hing di e en ly he e han on my sc een
and ha I don’ knowwha she’s go he e, so hen I el ha I ha e o explain wha I’m
seeing […]’(exce p 1, s uden 24B, CRR in e iew).
In e ms o he cogni i e aspec s o CPS, s uden s wi h a low p o iciency o e-
sou ce managemen skills may only hink o hemsel es when checking how hei
machine unc ions, he eby monopolising he use o esou ces, which seems o be
he si ua ion wi h pai 29, while mo e skilled s uden s would mo e likely ecognise
he need o sha ed use o he balls and hence would sha e hem equally (pai 24).
Mo eo e , o each a solu ion, he s uden s needed o iden i y he ela ionship be-
ween he en y and exi poin o he balls and de e mine i he e was consis ency
in how he machines unc ion; he pai needed o cons uc a way o ep esen ing
his in o ma ion and communica ing i o each pa ne , as well as being able o
unde s and he o he o ms o ep esen a ion ha he pa ne uses. Fo example,
o pai 29, s uden A p o ided a na a i e while s uden B lis ed pieces o
in o ma ion. S uden 29B desc ibed hei pa allel wo king on he ask as ollows
(exce p 2):
[…] Well my pa ne (didn’ ealise pe haps) ha I had checked i . Then he e a one
poin I s ill es ed and hen I w o e down whe e hey go, so ha i I d op he e om
L. And hen I ied o es he same ones like in hose i i goes always like in he
same way. Bu hen a some poin when hose pa ne ’s messages appea ed he e, so
hen I s ill ied wi h hose, and hen did i a he same ime (you know) and hen I
w o e i down like ha [as le e s and numbe s on cha ] so ha whe e hey go om
hose […] (exce p 2, s uden 29B, CRR in e iew).
Skil ul s uden s will also challenge he pa e ns and es he assump ions ha unde pin
hei obse a ions, which happened wi h s uden 29B (exce p 2). The inal s ep com-
p ised he s uden s compa ing hei ep esen a ions so ha a decision conce ning he
simila i y o he unc ioning o he clown machine could be made (solu ion). Pai 24 com-
munica ed h oughou hei ac i i ies, whe eas pai 29 only discussed hei sha ed unde -
s anding owa d he end. I he ask was no a o ced collabo a ion exe cise ( hanks o he
las concluding ques ion), s uden 29A would ha e been able o sol e he ask qui e inde-
penden ly because he also poin ed ou he ollowing in he CRR in e iew (exce p 3):
[…] Like one can see i om his, I ealised like a once wha he ask he e is. And
hen I jus ied i ou a ew imes how he hing was unc ioning and epo ed [gi es
a sho laugh] igh away. I don’ know whe he one could ha e like [a sho laugh]
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 16 o 29
buil some coope a ion he e like so ha one would i s ha e like old he
ins uc ion in he way one unde s ood i […] (exce p 3, pai 29A, CRR in e iew).
Case 2: balanced e sus unbalanced CPS p ocess in e ms o he dynamics o in e ac ion ob-
se ed in he asymme ic ask Oli e Oil
In con as o he Laughing Clowns ask, in he asymme ic ask Oli e Oil, he cumula-
i e accoun s ob ained om he di ec ed con en analysis o pai s 24 and 29 (Table 8)
we e in line wi h he ask design, e en emphasising mo e social aspec s han he o i-
ginal design o he Oli e Oil ask did. Howe e , when compa ing hese cumula i e ac-
coun s om he di ec ed con en analysis on he CRR da a o he mic o-le el
in e ac ion p ocesses isible in he ac i i y logs, he e we e e iden di e ences ega d-
ing he quali y o collabo a ion ac oss he wo pai s. Acco dingly, he mic o-in e ac ion
con en s e ealed quali a i ely di e en CPS p ocesses as balanced e sus unbalanced
dynamics o in e ac ion ( o mic o-in e ac ion pa e ns, see Figs. 8and 9). E en hough
he asymme y o he ask s yle was designed o o ce in e ac ion in pai s, he quali y o
Fig. 8 Balanced CPS p ocess in e ms o he dynamics o in e ac ion (pai 24, Oli e Oil). The exce p
includes a ime-s amped, condensed ac i i y pa e n (A = ac ion, C = cha ); dialogue ex ac ed om he
ac i i y log, and he speech bubbles con ain exce p s om he coded CRR da a. (T ansla ed om Finnish)
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 17 o 29
he ways in which he pai s nego ia ed he ask seems o a y. Pai 24 con inued o e-
semble a join explo a ion p ocess. Pai 24 nego ia ed hei explo a ions and join ly de-
eloped and discussed hei unde s andings o sol e he ask (see Fig. 8). In pai 29 (see
Fig. 9), he dynamics o he in e ac ion in hei p oblem-sol ing p ocess we e unbal-
anced in e ms o he powe ela ions being unequal posi ions o s a us (see e.g. Simp-
son e al. 2017). S uden A ended o assign he ole o ‘leade ’o ‘d i e ’ o himsel in
he sha ed ac i i y, assigning s uden B he ole o ‘ ollowe ’o ‘passenge ’(Schneide
and Pea 2013; Shae e al. 2011). Howe e , s uden B seemed o accep his ole. In
Figs. 8and 9, he quali a i e di e en dynamics o he in e ac ion a e exempli ied. Fig-
u e 8depic s he pa e n o pai 24, which can be cha ac e ised as a balanced, join
in e ac ion, wi h equal dis ibu ion o wo k and esponsibili ies. Figu e 9, in u n, shows
an example o he unbalanced na u e o he dynamics o he in e ac ion in pai 29. Fig-
u e 9gi es an example o an explici disag eemen (Richie 2002; Simpson e al. 2017)
in which s uden 29A sho ly ques ions his pa ne ’s p oposal o he possible s eps o
each he p oblem solu ion, a ou ing his own idea. Figu e 9displays he pi o al mo-
men o pai 29 when he idea o how o sol e he ask is c ys allised o s uden B.
Howe e , he idea is ejec ed by s uden A, who simply con inues o implemen his
own idea o how o each he solu ion. When s uden B seemingly asked o con i m-
a ion, ‘[…] o did we go some hing w ong […]’, s uden A me ely eplied using a deli-
ca e impe a i e: ‘[…] No, bu we’ e s ill on he map when we do as ollows: […]’.
Fig. 9 Unbalanced CPS p ocess in e ms o he dynamics o in e ac ion (pai 29, Oli e Oil). The exce p includes
a ime-s amped, condensed ac i i y pa e n (A = ac ion, C = cha ). The dialogue is ex ac ed om he ac i i y
log, and he speech bubbles con ain exce p s om he coded CRR da a. (T ansla ed om Finnish)
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 18 o 29
Based on he asymme ical ask design, a he beginning o he Oli e Oil ask, s uden B
depended on he ac ions and in e ac ion o s uden A. S uden A, e en hough highly ac i e,
did no uly in e ac wi h s uden B du ing he p oblem-sol ing e o s; he s uden also
ailed o espond o he co-s uden ’s e o s o sol e he ask. Ins ead, s uden A a emp ed
o di ec he p ocess, as is acknowledged by his pa ne in he CRR in e iew (exce p 4):
‘[…]Ijus hough asmypa ne oldme he elikequi ealo wha Iha e odo,soI
hough hen ha he like igu es his ou […]’(exce p 4, s uden 29B, CRR in e iew).
F om he cogni i e aspec s o CPS, p oblem analysis is he i s s ep in planning.
P oblem analysis equi es pa icipan s o explo e he ask and de ine sub asks and con-
s uc subgoals o success ully sol e he p oblem oge he . In pai 29, e en hough s u-
den A seems ocused on un a elling he ask, he subgoals se we e no aligned a he
pai le el. Also, be o e o ganising he necessa y s eps o sol e he p oblem, he s uden s
needed o sha e in o ma ion and explain he esou ces a ailable, as was e icien ly dis-
cussed by pai 24. Pai 29 did sha e in o ma ion bu did no sys ema ically discuss he
elemen s each had on his o he sc een, as was he case wi h pai 24. This caused mis-
unde s andings. S uden 29A e lec ed on hei wo king (exce p 5): ‘[…] We wen
h ough wha we ha e he e. [Pause] Wha we can see on ou sc een, bu ac ually we
wen h ough his only la e […]’(exce p 5, s uden 29A, CRR in e iew).
The p oblem analysis phase is ollowed by o ganising he necessa y s eps o sol e he
ask. Cause and e ec equi ed he s uden s o success ully hink o s eps ahead o hei
cu en p oblem s a e and wo k ou sub asks be o e ac ing. In he Oli e Oil ask, he
essen ial equi emen was, in ad ance, o wo k ou sequences o mo emen s o achie e
he goal, ha is, o ill he ja wi h 4 l o oil. Pai 24B e lec ed on his (see exce p 6):
‘[…] I like his kind o hings whe e you mus hink like, hose ew mo es so ha you’ll
ge hem, you each he inal esul […]’(exce p 6, s uden 24B, CRR in e iew).
As he ci cums ances change, p o icien s uden s we e be e able o adap hei ea-
soning and e-o ganise hei unde s anding o he ask a hand (i.e. e lec s and moni-
o s; Hesse e al. 2015). Fo example, pai 24 ecalled he ollowing (exce p s 7 and 8):
s uden A said, ‘[…] So hen I was s ill hinking ha i one could pou i om he
bucke , so I ealised ( ha no hing is coming om he e). The e we no iced i , o now I
ealised how we should do his you know […]’(exce p 7, s uden 24A, CRR in e iew).
S uden B, in u n, o mula ed hei easoning as ollows:
[…] Well hen he e was he hing ha i would ha e gone he e in o he bucke he
igh amoun , ha is ou li es, bu hen we s a ed o hink ha pe haps i ’s such a
glass con aine a e all, whe e we should ha e i . [Pause, he ideo shows ha 24b
has a ull glass con aine .] The e is now ha one li e he e in [pa ne ’s] bucke .
Now, she only should h ee mo e, so hen he e would be he ou li es in he
bucke . Bu hen we ealise ha i ough o be a glass con aine . (We ha e hough
i ) w ong […] (exce p 8, s uden 24B, CRR in e iew).
In he Oli e Oil ask, he cogni i e elemen , ela ionship, was isible in he cha because
s uden A o B ecognised and communica ed he signi ican de ails o wha ollows i he
ja con ains only 1 l o oil. In pai 29, al eady in he middle o he en i e p oblem-sol ing
p ocess, s uden B came up wi h a co ec , de ailed plan o he sequences o mo emen s
o be execu ed o achie e he desi ed goal s a e, bu s uden A did no ecognise his
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 19 o 29
e o , con inuing wi h his ongoing ac ions, so he idea was dismissed (Fig. 7). S uden
29B depic ed he ollowing (exce p 9):
[…] As I know ha i [own idea] is igh , and hen as he was ye ha ‘le ’s do like
his’so hen I began o hink ha which one he con aine ac ually is. And as he
seemed o be so ce ain he e in his explana ion. And I was ce ain as well, bu as he
was like ha ‘le ’s do like his […] (exce p 9, s uden 29B, CRR in e iew).
La e on, s uden 29A came up wi h he same idea, as s uden 29B s a ed (see exce p 10):
[…] Now he would ha e liked o do i like ha , hen [laugh e ]. Now he’s explaining
he same wha I explained o him al eady ea lie [a sho laugh]. As now he ells ‘I’ll
send you 1 and hen I’ll send 3’so he e i was. And I was like in ac I old you so!
[a sho laugh]! I knew his all igh , ha hen i will be sol ed. Tha I knew i all he
ime. I knew his be o e him […] (exce p 10, s uden 29B, CRR in e iew).
Discussion and conclusions
The aim o he cu en explo a o y case s udy was o examine he mic o-in e ac ions in pai s
du ing he online assessmen o CPS skills in he ATC21S assessmen en i onmen . The
p esen s udy shi ed om an indi idual and solu ion-o ien a ion ocus on he CPS p ocess
o he g oup- ela ional aspec s o CPS. Ou g ounding assump ions we e based on he de in-
i ion o he CPS cons uc as comp ising mul iple, in e ac ing subelemen s o skills a he
han a linea se o p ocesses (see Ca e e al. 2016; Hesse e al. 2015; Scoula e al. 2017). In
he cu en s udy, we asked how s uden pai s accomplished ambiguous CPS asks and how
hei nego ia ions owa d sha ed unde s anding we e mani es ed in mic o-in e ac ions du ing
he CPS p ocess. We we e in e es ed in whe he di e en se s o in e ac ions can unde lie
success ul comple ion and whe he he na u e o a ask s imula es di e en se s o in e ac-
ions. Ou case-based po ai s showed ha despi e he s uden s’compa able and ela i ely
high o mode a e CPS pe o mance ou come sco es, especially in social skills (sco ed be-
ween he le els o 5–6, wi h 6 being he highes skill le el), a ia ions in mic o-in e ac ions
occu ed ac oss pai s, o example as indi idual and join solu ion endea ou s and as bal-
anced and unbalanced CPS p ocesses in pai s. Also, he asymme ical ask ype, in compa i-
son o he symme ical ask ype, appea ed o be mo e e ec i e in o cing in e ac ions in
pai s. Howe e , his did no gua an ee he ac ual quali y o he mic o-in e ac ions.
Acco dingly, in small-g oup collabo a ion, he e a e mul iple in e ac ing elemen s
ha con ibu e o he s uc u e and low o collabo a ion, such as he a o emen ioned
elemen s ha a e ela ed o he join p oblem space (Al e man and Ha sch 2017;
Roschelle and Teasley 1995). As was also wi nessed in case 1 (Laughing Clowns), sig-
ni ican challenges in collabo a ion may a ise i pa icipan s do no a emp o coo din-
a e hei indi idual pe spec i es in he join p oblem space. Pa icipan s who sensed a
co-p esence pe cei ed hei pa ne s and paid a en ion o each o he (Al e man and
Ha sch 2017). As shown (e.g. Go man 1963), o achie e a sense o co-p esence is es-
sen ial o high-quali y collabo a ion. This is, howe e , mo e demanding when s uden s
a e ope a ing in online se ings, such as he con ex o he cu en s udy.
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 20 o 29

In CPS, he p ocess o cons uc ing and main aining a sha ed unde s anding o he ask
a hand is needed o collabo a i e lea ning o e ol e (Dillenbou g e al. 2016; Roschelle
and Teasley 1995). Building a sha ed unde s anding is highly con ex ual: he media may
acili a e o hampe in e ac ions. When compa ed wi h co-loca ed collabo a ion, in an on-
line se ing, he c ea ion o a sha ed unde s anding equi es addi ional e o om he pa -
icipan s (Dillenbou g e al. 2016) because o egula e in e ac ions and achie e ag eemen
is much mo e complex (Al e man and Ha sch 2017). He e, he e may be p essu e o
achie e join goals wi h educed le els o sha ing. Bu as Da is e al. (2015)s a ed,no -
mally, in join ac i i y, people b ing hei own, indi idual goals o he si ua ion, bu a he
same ime, hey join ly wo k owa d a de ini ion o he si ua ion ha wo ks o gi e he
in e ac ion cohe ence (p. 61). In case 2 (Oli e Oil), he oles aken and assigned (i.e. s u-
den A as a leade , s uden B as a ollowe , see Schneide and Pea 2013) may e lec ( ol-
lowing Simpson e al. 2017) he ways in which powe ela ions and in ellec ual au ho i y
a e nego ia ed in g oup in e ac ions in CSCL se ings. Despi e he many posi i e p omises
o CSCL, such as inc eased lea ning gains esul ing om join in e ac ion, Simpson e al.
(2017) no ed he dange s o CSCL en i onmen s, such as he possibili y o hampe ing he
lea ning oppo uni ies o hose s uden s who lack a oice and a e subjec ed o hei
co-s uden s’ins uc ions (see also Lange -Osuna 2016).
Consequen ly, as Dillenbou g e al. (2016) poin ed ou abou collabo a i e lea ning in
gene al, he asks a e p ima ily designed o acili a e sha ed meaning and collabo a ion.
In he CPS cons uc examined he e, he asks, e en hough being p ima ily designed
o measu e indi idual s uden ’s CPS skills in a collabo a i e si ua ion, canno be suc-
cess ully comple ed wi hou he pai (e.g. Ca e e al. 2015,2016; Hesse e al. 2015; Sco-
ula e al. 2017). Howe e , collabo a i e ac i i y and lea ning should no be conside ed
some hing ha would au oma ically happen: despi e he simila asks and equal suppo
p o ided o he pa icipan s, he pedagogical designs can be pe cei ed and in e p e ed
di e en ly by di e en pa icipan s (e.g. A aja and Pöysä-Ta honen 2013; Simpson e
al. 2017; Sinha e al. 2015). Fo example, he deg ee o sha ed unde s anding ha he
pai s needed o each was ela ed o he ype o ask hey needed o pe o m (Dillen-
bou g e al. 2016). In case 1, he symme y o he esou ces a ailable o he pai s migh
ha e dilu ed he e ec o in e dependence. In case 2 (Oli e Oil), in u n, he asymme y
o he ask ac i a ed posi i e in e ac ion pa e ns, such as sha ed nego ia ions and ue
ac s o collabo a ion, as expec ed om his s yle o ask o ensu e he emo al o , bu
no gua an eeing, unbalanced in e ac ions. In line wi h he indings o Simpson e al.
(2017), his seemingly in luenced he pe o mance and p oduc i i y o pai 29.
Taken oge he , success ul collabo a i e p oblem sol ing is a complex combina ion o
mul iple in e wined ac o s, such as cogni i e, social and emo ional ac o s (Lud igsen
2016). He e, o be e unde s and how pa icipan s p oduce lea ning in collabo a i e si u-
a ions equi es a ocus no only on how hey build sha ed meaning bu also on how hey
engage in his ac i i y (Dillenbou g e al. 2016;S ahl2007). Engagemen may appea a
a ious le els, ha is, a he beha iou al le el (i.e. e o and con ibu ing o he ask), so-
cial le el (i.e. he quali y o socio-emo ional in e ac ion and equi able pa icipa ion) and
cogni i e le el (i.e. planning, moni o ing and e alua ion) (e.g. Sinha e al. 2015). In
ela i ely limi ed collabo a i e si ua ions, such as in case 1 (Laughing Clowns), a a he
shallow in e ac ional quali y migh be enough o comple e he ela i ely simple and sho
ask o a symme ic s yle. Howe e , in he o he asks s udied, being iche and
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 21 o 29
asymme ical in hei designs, such as case 2 (Oli e Oil), he o he pa icipan ’s esou ces
and unde s anding become mo e c i ical. Bu in ou da a, pa e ns o in e ac ional di i-
cul ies o e en b eakdowns could be ound wi h he same pai ha had di icul ies in ask
1 (Laughing Clowns) bu because o he inc easing complexi y, no in such linea modes
as p esen ed he e. In e ms o success ul ask comple ion, when oubles a ise in he s u-
den s’in e ac ion, how pai s a e able and mo i a ed o epai hese encoun e s is c ucial.
Fu he mo e, deepening ou unde s anding o he appea ance collabo a i e aspec s o
CPS du ing an online assessmen o CPS skills, he cu en s udy con ibu es o he
me hods o analysis o help isualise he in e ac ion pa e ns wi hin a pai o e eal
hei collabo a i e app oach (o lack o ). We used a case s udy me hod (see e.g. Baška -
ada 2014; Ge ing 2004) o ca e ully obse e s uden pai s building sha ed meaning
and acqui ing CPS p ac ices in an online assessmen con ex . We applied mul iple da a
sou ces and phases o analysis and, in his way, succeeded in main aining a chain o
e idence o he audience o ollow (see Baška ada 2014; Yin 2009). Also, he me hods
used we e ich in he sense ha hey combined di ec e idence due o an objec i e
p ocess da a bu we e combined and seen in ela ion o he pa icipan s’subjec i e de-
sc ip ions, as cued by he e ospec i e epo ing (CRR) da a. This can be seen o en-
hance he alidi y o he cu en s udy and educe i s bias (see Baška ada 2014). Also,
ou indings ein o ce he ecen unde s anding ha o be e g asp he quali y o social
in e ac ion, one mus ex end he unde s anding om cumula i e accoun s o in e -
ac ion owa d g asping he empo ali y in he da a (e.g. Kapu 2011; Kapu e al. 2008;
Reimann 2009). Taken oge he , he dep h o he analysis is he p ima y s eng h o he
case s udy me hod (Ge ing 2004). As a case s udy, he aim is no o gene alise o pop-
ula ions bu a he o a i e a concep ualising he egula i ies o small-g oup p ocesses
pe aining o he CPS cons uc (see also S ahl 2017) and o ans o m aci knowledge
in o explici knowledge (Baška ada 2014). The Laughing Clowns case illus a es a qual-
i y con as o mic o-in e ac ions in he mos condensed and con inuous o m, as op-
posed o he mic o-in e ac ions in he o he asks o he bundle s uden s comp ised,
such as Oli e Oil, which equi ed mo e comp ehensi e ex apola ion. The examples
chosen he e, howe e , seem o be ep esen a i es o he gene al quali y o he da a co -
pus p oduced by hese wo pai s ac oss all he asks o he bundle.
Finally, because ou explo a o y case s udy was conduc ed in a highly s uc u ed as-
sessmen en i onmen online, one cha ac e ised by a challenging communica ion chan-
nel (cha ) and wi h pai s as he uni o analysis, o be able o gene alise he con as ing
cases beyond his special esea ch design equi es eplica ion wi h mo e subjec s and
addi ional case examples. Also, i should be no ed ha i is unclea whe he and how
he assessmen si ua ion, e en wi h i being olun a y, impac ed he ways in which he
s uden s collabo a ed. Fu he , because a CPS ac i i y is associa ed wi h bo h ask and
social egula ion, one o he u u e lines o in es iga ion in his con ex could be
whe he s uden s ope a ing in an online se ing may bene i om ools and suppo
s uc u es o enhance egula ion ac i i ies and awa eness be ween hem. Fo example,
o include no el me hods, such as dual-eye acking when wo king on online collabo a-
i e asks analysed in espec o iden i ied key in e ac ional e en s o ocusing on he
‘mic o-moni o ing’o a pa ne ’s beha iou in dyads (Schneide and Pea 2013), could
en ich he unde s anding o CPS. Mo eo e , in he cu en s udy, he challenges disco -
e ed in e ms o collabo a ion we e no pa icula ly new and pe ained o he ela i ely
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 22 o 29
Appendix 1
Table 9 Examples o he coded p ima y da a om he cued e ospec i e epo ing (CRR) da a
( om he Laughing Clowns ask)
Skill/elemen Beha iou indica o (as assessed in
he ATC21S en i onmen )
An example o he CRR da a cap u ed o coding
he ca ego y in he Laughing Clowns ask
In e ac ion In e ac ing wi h pa ne (e.g. p esence o
cha be o e allowing pa ne o make a
mo e)
‘Bu hen she [ he pa ne ] was again so much
sma e he e, ha she like s a ed o explain wha
she was seeing.’
‘In his ask i el su p isingly di icul jus o hink
ha one should always explain o he o he pe son
wha one is seeing.’
‘I I ecall I ead he ins uc ion and hen, hen I
s a ed o check i ou . and like wha , so i he e’s any
idea as o wha we we’ e supposed o do he e, and
hen we we e h owing he ball and hen discussed
abou i [a sho laugh]’
‘He e one somehow s a ed o be like, how would I
pu i , you know like (one immedia ely s a ed o ell
wha he e is and whe e) and desc ibe i like all he
ime asking ha wha do you see, like o ell wha is
happening on you own sc een’.
Audience
awa eness
Adap s con ibu ions o inc ease
unde s anding o pa ne
‘I unde s ood ha [ he pa ne ] has some hing
di e en ly he e han on my sc een and ha I don’
know wha she’s go he e, so hen I el ha I ha e
o explain wha I’m seeing. And I like wan ed o ask
wha – hen we h ew [ he pa ne h ew] he ball
again [a sho laugh] and I had his same ha I
couldn’ see a all wha was happening, and I see
ha he ball disappea s.’
‘As I ha e now pu , (is he e) I hink I oo, [ he
pa ne ] oo go . A e his I migh pu again o M3,
no I mean, wha did I ge he e? So yes, I go R3. No,
i was R2. Yes. Then he e I was eally hinking how i
migh , o in ac I was hinking whe he i would go
somehow like in a logical o de , so ha o me o
ins ance some M3 is always epea ed, hen he e will
be R2, some ce ain [pa e ns] like hese. And hen
a e his, now soon he e, I hink, [ he pa ne ] will
like ell how hese go, you know. Then I am like aha,
yes, igh , yes and hen, bu she jus ge s always R,
did she ge always R1 hen and o me R2 (o
somehow like ha ).’
‘I hink [ he pa ne ] did ell a some poin ha hey
go. Because she’s [ old] me, so she old ha hey go,
how he clown’s head goes. Then I jus hough ha
well, should we s ill make su e, so ha le ’s use all
hose balls.’
Responsibili y
ini ia i e
Takes esponsibili y o p og ess o
he g oup ask
‘A his poin we we e ponde ing i we should mo e
on o he nex one o s ill keep h owing hese balls,
bu . Yes bu hen we decided o h ow s ill one ball
each.’
‘I ied as quickly as possible so ha we’ll ge his
done.’
‘And hen we ega ded ha we should mo e on.
(This was indeed a di icul one.)’
Resou ce
managemen
Manages esou ces ‘A ha poin I was a li le a aid ha did she ealise
ha he balls an ou .’
‘Pe haps I was a bi in a hu y he e. [A sho laugh]
and he e we no iced ha hey [ he balls] we e
ac ually join ly o us. I hink we didn’ ead i qui e
p ope ly he ins uc ion om he e. As I was pu ing
his ball in o he [clown’s] mou h bu hen i
suddenly disappea ed as my pa ne had pu i .’
‘And hen we didn’ ealise [gi ing a laugh] ha hey
a e he same balls. So in p inciple as I ha e wo balls
le , hen she [ hough ] ha she, oo, has. [Pause] So
hen we jus ealised ha hey a e join ly o us.’
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 23 o 29
Table 9 Examples o he coded p ima y da a om he cued e ospec i e epo ing (CRR) da a
( om he Laughing Clowns ask) (Con inued)
Skill/elemen Beha iou indica o (as assessed in
he ATC21S en i onmen )
An example o he CRR da a cap u ed o coding
he ca ego y in he Laughing Clowns ask
Sys ema ici y Implemen s possible solu ions o a
p oblem
‘I like in a way ied, so ha I’d been eady mysel
he e now al eady o p ess ha ou compu e s a e
wo king di e en ly, bu hen [ he pa ne ] s ill wan ed
o y ou hose ows [L, M, R]. Uses hal o he balls
o co e he posi ions in a sequen ial o de .’
‘Well, we bo h had no iced ha i you d opped a ball
when he mou h is he e a M, so i will hen go o
numbe h ee. And hen we decided o es hose
o he le e s as well. As we had o ind ou whe he
he machines we e simila o no .’
‘He e you could pe haps see how di e en I and [ he
pa ne ] a e in a way, because I would ha e been a
his s age [poin s o he cha sc een] so o eady in
p inciple like o click ha we a e eady. And hen [ he
pa ne ] like wan ed ha I would es i wi h wo
mo e balls, al hough I had al eady no iced a his
s age ha okay, i wo ks his way, om M o 2 and
om R o 1, you know, so I don’ need o es i any
u he . So [ he pa ne ] hough ha I should s ill
wi h wo mo e balls, al hough I would ha e been like
eady a his poin . [Pause, on he ideo, she is
b owsing upwa ds on he cha sc een.] So he e I jus
checked ha [ he pa ne ] had also ea lie , no I mean
wha she had, ea lie some hing, I hink she es ed
s ill one some hing. I appea ed he e like in he
same way as wha she had go he e ea lie . So we
could see ha i was like.’
Rela ionships Iden i ies connec ions and pa e ns
be ween elemen s o knowledge
‘In a way so a his poin we bo h ha e ealised ha
i wo ked di e en ly, when (-) like when we bo h had
i . Because now again, o we’ e again ying o d op
he balls. The wo s uden s come o an ag eemen
on how hei machine wo ks.’
‘So hen we s a ed o conside ha whe he hese
compu e s a e hus wo king di e en ly, because
when [ he pa ne ] h ows a ball so on my sc een (a
ball disappea s), When I h ow a ball so hen – hen
well, my compu e is wo king, (bu ). We conside ed
ha he sc eens a e anyway qui e iden ical, he e a e
hose le , middle and igh , (-- hey a e he same).
[Pause] I can’ ecall wha I was hinking he e [gi ing
a laugh] bu p obably ha whe e is i coming ou
[ he ball]. and when i came ou om a di e en
poin , so (I hough hey we e wo king) di e en ly,
because i you h ow ha ball in he e and i comes
a once, (O ou I mean). (Bu hen i could be nice
you know also o look a bi ha which ball.)’
‘So I s a ed o pay a en ion o wha e e is
happening hen, launching hose balls. [Pause] And I
mo e –I launched his one ball and hen I ealised, i
made he R. I hink I ied again.’
Solu ion Co ec answe ‘Then jus as ha one didn’ go s aigh so I like
ealised i a ha poin [ ha he compu e s we e
unc ioning di e en ly]. Then I like con i med wha
he ou come was Selec ion o he co ec op ion by
S uden s A and B on how hei machines wo k.’
‘(we no iced he e ha ) hey’ e eally wo king
di e en ly, as [ he pa ne ] had go hose o he
le e s.’
‘I’d been eady mysel he e now al eady o p ess ha
ou compu e s a e wo king di e en ly.’
The ca ego isa ion ma ix o he di ec ed con en analysis applied he same beha iou al indica o s ha we e used in he
au oma ed coding p ocedu es in he ATC21S en i onmen (see Ca e e al. 2015)
Pöysä-Ta honen e al. Resea ch and P ac ice in Technology Enhanced Lea ning (2018) 13:12 Page 24 o 29