Resea ch A icle h ps://doi.o g/10.12973/ijem.10.2.1013
In e na ional Jou nal o Educa ional Me hodology
Volume 10, Issue 2, 213 – 225.
ISSN: 2469-9632
h p://www.ijem.com/
Cogni i e Technology o E alua e he Academic Lea ning o
Compu a ional Cogni ion in Psychology S uden s
Guadalupe Elizabe h Mo ales-
Ma inez*
Na ional Au onomous Uni e si y o Mexico,
MEXICO
Angel Ga cia-Collan es
Dis ance Uni e si y o Mad id, SPAIN
Ra ael Manuel Lopez-Pe ez
Fundación Uni e si a ia Beha io &
Law, SPAIN
Recei ed: No embe 19, 2023 ▪ Re ised: Janua y 2, 2024 ▪ Accep ed: Ma ch 14, 2024
Abs ac : This s udy illus a ed an al e na i e way o e alua e s uden s’ academic lea ning. I in ol ed he join and in e wined
applica ion o he na u al seman ic ne wo k echnique, compu e simula ions, and seman ic p iming expe imen s o assess he
cogni i e changes in knowledge s uc u es due o academic lea ning in wo g oups o psychology s uden s. The expe imen al
g oup was en olled in a cou se on compu a ional cogni ion, while he con ol g oup was obli ious o his cou se. The esul s
indica ed ha he cogni i e assessmen ools disc imina e he cogni i e changes p oduced as a esul o gene al aining
unde aken in a psychology deg ee e sus he in luence o a speci ic cou se. A e he cou se, he expe imen al g oup inc eased
hei echnical ocabula y, changed hei concep ual alua ion o de ine s ela ed o compu a ional heo ies o mind, and
eo ganized he ela ions among de ine s acco ding o he compu a ional cogni ion app oach. Also, his g oup p esen ed a
highe connec i i y index be ween he concep s o he seman ic ne wo k, hei concep ual ac i a ion le el and concep ual co-
ac i a ion pa e n changed, and hei access le el o he e alua ed schema’s concep s imp o ed. In con as , he con ol g oup
did no show signi ican changes in hei cogni i e pa e ns a e he cou se. These indings sugges ha cogni i e ools may be
help ul in he diagnosis o academic lea ning.
Keywo ds: Academic lea ning, cogni i e assessmen , na u al seman ic ne wo ks, psychology s uden s, seman ic p iming.
To ci e his a icle: Mo ales-Ma inez, G. E., Ga cia-Collan es, A., & Lopez-Pe ez, R. M. (2024). Cogni i e echnology o e alua e he
academic lea ning o compu a ional cogni ion in psychology s uden s. In e na ional Jou nal o Educa ional Me hodology, 10(2), 213-
225. h ps://doi.o g/10.12973/ijem.10.1.1013
In oduc ion
Lea ning assessmen is one o he mos exci ing challenges o he 21s cen u y educa ion, equi ing e olu ion in he
ein o eaching models and echnology ad ancemen s and in ega d o unde s anding socie y's eme ging needs. In
addi ion, de e mining aspec s o lea ning assessmen s and how hey should be e alua ed in educa ional se ings is a
majo , e y di icul ask (Na ional Resea ch Council [NRC], 2001; Nichols & Sug ue, 1999). Mos academic es s
measu e s uden s' sho - e m memo y skills as knowledge acquisi ion; o example, eache s adminis e mul iple-
choice, ue- alse, o open-ques ion es s a he end o a cou se o e i y whe he he s uden has e ained lea ned
knowledge. The co ec answe is p oo o he s uden 's acqui ed knowledge, bu such con en ional e alua ion does no
de e mine i a s uden has de eloped a long- e m cogni i e abili y; many s uden s use s a egies o pass exams wi hou
engaging in meaning ul long- e m lea ning (Ma zano, 1994; Ma zano & Cos a, 1988; Ma zano e al., 1990).
A con empo a y e alua ion o academic lea ning equi es he use o e alua ion ools o measu e abili ies ela ed o
cogni i e in o ma ion p ocessing, which a e cen al o aining 21s -cen u y s uden s, who li e in an economy la gely
dependen on in o ma ion managemen (A ieli-A ali, 2013). Howe e , he e a e sca ce al e na i es emb acing his
mode n ision o lea ning, echnology de elopmen s, and ad ancemen s in science lea ning. In line wi h his conce n,
he NRC (2001) used he ad ancemen o cogni i e science o e hink assessmen app oaches, since ad ances in
cogni i e science expanded he knowledge su ounding impo an lea ning dimensions. Fu he , i s ad ances in
measu emen echniques open possibili ies o unde s and and in e p e inc easingly complex e idence ela ing o he
academic pe o mance o s uden s.
*
Co esponding au ho :
Guadalupe Elizabe h Mo ales-Ma inez, Na ional Au onomous Uni e si y o Mexico (UNAM), Mexico.  gemo ama @ho mail.com
© 2024 The au ho (s); licensee IJEM by RAHPSODE LTD, UK. Open Access - This a icle is dis ibu ed unde he e ms and condi ions o he
C ea i e Commons A ibu ion License (h ps://c ea i ecommons.o g/licenses/by/4.0/).
214  MORALES-MARTINEZ ET AL. / Cogni i e Technology o E alua e he Academic Lea ning
In consonance wi h he NRC’s ision, Mo ales-Ma inez e al. (2023) highligh ed he use ulness o cogni i e assessmen
ools o e alua e he p ocess and esul s o lea ning, since measu emen ad ances in cogni i e psychology a e
compa ible wi h new echnologies. The in e wined use o cogni i e pa adigms o s udy he human mind wi h
compu a ional de elopmen s o e s an oppo uni y o c ea e o inno a e digi al me hods o app oxima e he lea ning
assessmen , as is illus a ed in he nex sec ion.
Li e a u e Re iew
The p oposal o linking cogni ion and lea ning heo ies wi h lea ning assessmen p ac ices and eaching is no a new
concep (see NRC, 2001), A ieli-A ali (2013) no ed how since he p e ious cen u y, he e ha e been sophis ica ed
app oaches o enchain cogni i e science wi h lea ning assessmen . Mo e ecen ly, Lopez-Rami ez e al. (2014)
p oposed he Ch onome ic Cons uc i e Cogni i e Lea ning E alua ion Model, o C3 LEM, which wo ks unde he
p inciples o se ial and pa allel human in o ma ion p ocessing and p o ides cogni i e ools ha explo e how he
s uden 's mind wo ks when o ming knowledge s uc u es.
Con en ionally, he C3 LEM sugges s using in e wined ch onome ic and men al ep esen a ion echniques o assess
academic lea ning p ocesses (selec ion, s o age, and e ie al o he in o ma ion s o ed in he s uden s' memo y). The
applica ion o C3-LEM in ol ed wo phases (Figu e 1): he cons uc i e cogni i e e alua ion and he ch onome ic
cogni i e e alua ion o knowledge. The i s assesses he meaning o ma ion on knowledge men al ep esen a ion and
e eals hei cogni i e cha ac e is ics ( he concep ual o ganiza ion, s uc u e, and dynamic) h ough men al
ep esen a ion echniques (Na u al Seman ic Ne wo ks) and compu a ional simula ions.
Figu e 1. Phases and Componen s o he C3-LEM (Mo ales-Ma inez, Angeles-Cas ellanos, e al., 2020)
The Na u al Seman ic Ne wo ks (NSN) echnique (Figue oa e al., 1976) is a me hodological app oach o s udying
men al ep esen a ion, which implies he eco e y o in o ma ion om human memo y by using concep ual clues
( a ge s). This echnique is use ul o explo ing he de elopmen o knowledge schemas acqui ed h oughou he
academic yea . Acco ding o Mo ales Ma ínez, López Pé ez, e al. (2020), applying an ini ial NSN in a cou se and
compa ing i o he inal NSN allows o obse a ion o he quali a i e (con en ) and quan i a i e cogni i e changes due
o academic lea ning. Compa ison o bo h NSN, ini ial and inal, in ol es con as ing se e al indica o s such as he
seman ic ichness (numbe o concep ual nodes), seman ic ele ance o he concep s (M alue), and he In e -Response
Times ( empo a y pa e ns o de ine s appea ance), among o he s.
The NSN echnique equi es a de ini ion ask; pa icipan s de ine co e a ge concep s o he academic cou se using
de ine s (nouns, e bs, adjec i es, p onouns). Following his, hey a e hese de ine s based on hei ela ionship
deg ee wi h he a ge (Mo ales-Ma inez, Angeles-Cas ellanos, e al., 2020). These sco es suppo he compu a ional
simula ions o he knowledge schema's beha io a he beginning and end o he academic yea (Lopez-Rami ez e al.,
2014). Lopez-Rami ez e al. (2015) used his echnique o explo e schema ic ac i i y in academic lea ning ac oss
di e en knowledge domains. Fu he mo e, compu e simula ions can indica e connec ions among concep s only
obse ed wi h hese ool ypes. Fo example, Gonzalez e al. (2013) obse ed ha high school s uden s es ablished
implici ela ionships among concep s o an NSN on mo al cou se. They obse ed he concep o pa en co-ac i a ed
police, e en hough hese concep s appea seman ically un ela ed. Howe e , when he esea che s conside ed he
social and cul u al con ex o he pa icipan s, hey no ed a ela ionship o psychological signi icance.
In gene al, he NSN analysis and he compu e simula ions allow us o obse e he changes in he men al
ep esen a ion o s uden s' knowledge p oduced by lea ning a knowledge domain. Combining hese echniques wi h
expe imen al cogni i e s udies inc eases he po en ial o e alua e o he ele an aspec s o academic knowledge as he
empo al pa e ns o schema ic beha io . Fo example, he implemen a ion o ch onome ic cogni i e e alua ion
s udies helps ob ain empo al ends in he capaci y o access in o ma ion om memo y.
In e na ional Jou nal o Educa ional Me hodology  215
Mo ales-Ma ínez and San os-Alcan a a (2015), ollowing he p oposal o E.-O. López-Ramí ez (pe sonal
communica ion, Augus 9 h, 2014), used he In e -Response Time (IRT) o NSN o analyze he in o ma ion accessibili y
le el. These au ho s epo ed ha he de ine s wi h he g ea es weigh end o appea be ween 30 and 40 seconds, and
hey appea ed be ween he hi d and i h posi ions on he lis . Howe e , he e was no discussion abou wha his
posi ioning means. I is unknown i his is he case o all knowledge domains and which a iables in luence he
eco e y o academic concep s.
In addi ion o IRT, eac ion imes (RT) a e ano he indica o o he ch onome ic cogni i e e alua ion; hese come om
schema ic wo d ecogni ion s udies applied be o e and a e he cou se. The RT can be classi ied h ough a neu al
ne wo k o disc imina e whe he he e was an in eg a ion o he academic con en in o s uden s’ long- e m memo y
s uc u es a he end o a cou se. To achie e his, he C3-LEM includes he applica ion o he seman ic p iming
pa adigm h oughou lexical decision asks (McNama a, 2005), which consis s o p esen ing wo d pai s wi h di e en
ela ionship ypes (e.g., associa i e, ca ego ical, schema ic, un ela ed). The expe imen al ask is o men ally ead he
i s (p ime) and he las ( a ge ) wo d and decide whe he he a ge is spelled co ec ly.
The ecogni ion imes o a ge s o e in o ma ion abou how he con ex p eceding schema ic wo ds a ec s s uden s'
in o ma ion p ocessing. Suppose he p esen a ion o a s imulus (wo d o image) is p eceded by ano he seman ically
ela ed s imulus. In ha case, s uden s will ecognize he second s imulus mo e quickly o accu a ely han in he case o
no seman ic ela ionship be ween he wo s imuli. In a C3-LEM s udy, he seman ic ela ionships be ween he
schema ic wo d pai s a e ele an . I he knowledge schema does no exis in he s uden 's memo y a he beginning o
he cou se, and by he school yea end, s uden s ha e in eg a ed he in o ma ion in o hei knowledge s uc u es, he
ecogni ion ime o schema ic pai s will signi ican ly educe by he cou se end.
Lopez (1996) and Lopez and Theios (1992) p oposed ha he seman ic p iming e ec p oduced by a schema ic
ela ionship is a concep e med “schema ic p iming.” In academic lea ning, schema ic p iming is p esen jus o hose
wo d ecogni ion asks ha in ol e concep s ela ed o he knowledge schema o he e alua ed cou se. The e idence
om cogni i e s udies shows ha when a s uden s o es he concep ual nodes lea ned in class in his long- e m
memo y, he wo d ecogni ion imes ela ed o he lea ned schema dec ease a he cou se end (e.g., see Gonzalez e al.,
2013; Mo ales Ma ínez, López Pé ez, e al., 2020). The opposi e happens when s uden s do no consolida e he
in o ma ion in hei memo y (U diales-Iba a e al., 2018). In sho , his ype o knowledge o ganiza ion cogni i e
phenomenon in long- e m memo y (schema ic p iming) can accoun o a lea ning p ocess on academic knowledge
schemas.
Techniques such as hose in ol ed in he C3-LEM can be aluable ools in he assessmen o , as, and o lea ning (see
Mo ales-Ma inez, 2020; Mo ales-Ma inez & Lopez-Rami ez, 2016; Mo ales-Ma inez e al., 2015, 2017), hus, i is
essen ial o accumula e e idence o he wo h o his cogni i e app oach o e alua e lea ning e ec i ely. The e o e,
using his e alua ion model, his esea ch explo ed he changes in s uden s' knowledge s uc u es due o he lea ning
conduc ed in a ace- o- ace psychology cou se. The i s ques ion in his esea ch was o de e mine he pa e n o
cogni i e knowledge s uc u e changes ha s uden s expe ience a e lea ning a speci ic opic (Compu a ional
Cogni ion). The second ques ion was o disc imina e he pa e n o cogni i e changes among s uden s en olled and no
en olled in he cou se; as he esul s o his s udy poin ed ou , he e a e di e ences in cogni i e pa e ns ela ed o
expe ise in his ield. The cogni i e measu emen ools accoun ed o cogni i e di e ences be ween he change
di ec ed by speci ic lea ning and he spon aneously p oduced by non-sys ema ic exposu e o gene al psychology
educa ion.
Me hodology
Resea ch Design
The p esen au ho s used a quali a i e and quan i a i e mixed me hod (C3-LEM) o measu e he cogni i e dimension o
academic lea ning in a Compu a ional Cogni ion cou se. Fi s , hey designed and applied an NSN s udy; hen, hey
pe o med a compu a ional simula ion on he da a o his i s s udy using a neu al ne wo k o cons ain sa is ac ion
p oposed by Lopez and Theios (1992). Finally, hey implemen ed expe imen al esea ch h ough he seman ic p iming
pa adigm.
Sample
Two g oups o i s -yea psychology s uden s pa icipa ed in his s udy (74 women and 30 men). The i s a ended a
compu a ional cogni ion cou se (expe imen al g oup), and he second was he con ol g oup, which did no ha e access
o he in o ma ion on he subjec e alua ed. The con ol and he expe imen al g oups came om di e en ins i u ions.
The pa icipan s had a mean age o 19 yea s ( ange 17 o 27 yea s, SD= 1.7). Pa icipa ion was olun a y, and all
pa icipan s ga e in o med consen .
216  MORALES-MARTINEZ ET AL. / Cogni i e Technology o E alua e he Academic Lea ning
Ins umen s
The au ho s selec ed en s imuli om he heo y discussed in he cou se (mind, compu a ion, compu a ional mind, HIP,
on Neumann, Tu ing machine, connec ionism, memo y, wo king memo y, and long- e m memo y) o design he NSN
s udy. Concep selec ion ollowed Mo ales-Ma inez's P o ocol o he Collec ion o Concep s (Mo ales-Ma inez, 2015),
which consis s o se e al concep ual analysis s eps on he subjec o e alua ion. This is a guide o selec ing a ge
concep s o c ea e he NSN s udy as well as 30 de ine s o design he seman ic p iming expe imen . The au ho s
o ganized he de ine s in p ime- a ge pai s (e.g., so wa e-p ocesses, algo i hm-language, and memo y-p ocesso ).
Fu he mo e, hey added 15 associa i e concep pai s (e.g., bee-s ing, ai plane-pilo , den is - oo h) and 15 un ela ed
wo d pai s (e.g., loo -sc een, moun ain-blood, wa -ele a o ).
P ocedu e
This esea ch in ol ed an announcemen , whe e he s uden s ecei ed an in i a ion o pa icipa e olun a ily, also hey
lea ned abou he s udy objec i es and bene i s, and he au ho s ga e a p i acy wa ning abou hei da a. Following
his, he s uden s who chose o pa icipa e ga e hei in o med consen and ecei ed speci ic ins uc ions o NSN and
seman ic p iming s udy asks. Finally, he s uden s comple ed bo h asks, and he au ho s pe o med a compu e
simula ion on he da a om he i s s udy.
Na u al Seman ic Ne wo k S udy: The Men al Rep esen a ion o Knowledge
The pa icipan s pe o med a concep ual de ini ion ask, which in ol ed de ining en a ge concep s ha emb aced he
compu a ional cogni ion opic. The a ge s appea ed andomly, one by one, and emained in he compu e sc een's
cen e o 60 seconds. The pa icipan de ined each a ge wi h e bs, nouns, adjec i es, and p onouns and la e
quali ied indi idually he concep ual quali y o each de ine conce ning i s a ge . As Lopez (1996) and Lopez and
Theios (1992) sugges ed, concep ual sco es a ied be ween one and en; he smalle he numbe , he lowe he quali y
o he schema ic ela ionship be ween he de ine and i s a ge .
Compu e Simula ion: The Schema ic Dynamics o Knowledge S uc u es
Da a ob ained om he NSN s udy ed a cons ain -sa is ac ion neu al ne wo k using EVCOG so wa e. The
compu a ional simula ion p ocedu e ollowed Lopez and Theios' (1992) o mula:
WIJ = -1n{[p(X=0 & Y = 1) p(X=1 & Y = 0)]*[p(X=1 & Y = 1) p(X=0 & Y = 0)]-1} [1]
X and Y ep esen pai s o concep s. Ob aining he associa ion g ade among X and Y equi es calcula ing he p(X = 1 & Y
= 0) alue by de e mining he join p obabili y ha X appea s when Y does no appea in a SAM g oup. Fu he mo e,
p(X = 0 & Y = 1) and p(X = 0 & Y = 0) a e calcula ed simila ly. Calcula ing he p(X = 1 & Y = 1) alue conside s a
hie a chical modula ion o he M alue in each SAM g oup and hei in e connec i i y h ough he neu o-compu a ional
ne wo k. These calcula ions se ed o con igu e a connec i i y ma ix (SASO Ma ix) (Seman ic Analyze o Schema a
Beha io ) (Lopez & Theios, 1992) use ul o he seman ic analysis o schema ic beha io .
Seman ic P iming S udy: The Men al Ch onome y o Academic Lea ning
Finally, a seman ic p iming s udy de e mined he deg ee o lea ned in o ma ion consolida ion in he s uden 's memo y.
The s udy p esen ed pai s o wo ds (p ime- a ge ) ha could ha e di e en ela ionship kinds (associa i e s.
schema ic s. un ela ed). Figu e 2 illus a es he expe imen al sequence.
Figu e 2. The Sequence o an Expe imen al Tes o he Seman ic P iming S udy
The expe imen al sequence consis ed o a black do in he compu e sc een cen e p esen ed o 500 ms, a e he i s
concep appea ed (p ime) o 250 ms. Finally, he las wo d ( a ge ) appea ed and emained on he sc een un il he
pa icipan pe o med he expe imen al ask, which consis ed o eading each pai o wo ds (p ime- a ge ) silen ly and
In e na ional Jou nal o Educa ional Me hodology  217
hen making a lexical judgmen abou he a ge . The pa icipan ca ego ized he las wo d as ‘wo d’ o ‘no wo d’. The
s udy du a ion anged om 7 o 9 minu es, depending on he pa icipan 's pe o mance.
Da a Analysis
Fi s , he au ho s pe o med a mul idimensional scaling analysis on he i s s udy's da a o inspec he o ganiza ion o
he pa icipan s' schema. A e , hey ca ied ou a compu a ional simula ion h ough a cons ain -sa is ying neu al
ne wo k. This kind o ool acili a es he obse a ion o knowledge schema ac i a ion and co-ac i a ion beha io .
Finally, a hi d analysis explo ed he ch onome ic pa e ns o in o ma ion p ocessing ela ed o he e alua ed cou se.
The au ho s ob ained he IRT ( he ime each s uden used o eco e each de ine o he NSN om hei memo y).
Following his, hey applied a mixed ANOVA on he RT ob ained in bo h he expe imen al and con ol g oups’ seman ic
p iming s udy o de e mine he consolida ion le el o in o ma ion in he s uden ’s memo y a he beginning and he end
o he cou se.
The au ho s ca ied ou analysis o a iance wi h conside a ion ha he le el o measu emen o he dependen
a iable was a io. The obse a ions had a condi ion o independence as he expe imen al condi ions and he
pa icipan s we e assigned andomly o his s udy. Rega ding equino mali y in he da a, he QQ plo showed ha he
da a dis ibu ion was no mal, while Le ene's es showed ha he a iances we e equal ac oss he expe imen al
condi ions o bo h he con ol g oup and he expe imen al g oup.
Resul s
The au ho s o ganized he esul s in h ee dimensions: he i s desc ibes he con en and o ganiza ion o he
knowledge schema on compu a ional cogni ion. The second is he obse a ion o he schema ic dynamics and
connec i i y pa e n o he NSN. The las dimension examines he concep ual accessibili y and consolida ion o he
in o ma ion in he lea ned schema.
Schema's Con en and O ganiza ion Analysis
A mul idimensional scaling analysis e ealed ha he expe imen al g oup p esen ed changes in he concep ual
o ganiza ion and quali y o he de ine s ela ed o compu a ional cogni ion. While hey used concep s om a mixed
psychology and echnology gene al schema in he cou se beginning, owa d he cou se's end, he s uden s o ganized
hei concep s in wo concep ual axes unde a specialized schema ela ed o compu a ional cogni ion (Figu e 3). One
axis in ol ed HIP (Human In o ma ion P ocessing) and PDP (Pa allel Dis ibu ed P ocessing) concep s. The second axis
inco po a ed concep s ela ed o he body-mind duali y issue om a cogni i e pe spec i e, o he wise known as he
ha dwa e-so wa e me apho .
Figu e 3. De ine s’ Concep ual O ganiza ion a he Cou se Beginning and he End o he Expe imen al and Con ol G oup

218  MORALES-MARTINEZ ET AL. / Cogni i e Technology o E alua e he Academic Lea ning
In con as , al hough he con ol g oup inco po a ed new concep s owa ds he cou se end, hey belonged o a gene al
psychology schema, and hei o ganiza ion emained ela i ely simila a he cou se beginning and end.
Connec i i y Pa e ns and Schema Dynamics
Academic lea ning exp ession also includes he o ma ion o new connec ions, he loss o exis ing ones, o he change in
connec ion weigh s. Figu e 4 shows he changes in he ini ial and inal connec ions be ween he a ge s e alua ed by
he expe imen al and con ol g oups. No ice in he igu e ha he expe imen al g oup inc eased he numbe o
connec ions on di e en a ge s. Fo example, a he beginning o he cou se, hey connec ed compu a ion only wi h
h ee a ge s (compu a ional mind, Tu ing machine, and HIP); a e he cou se, pa icipan s o med six new
connec ions ( on Neumann, long- e m memo y, wo king memo y, connec ionism, memo y, mind) wi h his same
concep o nine inal connec ions. Fu he mo e, hey los concep ual nodes, disconnec ed long- e m memo y, and on
Neumann, and o med a connec ion be ween he Tu ing machine and compu a ion. Also, pa icipan s inc eased o
dec eased he connec i i y s eng h among di e en a ge s. Fo example, he compu a ional mind- on Neumann pai
s eng hened hei ela ionship by inc easing he numbe o common de ine s om one o i e owa d he cou se end.
The con ol g oup gained and los connec ions and changed connec ion s eng h among di e en a ge pai s. Howe e ,
he con ol g oup had e y li le change in concep s such as HIP compa ed o he expe imen al g oup's pe o mance.
Figu e 4. Connec i i y G aph Ob ained Be o e and A e he Cou se by he Expe imen al and Con ol G oup
No e. A colo node ep esen s each a ge , and he node's size is ela ed o he connec i i y deg ee. The de ine s' numbe
connec ing each a ge pai is o e he linking line. The mo e de ine s connec each a ge pai , he da ke he
line.
On he o he hand, he au ho s analyzed wo aspec s o explo e he schema ic beha io : he le els o schema ac i a ion
be o e and a e he cou se and he co-ac i a ion pa e n among de ine s. In bo h cases, he SASO ma ix helped g aph
he schema beha io . The esul s indica ed ha only he expe imen al g oup's schema ac i i y was signi ican ly
modi ied. By con as , in he con ol g oup, he ac i a ion emained simila be ween he ini ial and inal measu emen s
(Figu e 5). The expe imen al g oup inc eased i s concep ual ac i a ion le el a he cou se end, as shown in Figu e 5,
whe e g ea e ac i a ion is p esen ed wi h a ed and highe g aph posi ion.
In e na ional Jou nal o Educa ional Me hodology  219
Figu e 5. Expe imen al and Con ol G oup Su ace Plo s
The au ho s illus a ed he co-ac i a ion schema pa e ns by ac i a ing common concep s wi h he highes M alues in
he expe imen al g oup (Figu e 6) and hese concep s expe ienced a meaning change a he cou se end. In his ega d,
he b ain wen om a ision di ec ly associa ed wi h memo y o a cogni i e science iew (neu al ne wo ks). The
compu e passed om a classical compu a ion schema owa d a human in o ma ion p ocessing ision. Howe e , hese
concep s did no expe ience any change in he con ol g oup.
220  MORALES-MARTINEZ ET AL. / Cogni i e Technology o E alua e he Academic Lea ning
Figu e 6. Pa e ns o Co-Ac i a ion o Two De ine s wi h he Two Highes M Values in Bo h G oups
Men al Ch onome y and Consolida ion o he Schema
A quali a i e analysis o he expe imen al g oup's IRT e ealed ha he ela ion among concep ual accessibili y o
de ine s in hei M alues is nega i e (Figu e 7). The co ela ion inc eased signi ican ly o he expe imen al g oup a
he end o he cou se ( = -.31) compa ed wi h he ini ial NSN ( = -.16). This means ha a e s uden s lea ned
compu a ional cogni ion schema, hey accessed de ine s wi h he highes M alue in a sho e ime han when he
concep s ob ained less seman ic ele ance. Simila ly, he con ol g oup exhibi ed a co ela ion change om he ini ial
NSN ( = -.42) o he inal NSN ( = -.55).
Figu e 7. Rela ionship Be ween he In e -Response Times and he M Values Ob ained o All he De ine s by he
Expe imen al and Con ol G oups a he Cou se Beginning and End
In e na ional Jou nal o Educa ional Me hodology  221
The expe imen al g oup changed in he pa e n o access ime and seman ic ele ance o de ine s (Figu e 7). Howe e ,
a s uden ’s es o dependen samples, pe o med on he access imes on he common de ine s ob ained be o e (M =
30, SD = 9.6) and a e he cou se (M = 27, SD = 3.1), indica ed ha he e was no a s a is ically signi ican di e ence.
The sca ci y o common de ine s and he high a iabili y in he IRT om he cou se beginning likely in luenced his
esul . On he o he hand, a s uden 's - es was also applied o he M alues ob ained in he common de ine s be o e (M
= 62, SD = 23) and a e (M = 148, SD = 50) he cou se, and he di e ence was s a is ically signi ican ( (8) = -4.17, p =
.004).
On he o he hand, he con ol g oup showed g ea e changes in he M alues han in he common de ine s' IRT
dis ibu ion. In his ega d, a s uden ’s - es ( (42) = .85) indica ed ha he e a e no signi ican di e ences be ween
he IRT means ob ained o he commons de ine s a he cou se beginning (M = 29, SD = 10) and he end (M = 28, SD =
9.2). Howe e , he e was a signi ican di e ence ( (42) = -2.77, p = .008) be ween he M alue means o he ini ial (M =
65 SD = 37) and inal NSN (M = 56 SD = 24). Figu e 8 shows hese changes in he in e ac ion pa e n be ween IRT and
M- alue, only o he common de ine s be ween he ini ial and inal NSN.
Figu e 8. Rela ionship Be ween he In e -Response Times and he M Values Ob ained o he Common De ine s by he
Expe imen al and Con ol G oups a he Beginning and End o he Cou se
The inal ch onome ic analysis comp ised a mixed ANOVA o 2(G oup: Expe imen al s. Con ol) x2(Cou se ime: S a
s. End) x3(Seman ic ela ionship: Associa i e s. Schema ic s. None) on eac ion imes ob ained in he seman ic
p iming expe imen (Table 1). The signi icance le el was a p≤.05. The analysis included only he eac ion imes o he
co ec answe s o 45 pa icipan s in each g oup. The au ho s elimina ed hose pa icipan s who did no ob ain a leas
70% co ec answe s in he expe imen al condi ions in any o he wo measu emen momen s.