Cognitive Technology to Evaluate the Academic Learning of Computational Cognition in Psychology Students

Resea ch A icle h ps://doi.o g/10.12973/ijem.10.2.1013 In e na ional Jou nal o Educa ional Me hodology Volume 10, Issue 2, 213 – 225. ISSN: 2469-9632 h p://www.ijem.com/ Cogni i e Technology o E alua e he Academic Lea ning o Compu a ional Cogni ion in Psychology S uden s Guadalupe Elizabe h Mo ales- Ma inez* Na ional Au onomous Uni e si y o Mexico, MEXICO Angel Ga cia-Collan es Dis ance Uni e si y o Mad id, SPAIN Ra ael Manuel Lopez-Pe ez Fundación Uni e si a ia Beha io & Law, SPAIN Recei ed: No embe 19, 2023 ▪ Re ised: Janua y 2, 2024 ▪ Accep ed: Ma ch 14, 2024 Abs ac : This s udy illus a ed an al e na i e way o e alua e s uden s’ academic lea ning. I in ol ed he join and in e wined applica ion o he na u al seman ic ne wo k echnique, compu e simula ions, and seman ic p iming expe imen s o assess he cogni i e changes in knowledge s uc u es due o academic lea ning in wo g oups o psychology s uden s. The expe imen al g oup was en olled in a cou se on compu a ional cogni ion, while he con ol g oup was obli ious o his cou se. The esul s indica ed ha he cogni i e assessmen ools disc imina e he cogni i e changes p oduced as a esul o gene al aining unde aken in a psychology deg ee e sus he in luence o a speci ic cou se. A e he cou se, he expe imen al g oup inc eased hei echnical ocabula y, changed hei concep ual alua ion o de ine s ela ed o compu a ional heo ies o mind, and eo ganized he ela ions among de ine s acco ding o he compu a ional cogni ion app oach. Also, his g oup p esen ed a highe connec i i y index be ween he concep s o he seman ic ne wo k, hei concep ual ac i a ion le el and concep ual co- ac i a ion pa e n changed, and hei access le el o he e alua ed schema’s concep s imp o ed. In con as , he con ol g oup did no show signi ican changes in hei cogni i e pa e ns a e he cou se. These indings sugges ha cogni i e ools may be help ul in he diagnosis o academic lea ning. Keywo ds: Academic lea ning, cogni i e assessmen , na u al seman ic ne wo ks, psychology s uden s, seman ic p iming. To ci e his a icle: Mo ales-Ma inez, G. E., Ga cia-Collan es, A., & Lopez-Pe ez, R. M. (2024). Cogni i e echnology o e alua e he academic lea ning o compu a ional cogni ion in psychology s uden s. In e na ional Jou nal o Educa ional Me hodology, 10(2), 213- 225. h ps://doi.o g/10.12973/ijem.10.1.1013 In oduc ion Lea ning assessmen is one o he mos exci ing challenges o he 21s cen u y educa ion, equi ing e olu ion in he ein o eaching models and echnology ad ancemen s and in ega d o unde s anding socie y's eme ging needs. In addi ion, de e mining aspec s o lea ning assessmen s and how hey should be e alua ed in educa ional se ings is a majo , e y di icul ask (Na ional Resea ch Council [NRC], 2001; Nichols & Sug ue, 1999). Mos academic es s measu e s uden s' sho - e m memo y skills as knowledge acquisi ion; o example, eache s adminis e mul iple- choice, ue- alse, o open-ques ion es s a he end o a cou se o e i y whe he he s uden has e ained lea ned knowledge. The co ec answe is p oo o he s uden 's acqui ed knowledge, bu such con en ional e alua ion does no de e mine i a s uden has de eloped a long- e m cogni i e abili y; many s uden s use s a egies o pass exams wi hou engaging in meaning ul long- e m lea ning (Ma zano, 1994; Ma zano & Cos a, 1988; Ma zano e al., 1990). A con empo a y e alua ion o academic lea ning equi es he use o e alua ion ools o measu e abili ies ela ed o cogni i e in o ma ion p ocessing, which a e cen al o aining 21s -cen u y s uden s, who li e in an economy la gely dependen on in o ma ion managemen (A ieli-A ali, 2013). Howe e , he e a e sca ce al e na i es emb acing his mode n ision o lea ning, echnology de elopmen s, and ad ancemen s in science lea ning. In line wi h his conce n, he NRC (2001) used he ad ancemen o cogni i e science o e hink assessmen app oaches, since ad ances in cogni i e science expanded he knowledge su ounding impo an lea ning dimensions. Fu he , i s ad ances in measu emen echniques open possibili ies o unde s and and in e p e inc easingly complex e idence ela ing o he academic pe o mance o s uden s. * Co esponding au ho : Guadalupe Elizabe h Mo ales-Ma inez, Na ional Au onomous Uni e si y o Mexico (UNAM), Mexico.  gemo ama @ho mail.com © 2024 The au ho (s); licensee IJEM by RAHPSODE LTD, UK. Open Access - This a icle is dis ibu ed unde he e ms and condi ions o he C ea i e Commons A ibu ion License (h ps://c ea i ecommons.o g/licenses/by/4.0/). 214  MORALES-MARTINEZ ET AL. / Cogni i e Technology o E alua e he Academic Lea ning In consonance wi h he NRC’s ision, Mo ales-Ma inez e al. (2023) highligh ed he use ulness o cogni i e assessmen ools o e alua e he p ocess and esul s o lea ning, since measu emen ad ances in cogni i e psychology a e compa ible wi h new echnologies. The in e wined use o cogni i e pa adigms o s udy he human mind wi h compu a ional de elopmen s o e s an oppo uni y o c ea e o inno a e digi al me hods o app oxima e he lea ning assessmen , as is illus a ed in he nex sec ion. Li e a u e Re iew The p oposal o linking cogni ion and lea ning heo ies wi h lea ning assessmen p ac ices and eaching is no a new concep (see NRC, 2001), A ieli-A ali (2013) no ed how since he p e ious cen u y, he e ha e been sophis ica ed app oaches o enchain cogni i e science wi h lea ning assessmen . Mo e ecen ly, Lopez-Rami ez e al. (2014) p oposed he Ch onome ic Cons uc i e Cogni i e Lea ning E alua ion Model, o C3 LEM, which wo ks unde he p inciples o se ial and pa allel human in o ma ion p ocessing and p o ides cogni i e ools ha explo e how he s uden 's mind wo ks when o ming knowledge s uc u es. Con en ionally, he C3 LEM sugges s using in e wined ch onome ic and men al ep esen a ion echniques o assess academic lea ning p ocesses (selec ion, s o age, and e ie al o he in o ma ion s o ed in he s uden s' memo y). The applica ion o C3-LEM in ol ed wo phases (Figu e 1): he cons uc i e cogni i e e alua ion and he ch onome ic cogni i e e alua ion o knowledge. The i s assesses he meaning o ma ion on knowledge men al ep esen a ion and e eals hei cogni i e cha ac e is ics ( he concep ual o ganiza ion, s uc u e, and dynamic) h ough men al ep esen a ion echniques (Na u al Seman ic Ne wo ks) and compu a ional simula ions. Figu e 1. Phases and Componen s o he C3-LEM (Mo ales-Ma inez, Angeles-Cas ellanos, e al., 2020) The Na u al Seman ic Ne wo ks (NSN) echnique (Figue oa e al., 1976) is a me hodological app oach o s udying men al ep esen a ion, which implies he eco e y o in o ma ion om human memo y by using concep ual clues ( a ge s). This echnique is use ul o explo ing he de elopmen o knowledge schemas acqui ed h oughou he academic yea . Acco ding o Mo ales Ma ínez, López Pé ez, e al. (2020), applying an ini ial NSN in a cou se and compa ing i o he inal NSN allows o obse a ion o he quali a i e (con en ) and quan i a i e cogni i e changes due o academic lea ning. Compa ison o bo h NSN, ini ial and inal, in ol es con as ing se e al indica o s such as he seman ic ichness (numbe o concep ual nodes), seman ic ele ance o he concep s (M alue), and he In e -Response Times ( empo a y pa e ns o de ine s appea ance), among o he s. The NSN echnique equi es a de ini ion ask; pa icipan s de ine co e a ge concep s o he academic cou se using de ine s (nouns, e bs, adjec i es, p onouns). Following his, hey a e hese de ine s based on hei ela ionship deg ee wi h he a ge (Mo ales-Ma inez, Angeles-Cas ellanos, e al., 2020). These sco es suppo he compu a ional simula ions o he knowledge schema's beha io a he beginning and end o he academic yea (Lopez-Rami ez e al., 2014). Lopez-Rami ez e al. (2015) used his echnique o explo e schema ic ac i i y in academic lea ning ac oss di e en knowledge domains. Fu he mo e, compu e simula ions can indica e connec ions among concep s only obse ed wi h hese ool ypes. Fo example, Gonzalez e al. (2013) obse ed ha high school s uden s es ablished implici ela ionships among concep s o an NSN on mo al cou se. They obse ed he concep o pa en co-ac i a ed police, e en hough hese concep s appea seman ically un ela ed. Howe e , when he esea che s conside ed he social and cul u al con ex o he pa icipan s, hey no ed a ela ionship o psychological signi icance. In gene al, he NSN analysis and he compu e simula ions allow us o obse e he changes in he men al ep esen a ion o s uden s' knowledge p oduced by lea ning a knowledge domain. Combining hese echniques wi h expe imen al cogni i e s udies inc eases he po en ial o e alua e o he ele an aspec s o academic knowledge as he empo al pa e ns o schema ic beha io . Fo example, he implemen a ion o ch onome ic cogni i e e alua ion s udies helps ob ain empo al ends in he capaci y o access in o ma ion om memo y. In e na ional Jou nal o Educa ional Me hodology  215 Mo ales-Ma ínez and San os-Alcan a a (2015), ollowing he p oposal o E.-O. López-Ramí ez (pe sonal communica ion, Augus 9 h, 2014), used he In e -Response Time (IRT) o NSN o analyze he in o ma ion accessibili y le el. These au ho s epo ed ha he de ine s wi h he g ea es weigh end o appea be ween 30 and 40 seconds, and hey appea ed be ween he hi d and i h posi ions on he lis . Howe e , he e was no discussion abou wha his posi ioning means. I is unknown i his is he case o all knowledge domains and which a iables in luence he eco e y o academic concep s. In addi ion o IRT, eac ion imes (RT) a e ano he indica o o he ch onome ic cogni i e e alua ion; hese come om schema ic wo d ecogni ion s udies applied be o e and a e he cou se. The RT can be classi ied h ough a neu al ne wo k o disc imina e whe he he e was an in eg a ion o he academic con en in o s uden s’ long- e m memo y s uc u es a he end o a cou se. To achie e his, he C3-LEM includes he applica ion o he seman ic p iming pa adigm h oughou lexical decision asks (McNama a, 2005), which consis s o p esen ing wo d pai s wi h di e en ela ionship ypes (e.g., associa i e, ca ego ical, schema ic, un ela ed). The expe imen al ask is o men ally ead he i s (p ime) and he las ( a ge ) wo d and decide whe he he a ge is spelled co ec ly. The ecogni ion imes o a ge s o e in o ma ion abou how he con ex p eceding schema ic wo ds a ec s s uden s' in o ma ion p ocessing. Suppose he p esen a ion o a s imulus (wo d o image) is p eceded by ano he seman ically ela ed s imulus. In ha case, s uden s will ecognize he second s imulus mo e quickly o accu a ely han in he case o no seman ic ela ionship be ween he wo s imuli. In a C3-LEM s udy, he seman ic ela ionships be ween he schema ic wo d pai s a e ele an . I he knowledge schema does no exis in he s uden 's memo y a he beginning o he cou se, and by he school yea end, s uden s ha e in eg a ed he in o ma ion in o hei knowledge s uc u es, he ecogni ion ime o schema ic pai s will signi ican ly educe by he cou se end. Lopez (1996) and Lopez and Theios (1992) p oposed ha he seman ic p iming e ec p oduced by a schema ic ela ionship is a concep e med “schema ic p iming.” In academic lea ning, schema ic p iming is p esen jus o hose wo d ecogni ion asks ha in ol e concep s ela ed o he knowledge schema o he e alua ed cou se. The e idence om cogni i e s udies shows ha when a s uden s o es he concep ual nodes lea ned in class in his long- e m memo y, he wo d ecogni ion imes ela ed o he lea ned schema dec ease a he cou se end (e.g., see Gonzalez e al., 2013; Mo ales Ma ínez, López Pé ez, e al., 2020). The opposi e happens when s uden s do no consolida e he in o ma ion in hei memo y (U diales-Iba a e al., 2018). In sho , his ype o knowledge o ganiza ion cogni i e phenomenon in long- e m memo y (schema ic p iming) can accoun o a lea ning p ocess on academic knowledge schemas. Techniques such as hose in ol ed in he C3-LEM can be aluable ools in he assessmen o , as, and o lea ning (see Mo ales-Ma inez, 2020; Mo ales-Ma inez & Lopez-Rami ez, 2016; Mo ales-Ma inez e al., 2015, 2017), hus, i is essen ial o accumula e e idence o he wo h o his cogni i e app oach o e alua e lea ning e ec i ely. The e o e, using his e alua ion model, his esea ch explo ed he changes in s uden s' knowledge s uc u es due o he lea ning conduc ed in a ace- o- ace psychology cou se. The i s ques ion in his esea ch was o de e mine he pa e n o cogni i e knowledge s uc u e changes ha s uden s expe ience a e lea ning a speci ic opic (Compu a ional Cogni ion). The second ques ion was o disc imina e he pa e n o cogni i e changes among s uden s en olled and no en olled in he cou se; as he esul s o his s udy poin ed ou , he e a e di e ences in cogni i e pa e ns ela ed o expe ise in his ield. The cogni i e measu emen ools accoun ed o cogni i e di e ences be ween he change di ec ed by speci ic lea ning and he spon aneously p oduced by non-sys ema ic exposu e o gene al psychology educa ion. Me hodology Resea ch Design The p esen au ho s used a quali a i e and quan i a i e mixed me hod (C3-LEM) o measu e he cogni i e dimension o academic lea ning in a Compu a ional Cogni ion cou se. Fi s , hey designed and applied an NSN s udy; hen, hey pe o med a compu a ional simula ion on he da a o his i s s udy using a neu al ne wo k o cons ain sa is ac ion p oposed by Lopez and Theios (1992). Finally, hey implemen ed expe imen al esea ch h ough he seman ic p iming pa adigm. Sample Two g oups o i s -yea psychology s uden s pa icipa ed in his s udy (74 women and 30 men). The i s a ended a compu a ional cogni ion cou se (expe imen al g oup), and he second was he con ol g oup, which did no ha e access o he in o ma ion on he subjec e alua ed. The con ol and he expe imen al g oups came om di e en ins i u ions. The pa icipan s had a mean age o 19 yea s ( ange 17 o 27 yea s, SD= 1.7). Pa icipa ion was olun a y, and all pa icipan s ga e in o med consen . 216  MORALES-MARTINEZ ET AL. / Cogni i e Technology o E alua e he Academic Lea ning Ins umen s The au ho s selec ed en s imuli om he heo y discussed in he cou se (mind, compu a ion, compu a ional mind, HIP, on Neumann, Tu ing machine, connec ionism, memo y, wo king memo y, and long- e m memo y) o design he NSN s udy. Concep selec ion ollowed Mo ales-Ma inez's P o ocol o he Collec ion o Concep s (Mo ales-Ma inez, 2015), which consis s o se e al concep ual analysis s eps on he subjec o e alua ion. This is a guide o selec ing a ge concep s o c ea e he NSN s udy as well as 30 de ine s o design he seman ic p iming expe imen . The au ho s o ganized he de ine s in p ime- a ge pai s (e.g., so wa e-p ocesses, algo i hm-language, and memo y-p ocesso ). Fu he mo e, hey added 15 associa i e concep pai s (e.g., bee-s ing, ai plane-pilo , den is - oo h) and 15 un ela ed wo d pai s (e.g., loo -sc een, moun ain-blood, wa -ele a o ). P ocedu e This esea ch in ol ed an announcemen , whe e he s uden s ecei ed an in i a ion o pa icipa e olun a ily, also hey lea ned abou he s udy objec i es and bene i s, and he au ho s ga e a p i acy wa ning abou hei da a. Following his, he s uden s who chose o pa icipa e ga e hei in o med consen and ecei ed speci ic ins uc ions o NSN and seman ic p iming s udy asks. Finally, he s uden s comple ed bo h asks, and he au ho s pe o med a compu e simula ion on he da a om he i s s udy. Na u al Seman ic Ne wo k S udy: The Men al Rep esen a ion o Knowledge The pa icipan s pe o med a concep ual de ini ion ask, which in ol ed de ining en a ge concep s ha emb aced he compu a ional cogni ion opic. The a ge s appea ed andomly, one by one, and emained in he compu e sc een's cen e o 60 seconds. The pa icipan de ined each a ge wi h e bs, nouns, adjec i es, and p onouns and la e quali ied indi idually he concep ual quali y o each de ine conce ning i s a ge . As Lopez (1996) and Lopez and Theios (1992) sugges ed, concep ual sco es a ied be ween one and en; he smalle he numbe , he lowe he quali y o he schema ic ela ionship be ween he de ine and i s a ge . Compu e Simula ion: The Schema ic Dynamics o Knowledge S uc u es Da a ob ained om he NSN s udy ed a cons ain -sa is ac ion neu al ne wo k using EVCOG so wa e. The compu a ional simula ion p ocedu e ollowed Lopez and Theios' (1992) o mula: WIJ = -1n{[p(X=0 & Y = 1) p(X=1 & Y = 0)]*[p(X=1 & Y = 1) p(X=0 & Y = 0)]-1} [1] X and Y ep esen pai s o concep s. Ob aining he associa ion g ade among X and Y equi es calcula ing he p(X = 1 & Y = 0) alue by de e mining he join p obabili y ha X appea s when Y does no appea in a SAM g oup. Fu he mo e, p(X = 0 & Y = 1) and p(X = 0 & Y = 0) a e calcula ed simila ly. Calcula ing he p(X = 1 & Y = 1) alue conside s a hie a chical modula ion o he M alue in each SAM g oup and hei in e connec i i y h ough he neu o-compu a ional ne wo k. These calcula ions se ed o con igu e a connec i i y ma ix (SASO Ma ix) (Seman ic Analyze o Schema a Beha io ) (Lopez & Theios, 1992) use ul o he seman ic analysis o schema ic beha io . Seman ic P iming S udy: The Men al Ch onome y o Academic Lea ning Finally, a seman ic p iming s udy de e mined he deg ee o lea ned in o ma ion consolida ion in he s uden 's memo y. The s udy p esen ed pai s o wo ds (p ime- a ge ) ha could ha e di e en ela ionship kinds (associa i e s. schema ic s. un ela ed). Figu e 2 illus a es he expe imen al sequence. Figu e 2. The Sequence o an Expe imen al Tes o he Seman ic P iming S udy The expe imen al sequence consis ed o a black do in he compu e sc een cen e p esen ed o 500 ms, a e he i s concep appea ed (p ime) o 250 ms. Finally, he las wo d ( a ge ) appea ed and emained on he sc een un il he pa icipan pe o med he expe imen al ask, which consis ed o eading each pai o wo ds (p ime- a ge ) silen ly and In e na ional Jou nal o Educa ional Me hodology  217 hen making a lexical judgmen abou he a ge . The pa icipan ca ego ized he las wo d as ‘wo d’ o ‘no wo d’. The s udy du a ion anged om 7 o 9 minu es, depending on he pa icipan 's pe o mance. Da a Analysis Fi s , he au ho s pe o med a mul idimensional scaling analysis on he i s s udy's da a o inspec he o ganiza ion o he pa icipan s' schema. A e , hey ca ied ou a compu a ional simula ion h ough a cons ain -sa is ying neu al ne wo k. This kind o ool acili a es he obse a ion o knowledge schema ac i a ion and co-ac i a ion beha io . Finally, a hi d analysis explo ed he ch onome ic pa e ns o in o ma ion p ocessing ela ed o he e alua ed cou se. The au ho s ob ained he IRT ( he ime each s uden used o eco e each de ine o he NSN om hei memo y). Following his, hey applied a mixed ANOVA on he RT ob ained in bo h he expe imen al and con ol g oups’ seman ic p iming s udy o de e mine he consolida ion le el o in o ma ion in he s uden ’s memo y a he beginning and he end o he cou se. The au ho s ca ied ou analysis o a iance wi h conside a ion ha he le el o measu emen o he dependen a iable was a io. The obse a ions had a condi ion o independence as he expe imen al condi ions and he pa icipan s we e assigned andomly o his s udy. Rega ding equino mali y in he da a, he QQ plo showed ha he da a dis ibu ion was no mal, while Le ene's es showed ha he a iances we e equal ac oss he expe imen al condi ions o bo h he con ol g oup and he expe imen al g oup. Resul s The au ho s o ganized he esul s in h ee dimensions: he i s desc ibes he con en and o ganiza ion o he knowledge schema on compu a ional cogni ion. The second is he obse a ion o he schema ic dynamics and connec i i y pa e n o he NSN. The las dimension examines he concep ual accessibili y and consolida ion o he in o ma ion in he lea ned schema. Schema's Con en and O ganiza ion Analysis A mul idimensional scaling analysis e ealed ha he expe imen al g oup p esen ed changes in he concep ual o ganiza ion and quali y o he de ine s ela ed o compu a ional cogni ion. While hey used concep s om a mixed psychology and echnology gene al schema in he cou se beginning, owa d he cou se's end, he s uden s o ganized hei concep s in wo concep ual axes unde a specialized schema ela ed o compu a ional cogni ion (Figu e 3). One axis in ol ed HIP (Human In o ma ion P ocessing) and PDP (Pa allel Dis ibu ed P ocessing) concep s. The second axis inco po a ed concep s ela ed o he body-mind duali y issue om a cogni i e pe spec i e, o he wise known as he ha dwa e-so wa e me apho . Figu e 3. De ine s’ Concep ual O ganiza ion a he Cou se Beginning and he End o he Expe imen al and Con ol G oup 218  MORALES-MARTINEZ ET AL. / Cogni i e Technology o E alua e he Academic Lea ning In con as , al hough he con ol g oup inco po a ed new concep s owa ds he cou se end, hey belonged o a gene al psychology schema, and hei o ganiza ion emained ela i ely simila a he cou se beginning and end. Connec i i y Pa e ns and Schema Dynamics Academic lea ning exp ession also includes he o ma ion o new connec ions, he loss o exis ing ones, o he change in connec ion weigh s. Figu e 4 shows he changes in he ini ial and inal connec ions be ween he a ge s e alua ed by he expe imen al and con ol g oups. No ice in he igu e ha he expe imen al g oup inc eased he numbe o connec ions on di e en a ge s. Fo example, a he beginning o he cou se, hey connec ed compu a ion only wi h h ee a ge s (compu a ional mind, Tu ing machine, and HIP); a e he cou se, pa icipan s o med six new connec ions ( on Neumann, long- e m memo y, wo king memo y, connec ionism, memo y, mind) wi h his same concep o nine inal connec ions. Fu he mo e, hey los concep ual nodes, disconnec ed long- e m memo y, and on Neumann, and o med a connec ion be ween he Tu ing machine and compu a ion. Also, pa icipan s inc eased o dec eased he connec i i y s eng h among di e en a ge s. Fo example, he compu a ional mind- on Neumann pai s eng hened hei ela ionship by inc easing he numbe o common de ine s om one o i e owa d he cou se end. The con ol g oup gained and los connec ions and changed connec ion s eng h among di e en a ge pai s. Howe e , he con ol g oup had e y li le change in concep s such as HIP compa ed o he expe imen al g oup's pe o mance. Figu e 4. Connec i i y G aph Ob ained Be o e and A e he Cou se by he Expe imen al and Con ol G oup No e. A colo node ep esen s each a ge , and he node's size is ela ed o he connec i i y deg ee. The de ine s' numbe connec ing each a ge pai is o e he linking line. The mo e de ine s connec each a ge pai , he da ke he line. On he o he hand, he au ho s analyzed wo aspec s o explo e he schema ic beha io : he le els o schema ac i a ion be o e and a e he cou se and he co-ac i a ion pa e n among de ine s. In bo h cases, he SASO ma ix helped g aph he schema beha io . The esul s indica ed ha only he expe imen al g oup's schema ac i i y was signi ican ly modi ied. By con as , in he con ol g oup, he ac i a ion emained simila be ween he ini ial and inal measu emen s (Figu e 5). The expe imen al g oup inc eased i s concep ual ac i a ion le el a he cou se end, as shown in Figu e 5, whe e g ea e ac i a ion is p esen ed wi h a ed and highe g aph posi ion. In e na ional Jou nal o Educa ional Me hodology  219 Figu e 5. Expe imen al and Con ol G oup Su ace Plo s The au ho s illus a ed he co-ac i a ion schema pa e ns by ac i a ing common concep s wi h he highes M alues in he expe imen al g oup (Figu e 6) and hese concep s expe ienced a meaning change a he cou se end. In his ega d, he b ain wen om a ision di ec ly associa ed wi h memo y o a cogni i e science iew (neu al ne wo ks). The compu e passed om a classical compu a ion schema owa d a human in o ma ion p ocessing ision. Howe e , hese concep s did no expe ience any change in he con ol g oup. 220  MORALES-MARTINEZ ET AL. / Cogni i e Technology o E alua e he Academic Lea ning Figu e 6. Pa e ns o Co-Ac i a ion o Two De ine s wi h he Two Highes M Values in Bo h G oups Men al Ch onome y and Consolida ion o he Schema A quali a i e analysis o he expe imen al g oup's IRT e ealed ha he ela ion among concep ual accessibili y o de ine s in hei M alues is nega i e (Figu e 7). The co ela ion inc eased signi ican ly o he expe imen al g oup a he end o he cou se ( = -.31) compa ed wi h he ini ial NSN ( = -.16). This means ha a e s uden s lea ned compu a ional cogni ion schema, hey accessed de ine s wi h he highes M alue in a sho e ime han when he concep s ob ained less seman ic ele ance. Simila ly, he con ol g oup exhibi ed a co ela ion change om he ini ial NSN ( = -.42) o he inal NSN ( = -.55). Figu e 7. Rela ionship Be ween he In e -Response Times and he M Values Ob ained o All he De ine s by he Expe imen al and Con ol G oups a he Cou se Beginning and End In e na ional Jou nal o Educa ional Me hodology  221 The expe imen al g oup changed in he pa e n o access ime and seman ic ele ance o de ine s (Figu e 7). Howe e , a s uden ’s es o dependen samples, pe o med on he access imes on he common de ine s ob ained be o e (M = 30, SD = 9.6) and a e he cou se (M = 27, SD = 3.1), indica ed ha he e was no a s a is ically signi ican di e ence. The sca ci y o common de ine s and he high a iabili y in he IRT om he cou se beginning likely in luenced his esul . On he o he hand, a s uden 's - es was also applied o he M alues ob ained in he common de ine s be o e (M = 62, SD = 23) and a e (M = 148, SD = 50) he cou se, and he di e ence was s a is ically signi ican ( (8) = -4.17, p = .004). On he o he hand, he con ol g oup showed g ea e changes in he M alues han in he common de ine s' IRT dis ibu ion. In his ega d, a s uden ’s - es ( (42) = .85) indica ed ha he e a e no signi ican di e ences be ween he IRT means ob ained o he commons de ine s a he cou se beginning (M = 29, SD = 10) and he end (M = 28, SD = 9.2). Howe e , he e was a signi ican di e ence ( (42) = -2.77, p = .008) be ween he M alue means o he ini ial (M = 65 SD = 37) and inal NSN (M = 56 SD = 24). Figu e 8 shows hese changes in he in e ac ion pa e n be ween IRT and M- alue, only o he common de ine s be ween he ini ial and inal NSN. Figu e 8. Rela ionship Be ween he In e -Response Times and he M Values Ob ained o he Common De ine s by he Expe imen al and Con ol G oups a he Beginning and End o he Cou se The inal ch onome ic analysis comp ised a mixed ANOVA o 2(G oup: Expe imen al s. Con ol) x2(Cou se ime: S a s. End) x3(Seman ic ela ionship: Associa i e s. Schema ic s. None) on eac ion imes ob ained in he seman ic p iming expe imen (Table 1). The signi icance le el was a p≤.05. The analysis included only he eac ion imes o he co ec answe s o 45 pa icipan s in each g oup. The au ho s elimina ed hose pa icipan s who did no ob ain a leas 70% co ec answe s in he expe imen al condi ions in any o he wo measu emen momen s.