Exploring the reliability and validity of multiple mini-interviews in admission to teacher education

This is a sel -a chi ed e sion o an o iginal a icle. This e sion may di e om he o iginal in pagina ion and ypog aphic de ails. Au ho (s): Ti le: Yea : Ve sion: Copy igh : Righ s: Righ s u l: Please ci e he o iginal e sion: CC BY 4.0 h ps://c ea i ecommons.o g/licenses/by/4.0/ Explo ing he eliabili y and alidi y o mul iple mini-in e iews in admission o eache educa ion © 2024 he Au ho s Published e sion Vilppu, Henna; Laakkonen, Ee o; Haa aja, Ee a; Tol anen, Asko; Me säpel o, Rii a-Leena Vilppu, H., Laakkonen, E., Haa aja, E., Tol anen, A., & Me säpel o, R.-L. (2024). Explo ing he eliabili y and alidi y o mul iple mini-in e iews in admission o eache educa ion. Eu opean Jou nal o Teache Educa ion, Ea ly online. h ps://doi.o g/10.1080/02619768.2024.2351068 2024 Full Te ms & Condi ions o access and use can be ound a h ps://www. and online.com/ac ion/jou nalIn o ma ion?jou nalCode=ce e20 Eu opean Jou nal o Teache Educa ion ISSN: (P in ) (Online) Jou nal homepage: www. and online.com/jou nals/ce e20 Explo ing he eliabili y and alidi y o mul iple mini-in e iews in admission o eache educa ion Henna Vilppu, Ee o Laakkonen, Ee a Haa aja, Asko Tol anen & Rii a-Leena Me säpel o To ci e his a icle: Henna Vilppu, Ee o Laakkonen, Ee a Haa aja, Asko Tol anen & Rii a- Leena Me säpel o (04 May 2024): Explo ing he eliabili y and alidi y o mul iple mini- in e iews in admission o eache educa ion, Eu opean Jou nal o Teache Educa ion, DOI: 10.1080/02619768.2024.2351068 To link o his a icle: h ps://doi.o g/10.1080/02619768.2024.2351068 © 2024 The Au ho (s). Published by In o ma UK Limi ed, ading as Taylo & F ancis G oup. Published online: 04 May 2024. Submi you a icle o his jou nal View ela ed a icles View C ossma k da a Explo ing he eliabili y and alidi y o mul iple mini-in e iews in admission o eache educa ion Henna Vilppu a , Ee o Laakkonen a , Ee a Haa aja b , Asko Tol anen c and Rii a- Leena Me säpel o d a Depa men o Teache Educa ion and Cen e o Resea ch on Lea ning and Ins uc ion (CERLI), Uni e si y o Tu ku, Tu ku, Finland; b Facul y o Educa ional Sciences, Uni e si y o Helsinki, Helsinki, Finland; c Me hodology Cen e o Human Sciences, Uni e si y o Jy äskylä, Jy äskylä, Finland; d Depa men o Teache Educa ion, Uni e si y o Jy äskylä, Jy äskylä, Finland ABSTRACT This s udy add essed he need o explo e he eliabili y and alidi y o he ecen ly enewed ap i ude es based on he mul iple mini- in e iew (MMI) o ma in Finnish eache educa ion p og ammes. U ilising a na ionwide da ase (N = 3306), we explo ed he eliabili y o he o ma using in aclass co ela ions and mul ile el modelling. Fu he mo e, we alida ed he in ended s uc u e o he MMI wi h con i ma o y ac o analysis and con as ed he sco es o he MMI wi h o he admission sco es, i.e. cogni i e measu es. The esul s showed mos ly small e ec s o clus e ing o he applican s acco d- ing o he di e en in e iewe s and ci cui s. The la ges a iance componen o he MMI o al sco e was he applican (73.3%), ollowed by measu emen e o (15.09%). The in ended s uc u e was alida ed, and he MMI seemed o measu e di e en a ibu es om o he admission measu es. Thus, he s udy demons a ed sa is ac o y eliabili y and alidi y o MMI in admission o eache educa ion, ye he s a ions need u he de elopmen . ARTICLE HISTORY Recei ed 12 June 2023 Accep ed 28 Ap il 2024 KEYWORDS Mul iple mini-in e iews; s uden admission; ini ial eache educa ion In oduc ion In Finland, many eache educa ion p og ammes ha e long main ained hei place as he mos compe i i e uni e si y p og ammes, wi h accep ance a es o less han 10%. This makes he admission p ocess a high-s akes si ua ion (Rees e al. 2016) and places special impo ance on he selec ion: how o selec he mos sui able candida es o eache educa ion om among housands o applican s (Lehane, Lysagh , and O’Lea y 2023). Recen ly, admission o eache educa ion p og ammes in Finland has been signi ican ly de eloped and uni ied among he eigh uni e si ies o e ing ini ial eache educa ion (ITE). Consequen ly, uni e si y-speci ic ap i ude es s and pe sonal o g oup in e iews ha e been eplaced by a common ap i ude es based on he mul iple mini-in e iew (MMI) o ma , by which he applican can apply o almos any o he eache educa ion p og ammes. CONTACT Henna Vilppu [email p o ec ed] Depa men o Teache Educa ion & Cen e o Resea ch on Lea ning and Ins uc ion (CERLI), Uni e si y Resea ch Fellow, Uni e si y o Tu ku, Tu ku, Finland EUROPEAN JOURNAL OF TEACHER EDUCATION h ps://doi.o g/10.1080/02619768.2024.2351068 © 2024 The Au ho (s). Published by In o ma UK Limi ed, ading as Taylo & F ancis G oup. This is an Open Access a icle dis ibu ed unde he e ms o he C ea i e Commons A ibu ion License (h p://c ea i ecommons.o g/ licenses/by/4.0/), which pe mi s un es ic ed use, dis ibu ion, and ep oduc ion in any medium, p o ided he o iginal wo k is p ope ly ci ed. The e ms on which his a icle has been published allow he pos ing o he Accep ed Manusc ip in a eposi o y by he au ho (s) o wi h hei consen . The basic idea o he MMI me hod is ha he applican goes h ough se e al s a ions wi h di e en in e iewe s, and mul iple a ibu es a e assessed using a p ede ined, s uc u ed sco ing ub ic (E a e al. 2004). These mul iple in e iews should educe he e ec o chance and in e iewe o si ua ional biases; his is seen as an impo an asse o MMI compa ed o adi ional pe sonal semis uc u ed o uns uc u ed in e iews. MMI has been widely used and s udied in he con ex o medical educa ion, bu ela i ely li le is known abou i s applica ion in admission o eache educa ion. Howe e , a couple o ecen s udies ha e shown p omising esul s conce ning i s elia- bili y and in e iewe and applican pe cep ions o he p ocedu e (Me säpel o, U iainen, e al. 2022), as well as i s abili y o measu e ac ual disposi ions in con as o wo k expe ience (Saling é and MacMa h 2021). Ne e heless, he u ili y o MMI in eache educa ion admissions calls o mo e esea ch. The pu pose o his s udy was o ill his gap and shed ligh on he eliabili y and alidi y o he MMI me hod as one pa o he admission p ocess wi h an ex ensi e sample, he en i e na ional popula ion o applican s in Finland. Speci ically, we aimed o s udy whe he MMI demons a es sa is ac o y elia- bili y, whe he i can measu e mul iple noncogni i e a ibu es, and whe he i assesses di e en a ibu es om he o he admission measu es. The MMI as a me hod in s uden selec ion Mul iple app oaches ha e been used in s uden selec ion. In he ela ed li e a u e, a dis inc ion is ypically made be ween measu ing he cogni i e and noncogni i e a i- bu es o applican s (e.g. Ba dach, Klassen, and Pe y 2022; Rei e e al. 2007). Cogni i e a ibu es usually e e o in ellec ual p owess, measu ed by high school g ade poin a e age (GPA), o pe o mance on a ce ain knowledge o cogni i e p ocessing es , while noncogni i e a ibu es a e o en used o encapsula e all he o he desi ed quali ies in an applican (E a e al. 2004). P e ious s udies on s uden admission o ITE do no show e y p omising esul s o selec ion solely based on cogni i e a ibu es (e.g. Ba dach and Klassen 2020; D’Agos ino and Powe s 2009) bu encou age he use o b oade app oaches ha include speci ic noncogni i e measu es, such as psychological eache cha ac e is ics (Ba dach, Klassen, and Pe y 2022). Howe e , iden i ying sui able ools o he e alua ion o noncogni i e a ibu es seems o be challenging (Klassen e al. 2020). In he domain o medical educa ion, he MMI o ma was de eloped o ackle p oblems ela ed o measu ing noncogni i e a ibu es on a single measu emen occasion, such as pe sonal o g oup in e iews (E a e al. 2004; Rei e e al. 2007). The MMI is a highly s uc u ed selec ion me hod in which applican s o a e h ough a se ies o s a ions whe e hey espond o p ede ined ques ions on a opic, p oblem, o case-based scena io. Fu he mo e, in e iewe s may assess candida es on mul iple a ibu es, bo h wi hin and ac oss in e iew s a ions (Oli e e al. 2014). S uc u e and design o he MMI The MMI u ilises mul iple s a ions wi h di e en in e iewe s who independen ly sco e each applican on a s anda d a ing scheme. Thus, i applies a mul iple independen sampling me hodology (Hanson e al. 2012). The e has been sligh a ia ion in he numbe and du a ion o he s a ions, which o en ep esen s a comp omise be ween 2H. VILPPU ET AL. easibili y (e.g. esou ces, pa icipan a igue) and eliabili y (Dodson e al. 2009). Typically, medical schools ha e adop ed MMIs wi h six o wel e s a ions, wi h du a ions o 5 o 15 minu es (Kno and Hissbach 2014). Conce ning eliabili y, a s a ion ime o 5 o 6 minu es has been p o en su icien (e.g. Dodson e al. 2009; Kno and Hissbach 2014). The s a ions a e designed o measu e, o example, he applican ’s abili y o sol e p oblems while communica ing hei ideas clea ly, as opposed o speci ic lea ned knowl- edge (E a e al. 2004). Pau e al. (2013) p o ided e idence ha he MMI can assess c i ical hinking abili y, communica ion skills, p o essionalism, and a i udes owa ds ce ain e hical and social dilemmas. Thus, he MMI aims o measu e mul iple noncogni i e a ibu es, which a e belie ed o be impo an o and p edic i e o u u e pe o mance in he p o ession (Oli e e al. 2014). Howe e , ew s udies ha e epo ed iden i ying he a ge a ibu es h ough li e a u e esea ch and s akeholde analysis (Kno and Hissbach 2014). The MMI s a ions ypically equi e applican s o espond o a p omp , which migh be a pic u e, an a icle, o a scena io (Saling é and McMa h 2021). S a ions may u ilise di e en ask o ma s, such as discussing a opic o dilemma wi h he in e iewe , answe ing s anda dised ques ions, collabo a ing wi h o he applican s, o answe ing p oblem-sol ing asks (Kno and Hissbach 2014). The applican ’s pe o mance on s a ions is ypically a ed on ancho ed Like scales. S a ion sco es may be measu ed ei he on a single scale o by summing o agg ega ing se e al subscales. The MMI is a esou ce-in ensi e me hod o assessmen as i equi es ex ensi e human esou ces and space, which may a ec , o example, he numbe o s a ions a anged (Uij dehaage, Doyle, and Pa ke 2011; Yamada e al. 2017). Se ing up an adequa e numbe o s a ions and ques ions wi hin s a ions, as well as p o iding a su icien amoun o aining o he examine s, can imp o e he eliabili y o he MMIs bu may be oo cos ly (Robe s e al. 2008). To dec ease he inancial cos s o he MMI, Rosen eld e al. (2008) sugges ed ec ui ing exis ing acul y o be examine s, using uni e si y p emises, and educing he du a ion o s a ions a he han hei numbe . Reliabili y and alidi y o MMIs In a high-s akes si ua ion, issues o eliabili y and alidi y equi e special a en ion. Gene ally, eliabili y e e s o he consis ency o he measu es used, while alidi y e e s o whe he he i ems used measu e wha hey a e in ended o measu e. Validi y may include a numbe o mo e speci ic componen s, such as con en alidi y (consis ency be ween he s a ion scena io and he a ge a ibu e o be measu ed) and cons uc alidi y (consis ency be ween he sco es and he a ge a ibu e; Hecke e al. 2009). In addi ion, he MMI li e a u e o en discusses p edic i e, disc iminan , and ace alidi y. P edic i e alidi y de e mines he ex en o which MMI iden i ies applican s who will display he desi ed a ibu es du ing aining and h oughou hei p o essional p ac ice (Came on e al. 2017). Disc iminan alidi y con as s MMI wi h o he selec ion measu es (B eil e al. 2020), and ace alidi y e e s o he accep abili y o he MMI by in e iewe s and applican s (Hecke e al. 2009). The use o he MMI o ma has yielded p omising esul s in s uden admissions o heal h p o essions. Compa ed o adi ional in e iews, MMIs o e imp o ed eliabili y and alidi y, which should be expec ed o a selec ion ins umen in a high-s akes selec ion EUROPEAN JOURNAL OF TEACHER EDUCATION 3 se ing (Pa e son e al. 2016). Fo example, he mul iple independen sampling me ho- dology is hough o con ibu e o i s eliabili y, as one assessmen does no in luence ano he (Hanson e al. 2012). Gene ally, he eliabili y o MMI has been epo ed o be accep able; howe e , he in e p e a ion o eliabili y in s udies has no always been clea ly explained (Pau e al. 2013). Fac o analysis is ecommended o iden i ying which s a ions assess he same a ibu es (Lemay e al. 2007), and gene alisabili y analysis is o en ca ied ou o compu e he o e all eliabili y o he MMI conside ing all sou ces o a iance (Pau e al. 2013), such as in e iewe , s a ion, o applican . In a ecen s udy pilo ing he MMI p ocedu e in Finnish eache educa ion, app oxima ely 63% o he a iance be ween he MMI sco es could be a ibu ed o he applican , while measu emen e o accoun ed o abou 21% o he a iance (Me säpel o, U iainen, e al. 2022). In mos o he s a ions, in e iewe and ci cui e ec s we e small (unde 10%), while he s a ion explained a ound 20% o he a iance, poin ing o a ying le els o di icul y be ween s a ions. The e has been disag eemen abou whe he MMIs can measu e only one a ibu e, one a ibu e pe s a ion, o one a ibu e pe social skill dimension, ega dless o s a ion (B eil e al. 2020). Fo example, measu es o di e en noncogni i e a ibu es assessed wi hin he same s a ion ha e been ound o be highly co ela ed, which migh limi he in e p e abili y o MMI sco es (Oli e e al. 2014). This has also led o he common p ac ice o epo ing o al sco es wi hin each s a ion ins ead o sepa a e a ibu e-based sco es. Thus, i emains unclea whe he MMI sco es a e able o cap u e one o mul iple a i- bu es. Oli e e al. (2014) epo ed e idence suppo ing he measu emen o mul iple a ibu es wi hin one s a ion bu highligh ed he need o clea ly de ine he aspec s o he noncogni i e a ibu es ha a e in ended o be assessed. Addi ionally, s udies by Lemay e al. (2007) and Hecke e al. (2009) p o ided simila e idence suppo ing he mul i- dimensionali y o MMI, ha is, he abili y o he o ma o assess mul iple noncogni i e a ibu es. In medical educa ion, consis en e idence has been epo ed conce ning he p edic- i e alidi y o MMIs. Acco ding o a sys ema ic e iew (Pa e son e al. 2016), pe o mance on MMIs was ela ed o subsequen pe o mance on bo h unde g adua e and pos g ad- ua e examina ions o clinical compe ence, as well as on o he examina ions. Kelly e al. (2014) p o ed he opposi e, showing ha MMI was no highly p edic i e o i s -yea success in he p og amme. Howe e , hey sugges ed ha p edic i e alidi y pa e ns may change as s uden s p og ess in hei s udies and gain mo e clinical expe ience. In he eache educa ion con ex , Saling é and MacMa h (2021) epo ed ew co ela ions be ween MMI and success measu es in he long p ac icum, sugges ing he need o mo e esea ch. Conce ning he disc iminan alidi y o MMI, he o ma has shown low o non-exis en associa ions wi h adi ional admission ools, such as cogni i e abili y, p e-en y academic quali ica ions (e.g. GPA), o in e iew (e.g. E a e al. 2004; Hecke e al. 2009; Pau e al. 2013; Rees e al. 2016). Howe e , sco es om MMIs wi h only in e iew s a ions we e s ongly associa ed wi h sco es om MMIs ha included di e en kinds o exe cises, demons a ing con e gence be ween di e en ask o ma s (Ga ni e al. 2012). Acco ding o B eil e al. (2020), mos p e ious s udies ha e ocused on he ela ionships be ween o e all MMI sco es and o he selec ion measu es, while i would be impo an o conside he co ela ion be ween indi idual s a ions and o he selec ion measu es i MMI is assumed o measu e se e al dis inc a ibu es. 4H. VILPPU ET AL. In he medical con ex , se e al s udies indica e good ace alidi y o MMI, cha ac e ising i as a mo e accep able me hod o admission assessmen han adi- ional in e iews by bo h in e iewe s and in e iewees (e.g. Do e e al. 2010; Kelly e al. 2014). In eache educa ion, MMI has also been pe cei ed posi i ely by bo h applican s and in e iewe s (Me säpel o, U iainen, e al. 2022). The special ad an- age o MMIs is ha applican s can imp o e hei pe o mance a he nex s a ion, ega dless o possible ailu es in p e ious s a ions (Pau e al. 2013; Rosen eld e al. 2008). Se e al s udies on he accep abili y o MMIs s a e ha MMI sco es a e no ela ed o applican s’ gende , disad an ages, p e ious school success, o access o paid coaching (Pau e al. 2013; Uij dehaage, Doyle, and Pa ke 2011). Howe e , some s udies ha e epo ed ha applican s o abo iginal o u al backg ound ecei e lowe sco es compa ed o o he applican s (Rees e al. 2016). Accep abili y can be inc eased by combining di e en ypes o ques ions wi hin he s a ions (Yamada e al. 2017). The cu en s udy: MMI as a pa o s uden admission o eache educa ion in Finland In Finland, s uden s a e selec ed o ITE ia a wo-phase selec ion p ocess. The i s phase includes he p eselec ion o applican s based on hei weigh ed ma icula ion exam sco es, which a e applied o a pe cen age (e.g. 60%) o applican s. They a e g an ed he igh o en e he second phase wi hou pa icipa ing in he i s phase sc eening. Applican s wi hou ma icula ion exams o high enough sco es may ake i s phase sc eening, which aims o measu e he applican s’ cogni i e compe encies, such as con- cep ual comp ehension. I is comp ised o a na ionally planned and coo dina ed w i en exam (VAKAVA) ha comp ises eading scien i ic ex s and answe ing mul iple-choice ques ions based on he ex s wi hin a ime limi . Based on he success in he i s phase, a leas wice as many applican s compa ed o he numbe o s udy places a e in i ed in he second phase. The inal sc eening o applican s akes place in he second phase, which ocuses on applican s’ noncogni i e a ibu es, such as social skills. The inal sc eening is based on a na ionally planned and coo dina ed MMI o ma in which applican s mo e h ough a sequence o s a ions wi h di e en in e iewe s and espond o s uc u ed asks (E a e al. 2004). The a ibu es measu ed in he MMI a e de i ed om he mul idimensional adap ed p ocess model (MAP) o eaching (Me säpel o, Poikkeus, e al. 2022), highligh ing applican s’ sui abili y o he eaching p o ession. In he inal selec ion decision, nei he he sco es om he p eselec ion no he i s phase sc eening a e conside ed; only he sco es o he MMI a e used o ank he s uden s. In bo h he i s and second phase sc eenings, he p ocedu es and sco ing a e uni o m ac oss uni e si ies, and bo h phases apply o all eache educa ion p og ammes in Finland. The applican s can simul aneously apply o se e al p og ammes wi hin he same o mul iple uni e si ies, bu hey mus ank hei choices and can only en e one p og amme in which hey sco e highe han i s cu -o . The cu en s udy add essed ques ions conce ning he eliabili y as well as cons uc and disc iminan alidi y o MMI as a pa o s uden admission o eache educa ion p og ammes in Finland. The MMI o ma was pilo ed in some eache educa ion uni s in EUROPEAN JOURNAL OF TEACHER EDUCATION 5 2019, a e which i was u he de eloped; since 2020, i has been u ilised na ionwide. Since he o ma has been ai ly ecen ly applied o he eache educa ion con ex bu is less s udied wi hin i , he s udy aimed o shed mo e ligh on he ma e by answe ing he ollowing esea ch ques ions: In he con ex o s uden admission o eache educa ion, (1) Does he MMI o ma demons a e eliabili y? ●How much o he a ia ion in MMI sco es in each s a ion is explained by he clus e ing o applican s o di e en in e iewe s and ci cui s? ●How much o he a ia ion in MMI sco es is a ibu able o he in e iewe , ci cui , s a ion, applican , and measu emen e o ? (2) Does he MMI o ma demons a e alidi y? ●Does he MMI measu e se e al noncogni i e a ibu es, as i is designed o do? ●To wha ex en does MMI assess a ibu es ha a e no assessed by o he admission a iables (ma icula ion exam sco es and VAKAVA sco es)? The i s esea ch ques ion add essed he eliabili y o he MMI in e ms o how much o he a iance be ween he sco es could be a ibu ed o he pa icipan s and how much o he o he ac o s. In a p e ious s udy, he la ges a iance componen in he MMI o al sco e was o he applican , bu i e lec ed only ma ginally sa is ac o y eliabili y (Me säpel o, U iainen, e al. 2022). A he ime o he ea lie s udy, he MMI p ocedu e was pilo ed o he i s ime in Finnish eache educa ion admissions, whe eas a he ime o he cu en s udy, he o ma had become es ablished. Thus, we expec ed sligh ly highe eliabili y in e ms o applican sco es. The second esea ch ques ion conce ning he alidi y o he MMI was di ided in o wo sub-ques ions. The i s aimed o examine whe he he da a s uc u e o he MMI sco es suppo s he p esence o mul iple noncogni i e a ibu es as independen es dimen- sions. Gi en he explici measu emen and supposed dis inc i eness o i e MMI dimen- sions, we expec ed a s onge model i o a i e- ac o solu ion han o a one- ac o solu ion, sugges ing ha only one a ibu e is measu ed; o o a ou - ac o solu ion, sugges ing ha each s a ion would measu e only one a ge a ibu e (ins ead o he wo a ibu es ha we e aimed a ; see Oli e e al. 2014). The second sub-ques ion pe ained o he ela ionship be ween he MMI and he o he admission measu es. Based on ea lie esea ch, we assumed ha he MMI measu es some hing o he han he cogni i e skills measu ed by he o he admission a iables, namely, ma icula ion exam sco es and VAKAVA sco es ( o GPA, see E a e al. 2004, 2012; Rei e e al. 2007). Thus, we expec ed only modes co ela ions be ween MMI s a ion sco es and o he admission measu es. Ma e ials and me hods Pa icipan s The pa icipan s o he s udy comp ised he en i e popula ion o applican s (N = 3309) who had applied o a leas one o he h ee la ges eache educa ion p og ammes in Finland (p ima y eache educa ion, ea ly childhood eache educa ion, o special 6H. VILPPU ET AL. needs eache educa ion) and had pa icipa ed in he second phase o s uden admission ( he ap i ude es o ganised in he MMI o ma in 2021). The da a we e d awn om he na ional egis e o s udy p og ammes leading o a deg ee, main- ained by he Finnish Na ional Agency o Educa ion. Complying wi h he Eu opean Gene al Da a P o ec ion Regula ion, he pa icipan s we e in o med abou he p oces- sing o hei pe sonal da a and gi en he oppo uni y o e use pa icipa ion. Th ee applican s eques ed non-pa icipa ion in he s udy, esul ing in a inal sample size o 3306. Measu es and p ocedu e The mul iple mini-in e iew The mul iple mini-in e iew (MMI) p ocedu e was designed based on a pilo s udy (Me säpel o, U iainen, e al. 2022), whe e i e a ibu es we e measu ed h ough i e i e- minu e s a ions. Howe e , due o esou ce-in ensi eness and a la ge numbe o appli- can s compa ed o many s udies in he medical ield (e.g. E a e al. 2004; Uij dehaage, Doyle, and Pa ke 2011), in he cu en s udy, he MMI ci cui was condensed in o ou s a ions, each las ing i e minu es, wi h a h ee-minu e u na ound be ween s a ions. While he numbe o s a ions was smalle han epo ed in se e al s udies (see, e.g. he e iews o Kno and Hissbach 2014; Pau e al. 2013), i is no excep ionally small, as e en h ee-s a ion MMIs ha e been used (e.g. Klassen and Kim 2021; Onyon, Wall, and Goodyea 2009). Each applican o a ed h ough he ou -s a ion ci cui and me wi h a di e en in e iewe a each s a ion. The MMIs we e o ganised in adi ional ace- o- ace mode (Kok e al. 2021) and synch onously a di e en uni e si ies in Finland, o alling 51 ci cui s and 204 in e iewe s. All he in e iewe s ecei ed one-day aining on he aims and implemen a ion o MMIs as well as on he adminis a ion and sco ing o hei s a ions. The s a ions we e designed by an admission commi ee o eigh senio s a mem- be s om di e en uni e si ies. S a ion de elopmen was based on he MAP ame- wo k (Me säpel o, Poikkeus, e al. 2022), which speci ies he key compe ence domains pe cei ed o be c i ical o he eaching p o ession. The model conside s high-quali y eaching lea ne -cen ed and cons uc i is , and emphasises eaching in e ac ions, s uden s’ ac i e ole in lea ning, and eache s’ emo ional and lea ning suppo o s uden s. S a ions we e designed acco ding o ce ain a ibu es ha we e conside ed c ucial in de eloping hese skills and ha indica ed applican s’ gene al sui abili y o he eaching p o ession. Each o he ou s a ions ocused on one a ge a ibu e (mo i a ion o pu sui o a eaching ca ee , skills in managing emo ions, in e cul u al compe ence, and social p oblem sol ing), which was assessed wi h a c i e ion- e e enced sco ing ub ic. Addi ionally, social skills we e assessed a each s a ion. The s a ions included sub asks ha comp ised di e en o ma s and con en ypes, such as e lec ion on ce ain opics. A each s a ion, applican s could ea n a maximum o 24 poin s on he measu ed a ibu e, and he same o al sco e applied o he a ibu e o social skills assessed a each s a ion (wi h a maximum o six poin s a each). Thus, he maximum MMI sco e was 120. The o al sco e o he applican and a ibu e- ela ed sco es we e u ilised in he analyses. Fo he i s s a ion, howe e , a o al sco e o 18 was applied, since he i s sub ask o he s a ion was no ela ed o he main a ge a ibu e o he s a ion (i.e. mo i a ion), bu o EUROPEAN JOURNAL OF TEACHER EDUCATION 7 eliably and consis en ly in he assessmen o mos a ibu es. Thus, he sco es gi en by one in e iewe did no esemble each o he mo e han he sco es o o he applican s by o he in e iewe s (Me säpel o, U iainen, e al. 2022). A S a ion 3, a ge ing applican s’ in e cul u al compe ence, he in aclass co ela ion was somewha highe (17%), hin ing ha in e iewe bias may ha e a ec ed he sco ing o some deg ee. The eliabili y o he MMI was u he s udied using mul ile el modelling. When he sou ces o a iance in he MMI o al sco e we e s udied, he e ec o he ci cui was minimal (1.1%) and ha o he in e iewe was ai ly small (<8%). Howe e , he s a ion explained abou one- ou h o he o e all a iance, highligh ing ha one applican ecei ed di e en sco es om di e en s a ions. This seems unde s andable, as he s a ions we e designed o measu e di e en noncogni i e a ibu es; hus, hei pe cei ed di icul y o he applican p obably a ied. The e o e, one could sco e high on social p oblem-sol ing bu low on in e cul u al compe ence, o ins ance. These esul s a e e y much in line wi h he p e ious s udy in he con ex o Finnish eache educa ion (Me säpel o, U iainen, e al. 2022). The applican s’ ue sco e a iance u ned ou o be 73.3%, which can be conside ed accep able (e.g. Do e e al. 2010) and shows imp o emen om he ea lie s udy when he MMI p ocedu e was pilo ed (63.3%; Me säpel o, U iainen, e al. 2022). Al hough he MMIs a e edesigned e e y yea and hus a e no di ec ly compa able, his can be seen as a posi i e de elopmen , indica ing ha he eliabili y o he p ocedu e has been s eng h- ened by expe ience. Wi h only ou 5-minu e s a ions and i e measu ed a ibu es aiming a cos -e ec i e implemen a ion, he esul s conce ning eliabili y a e encou aging. Howe e , he s a ions’ in e nal consis ency a ied ema kably (C onbach’s alphas 0.53– 0.77), and despi e he s uc u ed s a ions and c i e ion-based sco ing ub ic, app oxi- ma ely 15% o he a ia ion in he MMI o al sco e was s ill a ibu able o measu emen e o . In a-s a ion eliabili y is usually expec ed o be high (Pau e al. 2013), and some s udies epo esul s complian wi h his assump ion (e.g. Callwood e al. 2018; Do e e al. 2010; Lemay e al. 2007). Howe e , in a-s a ion (o domain) eliabili y is no o en s udied, and o he examples o a iabili y exis , such as in Dowell e al. (2012). The easons o inconsis ency migh be ela ed o di e en ask ypes applied in he s a ions, as well as di e en numbe o sub asks wi hin s a ions. These indings emphasise ha mo e a en- ion should be paid o de eloping he s a ions and hei sub asks. Ou esul s suppo he assump ion ha he MMI is able o assess and di e en ia e among mul iple noncogni i e a ibu es (Hecke e al. 2009; Lemay e al. 2007; Oli e e al. 2014). The co ela ion analyses showed a ela i ely low co ela ion be ween he a ge dimensions, sugges ing ha he s a ions measu e di e en a ibu es. Howe e , he co ela ions be ween social skills dimensions, measu ed a e e y s a ion, and each s a ion a ibu e we e he highes , sugges ing ha he dimension o social skills was he leas dis inguishable a ibu e o he i e measu ed a ibu es. Howe e , CFAs suppo ed he i e- ac o s uc u e o e he one- ac o model, sugges ing ha MMI would measu e only one a ge a ibu e as well as o e he ou - ac o model, whe e he dimension o social skills was embedded in he o he ou a ge a ibu es and no as a sepa a e dimension. When he MMI sco es we e con as ed wi h he cogni i e measu es o ma icula ion examina ion sco es and VAKAVA w i en en ance examina ion sco es, only weak o non- exis en associa ions be ween hem we e ound. This p o ides e idence ha he MMI measu es some hing o he han he cogni i e measu es o he applica ion phase (e.g. E a 14 H. VILPPU ET AL. e al. 2004, 2012; Rei e e al. 2007). Thus, he esul s suppo he ole o MMI in he admission p ocess as a supplemen a y measu e o cogni i e sc eening. The s eng h o he s udy lies in he sophis ica ed s a is ical modelling based on ep e- sen a i e da a om he en i e yea ly popula ion o applican s o he la ges eache educa- ion p og ammes in Finland. Howe e , one needs o conside he one-o na u e o he MMI: since he s a ions a e edesigned e e y yea , hei compa abili y o ea lie and u u e s udies is limi ed. Howe e , he measu ed a ibu es show a ce ain consis ency om yea o yea (c . Me säpel o, U iainen, e al. 2022), as hey a e based on he MAP model (Me säpel o, Poikkeus, e al. 2022). Fu he , al hough he s udy explo ed he eliabili y and alidi y o he MMI in e sa ile ways, i was in no accoun comp ehensi e. Fu u e s udies should he e o e be conduc ed, o example, o assess i s p edic i e alidi y, ha is, whe he he MMI sco es a e associa ed wi h s udy achie emen in eache educa ion as well as success and pe sis ence in he eache p o ession, o con e gence alidi y, ha is, whe he o he es s measu ing simila a ibu es gi e simila esul s. Addi ionally, he s udy did no conside he ai ness o he MMI o all applican s, ha is, whe he applican s’ age, gende , socio- economic s a us o e hnic backg ound had an e ec on hei pe o mance (c . Rees e al. 2016). Conclusion In conclusion, ou s udy p o ides encou aging suppo o he eliabili y and alidi y o using he MMI o ma as a pa o s uden admission o eache educa ion. Howe e , mo e esea ch and de elopmen wo k a e needed o imp o e he in e nal consis ency o he s a ions. Conside ing he esou ce in ensi y o he adi ional ace- o- ace MMI, o he solu ions could also be conside ed, such as an au oma ed in e iew g ounded in MMI me hodology pilo ed in heal hca e s uden selec ions (Callwood e al. 2022). Acknowledgmen s The au ho s wish o hank he Resea ch Council o Finland [#342191] o he inancial suppo o conduc ing his s udy. Fu he mo e, he au ho s wish o hank all he pa icipan s in he s udy. Disclosu e s a emen No po en ial con lic o in e es was epo ed by he au ho (s). Funding This wo k was suppo ed by he Resea ch Council o Finland unde G an [#342191]. Howe e , he unding agency had no in ol emen in s udy design, in he collec ion, analysis, and in e p e a ion o da a, no w i ing he epo o he decision o submi he a icle o publica ion. No es on con ibu o s Henna Vilppu PhD (educa ion), wo ks as a Uni e si y Resea ch Fellow a he Depa men o Teache Educa ion, Uni e si y o Tu ku, Finland. He esea ch in e es s conce n eaching and lea ning a EUROPEAN JOURNAL OF TEACHER EDUCATION 15 di e en highe educa ion con ex s, as well as eache educa ion and eache s’ p o essional de elopmen . Ee o Laakkonen Lic. Econ. (S a is ics), wo ks as a senio esea che a he Depa men o Teache Educa ion, Uni e si y o Tu ku, Finland. His esea ch in e es s conce n he applica ion o quan i a- i e esea ch me hods in beha iou al sciences, e.g., s a is ical mul i a ia e me hods, s uc u al equa ions models, s a is ical compu ing. Ee a Haa aja PhD (educa ion), wo ks as a uni e si y lec u e a he Facul y o Educa ional Sciences, Uni e si y o Helsinki, Finland. He esea ch in e es s conce n eache s’ p o essional ision and eache -s uden in e ac ion in ma hema ics educa ion, as well as eache educa ion and eache s’ p o essional de elopmen . Asko Tol anen PhD (S a is ics), wo ks as p o esso in Me hodology Cen e o Human Sciences, a he Uni e si y o Jy äskylä, Finland. His esea ch in e es conce n applying s uc u al equa ion modelling in he con ex o mul ile el and mix u e modelling. Rii a-Leena Me säpel o PhD (Psychology) and Adjunc P o esso , wo ks as a senio esea che a he Depa men o Teache Educa ion, Uni e si y o Jy äskylä, Finland. He esea ch in e es s conce n s uden selec ion o eache educa ion, eache compe ences and eache s’ p o essional de elopmen . Re e ences Ba dach, L., and R. Klassen. 2020. “Sma Teache s, Success ul S uden s? A Sys ema ic Re iew o he Li e a u e on Teache s’ Cogni i e Abili ies and Teache E ec i eness.” Educa ional Resea ch Re iew 30:100312. h ps://doi.o g/10.1016/j.edu e .2020.100312 . Ba dach, L., R. Klassen, and N. Pe y. 2022. “Teache s’ Psychological Cha ac e is ics: Do They Ma e o Teache E ec i eness, Teache s’ Well-Being, Re en ion, and In e pe sonal Rela ions? An In eg a i e Re iew.” Educa ional Psychology Re iew 34 (1): 259–300. h ps://doi.o g/10.1007/ s10648-021-09614-9 . B eil, S., B. Fo hmann, A. He el-Waszak, H. Ah ens, B. B ouwe , E. Schöne eld, B. Ma schall, and M. Back. 2020. “Cons uc Validi y o Mul iple Mini In e iews—In es iga ing he Role o S a ions, Skills, and Ra e s Using Bayesian G-Theo y.” Medical Teache 42 (2): 164–171. h ps://doi.o g/10. 1080/0142159X.2019.1670337 . B ennan, R. 2010. “Gene alizabili y Theo y and Classical Tes Theo y.” Applied Measu emen in Educa ion 24 (1): 1–21. h ps://doi.o g/10.1080/08957347.2011.532417 . Callwood, A., D. Cooke, S. Bolge , A. Lemanska, and H. Allan. 2018. “The Reliabili y and Validi y o Mul iple Mini In e iews (MMIs) in Values Based Rec ui men o Nu sing, Midwi e y and Pa amedic P ac ice P og ammes: Findings om an E alua ion S udy.” In e na ional Jou nal o Nu sing S udies 77:138–144. h ps://doi.o g/10.1016/j.ijnu s u.2017.10.003 . Callwood, A., L. Gillam, A. Ch is idis, J. Doul on, J. Ha is, M. Piano, A. Kubacki, e al. 2022. “Feasibili y o an Au oma ed In e iew G ounded in Mul iple Mini In e iew (MMI) Me hodology o Selec ion in o he Heal h P o essions: An In e na ional Mul ime hod E alua ion.” BMJ Open 12 (2): e050394. h ps://doi.o g/10.1136/bmjopen-2021-050394 . Came on, A., L. MacKeigan, N. Mi sakakis, and J. Pugsley. 2017. “Mul iple Mini-In e iew P edic i e Validi y o Pe o mance on a Pha macy Licensing Examina ion.” Medical Educa ion 51 (4): 379–389. h ps://doi.o g/10.1111/medu.13222 . D’Agos ino, J., and S. Powe s. 2009. “P edic ing Teache Pe o mance wi h Tes Sco es and G ade Poin A e age: A Me a-Analysis.” Ame ican Educa ional Resea ch Jou nal 46 (1): 146–182. h ps:// doi.o g/10.3102/0002831208323280 . Dodson, M., B. C o y, D. P ideaux, R. Ca ne, A. Wa d, and E. de Leeuw. 2009. “The Mul iple Mini-In e iew: How Long Is Long Enough?” Medical Educa ion 43 (2): 168–174. h ps://doi.o g/ 10.1111/j.1365-2923.2008.03260.x . 16 H. VILPPU ET AL. Do e, K., S. K euge , M. Ladhani, D. Rol son, D. Ku z, K. Kulasega am, A. Cullimo e, e al. 2010. “The Reliabili y and Accep abili y o he Mul iple Mini-In e iew As a Selec ion Ins umen o Pos g adua e Admissions.” Academic Medicine 85 (10): 60–63. h ps://doi.o g/10.1097/ACM. 0b013e3181ed442b . Dowell, J., B. Lynch, H. Till, B. Kumwenda, and A. Husbands. 2012. “The Mul iple Mini-In e iew in he UK Con ex : 3 Yea s o Expe ience a Dundee.” Medical Teache 34 (4): 297–304. h ps://doi.o g/10. 3109/0142159X.2012.652706 . E a, K., H. Rei e , J. Rosen eld, K. T inh, T. Wood, and G. No man. 2012. “Associa ion Be ween a Medical School Admission P ocess Using he Mul iple Mini-In e iew and Na ional Licensing Examina ion Sco es.” Jou nal o he Ame ican Medical Associa ion 308 (21): 2233–2240. h ps://doi. o g/10.1001/jama.2012.36914 . E a, K., J. Rosen eld, H. Rei e , and G. No man. 2004. “An Admissions OSCE: The Mul iple Mini In e iew.” Medical Educa ion 38 (3): 314–326. h ps://doi.o g/10.1046/j.1365-2923.2004.01776.x . Ga ni, N., A. Moshinsky, O. Eisenbe g, D. Zeigle , and A. Zi . 2012. “Reliabili y Es ima es: Beha iou al S a ions and Ques ionnai es in Medical School Admissions.” Medical Educa ion 46 (3): 277–288. h ps://doi.o g/10.1111/j.1365-2923.2011.04155.x . Haa aja, E., A. Tol anen, H. Vilppu, M. Kallio, J. Pel onen, and R.-L. Me säpel o. 2023. “Measu ing Highe -O de Cogni i e Skills wi h Mul iple-Choice Ques ions: Po en ial and Pi alls o Finnish Teache Educa ion En ance.” Teaching and Teache Educa ion 122:103943. h ps://doi.o g/10. 1016/j. a e.2022.103943 . Hanson, M., K. Kulasega am, D. Coombs, and J. He old. 2012. “Admissions File Re iew: Applying he Mul iple Independen Sampling (MIS) Me hodology.” Academic Medicine 87 (10): 1335–1340. h ps://doi.o g/10.1097/ACM.0b013e3182674629 . Hecke , K., T. Donnon, C. Fuen ealba, D. Hall, O. Illanes, D. Mo ck, and C. Muelling. 2009. “Assessmen o Applican s o he Ve e ina y Cu iculum Using a Mul iple Mini-In e iew Me hod.” Jou nal o Ve e ina y Medical Educa ion 36 (2): 166–173. h ps://doi.o g/10.3138/j me.36.2.166 . Kelly, M., J. Dowell, A. Husbands, J. Newell, S. O’Flynn, T. K opmans, F. Dunne, and A. Mu phy. 2014. “The Fai ness, P edic i e Validi y and Accep abili y o Mul iple Mini In e iew in an In e na ionally Di e se S uden Popula ion—A Mixed Me hods S udy.” BMC Medical Educa ion 14 (1): 267. h ps://doi.o g/10.1186/s12909-014-0267-0 . Killip, S., Z. Mah oud, and K. Pea ce. 2004. “Wha Is an In aclus e Co ela ion Coe icien ? C ucial Concep s o P ima y Ca e Resea che s.” The Annals o Family Medicine 2 (3): 204–208. h ps://doi. o g/10.1370/a m.141 . Klassen, R., and L. Kim. 2021. Teache Selec ion: E idence-Based P ac ices. Wiesbaden: Sp inge . Klassen, R., L. Kim, J. Rushby, and L. Ba dach. 2020. “Can We Imp o e How We Sc een Applican s o Ini ial Teache Educa ion?” Teaching and Teache Educa ion 87:102949. h ps://doi.o g/10.1016/j. a e.2019. 102949 . Kno , M., and J. Hissbach. 2014. “Mul iple Mini-In e iews: Same Concep , Di e en App oaches.” Medical Educa ion 48 (12): 1157–1175. h ps://doi.o g/10.1111/medu.12535 . Kok, K., L. Chen, F. I daya i Id is, N. H. Mumin, H. Ghani, I. Nazu ah Zulkipli, and M. Ann Lim. 2021. “Conduc ing Mul iple Mini-In e iews in he Mids o COVID-19 Pandemic.” Medical Educa ion Online 26 (1): 1. h ps://doi.o g/10.1080/10872981.2021.1891610 . Koo, T., and M. Li. 2016. “A Guideline o Selec ing and Repo ing In aclass Co ela ion Coe icien s o Reliabili y Resea ch.” Jou nal o Chi op ac ic Medicine 15 (2): 155–163. h ps://doi.o g/10.1016/j.jcm. 2016.02.012 . Lehane, P., Z. Lysagh , and M. O’Lea y. 2023. “A Validi y Pe spec i e on In e iews as a Selec ion Mechanism o En y o Ini ial Teache Educa ion P og ammes.” Eu opean Jou nal o Teache Educa ion 46 (2): 293–307. h ps://doi.o g/10.1080/02619768.2021.1920920 . Lemay, J.-F., J. Lockye , T. Collin, and K. B ownell. 2007. “Assessmen o Non-Cogni i e T ai s Th ough he Admissions Mul iple Mini-In e iew.” Medical Educa ion 41 (6): 573–579. h ps://doi.o g/10. 1111/j.1365-2923.2007.02767.x . Ma sh, H., A. Ma in, and J. Cheng. 2008. “A Mul ile el Pe spec i e on Gende in Class oom Mo i a ion and Clima e: Po en ial Bene i s o Male Teache s o Boys?” Jou nal o Educa ional Psychology 100 (1): 78–95. h ps://doi.o g/10.1037/0022-0663.100.1.78 . EUROPEAN JOURNAL OF TEACHER EDUCATION 17 Me säpel o, R.-L., A.-M. Poikkeus, M. Heikkilä, J. Husu, A. Laine, K. Lappalainen, M. Läh eenmäki, e al. 2022. “A Mul idimensional Adap ed P ocess Model o Teaching.” Educa ional Assessmen , E alua ion and Accoun abili y 34 (2): 143–172. h ps://doi.o g/10.1007/s11092-021-09373-9 . Me säpel o, R.-L., J. U iainen, A.-M. Poikkeus, J. Muo ka, A. Tol anen, and A. Wa inowski. 2022. “Mul iple Mini In e iew As a Selec ion Tool o Ini ial Teache Educa ion Admissions.” Teaching and Teache Educa ion 113:103660. h ps://doi.o g/10.1016/j. a e.2022.103660 . Mu hén, L., and B. Mu hén. 1998–2017. Mplus Use ’s Guide. 8 h ed. Los Angeles, CA: Mu hén & Mu hén. Oli e , T., K. Hecke , P. Hausdo , and P. Conlon. 2014. “Valida ing MMI Sco es: A e We Measu ing Mul iple A ibu es?” Ad ances in Heal h Sciences Educa ion 19 (3): 379–392. h ps://doi.o g/10. 1007/s10459-013-9480-6 . Onyon, C., D. Wall, and H. Goodyea . 2009. “Reliabili y o Mul i-S a ion In e iews in Selec ion o Junio Doc o s o Special y T aining.” Medical Teache 31 (7): 665–667. h ps://doi.o g/10.1080/ 01421590802578236 . Pa e son, F., A. Knigh , J. Dowell, S. Nicholson, F. Cousans, and J. Cleland. 2016. “How E ec i e A e Selec ion Me hods in Medical Educa ion? A Sys ema ic Re iew.” Medical Educa ion 50 (1): 36–60. h ps://doi.o g/10.1111/medu.12817 . Pau, A., K. Jee a a nam, Y. Chen, A. Fall, C. Khoo, and V. Nada ajah. 2013. “The Mul iple Mini-In e iew (MMI) o S uden Selec ion in Heal h P o essions T aining: A Sys ema ic Re iew.” Medical Teache 35 (12): 1027–1041. h ps://doi.o g/10.3109/0142159X.2013.829912 . Rees, E., A. Hawa den, G. Den , R. Hays, J. Ba es, and A. Hassell. 2016. “E idence Rega ding he U ili y o Mul iple Mini-In e iew (MMI) o Selec ion o Unde g adua e Heal h P og ams: A BME Sys ema ic Re iew.” Medical Teache 38 (5): 443–445. h ps://doi.o g/10.3109/0142159X.2016.1158799 . Rei e , H., K. E a, J. Rosen eld, and G. No man. 2007. “Mul iple Mini-In e iews P edic Cle kship and Licensing Examina ion Pe o mance.” Medical Educa ion 41 (4): 378–384. h ps://doi.o g/10.1111/ j.1365-2929.2007.02709.x . Robe s, C., M. Wal on, I. Ro hnie, J. C ossley, P. Lyon, K. Kuma , and D. Tille . 2008. “Fac o s A ec ing he U ili y o he Mul iple Mini-In e iew in Selec ing Candida es o G adua e-En y Medical School.” Medical Educa ion 42 (4): 396–404. h ps://doi.o g/10.1111/j.1365-2923.2008.03018.x . Rosen eld, J., H. Rei e , K. T inh, and K. E a. 2008. “A Cos E iciency Compa ison Be ween he Mul iple Mini-In e iew and T adi ional Admissions In e iews.” Ad ances in Heal h Sciences Educa ion 13 (1): 43–58. h ps://doi.o g/10.1007/s10459-006-9029-z . Saling é, B., and S. MacMa h. 2021. “Using Mul iple-Mini-In e iews o Admission in o Teache Educa ion.” Canadian Jou nal o Educa ion 44 (1): 150–173. h ps://doi.o g/10.53967/cje- ce. 44i1.4401 . Sch eibe , J., A. No a, F. S age, E. Ba low, and J. King. 2006. “Repo ing S uc u al Equa ion Modeling and Con i ma o y Fac o Analysis Resul s: A Re iew.” The Jou nal o Educa ional Resea ch 99 (6): 323–338. h ps://doi.o g/10.3200/JOER.99.6.323-338 . Uij dehaage, S., L. Doyle, and N. Pa ke . 2011. “Enhancing he Reliabili y o he Mul iple Mini-In e iew o Selec ing P ospec i e Heal h Ca e Leade s.” Academic Medicine 86 (8): 1032–1039. h ps://doi.o g/10.1097/ACM.0b013e3182223ab7 . Yamada, T., J. Sa o, H. Yoshimu a, T. Okubo, E. Hi aoka, T. Shiga, T. Kubo a, e al. 2017. “Reliabili y and Accep abili y o Six S a ion Mul iple Mini-In e iews: Pas -Beha iou al e sus Si ua ional Ques ions in Pos g adua e Medical Admission.” BMC Medical Educa ion 17 (1): 57. h ps://doi. o g/10.1186/s12909-017-0898-z. 18 H. VILPPU ET AL.