Mechatronic system usability evaluation

Jou nal o Technology and Science Educa ion JOTSE, 2023 – 13(1): 65-72 – Online ISSN: 2013-6374 – P in ISSN: 2014-5349 h ps://doi.o g/10.3926/jo se.1349 MECHATRONIC SYSTEM USABILITY EVALUATION Apos olos Tsaga is In e na ional Hellenic Uni e si y (G eece) sag[email p o ec ed] Recei ed June 2021 Accep ed July 2022 Abs ac This s udy aimed o e ine and alida e a Mecha onic Sys em Usabili y E alua ion (MSUE) ques ionnai e. A o al o 626 use s we e selec ed using andom sampling, om he a ea o Wes Thessaloniki, G eece. The alidi y o he ques ionnai e we e es ed wi h he con en and cons uc ion alidi y me hod. The eliabili y o he MSUE ques ionnai e ins umen we e es ed using es - e es and in e nal consis ency me hod. Fac o analysis esul ed a 25 ques ions ques ionnai e di ided in o i e axes o , namely e iciency, e ec i eness, Sa is ac ion, ease o use, and ease o lea n. The C onbach Alpha coe icien o he en i e scale was 0.819. The ques ionnai e was es ed and he esul s was shown a sui able ins umen o measu e usabili y on mecha onic sys ems. Keywo ds – Usabili y, Mecha onic sys ems, E iciency, Sa is ac ion, E ec i eness, Easy o use. To ci e his a icle: Tsaga is, A. (2023). Mecha onic sys em usabili y e alua ion. Jou nal o Technology and Science Educa ion, 13(1), 65-72. h ps://doi.o g/10.3926/jo se.1349 ---------- 1. In oduc ion The pu pose o he p esen esea ch is o c ea e and alida e a weigh ed ques ionnai e ha measu es he usabili y o he in e ac ion wi h mecha onic sys ems. Mecha onics is he in eg a ion o mechanical, elec ical, and compu e echnologies in o he design o complex p oduc s (Figu e 1). I is a combina ion o p ecision mechanical enginee ing, elec onic, con ol enginee ing and compu e s o he in elligen con ol o machines. Is he p ocess whe e an in eg a ed de elopmen o he enginee ing design is pu chased: The o m o mode n au oma ion sys ems. Mecha onics sys em is any elec omechanical sys em ha combines elemen s om elec onic de ices and he con ol is done wi h he help o so wa e. I usually includes a mic ocon olle , senso s, ac ua o s and he necessa y so wa e o con ol hem. Many imes i has a specially designed in e ace o con ol ei he h ough a PC o h ough special consoles. The mic ocon olle unde akes he p ocessing o he signals collec ed by he a ious senso s. The collec ion and he acquisi ion o he signals h ough he senso s is a key pa o he sys em. They a e hen p ocessed o gi e he app op ia e in o ma ion o he sys em by pe o ming a ious con ol ac ions. -65- Jou nal o Technology and Science Educa ion – h ps://doi.o g/10.3926/jo se.1349 Figu e 1. Mecha onic diag am A mecha onic sys em is use ul when i has a high deg ee o usabili y. When he in e ace is no easy o use, he use does no ocus on he con en o he in e ac ion, bu on he way o in e ac wi h he sys em and ha is no good o he esul o he p ocess (Luo, Liu, Kuo & Yuan, 2014). Sea ching in he li e a u e one will ind nume ous de ini ions o usabili y. As is men ioned in a ious esea ch and in in e na ional s anda ds, usabili y e e s o e iciency and e ec i eness in achie ing speci ic goals and use sa is ac ion (Cha ziky kou, 2020). No e ha he de ini ions o usabili y mos o en e e o Human Compu e In e ac ion o Human Robo ic In e ac ion o Human Machine In e ac ion. I is he i s ime speaking o Human Mecha onic Sys em In e ac ion. This can be done wi h a combina ion o he abo e echniques, because mecha onic sys ems a e a combina ion o he abo e a eas. Acco ding o a ious de ini ions: “Usabili y is he deg ee o which a p oduc o sys em can be used by speci ic use s o achie e speci ic goals wi h e iciency, e ec i eness and sa is ac ion in a pa icula con ex o use” (ISO 9241) The main dimensions o usabili y examined in he li e a u e o in e ac ion wi h compu e s, machines o obo ics include ease o lea ning (Nielsen, 1990; Guilleme e, 1995), ease o use (Nielsen, 1990; Guilleme e, 1995), u ili y (Guilleme e, 1995), use sa is ac ion (Nielsen, 1993), e iciency (Pa sazadeh e al., 2018) and e iciency (Pa sazadeh, Ali, Meh an & Teh ani, 2018). Acco ding o ISO 9241 (E gonomics o Human-Sys em In e ac ion), usabili y is de ined as “ he ex en o which a p oduc o sys em can be used by designa ed use s o achie e speci ic goals wi h e iciency, e ec i eness and sa is ac ion, in a speci ic con ex o use” (ISO 9241-210). The indi idual c i e ia men ioned in i , a e analyzed in he ease o lea ning, in he high e iciency o execu ion, in he low equency o use e o s, in he ease o e aining he knowledge o i s use and in he subjec i e use sa is ac ion (A ou is, Ka sanos, Tselios & Mous akas, 2015). Dix, Finlay, Abowd and Beale (2004) in hei book “Human-Compu e In e ac ion” (3 d edi ion) conside usabili y o be a complex concep and analyze i in ease o lea ning, lexibili y, and obus ness (Kou sambasis, 2015). Shneide man and Pleasan (2009), in hei book “Designing he Use In e ace” (5 h edi ion) de ine usabili y as he sum o he ollowing measu es (o me ics): ime o lea n, speed o pe o mance, a e o e o s by use s, e en ion o e ime and subjec i e sa is ac ion (Kou sambasis, 2015). Acco ding o A ou is, he e alua ion o he usabili y o a sys em includes he analysis o he cha ac e is ics o he sys em in ela ion o a ce ain con ex o use, he analysis o he in e ac ion p ocess and he analysis o e iciency, e ec i eness and use sa is ac ion (A ou is e al., 2015). Al hough he e a e many esea ch on he usabili y o human compu e in e ac ion, human machine in e ac ion and human obo in e ac ion, he e is no men ion on he usabili y o human mecha onic sys em in e ac ion, because he e m mecha onic sys em is new, is used he las yea s and combines di e en kind o echnologies, which includes pa o echnologies men ioned abo e. This esea ch -66- Jou nal o Technology and Science Educa ion – h ps://doi.o g/10.3926/jo se.1349 examines he “usabili y” o a mecha onic sys em in e ms o e iciency, e ec i eness, Sa is ac ion, ease o use, and ease o lea n [8, 9, 10]. (A ou is e al., 2015; Kou sambasis, 2015; Lund, 2001). The e ec i eness o a mecha onics sys em depends on he deg ee o which a sys em achie es i s goals. Use e ec i eness is de ined as he accu acy and comple eness wi h which use s achie e hei goals using he sys em. I e lec s he use 's abili y o use he sys em e ec i ely and is ela ed o his desi e o use i . This ac o is in luenced by o he ac o s, such as he suppo and managemen o he sys em, he ease o use o he sys em and he quali y o he se ices i p oduces. I is a powe ul ac o o sa is ac ion. In o he wo ds, e ec i eness e e s o he ex en o which use s' objec i e goals a e me . The e m e ec i eness is de ined as he p ope y, o abili y o deli e an expec ed esul . E iciency mainly conce ns he in e nal unc ion o he in e ac ion and exp esses he esou ces consumed o achie e a esul . How easily o di icul ly one can achie e he desi ed esul and how many esou ces will be consumed o achie e his. The sa is ac ion conce ns he subjec i e eeling ha he use de i es om i s use. Ease o use is he abili y o inexpe ienced use s o be able o use he sys em wi hou special knowledge. The in ui ion o he sys em ha allows hem o make use o i wi hou being pa icula ly knowledgeable also plays an impo an ole in achie ing his goal. Lea nabili y is he abili y o no ice use s o unde s and how o use he sys em and how o ge an ini ial le el o good pe o mance. I includes he indi idual p ope ies: p edic abili y, syn hesizabili y, amilia i y, gene alizabili y and consis ency (Kou sambasis, 2015). The s uc u e o he pape includes he sec ion 2 wi h he me hodology o ques ionnai e design, he sec ion 3 wi h he ques ionnai e de elopmen , sec ion 4 wi h he e alua ion o he sec ion 5 wi h he cons uc ion o he ques ionnai e and inally he conclusions whe e he esea che summa izes he esul s o he wo k desc ibed in he pape . 2. Me hodology o Ques ionnai e Design The de elopmen o he ques ionnai e includes se e al s ages. Αs desc ibed in igu e 2 includes he design o he ques ionnai e, he de elopmen o he ques ionnai e, he e alua ion o he ques ionnai e and he inal cons uc ion o he ques ionnai e (Figu e 2). Figu e 2. Ques ionnai e De elopmen Me hodology Ques ionnai e weigh ing includes pilo applica ion, alida ion and eliabili y check and ela ed adjus men s / imp o emen s. I is necessa y o conduc a pilo s udy wi h a ela i ely small numbe o pa icipan s (app oxima ely 30-50), in o de o make a p elimina y assessmen o he alidi y and eliabili y o he ques ionnai e and o co ec as many e o s and omissions as possible. The co ec ed ques ionnai e is hen subjec ed o a inal alidi y and eliabili y es , h ough a new pilo s udy. -67- Jou nal o Technology and Science Educa ion – h ps://doi.o g/10.3926/jo se.1349 Validi y is he ideli y wi h which he a ibu e we wan o measu e coun s, while eliabili y is he accu acy wi h which a ques ionnai e measu es an a ibu e. The alidi y o a measu emen scale, in simple wo ds, e e s o he deg ee o which wha i was made o measu e ac ually coun s. The e a e many ypes o alidi y. In he p esen s udy, con en alidi y and cons uc alidi y will be examined (Galanis, 2013). Con en alidi y e e s o he ex en o which a scale o measu emen measu es he o al o he eques o which i has been made. Du ing he p ocess o assessing he alidi y o he con en , a eco d is made o he da a ela ed o he speci ic concep examined in a ques ion. F om all hese da a, he mos ele an ones a e selec ed. The au ho s o he ques ionnai e ask a p e-selec ed eam o expe s o judge each i em o he ques ionnai e as “necessa y”, “use ul, bu no necessa y” o “necessa y”, and hen calcula e he “Con en Validi y Ra io” o each i em in he ques ionnai e, acco ding o he ollowing equali y Validi y is he ideli y wi h which he a ibu e we wan o measu e coun s, while eliabili y is he accu acy wi h which a ques ionnai e measu es an a ibu e. (1) In he equa ion, N o an i em (eg ques ion) symbolizes he o al numbe o expe s who judge he ques ionnai e i ems and ne symbolizes he numbe o expe s who cha ac e ize he speci ic i em in he ques ionnai e as “necessa y”. When he con en alidi y a io o an elemen o a ques ionnai e is equal o 0, hen hal o he expe s conside his elemen as “necessa y”. When he con en alidi y a io o an i em in a ques ionnai e is > 0, hen mo e han hal o he expe s conside his i em o be “ equi e”. When he con en alidi y a io o an i em in a ques ionnai e is <0, hen less han hal o he expe s conside his i em o be “ equi ed”. 10 expe s a e used in his esea ch. Acco ding o Galanis (2013) he minimum con en alidi y a io ha an i em mus ha e in o de o be included in he ques ionnai e is 0.62 (Galanis, 2013). Cons uc alidi y e e s o he deg ee o which a scale o measu emen accu a ely measu es he meaning we ha e de ined as measu ing. I is o some ex en subjec i e and he e o e equi es a signi ican numbe o s udies o be conduc ed in di e en coun ies, in di e en s udied popula ions and a di e en imes. In he p esen esea ch, he con ol o he alidi y o a concep ual cons uc ion is achie ed by ac o analysis (Pe ei a, Maia, Ma ques, Bos, Soa es, Gomes e al., 2008). The eliabili y o , in o he wo ds, he accu acy o a ques ionnai e e e s o he s abili y o , in o he wo ds, o he consis ency wi h which he ques ionnai e measu es he meaning o he a iable ha i claims o measu e. Inc easing he eliabili y o a ques ionnai e means educing he andom e o . We mus keep in mind ha eliabili y e e s o he esul s o measu ing a scale and no o he scale i sel . This means ha eliabili y is in luenced by he subjec s o he esea ch ( o example by he esponden s) and by he measu emen p o ocol. The e o e a scale can be eliable in one applica ion a ea and un eliable in ano he . The e a e di e en ypes o eliabili y and i s e alua ion in his esea ch o check he eliabili y o epe i i e measu emen s o con ol - es ( e es ) and he eliabili y o in e nal consis ency (in e nal consis ency). In he eliabili y es wi h he me hod o epe i i e measu emen s ( es - e es ) we apply, a o he imes, he scale wice o he same people, unde he same condi ions and inally we check he s a is ical co ela ion be ween he wo sco es. Calcula e he coe icien (Pea son coe icien ) o es - e es . In he con ol o eliabili y wi h he me hod o in e nal consis ency (in e nal consis ency) he homogenei y o he ques ions o he scale is e alua ed. We apply he scale once because he answe s a e no bina y bu mo e, we use C onbach's alpha. I he coe icien is <0.7 he in e nal consis ency eliabili y is ques ionable, while i i is> 0.7 i is accep able. O cou se, he highe he eliabili y (Galanis, 2013). 3. Ques ionnai e De elopmen The s udy was conduc ed in wo s ages conce ning he de elopmen o he s anda d esea ch ool and da a collec ion. As he wo k aims o collec da a o he alida ion o he esea ch ool, da a we e collec ed -68- Jou nal o Technology and Science Educa ion – h ps://doi.o g/10.3926/jo se.1349 om 626 mecha onics sys em use s. The use in ol ed an ATM wi hd awal sys em ha is known o many people. Simila s udies ha e shown ha he sample size is sa is ac o y (Hai , Ande son, Ta ham, Black, 1998; Go such, 1983; Kline, 1979) An ATM is simply a e minal wi h wo inpu de ices (ca d eade , keyboa d) and ou ou pu de ices (mic ophone, moni o , eceip p in e , slo and cash dispense ). These de ices a e connec ed o he p ocesso which is he hea o he ATM. All ATMs wo ldwide a e based on a cen al da abase sys em and o his eason he ATM communica es wi h he cen al p ocesso (se e ) which in u n communica es wi h he in e ne se ice p o ide (ISP) which is he b idge h ough which All ATMs a e a ailable o he ca dholde . When he use wan s o make a ansac ion, he p o ides he necessa y in o ma ion h ough his ca d, which is “ ead” by he buil -in ca d eade , and h ough he keyboa d, which selec s he ype o se ice he needs. The ATM o wa ds his in o ma ion o he se e whe e i ans e s he use 's eques o he espec i e bank. In case o wi hd awal o deposi , he bank sub ac s o adds he equi ed amoun om he use and communica es wi h he ATM p ocesso o comple e he ansac ion h ough he cash- ecei e . To pilo he ques ionnai e, a pilo applica ion was implemen ed wi h a ela i ely small numbe o pa icipan s (25 people) and a p elimina y assessmen o he alidi y and eliabili y o he ques ionnai e was ca ied ou , co ec ing as many e o s and omissions as possible. Some co ec ions we e made ega ding he comp ehension and co ec wo ding o he ques ions, he de e mina ion o he app op ia e ime o comple e, he in e es o he esponden s and he o e all appea ance. The co ec ed ques ionnai e was hen subjec ed o a inal alidi y and eliabili y check. 4. E alua ion o Ques ionnai e Validi y included alidi y o con en and alidi y o concep ual cons uc ion. In Con en Validi y, a p e-selec ed eam o 10 expe s was asked o a e each ques ion in he ques ionnai e as “necessa y”, “use ul bu no necessa y” o “unnecessa y”. The “Con en Validi y Ra io” was hen calcula ed o each ques ion acco ding o equa ion 1. Ques ions answe ed as equi ed wi h a alidi y a io o > 0.62 we e e ained and inco po a ed in o he p oposed usabili y assessmen ques ionnai e. Those deemed necessa y wi h a alidi y a io o <0.62, o deemed use ul bu unnecessa y o unnecessa y we e disca ded and dele ed om he ool (Galanis, 2013). In he nex phase, he alidi y o he concep ual cons uc ion was e alua ed. Fac o analysis was used since he da a had many dimensions (scale 1-5). Applying ac o analysis o he da a o a ques ionnai e, 5 ac o s eme ged ha exp ess indi idual dimensions o he concep measu ed by he s udy ques ionnai e. These ac o s eme ged based on he co ela ions p esen ed be ween he a ious elemen s o he ques ionnai e. Thus, a b oad concep was simpli ied and g ouped in o indi idual pa s o make i clea e . Table 1 below shows he ques ions and ac o s. By examining each ac o i em acco ding o he model and li e a u e amewo k, i ems unde ac o 1 can be placed unde he aspec o e iciency, ac o 2 unde he aspec o e ec i eness, ac o 3 unde he aspec o Sa is ac ion, ac o 4 unde he aspec o ease o use and ac o 5 unde ease o lea n. I ems loaded om each o he i e componen s ha e s ong, clea , and concep ual links. In o de o analyse he alid i ems o each componen , a Kaise –Meye –Olkin (KMO) es and Ba le ’s es o sphe ici y ha e been ca ied ou . Table 2 shows ha he KMO es esul ed in a alue o 0.916. This alue exceeded he ecommended alue o 0.6 (Kaise , 1970; Kaise , 1974), indica ing ha he sample was adequa e o es he ac o analysis. The es - e es me hod was used o check he eliabili y. Thus, he s abili y o he answe s was e alua ed. The same scale was applied wi h a ime di e ence o 15 days o he same indi iduals (20 indi iduals) unde he same condi ions and he s a is ical co ela ion be ween he alues o he esponses was checked. The pea son coe icien calcula ed was 0.923 which means a e y la ge co ela ion. -69- Jou nal o Technology and Science Educa ion – h ps://doi.o g/10.3926/jo se.1349 Ques ions Fac o s 1 2 3 4 5 Q1 The design o in e ac ion is simple wi h he essen ials. 0.847 Q2 The esponse speed o he sys em is sa is ac o y. 0.783 Q3 The execu e o he command is accu a ely. 0.821 Q4 The In e ac ion is comple e. 0.904 Q5 I wo ks as expec ed. 0.791 Q6 The gi en in o ma ion by he in e ace is sa is ac o y. 0.837 Q7 The in e ac ion is easy. 0.868 Q8 A wide ange o commands is co e ed. 0.793 Q9 The e is eedom o mo emen by he use . 0.862 Q10 The change o command is easy. 0.848 Q11 The e is lexibili y in in e ac ion. 0.786 Q12 The use p o ides sa is ac ion. 0.846 Q13 I is lexible in he choice o mo emen s. 0.831 Q14 In e ac ion is no a ec ed by en i onmen al condi ions. 0.871 Q15 The equipmen does no make mo emen s di icul . 0.823 Q16 Ope a ing he sys em is no edious. 0.799 Q17 Commands a e easy o emembe . 0.841 Q18 I is easy o use. 0.787 Q19 I is no edious o use. 0.773 Q20 No mis akes a e made du ing he in e ac ion. 0.826 Q21 I lea ned o use i quickly. 0.707 Q22 I easily emembe how o use i . 0.841 Q23 I quickly became a capable use . 0.785 Q24 I is easy o lea n. 0.837 Q25 I has no di icul unc ions. 0.773 Table 1. P incipal Componen Analysis Tes Value Kaise -Meye -Olkin Adequacy Sampling Measu e 0.916 Ba le ’s Tes o Sphe ici y App ox. Chi-Squa e 5135.417 d 190 Sig. .000 Table 2. Kaise –Meye –Olkin (KMO) and Ba le ’s es . Ques ions Fac o C onbach Alpha Q1 - Q5 E iciency 0.829 Q6 - Q9 E ec i eness 0.840 Q10 - Q13 Sa is ac ion 0.828 Q14 - Q20 Ease o Use 0.817 Q21 - Q25 Ease o Lea n 0.789 Table 3. C onbach Alpha The in e nal consis ency eliabili y was hen checked by calcula ing he C onbach alpha coe icien . The coe icien was calcula ed a 0.819 o he whole ques ionnai e, while o each axis sepa a ely he coe icien alues a e shown in Table 3. -70- Jou nal o Technology and Science Educa ion – h ps://doi.o g/10.3926/jo se.1349 In all cases we see ha we ha e alues g ea e han 0.78 and in some alues hey each up o 0.84. This means ha he ques ions ha e a e y high in e nal cohe ence bo h in each axis and in he o e all ques ionnai e. As can be seen om he analysis o he alidi y and eliabili y o he ques ionnai e, i can be concluded ha is a ool ha can be used o measu e he usabili y o he in e ac ion. 5. Ques ionnai e Cons uc ion The ques ionnai e s a s wi h a epo on he de ails o he su ey. I consis s o wo pa s, one e e ing o he gene al in o ma ion o he pa icipan s and he o he o he de ails o he usabili y assessmen . In he gene al ques ions he pa icipan s we e asked abou hei gende , age and educa ional le el. These da a a e c ucial in o ma ion o he esul s o he esea ch, hus iden i ying he independen a iables o he esea ch. The second pa ques ions e e o he de ails o he e alua ion o he usabili y o mecha onic sys ems di ided in o he 5 axes analyzed abo e. The ques ionnai e was cons uc ed using a i e-poin Like scale anging om 1 (s ongly disag ee) o 5 (s ongly ag ee). The ques ions Q1-Q5 e e s o E iciency and Q6-Q9 o he E ec i eness. The Q10-Q13 e e s o he Sa is ac ion, while he ques ions Q14-Q20 e e s o he Ease o Use. Finally he ques ions Q21-Q25 e e s o Ease o Lea n. 6. Conclusions This pape p esen s he cons uc ion and es ing o a ques ionnai e ha can be used o assess usabili y in in e ac ion wi h mecha onic sys ems. Combining e iciency, e ec i eness, ease o use, ease o lea ning and sa is ac ion, he ques ionnai e was buil o de ec he o e all usabili y o a sys em. All checks on he alidi y and eliabili y o he ques ionnai e we e pe o med and he indica o s showed ha i is an essen ial ool o e alua ing usabili y. Fo he es we e used among he o he 626 use s and he alue o he C onbach alpha coe icien was calcula ed a 0.819 gi en a e y high alidi y and eliabili y o he ool. As u he wo k o he esea ch could be men ioned he applica ion o his ques ionnai e in he in e ac ion wi h compu e in eg a ed sys ems, such p oduc ion lines, cons uc ion lines, obo ic assembly lines, e c. in o de o op imize he in e ac ion wi h hem. Decla a ion o Con lic ing In e es s The au ho s decla ed no po en ial con lic s o in e es conce ning he esea ch, au ho ship, and/o publica ion o his a icle. Funding The au ho ecei ed no inancial suppo o he esea ch, au ho ship, and/o publica ion o his a icle. Re e ences A ou is, N., Ka sanos, C., Tselios, N., & Mous akas, K. (2015). In oduc ion o Human-Compu e In e ac ion. Heallink, Ed. A hens: SEAB. A ailable a : h p://hdl.handle.ne /11419/4224 Cha ziky kou, M., (2020). Usabili y e alua ion o mecha onic sys em by p ima y school child en. 7 h ICMMEN In e na ional Con e ence. Thessaloniki, G eece. h ps://doi.o g/10.1051/ma eccon /202031801026 Dix, A.J., Finlay, J.E., Abowd, G.D., & Beale, R. (2004). Human-Compu e In e ac ion (3 d Edi ion). P en ice- Hall. Galanis, P. (2013). Validi y and eliabili y o ques ionnai es in epidemiological s udies. A chi es o Hellenic Medicine, 30(1), 97-110 Go such, R.L. (1983). Fac o Analysis (2nd ed.). Hillsdale, NJ: Law ence E lbaum Associa es. -71- Jou nal o Technology and Science Educa ion – h ps://doi.o g/10.3926/jo se.1349 Guilleme e, R.A. (1995). The e alua ion o usabili y in in e ac i e in o ma ion sys ems. In: Ca ey, J.M. (Ed.). Human ac o s in in o ma ion sys ems: Eme ging heo e ical bases (207-221). No wood, NJ: Ablex. Hai , J.F.., Ande son, R.E., Ta ham, R.L., Black, W.C. (1998). Mul i a ia e Da a Analysis (5 h ed.). Uppe Saddle Ri e , NJ: P en ice Hall. ISO 9241 (2010). E gonomics o Human-Sys em In e ac ion. In e na ional O ganiza ion o S anda diza ion. ISO 9241-210 (2010). E gonomics o human-sys em in e ac ion -- Pa 210: Human-cen ed design o in e ac i e sys ems. In e na ional O ganiza ion o S anda diza ion. A ailable a : h ps://www.iso.o g/s anda d/52075.h ml (Accessed: Augus 2018) Kline, P. (1979). Psychome ics and Psychology. London, UK: Academic P ess. Kaise , H. (1970). A second-gene a ion Li le Ji y. Psychome ika 35, 401-415. h ps://doi.o g/10.1007/BF02291817 Kaise , H. (1974). An index o ac o ial simplici y. Psychome ika 39, 31-36. h ps://doi.o g/10.1007/BF02291575 Kou sambasis, P. (2015). E alua ion o use -cen e ed in e ac i e sys ems. Seab Ed. A ailable a : h ps:// eposi o y.kallipos.g /handle/11419/2765 Lund, A., (2001). Measu ing usabili y wi h he USE ques ionnai e. Usabili y In e ace, 8(2), 3-6. Luo, G.H., Liu, E.Z.F., Kuo, H.W., & Yuan, S.M. (2014). Design and implemen a ion o a simula ion-based lea ning sys em o in e na ional ade. The In e na ional Re iew o Resea ch in Open and Dis ance Lea ning, 15(1). h ps://doi.o g/10.19173/i odl. 15i1.1666 Nielsen, J. (1990). E alua ing hype ex usabili y. In Jonassen, D.H., & Mandl, H. (Eds.), Designing hype media o lea ning (147-168), Be lin: Sp inge -Ve lag. h ps://doi.o g/10.1007/978-3-642-75945-1_9 Nielsen, J. (1993). Usabili y Enginee ing. London: Academic P ess. h ps://doi.o g/10.1016/B978-0-08-052029- 2.50009-7 Pa sazadeh, N., Ali, R., Meh an, R., & Teh ani, S.Z., (2018). The cons uc ion and alida ion o a usabili y e alua ion su ey o mobile lea ning en i onmen s. S udies in Educa ional E alua ion, 58, 97-111. h ps://doi.o g/10.1016/j.s ueduc.2018.06.002 Pe ei a, A.T., Maia, B.R., Ma ques, M., Bos, S.C., Soa es, M.J., Gomes, A. e al. (2008). Fac o s uc u e o he Ru e Teache Ques ionnai e in Po uguese child en. Re is a B asilei a de Psiquia ia, 30(4), 322-327. h ps://doi.o g/10.1590/S1516-44462008000400004 Shneide man, B., & Plaisan C. (2009). Designing he Use In e ace (5 h Edi ion). Addison-Wesley Published by OmniaScience (www.omniascience.com) Jou nal o Technology and Science Educa ion, 2023 (www.jo se.o g) A icle’s con en s a e p o ided on an A ibu ion-Non Comme cial 4.0 C ea i e commons In e na ional License. Reade s a e allowed o copy, dis ibu e and communica e a icle’s con en s, p o ided he au ho ’s and JOTSE jou nal’s names a e included. I mus no be used o comme cial pu poses. To see he comple e licence con en s, please isi h ps://c ea i ecommons.o g/licenses/by-nc/4.0/. -72-