Haptic Feedback in Eye Typing

Jou nal o Eye Mo emen Resea ch 9(1):3, 1-14 1 In oduc ion Real- ime eye acking enables he use o gaze as an inpu me hod in use in e aces. Fo people wi h se e e physical disabili ies such gaze-based in e ac ion p o ides means o communica e and in e ac wi h echnology and wi h o he people h ough he In e ne . Howe e , using he eyes o con ol is missing na u al eedback; i has o be p o ided by he applica ion. We we e in e es ed in s udying how hap ic eedback – in compa ison o he mo e commonly used isual o audi o y eedback – pe - o ms in ex en y by gaze. Eye yping The e a e many ways o en e ex by gaze, such as using a manual swi ch, blinking o owning o selec he ocused key, o commands based on gaze ges u es (i.e. se ies o eye mo emen s which a e mapped o ce ain le e s o commands), (Maja an a & Räihä, 2007). The mos common me hod, p e e ed by people wi h disabili- ies (Donegan, Gill, & Ellis, 2012, p. 285), is o use dwell- ime o ac i a e commands shown on he compu e sc een. Dwell- ime was also used in he cu en s udy. In he dwell selec ion me hod, he use i s ocuses his/he gaze on he desi ed le e on he i ual keyboa d. The use hen le s he gaze dwell on he le e in o de o selec i . The p ede ined dwell du a ion is used o sepa- a e in en ional commands om casual iewing. The du a ion depends on he expe ise and p e e ences o he use . No ices ypically use dwell imes ha a e be ween 500 and 1000 ms (Maja an a & Räihä, 2007). Howe e , a e aining, mos p e e a sho e dwell ime. In an expe imen o in es iga e use adjus ed dwell ime i was ound ha a e abou one hou o aining, pa icipan s who s a ed wi h a 1000 ms dwell ime educed i down o 500 ms and a e some u he p ac ice, mos s ayed a ound 300-400 ms (Maja an a, Ahola, & Špako , 2009). Expe ypis s may use dwell imes ha co espond o hei no mal ixa ion imes (Räihä & O aska, 2012). Such sho dwell imes equi e an accu a e eye acke as he e is li le possibili y o cancella ion o co ec ing gaze Hap ic Feedback in Eye Typing Päi i Maja an a Uni e si y o Tampe e Poika Isokoski Uni e si y o Tampe e Oleg Špako Uni e si y o Tampe e Ja i Kangas Uni e si y o Tampe e Jussi Ran ala Uni e si y o Tampe e Deepak Akkil Uni e si y o Tampe e Roope Raisamo Uni e si y o Tampe e P ope eedback is essen ial in gaze based in e aces, whe e he same modali y is used o bo h pe cep ion and con ol. We measu ed how ib o ac ile eedback, a o m o hap ic eedback, compa es wi h he commonly used isual and audi o y eedback in eye yping. Hap ic eedback was ound o p oduce esul s ha a e close o hose o audi o y eedback; bo h we e easy o pe cei e and pa icipan s liked bo h he audi o y ”click” and he ac ile “ ap” o he selec ed key. Implemen a ion de ails (such as he placemen o he hap ic ac ua o ) we e also ound impo an . Keywo ds: Gaze inpu , ib o ac ile eedback, hap ics, eye acking, ex en y DOI 10.16910/jem .9.1.3 ISSN 1995-8692This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional license. Jou nal o Eye Mo emen Resea ch Maja an a P., Isokoski, P., Ran ala, J., Špako , O., Akkil, D., Kangas, J., & Raisamo, R. (2016) 9(1):3, 1-14 Hap ic Feedback in Eye Typing 2 poin inaccu acies wi hin he e y sho ime ame (<300 ms) be ween he ini ial ocus on he key and he inal selec ion (Maja an a, Aula, & Räihä, 2004). Feedback As a gene al ule in usabili y enginee ing p ac ice, “ he sys em should con inuously in o m he use abou wha i is doing and how i is in e p e ing he use ’s in- pu ” (Nielsen, 1993, p. 134). Since he eyes a e p ima ily used o pe cep ion, addi ional use as a means o con ol- ling a compu e equi es ca e ul design o he gaze in e - ac ion in o de o p o ide adequa e eedback and o a oid alse ac i a ions. The alse ac i a ion p oblem is known as he Midas ouch p oblem in gaze in e ac ion li e a u e (Jacob, 1991): e e ywhe e he use looks, some hing ge s selec ed. Feedback plays an essen ial ole in coping wi h he Midas ouch challenge. Unless he sys em clea ly shows which i em is unde ocus, he use has li le chance in p e en ing alse selec ions. The na u al eedback ha is p esen in manual yping is missing in eye yping. In manual yping he use eels he inge ouch he key and he key go down, hea s he clicking sound o he key, and is also able o see he cha - ac e appea on he sc een a he same ime as he inge p esses he key. In eye yping none o he ac ile eedback is p esen . Also, he eedom o use eyes o obse ing isual eedback is limi ed. When using eyes o make selec ions, gaze is engaged in he yping p ocess: he use needs o look a he le e o selec i . In o de o e iew he ex w i en so a , he use needs o in e up yping and mo e his o he gaze away om he i ual keyboa d o he ex en y ield. P e ious esea ch has shown ha p ope eedback can acili a e he eye yping p ocess, by signi ican ly imp o - ing pe o mance and subjec i e sa is ac ion (Maja an a, MacKenzie, Aula, & Räihä, 2006). Maja an a e al. (2003) showed ha compa ed o isual eedback alone, a simple audi o y con i ma ion (“click” sound) on selec ion signi ican ly educed equency o swapping he gaze be ween he i ual keys and he yped ex . I he sys em gi es clea eedback on ocus and selec ion, i enables he use o concen a e on he yping ins ead o cons an ly e iewing he yped ex . Using dwell ime, he use only ini ia es he ac ion by dwelling on he i em. The ac ual selec ion is done by he sys em when he dwell ime exceeds he p ede ined du a- ion. Anima ed eedback (e.g. a p og ess ba ) indica es he p og ession o ime, making he dwell ime p ocess isible o he use (Lank o d, 2000; Hansen, Hansen, & Johansen, 2001). P ope eedback helps he use o keep he gaze ixa ed on he ocused i em o dwell-selec ion (Maja an a e al. 2006). Howe e , sho dwell imes do no lea e much ime o anima ed eedback. Thus, sho dwell imes may bene i om a sha p, dis inc eedback on selec ion (Maja an a el al. 2006). A me a-analysis o se e al di e en s udies (no ela - ed o eye yping) indica es ha complemen ing isual eedback wi h audi o y o ac ile eedback imp o es pe - o mance (Bu ke, e al., 2006). This is likely because human esponse imes o mul imodal eedback a e as e han o unimodal eedback. Hech , Reine and Ha ely (2006) ound an a e age esponse ime o 362 ms o unimodal isual, audi o y and hap ic eedback. Fo bi- modal and imodal eedback he a e age esponse imes we e 293 and 270 ms, espec i ely. Response imes a y also be ween modali ies. Resul s o s udies on measu ing pe cep ual la ency gene ally ag ee ha bo h audi o y and hap ic s imuli alone a e p ocessed as e han isual s im- uli, and ha audi o y s imuli a e p ocessed as e han hap ic s imuli (Hi sh & She ick, 1961; Ha a & Ha is, 2008). The la ency o pe cei ing he hap ic s imuli is dependen on he loca ion; a signal om inge s akes a di e en ime han a signal om he head (Ha a & Ha is, 2005). Hap ic eedback, i.e. eedback sensed h ough he sense o ouch, has been used ex ensi ely in communica- ion aids a ge ed o people wi h limi ed sigh o he blind (Jansson, 2008). We we e in e es ed in measu ing i and how hap ic eedback may acili a e eye yping. To ou knowledge, hap ic eedback has no been s udied in he con ex o eye yping. Hap ic eedback has been ound bene icial in i ual ouchsc een keyboa ds ope a ed by manual ouch. A good-quali y ac ile eedback imp o es yping speed and educes e o a e (Hoggan, B ews e , & Johns on, 2008; Koskinen, Kaa esoja, & Lai inen, 2008). Hap ic eedback can be used o in o m he use abou gaze e en s, p o ided he eedback is gi en wi hin he du a ion o a ixa ion (Kangas, Ran ala, Maja an a, Isokoski, & Raisamo, 2014c). I gaze is used o con ol a hand-held de ice such as a mobile phone, hap ic eed- back can be gi en ia he phone. Kangas e al. (2014b) ound ha hap ic eedback on gaze ges u es imp o ed DOI 10.16910/jem .9.1.3 ISSN 1995-8692This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional license. Jou nal o Eye Mo emen Resea ch Maja an a P., Isokoski, P., Ran ala, J., Špako , O., Akkil, D., Kangas, J., & Raisamo, R. (2016) 9(1):3, 1-14 Hap ic Feedback in Eye Typing 3 use pe o mance and sa is ac ion on handheld de ices. In addi ion o hand-held de ices, head-wo n eye acke s p o ide na u al physical con ac poin o hap ic eedback (Ran ala, Kangas, Akkil, Isokoski, & Raisamo, 2014). Finally, in addi ion o quan i a i e measu es, eedback also has an e ec on he quali a i e measu es. P ope eedback can ease lea ning and imp o e use sa is ac ion and use expe ience in gene al. This applies o isual and audi o y (Maja an a e al. 2006) as well as hap ic eed- back (Pakkanen, Raisamo, Raisamo, Salminen, & Su akka, 2010). Despi e he ea lie indings ci ed abo e, i was no clea ha hap ic eedback would wo k well in eye yping, whe e he ask is hea ily ocused on apid se ial poin ing wi h he eyes. In addi ion, he e is a good oppo uni y o gi ing isual and audi o y eedback, because he gaze s ays s a iona y du ing he dwell ime and audio is also easy o pe cei e. On he o he hand, i was emp ing o specula e ha adding he ac ile eedback ha is no na u- ally p esen in eye yping could imp o e he use expe i- ence and possibly also he eye yping pe o mance. Thus, we ound expe imen s necessa y o be e unde s and how hap ic eedback wo ks in eye yping. Expe imen 1: Explo a o y S udy The pu pose o he i s expe imen was o ge i s imp essions on using hap ic eedback in eye yping and o ind ou how well a hap ic con i ma ion o selec ion pe - o ms in compa ison wi h isual o audi o y con i ma- ions. Pa icipan s Twel e uni e si y s uden s (8 male, 4 emale, aged be ween 20 o 35, mean 24 yea s) we e ec ui ed o he expe imen . S uden s we e ewa ded wi h ex a poin s ha coun ed owa ds passing a cou se o hei pa icipa- ion in he expe imen . All we e no ices in eye yping; six had some expe ience in eye acking (e.g. ha ing pa icipa ed in a demons a ion). One pa icipan wo e eye glasses. None had p oblems in seeing, hea ing o ac ile pe cep ion and all we e able-bodied. Appa a us Eye acking. Tobii T60 (Tobii Technology, Sweden) was used o ack he eye mo emen s, oge he wi h a Windows XP lap op. The acke ’s buil -in 17-inch TFT moni o (wi h esolu ion o 1280x1024) was used as he p ima y moni o in he expe imen . Expe imen al so wa e. Al Typing, buil on ETU- D i e (de eloped by Oleg Špako , Uni e si y o Tampe- e) was used o un he expe imen and o log e en da a. Al Typing includes a se up mode whe e one can ad- jus he layou o he i ual keyboa d. The layou o he keyboa d was adjus ed so ha i included le e s and punc ua ion equi ed in he expe imen (see Figu e 1). In addi ion, he e we e a ew unc ion keys, di e en ia ed om he cha ac e s by g een backg ound (le e keys had blue backg ound). The backspace key (ma ked wi h an a ow symbol ‘!’) was used o dele e he las cha ac e . A shi key (⌂) changed he whole keyboa d in o capi als; a e selec ion o a key, he keyboa d e u ned o lowe case con igu a ion. A special ‘Ready’ key (☺) ma ked he end o he yping o he cu en sen ence and loaded he nex sen ence. I was loca ed sepa a ely om he o he keys o a oid acciden ally ending a sen ence p ema u ely. Dwell ime was used o selec a key. Based on pilo es s, he dwell ime o key selec ion was se o 860 ms. DOI 10.16910/jem .9.1.3 ISSN 1995-8692This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional license. Jou nal o Eye Mo emen Resea ch Maja an a P., Isokoski, P., Ran ala, J., Špako , O., Akkil, D., Kangas, J., & Raisamo, R. (2016) 9(1):3, 1-14 Hap ic Feedback in Eye Typing 4 Figu e 1. Al Typing i ual keyboa d o eye yping. Feedback P og ession o he dwell ime was indica ed by a da k blue anima ed, closing ci cle (see ‘n’ in Figu e 1). This anima ion was same o all condi ions. In he end o he anima ion, selec ion was con i med ei he by an audible click, a sho hap ic ib a ion, o isual lash o he key, as desc ibed below. Visual eedback. Visual eedback o selec ion was shown on he sc een as a “ lash” o he selec ed key ( he backg ound colo o he key changed o da k ed o 100 ms, wi h cons an in ensi y and one). The con i ma- ion eedback on selec ion was shown as soon as he dwell ime was exceeded (immedia ely a e he anima ed ci cle had closed). Audi o y eedback. Audi o y eedback played a ‘click’ sound o con i m selec ion ia o dina y desk op loud- speake s. The sound was simila o he de aul click sound used by Mic oso Windows (see Figu e 2, op). Hap ic Feedback. EAI C2 Tac o ib o ac ile ac ua- o , con olled wi h a Gigapo HD USB sound ca d, was used o he hap ic eedback. We chose ib o ac ile ac ua- o s o e o he ac ua ion echnologies such as shea (Win ield, Glassmi e, Colga e, & Peshkin, 2007) and inden a ion (Ran ala, e al., 2011) because ib o ac ile ac ua o s a e compac and easy o a ach o di e en body loca ions. The C2 ac ua o con e ed an audio ile (a 100 ms ile wi h a sinusoidal wa e, wi h a cons an ampli ude o 250 Hz) o ib a ion (see Figu e 2, bo om). Ou aim in c ea ing eedbacks was o make hem sui - able o he modali y and as sho as possible. The audi o- y eedback was easily pe cei ed e en a i s 15ms leng h. The isual eedback was se o 100 ms long o make i clea ly pe cei able; wi h a sho e du a ion i could be missed e.g. due o a blink. Also he hap ic eedback se o 100 ms du a ion would make i easy o pe cei e. Since i was possible ha he pa icipan hea d some sound om he ib o ac ile ac ua o s, and we wan ed o isola e he e ec o audi o y eedback om hap ic eed- back, he pa icipan s wo e hea ing p o ec o s. The hap ic ac ua o was placed on he pa icipan ’s do sal w is . I was held in place by an adjus able Velc o s ap. We con- side ed a w is band as a con enien way o a ach he hap ic ac ua o . Since some in elligen w is wa ches al eady include inbuil hap ic eedback (e.g. Apple Wa ch wi h hap ic ale s), his se up is also ele an o design in he w is wa ch o m ac o . The iming o ini ia ion o he eedback was he same o all eedback ypes ( hey s a ed immedia ely as he dwell ime had un ou ). The expe ienced “s eng h” (in ensi y) o each ype was se in pilo es s o be as simila as possible, al hough i may be conside ed a he subjec i e. In p ac ice, o example, he same signal ou - DOI 10.16910/jem .9.1.3 ISSN 1995-8692This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional license. Jou nal o Eye Mo emen Resea ch Maja an a P., Isokoski, P., Ran ala, J., Špako , O., Akkil, D., Kangas, J., & Raisamo, R. (2016) 9(1):3, 1-14 Hap ic Feedback in Eye Typing 5 pu ed as audio and hap ic would cause qui e di e en pe cei ed “s eng h”. The e o e, he hap ic wa e had o be ampli ied o p oduce an expe ience ha is simila in in ensi y wi h he audi o y click. In o he wo ds, a sound cons uc ed om a signal o e y low ampli ude is s ill easy o pe cei e bu ib o ac ile eedback cons uc ed om he same low ampli ude signal would be ba ely no iceable. Figu e 2. Signal wa e o ms o audi o y (on he op) and hap ic (on he bo om) eedback. Task The ask ha he pa icipan s comple ed in he expe - imen was o ansc ibe a ph ase o ex epea edly. The ph ases we e andomly d awn om he se o 500 ph ases o iginally published by MacKenzie and Souko e (2003). Because he pa icipan s we e Finnish, he Finn- ish ansla ion o he ph ase se was used. The ph ase se was ansla ed in Finnish by Isokoski and Linden (2004). A ew ypos we e co ec ed and he punc ua ion was uni ied o his expe imen . The s imulus ph ase was shown on op o he expe imen al so wa e and he ex w i en by he pa icipan appea ed in an inpu ield be- low he a ge ex (Figu e 1). P ocedu e The pa icipan s we e i s b ie ed abou he mo i a- ion o he s udy and he expe imen al p ocedu e. Each pa icipan was asked o ead and sign an in o med con- sen o m and o ill in a demog aphics o m. The pa icipan was sea ed in on o he eye acke so ha he dis ance be ween he pa icipan ’s eyes and he came a was abou 60-70 cm. The chai posi ion was ixed bu pa icipan ’s mo emen s we e no es ic ed. The eye acke was calib a ed be o e he ask s a ed and ecali- b a ed again be o e each es block. In he hap ic condi- ion, he se up was checked o make su e he hap ic eed- back was easily pe cei able. P io o he expe imen , he pa icipan had a chance o p ac ice eye yping. The se up and he ask du ing aining we e simila o he ac ual expe imen bu he eedback was somewha di e en : Du ing p ac ice, we used he de aul isual eedback gi en by Al Typing. When he pa icipan ocused on a key, i isually ose (as i he eye was pulling i up). A e he 1000 ms dwell ime used du ing he aining had elapsed, he key wen down (like i was p essed by an in isible inge ). A e 150 ms he key e u ned o i s de aul s a e. Du ing he expe imen , he ph ases we e p esen ed one a a ime. The pa icipan was ins uc ed o i s ead and memo ize he ph ase and hen ype i as as and as accu a ely as possible. The p esen ed ph ase emained isible du ing he ask, hus he pa icipan could ead i again i necessa y. The pa icipan s we e ins uc ed o co ec e o s i hey no iced hem immedia ely bu lea e e o s ha hey no iced la e in he middle o he ex . A e inishing he ph ase, he pa icipan ac i a ed he Ready key ha loaded he nex ph ase. The yping ime o each condi ion was se o i e minu es. Fo anal- ysis pu poses, his p ese yping ime included only ac i e yping ime ( om he i s en e ed key o he selec ion o he las key, be o e he selec ion o he Ready key). How- e e , i he p ese ime had elapsed du ing yping, he pa icipan could inish he las ph ase wi hou in e up- ion. A e each 5-minu e block o ph ases, he pa icipan illed in a ques ionnai e abou his/he subjec i e expe i- ence o he eedback used in ha block. The expe imen hen con inued wi h di e en eedback. A e all eedback condi ions had been comple ed, he pa icipan illed in a ques ionnai e compa ing he eed- back modes. The session ended wi h a sho in e iew. The explo a o y s udy included one session ha con- sis ed o he aining and h ee 5-minu e blocks (one o each eedback ype). The whole expe imen , including ins uc ions and pos - es ques ionnai es ook abou one hou . Design and Measu emen s The expe imen had a wi hin-subjec design whe e each pa icipan es ed all eedback condi ions. The o de DOI 10.16910/jem .9.1.3 ISSN 1995-8692This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional license. Jou nal o Eye Mo emen Resea ch Maja an a P., Isokoski, P., Ran ala, J., Špako , O., Akkil, D., Kangas, J., & Raisamo, R. (2016) 9(1):3, 1-14 Hap ic Feedback in Eye Typing 6 o he eedback ypes was coun e -balanced be ween pa icipan s. The eedback ype was he independen a iable. Dependen a iables included he pe o mance and use expe ience measu emen s desc ibed below. Speed. Tex en y a e was measu ed in wo ds pe mi- nu e (WPM), whe e a wo d is de ined as a sequence o i e cha ac e s, including le e s, punc ua ion and space (MacKenzie, 2003). The ph ase yping du a ion was measu ed as he in e al be ween he i s and las key selec ion. The selec ion o Ready key was no included, as i only se ed o s imuli changing, no o ex en y. Accu acy. The e o a e pe cen age was calcula ed by compa ing he ansc ibed ex wi h he p esen ed ex , using he minimum s ing dis ance me hod. E o a e only e alua ed he esul bu did no ake in o accoun he co ec ed e o s. Keys okes pe cha ac e (KSPC) was used o calcula e he o e head incu ed in co ec ing e o s (Souko e & MacKenzie, 2001). Subjec i e expe ience. Pa icipan s’ pe cei ed usabil- i y and subjec i e sa is ac ion was e alua ed wi h ques- ionnai es. The ques ionnai e was gi en a e each eed- back mode, conce ning pe cei ed speed and accu acy, lea ning, pleasu e, a ousal, concen a ion equi emen s, i edness, consis ency and unde s andabili y o each eedback, wi h a scale om 1 (e.g. e y low) o 7 (e.g. e y high). In he end (a e all condi ions), we also asked he pa - icipan s’ eedback mode p e e ences. They we e asked o indica e which eedback mode hey liked he bes and which he leas , which was mos clea , us wo hy and pleasan . We also asked i any o he eedback modes was mo e “dominan (s onge )” han he o he s. S a is ical analysis Repea ed measu es ANOVA was used o es o s a- is ically signi ican di e ences. Pai wise T- es s we e used o analyze di e ences be ween speci ic condi ions when he ANOVA sugges ed a di e ence among mo e han wo condi ions. Un inished ph ases and seman ically inco ec ph ases we e le ou o he analysis (8 ph ases ou o 287 ph ases in o al). Seman ic e o s in he yping occu ed when he pa icipan skipped, added, o mis- ecalled whole wo ds (e.g. “Ne e oo ich” e sus “Ne e oo much”). In a ew cases, calib a ion p oblems caused a p ema u e ending o he yping and equi ed a e-calib a ion be o e he es could con inue. Resul s Speed. The g and mean o yping speed was 8.88 wpm (0.86 s anda d de ia ion, wi h he ange o means o indi idual pa icipan s was 6.49 - 10.74 wpm). Mean alues o each eedback mode we e: audi o y 9.06 (0.83 SD), hap ic 8.90 (0.76 SD), and isual 8.67 (0.98 SD) wpm. The e we e no s a is ically signi ican di e ences be ween he eedback modes. Accu acy. The e was a ia ion in e o a es be ween pa icipan s. This is no su p ise as he pa icipan s we e no ices and hey only used each eedback mode once. The g and mean o indi idual e o a es was 0.48% (0.89 SD). The audi o y eedback mode had he lowes e o a e o 0.17% (0.40 SD), he hap ic eedback was in he middle wi h 0.43% (0.81 SD) and he isual eedback had he highes e o a e o 0.83% (1.46 SD). Howe e , hese di e ences we e no s a is ically signi ican . The pa icipan s co ec ed some o he e o s hey made. This was e lec ed in he keys okes pe cha ac e measu e. The g and mean was 1.06 KSPC (0.06 SD). The mean KSPC o audi o y, hap ic and isual eedback modes we e, espec i ely: 1.06 (0.06 SD), 1.05 (0.06 SD), and 1.06 (0.06 SD). The di e ences be ween eed- back modes we e small and no s a is ically signi ican . Subjec i e expe ience. The e was a lo o a ia ion in he pe cei ed a ibu es (each mode was a ed as he bes in a leas some ca ego y) and he di e ences be ween he eedback modes we e small. In gene al, hap ic eedback was pe cei ed posi i ely and did no di e a lo om o he eedback modes; i was expe ienced as a pleasu a- ble eedback mode ha was easy o lea n and he pa ici- pan s could us . Audi o y eedback and isual eedback we e p e e ed by h ee pa icipan s. Six pa icipan s chose hap ic eed- back as hei p e e ed eedback mode. This is in e es ing, because wo pa icipan s spon aneously commen ed ha he hap ic ac ua o placed on hei w is el uncom o a- ble, “as he second hea a e pulse” o “i ickles” ( h ee pa icipan s ma ked he hap ics as hei leas p e e ed op ion). One pa icipan sugges ed o gi e he hap ic eedback on inge , as i would “ esemble he na u al ac ile eedback while ouch yping”. One pa icipan expe ienced ha he audi o y eedback came a e he DOI 10.16910/jem .9.1.3 ISSN 1995-8692This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional license. Jou nal o Eye Mo emen Resea ch Maja an a P., Isokoski, P., Ran ala, J., Špako , O., Akkil, D., Kangas, J., & Raisamo, R. (2016) 9(1):3, 1-14 Hap ic Feedback in Eye Typing 7 ac ion; echnically, all eedback modes we e gi en a he same ime. Discussion The esul s a e pe haps bes summa ized by highligh - ing he gene al absence o signi ican di e ences be ween he eedback modes. The explo a o y s udy was sho , wi h a lo o a iabili y be ween pa icipan s making i possible ha i was no sensi i e enough o de ec di e - ences. Howe e , i was also possible ha di e ences be ween he eedback modes eally did no exis . The e could ha e been a leas wo easons o his: 1) all eed- back modes we e equi alen in e ms o pe o mance and subjec i e imp essions. 2) None o he eedback modes we e essen ial o accomplishing he ask and he e o e had no e ec on he ask. We suspec ha eason 2 was a leas pa ially ue. This was because o he closing ci cle eedback shown du ing he dwell ime. I allowed he pa icipan s o an icipa e when he selec ion would hap- pen. The mos e icien way o u ilize he sys em was p obably o an icipa e based on he ci cle and igno e he eedback ha ollowed he selec ion. In ac , du ing in e - iew in he end, some pa icipan s commen ed ha i was ha d o e alua e any di e ences be ween he eedback modes. Some o he pa icipan s commen ed ha hey mainly concen a ed on he anima ed eedback gi en on dwell ime p og ession. This suppo s ou easoning ha he eedback a e selec ion was no impo an o he comple ion o he ask. S ill, we lea ned ha in he es ed condi ion o dwell ime based eye yping hap ic eedback did no pe o m any wo se han he audi o y eedback. While s a is ically non-signi ican , he e was a sligh endency o wo se pe o mance wi h isual eedback, which was also e- lec ed in he pa icipan s’ p e e ences. In summa y, he i s explo a o y s udy showed ha hap ic eedback seems o wo k jus as well as he o he modes o eedback. Howe e , we we e le wi h he sus- picion ha pe haps he dwell- ime eedback in e e ed wi h he measu emen o he e ec s o he selec ion eed- back modes. Thus, o measu e he e ec o he selec ion eedback modes, we needed an expe imen ha empha- sized ha pa o he eedback. Expe imen 2: Feedback on Selec ion Based on he expe iences om he i s explo a o y s udy, we designed a ollow-up s udy on he e ec s o hap ic eedback on selec ion in eye yping. Pa icipan s Twel e olun ee s (4 male, 8 emale, aged be ween 24 o 54, mean 39 yea s) pa icipa ed in he expe imen . The olun ee s ecei ed mo ie icke s in e u n o hei pa icipa ion. Th ee wo e eye glasses and one had con ac lenses du ing he expe imen . The one wi h con ac lenses expe ienced some p oblems wi h eye acke calib a ion and equi ed a ew e-calib a ions du ing he expe imen . Pa icipan s included people wi h a ying backg ounds and expe ise le els in gaze-based in e ac ion and use o in o ma ion echnology in gene al (some pa icipan s we e o al no ices om ou side o he academia). Fi e had p e ious expe ience wi h eye acking bu no wi h eye yping. Mos had some expe ience in hap ics, usually om he ib a ion in cell phones (eigh epo ed using hap ic eedback du ing inpu in hei sma phones). Appa a us The eye acke and he expe imen al so wa e we e he same as in he i s expe imen . Feedback Taking in o accoun he lessons lea ned om he i s expe imen , we adjus ed he eedback and modi ied he se up. We emo ed he domina ing “closing ci cle” isual eedback on dwell. By emo ing he anima ed eedback on dwell (which also inhe en ly indica ed he selec ion in he end o dwell), we o ced he pa icipan s o concen- a e on he eedback on selec ion. Howe e , some eed- back on ocus was necessa y as he accu acy o he meas- u ed poin o gaze is no always pe ec and he use s hus needed some indica ion ha he gaze was poin ing a he co ec key. We used a simple, s a ic isual eedback indica ing ocus: when he use ixa ed on a key, i s backg ound colo changed o a sligh ly da ke colo . The colo s ayed da ke as long as he key was ixa ed and e u ned o i s de aul colo when he gaze mo ed away om he key. A sho delay o 100 ms was added be o e he eedback on ocus, because he change in colo was ha d o de ec i i happened immedia ely a e he gaze landed on he key. The delay was included in he dwell DOI 10.16910/jem .9.1.3 ISSN 1995-8692This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional license. Jou nal o Eye Mo emen Resea ch Maja an a P., Isokoski, P., Ran ala, J., Špako , O., Akkil, D., Kangas, J., & Raisamo, R. (2016) 9(1):3, 1-14 Hap ic Feedback in Eye Typing 8 ime (500 ms), hus he eedback on dwelling was shown o 400 ms be o e selec ion. Visual eedback. Visual eedback o selec ion was he same as in he i s expe imen . Audi o y eedback. Audi o y eedback was he same as in he i s expe imen . Hap ic eedback. In he i s expe imen , some pa ic- ipan s did no like he hap ic eedback on hei w is . The e o e, as sugges ed by he pa icipan s, we decided o gi e he eedback on he inge . The pa icipan placed his o he igh index inge on he ac ua o (Figu e 3). One pa icipan had an inju y on he index inge and used he middle inge ins ead. To educe sound om he ib a ion, he ac ua o was placed on a so cushion. We had he op ion o use hea ing p o ec o s i he pa icipan could hea he sound despi e he cushioning bu i was no needed. Again, we es ed he se up be o e he ac ual ex- pe imen wi h each pa icipan . Figu e 3. C2 Tac o hap ic ac ua o . Some pa icipan s complained ha he 100 ms con- s an ampli ude hap ic eedback was uncom o able be- cause i “ ickled” hei skin. Conside ing ha we mo ed he hap ic s imula ion o he inge ip, which is known o be mo e sensi i e, we needed o educe he in ensi y o he ac ile signal. In in o mal es ing we ound ha a signal o he same 250Hz equency wi h decaying ampli- ude ha eached 0 a ound 70 milliseconds (see Figu e 4, bo om) was equally easy o de ec , bu signi ican ly less “ icklish”. Figu e 4. Signal wa e o ms o audi o y (on he op) and hap ic (on he bo om) eedback. Design and Measu emen s We ex ended he expe imen om one session in o h ee so ha each pa icipan used each eedback condi- ion h ee imes. We hoped ha a longe expe imen would be e e eal he po en ial di e ences be ween eedback ypes and he e ec o lea ning. The expe imen ollowed a epea ed measu es design wi h coun e - balancing o he o de o he eedback ypes be ween sessions and pa icipan s. We an a couple o pilo pa icipan s o es he modi- ied se up. I was soon ound ha he o iginal 860 ms dwell ime el oo long al eady a e he i s session. A e a ew adjus men s, i seemed e en as sho as 500 ms would wo k as people seemed o lea n qui e as and hey we e eage o educe he dwell ime qui e soon. This is in line wi h p e ious s udies whe e pa icipan s ha e adjus ed he dwell ime al eady in he i s session (Maja- an a e al. 2009; Räihä & O aska, 2012). Howe e , we we e awa e ha 500 ms may be oo as o he pa ici- pan s’ i s encoun e ed wi h eye yping. The e o e, in he p ac ice session ha p eceded he h ee sessions analyzed below i was se o 860 ms. Al hough he anima ed ci cle was no a pa o he second expe imen , i was used as he only eedback in he p ac ice session. Task and P ocedu e. The ask and he p ocedu e we e he same as in he i s expe imen excep ha he numbe o sessions was now h ee. In o de o keep he ime in he lab easonable, he sessions we e o ganized in wo di e en days. The p ac ice phase and he i s expe i- men al session we e comple ed on he i s day. The se- cond and he hi d sessions we e comple ed on ano he day wi hin a week om he i s day. Be ween he second and he hi d session, pa icipan s ook a b eak o ew DOI 10.16910/jem .9.1.3 ISSN 1995-8692This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional license. Jou nal o Eye Mo emen Resea ch Maja an a P., Isokoski, P., Ran ala, J., Špako , O., Akkil, D., Kangas, J., & Raisamo, R. (2016) 9(1):3, 1-14 Hap ic Feedback in Eye Typing 9 minu es. Du ing he b eak he pa icipan s we e ins uc ed o s e ch o walk. Measu emen s. The same measu emen s we e ana- lyzed as in he i s expe imen (speed, accu acy, subjec- i e expe ience). In addi ion, we collec ed wo measu es on gaze beha io . Du ing eye yping, he use has o look a he i ual keys o selec hem. In o de o e iew he ex w i en so a , he use has o swi ch he gaze om he keyboa d o he ex en y ield. Read ex e en s (RTE) pe cha ac e is a measu e o he equency o gaze swi ching be ween he i ual keyboa d and he ex en y ield. Using dwell-selec ion, he use has o ixa e on he key o long enough o selec i . Re- ocus e en s (RFE) is he measu e o he numbe o p ema u e exi s pe cha ac- e , indica ing he use looked away om he cha ac e oo ea ly and hus had o e- ocus on i o selec i . All ocus-and-lea e e en s we e eco ded, no only inally selec ed cha ac e s. A single gaze sample ou side he key did no ye inc ease he RFE coun , because he expe i- men al so wa e used an in elligen algo i hm (desc ibed in (Räihä, 2015)) o il e ou ou lie s: single gaze sam- ples ou side o he cu en ly ocused key did no ese he dwell ime coun e o ha key bu only dec eased i s accumula ed ime by he ime spen ou side, allowing he dwelling o con inue when he gaze e u ned o he key. Thus, a eal ixa ion on he o he key was equi ed o a ocus/un ocus e en . The so wa e includes an op ion o he dwell- ime eedback delay, which was se o 100 ms in his expe imen . The eedback on ocus (change o he backg ound colo o he key) was only shown a e he minimum dwell ime was ull. The esul s we e analyzed using he same s a is ical me hods as in he i s expe imen . Resul s 19 ph ases (ou o 1048 in o al) we e le ou o he analysis because hey we e ei he un inished (due o poo calib a ion ha o ced e-calib a ion in he middle o he ph ase) o seman ically inco ec (e.g. he pa icipan mis emembe ed he ph ase and hus eplaced whole wo ds). Typing speed. The g and mean o yping speed was 11.20 wpm (SD 1.42), wi h he a e age o 11.80 wpm (SD 1.60) o audi o y, 11.34 wpm (SD 1.31) o hap ic and 10.46 wpm (SD 1.39) o isual eedback. The e- pea ed measu es ANOVA showed ha he eedback mode had a s a is ically signi ican e ec on he yping speed (F(1.73,18.99) = 10.02, p=.002, G eenhouse- Geisse co ec ed). Pai wise - es s showed ha isual eedback di e ed signi ican ly om audi o y (p=.003) and hap ic eedback (p=.006) bu he e was no s a is ical signi icance be ween audi o y and hap ic eedback (p=.135). Mos pa icipan s main ained a yping speed well abo e en wo ds pe minu e bu one pa icipan was clea ly slowe han he o he s ( eaching 5.56 wpm on a e age). Howe e , since ha pe son inished all condi- ions and he e ec (i.e. he a e age speed and de ia ion) was simila in all condi ions and sessions, he da a om his pa icipan we e included in he analysis. A small lea ning e ec was seen in he inc ease o he yping speed, om an a e age o 10.91 wpm (SD 1.46) in he i s session, 11.10 wpm (SD 1.42) in he second, and 11.59 wpm (SD 1.42) in he hi d ( inal) session (see Figu e 5). Howe e , he e ec o session on ex en y a e was no s a is ically signi ican (p=.314). Figu e 5. Typing speed in wo ds pe minu e pe session. Accu acy. The mean e o a e was ai ly low, 1.37% (SD 1.89), bu he e we e huge di e ences in he e o a es be ween pa icipan s as shown in Figu e 6. The e ec o eedback ype on e o a e was s a is ically signi ican ; F(1.10,12.13) = 8.63, p=.011 (G eenhouse- Geisse co ec ed). Visual eedback p oduced he highes e o a es wi h a mean o 2.14% (SD 2.27), compa ed o 1.05% (SD 1.62) o audi o y and 0.93% (1.77) o hap ic eedback. Again, pai wise analysis showed ha isual eedback di e ed signi ican ly om audi o y (p=.009) and hap ic eedback (p=.015) wi h no s a is ical di e - ence be ween audi o y and hap ic eedback (p=.349). The mean e o a es also dec eased sligh ly owa ds he end o he expe imen (2.00%, SD 2.12, i s ; 1.15%, SD 1.89, second; and 0.96%, SD 1.65, o he hi d session) bu his lea ning e ec was no s a is ically signi ican (p=.128). DOI 10.16910/jem .9.1.3 ISSN 1995-8692This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional license.