A Preliminary Investigation into Eye Gaze Data in a First Person Shooter Game

A P elimina y In es iga ion in o Eye Gaze Da a in a Fi s Pe son Shoo e Game Alan Kenny, Hend ik Koesling, Declan Delaney Depa men o Compu e Science Na ional Uni e si y o I eland, Maynoo h Maynoo h, Co. Kilda e, I eland E-mail: [email p o ec ed] Seamus McLoone and Tomás Wa d Depa men o Elec onic Enginee ing Na ional Uni e si y o I eland, Maynoo h Maynoo h, Co. Kilda e, I eland KEYWORDS Eye T acking, Ne wo ked Games, Psycho-Pe cep ual Measu es ABSTACT This pape desc ibes a s udy ca ied ou in which he eye gaze da a o se e al use s playing a simple Fi s Pe son Shoo e (FPS) game has been eco ded. This wo k shows he design and implemen a ion o a simple game and how he execu ion o he game can be synch onized wi h an eye acking sys em. The mo i a ion behind his wo k is o de e mine he exis ence o isual psycho-pe cep ual phenomena, which may be o some use in de eloping app op ia e in o ma ion limi s o dis ibu ed in e ac i e media comp ession algo i hms. Only 2 deg ees o he 140 deg ees o human ision has a high le el o de ail. I may be possible o de e mine he a eas o he sc een ha a use is ocusing on and ende i in high de ail o pay pa icula a en ion o i s con en s so as o se app op ia e dead eckoning limi s. Ou expe imen shows ha eye acking may allow o imp o emen s in ende ing and new comp ession algo i hms o be c ea ed o an online FPS game. 1. INTRODUCTION Ne wo ked in e ac i e compu e games a e an impo an class o Dis ibu ed In e ac i e Applica ion (DIA). These games gene a e a la ge quan i y o da a ha mus be communica ed ac oss he physical ne wo k be ween pa icipa ing game nodes o allow he game s a e o be eplica ed a emo e nodes. Howe e , ne wo k bandwid h is a limi ed esou ce, being pa icula ly sca ce a ypical end-use loca ions. A key esea ch a ea in dis ibu ed compu e games is, he e o e, conce ned wi h de ising new echniques and me hodologies ha educe he amoun o da a ha needs o be gene a ed. In his way he a ailable bandwid h is exploi ed ully, wi hou loss o consis ency o he pa icipan s. One popula echnique in oduced wi h he Dis ibu ed In e ac i e Simula ion (DIS) s anda d (IEEE 1993) and used in games such as Doom and T ibes is dead eckoning (Pan el and Wol 2002). This comp esses use dynamics in o ma ion by down-sampling he use ajec o y. O he echniques employed in DIAs include he hyb id s a egy model (Delaney e al. 2003), a ea o in e es managemen , da a comp ession and dynamic load balancing (Singhal and Zyda 1999). Mo e ecen ly in dis ibu ed media such as ideo o music, psycho pe cep ual phenomena ha e been employed o educe he quan i y o da a ha needs o be communica ed wi hou impac ing he quali y o he end- use expe ience. Examples include he mp3 s anda d o music and he MPEG o Di X s anda ds o ideo. Howe e , in DIAs such as online compu e games he exploi a ion o psycho-pe cep ual pa ame e s has ecei ed limi ed a en ion. This pape is pa o a la ge body o wo k ha hypo hesises ha knowledge o playe psycho-pe cep ion can be exploi ed o educe he amoun o da a ha needs o be ansmi ed be ween use s in dis ibu ed games. Visual in o ma ion p ocessing is one o he mos impo an componen s o how a use pe cei es a gaming expe ience. The con ibu ion o his pape is ha i desc ibes how eye mo emen da a can be eco ded and synch onized wi h game play in an ac ion game de eloped using he To que games engine. Eye acking is one a ea o esea ch ha has ye o be explo ed in his con ex , o he bes o ou knowledge, and has he po en ial o yield in e es ing and use ul psycho-pe cep ual esul s. Eye mo emen s in s a ic scenes, such as a compu e sc een, a e no pe o med con inuously, bu consis o "jumps" (saccades) and " es s" ( ixa ions). Eye- acking esul s a e ep esen ed as lis s o ixa ion da a. This da a con ains, o example, he ixa ion posi ion and du a ion, he cu en pupil size and he s a and end imes o each ixa ion. P e ious wo k on decision making in socce ma ch (Koesling and Höne 2003) has shown dis inc p e- and pos -decisional phases o use ac ion: once a a ge playe has been chosen, he isual ield na ows a ound his a ge o pos -decisional ac ion planning. The unde lying ac ion heo y o decision-making is known as he Rubicon Theo y and he poin ha sepa a es he p e- and pos - decisional phases is known as he Rubicon poin (Gollwi ze and Baye 1999). I is ou goal o de e mine i simila esul s could be ob ained wi h espec o compu e games. This would enable de elope s o de e mine en i ies and egions o he game en i onmen on which playe s ocus hei a en ion. Such in o ma ion could be used o de e mine scene ende ing p io i ies, le el o ende ing de ail and load balancing, as well as p o iding a pa ame e o assigning p io i ies o en i y ex apola ion in comp ession echniques such as dead eckoning o he hyb id s a egy model. The emainde o his pape is s uc u ed as ollows: sec ion 2 desc ibes he expe imen and how he da a acquisi ion was achie ed. The da a is analysed and discussed in de ail in sec ion 3. Finally conclusions a e made along wi h some p oposals o u u e wo k in sec ion 4. 2. METHODOLOGY The sys em consis ed o an SR Resea ch EyeLink®2 eye acke , a con ol Pe sonal Compu e (PC) and a display PC. The EyeLink®2 is a binocula ideo-based eye- acking sys em wi h a sampling a e o up o 500Hz. Du ing expe imen s, subjec s wea he eye- acke headse while hey iew s imuli on a compu e moni o sc een (Figu e 1). Small in a ed (IR) came as on he headse ansmi in o ma ion abou he subjec s' head and pupil posi ions o he eye acke . F om his da a he eye acke calcula es and eco ds he posi ions subjec s look a on he sc een. The con ol PC hos s he eye acke ope a ional so wa e and is used o con ol and calib a e he eye acke . The expe imen is un on he display PC, which is connec ed o he con ol PC ia an E he ne connec ion. The display PC o hese expe imen s was a lap op wi h an ex e nal moni o , mouse and keyboa d. Fou IR ma ke s we e a ached o he ex e nal moni o in o de o he eye acke o compensa e o head mo emen s. The ad an age o using an ex e nal moni o , keyboa d and mouse is ha i p o ides he ope a o wi h eal- ime con ol o e he expe imen , so ha he eye acke so wa e can be s a ed and s opped wi hou in e e ing wi h he subjec s con ols. Figu es 1: Expe imen Se -up A Fi s Pe son Shoo e game was c ea ed using he To que Game Engine (h p://www.ga agegames.com). To que is an indus ial game engine ha has been used o p oduce many games (Ma shall e al. 2004) including he awa d winning T ibes 2. The objec i e o he game was o ge as high a sco e as possible. A use could inc ease hei sco e by des oying enemy bo s o by collec ing okens, bu in o de o achie e a high sco e mo e di icul bo s had o be a acked. Each game las ed wo minu es, wi h a b ie pe iod o synch onisa ion a he s a and end o he game. Six olun ee s pa icipa ed, each playing he game wice; he i s un was conside ed a lea ning expe ience. This esul ed in a o al o wel e da a se s. Th ough he use o ques ionnai es i was disco e ed ha each olun ee had a good le el o compu e expe ise. Th ee olun ee s had keen in e es in ideo games and we e qui e p o icien a hem; he emainde had ela i ely li le expe ience. One o he g ea es challenges encoun e ed in pe o ming he expe imen was he synch onisa ion be ween he eye acke so wa e and he FPS game. These so wa e packages do no ha e easily accessible in e aces and he communica ion and p ocessing o da a is dic a ed by he eal- ime cons ain s o he expe imen . Once he eye acke p og am is ini ialised and he calib a ion p ocedu es ha e inished, he ex e nal To que applica ion is execu ed. A “STARTING GAME” message is sen o he eye acke ile so ha he eco ded eye da a can be synch onized wi h he applica ion ime. In addi ion, jus p io o he game play commencing and inishing, a se o anima ed poin s a e displayed on sc een, each las ing app oxima ely 900ms. The use is asked o ixa e on hese poin s. By loca ing his pa e n in he eye acke da a, he da a ela ing o he s a and end o game play can be isola ed. Th ee se s o da a we e eco ded o each ial: he eye acke da a, he To que eco ding and a cus om To que log ile. Eye acke da a was eco ded e e y 4ms. The To que eco ding is a playback o game e en s based on ne wo k samples and each eco ding was la e eco ded as an AVI ile using FRAPS. FRAPS is a u ili y specially designed o eco ding game oo age ( u he in o ma ion on FRAPS can be ound a h p://www. aps.com/). The To que log eco ds en i y posi ion e e y 500ms in addi ion o all e en s associa ed wi h i ems, weapons, damage and kills. 3. ANALYSIS In Figu e 2 a sample image is shown, which supe imposes he compu e game oo age wi h ma ke s ep esen ing eye ixa ions. The da ke c oss shows he le eye ixa ion poin and he da k iangle shows he igh eye ixa ion poin . In o de o gene a e such supe imposed ideo, he eye da a ela ing o he game play session had o be ex ac ed by hand using he synch oniza ion mechanism men ioned in sec ion 2. Pe l and Ma lab® sc ip s we e hen de eloped o s ip ou and ecombine he in o ma ion om he wo sepa a e da a. Figu e 2: Eye Fixa ion Da a Obse a ions o ini ial esul s om his expe imen a e summa ized in he ollowing pa ag aphs. The c osshai in a FPS game e ec i ely c ea es a na u al ixa ion poin . This coupled wi h he ac ha he use e ec i ely con ols he wo ld iew wi h he c osshai , esul s in he majo i y o ixa ions aking place nea he cen e o he sc een, whe e ‘nea cen e’ is aken o be he inne 400x300 ec angle om he 800x600 esolu ion sc een. In essence he con ol o he na u al ixa ion poin a ound he poin o mos ac i i y in he game esul s in he use using his as a i ual eye. Nea ly 88% o all ixa ions all wi hin his egion as shown in Figu e 3. This esul s in he use s spending 86% o ixa ion ime and 82% o he game ime wi hin he nea cen e egion. In e es ingly he egions o he sc een ep esen ing he use ’s heal h, message box and sco e ecei ed e y li le eye ixa ions, only 2% o all ixa ions all wi hin hese egions o which one use con ibu ed 53%. Figu e 3: Ra io o Nea Cen e Fixa ions o To al Fixa ions o he Twel e Da a Se s This phenomenon could be exploi ed by g aphics ende ing algo i hms (O’ Sulli an and Dingliana 2001). A eas wi hin he nea cen e egion could be ende ed in high quali y wi h he pe iphe y being ende ed in a lowe quali y, as he human pe iphe al ision is less awa e o he lowe quali y g aphics. I has been shown in p e ious esea ch ha he human eye only ocuses on a small a ea o a sc een in high de ail. This esul , based on human pe cep ion capabili ies, can be used o u he comp ess eal ime MPEG ideo s eams, which ha e app op ia ely-paced con en and equi e high esolu ion gain in he eye gaze window (Komogo se and Khan 2004). Resea ch in le el o de ail (LOD) managemen has also shown ha a use ’s subjec i e opinion o a scene is no hampe ed by lowe ing he de ail o non- impo an objec s in ha scene (B own e al. 2003). This would educe p ocessing equi emen s in g aphically in ense FPS i les, such as Doom 3 and Hal -li e 2, allowing de elope s o c ea e g aphically de ailed i les ha each a wide audience. Addi ionally, he human eye is mo e likely o no ice inconsis encies (i.e. due o la ency, ji e and packe loss) ha occu a ound he poin s o ixa ions. As mos ixa ions ake place wi hin his nea cen e egion o a FPS game i is likely ha any inconsis encies ha occu wi hin his egion a e mo e likely o be no iced han hose ha occu ou side his egion. In ne wo ked online games his could be exploi ed by “a ea-o -in e es ” algo i hms o educe he upda e packe s ha need o be exchanged be ween pa icipan s. P e ious wo k on a se o socce ideos has shown ha he e exis s up o 600ms o delay om when a pe son makes a decision abou execu ing a ask as indica ed by eye mo emen and when he ask is ac ually execu ed (Koesling e al. 2001). In hose expe imen s use s had only o make a single decision while iewing a se ies o six-second ideo clips. Howe e , in he expe imen s desc ibed he e he na u e o a FPS game means ha he ime be ween use decisions can be e y sho . This makes i di icul o consis en ly iden i y he Rubicon poin . As a consequence, i may be di icul o u ilize eye da a in o de o p e send in o ma ion du ing online mul iplaye games. Howe e , o he game gen es ha do no equi e wi ch ac ions may be mo e sui able o using eye da a o p edic ing use s’ ac ions. Fu he expe imen s o in es iga e he decision-making p ocess in games would ha e o be conduc ed o es his hypo hesis. Ou esul s also indica e ha , in gene al, use s exhibi ixa ion du a ions ha a e compa able o hose o o he media (Sibe and Jacob 2000). In Figu e 4 he a e age le eye ixa ion du a ion o each session is shown (a simila g aph can be de i ed om he igh eye ixa ions). As can be seen he a e age ixa ion du a ion o mos use s alls wi hin he ange o 300ms and 750ms, wi h only wo ou lie s. These ou lie s belong o an ex emely expe ienced FPS playe , which may explain hei occu ence. The esul s appea o con adic he in o ma ion displayed in he Figu e 4: A e age Le Eye Fixa ion Du a ion Figu e 5: F equency Dis ibu ion o Fixa ion Du a ions supe imposed ideos, whe e use s seem o ha e ela i ely long ixa ion du a ions ( ixa ions o up o en seconds being measu ed). This can be explained by he ac ha mos use s (excep he p e iously no ed ou lie s) ha e mo e ixa ions unde 300ms han be ween 300ms and 600ms o o e 600ms as seen in igu e 5. The a e age ixa ion du a ion minus he ixa ions unde 300ms is nea ly 1150ms, which mo e accu a ely ep esen s he da a ha was obse ed in he ideos. Ano he poin o in e es is he endency o highe game sco es o esul in longe ixa ion du a ions and consequen ly a lowe numbe o ixa ions as shown in igu e 6. I can also be seen ha playe s didn’ necessa ily pe o m be e on hei second a emp . While he sco ing mechanism used wi hin he game may no pe ec ly ep esen ask p o iciency all subjec s exhibi ed highe sco es o games wi h longe a e age ixa ion du a ions. Mo e accu a e measu es o ask p o iciency would ha e o be de eloped in o de o e i y his. The e may also be a co ela ion be ween he high pe cen ages o nea cen e ixa ions, he du a ion o he a e age ixa ion and he use ’s sco e. Using a playe ’s sco e as an indica ion o hei abili ies, he h ee bes playe s had an a e age o 91% nea cen e ixa ions and a e age ixa ion du a ion o jus o e 700ms. Addi ionally he wo s playe s had an a e age nea cen e ixa ion o 84% and a e age ixa ion du a ion o jus less han 470ms. Eye mo emen esea ch has al eady sugges ed ha high equency o ixa ions in a egion coupled wi h long du a ions indica es high ask di icul y and can be aken as an indica ion o in o ma ion complexi y, simila ly a smalle numbe o saccades indica es highe men al load (Pan e al. 2004). In ela ion o ou FPS game his may be explained as he mo e p o icien playe s ha ing g ea e con ol o e he i ual eye. As he ask o sco ing in ol ed disabling enemies and his is accomplished by ocusing he c osshai on he a ge i may indica e ha he be e a playe becomes a supplemen ing hei own eye mo emen wi h i ual eye mo emen he mo e accu a ely hey a e able o ocus on he ac i i y a ound he c osshai s. Fu he esea ch would ha e o be ca ied ou o de e mine i his is he case. Table 1: S a is ics Summa y Use Sco e N ea Cen e Fix. % Time in Nea Cen e Region % A g. L. Fix. ms Num. Fix. Unde 300 ms Num. Fix. Be ween 300 ms – 600 ms Num. Fix. O e 600 ms 1.1 3350 97 99 847 62 100 125 2.1 3700 85 95 628 194 93 84 3.1 1200 93 96 383 368 131 83 4.1 2100 84 88 426 302 155 91 5.1 2500 93 97 723 148 76 103 6.1 1100 72 86 468 249 141 78 1.2 5800 90 98 941 97 60 97 2.2 2800 87 92 526 220 140 87 3.2 1500 91 96 424 321 144 79 4.2 1200 79 85 370 368 167 81 5.2 2150 95 96 647 157 113 91 6.2 1900 85 90 754 165 58 88 Figu e 6: A e age Le Eye Fixa ion Ve sus Sco e 4. CONCLUSIONS AND FUTURE WORK In his pape a sys em o synch onising an eye acke wi h a comme cial game de elopmen engine has been desc ibed. P elimina y esul s sugges ha he sys em success ully co- egis e s eye gaze in o ma ion wi h he isual scene p esen ed o he use du ing hei game play. Analysis o he da a indica es ha use s o a FPS spend an ex emely la ge p opo ion o ime ocused a ound he cen e o he sc een, ha he e is some indica ion o he Rubicon heo y aking place wi hin games and ha he e may be a co ela ion be ween ask p o iciency and he du a ion o eye mo emen . Fu u e wo k will in es iga e psycho-pe cep ual phenomenon ha can be used o il e he in o ma ion ha needs o be ansmi ed in DIAs such as ne wo ked compu e games o main ain an adequa ely consis en global sha ed s a e. Speci ically i he poin o he Rubicon can be accu a ely and con inuously de e mined hen some in o ma ion abou use ac ions could be sen be o e hey ake place. While ou esul s ha e shown ha i may be di icul o achie e his in a as ac ion FPS game, o he game gen es wi h slowe in e ac ions o an FPS wi h mo e con olled in e ac ions may p o e easie o analyse. Fu u e expe imen s will aim o mo e accu a ely highligh he Rubicon e ec wi hin games. REFERENCES B own, R., Coppe , L. and Pham, B. 2003. “Visual A en ion- based Polygon Le el o De ail Managemen .” Fi s In e na ional Con e ence on Compu e G aphics and In e ac i e Techniques, Melbou ne, Aus alia. Delaney, D., Wa d, T. and McLoone, S. 2003. “Reducing Upda e Packe s in Dis ibu ed In e ac i e Applica ions using a Hyb id Model.” 16 h In e na ional Con e ence on Pa allel and Dis ibu ed Compu ing Sys ems, Reno, USA. Gollwi ze , P.M., and Baye , U. 1999. “Delibe a i e e sus implemen al mindse s in he con ol o ac ion.” Dual- p ocess heo ies in social psychology (pp. 403-422). New Yo k: Guil o d. IEEE 1993. IEEE S anda d o Dis ibu ed In e ac i e Simula ion - Applica ion P o ocols, in IEEE S d 1278- 1993, I.C. Socie y, Edi o . 1993, IEEE: New Yo k. Koesling, H. and Hone , O. 2003. “Bend i like Beckham – Mindse s and Visual A en ion in Decision-Making in Socce .” P oc. 12 h Eu opean Con e ence on Eye Mo emen s, Dundee, UK. Komogo se , O., and Khan, J. 2004. “P edic i e Pe cep ual Comp ession o Real Time Video Communica ion.” P oc. 12 h ACM In e na ional Con e ence on Media, New Yo k, USA. Ma shall, D., Delaney, D., McLoone, S. and Wa d, T. 2004. “Explo ing he Spa ial Densi y o S a egy Models in a Realis ic Dis ibu ed In e ac i e Applica ion.” In e na ional Symposium on Dis ibu ed Simula ion and Real-Time Applica ions, Budapes , Hunga y. O’Sullli an, C. and Dingliana, J. 2001. “Collisions and Pe cep ion.” ACM T ansac ions on G aphics, Volume 20, Issue 3. Pan, B. e al. 2004. “The De e minan s o Web Page Viewing Beha iou : An Eye T acking S udy.” P oc. Eye T acking Resea ch and Applica ions Symposium, San An onio, Texas. Pan el, L. and Wol , L. 2002. “On he Sui abili y o Dead Reckoning Schemes o Games.” Fi s Wo kshop on Ne wo k and Sys em Suppo o Games, B aunschweig, Ge many. Sibe , L. and Jacob, R. 2000. “E alua ion o Eye Gaze In e ac ion.” P oc. SIGCHI con e ence on Human Fac o s, The Hague ,Ne he lands . Singhal, S. and Zyda, M. 1999. “Ne wo ked Vi ual En i onmen s: Design and Implemen a ion.” ACM P ess SIGGRAPH Se ies. Addison-Wesley, Reading, Massachuse s. AUTHOR BIOGRAPHIES ALAN KENNY was bo n in Mullinga , I eland and wen o NUI, Maynoo h, whe e he s udied compu e science and so wa e enginee ing and ob ained his deg ee in 2004. Ha ing spen a sho s in wo king as a web de elope he e u ned o he Uni e si y o s udy a PhD in he a ea o psycho-pe cep ual measu es in ne wo ked games. His e-mail add ess is: [email p o ec ed] and his Web page can be ound a h p://www.cs.may.ie/~akenny. HENDRIK KOESLING g adua ed in compu e science and physics (M.Sc.) in 1997 and ecei ed a PhD in compu e science om Biele eld Uni e si y, Ge many, in 2003. He wo ked as a lec u e and esea che in he Neu oin o ma ics G oup and he CRC360 "Si ua ed A i icial Communica o s" a Biele eld Uni e si y un il 2004. He cu en ly holds a pos doc o al esea ch posi ion a he Compu e Science Depa men a NUI Maynoo h. His esea ch in e es s a e in isual pe cep ion/a en ion and cogni i e p ocesses in humans, based on he analysis and modeling o eye mo emen s. DECLAN DELANEY ecei ed his elec onic enginee ing deg ee om Uni e si y College Dublin in 1991 and his Mas e s deg ee in 1999. He has wo ked ab oad as bo h a so wa e enginee and p ojec manage . A p esen he lec u es in he Compu e Science Depa men in NUI Maynoo h. His cu en esea ch in e es s ocus on echniques o educing la ency in dis ibu ed ne wo ked applica ions and on he use o small g oup PBL in so wa e enginee ing educa ion. He is a cha e ed membe o he IEI. SEAMUS McLOONE ecei ed his B.Eng. honou s deg ee in Elec ical and Elec onic Enginee ing om Queen's Uni e si y Bel as in 1996 and his Ph.D., en i led Nonlinea Iden i ica ion using Local Model Ne wo ks, in Sep embe 2000. A p esen , he is employed as a lec u e in he Depa men o Elec onic Enginee ing in he Na ional Uni e si y o I eland (NUI), Maynoo h. His main esea ch in e es s include mul iple model app oaches o nonlinea sys em iden i ica ion and educing and masking he e ec s o la ency in dis ibu ed in e ac i e applica ions (DIAs). TOMAS WARD ecei ed he B.E., he M.Eng.Sc. and Ph.D deg ees om Uni e si y College, Dublin, I eland in 1994, 1996 and 1999 espec i ely. He wo ked as Resea ch Coo dina o in he Rehabili a ion Enginee ing Depa men a he Na ional Rehabili a ion Hospi al om 1996-1999. F om 1999-2000 he was an Assis an Lec u e in he Depa men o Compu e Science a he Na ional Uni e si y o I eland, Maynoo h. In 2000 he became a lec u e in he Depa men o Elec onic Enginee ing a NUI Maynoo h. His main esea ch in e es s lie in he a eas o nea -in a ed op ical and e ahe z medical imaging, he de elopmen o he apeu ic ools in medical ehabili a ion, emo ion syn hesis in a i icial speech and dis ibu ed en i onmen s o mul i-use in e ac ion.