A P elimina y In es iga ion in o Eye Gaze Da a in a Fi s Pe son Shoo e
Game
Alan Kenny, Hend ik Koesling, Declan Delaney
Depa men o Compu e Science
Na ional Uni e si y o I eland, Maynoo h
Maynoo h, Co. Kilda e, I eland
E-mail: [email p o ec ed]
Seamus McLoone and Tomás Wa d
Depa men o Elec onic Enginee ing
Na ional Uni e si y o I eland, Maynoo h
Maynoo h, Co. Kilda e, I eland
KEYWORDS
Eye T acking, Ne wo ked Games, Psycho-Pe cep ual
Measu es
ABSTACT
This pape desc ibes a s udy ca ied ou in which he
eye gaze da a o se e al use s playing a simple Fi s
Pe son Shoo e (FPS) game has been eco ded. This
wo k shows he design and implemen a ion o a simple
game and how he execu ion o he game can be
synch onized wi h an eye acking sys em. The
mo i a ion behind his wo k is o de e mine he
exis ence o isual psycho-pe cep ual phenomena,
which may be o some use in de eloping app op ia e
in o ma ion limi s o dis ibu ed in e ac i e media
comp ession algo i hms. Only 2 deg ees o he 140
deg ees o human ision has a high le el o de ail. I
may be possible o de e mine he a eas o he sc een ha
a use is ocusing on and ende i in high de ail o pay
pa icula a en ion o i s con en s so as o se
app op ia e dead eckoning limi s. Ou expe imen
shows ha eye acking may allow o imp o emen s in
ende ing and new comp ession algo i hms o be c ea ed
o an online FPS game.
1. INTRODUCTION
Ne wo ked in e ac i e compu e games a e an impo an
class o Dis ibu ed In e ac i e Applica ion (DIA).
These games gene a e a la ge quan i y o da a ha mus
be communica ed ac oss he physical ne wo k be ween
pa icipa ing game nodes o allow he game s a e o be
eplica ed a emo e nodes. Howe e , ne wo k
bandwid h is a limi ed esou ce, being pa icula ly
sca ce a ypical end-use loca ions. A key esea ch a ea
in dis ibu ed compu e games is, he e o e, conce ned
wi h de ising new echniques and me hodologies ha
educe he amoun o da a ha needs o be gene a ed. In
his way he a ailable bandwid h is exploi ed ully,
wi hou loss o consis ency o he pa icipan s. One
popula echnique in oduced wi h he Dis ibu ed
In e ac i e Simula ion (DIS) s anda d (IEEE 1993) and
used in games such as Doom and T ibes is dead
eckoning (Pan el and Wol 2002). This comp esses
use dynamics in o ma ion by down-sampling he use
ajec o y. O he echniques employed in DIAs include
he hyb id s a egy model (Delaney e al. 2003), a ea o
in e es managemen , da a comp ession and dynamic
load balancing (Singhal and Zyda 1999).
Mo e ecen ly in dis ibu ed media such as ideo o
music, psycho pe cep ual phenomena ha e been
employed o educe he quan i y o da a ha needs o be
communica ed wi hou impac ing he quali y o he end-
use expe ience. Examples include he mp3 s anda d
o music and he MPEG o Di X s anda ds o ideo.
Howe e , in DIAs such as online compu e games he
exploi a ion o psycho-pe cep ual pa ame e s has
ecei ed limi ed a en ion. This pape is pa o a la ge
body o wo k ha hypo hesises ha knowledge o
playe psycho-pe cep ion can be exploi ed o educe he
amoun o da a ha needs o be ansmi ed be ween
use s in dis ibu ed games. Visual in o ma ion
p ocessing is one o he mos impo an componen s o
how a use pe cei es a gaming expe ience. The
con ibu ion o his pape is ha i desc ibes how eye
mo emen da a can be eco ded and synch onized wi h
game play in an ac ion game de eloped using he
To que games engine.
Eye acking is one a ea o esea ch ha has ye o be
explo ed in his con ex , o he bes o ou knowledge,
and has he po en ial o yield in e es ing and use ul
psycho-pe cep ual esul s. Eye mo emen s in s a ic
scenes, such as a compu e sc een, a e no pe o med
con inuously, bu consis o "jumps" (saccades) and
" es s" ( ixa ions). Eye- acking esul s a e ep esen ed
as lis s o ixa ion da a. This da a con ains, o example,
he ixa ion posi ion and du a ion, he cu en pupil size
and he s a and end imes o each ixa ion. P e ious
wo k on decision making in socce ma ch (Koesling and
Höne 2003) has shown dis inc p e- and pos -decisional
phases o use ac ion: once a a ge playe has been
chosen, he isual ield na ows a ound his a ge o
pos -decisional ac ion planning. The unde lying ac ion
heo y o decision-making is known as he Rubicon
Theo y and he poin ha sepa a es he p e- and pos -
decisional phases is known as he Rubicon poin
(Gollwi ze and Baye 1999). I is ou goal o de e mine
i simila esul s could be ob ained wi h espec o
compu e games. This would enable de elope s o
de e mine en i ies and egions o he game en i onmen
on which playe s ocus hei a en ion. Such
in o ma ion could be used o de e mine scene ende ing
p io i ies, le el o ende ing de ail and load balancing,
as well as p o iding a pa ame e o assigning p io i ies
o en i y ex apola ion in comp ession echniques such
as dead eckoning o he hyb id s a egy model.
The emainde o his pape is s uc u ed as ollows:
sec ion 2 desc ibes he expe imen and how he da a
acquisi ion was achie ed. The da a is analysed and
discussed in de ail in sec ion 3. Finally conclusions a e
made along wi h some p oposals o u u e wo k in
sec ion 4.
2. METHODOLOGY
The sys em consis ed o an SR Resea ch EyeLink®2
eye acke , a con ol Pe sonal Compu e (PC) and a
display PC. The EyeLink®2 is a binocula ideo-based
eye- acking sys em wi h a sampling a e o up o
500Hz. Du ing expe imen s, subjec s wea he eye-
acke headse while hey iew s imuli on a compu e
moni o sc een (Figu e 1). Small in a ed (IR) came as
on he headse ansmi in o ma ion abou he subjec s'
head and pupil posi ions o he eye acke . F om his
da a he eye acke calcula es and eco ds he posi ions
subjec s look a on he sc een. The con ol PC hos s he
eye acke ope a ional so wa e and is used o con ol
and calib a e he eye acke . The expe imen is un on
he display PC, which is connec ed o he con ol PC ia
an E he ne connec ion. The display PC o hese
expe imen s was a lap op wi h an ex e nal moni o ,
mouse and keyboa d. Fou IR ma ke s we e a ached o
he ex e nal moni o in o de o he eye acke o
compensa e o head mo emen s. The ad an age o
using an ex e nal moni o , keyboa d and mouse is ha i
p o ides he ope a o wi h eal- ime con ol o e he
expe imen , so ha he eye acke so wa e can be
s a ed and s opped wi hou in e e ing wi h he subjec s
con ols.
Figu es 1: Expe imen Se -up
A Fi s Pe son Shoo e game was c ea ed using he
To que Game Engine (h p://www.ga agegames.com).
To que is an indus ial game engine ha has been used
o p oduce many games (Ma shall e al. 2004) including
he awa d winning T ibes 2. The objec i e o he game
was o ge as high a sco e as possible. A use could
inc ease hei sco e by des oying enemy bo s o by
collec ing okens, bu in o de o achie e a high sco e
mo e di icul bo s had o be a acked. Each game las ed
wo minu es, wi h a b ie pe iod o synch onisa ion a
he s a and end o he game. Six olun ee s
pa icipa ed, each playing he game wice; he i s un
was conside ed a lea ning expe ience. This esul ed in a
o al o wel e da a se s. Th ough he use o
ques ionnai es i was disco e ed ha each olun ee had
a good le el o compu e expe ise. Th ee olun ee s
had keen in e es in ideo games and we e qui e
p o icien a hem; he emainde had ela i ely li le
expe ience.
One o he g ea es challenges encoun e ed in
pe o ming he expe imen was he synch onisa ion
be ween he eye acke so wa e and he FPS game.
These so wa e packages do no ha e easily accessible
in e aces and he communica ion and p ocessing o
da a is dic a ed by he eal- ime cons ain s o he
expe imen . Once he eye acke p og am is ini ialised
and he calib a ion p ocedu es ha e inished, he
ex e nal To que applica ion is execu ed. A “STARTING
GAME” message is sen o he eye acke ile so ha
he eco ded eye da a can be synch onized wi h he
applica ion ime. In addi ion, jus p io o he game play
commencing and inishing, a se o anima ed poin s a e
displayed on sc een, each las ing app oxima ely 900ms.
The use is asked o ixa e on hese poin s. By loca ing
his pa e n in he eye acke da a, he da a ela ing o
he s a and end o game play can be isola ed.
Th ee se s o da a we e eco ded o each ial: he eye
acke da a, he To que eco ding and a cus om To que
log ile. Eye acke da a was eco ded e e y 4ms. The
To que eco ding is a playback o game e en s based on
ne wo k samples and each eco ding was la e eco ded
as an AVI ile using FRAPS. FRAPS is a u ili y
specially designed o eco ding game oo age ( u he
in o ma ion on FRAPS can be ound a
h p://www. aps.com/). The To que log eco ds en i y
posi ion e e y 500ms in addi ion o all e en s associa ed
wi h i ems, weapons, damage and kills.
3. ANALYSIS
In Figu e 2 a sample image is shown, which
supe imposes he compu e game oo age wi h ma ke s
ep esen ing eye ixa ions. The da ke c oss shows he
le eye ixa ion poin and he da k iangle shows he
igh eye ixa ion poin . In o de o gene a e such
supe imposed ideo, he eye da a ela ing o he game
play session had o be ex ac ed by hand using he
synch oniza ion mechanism men ioned in sec ion 2. Pe l
and Ma lab® sc ip s we e hen de eloped o s ip ou
and ecombine he in o ma ion om he wo sepa a e
da a.
Figu e 2: Eye Fixa ion Da a
Obse a ions o ini ial esul s om his expe imen a e
summa ized in he ollowing pa ag aphs.
The c osshai in a FPS game e ec i ely c ea es a
na u al ixa ion poin . This coupled wi h he ac ha he
use e ec i ely con ols he wo ld iew wi h he
c osshai , esul s in he majo i y o ixa ions aking
place nea he cen e o he sc een, whe e ‘nea cen e’
is aken o be he inne 400x300 ec angle om he
800x600 esolu ion sc een. In essence he con ol o he
na u al ixa ion poin a ound he poin o mos ac i i y
in he game esul s in he use using his as a i ual eye.
Nea ly 88% o all ixa ions all wi hin his egion as
shown in Figu e 3. This esul s in he use s spending
86% o ixa ion ime and 82% o he game ime wi hin
he nea cen e egion. In e es ingly he egions o he
sc een ep esen ing he use ’s heal h, message box and
sco e ecei ed e y li le eye ixa ions, only 2% o all
ixa ions all wi hin hese egions o which one use
con ibu ed 53%.
Figu e 3: Ra io o Nea Cen e Fixa ions o To al
Fixa ions o he Twel e Da a Se s
This phenomenon could be exploi ed by g aphics
ende ing algo i hms (O’ Sulli an and Dingliana 2001).
A eas wi hin he nea cen e egion could be ende ed in
high quali y wi h he pe iphe y being ende ed in a
lowe quali y, as he human pe iphe al ision is less
awa e o he lowe quali y g aphics. I has been shown
in p e ious esea ch ha he human eye only ocuses on
a small a ea o a sc een in high de ail. This esul , based
on human pe cep ion capabili ies, can be used o u he
comp ess eal ime MPEG ideo s eams, which ha e
app op ia ely-paced con en and equi e high esolu ion
gain in he eye gaze window (Komogo se and Khan
2004). Resea ch in le el o de ail (LOD) managemen
has also shown ha a use ’s subjec i e opinion o a
scene is no hampe ed by lowe ing he de ail o non-
impo an objec s in ha scene (B own e al. 2003). This
would educe p ocessing equi emen s in g aphically
in ense FPS i les, such as Doom 3 and Hal -li e 2,
allowing de elope s o c ea e g aphically de ailed i les
ha each a wide audience.
Addi ionally, he human eye is mo e likely o no ice
inconsis encies (i.e. due o la ency, ji e and packe
loss) ha occu a ound he poin s o ixa ions. As mos
ixa ions ake place wi hin his nea cen e egion o a
FPS game i is likely ha any inconsis encies ha occu
wi hin his egion a e mo e likely o be no iced han
hose ha occu ou side his egion. In ne wo ked online
games his could be exploi ed by “a ea-o -in e es ”
algo i hms o educe he upda e packe s ha need o be
exchanged be ween pa icipan s.
P e ious wo k on a se o socce ideos has shown ha
he e exis s up o 600ms o delay om when a pe son
makes a decision abou execu ing a ask as indica ed by
eye mo emen and when he ask is ac ually execu ed
(Koesling e al. 2001). In hose expe imen s use s had
only o make a single decision while iewing a se ies o
six-second ideo clips. Howe e , in he expe imen s
desc ibed he e he na u e o a FPS game means ha he
ime be ween use decisions can be e y sho . This
makes i di icul o consis en ly iden i y he Rubicon
poin . As a consequence, i may be di icul o u ilize
eye da a in o de o p e send in o ma ion du ing online
mul iplaye games. Howe e , o he game gen es ha do
no equi e wi ch ac ions may be mo e sui able o using
eye da a o p edic ing use s’ ac ions. Fu he
expe imen s o in es iga e he decision-making p ocess
in games would ha e o be conduc ed o es his
hypo hesis.
Ou esul s also indica e ha , in gene al, use s exhibi
ixa ion du a ions ha a e compa able o hose o o he
media (Sibe and Jacob 2000). In Figu e 4 he a e age
le eye ixa ion du a ion o each session is shown (a
simila g aph can be de i ed om he igh eye
ixa ions). As can be seen he a e age ixa ion du a ion
o mos use s alls wi hin he ange o 300ms and
750ms, wi h only wo ou lie s. These ou lie s belong o
an ex emely expe ienced FPS playe , which may
explain hei occu ence. The esul s appea o
con adic he in o ma ion displayed in he
Figu e 4: A e age Le Eye Fixa ion Du a ion Figu e 5: F equency Dis ibu ion o Fixa ion Du a ions
supe imposed ideos, whe e use s seem o ha e
ela i ely long ixa ion du a ions ( ixa ions o up o en
seconds being measu ed). This can be explained by he
ac ha mos use s (excep he p e iously no ed
ou lie s) ha e mo e ixa ions unde 300ms han be ween
300ms and 600ms o o e 600ms as seen in igu e 5.
The a e age ixa ion du a ion minus he ixa ions unde
300ms is nea ly 1150ms, which mo e accu a ely
ep esen s he da a ha was obse ed in he ideos.
Ano he poin o in e es is he endency o highe
game sco es o esul in longe ixa ion du a ions and
consequen ly a lowe numbe o ixa ions as shown in
igu e 6. I can also be seen ha playe s didn’
necessa ily pe o m be e on hei second a emp .
While he sco ing mechanism used wi hin he game may
no pe ec ly ep esen ask p o iciency all subjec s
exhibi ed highe sco es o games wi h longe a e age
ixa ion du a ions. Mo e accu a e measu es o ask
p o iciency would ha e o be de eloped in o de o
e i y his. The e may also be a co ela ion be ween he
high pe cen ages o nea cen e ixa ions, he du a ion o
he a e age ixa ion and he use ’s sco e. Using a
playe ’s sco e as an indica ion o hei abili ies, he h ee
bes playe s had an a e age o 91% nea cen e ixa ions
and a e age ixa ion du a ion o jus o e 700ms.
Addi ionally he wo s playe s had an a e age nea
cen e ixa ion o 84% and a e age ixa ion du a ion o
jus less han 470ms. Eye mo emen esea ch has
al eady sugges ed ha high equency o ixa ions in a
egion coupled wi h long du a ions indica es high ask
di icul y and can be aken as an indica ion o
in o ma ion complexi y, simila ly a smalle numbe o
saccades indica es highe men al load (Pan e al. 2004).
In ela ion o ou FPS game his may be explained as he
mo e p o icien playe s ha ing g ea e con ol o e he
i ual eye. As he ask o sco ing in ol ed disabling
enemies and his is accomplished by ocusing he
c osshai on he a ge i may indica e ha he be e a
playe becomes a supplemen ing hei own eye
mo emen wi h i ual eye mo emen he mo e
accu a ely hey a e able o ocus on he ac i i y a ound
he c osshai s. Fu he esea ch would ha e o be
ca ied ou o de e mine i his is he case.
Table 1: S a is ics Summa y
Use Sco e
N
ea Cen e
Fix. % Time in Nea
Cen e Region % A g. L. Fix.
ms Num. Fix.
Unde 300 ms Num. Fix. Be ween
300 ms – 600 ms Num. Fix. O e
600 ms
1.1 3350 97 99 847 62 100 125
2.1 3700 85 95 628 194 93 84
3.1 1200 93 96 383 368 131 83
4.1 2100 84 88 426 302 155 91
5.1 2500 93 97 723 148 76 103
6.1 1100 72 86 468 249 141 78
1.2 5800 90 98 941 97 60 97
2.2 2800 87 92 526 220 140 87
3.2 1500 91 96 424 321 144 79
4.2 1200 79 85 370 368 167 81
5.2 2150 95 96 647 157 113 91
6.2 1900 85 90 754 165 58 88
Figu e 6: A e age Le Eye Fixa ion Ve sus Sco e
4. CONCLUSIONS AND FUTURE WORK
In his pape a sys em o synch onising an eye acke
wi h a comme cial game de elopmen engine has been
desc ibed. P elimina y esul s sugges ha he sys em
success ully co- egis e s eye gaze in o ma ion wi h he
isual scene p esen ed o he use du ing hei game
play. Analysis o he da a indica es ha use s o a FPS
spend an ex emely la ge p opo ion o ime ocused
a ound he cen e o he sc een, ha he e is some
indica ion o he Rubicon heo y aking place wi hin
games and ha he e may be a co ela ion be ween ask
p o iciency and he du a ion o eye mo emen .
Fu u e wo k will in es iga e psycho-pe cep ual
phenomenon ha can be used o il e he in o ma ion
ha needs o be ansmi ed in DIAs such as ne wo ked
compu e games o main ain an adequa ely consis en
global sha ed s a e. Speci ically i he poin o he
Rubicon can be accu a ely and con inuously de e mined
hen some in o ma ion abou use ac ions could be sen
be o e hey ake place. While ou esul s ha e shown
ha i may be di icul o achie e his in a as ac ion
FPS game, o he game gen es wi h slowe in e ac ions
o an FPS wi h mo e con olled in e ac ions may p o e
easie o analyse. Fu u e expe imen s will aim o mo e
accu a ely highligh he Rubicon e ec wi hin games.
REFERENCES
B own, R., Coppe , L. and Pham, B. 2003. “Visual A en ion-
based Polygon Le el o De ail Managemen .” Fi s
In e na ional Con e ence on Compu e G aphics and
In e ac i e Techniques, Melbou ne, Aus alia.
Delaney, D., Wa d, T. and McLoone, S. 2003. “Reducing
Upda e Packe s in Dis ibu ed In e ac i e Applica ions
using a Hyb id Model.” 16 h In e na ional Con e ence on
Pa allel and Dis ibu ed Compu ing Sys ems, Reno, USA.
Gollwi ze , P.M., and Baye , U. 1999. “Delibe a i e e sus
implemen al mindse s in he con ol o ac ion.” Dual-
p ocess heo ies in social psychology (pp. 403-422). New
Yo k: Guil o d.
IEEE 1993. IEEE S anda d o Dis ibu ed In e ac i e
Simula ion - Applica ion P o ocols, in IEEE S d 1278-
1993, I.C. Socie y, Edi o . 1993, IEEE: New Yo k.
Koesling, H. and Hone , O. 2003. “Bend i like Beckham –
Mindse s and Visual A en ion in Decision-Making in
Socce .” P oc. 12 h Eu opean Con e ence on Eye
Mo emen s, Dundee, UK.
Komogo se , O., and Khan, J. 2004. “P edic i e Pe cep ual
Comp ession o Real Time Video Communica ion.”
P oc. 12 h ACM In e na ional Con e ence on Media, New
Yo k, USA.
Ma shall, D., Delaney, D., McLoone, S. and Wa d, T. 2004.
“Explo ing he Spa ial Densi y o S a egy Models in a
Realis ic Dis ibu ed In e ac i e Applica ion.”
In e na ional Symposium on Dis ibu ed Simula ion and
Real-Time Applica ions, Budapes , Hunga y.
O’Sullli an, C. and Dingliana, J. 2001. “Collisions and
Pe cep ion.” ACM T ansac ions on G aphics, Volume 20,
Issue 3.
Pan, B. e al. 2004. “The De e minan s o Web Page Viewing
Beha iou : An Eye T acking S udy.” P oc. Eye T acking
Resea ch and Applica ions Symposium, San An onio,
Texas.
Pan el, L. and Wol , L. 2002. “On he Sui abili y o Dead
Reckoning Schemes o Games.” Fi s Wo kshop on
Ne wo k and Sys em Suppo o Games, B aunschweig,
Ge many.
Sibe , L. and Jacob, R. 2000. “E alua ion o Eye Gaze
In e ac ion.” P oc. SIGCHI con e ence on Human
Fac o s, The Hague ,Ne he lands .
Singhal, S. and Zyda, M. 1999. “Ne wo ked Vi ual
En i onmen s: Design and Implemen a ion.” ACM P ess
SIGGRAPH Se ies. Addison-Wesley, Reading,
Massachuse s.
AUTHOR BIOGRAPHIES
ALAN KENNY was bo n in Mullinga , I eland and
wen o NUI, Maynoo h, whe e he s udied compu e
science and so wa e enginee ing and ob ained his
deg ee in 2004. Ha ing spen a sho s in wo king as a
web de elope he e u ned o he Uni e si y o s udy a
PhD in he a ea o psycho-pe cep ual measu es in
ne wo ked games. His e-mail add ess is:
[email p o ec ed] and his Web page can be
ound a h p://www.cs.may.ie/~akenny.
HENDRIK KOESLING g adua ed in compu e
science and physics (M.Sc.) in 1997 and ecei ed a PhD
in compu e science om Biele eld Uni e si y,
Ge many, in 2003. He wo ked as a lec u e and
esea che in he Neu oin o ma ics G oup and he
CRC360 "Si ua ed A i icial Communica o s" a
Biele eld Uni e si y un il 2004. He cu en ly holds a
pos doc o al esea ch posi ion a he Compu e Science
Depa men a NUI Maynoo h. His esea ch in e es s a e
in isual pe cep ion/a en ion and cogni i e p ocesses in
humans, based on he analysis and modeling o eye
mo emen s.
DECLAN DELANEY ecei ed his elec onic
enginee ing deg ee om Uni e si y College Dublin in

1991 and his Mas e s deg ee in 1999. He has wo ked
ab oad as bo h a so wa e enginee and p ojec manage .
A p esen he lec u es in he Compu e Science
Depa men in NUI Maynoo h. His cu en esea ch
in e es s ocus on echniques o educing la ency in
dis ibu ed ne wo ked applica ions and on he use o
small g oup PBL in so wa e enginee ing educa ion. He
is a cha e ed membe o he IEI.
SEAMUS McLOONE ecei ed his B.Eng. honou s
deg ee in Elec ical and Elec onic Enginee ing om
Queen's Uni e si y Bel as in 1996 and his Ph.D.,
en i led Nonlinea Iden i ica ion using Local Model
Ne wo ks, in Sep embe 2000. A p esen , he is
employed as a lec u e in he Depa men o Elec onic
Enginee ing in he Na ional Uni e si y o I eland (NUI),
Maynoo h. His main esea ch in e es s include mul iple
model app oaches o nonlinea sys em iden i ica ion and
educing and masking he e ec s o la ency in
dis ibu ed in e ac i e applica ions (DIAs).
TOMAS WARD ecei ed he B.E., he M.Eng.Sc. and
Ph.D deg ees om Uni e si y College, Dublin, I eland
in 1994, 1996 and 1999 espec i ely. He wo ked as
Resea ch Coo dina o in he Rehabili a ion Enginee ing
Depa men a he Na ional Rehabili a ion Hospi al om
1996-1999. F om 1999-2000 he was an Assis an
Lec u e in he Depa men o Compu e Science a he
Na ional Uni e si y o I eland, Maynoo h. In 2000 he
became a lec u e in he Depa men o Elec onic
Enginee ing a NUI Maynoo h. His main esea ch
in e es s lie in he a eas o nea -in a ed op ical and
e ahe z medical imaging, he de elopmen o
he apeu ic ools in medical ehabili a ion, emo ion
syn hesis in a i icial speech and dis ibu ed
en i onmen s o mul i-use in e ac ion.