In es iga ing Saccade-Onse Locked
EEG Signa u es o Face Pe cep ion
du ing F ee-Viewing in a Na u alis ic
Vi ual En i onmen
Debo a Nol e,
1
Vincen Schmid ,
1
Ai ana G asso-Clade a,
1
and
Pe e König
1,2
1
Ins i u e o Cogni i e Science, Uni e si y o Osnab ück, Osnab ück 49090, Ge many
and
2
Depa men o Neu ophysiology and Pa hophysiology, Uni e si y Medical Cen e
Hambu g-Eppendo , Hambu g 20246, Ge many
Abs ac
Cu en esea ch s i es o in es iga e cogni i e p ocesses unde na u al condi ions. Vi ual eali y and EEG
a e p omising echniques combining na u alis ic se ings wi h close expe imen al con ol. Howe e , many
ques ions and echnical challenges emain, e.g., a e saccade onse s a sui able eplacemen o ixa ion
onse s as key e en s in con inuous gaze ajec o ies (Amme e al., 2024), and consequen ly, can VR cap-
u e di e ences ac oss di e en s imulus ca ego ies associa ed wi h a ying saccade du a ions? To
add ess bo h ques ions, we in es iga e he N170 ace e ec in humans (14 males, 19 emales, ze o
di e se) using a ee- iewing and ee-mo emen imme si e VR s udy ha con ained houses, a ious
backg ound s imuli, and, no ably, s a ic and mo ing pedes ians o s udy ace pe cep ion unde na u alis ic
condi ions. Ou esul s show ha aligning ials o saccade onse s leads o mo e well-de ined ERPs han
ixa ion onse s, especially o he P100 componen , demons a ing ha saccade-onse ERPs a e a be e -
sui ed analysis me hod o his ype o expe imen . Fu he mo e, we obse e an e olu ion o ca ego y-
based di e ences, i.e., ace e sus backg ound saccade-onse ERPs, compa ible wi h p e ious epo s
bu ex ending in a la ge empo al window and including all elec ode si es a di e en poin s in ime. In
summa y, employing VR, EEG, and eye- acking o in es iga e di e ences ac oss ixa ion ca ego ies p o-
ides insigh s in o he ele ance o saccadic onse s as e en igge s and enhances ou unde s anding o
cogni i e p ocesses in na u alis ic se ings.
Key wo ds: ace pe cep ion; ixa ion-onse ERP; ee- iewing; N170; saccade-onse ERP; i ual eali y
Signi icance S a emen
Wi h he e o o in es iga ing and unde s anding cogni i e p ocesses unde na u alis ic condi ions,
combining i ual eali y and EEG can be ui ul in implemen ing ee- iewing s udies. The cu en
wo k combines hese echnologies o explo e key challenges in he con ex o ace pe cep ion in an
imme si e i ual en i onmen . Ou esul s show ha saccade-onse ERPs yield mo e p ecise measu e-
men s when analyzing con inuous eye- acking da a han ixa ion onse s. Fu he mo e, when p ocessing
ace compa ed wi h backg ound s imuli, dis inc empo al pa e ns encompassing all elec ode si es can
be obse ed, o e ing new insigh s in o ace pe cep ion. O e all, his wo k highligh s he po en ial o
in eg a ing VR and EEG o ad ance ou unde s anding o cogni i e p ocesses in na u alis ic se ings.
In oduc ion
In ecen yea s, a s ep has been aken o s udy and unde s and cogni i e p ocesses
unde na u al condi ions, cap u ing hem in dynamic, eal-wo ld en i onmen s (T omp
e al., 2018;Shamay-Tsoo y and Mendelsohn, 2019;Rounds e al., 2020;Ge e al.,
Con inued on nex page.
Recei ed Dec. 14, 2024; e ised July 8,
2025; accep ed July 17, 2025.
The au ho s decla e no compe ing
financial in e es s.
Au ho con ibu ions: D.N. and P.K.
designed esea ch; D.N. pe o med
esea ch; D.N., V.S., and A.G.-C.
analyzed da a; D.N., V.S., A.G.-C., and
P.K. w o e he pape .
We hank e e yone who con ibu ed o
he p ojec , specifically John Mad id-
Ca ajal, Jakob Li sch, E a on Bu le ,
Anneke Büü ma, Ma ke a Bece o a,
Reem Hjoj, and Ma ie Bensien o hei
help in collec ing he da a.
Fu he mo e, we hank Ma c Vidal De
Palol, Jakob Li sch, and Anna L. Ge
o hei suppo in designing he s udy,
A u Czeszumski o his inpu while
de eloping he au oma ed EEG
p ep ocessing pipeline, Jessica Simon
o in es iga ing di e en
p ep ocessing pa ame e s, Mo i z
Lönke o implemen ing he saccade
ampli ude calcula ions, and finally,
T acy Sánchez Pacheco o he inpu
and eedback on he isualiza ions and
TFCE analysis.
Resea ch A icle: Confi ma ion
No el Tools and Me hods
Sep embe 2025, 12(9). DOI: h ps://doi.o g/10.1523/ENEURO.0573-24.2025. 1 o 16
2022;S angl e al., 2023). A cen al aspec o his app oach a e ee- iewing pa adigms,
whe e subjec s mo e hei eyes and ac i ely choose whe e o di ec hei gazes (Ge
e al., 2022;Amme e al., 2024), allowing us o s udy he spon aneous and adap i e na u e
o eal-wo ld isual beha io (Shamay-Tsoo y and Mendelsohn, 2019;S angl e al., 2023).
Fo hese s udies, i ual eali y (VR) is eme ging as a powe ul ool, combining he high
expe imen al con ol o labo a o y se ups wi h ee- iewing expe iences o eal li e
(Bohil e al., 2011;Pan and Hamil on, 2018;Bell e al., 2020). Suppo ing he po en ial
o VR, ecen s udies demons a ed ha VR can p o ide findings simila o eal li e
(Nol e e al., 2025) and is sui able o analyzing eye- acking da a in na u alis ic en i on-
men s (Clay e al., 2019;Llanes-Ju ado e al., 2020;Nol e e al., 2024). Beyond eye mo e-
men s, in eg a ing VR wi h elec oencephalog aphy (EEG) allows o explo ing neu al
esponses o na u alis ic isual beha io (T omp e al., 2018;Rounds e al., 2020;
S angl e al., 2023) and measu ing fixa ion-onse e en – ela ed po en ials (ERPs; Nol e
e al., 2024). This highligh s he po en ial o combining VR and EEG o s udy cogni i e p o-
cesses unde na u alis ic, ee- iewing condi ions.
While VR has p o en help ul in in es iga ing ision and neu al p ocesses unde na u al-
is ic condi ions, many ques ions and echnical challenges emain. Fo ins ance, al hough
neu al p ocesses can be s udied wi h VR–EEG se ups (T omp e al., 2018;Rounds e al.,
2020;Nol e e al., 2024), he easibili y o using his combina ion o examine fixa ion-onse
ERP di e ences ac oss expe imen al condi ions emains o be explo ed. No ably, a ecen
magne oencephalog aphy (MEG) s udy in es iga ed fixa ion- and saccade-onse ERP di -
e ences du ing na u alis ic iewing o pic u es and ound saccade-onse ERPs be e
sui ed o s udying ea ly isual componen s (Amme e al., 2024). Building on hese find-
ings, a ques ion a ises: Do saccade onse s p o ide he op imal alignmen o ERP analysis
in ee- iewing s udies? Fu he mo e, i saccade onse s a e he p e e ed alignmen , can
VR cap u e di e ences ac oss s imulus ca ego ies a ying in saccade cha ac e is ics?
To add ess he fi s ques ion, we can assess he iming o a saccade-onse P100 in an
imme si e ee- iewing s udy. Employing hese saccade-onse ERPs o s udy a well-
es ablished e ec , such as he N170 ace e ec (Rossion and Jacques, 2008;Eime ,
2011), can ackle he second ques ion. The N170 e ec , desc ibed as a s onge ERP
esponse o aces (Rossion and Jacques, 2008;Eime , 2011) and bodies (Hie anen and
Nummenmaa, 2011) compa ed wi h o he s imuli, has been eplica ed in ee- iewing pic-
u e se ups (de Lissa e al., 2019;Aue bach-Asch e al., 2020;Ge e al., 2022) and wi h i -
ual humans (Whea ley e al., 2011). Thus, he N170 e ec is ideal o in es iga ing whe he
saccade onse s p o ide supe io empo al alignmen and whe he he ERPs can e eal
di e ences be ween expe imen al condi ions in an imme si e ee- iewing s udy.
The cu en expe imen was designed as a h ee-dimensional i ual ci y popula ed wi h
a a a s. Pa icipan s ad libi um explo ed he ci y cen e while we eco ded hei eye mo e-
men s and EEG signals. We explo ed he empo al alignmen o fixa ion- and
saccade-onse ERPs in line wi h p e ious esea ch (Amme e al., 2024) o de e mine he
mo e sui able op ion. Fu he mo e, o in es iga e di e ences be ween s imulus ca ego-
ies, we spli ou da a in o head, body, and backg ound s imuli (Ge e al., 2022). We
hypo hesized he highes N170 ampli ude o heads, ollowed by bodies, and he smalles
o backg ound s imuli. Ou esul s suppo ed a saccade-onse alignmen . Noise-le el di -
e ences ac oss s imulus ca ego ies p e en ed di ec ly es ing he N170 e ec ; ins ead, a
mass uni a ia e analysis e ealed di e ences be ween all h ee ca ego ies, pa ially sup-
po ing ou hypo hesis. O e all, hese esul s unde line he sui abili y o combining EEG
wi h VR bu highligh new me hodological challenges o ee- iewing s udies.
Ma e ials and Me hods
Subjec s. O e all, 61 subjec s we e in i ed o he lab o he expe imen . We could no
s a he eco ding o wo subjec s due o echnical di ficul ies. Ou o he emaining 59 sub-
jec s, a o al o 26 subjec s we e excluded: 5 qui due o mo ion sickness, 3 subjec s we e
excluded as hey did no ollow ask ins uc ions and le he cen al squa e o >10% o he
ime, and 18 had o be excluded due o da a issues ha occu ed du ing o a e he eco d-
ing including 8 subjec s ha we e excluded due o unsynch onized d i s be ween o wi hin
mul iple da a s eams eco ded. A e applying a conse a i e app oach o da a inclusion o
main ain high da a quali y, he final da ase included 33 subjec s (19 emales, ze o di e se;
mean age 22.63 ± 2.48 yea s). All subjec s had no mal o co ec ed o no mal ision, did no
Funding was p o ided by he Ge man
Fede al Minis y o Educa ion and
Resea ch o he p ojec SIDDATA
(Indi idualiza ion o S udies h ough
Digi al, Da a-D i en Assis an s), FKZ
16DHB2123, and by he EU Ho izon
2020 (MSCDA) esea ch and inno a ion
p og am unde G an Ag eemen
Numbe 861166 (INTUITIVE). This
esea ch was also suppo ed by he
Uni e si y o Osnab ück in coope a ion
wi h he Deu sche
Fo schungsgemeinscha (DFG,
Ge man Resea ch Founda ion) in he
con ex o unding he Resea ch
T aining G oup “Si ua ed Cogni ion”
unde he P ojec Numbe GRK
274877981.
Co espondence should be add essed
o Debo a Nol e a debo a.nol e@uni-
osnab ueck.de.
Copy igh © 2025 Nol e e al.
This is an open-access a icle
dis ibu ed unde he e ms o he
C ea i e Commons A ibu ion 4.0
In e na ional license, which pe mi s
un es ic ed use, dis ibu ion and
ep oduc ion in any medium p o ided
ha he o iginal wo k is p ope ly
a ibu ed.
Resea ch A icle: Confi ma ion 2 o 16
Sep embe 2025, 12(9). DOI: h ps://doi.o g/10.1523/ENEURO.0573-24.2025. 2 o 16
epo any neu ological diso de s, ga e w i en in o med consen be o e pa icipa ing, and we e ewa ded wi h mone a y
compensa ion o pa icipa ion hou s. The e hics commission o he Uni e si y o Osnab ück app o ed he s udy.
Expe imen al se up. A de ailed desc ip ion o he da a and expe imen al design can be ound in p e ious publica ions
(Nol e e al., 2024,2025). Below, we p o ide he essen ial aspec s ele an o he cu en s udy. The expe imen was de el-
oped using Uni y3D (Uni y Technologies, 2021) e sion 2019.4.21 1, employing he buil -in Uni e sal Rende Pipeline/Unli
wi h one cen al ligh sou ce. To main ain pe cep ual consis ency, we minimized shaded a eas. The i ual en i onmen
was displayed a a cons an 90 Hz ame a e ia he HTC Vi e P o Eye head-moun ed display (HMD; 110° field o
iew, esolu ion 1,440 × 1,600 pixels pe eye, e esh a e 90 Hz; HTC Co po a ion, 2018a). The ad an age o he HTC
Vi e P o Eye HMD is he in eg a ed Tobii eye- acke (0.5–1.1° accu acy, 110° field o iew), allowing us o ac i ely eco d
he subjec ’s eye mo emen s. Eye- acking was acili a ed using he SRanipal SDK ( 1.1.0.1; HTC Co po a ion, 2018b),
and spa ial acking was p o ided by he HTC Vi e Ligh house 2.0 sys em (HTC Co po a ion, 2018c). Pa icipan s mo ed
wi hin he i ual ci y using HTC Vi e con olle s 2.0 (HTC Co po a ion, 2018d; senso y eedback disabled), wi h he di ec-
ion o mo emen de e mined by he head o ien a ion. The da a we e eco ded on an Alienwa e Au o a Ryzen compu e
(Windows 10, 64-bi , build 19044, 6553 MB RAM; N idia RTX 3090 GPU, d i e e sion 31.0.15.2698; AMD Ryzen 9 3900X
12-Co e CPU). Simul aneously, EEG da a using a 10/20 64-channel Ag/AgCl-elec ode sys em wi h a Wa egua d cap
(ANT Neu o) and a Re a8 (TMSi) amplifie we e eco ded using he OpenVIBE acquisi ion se e ( 2.2.0; Rena d e al.,
2010) on a Dell P ecision 5820 Towe (Windows 10, 64 bi , build 19044; N idia RTX 2080 Ti GPU, d i e e sion
31.0.15.1694; In el Xeon W-2133 CPU). The EEG da a we e collec ed a 1,024 Hz wi h an a e age e e ence and a g ound
elec ode unde he le colla bone. Impedances we e kep below 10 kΩ. Synch oniza ion be ween he EEG and VR sys-
ems was achie ed using he LabS eamingLaye (LSL; Ko he, 2014). Th oughou he expe imen , pa icipan s we e
sea ed on a swi el chai o allow ull 360° body o a ion (Fig. 1A).
The expe imen al p ocedu e. The en i e expe imen las ed 2.5 h. A fi s a i al, pa icipan s filled ou in o med consen
shee s and ecei ed ins uc ions abou he expe imen . Following his, pa icipan s unde wen a 1 min mo ion sickness
es in he same i ual en i onmen bu in an un eachable pa o he ci y. They we e ins uc ed o mo e owa d a ed sphe e
a he end o a s ee . Only pa icipan s who epo ed no discom o o mo ion sickness a e his es p oceeded o he main
expe imen . Following his ini ial es , he EEG sys em was se up (see sec ion “EEG p ep ocessing” o de ails), which ook
mos o he ime. Finally, he main expe imen began wi h he eye- acke ’s calib a ion and subsequen fi e-poin alida ion.
The expe imen al du a ion las ed ∼40 min. The pa icipan s had 30 min o ad libi um explo e he cen al ci y squa e
(Fig. 1B, beige iles) unde he ins uc ion o beha e na u ally as i wai ing o a iend. E e y 5 min, he explo a ion was
paused o eye- acke alida ion and ecalib a ion and o he pa icipan o ake a b eak i needed. A e each pause, pa -
icipan s we e e u ned o hei p e ious loca ion in he ci y.
The i ual en i onmen . The i ual en i onmen was modeled o esemble a ci y cen e , popula ed wi h a ious back-
g ound objec s (e.g., buildings, oliage; Fig. 1B) and 140 pedes ians (Fig. 1C). Pedes ians we e sou ced om he Adobe
Figu e 1. Expe imen al se up. A, Pa icipan s we e sea ed on a swi el chai , wea ing an EEG cap and VR glasses. The EEG equipmen , specifically he
amplifie , was s o ed on he back o he chai . B, The walkable a ea was confined o he beige floo and comp ised he cen e o he VR scene.
C, Di e en pedes ians we e dis ibu ed h oughou he ci y squa e.
Resea ch A icle: Confi ma ion 3 o 16
Sep embe 2025, 12(9). DOI: h ps://doi.o g/10.1523/ENEURO.0573-24.2025. 3 o 16
Mixamo collec ion (Mixamo, 2008), displaying a ied ac i i y and anima ion le els anging om s a iona y and s a ic o
ac i ely mo ing h oughou he ci y. The pedes ians we e designed o ep esen ypical beha io s such as shopping,
mee ing iends, o elaxing on benches. The pedes ians did no eac o he pa icipan s o he han a oiding mo emen
collisions. The ac i e-mo ing pedes ians mo ed along p edefined pa hs. Each objec in he i ual en i onmen had a col-
lide , an in isible box o sphe e ma king he ou line o an objec , a ached o hem, wi h pedes ians ha ing a sepa a e
collide o hei heads and he es o hei bodies. This allowed us o sepa a ely in es iga e he neu al esponse owa d
he heads and bodies o he i ual a a a s. Pa icipan mo emen s wi hin he i ual en i onmen we e p og ammed o
mimic eal-li e displacemen s con olled by he pa icipan s’head o ien a ion and ma ched in speed o he mo ing pedes-
ians. The dimensions o he i ual ci y ma ched he eal wo ld, wi h one uni y uni co esponding o 1 m, allowing us o
indica e dis ances using me e s.
Using gaze e en s o de e mine EEG ial onse s. We eco ded EEG and eye- acking simul aneously o use he iming o
gaze e en s (fixa ions o saccades) as ial ma ke s. Due o he absence o ex e nal s imulus onse s (o compa able
e en s), we conside he da a eco ded du ing fixa ion (o saccade) and i s immedia e empo al con ex as a “ ial.”
This allowed us, o example, o in es iga e fixa ion ERPs (Dimigen, 2020;Ge e al., 2022). To his end, accu a e de ec ion
o e en onse s (fixa ions and saccades) in he eye- acking da a was essen ial. The e o e, we employed a eloci y-based
eye– acking algo i hm o ee- iewing and ee-explo a ion in a i ual en i onmen , which co ec s ansla ional mo e-
men in o ma ion supe imposed on he eye mo emen da a (Nol e e al., 2024; based on Voloh e al., 2020;Da e al., 2021;
Kesha a e al., 2023). Applying his algo i hm allowed us o di e en ia e be ween gazes (eye-s abiliza ion mo emen s,
om now on, simply e e ed o as fixa ions) and saccades. In de ail, he con inuous eye- acking da a we e segmen ed
in o smalle in e als (Da e al., 2021), and a da a-d i en h eshold was calcula ed o each o hese in e als (Voloh e al.,
2020;Kesha a e al., 2023). Consecu i e samples exceeding his h eshold we e classified as a saccade, and samples
below he h eshold we e classified as fixa ions. This p ocess esul ed in a sequen ial iden ifica ion o saccades and fix-
a ions h oughou he en i e eco ding. These e en s could hen be used as ial onse s o he EEG analysis. Specifically,
we compa ed ERPs aligned o fixa ion and o saccade onse , whe e he la e used he saccade onse p eceding each
fixa ion as he ial onse (Amme e al., 2024). This app oach allowed us o compa e iden ical ials, di e ing by a ime shi :
he ime poin ze o in saccade-onse ials happened se e al milliseconds be o e he co esponding ime poin o
fixa ion-onse ials. Consequen ly, i e en s, such as small saccades and he ma ching subsequen fixa ion onse s,
we e no de ec ed, hey would be p esen in and a ec bo h ypes o ERPs simila ly. The ials o bo h fixa ion- and
saccade-onse ERPs we e spli in o h ee dis inc s imulus ca ego ies: heads, co esponding o fixa ions on he heads
o pedes ians, bodies, and backg ound s imuli, encompassing e e y hing ha was no a pedes ian, allowing us o in es-
iga e he p esence o an N170 e ec in a ee- iewing expe imen conduc ed in VR. Fo saccade-onse ERPs, we used he
s imulus ca ego y o he fixa ion di ec ly succeeding he saccades. A sequence o a pa icipan ’s walking pa h and a ew
selec ed fixa ions can be seen in Figu e 2A.
Tempo al alignmen o EEG and eye- acking da a. Aligning he EEG and eye- acking da a wo ked; howe e , isual
inspec ion indica ed a small cons an linea d i be ween he o e all EEG and eye- acking (uni y) imelines. To co ec
his d i , we calcula ed he di e ence be ween he fi s EEG and eye- acking imes amps and be ween he las ones,
compu ed he de ia ion be ween hese di e ences, and applied i linea ly o he eye- acking imeline (Nol e e al.,
2024). Twen y-one subjec s displayed a mo e subs an ial d i , equi ing he s a –end de ia ion up o ou imes o o
adjus he imeline by one (11 ms) o wo (22 ms) sample(s), acco ding o he 90 Hz sampling a e, o e he cou se o a
30 min expe imen al session. No ably, his d i co ec ion was iden ical o fixa ion and saccade onse s. The final da ase
only included subjec s o which we we e confiden in aligning he wo da a s eams (also see abo e, Subjec s).
EEG p ep ocessing. P ep ocessing was pe o med in MATLAB (R2024a) using he EEGLab so wa e (Delo me and
Makeig, 2004; e sion 2020.0). EEG da a we e fi s loaded in o MATLAB, channels we e enamed acco ding o he
10-5 BESA s anda d sys em, and emp y channels we e emo ed. We hen impo ed a sepa a e igge file con aining
all ele an fixa ion o saccade-onse e en s de i ed om ou eye- acking da a (see abo e, Using gaze e en s o de e -
mine EEG ial onse s, o a de ailed explana ion). We applied a low-pass fil e a 128 Hz and a high-pass fil e a 0.5 Hz
(pop_eegfil new, using a hamming window; Widmann e al., 2015). Following he ecommenda ion o Klug and
Kloos e man (2022), we downsampled he EEG da a om 1,024 o 500 Hz o apply a line noise fil e om he “zapline
plus”plugin (Klug and Kloos e man, 2022, based on de Che eigné, 2020). We conduc ed his p ocedu e o au oma ically
emo e spec al peaks ∼50 Hz and, sepa a ely, 90 Hz. Then, ensu ing he da a we e e e enced o he a e age e e ence,
we applied au oma ed cleaning o noisy channels and da a segmen s using he “clean_ awda a”plugin (Ko he e al.,
2019). Ou da a included ac i e mo emen and con ained mo e noise han expec ed in a classic s a iona y labo a o y
se up. To his end, we chose o apply a conse a i e bu s c i e ion o 20, e e ing o he s anda d de ia ion cu o o
he emo al o bu s s ia a i ac subspace econs uc ion. Remo ed noisy segmen s we e sa ed o be used by he un old
oolbox (Ehinge and Dimigen, 2019; o a de ailed desc ip ion, see below, EEG analysis). A e channel emo al, he clean
da ase was e e e enced o he a e age e e ence once mo e. Using he AMICA plugin ( e sion 15, Palme e al., 2012), we
pe o med an independen componen analysis (ICA) on he cleaned da a o iden i y and emo e muscle, eye, hea , o
Resea ch A icle: Confi ma ion 4 o 16
Sep embe 2025, 12(9). DOI: h ps://doi.o g/10.1523/ENEURO.0573-24.2025. 4 o 16
emaining line o channel noise. Fo his s ep only, we high-pass fil e ed ou da a o 2 Hz (Dimigen, 2020). Componen s
labeled wi h 80% muscle ac i i y o abo e (mean, 16 componen s; SD, 7.407) o >90% o o he noise (ocula mo emen ,
mean, 2.121; SD, 0.331; channel noise, mean, 0.303; SD, 0.529; ca diac a i ac , mean, 0.060; SD, 0.242; line noise, mean,
0.030; SD, 0.174), as iden ified by ICLabel (Pion-Tonachini e al., 2019), we e emo ed au oma ically. ICA weigh s we e
hen ans e ed o he da ase fil e ed a 0.5 Hz. Finally, we in e pola ed he missing channels (sphe ical in e pola ion).
The desc ibed p ocedu e was epea ed o all subjec s be o e we applied u he s a is ical analysis.
EEG analysis. Fi s , we analyzed and compa ed ERPs aligned o fixa ion and saccade onse s, in es iga ing he ERP
wa e o ms o −300 o 500 ms su ounding each e en o indi idual subjec s and he a e aged ERPs ac oss subjec s.
Nex , o accoun o and co ec he e ec o o e lapping e en s due o ou ee- iewing pa adigm, we used a linea
model implemen ed by he un old oolbox (Ehinge and Dimigen, 2019) wi h he cu en e en ac o and he le els o back-
g ound, body, and head. This o e lap co ec ion was applied o −500 up o 1,000 ms su ounding saccade onse s (Ge
e al., 2022). As we in es iga e di e ences in saccade-onse ERPs, we did no model saccade ampli udes due o hei high
co ela ion wi h saccade du a ions (Ha is and Wolpe , 2006;Guad on e al., 2022).
To in es iga e di e ences ac oss ca ego ies a all elec odes and ime poin s (−500 o 1,000 ms su ounding saccade
onse ), we conduc ed a one- ac o epea ed–measu e ANOVA (1 × 3: head, body, and backg ound), wi h an alpha le el se
a 0.05. To accoun o he mul iple-compa ison p oblem, we applied a clus e -based pe mu a ion es inco po a ing
h eshold- ee clus e enhancemen (TFCE), as implemen ed ia he ep _TFCE MATLAB oolbox (Mensen and Kha ami,
2013). We pe o med 10,000 pe mu a ions, andomizing da a ac oss he h ee ac o s o each pe mu a ion, ollowed
by a one- ac o epea ed–measu e ANOVA. The esul ing F alues we e enhanced using TFCE (pa ame e s E= 0.666;
H= 1) based on ecommenda ions o Fs a is ics (Mensen and Kha ami, 2013). This p ocess gene a ed an empi ical
null dis ibu ion (H0) o TFCE-enhanced F alues, and he maximum F alue ac oss channels and ime poin s o each
Figu e 2. Dis ibu ion o gaze e en s. A, An example o di e en fixa ions on di e en objec s is plo ed on op o he co esponding image o he ci y cen e .
As a no e, he da a we e sligh ly adjus ed o isualiza ion pu poses only. The black line co esponds o he pa icipan ’s mo emen pa h, and he a ows
co espond o a ew selec ed fixa ions du ing his du a ion. A fixa ion on he ace o a pedes ian is highligh ed in ed. The blue a ows co espond o fix-
a ions di ec ed a di e en backg ound objec s. B–D, The dis ibu ion o backg ound (B), body (C), and head (D)fixa ions o e ime, displayed as cumu-
la i e dis ibu ion unc ions. Each line co esponds o one pa icipan .
Resea ch A icle: Confi ma ion 5 o 16
Sep embe 2025, 12(9). DOI: h ps://doi.o g/10.1523/ENEURO.0573-24.2025. 5 o 16

pe mu a ion was eco ded. The obse ed TFCE-enhanced F alues we e hen compa ed wi h he empi ical dis ibu ion,
wi h s a is ical significance de e mined as alues exceeding he 95 h pe cen ile o he null dis ibu ion.
Assessmen o ace s imuli cha ac e is ics. To alida e ou s imuli, we conduc ed an online su ey wi h a sepa a e g oup
o 12 pa icipan s (eigh emales, ze o di e se; mean age, 31.62 ± 13.30 yea s). A o al o 40 andomly selec ed acial
images we e shown: 10 o ou a a a s and 30 images selec ed om Go lini e al. (2023), consis ing o 10 each om h ee
ca ego ies: un ealis ic, semi ealis ic, and ealis ic aces. Pa icipan s a ed each image on h ee indices using a alida ed
ques ionnai e de eloped by Ho and MacDo man (2010): humanness (six i ems), ee iness (eigh i ems), and a ac i eness
( ou i ems), wi h seman ic di e en ial i ems assessed using a fi e-poin Like scale. Fo all s imulus ca ego ies, we cal-
cula ed a e age sco es o each pa icipan o each index. S a is ical di e ences we e e alua ed using a sepa a e
F iedman es pe index.
Code accessibili y. The code desc ibed in he pape is eely a ailable online a h ps://gi hub.com/debnol e/saccade-
onse _ERPs_o - ace_pe cep ion_ ee- iewing_VR. The code is a ailable as Ex ended Da a.
Resul s
Gaze e en s
Be o e analyzing ERPs, we fi s compa ed he di e en gaze e en s by examining he median and median absolu e
de ia ion (MAD) ac oss a ious aspec s o each ca ego y. One clea di e ence be ween he h ee ca ego ies—back-
g ound s imuli, bodies, and heads—was he numbe o ials. Backg ound s imuli had he mos ials, wi h a median o
3,968 ± 492. No ably, he body and head ca ego ies had ∼10 imes ewe ials han he backg ound ca ego y, wi h bodies
a e aging 741 ± 221 ials and heads 151 ± 133 ials. Al hough he e was conside able be ween-subjec a ia ion in he
numbe o ials o bo h he body (min, 290; max, 1,296) and head (min, 17; max, 913) ca ego ies, he numbe o fixa ions
di ec ed a bodies and heads was no significan ly co ela ed ac oss pa icipan s ( = 0.184; p= 0.305). This indica ed ha i
was no simply ha some subjec s gazed a pedes ians o e all mo e o less; ins ead, some subjec s ocused mo e on
heads, while o he s di ec ed mo e fixa ions owa d bodies. Despi e he di e ences in ial coun s, fixa ions in all h ee ca -
ego ies we e equally dis ibu ed ac oss he en i e expe imen al du a ion (see cumula i e dis ibu ion unc ions, Fig. 2B–D).
This balanced dis ibu ion was essen ial, enabling compa ison ac oss he h ee ca ego ies wi hou adjus ing o di e -
ences in expe imen al du a ion o pa icipan a igue. When examining he median and MAD o e en du a ions, fixa ion
du a ions we e simila ac oss all h ee ca ego ies: backg ound s imuli (0.186 s ± 0.013), bodies (0.2 s ± 0.016), and heads
(0.178 s ± 0.023). In con as , saccade du a ions di e ed, wi h backg ound s imuli ha ing he la ges saccade du a ions
(0.076 s ± 0.003), ollowed by bodies (0.066 s ± 0.001) and heads wi h he sho es saccade du a ions (0.056 s ± 0.015).
A simila pa e n eme ged o saccade ampli udes, whe e saccades owa d backg ound s imuli had he highes ampli-
udes (12.119° ± 2.104), ollowed by bodies (7.212° ± 1.618) and heads (5.35° ± 1.491). These findings highligh ed ha ,
despi e simila i ies in fixa ion du a ions and hei empo al dis ibu ion, he di e en s imulus ca ego ies we e associa ed
wi h di e en saccade pa e ns and a ying numbe s o e en s.
Nex , o in es iga e whe he fixa ion and saccade onse s a e subjec o a bias inhe en in he eye mo emen classifica-
ion algo i hm, we analyzed he dis ibu ion o eye mo emen eloci ies a e en onse s (Fig. 3). Specifically, we examined
Figu e 3. Va ia ion o eloci ies a ound e en onse s. The de ia ion o angula eloci ies su ounding e en onse s is displayed o (A) backg ound, (B) body,
and (C) head ials. The x-axis displays samples a ound e en onse s, wi h he fixa ion and saccade onse s aligned and ma ked by a black line. Veloci ies
aligned o fixa ion onse s a e shown in ed, while hose aligned o saccade onse s a e displayed in blue. Each line co esponds o one pa icipan , display-
ing he s anda d de ia ion ac oss all ials.
Resea ch A icle: Confi ma ion 6 o 16
Sep embe 2025, 12(9). DOI: h ps://doi.o g/10.1523/ENEURO.0573-24.2025. 6 o 16
he a iabili y ac oss ials. I we assume ha e en onse s a e well defined, we would expec o obse e a low a iabili y o
eloci ies a ound hese onse s. In con as , i he e en onse s a e no clea ly defined, we would an icipa e a highe a -
iabili y. By compa ing he a iabili y o eloci ies a fixa ion and saccade onse s, we aimed o de e mine whe he ou clas-
sifica ion algo i hm was mo e p ecise in defining one ype o e en o e he o he . As shown in Figu e 3, he eloci ies a
fixa ion onse s exhibi a ela i ely low a iabili y wi hin and also ac oss subjec s. The eloci ies a saccade onse s display a
simila bu ime-shi ed dis ibu ion, wi h low a iabili y wi hin and ac oss subjec s, one sample be o e he saccade onse
ollowed by a high a iabili y a he saccade onse . Fu he mo e, he dis ibu ions o da a o ei he e en do no o e lap.
These obse a ions p o ide suppo ha he classifica ion o fixa ions and saccades is no influenced by an ob ious bias.
Compa ing fixa ion- and saccade-onse ERPs
In in es iga ing ERPs, we fi s examined single subjec s o compa e he di e ence be ween fixa ion and saccade-onse
ERPs, ollowing he app oach o Amme e al. (2024). Fo isualiza ion pu poses, we selec ed h ee ep esen a i e subjec s
(Fig. 4). The fi s subjec (Fig. 4A,B) had 4,620 ials, spli in o 3,779 backg ound, 747 body, and 194 head ials. The second
subjec (Fig. 4C,D) had 3,802 backg ound, 609 body, and 151 head ials, while he hi d subjec (Fig. 4E,F) had 3,797
backg ound, 792 body, and 361 head ials. To compa e he di e ence be ween fixa ion- and saccade-onse ERPs, we
so ed each subjec ’sfixa ion-onse ials by he du a ion o he p eceding saccade, in line wi h Amme e al. (2024).
The EEG da a we e aligned and epoched using fixa ion onse s and hen o de ed based on saccade du a ions.
Figu e 4, A,C, and E, shows he esul s: fixa ions onse s a e ma ked by he s aigh black lines a ze o, while saccade
onse s a e indica ed as he p eceding cu ed black lines. I fixa ion onse s we e he op imal alignmen poin s, we would
expec he P100 ampli ude peaks o o m a s aigh line 100 ms a e he fixa ion onse . Howe e , ac oss all h ee subjec s,
he P100 ampli ude peaks ollowed he cu ed saccade-onse ajec o y, sugges ing ha saccade onse s p o ide mo e
sui able ime poin s o aligning indi idual ials in ou ee- iewing expe imen . In e es ingly, ials wi h e y sho saccades
isually di e om longe saccades, po en ially due o smalle changes o he isual inpu , smoo hing, o an o e lap o
saccadic and fixa ion ac i i y. The no ion ha saccade-onse ERPs we e mo e empo ally p ecise was u he suppo ed
by ime-shi ed bu highe P100 ampli udes compa ed wi h smalle , mo e smea ed-ou fixa ion–onse ERPs (Fig. 4B,D,F).
No ably, he saccade-onse ERP wa e o m o he head ca ego y had highe noise le els han he backg ound and body
ca ego ies. O e all, he single-subjec esul s suppo ed he idea ha saccade-onse ERPs migh be a be e -sui ed anal-
ysis me hod han fixa ion-onse ERPs o his ype o expe imen .
Nex , we in es iga ed he di e ence be ween fixa ion- and saccade-onse ERPs ac oss subjec s. Fo his, we fi s a e -
aged wi hin subjec s o accoun o he high a iabili y o head ials and hen a e aged ac oss subjec s. Fixa ion-onse
ERPs (Fig. 5A, do ed lines) show a b oad P100 componen ac oss all h ee s imulus ca ego ies, wi h backg ound s imuli
e oking he highes P100 peak and head s imuli he lowes . In compa ison, saccade-onse ERPs (Fig. 5A, solid lines) a e
shi ed in ime bu exhibi highe ampli udes ac oss all s imulus ca ego ies and P100 peaks ha a e mo e empo ally
ocused. In e es ingly, he di e ences in P100 peaks be ween he s imulus ca ego ies seen in fixa ion-onse ERPs disap-
pea wi h saccade-onse alignmen . Like he single-subjec esul s, he head ca ego y appea s o be he noisies , ega d-
less o he alignmen . The opog aphical analysis ac oss all channels (Fig. 5B,C) suppo ed his obse a ion, highligh ing
ha he p e e ence o saccade-onse ERPs is no es ic ed o only he single, selec ed elec ode. ERPs aligned o he
saccade onse elici highe ampli udes ac oss occipi al elec odes han hose aligned o he fixa ion onse . These findings
suppo he usage o saccade-onse e sus fixa ion-onse alignmen in EEG analyses due o hei impac on he o e all
ERP cu e. Saccade onse s lead o mo e well-defined ERPs, especially conce ning he P100 componen .
To s a is ically compa e fixa ion- and saccade-onse ERPs, we ollowed he app oach o Amme e al. (2024), g ouping
he da a in o 10 equally sized bins based on saccade du a ion. Wi hin each bin, ERPs we e a e aged fi s wi hin pa ici-
pan s and hen ac oss pa icipan s. We iden ified he hal -maximum poin o he P100 slope o each binned ERP wa e-
o m. We hen calcula ed he s anda d de ia ion o he hal -maximum ime poin s ac oss bins, sepa a ely o fixa ion- and
saccade-aligned condi ions. The s anda d de ia ion was 19.82 ms o fixa ion-onse ERPs and 8.32 ms o saccade-onse
ERPs. A pai ed-sample es confi med ha his di e ence was s a is ically significan (
(9)
= 7.828; p< 0.0001), indica ing
ha saccade-onse alignmen yields a mo e empo ally s able es ima e o he P100 componen .
Compa ing head, body, and backg ound ials o saccade-onse ERPs
Be o e in es iga ing he p esence o an N170 e ec , ypically associa ed wi h he pe cep ion o aces (Rossion and
Jacques, 2008;Eime , 2011), we assessed ha agg ega ing all fixa ions on heads, i espec i e o iewing angles, does
no obscu e any po en ial di e ences. To his end, inspec ing he iewing angle dis ibu ion ac oss pa icipan s
(Fig. 6A) e ealed ha mos fixa ions a e di ec ed owa d pedes ians’ aces. Specifically, compu ing he ci cula mean
wi hin pa icipan s, ollowed by he ci cula mean and he ci cula s anda d de ia ion ac oss pa icipan s, esul ed in a e -
age iewing angles o 13.210 ± 30.868°. Addi ionally, o exclude iewing dis ance as a po en ial influence, we compu ed
he median and MAD ac oss pa icipan s (5.209 ± 2.591 m; Fig. 6B), indica ing ha mos heads a e iewed a a close
ange. When we in es iga ed he saccade-onse ERPs o fixa ions on heads (151 ± 133 ials) compa ed wi h hose on
only aces (72 ± 64 ials), no isible di e ences o he han a sligh inc ease in noise le els eme ged, sugges ing ha agg e-
ga ing all head fixa ions is alid. Only inspec ing on al aces iewed a a close dis ance (<5 m) esul ed in ewe ials
(39 ± 38 ials) and an ERP wi h high noise le els, making he ERP cu e challenging o in e p e . O e all, ou esul s
Resea ch A icle: Confi ma ion 7 o 16
Sep embe 2025, 12(9). DOI: h ps://doi.o g/10.1523/ENEURO.0573-24.2025. 7 o 16
indica e no disce nible di e ences elici ed by iewing angles o dis ances, confi ming ha agg ega ing head fixa ions
is app op ia e.
To examine ca ego y-specific di e ences o saccade-onse ERPs, we fi s ocused on channels discussed in p e ious
li e a u e (Ge e al., 2022), pa icula ly PO7 (Fig. 7A) and PO8 (Fig. 7B). Compa ing he decon olu ed po en ials ac oss
Figu e 4. Fixa ion- and saccade-onse ERPs o a single subjec . A, All ials o one subjec a elec ode PO7, aligned o fixa ion onse , a e so ed acco ding
o saccade du a ion, wi h he fi s ials ( op o he y-axis) ha ing he longes saccade du a ions. Red indica es posi i e, and blue indica es nega i e ampli-
udes. The ials a e plo ed o e ime. The black e ical line co esponds o he fixa ion and, he e o e, ial onse . The ials a e smoo hed o isualiza ion
wi h a Gaussian fil e o 2 in he ydi ec ion and a Gaussian fil e o 5 in he xdi ec ion. The black-do ed line ep esen s he saccade onse in each ial.
B, Fixa ion- (do ed lines) and saccade-onse (solid lines) ERPs o he same subjec o elec ode PO7. The di e en ca ego ies, backg ound, body, and
head, a e indica ed by he di e en colo s. C,D, The same plo s o a second and (E,F) o a hi d subjec .
Resea ch A icle: Confi ma ion 8 o 16
Sep embe 2025, 12(9). DOI: h ps://doi.o g/10.1523/ENEURO.0573-24.2025. 8 o 16
ca ego ies a hese channels e ealed no isible disc epancies, wi h only minimal, i any, nega i e deflec ion ollowing he
ini ial posi i e peak, a ime-shi ed P100. In con as , o he elec odes displayed no able di e ences be ween ca ego ies.
Fo ins ance, a elec ode P2 (Fig. 7C), he head ace di e ged om he body and backg ound aces a e he saccade
onse un il eaching peak ampli udes ∼150 ms. Simila ly, a on al si es such as F7 (Fig. 7D), dis inc ions be ween he
head compa ed wi h bo h backg ound and body ca ego ies we e isible a ound he fixa ion onse , ∼50–80 ms a e he
saccade onse . No ably, ac oss all ou elec odes, he head ca ego y exhibi ed a highe noise le el and mo e conside able
be ween-subjec a iabili y han he o he wo ca ego ies, mos likely caused by he lowe numbe o ials. This a iabili y
o he head ca ego y and he obse ed opog aphical dis inc ions equi ed a s a is ical app oach beyond adi ional mea-
su es such as peak- o-peak compa isons.
Figu e 5. Fixa ion- e sus saccade-onse ERPs. A, Ac oss-subjec ERPs a channel PO7 o all h ee ca ego ies o he wo di e en onse s: fixa ion-onse
ERPs shown wi h do ed lines, saccade-onse ERPs wi h solid ones. B,C, Topoplo s ac oss all ials i espec i e o he s imulus ca ego y ( he a e age o all
backg ound, body, and head ials) o (B)fixa ion-onse and (C) saccade-onse ERPs, shown o h ee dis inc ime poin s: be o e he P100, a ound he
P100, and a he N170. Fo isualiza ion pu poses, we selec ed ime in e als based on he isual inspec ion o fixa ion- and saccade-onse ERPs, main-
aining a cons an di e ence be ween fixa ion and saccade onse s.
Resea ch A icle: Confi ma ion 9 o 16
Sep embe 2025, 12(9). DOI: h ps://doi.o g/10.1523/ENEURO.0573-24.2025. 9 o 16
Rubo M, Kä hne I, Munsch S (2023) A en ion o aces in images is
associa ed wi h pe sonali y and psychopa hology. PLoS One 18:
e0280427.
Sánchez Pacheco T, Sa ia Mosque a M, Gä ne K, Schmid V, Nol e
D, König SU, Pipa G, König P (2025) The impac o human agen s
on spa ial na iga ion and knowledge acquisi ion in a i ual en i-
onmen . F on Vi ual Real 6:1497237.
Schindle S, Zell E, Bo sch M, Kissle J (2017) Di e en ial e ec s o
ace- ealism and emo ion on e en - ela ed b ain po en ials
and hei implica ions o he uncanny alley heo y. Sci Rep
7:45003.
Shamay-Tsoo y SG, Mendelsohn A (2019) Real-li e neu oscience: an
ecological app oach o b ain and beha io esea ch. Pe spec
Psychol Sci 14:841–859.
Spie ing L, Dimigen O (2025) (Mic o)saccade- ela ed po en ials
du ing ace ecogni ion: a s udy combining EEG, eye- acking,
and decon olu ion modeling. A en Pe cep Psychophys
87:133–154.
S angl M, Maoz SL, Su hana N (2023) Mobile cogni ion: imaging he
human b ain in he ‘ eal wo ld’. Na Re Neu osci 24:347–362.
T omp J, Pee e s D, Meye AS, Hagoo P (2018) The combined use o
i ual eali y and EEG o s udy language p ocessing in na u alis ic
en i onmen s. Beha Res Me hods 50:862–869.
Uni y Technologies (2021) Uni y3D ( e sion 2019.4.21 1).
Voloh B, Wa son MR, Konig S, Womelsdo T (2020) MAD saccade:
s a is ically obus saccade h eshold es ima ion ia he median
absolu e de ia ion. J Eye Mo Res 12:3.
Whea ley T, Weinbe g A, Loose C, Mo an T, Hajcak G (2011) Mind
pe cep ion: eal bu no a ificial aces sus ain neu al ac i i y
beyond he N170/VPP. PLoS One 6:e17960.
Widmann A, Sch öge E, Maess B (2015) Digi al fil e design o elec-
ophysiological da a –a p ac ical app oach. J Neu osci Me hods
250:34–46.
Wu z RH, Joine WM, Be man RA (2011) Neu onal mechanisms o
isual s abili y: p og ess and p oblems. Philos T ans R Soc Lond
B Biol Sci 366:492–503.
Resea ch A icle: Confi ma ion 16 o 16
Sep embe 2025, 12(9). DOI: h ps://doi.o g/10.1523/ENEURO.0573-24.2025. 16 o 16