Neu al Ne wo ks 152 (2022) 150–159
Con en s lis s a ailable a ScienceDi ec
Neu al Ne wo ks
jou nal homepage: www.else ie .com/loca e/neune
Spa io empo al CNN wi h Py amid Bo leneck Blocks: Applica ion o
eye blinking de ec ion
S.E. Bekhouche b,c, I. Kajo b, Y. Ruichek b, F. Do naika a,c,d,∗
aSchool o Compu e and In o ma ion Enginee ing, Henan Uni e si y, Kai eng, China
bCIAD, Uni e si y Bou gogne F anche-Com é, UTBM, F-90010 Bel o , F ance
cUni e si y o he Basque Coun y UPV/EHU, San Sebas ian, Spain
dIKERBASQUE, Basque Founda ion o Science, Bilbao, Spain
a icle in o
A icle his o y:
Recei ed 28 No embe 2021
Recei ed in e ised o m 14 Feb ua y 2022
Accep ed 11 Ap il 2022
A ailable online 18 Ap il 2022
Keywo ds:
Eye blinking
Py amid Bo leneck Blocks
Spa io empo al CNN
Inc emen al SVD
Facial landma ks
abs ac
Eye blink de ec ion is a challenging p oblem ha many esea che s a e wo king on because i has he
po en ial o sol e many acial analysis asks, such as ace an i-spoo ing, d i e d owsiness de ec ion,
and some heal h diso de s. The e ha e been ew a emp s o de ec blinking in he wild scena io,
while mos o he wo k has been done unde con olled condi ions. Mo eo e , cu en lea ning
app oaches a e designed o p ocess sequences ha con ain only a single blink igno ing he case o
he p esence o mul iple eye blinks. In his wo k, we p opose a as amewo k o eye blink de ec ion
and eye blink e i ica ion ha can e ec i ely ex ac mul iple blinks om image sequences conside ing
se e al challenges such as ligh ing changes, a ie y o poses, and change in appea ance. The p oposed
amewo k employs as landma ks de ec o o ex ac mul iple acial key poin s including he ones
ha iden i y he eye egions. Then, an SVD-based me hod is p oposed o ex ac he po en ial eye
blinks in a mo ing ime window ha is upda ed wi h new images e e y second. Finally, he de ec ed
blink candida es a e e i ied using a 2D Py amidal Bo leneck Block Ne wo k (PBBN). We also p opose
an al e na i e app oach ha uses a sequence o ames ins ead o an image as inpu and employs
a con inuous 3D PBBN ha ollows mos o he s a e-o - he-a app oaches schemes. Expe imen al
esul s show he be e pe o mance o he p oposed app oach compa ed o he s a e-o - he-a
app oaches.
©2022 The Au ho (s). Published by Else ie L d. This is an open access a icle unde he CC BY-NC-ND
license (h p://c ea i ecommons.o g/licenses/by-nc-nd/4.0/).
1. In oduc ion
Eye blinking ac ion is one o he signi ican i al signs ha can
indica e se e al human heal h issues such as ed eye synd ome,
a igue, and d owsiness. Mo eo e , he signal o a sequence o
eye blinks is ex ac ed and employed in many applica ions such
as disabled people communica ion, ake ace de ec ion, and ace
an i-spoo ing. To app op ia ely de ec an eye blink, se e al image
and ideo p ocessing echniques should be pe o med be o e-
hand. The i s phase o a adi ional eye blinking scheme is o
de ec he ace o he a ge pe son/s. The e a e a lo o ace
de ec ion app oaches ha can be ound in he li e a u e whe e
many o hem (Chen, Huang, Peng, Zhou, & Zhang,2020;Jiang &
Lea ned-Mille ,2017;Koll eide , F on hale , Fa aj, & Bigun,2007;
Li, Tang, Wu, Liu, & He,2019;Viola & Jones,2001) show good and
obus pe o mances in he p esence o se e al challenges.
∗Co esponding au ho a : Uni e si y o he Basque Coun y UPV/EHU, San
Sebas ian, Spain.
E-mail add esses: [email p o ec ed] (S.E. Bekhouche),
[email p o ec ed] (I. Kajo), [email p o ec ed] (Y. Ruichek),
[email p o ec ed] (F. Do naika).
A e he ace being de ec ed, he goal is o co ec ly de ec
he eyes o u he p ocessing. Resea che s ha e p oposed many
echniques ha de ec he wo eyes di ec ly o indi ec ly by
de ec ing hei pupils and gazes. Such echniques can be di ided
in o wo classes: In a- ed based echniques and appea ance-
based echniques. The o me ca ego y in ol es he echniques
ha make use o came as equipped wi h in a- ed senso s o
ob ain se e al eye loca ion candida es based on hei co neal
e lec ions. Despi e hei good pe o mance in p o iding accu a e
eye loca ions, he equi emen o addi ional ha dwa e emains an
ob ious downside ha needs o be ackled. On he o he hand,
appea ance-based echniques p o ide mo e p ac ical amewo ks
ha can be easily implemen ed in a ious eal-wo ld applica-
ions. Likewise, such a ca ego y o echniques can be di ided
based on hei way o p ocessing in o wo main subca ego ies:
ea u e-based echniques and model-based echniques. The i s
subca ego y consis s o echniques ha ake ad an age o he
eye symme y concep when measu ing nume ous de ec ed local
image ea u es such as co ne , edge, and g adien . Such a subca -
ego y does no equi e any lea ning be o ehand which makes i s
echniques mo e eliable o deal wi h un ained scena ios. How-
e e , such echniques a e sensi i e o noise and highly dependen
h ps://doi.o g/10.1016/j.neune .2022.04.010
0893-6080/©2022 The Au ho (s). Published by Else ie L d. This is an open access a icle unde he CC BY-NC-ND license (h p://c ea i ecommons.o g/licenses/by-
nc-nd/4.0/).
S.E. Bekhouche, I. Kajo, Y. Ruichek e al. Neu al Ne wo ks 152 (2022) 150–159
on he accu acy o ea u e de ec ion whe e alsely de ec ed non-
eye ea u es lead o less s able pe o mance. On he con a y,
model-based echniques employ he global appea ance o he eye
o ace images. Se e al machine and deep lea ning ne wo ks ha e
been p oposed o ex ac he accu a e loca ion o he eye egion
in addi ion o o he acial ea u es. These ne wo ks a e usually
ained on aw eye images o se o acial ea u es and p o ide
accu a e and obus eye de ec ion in mos cases.
The las phase o an eye blink de ec ion amewo k is eye s a e
es ima ion whe e he s a e o he eye is iden i ied whe he i is
closed o open. Dozens o eye s a e es ima ion echniques a e
p oposed in he li e a u e and hey can be classi ied in o h ee
ca ego ies: empla e ma ching based, shape-based, and lea n-
ing based echniques. Templa e-based echniques compa e he
de ec ed eye images wi h empla es ha ep esen bo h eye
s a es and he simila i ies among hese images a e measu ed
o es ima e he inal eye s a e. On he o he hand, shape-based
echniques make use o di e en geome ic cha ac e is ics o
se e al shape ea u es such as ci cula shape, cu a u e, and
p ojec ion o pixel in ensi ies along wi h bo h ho izon al and
e ical di ec ions. The las ca ego y ep esen s he echniques
ha use bo h machine and deep lea ning app oaches o e i y he
s a e o de ec ed eye images whe e hei ne wo ks a e basically
ained on eye s a e sequences o (close–open–close) images.
Despi e hei good and obus pe o mances, he majo i y o hese
app oaches a e no designed o handle a sequence o eye blinks
in he same ideo. An e ec i e eye blink de ec ion amewo k
should be able o handle mul iple blinks in one gi en ideo in
addi ion o o he challenges such as appea ance changes and
illumina ion a ia ions.
In his pape , we aim o use a ision-based amewo k o
au oma ic eye blinking de ec ion whe e we p opose wo di e en
app oaches ha imp o e pe o mance in challenging scena ios.
The i s app oach s a s wi h cons uc ing a ea u e-based ma ix
ha con ains empo al changes o he eyes, hen uses SVD o
ex ac he eye signal o eye blink de ec ion. Finally, he de ec ed
blinks a e e i ied using a 2D py amidal bo leneck block ne wo k
(PBBN). The second app oach uses an end- o-end 3D PBBN o
decide whe he he e is a blink in a speci ic image sequence o
no . The main con ibu ions o his wo k can be summa ized in
he ollowings:
•In oducing 2D and 3D ligh CNNs called Py amidal Bo le-
neck Block Ne wo ks (PBBN) ha con ain Py amid Bo le-
neck blocks.
•P oposing mo ing windowed-singula alue decomposi ion
(SVD) o eye blinks de ec ion
•P oposing an end- o-end 3D PBBN o de e mine he exis-
ence o blink wi hin an image sequence
The emaining o he pape is o ganized as ollows:
Sec ion 2p esen s ela ed wo k on eye blinking. Then we in o-
duce and desc ibe he p oposed app oaches in Sec ion 3. Sec ion 4
p esen s he expe imen s and discusses he ob ained esul s.
Finally, Sec ion 5d aws some conclusions and poin s o u u e
di ec ions.
2. Rela ed wo k
Dozens o eye blink de ec ion echniques ha e been p oposed
in he li e a u e. These echniques can be classi ied in o di -
e en classes acco ding o hei inpu da a, way o p ocessing,
and he ea u es used. The e ha e been a a ie y o me hods
ha p oposed o de ec eye blinks by es ima ing he eye s a es
(open/close) using a single image only. Such es ima ion mainly
s a s by ex ac ing di e en se s o ea u es such as his og am
o o ien ed g adien s and local bina y pa e ns which a e ed
in o di e en machine lea ning echniques o lea n he di e ence
be ween bo h eye s a es. Recen ly, esea che s s a ed using di -
e en con olu ional neu al ne wo k a chi ec u es o enhance he
accu acy o he s a e es ima ion esul s.
Zhao, Wang, Zhang, Qi, and Wang (2018) p oposed a ame-
wo k based on deep lea ning o eye blink classi ica ion com-
posed o wo deep ne wo ks. Fi s , hey de ec he ace om
a ame using Viola–Jones, hen he eye egions a e c opped
using acial landma ks p o ided by a de o mable ace alignmen
sys em. The image is ed o hei i s ne wo k which consis s
o con olu ion laye s, pooling laye s and ully connec ed laye s.
The las ully connec ed laye is ollowed by a So max unc ion.
In he second ne wo k, a la en ec o ep esen ing he image’s
pixels is ed o mul iple ully connec ed laye s ollowed by a
So max unc ion. In he end, hey ob ain h ee ou pu s: (i) usion
o So max unc ions om he wo ne wo ks, (ii) he ou pu o
he i s ne wo k and (iii) he ou pu o he second ne wo k.
These h ee ou pu s a e passed o c oss-en opy loss unc ion
which calcula e he model e o o ain he whole sys em. This
amewo k could un in eal- ime, howe e i is ulne able o
ou doo scena ios.
Li, Chang, and Lyu (2018) in oduced a me hod o human eye
blinking de ec ion o expose he ake aces in ideos gene a ed by
deep ne wo ks. They de ec ed he ace using Dlib lib a y (King,
2009), hen ex ac ed he acial landma ks ia Kazemi algo i hm
(Kazemi & Sulli an,2014). These landma ks we e used o align
he ace and c op he eye egion. The c opped eye egion o
each ame was ed o hei p oposed Long- e m Recu en Con-
olu ional Ne wo ks (LRCN) which can memo ize he dynamic
in o ma ion om he inpu sequence. Gene ally, LRCN (Donahue
e al.,2015) is composed o a isual ea u e ex ac o using CNN
and sequence lea ning using a s ack o ecu en neu al ne wo ks
(RNNs). They used he i s ully connec ed laye o he VGG-
16 model (Simonyan & Zisse man,2014) o ex ac he ea u es.
Simila o Hu e al. (2020) and Li e al. (2018) p oposed o
use a RNN o cap u e empo al in o ma ion o eye blinking in
uncons ained scena ios. Ins ead o using deep ea u es ex ac ed
by CNNs, hey ex ac ed he eye ea u es using a ligh weigh
uni o m LBP desc ip o (Ahonen, Hadid, & Pie ikainen,2006).
The second ca ego y o me hods ep esen s he echniques
ha p ocess he whole ideo a he han a single image whe e he
changes wi h espec o eye appea ance ea u es o eye mo ion
a e acked and analyzed o cons uc a signal ha ep esen s
he blinking e en s o e ime. Lalonde, By ns, Gagnon, Teasdale,
and Lau endeau (2007) in oduced a mul i-senso app oach ha
de ec s eye blinks in low con as unde nea -in a ed images.
Ini ial eyes loca ions a e calcula ed by inding he minimum o he
la ge alleys in he ex ac ed ace p o iles ( ow-wise p ojec ion).
These ini ial eye loca ions a e used o iden i y wo eye egions
o in e es (ROI) in which SIFT ea u e poin s a e ex ac ed and
acked o e ime using Kalman il e o main ain he posi ion
alignmen among successi e ames. Then, mo ion de ec ion ol-
lowed by a h esholding p ocedu e a e pe o med in he acked
ROIs o iden i y he bes eye blobs based on se e al geome y
me ics such as a ea, posi ion, angle, and a ios. Finally, he op i-
cal lows in he selec ed blob egions a e compu ed o de e mine
he dominan di ec ion whe e e ical downwa d mo ion ec o s
a e used o indica e he exis ence o an eye blink. This app oach
shows high de ec ion a e and i pe o ms in nea eal ime.
Howe e , he usage o in a ed illumina ion may cause ha m o
eyes especially a close dis ances.
Lee, Lee, and Pa k (2010) p oposed a echnique ha de ec s
bo h ace and eye egions using Adaboos algo i hm ollowed
by illumina ion, bina iza ion, and mo phological ope a ions. They
in oduced wo ea u es o de ec he eye blink p ope ly. The
i s ea u e is ex ac ed by compu ing he heigh o wid h a io
151
S.E. Bekhouche, I. Kajo, Y. Ruichek e al. Neu al Ne wo ks 152 (2022) 150–159
Fig. 1. Eye sys em o e iew.
o each eye egion. The second is ob ained by compu ing he
cumula i e di e ence o he numbe o black pixels o e ime
based on he assump ion ha his di e ence co esponds o he
changes in he eye s a e. Fo be e de ec ion accu acy, he wo
ex ac ed ea u es a e ed o a suppo ec o machine (SVM)
which is adap i ely selec ed based on iew angle o he a ge
ace. This app oach shows obus ness o di e en acial poses
and di e en ligh ing condi ions. On he downside, he eye blink
de ec ion misde ec s many eye blinks due o he sensi i i y o
he p oposed cumula i e di e ence p ocedu e o came a loca ion,
eye size, and ini ial eye mask.
D u a o sky and Fogel on (2014) p esen ed a mo ion based
eye blink de ec ion me hod ha acks he ini ial eye egions
o e ime using a lock o KLT acke s. The acked eye egions
a e spli in o 6 blocks whe e he dominan mo ion ec o in
each block is ex ac ed by a e aging he local e ical mo ion
componen s loca ed in he p ocessed block. Then, a simple s a e
machine is ed wi h he a iance o he ex ac ed a e age mo ion
ec o s o de e mine he eye s a e and de ec he eye blink
acco dingly. Recen ly, he same au ho s enhanced hei wo k by
using Gunna -Fa neback acke which p o ides less ou lie s han
KLT acke and be e dis ibu ion o mo ion ec o s. Subse-
quen ly, all e ical componen s o ex ac ed mo ion ec o s a e
no malized by he in aocula dis ance and a e aged o cons uc
a wa e o m ha shows changes in i s magni udes while eye
blinks. Mo e ecen ly, he same au ho s p oposed ano he eye
blink de ec ion scheme ha uses op ical low o he mo ion
de ec ion phase and LSTM o he eye s a e es ima ion phase.
Thei p oposed app oaches achie ed high accu acy when hey a e
es ed on he exis ed da ase s in addi ion o hei p oposed one.
Howe e , he es ed da ase s a e limi ed o indoo ideos and
in ol es limi ed numbe o pe sons.
Chen, Wu, and Chien (2015) p oposed a se o schemes o eye
blink de ec ion and gaze es ima ion wi hou aking he ad an age
o in a ed illumina ion. A e he eyes being de ec ed, se e al
image p ep ocessing p ocedu es a e pe o med o elimina e he
noise caused by he changes in no mal-ligh condi ions and e-
lec ions. To ackle he challenges p esen ed while de ec ing eye
pa s unde isible ligh ing condi ions, hey modi ied S a bu s
algo i hm o make i mo e obus o such challenges. Using he
adap i e S a bu s ex ac ion algo i hm, hei p oposed echnique
co ec ly iden i ies bo h he i is and limbus ea u es. A e wa ds,
he aspec a io o he bounding box ha con ains he i is mask is
compu ed o e ime whe e la ge alues indica e eye-close s a es
while small alues indica e eye-open s a es.
Daza, Mo ales, Fie ez, and Tolosana (2020) p oposed an eye
blink classi ica ion app oach using a modi ied VGG16 a chi ec-
u e. They also p esen ed a da ase o eye blink classi ica ion
unde con olled condi ions using h ee di e en senso s, namely
2 came as (RGB and NIR) and elec oencephalog aphy (EEG) o
de ec he blink. Ryan e al. (2021) ocused on blink de ec ion
using e en came as by p oposing a ully con olu ional ga ed e-
cu en YOLO ne wo k o de ec eyes and hen ack hem. Then,
a ixed ime window is used o analyze he p esence/absence o
eye blink.
3. P oposed app oach
In his sec ion, we aim o p o ide a de ailed explana ion o
he p oposed app oach ha ackles se e al challenges in he
ield o eye blinking de ec ion. Ou app oach is di ided in o
wo main phases. The i s one in ol es se e al p ep ocessing
p ocedu es such as ace de ec ion and eye de ec ion while he
second phase in ol es wo p ocessing s ages: eye blinks de ec ion
ia mo ing-windowed SVD and eye blink e i ica ion ia 2D PBBN
ha e i ies he exis ence o eye blink in each sub-sequence
candida e ex ac ed in he i s s age. The gene al wo k low o
he p oposed app oach is shown in Fig. 1.
3.1. Face/ acial landma k de ec ion and eye egion ex ac ion
The i s ask o mos acial analysis app oaches is ace de-
ec ion. Fo his eason, he bes ace de ec ion app oach ha
is sui able o ou eye blink de ec ion app oach is selec ed. The
ace de ec o we op ed o is based on a Single Sho De ec o
(SSD) amewo k (Liu e al.,2016) using a ResNe model. A e
he ace being de ec ed, he eyes should be accu a ely localized
o a oid he nega i e impac ha alse eye localiza ion has on
he eye blinking sys em. The e o e, ou wo k is p oposed o e ec-
i ely ackle he eye localiza ion challenges such as obus ness in
uncon olled condi ions, compu a ion ime and sensi i i y o he
illumina ion changes.
Fo an e icien eyes localiza ion p ocess, we p opose o use
Kazemi algo i hm (Kazemi & Sulli an,2014), which de ec s 68
acial poin s wi h speci ic coo dina es ha su ound ce ain pa s
o he ace including he eyes and nose (see 2) which can be
152
S.E. Bekhouche, I. Kajo, Y. Ruichek e al. Neu al Ne wo ks 152 (2022) 150–159
Fig. 2. Posi ions o 68 acial landma ks.
compu ed in abou 1 millisecond. A e de ec ing he acial land-
ma ks in he inpu acial image as shown in Fig. 2, he ace pose
in he 2D image is ec i ied based on he eyes cen e simila
o Bekhouche, Oua i, Do naika, Taleb-Ahmed, and Hadid (2017).
Then, he landma ks om 37 o 42 and om 43 o 48 a e used
o c op he igh -eye image and he le -eye image espec i ely.
The me hod o c opping he eyes depends on padding he egion
ha su ounds he landma ks o he in ended eye by 25% in all
di ec ions. Finally, he c opped le and igh eyes a e esized
o 96 ×96 pixels and placed acco ding o hei imes amp in o
wo image se s ep esen ing bo h le and igh eye sequences
espec i ely.
3.2. Mo ing-windowed SVD
Suppose we ha e an image sequence A= {I1,I2,...,Ik} ∈
Rm×n×k ha con ains he c opped le / igh eye images whe e
m e e s o he image heigh , n e e s o he image wid h, and
kindica es he numbe o images. Thus, a ea u e-based ma ix
ha con ains empo al changes o he pixels in eye egions is
cons uc ed as ollows. A e p ope ly acking and segmen ing
he eye egions, he segmen ed eye egions a e di ided in o d
blocks. Then, he pixel ene gy (sum o he squa e pixels’ in en-
si ies) in each block is compu ed o cons uc a one-dimensional
ec o ha con ains he ene gy alues o all blocks in a single
c opped ame. Subsequen ly, he ex ac ed ec o s a e employed
o cons uc a k×dma ix B= {e1;e2;. . . ;ek}, whe e each ow
e is an ene gy obse a ion d-dimensional ec o . To ex ac he
eye change signal ha bes ep esen s he eye blinking e en , he
singula alue decomposi ion o ma ix Bis compu ed as ollows:
UTBV =S=diag(s1,...,sp)∈Rk×d(1)
whe e p=min{k,d}and s1≥s2≥ · · · ≥ sp≥0. The ma ices
U∈Rk×kand V∈Rd×da e he le and igh singula ec o s, e-
spec i ely. P ac ically, a educed-size SVD is u ilized in his pape
whe e he numbe o ows in he ma ix U∈Rk×kis educed o d
whe e Ubecome U∈Rk×d. As discussed in Kajo, Kamel, Ruichek,
and Malik (2018), he ma ix Ucon ains he same empo al
in o ma ion as he co esponding o iginal ma ix B. Gi en his
ac , he s uc u es o he le singula ec o s o ma ix Ushould
be u he in es iga ed. F om a signal p ocessing poin o iew,
he p ojec ion o ma ix Bon o he i s le singula ec o u1
subspace e eals he low- ank in o ma ion embedded in B. On he
o he hand, he p ojec ions o Bon o he emaining le singula
ec o s’ u2,u3. . . udsubspaces e eal he spa se in o ma ion ha
ep esen s he empo al changes in B. The e o e, he ec o ha
bes ep esen s he eye change signal is expec ed o be one o
hese ec o s. The le singula ec o s con ain bo h nega i e
and posi i e en ies wi h alues anging be ween −1 and 1. Fo
be e ep esen a ion and analysis o he es ima ed eye signal, he
en ies o he ec o s o in e es a e scaled o all on he in e al o
[0 1]. The scaled ec o s a e empo ally p ocessed using a mo ing
a e age il e o educe he ou lie s and emo e he noise. The
bes ec o ha ep esen s he eye change signal is de e mined
based on i s equency in o ma ion. To achie e his, a equency
es ima ion ia as Fou ie ans o m is pe o med on each ec o
o in e es and he ec o ha ing i s p inciple equency wi hin
a p ede ined in e al and has he la ges ampli ude, is ex ac ed.
The en ies ha co espond o he ames when he eye is closed
a e expec ed o ha e la ge alues while he en ies ha co -
espond o he ames when he eye is open a e expec ed o
ha e small alues. Based on his ac , a coa se peak analysis is
pe o med on he ex ac ed ec o -signal o ob ain a se C=
{A1,A2,...,Al} ∈ R96×96×k×l ha con ains lsub-sequence can-
dida es o size 96 ×96 ×kwhich a e expec ed o show he eye
blink e en s. Due o he ac ha he e i ica ion s age is designed
o deal wi h sequences wi h single po en ial eye blink, he ob-
ained candida es a e ed indi idually in o he p oposed 2D PBBN
o e i y he p esence/absence o an eye blink. Fig. 3 illus a es an
example o eye change signal ex ac ion om a gi en ideo.
To achie e he eal- ime equi emen s, a mo ing window
mechanism is used whe e he ini ial SVD componen s a e ex-
ac ed using he i s k ames and he i s ame in he sequence
is emo ed when a new ame is a i ed. The eye change signal
is upda ed e e y second and he new added pa is analyzed o
de ec new eye blink/s.
3.3. Py amidal con olu ion neu al ne wo k
Deep lea ning algo i hms ha e signi ican ly imp o ed he pe -
o mance in many compu e ision asks whe e deep lea n-
ing models can lea n mo e obus ea u es compa ed o classic
me hods. S a ing wi h LeNe (LeCun e al.,1995) hen Alexne
(K izhe sky, Su ske e , & Hin on,2012), mo e gene alized deep
a chi ec u es ha e been me ged like VGG (Simonyan & Zisse -
man,2014), ResNe (He, Zhang, Ren, & Sun,2016) and Incep ion
(Szegedy e al.,2015). Inspi ed by esidual block and bo leneck
esidual block (He e al.,2016), we p opose a simple block named
Py amid Bo leneck which can be applied o bo h 2D and 3D
inpu s. The idea behind he Py amid Bo leneck (PB) block is o
educe he o al numbe o blocks in an a chi ec u e which leads
o educe he numbe o he pa ame e s. The impo ance o a
smalle numbe o pa ame e s is mainly o sho en he in e ence
ime o he model and o i he size o he model o he size o he
aining se , because he eye blink da ase does no con ain many
samples o ain a model wi h a la ge numbe o pa ame e s.
The p oposed PBBN is composed o a s a ing block ha con-
ains a con olu ional laye ha il e s he 96 ×96 ×3 inpu
image wi h 64 ke nels o size 3 ×3 wi hou s ide, a ba ch no -
maliza ion laye which no malizes each inpu channel, ReLU laye
which pe o ms a h eshold ope a ion o he nega i e alues o
be se as 0 and a max-pooling laye o 3 ×3 wi h a s ide o
1×1. Then, mul iple PB blocks s a ed wi h he con olu ion o 64
ke nels and i doubles he ke nels a e each PB block. Finally, he
ne wo ks end wi h a global a e age pooling which is connec ed
wi h a ully connec ed laye ha has he size o he numbe o
classes o numbe o labels o he in ended ask.
The PB block is a bunch o b anches shaped like a py amid
so ha each b anch con ains mul iple laye s, he numbe o
153
S.E. Bekhouche, I. Kajo, Y. Ruichek e al. Neu al Ne wo ks 152 (2022) 150–159
Fig. 3. Eye signal ex ac ion om image sequence o eyes.
Table 1
A chi ec u e o an example o 3D PBBN ha con ains one py amid wi h wo b anches.
Block Laye Fil e s numbe Fil e size S ide size Ou pu
Inpu
3D Con 64 3 ×3×3×3 1 ×1×2 96 ×96 ×7×64
BN – – – 96 ×96 ×7×64
ReLU – – – 96 ×96 ×7×64
MaxPool 1 3 ×3×3 1 ×1×2 96 ×96 ×4×64
P1
B1 3D Con 64 1 ×1×3×64 2 ×2×1 48 ×48 ×4×64
BN – – – 48 ×48 ×4×64
B2
3D Con 64 3 ×3×3×64 1 ×1×1 96 ×96 ×4×64
BN – – – 96 ×96 ×4×64
ReLU – – – 96 ×96 ×4×64
3D Con 64 1 ×1×3×64 2 ×2×1 48 ×48 ×4×64
BN – – – 48 ×48 ×4×64
Add ADD – – – 48 ×48 ×4×64
ReLU – – – 48 ×48 ×4×64
Ou pu A gPool – – – 1 ×1×1×64
FC – – – 2
laye s changes acco ding o he numbe o PB. Le say we ha e
lb anches, he i s b anch has one con olu ion laye , he second
b anch has wo con olu ion laye s, and so on. Each con olu-
ion laye is ollowed by a ba ch no maliza ion laye and ReLU
laye excep he las con olu ion laye o each b anch whe e i
is ollowed only by ba ch no maliza ion. The las con olu ions
ha e il e s o size 1 ×1 wi h a s ide o 2 ×2. A e each
PB block, he channels dimension inc eases by double, and he
spa ial dimensions (i.e., h×w) a e educed o hal , and each PB
b anch s a s wi h con olu ion laye s ha ha e a il e size o
2l−1×2l−1, and i keeps educing he il e size by 2 o he
nex con olu ion o he same b anch whe e lis he numbe o
b anches. The PB block could be explained ma hema ically as a
gi en inpu x∈Rc×h×w, whe e cis he numbe o channels; h,w
a e he heigh and wid h, espec i ely. The new ea u e map F′is
compu ed as:
F′(x)=
l
∑
b=1
Fl(x) (2)
whe e lis he numbe o he b anches inside he Py amid Bo le-
neck and Flis se ies o lcon olu ions. The ou pu s o b anches a e
added elemen -wise oge he , hence he con olu ion laye s ha e
ze o-padding excep he las con olu ion laye o each b anch,
Fig. 4 illus a es an example o PB block wi h 4 b anches.
Rega ding he 3D Py amid Bo leneck Block, i has he same
cha ac e is ics as 2D one excep o he inpu ha con ains dep h
in o ma ion and hey also di e s in he spa ial size o he il e s
whe e hei size in 2D is l×las o 3D i is l×l×3, his is
due o he ac ha he dep h has small dimension compa ed
o he spa ial in o ma ion. Fig. 5 illus a es an example o 3D PB
block wi h 3 b anches. Simila o 2D PBBN, 3D PBBN is composed
o an opening block ha con ains a 3D con olu ional laye ha
il e s he 96 ×96 ×13 ×3 inpu sequence wi h 64 ke nels o
size 3 ×3×3 wi h s ide 1 ×1×2 o downsampling he
empo al dimension whe e 13 is he numbe o ames, a ba ch
no maliza ion laye , ReLU laye and a 3D max-pooling laye o
3×3×3 ha also downsamples he empo al dimension o hal
and main ains he spa ial and channels dimensions. Then, come
mul iple 3D PB blocks like 2D CNN ha down-sample he spa ial
dimension by hal and double channels dimension a e each 3D
Block. The 3D PBBN ends wi h global a e age pooling, a ully
connec ed laye ha has wo ou pu s and a so max laye (case o
eye blinking classi ica ion). Table 1 illus a es he a chi ec u e o
an example o a 3D PBBN ha con ains one py amid wi h wo
b anches. As he la e able shows, he downsampling o he
empo al dimension is done only in he i s 3D con olu ion laye
and he max-pooling laye . On he o he hand, he downsampling
o he spa ial dimension is done a e each PB block whe e he
ou pu will be downsized o hal . Finally, he global a e age pool-
ing laye will downsample bo h spa ial and empo al dimensions
o gene a e one ea u e map ha is connec ed o a ully connec ed
laye wi h an ou pu co esponding o he speci ic ask o he
model.
4. Expe imen s
We conside wo baseline asks o e alua e he p oposed wo k.
The i s ask ela es o eye blink classi ica ion whe e he objec-
i e is o de e mine he p esence o absence o an eye blinking
in a sho sequence o images ha con ain only single eye blinks.
As o he second ask, named eye blink de ec ion, he objec i e
is o de e mine he ime and du a ion o he de ec ed eye blink.
154
S.E. Bekhouche, I. Kajo, Y. Ruichek e al. Neu al Ne wo ks 152 (2022) 150–159
Fig. 4. Example o 2D Py amid Bo leneck Block wi h 4 b anches.
Fig. 5. Example o 3D Py amid Bo leneck Block wi h 3 b anches.
4.1. Eye blinking classi ica ion
4.1.1. Da ase s
In his wo k, we ocused on eal scena ios o eye blinking clas-
si ica ion, he e o e we chose HUST-LEBW da ase as a sui able
da ase o eye blinking classi ica ion in he wild. This da ase
was c ea ed using clips om 20 mo ies and TV se ies such as
The Ma ix, A Chinese Fai y Tale and Game o Th ones. These
clips we e spli in o a aining se and es ing se and each clip
is di ided in o mul iple sub-clips ideos, so he o al numbe
o ideos eaches 90. Each ideo is ei he wi h a esolu ion o
1280 ×720 o 1456 ×600 and he ac o s in he ideos appea
in di e en poses and unde di e en iewpoin s. Th ough all
he ideos, 1314 samples we e ex ac ed and each sample is
anno a ed wi h ei he a p esence o absence o eye blink. The
155
S.E. Bekhouche, I. Kajo, Y. Ruichek e al. Neu al Ne wo ks 152 (2022) 150–159
Table 2
Dis ibu ion o sequences in HUST-LEBW da ase .
Eye Blinking T ain Tes
Righ Yes 256 126
No 190 98
Le Yes 243 122
No 181 98
Table 3
Pe o mance o he di e en a ian s o he p oposed 3d PBBN in HUST-LEBW
da ase .
Ne wo k Pa ame e s F1Recall P ecision
3D PBBN P2B2 437 184 0.8463 0.8548 0.8379
3D PBBN P2B3 1 974 912 0.8640 0.8710 0.8571
3D PBBN P2B4 5 933 952 0.8265 0.8548 0.8000
3D PBBN P3B2 1 286 784 0.8509 0.8629 0.8392
3D PBBN P3B3 5 775 360 0.9105 0.8871 0.8730
3D PBBN P3B4 17 220 096 0.8245 0.8145 0.8245
de ails a e shown in Table 2. Each sample has a ime span o 13
ames.
4.1.2. E alua ion
To add ess he p oblem o eye blinking classi ica ion, we pe -
o med some expe imen s using di e en combina ions o he
p oposed 3D PBBN on HUST-LEBW da ase . As a classi ica ion
p oblem, he e alua ion o he pe o mance o hese expe imen s
is done using Recall,P ecision and F1me ics which a e compu ed
as ollows:
Recall =TP
TP +FN (3)
P ecision =TP
TP +FP (4)
F1=2
1
Recall +1
P ecision
(5)
Unlike he p oposed 2D PBBN, he p oposed 3D PBBN is ap-
plied on he agg ega e successi e ames o lea n he spa io em-
po al in o ma ion as we men ioned in Sec ion 3.3. Fo a ai
compa ison wi h o he wo ks, we ha e used a span ime o
13 ames as dep h o he inpu ideo sequence ha is ed o
he p oposed 3D PBBN. The aining o he ne wo k is simila
o he aining o 2D PBBN, howe e , we educed he mini-
ba ch o 16 samples. Also, he loss unc ion di e s, we used
c oss-en opy unc ion wi h mu ually exclusi e classes (blink/no-
blink). We ha e chosen o e alua e he pe o mance o di e en
combina ions o he p oposed 3D PBBN by changing he numbe
o py amids om 2 o 3 and he numbe o b anches o each
py amid om 2 o 4, which gi es ise 6 combina ions. Table 3
illus a es he esul s o each 3D PBBN combina ion on he eye
blinking classi ica ion p oblem.
To show he gene aliza ion abili y and he s abili y o he p o-
posed model, we conduc ed mo e expe imen s besides he la e
one. Speci ically, we conduc ed h ee g oups o expe imen s. The
esul s o hese expe imen s a e shown in Table 4.
In he i s g oup, we y o educe he andomness o he
aining o he deep ne wo k. To his end, aining and es ing
we e epea ed i e imes wi h he same aining/ es spli . The
a e age and s anda d de ia ion o ecall, p ecision and F1sco e
we e epo ed. In he second se o expe imen s, we aim o educe
he andomness in oduced by he selec ion o he aining se .
Fo his pu pose, we pe o m 5 di e en andom spli s (70% o
he aining and 30% o he es s) and epo he co esponding
a e age and s anda d de ia ion o he e alua ion me ics. In he
las se o expe imen s, we use he classical scheme o i e old
Fig. 6. ROC cu es o he eye blinking classi ica ion esul s o he 3 b anches
3D PBBN combina ions.
c oss- alida ion. As can be seen, he ob ained s anda d de ia ions
a e ela i ely small o all ypes o uns, indica ing ha he
solu ion p oposed by ou scheme is s able bo h in e ms o he
aining p ocess o he ne wo k and in e ms o he selec ed
aining images o ideos.
F om his able, we can obse e ha 3D PBBN P3B3 (3 py a-
mids wi h 3 b anches in each py amid) has he bes esul s among
he o he a ian s. Also, we can no ice ha he bes esul s a e ob-
ained om 3 b anches py amids ( ows 2 and 5 o Table 7). Fig. 6
illus a es he ROC cu es o he eye blinking classi ica ion esul s
o he h ee 3D PBBN combina ions, whe e i shows he p omising
po en ial pe o mance o he h ee combina ions o he p oposed
3D PBBN wi h h ee b anches. Fo a comp ehensi e e alua ion,
Table 5 p o ides compa ison be ween he bes combina ion (3D
PBBN P3P3) and he s a e-o - he-a app oaches whe e i e eals
ha he p oposed app oach is signi ican ly be e compa ed o
he es app oaches in he ecen benchma k.
4.2. Eye blinking de ec ion
4.2.1. Da ase s
The Epan-EyeBlink da ase 1was collec ed om You ube
ideos unning a ame a es o 30 ps. We collec ed and immed
18 ideos wi h a a ia ion o subjec s, poses, no glasses/glasses,
exp essions and illumina ion. Unlike o he publicly a ailable
da ase s, he ideos in he p oposed da ase ha e mul iple blinks
in hei sequences which allows s udying he ime and wid h
o each blink. The a e age ime o he ideos is 26 s, and he
a e age numbe o blinks is 15. Fig. 7 shows some samples om
ou Epan-EyeBlink da ase .
4.2.2. E alua ion
We use he same me ics as o eye blinking classi ica ion.
Howe e , he TP, FP, and FN a e de ined in a di e en way whe e
TP means he e is in e sec ion in ime dimension be ween he
p edic ed blink and g ound u h. FN and FP a e he numbe s o
missed blinks and he numbe o alse de ec ed blinks.
In his sec ion, we pe o med h ee e alua ions o eye blink-
ing de ec ion. The i s one is conce ned wi h he p oposed mo -
ing windowed-SVD app oach, he second one abou he p oposed
1h ps://gi hub.com/Bekhouche/Epan-EyeBlink.
156
S.E. Bekhouche, I. Kajo, Y. Ruichek e al. Neu al Ne wo ks 152 (2022) 150–159
Fig. 7. Samples om Epan-EyeBlink da ase .
Table 4
Pe o mance o he p oposed 3D PBBN using di e en aining s a egies in HUST-LEBW da ase .
S a egy Eye idx Recall P ecision F1 sco e
5- epe i ion Le 0.9049 ∓0.0093 0.8805 ∓0.0086 0.8925 ∓0.0135
Righ 0.8984 ∓0.0067 0.8844 ∓0.0063 0.8913 ∓0.0060
5- andom spli Le 0.9032 ∓0.0103 0.8770 ∓0.0191 0.8898 ∓0.0116
Righ 0.8952 ∓0.0181 0.8691 ∓0.0138 0.8819 ∓0.0142
5- old c oss alida ion Le 0.8968 ∓0.0177 0.8642 ∓0.0199 0.8801 ∓0.0135
Righ 0.8901 ∓0.0188 0.8593 ∓0.0203 0.8744 ∓0.0180
Table 5
Pe o mance compa ison among he di e en eyeblink e i ica ion me hods on HUST-LEBW da ase .
Me hod Eye idx Recall P ecision F1 sco e
Mo is ( e .) (Mo is, Blenkho n, & Zaidi,2002) (2002) Le 0.5246 0.4741 0.4981
Righ 0.5635 0.5064 0.5334
Mo is (ho .) (Mo is e al.,2002) (2002) Le 0.6393 0.5342 0.5821
Righ 0.5476 0.5107 0.5285
Mo is ( low) (Mo is e al.,2002) (2002) Le 0.4918 0.4918 0.4918
Righ 0.4286 0.4741 0.4502
Chau (Chau & Be ke,2005) (2005) Le 0.1721 1.0000 0.2937
Righ 0.2302 0.9656 0.3718
D u a o sky (D u a o sky & Fogel on,2014) (2014) Le 0.1190 0.4757 0.1904
Righ 0.0952 0.2860 0.1428
Daza (Daza e al.,2020) (2020) Le 0.9603 0.6080 0.7446
Righ 0.7950 0.7348 0.7637
Hu (Hu e al.,2020) (2020) Le 0.7805 0.7385 0.7589
Righ 0.8333 0.7778 0.8046
P oposed (3D PBBN) Le 0.9161 0.8812 0.8983
Righ 0.9048 0.8507 0.8769
Table 6
Resul s o he p oposed SVD app oach on Epan-EyeBlink da ase .
Video Recall P ecision F1
1 0.8205 0.8205 0.8205
2 1.0000 0.2405 0.3878
3 0.9048 0.4750 0.6230
4 0.9467 0.4863 0.6426
5 0.9200 0.4035 0.5610
6 1.0000 0.5778 0.7324
7 1.0000 0.0909 0.1667
8 0.8537 0.5512 0.6699
9 0.8148 0.4074 0.5432
10 1.0000 0.8636 0.9268
11 0.9245 0.6164 0.7396
12 1.0000 0.3483 0.5167
13 1.0000 0.1374 0.2416
14 1.0000 0.3226 0.4878
15 1.0000 0.6419 0.7819
16 0.7609 0.6604 0.7071
17 0.9635 0.6839 0.8000
18 1.0000 0.2815 0.4393
A e age 0.9394 0.4783 0.5993
2D PBBN, and he las one e alua ed he combina ion o he
p oposed mo ing windowed-SVD me hod and he p oposed 2D
PBBN. He ein, we e alua e he p oposed mo ing windowed SVD
on he Epan-EyeBlink da ase , he de ailed esul s a e p esen ed
in Table 6. The esul s show poo p ecision and high ecall owing
o he ac ha he p oposed mo ing windowed-SVD gi es a lo
o alse de ec ed blinks. On he o he hand, i de ec s mos o he
blinks.
In he case o he combina ion-based app oach, he objec i e
o he 2D PBBN is o e i y he exis ence o absence o eye blink o
enhance he pe o mance o he SVD app oach by il e ing mos
o he alse de ec ed blinks. The e o e, we i s ained a ligh 2D
PBBN (2 py amids and each py amid has 2 b anches) using some
images o he HUST-LEBW da ase . To make he da abase sui able
o aining he p oposed ligh 2D PBBN, we ook h ee images
om each sequence and labels hem simila o hei sequence
label, so ha we ha e 2610 images o aining and 1332 images
o alida ion. The bes -achie ed esul on he alida ion subse
was 91.14% ecall.
The esul s o he 2D PBBN on he Epan-EyeBlink da ase
a e gi en in Table 7. Then, he ained ne wo k is applied on
157
S.E. Bekhouche, I. Kajo, Y. Ruichek e al. Neu al Ne wo ks 152 (2022) 150–159
Table 7
Resul s o he p oposed PBBN app oach on Epan-EyeBlink da ase .
Video Recall P ecision F1
1 0.8478 1.0000 0.9176
2 1.0000 1.0000 1.0000
3 0.9545 0.9545 0.9545
4 0.9074 0.9608 0.9333
5 0.8846 0.8846 0.8846
6 1.0000 0.9808 0.9903
7 1.0000 0.6364 0.7778
8 0.7917 0.9500 0.8636
9 0.9107 0.9623 0.9358
10 1.0000 1.0000 1.0000
11 0.9381 0.9464 0.9422
12 1.0000 0.8750 0.9333
13 1.0000 0.6667 0.8000
14 1.0000 0.9268 0.9620
15 1.0000 0.9517 0.9752
16 0.7794 1.0000 0.8760
17 0.9638 0.9568 0.9603
18 1.0000 0.9143 0.9552
A e age 0.9432 0.9204 0.9257
Table 8
Resul s o he p oposed SVD+2D PBBN app oach on Epan-EyeBlink da ase .
Video Recall P ecision F1
1 0.8478 1.0000 0.9176
2 1.0000 1.0000 1.0000
3 0.9512 0.9750 0.9630
4 0.9699 0.8815 0.9236
5 0.9636 0.9298 0.9464
6 1.0000 0.9333 0.9655
7 1.0000 0.8485 0.9180
8 0.9130 0.9921 0.9509
9 0.9138 0.9815 0.9464
10 1.0000 0.8636 0.9268
11 0.9490 0.9371 0.9430
12 1.0000 0.8764 0.9341
13 1.0000 0.8550 0.9218
14 1.0000 0.8629 0.9264
15 1.0000 0.9256 0.9614
16 0.8281 1.0000 0.9060
17 0.9733 0.9430 0.9579
18 1.0000 0.9556 0.9773
A e age 0.9617 0.9312 0.9437
all he de ec ed blink candida es ob ained by he p oposed SVD
based me hod, and he esul s a e shown in Table 8. Due o he
high ecall o he p oposed 2D PBBN based e i ica ion phase,
he h esholding pa ame e s used in he peak analysis p oce-
du e applied on he ex ac ed eye signals, a e se o be as low
as possible. Such s ep gua an ees he de ec ion o he majo i y
exis ed eye blinks in an eye signal which is clea ly indica ed
by he high p ecision alues in Table 8. On he o he hand,
lowe ing he h esholding pa ame e s inc eases he likelihood o
alse de ec ions which is esul ed in low ecall alues as epo ed
in Table 8.
The pe o mance compa ison o he p oposed app oach wi h
he ecen s a e-o - he-a app oaches on he p oposed Epan-
EyeBlink da ase is p o ided in Table 9. We can obse e ha ou
p oposed SVD based me hod has a be e ecall han he o he
s a e-o - he-a app oaches howe e i s p ecision is e y low due
o he mul iple alse de ec ions. On he o he hand, he p oposed
PBBN has good p ecision and ecall. Whe e o e, he combina ion
o SVD +PBBN imp o es bo h p ecision and ecall and i has he
bes esul s among he o he wo ks.
5. Conclusion
In his pape , we p oposed di e en supe ised and unsupe -
ised lea ning app oaches o p o ide an e ec i e and obus eye
Table 9
Pe o mance compa ison among he di e en eyeblink de ec ion me hods on
Epan-EyeBlink da ase .
Me hod Recall P ecision F1
Li (Li e al.,2018) (2018) 0.8507 0.8153 0.8326
Maio (Maio , das
Chagas Mou a, San ana, & Lins,
2020) (2020)
0.8976 0.6120 0.7278
Hu (Hu e al.,2020) (2020) 0.8712 0.8636 0.8674
P oposed (SVD) 0.9394 0.4783 0.5993
P oposed (PBBN) 0.9432 0.9204 0.9257
P oposed (SVD-PBBN) 0.9617 0.9312 0.9437
blink de ec ion amewo k. Fi s , we p oposed an e icien 3D
model o de e mine he exis s o an eye blink in eye sequence
images as his model con ains a small numbe o pa ame e s
compa ed o known CNN models. Second, we inco po a ed he
unsupe ised lea ning using SVD which is e ec i ely employed o
ex ac he eye mo ion signal ha con ains unique pa e ns which
ep esen he eye blinks. Then, he supe ised lea ning based on
he 2D PBBN which is u ilized o e i y he de ec ed eye blink
candida es and enhance he de ec ion pe o mance in e ms o
ecall alues. Such usion o supe ised and unsupe ised lea ning
app oaches p o ides a obus eye blink de ec ion amewo k ha
is capable o handling se e al challenges such as di e en ligh -
ing condi ions, a ie y o appea ance, and mul i-blink sequences.
Mo eo e , a ailable da ase s wi hin his esea ch ield we e lim-
i ed o sequences wi h only one eye blink pe sequence which
in u n p e en s he e alua ion o he pe o mance o p oposed
echniques in he long- e m and in he p esence o he challenges
ha accompany hese sequences. The e o e, we in oduced a new
da ase ha in ol es se e al ideos wi h mul iple eye blinks in
each sequence in addi ion o di e en challenges. The expe imen-
al esul s indica e he e ec i eness and ou pe o mance o he
p oposed amewo k compa ed o s a e-o - he-a me hods.
As u u e wo k, we en ision he use o empo al ans o m-
e s ne wo ks and he imp o ed combina ions o CNN-LSTM o
eye blinking and o he ela ed applica ions such as yawning
and d owsiness de ec ion. One limi a ion o he s a e-o - he-a
eye blinking me hods is ha hey equi e on al ace. Thus, we
en ision ying o ackle and in es iga e his p oblem by using
non- on al aces.
Decla a ion o compe ing in e es
The au ho s decla e ha hey ha e no known compe ing inan-
cial in e es s o pe sonal ela ionships ha could ha e appea ed
o in luence he wo k epo ed in his pape .
Re e ences
Ahonen, T., Hadid, A., & Pie ikainen, M. (2006). Face desc ip ion wi h local bina y
pa e ns: Applica ion o ace ecogni ion. IEEE T ansac ions on Pa e n Analysis
and Machine In elligence,28(12), 2037–2041.
Bekhouche, S., Oua i, A., Do naika, F., Taleb-Ahmed, A., & Hadid, A. (2017). Py a-
mid mul i-le el ea u es o acial demog aphic es ima ion. Expe Sys ems
wi h Applica ions,80, 297–310.
Chau, M., & Be ke, M. (2005). Real ime eye acking and blink de ec ion wi h usb
came as:Technical epo , Bos on Uni e si y Compu e Science Depa men .
Chen, W., Huang, H., Peng, S., Zhou, C., & Zhang, C. (2020). YOLO-Face: a eal- ime
ace de ec o . The Visual Compu e , 1–9.
Chen, B.-C., Wu, P.-C., & Chien, S.-Y. (2015). Real- ime eye localiza ion, blink
de ec ion, and gaze es ima ion sys em wi hou in a ed illumina ion. In 2015
IEEE in e na ional con e ence on image p ocessing (ICIP) (pp. 715–719). IEEE.
Daza, R., Mo ales, A., Fie ez, J., & Tolosana, R. (2020). mEBAL: A mul imodal
da abase o eye blink de ec ion and a en ion le el es ima ion.
158
