Knowledge-Based Sys ems 242 (2022) 108246
Con en s lis s a ailable a ScienceDi ec
Knowledge-Based Sys ems
jou nal homepage: www.else ie .com/loca e/knosys
Deep lea ning based ace beau y p edic ion ia dynamic obus losses
and ensemble eg ession
F. Bougou zi b, F. Do naika a,c,d,∗, A. Taleb-Ahmed e
aSchool o Compu e and In o ma ion Enginee ing, Henan Uni e si y, Kai eng, China
bIns i u e o Applied Sciences and In elligen Sys ems, Na ional Resea ch Council o I aly, 73100 Lecce, I aly
cUni e si y o he Basque Coun y UPV/EHU, San Sebas ian, Spain
dIKERBASQUE, Basque Founda ion o Science, Bilbao, Spain
eUni . Poly echnique Hau s-de-F ance, Uni . Lille, CNRS, Cen ale Lille, UMR 8520 - IEMN, F-59313 Valenciennes, F ance
a icle in o
A icle his o y:
Recei ed 12 Oc obe 2021
Recei ed in e ised o m 20 Decembe 2021
Accep ed 19 Janua y 2022
A ailable online 31 Janua y 2022
Keywo ds:
Facial beau y p edic ion
Con olu ional neu al ne wo k
Deep lea ning
Ensemble eg ession
Robus loss unc ions
abs ac
In he las decade, se e al s udies ha e shown ha acial a ac i eness can be lea ned by machines.
In his pape , we add ess Facial Beau y P edic ion om s a ic images. The pape con ains h ee main
con ibu ions. Fi s , we p opose a wo-b anch a chi ec u e (REX-INCEP) based on me ging he a chi-
ec u e o wo al eady ained ne wo ks o deal wi h he complica ed high-le el ea u es associa ed
wi h he FBP p oblem. Second, we in oduce he use o a dynamic law o con ol he beha iou o
he ollowing obus loss unc ions du ing aining: Pa amSmoo hL1, Hube and Tukey. Thi d, we
p opose an ensemble eg ession based on Con olu ional Neu al Ne wo ks (CNNs). In his ensemble,
we use bo h he basic ne wo ks and ou p oposed ne wo k (REX-INCEP). The p oposed indi idual CNN
eg esso s a e ained wi h di e en loss unc ions, namely MSE, dynamic Pa amSmoo hL1, dynamic
Hube and dynamic Tukey. Ou app oach is e alua ed on he SCUT-FBP5500 da abase using he wo
e alua ion scena ios p o ided by he da abase c ea o s: 60%–40% spli and i e- old c oss- alida ion.
In bo h e alua ion scena ios, ou app oach ou pe o ms he s a e o he a on se e al me ics. These
compa isons highligh he e ec i eness o he p oposed solu ions o FBP. They also show ha he
p oposed dynamic obus losses lead o mo e lexible and accu a e es ima o s.
©2022 The Au ho s. Published by Else ie B.V. This is an open access a icle unde he CC BY-NC-ND
license (h p://c ea i ecommons.o g/licenses/by-nc-nd/4.0/).
1. In oduc ion
Fo cen u ies philosophe s, a is s, and scien is s ha e ied o
disco e he mys e y o beau y [1]. In ac , he beau y o he ace
is gaining mo e and mo e in e es due o he apid de elopmen
o plas ic su ge y and cosme ic indus y [2]. In ecen yea s, acial
beau y es ima ion and classi ica ion has become an in e es ing
esea ch opic in compu e ision and machine lea ning due o
i s g owing applica ions [3,4]. Applica ions o acial beau y es-
ima ion and p edic ion include: Cosme ic ecommenda ions [5],
scheduling o aes he ic su ge ies [6], acial beau i ica ion [7], and
Social Ne wo ks Se ices (SNS) (such as Facebook, Ins ag am,
and da ing websi es) [8]. In addi ion, au oma ic acial beau y
p edic ion (FBP) may ind applica ion when a ac i eness is a
basic equi emen , such as in ad e ising, magazine co e s, and
sc eening applican s o ce ain jobs, such as en e ainmen and
modelling [6].
Despi e he conside able p og ess in es ima ing and p edic ing
he beau y o aces, mo e labelled da a is needed o aining deep
∗Co esponding au ho a : Uni e si y o he Basque Coun y UPV/EHU, San
Sebas ian, Spain.
E-mail add ess: [email p o ec ed] (F. Do naika).
CNNs. To deal wi h he da a limi a ion, some esea che s used
ac i e da a augmen a ion. In addi ion, o he s used p e- ained
models ained on he ImageNe da abase [9]. These p e- ained
models a e capable o ex ac ing high-le el ea u es. In his pa-
pe , we p opose a sys em ha exploi s he di e si y o lea ne s.
We p esen wo main p oposals. Fi s , we p opose o combine
wo di e en CNN a chi ec u es in o a single a chi ec u e (called
he wo-b anch a chi ec u e) ha is ained end- o-end. Second,
we p opose o build an ensemble o eg essions whe e he i-
nal p edic ion is gi en by he a e age o all p edic ions. The
la e solu ion does no need o be ained on new alida ion
se s. Mo e speci ically, we p opose ensemble eg essions using
one-b anch a chi ec u es (ResneX -50 and Incep ion- 3) and ou
p oposed wo-b anch a chi ec u e (REX-INCEP) ained wi h di -
e en loss unc ions. Fou loss unc ions a e used in ou app oach,
namely MSE, dynamic Pa amSmoo hL1, dynamic Hube and dy-
namic Tukey. In summa y, he main con ibu ions o his pape
a e as ollows:
•We p opose Pa amSmoo hL1 eg ession loss unc ion
Pa amSmoo hL1. Mo eo e , we in oduce a dynamic law
ha changes he pa ame e s o he obus loss unc ion
du ing aining. Fo his pu pose, we use he cosine law
h ps://doi.o g/10.1016/j.knosys.2022.108246
0950-7051/©2022 The Au ho s. Published by Else ie B.V. This is an open access a icle unde he CC BY-NC-ND license (h p://c ea i ecommons.o g/licenses/by-
nc-nd/4.0/).
F. Bougou zi, F. Do naika and A. Taleb-Ahmed Knowledge-Based Sys ems 242 (2022) 108246
wi h he ollowing obus loss unc ions: Pa amSmoo hL1,
Hube and Tukey. This can sol e he p oblem o complexi y
in inding he bes loss unc ion pa ame e .
•We p opose wo b anches ne wo k (REX-INCEP) o ace
beau y es ima ion based on ResneX -50 and Incep ion- 3 a -
chi ec u es. The main ad an age o ou REX-INCEP a chi ec-
u e is i s abili y o lea n FBP ea u es a a high le el by using
ResneX and Incep ion blocks simul aneously, which p o ed
i s e iciency compa ed o se en CNN a chi ec u es. Mo e-
o e , ou REX-INCEP a chi ec u e p o ides he igh ade-
o be ween he pe o mance and he numbe o pa ame e s
o acial beau y p edic ion.
•We p opose an ensemble eg ession o acial beau y es i-
ma ion by me ging he p edic ed sco es o ne wo ks wi h
one b anch ne wo ks (ResneX -50 and Incep ion- 3) and
wo b anches ne wo k (REX-INCEP) ained wi h ou loss
unc ions (MSE, dynamic Pa amSmoo hL1, dynamic Hube ,
and dynamic Tukey). Al hough he indi idual eg ession
models a e ained wi h he same ixed hype pa ame e s,
he esul ing ensemble eg ession p o ides he mos accu-
a e es ima es compa ed o he indi idual models and o
s a e-o - he-a solu ions. We ha e made ou codes and p e-
p ocessed aces publicly a ailable using ou ace alignmen
scheme a h ps://gi hub.com/ a esbougou zi/CNN-ER_ o _
FBP. (Las accessed on No embe , 29 h 2021)
This pape is o ganized as ollows: Sec ion 2p esen s some e-
la ed wo k on acial beau y p edic ion. In Sec ion 3, we illus a e
he backbone CNN a chi ec u es used, he p oposed app oach,
and he p oposed dynamic obus losses. Sec ion 4includes: he
desc ip ion o he da abase and e alua ion me ics used, he
expe imen al se ings, and he 60%–40% spli and i e old c oss-
alida ion expe imen s. Sec ion 4is concluded by he compa ison
wi h s a e-o - he-a me hods. Finally, Sec ion 5concludes he
pape .
2. Rela ed wo k
Image-based es ima ion o he beau y o aces is a new p ob-
lem in compu e ision. The i s da abase ha ea s FBP as
a eg ession ask is om 2015 [10]. The me hods used in he
li e a u e o p edic and es ima e acial beau y a e ei he hand-
c a ed me hods [11–18] o deep lea ning me hods [12,19–21].
The hand-c a ed me hods can be classi ied as geome y-based
o appea ance-based me hods [16]. In [15], P. Aa abi e al. de-
eloped an au oma ic acial beau y a ing sys em based on he
ela ionships be ween acial ea u es ( ace, eyes, eyeb ows and
mou h) wi h he K-nea es neighbou algo i hm o lea n a beau y
mapping. D. Zhang e al. used ens o housands o emale and
male aces and assigned hem o a human ace shape subspace.
They hen used a quan i a i e me hod o analyse he e ec o
geome ic acial ea u es on human acial beau y using a simi-
la i y ans o ma ion in a ian shape dis ance measu e. In [16],
H. Yan p oposed a new CSOR (Cos -Sensi i e O dinal Reg ession)
o measu e he impo ance o samples in di e en classes. They
applied hei CSOR o ou ypes o ea u es, namely in ensi y,
LBP [22], SIFT [23], and LE [24]. L. Liang e al. [12] used geome -
ic ea u es (ex ac ed 18-dimensional a io ea u es om aces)
and appea ance ea u es (40 Gabo ea u e maps) wi h shallow
p edic o s, which a e linea eg ession (LR), Gaussian eg ession
(GR), and suppo ec o eg ession (SVR). These me hods we e
es ed using he SCUT-FBP5500 da abase.
In ecen yea s, deep lea ning a chi ec u es ha e been widely
used o e alua e he beau y o aces. In [12], L. Liang e al. p e-
sen ed hei ace beau y da abase (SCUT-FBP5500) wi h wo e al-
ua ion p o ocols (60%–40% spli and 5- old c oss alida ion). They
es ed h ee CNN a chi ec u es (Alexne [25], Resne -18 [26] and
ResneX -50 [27]). Thei esul s show ha he ResneX -50 a chi-
ec u e ou pe o med he o he wo deep a chi ec u es (Alexne
and Resne -18). Mo eo e , he deep a chi ec u es pe o med be -
e han he hand-c a ed ea u es hey used wi h di e en shal-
low eg esso s. K. Cao e al. used a esidual-in- esidual (RIR)
block o build a deepe ne wo k wi h mul i-le el skip connec ions
o p oduce be e g adien ansmission low. In addi ion, hey
used bo h channel-wise and space-wise a en ion mechanisms
o ind he inhe en co ela ion be ween ea u e maps. Thei
app oach was es ed on he SCUT -FBP5500 [12] da abase and
showed good pe o mance. In [21], L. Lin e al. p opose an R3CNN
a chi ec u e consis ing o wo main componen s. The i s com-
ponen is a eg ession componen ha con ains wo iden ical
eg ession subne s ha consis en ly map each ace image o a
beau y alue. The second componen is a anking componen ha
uses he Siamese ne wo k o lea n a pai wise anking o guide he
eg ession o he beau y p edic ion. Thei a chi ec u e showed
p omising esul s on SCUT-FBP [10] and he SCUT-FBP5500 [12]
da abases. In addi ion o supe ised lea ning, semi-supe ised
lea ning shows p omising esul s o ace beau y es ima ion [28,
29]. In [29], F. Do naika e al. p esen ed a mul i-laye ed local dis-
c iminan embedding algo i hm ha in eg a es ea u e selec ion
as he main s ep. Fea u e selec ion cap u es he mos ele an and
disc imina i e ea u es o an inpu ace image o ace desc ip o .
3. Me hodology
In his sec ion, we will p esen he used CNN a chi ec u es and
ou p oposed app oach and he p oposed dynamic obus losses.
3.1. Backbone CNN a chi ec u es
Since deep lea ning a chi ec u e ‘‘Alexne ’’ [25] won he Ima-
geNe challenge in 2012, nume ous CNN a chi ec u es ha e been
p oposed. In ou wo k, we will use wo popula a chi ec u es
(ResneX -50 [27] and Incep ion- 3 [30]) as building blocks o
ou solu ion. I is wo h no ing ha o he backbone a chi ec-
u es can also be used. In ou p oposal, we use he abo e p e-
ained models ained on he ImageNe challenge da abase [9].
To keep he pape sel -con ained, his sec ion b ie ly in oduces
he CNNs ResneX -50 and Incep ion- 3, which we e used as
backbone a chi ec u es in ou p oposed solu ion.
ResneX -50 a chi ec u e. The a chi ec u e o ResneX -50 is p e-
sen ed in [27], which is based on he ResneX module (Fig. 1). The
ResneX module pe o ms a se ies o ans o ma ions, each based
on a low-dimensional embedding and sha ing he same opology.
The esul s o all ans o ma ions a e combined by summa ion.
Incep ion- 3 a chi ec u e. The Incep ion- 3 a chi ec u e is p e-
sen ed in [30], which is based on he Incep ion module p esen ed
in [31]. The main idea o he incep ion a chi ec u e is o com-
bine di e en con olu ional laye s wi h di e en ke nel sizes and
pooling laye s in one incep ion module, as shown in Fig. 2.
3.2. Ou app oach
Ou global app oach is shown in Fig. 3. The ou pu sco e is
he a e age o mul iple sco es, which means ha we use an
ensemble o mul iple eg ession models. In ou implemen a ion,
we use six models. The e a e wo main con ibu ions in his
ensemble: (i) he deep ne wo k wi h wo b anches (REX-INCEP)
(see Sec ion 3.4) and (ii) he dynamic obus loss unc ions (see
Sec ion 3.5).
The i s wo sco es a e p edic ed by he ained deep ne -
wo ks ResneX -50 and Incep ion- 3 using he MSE loss unc ion
2
F. Bougou zi, F. Do naika and A. Taleb-Ahmed Knowledge-Based Sys ems 242 (2022) 108246
Fig. 1. A ResneX Module wi h ca dinali y = 32, wi h oughly he same
complexi y. A laye is shown as (# in channels, il e size, # ou channels) [27].
Fig. 2. The Thi d Incep ion Module used in Incep ion- 3 a chi ec u e [30].
and he dynamic Hube loss unc ion, espec i ely. The emaining
ou sco es a e es ima ed a e aining he p oposed wo-b anch
deep ne wo k (REX-INCEP) using ou loss unc ions: MSE, dy-
namic Pa amSmoo hL1, dynamic Hube , and dynamic Tukey. The
wo-b anch deep ne wo k consis s o ResneX -50 and incep ion-
3, which a e me ged in o a single a chi ec u e. As will be seen
in he expe imen al sec ion, using he wo con ibu ions wi hou
he ensemble esul s in pe o mance ha is be e han ha o
he s a e-o - he-a me hods. Using he ensemble shown in Fig. 3
will u he imp o e he esul s.
3.3. Face p ep ocessing
In he ace p ep ocessing phase, we used he scheme p oposed
in [32,33]. The p ocess is shown in Fig. 4. The e a e h ee s eps
o c op he ace egion om he aw ace image. Fi s , we use
he p o ided ace ea u es o align he eyes by pe o ming a 2D
o a ion o he ace based on he eye coo dina es. A e his 2D
o a ion o he image and he de ec ed poin s, he h ee a hes
poin s in he le , igh and bo om di ec ions a e selec ed as
he h ee bounda ies o he ace. We deno e he dis ance om
he lowe bounda y o he posi ion o he eyes as d1. The uppe
bounda y o he ace is se wi h a dis ance d2 om he eyes, which
is se as d2= 0.6 d1. Finally, ob ain he ace ROI, by c opping
he ace using he ou bounda ies and esizing he ob ained
box image o a ixed size ha depends on he inpu size o he
co esponding ne wo k.
3.4. Two b anches a chi ec u e
In ecen yea s, many success ul deep a chi ec u es ha e been
p oposed o many compu e ision asks.
To ain wo a chi ec u es simul aneously, we p opose wo
b anches a chi ec u e o exploi he di e en capabili ies o he
ne wo ks. Since FBP da a is limi ed, we p opose o exploi he
low-le el and high-le el ea u e ex ac ion capabili y o wo
powe ul a chi ec u es simul aneously. Fig. 5 summa izes ou
p oposed a chi ec u e wi h wo b anches. The i s and second
b anches a e he ResneX -50 and Incep ion- 3 a chi ec u es, e-
spec i ely, wi h he decision le els emo ed. In ou p oposed
a chi ec u e wi h wo b anches, we added he laye FC1 ha
maps he ou pu o he ResneX -50 b anch ( ec o o dimension
2048) o 1024 neu ons. Simila ly, we added laye FC2, which
maps he ou pu o he Incep ion- 3 b anch ( ec o o dimension
2048) o 1024 neu ons. FC1 and FC2 we e conca ena ed in o a
single ec o FC, which is ollowed by he FC2 laye ha pe o ms
he eg ession. No e ha he weigh s o bo h b anches a e he
weigh s o he p e- ained ResneX -50 and Incep ion- 3 models
( ained on he ImageNe challenge da abase [9].), while he
FC1, FC2 and FC3 laye s a e andomly ini ialized. Ou p oposed
ne wo k wi h wo b anches is called REX-INCEP a chi ec u e. In
he aining phase, we will ine- une his a chi ec u e o FBP.
3.5. Loss unc ions: he use o dynamic obus losses
Du ing con olu ional ne wo k aining, he loss unc ion mea-
su es he e o ( he loss) be ween he g ound u h and he
es ima ed alues. The CNNs aim o minimize he loss based on
he g adien s o he loss unc ion used o upda e he weigh s o
he ne wo k. In his sec ion, we will desc ibe he loss unc ions
used in ou expe imen s. We emphasize ha h ee o hem a e
obus loss unc ions. We will also in oduce a dynamic law ha
adjus s he pa ame e s o he obus losses du ing aining. The
losses a e compu ed o he ba ch size N,yideno es he g ound
u h sco e o he i h image, and ˆ
yideno es he es ima ed alue
co esponding o he i h image.
3.5.1. L1Loss unc ion
L1is one o he mos commonly used loss unc ions. The mos
impo an p ope y o he L1loss unc ion is i s obus ness o
ou lie s. Fo Nba ch size, L1loss unc ion is de ined by:
LL1=1
N
N
∑
i=1
|yi−ˆ
yi|(1)
3.5.2. Mean Squa ed E o (MSE) loss unc ion
MSE is also known as L2loss unc ion, i is mo e sensi i e o
ou lie s compa ed o L1. The MSE loss unc ion should be used
when he a ge da a a e no mally dis ibu ed a ound a mean and
when i is impo an o penalize ou lie s pa icula ly hea ily. Fo
Np edic ions, he MSE loss unc ion is de ined by:
LMSE =1
N
N
∑
i=1
(yi−ˆ
yi)2(2)
3.5.3. Dynamic pa ame e ized Smoo hL1 (Pa amSmoo hL1) loss unc-
ion
The loss unc ion Smoo hL1 c ea es a c i e ion ha uses a
quad a ic e m i he absolu e elemen -wise e o alls below 1,
and an L1 e m o he wise. I is less sensi i e o ou lie s han
he MSE loss unc ion, and in some cases p e en s exploding
g adien s [34]. The Smoo hL1 loss unc ion o Nimages is de ined
by:
LSmoo hL1=1
N
N
∑
i=1
zi(3)
3
F. Bougou zi, F. Do naika and A. Taleb-Ahmed Knowledge-Based Sys ems 242 (2022) 108246
Fig. 3. Gene al s uc u e o he p oposed app oach (CNN-ER).
Fig. 4. Face Region o In e es . The le image is an o iginal image om he da abase SCUT-FBP5500 [12]. The second image is he o a ed ace wi h i s 86 de ec ed
landma ks used o es ima e he h ee ace bounda ies ( igh , le and bo om). These bounda ies co espond o he h ee poin s ∗ma ked in blue. The hi d image
shows how he uppe bounda y o he ace is de e mined. I is loca ed a a dis ance d2=0.6d1 om he e ical posi ion o he eyes. The ou h image shows he
c opped and escaled ace image wi h 224 pixels. No e ha he dis ances D1and D2a e cons an o all c opped aces.
Fig. 5. Ou p oposed wo b anches ne wo k REX-INCEP.
whe e Nis he ba ch size and ziis gi en by:
zi={0.5 (yi−ˆ
yi)2,i |yi−ˆ
yi|<1
|yi−ˆ
yi| − 0.5,o he wise (4)
Since he h eshold may a y om one ask o ano he , we p o-
posed a Pa ame e ized Smoo hL1 loss unc ion de ined as ollows:
LPa a_Smoo hL1=1
N
N
∑
i=1
zi(5)
whe e Nis he ba ch size and ziis gi en by:
zi={0.5 (yi−ˆ
yi)2,i |yi−ˆ
yi| ≤ α
|yi−ˆ
yi| + 0.5α2−α, o he wise (6)
whe e αis unable pa ame e . Ou p oposed dynamic obus
loss unc ions a e based on he ollowing obse a ion. Du ing he
aining o Con Ne s, he obus loss unc ions can be adjus ed
as he aining p og esses. Namely, du ing aining, he model
e ol es and he ou lie examples may a y. In he ea ly s ages
o aining, he model is usually nei he e y s able no accu a e
enough o handle he ou lie examples. The e o e, i is ecom-
mended o use he quad a ic unc ion o loss. A he end o he
aining, he model may be mo e o less accu a e o deal wi h he
ou lie s. The e o e, i is ecommended o use he obus loss unc-
ion whe e he ange o non-ou lie e o s is ela i ely small. This
means ha he pa ame e o he obus loss unc ion s a s wi h
a maximum alue and dec eases mono onically as he aining
p og esses. F om a p ac ical poin o iew, i is ex emely di icul
o know he bes alue o αin ad ance. Howe e , he a ia ion
in e al [αmin, αmax]can be known in ad ance. The e o e, o make
he obus loss unc ion mo e adap i e o he aining p og ess,
we p opose a dynamic pa ame e α. This pa ame e ollows a
cosine law as a unc ion o he epoch numbe . The cu en alue
o αis gi en by:
αcu =αmin +1
2(αmax −αmin)(1+cos (ecu
ne
π))(7)
whe e αcu is he alue o αa he cu en epoch (ecu ). The la e
a ies be ween 1 and he o al numbe o epochs (ne). αmax and
αmin a e he maximum and minimum o he α alue. In his pape ,
we deno e he p oposed dynamic Pa ame e ized Smoo hL1 by
dynamic Pa amSmoo hL1. Fig. 6 shows he alues o αusing he
p oposed law (Eq. (7)) as a unc ion o epoch numbe . He e αmax
and αmin a e ixed a 0.7 and 0.3, espec i ely. Ou dynamic law
was inspi ed by he dynamic law used o con ol he lea ning a e
in s ochas ic g adien descen me hods [35].
3.5.4. Dynamic Hube loss unc ion
Simila o Pa amSmoo hL1, Hube is ano he loss unc ion ha
is less sensi i e o ou lie s in he da a han he L2loss unc ion L2.
Fo N aining images, he Hube loss unc ion is de ined by [36]:
LHube =1
N
N
∑
i=1
zi(8)
whe e Nis he ba ch size and ziis de ined by:
zi={0.5 (yi−ˆ
yi)2,i |yi−ˆ
yi| ≤ β
β|yi−ˆ
yi| − 0.5β2,o he wise (9)
4
F. Bougou zi, F. Do naika and A. Taleb-Ahmed Knowledge-Based Sys ems 242 (2022) 108246
Fig. 6. Dynamic Pa amSmoo hL1 wi h α ha dec eases om 0.7 o 0.3.
whe e βis a con olled pa ame e . Fig. 7, shows a isualiza ion o
he Hube loss unc ion wi h ou β alues (0.7, 0.5, 0.3 and 0.1)
and L2loss unc ion.
Simila o he Pa amSmoo hL1 loss, we p opose o use he
dynamic βgi en by he ollowing he equa ion:
βcu =βmin +1
2(βmax −βmin)(1+cos (ecu
ne
π))(10)
whe e βcu is he alue o βin he cu en epoch (ecu ), whe e ecu
inc eases om 1 o he o al numbe o epochs (ne). βmax and βmin
a e he maximum and minimum o he β alue.
3.5.5. Dynamic Tukey loss unc ion
The Tukey loss unc ion [37] has he p ope y o supp essing
he in luence o ou lie s du ing backp opaga ion by educing he
magni ude o hei g adien owa ds ze o. Ano he in e es ing
p ope y o his loss unc ion is he so cons ain s i imposes
be ween inlie s and ou lie s [38]. The Tukey loss unc ion is
de ined by:
LTukey =1
N
N
∑
i=1
zi(11)
whe e Nis he ba ch size and ziis gi en by:
zi={c2
6[1−(1 −(|yi−ˆ
yi|
c)2)3],i |yi−ˆ
yi| ≤ c
c2
6,o he wise
(12)
whe e cis an adjus able pa ame e . Simila o Pa amSmoo hL1
and Hube losses, we p opose o use dynamic cdu ing aining
h ough he equa ion:
ccu =cmin +1
2(cmax −cmin)(1+cos (ecu
ne
π))(13)
whe e ccu is he alue o ca he cu en epoch (ecu ), whe e ecu
inc eases om 1 o he o al numbe o epochs (ne). cmax and cmin
a e he maximum and minimum o he c alue.
4. Pe o mance e alua ion
4.1. Da abase and e alua ion p o ocols
To e alua e he pe o mance o ou app oach, we used he
da abase SCUT-FBP5500 [12]. I consis s o 5500 on al aces o
subjec s wi h di e en a ibu es: Age ( om 15 o 60), gende
(male/ emale), and e hnici y (Asian/Caucasian). Each ace image
was gi en a beau y sco e in he ange [1–5] by 60 olun ee s. In
addi ion, each ace image con ains 86 acial ea u es. Figs. 8(a),
8(b),8(d) and 8(c) show some ace samples wi h hei co -
esponding beau y a ings. The c ea o s o he SCUT-FBP5500
da abase p o ided wo e alua ion scena ios [12]. In he i s
scena io, he da a we e spli in o a aining spli and a es spli
(60%–40%). In he second scena io, he da a was spli in o 5 olds
o pe o m a i e- old c oss- alida ion. In ou analyses, we will
use bo h scena ios.
4.2. E alua ion me ics
To e alua e he pe o mance o each model, ou e alua ion
me ics a e used, namely: mean absolu e e o (MAE), oo mean
squa e e o (RMSE), Pea son co ela ion coe icien (PC) and he
ϵe o . Conside Y=(y1,y2,...,yn) he g ound- u h sco es
o he es ed nimages and ˆ
Y=(ˆ
y1,ˆ
y2,..., ˆ
yn) as hei co e-
sponding es ima ed sco es. Whe e n ep esen s he numbe o
ace images es ed. The e alua ion me ics a e de ined as ollows:
Mean Absolu e E o (MAE): MAE is de ined by:
MAE =1
n
n
∑
i=1
|yi−ˆ
yi|(14)
MAE is a scale-dependen accu acy measu emen , i.e. MAE uses
he same scale as he da a being measu ed.
Roo Mean Squa e E o (RMSE): RMSE is de ined by:
RMSE =1
n
n
∑
i=1
(yi−ˆ
yi)2(15)
The RMSE is ano he scale-dependen accu acy measu e. Unlike
MAE, he e ec o any e o on he RMSE is p opo ional o he
squa ed e o ; he e o e, la ge e o s ha e a disp opo iona ely
la ge e ec on he inal RMSE. Consequen ly, he RMSE is sensi i e
o ou lie s.
Pea son Co ela ion coe icien (PC): PC was de eloped by Ka l
Pea son [39] and i is de ined by:
PC =∑n
i=1(yi−yi) ( ˆ
yi−ˆ
yi)
√∑n
i=1(yi−yi)2√∑n
i=1(ˆ
yi−ˆ
yi)2
(16)
whe e yiand ˆ
yia e he means o he g ound- u h sco es and he
es ima ed sco es, espec i ely. PC has a alue be ween +1 and -1,
i is a s a is ic ha measu es he linea co ela ion be ween wo
a iables Yand ˆ
Y. A alue o +1 means a comple ely posi i e
linea co ela ion, 0 means no linea co ela ion, and −1 means
a comple ely nega i e linea co ela ion.
ϵ-e o : ϵ-e o is de ined by:
ϵ-e o =1
n
n
∑
i=1(1−exp ((yi−ˆ
yi)2
2σ2
i)) (17)
whe e σiis he s anda d de ia ion o he sco es o all a e s
o image i. The alue o he ϵe o is he accumula ion o he
e o s o he indi idual images ibased on he e m ϵ-e o i=
1−exp ((yi−ˆ
yi)2
2σ2
i
). When he absolu e e o o image iapp oaches
ze o (i.e., yi=ˆ
yi), ϵ-e o iis ze o. On he o he hand, when he
absolu e e o is la ge, ϵ-e o akes in o accoun he unce ain y
o he a e ep esen ed by σ2
i. Mo e p ecisely, he di ision by he
e m σ2
icon ibu es less o he alue o he ϵe o when he
unce ain y o he a e is la ge and ice e sa.
5
F. Bougou zi, F. Do naika and A. Taleb-Ahmed Knowledge-Based Sys ems 242 (2022) 108246
Fig. 7. Visualiza ion o wo loss unc ions: L2and Hube wi h ou β alues (0.7, 0.5, 0.3 and 0.1).
Fig. 8. Facial beau y samples om he SCUT-FBP5500 da abase, (a) Female Asian
samples and he co esponding sco es a e ( om le o igh ): 1.88, 3.00, 3.93,
and 4.28. (b) Male Asian samples and he co esponding sco es a e ( om le
o igh ): 1.73, 2.48, 3.53, and 4.43. (c) Female Caucasian samples and he
co esponding sco es a e ( om le o igh ): 1.93, 2.87, 3.63, and 4.7. (d) Male
Caucasian samples and he co esponding sco es a e ( om le o igh ): 1.88,
2.67, 3.27, and 4.43.
4.3. Expe imen al se up
All expe imen s we e pe o med on Py o ch [40] wi h an
NVIDIA Ge o ce GTX 1060 6 GB GPU. All ne wo ks a e ained
o 40 epochs using Adam op imize [41] and ba ch size o 15.
The ini ial lea ning a e is 1e−4 o 20 epochs, hen he lea ning
a e dec eases o 1e−5 o he nex 10 epochs, and o he
las 10 epochs he lea ning a e dec eases o 1e−6. Ac i e da a
augmen a ion is pe o med by o a ing he inpu ace by an angle
be ween [−5, 5]. Fo all expe imen s, he epo ed esul s co e-
spond o he bes PC o he es da a du ing he aining/ es ing
o he 40 epochs.
4.4. Expe imen al esul s on he 60%–40% spli scena io
In his sec ion, we limi he s udy o he p o ided 60%–40%
spli .
4.4.1. Raw inpu s he p oposed ace p ep ocessing
To in es iga e he e ec i eness o ou p oposed ace p ep o-
cessing me hod, we used ResneX -50 and Incep ion- 3 wi h loss
unc ion MSE o ain FBP on wo inpu image scena ios ( he
aw image and he c opped ace wi h ou p ep ocessing me hod).
The ob ained esul s a e summa ized in Table 1. F om hese
esul s, i can be seen ha ou p ep ocessing scheme imp o es
he esul s o bo h ResneX -50 and Incep ion- 3 a chi ec u es.
In o he wo ds, ou p oposed ace alignmen scheme can suppo
he aining o CNN a chi ec u es by disca ding he backg ound
ea u es and p io i izing he ace ea u es.
4.4.2. FBP using CNN a chi ec u es
In his sec ion, we compa e he pe o mance o se en CNN a -
chi ec u es (VGG-16 [42], Resne -50 [44], Resne -101 [44],
Resne -152 [44], Wide-Resne [43], Incep ion- 3 [30], ResneX -
50 [27]) using he s anda d MSE loss unc ion. The esul s a e
summa ized in Table 2. Based on hese esul s, we can conclude
ha Incep ion- 3 and ResneX -50 pe o m he bes compa ed
o he o he CNN a chi ec u es. Mo eo e , hese wo CNN a -
chi ec u es ha e a smalle numbe o pa ame e s han VGG-
16, Resne -101, Resne -152, Wide-Resne and simila o Resne -
50, which p o es he abili y o Incep ion- 3 and ResneX -50 o
lea n high-le el ea u es o FBP wi h an in e media e numbe o
pa ame e s.
Gi en he obse ed e iciency o he Incep ion- 3 and ResneX -
50 a chi ec u es in bo h pe o mance and numbe o ainable
pa ame e s (Table 2), we p opose o combine hese wo CNN
a chi ec u es (REX-INCEP). F om Table 2, we conclude ha ou
p oposed REX-INCEP achie es be e pe o mance han all CNN
a chi ec u es. Mo eo e , he numbe o ainable pa ame e s o
ou p oposed REX-INCEP is simila o Resne -101, Resne -152 and
Wide-Resne and less han VGG-16. These ad an ages p o e he
6
F. Bougou zi, F. Do naika and A. Taleb-Ahmed Knowledge-Based Sys ems 242 (2022) 108246
Table 1
Face beau y p edic ion using ResneX -50 and Incep ion- 3 ne wo ks wi h MSE loss unc ion and
wo inpu image scena ios (The aw image and he de ec ed ace wi h ou p ep ocessing scheme).
CNN a chi ec u e P e-p ocessing PC ↑MAE ↓RMSE ↓ϵ-e o ↓
ResneX -50 Raw image 0.9119 0.2126 0.2845 0.0853
ResneX -50 Face de ec ion 0.9146 0.2092 0.2763 0.0802
Incep ion- 3 Raw image 0.9108 0.2150 0.2831 0.0873
Incep ion- 3 Face de ec ion 0.9112 0.2147 0.2814 0.0833
Table 2
Compa ison be ween se en CNN a chi ec u es (VGG-16, Resne -50, Resne -101, Resne -152, Wide-
Resne , Incep ion- 3 and ResneX -50) and ou p oposed REX-INCEP app oach o Facial Beau y
P edic ion wi h MSE loss unc ion.
CNN a chi ec u e PC ↑MAE ↓RMSE ↓ϵ-e o ↓N o pa ams
VGG-16 [42] 0.9025 0.2229 0.2932 0.0891 134 M
Wide-Resne [43] 0.9066 0.2176 0.2889 0.0851 66 M
Resne -50 [44] 0.9087 0.2155 0.2850 0.0849 23 M
Resne -152 [44] 0.9069 0.2182 0.2880 0.0860 58 M
Resne -101 [44] 0.9095 0.2157 0.2852 0.0848 44 M
Incep ion- 3 [30] 0.9139 0.2125 0.2779 0.0819 25 M
ResneX -50 [27] 0.9146 0.2092 0.2763 0.0802 22 M
REX-INCEP (Ou a chi ec u e) 0.9159 0.2071 0.2739 0.0790 52 M
e iciency o ou p oposed REX-INCEP o FBP compa ed o CNN
a chi ec u es.
As shown in Fig. 5, ou p oposed REX-INCEP has wo b anches.
In he i s b anch, ou p oposed REX-INCEP a chi ec u e is able
o lea n high-le el ea u es o FBP by using a combina ion o
spli ing, ans o ma ion and agg ega ion mechanisms h ough
he ResneX block. In he second b anch, ou p oposed REX-INCEP
a chi ec u e is able o lea n high-le el ea u es o FBP by combin-
ing di e en con olu ional laye s wi h di e en ke nel sizes and
pooling laye s h ough he Incep ion blocks. The main ad an age
o ou REX-INCEP a chi ec u e is i s abili y o lea n high-le el
FBP ea u es using ResneX and Incep ion blocks simul aneously,
which p o ed i s e iciency compa ed o se en CNN a chi ec u es.
F om he esul s in Table 2, we conclude ha ou REX-INCEP
a chi ec u e p o ides he igh adeo be ween he pe o mance
and he numbe o pa ame e s o acial beau y p edic ion.
4.4.3. Dynamic s ixed loss pa ame e
In his sec ion, we compa e he pe o mance o Face Beau y
P edic ion wi h dynamic and ixed loss unc ions. In his se o
expe imen s, we choose a CNN a chi ec u e and a pa ame ic
obus loss unc ion. We hen compa e he pe o mance o wo
a ian s o his pa ame ic obus loss unc ion: (i) a loss unc ion
ha assumes a ixed pa ame e , and (ii) a loss unc ion ha
assumes a dynamic pa ame e using he cosine law. Speci ically,
we use he ResneX -50 ne wo k and he ollowing pa ame ic
loss unc ions: Pa amSmoo hL1, Hube and Tukey loss unc ions.
To p o ide a ai compa ison, he in e al o pa ame e a ia ion
associa ed wi h he dynamic scheme is also used by he ixed
pa ame e loss unc ion. This is achie ed by epea ing he aining
and es ing wi h se e al ixed alues om he same in e al.
We compa e he pe o mance ob ained wi h he dynamic
scheme wi h he a e age pe o mance associa ed wi h he
spanned ixed alues wi hin his in e al. Table 3 summa izes he
esul s ob ained. Fo he loss o Pa amSmoo hL1, he in e al o
αis ixed o [0.7–0.3]. The ixed pa ame e scheme spans he ol-
lowing alues {0.7,0.6,0.5,0.4,0.3}. Based on he esul s o he
loss unc ion Pa amSmoo hL1, we can see ha he pe o mance
o he dynamic scheme is be e han he a e age pe o mance
ob ained wi h he ixed loss. Simila o Pa amSmoo hL1, he
dynamic Hube loss unc ion wi h a βpa ame e in he in e al
[0.7–0.3] achie ed be e pe o mance han he mean pe o -
mance ob ained by he ixed β alues {0.7,0.6,0.5,0.4,0.3}. Fo
Tukey loss, he in e al o pa ame e cwas se o [2-1.5] and he
ixed c alues a e {2,1.7,1.5}. Simila o Pa amSmoo hL1 and he
Hube loss unc ion, he dynamic Tukey loss p o ided be e pe -
o mance han he Tukey loss wi h a ixed c alue. Mo eo e , he
dynamic Tukey loss unc ion adop ing he [2-1] achie ed be e
pe o mance han he dynamic Tukey loss unc ion adop ing he
in e al [2-1.5]. In ou app oach, he in e als o he α,βand c
pa ame e s o he h ee loss unc ions a e [0.7–0.3] [0.7–0.3] and
[2-1], espec i ely.
4.4.4. Two b anches s one b anch using i e loss unc ions
In his sec ion, we compa e he pe o mance o single-b anch
ne wo ks (ResneX -50 and Incep ion- 3) and ha o he p oposed
wo b anches ne wo k (REX-INCEP). Table 4 shows he pe o -
mances ob ained wi h he ResneX -50 ne wo k when i e loss
unc ions we e used. F om he esul s, i can be seen ha he
loss unc ion MSE gi es he bes pe o mance. Table 5 con ains
he esul s o he Incep ion- 3 ne wo k when i e loss unc ions
we e used. F om hese esul s, we can conclude ha he dynamic
Hube loss unc ion gi es he bes pe o mance.
Table 6 shows he esul s o ou p oposed wo b anches
ne wo k (REX-INCEP) when ou loss unc ions (MSE, dynamic
Pa amSmoo hL1, dynamic Hube and dynamic Tukey losses) a e
used. Among he esul s ob ained, dynamic Pa amSmoo hL1
achie ed he bes pe o mance. Mo eo e , o a gi en loss unc-
ion, he pe o mance o he wo-b anch solu ion was be e han
ha o he one-b anch solu ion. The excep ion is he dynamic
Hube loss. Howe e , his di e ence in pe o mance is e y small.
Compa ing he esul s o one-b anch ne wo ks and wo-b anch
ne wo ks, we ind ha he wo-b anch ne wo k achie es high
pe o mance o all loss unc ions, wi h Pa amSmoo hL1 being he
bes . In con as , he one-b anch ne wo ks achie ed compe i i e
pe o mance only o he MSE loss wi h he ResneX -50 ne wo k
and he dynamic Hube loss wi h he Incep ion- 3 ne wo k. This
p o es he e ec i eness o he p oposed REX-INCEP ne wo k,
which e ec i ely uses and ans o ms he ea u es gene a ed by
each a chi ec u e.
4.4.5. CNN ensemble
To inc ease he pe o mance o FBP, we will use an ensemble
o ained CNN a chi ec u es and use di e en loss unc ions.
In his scena io, he inal sco e is se o he a e age o he
acial beau y sco es p o ided by di e en models. In his g oup
o expe imen s, six models a e adop ed: he wo ained ne -
wo ks wi h one b anch and he bes loss unc ions (ResneX -
50 wi h MSE and Incep ion- 3 wi h dynamic Hube ) and ou
7
F. Bougou zi, F. Do naika and A. Taleb-Ahmed Knowledge-Based Sys ems 242 (2022) 108246
Table 3
Compa ison be ween dynamic and ixed pa ame e s o loss unc ions Pa amSmoo hL1, Hube and
Tukey using ResneX -50 ne wo k.
Loss unc ion Pa ame e PC ↑MAE ↓RMSE ↓ϵ-e o ↓
α= 0.7 0.9122 0.2127 0.2799 0.0814
α= 0.6 0.9141 0.2098 0.2772 0.0803
Pa amSmoo hL1 α= 0.5 0.9132 0.2110 0.2780 0.0815
α= 0.4 0.9101 0.2146 0.2825 0.0833
α= 0.3 0.9116 0.2150 0.2810 0.0831
Mean 0.9123 0.2126 0.2797 0.0819
dynamic α(0.7–0.3) 0.9140 0.2104 0.2777 0.0805
β= 0.7 0.9126 0.2114 0.2796 0.0812
β= 0.6 0.9130 0.2107 0.2780 0.0804
Hube β= 0.5 0.9144 0.2111 0.2770 0.0808
β= 0.4 0.9124 0.2122 0.2783 0.0839
β= 0.3 0.9110 0.2155 0.2811 0.0845
Mean 0.9127 0.2122 0.2788 0.0822
dynamic β(0.7–0.3) 0.9141 0.2116 0.2777 0.0814
c= 2.0 0.9128 0.2155 0.2810 0.0837
c= 1.7 0.9116 0.2133 0.2805 0.0821
Tukey c= 1.5 0.9126 0.2129 0.2808 0.0824
Mean 0.9123 0.2139 0.2808 0.0827
dynamic c(2–1.5) 0.9127 0.2120 0.2801 0.0819
dynamic c(2–1) 0.9133 0.2100 0.2780 0.0802
Table 4
Facial Beau y P edic ion using ResneX -50 Ne wo k wi h i e loss unc ions (L1, MSE, dynamic
Pa amSmoo hL1, dynamic Hube and dynamic Tukey losses).
Loss unc ion PC ↑MAE ↓RMSE ↓ϵ-e o ↓
L10.9126 0.2113 0.2783 0.0810
MSE 0.9146 0.2092 0.2763 0.0802
Dyn. Pa amSmoo hL1 (0.7–0.3) 0.9140 0.2104 0.2777 0.0805
Dyn. Hube (0.7–0.3) 0.9141 0.2116 0.2777 0.0814
Dyn. Tukey (2–1) 0.9133 0.2100 0.2780 0.0802
Table 5
Facial Beau y P edic ion using Incep ion- 3 Ne wo k wi h i e loss unc ions (L1, MSE, dynamic
Smoo hL1, dynamic Hube and dynamic Tukey).
Loss unc ion PC ↑L1↓RMSE ↓ϵ-e o ↓
L10.9103 0.2152 0.2832 0.0848
MSE 0.9112 0.2147 0.2814 0.0833
Dyn. Pa amSmoo hL1 (0.7–0.3) 0.9118 0.2129 0.2805 0.0836
Dyn. Hube (0.7–0.3) 0.9139 0.2125 0.2779 0.0819
Dyn. Tukey (2–1) 0.9124 0.2138 0.2794 0.0802
Table 6
Facial Beau y P edic ion using he p oposed wo b anches Ne wo k (REX-INCEP) wi h ou loss
unc ions (MSE, dynamic Pa amSmoo hL1, dynamic Hube and dynamic Tukey losses).
Loss unc ion PC ↑MAE ↓RMSE ↓ϵ-e o ↓
MSE 0.9159 0.2071 0.2739 0.0790
Dyn. Pa amSmoo hL1 (0.7–0.3) 0.9165 0.2065 0.2736 0.0789
Dyn. Hube (0.7–0.3) 0.9138 0.2113 0.2785 0.0818
Dyn. Tukey (2–1) 0.9149 0.2105 0.2766 0.0806
ained wo b anches ne wo ks wi h ou loss unc ions (REX-
INCEP wi h MSE, dynamic Pa amSmoo hL1, dynamic Hube and
dynamic Tukey). The solu ion based on he six models is called
CNN Ensemble Reg ession (CNN-ER) and he usion scheme e e s
o he a e aging o he p edic ed sco es om mo e han one
model. Fo each model, we selec ed he inal model a e aining
i wi h 40 epochs. The esul s a e summa ized in Table 7. This
able p esen s h ee di e en ensemble solu ions. Acco ding o
he esul s depic ed in his able, we ind he mean sco es o he
models wi h wo b anches pe o m be e han he mean sco es
o he models wi h one b anch. Mo eo e , he mean sco es o he
models wi h one and wo b anches (all six models) ou pe o med
he mean sco es o he models wi h one and wo b anches.
We also no e ha he mean sco es o he one b anch ne wo ks
pe o m be e han one and wo b anches ne wo ks (Tables 4,
5, and 6). This p o es he e ec i eness o ou p oposed CNN
ensemble me hod. I is wo h no ing ha he indi idual models
we e ained only on he aining se wi h a ixed numbe o
epochs (40 epochs).
4.5. Expe imen al esul s using he i e old c oss- alida ion scena io
In his sec ion, we will use he p o ided i e olds o pe o m
he c oss- alida ion expe imen s. Table 8 con ains he esul s
ob ained wi h each old, as well as he a e age o e he i e
olds using he ne wo ks wi h one b anch (ResneX -50 wi h MSE
loss unc ion and Incep ion- 3 wi h dynamic Hube loss unc ion)
and ou ne wo ks wi h wo b anches (REX-INCEP wi h MSE,
dynamic Pa amSmoo hL1, dynamic Hube and dynamic Tukey
loss unc ion). I is wo h no ing ha he p esen ed esul o each
old co esponds o he bes esul ob ained by PC o e he es
8
F. Bougou zi, F. Do naika and A. Taleb-Ahmed Knowledge-Based Sys ems 242 (2022) 108246
Table 7
Facial Beau y P edic ion using he p oposed CNN ensemble o di e en ained models on 60%–40%
da a spli .
Fusion scheme PC ↑MAE ↓RMSE ↓ϵ-e o ↓
One b anch (Mix u e 2 models) 0.9190 0.2054 0.2705 0.0777
Two b anches (Mix u e 4 models) 0.9194 0.2043 0.2699 0.0771
CNN-ER (Mix u e 6 models) 0.9207 0.2032 0.2683 0.0764
Table 8
Fi e- old c oss- alida ion o acial beau y p edic ion using ne wo ks wi h one
b anch (ResneX -50 wi h MSE loss and Incep ion- 3 wi h dynamic Hube loss)
and wo b anches (REX-INCEP wi h MSE, dynamic Pa amSmoo hL1, dynamic
Hube and dynamic Tukey losses).
A chi ec u e Fold PC ↑MAE ↓RMSE ↓ϵ-e o ↓
Fold 1 0.9149 0.2142 0.2779 0.0799
Incep ion- 3 Fold 2 0.9138 0.2074 0.2791 0.0798
wi h dynamic Fold 3 0.9203 0.2122 0.2801 0.0800
Hube loss Fold 4 0.9226 0.2072 0.2684 0.0783
unc ion Fold 5 0.9202 0.2056 0.2706 0.0777
Mean 0.9184 0.2093 0.2752 0.0791
Fold 1 0.9176 0.2072 0.2744 0.0765
ResneX -50 Fold 2 0.9114 0.2163 0.2865 0.0845
wi h MSE loss Fold 3 0.9204 0.2106 0.2759 0.0783
unc ion Fold 4 0.9228 0.2066 0.2686 0.0767
Fold 5 0.9204 0.2053 0.2702 0.0760
Mean 0.9185 0.2092 0.2751 0.0784
Fold 1 0.9190 0.2081 0.2722 0.0772
REX-INCEP Fold 2 0.9172 0.2068 0.2755 0.0783
wi h MSE loss Fold 3 0.9212 0.2085 0.2748 0.0783
unc ion Fold 4 0.9252 0.2045 0.2654 0.0767
Fold 5 0.9213 0.2049 0.2691 0.0756
Mean 0.9208 0.2066 0.2714 0.0772
Fold 1 0.9202 0.2070 0.2718 0.0763
REX-INCEP Fold 2 0.9167 0.2056 0.2766 0.0774
wi h dynamic Fold 3 0.9206 0.2095 0.2763 0.0780
Pa amSmoo hL1 Fold 4 0.9253 0.2053 0.2634 0.0759
loss unc ion Fold 5 0.9238 0.2011 0.2639 0.0742
Mean 0.9213 0.2057 0.2704 0.0764
Fold 1 0.9196 0.2078 0.2722 0.0767
REX-INCEP Fold 2 0.9167 0.2033 0.2742 0.0771
wi h dynamic Fold 3 0.9238 0.2084 0.2717 0.0775
Hube loss Fold 4 0.9282 0.1992 0.2605 0.0720
unc ion Fold 5 0.9227 0.2041 0.2666 0.0755
Mean 0.9222 0.2046 0.2690 0.0758
Fold 1 0.9178 0.2079 0.2738 0.0766
REX-INCEP Fold 2 0.9166 0.2082 0.2782 0.0790
wi h dynamic Fold 3 0.9222 0.2089 0.2722 0.0773
Tukey loss Fold 4 0.9242 0.2036 0.2648 0.0755
unc ion Fold 5 0.9205 0.2076 0.2702 0.0770
Mean 0.9203 0.2072 0.2718 0.0771
da a du ing he aining o 40 epochs. The c oss- alida ion esul s
can p o ide a be e compa ison be ween he ne wo ks and he
loss unc ions. Compa ing he ne wo ks, we can ind ha he
ained ne wo ks wi h wo b anches ou pe o m he ne wo ks
wi h one b anch. This is consis en wi h he conclusion ound in
he 60%–40% spli scena io.
Mo eo e , we can obse e ha ResneX -50 wi h he MSE loss
unc ion and Incep ion- 3 wi h he dynamic Hube loss unc ion
achie e simila esul s. Based on he esul s ob ained wi h he
wo b anch solu ions, we can conclude ha he dynamic Hu-
be loss unc ion achie es he bes pe o mance. On he o he
hand, he dynamic loss unc ion Pa amSmoo hL1 ou pe o ms he
dynamic loss unc ions Tukey and MSE, which ob ained simila
esul s.
Table 9 shows he pe o mances achie ed by CNN ensembles.
Simila o he ensemble expe imen s in he p e ious scena io, we
conside an ensemble o one b anch ne wo ks, an ensemble o
wo b anch ne wo ks, and he mix u e ensemble o all ne wo ks.
Compa ing he esul s o he 8and 9, we can make he ollowing
obse a ions:
Table 9
Fi e olds c oss- alida ion o Facial Beau y P edic ion using he p oposed CNN
ensemble o di e en ained models.
Fusion scheme Fold PC ↑MAE ↓RMSE ↓ϵ-e o ↓
Fold 1 0.9205 0.2065 0.2703 0.0753
Fold 2 0.9170 0.2060 0.2763 0.0788
One b anch Fold 3 0.9245 0.2029 0.2691 0.0745
(2 models) Fold 4 0.9263 0.2015 0.2630 0.0747
Fold 5 0.9234 0.2010 0.2652 0.0736
Mean 0.9223 0.2036 0.2688 0.0754
Fold 1 0.9223 0.2035 0.2683 0.0743
Fold 2 0.9202 0.2018 0.2713 0.0757
Two b anches Fold 3 0.9252 0.2053 0.2698 0.0751
(4 models) Fold 4 0.9289 0.1995 0.2586 0.0730
Fold 5 0.9253 0.1994 0.2622 0.0725
Mean 0.9244 0.2019 0.2660 0.0741
Fold 1 0.9232 0.2026 0.2667 0.0735
Fold 2 0.9204 0.2016 0.2710 0.0756
CNN-ER Fold 3 0.9264 0.2029 0.2675 0.0738
(Mix u e Fold 4 0.9292 0.1990 0.2583 0.0727
6 models) Fold 5 0.9257 0.1984 0.2615 0.0720
Mean 0.9250 0.2009 0.2650 0.0735
Table 10
Compa ison wi h he s a e-o - he-a s me hods using he 60%–40% spli .
Me hod PC ↑MAE ↓RMSE ↓
LR [12] 0.5948 0.4289 0.5531
GR [12] 0.6738 0.3914 0.5085
SVR [12] 0.6668 0.3898 0.5132
Alexne [12] 0.8298 0.2938 0.3819
Resne -18 [12] 0.8513 0.2818 0.3703
ResneX -50 [12] 0.8777 0.2518 0.3325
CNN wi h SCA [19] 0.8780 0.2517 0.3320
Dynamic Pa amSmoo hL1* (Ou s) 0.9165 0.2065 0.2736
CNN-ER (Ou s) 0.9207 0.2032 0.2683
Dynamic Pa amSmoo hL* is ou REX-INCEP ne wo k ha was ained using he
dynamic Pa amSmoo hL1 loss unc ion.
•F om he esul s o he one-b anch ensemble, i can be
seen ha he usion scheme ou pe o ms he indi idual one-
b anch ne wo ks (ResneX -50 wi h MSE loss unc ion and
Incep ion- 3 wi h dynamic Hube loss unc ion).
•Fo he wo-b anch ensemble esul s, he usion scheme
ou pe o ms all he esul s ob ained by he single wo-
b anch ne wo ks (REX-INCEP wi h MSE, dynamic
Pa amSmoo hL1, dynamic Hube and dynamic Tukey loss
unc ions).
•F om he esul s o he mix u e ensemble (all six models),
i is clea ha he usion scheme ou pe o ms no only he
one- and wo-b anch ne wo ks, bu also hei used models.
The abo e obse a ions p o e he e ec i eness o he p o-
posed usion scheme. This also shows he e iciency o using wo
b anch ne wo ks wi h di e en loss unc ions.
4.6. Compa ison wi h s a e-o - he-a me hods
In his sec ion, we compa e ou p oposed me hods wi h he
s a e-o - he-a me hods in bo h scena ios: 60%–40% spli and
i e- old c oss- alida ion.
Table 10 shows a compa ison be ween ou me hod and he
s a e-o - he-a me hods using he 60%–40% spli . The compa i-
son shows ha ou app oach (CNN-ER) ou pe o ms he s a e-
9
