Accu acy Inc ease on E ol ing P oduc Uni
Neu al Ne wo ks ia Fea u e Subse Selec ion
An onio J. Tall´on-Balles e os1,2(B),Jos´e C. Riquelme2, and Robe o Ruiz1
1A ea o Compu e Science, Pablo de Ola ide Uni e si y, Se ille, Spain
[email p o ec ed]
2Depa men o Languages and Compu e Sys ems,
Uni e si y o Se ille, Se ille, Spain
Abs ac . A amewo k ha combines ea u e selec ion wi h e olu ion-
a y a ificial neu al ne wo ks is p esen ed. This pape copes wi h neu al
ne wo ks ha a e applied in classifica ion asks. In machine lea ning
a ea, ea u e selec ion is one o he mos common echniques o p e-
p ocessing he da a. A se o fil e s ha e been aken in o conside a ion
o assess he p oposal. The expe imen a ion has been conduc ed on nine
da a se s om he UCI eposi o y ha epo es e o a es abou fi -
een pe cen o abo e wi h e e ence classifie s such as C4.5 o 1-NN.
The new p oposal significan ly imp o es he baseline amewo k, bo h
app oaches based on e olu iona y p oduc uni neu al ne wo ks. Also
se e al classifie s ha e been ied in o de o illus a e he pe o mance
o he diffe en me hods conside ed.
1 In oduc ion
Many echniques add essing he classifica ion p oblem [1] ha e been p esen ed
by he machine lea ning communi y. Depending on he na u e o he algo i hm
we can dis inguish, among o he s, neu al ne wo ks, ule-based classifie s and
decision ees. Neu al ne wo ks models play an impo an ole in pa e n ecog-
ni ion [13]. The possible inpu s o an A ificial Neu al Ne wo k (ANN) could
be ex emely la ge in he con ex o many p ac ical p oblems. The e may be
some edundancy among diffe en inpu s. A huge numbe o inpu s o an ANN
inc ease i s size and hus equi e mo e aining da a and longe aining imes in
o de o achie e easonable gene aliza ion abili y. P e-p ocessing is o en needed
o educe he numbe o inpu s o an ANN.
This pape aims a imp o ing he accu acy and ge ing simple neu al mod-
els wi h a lowe numbe o inpu s and, i possible, con aining a lowe numbe
o hidden neu ons. The kind o he neu al ne wo ks ha a e in he scope o
he cu en wo k a e eed- o wa d neu al ne wo ks composed by p oduc uni s
in he hidden laye . Basically, he aining o he models is ca ied ou by an
e olu iona y p og amming algo i hm [4]. Mo e conc e ely we u ilise a amewo k
ollowing a mas e -sla e app oach, whe e he mas e dis ibu es a configu a ion
o sla e p ocesses. A p elimina y s udy o his base app oach is desc ibed in [16].
Now, he no el ing edien is a p ep ocessing phase p io o he aining o he
classifica ion models.
The emainde o his pape is o ganised as ollows: Sec . 2desc ibes some
concep s abou he aining o P oduc Uni Neu al Ne wo ks (PUNNs), he
expe imen al design dis ibu ion and ea u e selec ion; Sec . 3p esen s p oposal;
Sec . 4de ails he conduc ed expe imen a ion; hen Sec . 5shows and analyzes
he esul s ob ained; finally, Sec . 6s a es he concluding ema ks.
2 Me hodology
2.1 P oduc Uni Neu al Ne wo ks and T aining P ocedu e
Among he diffe en ypes o neu al ne wo k a chi ec u es, he mos popula
a e eed- o wa d ones. Wi hin his kind, single hidden-laye ne wo ks a e e y
powe ul due o hei uni e sal app oxima ion p ope y. Mul iplica i e ANNs
[17] con ain nodes ha mul iply hei inpu s ins ead o adding hem. This class
o ne wo ks comp ises such ypes as sigma-pi ne wo ks and p oduc uni neu al
ne wo ks. The la e ype was in oduced by R. Du bin and D. Rumelha [5]
and is he s udy objec o he cu en pape . The aining o he neu al ne wo ks
is pe o med by means o an e olu iona y p og amming algo i hm o simul ane-
ously lea n he a chi ec u e and weigh s o he PUNN classifica ion model. The
neu al ne wo k opology is a h ee-laye a chi ec u e, wi h k (numbe o ea u es
o he p oblem a hand) nodes in he inpu laye , m ones and a bias one in he
hidden laye and a numbe o nodes equals o he numbe o classes minus one
in he ou pu laye . The m alue is de e mined by he aining algo i hm. The
ans e unc ion o each node in he hidden and ou pu laye s is he iden i y
unc ion. We ha e conside ed a s anda d so -max ac i a ion unc ion, associa ed
wi h he gne wo k model wi h Jclasses, gi en by:
gj(x)= exp j(x)
J
j=1 exp j(x)j=1, ..., J (1)
whe e j(x) is he ou pu o node j o pa e n xand gj(x) is he p obabili y
ha his pa e n belongs o class j. Gi en a aining se D=(xi,yi)i=
1, ..., N, a unc ion o c oss-en opy e o is used o e alua e a ne wo k g wi h
he ins ances o a p oblem, which is eflec ed in he ollowing exp ession:
l(g)=−1
N
N

i=1
J

j=1
(yj
iln(gj(xi))) (2)
and subs i u ing gjdefined in (2),
l(g)=−1
N
N

i=1
⎛
⎝−
J

j=1
yj
i j(xi)+ln(
J

j=1
exp j(xi))⎞
⎠(3)
whe e yj
iis he a ge alue o class jwi h pa e n xi(yj
i=1i xi∈class j
and yj
i= 0 o he wise), j(xi) is he ou pu alue o he neu al ne wo k o he
ou pu neu on jwi h pa e n xi. Obse e ha so -max ans o ma ion p oduces
p obabili ies ha sum o one and he e o e he ou pu s can be in e p e ed as
he condi ional p obabili y o class membe ship. Thus, he numbe o nodes in
he ou pu laye is equal o he numbe o classes minus one in he p oblem.
Since he EA objec i e is o minimize he chosen e o unc ion, a fi ness unc ion
is used in he o m A(g)=(1+l(g))−1.
The main issues abou he e olu iona y aining algo i hm a e b iefly
explained nex . The sea ch begins wi h a andom ini ial popula ion and, o
each i e a ion, he popula ion is modified using a popula ion-upda e algo-
i hm ounded on pa ame ic and s uc u al mu a ions. The algo i hm loops
a e epea ed un il he maximum numbe o gene a ions, in each case, is eached
o un il he bes indi idual o he popula ion mean fi ness does no imp o e
du ing gen −wi hou −imp o ing (20 in his pape ) gene a ions. The popu-
la ion is subjec ed o he ope a ions o eplica ion and mu a ion. Mo e de ails
and common pa ame e alues o he algo i hm a e ound in [16]. C osso e is
no used due o i s po en ial disad an ages in e ol ing a ificial ne wo ks. Wi h
hese p ope ies he algo i hm alls in o he class o e olu iona y p og amming.
2.2 Expe imen al Design Dis ibu ion
The s a ing amewo k o he cu en wo k is named Expe imen al Design Dis-
ibu ion (EDD) and ollows a mas e -sla e p og amming model. The mas e
p ocess p epa es a base configu a ion ha is dis ibu ed o all he sla e p ocesses
ha upda e he ecei ed configu a ion. Depending on he iden i y o he sla e
he ask o be pe o med is diffe en in he sense ha may ac on a conc e e
pa ame e doing a specific ope a ion wi h a single alue o he base configu-
a ion. Nex , each p ocess o e e y ype uns he aining algo i hm desc ibed
in he p e ious subsec ion using he p ope (base/upda ed) configu a ion. The
ad an age o his amewo k is ha a single configu a ion file and he numbe o
sla es o be spawned is equi ed. EDD is able o dis ibu e wo o h ee pa ame-
e s o e a maximum o eigh compu ing nodes, ha is, each p ocess is mapped
o one p ocesso ha is used in a exclusi e way. We may ha e one mas e and
se en sla e p ocesses. This is he fi s app oach published in [16]. The e we came
o he empi ical conclusion ha is e y use ul o dis ibu e h ee pa ame e s.
F om he eigh configu a ions o ha p oposal, he configu a ions ha do no
educe he numbe o gene a ions ge be e esul s. This ac mo i a es us o
only conside he eina e he fi s ou configu a ions o he app oach deli e -
ing h ee pa ame e s among he p ocessing sys em. In o he wo ds, i is jus
he same ha asse ing ha he maximum numbe o neu ons in he hidden
laye and he pa ame e alue associa ed wi h he pa ame ical mu a ion a e
dis ibu ed among ou compu a ion nodes.
2.3 Fea u e Selec ion
Fea u e selec ion may be defined as he p oblem o picking up a subse o ea u es
ha a e necessa y and sufficien o desc ibe he a ge concep [9]. A axonomy
o he ea u e selec ion algo i hms may be based on he a ibu e e alua ion
measu e: depending on he ype (fil e o w appe echnique) o on he way
ha ea u es a e e alua ed (indi idual o subse e alua ion). The fil e model
elies on gene al cha ac e is ics o he da a (such as consis ency, co ela ion and
dis ance) o assess and selec ea u e subse s wi hou in ol ing any da a mining
algo i hm. The w appe model equi es a p ede e mined mining algo i hm and
uses i s pe o mance as e alua ion c i e ion. This pape pays a en ion o ea u e
subse selec ion implemen ed as fil e s. In his con ex , i is a ac ha wo
kinds o ea u es a e gene ally pe cei ed as being unnecessa y: ea u es ha a e
i ele an o he a ge concep , and ea u es ha a e edundan gi en o he
ea u es. BIRS (Bes Inc emen al Ranked Subse [15]) me hod was p oposed in
a p e ious wo k o ob ain ele an ea u es and o emo e edundancy. These
ea u es selec ed a e conside ed as inpu a iables o he ne wo k models ha we
ge in his pape ia EDD amewo k. Since BIRS belongs o a hyb id ca ego y,
he selec ion p ocess does no ollow he ypical pa hs and is di ided in o wo
phases: in he fi s one, ea u es a e e alua ed indi idually, p o iding a anking
based on a c i e ion; in phase wo, a ea u e subse e alua o is applied o a
ce ain numbe o ea u es in he p e ious anking acco ding o a sea ch s a egy.
BIRS can use any e alua o in he wo s ages. In he cu en con ibu ion, BIRS
uses as a subse e alua o CFS (Co ela ion-based Fea u e Selec ion [6]) and
CNS (CoNSis ency based measu e [10] - ha a e es ablished on co ela ion and
consis ency concep s- a he second phase, and SOAP (Selec ion O A ibu es
by P ojec ion [14]) measu e and he own subse e alua o a he fi s phase as a
anking e alua o . Thus, in he expe imen s, spBI CNS indica es ha SOAP is
u ilised as an indi idual measu e in he fi s pa o BIRS, and CNS is employed
as a subse e alua o in he second pa . In he same way, c BI CFS deno es
ha CFS e alua o will be used in bo h pa o he BIRS algo i hm.
3 P oposal Desc ip ion
The cu en pape in oduces Expe imen al Design Dis ibu ion wi h Fea u e
Selec ion (EDDFS) amewo k, a combina ion o some FS me hods, one by one
independen ly, wi h EDD. Fi s o all, some ea u e selec o s a e applied in an
independen way o he aining se o all da a se s in o de o ob ain a lis o
a ibu es, o each o hem, ha i is conside ed o aining and es phases.
In his way, wo subse s ( aining and es subse ) a e gene a ed, whe e only
mos ele an ea u es a e included. I is impo an o ema k ha he ea u e
selec ion is pe o med only wi h aining da a; he es subse has exac ly he
same ea u es as he educed aining se . These subse s a e aken as inpu
o he e olu iona y algo i hm. EDDFS me hodology ope a es wi h ou ea u e
selec o s. As a esul o he FS s age, a lis o ele an ea u es is ob ained
wi h each o he FS me hods o each da a se . The EDDFS p ope ies a e he
Table 1. Configu a ions o he EDD (baseline) and EDDFS amewo ks
F amewo k
EDD EDDFS
Con igu a ion 1 2 3 4 1234
Neu ons (neu)neu neu +1 neu neu +1 neuneu+1 neuneu+1
Gene . (gen)gen gen gen gen gengengengen
α21 1 1.5 1.5 1 1 1.5 1.5
ollowing: (a) PUNNs ha e been u ilised, wi h a numbe o neu ons in he inpu
laye equal o he numbe o a iables in he p oblem; a hidden laye wi h a
numbe o nodes ha depends on he da a se o be classified and he numbe
o selec ed ea u es; and he numbe o nodes in he ou pu laye equal o he
numbe o classes minus one because a so -max ype p obabilis ic app oach
has been used; (b) ou expe imen s ha e been pe o med o each p oblem,
whe e wo diffe en alues ha e been used o -associa ed wi h he esidual o
he upda ing exp ession o he ou pu -laye weigh s- and he numbe o neu ons
in he hidden laye ; (c) i employs simila e minology o a o emen ioned EDD;
(d) ou diffe en configu a ions (1’, 2’, 3’ and 4’) a e applied o subse s ob ained
wi h each o he selec o s, o each da a se . The pa ame e s o each configu a ion
a e neu,gen and α2. The fi s wo ones ake specific alues depending on he
da a se and he las one depends on he configu a ion numbe (1’, ...). Table 1
shows he main aspec s o bo h EDD and EDDFS configu a ions.
4 Expe imen a ion
Table 2summa izes he da a se s employed. All o hem ha e been downloaded
om he Uni e si y o Cali o nia a I i ine (UCI) eposi o y [2]. Since we
a e using neu al ne wo ks, all nominal a iables ha e been con e ed o bina y
ones; due o his, some imes he numbe o inpu s is g ea e han he numbe o
ea u es. Also, he missing alues ha e been eplaced in he case o nominal a i-
ables by he mode o , when conce ning con inuous a iables, by he mean, aking
in o accoun he ull da a se . These da a se s ha e in common ha p esen e o
a es in es phase abou 15 % o abo e wi h e e ence classifie s such as C4.5 o
1-NN wi hou ea u e selec ion. The numbe o samples in he aining and es
se s ensues om he spli ing o he da a se s ollowing a expe imen al design ia
a c oss alida ion echnique called hold −ou ha consis s o di iding he da a
in o wo se s: a aining and a es se . In ou case, he sizes o he aining and
es se s a e h ee and one qua e s o he numbe o pa e ns in he p oblem,
espec i ely; hese pe cen ages a e simila o hose used in [11]. Mo e exac ly,
we ha e u ilised a s a ified holdou whe e he wo se s a e s a ified [7] so ha
he class dis ibu ion o he samples in each se is app oxima ely he same as in
he o iginal da a se .

Table 2. Summa y o he da a se s and specific pa ame e alues o EDD and EDDFS
amewo ks
Da a se Size T ain. Tes Fea . Inp. Cl. neu; gen neu’; gen’
B eas 286 215 71 9 15 2 9; 500 7; 500
Hea 270 202 68 13 13 2 6; 500 4; 25
Hepa i is 155 117 38 19 19 2 3; 100 3; 100
Pa kinsons 195 146 49 23 22 2 6; 500 3; 500
Pima 768 576 192 8 8 2 3; 120 3; 120
P omo e 106 80 26 58 114 2 11; 500 5; 300
Wa e o m 5000 3750 1250 40 40 3 3; 500 3; 500
Winequali y- ed 1599 1196 403 11 11 6 6; 300 4; 300
Yeas 1484 1112 372 8 8 10 11; 500 11; 500
Rega ds o EDD me hodology, he conc e e alues o neu and gen pa ame e s
depend on he da a se and a e shown in he eigh h las columns o Table 2.The
decision abou he numbe o neu ons in he hidden-laye is a e y difficul ask
in he scope o neu al ne wo ks; we ha e done a p elimina y s udy spli ing he
aining se in wo s a ified se s wi h h ee and one qua e s on he pa e ns
and explo ing he ange [2–12] o he numbe o hidden neu ons. Conce ning he
numbe o gene a ions, we ha e defined h ee kinds o alues: small (100–120),
medium (300) and la ge (500). In EDDFS, again he e a e wo pa ame e s:
neu and gen, whose alues, neu’ and gen’, a e defined o each da a se .The
assignmen o alues in EDD is no i ial, bu now in EDDFS his decision is
mo e difficul because he e a e ou FS me hods and he alues a e common
o all o hem. Kwak and Choi [8] ha e also conside ed his idea. The p oblem
o finding he bes a chi ec u es in neu al ne wo ks ha employ inpu ea u e
selec ion emains unsol ed. In mos o cases he numbe o neu ons is defined
by us wi h a lowe alue han model EDD. The e is no heu is ics o guide his
p ocess, so we ha e used alues a a guess. Rega ds he numbe o gene a ions,
he alues a e he same han in p e ious me hodology excep in he case o
P omo e and Hea da a se s. In he o me , he dimensionali y educ ion is
e y impo an and he gene a ion numbe has been change o medium alue.
In he la e he algo i hms con e ges soon and a e y small alue (25) is u ilised.
Table 3depic s he me hods used in he expe imen a ion ega ding he da a
p epa a ion s age by means ea u e selec ion. The e a e ou ones wi h and one
wi hou ea u e selec ion ha belong espec i ely o EDDFS ( he cu en p o-
posal) and EDD amewo ks. Las column defines an abb e ia ed name o each
o hem ha is employed in nex sec ions.
As p e iously men ioned, ou FS me hods ha e been applied o each da a
se . Table 4illus a es, o each da a se , he numbe o inpu s o he o iginal
ain se (see column labelled F0) and hose ha ha e been ob ained wi h he
diffe en ea u e selec o s (see columns labelled F1-4) along wi h he educ ion
Table 3. Lis o fil e s based on ea u e subse selec ion employed in he empi ical
s udy
Fea u e selec o name Sea ch me hod Subse e alua o F amewo k Abb e ia ion
−None None EDD F0
spBI CFS spBI CFS EDDFS F1
c BI CFS c BI CFS EDDFS F2
spBI CNS spBI CNS EDDFS F3
cnBI CNS cnBI CNS EDDFS F4
pe cen age in he inpu s o each selec o compa ed o he o iginal da a se . Las
ow shows he a e age o he numbe o inpu s o educ ion pe cen age o he
es bed o each expe imen ed me hod on his pape . The educ ion pe cen age
o he numbe o inpu s is defined as:
Reduc ion o Inpu s(%) = 1−Inpu s(Fi)
Inpu s(F0)100 i=1, ..., 4 (4)
whe e iis he FS me hod index and Inpu s(j) ep esen s he numbe o
inpu s o a gi en da a se wi h me hod j. In all cases, FS me hods success-
ully dec eased he da a dimensionali y by selec ing, in mean, much less han
he hal o he o iginal ea u es. P ecisely, he numbe o selec ed ea u es fluc u-
a es be ween a qua e and a hi d o he o iginal ea u es. F2 me hod achie es a
educ ion pe cen age, on a e age, o 63.34 % ( om 27.78 o 6.56 ea u es in a e -
age), which is he highes o e all a e age alue ob ained. Indi idually, P omo e
da a se has he highes educ ion a e, abo e a 92 % in all cases.
Table 4. Numbe and educ ion (%) o inpu s wi h EDD (baseline) and EDDFS
amewo ks
Da a se Inpu s Reduc ion (%)
F0F1F2F3F4F1F2F3F4
B eas 15 4 4 2 2 73.33 73.33 86.67 86.67
Hea 13 7 7 8 9 46.15 46.15 38.46 30.77
Hepa i is 19 10 10 11 5 47.37 47.37 42.11 73.68
Pa kinsons 22 5 5 7 6 77.27 77.27 68.18 72.73
Pima 8 3 3 4 5 62.50 62.50 50.00 37.50
P omo e 114 7 7 8 7 93.86 93.86 92.98 93.86
Wa e o m 40 14 14 15 15 65.00 65.00 62.5 62.50
Winequali y− ed 11 5 5 8 8 54.55 54.55 27.27 27.27
Yeas 8 5 4 7 7 37.50 50.00 12.50 12.50
A e age 27.78 6.67 6.56 7.78 7.11 61.95 63.34 53.41 55.28
We ollow he guidelines poin ed ou by J. Demˇsa [3] o pe o m nonpa ame -
ic s a is ical es s. Iman-Da enpo and Bon e oni-Dunn (Dunn, 1961) es s
ha e been pe o med. The c i ical diffe ence (CD) o Bon e oni-Dunn es can
be compu ed om c i ical alues, kand N. The conside ed significance le els
ha e been 0.05 o Iman-Da enpo es , and 0.05 and 0.10 o he pos -hoc
me hod.
5 Resul s
This sec ion depic s he esul s ob ained, measu ed in accu acy in he es se o
in he es subse depending on ha ea u e selec ion has been conside ed o no .
Fi s o all, we p esen he esul s ob ained wi h EDD and EDDFS. A e ha ,
a s a is ical analysis compa es EDD e sus EDDFS o de e mine whe he he e
a e significan diffe ences be ween applying o no ea u e selec ion wi h PUNN.
Nex , a second expe imen compa es, o each ea u e selec o , he bes mean
alues ob ained wi h he cu en p oposal o o he classifie s using he same
educed da a se s. Hence, in ega d o EDD, he esul s ha e been ex ac ed
om nex subsec ion o F1-4 me hods.
5.1 Resul s Applying EDD and EDDFS
The esul s ob ained by applying he EDD amewo k [16] a e p esen ed, along
wi h hose ob ained wi h EDDFS. In he case o EDD, he e we e 8 configu a-
ions, deno ed in he ollowing way: 1, 2, ... 8. As al eady men ioned, his pape
only deals wi h he fi s ou configu a ions. In EDDFS, he ou exis ing con-
figu a ions a e 1’, ..., 4’. Table 5shows he mean and s anda d de ia ion (SD) o
he es accu acies o each da a se o a o al o 30 uns. F om he analysis o
he da a, i can be concluded, om a pu ely desc ip i e poin o iew, ha he
EDDFS amewo k ob ains bes esul s o all da a se s. Always, he SD educ-
ion wi h EDDFS is clea and i exp esses mo e homogeneous esul s compa ed
o EDD.
S a is ical Analysis. In his subsubsec ion we compa e EDD and EDDFS
me hodologies by means o nonpa ame ic s a is ical es s. To de e mine whe he
he e a e significan diffe ences we apply an Iman-Da enpo es . I compa es
he a e age anks o he algo i hms, whe e a low ank alue indica es a good
algo i hm pe o mance and a high alue a bad algo i hm pe o mance. The a e -
age anks o all me hods, wi hou (F0) and wi h FS (F1-4) a e 4.78, 2.33, 2.56,
3.06 and 2.28, espec i ely. Acco ding o Iman-Da enpo es esul s, since he
s a is ic FF=6.10 is highe han he c i ical alue a (F(4,32) = 2.67) he null-
hypo hesis is ejec ed. The e o e, we apply a pos -hoc Bon e oni-Dunn es ha
compa es a numbe o me hods wi h a con ol me hod, by de e mining whe he
he a e age anks diffe by a leas he CD. In ou case, we make a compa i-
son o he me hods ha employ FS (F1-4) e sus he con ol me hod (F0) ha
does no use FS. CDs ob ained by Bon e oni-Dunn es a e 1.86 (a α=0.05)
Table 5. Resul s ob ained in he es -bed by means o EDD and EDDFS amewo ks
Da a se Fil e Mean±SD
Con ig.
1/12/23/34/4
B eas F0 63.85 ±3.81 63.00 ±3.24 64.27 ±3.89 63.43 ±3.80
F1 70.84 ±1.92 70.93 ±1.59 70.18 ±1.77 70.00 ±1.92
F2 70.84 ±1.92 70.93 ±1.59 70.18 ±1.77 70.00 ±1.92
F3 69.20 ±0.48 69.10 ±0.35 69.06 ±0.25 69.06 ±0.25
F4 69.20 ±0.48 69.10 ±0.35 69.06 ±0.25 69.06 ±0.25
Hea F0 75.93 ±2.40 75.83 ±3.27 76.23 ±2.48 76.03 ±3.50
F1 76.23 ±1.86 76.47 ±2.12 75.93 ±2.33 77.50 ±2.01
F2 76.23 ±1.86 76.47 ±2.12 75.93 ±2.33 77.50 ±2.01
F3 76.08 ±2.50 75.98 ±2.30 76.47 ±2.01 75.59 ±2.37
F4 77.40 ±2.10 76.76 ±2.09 77.89 ±2.49 77.99 ±1.79
Hepa i is F0 84.47 ±4.49 85.52 ±4.67 84.47 ±4.55 84.29 ±5.33
F1 88.77 ±2.49 88.77 ±2.93 88.95 ±2.80 89.91 ±2.59
F2 88.77 ±2.49 88.77 ±2.93 88.95 ±2.80 89.91 ±2.59
F3 89.04 ±2.40 89.30 ±2.49 89.56 ±2.13 89.47 ±2.85
F4 86.67 ±1.53 86.67 ±1.82 86.40 ±1.40 86.32 ±1.45
Pa kinsons F0 79.66 ±5.01 78.16 ±4.77 78.98 ±4.05 79.32 ±4.76
F1 79.66 ±2.37 79.32 ±2.32 79.93 ±2.15 79.05 ±2.83
F2 79.66 ±2.37 79.32 ±2.32 79.93 ±2.15 79.05 ±2.83
F3 80.27 ±4.23 81.84 ±4.20 80.61 ±3.07 80.14 ±3.90
F4 78.03 ±1.16 80.07 ±2.82 78.78 ±1.98 79.66 ±3.24
Pima F0 77.33 ±2.36 78.61 ±1.88 76.96 ±1.67 77.69 ±1.79
F1 79.54 ±0.90 79.49 ±0.79 79.60 ±0.87 79.89 ±0.92
F2 79.54 ±0.90 79.49 ±0.79 79.60 ±0.87 79.89 ±0.92
F3 75.48 ±1.42 75.19 ±1.26 75.00 ±1.17 75.31 ±1.51
F4 78.42 ±1.09 78.76 ±1.13 78.73 ±1.06 78.71 ±1.47
P omo e F0 59.74 ±9.30 58.21 ±9.67 60.51 ±10.00 55.51 ±10.03
F1 84.48 ±3.97 84.62 ±3.78 83.20 ±3.97 82.94 ±4.12
F2 84.48 ±3.97 84.62 ±3.78 83.20 ±3.97 82.94 ±4.12
F3 67.43 ±5.77 67.05 ±5.11 66.15 ±5.56 67.94 ±5.74
F4 75.89 ±4.39 75.51 ±4.45 76.41 ±3.74 76.53 ±4.55
Wa e o m F0 81.43 ±2.10 82.78 ±0.64 82.05 ±1.64 84.32 ±1.73
F1 84.97 ±1.13 86.54 ±0.48 84.92 ±0.98 86.30 ±0.95
F2 84.97 ±1.13 86.54 ±0.48 84.92 ±0.98 86.30 ±0.95
F3 85.39 ±1.41 85.78 ±0.74 85.20 ±1.14 86.37 ±0.84
F4 84.87 ±0.93 86.75 ±0.57 85.55 ±1.21 85.66 ±0.80
(Con inued)