FACULDADE DE ENGENHARIA DA UNIVERSIDADE DO PORTO
Au oma ed Fea u e Enginee ing o
Classi ica ion P oblems
Guilhe me Felipe do Nascimen o Reis
Mes ado em Engenha ia de So wa e
Supe iso : Ca los Manuel Milhei o de Oli ei a Pin o Soa es
July 29, 2019
Au oma ed Fea u e Enginee ing o Classi ica ion
P oblems
Guilhe me Felipe do Nascimen o Reis
Mes ado em Engenha ia de So wa e
July 29, 2019
Abs ac
The s udy on ea u e gene a ion has g own o e he las yea s.I is en i ely dependen on domain
knowledge and done manually, so i is ime consuming and no scalable. In u n, me a-lea ning
helps o lea n h ough di e en domains and can b ing bene i s o his a ea.
We p esen wo me a-lea ning app oaches o suppo ea u e selec ion. The i s p edic s
whe he a gi en au oma ed ea u e enginee ing app oach will imp o e he esul s. The second
p edic s i a single ea u e will imp o e he esul s.
We es hese app oaches on 100 da a se s om di e en domains. Ou expe imen s showed
ha i is possible o use me a-lea ning in he ea u e selec ion p ocess, and can in o m us whe he
o no we should gene a e ea u es wi h a he gi en me hod.
On he o he hand, he esul s o he second app oach, which p edic s he use ulness o each
indi idual ea u e a e no posi i e. The esul s show ha me a-lea ning can be used o aid he
gene a ion and selec ion o ea u es. Howe e , ou app oach can s ill be imp o ed, being mo e
p ecise in he p edic ions a he me a-le el and be e esul s a he base le el.
Ou code is a ailable a h ps://gi hub.com/gui elipe /au oma ed- ea u e-enginee ing
Keywo ds: da a mining, machine lea ning, me alea ning, KDD, ea u e enginee ing, au oma ed
ea u e enginee ing
i

ii
Resumo
O es udo sob e ge ação de ea u es em aumen ado con o me os anos. To almen e dependen e
de conhecimen o de domínio e ei o de o ma manual, po isso consome mui o empo e não é
escalá el. Po sua ez, me a-lea ning auxilia o ap endizado a a és di e en es domínios.
Nos ap esen amos duas abo dagens de me a-lea ning pa a auxílio na seleção de ea u es. A
p imei a p e ê se uma de e minada abo dagem de ge ação au omá ica de ea u es melho a á os
esul ados. O segundo p e ê se um único ecu so melho a á os esul ados.
Nós es amos essas abo dagens em 100 conjun os de dados de di e en es domínios. Nossos
expe imen os mos a am que é possí el usa o me a-lea ning no p ocesso de seleção de ecu sos,
e pode nos in o ma se de emos ou não ge a ecu sos com um de e minado mé odo.
Po ou o lado, os esul ados da segunda abo dagem, o qual p e ê a u ilidade de cada ca -
ac e ís ica indi idual, não são posi i os. Os esul ados mos am que o me a-lea ning pode se
usada pa a auxilia na ge ação e seleção de ecu sos. No en an o, nossa abo dagem ainda pode se
melho ada, sendo mais p ecisa nas p e isões no me a-le el e melho es esul ados no base le el.
Nosso código es a disponí el em h ps://gi hub.com/gui elipe /au oma ed- ea u e-enginee ing.
Keywo ds: da a mining, machine lea ning, me alea ning, KDD, ea u e enginee ing, au oma ed
ea u e enginee ing
iii
i
Acknowledgemen s
Fi s , I would like o hank my amily and wi e o all he suppo and ad ice h oughou my
jou ney. De ini ely wi hou hem, I would no be he pe son I am oday.
I also wan o hank all he men o s who ha e appea ed in my li e, all o hem helped me
o become he p o essional ha I am oday. Mainly, P o . João Rod igues, who unknowingly
in oduced me o enginee ing when I was s ill in high school. The P o . Tiago Hen ique, who
imp o ed me du ing my enginee ing ca ee when I was s ill in Bachelo , and my P o esso s Ca los
Soa es and Rui Quin ino who helped and guided me a lo in his wo k.
Fo all hose who helped me and encou aged me in my pa h, I hank you om my hea , and I
will ce ainly always emembe you.
Guilhe me Reis
xii LIST OF TABLES

Abb e ia ions
Au oML Au oma ion Machine Lea ning
DFS Deep Fea u e Syn hesis
CRISP-DM C oss Indus y S anda d P ocess o Da a Mining
NA No A ailable
NHST Null Hypo hesis Signi icance Tes
ML Machine Lea ning
xiii
Chap e 1
In oduc ion
One o he ac o s o a success ul applica ion o a machine lea ning app oach is a well p epa ed
da a se . The da a needs o be cleaned and p epa ed o he algo i hm o induce a good model.
P epa ing a da a se co ec ly usually akes a signi ican amoun o ime, and i equi es solid
knowledge abou he domain. One pa icula ly impo an ask is ea u e enginee ing, which con-
sis s o c ea ing new a iables ha make i easie o he algo i hm o ob ain a good model. The e
a e wo app oaches o ea u e enginee ing: manual and au oma ed. In he manual app oach, one
ea u e is buil a a ime. This is ime-consuming, challenging and e o -p one p ocess and depends
on domain knowledge. The au oma ed app oach can ex ac s many ea u es om a da a se in a
sys ema ic way which can be done wi hou human in e en ion. Addi ionally, i can be applied o
any p oblem and he building p ocess is quicke and no dependen on domain knowledge.
Se e al au oma ion s a egies ha e been de eloped ecen ly, o dec ease he e o de o ed o
ea u ing enginee ing. Howe e , hese me hods commen ed on in his hesis and ou me hod do
no gua an ee ha he se o ea u es gene a ed will lead o be e models [6,11,8].
“Au oma ing ea u e enginee ing op imizes he p ocess o building and deploying ac-
cu a e machine lea ning models by handling necessa y bu edious asks so da a sci-
en is s can ocus mo e on o he impo an s eps.” [6]
1.1 Mo i a ion
On ea u e enginee ing, i is possible o au oma e pa o he p ocess which does no depend on
domain knowledge.
A suba ea o machine lea ning ha deals wi h he p oblem o ex ac ing knowledge abou
lea ning p oblems which is independen o he domain a e me a-lea ning. Me a-lea ning consis s
o lea ning abou he beha iou o machine lea ning (ML) algo i hms by collec ing (me a)da a
abou ins ances o hose p ocesses and applying ML algo i hms o ha me ada a [15]. Howe e ,
me a-lea ning app oaches ha e ne e been applied o he p oblem o ea u e enginee ing.
1
2In oduc ion
1.2 Objec i es
Gi en a me hod o au oma ically gene a e ea u es, ou goal is o use a me a-lea ning app oach o
decide whe he i will lead o be e models compa ed o he o iginal da a se .
Ou p elimina y esea ch ques ion is he ollowing:
•Does he sys ema ic combina ion o ea u es using simple ope a ions lead o be e p edic i e
models han he o iginal da a se ?
A e , we speci ically add ess he ollowing ques ions:
•Can a me a-lea ning app oach p edic when a me hod o he sys ema ic gene a ion o ea-
u es leads o be e p edic i e models?
•Can a me a-lea ning app oach p edic when a me hod o au oma ic gene a ion o ea u es
leads o be e p edic i e models?
•Can a me a-lea ning app oach p edic when an indi idual ea u e leads o a be e p edic i e
model?
The con ibu ions o his wo k a e:
•Applica ion o a me a-lea ning app oach o he p oblem o deciding whe he a me hod o
au oma ic ea u e gene a ion should be applied and which ea u es should be used o im-
p o emen o he model;
•De elopmen o a se o me a- ea u es ha a e speci ic o he p oblem add essed he e;
•Empi ical e alua ion o he app oach p oposed using a simple me hod o au oma ic ea u e
gene a ion.
In sec ion 2, we discuss he backg ound and ela ed wo k; In sec ion 3, we explain he sys em-
a ic ea u e gene a ion and he explo a o y da a analysis. Fo he sec ions 4and 5we discuss he
app oaches p oposed and he expe imen al se up used and esul s. Finally, in sec ion 6, we p esen
he conclusion and u u e wo k.
Chap e 2
Backg ound and Rela ed Wo k
O e he las yea s, he demands o da a mining ha e been inc easing and also he g owing in e es
in au oma ed machine lea ning, see he igu e 2.1 1. Da a mining uses a conside able numbe o
da a analysis echniques o he p ocess o iden i ying speci ic pa e ns in a massi e amoun o
da a.
In his chap e , we discuss an analysis o he da a mining p ocess and mining echniques 2.1,
he ac ual s a e o au oma ed machine lea ning 2.4, me a-lea ning and au oma ed ea u e engi-
nee ing 2.3.
2.1 Da a Mining
Da a mining can use machine lea ning echniques, bu i is possible o use da a managemen ech-
niques (e.g. ela ional da abase managemen sys em) . Ne e heless, we a e going o alk only
abou machine lea ning echniques.
The e a e wo p ima y ype o asks o da a mining, p edic i e and desc ip i e.p edic i e
has he in en o p edic based on he his o ical da a, and he mos common asks a e, classi ica ion
and eg ession. Reg ession is used o p edic a nume ic alue. Fo example, a new ca model is
added o he da abase, and he sys em needs o sugges he p ice o a ca wi h i e doo s, elec ic
di ec ion, om a speci ic b and, wi h 100.000 Kms. Reg ession should be used o ha case.
Classi ica ion is used o de e mine a class o a da a poin . Fo example, a speci ic en y o he
lowe s da abase does no ha e a classi ica ion, and he sys em needs o iden i y wha lowe i is
o comple e he da a. Classi ica ion should be used o ha case.
Desc ip i e da a mining is used o o ganise he da a and unde s and be e he in o ma ion
ha he da abase has; i is also di ided in o wo echniques, associa ion and clus e ing. Clus e ing
iden i ies g oups o simila da a poin s. Da a poin s in a clus e a e mo e simila among hemsel es
han o da a poin s in o he clus e s. Associa ion analyses he associa ion be ween i ems. Fo
1A ailable a h ps:// ends.google.p / ends/explo e?q=Au oML
3

4Backg ound and Rela ed Wo k
Figu e 2.1: Google T ends abou Au oML
.
example, in a comme cial da abase associa ion can iden i y p oduc s ha a e gene ally pu chased
oge he , a amous s o y analysed a ma ke da abase and disco e ed ha usually clien s who buy
diape s also buy bee .
The p ocess o da a mining is in e ac i e and challenging. The mos common p ocess model
in ol es six phases, whe e each one has speci ics deli e ies o he nex phase. The p ocess model
is also known as C oss Indus y S anda d P ocess o Da a Mining (CRISP-DM), see li e cycle on
igu e 2.2.
Business unde s anding - The goal o his phase is unde s anding he business pe spec i e and
hen ansla ing i o a da a mining p oblem.
Da a unde s anding - The ocus o his phase is o collec and ha e ini ial con ac wi h he da a.
Analyse da a quali y and iden i y possible p oblems.
Da a p epa a ion - The goal o his phase is o clean he da a, gene a e mo e ea u es, selec he
ea u es c ea ed o eed he algo i hms o he nex phase. This phase is he cen e o ou
wo k.
Modelling - The algo i hms a e selec ed, and hei pa ame e s uned o an op imal esul .
E alua ion - Be o e he inal deploymen , his s ep will assess whe he he business goals ha e
been me .
Deploymen - The goal o his phase is o plan deploymen , p oduce a inal epo and e iew he
p ojec o e alua e wha wen igh and wha wen w ong.
All o his in o ma ion and mo e de ailed p ocess o CRISP-DM can be ound in he book CRISP-
DM 1.0 [14].
2.2 Au oma ed Machine lea ning 5
Figu e 2.2: Phases o CRISP-DM model [14]
2.2 Au oma ed Machine lea ning
Machine lea ning is ime-consuming and needs many esou ces o ge he ask done. The e o e,
companies like IBM ha e been in es ing in Au oma ed Machine Lea ning esea ch [11]. Au o-
ma ed machine lea ning (Au oML) goal is o change his scena io, de eloping accessible machine
lea ning.
Cu en ly, Au oML ocuses on da a p epa a ion and modelling. In da a p epa a ion, i is o en
used o au oma e he p ocess o da a p ep ocessing, ea u e gene a ion, and selec ion. Fo model
aining, i is used o au oma e model selec ion and hype -pa ame e op imisa ion.
2.2.1 Da a p ep ocessing
The da a collec ed is no always sui able o un a machine lea ning p ocess. The e o e, i is nec-
essa y o ans o m he da a. P ep ocessing in ol es many di e en asks such as scaling, missing
alue impu a ion, ou lie de ec ion, binning and ca ego ical da a ans o ma ion [2].
Scaling - Scaling echniques ensu e ha he nume ical ea u es a e weigh ed p opo iona ely
by he algo i hms. The e ms used o ha echnique a e s anda disa ion and no malisa ion.
Missing alues - Missing alues o en occu s in da a se s. This can happen because o a de ice
bug, yping e o s o e en he use choosing no o p o ide some in o ma ion.
1. Use a cons an : I is possible o ill all he missing alues wi h a cons an . The cons an can
be a disp opo iona e numbe o a no a ailable (NA) o sepa a e hose o he es o he
da a se .
2. Use a mean/median alue o ill: Nume ical alues can be eplaced wi h mean o median
and ca ego ical alues can be eplaced by he mode.
6Backg ound and Rela ed Wo k
Figu e 2.3: Fea u e selec ion p ocess [18]
3. Remo e/Dele e Da a: This echnique is ecommended when only a ew da a a e missing,
he elimina ion can be applied o a speci ic ow o column. Howe e , when oo much da a
a e missing, i may educe he amoun o a ailable da a oo much, a ec ing he quali y o
he models gene a ed.
4. Use da a mining o p edic he missing alue: I is possible o use machine lea ning algo-
i hms o p edic he mos p obable alue o missing da a. Howe e , when used o oo
much missing da a i can cause o e i ing on he inal model because he da a used o ill
he emp y da a was he same o ain and es .
5. Add lag: Indica e he missing da a wi h a lag. Inse a lag (e.g.: ’missing alue’ s ing
o ou lie numbe ) in he missing da a can iden i y whe e he da a was missed, aiding o
iden i y he lack o in o ma ion and assess he da a.
Ou lie de ec ion - Ou lie s a e hose alues " ha a e so di e en om he o he s ha dis o
he da a dis ibu ion. Those alues can lead o low pe o mance models. Ou lie s ha e wo ypes,
uni a ia e and mul i a ia e. A uni a ia e ou lie , as he name implies, is an ou lie based on a
single a iable. A mul i a ia e ou lie is a mix u e o alues on wo o mo e a iables.
Binning - Binning goal is o disc e izes con inuous alues, i is a p ocess o g oup a con inuous
numbe in smalle in e al, o example, di iding he age o a su ey in o small in e als. This is
use ul o algo i hms which ha e di icul ies in analysing nume ical a iables, such as nai e Bayes.
Two common binning me hods a e:
•Equiwid h binning: The bins a e di ided in o nin e als o equal size.
•Equi equency binning: The bins a e di ided in o ng oups, each con aining he same num-
be o alues.
All o his in o ma ion and mo e de ail can be ound [2].
2.2.2 Fea u e selec ion
Fea u e selec ion is used o emo e unnecessa y and edundan a ibu es om he da a because
some a ibu es dec ease pe o mance ins ead o inc easing i . Fu he mo e, he complexi y o he
2.2 Au oma ed Machine lea ning 7
Table 2.1: Example o ea u e gene a ion ex ac ing in o ma ion om a da e a ibu e
Use ID Penul ima e login Las Login Di Login Las Login is Weekend?
865 12/12/16 03/01/17 22 FALSE
1954 18/03/16 23/07/16 127 TRUE
1307 25/11/17 11/03/18 106 TRUE
1966 29/03/16 26/04/16 28 FALSE
51 07/01/16 23/02/16 47 FALSE
model can inc ease wi h mo e a ibu es, no helping he da a scien is o unde s and o explain he
model. So, ea u e selec ion can help o ha e a be e pe o mance wi h ewe ea u es and less
complexi y in he model [4] (Figu e 2.3).
Th ee ypes o me hods can be used: Fil e me hods, w appe me hods and embedded me hods.
Fil e Me hods These me hods analyse each ea u e independen ly and assign a sco e o each.
These sco es a e used o ank he ea u es and he bes ones a e selec ed. The sco es can be
based on he in o ma ion gain measu e.
W appe Me hods These me hods e alua e combina ions o ea u es. Di e en combina ions
a e c ea ed and compa ed by lea ning and e alua ing a model. The pe o mance o he
model is used o guide he sea ch o he bes se o ea u es. W appe me hods include
ecu si e ea u e elimina ion, dele ing and adding ea u es, o using a andom algo i hm o
help he sea ch.
Embedded Me hods These a e o en egula isa ion mechanisms embedded in he algo i hms. I
consis s o adding a penal y o complex models, helping o educe o e i ing and a iance.
Howe e , i adds bias o he model. An example o his me hod is Ridge Reg ession.
2.2.3 Fea u e gene a ion
The ea u e gene a ion, also known as ea u e cons uc ion, adds mo e ea u es in o he aw da a
and i can esul in mo e in o ma ion ha leads o models wi h be e accu acy. Howe e , adding
ea u es inc eases he dimensionali y o he da a se and inc eases he chance o o e i ing, known
as he cu se o dimensionali y. To minimise he e ec o ea u e c ea ion, one app oach is ea u e
selec ion, which was desc ibed ea lie .
To illus a e he ea u e enginee ing dilemma, conside Table 2.1. He e we ha e i e columns
showing he beha iou o access o some use s, and he i s h ee columns a e he aw da a, he
las wo columns a e he new ea u es gene a ed in he p ocess o ea u e enginee ing. We can
see ha he use s who access he pla o m on weekends also s ay longe wi hou using i . When
s udying only he o iginal da a, we do no see his pa e ns. New ea u es can help o show hose
pa e ns and help o build be e models.
Nume ical - Many ma hema ical ope a ions can be applied o ans o m single and g oups o
a iables. Fo example, log, a io, addi ion, a iance o wo numbe s o any ma hema ical
14 Backg ound and Rela ed Wo k
Figu e 2.7: Example o Decision ee aids he decision be ween wo algo i hms
ees o p esen a be e p edic i e esul . The Decision ee in mos o he cases ha e a high
a iance, and i happens when we ex ac di e en aining and es se s om he same da a se ,
esul ing in di e en esul s.
Sadly, he Decision ee can cause poo pe o mance o a new da a se . Howe e , Random
o es is eage o sol e he weakness o Decision ee educing he a iance and bias e o s, by
choosing a subsample o he ea u e space o each spli . [3]
2.4.6 Resul s analysis
In his sec ion, we summa ise he esul s o he ela ed pape s. They a e discussed sepa a ely
because hey a e no all ela able.
Explo eKIT [8] - The au ho s buil ou app oaches and he bes app oach was he ML_ ull,
ob aining an a e age e o educ ion o 17.4%–29.3%.
Deep Fea u e Syn hesis [6] - The Da a Science Machine, also known as Deep Fea u e Syn hesis,
was one o he i s p ojec s wi h success ul esul s on au oma ed ea u e gene a ion. The
algo i hm has been es ed in Kaggle compe i ions and i achie ed 90% o he bes sco e
achie ed by any compe i o in wo ou o h ee compe i ions. The bes pe o mance was on
KDD15 challenge bea ing 86% o o he compe i o s.
OneBu on Machine [11] - The One Bu on Machine uses wo app oaches. I was es ed in di e -
en da abases, one o hem was he KDD 2014 cup o he compa ison wi h he Deep ea u e

2.4 Au oma ed ea u e enginee ing 15
syn hesis p ojec . The One bu on machine compa e hei esul s wi h he esul s o da a
scien is s in he KDD 14 Challenge, and hei esul s a e in he op 17% while he DFS was
in he op 30%. The esul s show ha hese app oaches a e compe i i e wi h a la ge numbe
o expe ienced da a scien is s.
Au oLea n [9] - The Au olea n algo i hm aims o use ewe ea u es o be e accu acy. I
achie ed 13.28% and 5.87% imp o emen in e ms o accu acy, agains he o iginal ea-
u es. I also ob ained compe i i e esul s when compa ed wi h Explo eKi and o he s. The
algo i hm was es ed on 25 da a se s.
Cogni o [10]- I ob ained imp o emen s be ween 7% - 40% compa ed wi h he o iginal ea u es
on di e en da a se s.
Au oma ic ea u e gene a ion o machine lea ning based op imising compila ion [12] - The goal
o his p ojec is o use Au oML o de ec ing he bes loop un olling in GCC 4.3.1. The
no el algo i hm applied in his p ojec was able o achie e 75% o he maximum eloci y
o he compile on a e age. The benchma ks show ha he algo i hm imp o ed 35% o he
speed.
In his hesis will be used one o he Bayesian es s p esen ed in he a icle, called Bayesian
co ela ed - es ha will be used o compa e pe o mance be ween he esul s ob ained by 10- old
c oss- alida ion and he baseline o each old.
2.4.7 P og amming languages
The p og amming languages used in he ela ed wo k we e Py hon, Ja a and C/C++. The e is no
mo i a ion o he selec ion o he p og amming language. Thus, he discussion abou he be e
language o Au oma ed Machine Lea ning is open.
The choice o p og amming language is no ele an o his hesis. Howe e , he choices in he
ela ed wo k a e well di ided. Two pape s use Py hon, Deep Fea u e Syn hesis [6] and Au oLea n
[9], wo pape s use Ja a, Cogni o [10] and Explo eKi [8], and he only one ha uses C/C++ is
Au oma ic Fea u e Gene a ion o Machine Lea ning Based Op imizing Compila ion [12]. The e
is ano he one which ha did no iden i y which p og amming language was used [11].
The Vision Mobile company eleased a epo in Ap il 20172, showing he popula i y o each
language o machine lea ning. Howe e , i can change depending on he backg ound o he in-
di idual in cha ge o he de elopmen p ocess. Fo example, Py hon was he p e e ed language
o machine lea ning on 38% o answe s and used mos ly by da a scien is s and junio p o ession-
als. F on -end de elope s use ja asc ip wi h 16%, Elec onics enginee s p e e C/C++ (8%), da a
analys s and s a is ician p io i ise R (14%). In conclusion, he e is no co ec language.
2A ailable a h ps:// isionmobile.com/ epo s/s a e-de elope -na ion-q1-2017
16 Backg ound and Rela ed Wo k
Chap e 3
Sys ema ic ea u e gene a ion
In his chap e , we p esen ou app oach o sys ema ic ea u e enginee ing. This me hod will be
used as ea u e gene a ion me hod o he me a-lea ning app oaches used in he ollowing chap e s.
We also p esen an explo a o y da a analysis o he empi ical esul s ob ained wi h his me hod.
The app oach was implemen ed in Py hon1, using some lib a ies o assis he de elopmen , such
as I e ools 2, Sciki Lea n 3and Pandas 4. The ML algo i hm used was Random Fo es wi h 100
es ima o s and all de aul Sciki -lea n pa ame e s . The e alua ion me ic used was accu acy and
i was es ima ed wi h s a i ied c oss- alida ion.
3.1 Expe imen al Se up
We analysed 100 supe ised classi ica ion da a se s collec ed om he OpenML pla o m [5]. We
selec ed da a se s wi h 30 o ewe classes (Figu es 3.1 and 3.2 ). Since we ha e decided o ocus
on ope a ions on nume ical a iables, we decided o emo e all a iables which a e no nume ical
3.2. Fo example, conside ing we ha e a da a se wi h 20 a ibu es on he o al, bu only 5 o hem
a e nume ical a ibu es, ou o iginal da a se will con ain only he 5 nume ical a ibu es. The
da a se s we e collec ed om he OpenML pla o m [5]. The ull lis o da a se s can be ound in
appendix A.
3.2 App oach
Ou app oach can be applied o a iables o any na u e. Howe e , o simplici y, in his p ojec ,
we will ocus on nume ical a iables, wi hou loss o gene ali y.
1h ps://www.py hon.o g/
2h ps://docs.py hon.o g/3/lib a y/i e ools.h ml
3h ps://sciki -lea n.o g/s able/ - e sion 0.19.2
4h ps://pandas.pyda a.o g/
17
18 Sys ema ic ea u e gene a ion
Figu e 3.1: Analysis o da a se s wi hou elimina ing highly co ela ed ea u es.
Fo mally, gi en a da a se D:= (X,Y), wi h a a ge a iable Y, and a se o a iables X:=
(x1,x2,...,xm)⊆Rm, whe e mis he numbe o a iables. Addi ionally, gi en a bina y unc ion τo:
R×R→Rou app oach gene a es a se o new ea u es Excon aining all possible combina ions
o he o iginal a iables, wi hou loss o gene ali y, using ha unc ion:
Ex= (o,i,j)| (o,i,j)=τo(xi,xj),∀i6=j∈{1,...,m},o(3.1)
A e he gene a ion o new ea u es, he app oach combine he new ea u es o he o iginal
ea u es, X+= (X,Ex), hen o each o iginal da a se we ha e D+= (X+
,Y). This app oach is
expec ed o gene a e a e y la ge numbe o a iables, many o which may be edundan . Fo
ins ance, i he unc ion used o combine a iables is commu a i e (e.g. p oduc ), hen Eo,i,j
x=
o(xi,xj) = o(xj,xi) = Eo,i,j
x, which means ha hese wo new ea u es a e edundan . Thus, we
apply a il e ea u e selec ion me hod o elimina e edundan ea u es in Ex[7]:
Xx, il = il e (X+)(3.2)
3.3 Explo a o y da a analysis
Figu e 3.1 shows he dis ibu ion o he numbe o ea u es in he da a se s, be o e and a e he
addi ion o new ea u es. The i s plo shows he dis ibu ion o classes, he majo i y o he da a
se s has be ween 2 and 5 classes, bu he e is an ou lie , coun ing 26 classes. In he same igu e,
i is possible o obse e he di e ence be ween he ea u es on he o iginal and he ex ended da a
se s.
The ex ac ion o ea u es does no ake oo much ime. The algo i hm o c ea e new ea u es
and e alua e he o iginal and he new da a se o he 100 da a se s, ook 4 hou s. The p ocess
gene a ed 132.206 ea u es, which would be impossible o a human o pe o m in such a sho
3.3 Explo a o y da a analysis 19
Figu e 3.2: Analysis o he da a se s wi h highly co ela ed ea u es emo ed.
ime. Howe e , a la ge numbe o new ea u es is expec ed. Fo each bina y unc ion, in a da a se
wi h m a iables, a combina ion o m, wo-by- wo, will be gene a ed. Fo example, he o iginal
hill- alley da a se has 100 nume ic ea u es. The ex ension da a se we gene a ed has 19,900
ea u es in o al.
Figu e 3.2 shows he dis ibu ion o he numbe o ea u es in he da a se s, be o e and a e
he ex ension a e applying a il e -based ea u e selec ion me hod, as desc ibed in Sec ion 2.2.2.
In his case, he o iginal hill- alley da a se has 100 nume ic ea u es, and a e he ex ension
and il e ing p ocess, i esul s in 9902 ea u es on he o al. The elimina ion o highly co ela ed
ea u es can educe he compu a ional cos o ou model.
3.3.1 Base-le el pe o mance
We analysed he base-le el pe o mance on he wo e sions o he ex ended da a se s, wi h and
wi hou ea u e selec ion. The da a was classi ied wi h wo op ions 1 and 0, which 1 is when
he sys ema ic ea u e gene a ion app oaches ha e a be e pe o mance han he o iginal da a se
and 0 when he o iginal da a se has a be e pe o mance han he ex ended da a se . Fo all 100
da a se s analysed we ha e a balanced esul s on bo h e sions. Fo he il e ed s a egy, we see
52 imp o emen s om he o iginal da a se (class 1) and 48, which he o iginal is be e han
he ex ended ea u es (class 0). Fo he non- il e ed s a egy, we see 51 imp o emen s om he
o iginal da a se (class 1) and 49, which he o iginal is be e han he ex ended ea u es (class 0).
We see a sligh di e ence be ween hem. Mo e de ails a e a ailable in Appendix A.
Howe e , we can see a mo e signi ican dis inc ion in he di e ence be ween he accu acy
o he o iginal da a se and ex ended da a se . In he il e ed da a se , we ha e a maximum im-
p o emen on he accu acy o 48.5 pe cen age poin s and a maximum educ ion o 9 poin s. Fo
he non- il e ed s a egy, we had a maximum imp o emen on he accu acy o 40.5 poin s and a
maximum educ ion o 15.5 poin s.

20 Sys ema ic ea u e gene a ion
Figu e 3.3: Explo a o y analysis o he del a be ween he o iginal accu acy s ex ended accu acy.
Analysing he a e age o bo h s a egies conside ing he 100 da a se s, he sys ema ic ea u e
gene a ion esul s in an o e all imp o emen o 0.97 poin s o he il e ed s a egy and 0.68 poin s
o imp o emen o he non- il e ed s a egy. The hi d qua ile shows ha we ob ain gains o 0.83
and 0.57 pe cen age poin s in he on he il e ed and non- il e ed s a egies, espec i ely.
In spi e o his, he e a e many da a se s in which he ex ension does no lead o imp o emen s,
which means ha , gi en a da a se , i is necessa y o decide whe he i is wo hwhile o gene a e
new ea u es o no . We add ess his p oblem wi h me a-lea ning app oaches in he ollowing
chap e s. The il e ed ex ended da a se leads us o conclude ha il e ing high co ela ed ea u es
lead o a be e model. Also, we can conclude ha he sys ema ic combina ion o ea u es using
simple ope a ions can lead us o be e p edic i e models.
Chap e 4
Me a-lea ning o selec comple e se s o
ea u es
In his chap e , we p esen ou me a-lea ning app oach o selec comple e se s o ea u es. In he
p e ious chap e (Chap e 3), we obse ed ha many da a se s a e he ex ension did no lead o a
be e model, leading us o a new p oblem: Can a me a-lea ning app oach p edic when a me hod
o au oma ic gene a ion o ea u es leads o be e p edic i e models?
4.1 P oblem de ini ion
The p oblem is o an icipa e whe he he gene a ion o au oma ic ea u es will gene a e any gain
in he pe o mance o he model o no . Besides aiming o imp o e accu acy, we can educe he
compu a ional esou ces equi ed o lea ning he models.
4.2 App oach
We a e going o add ess he p oblem using a me a-lea ning app oach, bu he ypical me a-lea ning
app oaches, me a- ea u es cha ac e ise a single da a se . In ou case, we ha e wo da a se s: he
o iginal and he ex ended da a se s. Mos impo an ly, we expec ha i is he combined a io in he
me a- ea u es be ween he o iginal and ex ended me a-da a ha de e mines whe he he algo i hm
will be able o ob ain a be e model on he ex ended one o no .
Gi en he se s o da a se s {D1,D2,...,Dm}, he se s o combina ion be ween o iginal da a and
new ea u es X+
1,X+
2,...,X+
m, he se s o ex ended da a se s D+
1,D+
2,...,D+
mand o each p
a unc ion m i ha compu es a gi en me a- ea u e (e.g., a e age co ela ion), we apply i o he
o iginal da a se , XX
i,j=m j(Di)∀i,j, o he ex ended one, XX+
i,j=m j(D+
i)∀i,jand hen we combine
using he unc ion, X(X,X+)
i,j=T(XX
i,j,XX+
i,j). The unc ion used o combine he cha ac e is ics o
21
22 Me a-lea ning o selec comple e se s o ea u es
he o iginal and de i ed da a se s should quan i y he di e ences be ween hem, so a a io o a
sub ac ion a e sui able unc ions, and in his hesis, we choose he a io.
Fo all da a se s, all ele an in o ma ion was eco ded, making i possible o compa e he
o iginal and he ex ended da a se s, bo h in e ms o hei cha ac e is ics as well as in e ms o
he pe o mance o he algo i hm. The me a-le el class a ibu e holds he inal esul s: 1 i i is
wo hwhile o do he au oma ic gene a ion o ea u es o 0 i he algo i hm should be applied o
he o iginal da a se . We wan o apply he me a-lea ning app oach o p edic when a me hod o
he sys ema ic gene a ion o ea u es leads o be e p edic i e models.
4.3 Expe imen al se up
Fo he gene a ion o me a- ea u es, we used he Me a-Fea u e Ex ac o (MFE) [15] R package.
I p o ides i e dis inc g oups o me a- ea u es: gene al, s a is ical, in o ma ion- heo e ic, model-
based, landma king. Howe e , we do no use all he g oups o a iables o his expe imen , and
we ha e some es ic ions in he selec ed g oups.
The domain o in o ma ion- heo e ic g oup is only o ca ego ical da a [15], and in ou expe -
imen , we used only nume ical da a. This lead us o elimina e he whole g oup. The Landma king
g oup measu es he pe o mance o a decision ee o each da a se , in ou case i leads o a sig-
ni ican compu a ional cos , and, hus, we decided o elimina e hese me a- ea u es as well o his
p ojec . Ne e heless, we belie e hey may be use ul and should be conside ed in u u e ex ensions
o his wo k.
Addi ionally, as we ha e a easonable a ie y o da a se s, some me a- ea u es gene a ed in-
alid esul s. On ha case, me a- ea u es wi h mo e han 10% o no a ailable o no appli-
cable we e also elimina ed, namely: gene al.num oca , s a is ical.sd a io, s a is ical.gmean.sd,
s a is ical.gmean.mean, s a is ical.wlambda, s a is ical.can co .sd, s a is ical.skewness.sd, s a is i-
cal.n co a , s a is ical.ku osis.sd, s a is ical.co .sd, s a is ical.co .mean, s a is ical.skewness.mean,
s a is ical.ku osis.mean, s a is ical.hmean.sd. Mos o hese a e based on s anda d de ia ion o
mean and we e caused by di isions by ze o, as expec ed [15].
Also, in o de o emo e noise, we applied a d op o highly co ela ed ea u es, pe o med wi h
a h eshold o 95% o me a- ea u es o he o iginal da a se . Fo each da a p epa a ion ope a ions,
we pe o med wo hypo hesis, he i s was pe o med o he base da a se wi hou il e ing a he
base le el, and he second hypo hesis we added he il e men ioned abo e, a he base le el da a
se .
4.4 Resul s
Ou me a-lea ning app oach can p edic om he me ada a o he o iginal da a se and ex ended
ea u es, whe he using he new ea u es p oduces a be e model, as we can see om he model
4.4 Resul s 23
Table 4.1: Pe o mance achie ed by applying he me a-lea ning app oach.
No il e Fil e ed
A e age (%) sd (+/-) A e age (%) sd (+/-)
Base 60.94 (0.21) 61.93 (0.30)
Base - add a io 62.14 (0.19) 56.38 (0.31)
Base - d op NA 60.83 (0.23) 62.91 (0.33)
Base - d op co ela ions 58.85 (0.21) 60.04 (0.34)
Base - add a io
& d op NA 60.92 (0.19) 62.18 (0.33)
Base - add a io
& d op co ela ions 55.05 (0.24) 61.09 (0.37)
Base - add a io
& d op NA
& d op co ela ions
57.14 (0.25) 61.20 (0.37)
pe o mance in able 4.1. Recalling ha ou baseline is 52%, and we a e using 10- old c oss-
alida ion, ou app oach ob ained an accu acy o 62.91% ( +/- 0.33) when he da a se o d op he
highly co ela ed ea u es.
Fo he able 4.1 we can conside he base da a as he o iginal XX
i,jand he ex ended XX+
i,j
me ada a on he same da a se . F om his base me a-da a, we pe o med di e en da a p epa a ion
ope a ions, emo e all no a ailable gene a ed by he MFE package and emo e highly co ela ed
ea u es, and mos impo an ly we added he combina ion a io as he equa ion X(X,X+)
i,j. We applied
hese p epa a ion ope a ions o he wo a ian s o he me a-da ase s il e and wi hou he il e .
Howe e , his p ocess s ill needs he gene a ion o ea u es o compu e he me a- ea u es.
This leads o addi ional compu a ional cos . E en i his is signi ican ly smalle han he cos
o lea ning he models, we need o in es iga e i success ul ecommenda ions can be ob ained
wi hou he gene a ion o he new ea u es. In o he wo ds, i he me a-lea ning p ocess uses only
he me ada a o he o iginal da a se o p edic whe he o no o add new ea u es.
To p edic when o no o add he new ea u es, we apply se e al app oaches o he o iginal
da a se , o be able o emo e any noise ha impac s he pe o mance o he model. The e o e,
conside ing he o iginal da a se , we ex ac ed all me a-da a as we saw on he p e ious sec ion
XX
i,j=m j(Di)∀i,j, including only he gene al, s a is ical and model-based g oups o me a- ea u es.
Resul s in able 4.2 show ha i is possible, by using only he me a-da a o he o iginal da a se
o p edic when he me hod o au oma ic gene a ion o ea u es leads o be e p edic i e models
Table 4.2: Pe o mance achie ed using only he me ada a o he o iginal da a se .
No il e Fil e ed
A e age (%) sd (+/-) A e age (%) sd (+/-)
Base 63.14 (0.26) 59.04 (0.27)
Base - d op NA 58.94 (0.23) 61.11 (0.33)
Base - d op co ela ions 66.96 (0.25) 62.95 (0.24)
Base - d op NA and co ela ions 65.05 (0.25) 61.31 (0.31)
30 Me a-lea ning o selec indi idual ea u es
se s o ea u es bu a single ea u e and we ha e collec ed he pe o mance o each ea u e o
each da a se , so we decided o use only he ea u es ha supposedly lead o an imp o emen in
pe o mance. This app oach needs u he esea ch o add ess his issue.
Un o una ely, using only he ea u es ha ou app oach sugges s, he a e age accu acy d opped
by 0.72 pe cen age poin s compa ed o applying all gene a ed ea u es. Fu he mo e, he same
me hod s ill d opping on an a e age o 0.043 pe cen age poin s compa ed o he o iginal da a se .
See igu e 5.1 ha illus a es he accu acy di e ence o he wo app oaches.
Acco ding o able 5.2, he selec ed ea u es had a maximum loss o 24 pe cen age poin s and
a maximum gain o 10.4 pe cen age poin s compa ed o he da a se wi h all he ea u es c ea ed.
Howe e , compa ing he selec ed esul s wi h he o iginal da a se esul s, we ha e a maximum
loss o 7.3 pe cen age poin s, while we ha e a maximum gain o 36.7 pe cen age poin s.
Again, ou bigges imp o emen was in he Hill- alley da a se eaching an accu acy o 93.48%,
losing -3.79 pe cen age poin s when compa ed o all gene a ed ea u es, which has an accu acy o
97.27%, and i compa ed o he o iginal da a se i is a gain o 36.7 pe cen age poin s. See de ailed
esul s in Appendix A.
These esul s indica e ha e en wi h he wo se esul compa ed o using all he ea u es, and
we can some imes imp o e he esul s o he o iginal da a se and he ex ended da a se . We do
ha e a highe a e age loss wi h his app oach, bu i shows us ha he app oach can be imp o ed
and maybe we can o e come some esul s. The o e all esul s indica e ha e en we collec ed
p omising esul s on he me a-le el, i leads us o a wo se esul on he base-le el. This is p elimi-
na y wo k and se e al challenges we e add essed wi h simple solu ions ha could cause some loss
o pe o mance, as he selec ion o he inal se o ea u es and da a cha ac e isa ion. Howe e , i
shows us ha he app oach can be imp o ed and i needs u he in es iga ions.

5.4 Resul s 31
Figu e 5.1: Compa ison o he accu acy om selec ed ea u es, comple e se o ea u es and
o iginal ea u es
32 Me a-lea ning o selec indi idual ea u es
Chap e 6
Conclusions and Fu u e Wo k
P epa ing a da a se co ec ly usually akes a signi ican amoun o ime and i equi es subs an ial
knowledge abou he domain. An essen ial ask on his p ocess is ea u e enginee ing, which
consis s o gene a ing new ea u es om he exis ing ones. This is one o he mos complex and
ime-consuming asks oday.
The e is an inc ease o esea ch on ea u e enginee ing o e he las yea s, wi h a g owing
in e es in au oma ed machine lea ning. Ou esul s show ha au oma ed app oaches can ex ac
many ea u es om he aw da a se independen ly o he p oblem and domain knowledge, o en
leading o mo e accu a e models.
P e ious esea ch has ocused on applying di e en ope a ions on he aw da a se . In his
hesis, we p opose a me a-lea ning app oach o au oma e he p ocess o ea u e enginee ing. Thus
gi en a da a se we sys ema ically gene a e a se o new ea u es, and as i can gene a e a as
numbe o ea u es we use a il e o d op he highly co ela ed ea u es using he il e co ela ion
me hod.
Ou app oach uses me a- ea u es ha desc ibe he ex ension and he o iginal da a se s, and a
combina ion o bo h o quan i y he di e ences be ween hem. In his p ojec , we used a ios o
ha pu pose. This app oach was used o answe : "Can a me a-lea ning app oach p edic when a
me hod o he sys ema ic gene a ion o ea u es leads o be e p edic i e models?". The empi ical
s udy ca ied ou ocuses only on nume ical a iables. Howe e , he app oach can be applied o
a iables o any na u e.
We de eloped ano he me a-lea ning app oach o he p oblem o p edic ing whe he a new ea-
u e will lead o mo e accu a e models. This p oblem has he addi ional challenge o cha ac e ising
a single a iable, while ypical me a- ea u es cha ac e ise da a se s. We ex ac ed he in o ma ion
o each new ea u e and he ex ension da a se wi hou he speci ic ea u e, also combining hem
o collec he di e ence be ween he da a se s. This app oach was used o answe ou ques ion
"Can a me a-lea ning app oach p edic when an indi idual ea u e leads o a be e p edic i e
model?"
33
34 Conclusions and Fu u e Wo k
6.1 Resul s
This hesis p oposed o iden i y he e ec s o au oma ed ea u e gene a ion using me a-lea ning
as he app oach. Based on empi ical esul s, and we can assume ha a sys ema ic combina ion o
ea u es using simple ope a ions can lead o be e p edic i e models. Applying a simple il e o
highly co ela ed ea u es i is possible o imp o e he model pe o mance.
Addi ionally, he empi ical esul s show ha he me a-lea ning app oach can p edic when a
me hod o he sys ema ic gene a ion o ea u es leads o be e p edic i e models. The esul s
also indica e ha he me a-lea ning app oach can p edic when a me hod o au oma ic gene a ion
o ea u es leads o be e p edic i e models wi hou applying i , using only he o iginal da a se
wi hou he me ada a o he new ea u es.
On he o he hand, we did no ge sa is ac o y esul s on he p oblem o p edic ing whe he a
ea u e will lead o be e models. E en hough he esul s a he me a-le el we e p omising, he
selec ed ea u es did no lead o be e base-le el models.
Howe e , his is a oo complex p oblem and we decided o simpli y, causing some limi a ions
o he p oblem. Limi a ions such as he cha ac e isa ion o he da a and he compu a ional cos o
gene a e a me a-da a o each ea u e. In addi ion, he esul s indica e ha he e is mo e esea ch
o be done o he selec ion o ea u es using he me a-lea ning app oach, applying di e en ech-
niques o he same app oach.
6.2 Fu u e Wo k
Fo u u e wo k, we see he a emp o elimina e some limi a ions in he cu en wo k.
Fo he selec ion o indi idual ea u es, we would like o add ess a mo e ealis ic app oach:
ins ead o emo ing only a single ea u e, emo e a andom ac ion o ea u es om he da a se ,
epea ing he p ocess many imes. Thus, a oiding he es ic ion, we had o gene a e me ada a o
a single ea u e, as in es iga e i i is possible o build new me a- ea u es o an indi idual ea u e
o a new app oach o he selec ion o ea u es.
New app oaches should also seek o mi iga e ime consump ion in me ada a gene a ion, mak-
ing his app oach mo e p ac ical. Mo eo e , i would be good o implemen o he ea u e gen-
e a ion ans o ma ions such as una y and highe -o de ans o ma ions. Addi ionally, we could
combine he ea u e gene a ion ools wi h he me a-lea ning app oach ci ed he e.
Du ing he p ocess o analysis o his wo k, i was possible o see he impac ha a single da a
se gene a es on he inal esul , so an ob ious s ep would be o add new da a se s, mainly da a se s
wi h a la ge olume.
He e, we only used he Random Fo es algo i hm. Howe e , he e a e many algo i hms ha
we can use o his p oblem and should be conside ed o make a compa ison in u u e wo k.
Appendix A
Base le el accu acy
The ables A.1,A.2,A.3 a e ela ed wi h he o iginal and ex ended ea u es wi hou he il e o
highly co ela ed ea u es. The ables A.4,A.5,A.6 is abou he o iginal and ex ended ea u es
wi h he il e o highly co ela ed ea u es. The ables A.7,A.8,A.6 is abou he accu acy o
each da a se using only he indi idual ea u es ha supposedly would lead a be e pe o mance
o he model.
35

36 Base le el accu acy
Table A.1: Accu acy o each da a se es ed wi hou il e on base-le el.
Da a se s Ex ended O iginal Ex ended s O iginal Class
abalone 63.97 63.44 0.53 1
aids 63.33 60 3.33 1
ai lines 59.94 60 -0.06 0
analca da a_c edi sco e 98.89 98.89 0 0
au ho ship 98.1 99.17 -1.07 0
au oMpg 87.92 88.67 -0.75 0
au oUni -au7-700 48.61 48.99 -0.38 0
backache 86.28 86.75 -0.47 0
blood- ans usion-se ice-cen e 68.06 66.6 1.46 1
bond a e 67.81 63.57 4.24 1
b eas -w 96.01 96.29 -0.28 0
chu n 96.28 95.74 0.54 1
cmc 52.34 51.79 0.55 1
c edi -g 69.9 70.8 -0.9 0
cyyoung 82.97 82.75 0.22 1
diabe es 76.43 75.78 0.65 1
diabe es130US 35.59 41.25 -5.66 0
ecoli 86.7 87.57 -0.87 0
eeg-eye-s a e 58.64 59.21 -0.57 0
elec ici y 69.31 69.41 -0.1 0
glass 74.61 82.13 -7.52 0
habe man 67.38 64.78 2.6 1
hea -s a log 83.33 84.07 -0.74 0
hepa i is 75.03 75.03 0 0
houses 94.67 94.28 0.39 1
kddcup09_upselling 92.44 92.47 -0.03 0
le e 97.28 96.57 0.71 1
mammog aphy 98.83 98.75 0.08 1
mo phological 99.85 99.85 0 0
page-blocks 96.89 96.75 0.14 1
phoneme 91.17 91.36 -0.19 0
p nn_c abs 99.5 80 19.5 1
p o b 63.4 59.67 3.73 1
m sa_sleepda a 36.76 36.25 0.51 1
sa elli e 99.33 99.29 0.04 1
sa image 91.66 91.85 -0.19 0
sona 69.95 73.72 -3.77 0
ae 81.14 80.43 0.71 1
eachingassis an 63.99 65.2 -1.21 0
i anic 64.4 64.17 0.23 1
ehicle 78.25 75.99 2.26 1
innie 80.47 80.47 0 0
olcanoes 96.37 96.44 -0.07 0
wall- obo -na iga ion 99.58 99.76 -0.18 0
wholesale-cus ome s 92.26 91.82 0.44 1
wine 96.14 96.73 -0.59 0
yeas 58.85 59.05 -0.2 0
Base le el accu acy 37
Table A.2: Accu acy o each da a se es ed wi hou il e on base-le el [con inui y].
Da a se s Ex ended O iginal Ex ended s O iginal Class
allbp 97.77 97.67 0.1 1
hy oid-ann 99.79 99.58 0.21 1
au os 69.15 67.83 1.32 1
op digi s 97.68 97.69 -0.01 0
engine 81.44 86.2 -4.76 0
calenda dow 58.88 58.11 0.77 1
led-display-domain 71.51 71.77 -0.26 0
sma phone-based
_ ecogni ion_o _
human_ac i i ies
96.67 97.22 -0.55 0
s eel-pla es- aul 97.16 91.52 5.64 1
olcanoes-d4 94.04 94.2 -0.16 0
BNG_b eas -w 98.61 98.61 0 0
BNG_cmc 52.87 52.02 0.85 1
qsa -biodeg 85.4 84.64 0.76 1
kc2 80.19 80.79 -0.6 0
ozone-le el-8h 93.45 92.66 0.79 1
hill- alley 97.27 56.78 40.49 1
wdbc 96.85 96.15 0.7 1
clima e-model
-simula ion-c ashes 91.12 91.31 -0.19 0
spambase 93.59 94.09 -0.5 0
ilpd 68.24 68.26 -0.02 0
kc1 83.46 83.13 0.33 1
pc1 93.51 93.7 -0.19 0
pc3 90.08 89.95 0.13 1
pc4 91.29 91.02 0.27 1
bankno e
-au hen ica ion 99.85 99.34 0.51 1
mozilla4 84.89 84.55 0.34 1
pendigi s 99.37 99.07 0.3 1
balance-scale 84.5 68.42 16.08 1
ca dio ocog aphy 100 100 0 0
i s -o de
- heo em-p o ing 57.71 57.66 0.05 1
wil 98.39 98.16 0.23 1
hy oid-allhype 70.29 70.68 -0.39 0
hy oid-allbp 70.29 70.68 -0.39 0
hy oid-allhypo 70.29 70.68 -0.39 0
hy oid-all ep 70.29 70.68 -0.39 0
hy oid-dis 70.29 70.68 -0.39 0
sa elli e_image 89.9 89.68 0.22 1
seismic-bumps 91.26 92.19 -0.93 0
segmen 98.1 97.88 0.22 1
38 Base le el accu acy
Table A.3: Accu acy o each da a se es ed wi hou il e on base-le el [con inui y].
Da a se s Ex ended O iginal Ex ended s O iginal Class
b eas - issue 51.01 66.48 -15.47 0
c edi -app o al 75.12 73.94 1.18 1
banana 89 89.68 -0.68 0
pc1_ eq 67.46 66.49 0.97 1
pa kinsons 84.06 83.59 0.47 1
i is 96 96.67 -0.67 0
jungle_chess_2pcs_ aw_endgame
_comple e 71.61 68.22 3.39 1
oil_spill 95.19 95.83 -0.64 0
ho acic-su ge y 83.62 84.04 -0.42 0
jm1 79.18 79.05 0.13 1
JapaneseVowels 93.23 92.52 0.71 1
eucalyp us 22.13 20.79 1.34 1
all ep 98.54 97.8 0.74 1
use -knowledge 90.73 90.2 0.53 1
Base le el accu acy 39
Table A.4: Accu acy o each da a se es ed wi h il e on base-le el.
Da a se s Ex ended Fil e ed O iginal Fil e ed Ex ended s O iginal Class
abalone 64.35 54.66 9.69 1
aids 60.83 60 0.83 1
ai lines 59.81 60 -0.19 0
analca da a_c edi sco e 98.89 98.89 0 0
au ho ship 98.81 99.17 -0.36 0
au oMpg 89.71 86.64 3.07 1
au oUni -au7-700 46.29 49.28 -2.99 0
backache 85.63 86.75 -1.12 0
blood- ans usion
-se ice-cen e 67.13 67.53 -0.4 0
bond a e 64.38 63.57 0.81 1
b eas -w 95.87 96.29 -0.42 0
chu n 95.84 94.96 0.88 1
cmc 52.54 51.79 0.75 1
c edi -g 69.9 70.8 -0.9 0
cyyoung 83.72 84.97 -1.25 0
diabe es 76.3 75.78 0.52 1
diabe es130US 44.88 41.25 3.63 1
ecoli 86.93 87.89 -0.96 0
eeg-eye-s a e 57.03 54.57 2.46 1
elec ici y 68.92 69.41 -0.49 0
glass 74.47 82.13 -7.66 0
habe man 69.36 64.78 4.58 1
hea -s a log 82.22 84.07 -1.85 0
hepa i is 75.03 75.03 0 0
houses 94.63 94.26 0.37 1
kddcup09_upselling 92.46 92.47 -0.01 0
le e 97.24 96.57 0.67 1
mammog aphy 98.84 98.75 0.09 1
mo phological 99.85 99.75 0.1 1
page-blocks 97.09 96.77 0.32 1
phoneme 91.52 91.36 0.16 1
p nn_c abs 99.5 75.5 24 1
p o b 62.8 59.67 3.13 1
m sa_sleepda a 36.25 36.25 0 0
sa elli e 99.45 99.29 0.16 1
sa image 90.96 90.08 0.88 1
sona 70.11 73.72 -3.61 0
ae 79.1 80.43 -1.33 0
eachingassis an 60.66 65.2 -4.54 0
i anic 64.55 64.17 0.38 1
ehicle 78.61 75.53 3.08 1
innie 80.98 80.47 0.51 1
olcanoes 96.34 96.44 -0.1 0
46 REFERENCES
[12] Hugh Lea he , Edwin Bonilla, and Michael O’Boyle. Au oma ic ea u e gene a ion o ma-
chine lea ning based op imizing compila ion. In P oceedings o he 7 h annual IEEE/ACM
In e na ional Symposium on Code Gene a ion and Op imiza ion, pages 81–91. IEEE Com-
pu e Socie y, 2009.
[13] Ca los Soa es Rica do Vilal a Pa el B azdil, Ch is ophe Gi aud-Ca ie . Me alea ning: Ap-
plica ions o Da a Mining. Sp inge Be lin Heidelbe g, 2009.
[14] Randy Ke be Thomas Khabaza Thomas Reina z Colin Shea e Pe e Chapman, Julian Clin-
on and Rüdige Wi h. CRISP-DM 1.0 - S ep-by-s ep da a mining guide. CRISP-DM Con-
so ium, Fi s edi ion, 2009.
[15] Ad iano Ri olli, Luís PF Ga cia, Ca los Soa es, Joaquin Vanscho en, and And é CPLF
de Ca alho. Towa ds ep oducible empi ical esea ch in me a-lea ning. a Xi p ep in
a Xi :1808.10406, 2018.
[16] Ros isla S íz. Me alea ning o da a mining and kdd. pages 1–21, 2013.
[17] Rica do Vilal a, Ch is ophe G Gi aud-Ca ie , Pa el B azdil, and Ca los Soa es. Using me a-
lea ning o suppo da a mining. IJCSA, 1(1):31–45, 2004.
[18] Mlle Bouaguel Waad. On Fea u e Selec ion Me hods o C edi Sco ing. PhD hesis, Uni-
e si é de Tunis - Ins i u Supé ieu de Ges ion, 2015.