Mas e in A i icial In elligence
An In elligen Decision Suppo
Sys em o Machine Lea ning
Algo i hms Recommenda ion
And ei Mihai
ad ised by
D . Miquel S`anchez-Ma `e,
Depa men o Compu e Science, UPC
FACULTAT D’INFORM`
ATICA DE BARCELONA (FIB)
FACULTAT DE MATEM`
ATIQUES (UB)
ESCOLA T`
ECNICA SUPERIOR D’ENGINYERIA (URV)
UNIVERSITAT POLIT`
ECNICA DE CATALUNYA (UPC) –
Ba celonaTech
UNIVERSITAT DE BARCELONA (UB)
UNIVERSITAT ROVIRA i VIRGILI (URV)
Janua y/Feb ua y 2017
Abs ac
Machine lea ning is a e y cen al opic in A i icial In elligence and e en
compu e science in gene al. Nowadays, i s use in Big Da a p oblems is qui e
well known. Howe e , while he big da a, and machine lea ning p oblems in
gene al, a e qui e a ied and in needing o di e en kinds o solu ions, he e a e
as well many di e en me hods in machine lea ning ha can be used. In his
wo k, we p opose an applica ion ha migh help deciding on which machine
lea ning me hods a use needs o a speci ied p oblem.
The applica ion is an In elligen Decision Suppo Sys em o Machine
Lea ning Algo i hm Recommenda ion o which we p esen he design, which
is cen e ed a ound he combined use o he Case-Based Reasoning and Rule-
Based Reasoning, o he ecommending p ocess, while also ying o make
he sys em easy o use and manage. We p esen a p o o ype o such a sys-
em, and he implemen a ion de ails o he wo ecommende algo i hms. The
p elimina y es ing o he p o o ype shows i o be a p omising ool.
Con en s
1 In oduc ion ............................. 3
1.1 P oblem o be sol ed .................... 3
1.2 Mo i a ion .......................... 3
1.3 Issues o he wo k ...................... 4
2 S a e o he a ........................... 5
2.1 Rela ed Wo k ........................ 5
2.2 Tools and echnologies used ................ 7
3 Machine Lea ning Recommende P oposal ............ 11
3.1 O e iew .......................... 11
3.2 High-le el Design ...................... 13
3.3 Abou In elligen Algo i hms used ............. 20
4 De elopmen o he Recommende ................. 23
4.1 Gene al Sys em Implemen a ion .............. 23
4.2 De ails on In elligen Me hods ............... 29
4.3 Abou Da a ......................... 33
5 Expe imen al E alua ion ...................... 37
6 Sus ainabili y Analysis ....................... 39
7 Tempo al Plani ica ion ....................... 43
8 Conclusions and Fu u e Wo k ................... 45
Re e ences ................................. 47
1

CONTENTS
2
An IDSS o Machine Lea ning Algo i hms Recommenda ion
1 In oduc ion
In his chap e we p esen he p oblem ha we ackle in his hesis, wha
a e ou mo i a ions behind his decision and wha a e he issues in sol ing his
p oblem.
1.1 P oblem o be sol ed
Nowadays, one o he main opics in A i icial in elligence is Machine Lea n-
ing. I is used in many ields, mo e p ominen ly in he inc easing ield o Big
Da a. Machine lea ning o big da a is used in many comme cial sec o s such
as e ail, manu ac u ing, a el, inancial se ices o ene gy and u ili ies [15].
While he e a e many and a ied kinds o p oblems equi ing di e en kinds
o big da a and machine lea ning solu ions, he e a e as well many di e en
me hods in machine lea ning ha can be used [23].
Among he ple ho a o me hods and algo i hms ha can be used in machine
lea ning he e a e o cou se ca ego ies and ypes o me hods. Some ypes o
algo i hms a e sui able o some p oblems, some o he ype o algo i hms,
o o he p oblems [23]. Expe ienced machine lea ning scien is migh ha e
knowledge o his kind, bu o new use s i migh no be ob ious whe e o
ind such in o ma ion.
In his wo k, we p opose an applica ion ha migh help deciding on which
machine lea ning me hods a use needs o a speci ied p oblem. This applica-
ion could be used by s uden s, beginne s in machine lea ning o simply use s
ha ha e no expe ise in machine lea ning.
1.2 Mo i a ion
As a s uden o A i icial In elligence I ha e been gi en di e en asks ha
needed o be sol ed wi h machine lea ning me hods. While in some cases
we would ecei e ad ises (knowledge om an expe ienced use ) in o he s we
jus had o choose he me hod. Howe e , as s a ed be o e, he e a e many
me hods o machine lea ning, and we ended up choosing one ha jus seemed
in e es ing o simply he i s which seemed o wo k.
Finding in o ma ion on wha is he bes algo i hm o he p oblem did no
seem o be i ial. O cou se, he e is no way in which o pe ec ly p edic
which is he bes algo i hm o a gi en ask and no algo i hm is bes o all
asks [32]. Ye , a choice o wha algo i hm o use has o be made. E en i
many p oblems using machine lea ning a e sol ed by ial and e o [11], a
choice has o be made on whe e o s a . And so he ques ion a ose: could a
p ac ical In elligen Decision Suppo Sys em (IDSS) be c ea ed o help make
his choice?
3
1. INTRODUCTION
And a e some esea ch, we ound ha , be ween chea shee s o algo i hms
and au oma ic algo i hm selec ion (see chap e 2), he e could be po en ial
behind such an In elligen Decision Suppo Sys em.
1.3 Issues o he wo k
The bigges p oblem o such wo k is he ex emely high numbe o a ia ions
o p oblem/solu ion pai s in machine lea ning (whe e solu ion e e s o he al-
go i hm used). This high a ia ion makes i ha d o pinpoin some clea ules
o (maybe e en ha de ) o au oma ically ex ac hese ules. This leads o
agueness and spa si y and simply ecommending he bes me hod o a p ob-
lem, in an e ec i e way, seems no o be doable wi h he cu en echnology
[21].
And his leads o a second issue, somehow de i ed om he i s one. I is
ha d o ga he he da a needed. E en i he sys em is supposed o ecommend
app oxima ions, he da a needed o uel such a sys em has o come om expe
knowledge and p e iously made expe iences and s udies. Ye , because o he
g ea a ia ion, hese a e qui e spa se, and ying o ind clea in o ma ion on
mos ype o p oblems and mos ypes o me hods in machine lea ning seems
o be e y ha d.
And he las issue we wan o men ion is he cu en g ow h o machine
lea ning. Machine lea ning is a e y in e es ing opic, used and usable in many
ields (as men ioned in sec ion 1.1), and he e is a lo o wo k concen a ed on
his ield, o make i mo e powe ul, wi h be e me hods, applicable on mo e
and mo e p oblems and o wo k be e in olde exis ing p oblems. Simply
pu , Machine Lea ning g ows and changes a a as a e nowadays. While his
may be good o he ield, as i is expanding and ge ing be e , he as pace
o change in i is qui e an issue o he ype o applica ion discussed in his
wo k. Tha is, while ying o c ea e an in elligen decision suppo sys em
o ecommend machine lea ning me hods, machine lea ning is changing. And
his leads o he isk o making he in elligen decision suppo sys em obsole e
as e han i can each i s po en ial. A way o mi iga e, o e en o e come, his
p oblem is by making he sys em easily ex ensible, so ha i can be upda ed
as e o he new echnology.
4
An IDSS o Machine Lea ning Algo i hms Recommenda ion
2 S a e o he a
Nowadays, wi h so many algo i hms and machine lea ning me hods, mos
o he p oblems a e sol ed by ial and e o , in o de o ind he bes lea ne ,
and i may e en be conside ed among he bes p ac ices in machine lea ning
[8]. When aced wi h a p oblem, a machine lea ning expe will y di e en
algo i hms ha in his knowledge could be good o ha p oblem and choose he
bes one. No wi hs anding, wha happens when he e is no enough knowledge
on he p oblem o he use is no a machine lea ning expe ?
While i seems ha he e is no ac ual esea ch o answe his ques ion, bo h
in ou expe ience and esea ch on ”Ques ion and Answe ” websi es such as
Quo a[1] o S ackExchange[2], wha he e is o do is o consul an expe ha
migh ha e some knowledge o simply sea ch he in o ma ion on he in e ne
ying o ind he knowledge you need (whe e he websi es we ha e men ioned
end o help).
In his con ex we hough abou an In elligen Decision Suppo Sys em
o help wi h machine lea ning, especially o s uden s o beginne s.
2.1 Rela ed Wo k
O cou se, ying o ind he bes machine lea ning algo i hm o a p oblem
is no a new idea, qui e he con a y. E en since he ea lie days o machine
lea ning, when mo e and mo e lea ning sys ems s a ed o be de eloped, i
became appa en ha hese sys ems usually a e only e ec i e on a na ow
ange o p oblems, o which hey we e designed [24]. So e o s we e being
made o c ea e a mo e gene al machine lea ning algo i hm, o an algo i hm
ha could gene a e he bes machine lea ning algo i hm needed [11].
In o de o ge he bes ou o machine lea ning o a ce ain p oblem, choos-
ing he bes algo i hm and hype -pa ame e s can be iewed as an op imiza ion
p oblem. In [21], Luo explains qui e well his app oach and summa izes mos
o he wo k done in his di ec ion. Luo ga e 4 main unc ionali ies ha an au-
oma ed machine lea ning sys em can ha e: e icien ly handle big da a, handle
a wide ange o algo i hms, handle a ious ypes o hype -pa ame e s, handle
any numbe o hype -pa ame e alue combina ions. O all he algo i hms Luo
p esen ed, none o hem we e able o ha e all hese unc ionali ies. While some
o hese sys ems can be used in some domains, he e is s ill wo k o be done
be o e hey can be used e icien ly in p ac ice, especially o big da a (in which
[21] had in e es ).
The use in p ac ice o such sys ems can also enable non expe s o use
machine lea ning easily [9]. The e a e ools ha we e c ea ed o his pu pose,
such as Weka,RapidMine o Sciki Lea n. These kinds o ools we e designed
o gi e easy access o machine lea ning me hods, e en o non compu e sci-
en is s. Howe e , only access o a lo o machine lea ning me hods wi hou
he knowledge how o use i only sol es a pa o he p oblem. Tha ’s why
5
3. MACHINE LEARNING RECOMMENDER PROPOSAL
bu mul iple algo i hms ha should be iewed as op ions o wha migh wo k.
In addi ion, he sys em ou pu s o he da a mining s eps ha migh be ele an .
Fo example, ea u e scaling be o e eeding he da a o he algo i hms, i some
o he ecommended algo i hms a e no scale in a ian . In addi ion, he sys em
migh ou pu o he in o ma ion ha migh be ele an o he use , such as
di e en esou ces. Fo ins ance, i i ecommends Suppo Vec o Machines
wi h a non-linea ke nel, i migh also gi e a esou ce whe e di e en ypes o
ke nels a e explained, so he use can mo e easily implemen he ecommended
algo i hm.
In sho , he p oposed sys em akes in o accoun me a-da a abou he p ob-
lem, which can a y g ea ly, depending on he p oblem, and e u ns mul iple
ecommended algo i hms, possible wi h some me a-da a (hype pa ame e s)
con igu a ions o hose algo i hms, possible ecommenda ions o o he s eps
o he da a mining p ocess, and possibly o he in o ma ion ha migh be el-
e an o he use .
As echnologies o powe he p oposed sys em we e chosen Rule-Based Rea-
soning and Case-Based easoning. A e y impo an cha ac e is ic in bo h
me hods is ha hey can easily handle inpu da a wi h a ying a ibu es.
Unlike o he ypes o me hods, in e ence on ules and cases’ e ie al can be
done wi h highly di e en inpu wi hou e o , i enough ules and cases a e
p o ided.
Conside ing ha mos po en ial use s, in he absence o a sys em such as
he one p oposed he e, used ” ules o humb” ound on he in e ne o gi en
by expe s in o de o s a implemen ing he da a-mining p ocess, we hough
ha a ule-based sys em would be a good i o ou sys em. E en i such ules
a e no pe ec , hey a e good enough o guide use s. Ga he ing many such
a ules oge he in one ule-based sys em can gi e be e guidance han he
spa se ules o humb one migh ound, and his guidance is much easie o
each and use. One o he majo d awbacks in ule-based sys ems is ha hey
become unmanageable as he numbe o ules inc eases g ea ly, bo h in e ms
o compu a ional ime and ule base managemen . On he o he hand, ou
sys em does no need o gi e he pe ec answe , so ha an ex ensi e se o
ules o co e all possibili ies is no needed. Thus, some ules o mo e gene al
ecommenda ions and maybe o some speci ic p e alen p oblems, should be
enough o a good-enough guiding. Conside ing his, we don’ expec he size
o he ule base o become unmanageable.
Ano he d awback o a ule-sys em is he ac ha hey a e no lexible,
hey canno adap o an changing en i onmen . And he wo ld o machine
lea ning and da a-mining is qui e dynamic, especially nowadays when A i icial
In elligence is being de eloped apidly. While he e migh be some ules o
humb ha pass he es o ime, a be e guidance is gi en i he ules a e
adap ed o he new algo i hms and a ia ions, and o di e en p oblems. Thus,
in o de o such a sys em o wo k, he e is need o cons an upda e o he ule
12

An IDSS o Machine Lea ning Algo i hms Recommenda ion
base by expe s.
Al hough, his is exac ly one o he s eng hs o he case-based easone .
We decided o add a case-based easone because we hough ha ecommen-
da ions om gene al ules migh no be enough. The case-based easone is
hough o gi e ecommenda ions om eal cases, i.e. om o he expe iences.
Thus, he sys em would gi e guidance based on ules o humb and p e ious
expe ience. We obse ed ha he wo sys ems a e qui e complemen a y. While
he e is cons an need o expe s o manage he case base, he sys ems encou -
ages use s o add hei inal solu ion o he case base, so ha , i he sys em is
used, he case base is always up o da e wi h he new ypes o p oblems and
solu ions. On he o he hand, while he ule base does no need an excessi e
amoun o ules o unc ion p ope ly, he case base ends o g ow cons an ly,
po en ially ge ing o big o e ec i ely use i o easoning, o o manage i .
This could be o e come by ha ing expe s ha manage he case base o by
ha ing an au oma ed p ocess ha emo es less use ul cases.
As his sys em is o be used mainly by non-expe s, ease o use was also
aken in o accoun . The me a-da a o he p oblem is gi en o he sys em using
an in o ma ion ex ac ion app oach. The use is gi en a i s some gene al
ques ions abou he p oblem, such as he goal o he numbe o samples, and
mo e speci ic ques ions appea as he use gi es answe . This unc ionali y is
also powe ed by a ule-based sys ems. Mo eo e , he e is he need o cons an ly
manage he ule base and case base, so he sys em p o ides ools o ease he
managing o ules and cases.
3.2 High-le el Design
In his sec ion we will p esen he high-le el design o ou p oposed sys em.
Tha is, how we en ision he sys em should wo k and o mally p esen i .
We ha e clea ly s a ed in he i s wo sec ions who a e he use s o ou
p oposed sys em. On op o ha , he e should be adminis a o s and/o e-
sea che s ha manage he ule base and case base so ha he ecommenda ions
a e up o da e. The di e en ways ha ou sys em can be used can be o mally
desc ibed by a use-case diag am.
Figu e 1shows his sys em’s use-case diag am. The wo main ac o s a e
he use and he adminis a o / esea che , who has expe knowledge abou
machine lea ning and di e en algo i hms. The usual use mainly needs ec-
ommenda ion abou a speci ic p oblem, which needs ha he use en e s me a-
da a abou he p oblem. Also, he use needs o be able (and should be en-
cou aged o) o inse cases in he case base, ha is, inse he inal solu ion o
his/he p oblem. The expe (adminis a o o esea che ) needs o be able o
manage he case base, ha is, o add cases, change cases (i hey a e ou da ed
o inco ec and so on) and dele e cases (i hey a e no needed anymo e o i
hey a e edundan ). I also needs o be able o manage he ule base and ac
13
3. MACHINE LEARNING RECOMMENDER PROPOSAL
Figu e 1: Use Case Diag am.
base in o de o keep he ules up o da e and also keep he in o ma ion ex-
ac o up o da e wi h he mode n p oblems (new ype o me a-da a needed)
and algo i hms.
In o de o accommoda e hese needs, he sys em needs clea ly sepa a ed
sub-sys ems ha can be accessed and con olled h ough a G aphical Use
In e ace (GUI). The sys em has a ule-based sub-sys em ha includes an
in o ma ion ex ac o and he ecommende . The sys em also has a sepa a e
case-based easone ha can gi e ecommenda ion based on a gi en p oblem.
Also, he sys em has a G aphical Use In e ace wi h which he ac o s can
in e ac wi h he sys em. In o de o combine he unc ionali ies o hese sub
sys ems i needs a con olle ha links he inpu s om he G aphical Use
In e ace o he igh sys em ( ule-based easone o case-based easone ) can
ge he answe s and gi e hem o he GUI in a way hey can be displayed
14
An IDSS o Machine Lea ning Algo i hms Recommenda ion
p ope ly. The gene al low o he sys em can be seen in Figu e 2.
Figu e 2: Gene al low o he sys em
Now, we will go a bi in o de ails on how he sepa a e sys ems a e designed.
We will no go in o de ails o he G aphical Use In e ace, as i eally de-
penden on he implemen a ion and we will p esen mo e when we alk abou
he sys em p o o ype we de eloped. The h ee main sys ems a e he case-
based easone , he ule-based easone and he con olle . The con olle is
ac ually coo dina ing he whole ecommending p ocess, so when p esen ing
he con olle design, we ac ually p esen he design o he ecommenda ion
p ocess.
One o he subsys ems is he ule-based easone . The ule-based easone
has wo oles: in o ma ion ex ac ion and ecommenda ion. In Figu e 3 he
gene al design o he ule-based easone is p esen ed. The e a e h ee main
pa s o he ule-based easone : he in o ma ion ex ac ion wo k low, he ule-
based ecommende wo k low and he ule base and ac base managemen .
Simply pu , he managemen o he ule base and ac base a e gene al CRUD
ope a ions: c ea e new ules and ac s, upda e some ules and ac s wi h new
in o ma ion o co ec hem and dele e obsole e o w ong ules and ac s.
The e a e wo ule bases and one ac base. The ule-based ecommende uses
one ule base, con aining ules abou wha ecommenda ion o gi e in wha
si ua ion. The e is no s a ic ac base o he ule-based ecommende , as all
he ac s a e gi en by he use , h ough he G aphical Use In e ace. The e is
one ule base o he in o ma ion ex ac o , con aining ules exp essing wha
15
3. MACHINE LEARNING RECOMMENDER PROPOSAL
Figu e 3: The ule-based subsys em design.
ques ions o ask he use in wha si ua ion. The e is also a s a ic ac base o
he in o ma ion ex ac o con aining in o ma ion such as wha a e he possible
answe s o di e en ques ions.
The e needs o be a subsys em ha can manage hese in o ma ion bases,
which is ep esen ed by he ”Managemen ” squa e in he igu e. The ”Man-
agemen ” subsys em ge s eques s o modi y he in o ma ion bases om he
use ( h ough he G aphical Use In e ace and he con olle ) and i pe o ms
managemen ope a ions on he bases, also assu ing hei pe sis ence.
The Rule-Based Recommende subsys em ecei es ac s abou he p oblem,
no necessa ily in a ep esen a ion ha he in e ence engine can use, so some
con e sion migh be necessa y. Then i uses he in e ence engine o in e ac s
om he gi en da a and he exis ing ules abou ecommenda ion. Finally,
he ule-based ecommende assembles he in e ed ac s in a solu ion usable
by he con olle (which will pass i o he G aphical Use In e ace which will
display i o he use ).
Finally, he e is he In o ma ion Ex ac o subsys em ha ecei es ac s
abou he p oblem, and simila ly o he ules ecommende , migh need o
con e hem in o de o become usable by he in e ence engine. Then i uses
hese ac s and he ac s in he ac base o in e wha o he ques ions need o
be displayed o he use , based on he exis ing ules.
16
An IDSS o Machine Lea ning Algo i hms Recommenda ion
Figu e 4: The case-based easone subsys em design.
The second pa o ou p oposed sys em is he Case-Based Reasone . In ig-
u e 4 he gene al design o he case-based easone subsys em can be iewed.
The Case-Based Reasoning is isibly simple han he Rule-Based Reasoning,
since he Case-Based Reasoning only has one main goal: gi ing ecommenda-
ions (as opposed o he Rule-Based Reasoning, which also is used o in o -
ma ion ex ac ion). The e is one case base, ha he case-based ecommende
uses. Each case con ains a pai o p oblem-solu ion, ha we e added by he
sys em’s adminis a o s, esea che o expe s, o (and his should be ue o
mos cases) added by he use s hemsel es, once hey success ully inished a
da a-mining ask, so hei solu ion can be used in he u u e by ano he use .
As men ioned, his case base needs o be able o be managed by adminis a-
o s, esea che s o expe s, so he e is need o a ”Managemen ” subsys em, as
in he case o he ule-based ecommende abo e. Ve y simila ly o he a o e-
men ioned, his managemen subsys em ecei es CRUD managemen eques s
om he use , h ough he GUI and he con olle , and modi ies acco dingly
he case base, assu ing ha he modi ica ions a e pe sis en .
The case-based ecommende ecei es da a abou he p oblem om he
use and, like in he p e ious cases, i migh need o con e he ecei ed da a
o be p ope ly used by he easone , depending on he implemen a ion. Then
i uses he gi en da a o sea ch o simila cases. The simila i y is compu ed
17

3. MACHINE LEARNING RECOMMENDER PROPOSAL
compa ing he p oblem pa o he case, wi h he gi en p oblem. The e migh
be a need o comple ely igno e some exis en cases in he case base, which may
look simila bu a e o no ac ual use o he gi en p oblem. Fo example, e en
i he da ase is he same, i he goal is dimensionali y educ ion ins ead o
classi ica ion, hen he case should be igno ed, as algo i hms o classi ica ion
may pe o m poo ly on dimensionali y educ ion asks, e en i i is he same
da ase .
Figu e 5: The gene al ecommenda ion low. The con olle pa o he sys em
akes he ole o ” he” ecommende .
And inally, he e is he Con olle pa o he sys em, ha makes he link
wi h all o he o he pa s. We iewed he con olle as playing he ole o
” he” ecommende , as in, being he o e all ecommende ha he use uses,
encompassing in i s s uc u e he in o ma ion ex ac o , he ule-based ec-
ommende and he case-based ecommende . The con olle also has he ole
18
An IDSS o Machine Lea ning Algo i hms Recommenda ion
o pass on he managemen in o ma ion o he espec i e subsys em, bu his
pu pose is mo e o a seconda y one, he p ima y one being ha o coo dina -
ing he ecommenda ion p ocess and gi ing o display o he G aphical Use
In e ace he inal solu ion i.e. he ecommenda ion eques ed by he use . In
Figu e 5, he low o a comple e ecommenda ion as well as he con olle ’s
ole as ” he” ecommende can be seen.
Now, we will ocus only on he ecommende pa o he con olle , as he
managemen pa is less in e es ing and qui e i ial (as he con olle only
passes he eques om he GUI o he espec i e subsys em). In ou iew
o he sys em, he ecommenda ion is pe o med in h ee s eps. Fi s , he
in o ma ion abou he p oblem is ex ac ed, hen he solu ions a e eques ed
om he wo ecommende s and inally hese solu ions a e combined o o m
a inal solu ion o gi e o he use .
•In he i s s ep, he use sends wi h he help o he G aphical Use
In e ace some da a abou he p oblem. The Con olle han passes
hese da a o he in o ma ion ex ac o , ha in e s wha o he ques ions
migh be asked abou he p oblem o he use . Then he ques ions a e
passed o he G aphical Use In e ace o be displayed o he use . Based
on he new answe s om he use , his p ocess migh be epea ed, un il
enough da a is ga he ed om he use abou he p oblem.
•The second phase o he ecommenda ion p ocess also s a s a he
signal o he use , as he use has o inish uploading he da a o his/he
p oblem in o he sys em. Then he use makes a eques o a ecommen-
da ion. When he con olle ecei es he ecommenda ion, i akes he
inse ed p oblem da a and passes hem o he case-based easone , e-
ques ing o a ecommenda ion, and o he ule-based easone , eques -
ing o a ecommenda ion. The o de o he eques is no impo an ,
he ecommende s can be eques ed in any o de . The ecommende s
can e en wo k in pa allel i he implemen a ion enables i and i seems
necessa y, o ha i imp o es he ecommenda ion p ocess. In any case,
he nex phase s a s when he con olle ecei es bo h o he solu ions
om he wo ecommende s.
•When he con olle has ecei ed bo h ecommenda ions, he hi d s ep
s a s. While solu ions ha e al eady been gi en, he con olle can gi e
only one solu ion o he GUI o displaying o he use . The e o e, o
his, he wo solu ions gi en need o be me ged. Thus, we en isioned an-
o he subsys em ha we named ”Solu ion/Recommenda ion Combine ”.
This subsys em akes wo solu ions and ou pu s only one, combining he
ecommenda ions gi en in he wo solu ions. We did no p o ide a gen-
e al design o he Recommenda ions Combine , as i is qui e dependen
on he ep esen a ion o he solu ion in a pa icula implemen a ion o
he sys em.
19
3. MACHINE LEARNING RECOMMENDER PROPOSAL
3.3 Abou In elligen Algo i hms used
In his sec ion we will ocus mo e on he ole o he ule-based sys em and
he case-based sys em in ou p oposed me hodology. Also, e en i no ech-
nically conside ed an in elligen me hod, we will sho ly s udy mo e closely
he combina ion o he ou pu o he Rule-Based Reasoning sys em and he
Case-Based Reasoning sys em.
Rule-Based Sys em The i s in elligen sys em used in ou sys em is he
ule-based sys em ha suppo s ecommenda ion and in o ma ion ex ac ion.
We en isioned his ule-based sys em o be some hing akin o an expe sys em,
mimicking he ques ions ha an expe may ask abou a gi en p oblem ( he
in o ma ion ex ac o ) and he ules o humb he may gi e in o de o help
wi h he ask ( he ecommende ).
The e a e wo main componen s ha a e a pa o a ule-based sys em:
he in e ence engine (bo h o wa d and backwa d) and he knowledge gi en
in o m o ules and ac s. The in e ence engine is a gene al algo i hm ha
gi en a se o ules and ac s, can in e new ac s ollowing some kind o logic
o malism (e.g. i s -o de logic). O cou se, he ules and ac s a e supposedly
ollowing he same logic o malism. The in e ence engine is independen o
he p oblem and o he ules and ac s i is needed o be used on. The only
link be ween he in e ence engine and he ac ual p oblem i is used o sol e
is he logic o malism. Tha is, he da a needs o be able o be ep esen ed
in he o malism ha he engine wo ks wi h. The e a e much mo e p oblems
o be sol ed wi h a ule-based sys em han he e a e logic o malisms, hus
he e can be one p oblem o c ea e an in e ence engine o a logic o malism,
and a comple ely sepa a e p oblem o ep esen a speci ic p oblem in a logic
o malism and use he in e ence engine made o ha o malism o sol e i .
Tha is why he second main componen when using a ule-based sys em is
he ep esen a ion o you p oblem as ules and ac s in he logic o malism o
choice, in a way ha using an in e ence engine, he p oblem can be sol ed.
While an al eady exis en in e ence engine can be used in he sys em, he
p oblem ha he ule engine poses is he ansla ion o he in o ma ion he
sys em uses in o a logical o malism. So he ques ion is, how o ep esen
he p oblem me a-da a as ac s in his logical o malism, how o ep esen he
solu ion o he ecommenda ion as ac s, and how o use he pa icula logical
o malism o ep esen he ules ha lead om one o ano he ( om me a-
da a o ecommenda ion). Also, he in o ma ion ex ac ion uses a ule-based
sys em. The same in e ence engine, o a di e en one can be used o powe he
in o ma ion ex ac o . In any case, he ques ion o how o ep esen ques ions
and possible answe s emains.
I is e y impo an o no e, ha he way o ep esen he ac s and ules o
he in o ma ion ex ac o mus be linked o he ep esen a ion o ac s and ules
o he ecommende . Because, he in o ma ion ex ac o may gi e possible
answe s o a ques ion i sends o be asked o he use , and he answe he use
20
An IDSS o Machine Lea ning Algo i hms Recommenda ion
gi es will be used as me a-da a ac abou he p oblem by he ecommende .
Thus he e needs o be a kind o unde s anding be ween he ac s gene a ed by
he in o ma ion ex ac o and he ac s ha he ecommende can use. This
unde s anding needs o be implemen ed (a leas in pa ) a he o maliza ion
le el, so ha he ac s a e ep esen ed in he same way, and he expe en e ing
he ac s needs o make su e ha he ac s om he in o ma ion ex ac o a e
unde s ood by he ecommende .
Case-Based Reasoning Case-Based Reasoning is he second in elligen
me hod used in ou p oposed sys em. In he adi ional CBR sys em he e
a e ou basic s eps: e ie e, euse, e ise and e ain [26].
In he e ie ing phase we need o ind he cases ha had he mos simila
p oblem o he one gi en by he use . This is he mos impo an s ep in
ou sys em’s Case-Based Reasoning, as he be e ma ch he easone inds o
he p oblem, he mo e will he solu ion gi en help he use . I is c i ical o
de elop a good simila i y unc ion be ween p oblems. While his is dependen
on he implemen a ion, he e a e some highe le el decisions ha need o
be made, based on knowledge o da a-mining p oblems and algo i hms. Fo
example, as men ioned be o e, maybe some cases should no be eligible based
on he goal (i he p oblem has he goal o dimensionali y educ ion and he
p oblem in he case had he goal o classi ica ion, hen maybe he solu ion o
he case is no applicable o he p oblem). O , maybe some me a-da a needs
o be gi en di e en weigh s, o example, gi ing mo e weigh o he da a ype
(image da a, sound da a e c.) han o he numbe o samples, al hough bo h
migh be mo e impo an han he le el o noise in he da a. These decisions
a e made ha ing knowledge abou di e en p oblems and machine lea ning
algo i hms. While he e is no pe ec simila i y unc ion, expe knowledge can
help. Wha is e en mo e impo an is ha he simila i y unc ion’s e icacy
my change in ime as di e en ypes o p oblems and algo i hms a ise ha a e
maybe a ec ed by di e en da a in he p oblem. E en i we did no design ou
sys em o i now, i may be a u u e de elopmen he addi ion o a G aphical
Use In e ace sus ained modi ica ion o he simila i y unc ion, so ha he
expe s ha manage he case base, migh also manage he simila i y unc ion,
and upda e i i needed.
The euse phase in ou case, e e s o e u ning a solu ion o he con olle .
This can simply be a s aigh o wa d ac ion, such as e u ning he solu ion
o he case wi h he p oblem mos simila o he gi en p oblem, o migh be
mo e complex, such as combining he solu ion o he wo o h ee bes cases.
The e ise case is no done au oma ically by he sys em. Ins ead i is done
manually by he use . The use akes he combined ecommenda ion o he
ule-based easone and he case-based easone and ies o sol e i s p oblem,
o comple e i s da a-mining ask. Then, when hey managed o success ully
comple e he ask hey had, hey will add a e ised e sion o he solu ion,
o he case base; he solu ion ha ac ually was he bes o hem. Thus he
21
4. DEVELOPMENT OF THE RECOMMENDER
class Da a, o ep esen his o he da a abou he p oblem ha migh
be en e ed by he use , and he P oblem would ha e a lis o hese Da a
objec s. A Da a objec has a name, which usually is he ques ion ha is
asked o he use , a alue, which is he alue gi en by he use , a lis o
s ing ep esen ing he possible alues, and ano he s ing ep esen ing
an id code. The possible alues a iable, and he idCode a iable a e
needed in o de o be able o use he da a wi h he ule-based ecom-
mende . Inside he ules o maliza ion, he da a will be iden i ied by i s
id code a he han i s name. And, i i would be a nominal a ibu e, i
is impo an ha he alues gi en by he use a e ma ching wi h he ones
inside he ules, and o ha eason he e a e possible alues gi en o he
use . O cou se, his means ha he e is a need o coo dina ion be ween
he in o ma ion ex ac o and he ule-based ecommende , so ha he
possible alues gi en can be ecognized by he ules ecommende .
We needed o s ic ly decide wha would a ecommenda ion mean. Wha
should a solu ion o he use ’s p oblem be. As s a ed in a p e ious chap e , we
iewed he ecommenda ion mo e as a guideline a he han an exac answe
ha he use needs o use, and ha he ecommenda ion means one o mo e
algo i hms, some o hei hype pa ame e s, some o he da a mining s eps and
maybe some o he in o ma ion ele an o he use . Thus, we decided ha he
Solu ion should be ep esen ed as a class ha has h ee a ibu es:
1. Recommended algo i hms, which is a lis o Algo i hm objec s,
2. Recommended o he da a mining s eps, which is a lis o Da aMining-
S ep objec s,
3. A s ing ha ep esen s he o he ele an in o ma ion.
An Algo i hm objec has:
1. a name, which is a alue om he Enume a ion ha con ains he possible
algo i hm names,
2. a lis o Hype pa ame e objec s.
AHype pa ame e objec has a name, a alue and de ails, all being
s ings. Hype pa ame e would be be e ep esen ed as a S uc , bu he e
a e no S uc s uc u es in Py hon, hus Hype pa ame e is a class wi h
public a ibu es.
ADa aMiningS ep objec is simila o a Hype pa ame e objec . I
has h ee a ibu es: name, solu ion and de ails.
28

An IDSS o Machine Lea ning Algo i hms Recommenda ion
And inally, he main.py ile is used o s a up he G aphical Use In e ace
and c ea e he con olle . The unc ions ile con ains h ee unc ions, one ha
ans o ms a use inse ed s ing o boolean ( hus co e ing cases such as T ue,
ue, yes e c.), one ha checks i a s ing is a numbe , and inally one ha
p epa es wo use inse ed s ings o compa ison ( hus emo ing cases and
some punc ua ion).
4.2 De ails on In elligen Me hods
In his sec ion we will p esen in mo e de ail ou app oach a implemen ing
he needed in elligen me hods.
Rule-Based Recommende The i s implemen ed me hod is he ule-
based ecommende . As we men ioned be o e, we ha e used an al eady im-
plemen ed in e ence engine, mo e speci ically, he one used by he pyDa alog
package, which is a Py hon implemen a ion o he da alog logical p og amming
language. Thus, he challenge was o ind a way o ep esen he p oblem and
solu ion as ac s in o de o be able o c ea e ac s om he p oblem da a and
o c ea e a solu ion om he in e ed ac s.
Fo he P oblem, we decided o ep esen he a ibu es o he p oblem as
a p edica e and he alue o he a ibu es as he p edica e pa ame e . So, o
example, is a p oblem would ha e he goal classi ica ion (p oblem.goal =
Goal.classi ica ion) hen in he ules i would be ep esen ed as a p ed-
ica e goal wi h he pa ame e classi ica ion (goal(’classi ica ion’)).
A special case a e he a ibu es ha a e no nominal, ha is, he a ibu es
ha a e nume ical. In ha case, a pyDa alog unc ion is used, which is a
p edica e wi h a pa ame e ha can ha e a alue (qui e simila o Py hon’s
buil -in dic ). So, o example, he a ibu e numbe o samples wi h a alue
o 500 in pyDa alog will be ep esen ed as he ac numbe [’sample’] =
500. This is use ul because hen he unc ion’s alue can be used any numbe ,
mos impo an ly o be compa ed o o he numbe s, some hing which a no mal
p edica e pa ame e is unable o do. Fo he Da a objec o he p oblem, he
idCode is used as a pa ame e and he alue as he pa ame e . I he alue
should be a numbe , hen he idCode should be ’numbe o ...’.
The solu ion is a bi mo e complex, since i has a much mo e complex
s uc u e. The e is he Algo i hm o be ep esen ed, he Da aMiningS ep and
he s ing wi h o he ele an in o ma ion. The la e is ac ually he easies
o ep esen , as a p edica e named adi ionalIn o wi h one pa ame e , which is
he addi ional in o ma ion ha should be p esen ed o he use . The o he
ecommended s eps a e ep esen ed simila ly o he o he ecommended in o -
ma ion: a p edica e named o he S eps wi h h ee pa ame e s, i s he name,
he second is he solu ion, and he hi d is he he de ails. Then he e is he
Algo i hm which has a name and a lis o hype pa ame e s. So he e a e 2
p edica es o ep esen he Algo i hm. Fi s is ecommenda ion wi h one pa-
ame e which ep esen s he algo i hm name. The second is hype pa ame e
29
4. DEVELOPMENT OF THE RECOMMENDER
which has 4 pa ame e s. The i s pa ame e ep esen s he algo i hm i is
mean o hen he second is he name, he hi d is he alue and he las one
ep esen s he de ails o he hype pa ame e .
In o de o ans o m he P oblem in o pyDa alog ac s, i is a simple need
o s a e he ac s. I is also needed o s o e wha ac s ha e been added so ha
a e he ecommenda ion p ocess is inished, hose ac s need o be e ac ed,
so as o no in e e e wi h he nex eques ed ecommenda ion. In o de o
ans o m he ac s in o a Solu ion, he knowledge bases need o be que ied,
p ocess which will make use o he in e ence engine. a que y in pyDa alog is
simply he call o a p edica e wi h some pa ame e s. I any o he pa ame e s
is a a iable hen i will e u n he alues o he a iable pa ame e s o which
he p edica e is ue. O he wise i will e u n T ue o False, i he ac can
be in e ed om he knowledge bases o no . Fo example, in he que y hy-
pe pa ame e (’Suppo Vec o Machines’,’ke nel’,X,Y), X and
Y a e pa ame e s, so he que y will e u n all (X,Y) pai s o which he p edi-
ca e is ue, ha is, all he ecommended ke nel o he suppo ec o machine
algo i hm. We need ou di e en que ies:
•one o ind all he ecommended algo i hms,
•one o ind all he ecommended hype pa ame e s o each ecommended
algo i hm,
•one o ind all he ecommended o he da a mining s eps
•one o ind all he o he in o ma ion ha migh be ele an
The ecommende pe o ms all hese que ies, and hen ans o ms he ga he ed
da a in a Solu ion objec which is he e u ned o he RuleCon olle .
Finally, he ules need o be w i en in a P olog-like syn ax. An algo i hm
is ecommended i he e is some da a abou he p oblem p esen ed ( he ec-
ommended p edica e wi h ha algo i hm as pa ame e becomes ue i some
ac s ha ep esen da a abou he p oblem a e ue). And in a simila way
all he o he ules need o be de ined.
The in o ma ion ex ac o wo ks mos ly in he same way as he ecom-
mende , simply wi h o he p edica es, ules and que ies. The idea is ha he
in o ma ion ex ac o ecei es a P oblem objec con aining some da a bu also
e u ns a P oblem objec , ha has a modi ied lis o Da a objec s. The new
Da a objec s in he lis ha e a name an idCode and possibleValues, bu
don’ ha e a alue, so hey ac as ques ions o he use .
In he ac base, we ha e all he possible da a ha migh be asked om
he use . The e is he ’ques ion’ p edica e ha has wo pa ame e s, i s is
he idCode o he Da a and he second is he ’name’ o he Da a which
is ac ually he ques ion ha will be p esen ed o he use . Then he e is he
possibleAnswe p edica e ha has wo pa ame e s: he i s is he idCode
30
An IDSS o Machine Lea ning Algo i hms Recommenda ion
o he Da a and he second is one o he possible answe s ha Da a can ha e.
I is impo an o no e ha he idCode used he e as a pa ame e is used in
he ecommende as a p edica e, so when w i ing ules in he ecommende o
ac s o he in o ma ion ex ac o , i is impo an o be awa e o he o he
and coo dina e hem. Possibly in a GUI aided managemen o he knowledge
bases, his s ep would no be needed. Fo ules, he in o ma ion ex ac o s
has one p edica e, askThis, which akes one pa ame e , he idCode o he
Da a.
The in o ma ion ex ac o ’s main ole is o upda e he P oblem wi h
new Da a objec s which s and as ques ions. The p ocess goes as ollows (in
pseudo-code):
Algo i hm 1 In o ma ion Ex ac o
unc ion FindNewQues ions(P oblem gi enP oblem)
ac s ← o malizeToFac s(gi enP oblem)
idCodes ←askThis( ac s) // he ids o he ques ions o be asked
o idCode in idCodes do
name ←name(idCode)
possibleAnswe s ←possibleAnswe s(idCode)
Da a newQues ion ←new Da a(idCode, name, possibleAnswe s)
gi enP oblem.add(newQues ion)
e u n gi enP oblem
Case-Based Reasone The second in elligen me hod we implemen ed is
he case-based easone . In ou implemen a ion he case-based easone wo ks
as ollows: he case base is il e ed depending on some o he da a o he
p oblem ecei ed, so only a subse o he case base emains (called eligible
cases), hen hese cases ecei e a sco e, based on he simila i y be ween hei
p oblem and he gi en p oblem, hen, based on he sco e, a solu ion is c ea ed
om he bes cases ound. Thus he ask was o de ine a il e , a simila i y
unc ion be ween wo p oblems and a me hod o gene a e a Solu ion om he
cases wi h he bes sco e.
In o de o il e he case base, we hough ha he only applicable me hod
is o il e depending on he goal. So only he cases which had a P oblem wi h
he same goal as he gi en p oblem can be eligible cases. Pe haps he e migh
be be e me hods o il e he case base, based on combina ions o a ibu es o
he p oblem, bu he de ini ion o such il e s equi es mo e expe knowledge
han we had a he ime o he implemen a ion o his i s p o o ype.
The bigges challenge is o implemen a good simila i y unc ion. Wha
a ibu es should be gi en a p io i y o e he o he ? Thus, gi en he knowledge
we had access o in his i s phase, we decided ha he da a ype should ha e
he bigges weigh . We decided his because knowledge abou he ask he
machine lea ning p oblem is ying o sol e can, mos o he imes, imp o e he
31
4. DEVELOPMENT OF THE RECOMMENDER
e iciency o he machine lea ning. Then, he second mos impo an ea u es
a e he numbe o samples and he numbe o ea u es, as di e en machine
lea ning algo i hms beha e in di e en ways depending on he p opo ion o
he samples o ea u es, o some algo i hms pe o m eally well only wi h a
small numbe o samples, while o he algo i hms pe o m eally well wi h a big
numbe o samples. Then he o he ea u es ha we hough would weigh a bi
mo e han he es a e he ac i he e a e ime o memo y es ic ions. Tha
is, i he aining ime needs o be sho (e.g. o he cases whe e he e is need
o online aining) and i he algo i hm has a i s disposal wha e e amoun
o memo y, o i he machines he aining is pe o ming on a e less equipped
in e ms o ha dwa e. We ook his in o conside a ion because he e a e some
algo i hms ha pe o m eally well, bu a e big esou ces consume s, so, i
he e a e no many esou ces a ailable, o he algo i hms should be p e e ed.
Then, all he o he ea u es a e scaled he same. The sco e o e e y case goes
om 0 o 1, bu he maximum is 0.5 i he da a ype is di e en .
As he me hod o gene a e he solu ion, we chose simply o ake he Solu-
ion o he case wi h he bigges sco e. I may no be he bes me hod, bu
i is ha d o au oma e he p ocess o inding he bes pa o mul iple solu-
ions, and build ano he one ou o hem. One idea o a u u e wo k would
be o ha e solu ion gene a ion aided by a ule-based sys em, in whe e he e
a e ules abou wha pa o a solu ion is be e in which case. This would
be ha d, because inding such ules is a om i ial. Ano he ideea would
be o ha e a supe ised algo i hm o lea n o gene a e a solu ion om he
i s bes h ee cases. Howe e , his equi es a da ase o da a con aining a
p oblem, h ee no -so-good solu ion and a be e solu ion c ea ed om hose
h ee solu ions. And such a da ase seems ex emely ha d o c ea e. Thus, o
he i s p o o ype, we decided o go o he simple ’bes case’s solu ion’.
Solu ions Combine I is igh ly linked o he o he in elligen me hods,
so we will gi e some de ails abou i in his subsec ion. In o de o combine
he solu ions, we needed o combine he h ee di e en pa s o he solu ions
independen ly: he ecommended algo i hms lis , he ecommended o he da a
mining s eps lis and he o he ele an in o ma ion.
The la e is he easies , as he wo s ings can be conca ena ed and all
he ele an in o ma ion will be a ailable in he inal solu ion. The o he da a
mining s eps lis is, simila ly, qui e easy. The lis s can be simply conca ena ed,
we jus ha e o make su e o emo e he duplica es, which a e he s eps wi h
he same name and alue. We suppose ha ha ing he same name and alue
will ha e mos ly he same in o ma ion in he desc ip ion, so we will jus keep
he one wi h he longe desc ip ion, as we suppose i is he one ha gi es mos
de ails.
Wi h he algo i hms mo e ques ions a ise. We ha e decided o display on
he G aphical Use In e ace a mos h ee ecommended algo i hms. We made
his decision because we hough ha , i we display oo many ecommended
32
An IDSS o Machine Lea ning Algo i hms Recommenda ion
algo i hms o he use , hen his de ea s he pu pose o he sys em i sel , as
i was speci ically designed o help he use choose be ween he my iad o
di e en algo i hms and hype -pa ame e combina ions, so i i ecommends
many algo i hms i is no eally use ul. E en i , in ano he implemen a ion, i
would display all he ecommended algo i hms, in o de o a oid he p e iously
desc ibed p oblem, hese algo i hms would ha e o be o de ed in some way,
o gi en a sco e o some hing simila . So he challenge is o ind a me hod o
ank hese algo i hms.
Fo he Case-Based Reasoning solu ion, we can suppose ha he o de in
which hey a e en e ed by he use o he adminis a o is he o de o bes o
wo s , so hese algo i hm can be conside ed o be al eady so ed. On he o he
hand, he algo i hms ha a e in he ule-based ecommende solu ion a e in a
andom o de . Wi h ou cu en expe knowledge we could no ind any kind
o heu is ic o jus ank some algo i hms and hei hype pa ame e s by hei
e iciency. So, we decided o y an heu is ic ha seemed o be in a o o
he mo e subjec i e pu pose o ou p oposed sys em o gi ing suppo ins ead
o going o e iciency o he ecommended algo i hm: we o de hem by he
amoun o de ails he e a e in he algo i hm. This is because we hink ha
mo e de ails abou an algo i hm he easie will be o he use o unde s and
and o choose, a he han less in o ma ion which migh no help as much.
Also, we so his s ep a e we combine he same algo i hms. This is qui e a
isky s ep, as some imes he e a e wo di e en combina ions o hype pa am-
e e s ha he use should y, bu o he imes his helps o emo e edundan
in o ma ion and complemen in o ma ion om one sou ce wi h he o he . We
decided o go o he combina ion o he algo i hms also because his sys em
was based on he idea o combining he wo di e en ecommenda ion algo-
i hms, so we belie e ha i may esul in be e o mo e exac ecommended
algo i hms.
4.3 Abou Da a
Un il now we ha e alked abou how he p oposed sys em wo ks in his
speci ic implemen a ion and how i mo es a ound and p ocesses da a: asking
new ques ions abou he gi en p oblem, compa ing he p oblem wi h al eady
exis en cases and in e ing ac s abou he solu ion using ac s abou he
p oblem and some ules. All his elies hea ily on he da a ha he sys em
uses: al eady exis en cases in he case base and ules and ac s o he ule-
based ecommende and he in o ma ion ex ac o .
We ha e p e iously men ioned ha his kind o da a is o be en e ed by
esea che s o expe s, as we belie e ha his kind o da a needs expe knowl-
edge in he domains o machine lea ning and da a mining. On he o he hand,
his o he da a can be en e ed by a sys em adminis a o ha ex ac s he
da a om di e en sou ces, such as jou nals, chea shee s o ques ion and an-
swe websi es (as we men ioned in chap e 2). And his ’adminis a o ’ ole
we ook as we implemen ed he sys em, in o de o en e some ini ial da a in
33

4. DEVELOPMENT OF THE RECOMMENDER
he sys em.
The ga he ing o ini ial da a can be qui e ime consuming, so, o an ini ial
es o he sys em we decided o ini ialize he sys em da a only wi h some
basic da a so o es i he unc ionali ies a e wo king p ope ly and i he
ecommende wo ks.
Fo he ule da a, we hough ha da a om chea shee s should be enough
o a i s p o o ype. We ha e ga he ed mos o he da a om he Sciki
Lea n chea shee . Al hough some s aigh - o wa d ules can be ex ac ed
om he chea shee , i is no i ial o ep esen hem in he pyDa alog
logic o maliza ion. The e is need o ex ac wha a e ac ual p oblem da a
ha seems o make he di e ence in wha algo i hm o hype pa ame e o
use. Then we had o ex ac wha a e he ac ual ecommended algo i hms
and hype pa ame e s, as he e ms p esen ed migh be sligh ly di e en han
he ac ual used algo i hms. Fo example, in he Sciki Lea n chea shee hey
men ion he SVC and he SVR algo i hms, while hey a e bo h Suppo Vec o
Machines, bu one used o classi ica ion and he o he o eg ession. So hese
cases ha e o be disco e ed and aken in o accoun , and he di e ence be ween
he SVC and SVR has o be made h ough he hype pa ame e s o he Suppo
Vec o Machine.
In addic ion wi h he ules ex ac ion om he chea shee , we also looked
h ough he de ails o he ecommended algo i hms (Sciki Lea n’s websi e
o e s de ails o e e y algo i hm men ioned in he chea shee ) in o de o
ex ac mo e in o ma ion, maybe e en o he ecommended da a mining s eps
( o examples o some algo i hms i is ecommended o scale he da a i s ) o
o he ele an in o ma ion (such as a lis o non-linea ke nels o he Suppo
Vec o Machines - yes, he e migh be a ecommended ke nel, bu maybe ha
one is no he bes one o he use ’s ask and maybe ha ing he in o ma ion
o o he ke nels will help o ind he be e solu ion).
Fo he case base, i is a bi mo e icky as he e is no eposi o y con aining
da a mining p oblems people ha e had a some poin and hei solu ions. This
in o ma ion can be ound in a icles, machine lea ning compe i ion esul s and
o he such places. One p oblem wi h ex ac ing cases om a icles is ha he
p oblems in he a icles a e e y speci ic, so he e would be need o many
such speci ic cases o co e a mo e gene al kind o p oblem. Finding enough
a icles on a speci ic opic, ha ea di e en aspec s o ha opic and hi e
di e en solu ions is no i ial.
Fo he p o o ype i s we decided o only sea ch cases only o some ypes
o p oblems. We decided o go wi h classi ica ion p oblems, as hey a e one
o he mo e nume ous ype o p oblem and he e is qui e some esea ch done
in classi ica ion p oblems o many kinds. S ill, o he i s es s we decided
o ex ac cases om he s udy o [5] whe e he e is a e iew o mul iple
classi ica ion algo i hms. We decided o do so because in his s udy he e a e
34
An IDSS o Machine Lea ning Algo i hms Recommenda ion
se e al ypes o classi ica ion algo i hms used and he da ase s used consis o
eal da a and a e o a ious ypes (images, biological, mul i a ia e, e c.), and
e e y classi ica ion algo i hm was qui e ho oughly es ed wi h many di e en
hype pa ame e combina ions. Thus, we could ex ac a case as ollows: he
p oblem had he de ails o he da ase gi en and he de ails o he expe imen s
(e.g. using wo classes), and he solu ion was c ea ed combining he de ails o
he bes wo o h ee algo i hm-hype pa ame e combina ions.
35
4. DEVELOPMENT OF THE RECOMMENDER
36
An IDSS o Machine Lea ning Algo i hms Recommenda ion
5 Expe imen al E alua ion
The e is no simple objec i e way o e alua e his sys em. This sys em
was design o help di e en kind o non-expe s o choose a machine lea ning
algo i hm. I was no designed o gi e exac ecommenda ions, o o gi e he
bes algo i hm. The sys em is be e i i can help a ge use o mo e e icien ly
sol e hei da a-mining ask. In o de o p ope ly e alua e he sys em, an
expe imen simila o an usabili y es would need o be pe o med.
Some po en ial use s would ha e o ag ee o pa ake in o an expe imen
ega ding his new sys em. Then hese po en ial use s will be p o ided wi h he
e sion o he sys em ha we wan o es . I is impo an ha he use s would
do hei no mal da a-mining asks as usual. They will ha e o be moni o ed
in some way while using he sys em, maybe by comple ing di e en su eys
while and a e hey inished hei da a-mining ask, and he compa ison o
hei ound solu ion and he sys em ecommended solu ion. A e a pe iod, he
da a is ga he ed and analyzed. Only hen we could eally assess he e alua ion
o he sys em, as i is c ucial o know how he sys em helped he use s achie e
hei goal. We we e no able o execu e such an expe imen because o lack o
use s eady o ake he es .
A mo e in ui i e app oach would be o ha e some cases ha a e no in he
case base and o en e he p oblem in o he sys em hen compa e he ou pu
o he exis ing bes ou pu . This may look simila o usual es ing, bu he
ac is ha his is no a case o ma ching he ou pu wi h he g ound u h,
as i usually is. The ac ual bes solu ion o he p oblem may be e y speci ic
and e en unique. The ou pu o he sys em is mean o guide so ha he bes
solu ion is ound as e . Thus, he compa ison o he eal solu ion wi h he
ecommenda ion needs o be done manually and o assess how good would be
he ecommenda ion in o de o ind he eal solu ion. An au oma ed sys em
could be pu in place, maybe o sco e he simila i y o he wo, bu i would be
a bes an app oxima ion o he pe o mance o he sys em. Also, when mea-
su ing he pe o mance o he sys em, i is needed o ake in o accoun wha is
he cu en s a e o he knowledge bases and he case base. Fo example, i he
case base con ains only cases abou p edic ion and classi ica ion o images, and
he same he knowledge bases, hen i would no make sense o y o asses he
sys em’s pe o mance wi h cases wi h a di e en scope (such as dimensionali y
educ ion o ex ) as i will ob iously gi e bad ecommenda ions.
In he case o ou i s p o o ype, he ule base con ains jus some gene al
ules abou gene al da a mining asks and he case base con ains only bina y
classi ica ion cases. We decided o lea e one o he cases ou o manual es ing.
We en e ed he p oblem as he inpu , and ecei ed he ecommenda ion. The
i s ecommended algo i hm was simila o he bes solu ion (AdaBoos wi h
Decision T ees) and he second ecommended algo i hm was simila o he
hi d bes algo i hm (SVM wi h linea ke nel). We decided ha i was a ai ly
good esul o con inue wi h he p ojec , and u he es ing will be done when
he e will be mo e cases and ules added o he case base and knowledge bases.
37
7. TEMPORAL PLANIFICATION
is supposed o be qui e ligh weigh , he e o e es ing i should no ake oo
much ime. We planned o inish he implemen a ion and es ing be o e 1s o
Janua y, bu i ook sligh ly longe han expec ed, so we only inished on he
4 h o Janua y.
Finally, we planned o inish he mas e hesis epo un il he 20 h o
Janua y, which was he o icial deadline, and hen o p epa e o he o al
p esen a ion un il he end o Janua y, when he o al de ense o he hesis akes
place.
44

An IDSS o Machine Lea ning Algo i hms Recommenda ion
8 Conclusions and Fu u e Wo k
In his wo k we ha e p esen ed he concep o an In elligen Decision Suppo
Sys em o Machine Lea ning Algo i hm Recommenda ion and p oposed a
p o o ype o such a sys em. In ou expe ience, he e is a lack o ools o
non-expe s o help hem choose be ween he many me hods a ailable o sol e
machine lea ning, and in pa icula da a mining, asks. Thus we had he idea
o design a sys em ha would help wi h hese p ocesses. We ha e p esen ed
his design, which is cen e ed a ound he combined use o he Case-Based
Reasoning and Rule-Based Reasoning, o he ecommending p ocess, while
also ying o make he sys em easy o use and manage, o example, by using
an in o ma ion ex ac o ; while insis ing on he ac ha he sys em is no
designed o gi e he bes possible algo i hm o solu ion, bu o help inding
i . We ha e p esen ed a p o o ype o such a sys em, and he implemen a ion
de ails o he wo ecommende algo i hms. The p elimina y es ing o he
p o o ype has shown i o be a p omising ool.
Fu u e Wo k. The mos ob ious u u e wo k would be he con inua ion o
he de elopmen o he sys em om he exis ing p o o ype, o a eal usable
se ice. We ha e also men ioned h oughou ou wo k some o he many
imp o emen s ha can be made o he sys em. Fi s he G aphical Use
In e ace would need o be g ea ly imp o ed, as he one in he p o o ype is
qui e simpleS. I needs o become mo e use iendly, and he e a e wo majo
unc ionali ies o be added o i : a be e (o maybe an ac ual) knowledge bases
manage , ha wi h which one can add and manage ules and ac s h ough
he GUI and no h ough a ex edi o ; and imp o e he manage o he case
base, because as he e will be an inc easing numbe o cases, a simple lis is
ine icien o managing, so unc ionali ies such as il e s, sea ch and maybe
agging o cases need o be added, as well as as e case base o malisms.
Beyond he simple imp o emen o he p o o ype, he e is plen y o space
o imp o emen o he in elligen me hods. The case-based easone migh be
changed o be an adap i e case-based easone , so ha a pa o he manage-
men o he case base is done au oma ically. Also, he case-based easone is
using a e y simple me hod o adap a ion o he solu ion. I only uses he
solu ion o he case wi h he bes sco e. The e migh be a good u u e wo k
in esea ching how can se e al bes solu ions be combined in o de o ge a
be e solu ion. Some ideas we e using a lea ning algo i hm (such as neu al
ne wo ks) using a ule sys em o maybe o use online lea ning ( o adap he
combina ion algo i hm e e y ime a use in oduces a new case by compa ing
he gi en case and he ecommended one). The combina ion o he ule-based
ecommende solu ion and he case-based easone solu ion migh also need
imp o emen , o example, o make an algo i hm o decide when o combine
algo i hms so ha hei s espec i e de ails a e added and he ecommenda ion
is be e , and when o keep hem sepa a ed, so ha hey ep esen di e en
algo i hm-hype pa ame e s combina ions o he same algo i hm.
45
8. CONCLUSIONS AND FUTURE WORK
46
Re e ences
[1] Wha a e he mos impo an machine lea ning algo i hms?, Quo a web-
si e, (2010). Websi e Link.
[2] How can i go abou applying machine lea ning algo i hms o s ock ma -
ke s?, S ack exchange websi e, (2011). Websi e Link.
[3] D. W. Aha, C. Ma ling, and I. Wa son,Case-based easoning com-
men a ies: in oduc ion, The Knowledge Enginee ing Re iew, 20 (2005),
pp. 201–202. Full Tex Link.
[4] D. B idge, M. H. Goke , L. McGin y, and B. Smy h,Case-based
ecommende sys ems, The Knowledge Enginee ing Re iew, 20 (2006),
pp. 315–320.
[5] R. Ca uana and A. Niculescu-Mizil,An empi ical compa ison o
supe ised lea ning algo i hms, ICML ’06 P oceedings o he 23 d in e -
na ional con e ence on Machine lea ning, (2006), pp. 161–168. Pd Link.
[6] W. Chee ham and I. Wa son,Fielded applica ions o case-based ea-
soning, The Knowledge Enginee ing Re iew, 20 (2006), pp. 321–323. Pd
Link.
[7] P. Desmond,Measu ing he ull en i onmen al impac o you da a cen-
e , Schneide Elec ic Blog, (2013). Websi e Link.
[8] P. Domingos,A ew use ul hings o know abou machine lea ning, Com-
munica ions o he ACM, 55 (2012), pp. 78–87. Pd Link.
[9] M. Feu e , A. Klein, K. Eggenspe ge , J. T. Sp ingenbe g,
M. Blum, and F. Hu e ,E icien and obus au oma ed machine
lea ning, NIPS’15 P oceedings o he 28 h In e na ional Con e ence on
Neu al In o ma ion P ocessing Sys ems, (2015), pp. 2755–2763. Pd Link.
[10] C. Gi aud-Ca ie ,The da a mining ad iso : me a-lea ning a he
se ice o p ac i ione s, P oceedings o he Fou h In e na ional Con e -
ence on Machine Lea ning and Applica ions, (2006), pp. 113–119. Pd
Link.
[11] C. Gi aud-Ca ie ,Me alea ning - a u o ial, (2008). Pd Link.
47
REFERENCES
[12] J. M. He nansaez, J. A. Bo , and A. F. Ska me a,Me ala: a
j2ee echnology based amewo k o web mining, Re is a Colombiana de
Compu acion, 5 (2004). Pd Link.
[13] A. Hol , I. Bichinda i z, R. Schmid , and P. Pe ne ,Medical
applica ions in case-based easoning, The Knowledge Enginee ing Re iew,
20 (2005), pp. 289–292. .
[14] E. B. U. Islam,Compa ison o con en ional and mode n load o ecas -
ing echniques based on a i icial in elligence and expe sys ems, IJCSI
In e na ional Jou nal o Compu e Science Issues, 8 (2011), pp. 504–513.
Pd Link.
[15] J. Joseph, O. Sha i , A. Kuma , S. Gadka i, and A. Mohan,
Using big da a o machine lea ning analy ics in manu ac u ing, (2014).
Pd Link.
[16] A. Kaklauskas,Biome ic and in elligen decision making suppo , In-
elligen Sys ems Re e ence Lib a y, 81 (2015). .
[17] M. A. Khan,Planning o and moni o ing o p ojec sus ainabili y: A
guideline on concep s, issues and ools, p oduced unde he UNDP sup-
po ed esul s-based Moni o ing and E alua ion ac i i y a he Moni o -
ing and P og ess Re iew Di ision o he Minis y o Plan Implemen a ion,
(2000). Websi e Link.
[18] A. L. Kidd,Knowledge acquisi ion o expe sys ems: A p ac ical hand-
book, (1987). Google Books link.
[19] J. U. Kie z, F. Se ban, A. Be ns ein, and S. Fische ,Da a min-
ing wo k low empla es o in elligen disco e y assis ance in apidmine ,
P oceedings o RCOMM’10, (2010), pp. 19 – 26. Pd Link.
[20] K. Langbo g-Hansen,The en i onmen al impac o unning so wa e,
Schneide Elec ic Blog, (2013). Websi e Link.
[21] G. Luo,A e iew o au oma ic selec ion me hods o machine lea ning
algo i hms and hype -pa ame e alues, (2016). Pd Link.
[22] M. Lu z,Lea ning py hon, (2008). Pd Link.
[23] J. Qiu, Q. Wu, G. Ding, Y. Xu, and S. Feng,Using big da a
o machine lea ning analy ics in manu ac u ing, EURASIP Jou nal on
Ad ances in Signal P ocessing, (2014). Pd Link.
[24] L. Rendell, R. Seshu, and D. Tcheng,Laye ed concep -lea ning and
dynamically- a iable bias managemen , P oceedings o he 10 h in e na-
ional join con e ence on A i icial in elligence, 1 (1987), pp. 308–314.
Pd Link.
48
An IDSS o Machine Lea ning Algo i hms Recommenda ion
[25] M. M. Rich e and A. Aamod ,Case-based easoning ounda ions,
The Knowledge Enginee ing Re iew, 20 (2005), pp. 203–207. Pd Link.
[26] T. Ro h–Be gho e and I. Iglezakis,Six s eps in case–based ea-
soning: Towa ds a main enance me hodology o case–based easoning
sys ems, P o essionelles Wissens managemen : E ah ungen und Visio-
nen (P oceedings o he 9 h Ge man Wo kshop on Case-Based Reasoning
(GWCBR)), (2001), pp. 198–208. Pd Link.
[27] S. Sahin, M. Tolun, and R. Hassanpou ,Hyb id expe sys ems:
A su ey o cu en app oaches and applica ions, Expe Sys ems wi h
Applica ions, 39 (2012), pp. 4609–4617. Pd Link.
[28] F. Se ban, J. Vanscho en, J.-U. Kie z, and A. Be ns ein,A
su ey o in elligen assis an s o da a analysis, ACM Compu ing Su eys
(CSUR), 45 (2013), p. 31. Pd Link.
[29] R. S iz,Me alea ning o da a mining and kdd, (2012). Pd Link.
[30] C. Tho n on, F. Hu e , H. H. Hoos, and K. Ley on-B own,
Au o-weka: Combined selec ion and hype pa ame e op imiza ion o clas-
si ica ion algo i hms, KDD ’13 P oceedings o he 19 h ACM SIGKDD
in e na ional con e ence on Knowledge disco e y and da a mining, (2013),
pp. 847–855. Pd Link.
[31] C. C. Ven e s, C. Jay, L. M. S. Lau, M. K. G i i hs,
V. Holmes, R. R. Wa d, J. Aus in, C. E. Dibsdale, and J. Xu,
So wa e sus ainabili y: The mode n owe o babel, CEUR Wo kshop P o-
ceedings. RE4SuSy: Thi d In e na ional Wo kshop on Requi emen s En-
ginee ing o Sus ainable Sys ems, (2014), pp. 7 – 12. Pd Link.
[32] D. H. Wolpe and W. G. Mac eady,No ee lunch heo ems o
op imiza ion, IEEE ansac ions on e olu iona y compu a ion, 1 (1997),
pp. 67–82. Pd Link.
[33] T. Yu, T. Jan, S. Simo , and J. Debenham,Inco po a ing p io
domain knowledge in o induc i e machine lea ning, (2007). Pd Link.
[34] R. Zucke , J.-U. Kie z, and A. Vadu a,Mining ma : Me ada a-
d i en p ep ocessing, (2001). Pd Link.
49