Sch i en eihe CIplus, Band 3/2015
He ausgebe : T. Ba z-Beiels ein, W. Konen, H. S enzel, B. Naujoks
Lea ning Model-Ensemble
Policies wi h Gene ic
P og amming
Oli e Flasch, Ma ina F iese, Ma in Zae e e ,
Thomas Ba z-Beiels ein, J¨u gen B anke
Lea ning Model-Ensemble Policies wi h Gene ic
P og amming
Oli e Flasch1, Ma ina F iese1, Ma in Zae e e 1,
Thomas Ba z-Beiels ein1, and J¨u gen B anke2
1Facul y o Compu e and Enginee ing Sciences
Cologne Uni e si y o Applied Sciences, 51643 Gumme sbach, Ge many
[email p o ec ed]
2Wa wick Business School, Uni e si y o Wa wick
Co en y, CV4 7AL, UK
[email p o ec ed]
Abs ac . We p opose o apply yped Gene ic P og amming (GP) o
he p oblem o inding su oga e-model ensembles o global op imiza ion
on compu e-in ensi e a ge unc ions. In a model ensemble, base-models
such as linea models, andom o es models, o K iging models, as well
as p e- and pos -p ocessing me hods, a e combined. In heo y, an op i-
mal ensemble will join he s eng hs o i s comp ising base-models while
a oiding hei weaknesses, o e ing highe p edic ion accu acy and o-
bus ness. This s udy de ines a g amma o model ensemble exp essions
and sea ches he se o op imal ensembles ia GP. We pe o med an
ex ensi e expe imen al s udy based on 10 di e en objec i e unc ions
and 2 se s o base-models. We a i e a p omising esul s, as on unseen
es da a, ou ensembles pe o m no signi ican ly wo se han he bes
base-model.
Keywo ds: Ensemble Me hods, Gene ic P og amming, Su oga e-Model-Based
Op imiza ion
1 In oduc ion
The applica ion o su oga e-model based me hods o he pu pose o sol ing
cos ex ensi e and ime consuming op imiza ion p oblems is an es ablished ech-
nique. Su oga e models, as employed in me hods like E icien Global Op imiza-
ion (EGO) [19] o Sequen ial Pa ame e Op imiza ion (SPO) [1] o e se e al
ad an ages. While hei main goal is o en o educe he numbe o cos ly a -
ge unc ion e alua ions, one can also bene i om he in o ma ion su oga e
models p o ide on he p oblem. This can include in o ma ion on he shape o
he op imized landscape, in e ac ions be ween pa ame e s o hei indi idual
impo ance.
A ques ion ha o en a ises wi h such app oaches is he choice o he model-
ing echnique (e.g., ee-based, Suppo Vec o Machines o K iging). This pape
2 F iese e al.
will use ”base-models” o e e o such di e en model ypes. Depending on he
ask a hand, a ce ain indi idual base-model migh be mos p omising. In many
cases howe e i may be unclea which o se e al candida es is bes sui ed. Fo
ins ance, one base-model migh p o ide be e global app oxima ion han an-
o he , while ano he base-model can handle discon inuous egions o he design
space wi h mo e ease. Thus i is o in e es o de elop me hods wi h which o
choose om o combine subs an ially di e en base-models.
Rele an c oss e e ences o simila p oblems as well as p e ious s udies on
his issue a e e iewed in Sec. 2.
In his e y i s s ep owa ds lea ning a meaning ul ensemble policy, a Ge-
ne ic P og amming (GP) sys em is used o combine he di e en models wi h
e y simple ope a o s. The me hods o his end a e in oduced in Sec. 3. We limi
ou sel es o he modeling s age, due o he complexi y o he p oblem. Employ-
ing gene a ed ensemble policies in an ac ual op imiza ion amewo k will emain
o a de ailed in es iga ion in ollowing publica ions.
Thus, ou app oach is es ed o i s success in modeling simple nume ical es
unc ions. The p oblem se up, including es unc ions and employed modeling
echniques, a e desc ibed in Sec. 4. Expe imen esul s a e gi en in Sec. 5 and
analyzed in Sec. 6. This ex closes wi h a summa y and ou look in Sec. 7.
2 P e ious Resea ch
Ensembles a e no a new opic, nei he in modeling no in su oga e model
suppo ed op imiza ion.
As he cos ly op imiza ion p oblems a hand wa an only e y spa se sam-
pling o da a, some classical ensemble app oaches like boo s ap agg ega ing
(bagging) [2] o boos ing [20, 28] a e no well applicable. Also, bagging is o en,
al hough no always [22, 6], employed o build ensembles o homogeneous base-
models, i.e. models o he same ype (e.g., only ee o only linea ). In a simila
way boos ing has been used o combine homogeneous base-models o a s onge
se o models, o ins ance as applied in e olu iona y op imiza ion by Holeˇna e
al. [17].
Wi h a di e en concep , Ong e al. [27] combined homogeneous base-models
by building hem locally a ound indi iduals o an e olu iona y algo i hm’s pop-
ula ion, hus building di e en models in di e en egions o he design space.
Ano he app oach o spli ing he design space is ha o G amacy e al. [15].
The e, ees ha spli he design space a e lea ned, applying Gaussian p ocess
o linea models in he a ious di isions.
Two di e en ensemble policies ha e been es ed by Lim e al. [26] o he
pu pose o enhancing a Su oga e Assis ed Meme ic Algo i hm. They combined
base-models ei he by agg ega ion o by selec ing he bes solu ion om each
model o e alua ion on he eal a ge unc ion.
Focusing on a he e ogeneous se o base-models, Go issen e al. [14] combine
models using a gene ic algo i hm. In hei implemen a ion, ensembles occu when
Lea ning Model-Ensemble Policies wi h Gene ic P og amming 3
wo model ypes a e selec ed o ecombina ion. The ensembles c ea ed a e sim-
ply an a e aged combina ion o he models.
Se e al al e na i e app oaches we e ecen ly in es iga ed by F iese e al. [12].
They g ouped app oaches in o wo ca ego ies, single-e alua ion and mul iple-
e alua ion. Single-e alua ion app oaches iew he p oblem o choosing a model
om he ensemble as a mul i a med bandi p oblem [13]. Tha means, only in o -
ma ion om one model is chosen in each sequen ial op imiza ion s ep. Mul iple-
e alua ion app oaches ain e e y model in each s ep, hus using in o ma ion
om all o hem.
The single-e alua ion app oach is also aken by Hess e al. [16], who in oduce
hei algo i hm e med PROGRESS. PROGRESS is an enhanced algo i hm o
sol ing mul i a med bandi p oblems, speci ically adap ed o he p oblem o
selec ing a su oga e model o op imiza ion.
The e a e u he app oaches, including some no solely in ended o being
employed in su oga e model op imiza ion.
One example is he app oach o F ayman e al. [10]. F ayman [10] e . al use a
neu al ne wo k (Mul i Laye Pe cep on MLP) o ain an ensemble combining
linea , MLP, logis ic eg ession and k-nea es neighbo models. This app oach
can be classi ied as a a ia ion o s acked gene aliza ion, o model s acking [29,
3].
3 Me hods
This sec ion desc ibes he a ious me hods used in his wo k, including model
ensembles and model combina ion policies. I is shown how model combina ion
policies a e e alua ed and how hey a e e ol ed by GP.
3.1 Model Ensembles
I s ill emains unclea how an ideal policy would combine in o ma ion om
undamen ally di e en base-models in a meaning ul way. While heo e ical con-
side a ions o ideas om o he ields (i.e. mul i a med bandi p oblems) do yield
Ensemble-Policies ha wo k o some ex end, we would like o sugges a way o
lea n such policies in a less cons ic ed en i onmen .
The idea p esen ed he e a emp s o do so wi h a GP app oach. Tha is, he
di e en models a e iewed as pa ame e s in a symbolic eg ession p oblem.
Wi h he e ms o F iese e al. we will he ein a emp o lea n a policy o
a mul iple-e alua ion app oach [12] Mul iple-e alua ion app oaches ain e e y
model in each s ep, combining o selec ing om he in o ma ion yielded. As
single e alua ion app oaches need se e al op imiza ion s eps o lea n he igh
beha io , hey may o en be in easible o he cos ly p oblems o conce n in
su oga e model based op imiza ion.
In his pape we call he se o a ailable base-models M. A speci ic base-
model will be e e ed o as mi, whe e i ep esen s he index o he model ype
(e.g., 1 is a linea model, 2 is a K iging model, e c.). Du ing expe imen s, each
4 F iese e al.
miwill be ained on a ixed aining da a se and yield a ec o o esponses yi
o a second, independen alida ion da a se .
A policy combining se e al miwill be e e ed o as πk. He e, k ep esen s a
coun e , as se e al policies can be lea ned, o ins ance by es a ing a GP un
wi h a di e en seed.
In he ollowing, he models mi, as well as policies πka e desc ibed as eal-
alued unc ions de ined in d-dimensional bounded pa ame e space D⊂Rd.
The dimensionali y d, as well as he bounds, o egion o in e es , o Da e
dependen on he objec i e unc ion.
xd xd< xd≥
mean
LM
x
mean
*
RF
x
1.88
decision
x2−0.59 −1.75 0.03
Fig. 1. Example o a GP-gene a ed policy shown as an exp ession ee.
Fig. 1 shows he exp ession ee o he ollowing GP-e ol ed example policy:
mean[LM(x),mean[RF(x)×1.88,decision(x,2,−0.59,−1.75,0.03)]]
This policy combines a linea model (LM) wi h a andom o es model (RF) by
calcula ing he a i hme ic mean o he model ou pu s. The RF ou pu is u he
pos -p ocessed by scaling and mixing (also ia a i hme ic mean) wi h he ou pu
o a simple GP-e ol ed decision ee. As we a e only using 2D es unc ions in
his s udy, base-models and model ensembles can be isualized as su ace plo s.
Lea ning Model-Ensemble Policies wi h Gene ic P og amming 5
In hese plo s, he i s dimension o he bounded pa ame e space Dis assigned
o he X (wid h) axis, he second dimension o D o he Y (dep h) axis and he
model ou pu o he Z (heigh ) axis.
As an example, we applied he ensemble policy o Fig. 1 o he 2D Weie s ass
unc ion wi h he bounds (domain) shown in Tab. 1. Fig. 2 p o ides a su ace
plo o his es unc ion. As ou example policy e e s o he LM and RF base-
models, bo h models ha e o be i ed o aining da a. Su ace plo s o hese
base-model i s a e shown in Fig. 3 and Fig. 4, espec i ely. Based on hese i s,
he ou pu o ou example policy is shown as a su ace plo in Fig. 5.
Fig. 2. Su ace plo o he 2D Weie s ass es unc ion.
6 F iese e al.
Fig. 3. Su ace plo o a linea model (LM) i o he 2D Weie s ass es unc ion.
Lea ning Model-Ensemble Policies wi h Gene ic P og amming 7
Fig. 4. Su ace plo o a andom o es (RF) model i o he 2D Weie s ass es
unc ion.
8 F iese e al.
Fig. 5. Su ace plo o an ensemble i o he 2D Weie s ass es unc ion based on he
policy shown in Fig. 1. The base-model i s used in he ensemble a e shown in Fig. 3
and Fig. 4.
Lea ning Model-Ensemble Policies wi h Gene ic P og amming 15
se : small
se : ull
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
1
1
10
10
0.1
1
1e−05
1e−01
1e+03
1e+07
10
1
10
100
1000
10000
1e−12
1e−08
1e−04
1e+00
: weie s ass
: g iewangk
: b anin
: mex_ha
: ackley
: discus
: as igin
: ko anchek
: osenb ock
: sphe e
MARS
RF
LM
EnsVal
EnsTs
MARS
RF
LM
EnsVal
EnsTs
SVM
KF
MLP
base models and model ensembles
SRMSE
Fig. 7. Boxplo o expe imen al esul s. EnsVal a e he model ensembles on alida ion
da a, i.e. hei i ness as seen by he GP sys em. EnsTs a e he model ensembles on
alida ion da a. Each box is based on 48 ∗47 = 2256 alues. Only he EnsVal boxes
a e based on jus 48 alues, which a e he i ness alues as seen by he GP sys em.
16 F iese e al.
a single excep ion) pe o m no be e han each bes base-model on es da a.
While he base-models mos ly a oid o e - i ing, he oo well adap ed s uc u e
o a policy can ein oduce ha p oblem. Enabling he complexi y c i e ion, o
c oss alida ing he i ness alues migh be possible ways o deal wi h ha issue.
Howe e , since complexi y is al eady limi ed due o he ee dep h limi i would
be mo e p omising o look a be e alida ion o i ness alues.
S ill, e en i o e - i ing is a oided app op ia ely, i migh occu ha ensem-
bles will be unable o ou pe o m he bes base-model. The esul s on Discus
and Sphe e unc ion a e good examples o ha . The e, i is unlikely ha he
LM base-model can be ou pe o med by a he e ogeneous ensemble policy.
7 Summa y and Ou look
Fo he pu pose o op imizing cos ly global op imiza ion p oblems, i is o in e -
es how o combine a se o he e ogeneous base-models, which a e hen exploi ed
in a su oga e model op imiza ion amewo k. The goal o his pape was o make
a i s s ep owa ds lea ning ensemble policies o su oga e based op imiza ion,
using gene ic p og aming. Tha i s s ep consis ed in es ing he necessa y me h-
ods o build such an ensemble policy. The e o in app oxima ing nume ical es
unc ions was used as a quali y indica o .
The necessa y g amma o model ensemble exp essions was de ined, and i
was shown ha he sugges ed amewo k is wo king. Howe e , ou app oach only
p oduced a signi ican imp o emen in one o wen y cases. The main p oblem,
which occu s ega dless o he limi ed complexi ies (i.e. he ee dep h limi ),
was o e - i ing. I can also be no iced ha se e al o he employed nume ical
es unc ions a e bes sol ed wi h a s a egy ha simply selec s om he base
models ins ead o lea ning any mo e complex policies.
The e o e, we plan o i s imp o e he app oach, in an a emp o a oid he
occu ing o e i ing. Since complexi y is al eady somewha limi ed, we would
sugges o c oss alida e i ness alues o each ensemble policy du ing he GP
un. I is also o in e es o ex end he p ocess sugges ed he e. Tha includes
imp o ing he used se o GP ope a o s, as well as o conside using u he
in o ma ion p o ided by he base models. Fo ins ance, K iging yields an e o
es ima e. In eg a ing such an e o es ima e easonably would be bene icial, as
i is o en success ully used in su oga e model based op imiza ion o balance
explo a ion and exploi a ion. In connec ion wi h his, a p ocess o iden i ying
ea u es o he op imized landscape could be inco po a ed. This would allow
o ac ually lea n policies which a e able o wo k on mo e han jus one a ge
unc ion.
Besides, due o he limi a ions o he employed es unc ions, using eal
wo ld p oblems would be signi ican ly mo e in e es ing. No only would hey
indica e beha io mo e ele an o p ac ical applica ions. Real wo ld p oblems
migh also yield landscapes which a e mo e in e es ing o be app oxima ed by
an ensemble.
Lea ning Model-Ensemble Policies wi h Gene ic P og amming 17
In addi ion, he e o measu e employed should be u he in es iga ed. I
is no clea whe he i is he bes choice when su oga e model op imiza ion is
he applica ion. Al e na i ely, some kind o pe mu a ion e o migh be mo e
sui ed.
In he long un o cou se he model policies ound should be es ed o hei
ac ual pu pose, which is su oga e model op imiza ion. The e o e, a second s ep
could in ol e es ing such policies as su oga es in an op imiza ion amewo k
like SPO. O else, he ac ual op imiza ion pe o mance i sel could be used as a
i ness measu e in he GP sys em. Tha , howe e , would aise he compu a ional
e o conside ably.
Finally, we conside o es his app oach o a a he di e en applica ion,
which is ensembles o ime se ies p edic ion algo i hms.
Acknowledgmen s This wo k has been kindly suppo ed by he Fede al Min-
is y o Educa ion and Resea ch (BMBF) unde he g an s MCIOP (FKZ 17N0311)
and CIMO (FKZ 17002X11).
Re e ences
1. T. Ba z-Beiels ein, K. E. Pa sopoulos, and M. N. V aha is. Design and analy-
sis o op imiza ion algo i hms using compu a ional s a is ics. Applied Nume ical
Analysis and Compu a ional Ma hema ics (ANACM), 1(2):413–433, 2004.
2. L. B eiman. Bagging p edic o s. Machine Lea ning, 24(2):123–140, 1996.
3. L. B eiman. S acked eg essions. Machine Lea ning, 24:49–64, 1996.
4. L. B eiman. Random o es s. Machine Lea ning, 45(1):5 –32, 2001.
5. C.-C. Chang and C.-J. Lin. LIBSVM: A lib a y o suppo ec o machines. ACM
T ansac ions on In elligen Sys ems and Technology, 2:27:1–27:27, 2011.
6. A. L. V. Coelho and D. S. C. Nascimen o. Le e s: On he e olu iona y design o
he e ogeneous bagging models. Neu ocompu ., 73(16-18):3319–3322, Oc . 2010.
7. K. Deb, A. P a ap, S. Aga wal, and T. Meya i an. A as eli is mul i-objec i e ge-
ne ic algo i hm: Nsga-ii. IEEE T ansac ions on E olu iona y Compu a ion, 6:182–
197, 2000.
8. O. Flasch, O. Me smann, and T. Ba z-Beiels ein. RGP: An open sou ce gene ic
p og amming sys em o he R en i onmen . In M. Pelikan and J. B anke, edi o s,
Gene ic and E olu iona y Compu a ion Con e ence, GECCO 2010, P oceedings,
Po land, O egon, pages 2071–2072. ACM, 2010.
9. A. Fo es e , A. Sobes e , and A. Keane. Enginee ing Design ia Su oga e Mod-
elling. Wiley, 2008.
10. Y. F ayman, B. F. Rol e, and G. I. Webb. Sol ing eg ession p oblems using com-
pe i i e ensemble models. In P oceedings o he 15 h Aus alian Join Con e ence
on A i icial In elligence: Ad ances in A i icial In elligence, AI ’02, pages 511–522,
London, UK, UK, 2002. Sp inge -Ve lag.
11. J. H. F iedman. Mul i a ia e adap i e eg ession splines. Ann. S a ., 19(1):1–141,
1991.
12. M. F iese, M. Zae e e , T. Ba z-Beiels ein, O. Flasch, P. Koch, W. Konen, and
B. Naujoks. Ensemble based op imiza ion and uning algo i hms. In F. Ho mann
and E. H¨ulle meie , edi o s, P oceedings 21. Wo kshop Compu a ional In elligence,
pages 119–134. Uni e si ¨a s e lag Ka ls uhe, 2011.
18 F iese e al.
13. J. Gi ins. Bandi p ocesses and dynamic alloca ion indices. J. R. S a is . Soc. B,
41:148–164, 1979.
14. D. Go issen, T. Dhaene, and F. D. Tu ck. E olu iona y model ype selec ion o
global su oga e modeling. J. Mach. Lea n. Res., 10:2039–2078, Dec. 2009.
15. R. B. G amacy and H. K. H. Lee. Bayesian eed Gaussian p ocess models. Tech-
nical epo , Dep . o Applied Ma h & S a is ics, Uni e si y o Cali o nia, San a
C uz, 2006.
16. S. Hess, T. Wagne , and B. Bischl. P og ess: P og essi e ein o cemen -lea ning-
based su oga e selec ion. In P oceedings o Se en h Lea ning and In elligen Op-
imiza ioN Con e ence LION 2013, 2013.
17. M. Holeˇna, D. Linke, and N. S ein eld . Boos ed neu al ne wo ks in e olu iona y
compu a ion. In P oceedings o he 16 h In e na ional Con e ence on Neu al In o -
ma ion P ocessing: Pa II, ICONIP ’09, pages 131–140, Be lin, Heidelbe g, 2009.
Sp inge -Ve lag.
18. In o ma ion echnology – Syn ac ic me alanguage – Ex ended BNF, 1996.
19. D. Jones, M. Schonlau, and W. Welch. E icien global op imiza ion o expensi e
black-box unc ions. Jou nal o Global Op imiza ion, 13:455–492, 1998.
20. M. Kea ns. Though s on hypo hesis boos ing. Unpublished manusc ip , 1988.
21. M. Keijze . Scaled symbolic eg ession. Gene ic P og amming and E ol able Ma-
chines, 5(3):259–269, sep 2004.
22. S. Ko sian is, D. Kanellopoulos, and I. Zaha akis. Bagged a e aging o eg ession
models. In I. Maglogiannis, K. Ka pouzis, and M. B ame , edi o s, A i icial In el-
ligence Applica ions and Inno a ions, olume 204 o IFIP In e na ional Fede a ion
o In o ma ion P ocessing, pages 53–60. Sp inge US, 2006.
23. J. Koza. Gene ic P og amming: On he P og amming o Compu e s by Means o
Na u al Selec ion. MIT P ess, Camb idge MA, 1992.
24. B. Lang. Mono onic mul i-laye pe cep on ne wo ks as uni e sal app oxima o s.
In W. Duch, J. Kacp zyk, E. Oja, and S. Zad ozny, edi o s, A i icial Neu al Ne -
wo ks: Fo mal Models and Thei Applica ions ? ICANN 2005, olume 3697 o Lec-
u e No es in Compu e Science, pages 31–37. Sp inge Be lin Heidelbe g, 2005.
25. R. V. Len h. Response-su ace me hods in R using sm (upda ed o e sion 1.40).
Technical epo , The Uni e si y o Iowa, 2010.
26. D. Lim, Y. soon Ong, B. Sendho , and Y. Jin. A s udy on me amodeling ech-
niques, ensembles, and mul i-su oga es in e olu iona y compu a ion. In In The
9 h Annual Con e ence on Gene ic and E olu iona y Compu a ion (GECCO-2007,
pages 1288–1295. ACM P ess, 2007.
27. Y. S. Ong, P. B. Nai , and A. J. Keane. E olu iona y op imiza ion o compu a-
ionally expensi e p oblems ia su oga e modeling. AIAA Jou nal, 41(4):687–696,
2003.
28. R. E. Schapi e. A b ie in oduc ion o boos ing. In P oceedings o he 16 h
in e na ional join con e ence on A i icial in elligence - Volume 2, IJCAI’99, pages
1401–1406, San F ancisco, CA, USA, 1999. Mo gan Kau mann Publishe s Inc.
29. D. H. Wolpe . S acked gene aliza ion. Neu al Ne wo ks, 5:241–259, 1992.
Kon ak /Imp essum
Diese Ve ¨o en lichungen e scheinen im Rahmen de Sch i en eihe ”CIplus”. Alle
Ve ¨o en lichungen diese Reihe k¨onnen un e
www.ciplus- esea ch.de
ode un e
h p://opus.bsz-bw.de/ hk/index.php?la=de
abge u en we den.
K¨oln, Janua 2012
He ausgebe / Edi o ship
P o . D . Thomas Ba z-Beiels ein,
P o . D . Wol gang Konen,
P o . D . Bo is Naujoks,
P o . D . Ho s S enzel
Ins i u e o Compu e Science,
Facul y o Compu e Science and Enginee ing Science,
Cologne Uni e si y o Applied Sciences,
S einm¨ulle allee 1,
51643 Gumme sbach
u l: www.ciplus- esea ch.de
Sch i lei ung und Ansp echpa ne / Con ac edi o s o ice
P o . D . Thomas Ba z-Beiels ein,
Ins i u e o Compu e Science,
Facul y o Compu e Science and Enginee ing Science,
Cologne Uni e si y o Applied Sciences,
S einm¨ulle allee 1, 51643 Gumme sbach
phone: +49 2261 8196 6391
u l: h p://www.gm. h-koeln.de/~ba z/
eMail: homas.ba z-beiels ein@ h-koeln.de
ISSN (online) 2194-2870
