scieee Science in your language
[en] (orig)

A new Taxonomy of Continuous Global Optimization Algorithms

Stork, Jörg,Eiben, A.E.,Bartz-Beielstein, Thomas · 2018

Abstract

Surrogate-based optimization and nature-inspired metaheuristics have become the state of the art in solving real-world optimization problems. Still, it is difficult for beginners and even experts to get an overview that explains their advantages in comparison to the large number of available methods in the scope of continuous optimization. Available taxonomies lack the integration of surrogate-based approaches and thus their embedding in the larger context of this broad field. This article presents a taxonomy of the field, which further matches the idea of nature-inspired algorithms, as it is based on the human behavior in path finding. Intuitive analogies make it easy to conceive the most basic principles of the search algorithms, even for beginners and non-experts in this area of research. However, this scheme does not oversimplify the high complexity of the different algorithms, as the class identifier only defines a descriptive meta-level of the algorithm search strategies. The taxonomy was established by exploring and matching algorithm schemes, extracting similarities and differences, and creating a set of classification indicators to distinguish between five distinct classes. In practice, this taxonomy allows recommendations for the applicability of the corresponding algorithms and helps developers trying to create or improve their own algorithms.

Read accessible full text

A new Taxonomy of Continuous Global Optimization Algorithms

Author: Stork, Jörg,Eiben, A.E.,Bartz-Beielstein, Thomas
Year: 2018
Source: https://cos.bibl.th-koeln.de/files/753/Stor16acos-submitted.pdf
CIplus
Band 4/2018
A new Taxonomy o Con inuous Global
Op imiza ion Algo i hms
Jö g S o k, A.E. Eiben, and Thomas Ba z-Beiels ein
A new Taxonomy o Con inuous Global Op imiza ion
Algo i hms
Jö g S o k1, A.E. Eiben2, and Thomas Ba z-Beiels ein1
1TH Köln, S einmüelle allee 1, 51648 Gumme sbach, Ge many
2VU Ams e dam, De Boelelaan 1105, 1081 HV Ams e dam, Ne he lands
joe [email p o ec ed], [email p o ec ed], [email p o ec ed]
Abs ac
Su oga e-based op imiza ion and na u e-inspi ed me aheu is ics ha e become he s a e
o he a in sol ing eal-wo ld op imiza ion p oblems. S ill, i is di icul o beginne s
and e en expe s o ge an o e iew ha explains hei ad an ages in compa ison o he
la ge numbe o a ailable me hods in he scope o con inuous op imiza ion. A ailable ax-
onomies lack he in eg a ion o su oga e-based app oaches and hus hei embedding in
he la ge con ex o his b oad ield. This a icle p esen s a axonomy o he ield, which
u he ma ches he idea o na u e-inspi ed algo i hms, as i is based on he human beha io
in pa h inding. In ui i e analogies make i easy o concei e he mos basic p inciples o he
sea ch algo i hms, e en o beginne s and non-expe s in his a ea o esea ch. Howe e ,
his scheme does no o e simpli y he high complexi y o he di e en algo i hms, as he
class iden i ie only de ines a desc ip i e me a-le el o he algo i hm sea ch s a egies. The
axonomy was es ablished by explo ing and ma ching algo i hm schemes, ex ac ing simi-
la i ies and di e ences, and c ea ing a se o classi ica ion indica o s o dis inguish be ween
i e dis inc classes. In p ac ice, his axonomy allows ecommenda ions o he applicabil-
i y o he co esponding algo i hms and helps de elope s ying o c ea e o imp o e hei
own algo i hms.
1 In oduc ion
Con inuous global op imiza ion (CGO) ackles a ious di icul p oblems eme ging om he
con ex o complex physical o chemical p ocesses. Sol ing op imiza ion p oblems o his kind
necessa ily elies on pe o ming eal-wo ld expe imen s o on using compu e simula ions,
which a e equen ly employed in black-box ashions. A undamen al challenge in such sys-
ems is he high cos o unc ion e alua ions. Whe he we a e p obing he eal physical sys em
o que ying he simula o , he ime needed o ecei e an objec i e unc ion alue is ypically
e y high and can ange om hou s o mon hs. CGO me hods o such p oblems hus need
o ul ill a ce ain se o equi emen s. They need o wo k wi h black-box s yle p obes only, so
wi hou any u he in o ma ion on he s uc u e o he p oblem. Fu he hey mus app oach
he icini y o he global op imum wi h a limi ed numbe o unc ion e alua ions.
The imp o emen o compu a ional powe in he las decades has been in luencing he de-
elopmen o algo i hms. A massi e amoun o compu a ional powe became a ailable o
1
esea che s wo ldwide h ough mul i-co e desk op machines, pa allel compu ing, and high-
pe o mance compu ing clus e s. This has imp o ed he ollowing ields o esea ch: Fi s ly, he
de elopmen o mo e complex, na u e-inspi ed, and gene ally applicable heu is ics, so called
me aheu is ics. Secondly, majo ad ances in he ield o accu a e, da a-d i en app oxima ion
models, so-called su oga e models and hei embodimen in an op imiza ion p ocess.
Nowadays, CGO di e s la gely om ea ly app oaches. Fo example, mul i-s aged me hods
e alua e objec i e unc ions no di ec ly on he p oblem. They u ilize a combina ion o su -
oga e modeling wi h classical o me aheu is ic op imiza ion me hods o maximize he use o
a ailable p oblem in o ma ion ins ead. These amewo ks, such as sequen ial pa ame e op i-
miza ion (Ba z-Beiels ein, Lasa czyk, and P euß, 2005) o he su oga e managemen amewo k
(Booke , Dennis J , F ank, Se a ini, To czon, and T osse , 1999; Se a ini, 1999), de ine a new
class o algo i hms ha a e no well in eg a ed in p e ious axonomies.
In his a icle we will p opose a new axonomy on basis o algo i hm ea u es and display
plausible desc ip ions ounded on he na u al human beha io in pa h inding. To es ablish a
comp ehensi e axonomy, we ocus on iden i ying key elemen s o algo i hm design and u i-
lizing hese o de ine a clea sepa a ion be ween a small numbe o algo i hm classes. Al hough
abs ac ion is necessa y o de eloping ou esul s, we will p esen esul s ha a e use ul o
p ac i ione s. The u ilized abs ac ion allows us o p esen simply comp ehensible ideas on
how he indi idual classes di e and mo eo e , how he espec i e algo i hms pe o m hei
sea ch. Fo his pu pose, we di ide CGO algo i hms in o i e in ui i e classes: Exac , Wande e ,
Guide, Ca og aphe , and Hyb id. This a icle pa icula ly add esses di e en kinds o ead-
e s: beginne s will ind an in ui i e axonomy o CGO algo i hms, especially wi h ega d o
common me aheu is ics and newe de elopmen s in he ield o su oga e-based op imiza ion.
Fo ad anced eade s, we also discuss he sui abili y o ce ain classes o speci ic p oblem
p ope ies o p o ide basic knowledge o easonable algo i hm selec ion. An ex ensi e lis
o e e ences is p o ided o he expe ienced use s. The axonomy can be used o c ea e eal-
is ic compa isons and benchma ks o he di e en classes o algo i hms. I u he p o ides
insigh s o use s, who aim o de elop new sea ch s a egies, ope a o s and algo i hms.
The goal o global op imiza ion is o ind he o e all bes solu ion, i.e., o he common ask
o minimiza ion, o disco e decision a iable alues which minimize he objec i e unc ion
alue. We deno e he global sea ch space as compac se S={x|xl≤x≤xu}wi h xl,xu∈
R
n
being he explici , ini e lowe and uppe bounds on x.
Gi en a eal alued objec i e unc ion :
R
n→
R
wi h eal alued inpu ec o s xwe a emp
o ind he loca ion x∈
R
nwhich minimizes he unc ion: a g min (x),x∈ S.
Finding he global op imum is always he ul ima e goal and as such desi able, bu o many
p ac ical p oblems a solu ion imp o ing he cu en bes solu ion in a gi en budge o e alua-
ions o ime will s ill be a success. Pa icula in CGO he global op imum commonly canno be
iden i ied exac ly, hus mode n heu is ics a e designed o spend hei esou ces as e icien ly as
possible o app oxima e nea -bes solu ions, while inding he global op imum is ne e gua an-
eed. The emainde o his a icle is o ganized as ollows: Sec ion 2 p esen s he de elopmen
o op imiza ion algo i hms and hei co e concep s. Sec ion 3 mo i a es a new axonomy by
e iewing he his o y o a ailable CGO axonomies, illus a es algo i hm design aspec s and
p esen s ex ac ed classi ica ion ea u es. Sec ions 4 o 8 in oduce he i e di e en classes o
he new axonomy wi h examples and sugges ions ega ding hei applicabili y. Sec ion 9 sum-
ma izes and concludes he a icle wi h he ecen ends and challenges in CGO and cu en ly
impo an esea ch ields.
2
2 E olu ion o Op imiza ion Algo i hms
In o de o de elop a axonomy, i is necessa y o unde s and he me hodology and de elop-
men his o y o he co esponding algo i hms. Be o e p esen ing he equi emen s o he new
axonomy in Sec ion 3, we will desc ibe he undamen al p inciples o mode n sea ch algo-
i hms, pa icula he elemen s and backg ounds o su oga e-based op imiza ion.
2.1 Heu is ics and Me aheu is ics
In mode n compu e -aided op imiza ion, heu is ics and me aheu is ics a e well es ablished
solu ion echniques. Al hough p esen ing solu ions which a e no gua an eed o be op imal
o pe ec , hei gene al applicabili y and abili y o p esen ing as su icien solu ions make
hem e y a ac i e o applied op imiza ion, pa icula o indus ial p oblems. They a e buil
upon he p inciple o ial and e o , whe e solu ion candida es a e e alua ed and ewa ded
wi h a i ness. The e m i ness has i s o igins in e olu iona y compu a ion (Eiben and Smi h,
2015), whe e he i ness desc ibes he compe i i e abili y o an indi idual in he ep oduc ion
p ocess. The i ness is in i s simples o m he objec i e unc ion alue y= (x)in ela ion
o he op imiza ion goal, e.g., in a minimiza ion p oblem smalle alues ha e a highe i ness.
Mo eo e , i can be pa o he sea ch s a egy, e.g., scaled o adjus ed by addi ional unc ions,
pa icula o mul i-objec i e o cons ained op imiza ion.
Heu is ics can be de ined as p oblem-dependen algo i hms, which a e de eloped o adap ed
o he pa icula i ies o a speci ic op imiza ion p oblem o p oblem ins ance (Pea l, 1985). Typ-
ically, heu is ics pe o m e alua ions in a sys ema ic manne , al hough u ilizing s ochas ic el-
emen s. Heu is ics use his p inciple o p o ide as , no necessa ily exac (i.e., no op imal)
nume ical solu ions o op imiza ion p oblems. Mo eo e , heu is ics a e o en g eedy o p o-
ide as solu ions, bu ge apped in local op ima and ail o ind he global op imum.
In hei s a ing days in he 1960s, heu is ics we e no conside ed as eliable p oblem sol e s,
because mos esea che s in academia p e e ed classical ma hema ical app oaches and only
some p ac i ione s used heu is ics o ge as , possibly inaccu a e solu ions (Zanakis and E ans,
1981). This si ua ion changed in he 1970s. Heu is ic op imiza ion became a majo pa o
academic esea ch. Possible easons o his change migh be:
•The need o sol e mo e sophis ica ed nonde e minis ic p oblems, which had a polyno-
mial un ime and hus could no be sol ed e icien ly wi h exac algo i hms (Fomin and
Kaski, 2013).
•The a ailabili y and easy access o academics o mo e compu a ional powe .
Fu he ad an ages o heu is ics we e summa ized by Zanakis and E ans (1981). The mos
impo an a e:
•Simplici y o he algo i hm.
•Accu acy, i.e., small e o o inal solu ion.
•Robus ness, i.e., good solu ions wi hin easonable ime o di e en p oblems.
•Speed, i.e., du a ion o he compu a ion.
While heu is ics a e de eloped and op imized o e icien ly sol e a ce ain p oblem, he im-
p o ed a ailabili y o compu e esou ces ga e ise o highe -le el heu is ics, he me aheu is-
ics. Me aheu is ics can be de ined as p oblem independen , gene al pu pose op imiza ion al-
go i hms. They a e applicable o a wide ange o p oblems and p oblem ins ances. The e m
me a desc ibes he highe -le el gene al me hodology, which is u ilized o guide he unde lying
heu is ic s a egy (Talbi, 2009).
3

They sha e he ollowing cha ac e is ics (Boussaïd, Lepagno , and Sia y, 2013):
•The algo i hms a e na u e-inspi ed; hey ollow ce ain p inciples om na u al phenom-
ena o beha io s (e.g., biological e olu ion, physics, social beha io ).
•The sea ch p ocess in ol es s ochas ic pa s; i is based on p obabili y dis ibu ions and
andom p ocesses.
•They do no use he g adien o Hessian o he objec i e unc ion o ely on in o ma ion
o he p ocess which is a ailable be o e he s a o he op imiza ion un, so-called a p io i
in o ma ion.
•As hey a e mean o be gene al applicable sol e s, hey include a se o con ol pa ame-
e s o adjus he sea ch s a egy.
Du ing he emainde o his a icle we will ocus on heu is ic, espec i ely me aheu is ic algo-
i hms.
2.2 Mode n Op imiza ion Algo i hms
Based on he undamen als o heu is ics and me aheu is ics, we a e able o iden i y a simila i y
in he design o mode n op imiza ion algo i hms, which a ge s a la ge class o p oblems. Im-
po an ly, we ha e o conside he No F ee Lunch Theo em (Wolpe and Mac eady, 1997), which
s a es ha he e is no op imiza ion algo i hm ha is supe io o all o he s i hei pe o mance
is a e aged o e all possible p oblems. Consequen ly, any algo i hm needs o be adap ed o
he s uc u e o he p oblem a hand o achie e op imal pe o mance. This can be conside ed
du ing he cons uc ion o an algo i hm, be o e he op imiza ion by pa ame e uning o du ing
he un by pa ame e con ol (Ba z-Beiels ein e al., 2005; Eiben, Hin e ding, and Michalewicz,
1999). Tö n and Zilinskas (1989) men ion h ee p inciples o he cons uc ion o an op imiza-
ion algo i hm:
1. An algo i hm u ilizing all a ailable a p io i in o ma ion will ou pe o m a me hod using
less in o ma ion.
2. I no a p io i in o ma ion is a ailable, he in o ma ion is comple ely based on e alua ed
candida e poin s and hei objec i e alues.
3. Gi en a ixed numbe o e alua ed poin s, op imiza ion algo i hms will only di e om
each o he in he dis ibu ion o candida e poin s.
As mos mode n algo i hms ocus on handling p oblems whe e li le o no a p io i in o ma ion
is gi en, he p inciples displayed abo e lead o he conclusion ha he mos c ucial design
aspec o any algo i hm is o ind a s a egy o dis ibu e he ini ial candida es in he sea ch
space and o gene a e new candida es based on a ia ion o solu ions. These wo p ocedu es
de ine he sea ch s a egy, which needs o ollow he wo compe ing goals o explo a ion and
exploi a ion. In gene al, he main goal o any me hod is o each hei a ge wi h high e iciency,
i.e., o disco e op ima as and accu a e wi h as li le esou ces as possible. Mo eo e , he goal
is no manda o y inding he global op imum, which is a demanding and expensi e ask o
many p oblems, bu o iden i y a aluable local op imum o o imp o e he cu en a ailable
solu ion. We will explici ly discuss he design o mode n op imiza ion algo i hms in Sec ion
3.3.
4
2.3 Exac Algo i hms
Exac algo i hms, also e e ed o as comple e algo i hms (Neumaie , 2004), a e a special class
o de e minis ic, sys ema ic and non-heu is ic op imiza ion algo i hms. I su icien a p io i in o -
ma ion abou he objec i e unc ion is a ailable, hey ha e a gua an ee o inding he global
op imum wi hin using a p edic able amoun o esou ces, such as unc ion e alua ions o com-
pu a ion ime (Fomin and Kaski, 2013). I hey a e applicable o he p oblem, hese algo i hms
a e mo e eliable han heu is ics, as hey allow con e gence p oo s o inding he global op i-
mum. Wi hou a ailable a p io i in o ma ion, he s opping c i e ium needs o be de ined by a
heu is ic app oach, which so ens he gua an ee o inding he op imum. Mo eo e , i is he-
o e ical possible o apply hese algo i hms o he class o black-box p oblems wi h he gi en
abili y o ind he global op imum wi h ce ain y a e ini e ime. Howe e , hey will need
exponen ial compu a ion ime due o an expensi e, dense sea ch. This ende s hem no ap-
plicable o many esou ce-limi ed applica ions. The exac class, p esen ed in Sec ion 4, con ains
he ela ed algo i hms.
2.4 Su oga e-based Op imiza ion Algo i hms
Su oga e-based op imiza ion algo i hms a e designed o p ocess expensi e and complex p ob-
lems, which a ise om eal-wo ld applica ions and sophis ica ed compu a ional models. Real-
wo ld p oblems a e commonly black-box, which means ha hey only p o ide e y spa se
domain knowledge. Consequen ly, p oblem in o ma ion needs o be exploi ed by expe imen s
o unc ion e alua ions.
Su oga e-based op imiza ion is de eloped o op imally exhaus he a ailable in o ma ion by
u ilizing a su oga e model. A su oga e model is an app oxima ion which subs i u es he o ig-
inal expensi e objec i e unc ion, eal-wo ld p ocess, physical simula ion, o compu a ional
p ocess du ing he op imiza ion. In gene al, su oga es a e ei he simpli ied physical o nu-
me ical models based on knowledge abou he physical sys em, o empi ical unc ional models
based on knowledge acqui ed om e alua ions and spa se sampling o he pa ame e space
(Sønde gaa d, Madsen, and Nielsen, 2003). In his wo k, we ocus on he la e . The e ms su -
oga e model,me a-model, esponse su ace model and pos e io dis ibu ion a e used synonymous
in he common li e a u e (Mockus, 1974; Jones, 2001; Ba z-Beiels ein and Zae e e , 2017). We
will b ie ly e e o a su oga e model as su oga e. Fu he mo e, we assume ha i is c ucial
o dis inguish be ween he use o an explici su oga e o he objec i e unc ion and gene al
model-based op imiza ion (Zlochin, Bi a a i, Meuleau, and Do igo, 2004), which addi ionally
e e s o me hods, whe e a s a is ical model is used o gene a e new candida e solu ions (c .
Sec ion 3.3). As hese wo de ini ions o model-based op imiza ion a e equen ly used in a
non-consis en manne , we will clea ly dis inguish be ween he wo di e en e ms su oga e-
based and model-based o a oid con usions. Ano he e m p esen in he li e a u e is su oga e-
assis ed op imiza ion, which mos ly e e s o he applica ion o su oga es in popula ion-based
e olu iona y compu a ion (Jin, 2011), whe e e olu iona y op imiza ion and su oga e-based
op imiza ion a e applied in an hyb id app oach (see Sec ion 8).
Impo an publica ions ea u ing o e iews o su eys on su oga es and su oga e-based op-
imiza ion we e p esen ed by Sacks, Welch, Mi chell, and Wynn (1989), Jones (2001), Queipo,
Ha ka, Shyy, Goel, Vaidyana han, and Tucke (2005), Fo es e and Keane (2009). Su oga e-
based op imiza ion is commonly applied, bu no limi ed o he case o complex eal-wo ld
op imiza ion applica ions, whe e wo ypical p oblem laye s and a su oga e laye can be de-
5
(op ional)
uning
p ocedu e
op imiza ion algo i hm
s(x)
<la exi sha1_base64="4W8GsRo EQsVqEn/ccCTpFCRcRA=">AAACz3ichVFNT8JAEH3UL8A 1KMXIpjghbRc9EjiR7yYQCI CRLTlgU pW3agiLRePXmV +Z/hYP q7FRIlxm+3M nnzdmbH8Gw CFX1LaHMzS8sLiVT6eWV1bX1zMZmPXCH ilqpmu7 PQA2FbjqiFVmiLpucL WDYomH0D6N4YyT8wHKd83Dsi ZA7zlW1zL1kFA9HxRu9/KXmZxaVOXKzjpa7OQQ 4qbeccFOnBhYogBBByE9G3oCPi1oEGFR6yNCTG niXjA dIM3dIliBDJ9 n 8dTK0YdniPNQGab MXm9pmZxS73iVQ0yI5uF QD2g/uO4n1/ xhIpWjCse0BhVTU GMeIg M 7LHMTMaS3/Z0ZdhejiQHZjsT5PIlG 5 OESM+sb6MZHEsmT1qGPI84gs4 DVWEL3yVCE O+7Q6 IKqeLEij 1 N o9VkPx6z9HuqsUy8VNbWoVUu5shYPPIl 7KDAqe6jjFNUWIeJazzjBa9KVblRHpTHL6qSiHO28GMpT59LPpA0</la exi >
<la exi sha1_base64="4W8GsRo EQsVqEn/ccCTpFCRcRA=">AAACz3ichVFNT8JAEH3UL8A 1KMXIpjghbRc9EjiR7yYQCI CRLTlgU pW3agiLRePXmV +Z/hYP q7FRIlxm+3M nnzdmbH8Gw CFX1LaHMzS8sLiVT6eWV1bX1zMZmPXCH ilqpmu7 PQA2FbjqiFVmiLpucL WDYomH0D6N4YyT8wHKd83Dsi ZA7zlW1zL1kFA9HxRu9/KXmZxaVOXKzjpa7OQQ 4qbeccFOnBhYogBBByE9G3oCPi1oEGFR6yNCTG niXjA dIM3dIliBDJ9 n 8dTK0YdniPNQGab MXm9pmZxS73iVQ0yI5uF QD2g/uO4n1/ xhIpWjCse0BhVTU GMeIg M 7LHMTMaS3/Z0ZdhejiQHZjsT5PIlG 5 OESM+sb6MZHEsmT1qGPI84gs4 DVWEL3yVCE O+7Q6 IKqeLEij 1 N o9VkPx6z9HuqsUy8VNbWoVUu5shYPPIl 7KDAqe6jjFNUWIeJazzjBa9KVblRHpTHL6qSiHO28GMpT59LPpA0</la exi >
<la exi sha1_base64="4W8GsRo EQsVqEn/ccCTpFCRcRA=">AAACz3ichVFNT8JAEH3UL8A 1KMXIpjghbRc9EjiR7yYQCI CRLTlgU pW3agiLRePXmV +Z/hYP q7FRIlxm+3M nnzdmbH8Gw CFX1LaHMzS8sLiVT6eWV1bX1zMZmPXCH ilqpmu7 PQA2FbjqiFVmiLpucL WDYomH0D6N4YyT8wHKd83Dsi ZA7zlW1zL1kFA9HxRu9/KXmZxaVOXKzjpa7OQQ 4qbeccFOnBhYogBBByE9G3oCPi1oEGFR6yNCTG niXjA dIM3dIliBDJ9 n 8dTK0YdniPNQGab MXm9pmZxS73iVQ0yI5uF QD2g/uO4n1/ xhIpWjCse0BhVTU GMeIg M 7LHMTMaS3/Z0ZdhejiQHZjsT5PIlG 5 OESM+sb6MZHEsmT1qGPI84gs4 DVWEL3yVCE O+7Q6 IKqeLEij 1 N o9VkPx6z9HuqsUy8VNbWoVUu5shYPPIl 7KDAqe6jjFNUWIeJazzjBa9KVblRHpTHL6qSiHO28GMpT59LPpA0</la exi >
<la exi sha1_base64="4W8GsRo EQsVqEn/ccCTpFCRcRA=">AAACz3ichVFNT8JAEH3UL8A 1KMXIpjghbRc9EjiR7yYQCI CRLTlgU pW3agiLRePXmV +Z/hYP q7FRIlxm+3M nnzdmbH8Gw CFX1LaHMzS8sLiVT6eWV1bX1zMZmPXCH ilqpmu7 PQA2FbjqiFVmiLpucL WDYomH0D6N4YyT8wHKd83Dsi ZA7zlW1zL1kFA9HxRu9/KXmZxaVOXKzjpa7OQQ 4qbeccFOnBhYogBBByE9G3oCPi1oEGFR6yNCTG niXjA dIM3dIliBDJ9 n 8dTK0YdniPNQGab MXm9pmZxS73iVQ0yI5uF QD2g/uO4n1/ xhIpWjCse0BhVTU GMeIg M 7LHMTMaS3/Z0ZdhejiQHZjsT5PIlG 5 OESM+sb6MZHEsmT1qGPI84gs4 DVWEL3yVCE O+7Q6 IKqeLEij 1 N o9VkPx6z9HuqsUy8VNbWoVUu5shYPPIl 7KDAqe6jjFNUWIeJazzjBa9KVblRHpTHL6qSiHO28GMpT59LPpA0</la exi >
s(x)
<la exi sha1_base64="4W8GsRo EQsVqEn/ccCTpFCRcRA=">AAACz3ichVFNT8JAEH3UL8A 1KMXIpjghbRc9EjiR7yYQCI CRLTlgU pW3agiLRePXmV +Z/hYP q7FRIlxm+3M nnzdmbH8Gw CFX1LaHMzS8sLiVT6eWV1bX1zMZmPXCH ilqpmu7 PQA2FbjqiFVmiLpucL WDYomH0D6N4YyT8wHKd83Dsi ZA7zlW1zL1kFA9HxRu9/KXmZxaVOXKzjpa7OQQ 4qbeccFOnBhYogBBByE9G3oCPi1oEGFR6yNCTG niXjA dIM3dIliBDJ9 n 8dTK0YdniPNQGab MXm9pmZxS73iVQ0yI5uF QD2g/uO4n1/ xhIpWjCse0BhVTU GMeIg M 7LHMTMaS3/Z0ZdhejiQHZjsT5PIlG 5 OESM+sb6MZHEsmT1qGPI84gs4 DVWEL3yVCE O+7Q6 IKqeLEij 1 N o9VkPx6z9HuqsUy8VNbWoVUu5shYPPIl 7KDAqe6jjFNUWIeJazzjBa9KVblRHpTHL6qSiHO28GMpT59LPpA0</la exi >
<la exi sha1_base64="4W8GsRo EQsVqEn/ccCTpFCRcRA=">AAACz3ichVFNT8JAEH3UL8A 1KMXIpjghbRc9EjiR7yYQCI CRLTlgU pW3agiLRePXmV +Z/hYP q7FRIlxm+3M nnzdmbH8Gw CFX1LaHMzS8sLiVT6eWV1bX1zMZmPXCH ilqpmu7 PQA2FbjqiFVmiLpucL WDYomH0D6N4YyT8wHKd83Dsi ZA7zlW1zL1kFA9HxRu9/KXmZxaVOXKzjpa7OQQ 4qbeccFOnBhYogBBByE9G3oCPi1oEGFR6yNCTG niXjA dIM3dIliBDJ9 n 8dTK0YdniPNQGab MXm9pmZxS73iVQ0yI5uF QD2g/uO4n1/ xhIpWjCse0BhVTU GMeIg M 7LHMTMaS3/Z0ZdhejiQHZjsT5PIlG 5 OESM+sb6MZHEsmT1qGPI84gs4 DVWEL3yVCE O+7Q6 IKqeLEij 1 N o9VkPx6z9HuqsUy8VNbWoVUu5shYPPIl 7KDAqe6jjFNUWIeJazzjBa9KVblRHpTHL6qSiHO28GMpT59LPpA0</la exi >
<la exi sha1_base64="4W8GsRo EQsVqEn/ccCTpFCRcRA=">AAACz3ichVFNT8JAEH3UL8A 1KMXIpjghbRc9EjiR7yYQCI CRLTlgU pW3agiLRePXmV +Z/hYP q7FRIlxm+3M nnzdmbH8Gw CFX1LaHMzS8sLiVT6eWV1bX1zMZmPXCH ilqpmu7 PQA2FbjqiFVmiLpucL WDYomH0D6N4YyT8wHKd83Dsi ZA7zlW1zL1kFA9HxRu9/KXmZxaVOXKzjpa7OQQ 4qbeccFOnBhYogBBByE9G3oCPi1oEGFR6yNCTG niXjA dIM3dIliBDJ9 n 8dTK0YdniPNQGab MXm9pmZxS73iVQ0yI5uF QD2g/uO4n1/ xhIpWjCse0BhVTU GMeIg M 7LHMTMaS3/Z0ZdhejiQHZjsT5PIlG 5 OESM+sb6MZHEsmT1qGPI84gs4 DVWEL3yVCE O+7Q6 IKqeLEij 1 N o9VkPx6z9HuqsUy8VNbWoVUu5shYPPIl 7KDAqe6jjFNUWIeJazzjBa9KVblRHpTHL6qSiHO28GMpT59LPpA0</la exi >
<la exi sha1_base64="4W8GsRo EQsVqEn/ccCTpFCRcRA=">AAACz3ichVFNT8JAEH3UL8A 1KMXIpjghbRc9EjiR7yYQCI CRLTlgU pW3agiLRePXmV +Z/hYP q7FRIlxm+3M nnzdmbH8Gw CFX1LaHMzS8sLiVT6eWV1bX1zMZmPXCH ilqpmu7 PQA2FbjqiFVmiLpucL WDYomH0D6N4YyT8wHKd83Dsi ZA7zlW1zL1kFA9HxRu9/KXmZxaVOXKzjpa7OQQ 4qbeccFOnBhYogBBByE9G3oCPi1oEGFR6yNCTG niXjA dIM3dIliBDJ9 n 8dTK0YdniPNQGab MXm9pmZxS73iVQ0yI5uF QD2g/uO4n1/ xhIpWjCse0BhVTU GMeIg M 7LHMTMaS3/Z0ZdhejiQHZjsT5PIlG 5 OESM+sb6MZHEsmT1qGPI84gs4 DVWEL3yVCE O+7Q6 IKqeLEij 1 N o9VkPx6z9HuqsUy8VNbWoVUu5shYPPIl 7KDAqe6jjFNUWIeJazzjBa9KVblRHpTHL6qSiHO28GMpT59LPpA0</la exi >
su oga e model
1(x)
<la exi sha1_base64="58e JI4/e+23 IDakVSEuBI2PGA=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 2MW7w4KnVzeKBlq6bOOmTh5SlY1yH3QDXUpIJuG5BEnnwR8lxjF+FpkkkEhsDZNgEXwHBXn9EBZ5A7B4mAwoAP8+zi1E THWW GK GLS52hEyd9 HPlKIF yVw49hP7H Fdb/94aJUpYVjmE KGaU4gVwQbdgzM 0Eua0l mZsi BPTpS3TioL1SI7NP+0TlBJAI2UBGdThWzDw1LnUd4AR+2hg kK08VdNVxF5Ypy5WKnygy6EWw8 VRD8Zs/h3q FM l0yjZF6W8xUzGXiadmmPipjqIVXonKqoQ07zhV7pTb Sx qj9 RN1VJJzg79W zF9SgkMs=</la exi >
<la exi sha1_base64="58e JI4/e+23 IDakVSEuBI2PGA=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 2MW7w4KnVzeKBlq6bOOmTh5SlY1yH3QDXUpIJuG5BEnnwR8lxjF+FpkkkEhsDZNgEXwHBXn9EBZ5A7B4mAwoAP8+zi1E THWW GK GLS52hEyd9 HPlKIF yVw49hP7H Fdb/94aJUpYVjmE KGaU4gVwQbdgzM 0Eua0l mZsi BPTpS3TioL1SI7NP+0TlBJAI2UBGdThWzDw1LnUd4AR+2hg kK08VdNVxF5Ypy5WKnygy6EWw8 VRD8Zs/h3q FM l0yjZF6W8xUzGXiadmmPipjqIVXonKqoQ07zhV7pTb Sx qj9 RN1VJJzg79W zF9SgkMs=</la exi >
<la exi sha1_base64="58e JI4/e+23 IDakVSEuBI2PGA=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 2MW7w4KnVzeKBlq6bOOmTh5SlY1yH3QDXUpIJuG5BEnnwR8lxjF+FpkkkEhsDZNgEXwHBXn9EBZ5A7B4mAwoAP8+zi1E THWW GK GLS52hEyd9 HPlKIF yVw49hP7H Fdb/94aJUpYVjmE KGaU4gVwQbdgzM 0Eua0l mZsi BPTpS3TioL1SI7NP+0TlBJAI2UBGdThWzDw1LnUd4AR+2hg kK08VdNVxF5Ypy5WKnygy6EWw8 VRD8Zs/h3q FM l0yjZF6W8xUzGXiadmmPipjqIVXonKqoQ07zhV7pTb Sx qj9 RN1VJJzg79W zF9SgkMs=</la exi >
<la exi sha1_base64="58e JI4/e+23 IDakVSEuBI2PGA=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 2MW7w4KnVzeKBlq6bOOmTh5SlY1yH3QDXUpIJuG5BEnnwR8lxjF+FpkkkEhsDZNgEXwHBXn9EBZ5A7B4mAwoAP8+zi1E THWW GK GLS52hEyd9 HPlKIF yVw49hP7H Fdb/94aJUpYVjmE KGaU4gVwQbdgzM 0Eua0l mZsi BPTpS3TioL1SI7NP+0TlBJAI2UBGdThWzDw1LnUd4AR+2hg kK08VdNVxF5Ypy5WKnygy6EWw8 VRD8Zs/h3q FM l0yjZF6W8xUzGXiadmmPipjqIVXonKqoQ07zhV7pTb Sx qj9 RN1VJJzg79W zF9SgkMs=</la exi >
1(x)
<la exi sha1_base64="58e JI4/e+23 IDakVSEuBI2PGA=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 2MW7w4KnVzeKBlq6bOOmTh5SlY1yH3QDXUpIJuG5BEnnwR8lxjF+FpkkkEhsDZNgEXwHBXn9EBZ5A7B4mAwoAP8+zi1E THWW GK GLS52hEyd9 HPlKIF yVw49hP7H Fdb/94aJUpYVjmE KGaU4gVwQbdgzM 0Eua0l mZsi BPTpS3TioL1SI7NP+0TlBJAI2UBGdThWzDw1LnUd4AR+2hg kK08VdNVxF5Ypy5WKnygy6EWw8 VRD8Zs/h3q FM l0yjZF6W8xUzGXiadmmPipjqIVXonKqoQ07zhV7pTb Sx qj9 RN1VJJzg79W zF9SgkMs=</la exi >
<la exi sha1_base64="58e JI4/e+23 IDakVSEuBI2PGA=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 2MW7w4KnVzeKBlq6bOOmTh5SlY1yH3QDXUpIJuG5BEnnwR8lxjF+FpkkkEhsDZNgEXwHBXn9EBZ5A7B4mAwoAP8+zi1E THWW GK GLS52hEyd9 HPlKIF yVw49hP7H Fdb/94aJUpYVjmE KGaU4gVwQbdgzM 0Eua0l mZsi BPTpS3TioL1SI7NP+0TlBJAI2UBGdThWzDw1LnUd4AR+2hg kK08VdNVxF5Ypy5WKnygy6EWw8 VRD8Zs/h3q FM l0yjZF6W8xUzGXiadmmPipjqIVXonKqoQ07zhV7pTb Sx qj9 RN1VJJzg79W zF9SgkMs=</la exi >
<la exi sha1_base64="58e JI4/e+23 IDakVSEuBI2PGA=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 2MW7w4KnVzeKBlq6bOOmTh5SlY1yH3QDXUpIJuG5BEnnwR8lxjF+FpkkkEhsDZNgEXwHBXn9EBZ5A7B4mAwoAP8+zi1E THWW GK GLS52hEyd9 HPlKIF yVw49hP7H Fdb/94aJUpYVjmE KGaU4gVwQbdgzM 0Eua0l mZsi BPTpS3TioL1SI7NP+0TlBJAI2UBGdThWzDw1LnUd4AR+2hg kK08VdNVxF5Ypy5WKnygy6EWw8 VRD8Zs/h3q FM l0yjZF6W8xUzGXiadmmPipjqIVXonKqoQ07zhV7pTb Sx qj9 RN1VJJzg79W zF9SgkMs=</la exi >
<la exi sha1_base64="58e JI4/e+23 IDakVSEuBI2PGA=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 2MW7w4KnVzeKBlq6bOOmTh5SlY1yH3QDXUpIJuG5BEnnwR8lxjF+FpkkkEhsDZNgEXwHBXn9EBZ5A7B4mAwoAP8+zi1E THWW GK GLS52hEyd9 HPlKIF yVw49hP7H Fdb/94aJUpYVjmE KGaU4gVwQbdgzM 0Eua0l mZsi BPTpS3TioL1SI7NP+0TlBJAI2UBGdThWzDw1LnUd4AR+2hg kK08VdNVxF5Ypy5WKnygy6EWw8 VRD8Zs/h3q FM l0yjZF6W8xUzGXiadmmPipjqIVXonKqoQ07zhV7pTb Sx qj9 RN1VJJzg79W zF9SgkMs=</la exi >
physical eal wo ld p ocess
con ol pa ame e s op imized con ol
pa ame e s
decision
a iables
op imized
a iables
p ocess pa ame e s,
es ima ed ou pu
simula ed
inpu
inpu ou pu
candida e solu ions p edic ed i ness
app oxima ion and e i ica ion su oga e-based op imiza ion
2(x)
<la exi sha1_base64="OYhAsR8zTgd/F7lNWP Ba 3g+Sg=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 1Mu3h0UO m8UTLU0mcdM3HylKxqkPugG+pSQDYNySNOPgn4LjGK8bXIJINCYG2aAI gOS O6YGyyB2CxcFgQA 493FqJaiPs9SMVbaNW1zsCJk67WO KUULbHk hx/D mL K6z/7w0TpSw HMNaUMwoxQ ggm7BmJ pJcxpL MzZVeCenSkunFQX6gQ2a 9o3OCSARsoCI6nSpmHxqWOo/wAj5sDRXIV54q6K jLixTlisVP1Fk0I g5eujHozZ/D UWadeLplGybws5y mM A07dIeFTHVQ6 QOVVRh5zmC73Sm3aljbVH7embqqWSnB36 bTnL9cKkMw=</la exi >
<la exi sha1_base64="OYhAsR8zTgd/F7lNWP Ba 3g+Sg=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 1Mu3h0UO m8UTLU0mcdM3HylKxqkPugG+pSQDYNySNOPgn4LjGK8bXIJINCYG2aAI gOS O6YGyyB2CxcFgQA 493FqJaiPs9SMVbaNW1zsCJk67WO KUULbHk hx/D mL K6z/7w0TpSw HMNaUMwoxQ ggm7BmJ pJcxpL MzZVeCenSkunFQX6gQ2a 9o3OCSARsoCI6nSpmHxqWOo/wAj5sDRXIV54q6K jLixTlisVP1Fk0I g5eujHozZ/D UWadeLplGybws5y mM A07dIeFTHVQ6 QOVVRh5zmC73Sm3aljbVH7embqqWSnB36 bTnL9cKkMw=</la exi >
<la exi sha1_base64="OYhAsR8zTgd/F7lNWP Ba 3g+Sg=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 1Mu3h0UO m8UTLU0mcdM3HylKxqkPugG+pSQDYNySNOPgn4LjGK8bXIJINCYG2aAI gOS O6YGyyB2CxcFgQA 493FqJaiPs9SMVbaNW1zsCJk67WO KUULbHk hx/D mL K6z/7w0TpSw HMNaUMwoxQ ggm7BmJ pJcxpL MzZVeCenSkunFQX6gQ2a 9o3OCSARsoCI6nSpmHxqWOo/wAj5sDRXIV54q6K jLixTlisVP1Fk0I g5eujHozZ/D UWadeLplGybws5y mM A07dIeFTHVQ6 QOVVRh5zmC73Sm3aljbVH7embqqWSnB36 bTnL9cKkMw=</la exi >
<la exi sha1_base64="OYhAsR8zTgd/F7lNWP Ba 3g+Sg=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 1Mu3h0UO m8UTLU0mcdM3HylKxqkPugG+pSQDYNySNOPgn4LjGK8bXIJINCYG2aAI gOS O6YGyyB2CxcFgQA 493FqJaiPs9SMVbaNW1zsCJk67WO KUULbHk hx/D mL K6z/7w0TpSw HMNaUMwoxQ ggm7BmJ pJcxpL MzZVeCenSkunFQX6gQ2a 9o3OCSARsoCI6nSpmHxqWOo/wAj5sDRXIV54q6K jLixTlisVP1Fk0I g5eujHozZ/D UWadeLplGybws5y mM A07dIeFTHVQ6 QOVVRh5zmC73Sm3aljbVH7embqqWSnB36 bTnL9cKkMw=</la exi >
2(x)
<la exi sha1_base64="OYhAsR8zTgd/F7lNWP Ba 3g+Sg=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 1Mu3h0UO m8UTLU0mcdM3HylKxqkPugG+pSQDYNySNOPgn4LjGK8bXIJINCYG2aAI gOS O6YGyyB2CxcFgQA 493FqJaiPs9SMVbaNW1zsCJk67WO KUULbHk hx/D mL K6z/7w0TpSw HMNaUMwoxQ ggm7BmJ pJcxpL MzZVeCenSkunFQX6gQ2a 9o3OCSARsoCI6nSpmHxqWOo/wAj5sDRXIV54q6K jLixTlisVP1Fk0I g5eujHozZ/D UWadeLplGybws5y mM A07dIeFTHVQ6 QOVVRh5zmC73Sm3aljbVH7embqqWSnB36 bTnL9cKkMw=</la exi >
<la exi sha1_base64="OYhAsR8zTgd/F7lNWP Ba 3g+Sg=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 1Mu3h0UO m8UTLU0mcdM3HylKxqkPugG+pSQDYNySNOPgn4LjGK8bXIJINCYG2aAI gOS O6YGyyB2CxcFgQA 493FqJaiPs9SMVbaNW1zsCJk67WO KUULbHk hx/D mL K6z/7w0TpSw HMNaUMwoxQ ggm7BmJ pJcxpL MzZVeCenSkunFQX6gQ2a 9o3OCSARsoCI6nSpmHxqWOo/wAj5sDRXIV54q6K jLixTlisVP1Fk0I g5eujHozZ/D UWadeLplGybws5y mM A07dIeFTHVQ6 QOVVRh5zmC73Sm3aljbVH7embqqWSnB36 bTnL9cKkMw=</la exi >
<la exi sha1_base64="OYhAsR8zTgd/F7lNWP Ba 3g+Sg=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 1Mu3h0UO m8UTLU0mcdM3HylKxqkPugG+pSQDYNySNOPgn4LjGK8bXIJINCYG2aAI gOS O6YGyyB2CxcFgQA 493FqJaiPs9SMVbaNW1zsCJk67WO KUULbHk hx/D mL K6z/7w0TpSw HMNaUMwoxQ ggm7BmJ pJcxpL MzZVeCenSkunFQX6gQ2a 9o3OCSARsoCI6nSpmHxqWOo/wAj5sDRXIV54q6K jLixTlisVP1Fk0I g5eujHozZ/D UWadeLplGybws5y mM A07dIeFTHVQ6 QOVVRh5zmC73Sm3aljbVH7embqqWSnB36 bTnL9cKkMw=</la exi >
<la exi sha1_base64="OYhAsR8zTgd/F7lNWP Ba 3g+Sg=">AAAC0XichVFLT8JAEB7qC/CFe TSCCZ4IS0XPZL4iBcTjPJIkJB WbCh 7QLEYmJ8e Nq/4x/S0e/HY JkoM22xn9p p3ZsULXiYVh Ke0hcWl5ZV0J u6 6xmd a s BMLJ5zQ7cIGpaLOau4/OacITLm2HEmWe5 GENjmW8MeJR7AT+ RiH O2x u/0HJsJQM1C 1Mu3h0UO m8UTLU0mcdM3HylKxqkPugG+pSQDYNySNOPgn4LjGK8bXIJINCYG2aAI gOS O6YGyyB2CxcFgQA 493FqJaiPs9SMVbaNW1zsCJk67WO KUULbHk hx/D mL K6z/7w0TpSw HMNaUMwoxQ ggm7BmJ pJcxpL MzZVeCenSkunFQX6gQ2a 9o3OCSARsoCI6nSpmHxqWOo/wAj5sDRXIV54q6K jLixTlisVP1Fk0I g5eujHozZ/D UWadeLplGybws5y mM A07dIeFTHVQ6 QOVVRh5zmC73Sm3aljbVH7embqqWSnB36 bTnL9cKkMw=</la exi >
compu a ional model
(op ional) uning o pa ame e s (op ional) di ec op imiza ion
Figu e 1: A su oga e-based op imiza ion p ocess o a eal-wo ld p ocess wi h he di e en objec i e unc ion
laye s and ou lined inpu s and ou pu s; he complexi y o he objec i e unc ions is isible by he dec easing size
o hei boxes. The ull g ey a ows illus a e he app oxima ion and e i ica ion pa hs, he yellow dashed and ed
do ed a ows indica e he su oga e-based op imiza ion and op ional di ec op imiza ion. The blue dashed a ows
show he op ional pa ame e uning o he op imiza ion algo i hm o su oga e modeling p ocess.
6
ined. The de ined laye s can be ans e ed o di e en compu a ional p oblems wi h expen-
si e unc ion e alua ions, such as complex algo i hms o machine lea ning asks. Each laye
can be he a ge o an op imiza ion o used o e ie e in o ma ion o guide he op imiza ion
p ocess. Figu e 1 illus a es he di e en laye s o objec i e unc ions and he su oga e-based
op imiza ion p ocess o eal-wo ld p oblems. In his case, he objec i e unc ion laye s, om
he bo om up, a e:
L1 The eal-wo ld applica ion 1(x), gi en by he physical p ocess i sel o a physical model.
Di ec op imiza ion is o en expensi e o e en impossible, due o e alua ions in ol ing
esou ce demanding p o o ype building o e en haza dous expe imen s.
L2 The compu a ional model 2(x), gi en by a simula ion o he physical p ocess o a complex
compu a ional model , e.g., a compu a ional luid dynamics model o s uc u al dynamics
model. A single compu a ion may ake minu es, hou s, o e en weeks o compu e.
L3 The su oga e s(x), gi en by a da a-d i en eg ession model. The accu acy hea ily depends
on he unde lying su oga e ype and numbe o a ailable in o ma ion (i.e., unc ion e al-
ua ions). The op imiza ion is, compa ed o he o he laye s, ypically cheap. Su oga es a e
cons uc ed ei he o he eal-wo ld applica ion 1(x)o he compu a ional model 2(x).
Fu he mo e, he su oga e-based op imiza ion cycle includes he op imiza ion p ocess i sel ,
which is gi en by an adequa e op imiza ion algo i hm o he selec ed objec i e unc ion laye .
No su oga e-based op imiza ion is pe o med, i he op imiza ion is di ec ly applied o 1(x)
o 2(x). The su oga e-based op imiza ion uses 1(x)o 2(x) o e i ica ion o p omising
solu ion candida es. Mo eo e , he con ol pa ame e s o he op imiza ion algo i hm o e en
he comple e op imiza ion cycle including he su oga e modeling p ocess can be uned (Ba z-
Beiels ein e al., 2005).
Each laye imposes di e en e alua ion cos s and solu ion accu acies: he eal-wo ld p oblem
is he mos expensi e o e alua e, while he su oga e is he cheapes o e alua e. The main
bene i o using su oga es is hus he educ ion o needed expensi e unc ion e alua ions on
he objec i e unc ion 1(x)o 2(x)du ing he op imiza ion. The s udies by Loshchilo , Schoe-
naue , and Sebag (2012), Ma sden, Wang, Dennis J , and Moin (2004), Ong, Nai , Keane, and
Wong (2005) and Won and Ray (2004) ea u e benchma k compa isons o su oga e-based op-
imiza ion. Ne e heless, he model cons uc ion and upda ing o he su oga es also equi es
compu a ional esou ces, as well e alua ions o e i ica ion on he mo e expensi e unc ion
laye s. An ad an age o su oga e-based op imiza ion is he a ailabili y o he su oga e model,
which can be u ilized o gain u he global insigh in o he p oblem, which is pa icula ly alu-
able o black-box p oblems. Fo ins ance, he su oga e can be u ilized o iden i y impo an
decision a iables o isualize he na u e o he p oblem, i.e., he i ness landscape.
A common op imiza ion p ocess using su oga es is ou lined by he ollowing s eps:
1. Sampling he objec i e unc ion a kposi ions wi h yi= 1(xi)o yi= 2(xi),1≤i≤k
o gene a e a se o obse a ions D ={(xi,yi),1≤i≤k}. The sampling design plan is
commonly selec ed acco ding o he su oga e.
2. Selec ing a sui able su oga e. The selec ion o he co ec su oga e ype can be a com-
pu a ional demanding s ep in he op imiza ion p ocess, as o en no p io in o ma ion
indica ing he bes ype is a ailable. Common ypes o su oga es will be p esen ed in
Sec ion 7.
7
pa icle swa m op imiza ion (Kennedy and Ebe ha , 1995; Shi and Ebe ha , 1998) uses he
mo emen o bi d locks as a ole model; an colony op imiza ion (Do igo, Bi a a i, and S u -
zle, 2006) mimics, as he name sugges s, he ingenious pa h inding and ood sea ch p inciples
o an popula ions. These wo examples indica e ha analogies a e use ul o inspi e de elope s
o c ea e new sea ch s a egies. They a e also help ul o explain he beha io o hese sea ch
algo i hms, which makes hem aluable o be used in a comp ehensible axonomy.
3.3 The Fi e Elemen s o Algo i hm Design
Any mode n op imiza ion algo i hm, as de ined in Sec ion 2.2, can be educed o he ou key
sea ch s a egy elemen s Ini ializa ion,Va ia ion,E alua ion and Selec ion. All hese key elemen s
a e con olled by a i h elemen : he con ol pa ame e s o he di e en unc ions and ope a o s
in each elemen . Algo i hm 3.1 displays he key elemen s and he abs ac ed undamen al
s uc u e o op imiza ion algo i hms (Ba z-Beiels ein and Zae e e , 2017). These s uc u e and
elemen s can be mapped o any mode n op imiza ion algo i hm. E en i he sea ch s a egy is
inhe en ly di e en o elemen s do no ollow he illus a ed o de o appea mul iple imes
pe i e a ion.
Algo i hm 3.1: Gene al Op imiza ion Algo i hm
1se ini ial con ol pa ame e s
2begin
3 = 0
4ini ialize candida e(s)
5e alua e ini ial candida e(s)
6while no e mina ion-condi ion do
7 = + 1
8 a ia e solu ions o ge new candida e(s)
9e alua e new candida e(s)
10 selec solu ion(s) o nex i e a ion
11 op ional: upda e con ol pa ame e s
12 end
13 end
The ini ializa ion o he sea ch de ines s a ing loca ions o a schema o he ini ial candida e
solu ions. Two common s a egies exis :
1. I he e is no a p io i knowledge abou he p oblem and i s sea ch space, he bes op ion
is o use s a egically andomized s a ing poin s. Pa icula ly in e es ing o su oga e-
based op imiza ion a e sys ema ic ini ializa ion schemes by me hods om he ield o
design o expe imen s.
2. I domain knowledge o o he a p io i in o ma ion is a ailable, such as in o ma ion om
he p ocess o da a om p e ious op imiza ion uns, he algo i hm should be ini ialized
u ilizing his in o ma ion o ull ex end, e.g., by using a selec ion o hese solu ions, such
as hese wi h bes i ness. In su oga e-based op imiza ion a ailable da a can be used o
he ini ial modeling.
14

The ini ial candida es ha e a la ge impac on he balance be ween explo a ion and exploi a-
ion. Space- illing designs wi h la ge amoun s o andom candida es o sophis ica ed design
o expe imen s me hods will lead o a ini ial explo a ion o he sea ch space. S a ing wi h a
single candida e will p esumably lead o an exploi a ion o he neighbo hood o he selec ed
candida e loca ion. Hence, algo i hms using he i s scheme a e in gene al mo e obus , while
he la e a e sensi i e o he selec ion o he s a ing candida e, pa icula in mul i-modal land-
scapes. The obus ness can be u he inc eased by mul i-s a s a egies, which a e pa icula
common o single-candida e algo i hms, and also equen ly ecommended o popula ion-
based algo i hms (Hansen, Auge , Ros, Finck, and Pošík, 2010a).
The a ia ion du ing he sea ch p ocess de ines he me hods o gene a ing new candida es,
wi h special ega d on how a ailable o ob ained in o ma ion abou he objec i e unc ion is
used. A s anda d app oach is he a ia ion o exis ing obse a ions, as i u ilizes, and o a
ce ain ex end p ese es, he in o ma ion o p e ious i e a ions. E en he simples wande e
class algo i hms (Sec 5.1), which do no equi e any global in o ma ion o s o ed knowledge o
o me i e a ions, u ilize he las ob ained solu ion o gene a e new candida e(s). Sophis ica ed
algo i hms gene a e new candida es on he basis o exploi ed and s o ed global knowledge
abou he objec i e unc ion and i ness landscape. This can be conduc ed explici ly by ei he
keeping an a chi e o all a ailable o selec ed obse a ions, o implici ly by using dis ibu-
ion o da a models o a ailable obse a ions. Ano he op ion o gene a e new candida es is
combining in o ma ion o mul iple candida es by dedica ed unc ions o ope a o s, pa icula
p esen in he guide class (Sec 6.1). The exac ope a o s o gene a ion and a ia ion o can-
dida e solu ions a e a ious and a key aspec o keeping he balance be ween explo a ion and
exploi a ion in a sea ch s a egy.
The e alua ion de ines how he i ness o he candida es is compu ed and which objec i e unc-
ion is u ilized. The e alua ion is he key aspec o any algo i hm, as i de ines he basis o any
in o ma ion gain and has a huge in luence on he sea ch s a egy. Fo black-box p oblems, he
e alua ion o candida es is he only op ion o exploi any p oblem in o ma ion. We di e en i-
a e be ween a di ec e alua ion o he objec i e and indi ec e alua ion by using he p edic ed
i ness p o ided by a su oga e. How e alua ion is pe o med depends mainly on he unde ly-
ing p oblem and is la gely in luenced by he design o he objec i e unc ion. Impo an aspec s
in eal wo ld p oblems a e noise,cons ain s and mul iple objec i es.
While mos compu e expe imen s can be seen as de e minis ic, i.e., i e a ions using he same
alue se o he associa ed decision a iables should deli e he same esul s, eal-wo ld p ob-
lems a e o en non-de e minis ic. They include non-obse able dis u bance a iables and s ochas-
ic noise. Typical noise handling echniques include mul iple e alua ions o solu ions o educe
he s anda d de ia ion and special sampling echniques. The in e es ed eade can ind a su -
ey on noise handling by A nold and Beye (2003).
Mo eo e , many eal-wo ld p oblems equen ly include di e en cons ain s, which need o
be conside ed du ing he op imiza ion p ocess. Cons ain handling echniques can be di ec ly
pa o he op imiza ion algo i hm, bu mos algo i hms a e designed o minimize he objec i e
unc ion and cons ain handling is added on op. Thus, i is o en in eg a ed by adjus ing he
i ness, e.g., by penal y e ms. Di e en echniques o cons ain handling a e discussed by
Coello (2002) and A nold and Hansen (2012).
The e alua ion o mul iple objec i es can include se e al co ela ed objec i e unc ions and
usually deli e s a se o non-domina ed solu ions, a so-called Pa e o-se (Naujoks, Beume, and
Emme ich, 2005). In his case, a so-called decision make is u ilized o compu e he i ness o a
15
solu ion and selec solu ions om he Pa e o-se (Fonseca, Fleming, e al., 1993).
The selec ion de ines he p inciple o choosing he solu ions which will be used in he nex
i e a ion. We use he e m selec ion, which has i ’s o igins in e olu iona y compu a ion. Beside
he simples s a egy o choosing he solu ion(s) wi h he bes i ness, ad anced selec ion s a e-
gies ha e eme ged, which a e pa icula p esen in me aheu is ics (Boussaïd e al., 2013). These
selec ion s a egy a e pa icula common in algo i hms wi h se e al candida es pe a ia ion
s ep, hus he mos sophis ica ed selec ion me hods we e in oduced in he scope o e olu-
iona y compu a ion (Eiben and Smi h, 2015). A common s a egy is based on ela i e i ness
compa isons, so-called anked selec ion. De ailed examples o selec ion s a egies a e gi en in
Sec ion 6.2.
Con ol pa ame e s de e mine how he sea ch can be adap ed and imp o ed by con olling
he abo e men ioned key elemen s. We dis inguish be ween in e nal and ex e nal pa ame e s:
Ex e nal pa ame e s, also known as o line pa ame e s, can be adjus ed by he use and need
o be se a p io i o he op imiza ion un. Typical ex e nal pa ame e s include he numbe
o candida es and se ings in luencing he abo e men ioned key elemen s. Beside common
heo y-based de aul s (Schwe el, 1993), hey a e usually se by ei he u ilizing a ailable domain
knowledge, ex ensi e a p io i benchma k expe imen s (Gämpe le, Mülle , and Koumou sakos,
2002), o educa ed guessing. Sophis ica ed uning me hods we e de eloped o exploi good
pa ame e se ings in an au oma ed ashion. Well known examples a e sequen ial pa ame e
uning (Ba z-Beiels ein e al., 2005), i e a ed acing o au oma ic algo i hm uning (López-
Ibáñez, Dubois-Lacos e, Cáce es, Bi a a i, and S ü zle, 2016), bonesa (Smi and Eiben, 2011) o
SMAC (Hu e , Hoos, and Ley on-B own, 2011).
In di e ence, in e nal pa ame e s a e no mean o be changed by he use . They a e ei he
ixed o a ce ain alue, which is usually based on physical cons an s o ex ensi e es ing by
he au ho s o he algo i hm, o a e sel -adap i e. Sel -adap ion o online con ol changes he
pa ame e s online du ing he sea ch p ocess on basis o he ga he ed knowledge o exploi ed
p oblem in o ma ion (Eiben e al., 1999) wi hou use in luence. Algo i hms using sel -adap i e
schemes hus end o gain ou s anding gene aliza ion abili ies and a e especially in e es ing
o black-box p oblems, whe e no in o ma ion abou he objec i e unc ion p ope ies is a ail-
able (Hansen, Mülle , and Koumou sakos, 2003). In gene al, he se ings o algo i hm con ol
pa ame e s di ec ly a ec he balance be ween explo a ion and exploi a ion du ing he sea ch
and a e c ucial o he pe o mance.
3.4 Fea u es o an In ui i e Taxonomy
Taxonomies a e o en based on he subjec i e au ho ’s expe ience and o me de ini ions, as
well as mo e impa ial ea u es and simila i ies. P io o es ablishing ou new axonomy, we
de ined a se i e essen ial classi ica ion ea u es (CF), which a e based on o me a ailable ax-
onomies as well as signi ican ea u es and simila i ies o CGO algo i hms. They a e in ended
o gi e a good unde s anding on how we sepa a ed ou classes o c ea e a dis inc axonomy,
which s ill emains comp ehensible and in ui i e. P io o each class desc ip ion we will ou -
line he hei ea u es in a ex box as exempli ied in Figu e 3. We will e e o ou classi ica ion
ea u es as CF-I o CF-V:
16
CF I) Use o In o ma ion:
The in o ma ion ea u e has ou possible ca ego ies: The i s ca ego y is memo yless. The
e m desc ibes algo i hms, which only use he a ailable in o ma ion o he p io i e a ion
(o ini ializa ion).
The second is explici memo y. I de ines hose algo i hms, which in o ma ion o p io
i e a ions in a di ec ashion, e.g., by main aining an a chi e o all obse a ions.
Thi d is implici memo y; hese algo i hms combine in o ma ion o se e al i e a ions and
solu ions by ope a o s, unc ions o models.
Finally, algo i hms which equi e a p io i in o ma ion abou he objec i e unc ion, such
as he alue o he op imum.
CF II) Candida e E alua ion:
The candida e e alua ion ea u e de ines i he objec i e unc ion alue is only di ec ly
calcula ed o also indi ec ly app oxima ed. The app oxima ion o he i ness du ing he
candida e a ia ion phase can g ea ly lowe he necessa y amoun o objec i e unc ion
e alua ions, bu no op imiza ion p ocess can be eliable and success ul wi hou e i ica-
ion o hese candida es wi h he objec i e unc ion
CF III) Type o Candida e:
This ea u e e e s o he numbe and ype o candida e solu ions used in he a ia ion and
main ained in each i e a ion. I has h ee ca ego ies:
The i s is single and implies ha he a ia ion is based on a single candida e solu ion.
Mo eo e , hese algo i hms main ain only a single solu ion o hei nex i e a ion.
The second ype is popula ion, whe e in each i e a ion he a ia ion is based on se e al
candida e solu ions and mo eo e , se e al solu ions a e s o ed o he nex i e a ion.
The mos sophis ica ed ype a e model-based algo i hms, which u ilize a candida e dis-
ibu ion model o he a ia ion, which is s o ed and adap ed in each i e a ion. The
candida e e alua ion is no a ec ed and emains di ec .
CF IV) Region o Sea ch:
This ea u e desc ibes he e ec i e sea ch egion o an algo i hm.
Local algo i hms ha e no ope a o s o unc ions o explo a ion and a e hus no capable
o escaping he so-called egion o a ac ion o an op imum.
Global algo i hms ha e he abili y o ind op ima in mul i-modal landscapes by in oduc-
ing ope a o s o unc ions o balance explo a ion and exploi a ion.
CF V) P oblem P ope ies:
This ea u e is a collec ion o objec i e unc ion p ope ies. Algo i hms which a e e icien
in sol ing a p oblem wi h he gi en p ope y a e assigned wi h he ela ed ea u e.
•Domain Knowledge: This ea u e desc ibes p oblems wi h known unc ion p ope -
ies, such as a ma hema ical p oblem o mula ion o in o ma ion abou he numbe
and objec i e unc ion alue o op ima. This knowledge can be exploi ed and used
o a e icien o exac sea ch p ocess.
•Unimodal: This e m desc ibes objec i e unc ions wi h a single op imum in a linea
o con ex sea ch space.
17
•Mul imodal: P oblems a e called mul imodal when hey ha e se e al local and/o
global op ima.
•Black-Box: P oblems a e called black-box i hey do no p o ide any domain knowl-
edge and all in o ma ion needs o be ga he ed by objec i e unc ion e alua ions.
Many eal-wo ld p oblems can be associa ed wi h his cha ac e is ic.
•Discon inui ies: Func ions who ha e jumps in hei objec i e unc ion alues a e
discon inuous and no di e en iable.
•Noisy: Noisy objec i e unc ion do no e u n de e minis ic unc ion alues. Mul-
iple e alua ions o he same candida e on a noisy unc ion can lead o di e en
esul s.
•Mul i-Objec i e: Mul i-Objec i e p oblems ha e se e al (compe ing) objec i es.
•Expensi e: Expensi e p oblems ha e a high cos o each unc ion e alua ion in
e ms o ei he physical esou ces o compu a ion ime.
Classi ica ion Fea u es:
I Use o In o ma ion: a p io i, memo yless, explici /implici memo y
II Candida e E alua ion: di ec , indi ec
III Type o Candida e: single, popula ion, model
IV Sea ch Space: local, global
V P oblem P ope ies: unimodal, mul imodal, black-box, ...
Figu e 3: Fundamen al ea u es o algo i hm classi ica ion
Fo an o e iew o all classes and he connec ed classi ica ion ea u es CF I-IV, we u ilize a
decision ee, illus a ed in Figu e 4. In his igu e, we u ilized he ea u es I o IV as classi i-
ca ion nodes which conclude in ou new axonomy wi h he in oduced classes exac ,wande e ,
guide and ca og aphe . Mo eo e , he associa ed p oblem p ope ies a e displayed. The igu e is
in ended o p o ide a as o e iew on how he new axonomy wo ks. New algo i hms could
be easily included by assigning he abo e lis ed ea u es and hen using he illus a ed decision
ee. We know ha no all algo i hms will i in o ou axonomy, as hey ha e p ope y com-
bina ions, which a e no displayed in ou scheme. They may also belong o he hyb id class,
which ep esen combina ions o me hods om he displayed classes. The hyb id class is no
shown in he igu e, as i has no dis inc p ope ies.
18
Wande e
IV: Region o
Sea ch
global
Global Me hods
Simula ed
Annealing
local
Local Me hods
Line Sea ch,
Quasi-New on,
I e a i e Hill Climbe
unimodal, mul imodal
memo yless
no memo y o p io
obse a ions
Exac Guide Ca og aphe
I: Use o
In o ma ion
Exac Me hods
G id Sea ch
B anch and Bound
explici o implici
memo y o p io
obse a ions
II: Candida e
E alua ion
di ec e alua ion;
u ilizing objec i e
unc ion
III: Type o
Candida e
indi ec e alua ion;
u ilizing su oga e o
objec i e unc ion
Su oga e-based
Efficien Global
Op imiza ion,
Bayesian
Op imiza ion,
Sequen ial Pa ame e
Op imiza ion
model
Model-based
Es ima ion o
Dis ibu ion
Algo i hms,
Co a iance Ma ix
Adap ion- E olu ion
S a egy, An
Colony Op imiza ion
single
Single-solu ion
Tabu-Sea ch,
Va iable
Neighbo hood
Sea ch
popula ion
Popula ion-based
E olu iona y
Algo i hms, Pa icle
Swa m Op imiza ion
domain
knowledge
black-box, noisy, discon inui ies, mul i-objec i e expensi e
Objec i e
Func ion
Class Subclass
a p io i
in o ma ion abou
p oblem p ope ies
Algo i hm complexi y inc eases, objec i e unc ion cha ac e is ics add up
Figu e 4: Algo i hmic classi ica ion including he classes exac , wande e , guide, and ca og aphe . The de ined
classi ica ion ea u es I o IV a e used as nodes o a decision ee wi h he subclasses and ela ed example algo-
i hms as he inal lea es. Below he subclass he co esponding main classes and he associa ed objec i e unc ion
cha ac e is ics a e illus a ed. The objec i e unc ion cha ac e is ics a e displayed bo om-up, i.e., he displayed
cha ac e is ics a e adding up and ge ing mo e sophis ica ed om le o igh .
19

4 The Exac Class
Exac Class:
I Use o In o ma ion: a p io i
II Candida e E alua ion: di ec
III Type o Candida e: single
IV Sea ch Space: global
V P oblem P ope ies: unimodal,mul imodal,domain knowledge
The gene al desc ip ion o exac algo i hms was p esen ed in Sec ion 2.3. Fo an e icien sea ch,
exac algo i hms need a p io i in o ma ion abou he objec i e unc ion. The e o e, hey a e
only sui able o a limi ed class o p oblems whe e his in o ma ion is a ailable. Mo eo e , he
applica ion o exac algo i hms is always a adeo decision be ween compu a ion ime and
p ecision.
Fo example: Gi en a nonde e minis ic polynomial (NP)-ha d p oblem, usually no exac algo-
i hm exis s ha is able o ind he bes solu ion in polynomial ime. The a eling salesman
p oblem is a common combina o ial NP-ha d p oblem, whe e he goal is o ind he minimal
leng h ou h ough a ixed numbe o ci ies. An exac algo i hm could sol e his p oblem by
calcula ing e e y possible ou and selec ing he bes . Despi e p o iding he bes solu ions, his
s a egy would use a lo o compu a ion ime. Pa icula i he numbe o ci ies ge s la ge, his
p oblem canno be sol ed exac in easonable ime.
I should be no ed ha e en i exac algo i hms a e no e icien o NP-ha d and black-box
p oblems, hey a e commonly pa o su oga e-based op imiza ion amewo ks, because he
models p o ide he equi ed in o ma ion o an exac sea ch o a e compu a ionally e y cheap
o e alua e (see Sec ion 2.4 and Figu e 1). Fo example, e icien global op imiza ion (Sec ion 7.2.1
)and bayesian op imiza ion (Sec ion 7.2.2) can use exac algo i hms.
Two common me hods om he amily o exac me hods a e g id sea ch and b anch-and-bound.
G id sea ch combines a mul i-s a local op imiza ion wi h an inc easingly sub le sampling
g id o s a ing poin s. B anch-and-bound op imiza ion is conduc ed by spli ing he o igi-
nal p oblem ecu si ely in o subp oblems wi h he goal o excluding o sol ing hem, un il
i is gua an eed ha no subp oblem can lead o a be e solu ion. I is known as b anch-and-
bound (Lawle and Wood, 1966) as lowe bounds on he objec i e unc ion a e compu ed. Mo e
complex exac sea ch me hods combine b anching and local op imiza ion, Lipschi zian op i-
miza ion, con exi y o in e al analysis (Neumaie , 2004; Floudas, 2013; Hansen and Wals e ,
2003; Ho s and Tuy, 2013).
Example 4.1 (Di iding Rec angles).An example o an exac algo i hm is di iding ec angles
(DIRECT), ini ially p oposed by Jones, Pe unen, and S uckman (1993) as a modi ica ion o
Lipschi zian op imiza ion. While assuming ha he objec i e unc ion is Lipschi z con inuous,
he algo i hm does no need a speci ica ion o he Lipschi z cons an , as i is sampled du ing
he op imiza ion un. The algo i hm uses hype cubes o di ide he sea ch space. The cen e
cio each hype cube is sampled and gi en a i ness, based on he objec i e unc ion alue and
he size o he associa ed hype cube. Based on his i ness, he hype cube which is mos likely
o inhe i he op imum is selec ed and u he di ided in o smalle hype cubes. Then hei
i ness is sampled and he p ocess epea s, un il a s opping c i e ion is me o he algo i hm has
con e ged.
20
5 The Wande e Class
The wande e class encompasses algo i hms wi h non-complex sea ch s a egies which gene -
a e and main ain a single candida e pe i e a ion. New candida es a e gene a ed in he icini y
o he cu en solu ion by a s ochas ic p ocess which is independen o p e ious sea ch s eps.
The consecu i e candida es desc ibe a ajec o y in he sea ch space ha o ms in he ideal case
a di ec line o he op imum. In hei sea ch, hey only use he local objec i e unc ion in o ma-
ion abou he p io solu ion. Thus, hese algo i hms do no use global in o ma ion abou he
p oblem in he a ia ion o selec ion s eps.
Analogy 1 (The Wande e ).
The in ui i e desc ip ion o a wande e is a single indi idual who wande s h ough he land-
scape o ind he mos a ac i e place in a gi en a ea. Du ing i s sea ch i only u ilizes local
in o ma ion abou i s cu en posi ion o ind he bes di ec ion. I he goal o his indi idual is
o ind he highes moun ain, i will likely ollow he ascending way, because i di ec ly sa is ies
he cu en objec i e. I does no memo ize ga he ed in o ma ion, so ha he e is a chance ha
i will ci cle a posi ion, e isi a place o ge s comple ely los .
We sepa a e be ween he local and global subclass o he wande e : While he local wande e
keeps a g eedy selec ion, he global wande e also allows he accep ance o non-imp o ing can-
dida es.
5.1 Local Wande e
Local Wande e :
I Use o In o ma ion: memo yless
II Candida e E alua ion: di ec
III Type o Candida e: single
IV Sea ch Space: local
V P oblem P ope ies: unimodal
The local wande e subclass consis s o basic local op imiza ion algo i hms, which include clas-
sical g adien -based algo i hms as well as de e minis ic o s ochas ic hill-climbing algo i hms.
These algo i hms a e designed o as con e gence o a local op imum si ua ed in a egion o
a ac ion A ⊆ S and ha e no explici s a egy o explo a ion.
G adien -based me hods, auch as quasi-New on Me hods (Shanno, 1970), di ec ly compu e o
app oxima e he g adien s o he objec i e unc ion o ind he s eepes di ec ion o he op i-
mum. Di ec -Sea ch me hods pe o m an i e a i e and g adien - ee sea ch by using a minimal
amoun o in o ma ion abou he objec i e unc ion. O e iews o di ec sea ch me hods we e
p esen ed by Lewis, To czon, and T osse (2000) and Kolda e al. (2003). Mo eo e , he (1+1)
e olu ion s a egy wi a basic selec ion ope a o ( u he explained in Sec ion 6.2) can be associ-
a ed wi h his class.
Example 5.1 (I e a ed S ochas ic Hill-Climbe ).The i e a ed s ochas ic hill-climbe (Michalewicz
and Fogel, 2013) is a ypical example o he local wande e subclass. I has a elemen a y al-
go i hm design, whe e he sea ch a ia ion is s ochas ic and he selec ion ypically g eedy. In
each i e a ion o he algo i hm a new candida e x is c ea ed by sampling om a p obabili y
dis ibu ion Da ound he p io solu ion x −1. The a iance o he dis ibu ion, which is o en
21
uni o m o Gaussian, de ines he so-called s ep size o he a ia ion and is he mos impo an
con ol pa ame e o his algo i hm. The g eedy selec ion wo ks as ollows: I he new candida e
x has a be e i ness alue han he p io solu ion x −1, i is accep ed as new solu ion and he
i e a ion is epea ed. I he i ness is no imp o ed, he p io solu ion is kep o he nex i e -
a ion. This g eedy selec ion scheme is based on a compa a i e, anking-based selec ion o he
candida es wi h no in luence o he absolu e di e ence in he objec i e unc ion alues. This
implies in a iance o linea ans o ma ions and scaling o he objec i e unc ion. As he basic
scheme includes no ope a o o escape om a local op imum o explo a ion, his algo i hm will
mos likely con e ge o a local op imum. The e mina ion condi ion is usually se o a numbe
o non-imp o ing i e a ions, a e which is assumed ha an op imum has been ound. The
gene alized design o he i e a ed hill-climbe is ou lined in Algo i hm 5.1.
Algo i hm 5.1: I e a ed Hill-Climbe
1begin
2 = 0
3ini ialize x wi h ( andom) candida e x∈ S
4e alua e x
5while no e mina ion-condi ion do
6 = + 1
7sample new candida e x om p obabili y dis ibu ion Da ound he cu en
solu ion x −1
8e alua e candida e x
9i new solu ion imp o es he i ness y <y −1 hen
10 accep new solu ion
11 else
12 keep old solu ion x =x −1
13 adjus a ia ion s ep size/ a iance o p obabili y dis ibu ion (op ional)
14 end
15 end

As he name implies, he local wande e is in he i s place sui able o unimodal unc ions o
o exploi local op ima. I can be applied o global op imiza ion o mul imodal landscapes i
an adequa e mul i-s a s a egy is used. These mul i-s a s a egies ypically demand a high
numbe o unc ion e alua ions and a e only easonable o be used o p oblems wi h ela i ely
cheap objec i e unc ions, such as in su oga e-based op imiza ion. Theo e ically, a local wan-
de e could es ablish a global sea ch by sampling he candida es om a dis ibu ion wi h a
dispe sion o e he comple e sea ch space, which is equi alen o a e y la ge s ep size o he
a ia ion. This s a egy would be ine icien as i leads o andom sampling o candida es wi h-
ou conside ing any local in o ma ion and di ec ion o sea ch. Thus, i is common o limi he
dispe sion o he p obabili y dis ibu ion o he icini y o he cu en solu ion, o o m a small
neighbo hood which is e y small compa ed o he comple e sea ch space. This leads o he
ou lined hill-climbing sea ch s a egy which pe o ms a ajec o y o small, i ness-imp o ing
s eps. The maximal s ep size is consequen ly an impo an con ol pa ame e , which is o en
designed as being adap i e in many e sions o he algo i hm (c . mu a ion ope a o in Sec ion
22
6.2). In gene al, local wande e s a e o en pa o sophis ica ed algo i hms as as con e ging
local sea ch s a egy.
5.2 Global Wande e
Global Wande e :
I Use o In o ma ion: memo yless
II Candida e E alua ion: di ec
III Type o Candida e: single
IV Sea ch Space: global
V P oblem P ope ies: mul imodal
The global wande e subclass encompasses algo i hms which implemen ope a o s o balance
explo a ion and exploi a ion in he selec ion p ocess o candida es. They di e om local wan-
de e s by hei explo a i e sea ch s a egies, which u he enable global op imiza ion. Simi-
la o he local e sion, he global wande e u ilizes a s ochas ic a ia ion o candida es in he
neighbo hood o he cu en solu ion wi hou conside ing s o ed o modeled global in o ma-
ion. Explo a ion is achie ed by in oducing ope a o s o unc ions which allow o expand he
sea ch space and escape he egion o a ac ion o a local op imum. The well-known simula ed
annealing (SANN) will be used o exempli ying his app oach.
Example 5.2 (Simula ed Annealing).Ki kpa ick e al. (1983) in oduced SANN as a sea ch p o-
cedu e o global combina o ial op imiza ion. I is known o be a signi ican con ibu ion o he
ield o me aheu is ic sea ch algo i hms. The basic sea ch p ocedu e o e e y SANN me hod
is inspi ed by annealing in me allu gy, whe e a ma e ial is i s hea ed, hen cooled. Following
his analogy, he mos impo an elemen o con ol he sea ch is an adap i e pa ame e : he
empe a u e (T). Analogous o he he modynamic ene gy in a hea ed ma e ial, i inc eases he
possible mo emen o he candida es.
The con inuous e sion (Go e, Fe ie , and Roge s, 1994; Sia y, Be hiau, Du din, and Haussy,
1997; Van G oenigen and S ein, 1998) o he SANN algo i hm basically ex ends he i e a ed
s ochas ic hill climbe . I includes a new elemen o allow global sea ch, he so-called accep ance
unc ion P(x −1,x , T). The accep ance unc ion is used du ing he selec ion and de e mines he
p obabili y o accep ing an in e io candida e as solu ion by u ilizing T as a pa ame e . This
dynamic selec ion allows o escape local op ima s eps by accep ing mo emen in he opposi e
di ec ion o imp o emen , which is he undamen al di e ence o a hill climbe . A common
example o an accep ance unc ion is he so-called Me opolis unc ion:
P(x −1,x , T) = min 1,exp − (x )− (x −1)
T (1)
The Me opolis unc ion always accep s i ness imp o ing s eps owa ds he minimum ( (x )−
(x −1)≤0) and mo eo e , has a p obabili y o accep ascending ( (x )− (x −1)>0) s eps
based on T. Highe T alues hus inc ease he p obabili y o accep an in e io candida e. In
con as o ank-based selec ion schemes, his unc ion is scale-based and u ilizes absolu e di -
e ences in he i ness, which ende s i sensi i e o linea ans o ma ions o he objec i e unc-
ion, e.g., a mul iplica ion by a scala .
A he end o each i e a ion, a so-called cooling ope a o Cadap s T. This ope a o can be used
o balance explo a ion and exploi a ion (Hende son, Jacobson, and Johnson, 2003). A common
23
and λo sp ing candida es acco dingly o hei anked i ness, whe eby he op µsolu ions a e
kep o he nex gene a ion. In he (µ, λ)-s a egy all pa en s µa e disca ded and he su i o s
a e selec ed only om he o sp ing λacco ding o hei anked i ness. This equi es o c ea e
λ≥µcandida es in he ecombina ion s ep o p e en ex inc ion o he popula ion. Fu he , in
age-based selec ion s a egies, each solu ion only su i es a de ined numbe o i e a ions be o e
i ge s disca ded. This is o en combined wi h i ness-based selec ion, as in he (µ, λ)-s a egy,
whe e each solu ion su i es only one i e a ion. Aging adds a handicap o old domina ing so-
lu ions which su i ed se e al gene a ions. This depic s an explo a i e s a egy o escape local
op ima and inc ease he di e si y o he popula ion. 
EAs a e e y lexible in hei implemen a ion and adap able by uning. They a e obus and
sui able o sol e a la ge class o p oblems, including mul imodal, mul i-objec i e, dynamic
and black-box p oblems, e en wi h noise o discon inui ies in he i ness landscape (Jin and
B anke, 2005; Ma le and A o a, 2004). Fu he , hey ha e success ully been applied o a la ge
numbe o di e en indus ial p oblems (Fleming and Pu shouse, 2002) bu ypically equi e
a ela i ely la ge numbe o unc ion e alua ions o con e ge. This makes hem no he i s
choice o expensi e p oblems whe e he numbe o e alua ions is s ongly limi ed.
The lexibili y and obus ness o EAs is caused by di e en mechanisms and s a egies o con-
olling he balance be ween explo a ion and exploi a ion. A good o e iew is p esen ed in
he su ey by ˇ
C epinšek, Liu, and Me nik (2013). The de ailed su ey classi ies he di e en
a ailable e olu iona y app oaches and p esen s an in ensi e discussion which mechanisms
in luence explo a ion and exploi a ion. Theo e ical aspec s o e olu iona y ope a o s a e dis-
cussed by Beye (2013). The pe o mance o EAs is in luenced by pa ame e uning and con ol,
e.g., he se ing o popula ion size, mu a ion s eng h and selec ion p obabili y. An ex ensi e
o e iew o he di e en on- and o line uning app oaches o pa ame e con ol in EAs was
published by Eiben e al. (1999).
Fu he common s a egies o con olling explo a ion and exploi a ion and mul imodal op i-
miza ion a e so-called niching s a egies, which u ilize sub-popula ions o main ain he di e -
si y o he popula ion, in es iga e se e al egions o he sea ch space in pa allel o conduc
de ined asks o explo ing and exploi ing (Shi and Bäck, 2005; Filipiak and Lipinski, 2014).
6.3 Model-based Guide
Model-based Guide
I Use o In o ma ion: implici memo y by he model/dis ibu ion
II Candida e E alua ion: di ec
III Type o Candida e: model/dis ibu ion
IV Sea ch Space: global
V P oblem P ope ies: mul imodal, black-box, noisy, discon inui ies, mul i-objec i e
The model-based guide class encompasses algo i hms, which explici ly u ilize ma hema ical o
s a is ical models. The dis inc ion o su oga e-based op imiza ion is based on he assump ion,
ha hese models do no depic di ec app oxima ions o he unde lying objec i e unc ion and
o no a ge o model he comple e i ness landscape. Ins ead, hey a e specialized local models
which a e used in he a ia ion . They a e no explici ly sea ched o op ima, al hough o en he
mean o he dis ibu ion is u ilized as p edic ed op imal solu ion.
30

A common class o model-based algo i hms a e es ima ion o dis ibu ion algo i hms (EDA) (La -
añaga and Lozano, 2001). They gene ally belong o he la ge ield o EAs (Sec ion 6.2). The
main di e ence o EAs is ha he a ia ion ope a o s, such as ecombina ion o mu a ion, a e
no di ec ly applied o he candida es, bu o dis ibu ion models. The dis ibu ion models a e
build using in o ma ion o p io e alua ed popula ions. Di e en dis ibu ion models can be
u ilized, e.g., Bayesian ne wo ks o mul i a ia e Gaussian dis ibu ions. Fu he common ex-
amples o model-based algo i hms a e he co a iance ma ix adap ion - e olu ion s a egy (CMA-
ES) (Hansen e al., 2003) and an colony op imiza ion o con inuous domains ACOR(Do igo e al.,
2006; Socha and Do igo, 2008). ACORis based on he social beha io o an colonies and hei
communica ion ia phe omones. An s sea ch o ood a ound hei nes in a andom manne
and lea e ails o phe omones on hei way o ma k and enable o he an s o ollow hese
ails. The idea o hese phe omone ails a e ans e ed o use special dis ibu ion models
and a ia ion ope a o s. As he gene al idea o model-based algo i hms is simila , we will ou -
line hei mechanisms on he example o a gene alized EDA and gi e u he insigh on he
pa icula i ies o ACORand he CMA-ES.
Example 6.3 (Es ima ion o Dis ibu ion Algo i hms).The gene al idea behind he dis ibu-
ion model is ha i is bene icial o lea n he s uc u al in o ma ion o he unde lying popula-
ion (La añaga and Lozano, 2001; Hauschild and Pelikan, 2011). The s uc u al in o ma ion
allows o acqui e quali y knowledge abou he dependences among he a iables. Mo eo e ,
his in o ma ion is used o gene a e new candida es and hus, o guide he sea ch o he op i-
mum. A gene al EDA is ou lined in Algo i hm 6.3.
Algo i hm 6.3: Es ima ion o Dis ibu ion Algo i hm
1begin
2 = 1
3ini ialize µ andom popula ion P ={x ,1:µ}⊆S
4e alua e µ andom candida es P
5while no e mina ion-condi ion do
6selec pa en s om popula ion P∗
⊆ P
7es ima e dis ibu ion pa ame e s o pa en s P∗
, such as mean, a iance and
co a iance
8mu a e o adap pa ame e s o dis ibu ion G
9sample dis ibu ion unc ion G o ge λo sp ing O={x∗
,1:λ}
10 e alua e o sp ing
11 selec su i o s om P ∪ O o nex gene a ion P +1
12 = + 1
13 end
14 end
The algo i hm s a s wi h popula ion o µsolu ions, ei he ini ialized a andom o based on
exposed p oblem knowledge. Simila o an EA, pa en s a e selec ed, ypically based on anking
selec ion. I is common o use he i es solu ions as pa en s (c . selec ion in EAs, Sec 6.2).
Ins ead o a di ec ecombina ion o mu a ion, he pa en s a e used o cons uc a dis ibu ion
model. Fo example, ypical pa ame e s o a mul i a ia e Gaussian dis ibu ion, such as mean,
a iance and co a iance o he selec ed pa en popula ion a e compu ed. In ACORGaussian
31
ke nel unc ions o each dimension o he decision a iable space a e cons uc ed. The dis i-
bu ion pa ame e s a e hen a ge o he a ia ion ope a o s. This a ia ion is he c ucial aspec
o he sea ch s a egy, simila o he ecombina ion and mu a ion in he basic EA. Fo exam-
ple, he CMA-ES adap s he pa ame e s du ing each i e a ion ollowing he his o y o p io
success ul i e a ions, he so-called e olu ion pa hs. These e olu ion pa hs a e basically exponen-
ially smoo hed sums o each dis ibu ion pa ame e o e he consecu i e p io i e a i e s eps
(c . adap i e mu a ion s eng h in Sec ion 6.2). They hus u ilize he in o ma ion o se e al
success ul sea ch s eps, which is in ended o quickly app oach an op imum.
In he nex s ep, a numbe o λo sp ing candida es a e gene a ed by sampling he adap ed
dis ibu ion. A e wa ds he candida es a e e alua ed and su i o s o he nex gene a ion
a e selec ed, again based on su i o selec ion ope a o s, as explained in (Sec ion 6.2). The
CMA-ES uses only ank-based selec ion schemes, which makes i insensi i e o scaling o he
objec i e unc ion. 
In gene al, model-based guides y o combine he bene i s o s a is ical models and hei ca-
pabili y o s o ing and p ocessing in o ma ion wi h popula ion-based sea ch ope a o s. They
a e high le el me aheu is ics and ad anced EAs which in ended o be lexible, obus and ap-
plicable o a la ge class o p oblems, pa icula hose wi h unknown unc ion p ope ies. This
makes hem e y success ul in popula black-box benchma ks (Hansen e al., 2010a). Fo ex-
ample, he design o he CMA-ES is seeks o make he algo i hm pe o mance obus and no
dependen on he objec i e unc ion o uning. The a ious con ol pa ame e s o he algo i hm
we e p e-de ined on basis o heo e ical aspec s and p ac ical benchma ks.
7 The Ca og aphe Class
Ca og aphe Class:
I Use o In o ma ion: implici /explici memo y by su oga e and solu ion a chi e
II Candida e E alua ion: indi ec
III Type o Candida e: single
IV Sea ch Space: global
V P oblem P ope ies: mul imodal, black-box, noisy, expensi e
Ca og aphe algo i hms di e om all o he de ined classes in hei ocus on acqui ing, ga he -
ing and u ilizing global in o ma ion abou he i ness landscape. They u ilize p io e alua ed
candida es and model he acqui ed in o ma ion o p edic he i ness o new candida es. These
models a e hen used o an e icien indi ec sea ch in which ypically a single new candida e
is p oposed in each i e a ion, ins ead o pe o ming mul iple, di ec and localized sea ch s eps.
Analogy 3 (The Ca og aphe ).
The in ui i e idea o he ca og aphe is a specialis who sys ema ically measu es a landscape
by aking samples o he heigh o c ea e a opological map. This map esembles he eal land-
scape wi h a gi en app oxima ion accu acy and is ypically exac a he sampled loca ions (i
he measu emen s a e wi hou a iance) and models he emaining landscape by eg ession.
I can hen be examined and u ilized by any o he indi idual, such as a wande e o guide, o
ind a desi ed loca ion. One could hink o a guide using a pape map o na iga ion sys em o
ind he place o in e es .
32
As illus a ed in Sec ion 2.4, he su oga es s(x)depic he maps o he i ness landscape o an
objec i e unc ion ( 1(x)o 2(x)) in an algo i hmic amewo k. In his Sec ion, we will i s
gi e a b ie in oduc ion o common su oga e models and hen ou line ypical ca og aphe
amewo ks and hei sea ch s a egies.
7.1 Su oga e Models
The su oga e is he co e elemen o any su oga e-based op imiza ion and essen ial o hei
pe o mance. A pe ec su oga e p o ides an excellen i o p io obse a ions, whils ideally
possessing supe io in e pola ion and ex apola ion abili ies. Howe e , he la ge numbe o
a ailable su oga e models all ha e signi ican di e ing cha ac e is ics, ad an ages and disad-
an ages. Model selec ion is hus a complica ed and di icul ask. I no domain knowledge is
a ailable, such as in eal black box op imiza ion, i is o en ine i able o es di e en su oga es
o hei applicabili y.
Su oga es a e buil on basis o p io obse a ions, which p o ide in o ma ion abou he i -
ness landscape o he p oblem. Thus, he ini ial candida es a e commonly selec ed ollowing
di e en in o ma ion c i e ia and sui able expe imen al design. Fo example, linea eg ession
models can be build wi h ac o ial designs, while Gaussian p ocess models a e bes coupled
wi h space- illing designs, such as La in hype cube sampling (Mon gome y, Mon gome y, and
Mon gome y, 1984; Sacks e al., 1989).
Common models a e: linea , quad a ic o polynomial eg ession, Gaussian p ocesses (also
known as K iging) (Sacks e al., 1989; Fo es e , Sobes e , and Keane, 2008), eg ession ees
(B eiman, F iedman, S one, and Olshen, 1984), a i icial neu al ne wo ks and adial basis unc-
ion ne wo ks (Haykin, 2004; Ho nik, S inchcombe, and Whi e, 1989) including deep lea n-
ing ne wo ks (Collobe and Wes on, 2008; Hin on, Deng, Yu, Dahl, Mohamed, Jai ly, Senio ,
Vanhoucke, Nguyen, Saina h, e al., 2012; Hin on, Osinde o, and Teh, 2006) and symbolic e-
g ession models (Augus o and Ba bosa, 2000; Flasch, Me smann, and Ba z-Beiels ein, 2010;
McKay, Willis, and Ba on, 1995), which a e usually op imized by gene ic p og amming (Koza,
1992).
Fu he , a lo o e o in cu en s udies is o esea ch he bene i s o model ensembles, which
combine se e al dis inc models (Goel, Ha ka, Shyy, and Queipo, 2007; Mülle and Shoemake ,
2014; F iese, Ba z-Beiels ein, and Emme ich, 2016). The goal is o c ea e a sophis ica ed p e-
dic o ha su passes he pe o mance o a single model. A well-known example a e andom
o es s (F eund and Schapi e, 1997), which use bagging o i a la ge numbe o decision ees
(B eiman, 2001). We ega d ensemble modeling as s a e-o - he-a o cu en esea ch, as hey
a e able o combine he ad an ages o di e en models o gene a e ou s anding esul s in bo h
classi ica ion and eg ession. The d awback o hese ensemble me hodologies is ha hey a e
compu a ional expensi e and pose a demanding p oblem in ega d o e icien model selec ion,
e alua ion and combina ion.
7.2 Ca og aphe Algo i hms
Ca og aphe algo i hms a e su oga e-based op imiza ion me hodologies, which explici ly
use a su oga e in hei op imiza ion cycle, ollowing he gene al p inciple ou lined in Sec ion
2.4. They a e ei he ixed algo i hms designed a ound a ce ain model, such as K iging in he
well-known e icien global op imiza ion (EGO) (Jones e al., 1998), o amewo ks wi h a choice o
33
possible su oga es and op imiza ion me hods. We p esen wo common amewo ks and dis-
cuss hei pa icula i ies: gene al bayesian op imiza ion (Mockus, 1974) and sequen ial pa ame e
op imiza ion (Ba z-Beiels ein e al., 2005; Ba z-Beiels ein, 2010).
Fo es e and Keane (2009) and Ba z-Beiels ein and Zae e e (2017) gi e o e iews o su oga e-
based op imiza ion, di e en su oga e models and in ill c i e ia. Mo eo e , hey ma ch su o-
ga es o p oblem classes and gi e hin s abou hei indi idual applicabili y. In gene al, he se-
lec ion o an adequa e model, expe imen al design and op imize equi es bo h domain knowl-
edge and expe ise. We will ocus on he abo e-men ioned amewo ks as hey deli e a good,
ye no comple e, iew o he su oga e-based sea ch s a egy.
7.2.1 E icien Global Op imiza ion
EGO (Jones e al., 1998) was mo i a ed by he u ge o de elop a me hodology o op imize
expensi e black-box unc ions. I u ilizes K iging su oga es and mo i a es he use expec ed
imp o emen as in ill c i e ia. In gene al, he algo i hm consis s o wo phases: i s , he ini ial-
iza ion by La in hype cube sampling and he cons uc ion o a K iging su oga e; second, he
i e a i e imp o emen o he bes solu ion u ilizing he su oga e. Be o e s a ing he second
phase, Jones e al. (1998) sugges s o analyze he su oga e model i . I he i is no sa is ac o y,
i can be ied o imp o e i by a uning o he model pa ame e s o ans o ma ion o he da a.
Algo i hm 7.1: E icien Global Op imiza ion
1begin
// phase 1: ini ial su oga e building
2 = 1
3ini ialize kcandida e solu ions {x1:k}by la in hype cube sampling
4e alua e hem on objec i e unc ion yi= 1(xi)o yi= 2(xi),1≤i≤k;
5build ini ial K iging su oga e s wi h ini ial obse a ions D ={(xi,yi),1≤i≤k}
6analyze and imp o e model i (op ional)
// phase 2: use and upda e su oga e
7while no e mina ion-condi ion do
8i >1 hen
9upda e he K iging su oga e s wi h he se o all obse a ions D
10 end
11 calcula e expec ed imp o emen in ill c i e ia on su oga e s
12 op imize EI o maximum by b anch and bound; use op imum as candida e x
13 e alua e x on he objec i e unc ion y = 1(x )o y = 2(x )
14 add new solu ions o he se o all obse a ions D +1 ={D ,(x ,y )}
15 = + 1
16 end
17 end
The second phase s a s he i e a i e op imiza ion p ocess, as desc ibed in Sec ion 2.4. Du -
ing he a ia ion, a new candida e is sea ched by op imizing he expec ed imp o emen in ill
c i e ia o he K iging su oga e. Fo his op imiza ion, he exac b anch and bound me hod is
used (c . exac class, Sec ion 4). Expec ed imp o emen is mo i a ed as in ill c i e ia, because i
gua an ees a balance o explo a ion and exploi a ion by u ilizing bo h he p edic ed bes mean
34
alue o he model, as well as he model unce ain y. An example o he comple e me hod-
ology is ou lined in Algo i hm 7.1. The sea ch s a egy o EGO is a undamen al example o
mos su oga e-based op imiza ion which a e applicable o expensi e op imiza ion p oblems.
Howe e , we sugges o use ad anced amewo ks based on his base e sion o EGO. These
amewo ks a e mo e lexible and applicable o a la ge class o p oblem.
7.2.2 Bayesian Op imiza ion
The e m Bayesian op imiza ion (BO) was in oduced by Mockus (1974, 1994, 2012) and de-
sc ibes no a single, bu a scheme o algo i hms, which we ega d as su oga e-based, pa ic-
ula hese based on Gaussian p ocesses. While he gene al BO scheme hus emains simila
o he ou lined algo i hm in Sec ion 2.4, BO di e s in he unde lying e minology: In BO, he
uses selec s and ini ial, so-called p io dis ibu ion, which should suppo he a p io i belie s
abou he unde lying unknown objec i e unc ion. Gaussian dis ibu ions a e sugges ed and a
common choice. Algo i hm 7.2 displays a gene al BO algo i hm.
Algo i hm 7.2: Bayesian Op imiza ion
1begin
// phase 1: ini ializa ion
2 = 1
3selec p io dis ibu ion s −1(x)
4ini ialize kcandida e solu ions {x1:k}
5e alua e hem on objec i e unc ion yi= 1(xi)o yi= 2(xi),1≤i≤k
6upda e p io wi h D ={(xi,yi),1≤i≤k} o ge ini ial pos e io dis ibu ion s (x);
// phase 2: op imiza ion
7while no e mina ion-condi ion do
8i >1 hen
9upda e pos e io dis ibu ion s wi h he se o all obse a ions D
10 end
11 calcula e in ill unc ion on pos e io dis ibu ion s (x)
12 op imize in ill c i e ion; use op imum as candida e x
13 e alua e x on he objec i e unc ion y = 1(x )o y = 2(x )
14 add new solu ions o he se o all obse a ions D +1 ={D ,(x ,y )}
15 = + 1
16 end
17 end
This p io dis ibu ion is upda ed by sampled obse a ions o acqui e he pos e io dis ibu-
ion. The op imiza ion cycle includes he op imiza ion o he acquisi ion (o in ill) unc ion o
maximize u ili y o minimize isk ( i ness). Typical choices include he p obabili y o imp o e-
men (Kushne , 1964), expec ed imp o emen (Jones e al., 1998) and con idence bounds (Cox
and John, 1997). Algo i hms such as EGO can be seen as applied a ian s o BO. I is widely
applicable o di e en applica ions, including expensi e op imiza ion p oblems (Lizo e, 2008;
Khan, Goldbe g, and Pelikan, 2002) and machine lea ning (Snoek, La ochelle, and Adams,
2012; Swe sky, Snoek, and Adams, 2013). B ochu, Co a, and De F ei as (2010) gi e a u o ial on
BO wi h di e en applica ion examples.
35

7.2.3 Sequen ial Pa ame e Op imiza ion Toolbox
The sequen ial pa ame e op imiza ion oolbox (SPOT), de eloped by Ba z-Beiels ein (2010), is
a dynamic su oga e-based op imiza ion amewo k, which was ini ially in ended o o line
uning o algo i hm con ol pa ame e s. Va ious me hods o ini ial sampling designs, di e en
models and op imiza ion echniques a e included.
SPOT is s ongly in luenced by s a is ical me hods om design o expe imen s, whe e i is
a emp ed o p o e a ce ain s a is ical hypo hesis on basis o es ing. He eby, he a ailable
budge o expe imen s (i.e., unc ion e alua ions) is used sequen ially o imp o e a solu ion
and upda e he su oga e. This is done un il su icien knowledge abou he sea ch space is
a ailable o accep o ejec he ini ial s a ed hypo hesis.
The o e all design was dedica ed o algo i hm uning and ollows wo goals: one was imp o -
ing he e iciency o an algo i hm, i.e., disco e ing he algo i hm pa ame e s o sol e a de ined
p oblem ins ance as as as possible. The o he goal was imp o ing he obus ness o an iden-
i ied pa ame e se up, i.e., o sol ing di e en p oblem ins ances which o example di e
in hei egion o in e es o sea ch space dimensionali y. To une s ochas ic algo i hms, SPOT
in eg a es noise handling echniques by dynamic e-sampling o solu ions. This design can be
ans e ed o gene al su oga e-based op imiza ion, as he me hods ackle he p esen balanc-
ing p oblem o explo a ion ( obus ness) and exploi a ion (e iciency).
The gene al amewo k o SPOT is simila o he gene al su oga e-based op imiza ion algo-
i hms EGO and BO, which a e di ided in ini ializa ion and i e a i e op imiza ion phases.
SPOT also explici ly includes a p io pa ame iza ion phase, whe e he uses has o choose he
su oga e and ini ial sampling design. SPOT de ines a lexible amewo k and is hus appli-
cable o a la ge ange o p oblems, such as he men ioned algo i hm uning and indus ial
op imiza ion.
8 The Hyb id Class
Hyb id Class:
I Use o In o ma ion:implici /explici memo y by su oga e o candida e a chi e
II Candida e E alua ion: di ec /indi ec
III Type o Candida e: single ,popula ion,dis ibu ion model
IV Sea ch Space: global
V P oblem P ope ies: mul imodal, black-box, noisy, discon inui ies, mul i-objec i e, ex-
pensi e
The hyb id class depic s combina ions o algo i hms om he p e iously men ioned classes.
Hyb id algo i hms we e de eloped as a s a egy o imp o e o ackle indi idual algo i hm
weaknesses. The algo i hms a e o en gi en dis inc i e oles o explo a ion and exploi a ion,
as hey a e combina ions o an explo a i e global sea ch me hod pai ed wi h a local sea ch
algo i hm. Fo example, popula ion-based algo i hms wi h ema kable explo a ion abili ies
can be pai ed wi h local algo i hms wi h as con e gence. This app oach gi es some bene i s,
as he combined algo i hms can be adap ed o uned o ul ill hei dis inc asks. Mo eo e ,
he concep s can be easily adap ed o pa allel amewo ks.
One o he mos success ul ype o hyb ids a e he su oga e-assis ed e olu iona y algo i hms (Em-
me ich, Giannakoglou, and Naujoks, 2006; Lim, Jin, Ong, and Sendho , 2010). An o e iew
36
o su oga e-assis ed op imiza ion is gi en by Jin (2011), including se e al examples o eal-
wo ld applica ions. An example is ou lined in Algo i hm 8.1.
Algo i hm 8.1: Su oga e-Assis ed E olu iona y Algo i hm
1begin
2 =0
3ini ialize µ andom popula ion P ={x ,1:µ}⊆S
4e alua e popula ion P
5while no e mina ion-condi ion do
6 un e olu iona y algo i hm o ind λo sp ing O ={xi, µ < i ≤µ+λ}
7build su oga e s (x)( wi h cu en obse a ions D ={(xi,yi),1≤i≤k})
8p edic i ness o o sp ing O using su oga e s (x)
9selec λ∗bes o sp ing O∗
based on p edic ed i ness
10 e alua e selec ed o sp ing O∗
wi h objec i e unc ion
11 un local op imize om x∈ O∗
as s a ing solu ion o ge e ined solu ions O∗∗
(op ional)
12 selec su i o s om P ∪ O∗
∪ O∗∗
o nex gene a ion P +1
13 = + 1
14 end
15 end
Example 8.1 (Su oga e-Assis ed E olu iona y Algo i hm).In his hyb id sea ch s a egy, a
local su oga e is buil upon he cu en pa en popula ion and u ilized o p edic he i ness o
a numbe o λo sp ing candida es. The selec ion is hen based on he p edic ed i ness o he
su oga e. Op ionally, a local op imize can be used o u he e ine he compu ed solu ions.
Ex ensi e use o local sea ch leads o a as con e gence o local op ima. This hyb id s a egy
can be al e ed by using he su oga e only o a pa o he gene a ed o sp ing, while he o he
pa is e alua ed wi h he eal i ness unc ion.
Addi ional examples o hyb ids can be ound in he li e a u e, co e ing all possible class com-
bina ions:
Meme ic algo i hms, as de ined by Mosca o e al. (1989), a e a class o sea ch me hods which
combine popula ion-based guides wi h a local wande e . An ex ensi e o e iew o meme ic
algo i hms is gi en by Molina, Lozano, Ga cía-Ma ínez, and He e a (2010). They desc ibe
how di e en hyb id algo i hms can be cons uc ed by looking a sui able local sea ch algo-
i hms wi h special ega d o hei con e gence abili ies.
Ba z-Beiels ein, P euss, and Rudolph (2006) desc ibe a hyb id app oach an e olu ion s a egy
o explo a ion wi h quasi-New on me hod o exploi a ion. The algo i hm uns he ES and he
local sea ch in a consecu i e way and he budge o e alua ions o each me hod is con olled
by a con ol pa ame e . They pe o med expe imen s in which hey a ied he budge pa am-
e e o es i his hyb id app oach can be supe io o unning bo h me hods indi idually. They
came o he conclusion, ha hyb idiza ion can be bene icial o di icul objec i e unc ions, as
he ES p o ides in o ma ion abou in e es ing egions whe e he local sea ch is hen applied.
The su oga e managemen amewo k (Booke e al., 1999; Se a ini, 1999) u ilizes a combina ion
o a global su oga e-based algo i hm wi h an exac local pa e n sea ch. Mo eo e , i uses
i ness space ans o ma ions om con inuous o combina o ial by in oducing a ini e mesh
37
o possible solu ions. The key concep is o lowe he op imiza ion cos s by educing he eal
unc ion e alua ions using a su oga e, while a he same ime e aining he bene i s o a com-
bina o ial sea ch space and pa e n sea ch, i.e., he obus con e gence beha io . The algo i hm
u ilizes wo dis inc global and local sea ch phases, which a e execu ed du ing he sequen ial
op imiza ion. In he global sea ch s ep he selec ed in ill c i e ion (e.g. expec ed imp o emen )
is op imized in con inuous space and he nea es mesh poin is selec ed. In he local poll s ep,
a se o candida es si ua ed on he mesh a ound he cu en bes solu ion is e alua ed di ec ly
on he objec i e.
Taddy, Lee, G ay, and G i in (2009) combine su oga e-based op imiza ion based on eed Gaus-
sian p ocesses(TGP) wi h exac asynch onous pa allel pa e n sea ch (APPS) in a pa allel sea ch
amewo k. The algo i hm s a s wi h a space illing ini ial sampling using a la in hype cube
design, hen uns TGP and APPS in pa allel. In his case, TGP is used o p edic a anked lis
o a ixed numbe o new candida es, while APPS is used o pe o m local op imiza ion uns.
The budge o e alua ion and compu a ion is spli be ween hese wo componen s and all
obse a ions a e s o ed in an sha ed a chi e.
Hyb id algo i hms a e applicable o a la ge class o p oblems, de ined by which class hei
algo i hms o igina e om. Thei downside is hei la ge complexi y and he isk, ha hei
highe complexi y does no lead o imp o ed pe o mance, due o he di icul balancing and
equi ed uning o he dis inc algo i hms. Thei complex sea ch s a egies wi h a la ge numbe
o con ol pa ame e can make hem di icul o une. The algo i hm i sel becomes a black-box,
as he unde lying sea ch s a egy and he con e gence beha io is in luenced by nume ous
ope a o s and di icul o comp ehend.
9 Concluding Rema ks
In his wo k, we p esen ed an o e iew o con inous global op imiza ion algo i hms wi h ocus
on explaining hei sea ch s a egies using a new in ui i e axonomy. We de ined a se o i e
classes: exac , wande e , guide, ca og aphe , and hyb id in he Sec ions 4 o 8 and ou lined hei
indi idual p ope ies and exempli ied algo i hms o each o he p oposed classes.
To sho ly ecapi ula e: he exac class u ilizes (a p io i) p oblem in o ma ion o sol e a p ob-
lem wi h a gua an ee o inding he op imum. The heu is ic sea ch s a egies o he class o
wande e algo i hms a e sui able o as con e gence in a unimodal sea ch space and o en pa
o o he algo i hms. The well-known me aheu is ics om he guide class we e de eloped o
gene al applicabili y, pa icula o mul i-modal p oblems wi h unknown p ope ies. The ca -
og aphe class ocuses on su oga e-based algo i hms and ega ded amewo ks o p oblems
wi h expensi e unc ion e alua ions. Las bu no leas , we ook a look a hyb id class algo-
i hms, which y o combine he s eng hs o di e en algo i hms o o e come hei indi idual
weaknesses.
In gene al, i is bene icial o each use o iden i y i an op imiza ion algo i hm is sui able o
hei p oblem be o e applying hem. To suppo use s in selec ing a sui able algo i hm, we
poin ed ou he p os and cons o he di e en sea ch s a egies, he indi idual algo i hm ea-
u es and ypical cha ac e is ics o CGO p oblems hey a e able o handle.
A his poin , we also wan o highligh a new p omising esea ch a ea in he ield o algo i hms,
which is au oma ed algo i hm selec ion and pa icula au oma ic algo i hm con igu a ion. Bo h ideas
ackle he p oblem o selec ing he co ec sea ch s a egy o a gi en p oblem. Au oma ed
algo i hm selec ion ies o ind he mos sui able algo i hm o a ce ain p oblem based on
38
machine lea ning and exploi ed p oblem in o ma ion, such as explo a i e landscape analysis.
This me hod o algo i hm selec ion has shown o be able o ou pe o m a single algo i hm on
a se o benchma k unc ions (Ke schke and T au mann, 2017). An e en mo e p omising e-
sul was p esen ed by an Rijn, Wang, an S ein, and Bäck (2017), whe e algo i hm con igu a ion
was used o selec algo i hmic componen s o c ea ing a sea ch s a egy ou pe o ming a ail-
able algo i hms. The pa icula in e es ing idea is he e, ha sea ch ope a o s o algo i hms a e
iden i ied, ex ac ed and hen again combined o a new sea ch s a egy. The whole p ocedu e
also shows he s ong connec ions be ween di e en named algo i hms, pa icula in he a ea
o bio-inspi ed me aheu is ics.
In e es ing challenges o u u e algo i hm design a ise om p oblems in enginee ing applica-
ions, whe e he da a a ailable is es ic ed o ce ain condi ions, such as s eaming and online
da a and dynamic p oblems. The need o new op imiza ion app oaches eme ges om apid
de elopmen o communica ing senso s and machines in he ield o enginee ing, also known
as in e ne o hings (A zo i, Ie a, and Mo abi o, 2010). Sui able op imiza ion algo i hms need o
be di ec ly included in he p oduc ion cycle, adap ing o gene a e obus solu ions in challeng-
ing dynamic en i onmen s wi h mo ing op ima. Dynamic, su oga e-based online lea ning,
whe e a complex s a ic su oga e is cons uc ed and combined wi h ime- a ying modeling, is
s ill an open issue (Jin and B anke, 2005). Nowadays, cloud compu ing and high-pe o mance
compu ing clus e s a e a ailable o a wide ange o use s. Many op imiza ion algo i hms a e
no ye i ed o he needs o pa allel compu a ion and need o be adap ed Rehbach, Zae e e ,
S o k, and Ba z-Beiels ein (2018). The la ge and success ul ield o deep lea ning ne wo ks (Le-
Cun, Bengio, and Hin on, 2015; Schmidhube , 2015) decla es a comple e new ield om which
e y complex and di icul op imiza ion p oblems a ise. The ex ension o su oga e-based op i-
miza ion o hese ields, e.g., pa allel amewo ks and deep lea ning, is an in e es ing esea ch
opic. Fu he , we iden i y a lack in he ield o ealis ic benchma ks, which a e based on eal-
wo ld da a se s, which would allow a ealis ic compa ison o di e en algo i hmic app oaches.
10 Acknowledgemen s
Thanks o all ou colleagues o help ul p oo eading, ema ks and w i ing sugges ions. This
wo k is pa o a p ojec ha has ecei ed unding om he Eu opean Union’s Ho izon 2020
esea ch and inno a ion p og am unde g an ag eemen no. 692286.
Re e ences
F. A che i and F. Schoen. A su ey on he global op imiza ion p oblem: gene al heo y and
compu a ional app oaches. Annals o Ope a ions Resea ch, 1(2):87–110, 1984.
D. V. A nold and H.-G. Beye . A compa ison o e olu ion s a egies wi h o he di ec sea ch
me hods in he p esence o noise. Compu a ional Op imiza ion and Applica ions, 24(1):135–159,
2003.
D. V. A nold and N. Hansen. A (1+ 1)-cma-es o cons ained op imisa ion. In P oceedings o he
14 h annual con e ence on Gene ic and e olu iona y compu a ion, pages 297–304. ACM, 2012.
J. A o a, O. Elwakeil, A. Chahande, and C. Hsieh. Global op imiza ion me hods o enginee ing
applica ions: a e iew. S uc u al op imiza ion, 9(3-4):137–159, 1995.
39
I. Rechenbe g. E olu ionss a egie–Op imie ung echnische Sys eme nach P inzipien de biologischen
E olu ion. F ommann-Holzboog, 1973.
I. Rechenbe g. E olu ionss a egie ’94. F ommann-Holzboog, 1994.
F. Rehbach, M. Zae e e , J. S o k, and T. Ba z-Beiels ein. Compa ison o pa allel su oga e-
assis ed op imiza ion app oaches. In P oceedings o he Gene ic and E olu iona y Compu a ion
Con e ence, pages 1348–1355. ACM, 2018.
G. Rozenbe g, T. Bck, and J. N. Kok. Handbook o na u al compu ing. Sp inge Publishing Com-
pany, Inco po a ed, 2011.
J. Sacks, W. J. Welch, T. J. Mi chell, and H. P. Wynn. Design and analysis o compu e expe i-
men s. S a is ical science, pages 409–423, 1989.
J. Schmidhube . Deep lea ning in neu al ne wo ks: An o e iew. Neu al ne wo ks, 61:85–117,
2015.
M. Schonlau. Compu e Expe imen s and Global Op imiza ion. PhD hesis, Uni e si y o Wa e loo,
On a io, Canada, 1997.
H.-P. Schwe el. Nume ische Op imie ung on Compu e -Modellen mi els de E olu ionss a egie:
mi eine e gleichenden Ein üh ung in die Hill-Climbing-und Zu allss a egie. Bi khäuse , 1977.
H.-P. P. Schwe el. E olu ion and op imum seeking: he six h gene a ion. John Wiley & Sons, Inc.,
1993.
D. B. Se a ini. A amewo k o managing models in nonlinea op imiza ion o compu a ionally expen-
si e unc ions. PhD hesis, Rice Uni e si y, 1999.
D. F. Shanno. Condi ioning o quasi-new on me hods o unc ion minimiza ion. Ma hema ics
o compu a ion, 24(111):647–656, 1970.
Y. Shi and R. Ebe ha . A modi ied pa icle swa m op imize . In E olu iona y Compu a ion P o-
ceedings, 1998. IEEE Wo ld Cong ess on Compu a ional In elligence., The 1998 IEEE In e na ional
Con e ence on, pages 69–73. IEEE, 1998.
O. M. Shi and T. Bäck. Niching in e olu ion s a egies. In P oceedings o he 7 h annual con e ence
on Gene ic and e olu iona y compu a ion, pages 915–916. ACM, 2005.
P. Sia y and G. Be hiau. Fi ing o abu sea ch o op imize unc ions o con inuous a iables.
In e na ional jou nal o nume ical me hods in enginee ing, 40(13):2449–2457, 1997.
P. Sia y, G. Be hiau, F. Du din, and J. Haussy. Enhanced simula ed annealing o globally min-
imizing unc ions o many-con inuous a iables. ACM T ansac ions on Ma hema ical So wa e
(TOMS), 23(2):209–228, 1997.
S. Smi and A. Eiben. Mul i-p oblem pa ame e uning using bonesa. In A i icial E olu ion,
pages 222–233, 2011.
J. Snoek, H. La ochelle, and R. P. Adams. P ac ical bayesian op imiza ion o machine lea ning
algo i hms. In Ad ances in neu al in o ma ion p ocessing sys ems, pages 2951–2959, 2012.
46

K. Socha and M. Do igo. An colony op imiza ion o con inuous domains. Eu opean jou nal o
ope a ional esea ch, 185(3):1155–1173, 2008.
J. Sønde gaa d, K. Madsen, and H. B. Nielsen. Op imiza ion using su oga e models-by he Space
Mapping echnique. PhD hesis, Technical Uni e si y o Denma k, 2003.
K. Swe sky, J. Snoek, and R. P. Adams. Mul i- ask bayesian op imiza ion. In Ad ances in neu al
in o ma ion p ocessing sys ems, pages 2004–2012, 2013.
M. A. Taddy, H. K. Lee, G. A. G ay, and J. D. G i in. Bayesian guided pa e n sea ch o obus
local op imiza ion. Technome ics, 51(4):389–401, 2009.
E.-G. Talbi. Me aheu is ics: om design o implemen a ion, olume 74. John Wiley & Sons, 2009.
A. Tö n and A. Zilinskas. Global Op imiza ion. Sp inge , 1989.
J. Van G oenigen and A. S ein. Cons ained op imiza ion o spa ial sampling using con inuous
simula ed annealing. Jou nal o En i onmen al Quali y, 27(5):1078–1086, 1998.
S. an Rijn, H. Wang, B. an S ein, and T. Bäck. Algo i hm con igu a ion da a mining o cma
e olu ion s a egies. In P oceedings o he Gene ic and E olu iona y Compu a ion Con e ence,
GECCO ’17, pages 737–744, New Yo k, NY, USA, 2017. ACM. ISBN 978-1-4503-4920-8. doi:
10.1145/3071178.3071205. URL h p://doi.acm.o g/10.1145/3071178.3071205.
D. H. Wolpe and W. G. Mac eady. No F ee Lunch Theo ems o Op imiza ion. IEEE T ansac-
ions on E olu iona y Compu a ion, 1(1):67–82, Ap . 1997.
K. S. Won and T. Ray. Pe o mance o k iging and cok iging based su oga e models wi hin he
uni ied amewo k o su oga e assis ed op imiza ion. In E olu iona y Compu a ion, 2004.
CEC2004. Cong ess on, olume 2, pages 1577–1585. IEEE, 2004.
S. H. Zanakis and J. R. E ans. Heu is ic “Op imiza ion”: Why, When, and
How o Use I . In e aces, 11(5):84–91, 1981. doi: 10.1287/in e.11.5.84. URL
h p://dx.doi.o g/10.1287/in e.11.5.84.
A. Žilinskas. A e iew o s a is ical models o global op imiza ion. Jou nal o Global Op imiza-
ion, 2(2):145–153, 1992.
M. Zlochin, M. Bi a a i, N. Meuleau, and M. Do igo. Model-based sea ch o combina o ial
op imiza ion: A c i ical su ey. Annals o Ope a ions Resea ch, 131(1-4):373–395, 2004.
47
Kon ak /Imp essum
Diese Ve ö en lichungen e scheinen im Rahmen de Sch i en eihe "CIplus". Alle Ve ö -
en lichungen diese Reihe können un e
h ps://cos.bibl. h-koeln.de/home
abge u en we den.
Die Ve an wo ung ü den Inhal diese Ve ö en lichung lieg beim Au o .
Da um de Ve ö en lichung: 31.08.2018
He ausgebe / Edi o ship
P o . D . Thomas Ba z-Beiels ein,
P o . D . Wol gang Konen,
P o . D . Bo is Naujoks,
P o . D . Ho s S enzel
Ins i u e o Compu e Science,
Facul y o Compu e Science and Enginee ing Science,
TH Köln,
S einmülle allee 1,
51643 Gumme sbach
u l:
www.ciplus- esea ch.de
Sch i lei ung und Ansp echpa ne / Con ac edi o ’s office
P o . D . Thomas Ba z-Beiels ein,
Ins i u e o Compu e Science,
Facul y o Compu e Science and Enginee ing Science,
TH Köln,
S einmülle allee 1, 51643 Gumme sbach
phone: +49 2261 8196 6391
u l:
h p://www.spo se en.de
eMail: homas.ba z-beiels ein@ h-koeln.de
ISSN (online) 2194-2870