CIplus
Band 6/2016
F om Real Wo ld Da a o Tes Func ions
And eas Fischbach, Ma in Zae e e , Jö g S o k, Ma ina F iese,
Thomas Ba z-Beiels ein
F om Real Wo ld Da a o Tes Func ions
And eas Fischbach, Ma in Zae e e , Jö g S o k, Ma ina F iese,
Thomas Ba z-Beiels ein
SPOTSe en Lab, Dep . o Comp. Sci. and Eng. Sci.
TH Köln
E-Mail: {and eas. ischbach, ma in.zae e e , joe g.s o k, ma ina. iese,
homas.ba z-beiels ein}@ h-koeln.de
www.spo se en.de
1 In oduc ion
When esea che s and p ac i ione s in he ield o compu a ional in elligence
a e con on ed wi h eal-wo ld p oblems, he ques ion a ises which me hod
is he bes o apply. Nowadays, he e a e se e al, well es ablished es
sui es and well known a i icial benchma k unc ions a ailable. Howe e ,
ele ance and applicabili y o hese me hods o eal-wo ld p oblems emains
an open ques ion in many si ua ions. Fu he mo e, he gene alizabili y o
hese me hods canno be aken o g an ed. Some p elimina y ideas abou
gene alizabili y a e discussed in [1, 2].
This pape desc ibes a da a-d i en app oach o he gene a ion o es
ins ances, based on eal-wo ld da a, as depic ed in Figu e 1. The es
ins ance gene a ion uses da a-p ep ocessing, ea u e ex ac ion, modeling,
and pa ame e iza ion. I was applied o se e al eal-wo ld scena ios, e.g., in
he con ex o gene ic p og amming [
3
]. In his wo k we apply his concep
on a classical design o expe imen eal-wo ld p ojec and gene a e es
ins ances o benchma king, i.e., design o expe imen me hods and model
i ness. Bu i can also be used o compa e and analyze se e al su oga e
echniques and op imiza ion algo i hms as well.
In mos cases, complex and expensi e eal-wo ld p oblems do no p o ide
su icien da a o compa ison o me hods. Thus, ou goal is ou goal is o
c ea e a oolbox con aining mul iple da a se s o eal-wo ld p ojec s. Wi h
ha oolbox, esea che s a e g an ed access o bo h he da a se s and he
de i ed es unc ions.
This wo k mainly ocuses on he ollowing ques ions:
P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016 1
C ea eini ialdesign
E alua eobjec i e unc ion
Buildsu oga emodel
E alua emodel i ness
Real‐wo ldDa ase
C ea epa ame e izedmodel
Gene a e es ins ances
2
3
4
5
1
Figu e 1: Simpli ied es p ocess o su oga e models i ed upon e alua ion on
ini ial designs wi h eal-wo ld da a based es ins ances. An op ional
alida ion s ep can be added be o e he es ins ances a e used in s ep 3.
(Q-1)
Wha is he cha ac e is ic o a a ia ion o a ce ain model pa ame e
in e ms o he models i ness landscape?
(Q-2)
How can simila i y be ween models be compu ed and wha a e use ul
h esholds o sepa a e simila models om almos equal models on
he one hand, and comple ely di e en models on he o he hand?
Conside ing an example applica ion o he es unc ion gene a o , he
design o expe imen used o ga he he e e ence da a se used in his wo k
will be u he analyzed:
(Q-3)
Which design o expe imen wo ks bes o he unde lying eal-wo ld
p oblem?
The emainde o his wo k is o ganized as ollows. Sec ion 2 gi es a
li e a u e e iew. Sec ion 3 desc ibes he p ocess o he es unc ion
gene a ion and applied modeling echniques as well as model simila i y
measu es used. Sec ion 4 illus a es he e e ence da a se and he example
applica ion, he e alua ion o design o expe imen me hods. Sec ion 5
concludes he wo k and gi es a sho ou look o u u e wo k.
2P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016
2 Rela ed Wo k
In he esea ch ield o benchma king eal-pa ame e p oblems many con i-
bu ions add ess he andom gene a ion o p oblem ins ances based on use
de ined pa ame e s, like he Max-Se o Gaussian Landscape Gene a o [
4
],
o he K igi ie [5], bo h using Gaussian P ocesses.
The Max-Se o Gaussian Landscape Gene a o compu es he uppe en-
elope o
𝑚
weigh ed Gaussian p ocess ealiza ions and can be used o
gene a e con inuous, bound-cons ained op imiza ion p oblems. The land-
scape gene a o is pa ame e ized o con ol, e.g., he numbe o Gaussian
componen s and implici ly, he numbe o local op ima, he occu ences
and a ia ions o hills and he peaks, and he global op imum.
The K igi ie ealizes a p ocedu e o gene a ing nonlinea objec i e unc-
ions. I s idea is based on he con enien supposi ion ha objec i e
unc ions a e ealiza ions o s ochas ic p ocesses. The use speci ies an
unde lying end, a s ochas ic p ocess and a ini e numbe o poin s a
which he p ocess will be obse ed. The K igi ie c ea es a noise e m and
uses he end and he noise e m o p oduce an objec i e unc ion.
A comple ely di e en app oach is add essed by he Real-Pa ame e Black-
Box Op imiza ion Benchma king [
6
]. The o ganize s o his benchma king
challenge choose and implemen a benchma king unc ion es bed, ypically
co e ing a i icial es unc ions like Sphe e, Rosenb ock, Ras igin, e c.
Pa icipan s hen ha e o apply hei black-box p oblem sol e s and hei
esul s will be ga he ed and compa ed.
All hese app oaches do no ely di ec ly on eal-wo ld p oblems. Ou wo k
ealizes one s ep owa ds closing a gap in means o alida ing me hods
ega ding hei applicabili y and gene alizabili y in p ac ical deploymen .
3 Tes Func ion Gene a ion
The simpli ied p ocess o eal-wo ld da a based es unc ion gene a ion is
depic ed in Figu e 2. I consis s o he ollowing s eps:
1.
P ocess a da a se o a eal-wo ld p oblem,
X
is he design ma ix
and
Y
is a ec o o co esponding ou come alues o he unde lying
p ocess.
P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016 3
Indus ial Da a
xi,y
i
xi,y
i
Le el 0 model
e.g. K iging o linea
model
gene a e
↵↵
Le el 1
model1
model1
↵1
↵1
Le el 1
model2
model2
↵2
↵2
Le el 1
modeln
modeln
↵n
↵n
de ine
bounds
gene a ion
Figu e 2: Resul ing hie a chy o he gene a ion p ocess o di e en models o be
used as es unc ion ins ances.
2.
Build a model, u he deno ed as le el 0 model, ha is sui able o
eg ession and in e pola ion pu poses o he da a. In his wo k he
K iging echnique is used o build he model.
3.
C ea e a pa ame e
𝛼
o a y he p e iously i ed le el 0 model. The
pa ame e
𝛼
is a scala o ec o , ha pe u bs he gene a ed model.
I may, e.g., de ine a change in pa ame e s o o he a iables o he
de i ed model. Fi s , bounds a e gene a ed o
𝛼
, ensu ing nume ical
obus ness. Then,
𝑛
ins ances
𝛼𝑖
wi h
𝑖
= 1
, ..., 𝑛
a e andomly
c ea ed wi hin he chosen bounds. Finally, a andomly selec ed
subse o he desi ed numbe o
𝑚 < 𝑛
ins ances is chosen. This
ensu es ha he ins ances which need o be sol ed by he e alua ed
me hods, a e ne e known in ad ance.
4.
Apply he selec ed
𝛼
ins ances on he le el 0 model o e ie e he
desi ed numbe o
𝑚
le el 1 model ins ances, each coupled wi h an
𝛼𝑗
,
𝑗
= 1
, ..., 𝑚
. The p ocess ensu es he ul illmen o he equi emen s
on he simila i y demands o he models by compu ing simila i y
measu es and disca ding in easible 𝛼ins ances.
The emainde o his sec ion desc ibes he K iging modeling echnique,
4P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016
he model a ia ion and he model simila i y e alua ion.
3.1 K iging
O en used o he pu pose o eg ession and in e pola ion, K iging is a
modeling me hod based on Gaussian p ocesses. In he ollowing we will
s ick closely o he desc ip ions by Fo es e e al. [
7
]. Fu he de ails can
be ound in hei book. Gi en a se o
𝑛
solu ions
X
=
{x(𝑖)}𝑖=1...𝑛
in a
𝑘
-dimensional con inuous sea ch space wi h obse a ions
y
=
{𝑦(𝑖)}𝑖=1...𝑛
,
K iging ies o de e mine an exp ession o a p edic ed alue a an unknown
loca ion by in e p e ing he obse a ions
y
as ealiza ions o a s ochas ic
p ocess. The s ochas ic p ocess is de ined by he se o andom ec o s
Y
=
{𝑌
(
x(𝑖)
)
}𝑖=1...𝑛
. The co ela ion o he andom a iables
𝑌
(
·
)is
modeled as ollows [7]:
co [︁𝑌(x(𝑖)), 𝑌 (x(𝑙))]︁= exp ⎛
⎝−
𝑘
∑︁
𝑗=1
𝜃𝑗|𝑥(𝑖)
𝑗−𝑥(𝑙)
𝑗|𝑝𝑗⎞
⎠.(1)
The ma ix ha collec s co ela ions o all pai s
{
(
𝑖, 𝑙
)
}
is called he co e-
la ion ma ix Ψ. I is used in he K iging p edic o
^𝑦(x) = ^𝜇+𝜓𝑇Ψ−1(y−1^𝜇),(2)
whe e
^𝑦
(
x
)is he p edic ed unc ion alue o a new sample
x
,
^𝜇
is he
maximum likelihood es ima e (MLE) o he mean and
𝜓
is he ec o o
co ela ions be ween aining samples
X
and he new sample
x
. The wid h
pa ame e
𝜃
=
(𝜃1, . . . , 𝜃𝑗, . . . , 𝜃𝑘)𝑇
de e mines how a he in luence o
each sample poin
x
sp eads. In de ail, he la ge he wid h pa ame e is,
he as e a e he po en ial changes in he p edic ed alue. The smalle he
wid h pa ame e is, he slowe a e he po en ial changes in he p edic ion.
Since he e is one
𝜃𝑖
o each dimension, his pa ame e can con ol he
ac i i y in each dimension. The pa ame e
𝑝𝑗
is usually ixed a
𝑝𝑗
=
2, and de ines he shape o he co ela ion unc ion: A
𝑝𝑗
= 2, he
co ela ion unc ion is mo e smoo h, whe eas
𝑝𝑗
= 1 is less smoo h. In
case o noise, he pa ame e
𝜆
is added o he diagonal o he co ela ion
ma ix
Ψ
. This allows he model a mo e smoo h i h ough obse a ions
( eg ession), in con as o he de aul which ep oduces all aining da a
exac ly (in e pola ion). Classically,
𝜆
is used o deal wi h noisy da a. Bu
i can as well be used o smoo hen mo e ugged i ness landscapes.
P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016 5
All model pa ame e s a e de e mined by Maximum Likelihood Es ima ion
(MLE). Fo 𝜃, 𝜆 and 𝑝, MLE equi es nume ical op imiza ion.
3.2 Model a ia ions
The main pa ame e s con olling he beha io o he model a e
𝜃
and
𝜆
. The majo goal o he es unc ion gene a o is he deploymen upon
eal-wo ld da a, which is usually noisy. So he a ia ion o he pa ame e
𝜆
is a na u al choice a a i s glance. In addi ion he a ia ion o he
wid h pa ame e
𝜃
seems impo an o change he model (sligh ly) by
main aining he gene al cha ac e is ic o he i ness landscape unde ce ain
ci cums ances. The bounds o he a ia ions o he pa ame e s ha e o be
de ined ca e ully, o he wise he model can show signs o degene a ion.
The es unc ion gene a o will compu e lowe and uppe bounds o
𝛼
acco ding o he i ed le el 0 model. The i s componen o
𝛼
ep esen s
he
𝜆
alue and he emaining componen s ep esen he co esponding
𝜃
alues. They will be added o hei co esponding alues o he le el
0 model o e ie e he al e ed le el 1 model. A e wa ds he co ela ion
Ma ix
Ψ
is ecalcula ed, so ha he changes ake e ec be o e p edic ions
a e made. The es unc ion gene a ion e u ns he desi ed numbe o
simila unc ions, andomly d awn om he sea ch space de ined by he
le el 0 model and he bounds o
𝛼
. I he gi en bounds do no allow he
c ea ion o su icien easible ins ances, an e o message is p oduced.
An example is shown in Fig. 3. He e, he one-dimensional unc ion
𝑓(𝑥)=(−18𝑥−2)2sin(20𝑥−4)
is i s sampled by 11 equidis an poin s. The de i ed le el 0 K iging model
has
𝜆
= 0 and
𝜃
= 100. To de i e he le el 1 models, he bounds o
𝑎𝑙𝑝ℎ𝑎
a e se o
𝑎𝑙𝑝ℎ𝑎𝑙𝑜𝑤
= [0
,−
90] and
𝑎𝑙𝑝ℎ𝑎ℎ𝑖𝑔ℎ
= [1
,
900] so ha
𝜆
will be se
o alues be ween ze o and one, and
𝜃
be ween en and one housand. Fo
demons a ion pu poses, he ex eme alues o
𝑎𝑙𝑝ℎ𝑎
a e chosen, as shown
abo e each plo in Fig. 3. The plo shows ha di e en
𝛼
alues a ec he
uggedness o he unc ion and a y he numbe o local op ima.
6P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016
0.0 0.2 0.4 0.6 0.8 1.0
−200 0 100 300
Le el 0 Model
x
(x)
−200 0 100 300
y (x)
0.0 0.2 0.4 0.6 0.8 1.0
−200 0 100 300
Le el 1 Model, α = [0.2,0]
x
(x)
−100 0 100 200
y (x)
0.0 0.2 0.4 0.6 0.8 1.0
−200 0 100 300
Le el 1 Model, α = [1,0]
x
(x)
−50 0 50 100
y (x)
0.0 0.2 0.4 0.6 0.8 1.0
−200 0 100 300
Le el 1 Model, α = [0.2,−90]
x
(x)
−10 0 10 20 30
y (x)
0.0 0.2 0.4 0.6 0.8 1.0
−200 0 100 300
Le el 1 Model, α = [1,−90]
x
(x)
0 5 10 15 20
y (x)
0.0 0.2 0.4 0.6 0.8 1.0
−200 0 100 300
Le el 1 Model, α = [1,900]
x
(x)
−50 0 50 100 150
y (x)
Figu e 3: A le el 0 K iging model and se e al de i ed le el 1 models. The solid
line is he ue unc ion 𝑓(𝑥), ci cles indica e obse a ions used o i
he le el 0 model and he dashed line indica es he le el 1 model ^𝑦(𝑥).
P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016 7
0.00
0.25
0.50
0.75
1.00
0.00 0.25 0.50 0.75 1.00
Lambda a ia ion
Measu e alue
0.00
0.25
0.50
0.75
1.00
−2 0 2 4
The a 1 a ia ion
Measu e alue
Measu e line ype
mae
pea son.
mse
spea man
. es .p
Measu e symbol
mae
pea son.
mse
spea man
. es .p
Figu e 4: E ec s o he a ia ion o model pa ame e s on he chosen simila i y
measu es MAE, RMSE, Pea sons , Spea man and he - es p alue.
Le : E ec s o 𝜆changes. Righ : E ec s o 𝜃1changes.
A de ailed look a he esul able om he sc eening expe imen was aken,
o decide which h esholds o each o he measu es should be aken o
disca d es unc ion ins ances. In he summa y o he measu es, depic ed
in Table 4, he dis ibu ion o he measu es can be seen. The co ela ion
coe icien s does no become e y low, wi h hei mean a a ound 0
.
95. The
p alue o he - es s dis ibu es on he whole scale om 0 o +1.
Table 4: Summa y o s a is ical measu es o simila i y ga he ed on sc eening
expe imen s o 𝛼.
mse pea son. spea man . es .p
Min. :0.0000 Min. :0.8732 Min. :0.8560 Min. :0.0000006
1s Qu.:0.1819 1s Qu.:0.9268 1s Qu.:0.9283 1s Qu.:0.0189555
Median :0.2430 Median :0.9524 Median :0.9564 Median :0.1431840
Mean :0.2376 Mean :0.9486 Mean :0.9506 Mean :0.3314180
3 d Qu.:0.2935 3 d Qu.:0.9729 3 d Qu.:0.9749 3 d Qu.:0.6384715
Max. :0.4120 Max. :1.0000 Max. :1.0000 Max. :1.0000000
A ull linea model wi h in e ac ion e ms be ween all pa ame e s and main
14 P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016
0.00
0.25
0.50
0.75
1.00
−2 0 2 4
The a 4 a ia ion
Measu e alue
Measu e line ype
mae
pea son.
mse
spea man
. es .p
Measu e symbol
mae
pea son.
mse
spea man
. es .p
Figu e 5: E ec s o he a ia ion o 𝜃4.
Table 5: Th esholds o measu es p e en ing oo simila and degene a ed es
unc ions.
mse pea son. spea man . es .p
lowe bound 0.2 0.85 0.85 0.5
uppe bound 0.35 0.92 0.92 0.9
e ec s up o second deg ee p edic ing he RMSE and Spea man co ela ion
coe icien e eals an adjus ed coe icien o de e mina ion o abou 0
.
85
and 0.74 espec i ely.
Fo he design o expe imen applica ion, he bounds o he measu es we e
se es shown in Table 5. Wi h hese h esholds 20 andom es unc ions
we e d awn. The impac o he chosen designs on he RMSE o he de i ed
linea models compa ed o he es unc ion a e shown in Figu e 6. I can
be seen ha he base design, applied o c ea e he aining da a in he
eal-wo ld p ojec , could no be ou pe o med by any o he design ype. A
a i s glance i migh be su p ising, ha designs wi h a la ge design size,
e.g. he Uni o m 81 o e en he LHD 81, pe o m wo se. Bu hese designs
we e no se up o i second o de linea eg ession models and would
he e o e no be he i s choice in such a se up. The esul s would su ely
P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016 15
base
FFD 2
FFD 3
LHD 16
LHD 67
LHD 81
Uni o m 16
Uni o m 67
Uni o m 81
0.00 0.05 0.10 0.15 0.20
RMSE
Design ype
Figu e 6: Resul ing RMSE alues o models based on di e en design o
expe imen me hods each applied on he same se o 20 andom es
unc ions.
look di e en i he le el 2 model would as well be a K iging model.
5 Conclusion
In his wo k he gene a ion p ocess o es unc ion ins ances based on a
eal-wo ld indus ial da a se is desc ibed. Based on his da a a K iging
model is i ed and al e ed acco ding o a a ia ion pa ame e
𝛼
, by adding
i s componen s o he co esponding model pa ame e . The gene a ed
ins ances will be disca ded o kep acco ding o h esholds o se e al
model simila i y measu es. This leads inally o a es unc ion ins ance
pool ha can be used o benchma k, i.e, design o expe imen and modeling
me hods, o hei p ac ical use on he unde lying p oblem.
The majo esea ch Ques ions add essed in his wo k we e:
(Q-1) Wha is he cha ac e is ic o a a ia ion o a ce ain model
pa ame e in e ms o he models i ness landscape?
The a ia ion o he K iging model pa ame e s
𝜆
and
𝜃
lead o
al e ed models ha we e s ill co ela ed a a he high coe icien
a es a ound 0
.
9. This was ensu ed by he de ini ion o lowe and
uppe bounds ela i e o he pa ame e alues o he base le el 0
16 P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016
model. This kind o limi he amoun o changes applicable o he
model and can be p oblema ic o example when a la ge numbe o
es ins ances is needed.
(Q-2) How can simila i y be ween models be compu ed and wha
a e use ul h esholds o sepa a e simila models om al-
mos equal models on he one hand, and comple ely di e -
en models on he o he hand?
This can kind o simple be isually analyzed o one o wo dimen-
sional p oblems, bu is ge ing ha de in gene al o la ge numbe
o dimensions. S a is ical measu es can help o dis inguish models
ha a e oo simila o oo di e en . Fu he analysis o he esul -
ing i ness landscapes in sense o , e.g., numbe o local op ima o
g adien s, can help o judge he simila i y o wo di e en models.
(Q-3) Which design o expe imen wo ks bes o he unde lying
eal-wo ld p oblem
The applica ion o di e en design o expe imen me hods has shown,
ha he base design, used o ga he he aining da a se , was no
ou pe o med by any o he design me hod. E en la ge designs like
a Full ac o ial designs a h ee le els pe ac o and La in hype cube
designs wi h 81 poin s could no domina e he esul s. This o cou se
has o be u he analyzed by al e ing he model echniques o he
le el 2 models. E en sligh changes o he base design can now be
analyzed and migh lead o in e es ing new design poin s o he
cus ome in u u e eal-wo ld expe imen s.
In e es ing u u e wo k include he shi o he simila i y compu a ion o
he beginning o he ins ance gene a ion. The Ma ix compu a ions o al e
K iging models can be ega ded as ime consuming, especially wi h high
dimensional da a. So i would be bene icial o compu e an expec a ional
alue o he simila i y o wo models.
In addi ion, i ness landscape analysis is an in e es ing opic o include
in he wo k. The i ness landscapes compa ison based on he RMSE o
co ela ions o unc ion alues can be ex ended by in e es ing ea u es o
add o o a oid, e.g., he numbe o local op ima, plane a eas o la ge
g adien s.
Finally, di e en a ia ion me hods on he models, e.g. o a ing, scaling,
dis o ion o he inpu space should be applied. Conside ing K iging models,
condi ional simula ion, could be applied o deli e possible ealiza ions o a
Gaussian p ocess wi h a ce ain p obabili y and he e o e be e y sui able
P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016 17
o deli e model a ia ions based on eal-wo ld da a.
Re e ences
[1]
M. Chia andini and Y. Goegebeu , “Mixed Models o he Analysis
o Op imiza ion Algo i hms,” in Expe imen al Me hods o he Analy-
sis o Op imiza ion Algo i hms (T. Ba z-Beiels ein, M. Chia andini,
L. Paque e, and M. P euss, eds.), pp. 225–264, Ge many: Sp inge ,
2010.
[2]
T. Ba z-Beiels ein, “How o C ea e Gene alizable Resul s,” in
Sp inge Handbook o Compu a ional In elligence, pp. 1127–1142,
Be lin, Heidelbe g: Sp inge Be lin Heidelbe g, 2015.
[3]
O. Flasch, “A Modula Gene ic P og amming Sys em,” May 2015.
Disse a ion zu E langung des G ades eines Dok o s de Ingenieu -
wissenscha en de Technischen Uni e si ä Do mund, Fakul ä ü
In o ma ik.
[4]
M. Gallaghe and B. Yuan, “A gene al-pu pose unable landscape
gene a o ,” IEEE T ansac ions on E olu iona y Compu a ion, ol. 10,
pp. 590–603, Oc . 2006.
[5]
M. W. T osse , I. o Compu e Applica ions in Science, and En-
ginee ing., The K igi ie [mic o o m] : a p ocedu e o gene a ing
pseudo andom nonlinea objec i e unc ions o compu a ional expe i-
men a ion / Michael W. T osse . Ins i u e o Compu e Applica ions
in Science and Enginee ing, NASA Langley Resea ch Cen e ; Na ional
Technical In o ma ion Se ice, dis ibu o Hamp on, VA : Sp ing ield,
VA, 1999.
[6]
N. Hansen, A. Auge , S. Finck, and R. Ros, “Real-pa ame e black-box
op imiza ion benchma king 2009: Expe imen al se up,” Tech. Rep.
RR-6828, INRIA, 2009.
[7]
A. Fo es e , A. Sobes e , and A. Keane, Enginee ing Design ia
Su oga e Modelling. Wiley, 2008.
[8]
T. Chai and R. R. R. D axle , “Roo mean squa e e o (RMSE) o
mean absolu e e o (MAE)? – A gumen s agains a oiding RMSE in
he li e a u e,” Geoscien i ic Model De elopmen , ol. 7, pp. 1247–1250,
June 2014.
18 P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016
[9]
M. A. Rebolledo Coy, S. K ey, T. Ba z-Beiels ein, O. Flasch, A. Fis-
chbach, and J. S o k, “Modeling and Op imiza ion o a Robus Gas
Senso ,” in Bioinspi ed Op imiza ion Me hods and hei Applica ions
(G. Papa and M. Me nik, eds.), pp. 267–278, May 2016.
[10]
R. Tibshi ani, “Reg ession sh inkage and selec ion ia he lasso,”
Jou nal o he Royal S a is ical Socie y Se ies B (Me hodolgical), ol. 58,
no. 1, pp. 267–288, 1996.
[11]
A. Fischbach, J. S o k, M. Zae e e , S. K ey, and T. Ba z-Beiels ein,
“Analyzing Capabili ies o La in Hype cube Designs Compa ed o Clas-
sical Expe imen al Design Me hods ,” in 25. Wo kshop Compu a ional
In elligence (F. Ho mann and E. Hülle meie , eds.), pp. 255–270, 2015.
P oc. 26. Wo kshop Compu a ional In elligence, Do mund, 24.-25.11.2016 19
Kon ak /Imp essum
Diese Ve ö en lichungen e scheinen im Rahmen de Sch i en eihe "CIplus". Alle Ve ö -
en lichungen diese Reihe können un e
h ps://cos.bibl. h-koeln.de/home
abge u en we den.
Köln, Janua 2012
Die Ve an wo ung ü den Inhal diese Ve ö en lichung lieg beim Au o .
Da um de Ve ö en lichung: 14.11.2016
He ausgebe / Edi o ship
P o . D . Thomas Ba z-Beiels ein,
P o . D . Wol gang Konen,
P o . D . Bo is Naujoks,
P o . D . Ho s S enzel
Ins i u e o Compu e Science,
Facul y o Compu e Science and Enginee ing Science,
TH Köln,
S einmülle allee 1,
51643 Gumme sbach
u l:
www.ciplus- esea ch.de
Sch i lei ung und Ansp echpa ne / Con ac edi o ’s office
P o . D . Thomas Ba z-Beiels ein,
Ins i u e o Compu e Science,
Facul y o Compu e Science and Enginee ing Science,
TH Köln,
S einmülle allee 1, 51643 Gumme sbach
phone: +49 2261 8196 6391
u l:
h p://www.spo se en.de
eMail: homas.ba z-beiels ein@ h-koeln.de
ISSN (online) 2194-2870
