CIplus
Band 8/2017
Su oga e-Assis ed Lea ning o Neu al
Ne wo ks
Jö g S o k, Ma in Zae e e , And eas Fischbach, F ede ik Rehbach,
Thomas Ba z-Beiels ein
Su oga e-Assis ed Lea ning o Neu al Ne wo ks
Jö g S o k, Ma in Zae e e , And eas Fischbach, F ede ik
Rehbach, Thomas Ba z-Beiels ein
SPOTSe en Labs, TH Köln
S einmülle allee 1, 51643 Gumme sbach
E-Mail: i s name.las name@ h-koeln.de
In oduc ion
Su oga e-assis ed op imiza ion has p o en o be e y success ul i applied
o indus ial p oblems. The use o a da a-d i en su oga e model o an
objec i e unc ion du ing an op imiza ion cycle has many bene i s, such
as being cheap o e alua e and u he p o iding bo h in o ma ion abou
he objec i e landscape and he pa ame e space. In p elimina y wo k, i
was esea ched how su oga e-assis ed op imiza ion can help o op imize
he s uc u e o a neu al ne wo k (NN) con olle [
7
]. In his wo k, we will
ocus on how su oga es can help o imp o e he di ec lea ning p ocess
o a anspa en eed- o wa d neu al ne wo k con olle . As an ini ial
case s udy we will conside a manageable eal-wo ld con ol ask: he
ele a o supe iso y g oup p oblem (ESGC) using a simpli ied simula ion
model [
3
]. We use his model as a benchma k which should indica e he
applicabili y and pe o mance o su oga e-assis ed op imiza ion o his
kind o asks. While he op imiza ion p ocess i sel is in his case no
conside ed expensi e, he esul s show ha su oga e-assis ed op imiza ion
is capable o ou pe o ming me aheu is ic op imiza ion me hods o a low
numbe o e alua ions. Fu he he su oga e can be used o signi icance
analysis o he inpu s and weigh ed connec ions o u he exploi p oblem
in o ma ion.
Mo i a ion
Recen ad ancemen s in obo ics and con ol ha e shown, ha me hods
om he ield o compu a ional in elligence a e becoming mo e and mo e
P oc. 27. Wo kshop Compu a ional In elligence, Do mund, 23.-24.11.2017 1
signi ican . Robo con ol policies a e no longe jus ained by machine
lea ning algo i hms. Ra he , obo s lea n how o sol e a ce ain ask by
hemsel es, e.g., by me hods o e olu iona y obo ics [
4
]. In an eal wo ld
en i onmen e olu iona y lea ning o con ol policies can be cos ly, as he i -
ness o a ce ain obo ac ion can only be e alua ed a e a sequence o ime
s eps, which can easily be in minu es o hou s. Thus, hese lea ning p o-
cesses pose a di icul op imiza ion p oblem and s anda d lea ning me hods
a e no sui able o he ime equi emen s o hese asks. Neu al ne wo ks
a e a well es ablished ype o con olle in e olu iona y obo ics. He e, he
se o coe icien s and he opology o he ne wo k need o be op imized
o op imal pe o mance. Mo e ecen and sophis ica ed app oaches o
de eloping and lea ning o con olle s, such as neu oe olu ion o augmen ing
opologies [
18
] we e in en ed o handle hese op imiza ion p ocesses, bu
hey s ill need many e alua ions o adap he neu al ne wo ks.
∙
Ou hypo hesis is ha assis ing his lea ning p ocess by means o
su oga e-assis ed op imiza ion, which has p o en o be able o pe -
o m signi ican ly well in expensi e indus ial op imiza ion asks
[14, 15], should be bene icial.
∙
As a second hypo hesis, we assume ha hese su oga e models can
help o e ie e addi ional use ul in o ma ion abou he objec i e
unc ion, e.g., impo ance o ce ain inpu s.
We wan o es his hypo hesis based on expe imen s wi h a small eal-wo ld
ask simula o , which is implemen ed as a simple neu al ne wo k and no
expensi e o e alua e. The esul s can be ans e ed o mo e sophis ica ed
asks, like a eal wo ld lea ning p ocess. The esul s should indica e
he applicabili y and basic pe o mance o su oga e-assis ed op imiza ion
me hods in compa ison o s a e-o - he-a op imiza ion algo i hms. The
a iable impo ance in o ma ion p o ided by he su oga e models is also
analyzed wi h ega d o hei use ulness. Fo ins ance, a iable impo ance
could be help ul o iden i y especially impo an o de ec i e inpu s senso s
o a physical con olle , e.g., in an e olu iona y obo ics ask.
2P oc. 27. Wo kshop Compu a ional In elligence, Do mund, 23.-24.11.2017
The Ele a o Supe iso y G oup P oblem
Gene al Desc ip ion
Today, ele a o sys ems a e p esen e e ywhe e in u ban a eas. They need
o be op imized o achie e he desi ed se ice quali y in e ms o wai ing
ime o he cus ome s, as well as in e ms o ene gy e iciency. They a e
con olled by an ele a o g oup con olle , which assigns he ele a o ca s
o ce ain loo s and des ina ions on basis o he cus ome se ice calls. The
ESGC p oblem as in oduced by [
3
] is a so-called des ina ion call sys em,
whe e he cus ome can choose hei desi ed des ina ion on he loo le el
ou side he ele a o ca s. In he in oduced p oblem ins ance, he con olle
is implemen ed as a sophis ica ed neu al ne wo k NN, whe e he speci ic
s uc u e and weigh s depic a ce ain con ol s a egy. The op imiza ion
o hese weigh s imposes a se o challenges, which ende his ask highly
complex:
∙
The opology o he i ness unc ion is o a high ex en non-linea as
well as mul i-modal.
∙
The a ic load is dynamic and s ochas ic, as cus ome s do no a i e
in a de e minis ic manne .
∙
G adien -based me hods canno be applied success ully o his op i-
miza ion p oblem.
∙
The simula o is compu a ional expensi e, which limi s he numbe
o unc ion e alua ions.
As consequence o he complexi y o such simula o s [
3
] in oduced a
simpli ied alida ion model o an ESGC sys em, he sequen ial ing(S-
Ring).
S-Ring Pe cep on Simula o
The S-Ring was in oduced o benchma k di e en ESGC algo i hms inde-
penden o a ce ain ele a o / loo con igu a ion. I uses a simpli ied NN
o con ol he ele a o s, whe e he connec ion weigh s can be modi ied and
ep esen he a iables o an op imiza ion p oblem. Each weigh se ing
P oc. 27. Wo kshop Compu a ional In elligence, Do mund, 23.-24.11.2017 3
will esul in a ce ain con ol s a egy which is es ed on simula ions o
di e en a ic si ua ions. The S-Ring has low compu a ional cos s, which
allows us o use an ESGC ins ance as a benchma k o a la ge a ie y o
algo i hms. Using di e en a ic si ua ions will lead o a i ness unc ion
which is subjec o noise. The S-Ring op imiza ion p oblem can be de ined
as ollows [3]:
𝐹(𝑛, 𝑚, 𝑝, 𝑤) = 𝐸(︃𝑡
∑︁
𝑖
𝑐𝑖)︃(1)
whe e
𝑛
is he numbe o ele a o s,
𝑚
he numbe o loo s,
𝑝
he p obabili y
o an a i ing cus ome pe loo and
𝑤
he NN weigh ec o , which depic s
he con ol policy. This objec i e unc ion e alua es he a e age wai ing
ime o all cus ome s
𝑐𝑖
du ing a simula ed a ic si ua ion wi h
𝑡
s eps. Fo
a gi en se o
𝑛, 𝑚, 𝑝
he pe o mance is only in luenced by he weigh ec o
o he NN con olle . Thus, he simpli ied p oblem, as u he used du ing
his pape , can be w i en as
𝐹
=
𝐹
(
𝑤
). The pa ame e s
𝑛, 𝑚, 𝑝
we e
se as ollows: Table 1 also displays he numbe o ime s eps o a single
Table 1: S-Ring Con igu a ion
nFloo s nEle a o s p obNewCus ome nI e a ions
6 2 0.3 10000
simula ion un, which was se a he high o simula e a longe pe iod. Fo
each simula ion un, he exac same pe iod was used, esembling a ce ain
ixed ime- ame, e.g. a ce ain day in a yea . By choosing a ixed ime
ame, we emo ed he noise o he p oblem, which ende s he p oblem
simple o op imize. Mo eo e , he p oblem was adap ed by se ing he
desi ed cus ome se ice quali y o he g ound loo o a high p io i y, while
he second loo was se o a lowe p io i y. This should simula e a ypical
eal wo ld ho el scena io, whe e i is wan ed ha a i ing cus ome s in he
lobby a e as se ed. The second loo displays an in e nal se ice a ea,
which is o low p io i y o he quali y o se ice. As a side e ec , his
educes he dimensionali y o he op imiza ion p oblem om 12 o 10.
4P oc. 27. Wo kshop Compu a ional In elligence, Do mund, 23.-24.11.2017
Me hods o Lea ning o Neu al Ne wo ks
A s anda d me hod o lea ning NN con olle is backp opaga ion. Back-
p opaga ion op imizes he weigh s by u ilizing a se o aining da a ha
con ains inpu alues wi h co esponding ou pu s. In case o he S- ing op-
imiza ion, ou ask is no machine lea ning, bu o ind he (single) global
op imum o he gi en i ness opology o a ime dependen simula ion
p oblem. We ecei e a i ness alue only a e e alua ing he weigh s in
a designa ed simula ion un. This means, he e is no clea mapping om
inpu o ou pu da a, as he ou pu only de ines a ce ain con ol policy
and he inal ac ion changes dynamically in e e y ime s ep. Thus we will
need o use mo e sophis ica ed me hods: me aheu is ic op imiza ion and
su oga e-assis ed op imiza ion.
Me aheu is ic Op imiza ion
Me aheu is ics a e sophis ica ed heu is ics, which a e o en inspi ed by
na u e. They u ilize s ochas ic p ocesses ( andomiza ion) and usually do
no equi e any g adien in o ma ion. Me aheu is ics a e known o be
gene al sol e s which apply o a la ge a ie y o global p oblems wi hou
needing a p io i in o ma ion. They a e sui able o highly non-linea and
mul i-modal p oblems, as well as so-called black-box p oblems, whe e no
in o ma ion abou he opology o he objec i e unc ion is known. No
algo i hm is able o deli e hei bes pe o mance o e e y p oblem
wi hou adap ing hei con ol pa ame e s; By pa ame e uning [
2
,
6
], we
can exploi bene icial pa ame e se ings, bu i is e y ime-demanding.
To p o ide eliable esul s wi hou pu ing a lo o e o in o algo i hm
uning, we selec ed ou di e en s a e-o - he-a R implemen a ions o
common me aheu is ics om he ange o simula ed annealing me hods
and e olu iona y algo i hms o ou compa ison. Simula ed annealing [
9
] is
inspi ed by annealing p ocesses in me allu gy, whe e ma e ials a e hea ed
and cooled o change hei physical s uc u e. Simula ed annealing ollows
he base p inciple o an g eedy s ochas ic algo i hm, bu implemen s a
con ol s a egy which allows o accep also solu ions wi h lesse i ness.
This allows o escape local op ima and es ablishes a global sea ch s a egy.
E olu iona y algo i hms [
1
] a e based on he p inciples o na u al selec ion:
in each gene a ion, a popula ion o indi iduals (e.g. solu ions
𝑤
) is e ol ed
P oc. 27. Wo kshop Compu a ional In elligence, Do mund, 23.-24.11.2017 5
by mu a ion, ecombina ion and selec ion s eps. The selec ed packages
a e DEop im,GA,GenSA and genoud. DEop im and GA we e chosen
due o pe sonal p e e ence, while he wo la e we e chosen based on he
su ey on Con inuous Global Op imiza ion in R by Mullen [
12
]. GenSA
and genoud pe o med bes on a se o di e en op imiza ion p oblems.
∙DEop im
[
13
] is an R-implemen a ion o he di e en ial e olu ion
algo i hm [
19
], which belongs o he class o e olu iona y algo i hms.
I is designed o global op imiza ion using eal ec o s.
∙GA
[
16
] is a package which implemen s an gene ic algo i hm an
allows op imiza ion o eal and in ege p oblems.
∙GenSA p o ides a e sion o gene alized simula ed annealing [20].
∙ genoud
[
11
,
17
] is an R-package which p o ides and implemen a ion
o a so-called hyb id algo i hm. This algo i hm combines e olu iona y
algo i hms wi h he de i a i e-based quasi-New on me hod B oyden-
Fle che -Gold a b-Shanno (BFGS).
Su oga e-Assis ed Op imiza ion
Su oga e-assis ed op imiza ion algo i hms employ da a d i en models
o ligh en he bu den o expensi e objec i e unc ion e alua ions. One
amewo k o su oga e-assis ed op imiza ion is Sequen ial Pa ame e
Op imiza ion (SPO) [2]. SPO p o ides a lexible amewo k ha employs
me hods om he ields o design o expe imen , op imiza ion, and s a is ics.
In essence, SPO s a s by gene a ing an ini ial design o expe imen , hen
builds a su oga e model (e.g., a linea model o K iging). Then, he
su oga e model is op imized o sugges a p omising candida e solu ion,
which is a e wa ds e alua ed by he expensi e objec i e unc ion. These
las s eps (model building, op imiza ion and e alua ion) a e i e a ed un il
some budge o e alua ions is exhaus ed. Figu e 1 shows he op imiza ion
cycle o he unde lying ESGC p oblem.
6P oc. 27. Wo kshop Compu a ional In elligence, Do mund, 23.-24.11.2017
Complex ESGC
Simula o
S-Ring Simula ion Model
P e-de ined S uc u e
Neu al Ne wo k
Su oga e Model
Op imiza ion
Fi ing and Upda ing Model
Su oga e P edic ion
Bes P edic ed Weigh
Vec o
Objec i e Func ion
In o ma ion / Va iable
Impo ance
Figu e 1: Su oga e-Assis ed Op imiza ion Cycle. The ESGC Simula o is
app oxima ed by he S-Ring simula o . The i ness opology is i ed
by he su oga e on basis o he ini ial design and sequen ial upda es.
The sequen ial weigh ec o s a e compu ed by an op imiza ion o he
su oga e.
The expe imen s make use o SPOT, he R implemen a ion o SPO. Fo
his s udy, we ha e chosen o in es iga e h ee di e en su oga e models
wi hin he SPO amewo k.
∙Second o de model wi h s ep-wise eg ession:
Fi s ly, we
build second o de linea eg ession models. The model is i s build
wi h all i s o de e ec s, quad a ic e ec s as well as second o de
in e ac ions. E.g., o wo pa ame e s
𝑥1
and
𝑥2
a model o he o m
𝑦
(
𝑥
) =
𝑐1𝑥1
+
𝑐2𝑥2
+
𝑐3𝑥2
1
+
𝑐4𝑥2
2
+
𝑐5𝑥1𝑥2
is de e mined. This ull
model is u he e ined by backwa ds, s epwise a iable selec ion
based on he Akaike in o ma ion c i e ion. The s epwise a iable
selec ion is skipped whene e he da a size is insu icien . While
he esul ing models a e compa a i ely simple, one ad an age is he
compa a i ely low compu a ional e o .
∙Random Fo es :
Secondly, we use a Random Fo es [
5
] model.
Random Fo es s a e ensembles o decision ees. We use he de aul
se ings o he andomFo es R-package [
10
]. Random Fo es s a e able
o lea n non-linea dependencies in he da a, a e ypically nume ically
obus and as o compu e, and can handle disc e e inpu a iables.
P oc. 27. Wo kshop Compu a ional In elligence, Do mund, 23.-24.11.2017 7
a iable impo ance compa ison shows, ha he models a e able o ex ac
knowledge beyond he bes ound pa ame e se ing. To alida e he gi en
esul s, we will need o use a designa ed expe imen al design, which will
be pa o u u e esea ch.
Compu a ion Time Compa ison
An impo an aspec o e e y op imiza ion echnique is he o al compu-
a ion ime. Table 4 shows app oxima ed alues o he me aheu is ics
and he su oga e-assis ed op imiza ion wi h he espec i e models. As
he p oblem i sel has nea ly no compu a ion ime, he indica ed alues
a e mainly caused by he op imiza ion algo i hms. As he alues indica e,
su oga e-assis ed op imiza ion is in compa ison e y expensi e. The model
i ing, upda ing and op imiza ion p ocess is compu a ionally expensi e,
pa icula ly o a highe numbe o samples. This is especially isible o
K iging, which is e y sensi i e o highe sample sizes.
Table 4: Algo i hm Compu a ion Time. All alues a e app oxima ed.
Algo i hm No. E alua ions Compu a ion Time
S-Ring P oblem C 1 <0.001 seconds
S-Ring P oblem R 1 <1second
Me aheu is ics 100 0.1 second
Me aheu is ics 1000 1 second
Me aheu is ics 1𝑒+5 1-2 minu es
su RF 100 1 minu e
su SO 100 4 minu es
su KR 100 8 minu es
su RF 1000 1 hou
su SO 1000 4 hou s
su KR 1000 > 1 day
A his poin , we also ha e o conside ha he SPOT implemen a ion is
a R- amewo k, which is in e ms o compu a ion ime much in e io o
14 P oc. 27. Wo kshop Compu a ional In elligence, Do mund, 23.-24.11.2017
C o C++ based implemen a ions. Fo ins ance, a e-implemen a ion o
he S-Ring simula o in R, which is no mally implemen ed in C, is abou
1000 imes slowe . We can assume, ha an op imized e sion would be
signi ican as e . Mo eo e , SPO pe o ms sequen ial op imiza ion, while
he me aheu is ics a e able o conduc pa allel e alua ions.
Conclusion
In acco dance o ou hypo heses, he esul s show ha su oga e-assis ed
op imiza ion is a bene icial app oach o he unde lying NN con ol op-
imiza ion ask. The es ed algo i hms we e capable o ou pe o ming
me aheu is ic op imiza ion me hods. Fu he mo e, he su oga e can be
used o signi icance analysis o he inpu s and weigh ed connec ions o
u he exploi p oblem in o ma ion. We can hus assume ha su oga e-
assis ed op imiza ion is able o p o ide a g ea e unde s anding o he
lea ning p ocess. The clea downside o he su oga e-assis ed op imiza ion
is he la ge compu a ion ime, which is mo e han 10000 imes la ge
han hese o me aheu is ic op imiza ion. Howe e , his huge downside
becomes less signi ican in scena ios whe e he objec i e unc ion e alu-
a ions become e y expensi e, e.g., in he a ea o se e al minu es. The
model i ing and op imiza ion p ocess could be conduc ed simul aneously
o he eal- ime e alua ions. Also, no conside ed he e a e op imized and
pa allel su oga e-assis ed app oaches, which could conside able imp o e
he compu a ion ime. In his s udy, we used he de aul pa ame e s o
all gi en algo i hms, whe eby no ex ensi e esea ch was made o op imize
he SPOT de aul pa ame e s, while he me aheu is ic implemen a ions
de aul pa ame e s a e commonly op imized o include sel -adap i e p oce-
du es o show compa able esul s. An ex ended s udy o iden i y gene ally
good se ings o la ge se o p oblems could be help ul o u he imp o e
gene al pe o mance. In u u e esea ch, we will es he applicabili y o
a la ge ange o di icul p oblems om he a ea o a i icial in elligence
and e olu iona y obo ics. Mo eo e , we will s udy he use ulness o he
ex ac ed a iable impo ance o e olu iona y obo ics.
P oc. 27. Wo kshop Compu a ional In elligence, Do mund, 23.-24.11.2017 15
Acknowledgemen s
This wo k is pa o a p ojec ha has ecei ed unding om he Eu o-
pean Unions Ho izon 2020 esea ch and inno a ion p og am unde g an
ag eemen no. 692286.
Re e ences
[1]
T. Back. E olu iona y algo i hms in heo y and p ac ice: e olu ion
s a egies, e olu iona y p og amming, gene ic algo i hms. Ox o d uni-
e si y p ess, 1996.
[2]
T. Ba z-Beiels ein, C. W. Lasa czyk, and M. P euß. Sequen ial
pa ame e op imiza ion. In E olu iona y Compu a ion, 2005. The
2005 IEEE Cong ess on, olume 1, pages 773–780. IEEE, 2005.
[3]
T. Ba z-Beiels ein, M. P euss, and S. Ma kon. Valida ion and op i-
miza ion o an ele a o simula ion model wi h mode n sea ch heu is ics.
Me aheu is ics: P og ess as Real P oblem Sol e s, pages 109–128, 2005.
[4]
J. C. Bonga d. E olu iona y obo ics. Communica ions o he ACM,
56(8):74–83, 2013.
[5] L. B eiman. Random Fo es s. Machine Lea ning, 45(1):5–32, 2001.
[6]
Á. E. Eiben, R. Hin e ding, and Z. Michalewicz. Pa ame e con-
ol in e olu iona y algo i hms. IEEE T ansac ions on e olu iona y
compu a ion, 3(2):124–141, 1999.
[7]
O. Flasch, T. Ba z-Beiels ein, A. Da yan, P. Koch, W. Konen, T. D.
Oye oyan, and M. Tamu an. Compa ing ci me hods o p edic ion
models in en i onmen al enginee ing. In P oc. o CEC, 2010.
[8]
A. Fo es e , A. Sobes e , and A. Keane. Enginee ing Design ia
Su oga e Modelling. Wiley, 2008.
[9]
S. Ki kpa ick, C. D. Gela , and M. P. Vecchi. Op imiza ion by
simula ed annealing. science, 220(4598):671–680, 1983.
16 P oc. 27. Wo kshop Compu a ional In elligence, Do mund, 23.-24.11.2017
[10]
A. Liaw and M. Wiene . Classi ica ion and Reg ession by andomFo es .
R News, 2(3):18–22, 2002.
[11]
W. R. Mebane J and J. S. Sekhon. Gene ic op imiza ion using
de i a i es: he genoud package o . Jou nal o S a is ical So wa e,
42(11):1–26, 2011.
[12]
K. M. Mullen. Con inuous global op imiza ion in . Jou nal o
S a is ical So wa e, 60(6):1–45, 2014.
[13]
K. M. Mullen, D. A dia, D. L. Gil, D. Windo e , and J. Cline. Deop im:
An package o global op imiza ion by di e en ial e olu ion. 2009.
[14]
Y. S. Ong, P. B. Nai , and A. J. Keane. E olu iona y op imiza ion o
compu a ionally expensi e p oblems ia su oga e modeling. AIAA
jou nal, 41(4):687–696, 2003.
[15]
N. V. Queipo, R. T. Ha ka, W. Shyy, T. Goel, R. Vaidyana han, and
P. K. Tucke . Su oga e-based analysis and op imiza ion. P og ess in
ae ospace sciences, 41(1):1–28, 2005.
[16]
L. Sc ucca. Ga: a package o gene ic algo i hms in . Jou nal o
S a is ical So wa e, 53(4):1–37, 2013.
[17]
J. S. Sekhon and W. R. Mebane. Gene ic op imiza ion using de i a i es.
Poli ical Analysis, 7:187–210, 1998.
[18]
K. O. S anley and R. Miikkulainen. E ol ing neu al ne wo ks h ough
augmen ing opologies. E olu iona y compu a ion, 10(2):99–127, 2002.
[19]
R. S o n and K. P ice. Di e en ial e olu ion–a simple and e icien
heu is ic o global op imiza ion o e con inuous spaces. Jou nal o
global op imiza ion, 11(4):341–359, 1997.
[20]
Y. Xiang, S. Gubian, B. Suomela, and J. Hoeng. Gene alized simula ed
annealing o global op imiza ion: The gensa package. R Jou nal, 5(1),
2013.
P oc. 27. Wo kshop Compu a ional In elligence, Do mund, 23.-24.11.2017 17
Kon ak /Imp essum
Diese Ve ö en lichungen e scheinen im Rahmen de Sch i en eihe "CIplus". Alle Ve ö -
en lichungen diese Reihe können un e
h ps://cos.bibl. h-koeln.de/home
abge u en we den.
Die Ve an wo ung ü den Inhal diese Ve ö en lichung lieg beim Au o .
Da um de Ve ö en lichung: 18.12.2017
He ausgebe / Edi o ship
P o . D . Thomas Ba z-Beiels ein,
P o . D . Wol gang Konen,
P o . D . Bo is Naujoks,
P o . D . Ho s S enzel
Ins i u e o Compu e Science,
Facul y o Compu e Science and Enginee ing Science,
TH Köln,
S einmülle allee 1,
51643 Gumme sbach
u l:
www.ciplus- esea ch.de
Sch i lei ung und Ansp echpa ne / Con ac edi o ’s office
P o . D . Thomas Ba z-Beiels ein,
Ins i u e o Compu e Science,
Facul y o Compu e Science and Enginee ing Science,
TH Köln,
S einmülle allee 1, 51643 Gumme sbach
phone: +49 2261 8196 6391
u l:
h p://www.spo se en.de
eMail: homas.ba z-beiels ein@ h-koeln.de
ISSN (online) 2194-2870
This p ojec has ecei ed unding om he Eu opean Union’s Ho izon 2020
esea ch and inno a ion p og amme unde g an ag eemen No 692286.
