This is a sel -a chi ed e sion o an o iginal a icle. This e sion
may di e om he o iginal in pagina ion and ypog aphic de ails.
Au ho (s):
Ti le:
Yea :
Ve sion:
Copy igh :
Righ s:
Righ s u l:
Please ci e he o iginal e sion:
CC BY 4.0
h ps://c ea i ecommons.o g/licenses/by/4.0/
DESDEO: The Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e
Op imiza ion
© 2021 he Au ho s
Published e sion
Misi ano, Gio anni; Saini, Bhupinde Singh; A sa , Beki ; Sha azipou , Babooshka;
Mie inen Kaisa
Misi ano, Gio anni, Saini, Bhupinde Singh, A sa , Beki , Sha azipou , Babooshka, Mie inen
Kaisa. (2021). DESDEO: The Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e
Op imiza ion. IEEE Access, 9, 148277-148295. h ps://doi.o g/10.1109/ACCESS.2021.3123825
2021
Recei ed Oc obe 1, 2021, accep ed Oc obe 20, 2021, da e o publica ion Oc obe 27, 2021, da e o cu en e sion No embe 8, 2021.
Digi al Objec Iden i ie 10.1109/ACCESS.2021.3123825
DESDEO: The Modula and Open
Sou ce F amewo k o In e ac i e
Mul iobjec i e Op imiza ion
G. MISITANO , B. S. SAINI , B. AFSAR , B. SHAVAZIPOUR , AND K. MIETTINEN
Facul y o In o ma ion Technology, Uni e si y o Jy äskylä, 40014 Jy äskylä, Finland
Co esponding au ho : G. Misi ano (gio [email p o ec ed])
This wo k was suppo ed by he Academy o Finland unde G an 322221.
ABSTRACT In e ac i e mul iobjec i e op imiza ion me hods inco po a e p e e ences om a human deci-
sion make in he op imiza ion p ocess i e a i ely. This allows he decision make o ocus on a subse o
solu ions, lea n abou he unde lying ade-o s among he con lic ing objec i e unc ions in he p oblem
and adjus p e e ences du ing he solu ion p ocess. Inco po a ing p e e ence in o ma ion allows compu ing
only solu ions ha a e in e es ing o he decision make , dec easing compu a ion ime signi ican ly. Thus,
in e ac i e me hods ha e many s eng hs making hem iable o a ious applica ions. Howe e , he e is a
lack o exis ing so wa e amewo ks o apply and expe imen wi h in e ac i e me hods. We ill a gap in he
op imiza ion so wa e a ailable and in oduce DESDEO, a modula and open sou ce Py hon amewo k o
in e ac i e mul iobjec i e op imiza ion. DESDEO’s modula s uc u e enables implemen ing new in e ac i e
me hods and eusing p e iously implemen ed ones and hei unc ionali ies. Bo h scala iza ion-based and
e olu iona y me hods a e suppo ed, and DESDEO allows hyb idizing in e ac i e me hods o bo h ypes in
no el ways and enables e en swi ching he me hod du ing he solu ion p ocess. Mo eo e , DESDEO also
suppo s de ining mul iobjec i e op imiza ion p oblems o di e en kinds, such as da a-d i en o simula ion-
based p oblems. We discuss DESDEO’s modula s uc u e in de ail and demons a e i s capabili ies in ou
ca e ully chosen use cases aimed a helping eade s un amilia wi h DESDEO ge s a ed using i . We also
gi e an example on how DESDEO can be ex ended wi h a g aphical use in e ace. O e all, DESDEO o e s
a much-needed oolbox o esea che s and p ac i ione s o e icien ly de elop and apply in e ac i e me hods
in new ways – bo h in academia and indus y.
INDEX TERMS Da a-d i en mul iobjec i e op imiza ion, e olu iona y compu a ion, in e ac i e me hods,
mul i-c i e ia decision making, nonlinea op imiza ion, open sou ce so wa e, Pa e o op imiza ion.
I. INTRODUCTION
Op imiza ion in many eal-li e p oblems is ypically cha -
ac e ized by se e al con lic ing objec i es o be consid-
e ed simul aneously. In hese mul iobjec i e op imiza ion
p oblems, he p esence o con lic ing objec i es esul s in
many so-called Pa e o op imal solu ions wi h di e en ade-
o s ins ead o a single op imal solu ion. These solu ions
a e incompa able wi hou addi ional in o ma ion. The e o e,
he e is a need o a domain expe , e e ed o as a decision
make (DM), o ul ima ely choose one o he Pa e o op imal
solu ions as he inal one based on his/he p e e ences.
The associa e edi o coo dina ing he e iew o his manusc ip and
app o ing i o publica ion was Huaqing Li .
Di e en ypes o me hods ha e been de eloped o sol -
ing mul iobjec i e op imiza ion p oblems in he mul iple
c i e ia decision making (MCDM) (e.g., [1]–[3]) and e o-
lu iona y mul iobjec i e op imiza ion (EMO) (e.g., [4], [5])
communi ies. Mos MCDM me hods inco po a e a DM’s
p e e ences o ocus on subse s o he Pa e o op imal solu ions
e lec ing he in e es s o he DM. These me hods ha e a
s ong heo e ical backg ound and can gua an ee Pa e o op i-
mali y (see, e.g., [3]). Mos MCDM me hods use so-called
scala iza ion o scala izing unc ions o ans o m he o igi-
nal mul iobjec i e op imiza ion p oblem wi h he p e e ence
in o ma ion in o a scala ized p oblem (wi h a single objec-
i e) o be op imized. A e his ans o ma ion, an app op i-
a e single-objec i e op imiza ion me hod is o be used o sol e
he scala ized p oblem. By ca e ully selec ing he scala izing
VOLUME 9, 2021 This wo k is licensed unde a C ea i e Commons A ibu ion 4.0 License. Fo mo e in o ma ion, see h ps://c ea i ecommons.o g/licenses/by/4.0/ 148277
G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
unc ion, one can gua an ee ge ing a Pa e o op imal solu-
ion o he o iginal p oblem so ha he DM’s p e e ences
a e conside ed. Wi h di e en p e e ences, one can ypically
ge di e en Pa e o op imal solu ions. Fo compa isons o
di e en scala izing unc ions, see, e.g., [6], [7]. In con as ,
EMO me hods handle a popula ion o solu ions a a ime and
gene a e se e al app oxima ed Pa e o op imal solu ions o
ep esen di e en Pa e o op imal solu ions. They o en s a
om a andom se o solu ions and use di e en selec ion,
mu a ion and ecombina ion ope a o s o c ea e he nex gen-
e a ion o solu ions. Because o hei heu is ic na u e, hey
canno gua an ee Pa e o op imali y, bu hey can be applied o
challenging p oblems wi h, e.g., discon inuous o noncon ex
unc ions.
One can classi y di e en mul iobjec i e op imiza ion
me hods based on when a DM wi h p e e ence in o ma ion
akes pa in he solu ion p ocess [1], [3]. A no p e e ence
me hod is applied in absence o p e e ences. The DM may
p o ide his/he p e e ences be o e o a e he solu ion p o-
cess in a p io i o a pos e io i me hods, espec i ely. In a p i-
o i me hods, he DM p o ides hopes and expec a ions, and he
me hod ies o ind he bes ma ching solu ion. In con as ,
a ep esen a i e se o Pa e o op imal solu ions is gene a ed
in a pos e io i me hods o he DM o choose om.
The ou h class o me hods, known as in e ac i e mul i-
objec i e op imiza ion me hods, in ol es he DM du ing he
solu ion p ocess. In his way, he DM i e a i ely p o ides
his/he p e e ences while g adually gaining u he insigh
in o he p oblem and lea ning abou hidden limi a ions such
as he easibili y o he p e e ences and a ainable solu-
ions [8]. The e o e, he DM has a chance o modi y his/he
p e e ences based on new insigh and lea ning. Mo eo e ,
he cogni i e load se on he DM (a a ime) is usually
low compa ed o o he me hods, e.g., a pos e io i me hods.
Indeed, he DM can ocus he sea ch on a subse o solu ions
and only conside Pa e o op imal solu ions o in e es . This
also sa es compu a ional esou ces. Because o hese easons,
we conside he e in e ac i e me hods. As men ioned, hey
consis o i e a ions. A each i e a ion, he DM sees a solu ion
o some solu ions e lec ing he p o ided p e e ences and can
adjus he p e e ences o e en ually ind he mos p e e ed
solu ion. Thanks o lea ning, he con idence o he DM g ows
du ing he solu ion p ocess.
The DM can p o ide a ious ypes o p e e ence in o ma-
ion. Examples o hem include so-called e e ence poin s
whose componen s ep esen desi ed alues o objec i e
unc ions (also called aspi a ion le els), anges o accep able
objec i e unc ion alues, classi ica ion, pai wise compa -
isons and selec ing desi ed o undesi ed solu ions ou o a
subse , o name a ew (see, e.g., [3], [9]).
O e he yea s, di e en in e ac i e me hods ha e been
de eloped in he li e a u e, and hey ha e shown hei po en-
ial in a ious applica ions, see, e.g., [10]. They di e
om each o he mainly in e ms o p e e ence in o ma-
ion used, how solu ions e lec ing p e e ences a e gen-
e a ed, and wha kind o in o ma ion is p o ided o he
DM [3], [8], [11]. Howe e , hei implemen a ions a e done
in isola ion, and hey a e no eadily a ailable. E en hough
mos in e ac i e me hods u ilize simila componen s (such
as ypes o p e e ence in o ma ion, scala izing unc ions,
sampling echniques), each me hod has a di e en way
o implemen a ion. These issues slow down he p ac ical
usage o in e ac i e me hods om di e en pe spec i es
and in oduces a ious challenges, which we ha e lis ed as
ollows:
1) I is no easy o ind implemen a ions o di e en in e -
ac i e me hods o be applied.
2) Iden i ying he mos sui able in e ac i e me hods o be
used in a ious eal-li e applica ions is challenging.
3) Compa ing in e ac i e me hods is di icul because o
he lack o ha ing a ious in e ac i e me hods wi hin
he same amewo k.
4) U ilizing he implemen ed me hods o some pa s o
hei implemen a ion in new de elopmen s is ha d,
so e e y new cons uc ion needs o be s a ed om
sc a ch.
5) The lack o openness limi s applicabili y.
6) The i e a i e na u e o he in e ac i e me hods, oge he
wi h some s anda d componen s, enables swi ching
be ween me hods in di e en i e a ions o he solu ion
p ocess, a leas in heo y. None heless, sepa a e imple-
men a ions ha e been p e en ing he chance o es ing
his exci ing idea.
To he bes o ou knowledge, only one amewo k has
been de eloped o in e ac i e me hods, which, o some
ex en , aims o add ess he lis ed issues. I is he so-called
DESDEO amewo k [12]. Howe e , he e sion discussed
in [12] had p ac ical issues in i s implemen a ion, o e all
s uc u e, and modula i y and was, hus, no eady o b oade
usage and ex ensions. Fo hese easons, he e was a need o
i s e-s uc u e and hen o e-implemen a new DESDEO
amewo k, which is in oduced in his pape . The new ame-
wo k has a clea po en ial in add essing all he six lis ed
challenges.
The new DESDEO amewo k implemen ed in Py hon [13]
has a modula s uc u e and , hus, in ol es eusable modules
ha can be u ilized o implemen ing new in e ac i e me hods
o modi ying he exis ing ones. DESDEO enables sol ing
compu a ionally expensi e simula ion-based and da a-d i en
p oblems using su oga e models, including unce ain y con-
side a ions. I con ains implemen a ions o se e al old and
new in e ac i e me hods by a ious de elope s co e ing
me hods o bo h MCDM and EMO ypes. Thanks o he mod-
ula s uc u e, new o e ised me hods can be con enien ly
included in he amewo k.
DESDEO consis s o packages and modules. We in oduce
hem and also demons a e how DESDEO can be applied
o sol e p oblems wi h analy ical exp essions as well as
da a-d i en and simula ion based p oblems. The s eng hs o
DESDEO include he op ion o hyb idize scala iza ion based
and e olu iona y me hods and he con enience o compa ing
di e en me hods in he same en i onmen . Fo ins ance,
148278 VOLUME 9, 2021
G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
he e is no need o speci y he p oblem o be sol ed o each
me hod sepa a ely.
The modula s uc u e enables hyb idiza ion be ween di -
e en ypes o me hods. By hyb idiza ion, we mean he
abili y o use inal o in e media e esul s o one me hod
in ano he me hod, such as gene a ing app oxima ed Pa e o
op imal solu ions u ilizing an EMO me hod and using he
solu ions in an MCDM me hod o swi ching he me hod du -
ing he solu ion p ocess, e.g., when he DM wan s o change
he ype o p e e ence in o ma ion. This opens up new oppo -
uni ies o u ilizing di e en ea u es o a ious me hods
while he DM is no limi ed o using only one me hod o one
ype o p e e ences. Va ious isualiza ions and a g aphical
use in e ace a e also being de eloped wi h a simila modula
s uc u e in mind. The me hods implemen ed in DESDEO
can be u ilized by anyone who has basic p og amming skills
in Py hon, which has become a widely-used p og amming
language in da a and business analy ics. Since he amewo k
is open sou ce, i is eadily a ailable o a ious applica-
ions and can be con enien ly ailo ed o di e en p oblems,
i needed. I is na u ally also open o new con ibu ions and
anybody in e es ed is welcome o con ibu e.
The es o he pape is s uc u ed as ollows. In Sec ion II ,
we ou line he gene al concep s and no a ions o mul iobjec-
i e op imiza ion ha we use in his pape , b ie ly e iew he
ela ed open sou ce amewo ks o mul iobjec i e op imiza-
ion in he li e a u e, and o e iew some in e ac i e me hods
( e e ed o in his pape ). Sec ion III is a ge ed a eade s
in e es ed in con ibu ing o he de elopmen o DESDEO.
Fo his, he amewo k s uc u e in oducing packages, mod-
ules, and ex e nal dependencies is desc ibed in de ail. Those
who only wish o apply he amewo k o sol ing mul iob-
jec i e op imiza ion p oblems can skip Sec ion III and ocus
on ou di e se illus a i e use cases ou lined in Sec ion IV.
In Sec ion IV, we also gi e a basic example o a g aphical
use in e ace ha can be implemen ed o ease in e ac ion
be ween he in e ac i e me hods in DESDEO and he DM.
In Sec ion V, we discuss he po en ial o he modula ame-
wo k, such as adjus ing o hyb idizing me hods and c ea -
ing use in e aces o a ious in e ac i e me hods. Finally,
we conclude in Sec ion VI.
II. BACKGROUND
In his sec ion, we i s in oduce he main no a ion and
concep s used in his pape . We hen su ey he s a e-o -
he-a o open sou ce so wa e amewo ks a ailable o
mul iobjec i e op imiza ion. Finally, we e y b ie ly ou line
some o he in e ac i e me hods e e ed o in he use cases
conside ed in Sec ion IV.
A. MULTIOBJECTIVE OPTIMIZATION
We conside he ollowing o m o mul iobjec i e op imiza-
ion p oblems minimizing k≥2 objec i e unc ions [3]:
min (x)=( 1(x),..., k(x))
s. . x∈S,(1)
whe e i:S→R(i=1,...,k) a e objec i e unc ions
and x=(x1,...,xn)Tis a ec o o ndecision a iables in
he easible egion S⊂Rnde ined by cons ain unc ions.
Wi hou loss o gene ali y, we he e assume ha all unc ions
a e o be minimized. I some unc ion iis o be maximized,
i is equi alen o minimize − i.
A decision ( a iable) ec o x∗∈Sis called Pa e o op imal
i he e exis s no x∈S, so ha o all i, i(x)≤ i(x∗)
and o some j, j(x)< j(x∗). The image o Pa e o op imal
decision ec o s in he objec i e space Rkis called a Pa e o
on and i consis s o Pa e o op imal objec i e ec o s. In he
de ini ion o Pa e o op imali y, a solu ion is no domina ed
by any o he easible solu ion. As we deal wi h e olu iona y
me hods ha canno gua an ee Pa e o op imali y, we also use
he e m nondomina ed solu ions. They a e no domina ed by
any solu ion in he solu ion se conside ed ( ypically e e ed
o as a popula ion), bu a e no necessa ily Pa e o op imal.
The bes and he wo s possible alues o objec i e unc-
ions in he Pa e o on a e ep esen ed by an ideal and a nadi
poin , espec i ely. The componen s o he ideal poin can
be calcula ed by op imizing each objec i e unc ion subjec
o Sas a single-objec i e op imiza ion p oblem. In con as ,
compu ing he nadi poin is di icul in p ac ice as he se
o all Pa e o op imal solu ions is unknown. Howe e , some
me hods (e.g., a payo able [14]) a e a ailable ha can
app oxima e he nadi poin (see e.g., [3] and e e ences
he ein).
B. DATA-DRIVEN MULTIOBJECTIVE OPTIMIZATION
As men ioned in he in oduc ion, DESDEO can be applied
o sol e di e en ypes o mul iobjec i e op imiza ion p ob-
lems. Typically, he analy ical o ms o objec i e unc ions
and cons ain s canno be o mula ed in mos eal-li e p ob-
lems. In some cases, simula ion models can be used o
e alua e unc ion alues. In o he cases, he objec i e o con-
s ain s alues mus be gained om some eal expe iences (o
labo a o y expe imen s). In ei he case, e alua ing he unc-
ion alues is usually expensi e om di e en pe spec i es.
The e o e, so-called su oga e models can be u ilized ins ead
o he o iginal expensi e models o expe imen s.
On he o he hand, in oday’s digi al socie ies, a ious da a
om di e en sou ces a e con inuously eco ded, which can
be used as a new sou ce o in o ma ion in decision making.
Making he mos o he da a a ailable can lead o da a-d i en
op imiza ion p oblems. In his case, no o he in o ma ion han
he da a is a ailable, gi ing no o he op ion han i ing su o-
ga e models o o mula e unc ions o op imiza ion p oblems
based on he da a. Then, su oga e models app oxima e he
objec i e o cons ain alues.
Di e en ypes o su oga e models, such as p obabilis ic
(e.g., Bayesian ne wo k [15] and Ma ko chain Mon e Ca lo)
o machine lea ning echniques (e.g., adial basis unc-
ions [16], K iging o Gaussian p ocesses [17], [18], suppo
ec o eg ession [19], and neu al ne wo ks [20], [21]) exis
and can be u ilized o de i e unc ions o mul iobjec i e
op imiza ion p oblems. Mos o hese echniques a e eely
VOLUME 9, 2021 148279
G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
a ailable in di e en Py hon packages and lib a ies, which
can be used wi hin he DESDEO amewo k.
C. LITERATURE REVIEW ON OPEN SOURCE
FRAMEWORKS FOR MULTIOBJECTIVE
OPTIMIZATION
We ha e su eyed open sou ce amewo ks o mul iobjec i e
op imiza ion p oblems. We do no conside closed sou ce
and comme cial so wa e implemen a ions because hey do
no p o ide an oppo uni y o adjus he me hods o one’s
needs in he way open sou ce so wa e does. Se e al open
sou ce so wa e amewo ks ha e been p oposed in he li -
e a u e. Each o hem has i s own s eng hs and limi a ions
and di e s in some nuances om he o he s. In gene al, many
aspec s should be conside ed when selec ing an app op ia e
amewo k o one’s needs. Fo example, amilia i y wi h he
p og amming language used o implemen he amewo k, he
cha ac e is ics o he p oblem o be sol ed, he a ailabili y
o isualiza ion ools, and an exempla y use in e ace can
in luence he selec ion o a amewo k.
Table 1summa izes well-known open sou ce amewo ks
p oposed o sol ing mul iobjec i e op imiza ion p oblems.
We also lis some common amewo ks wi h a modula s uc-
u e, whe e mul iobjec i e op imiza ion me hods can be c e-
a ed. Besides he name and he p og amming language used,
he able lis s whe he he amewo ks ocus on mul iobjec i e
op imiza ion, include MCDM o EMO ypes o me hods, p o-
ide a decision-making mechanism whe e a DM can p o ide
his/he p e e ence in o ma ion and choose he mos p e e ed
solu ion, isualiza ion ools, and a use in e ace. The able
also s a es whe he he amewo k has a modula s uc u e
o no . In he ollowing, we b ie ly desc ibe each o he
amewo ks.
DEAP [22] and Inspy ed [23] do no ocus speci ically
on mul iobjec i e op imiza ion bu p o ide Py hon imple-
men a ions o e.g., gene ic algo i hms, simula ed annealing,
and di e en ial e olu ion. The (a pos e io i) EMO me hod
NSGA-II o mul iobjec i e op imiza ion is also included.
Since hese wo amewo ks ha e been de eloped wi h a
modula s uc u e, mo e mul iobjec i e op imiza ion me hods
can be de eloped by using he modules a ailable in he
amewo k. Inspy ed includes u he na u e-inspi ed op i-
miza ion algo i hms such as pa icle swa m op imiza ion and
an colony op imiza ion.
Op Sol e [24] has been implemen ed in he Julia lan-
guage. I in eg a es se e al exac algo i hms o mul iobjec-
i e linea op imiza ion p oblems (including mixed-in ege
p oblems).
Pla ypus [25] in ol es Py hon implemen a ions o se e al
well-known EMO me hods concen a ing, hus, on mul i-
objec i e op imiza ion. I also includes an analysis ool o
isual compa ison o EMO me hods by applying some pe -
o mance indica o s.
MOEA [26] is a Ja a-based amewo k ha enables au o-
ma ic pa alleliza ion o me hods ac oss mul iple p ocesso
co es. I includes mos o he s a e-o - he-a a pos e io i
EMO me hods.
PyGMO [27] is a Py hon ex ension o PaGMO (C++) [28]
which has implemen a ions o a a ie y o single- and mul-
iobjec i e op imiza ion me hods and eal-li e enginee ing
p oblems in an objec -o ien ed a chi ec u e. Au oma ic pa -
alleliza ion o he implemen ed me hods enables using he
unde lying mul ico e a chi ec u e e icien ly.
jMe alPy [29] ex ends he Ja a-based amewo k
jMe al [30] (which con ains me aheu is ic me hods like e o-
lu iona y me hods) o mul iobjec i e op imiza ion o be used
in Py hon. jMe alPy p o ides imp o ed da a analysis, in e ac-
i e isualiza ion o Pa e o op imal solu ions, and inc eased
compu a ional pe o mance by applying lib a ies a ailable in
Py hon. Addi ionally, jMe alPy acili a es pa allel compu ing
o compu a ionally expensi e p oblems.
Pymoo [31] is a mul iobjec i e op imiza ion amewo k in
Py hon and o e s e olu iona y me hods o single- and mul-
iobjec i e op imiza ion p oblems. I in ol es se e al isu-
aliza ion echniques o illus a ing esul s and well-known
indica o s o compa e he pe o mance o he me hods.
Finally, Pla EMO [32] is an open sou ce amewo k
de eloped in MATLAB including many EMO me hods,
widely used pe o mance indica o s, and benchma k p ob-
lems. I also has a g aphical use in e ace. Howe e , one
should no e ha e en hough he implemen a ion is openly
a ailable, a MATLAB license is equi ed o use i . The e-
o e, while being comme cial so wa e, Pla EMO s ill allows
adjus ing i s implemen a ion o mee speci ic needs.
The amewo ks men ioned so a do no con ain in e -
ac i e me hods. They include ei he MCDM o EMO ypes
o me hods, bu no bo h, and only one o he amewo ks
comes wi h a use in e ace. As his summa y shows, o e all,
DESDEO is unique since i is he only open sou ce ame-
wo k including in e ac i e me hods. Thus, DESDEO ills
a gap in he so wa e a ailable in he mul iobjec i e op i-
miza ion communi y. DESDEO has a clea modula s uc u e
making i easy o use s and de elope s o con ibu e new
con en s. Impo an ly, DESDEO in ol es bo h MCDM and
EMO ypes o me hods, enabling hyb idizing and swi ching
be ween me hods depending on needs and applica ion a eas.
Mo eo e , elemen s o building cus om g aphical use in e -
aces o e icien in e ac ion be ween he DM and in e ac i e
me hods a e a planned u u e inclusion in DESDEO. These
elemen s a e cu en ly unde ac i e de elopmen and a e o
be included as addi ional packages in DESDEO e en ually.
The e o e, isualiza ion and use in e ace (UI) i ems o
DESDEO a e in pa en heses in Table 1 o he ime being.
Howe e , specialized non-modula g aphical use in e aces
ha e been de eloped o DESDEO in he pas as seen in
Sec ion IV-F.
D. SOME INTERACTIVE METHODS IMPLEMENTED
As men ioned ea lie , di e en in e ac i e mul iobjec i e
op imiza ion me hods ha e been implemen ed in DESDEO.
In his sec ion, we b ie ly in oduce a ew ha a e u ilized
148280 VOLUME 9, 2021

G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
TABLE 1. Summa y o open sou ce op imiza ion amewo ks. In he able, MO s ands o mul iobjec i e op imiza ion.
la e in Sec ion IV: he e e ence poin me hod [33], he
synch onous NIMBUS me hod [34] and he NAUTILUS
amily [35] (pa icula ly E-NAUTILUS [36] ) om MCDM
me hods, and RVEA [37] and NSGA-III [38] om EMO
me hods.
The e e ence poin me hod [33] is a popula in e ac i e
mul iobjec i e op imiza ion me hod in which he DM p o-
ides p e e ences as desi ed objec i e unc ion alues cons i-
u ing a e e ence poin . Then, a each i e a ion, k+1 Pa e o
op imal solu ions e lec ing he e e ence poin a e ound by
u ilizing an achie emen scala izing unc ion. The DM can
i e a e (i.e., compa e solu ions and p o ide new e e ence
poin s) un il he mos p e e ed solu ion is ound.
In NIMBUS, s a ing om a Pa e o op imal solu ion, a DM
exp esses his/he p e e ences by classi ying he objec i e
unc ions co esponding o he Pa e o op imal solu ion in o
up o i e p e e ence classes o indica e how he cu en
objec i es should change o be mo e p e e able o he DM.
In each i e a ion o NIMBUS, based on he DM’s p e e ences,
1–4 Pa e o op imal solu ions a e gene a ed and shown o he
DM ( he DM decides how many new solu ions (s)he wan s o
see). Besides classi ica ion, he DM can ask o he desi ed
numbe o solu ions gene a ed be ween any wo Pa e o op i-
mal ones. Like o he in e ac i e me hods, he solu ion p ocess
con inues un il he DM has ound his/he mos p e e ed
solu ion.
The NAUTILUS amily [35] con ains in e ac i e ade-
o - ee me hods. This means ha he DM does no deal
wi h Pa e o op imal solu ions bu g adually app oaches he
Pa e o on s a ing om an in e io solu ion (like a nadi
poin ). Then, ollowing he DM’s p e e ences, all objec i es
a e simul aneously imp o ed un il a Pa e o op imal solu ion is
eached. Du ing he solu ion p ocess, he anges o objec i e
unc ion alues ha s ill can be eached wi hou ading-o
na u ally sh ink. Once a Pa e o op imal solu ion is eached,
he solu ion p ocess s ops since i is no longe possible o p o-
ceed wi hou ading-o . NAUTILUS a ian s a y ega ding
he ypes o p e e ence in o ma ion used and how solu ions
a e gene a ed in each i e a ion (see [35] o a compa ison
o he di e ences). Fo example, in each i e a ion o he
o iginal NAUTILUS [39] me hod, he DM anks he objec i e
unc ions based on he p e e ed imp o emen o he cu en
objec i e alues. In con as , in NAUTILUS 2 [40], a ios o
imp o emen a e p o ided by he DM. In E-NAUTILUS [36],
which is pa icula ly de eloped o handling compu a ionally
expensi e p oblems, he DM can compa e mul iple solu ions
( e e ed o as in e media e poin s) a each i e a ion. Finally,
NAUTILUS Na iga o [41] in eg a es NAUTILUS wi h na -
iga ion ideas [42], whe e he DM sees anges o objec i e
unc ion alues ha a e s ill eachable om he cu en i e a-
ion poin sh inking in eal- ime and p o ides p e e ences as
desi ed aspi a ion le els and bounds no o be exceeded.
Besides MCDM ype o me hods, a ious in e ac i e EMO
me hods ha e also been de eloped and implemen ed in
DESDEO. They include in e ac i e e sions [43] o he e -
e ence ec o -guided e olu iona y algo i hm (RVEA) [37]
and NSGA-III [38]. RVEA and NSGA-III a e o iginally a
pos e io i me hods. The in e ac i e e sion o NSGA-III has
been implemen ed, co esponding o how RVEA was made
in e ac i e in [43]. The main ype o p e e ence in o ma ion
used in bo h is a e e ence poin , bu o he p e e ence ypes
a e also a ailable o RVEA.
III. STRUCTURE OF THE DESDEO FRAMEWORK
In his sec ion, we desc ibe he s uc u e o he DESDEO
amewo k, including packages o he amewo k and he
modules in each package. In addi ion, we discuss he pu pose
o each package and i s dependencies. We also conside he
implemen a ion o he DESDEO amewo k and i s ex e nal
dependencies. Las ly, we discuss he a chi ec u al choices
made in DESDEO ha any aspi ing de elope and use o
he amewo ks should be awa e o . This sec ion is in ended
mos ly o hose in e es ed in con ibu ing o he amewo k’s
de elopmen . Those in e es ed only in u ilizing he ame-
wo k o sol ing mul iobjec i e op imiza ion p oblems may
p oceed o Sec ion IV.
A. PACKAGES AND MODULES
In he modula s uc u e o DESDEO, each package is a
collec ion o modules, which con ain class and unc ion
de ini ions o ackle speci ic asks in modeling and sol ing
mul iobjec i e op imiza ion p oblems in e ac i ely. The main
VOLUME 9, 2021 148281
G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
packages, called co e packages, and hei indi idual modules
a e p esen ed in Figu e 1. Each package has a well-de ined
pu pose and is buil o add ess a ce ain se o asks in in e ac-
i e mul iobjec i e op imiza ion me hods. The modules may
depend on o he packages lowe in he s uc u e, as shown in
Figu e 2.
FIGURE 1. The main s uc u e o he DESDEO amewo k wi h packages
and modules included in each package. Fu he packages and modules
can be added as needed.
FIGURE 2. The packages o DESDEO and hei dependencies on each
o he .
In Figu e 2, he a ows ep esen he in e nal dependen-
cies o packages in DESDEO, e.g., he package desdeo-emo
depends on bo h he packages desdeo- ools and desdeo-
p oblem. A modula s uc u e allows use s o choose which
pa s o he amewo k o use. Fo example, o model a
mul iobjec i e op imiza ion p oblem, one can use he desdeo-
p oblem package and a oid he needless inclusion o he o he
packages. Addi ionally, ha ing he amewo k s uc u ed in
a modula ashion eases he de elopmen o he amewo k
by encapsula ing ea u es and unc ionali ies ela ed o in e -
ac i e mul iobjec i e op imiza ion in hei own espec i e
packages.
In wha ollows, we desc ibe packages included in
DESDEO and hei dependencies on o he packages. The
packages also depend on exis ing popula Py hon packages,
which a e discussed u he a he end o his sec ion.
The desdeo-p oblem package con ains ea u es ela ed o
he o mula ion and modeling o mul iobjec i e op imiza-
ion p oblems. P oblems can be analy ical exp essions o
unc ions depending on decision a iables, o modeled based
on collec ed da a ela ed o he mul iobjec i e op imiza ion
p oblem (ei he u ilizing da a a ailable o da a ob ained
by unning a p oblem-speci ic simula o ). The p oblem can
na u ally also ha e cons ain unc ions de ining a easible
egion. Tools o p oblem o mula ion can be ound in he
module p oblem. As al eady men ioned, su oga e models
may be ained and used o model unc ions o a mul iob-
jec i e op imiza ion p oblem based on da a. Fo example,
Gaussian eg ession is a ailable as a su oga e model bu
any o he machine lea ning- ocused package can be used o
ain su oga e models. The ools o building su oga es can
be ound in he su oga emodels module. Mo eo e , com-
monly u ilized es p oblems in mul iobjec i e op imiza ion
can be ound in he es p oblems module. Such p oblems
include, o example, he DTLZ p oblems [44]. The desdeo-
p oblem package does no depend on any o he package in he
DESDEO amewo k.
The desdeo- ools package con ains u ili y ools ha a e
expec ed o be used du ing any phase o he op imiza ion
p ocess, i espec i e o he me hod ype (MCDM o EMO)
used o op imiza ion. Such ools include abs ac ions o a -
ious p e e ence elici a ion echniques, scala izing unc ions,
and nondomina ed so ing. The in e ac ion module con ains
me hods o ease in e ac ion be ween a DM and an in e ac-
i e mul iobjec i e op imiza ion me hod. The scala iza ion
module con ains scala iza ion ools o ans o ming mul i-
objec i e op imiza ion p oblems in o single-objec i e p ob-
lems (inco po a ing p e e ence in o ma ion). As men ioned
in he in oduc ion, we can ge Pa e o op imal solu ions by
using app op ia e scala iza ion unc ions, such as achie e-
men scala izing unc ions [45] and he scala iza ion unc ion
o he ε-cons ain me hod [2]. The maps module con ains
ools o ans o ming objec i es om one space o ano he ,
such as e.g., he so-called p e e ence inco po a ed space [46].
Finally, he sol e module con ains ools o sol ing scala -
ized p oblems. These sol e s mus be app op ia e o he cha -
ac e is ics o he p oblem in ques ion (conside ing, e.g., he
ype o a iables and he na u e o unc ions in ol ed). The
desdeo- ools package does no depend on any o he package
in he DESDEO amewo k.
The desdeo-emo package is he eposi o y o e olu iona y
algo i hms (EAs) and ools which a e speci ically used wi h
EMO me hods. Besides in e ac i e EMO me hods, i has
148282 VOLUME 9, 2021
G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
implemen a ions o basic (a pos e io i) EMO me hods and
some a p io i me hods as hey can be used as elemen s
o in e ac i e ones. The package con ains he ollowing
modules: popula ion, ecombina ion,selec ion,EAs,su o-
ga emodelling, and u ili ies. The i s h ee modules con-
ain abs ac ions ep esen ing he popula ion, c osso e and
mu a ion ope a o s as well as selec ion ope a o s. We use
hese abs ac ions as building blocks o implemen a ious
e olu iona y algo i hms in he EAs module. New EAs can be
implemen ed by ei he modi ying he implemen a ions in he
EAs module o by using he building blocks in o he mod-
ules in en i ely new ways. The su oga emodelling module
implemen s ce ain EA based me hods which a e speci ically
designed o ain su oga e models. Finally, u ili ies con ains
miscellaneous ools ha a e used by one o mo e EMO me h-
ods, bu do no i in he o he modules. The desdeo-emo
package depends on he desdeo-p oblem and desdeo- ools
packages.
As he name sugges s, he desdeo-mcdm package con-
ains implemen a ions o in e ac i e mul iobjec i e op imiza-
ion me hods o he MCDM ype (in ol ing scala iza ion
unc ions o gene a e Pa e o op imal solu ions). The me h-
ods hemsel es a e in he in e ac i e module. Fo example,
he synch onous NIMBUS and me hods belonging o he
NAUTILUS amily a e implemen ed in his module. The
u ili ies module con ains a ious u ili ies o en needed in
MCDM me hods. Fo ins ance, he u ili ies include a payo
able me hod o compu ing an ideal and an app oxima ion
o he nadi poin . The desdeo-mcdm package depends on he
desdeo-p oblem and desdeo- ools packages.
Besides he co e packages o DESDEO discussed so a ,
o he packages can be, and ha e also been, de eloped based
on he packages discussed. Examples o hese packages
consis o specialized g aphical UIs and new expe imen al
in e ac i e mul iobjec i e op imiza ion me hods no ma u e
enough o be included in DESDEO ye . Due o hei expe i-
men al na u e, we will no discuss hese addi ional packages
u he he e.
As men ioned, he DESDEO amewo k has been imple-
men ed in Py hon and is a ailable online as open sou ce
so wa e on Gi Hub.1The amewo k makes use o exis -
ing Py hon lib a ies in he SciPy ecosys em, mos no ably
NumPy [47], SciPy ( he lib a y) [48] and Pandas [49].
NumPy o e s nume ically e icien da a s uc u es, which
enables an e icien handling o a ay-like s uc u es p esen
e e ywhe e in he DESDEO amewo k. SciPy o e s exis -
ing compu a ional ou ines. Fo example, i o e s excellen
op imiza ion ou ines o op imizing cons ained p oblems
wi h a single objec i e. As men ioned, his kind o p oblem
eme ges, o ins ance, when scala izing a mul iobjec i e op i-
miza ion p oblem. In u n, Pandas has excellen and e icien
da a manipula ion ou ines. They a e needed especially when
ep esen ing da a-d i en mul iobjec i e op imiza ion p ob-
lems, which may some imes consis o la ge amoun s o da a
1h ps://gi hub.com/indus ial-op imiza ion-g oup/DESDEO
equi ing ex ensi e ea u e enginee ing be o e modeling a
mul iobjec i e op imiza ion p oblem.
Fo a mo e de ailed desc ip ion o each package and
module ound in DESDEO, he eade is encou aged o
check DESDEO’s main documen a ion. The documen a-
ion is ound online (h ps://desdeo. ead hedocs.io/en/la es /)
whe e he indi idual documen a ion o each co e package can
be ound wi h addi ional de ails abou implemen ed classes
and unc ions.
B. ARCHITECTURAL DECISIONS IN DESDEO
A couple o choices ha e been made du ing he de elopmen
o DESDEO. The use o he amewo k should keep hem in
mind while de eloping o using he amewo k.
As men ioned in Sec ion II, objec i e unc ions in mul-
iobjec i e op imiza ion p oblems can ei he be minimized
o maximized, bu wi hin he op imiza ion me hods in
DESDEO, unc ions a e always assumed o be minimized.
This means ha we con e unc ions o be maximized o
unc ions o be minimized and in e nally only deal wi h mini-
miza ion p oblems. This choice has been made o emo e any
possible so wa e bugs, con usion, and guesswo k ela ed o
keeping ack whe he an objec i e is o be minimized o max-
imized, ans o ming p oblems om one ype o ano he , and
pa sing p e e ence in o ma ion. Na u ally, when displaying
in o ma ion ela ed o a mul iobjec i e op imiza ion p oblem
and i s solu ions o a DM, he objec i es a e p esen ed in hei
o iginal o m. The ask o making he con e sion whene e
needed (also in he p e e ence in o ma ion) is he esponsi-
bili y o he UI.
As in e ac i e mul iobjec i e op imiza ion me hods a y in
he ype o in e ac ion and p e e ence in o ma ion equi ed
om he DM, abs ac ion o in e ac ion has been kep simple
and non- es ic i e in DESDEO. Each in e ac i e me hod
has a leas wo (objec ) me hods: s a and i e a e.
As he name sugges s, he o me is always used o s a a
me hod a e i has been ins an ia ed. Likewise, he i e a e
me hod is hen used o any subsequen in e ac ions a e
s a ing he me hod. Bo h he s a and i e a e me h-
ods e u n a leas one eques (Py hon) objec . These
objec s con ain all he necessa y in o ma ion o ca y ou
a equi ed in e ac ion wi h he in e ac i e me hod in hei
con en a ibu e, which is a Py hon dic iona y. The con-
en s o a eques may a y depending on he in e ac i e
me hod, bu each con en dic iona y in DESDEO comes
a leas wi h a message en y mean o gi e a hin o
he use o wha is expec ed o hem in e ac ion-wise. Each
eques objec has a esponse a ibu e, which is also
a dic iona y. The esponse dic iona y has i s own en ies,
which he use mus de ine o con inue i e a ing he in e ac-
i e me hod. A e he en ies o he esponse ha e been
de ined, he i e a e me hod can be in oked by gi ing i he
eques con aining he esponse wi h de ined en ies as
an a gumen . The i e a e me hod will hen e u n a new
eques . Examples o his eques - esponse s uc-
u e can be ound in he use cases in Sec ion IV. Howe e ,
VOLUME 9, 2021 148283
G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
i is no expec ed ha a DM onesel would di ec ly handle
hese Py hon dic iona ies. Ins ead, i is expec ed ha some
ex e nal in e ace is used o acili a e in e ac ion be ween a
DM and DESDEO. The eques and esponse should
be mainly used o s o e and communica e in o ma ion o and
om in e ac i e me hods a ailable in DESDEO.
IV. USE CASES
In his sec ion, we demons a e how one can use he DESDEO
amewo k o de ine di e en ypes o p oblems and sol e
hem by applying in e ac i e mul iobjec i e op imiza ion
me hods. Fo simplici y, we use a i e pollu ion p oblem
wi h i e objec i e unc ions and wo decision a iables
p esen ed in Sec ion IV-A. In Sec ion IV-B, we desc ibe
how o de ine a p oblem wi h an analy ical o mula ion and
sol e i using he synch onous NIMBUS me hod [34]. This
me hod is in he desdeo-mcdm package and inco po a es
classi ica ion ypes o p e e ences. Sec ion IV-C is de o ed
o de ining and sol ing a da a-d i en p oblem. The in e -
ac i e RVEA [43] me hod in he desdeo-emo package is
applied, whe e p e e ence in o ma ion is gi en as a e e ence
poin . In Sec ion IV-D, we conside some challenges o com-
pu a ionally expensi e p oblems and ollow he h ee-s age
app oach [50], whe e i s , NSGA-III is u ilized in a p e-
decision-making s age o gene a e nondomina ed solu ions.
Then, a compu a ionally inexpensi e su oga e p oblem is
o med. In he decision-making s age, he DM applies he
in e ac i e E-NAUTILUS [36] me hod o sol e he su oga e
p oblem. The e can also be a pos -decision-making s age o
assu e he Pa e o op imali y o he inal solu ion. He e we
conside he i s wo s ages as a hyb id way o using me hods
wi hin he DESDEO amewo k. Las ly, in Sec ion IV-E we
demons a e how he DM can swi ch in e ac i e me hods in
DESDEO o exp ess his/he p e e ences in di e en ways.
This example also illus a es some o he ad an ages he
DESDEO en i onmen p o ides. Finally, we discuss some
g aphical use in e aces in Sec ion IV-F.
As men ioned, he main documen a ion o DESDEO can be
ound online (h ps://desdeo. ead hedocs.io/en/la es /). The
documen a ion o each co e package discussed in Sec ion III,
can be eadily accessed h ough he main documen a ion.
We ad ice he eade o check he documen a ion o any
addi ional de ails ela ed o he use cases conside ed in
Sec ions IV-B,IV-C,IV-D, and IV-E. The examples shown
in hese sec ions can also be ound online in a Jupy e
No ebook.2
A. THE RIVER POLLUTION PROBLEM
The i e pollu ion p oblem [51] conside s a i e close o a
ci y. The e a e wo sou ces o pollu ion: indus ial pollu ion
om a ishe y and municipal was e om he ci y and wo
ea men plan s (in he ishe y and he ci y). The pollu ion
is epo ed in pounds o biochemical oxygen demanding
2h ps://desdeo. ead hedocs.io/en/la es /no ebooks/ ou _simple_use_
cases.h ml
ma e ial (BOD), and wa e quali y is measu ed in dissol ed
oxygen concen a ion (DO).
Cleaning wa e in he ci y inc eases ax a e, and cleaning
in he ishe y educes he e u n on in es men . The p oblem
is o imp o e he DO le el in he ci y and a he municipali y
bo de ( 1and 2, espec i ely) while, a he same ime, max-
imizing he pe cen e u n on in es men a he ishe y ( 3)
and minimizing addi ion o he ci y ax ( 4). We conside a
a ian o he p oblem [52] wi h one mo e objec i e o ensu e
he ea men plan s’ e iciency by keeping he p opo ional
amoun o BOD emo ed om he wa e close o he ideal
alue o 0.65 ( 5). The co esponding mul iobjec i e op i-
miza ion p oblem whe e all objec i es ha e been con e ed
o be minimized is as ollows:
min 1(x)= −4.07 −2.27x1
min 2(x)= −2.60 −0.03x1−0.02x2
−0.01
1.39 −x2
1
−0.30
1.39 −x2
2
min 3(x)= −8.21 +0.71
1.09 −x2
1
min 4(x)= −0.96 +0.96
1.09 −x2
2
min 5(x)=max{|x1−0.65|,|x2−0.65|}
s. . 0.3≤x1,x2≤1.0,(2)
whe e he p opo ional amoun s o BOD emo ed om wa e
in he wo ea men plan s a e, espec i ely, he decision
a iables x1and x2.
B. USE CASE 1: PROBLEM WITH AN ANALYTICAL
FORMULATION
DESDEO has good suppo o de ining and op imizing
p oblems wi h analy ical o mula ions. DESDEO p o ides
indi idual classes o de ine componen s o p oblem (1),
i.e., objec i e unc ions, a iables, and cons ain unc ions
sepa a ely. Box-cons ain s o a iables a e also suppo ed.
He e we analy ically de ine (2) and use modules o he
desdeo-p oblem package and NumPy. The impo s needed
a e shown in Sou ce code 1. No ice ha his p oblem has only
box-cons ain s.
SOURCE CODE 1. Needed impo s o a p oblem de ined analy ically. The
class MOP oblem is used o de ine a p oblem, he class Va iable i s
decision a iables, and he class Scala Objec i e he objec i e
unc ions.
We de ine he i e objec i e unc ions as shown in Sou ce
code 2as indi idual unc ions. These unc ions a e expec ed
o e u n a 1-dimensional NumPy a ay wi h each elemen
ep esen ing he espec i e objec i e alue when e alua ed
wi h one o mo e decision a iable ec o s. These decision
148284 VOLUME 9, 2021
G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
FIGURE 4. The GUI o he E-NAUTILUS me hod implemen ed in plo ly-dash. A oy mul iobjec i e op imiza ion p oblem wi h h ee objec i es (INCOME,
QUALITY, VOLUME) o be maximized is shown.
shown o a single i e a ion in he web in e ace and he
console a e i ually he same. In p ac ice, he p esen ed
in e ace simply handles he eques s and esponses
(discussed in Sec ion III-B) as was done in Sec ion IV-D.
In he E-NAUTILUS GUI, we ha e a di e en mul iobjec i e
op imiza ion p oblem wi h h ee objec i es o be maximized
VOLUME 9, 2021 148291

G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
ins ead o p oblem (2). We ha e chosen a p oblem wi h
ewe objec i es o simplici y. No e ha he a ows a e he
unc ion names emind o he maximiza ion.
The in e ace desc ibed o E-NAUTILUS is a ailable
online (h ps://desdeo.i .jyu. i/dash) alongside an in e ace
implemen a ion o NAUTILUS Na iga o as well. The
sou ce code o he in e ace shown is a ailable on Gi Hub
online.3To es he in e aces, we ha e p o ided he in e es ed
eade wi h oy da a online.4
V. POTENTIAL OF THE DESDEO FRAMEWORK
Because he DESDEO amewo k con ains a ious in e ac-
i e me hods, i enables e sa ile ways o applying hem.
As said, he DM can con enien ly swi ch he me hod du ing
he solu ion p ocess. This can be desi able i (s)he wan s o
change he ype o p e e ence in o ma ion in he middle o he
solu ion p ocess o ge di e en ypes o in o ma ion abou
he p oblem. This opens up as possibili ies when he DM is
no o ced o s ick wi h a single me hod o be applied bu can
selec me hods ha bes sui he di e en phases o he solu-
ion p ocess (e.g., lea ning and decision phases [54]). This
po en ial has been conside ed in [55], whe e a gene ic mul i-
agen a chi ec u e o in e ac i e me hods was p oposed o
suppo DMs in selec ing he mos sui ed in e ac i e me hod
based on p e e ed p e e ence ype and hei needs in di e en
phases du ing he solu ion p ocess. Wi hou a amewo k like
DESDEO, swi ching he me hod is incon enien ; he p oblem
o be sol ed mus be connec ed o indi idual mul iobjec i e
op imiza ion me hods sepa a ely, and he solu ion his o y
wi h he p e ious me hod is no easily a ailable.
DESDEO has clea po en ial in allowing esea che s o
hyb idize EMO and MCDM me hods in no el ways. This
po en ial is no jus limi ed o he example seen in Sec ion IV,
whe e an EMO me hod was used o compu e a ep esen a ion
o a Pa e o on , which was hen explo ed using an MCDM
me hod. Mo e inno a i e and ad anced ways o combining
no jus me hods bu also hei indi idual componen s a e
possible. This is because o he modula ashion in which
he a ious mul iobjec i e op imiza ion me hods ha e been
implemen ed in DESDEO. Combining indi idual compo-
nen s enables he de elopmen o new in e ac i e me hods,
which can be also included in DESDEO ex ending he ame-
wo k u he . The IOPIS algo i hm, desc ibed in [46], is an
example o such a me hod.
Mo eo e , DESDEO o e s a p omising basis o imple-
men ing new in e ac i e mul iobjec i e op imiza ion me hods
ha a e no based on combining exis ing componen s. Due o
he modula s uc u e, a de elope can easily euse al eady
implemen ed componen s and only add hose ha a e no ye
a ailable (i needed). Fo example, he desdeo- ools package
has a wide a ie y o di e en ools anging om achie emen
scala izing unc ions o as nondomina ed so ing, which
3h ps://gi hub.com/indus ial-op imiza ion-g oup/desdeo-dash
4h ps://gi hub.com/indus ial-op imiza ion-
g oup/DESDEO/blob/mas e /docs/no ebooks/da a/ oy_da a.cs
can p o e use ul in implemen ing new me hods. In addi ion,
expe imen ing wi h new me hods and ideas in mul iobjec i e
op imiza ion is also made easy hanks o DESDEO and he
eusabili y o i s componen s. DESDEO can also encou age
and lowe he h eshold o esea che s o implemen hei
me hods as open sou ce code, con ibu ing o he openness o
he esea ch conduc ed in mul iobjec i e op imiza ion. This
way, DESDEO has he po en ial and is on a good ack o
becoming a cen al hub o open implemen a ions o in e ac-
i e mul iobjec i e op imiza ion me hods.
Apa om being in e es ing om an academic pe -
spec i e, DESDEO can na u ally be u ilized o modeling
and sol ing eal-li e p oblems om any ield as long as
he p oblem can be modeled as a mul iobjec i e op imiza ion
p oblem. Depending on he ype and equi emen s o he
p oblem, DESDEO migh s ill lack ce ain ea u es neces-
sa y o modeling and sol ing he p oblem, which is also
one o he cu en limi a ions o DESDEO. Howe e , due
o DESDEO’s modula s uc u e and open sou ce na u e,
implemen ing hese missing ea u es is possible by anyone.
Fo example, he unde lying op imiza ion me hods o single-
objec i e op imiza ion p oblems a ising in a ious in e ac i e
me hods in DESDEO can be changed o be e accoun o
he ype o p oblem being sol ed. Simila ly, he c osso e and
mu a ion ope a ions in EMO me hods can also be cus omized
i need be. Las ly, in modeling a da a-d i en mul iobjec-
i e op imiza ion p oblem, almos any su oga e model can
be implemen ed and used. Ob iously, exis ing ea u es in
DESDEO can be combined wi h new ea u es as well allow-
ing p ac i ione s o sa e ime and help hem ocus on sol ing
he p oblem a hand. In his way, DESDEO can be ex ended o
accoun o any kind o mul iobjec i e op imiza ion p oblem
om any ield while dec easing he po en ial wo kload o
p ac i ione s.
Being a so wa e amewo k, DESDEO has a lea ning
cu e o i , which means ha a ce ain le el o p o iciency
in Py hon and mul iobjec i e op imiza ion is o be expec ed
om he use . This clea ly limi s he size o he po en ial use
base o DESDEO and is, he e o e, one o he amewo k’s
majo limi a ions a he p esen ime. We al eady o e a
w i en documen a ion o DESDEO’s ea u es, bu o make
DESDEO e en mo e accessible, we plan on including mo e
opical guides in he documen a ion on how o use DESDEO
(such as he ones p esen ed in Sec ion IV) and conside p o-
ducing u o ial ideos on how o use DESDEO in he u u e.
This should help b oaden DESDEO’s use base and allow
use s o ex end DESDEO o mee hei indi idual needs.
All o his will help DESDEO g ow u he as a so wa e
amewo k.
Compa ison and iden i ying he bes sui ed me hod o a -
ious needs a e impo an . DESDEO does o e e y p omising
oppo uni ies o compa ing and alida ing di e en in e ac-
i e me hods. This is i al and demanding because he DM
plays an impo an ole in he solu ion p ocess and conduc ing
expe imen s wi h human pa icipan s is challenging. To be
able o compa e in e ac i e me hods, hei pe o mance needs
148292 VOLUME 9, 2021
G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
o be e alua ed and alida ed using app op ia e quali y indi-
ca o s. To he bes o ou knowledge, no quali y indica o s
o in e ac i e me hods ha e been p oposed. Fo such quali y
indica o s, he desi able p ope ies ha quali y in e ac i e
solu ion p ocesses should be de ined. In [56], a sys ema ic
li e a u e e iew o he assessmen s o in e ac i e me hods
is p o ided along wi h desi able p ope ies o in e ac i e
me hods. This can be conside ed as he ini ial s ep owa ds
de eloping quali y indica o s o in e ac i e me hods. Mo e-
o e , he e has been some in e es in compa ing in e ac-
i e me hods wi h so-called a i icial DMs in he li e a u e
(e.g., [57]–[59]). Wi hin he DESDEO amewo k, an a i-
icial DM has ecen ly been p oposed o compa e e e -
ence poin -based in e ac i e EMO me hods [60]. DESDEO
p o ides an excellen pla o m o compa isons because i
in ol es a ious in e ac i e me hods wi hin he same ame-
wo k. To u ilize he oppo uni ies a ailable, we need a i icial
DMs capable o handling di e en ypes o p e e ences and
me hods.
VI. CONCLUSION
In his pape , we ill a gap in he op imiza ion so wa e
a ailable. We in oduced DESDEO: an open sou ce mul i-
objec i e op imiza ion amewo k implemen ed in Py hon.
DESDEO makes in e ac i e mul iobjec i e op imiza ion
me hods openly a ailable o bo h use s and de elope s.
We in oduced he modula s uc u e o DESDEO and i s
di e en packages and hei modules. We also desc ibed
he pu pose o each package and i s dependencies and he
amewo k’s ex e nal dependencies. Besides, wi h a i e-
objec i e op imiza ion p oblem, we demons a ed how o use
he DESDEO amewo k o de ine di e en ypes o p ob-
lems (i.e., wi h analy ical exp essions, da a-d i en, and com-
pu a ionally expensi e p oblems) and sol e hem by applying
and hyb idizing in e ac i e mul iobjec i e op imiza ion me h-
ods o MCDM and EMO ypes.
The modula i y o DESDEO eases de eloping new
me hods and o e s a con enien possibili y o compa ing di -
e en in e ac i e me hods. Fu he mo e, implemen ing di -
e en ypes o me hods in he same amewo k, as done in
DESDEO, will s a a new e a in hyb idiza ion and allows
he DM o swi ch be ween me hods in a ious i e a ions o
he solu ion p ocess.
We also no ed ha o e icien in e ac ion wi h he DM,
he e is a need o in e ac i e isualiza ion ools and sui able
(g aphical) UIs in mul iobjec i e op imiza ion, which is lack-
ing in he li e a u e. We a e add essing his p ac ical conce n
by ac i ely de eloping a D3 (h ps://d3js.o g/) based Type-
sc ip lib a y o in e ac i e isualiza ion componen s, such
as in e ac i e pa allel coo dina e plo s wi hin DESDEO. Ou
p ima y goal wi h his lib a y is o p o ide he mul iobjec i e
op imiza ion communi y wi h new and needed ools o build
hei own in e aces o in e ac i e mul iobjec i e op imiza-
ion; simila o he example seen in Sec ion IV-F. To acili a e
he use o he packages in DESDEO o be ex ended o o he
so wa e, such as web based in e aces, we a e also wo king
on a web API (applica ion p og amming in e ace) h ough
which we can expose in e ac i e me hods in DESDEO o
enable hei use in a a ie y o applica ions. The in e es ed
eade can ollow he la es de elopmen s o DESDEO ia
i s homepage (desdeo.i .jyu. i). The ealiza ion o his ision
should make in e ac i e mul iobjec i e op imiza ion me hods
much mo e accessible in he u u e, no jus o esea che s
de eloping hem, bu also o he needs o applica ions in
a ious ields.
ACKNOWLEDGMENT
The au ho s would like o hank all o hose who ha e
con ibu ed o DESDEO in he pas . Especially, hey would
like o hank Gioma a Lá aga, Johanna Sil ennoinen,
Pouya Aghaei Pou , Juuso Pajasmaa, S e an O ayagich, and
An i Luopajä i. They would also like o hank Vesa Ojaleh o
o his pionee ing wo k in de eloping he old e sion o he
DESDEO amewo k. This wo k is a pa o he hema ic
esea ch a ea Decision Analy ics U ilizing Causal Models
and Mul iobjec i e Op imiza ion (DEMO, jyu. i/demo) a he
Uni e si y o Jy äskylä.
REFERENCES
[1] C.-L. Hwang and A. Masud, Mul iple Objec i e Decision Making–Me hods
and Applica ions: A S a e-o - he-A Su ey. Be lin, Ge many: Sp inge ,
1979.
[2] V. Chankong and Y. Y. Haimes, Mul iobjec i e Decision Making: Theo y
and Me hodology. New Yo k, NY, USA: Else ie , 1983.
[3] K. Mie inen, Nonlinea Mul iobjec i e Op imiza ion. Bos on, MA, USA:
Kluwe , 1999.
[4] C. A. C. Coello, G. B. Lamon , and D. A. Van Veldhuizen, E olu iona y
Algo i hms o Sol ing Mul i-Objec i e P oblems, 2nd ed. New Yo k, NY,
USA: Sp inge , 2007.
[5] K. Deb, Mul i-Objec i e Op imiza ion Using E olu iona y Algo i hms.
Chiches e , U.K.: Wiley, 2001.
[6] K. Mie inen and M. M. Mäkelä, ‘‘On scala izing unc ions in mul iobjec-
i e op imiza ion,’’ OR Spec ., ol. 24, no. 2, pp. 193–213, May 2002.
[7] F. Ruiz, M. Luque, and J. M. Cabello, ‘‘A classi ica ion o he weigh ing
schemes in e e ence poin p ocedu es o mul iobjec i e p og amming,’’
J. Ope . Res. Soc., ol. 60, no. 4, pp. 544–553, Ap . 2009.
[8] K. Mie inen, J. Hakanen, and D. Podkopae , ‘‘In e ac i e nonlinea mul i-
objec i e op imiza ion me hods,’’ in Mul iple C i e ia Decision Analysis:
S a e o he A Su eys, 2nd ed., S. G eco, M. Eh go , and J. Figuei a,
Eds. New Yo k, NY, USA: Sp inge , 2016, pp. 931–980.
[9] M. Luque, F. Ruiz, and K. Mie inen, ‘‘Global o mula ion o in e ac-
i e mul iobjec i e op imiza ion,’’ OR Spec ., ol. 33, no. 1, pp. 27–48,
Jan. 2011.
[10] J. B anke, K. Deb, K. Mie inen, and R. Slowinski, Eds., Mul iobjec i e
Op imiza ion: In e a i e and E olu iona y App oaches. Be lin, Ge many:
Sp inge , 2008.
[11] B. Xin, L. Chen, J. Chen, H. Ishibuchi, K. Hi o a, and B. Liu, ‘‘In e ac-
i e mul iobjec i e op imiza ion: A e iew o he s a e-o - he-a ,’’ IEEE
Access, ol. 6, pp. 41256–41279, 2018.
[12] V. Ojaleh o and K. Mie inen, ‘‘DESDEO: An open amewo k o in e -
ac i e mul iobjec i e op imiza ion,’’ in Mul iple C i e ia Decision Making
and Aiding, S. Hube , M. J. Geige , and A. T. de Almeida, Eds. Cham,
Swi ze land: Sp inge , 2019, pp. 67–94.
[13] G. Rossum, ‘‘Py hon e e ence manual,’’ NLD, Cen um oo Wiskunde
en In o ma ica, Ams e dam, The Ne he lands, Tech. Rep., 1995.
[14] R. Benayoun, J. de Mon gol ie , J. Te gny, and O. La i che , ‘‘Linea
p og amming wi h mul iple objec i e unc ions: S ep me hod (STEM),’’
Ma h. P og am., ol. 1, no. 1, pp. 366–375, Dec. 1971.
[15] X. Wang, Y. Jin, S. Schmi , and M. Olho e , ‘‘An adap i e Bayesian
app oach o su oga e-assis ed e olu iona y mul i-objec i e op imiza ion,’’
In . Sci., ol. 519, pp. 317–331, May 2020.
VOLUME 9, 2021 148293
G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
[16] S. N. Qasem, S. M. Shamsuddin, S. Z. M. Hashim, M. Da us, and
E. Al-Shamma i, ‘‘Meme ic mul iobjec i e pa icle swa m op imiza ion-
based adial basis unc ion ne wo k o classi ica ion p oblems,’’ In . Sci.,
ol. 239, pp. 165–190, Aug. 2013.
[17] J. Knowles, ‘‘Pa EGO: A hyb id algo i hm wi h on-line landscape app ox-
ima ion o expensi e mul iobjec i e op imiza ion p oblems,’’ IEEE T ans.
E ol. Compu ., ol. 10, no. 1, pp. 50–66, Feb. 2006.
[18] M. Li, G. Li, and S. Aza m, ‘‘A k iging me amodel assis ed mul i-objec i e
gene ic algo i hm o design op imiza ion,’’ J. Mech. Design, ol. 130,
no. 3, pp. 1–10, Ma . 2008.
[19] H. Ay uğ and S. Sayın, ‘‘Using suppo ec o machines o lea n he
e icien se in mul iple objec i e disc e e op imiza ion,’’ Eu . J. Ope . Res.,
ol. 193, no. 2, pp. 510–519, Ma . 2009.
[20] G. Kou akos and A. Man oglou, ‘‘De elopmen o a mul i-objec i e op i-
miza ion algo i hm using su oga e models o coas al aqui e manage-
men ,’’ J. Hyd ol., ol. 479, pp. 13–23, Feb. 2013.
[21] K. Mi a and S. Majumde , ‘‘Successi e app oxima e model based mul i-
objec i e op imiza ion o an indus ial s aigh g a e i on o e indu a ion
p ocess using e olu iona y algo i hm,’’ Chem. Eng. Sci., ol. 66, no. 15,
pp. 3471–3481, Aug. 2011.
[22] F.-A. Fo in, F.-M. De Rain ille, M.-A. G. Ga dne , M. Pa izeau, and
C. Gagné, ‘‘DEAP: E olu iona y algo i hms made easy,’’ J. Mach. Lang.
Res., ol. 13, pp. 2171–2175, Jul. 2012.
[23] A. Ga e , Inspy ed: Bio-inspi ed algo i hms in Py hon.
Accessed: No . 19, 2020. [Online]. A ailable: h ps://gi hub.com/
aa onga e /inspy ed
[24] X. Gandibleux, G. Soleilhac, A. P zybylski, and S. Ruzika, ‘‘ Op Sol e :
An open sou ce so wa e en i onmen o mul iobjec i e ma hema ical
op imiza ion,’’ in P oc. 21s Con . In . Fed. Ope . Res. Socie ies (IFORS),
2017, pp. 17–21.
[25] D. Hadka. Pla ypus: Mul iobjec i e Op imiza ion in Py hon.
Accessed: No . 19, 2020. [Online]. A ailable: h ps://pla ypus.
ead hedocs.io
[26] D. Hadka. MOEA F amewo k: A F ee and Open Sou ce Ja a F amewo k
o Mul iobjec i e Op imiza ion. Accessed: Dec. 1, 2020. [Online]. A ail-
able: h p://moea amewo k.o g/
[27] D. Izzo and F. Biscani. PyGMO: Py hon Pa allel Global Mul i-
objec i e Op imize . Accessed: No . 19, 2020. [Online]. A ailable:
h ps://esa.gi hub.io/pygmo
[28] F. Biscani, D. Izzo, and C. H. Yam, ‘‘A global op imisa ion oolbox o
massi ely pa allel enginee ing op imisa ion,’’ 2010, a Xi :1004.3824.
[29] A. Bení ez-Hidalgo, A. J. Neb o, J. Ga cía-Nie o, I. O egi, and J. Del Se ,
‘‘JMe alPy: A Py hon amewo k o mul i-objec i e op imiza ion
wi h me aheu is ics,’’ Swa m E ol. Compu ., ol. 51, Dec. 2019,
A . no. 100598.
[30] J. J. Du illo and A. J. Neb o, ‘‘jMe al: A Ja a amewo k o mul i-
objec i e op imiza ion,’’ Ad . Eng. So w., ol. 42, no. 10, pp. 760–771,
2011.
[31] J. Blank and K. Deb, ‘‘Pymoo: Mul i-objec i e op imiza ion in Py hon,’’
IEEE Access, ol. 8, pp. 89497–89509, 2020.
[32] Y. Tian, R. Cheng, X. Zhang, and Y. Jin, ‘‘Pla EMO: A MATLAB pla -
o m o e olu iona y mul i-objec i e op imiza ion,’’ IEEE Compu . In ell.
Mag., ol. 12, no. 4, pp. 73–87, No . 2017.
[33] A. P. Wie zbicki, ‘‘A ma hema ical basis o sa is icing decision making,’’
Ma h. Model., ol. 3, no. 5, pp. 391–405, 1982.
[34] K. Mie inen and M. M. Mäkelä, ‘‘Synch onous app oach in in e ac-
i e mul iobjec i e op imiza ion,’’ Eu . J. Ope . Res., ol. 170, no. 3,
pp. 909–922, May 2006.
[35] K. Mie inen and F. Ruiz, ‘‘NAUTILUS amewo k: Towa ds ade-o - ee
in e ac ion in mul iobjec i e op imiza ion,’’ J. Bus. Econ., ol. 86, nos. 1–2,
pp. 5–21, Jan. 2016.
[36] A. B. Ruiz, K. Sindhya, K. Mie inen, F. Ruiz, and M. Luque, ‘‘E-
NAUTILUS: A decision suppo sys em o complex mul iobjec i e op i-
miza ion p oblems based on he NAUTILUS me hod,’’ Eu . J. Ope . Res.,
ol. 246, no. 1, pp. 218–231, Oc . 2015.
[37] R. Cheng, Y. Jin, M. Olho e , and B. Sendho , ‘‘A e e ence ec o guided
e olu iona y algo i hm o many-objec i e op imiza ion,’’ IEEE T ans.
E ol. Compu ., ol. 20, no. 5, pp. 773–791, Oc . 2016.
[38] K. Deb and H. Jain, ‘‘An e olu iona y many-objec i e op imiza ion algo-
i hm using e e ence-poin -based nondomina ed so ing app oach, pa
I: Sol ing p oblems wi h box cons ain s,’’ IEEE T ans. E ol. Compu .,
ol. 18, no. 4, pp. 577–601, Ap . 2013.
[39] K. Mie inen, P. Eskelinen, F. Ruiz, and M. Luque, ‘‘NAUTILUS me hod:
An in e ac i e echnique in mul iobjec i e op imiza ion based on he nadi
poin ,’’ Eu . J. Ope . Res., ol. 206, no. 2, pp. 426–434, Oc . 2010.
[40] K. Mie inen, D. Podkopae , F. Ruiz, and M. Luque, ‘‘A new p e e ence
handling echnique o in e ac i e mul iobjec i e op imiza ion wi hou
ading-o ,’’ J. Global Op im., ol. 63, no. 4, pp. 633–652, Dec. 2015.
[41] A. B. Ruiz, F. Ruiz, K. Mie inen, L. Delgado-An eque a, and V. Ojaleh o,
‘‘NAUTILUS Na iga o : F ee sea ch in e ac i e mul iobjec i e op imiza-
ion wi hou ading-o ,’’ J. Global Op im., ol. 74, no. 2, pp. 213–231,
Jun. 2019.
[42] M. Ha ikainen, K. Mie inen, and K. Klam o h, ‘‘In e ac i e noncon ex
Pa e o na iga o o mul iobjec i e op imiza ion,’’ Eu . J. Ope . Res.,
ol. 275, no. 1, pp. 238–251, May 2019.
[43] J. Hakanen, T. Chugh, K. Sindhya, Y. Jin, and K. Mie inen, ‘‘Connec ions
o e e ence ec o s and di e en ypes o p e e ence in o ma ion in in e -
ac i e mul iobjec i e e olu iona y algo i hms,’’ in P oc. IEEE Symp. Se .
Compu . In ell. (SSCI), Dec. 2016, pp. 1–8.
[44] K. Deb, L. Thiele, M. Laumanns, and E. Zi zle , ‘‘Scalable es p oblems
o e olu iona y mul iobjec i e op imiza ion,’’ in E olu iona y Mul iobjec-
i e Op imiza ion: Theo e ical Ad ances and Applica ions, A. Ab aham,
L. Jain, and R. Goldbe g, Eds. London, U.K.: Sp inge , 2005, pp. 105–145.
[45] A. P. Wie zbicki, ‘‘On he comple eness and cons uc i eness o pa ame ic
cha ac e iza ions o ec o op imiza ion p oblems,’’ Ope .-Res.-Spek um,
ol. 8, no. 2, pp. 73–87, Jun. 1986.
[46] B. S. Saini, J. Hakanen, and K. Mie inen, ‘‘A new pa adigm in in e ac-
i e e olu iona y mul iobjec i e op imiza ion,’’ in Pa allel P oblem Sol -
ing F om Na u e—PPSN XVI, T. Bäck, M. P euss, A. Deu z, H. Wang,
C. Doe , M. Emme ich, and H. T au mann, Eds. Cham, Swi ze land:
Sp inge , 2020, pp. 243–256.
[47] S. an de Wal , S. C. Colbe , and G. Va oquaux, ‘‘The NumPy a ay:
A s uc u e o e icien nume ical compu a ion,’’ Compu . Sci. Eng.,
ol. 13, no. 2, pp. 22–30, 2011.
[48] P. Vi anen, R. Gomme s, T. E. Oliphan , M. Habe land, T. Reddy,
D. Cou napeau, E. Bu o ski, P. Pe e son, W. Weckesse , J. B igh , and
S. J. Van De Wal , ‘‘SciPy 1.0: Fundamen al algo i hms o scien i ic
compu ing in Py hon,’’ Na u e Me hods, ol. 17, no. 3, pp. 261–272,
Feb. 2020.
[49] W. McKinney, ‘‘Da a s uc u es o s a is ical compu ing in Py hon,’’ in
P oc. 9 h Py hon Sci. Con ., S. an de Wal and J. Millman, Eds. SciPy,
2010, pp. 56–61.
[50] I. S epona ič˙
e, S. Ruuska, and K. Mie inen, ‘‘A solu ion p ocess o
simula ion-based mul iobjec i e design op imiza ion wi h an applica ion
in he pape indus y,’’ Compu .-Aided Des., ol. 47, pp. 45–58, Feb. 2014.
[51] S. C. Na ula and H. R. Weis o e , ‘‘A lexible me hod o nonlinea mul-
ic i e ia decision-making p oblems,’’ IEEE T ans. Sys ., Man, Cybe n.,
ol. 19, no. 4, pp. 883–887, Jul. 1989.
[52] K. Mie inen and M. M. Mäkelä, ‘‘In e ac i e me hod NIMBUS o nondi -
e en iable mul iobjec i e op imiza ion p oblems,’’ in Mul ic i e ia Anal-
ysis, J. Clímaco, Ed. Be lin, Ge many: Sp inge , 1997, pp. 310–319.
[53] M. D. McKay, R. J. Beckman, and W. J. Cono e , ‘‘A compa ison o h ee
me hods o selec ing alues o inpu a iables in he analysis o ou pu
om a compu e code,’’ Technome ics, ol. 21, no. 2, pp. 239–245, 1979.
[54] K. Mie inen, F. Ruiz, and A. P. Wie zbicki, ‘‘In oduc ion o mul iobjec-
i e op imiza ion: In e ac i e app oaches,’’ in Mul iobjec i e Op imiza ion:
In e a i e and E olu iona y App oaches, J. B anke, K. Deb, K. Mie inen,
and R. Slowinski, Eds. Be lin, Ge many: Sp inge , 2008, pp. 27–57.
[55] B. A sa , D. Podkopae , and K. Mie inen, ‘‘Da a-d i en in e ac i e mul-
iobjec i e op imiza ion: Challenges and a gene ic mul i-agen a chi ec-
u e,’’ P oc. Compu . Sci., ol. 176, pp. 281–290, Jan. 2020.
[56] B. A sa , K. Mie inen, and F. Ruiz, ‘‘Assessing he pe o mance o in e -
ac i e mul iobjec i e op imiza ion me hods: A su ey,’’ ACM Compu .
Su eys, ol. 54, no. 4, p. 85, 2021.
[57] C. Ba ba-González, V. Ojaleh o, J. M. Ga cía-Nie o, A. J. Neb o,
K. Mie inen, and J. F. Aldana-Mon es, ‘‘A i icial decision make d i en
by PSO: An app oach o es ing e e ence poin based in e ac i e me h-
ods,’’ in P oc. 15 h In . Con . Pa allel P oblem Sol ing Na u e—PPSN
XV, A. Auge , C. M. Fonseca, N. Lou enço, P. Machado, L. Paque e, and
D. Whi ley, Eds. Cham, Swi ze land: Sp inge , 2018, pp. 274–285.
[58] S. Hube , M. J. Geige , and M. Se aux, ‘‘Simula ion o p e e ence
in o ma ion in an in e ac i e e e ence poin -based me hod o he bi-
objec i e in en o y ou ing p oblem,’’ J. Mul i-C i e ia Decis. Anal.,
ol. 22, nos. 1–2, pp. 17–35, Jan. 2015.
[59] V. Ojaleh o, D. Podkopae , and K. Mie inen, ‘‘Towa ds au oma ic es ing
o e e ence poin based in e ac i e me hods,’’ in Pa allel P oblem Sol ing
F om Na u e—PPSN XIV, J. Handl, E. Ha , P. R. Lewis, M. López-Ibáñez,
G. Ochoa, and B. Paech e , Eds. Cham, Swi ze land: Sp inge , 2016,
pp. 483–492.
148294 VOLUME 9, 2021
G. Misi ano e al.: DESDEO: Modula and Open Sou ce F amewo k o In e ac i e Mul iobjec i e Op imiza ion
[60] B. A sa , K. Mie inen, and A. B. Ruiz, ‘‘An a i icial decision make
o compa ing e e ence poin based in e ac i e e olu iona y mul iobjec-
i e op imiza ion me hods,’’ in E olu iona y Mul i-C i e ion Op imiza ion,
H. Ishibuchi, Q. Zhang, R. Cheng, K. Li, H. Li, H. Wang, and A. Zhou, Eds.
Cham, Swi ze land: Sp inge , 2021, pp. 619–631.
G. MISITANO ecei ed he M.Sc. deg ee om
he Uni e si y o Jy äskylä, in 2020, whe e
he is cu en ly pu suing he Doc o al deg ee
wi h he Mul iobjec i e Op imiza ion G oup.
His esea ch in e es includes he in e p e abili y
aspec s o in e ac i e mul iobjec i e op imiza ion.
This includes, bu is no limi ed o, esea ch-
ing new ways o make in e ac i e mul iobjec i e
op imiza ion me hods less opaque o he decision
make and analys alike. In addi ion, he is in e -
es ed in s udying how o apply in e p e able and explainable a i icial in el-
ligence o mul iobjec i e op imiza ion in gene al. He is also one o he main
con ibu o s o he DESDEO F amewo k.
B. S. SAINI ecei ed he M.Tech. deg ee om IIT
Kha agpu , in 2018. He is cu en ly pu suing he
Doc o al deg ee wi h he Mul iobjec i e Op imiza-
ion G oup, Uni e si y o Jy äskylä. His esea ch
in e es s include mul iobjec i e op imiza ion, da a
isualiza ion, da a-d i en op imiza ion, and de el-
opmen o e olu iona y algo i hms. He has wo ked
on many open sou ce implemen a ions o he
me hods om he a o emen ioned opics wi h a
ocus on modula i y and in e p e abili y. He is also
one o he p ima y con ibu o s o he DESDEO F amewo k.
B. AFSAR ecei ed he Ph.D. deg ee in compu e
enginee ing om Ege Uni e si y, Izmi , Tu key,
in 2014. He is cu en ly a Pos doc o al Resea che
wi h he Mul iobjec i e Op imiza ion G oup, Uni-
e si y o Jy äskylä. His main esea ch in e es s
include mul iobjec i e op imiza ion, da a-d i en
mul i-c i e ia decision-making, e olu iona y com-
pu a ion, in e ac i e mul iobjec i e op imiza ion
me hods and hei applica ions, and mul i-agen
sys ems. He is also wo king on assessing he
pe o mance o in e ac i e mul iobjec i e op imiza ion me hods wi h bo h
a i icial and human decision-make s.
B. SHAVAZIPOUR ecei ed he Ph.D. deg ee
in ope a ions esea ch om he Uni e si y o
Cape Town, Cape Town, Sou h A ica, in 2018.
He is cu en ly a Pos doc o al Resea che wi h
he Mul iobjec i e Op imiza ion G oup, Uni e -
si y o Jy äskylä. His p incipal esea ch in e es s
include mul iobjec i e op imiza ion and mul i-
c i e ia decision-making bo h in heo y and appli-
ca ions, ma hema ical p og amming, da a analysis
and impac s upon he da a en elopmen analysis,
scena io planning, and decision-making unde (deep) unce ain y.
K. MIETTINEN ecei ed he Ph.D. deg ee in
ma hema ical in o ma ion echnology om he
Uni e si y o Jy äskylä (JYU), Finland. She is cu -
en ly a P o esso o indus ial op imiza ion wi h
JYU. She heads he Resea ch G oup on Mul iob-
jec i e Op imiza ion and is he Di ec o o he he-
ma ic esea ch a ea Decision Analy ics u ilizing
Causal Models and Mul iobjec i e Op imiza ion
(DEMO, jyu. i/demo) a JYU. Wi h he g oup,
she de elops an open sou ce so wa e amewo k
o in e ac i e mul iobjec i e op imiza ion me hods (desdeo.i .jyu. i). She
has au ho ed abou 190 e e eed jou nals, p oceedings, and collec ion
pape s; edi ed 17 p oceedings, collec ions, and special issues; and w i en a
monog aph on nonlinea mul iobjec i e op imiza ion. He esea ch in e es s
include heo y, me hods, applica ions, and so wa e o nonlinea mul iob-
jec i e op imiza ion. She is a membe o he Finnish Academy o Science
and Le e s, Sec ion o Science, and he S ee ing Commi ee o E olu iona y
Mul i-C i e ion Op imiza ion. She has been he P esiden o he In e na ional
Socie y on Mul iple C i e ia Decision Making (MCDM). She has ecei ed
he Geo g Can o Awa d o he In e na ional Socie y on MCDM o de elop-
ing inno a i e ideas. She belongs o he edi o ial boa d o se en in e na ional
jou nals.
VOLUME 9, 2021 148295