Bubel, Ma in e al.
A icle — Published Ve sion
Op imal expe imen al design: om design poin o design
egion
S a is ical Pape s
P o ided in Coope a ion wi h:
Sp inge Na u e
Sugges ed Ci a ion: Bubel, Ma in e al. (2025) : Op imal expe imen al design: om design poin o
design egion, S a is ical Pape s, ISSN 1613-9798, Sp inge , Be lin, Heidelbe g, Vol. 66, Iss. 5,
h ps://doi.o g/10.1007/s00362-025-01725-7
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/323273
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h p://c ea i ecommons.o g/licenses/by/4.0/
REGULAR ARTICLE
S a is ical Pape s (2025) 66:109
h ps://doi.o g/10.1007/s00362-025-01725-7
Abs ac
Op imal expe imen al designs a e used in chemical enginee ing o ob ain p ecise
ma hema ical models. The op imal design consis s o design poin s wi h a maximal
amoun o in o ma ion and hus lead o mo e p ecise models han s a is ical designs.
In gene al, he op imal design depends on an unce ain es ima e o unknown model
pa ame e s
θ
. The op imal designs a e he e o e also unce ain and con inuously
shi in he design space, as he alue o
θ
changes. We p esen wo app oaches o
cap u e his beha io when compu ing op imal designs, a global clus e ing app oach
and a local app oxima ion o he con idence egions. Bo h me hods ind an op imal
design and assign he op imal design poin s con idence egions which can be used
by an expe imen e o decide which design poin s o use. The clus e ing app oach
equi es a Mon e Ca lo sampling o he unce ain pa ame e s and hen iden i ies
egions o high weigh densi y in he design space. The local app oxima ion o
he con idence egions is ob ained ia an e o p opaga ion using he de i a i es
o he op imal design poin s and weigh s. We apply he in oduced app oaches o
ma hema ical examples as well as o an applica ion example modeling apo -liquid
equilib ia.
Keywo ds Con idence egions · Robus op imiza ion · Op imal expe imen al
design
Recei ed: 12 July 2024 / Re ised: 12 May 2025
© The Au ho (s) 2025
Op imal expe imen al design: om design poin o design
egion
Ma inBubel1· PhilippSeu e 1· GlebKa po 2· JanSchwien ek1·
MichaelBo z1· I anOselede s2,3
Ma in Bubel
[email p o ec ed].de
1 F aunho e Ins i u ü Techno- und Wi scha sma hema ik, F aunho e -Pla z 1,
D-67663 Kaise slau e n, Ge many
2 Skol ech, Moscow, Russia
3 A i icial In elligence Resea ch Ins i u e AIRI, Moscow, Russia
1 3
M. Bubel e al.
1 In oduc ion
In mode n indus y, decision-making has inc easingly become model-based, pa icu-
la ly in he p ocess indus y, whe e ma hema ical models a e employed o he design
and op imiza ion o p ocesses. These models, whe he ully empi ical o physically
mo i a ed, ypically con ain unknown pa ame e s ha equi e calib a ion o align
wi h he ac ual sys em. Model calib a ion elies on expe imen al da a ga he ed om
we -lab expe imen s o simula ions such as compu a ional luid dynamics (CFD)
simula ions. Al hough models a e o en u ilized o op imiza ion - such as minimiz-
ing ene gy cos s while main aining h oughpu - he c i ical ask o model calib a ion
mus p ecede his. The e m expe imen al design e e s o a se ies o expe imen s
conduc ed speci ically o model calib a ion. The e ec i eness o an expe imen al
design di ec ly impac s he eliabili y and unce ain y o he calib a ed model Bubel
e al. (2024). Gi en ha expe imen s can be cos ly, he e is a signi ican in e es
in op imal expe imen al designs (OED) speci ically ailo ed o model calib a ion.
While he compu a ion o op imal expe imen al designs is s aigh o wa d o models
ha a e linea in hei pa ame e s, see e.g. Fedo o and Leono (2013), o nonlin-
ea models, app oxima ions a e equi ed, as he e is no gene al closed o m exp es-
sion o model unce ain y Bubel e al. (2025). A common app oach o compu ing
op imal expe imen al designs o nonlinea eg ession models in ol es linea iza ion
o he model by di e en ia ion w. . . he model pa ame e s, which c ea es a depen-
dency on hose pa ame e s’ alues. Consequen ly, i is o en mo e e icien o conduc
expe imen s and calib a e he model i e a i ely, u ilizing sequen ial upda es on model
pa ame e s and he esul ing op imal expe imen al designs.
This wo k low, as depic ed in Fig. 1, begins wi h exis ing da a o a se o ini ial
expe imen s, ollowed by i e a i e model calib a ion on he a ailable da a and he
compu a ion o new op imal expe imen al designs, and has been applied in mul iple
wo ks, e.g. A kinson (2008); Vana e e al. (2020); Bubel e al. (2024). The ansi-
ion om linea o nonlinea models is ho oughly e iewed in Fedo o and Leono
Fig. 1 I e a i e wo k low o model alida ion and
adjus men
1 3
109 Page 2 o 28
Op imal expe imen al design: om design poin o design egion
Fedo o and Leono (2013). Fo linea models, he op imal expe imen al design
emains independen o he e e ence pa ame e alue. In con as , o nonlinea
models, he op imal design is in luenced by hese pa ame e s, necessi a ing he dis-
cussion o model-based op imal expe imen al designs. Many me hods o compu ing
hose, e.g. hose o Yang and S u ken Yang and S u ken (2011) o Dua e e al. Dua e
e al. (2018), a e g ounded in he Kie e -Wol owi z equi alence heo em Kie e and
Wol owi z (1960), which ou lines necessa y and su icien condi ions o OED op i-
miza ion. T adi ional algo i hms o compu ing op imal designs include he e ex
di ec ion me hod (VDM) in oduced by Fedo o Fedo o and Leono (2013) and
Wynn Wynn (1970), he e ex exchange me hod (VEM) Böhning (1986), and he
mul iplica i e algo i hm (MUL) Pukelsheim and To sney (1991); Sil ey e al. (1978).
Recen adap a ions ha e aimed o enhance con e gence and educe un imes, gi ing
ise o me hods such as he cock ail algo i hm Yu (2011), he Yang-Biede mann-Tang
algo i hm Yang e al. (2013), he Weigh ed-Disc e iza ion-App oach Vana e e al.
(2020), MaxVol Mikhale and Oselede s (2018); Vana e e al. (2020), and a andom-
ized exchange algo i hm Ha man e al. (2020).
When dealing wi h nonlinea eg ession models, he dependence o op imal
expe imen al designs on e e ence pa ame e alues, which may be unce ain, in o-
duces unce ain y in o he designs hemsel es. To add ess his, he objec i e unc ion
in he op imiza ion o expe imen al design is modi ied o accoun o unce ain y.
Two p edominan app oaches a e he a e age case app oach Fedo o and Leono
(2013); Asp ey and Macchie o (2002), which eplaces he objec i e unc ion wi h
i s expec ed alue conce ning he unce ain pa ame e s, and he wo s case app oach
Kö kel e al. (2004); Asp ey and Macchie o (2002); Dua e and Wong (2014), which
minimizes he wo s case scena io, also known as he minimax app oach. Se e al
algo i hmic schemes o he wo s case app oach ha e been de eloped Asp ey and
Macchie o (2002); Kö kel e al. (2004); Dua e and Wong (2014), while ecen
wo k has explo ed he condi ional alue a isk in OED con ex s Valenzuela e al.
(2015); Kusumo e al. (2022). The e a e also me hods ha combine expec a ion and
a iance, as p esen ed by Mesbah and S ei (2015). Al e na i e s a egies o man-
age unce ain y in OED include a me hod p oposed by Go u Mukkula and Paulen
(2019), which uses Mon e-Ca lo simula ions o compu e he con idence egion o he
pa ame e es ima e
˜
θ
, p o iding a mo e p ecise es ima e compa ed o linea iza ion
echniques using Fishe in o ma ion ma ices. Addi ionally, mul i-s age designs in o-
duced by Go u Mukkula e al. (2021) in ol e pe o ming a subse o expe imen s
be o e obse ing he pa ame e alue
θ
, wi h he emaining expe imen s planned
based on he assumed knowledge o
θ
. D o and S einbe g D o and S einbe g (2006)
p esen an app oach ha compu es se e al op imal designs o a ying pa ame e
alues, subsequen ly applying a clus e ing algo i hm o ob ain op imal expe imen s.
This me hod was adap ed by Biede mann and Woods Biede mann and Woods (2011),
wi h a summa y a ailable in Chap e 14.6 o he Handbook o Design and Analysis o
Expe imen s Biede mann and Yang (2015). In his pape , we in oduce wo addi ional
app oaches o manage unce ain y in OED. These me hods a e ounded on he con-
cep o eplacing op imal design poin s
xi
wi h op imal egions
Xi⊂X
wi hin he
design space. The i s app oach u ilizes clus e ing o a e age op imal designs ac oss
se e al sampled scena ios, iden i ying egions in he design space wi h high weigh
1 3
Page 3 o 28 109
M. Bubel e al.
densi y. The second app oach examines sensi i i ies in he op imal design ela i e o
he unce ain pa ame e s
θ
. This me hod o e s he ad an age o equi ing only local
in o ma ion abou
θ
. By employing a linea app oxima ion, we can assign each op i-
mal design poin a con idence ellipsoid, which se es as supplemen a y in o ma ion
o de e mining which expe imen s o conduc .
2 Op imal expe imen al design
In his sec ion we gi e a b ie in oduc ion in o he heo y o op imal expe imen al
design. We ocus on he main de ini ions and esul s and also b ie ly discuss op i-
mal expe imen al design unde unce ain y. The de ini ions and esul s mainly ollow
Chap e s 1 and 2 o Fedo o and Leono (2013), bu can also be ound simila ly in
mos OED li e a u e A kinson (2008); Dua e and Wong (2015); Vana e e al. (2020).
In design o expe imen s we conside a pa ame e ized model unc ion
(x, θ)
, which maps om a design space
X⊂Rd
X o an ou pu space
Y⊂Rd
y,
wi h
θ∈Θ⊂Rd
a e he unknown model pa ame e s om he se
Θ
, i.e.
:
X
×Θ→Rd
y,
(
x, θ
)→
(
x, θ
).
(1)
To es ima e he unknown pa ame e s we pe o m expe imen s o design poin s
x1,...,x
n
and ake measu emen s
y1,...,y
n
. We assume ha he exis ence o a se o
ue pa ame e s
θ
such ha he obse a ions a e gi en by
yi= (xi,θ
)+εi∈Rd
y,
whe e he
εi∼N(0,Σ)
a e andom independen iden ically dis ibu ed (i.i.d.) mea-
su emen e o s wi h co a iance ma ix
Σ∈Rd
y
×d
y.
2.1 Locally op imal expe imen al designs
A design (o expe imen al plan)
ξN
o he model consis s o design poin s
xi∈X
and epe i ions
i
o he expe imen s, wi h
∑ i=N
he o al numbe o expe i-
men s. Al e na i ely we can also w i e he numbe o epe i ions ia
p
i
= i
N
. We
obse e ha he alues
pi
sum o 1, and he design
ξN
hus is a (disc e e) p ob-
abili y measu e on he design space X. As is ypical in OED we elax he no ion o
a design and de ine a design measu e
ξ
as a p obabili y measu e on X. We in oduce
Ξ(X)
as he space o all design measu es. Fo any ini e disc e e design measu es
ξ=∑N
i=1
ω
i
δ
xi
, gi en by a ini e se o design poin s
x1,...,x
N∈X
and weigh s
ω1,...,ω
N
, he suppo o
ξ
is gi en by
supp (ξ)={x1,...,x
N}.
(2)
The e iciency o an expe imen al design
ξ
o model calib a ion is equi alen o he
unce ain y o he leas -squa es es ima o
θLSE
. Fo nonlinea eg ession models,
he unce ain y o he leas -squa es es ima o is no known in closed o m, bu can be
app oxima ed using he Fishe in o ma ion ma ix, which is de ined as
1 3
109 Page 4 o 28
Op imal expe imen al design: om design poin o design egion
m(
x, θ
)=
D
θ
(
x, θ
)T·Σ−1·
D
θ
(
x, θ
),
(3)
whe e
Σ
deno es he co a iance ma ix o he measu emen e o s and
Dθ (x, θ)
deno es he Jacobian ma ix o he model wi h espec o he model pa ame e s
θ
.
Consequen ly, he co a iance he o leas -squa es es ima o
θLSE
depends on he
expe imen al design
ξ
as
Co θLSE
≈
(M(ξ,θ))−1=
(∫X
m(x, θ)ξ(dx)
)−1
.
(4)
Commonly, he unce ain y o he leas -squa es es ima o is quan i ied in e ms o
a sui able scala unc ion
Ψ
, usually e e ed o as design c i e ia, see o ins ance
Example 2.1 in Fedo o and Leono (2013). In his wo k, we use he log-D-c i e ion
ΨD
, which is de ined as
Ψ
logD
= log (de (
M
−1)).
(5)
While, om a heo e ical pe spec i e, he op ima o he log-D and he D-c i e ion a e
he same, we p e e he log-D-c i e ion o nume ical easons. The in e es ed eade
is e e ed o he wo k o Amen e al Amen e al. (2024), which ea u es a igo ous
compa ison o objec i e unc ion and hei log-based al e na i es.
Using he app oxima ion o pa ame e unce ain y (4) and he scala mapping (5),
he esul ing op imiza ion p oblem o OED is gi en by
ξ∗= a g min
ξΨ
logD
(
M
(
ξ,
˜
θ
)).
(6)
In (6),
˜
θ
deno es he e e ence alue o which we compu e he Fishe in o ma ion
ma ix
M(ξ, ˜
θ)
. Gi en a pa ame e es ima e
˜
θ
, he solu ion o (6) is conside ed a
locally op imal design, as i is only alid o
θ=˜
θ
.
2.2 Op imal expe imen al designs unde unce ain y
In he p e ious sec ion we ha e conside ed locally op imal designs, which depend on
a e e ence pa ame e alue
˜
θ
. Fo he ask o model calib a ion, he e e ence pa am-
e e alue usually is chosen as he leas -squa es es ima e
˜
θ=θLSE
, which usually is
unce ain. I ,
θLSE
is a om he ue pa ame e alue
θ
, local op imal expe imen al
designs can esul in non-op imal design o he ue pa ame e s
θ
, esul ing in a less
e icien model calib a ion p ocess. An example o his has been shown o ins ance
by Kö kel e al. in Kö kel e al. (2004).
Hedging agains unce ain y in he e e ence pa ame e alue includes he consid-
e a ion o unce ain y in local op imal expe imen al designs. Two classical app oaches
o handle unce ain y in OED a e he a e age case- and he wo s case app oach, p e-
sen ed e.g. in Fedo o and Leono (2013); Asp ey and Macchie o (2002); Dua e
and Wong (2014, 2015). The a e age case design is he solu ion o
1 3
Page 5 o 28 109
M. Bubel e al.
a g min ξ
Eθ
[Ψ(
M
(
ξ,θ
))] ,
(7)
while wo s -case op imal design a e ob ained by sol ing
a g min
ξ
sup
θ∈Θ
Ψ(
M
(
ξ,θ
)).
(8)
Fo bo h app oaches, unde mild assump ions, an equi alence heo em can be o -
mula ed simila o hose o locally op imal designs, see o ins ance Chap e 2.9 in
Fedo o and Leono (2013) o he a e age case p oblem and Asp ey and Macchie o
(2002); Dua e and Wong (2014) o he wo s case p oblem.
Compa ing bo h app oaches, we ind ha he a e age case app oach is known
o be oo op imis ic as i ends o igno e single bad scena ios. This is pa icula ly
p oblema ic, i he ue scena io
θ=θ
is one o hose bad scena ios. In con as , he
wo s case app oach ends o be oo pessimis ic as i only ocuses on he wo s case
scena io, gene ally lea ing oo much oom o imp o emen in a o able scena ios.
While hese app oaches ha e all been success ully applied o op imal expe imen-
al design, hey su e om wo p oblems. Fi s , he men ioned app oaches gi e li le
in o ma ion on he indi idual scena ios and i is somewha unclea , how an op imal
design will pe o m in he unknown ue scena io. Second, he app oaches all equi e
op imal designs in (almos e e y) pa ame e alue
θ
, o simila dis ibu ion in o ma-
ion, o be e alua ed. This in o ma ion can be compu a ionally expensi e o ob ain.
Tha ’s why we p opose wo al e na i e app oaches o handle he unce ain y in he
model pa ame e s
θ
in his wo k, o a i e a a be e unde s anding o unce ain y
con ained in expe imen al designs.
Addi ionally, no e ha while we ocus on he unce ain y in he model pa ame e s
θ
h oughou his wo k, he model unc ion may con ain u he unce ain pa am-
e e s. The p esen ed me hods can easily be ex ended o such pa ame e s.
3 Compu ing op imal egions in he design space
In his sec ion we in oduce wo al e na i e app oaches o handle he unce ain ies
o he pa ame e s
θ
. Ins ead o ocusing on he design c i e ion
Ψ
and applying an
unce ain y app oach o he objec i e unc ion o he op imiza ion p oblem, we wan
o ocus on he op imal design poin s
x∗
i(θ)
. A simila app oach has been p esen ed
in D o and S einbe g (2006) by D o and S einbe g and la e been modi ied by Bie-
de mann and Woods in Biede mann and Woods (2011). They p opose a Mon e-Ca lo
sampling o op imal designs in he design space and subsequen clus e ing algo i hms
o iden i y a obus design.
In he ollowing we assume he unce ain pa ame e s
θ
o be andom a iables
wi h a known p obabili y dis ibu ion. This can o example be a p io dis ibu ion, as
is used o Bayesian design c i e ia.
Fo he wo app oaches de eloped in his wo k, we s a e he ollowing assump ions
Assump ion 1 (i) The design space X is compac .
1 3
109 Page 6 o 28
Op imal expe imen al design: om design poin o design egion
(ii) The mapping
x→ m(x, ˜
θ)
is con inuous o he e e ence alue
˜
θ∈Θ
.
(iii) The e exis s a design
ξ∈Ξ in(
X,
˜
θ
)
o he e e ence alue
˜
θ∈Θ
.
I he abo e assump ions hold o all
θ∈Θ
, we can ind a ini e disc e e op imal
design
ξ∗(θ)
o each alue o
θ
. The design poin s
x∗
i(θ)
and weigh s
ω∗
i(θ)
a e also
andom a iables, as hey depend on
θ
. We expec he design poin s o smea in he
design space, as he alue o
θ
changes. The app oaches p esen ed in his chap e aim
a iden i ying hese egions in he design space.
As each egion is associa ed wi h a andom a iable
x∗
i(θ)
, we wan o app oxi-
ma e he con idence egion o each design poin
x∗
i(θ)
. Compu ing he exac con i-
dence egions is o e ly ambi ious, as he egions may o e lap, and he numbe o
design poin s in an op imal design can change in he alue o
θ
. An exac ep esen a-
ion can only be ound unde s ong ma hema ical assump ions. Howe e , a simple
app oxima ion is possible and gi es he necessa y insigh , in o how he unce ain y o
θ
e ec s each design poin
x∗
i(θ)
and he weigh s
ω∗
i(θ)
, and by ex ension he op imal
design
ξ∗(θ)
.
In bo h p esen ed app oaches we discuss app oxima ions o he mean
µ
and he
co a iance ma ix
Σ
assigned o he design poin s
x∗
i(θ)
. The mean
µ
can hen be
used as ep esen a i e expe imen , whe eas he co a iance ma ix gi es in o ma ion
on he unce ain y assigned o he expe imen . I he design poin
x∗
i(θ)
is no mally
dis ibu ed, i s con idence egion is an ellipsoid spanned by he ma ix
Σ
wi h cen e
µ
. Fo a le el
α∈[0,1]
he con idence ellipsoid is desc ibed by he equa ion
Cα=
y
(
y
−
µ
)TΣ−1(
y
−
µ
)≤
χ
2
1−α(
k
),
(9)
whe e
χ2
1−α(
k
)
deno es he
1−α
quan ile o he
χ
-squa ed dis ibu ion wi h k
deg ees o eedom.
In gene al, he design poin s will no be no mally dis ibu ed. Howe e , he ellip-
soid
Cα
can be used o app oxima e he ac ual con idence egions, o help in e p e
he size o he unce ain y gi en by he co a iance ma ix.
3.1 Clus e ing o design poin s
As i s app oach o iden i y he con idence egions in he design space we p opose a
Mon e-Ca lo based sampling app oach. A e wa ds we apply a clus e ing algo i hm
o iden i y he egions in he design space. In D o and S einbe g (2006); Biede mann
and Woods (2011) D o , S einbe g, Biede mann and Woods also p opose clus e ing
app oaches o iden i y obus designs om a se o sampled op imal designs. They
sugges he k-means clus e ing algo i hm, which iden i ies k clus e s among a se o
design poin s.
We use an al e na i e clus e ing algo i hm. We sample he unce ain pa ame e s
θ
and compu e op imal designs
ξj=ξ∗(θj)
o each sampled scena io
θ1,...,θM
. By
using he equi alence heo em Kie e and Wol owi z (1960) we assume he designs
ξj
o be disc e e designs wi h a bounded numbe o suppo poin s. Wi h he op imal
designs, we can compu e he a e age design
1 3
Page 7 o 28 109
M. Bubel e al.
ξ
=1
M·
M
∑
j
=1
ξj
.
(10)
This design is again a design measu e, which may howe e consis o a as amoun
o suppo poin s, each wi h e y small weigh . Such a design in gene al canno be
used o iden i y expe imen s in applica ions.
Using he clus e ing algo i hm, we ind egions in he design space wi h a high
a e age weigh , o equi alen ly, egions wi h a high weigh densi y. In o de o iden-
i y hese egions we ha e ound he algo i hm DBSCAN Es e e al. (1996); Schube
e al. (2017) o be a sensible choice. This algo i hm akes a lis o poin s wi h co e-
sponding weigh s and iden i ies clus e s in egions wi h high weigh densi y.
We b ie ly discuss how he algo i hm wo ks. The algo i hm depends on wo pa am-
e e s, a sea ch adius
>0
and a minimum weigh
ωmin
. Fo each design poin
xi
he
algo i hm compu es he o al weigh ound in a sphe e o adius a ound his poin
xi
. I his weigh exceeds he h eshold
ωmin
, he poin is assigned as clus e poin . I
he o al weigh is less hen
ωmin
, he poin is disca ded. The algo i hm is well docu-
men ed and p e iously published, hus we e e o he li e a u e, o ins ance Es e
e al. (1996); Schube e al. (2017), as well as he implemen a ion ound in sciki -
lea n Ped egosa and Va oquaux (2011) o de ails.
Compa ed o he k-means algo i hm used by D o and S einbe g D o and S ein-
be g (2006), he DBSCAN algo i hm has he ad an age, ha i does no equi e he
numbe o clus e s o be speci ied. Addi ionally, i also conside s he op imal weigh s
when iden i ying he clus e s, whe eas he k-means algo i hm only wo ks on he sup-
po poin s, he eby assigning each poin he same signi icance.
A e pe o ming he clus e ing we ob ain a numbe o clus e s
cj
, each gi en
by a lis o indices
I
c
j={
i
1
,...,i
N
j}
, such ha he clus e
cj
consis s o poin s
xi,i∈Icj,
and co esponding weigh s
ωi,i∈Icj
. These clus e s desc ibe egions
wi h high weigh -densi y. In a e age he op imal designs
ξ1,...,ξM
alloca e weigh
o hese egions and hus i appea s sensible, o pe o m an expe imen om wi hin
hese egions.
Fo each clus e
cj
we compu e some signi ican pa ame e s.
●Fi s , we compu e he o al weigh o he clus e , gi en by
ω(
cj
)= ∑
i
∈
Ic
j
ωi
.
(11)
This alue indica es he impo ance o he clus e wi h espec o he design
ξ
. The
mo e weigh a clus e has, he highe he p io i y o pe o ming expe imen s om
his clus e .
●Then, we conside he weigh ed mean o he clus e ,
1 3
109 Page 8 o 28
Op imal expe imen al design: om design poin o design egion
In his second egion he e a e many poin s wi h small weigh and he e is no clea
poin o p io i ize.
We now apply he clus e ing algo i hm wi h a sea ch adius o
=0.035
and a
minimal weigh h eshold o
ωmin =0.1
. The algo i hm inds wo clus e s. The co -
esponding clus e means, weigh s and con idence in e als a e plo ed in Fig. 3aand
he clus e s a e also gi en gi en in Table 2. The clus e s ha e a o al weigh o 0.955.
Some design poin s wi h posi i e a e age weigh s co esponding in sum o 0.045 a e
no co e ed by he wo clus e s.
Las we use he implici unc ion heo em o compu e he de i a i es
Dθx∗
i
and
Dθω∗
i
and hen app oxima e he con idence in e als. Fo he de i a i es we
ob ain
Dθx∗
2≈(0,0)T
,
Dθω∗
1≈0
and
Dθω∗
2≈0
. Fo he poin
x∗
1
we compu e
Dθx∗
1≈(0,0.0625)T
. These all co espond o he analy ical de i a i es, which a e
0 o he cons an weigh s and he poin
x∗
2
and
Dθx∗
1= (0,θ−2
2)
. Via he assumed
dis ibu ion o
θ
we app oxima e he con idence in e als. The esul s o he op imal
design poin s a e plo ed in Fig. 3b
We obse e, ha bo h he clus e ing app oach as well as he app oxima ion o he
con idence egions co ec ly depic he e ec o he unce ain pa ame e s. Bo h me h-
ods iden i y he op imal poin
x∗
2=1.0
, which does no change in he pa ame e s
θ
.
This beha io is mi o ed in bo h me hods, as bo h con idence in e als ha e leng h
0. Also he second op imal design egion, cen e ed a ound
x∗
1=0.75
, is iden i ied.
The co esponding design poin changes in
θ
as is indica ed by he compu ed con i-
dence in e als.
As second ma hema ical example we conside an exponen ial unc ion wi h addi-
ional powe e m, gi en by
2:(
x,
(
θ
1
,θ
2)) →
θ
1·exp(−
θ
1·
x
)+
x
θ
2
,
(24)
wi h
X= [0.8,5] ⊂R
and
θ∈R2
.
Fo illus a ion pu poses, we conside he special case whe e only he second
pa ame e
θ2
is o conside able unce ain y. This means, he pa ame e s a e samples
o a no mal dis ibu ion wi h mean
θ0= (1.05,−1.28)T
and co a iance ma ix
Σ=(0
.
00
.
0
0.00.01
).
(25)
This allows o examining he in luence o he unce ain y o he pa ame e
θ2
on he
op imal design. No e, ha he a iance
Va [θ1]=0
, whe eby he pa ame e
θ1
is
de e minis ic and no unce ain. The p esen ed app oaches can also be applied in his
se ing, whe e some o he pa ame e s
θ
a e ixed a hei mean
θ0
.
Using he mean pa ame e alue
θ0
as e e ence alue, we ob ain an op imal
design wi h 3 op imal suppo poin s. To compu e hese alues we eplace he design
space X wi h a ine g id o 841 candida e poin s. The g id is gi en by
Rep esen a i e Weigh Va iance
1 0.7486 0.455 0.052
2 1.0 0.5 0.0
Table 2 Compu ed clus e s o
he exponen ial unc ion
1
wi h
100 sampled pa ame e alues
1 3
Page 15 o 28 109
M. Bubel e al.
Xg id ={0.8+0.005 ·j|j=0,...,840}.
(26)
The alues o he op imal weigh s and he op imal suppo poin s a e gi en in Table 3.
We see, ha he op imal design o he e e ence alue
˜
θ= (1.05,−1.28)T
con-
sis s o h ee suppo poin s. This can howe e change, e en o small changes in
he alue o he e e ence pa ame e . Fo example, he op imal designs o he e e -
ence pa ame e s
(1.05,−1.26)T
and
(1.05,−1.4)T
consis o only 2 op imal design
poin s. Co esponding esul s can be seen in Fig. 4.
We sample
M= 100
poin s
θj
om he dis ibu ion o
θ
and compu e he op imal
designs o he co esponding e e ence alues. Again we use he g id
Xg id
o com-
pu e he co esponding designs
ξj=ξ∗(θj)
. We a e age he designs o ob ain
ξ
=1
M·
M
∑
j
=1
ξj
.
(27)
The a e aged weigh s a e plo ed in Fig. 5a. We obse e in his igu e ha we ha e
h ee egions in he design space which a e o in e es , he bo de s
x=0.8
and
x=5.0
, as well as a small egion a app oxima ely
x≈1.85
. The loca ion o he
h ee op imal design poin s only a ies sligh ly when he alue o
θ
changes.
We apply he clus e ing app oach o he a e aged weigh s, wi h a adius o
=0.05
and a minimum weigh o
ωmin =0.1
. Th ee clus e s a e iden i ied, ma ch-
ing he obse a ions om he a e age design. The clus e ep esen a i es, weigh s
and con idence in e als a e plo ed in Fig. 5a and a e also gi en in Table 4. No e, ha
he plo ed clus e ep esen a i es o e lap he a e age weigh s on he bo de s o he
design space. The clus e s ha e a o al weigh o 1.0 and hus co e all design poin s
wi h posi i e a e age weigh .
Nex we compu e he de i a i es
Dθx∗
i
and
Dθω∗
i
and hen app oxima e he co -
esponding con idence in e als locally. Fo he compu a ion o he con idence in e -
Fig. 4 Op imal designs o he expo-
nen ial unc ion
2
and e e ence pa-
ame e s
˜
θ= (1.05,−1.28)T
( ci cles),
˜
θ= (1.05,−1.26)T
( lowe iangles) and
˜
θ= (1.05,−1.4)T
( uppe iangles)
Design poin
x∗
i
Weigh
ω∗
i
1 0.8 0.1298
2 1.855 0.4878
3 5.0 0.3824
Table 3 Op imal design o he
exponen ial unc ion
2
1 3
109 Page 16 o 28
Op imal expe imen al design: om design poin o design egion
als we only conside he de i a i es wi h espec o he second pa ame e
θ2
and he
co esponding en ies o he co a iance ma ix
Σ
. This is done, as
θ1
has a iance 0
and is unco ela ed o he pa ame e
θ2
. The con idence in e als o he design poin s
a e gi en in Fig. 5b, he con idence in e als o he weigh s a e gi en in Table 5.
We obse e, ha bo h app oaches iden i y h ee egions in he design space which
a e o in e es . This ma ches he obse a ion om he a e age design. The wo poin s
on he bo de o he design space,
x∗
1=0.8
and
x∗
3=5.0
, a e co ec ly iden i ied
by bo h me hods. Bo h me hods assign hese poin s con idence in e als o leng h 0.
Howe e , he local app oxima ion assigns he op imal poin
x∗
2=1.855
a la ge con i-
dence in e al. The a e age design and he clus e ing app oach howe e indica e, ha
his poin only changes sligh ly. This de ia ion is p obably due o he local beha -
io o he implici unc ion app oach, whe eas he clus e ing app oach has access o
global da a.
F om he localized app oxima ion we also ob ain con idence in e als o he op i-
mal weigh s. The weigh s
ω∗
1
and
ω∗
3
ha e la ge con idence in e als. This indica es,
ha hese weigh s a y s ongly in he unce ain pa ame e
θ
. The weigh
ω∗
2
on he
o he hand has a smalle con idence in e al and we hus conclude i is mo e s able.
Design poin
x∗
i
Weigh
ω∗
Con idence in e al o
ω∗
1 0.8 0.2245
[−6
.
0242
,
6
.
2837]
2 1.855 0.4978 [0.1213, 0.8543]
3 5.0 0.2777
[−5.4051,6.1698]
Table 5 Con idence in e als
o he op imal weigh s
ω∗
o
he ma hema ical exponen ial-
powe model
Rep esen a i e Weigh Va iance
1 0.8 0.2245 0.0
2 1.8594 0.4978 0.014
3 5.0 0.2777 0.0
Table 4 Compu ed clus e s o
he exponen ial unc ion
2
and
100 sampled pa ame e alues
Fig. 5 Resul ing op imal designs om he Mon e Ca lo sampling, he clus e ing app oach and he ap-
p oxima ed con idence egions o he exponen ial unc ion
2
1 3
Page 17 o 28 109
M. Bubel e al.
We ha e seen, ha he numbe o suppo poin s changes in he pa ame e alue
θ
. The disappea ance and in pa icula he appea ance o such poin s is no a local
beha io , and hus we canno expec he local me hod o co ec ly p edic his beha -
io o app oxima e i . Howe e , he la ge con idence egions assigned o he weigh s
ω∗
1
and
ω∗
3
, indica e ha hese weigh s may go o 0. Using he app oxima ed g adien s
we can e en es ima e o which pa ame e alues his beha io occu s.
4.2 Applica ion example – he lash
As applica ion example we conside a lash, which o example can be ound in Vana-
e e al. (2020); Seu e e al. (2021, 2024). We gi e a b ie desc ip ion o he model
and he OED se up, whe e we closely ollow Vana e e al. (2020). The lash is a
p essu ized con aine , in o which a liquid inpu s eam
˙
F
wi h composi ion ec o
xF
en e s. A hea du y
˙
Q
is applied in he con aine so ha apo and liquid phase a e in
equilib ium. We ob ain wo ou pu s eams, a liquid ou pu s eam
˙
L
a concen a ion
yL
and a apo ou pu s eam
˙
V
a concen a ion
yV
. The apo -liquid equilib ium
is desc ibed by he so called MESH equa ions. Fo an inpu s eam consis ing o wo
componen s (me hanol and wa e ) hese equa ions a e gi en as Biegle e al. (1997)
●Mass balances
˙
F
·x
F
m
=˙
V·y
V
m
+˙
L·y
L
m
˙
F·x
F
w=
˙
V·y
V
w+
˙
L·y
L
w
(28)
●Equilib ium
P
·y
V
m
=
P
0
m
(
T
)
·y
L
m·γm
(
y
L
m,y
L
w,T,θ
)
P·y
V
w=P
0
w(T)·y
L
w·γw(y
L
m,y
L
w,T,θ)
(29)
●Summa ion
xF
m+xF
w=yV
m+yV
w=yL
m+yL
w=1
(30)
●Hea balance
˙
Q+˙
F·HL
(
x
F
m,x
F
w,T
F
)=
V·HV(y
V
m,y
V
w,T)+
˙
L·HL(y
L
m,y
L
w,T)
(31)
The empe a u e
TF
deno es he empe a u e o he inpu eed, he alue T deno es
he empe a u e a equilib ium in he lash. The unc ions
P0
m,P0
w
deno e he apo
p essu e o he pu e componen s and he unc ions
HL
and
HV
deno e he en halpies
o he mola liquid and apo s eams. The model pa ame e s
θ=(a12,a
21,b
12,b
21)
deno e pa ame e s o he ac i i y coe icien model
γ
Renon and P ausni z (1968)
used o desc ibe he he modynamic equilib ium.
We can ew i e he MESH as gi en abo e as an implici model
1 3
109 Page 18 o 28
Op imal expe imen al design: om design poin o design egion
0=
g
(
x, s, θ
),
(32)
whe e
s:== (yV,yL,T,x
F
w)
is a se o s a e a iables, he inpu a iable
x:== (xF
m,P)
consis s o he concen a ion o me hanol in he eed and he p es-
su e a which he lash ope a es, and g co esponds o he ou Mesh equa ions. We
de ine he mapping
h:R6→R2,s→ (T,yV
m)
and he model unc ion
(
x, θ
) :==
h
(
s
(
x, θ
)),
(33)
whe e
s(x, θ)
deno es an solu ion o he implici model (32) gi en he inpu s
x=(xF
m,P)
and he model pa ame e s
θ
. An exhaus i e discussion on he exis ence
o de i a i es o his sys em can be ound in Schmid 2024 Schmid e al. (2024). The
lash model example demons a es, ha he p oposed me hods can be s aigh o -
wa dly ex ended o implici ly de ined models, which a e commonly encoun e ed in
p ac ical case s udies. Op imal expe imen al design o implici model unc ions has
also been conside ed by Dua e Dua e e al. (2021).
As esul ing design space we selec
X= [0.0,1.0] ×[0.5,5.0]
, i.e. he mola ac-
ion
xF
m
is chosen om wi hin he ange (0.0 o
1
.
0) mol
mol
and p essu e om he ange
(0.5 o 5.0) ba . As ou pu s o he model we measu e he alues
(
T,y
V
m)=
(
x, θ
)
,
he empe a u e T a equilib ium and he mola ac ion
yV
m
o me hanol in he apo
ou pu s eam, y. We assume he model pa ame e s
θ=(
a
12
,a
21
,b
12
b
21)T
o be no -
mally dis ibu ed wi h mean
E[θ]=(−3.9402,6.3037,1337.558,−1891.945)T.
(34)
The alues o he s anda d de ia ions a e ob ained by assigning each pa ame e a 5
pe cen de ia ion om i s nominal alue, i.e.
Co [θ]ii = (0.05 ·θ0
i)2
, esul ing in
he co a iance ma ix
Co [θ]≈
0
.
0388 0 0 0
00.0993 0 0
0 0 4472.65 0
0 0 0 8948.64
.
(35)
Using he expec a ion
θ0
as e e ence alue we can compu e an op imal design. Fo
his pu pose we eplace he design space X wi h a uni o m g id, wi h 101 g id poin s
in he
xF
dimension and 91 g id poin s in he P dimension. The esul ing g id is gi en
by
X
g id =
i
100,j
20
T
i=0,...,100,
j= 10,...,100
(36)
and he op imal design
ξ∗
is gi en in Table 6.
Fo he lash we sample
M= 200
di e en alues
θj
o he model pa ame e s
om he desc ibed dis ibu ion
N(θ0,Σθ)
and compu e he co esponding op imal
1 3
Page 19 o 28 109
M. Bubel e al.
designs
ξj
on he speci ied g id
Xg id
. In Fig. 6a he co esponding suppo poin s a e
plo ed. The size o he poin s and he colo ing he eby indica e he weigh assigned o
each poin . As all compu ed op imal design poin s ha e a mola ac ion
xF
m≤0.6
,
we only plo he ele an pa o he design space and exclude all alues
xF
m>0.6
.
Nex , we apply he clus e ing algo i hm o he a e aged weigh s. The esul o his
me hod can be seen in Fig. 6b. He e, each clus e
cj
is ep esen ed ia i s mean poin
cj,mean
i s weigh
ω(cj)
and i s con idence ellipsoid. Fo he clus e ing we ha e used
a adius
=0.04
and a minimal weigh
ωmin =0.04
. The esul s a e also gi en in
Table 7. The clus e s ha e a o al weigh o 0.9015. Thus design poin s wi h posi i e
a e age weigh s co esponding in sum o 0.0985 a e no co e ed by he clus e s.
Las we compu e he de i a i es
Dθx∗(θ0),D
θω∗(θ0)
and app oxima e he con-
idence egions ia he implici unc ion heo em. The con idence in e als o he
Rep esen a i e Weigh
Co 11
Co 12
Co 22
1
(0
.
1533
,
0
.
5)T
0.2030 0.0023 0.0 0.0
2
(0
.
3310
,
1
.
53)T
0.1775 0.0005
−
0.0022 0.0475
3
(0
.
2224
,
1
.
65)T
0.0495 0.0002
−
0.0005 0.0056
4
(0
.
0746
,
5
.
0)T
0.2330 0.0001 0.0 0.0
5
(0
.
3464
,
5
.
0)T
0.2385 0.0014 0.0 0.0
Table 7 Compu ed clus e s o
he lash
Fig. 6 Resul ing op imal designs om he Mon e Ca lo sampling, he a e aging o weigh s and he
clus e ing app oach o he lash
Design poin
x∗
i
Weigh
ω∗
i
1
(0.07,5.0)T
0.2317
2
(0.1,2.0)T
0.0493
3
(0.16,0.5)T
0.2497
4
(0.33,1.7)T
0.2257
5
(0.34,5.0)T
0.2435
Table 6 Op imal design o he
lash on he g id
Xg id
and
e e ence pa ame e s
θ0
1 3
109 Page 20 o 28
Op imal expe imen al design: om design poin o design egion
weigh s a e gi en in Table 8. Un o una ely he app oxima ed con idence egions a e
e y la ge and canno be plo ed app op ia ely, hus we o ego such a plo .
F om he esul s we obse e ha he clus e ing app oach gi es easonable app oxi-
ma ions o he con idence egion. We alida e hese esul s by compu ing he alues
Y
j
:== ∑
i
∈
Ij
ωi
(
θj
),
whe e
θj,j =1,...,200,
deno e he sampled pa ame e alues om he clus e ing
app oach and
Ij
deno es he indices o he op imal design poin s
x∗
i(θj)
, which a e
wi hin one o he app oxima ed con idence egions. Fo he alues
Yj
some s a is i-
cal igu es a e gi en in Table 9. We obse e, ha o hal he samples we achie e a
co e age o
94%
, and we only ha e a co e age o less han
84%
in one qua e o
e alua ed cases.
In con as o he ma hema ical examples, he local app oach does no yield ea-
sonable app oxima ions, esul ing in con idence ellipsoids ha a e excessi ely la ge
and exceed he physically plausible ange o he concen a ion alues
x1
. We a i-
bu e his o wo main easons. Fi s , he a iance conside ed in he lash example is
subs an ial, leading o app oxima ion e o s in local app oach. Second, he nonlin-
ea i y o he op imal expe imen al design in
θ
con ibu es o u he app oxima ion
e o s when employing linea iza ion-based me hods. Bo h o hese e ec s we e dis-
cussed in Sec . 3.3, whe e we no ed ha he implici unc ion app oach is only locally
accu a e. Fo his speci ic example, we conclude ha he p ima y eason o he ail-
u e o he local app oach is he high nonlinea i y p esen in he NRTL model and
i s pa ame e iza ion. As analyzed by Laba a e al. Laba a e al. (2022), he NRTL
model can exhibi highly nonlinea beha io , which appea s o be he case wi h he
chosen pa ame e s in his example. While pa ame e ans o ma ions, as p oposed
by Ba es and Wa s Ba es and Wa s (1981), may mi iga e nonlinea i y, hey a e no
uni e sally applicable o all nonlinea eg ession models, including he NRTL model;
S a is ical igu es
mean 0.914175
s anda d de ia ion 0.095014
minimum 0.499907
25% quan ile 0.848292
50% quan ile 0.942115
75% quan ile 1.000000
maximum 1.000000
Table 9 Mean, S anda d De ia-
ion and quan iles o he alues
Yj
Design poin
x∗
i
Weigh
ω∗
Con idence in e al o
ω∗
1
(0.07,5.0)T
0.2317
[0.1807,0.2828]
2
(0.1,2.0)T
0.0493
[−0.1309,0.2296]
3
(0.16,0.5)T
0.2497
[0.2476,0.2518]
4
(0.33,1.7)T
0.2257
[0.1272,0.3243]
5
(0.34,5.0)T
0.2435
[0.2103,0.2767]
Table 8 Con idence ellipsoids
o he op imal weigh s
ω∗
o
he lash
1 3
Page 21 o 28 109
M. Bubel e al.
hence, we canno ci cum en he nonlinea i y in his ins ance. To de i e easonable
in o ma ion om he local app oach, we ecommend dec easing he co a iance o he
model pa ame e s. Fo his pa icula example, we ind ha mul iplying he diagonal
en ies in (35) by a ac o o 0.1 o 0.01 is easonable o achie e con idence ellipsoids
compa able in size o hose ob ained om he clus e ing app oach. Subsequen ly, he
compu ed local de i a i es (see Appendix A) o he op imal design poin s om he
lash example can be u ilized o de e mine how much and in which di ec ion he op i-
mal design poin s shi unde unce ain y. Al hough some in e e ence is necessa y
(scaling o pa ame e co a iance), he expe imen e may s ill gain insigh s in o which
expe imen s a e mo e and less s able, espec i ely. Ul ima ely, he chosen noise le el
bo h e lec s a ealis ic scena io om he p ocess enginee ing domain and nicely
demons a es he supe io s abili y o he clus e ing app oach o e he local app oach,
which is why we decided o keep he ela i ely la ge pa ame e co a iances in (35)
o he lash example.
5 Conclusion
In his pape we ha e p esen ed wo app oaches o handle unce ain y and unknown
model pa ame e s in op imal expe imen al design.
The i s app oach is a clus e ing app oach which u ilizes he DBSCAN algo i hm.
This app oach i s a e ages he op imal weigh s and hen u ilizes he algo i hm o
iden i y egions in he design space wi h a high weigh densi y. We iden i y each
egion wi h a ep esen a i e poin , weigh and co a iance ma ix. The clus e s co -
espond o design egions in which OED assigns a la ge weigh o e all sampled
scena ios. Pe o ming expe imen s in hese egions hus seems easonable, ega d-
less o he ue alue o he unce ain pa ame e s. Via he co a iance ma ix we also
ob ain in o ma ion on how unce ain each clus e is and can app oxima e con idence
egions.
The main disad an age o his app oach is, ha we need o sample he pa ame e
space
Θ
and compu e op imal designs o each o he samples. Simila o al e na i e
app oaches like he wo s -case design o he a e age case designs his can be compu-
a ionally expensi e, in pa icula in high dimensional se ings.
The second app oach app oxima es he con idence ellipsoids o he op imal design
poin s
x∗
i
and op imal weigh s
ω∗
i
. This app oach u ilizes he implici unc ion heo-
em o compu e he de i a i es
Dθx∗
i
and
Dθω∗
i
. As his is a local app oach in
x∗
i,ω∗
i
and in he nominal pa ame e
θ0
, we equi e signi ican ly less e alua ions o he
model and i s Jacobian ma ix
Dθ (x, θ)
. In he examples we obse e, ha his
app oach p o ides addi ional in o ma ion o each compu ed op imal design
ξ∗
. Via
he con idence ellipsoids an expe imen e can make a mo e in o med decision on
which expe imen s o pe o m.
Howe e , his app oach only is a local app oach and may lead o w ong and
misleading app oxima ions. This is pa icula ly he case, when he op imal design
poin s and op imal weigh s a e e y non-linea in
θ0
. La ge a iances in he unce -
ain pa ame e s can also be p oblema ic, as he linea iza ions applied ge wo se o
1 3
109 Page 22 o 28
Op imal expe imen al design: om design poin o design egion
la ge de ia ions om he nominal (expec ed) scena io. This e ec is seen in he lash
example o his con ibu ion.
Combining bo h app oaches, he da a poin s om he clus e s can be used o com-
pu e highe momen s a each clus e . These momen s can be used o in es iga e he
non-linea i y o each op imal design poin . This can hen indica e, how much we us
he local app oxima ions a each poin .
Las , we ha e seen ha in some examples he weigh
ω∗
i
assigned o an op imal
design poin can go o 0 as
θ
changes. The op imal design poin
x∗
i
is hen no longe
pa o he design and is emo ed. Via he g adien and con idence in e als on he
op imal weigh s
ω∗
i
, we can p edic unde which pa ame e changes he weigh con-
e ges o 0. The opposi e beha io , when a design poin is suddenly assigned some
posi i e weigh , is mo e di icul o p edic and can be pa o u u e esea ch.
De i a i es o designs ia he implici unc ion heo em
In his Appendix we gi e de ails on he applica ion o he implici unc ion heo em o
he Ka ush-Kuhn-Tucke (KKT) condi ions o he op imal expe imen al design p ob-
lem. This ollows ideas om he so-called pa ame ic op imiza ion p oblem Ja e and
S oe (2004).
We conside he op imiza ion p oblem
min
ω,x
1,...,xn
˜
Ψ(
ω,x1,...,x
n,θ
0)
s. . gi(ω)≤0 o i=0,...,n
h
j
(x
i
)
≤
0
o
i=1,...,n, j =1,...,2
·
d
x,
(37)
in oduced o he implici unc ion app oach. To imp o e no a ion we deno e
x=(x1,...,x
n)∈Rn×d
x.
The Lag angian unc ion o his p oblem is gi en by
Lθ:(
ω,
x
,λ
)→
L
θ(
ω,
x
,λ
),
(38)
wi h
L
θ
(
ω,
x
,λ
)=
˜
Ψ(ω,x,θ)+
n
∑
i=0
λi·gi(ω)+
n
∑
i=1
2·dx
∑
j=1
λij ·hj(xi)
.
(39)
As he poin s
x∗
i(θ0)
and weigh s
ω∗
i(θ0)
a e op imal o he alue
θ0
, we ind
Lag ange mul iplie s
λ∗
such ha he KKT condi ions o
Lθ
0 a e ul illed. Then
D
ωiLθ0
(
ω
∗
,
x
∗,λ
∗)=0,
D
x
i
Lθ0
(
ω∗,x
∗
,λ
∗
)=0.
(40)
1 3
Page 23 o 28 109
M. Bubel e al.
In he ollowing we deno e by
I
∗=
(i, j)
hj(xi)=0,i
=1
,...,n,
j=1,...,2
·
dx
(41)
he indices o he ac i e cons ain s. The co esponding Lag ange mul iplie s a e
deno ed by
λI∗=(
λ
ij
,
(
i, j
)∈
I
∗),
(42)
and he cons ain s by
hI∗(
x
1
,...,x
n)=(
h
j(
x
i)
,
(
i, j
)∈
I
∗).
(43)
Fo he implici unc ion heo em we de ine he unc ion
F
:(ω,x, λ, θ)
→ (
DωLθ
(
ω,
x
,λ
)
DxLθ0(ω,x,λ)
)
(44)
We conside he de i a i e o F wi h espec o
(ω∗,x∗,λ
∗
0,λ
∗
I∗)
, gi en by he ma ix
DF
:==
(
DF∗
(
Dh∗
)
T
Dh∗0
),
(45)
whe e we deno e
DF∗=(
DωF
(
ω
∗
,
x∗
,λ
∗
,θ
0)
,D
x
F
(
ω
∗
,
x∗
,λ
∗
,θ
0)),
(46)
and
Dh
∗=
(
Dωg0
(
ω
∗) 0
0DxhI∗(x∗)
).
(47)
We hen conside he de i a i e o F wi h espec o
θ
, gi en by
DG
:==
(
DθF
(
ω
∗
,
x∗
,λ
∗
,θ
0)
0
0
)
(48)
I he ma ix DF is non-singula , we can apply he implici unc ion heo em and
compu e he de i a i es
Dθω
∗(
θ
0)
Dθx∗(θ0)
Dθλ∗
0(θ0)
D
θ
λ
∗
I∗(
θ0
)
=−DF−1·
DG
(49)
1 3
109 Page 24 o 28
