The continuous stochastic gradient method: part II–application and numerics

G ieshamme , Max; P lug, Lukas; S ingl, Michael; Uihlein, And ian A icle — Published Ve sion The con inuous s ochas ic g adien me hod: pa II– applica ion and nume ics Compu a ional Op imiza ion and Applica ions P o ided in Coope a ion wi h: Sp inge Na u e Sugges ed Ci a ion: G ieshamme , Max; P lug, Lukas; S ingl, Michael; Uihlein, And ian (2023) : The con inuous s ochas ic g adien me hod: pa II–applica ion and nume ics, Compu a ional Op imiza ion and Applica ions, ISSN 1573-2894, Sp inge US, New Yo k, NY, Vol. 87, Iss. 3, pp. 977-1008, h ps://doi.o g/10.1007/s10589-023-00540-w This Ve sion is a ailable a : h ps://hdl.handle.ne /10419/312463 S anda d-Nu zungsbedingungen: Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen Zwecken und zum P i a geb auch gespeiche und kopie we den. Sie dü en die Dokumen e nich ü ö en liche ode komme zielle Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich machen, e eiben ode ande wei ig nu zen. So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen (insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en, gel en abweichend on diesen Nu zungsbedingungen die in de do genann en Lizenz gewäh en Nu zungs ech e. Te ms o use: Documen s in EconS o may be sa ed and copied o you pe sonal and schola ly pu poses. You a e no o copy documen s o public o comme cial pu poses, o exhibi he documen s publicly, o make hem publicly a ailable on he in e ne , o o dis ibu e o o he wise use he documen s in public. I he documen s ha e been made a ailable unde an Open Con en Licence (especially C ea i e Commons Licences), you may exe cise u he usage igh s as speci ied in he indica ed licence. h ps://c ea i ecommons.o g/licenses/by/4.0/ Compu a ional Op imiza ion and Applica ions (2024) 87:977–1008 h ps://doi.o g/10.1007/s10589-023-00540-w The con inuous s ochas ic g adien me hod: pa II–applica ion and nume ics Max G ieshamme 1·Lukas P lug1,2 ·Michael S ingl1·And ian Uihlein1 Recei ed: 7 Feb ua y 2023 / Accep ed: 26 Oc obe 2023 / Published online: 24 No embe 2023 © The Au ho (s) 2023, co ec ed publica ion 2024 Abs ac In his con ibu ion, we p esen a nume ical analysis o he con inuous s ochas- ic g adien (CSG) me hod, including applica ions om opology op imiza ion and con e gence a es. In con as o s anda d s ochas ic g adien op imiza ion schemes, CSG does no disca d old g adien samples om p e ious i e a ions. Ins ead, design dependen in eg a ion weigh s a e calcula ed o o m a con ex combina ion as an app oxima ion o he ue g adien a he cu en design. As he app oxima ion e o anishes in he cou se o he i e a ions, CSG ep esen s a hyb id app oach, s a ing o like a pu ely s ochas ic me hod and beha ing like a ull g adien scheme in he limi . In his wo k, he e iciency o CSG is demons a ed o p ac ically ele an applica ions om opology op imiza ion. These se ings a e cha ac e ized by bo h, a la ge num- be o op imiza ion a iables and an objec i e unc ion, whose e alua ion equi es he nume ical compu a ion o mul iple in eg als conca ena ed in a nonlinea ashion. Such p oblems could no be sol ed by any exis ing op imiza ion me hod be o e. Las ly, wi h ega ds o con e gence a es, i s es ima es a e p o ided and con i med wi h he help o nume ical expe imen s. Keywo ds S ochas ic g adien scheme ·Con e gence analysis ·S ep size ule · Back acking line sea ch ·Cons an s ep size BAnd ian Uihlein [email p o ec ed] Max G ieshamme [email p o ec ed] Lukas P lug [email p o ec ed] Michael S ingl [email p o ec ed] 1Depa men o Ma hema ics, Chai o Applied Ma hema ics, F ied ich-Alexande -Uni e si ä E langen-Nü nbe g (FAU), E langen, Ge many 2FAU Compe ence Cen e Scien i ic Compu ing, F ied ich-Alexande -Uni e si ä E langen-Nü nbe g (FAU), E langen, Ge many 123 978 M. G ieshamme e al. Ma hema ics Subjec Classi ica ion 65K05 ·90C06 ·90C15 ·90C30 1 In oduc ion In his pape , we p esen a nume ical analysis o he Con inuous S ochas ic G adien (CSG) me hod, which was i s p oposed in [1]. La e , in [2], i was shown ha he e o in he CSG g adien and objec i e unc ion app oxima ion anishes du ing he cou se o he i e a ions. This key p ope y o CSG yields s ong con e gence esul s known om classic g adien me hods, e.g., con e gence o he sequence o i e a es o cons an s ep sizes, which a e beyond he scope o s anda d s ochas ic app oaches known om li e a u e, like he S ochas ic G adien (SG) me hod [3], o he S ochas ic A e age G adien (SAG) me hod [4]. Fu he mo e, he app oxima ion p ope y o CSG signi ican ly inc eases he se o possible applica ions, allowing o mo e complex s uc u es in he op imiza ion p oblem han he schemes lis ed be o e. While CSG was shown o pe o m be e han a ious s ochas ic op imiza ion app oaches on academic examples [2], i emains o see i his is also he case o mo e in ol ed applica ions. Fo his pu pose, we conside se e al op imiza ion p oblems a ising in he con ex o op imal nanopa icle design. These applica ions ocus on op imiza ion wi h espec o he esul ing colo o a pa icula e p oduc , as i ep esen s one o he mos p ominen ields o esea ch wi hin his se ing [5–10]. Mo eo e , all con e gence esul s s a ed in [2] p o ide no insigh on he a e o con e gence. Since his plays a c ucial ole o he p ac icabili y o CSG, i is o g ea impo ance o u he analyze his quan i y. In his con ibu ion, we conjec u e es ima ed con e gence a es o he gene al CSG me hod and e i y hem nume ically. 1.1 S uc u e o he pape Sec ion 2in oduces he applica ion om nanopa icle op ics, men ioned abo e. Two di e en me hods o model he pa icle, a ying g ea ly in compu a ional e o and design dimension, a e p esen ed. A e de ailing he se ing and challenges in he low-dimensional op imiza ion p oblem, we compa e he esul s o he CSG me hod o di e en app oachesbasedon he mincon algo i hmp o idedbyMATLAB(Sec .2.7). La e on,weanalyze hehigh-dimensionalp oblem o mula ionpu elywi hin heCSG amewo k, since a compa ison wi h gene ic de e minis ic op imiza ion schemes is ou o scope, due o he associa ed compu a ional complexi y. A e wa ds, Sec .3sho ly co e s echniques o es ima e he g adien app oxima- ion e o du ing he op imiza ion, be o e we ocus on he con e gence a e o CSG in Sec .4. While he expec ed a es s a ed he ein a e no p o en, we p esen de ailed nume ical examples o solidi y ou claims. Fu he mo e, we analyze how he con e - gence a edependson hedimensiono in eg a ionandhow oa oidslowcon e gence, i he objec i e unc ion admi s addi ional s uc u e. 123 The con inuous s ochas ic g adien me hod 979 2 Nanopa icle design op imiza ion Since he design o a nanopa icle, i.e., i s shape, size, ma e ial dis ibu ion, e c., hea ily impac s i s op ical p ope ies, he ask o op imizing a nanopa icle design wi h espec o a speci ic op ical p ope y a ises na u ally [11]. In his sec ion, we a e in e es ed in using hema i e nanopa icles o op imize he colo o a pain ilm [12]. Thus, we s a by in oducing ou main amewo k o his applica ion. 2.1 Colo spaces Fi s o , we should explain wha op imal colo means in ou se ing. The e a e se e al di e en me hods o desc ibe colo ma hema ically, e.g., assigning each colo an RGB ep esen a ion ec o ∈R3, whe e he h ee componen s o co espond o he ed, g een and blue alue o he colo . In ou applica ion, we a e in e es ed in he colo o he pain ilm as i appea s o he human eye. The e o e, he unde lying colo space should be chosen based on he ollowing p ope y: I he Euclidean dis ance be ween he ep esen a ion ec o s o wo colo s is small, he colo s should be almos indis inguishable o he human eye. As i u ns ou , he RGB colo space is a e y poo choice wi h espec o his ea u e. Hence, we ins ead choose he CIELAB colo space [13], which was in oduced by he In e na ional Commission o Illumina ion (Commission In e na ionale de l’Eclai age, CIE), as i was designed wi h his exac pu pose in mind. The CIELAB ep esen a ion o a colo consis s o h ee alues L,aand b. He e, Lco esponds o he ligh ness o a colo and anges om 0 (black) o 100 (whi e). The alues o aand b, ypically wi hin he ange o ±150, desc ibe he colo s posi ion wi h espec o he opponen colo pai s g een- ed and blue-yellow. A sho o e iew is gi en in Fig.1. Ano he colo space, which na u ally a ises om ou se ing, is he CIE 1931 XYZ colo space [14]. The alues o X, Y and Z can be calcula ed by in eg a ing he op ical p ope ies o a pa icle o e he spec um o isible ligh (400–700 nm), which we deno e by . Each o hese in eg a ions is weigh ed by he co esponding colo ma ching unc ions x,y,z:→R. Thus, in ou applica ion, we will i s calcula e he CIE 1931 XYZ ep esen a ion o he esul ing colo and hen use he (nonlinea ) colo space ans o ma ion : R3→R3wi h (X,Y,Z)=(L,a,b), o wo k in he CIELAB colo space. Fo his ans o ma ion, we de ine a e e ence whi e poin ⎛ ⎝ X Y Z ⎞ ⎠=⎛ ⎝ 94.72528492 100 107.13012997⎞ ⎠ and deno e he ela i e XYZ alues by ˜ X=X X ,˜ Y=Y Y ,and ˜ Z=Z Z . 123 980 M. G ieshamme e al. Fig. 1 Resul ing colo o a ious di e en alues o aand b. Posi i e alues o a esul in ed colo s, while colo s co esponding o nega i e alues o aappea g een. Simila ly, posi i e b alues yield yellow colo s, while nega i e b alues shi he colo in o he blue spec um. In his igu e, we ixed L=50 U ilizing he in ended CIE pa ame e s =216 24389 and κ=24389 27 , he LAB colo alues a e hen gi en by L=116 (˜ Y)−16,a=500 (˜ X)− (˜ Y)and b=200 (˜ Y)− (˜ Z), whe e :R→Ris de ined as ( )=3 √ i > κ +16 116 o he wise . 2.2 Mie heo y and disc e e dipole app oxima ion Gi en a nanopa icle shape and ma e ial, we can use he ime-ha monic Maxwell’s equa ions o calcula e i s op ical p ope ies. Speci ically, in ou se ing, we a e in e - es ed in he abso p ion (Abs), sca e ing (Sca) and geome y ac o (Geo) [15, Sec ion 2.8]. These p ope ies desc ibe he in e ac ions o a pa icle wi h ligh and a e he e o e dependen no only on he pa icle’s design, bu also i s o ien a ion w. . . he incoming ligh wa e as well as he wa eleng h o said ligh . The ime equi ed and p ecision achie ed in hei nume ical calcula ion a e, o cou se, dependen on ou model o he nanopa icle and he me hod used o sol e Maxwell’s equa ions. Fo ou se ing, we choose wo di e en app oaches. On he one hand, we will use he disc e e dipole app oxima ion (DDA) [16–18], in which he pa icle is disc e ized in o an equidis an g id o dipole cells. Thus, DDA allows he analysis o a bi a y pa icle shapes and ma e ial dis ibu ions. The down- side lies wi hin he compu a ional complexi y o he me hod, which scales wi h he o al numbe o dipoles and he e o e g ows apidly when inc easing he esolu ion. 123 The con inuous s ochas ic g adien me hod 981 While he CSG me hod is s ill capable o sol ing he esul ing op imiza ion p ob- lem in ou expe imen s, he emendous compu a ional cos associa ed o he DDA app oach se e ely impede a de ailed analysis o he p oblem. Especially, he e is no compu a ionally easible, gene ic op imiza ion scheme o compa e ou esul s wi h. Howe e , we wan o no e ha op imiza ion in he DDA model has al eady been done in a sligh ly simple se ing, whe e he ull in eg al o e was eplaced by summa ion o e a small numbe o di e en wa eleng hs [19]. On he o he hand, Mie heo y [20,21] p o ides a nume ically cheap al e na i e, a he p ice o a mo e es ic i e se ing. In Mie heo y, one only conside s adially symme ic pa icles. In his special se ing, i is possible o ind analy ic solu ions based on se ies expansions o he ime-ha monic Maxwell’s equa ions. The e o e, in ou i s app oach, we will only conside co e-shell pa icles, as he u iliza ion o Mie heo y allows o a much deepe analysis o he esul ing op imiza ion p oblem and compa ison o de e minis ic op imiza ion app oaches, which ely on disc e iza ion o he in eg als. 2.3 Nanopa icles in pain ilm—Kubelka–Munk heo y As men ioned abo e, he XYZ colo alues o he pain ilm can be calcula ed by in eg a ion o he co esponding colo ma ching unc ions x,y,zand he impo an op ical p ope ies o he nanopa icle. The p ecise me hod o ob ain X, Y and Z is gi en by he Kubelka–Munk heo y [22], augmen ed by a Saunde son co ec ion [23]. Fo a pain ilm, in which nanopa icles wi h design ua e o ien ed in di ec ion ν∈S2, ha is illumina ed by ligh wi h wa eleng h λ∈, he esul ing colo can be exp essed by he Kand S alue K(u,λ,ν)=Abs(u,λ,ν) and S(u,λ,ν)=Sca(u,λ,ν) 1−Geo(u,λ,ν)  ia he e lec ance R∞(u,λ,ν)=1+8 3 K(u,λ,ν) S(u,λ,ν) −8 3 K(u,λ,ν) S(u,λ,ν)2 +16 3 K(u,λ,ν) S(u,λ,ν) . Now, X, Y and Z can be ob ained by X(u,ν)= x(λ)(1−ρ0−ρ1)R∞(u,λ,ν)+ρ0 1−ρ1R∞(u,λ,ν) dλ, Y(u,ν)= y(λ)(1−ρ0−ρ1)R∞(u,λ,ν)+ρ0 1−ρ1R∞(u,λ,ν) dλ, Z(u,ν)= z(λ)(1−ρ0−ρ1)R∞(u,λ,ν)+ρ0 1−ρ1R∞(u,λ,ν) dλ, whe e ρ0and ρ1a e ma e ial pa ame e s. In ou se ing, which we in oduce in he nex sec ion, we ha e ρ0=0.04 and ρ1=0.6. Mo eo e , x,yand za e he colo ma ching unc ions, as gi en in [24]. 123 982 M. G ieshamme e al. Fig. 2 Radially symme ic co e-shell nanopa icle. The inne co e (blue) has adius Rin he ange o 1–75 nm and consis s o wa e . The hickness o he hema i e shell ( ed) is deno ed by dand anges om 1 o 250nm 2.4 P oblem o mula ion In ou i s se ing, we conside a adially symme ic co e-shell nanopa icle (see Fig.2), whe e heinne co e consis so wa e , while he ou e shell ismade o hema i e. Thus, he design uconsis s o he adius R(1–75 nm) o he co e and he hickness d(1–250 nm) o he ou e hema i e shell, i.e., we ha e u=(R,d)∈U=[1,75]× [1,250]. Due o he symme y o he pa icle, i s op ical p ope ies do no depend on he o ien a ion ν∈S2, which is why we omi i in ou u he analysis o his se ing. As an addi ional laye o di icul y, we can, in p ac ice, no expec all nanopa icles p esen in he pain ilm o be iden ical copies o design u. Ins ead, when ying o p oduce nanopa icles o a speci ic design in la ge quan i ies, one usually ends up wi h a mix u e o pa icles o di e en designs, ollowing a ce ain p obabili y dis ibu ion μu, which is dependen on he in ended design u. We model his aspec by assuming ha , gi en a design u=(R,d), he pa icles p esen in he pain ilm ollow a unca ed no mal dis ibu ion on he space o eason- able designs R×D=[10−4,150]×[10−4,500]cen e ed a ound u, i.e., ˜ R∼NR(R,1 10 R)and ˜ d∼ND(d,1 10d). T unca ing he no mal dis ibu ion o he space R×Dci cum en s nonphysical pa - icles appea ing in he design dis ibu ions, like designs wi h nega i e componen s. F om a nume ical poin o iew, he impac is negligible, as he combined weigh o all excluded designs is below ypical machine p ecision, since a design componen mus de ia e om he a e age by mo e han 9 s anda d de ia ions in o de o be ejec ed. As he pain ilm no longe consis s o iden ical pa icles, he Kand S alues in he Kubelka–Munk model need o be eplaced by hei a e aged coun e pa s K(u,λ)=R×D Abs(˜ R,˜ d,λ)dμu(˜ R,˜ d) and S(u,λ)=R×D Sca(˜ R,˜ d,λ) 1−Geo(˜ R,˜ d,λ) dμu(˜ R,˜ d), 123 The con inuous s ochas ic g adien me hod 983 be o e calcula ing he e lec ance R∞(u,λ)and in eg a ing i o e . The objec i e in ou applica ion is o p oduce a pain o b igh ed colo . Thus, he comple e op imiza ion p oblem eads max u∈U 1 20 L(u)+19 20 a(u). (1) Due o he compac ness o U,Rand D,[2, Assump ion 2.2] is ob iously sa is ied. Fu - he mo e, he mapping om a design u, wa eleng h λand o ien a ion ν o he op ical p ope ies Abs, Sca and Geo is smoo h [25, Eqs. 1a, 1b, 1c]. Since e e y admissible design has a hema i e shell o posi i e hickness, we ob ain a lowe bound on Abs and Sca. By de ini ion, he geome y ac o is always smalle han 1 in absolu e alue. Consequen ly, R∞depends smoo hly on Abs, Sca and Geo. Now, by cons uc ion, R∞ admi s alues in [0,1]only. The colo ma ching unc ions x,y,za e gi en poin wise and can hus be in e pola ed wi h Lipschi z con inuous de i a i e. As a esul , X, Y, Za eL-smoo h unc ion w. . . all a gumen s. Finally, he unc ion , appea ing in he de ini ion o he colo ans o ma ion mapping , is cons uc ed in an L-smoo h ash- ion as well, showing ha [2, Assump ion 2.3] is sa is ied o ou se ing. By choosing in eg a ion weigh s p esen ed in [2, Sec ion 3], we can also sa is y [2, Assump ion 2.4]. 2.5 Challenges The highly condensed ashion, in which (1) is o mula ed, may obscu e a lo o he di icul ies ha a ise when ying o sol e i . To ge a be e unde s anding o he p oblem, le us i s analyze he abs ac s uc u e o he objec i e unc ion J(u)= 1 20 L(u)+19 20 a(u): ⎛ ⎝ Abs Sca Geo⎞ ⎠ in eg a e R×D −−−−→K SKubelka- Munk −−−−−→R∞ in eg a e  −−−−→⎛ ⎝ X Y Z⎞ ⎠ colo ans . −−−−→⎛ ⎝ L a b⎞ ⎠−→ J(u). Since calcula ing J(u)and ∇J(u) equi es in eg a ing he op ical p ope ies in mul i- ple dimensions and since e alua ing said p ope ies o any combina ion o ˜ R,˜ dand λ equi es sol ing he ime-ha monic Maxwell’s equa ions, s anda d de e minis ic app oaches, e.g., ull g adien me hods, un in o a p edisc e iza ion p oblem. On he one hand, he numbe o in eg a ion poin s needs o be su icien ly la ge o ou se ing. In Fig.3, a slice h ough he objec i e unc ion o a ixed alue o R and se e al di e en amoun s o in eg a ion poin s is shown. While we ac ually do no ca e oo much abou he app oxima ion e o esul ing om a small numbe o in eg a ion poin s, he a i icial local maxima in oduced in o he objec i e unc ion by he disc e iza ion se e ely impac he quali y o he op imiza ion. In o he wo ds, many solu ions o he disc e ized p oblem a e comple ely un ela ed o solu ions o (1). We wan o no e ha , e en hough no all o he s a iona y poin s in Fig.3co espond o s a iona y poin s o (1), he p edisc e iza ion s ill leads o e y la egions in he 123 984 M. G ieshamme e al. Fig. 3 Objec i e unc ion alues o ixed co e adius o 3 nm. Di e en g aphs co espond o di e en disc e iza ions. The label o a cu e shows in o how many poin s he in eg als o e ,Rand Dha e been spli , espec i ely. Each o he disc e iza ions in oduces a i icial s a iona y poin s in o he objec i e unc ion objec i e unc ions, which hinde he pe o mance o many sol e s. In Fig.4, his e ec is displayed. On he o he hand, he numbe o in eg a ion poin s is hea ily es ic ed by he compu a ional cos associa ed o he e alua ion o Abs, Sca and Geo. While medium esolu ions (253∼15000 poin s in o al) a e s ill nume ically ac able o simple Mie pa icles, hey a e ou igh impossible o achie e in he mo e gene al DDA se ing, which we wan o conside la e . Fo compa ison: The op imiza ion in [19] was ca ied ou using a disc e iza ion consis ing o 20 poin s in o al. We wan o emphasize ha s anda d SG- ype schemes, o e en he S ochas ic Com- posi ion G adien Descen (SCGD) me hod [26], which was used o he compa ison o composi e objec i e unc ions in [2, Sec ion 7.2], a e no capable o sol ing (1). The eason o his lies in he special s uc u e o J, which consis s o se e al in eg als nes ed in nonlinea unc ions. 2.6 Disc e iza ion Fo he easons men ioned abo e, we will only compa e he esul s ob ained by CSG o gene icde e minis ic op imiza ion schemes o a iouschoiceso disc e iza ion. Since he in eg a ion o e admi s no special s uc u e, we always choose an equidis an pa i ion o his dimension o in eg a ion. Howe e , o he in eg a ion o e R×D, we can use ou knowledge o μu o achie e a be e app oxima ion o he ue in eg al. Ins ead o di iding R×Din o an equidis an g id, we u ilize he ac ha ˜ Rand ˜ d ollow unca ed one-dimensional no mal dis ibu ions wi h pa ame e s independen om each o he . Since, o a no mal dis ibu ion, 99.7% o all weigh is concen a ed in he 3σ-in e al a ound he mean alue, we may only disc e ize his po ion o he ull domain in each s ep. Mo eo e , we know he p ecise densi y unc ion o bo h ˜ Rand ˜ d. Thus, gi en a design un=(Rn,dn), we will pa i ion Rn−3 10 Rn,Rn+3 10 Rnand 123 The con inuous s ochas ic g adien me hod 991 and S(u,λ)=1 S2S2Sca(u,λ,ν) 1−Geo(u,λ,ν) dν. He e, S2deno es he uni sphe e and he pa icle o ien a ion νisassumed obedis- ibu ed uni o mly andom o e all possible di ec ions. The design domain is a ball o 300 nm diame e , disc e ized in o n0=65752 dipole cells. The design u∈[ε, 1]n0=: Ugi es he ela i e amoun o hema i e o wa e in each cell, wi h ε=10−4. The op ical p ope ies o in e media e (g ey) ma e ial u(i)∈(0,1)a e gene a ed by linea in e pola ion be ween he espec i e p ope ies o wa e and hema i e. Consequen ly, each admissible design con ains a posi i e amoun o hema i e, esul ing in lowe bounds o Abs and Sca. As s a ed in Sec .2.4,[2, Assump ions 2.2–2.4] a e sa is ied, since changing om Mie heo y o he DDA model does no in e e e wi h he smoo hness o Abs, Sca and Geo w. . . (u,λ,ν),see[19, 28]. Gene ally, one would combine il e ing echniques and g eyness penaliza ion o ob ain a smoo h inal design wi hou in e media e ma e ial (see, e.g., [29]). Howe e , we explici ly e ain om doing so o p esen a clea analysis o he CSG pe o mance, wi hou in e e ence om seconda y laye s o smoo hing echniques. As men ioned abo e, he change o he DDA model signi ican ly inc eases he com- pu a ional cos o e alua ing Sca, Abs and Geo o a gi en (u,λ,ν)∈U××S2. Thus, he de e minis ic app oaches used in he p e ious se ing a e no longe compu- a ionally easible. Fig. 11 Rep esen a ion o he ini ial designs ( op ow). Red boxes co espond o cells consis ing pu ely o hema i e, while g ey boxes indica e an a i icial in e media e ma e ial, consis ing o 50% hema i e and 50% wa e . Fo la e e e ences, we deno e he ini ial designs by pla e (100%),pla e (50%) and sc ewd i e (50%), espec i ely. The di e en inal designs, ob ained by 5.000 i e a ions o SCIBL-CSG wi h ou e no m (a) a e shown in he bo om ow. Fo be e isibili y, cells wi h less han 50% hema i e a e conside ed as pu e wa e and le ou o he isualiza ion. Fo each inal design, he amoun o cells disca ded in his ashion is less han 100 (less han 0.15% o all cells) 123 992 M. G ieshamme e al. Fu he mo e, we wan o use his example o analyze he impac o he chosen no m on U××S2, appea ing in he nea es neighbo calcula ion, which was al eady men ioned in [2, Sec ion 3.5]. To be p ecise, calcula ing he CSG in eg a ion weigh s equi es he de ini ion o an ou e no m  (u∗,λ ∗,ν∗) Ou =cuu∗U+cλλ∗+cνν∗S2, Fig. 12 Objec i e unc ion app oxima ion o he sc ewd i e (50%) design. The blue and o ange cu e show he esul s o CSG wi h ixed s ep size τ=0 and di e en coe icien s o he ou e no m ·Ou . Fo Mon e Ca lo, each inne in eg al o e S2was app oxima ed using 40 andom di ec ions. The ue objec i e unc ion alue J∗≈37.84 is indica ed by he dashed line. The Mon e Ca lo esul s a e unca ed o he sake o eadabili y, as i equi es o e 8.000 e alua ions o each a good app oxima ion o J∗ Fig. 13 CSG objec i e unc ion app oxima ions du ing he op imiza ion p ocess o all ini ial designs and choice (a) o ·Ou , i.e., cu=1, cλ=100 and cν=100. The dashed lines indica e he objec i e unc ion alues o each ini ial design, espec i ely 123 The con inuous s ochas ic g adien me hod 993 whe e ·U,·and ·S2deno e no ms on he co esponding inne spaces and cu,cλ,cν>0. In his applica ion, we choose he Euclidean no m ·2 o each inne space. Addi ionally, we ix cu=1, bu conside di e en coe icien s cλand cν. Fo he op imiza ion, we conside h ee di e en ini ial designs, which a e shown in Fig.11, op ow. The objec i e unc ion alue as well as he alues o L,aand b o hese designs we e compu ed using he CSG me hod wi h ixed design, i.e., wi h cons an s ep size τ=0, and e i ied by Mon e Ca lo (see, e.g., [30]) in eg a ion. Fo one o he ini ial designs, he objec i e unc ion alue app oxima ion o CSG and Mon e Ca lo in eg a ion wi h espec o he numbe o e alua ions and di e en choices o ·Ou is shown in Fig.12. Fig. 14 Top le o bo om igh : Design e olu ion du ing he op imiza ion p ocess o he sc ewd i e (50%) ini ial design and ou e no m (a). The design snapsho s we e aken e e y 200 i e a ions. Red boxes ep esen design cells consis ing o pu e hema i e. In e media e ma e ial is indica ed ia a colo g adien , whe e a cell illed wi h 50% wa e and 50% hema i e is colo ed g ey. Based on his g adien , depending on he a io o hema i e and wa e in a cell, he cell colo is shi ed o ed (mo e hema i e) o blue (mo e wa e ) 123 994 M. G ieshamme e al. Fig. 15 Euclidean dis ance (a e di iding by √dim(U) o scaling) be ween in e media e designs and he espec i e inal design du ing he SCIBL-CSG op imiza ion p ocess, ca ied ou wi h ou e no m (a) Each design was op imized wi h SCIBL-CSG, using inexac hyb id weigh s o he in eg a ion o e S2and exac hyb id weigh s o he in eg a ion o e .Fo ·Ou , we conside ed ou di e en choices o he pa ame e s: (a) cu=1, cλ=100 and cν=100 (b) cu=1, cλ=1 and cν=1 (c) cu=1, cλ=1 100 and cν=1 (d) cu=1, cλ=1 100 and cν=1 100 The esul s in case (a) o all h ee ini ial designs a e p esen ed in Fig.13 and he espec i e design e olu ion o he ini ial design sc ewd i e (50%), shown in Fig.11 op ow, is depic ed in Fig.14. The co esponding inal designs, ob ained a e 5.000 SCIBL-CSG i e a ions, a e p esen ed in Fig.11, bo om ow. As a second measu e o con e gence in he design space, he e olu ion o he no m dis ance o he espec i e inal designs a e shown in Fig.15 o all h ee ini ial designs. Compa ing Figs.12 and 13, we no ice ha CSG, using an app op ia e ou e no m, inds an op imized design almos as as as i compu es he objec i e unc ion alue o a gi en design. In o he wo ds: The ull op imiza ion p ocess is only sligh ly mo e expensi e ha he simple e alua ion o a single design. Mo eo e , CSG inds an op imal solu ion o (2) long be o e he Mon e Ca lo app oxima ion o he ini ial objec i e unc ion alue is con e ged. I should, o cou se, also be no ed, ha choosing ·Ou should be done wi h cau ion, as Fig.16 shows. While case (a) is, o he bes o ou knowledge, no op imal by any means, cases (b) and (c) clea ly show wo se esul s. Choosing ·Ou ex emely poo ly, i.e., case (d), can e en ha e de as a ing e ec s on he pe o mance, see Fig.17. This, howe e , could also imply ha he pe o mance migh be signi ican ly imp o ed, i p oblem speci ic inne and ou e no ms would be chosen. Especially 123 The con inuous s ochas ic g adien me hod 995 Fig. 16 CSG objec i e unc ion alue app oxima ion du ing he op imiza ion p ocess o he pla e (100%) ini ial design. The dashed line shows he ini al objec i e unc ion alue, whe eas he di e en g aphs co espond o he choices (a), (b) and (c) o ·Ou Fig. 17 Resul s o he pla e (100%) ini ial design p esen ed in Fig.16, augmen ed by he CSG objec i e unc ion alue app oxima ion in he case ha ·Ou was chosen acco ding o (d) in e en mo e complex se ings, echniques o ob ain such no ms a p io i, o e en du ing he op imiza ion p ocess i sel , ep esen one o he mos impo an poin s o u he esea ch. 123 996 M. G ieshamme e al. 3 Online e o es ima ion Be o e we go in o heo e ical de ails, we i s collec a ew key p ope ies and esul s conce ning CSG, which we e shown in [2]. In a i s simple se ing, we conside op imiza ion p oblems o he o m min J(u) s. . u∈U⊂Rdo o some do∈N.(3) Addi ionally, we assume ha Uis compac , and o some d ∈N, he e exis s an open an bounded se X⊂Rd and a measu e μwi h supp(μ) ⊂X, such ha Jcan be w i en as J(u)=Xj(u,x)μ(dx). The de ailed se o assump ions is gi en in [2, Sec ion 2]. Fo now, i is only impo an ha ∇1j:U×X→Rdois bounded and Lipschi z con inuous, i.e., he e exis C,Lj>0 wi h ∇1j(u,x)≤C, ∇1j(u1,x1)−∇1j(u2,x2)≤Lju1−u2U+x1−x2X o all (u,x), (u1,x1), (u2,x2)∈U×X. Due o he ini e dimension o all appea ing spaces, we can choose a bi a y no ms on U,Xand Rdo, and simply deno e hem by · U,· Xand ·, espec i ely, unless speci ic choices a e made in nume ical expe imen s. Du ing he op imiza ion p ocess, CSG compu es design dependen in eg a ion weigh s αkk=1...,n(c . [2, Sec ion 3]) o build an app oxima ion ˆ Gn o he ue objec i e unc ion g adien , based on he a ailable samples om p e ious i e a ions ∇1j(uk,xk)k=1,...,n.Tobep ecise,weha e ∇J(u)=X∇1j(u,x)μ(dx)≈ n  k=1 αk∇1j(uk,xk)=: ˆ Gn. I was shown in [2, Lemma 4.6], ha ∇J(un)−ˆ Gn→0 o n→∞almos su ely. Ca e ully in es iga ing he me hods o ob ain he in eg a ion weigh s, we obse e ha   ∇J(un)−ˆ Gn  =   X∇1j(un,x)μ(dx)−ˆ Gn    =     n  i=1Mi∇1j(un,x)μ(dx)− n  i=1∇1j(ui,xi)νn(Mi)     , 123 The con inuous s ochas ic g adien me hod 997 whe e νndeno es he measu e associa ed o one o he measu es lis ed in [2, Sec ion 3.6], depending on he choice o in eg a ion weigh s, and Mk:= x∈X:un−ukU+x−xkX <un−ujU+x−xjX o all j∈{1,...,n} {k}. By cons uc ion, Mkcon ains all poin s x∈X, such ha (un,x)is close o (uk,xk) han o any o he p e ious poin we e alua ed ∇1ja . Fo exac in eg a ion weigh s, we ha e νn=μand hus   ∇J(un)−ˆ Gn  =     n  i=1Mi∇1j(un,x)μ(dx)− n  i=1Mi∇1j(ui,xi)μ(dx)     ≤ n  i=1Mi∇1j(un,x)−∇1j(ui,xi)μ(dx) ≤ n  i=1Mi Lj·sup x∈Mi Zn(x)μ(dx) =Lj n  i=1 μ(Mi)sup x∈Mi Zn(x) ≤Ljsup x∈X Zn(x). He e, Znis gi en by Zn(x):= min k∈{1,...,n}un−ukU+x−xkX. In o he wo ds, he app oxima ion e o can be bounded in e ms o he Lipschi z cons an o ∇1jand he quan i y Zn, which ela es o he size o Vo onoi cells [31] wi h posi i e in eg a ion weigh s. Bo h Ljand supx∈XZn(x)can be e icien ly app oxima ed du ing he op imiza ion p ocess, e.g. by ini e di e ences o he samples ∇1j(ui,xi)i=1,...,nand by sup x∈X Zn(x)≈max k=1,...,nZn(xk), yielding an online e o es ima ion. Such an app oxima ion may, o example, be used in s opping c i e ia. 4 Con e gence a es Th oughou his sec ion, we assume [2, Assump ions 2.1–2.4] o be sa is ied. Mo e- o e , o he en i e sec ion, le (un)n∈Nco espond o he CSG i e a es p oduced o a 123 998 M. G ieshamme e al. ixed andom sequence (xn)n∈N. Then, wi h p obabili y 1, we ha e  ˆ Gn−∇J(un) →0, see [2, Lemma 4.6] 4.1 Theo e ical backg ound In he con e gence analysis p esen ed in [2], we ha e al eady seen ha he ashion in which he g adien app oxima ion ˆ Gnis calcula ed in CSG is c ucial o ˆ Gn− ∇J(un)→0 and ha his p ope y o CSG in u n is he key o all ad an ages CSG o e s in compa ison o classic s ochas ic op imiza ion me hods, like con e gence o cons an s eps, back acking, mo e in ol ed op imiza ion p oblems, e c. The p ice we pay o his ea u e lies wi hin he dependency o ˆ Gnon he pas i e a es. Fo compa ison, he sea ch di ec ion ˆ GSG nin a s ochas ic g adien descen me hod is gi en by ˆ GSG n=∇ 1j(un,xn). Thus, i is independen o all p e ious s eps and ul ills EXˆ GSG n=EX∇1j(un,·)=∇J(un), i.e., i is an unbiased sample o he ull g adien . The combina ion o hese p ope ies allows o a s aigh o wa d con e gence a e analysis, see, e.g., [32]. Incon as , ˆ Gnisingene alno anunbiasedapp oxima ion o∇J(un)andmo eo e no independen o ui,xi)i=1,...,n−1. The main p oblem in inding he con e gence a e o un+1−unU→0 is, ha his quan i y depends on he app oxima ion e o ˆ Gn−∇J(un), which, as we ha e seen in Sec .3, depends on Zn. Since Zni sel is deeply connec ed o minkun−ukU, we un in o a ci cula a gumen . The e o e,up onow,wea eno able op o econ e gence a es o heCSGi e a es. We can, howe e , s a e a p edic ion o his a e and p o ide nume ical e idence. Conjec u e 4.1 We conjec u e ha he CSG me hod, applied o p oblem (3),usinga cons an s ep size τ<2 Land empi ical in eg a ion weigh s, ul ills un+1−unU=Oln(n)·n−1 max{2,d } wi h p obabili y 1. To mo i a e his claim, no e ha , in he p oo o [2, Lemma 4.6], i was shown ha he e exis s C>0 such ha   ˆ Gn−∇J(un)  ≤CX Zn(x)μ(dx)+dW(μn,μ) , 123 The con inuous s ochas ic g adien me hod 999 whe e dWdeno es he Wasse s ein dis ance o he wo measu es μnand μ.By[33, Theo em 1], he empi ical measu e μnsa is ies EdW(μn,μ) ≤C(d )·Xx3 Xμ(dx)1 3·⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 1 √ni d =1, ln(1+n) √ni d =2, n−1 d i d ≥3. This esul is he main mo i a ion o Conjec u e 4.1. I can be shown ha he a e n−1/d o d ≥3issha pi μco esponds o a uni o m dis ibu ion on X. Thus, in his case, i is easonable o assume a uni o m dis ibu ion also co esponds o he wo s - case a e o XZn(s)μ(dx)→0. Assuming ha he di e ence in designs appea ing in Znis negligible due o he o e all con e gence o CSG, we ob ain he a e sup x∈X Zn(x)=Oln(n)·n−1 max{2,d }. To see his, we ill X⊂Rd wi h balls (w. . . he no m · X) o adius ε>0 and deno e by N(ε) ∈N he numbe o cells. Due o he dimension o X,weha e ON(ε)=ε−d . Now, o achie e supx∈XZn(x)<ε, we need each o hese cells o con ain a leas one o he sample poin s (xi)i=1,...,n. I is well-known ha he expec ed numbe o samples we need o d aw o his o happen is gi en by N(ε) N(ε)  k=1 1 k=O−ε−d ln(ε), whe e we used n  k=1 1 k=Oln(n) o n→∞. In o he wo ds, he con e gence a es o XZn(x)μ(dx)→0 and dW(μn,μ)→0 a e compa able. Now ha we mo i a ed he a es claimed in Conjec u e 4.1 o he app oxima ion e o ˆ Gn−∇J(un), we use he ollowing p oposi ion o show ha he a es o un+1−unU→0 can no be wo se. P oposi ion 4.2 Assume ha he app oxima ion e o ˆ Gn−∇J(un)sa is ies ˆ Gn−∇J(un)=Oln(n)·n−1 max{2,d }. Then, unde he assump ions o Conjec u e 4.1, i holds un+1−unU=Oln(n)·n−1 max{2,d }. 123 1000 M. G ieshamme e al. P oo Assume o con adic ion ha his is no he case. Thus, he e exis s N∈N such ha   ∇J(un)−ˆ Gn  ≤1 21 τ−L 2un+1−unU o all n≥N.(4) By he descen lemma [34, Lemma 5.7], he cha ac e is ic p ope y o he p ojec ion ope a o [34, Theo em 6.41] and he Cauchy-Schwa z inequali y, we ob ain J(un+1)−J(un) ≤∇J(un)(un+1−un)+L 2un+1−un2 U =ˆ G n(un+1−un)+L 2un+1−un2 U+∇J(un)−ˆ Gn(un+1−un) ≤L 2−1 τun+1−un2 U+  ∇J(un)−ˆ Gn  ·un+1−unU =L 2−1 τun+1−unU+  ∇J(un)−ˆ Gn  un+1−unU. Combining his wi h (4)gi esJ(un+1)≤J(un) o all n≥N, since L 2<1 τ. Thus, he sequence o objec i e unc ion alues J(un)n∈Nis mono onically dec easing o all n≥N. By con inui y o Jand compac ness o U,Jis bounded and J(un)→¯ J o some ¯ J∈R. The e o e, −∞ <¯ J−J(uN)=∞  n=NJ(un+1−J(un)≤1 2L 2−1 τ∞  n=Nun+1−un2 U. Hence, he se ies ∞  n=Nun+1−un2 U con e ges, con adic ing un+1−unU= Oln(n)·n−1 max{2,d }. 4.2 Nume ical e i ica ion We wan o e i y he p oclaimed a es nume ically. Fo his pu pose, we conside wo op imiza ion p oblems ha can easily be scaled o high dimensions. The i s p oblem is gi en by min u∈U 1 2X u−x  2 2dx,(5) 123 The con inuous s ochas ic g adien me hod 1007 Acknowledgemen s The esea ch was unded by he Deu sche Fo schungsgemeinscha (DFG, Ge man Resea ch Founda ion)—P ojec -ID 416229255—CRC 1411). Funding Open Access unding enabled and o ganized by P ojek DEAL. Da a a ailabili y s a emen In ou nume ical expe imen s ela ed o con e gence a es, only simple aca- demic examples we e used o isualize he heo e ical esul s. These can be ep oduced based on he gi en algo i hms. Fo he nanopa icle design op imiza ion, he co esponding da a is a ailable a h ps://doi.o g/ 10.5281/zenodo.10032613. Decla a ion Con lic o in e es The au ho s ha e no ele an inancial o non- inancial in e es s o disclose. Open Access Thisa icleislicensedunde aC ea i eCommonsA ibu ion4.0In e na ionalLicense,which pe mi s use, sha ing, adap a ion, dis ibu ion and ep oduc ion in any medium o o ma , as long as you gi e app op ia e c edi o he o iginal au ho (s) and he sou ce, p o ide a link o he C ea i e Commons licence, and indica e i changes we e made. The images o o he hi d pa y ma e ial in his a icle a e included in he a icle’s C ea i e Commons licence, unless indica ed o he wise in a c edi line o he ma e ial. I ma e ial is no included in he a icle’s C ea i e Commons licence and you in ended use is no pe mi ed by s a u o y egula ion o exceeds he pe mi ed use, you will need o ob ain pe mission di ec ly om he copy igh holde . To iew a copy o his licence, isi h p://c ea i ecommons.o g/licenses/by/4.0/. Re e ences 1. P lug, L., Be nha d , N., G ieshamme , M., S ingl, M.: CSG: a new s ochas ic g adien me hod o he e icien solu ion o s uc u al op imiza ion p oblems wi h in ini ely many s a es. S uc . Mul idiscip. Op im. 61(6), 2595–2611 (2020). h ps://doi.o g/10.1007/s00158-020-02571-x 2. G ieshamme , M., P lug, L., S ingl, M., Uihlein, A.: The con inuous s ochas ic g adien me hod: pa I–con e gence heo y. Compu . Op im. Appl. (2023). h ps://doi.o g/10.1007/s10589-023-00542-8 3. Robbins, H., Mon o, S.: A s ochas ic app oxima ion me hod. Ann. Ma h. S a . 22, 400–407 (1951). h ps://doi.o g/10.1214/aoms/1177729586 4. Schmid , M., Le Roux, N., Bach, F.: Minimizing ini e sums wi h he s ochas ic a e age g adien . Ma h. P og am. 162(1-2, Se . A), 83–112 (2017). h ps://doi.o g/10.1007/s10107-016-1030-6 5. Zhao, Y., Xie, Z., Gu, H., Zhu, C., Gu, Z.: Bio-inspi ed a iable s uc u al colo ma e ials. Chem. Soc. Re . 41, 3297–3317 (2012). h ps://doi.o g/10.1039/C2CS15267C 6. Wang, J., Sul an, U., Goe li ze ,E.S.A., Mbah, C.F., Engel, M.S.,Vogel,N.: S uc u al colo o colloidal clus e s as a ool o in es iga e s uc u e and dynamics. Ad . Func . Ma e . 30 (2019) 7. England, G.T., Russell, C., Shi man, E., Kay, T., Vogel, N., Aizenbe g, J.: The op ical Janus e ec : asymme ic s uc u al colo e lec ion ma e ials. Ad . Ma e . 29 (2017). h ps://doi.o g/10.1002/adma. 201606876 8. Xiao, M., Hu, Z., Wang, Z., Li, Y., To mo, A.D., Thomas, N.L., Wang, B., Gianneschi, N.C., Shawkey, M.D., Dhinojwala, A.: Bioinspi ed b igh noni idescen pho onic melanin sup aballs. Sci. Ad . 3(9), 1701151 (2017). h ps://doi.o g/10.1126/sciad .1701151 9. Goe li ze , E.S.A., Klupp-Taylo , R.N., Vogel, N.: Bioinspi ed pho onic pigmen s om colloidal sel - assembly. Ad . Ma e . 30(28), 1706654 (2018). h ps://doi.o g/10.1002/adma.201706654 10. Uihlein, A., P lug, L., S ingl, M.: Op imizing colo o pa icula e p oduc s. PAMM 22(1), 202200047 (2023). h ps://doi.o g/10.1002/pamm.202200047 11. Taylo , R.K., Sei , F., Zhu omskyy, O., Peschel, U., Leuge ing, G., Peuke , W.: Pain ing by numbe s: Nanopa icle-based colo an s in he pos -empi ical age. Ad . Ma e . 23(22–23), 2554–2570 (2011). h ps://doi.o g/10.1002/adma.201100541 12. Buxbaum, G.: Indus ial ino ganic pigmen s. Wiley, New Je sey (2008) h ps://doi.o g/10.1002/ 3527603735 13. Colo ime y, C.: Repo no: Cie pub no 15. CIE Cen al Bu eau, Vienna (2004) 14. CIE Commission In e na ionale de l’Éclai age P oceedings (1931) 123 1008 M. G ieshamme e al. 15. Mishchenko, M.I., T a is, L.D., Lacis, A.A.: Sca e ing, Abso p ion, and Emission o Ligh by Small Pa icles. Camb idge Uni e si y P ess, Camb idge (2002) 16. DeVo e, J.R.: Re ac i e indices o u ile and sphale i e. J. Op . Soc. Am. 41(6), 416–419 (1951). h ps://doi.o g/10.1364/JOSA.41.000416 17. Pu cell, E.M., Pennypacke , C.R.: Sca e ing and abso p ion o ligh by nonsphe ical dielec ic g ains. As ophys. J. 186, 705–714 (1973). h ps://doi.o g/10.1086/152538 18. Yu kin, M.A., Hoeks a, A.G.:Thedisc e e-dipole-app oxima ioncodeADDA: capabili ies andknown limi a ions. J. Quan . Spec osc. Radia . T ans e 112(13), 2234–2247 (2011). h ps://doi.o g/10.1016/ j.jqs .2011.01.031 19. Nees, N., P lug, L., Mann, B., S ingl, M.: Mul i-ma e ial design op imiza ion o op ical p ope ies o pa icula e p oduc s by disc e e dipole app oxima ion and sequen ial global p og amming. S uc . Mul idiscip. Op im. (2022). h ps://doi.o g/10.1007/s00158-022-03376-w 20. Mie,G.: Bei äge zu op ik übe medien,speziell kolloidale me allösungen. Ann. Phys.330,377–445 (1908). h ps://doi.o g/10.1002/andp.19083300302 21. He ge , W., W ied , T.: The Mie Theo y: Basics and Applica ions. Sp inge Se ies in Op ical Science. Sp inge , Be lin (2012). h ps://doi.o g/10.1007/978-3-642-28738-1 22. Kubelka, P., Munk, F.: An a icle on op ics o pain laye s. Z. Tech. Phys. 12(593–601), 259–274 (1931) 23. Ga cía-Valenzuela, A., Cuppo, F., Oli a es, J.: An assessmen o saunde son co ec ions o he di use e lec ance o pain ilms. In: Jou nal o Physics: Con e ence Se ies, ol. 274, p. 012125 (2011). h ps:// doi.o g/10.1088/1742-6596/274/1/012125. IOP Publishing 24. on Illumina ion (CIE), I.C.: CIE 1964 colou -ma ching unc ions , 10 deg ee obse e . In e na ional Commission on Illumina ion (CIE). h ps://doi.o g/10.25039/cie.ds.sqksu2n5 25. Wiscombe, W.J.: Imp o ed mie sca e ing algo i hms. Appl. Op . 19(9), 1505–1509 (1980) 26. Wang, M., Fang, E.X., Liu, H.: S ochas ic composi ional g adien descen : algo i hms o minimizing composi ions o expec ed- alue unc ions. Ma h. P og am. 161(1-2, Se . A), 419–449 (2017). h ps:// doi.o g/10.1007/s10107-016-1017-3 27. Schä e , J., Lee, S.-C., Kienle, A.: Calcula ion o he nea ields o he sca e ing o elec omagne ic wa es by mul iple in ini e cylinde s a pe pendicula incidence. J. Quan . Spec osc. Radia . T ans e 113(16), 2113–2123 (2012). h ps://doi.o g/10.1016/j.jqs .2012.05.019 28. D aine, B.T., Fla au, P.J.: Disc e e-dipole app oxima ion o sca e ing calcula ions. JOSA A 11(4), 1491–1499 (1994) 29. Sigmund, O.: Mo phology-based black and whi e il e s o opology op imiza ion. S uc . Mul idiscip. Op im. 33(4), 401–424 (2007). h ps://doi.o g/10.1007/s00158-006-0087-x 30. Ca lisch, R.E.: Mon e ca lo and quasi-mon e ca lo me hods. Ac a Nume . 7, 1–49 (1998). h ps://doi. o g/10.1017/S0962492900002804 31. Bu ough, P., McDonnell, R., Lloyd, C.: 8.11 nea es neighbou s: Thiessen (di ichle / o oni) polygons. P inc. Geog aph. In . Sys . (2015) 32. Bo ou, L., Cu is, F.E., Nocedal, J.: Op imiza ion me hods o la ge-scale machine lea ning. SIAM Re . 60(2), 223–311 (2018). h ps://doi.o g/10.1137/16M1080173 33. Fou nie , N., Guillin, A.: On he a e o con e gence in wasse s ein dis ance o he empi ical measu e. P obab. Theo y Rela . Fields 162(3), 707–738 (2015). h ps://doi.o g/10.1007/s00440-014-0583-7 34. Beck, A.: Fi s -o de Me hods in Op imiza ion. MOS-SIAM Se ies on Op imiza ion, ol. 25, p. 475. Socie y o Indus ialandAppliedMa hema ics(SIAM):Ma hema icalOp imiza ionSocie y,Philadel- phia (2017). h ps://doi.o g/10.1137/1.9781611974997.ch1 Publishe ’s No e Sp inge Na u e emains neu al wi h ega d o ju isdic ional claims in published maps and ins i u ional a ilia ions. 123