Leveraging interpretable machine learning in intensive care

Bohlen, Lasse; Rosenbe ge , Julian; Zschech, Pa ick; K aus, Ma hias
A icle — Published Ve sion
Le e aging in e p e able machine lea ning in in ensi e
ca e
Annals o Ope a ions Resea ch
P o ided in Coope a ion wi h:
Sp inge Na u e
Sugges ed Ci a ion: Bohlen, Lasse; Rosenbe ge , Julian; Zschech, Pa ick; K aus, Ma hias (2024) :
Le e aging in e p e able machine lea ning in in ensi e ca e, Annals o Ope a ions Resea ch, ISSN
1572-9338, Sp inge US, New Yo k, NY, Vol. 347, Iss. 2, pp. 1093-1132,
h ps://doi.o g/10.1007/s10479-024-06226-8
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/323300
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h p://c ea i ecommons.o g/licenses/by/4.0/
Annals o Ope a ions Resea ch (2025) 347:1093–1132
h ps://doi.o g/10.1007/s10479-024-06226-8
ORIGINAL RESEARCH
Le e aging in e p e able machine lea ning in in ensi e ca e
Lasse Bohlen1·Julian Rosenbe ge 2·Pa ick Zschech3·Ma hias K aus2
Recei ed: 31 Ma ch 2023 / Accep ed: 15 Augus 2024 / Published online: 19 Sep embe 2024
© The Au ho (s) 2024
Abs ac
In heal hca e, especially wi hin in ensi e ca e uni s (ICU), in o med decision-making by
medical p o essionals is c ucial due o he complexi y o medical da a. Heal hca e analy -
ics seeks o suppo hese decisions by gene a ing accu a e p edic ions h ough ad anced
machine lea ning (ML) models, such as boos ed decision ees and andom o es s. While
hese models equen ly exhibi accu a e p edic ions ac oss a ious medical asks, hey o en
lack in e p e abili y. To add ess his challenge, esea che s ha e de eloped in e p e able ML
models ha balance accu acy and in e p e abili y. In his s udy, we e alua e he pe o mance
gap be ween in e p e able and black-box models in wo heal hca e p edic ion asks, mo ali y
and leng h-o -s ay p edic ion in ICU se ings. We ocus speci ically on he amily o gene -
alized addi i e models (GAMs) as powe ul in e p e able ML models. Ou assessmen uses
he publicly a ailable Medical In o ma ion Ma o In ensi e Ca e da ase , and we analyze
he models based on (i) p edic i e pe o mance, (ii) he in luence o compac ea u e se s
(i.e., only ew ea u es) on p edic i e pe o mance, and (iii) in e p e abili y and consis ency
wi h medical knowledge. Ou esul s show ha in e p e able models achie e compe i i e
pe o mance, wi h a mino dec ease o 0.2–0.9 pe cen age poin s in a ea unde he ecei e
ope a ing cha ac e is ic ela i e o s a e-o - he-a black-box models, while p ese ing com-
ple e in e p e abili y. This emains ue e en o pa simonious models ha use only 2.2%
o pa ien ea u es. Ou s udy highligh s he po en ial o in e p e able models o imp o e
decision-making in ICUs by p o iding medical p o essionals wi h easily unde s andable and
e i iable p edic ions.
Keywo ds Heal hca e analy ics ·In e p e able machine lea ning ·Gene alized addi i e
models ·Leng h-o -s ay p edic ion ·Mo ali y p edic ion
Lasse Bohlen and Julian Rosenbe ge ha e con ibu ed equally o his wo k.
BLasse Bohlen
[email p o ec ed]
Julian Rosenbe ge
julian. osenbe ge @u .de
Pa ick Zschech
[email p o ec ed]
Ma hias K aus
ma hias.k aus@u .de
1F ied ich-Alexande -Uni e si ä E langen-Nü nbe g, Lange Gasse 20, 90403 Nü nbe g, Ge many
2Uni e si ä Regensbu g, Bajuwa ens aße 4, 93053 Regensbu g, Ge many
3Uni e si ä Leipzig, G immaische S . 12, 04109 Leipzig, Ge many
123
1094 Annals o Ope a ions Resea ch (2025) 347:1093–1132
1 In oduc ion
Heal hca e sys ems ac oss he globe ace a mul i ude o complex challenges, such as ca e
quali y dispa i ies, demog aphic shi s, and adminis a i e obs acles including esou ce sho -
ages, ising cos s, and insu icien in as uc u e (Ronca olo e al., 2017). These issues a e
in ensi ied by inc easing demand o heal hca e se ices, sophis ica ed medical echnology
complica ing physicians’ wo k lows, and heigh ened expec a ions o pa ien -cen e ed ca e.
In he con ex o in ensi e ca e uni s (ICUs), hese challenges a e u he heigh ened due o
pa ien acui y, he need o specialized s a , and p essu e on esou ce alloca ion. ICUs accoun
o app oxima ely 14% o o al hospi al expenses, making hem one o he mos c i ical and
cos ly componen s o heal hca e sys ems (Halpe n & Pas o es, 2010). Consequen ly, e ec-
i e ICU managemen is essen ial o op imizing pa ien ou comes and ensu ing heal hca e
sys ems’ sus ainabili y (Be simas e al., 2021). Howe e , op imizing esou ce u iliza ion
in ICUs emains a daun ing ask due o he u gen and unp edic able na u e o in ensi e
ca e, high cos s associa ed wi h main aining su icien esou ces, and cons an demand o
specialized s a (Bai e al., 2018).
One p omising s a egy o add ess ICU challenges is he implemen a ion o machine
lea ning (ML) models. ML models, wi h hei abili y o make p edic ions mo e quickly
and on a la ge scale han humans, a e inc easingly conside ed indispensable o mode n
heal hca e sys ems (Malik e al., 2018). Ye , ad anced ML models such as boos ed decision
ees and andom o es s a e o en ega ded as he p ima y ML models wi h supe io p edic i e
pe o mance (e.g., Hyland e al., 2020), despi e hei black-box cha ac e is ics, which ende
hei decision logic di icul o humans o unde s and. This pe cep ion has led o a widesp ead
belie ha in e p e abili y mus be comp omised o achie e accu a e p edic ions (Gunning
&Aha,2019), signi ican ly impeding ML adop ion in heal hca e (Kundu, 2021). While
his iewpoin has been challenged by nume ous esea che s (e.g., Ca uana e al., 2015;
Rudin, 2019), he cu en li e a u e lacks an in-dep h analysis examining he pe o mance
gap be ween black-box and in e p e able models.
A undamen al p inciple o achie ing in e p e able ML models is o use he simples
possible model ha adequa ely explains he da a (B un on & Ku z, 2019). This concep is
also known as model pa simony. In gene al, he e a e wo objec i es o ocus on in o de
o ob ain a pa simonious model. Fi s , he ML model i sel should be in e p e able, i.e., o
limi ed complexi y, so ha humans can unde s and he model, and second, he numbe o
ea u es used by he model o compu e an ou pu should be small, i.e., using a compac ea u e
se .
Ou s udy seeks o in es iga e he pe o mance di e ences along hese wo objec i es.
We compa e black-box and in e p e able models, as well as he ole o compac ea u e se s.
By illus a ing ha in e p e able models wi h a minimal se o inpu ea u es can a ain com-
pa able accu acy, we aim o p omo e u he esea ch and de elopmen o in e p e able ML
models o heal hca e applica ions. This wo k con ibu es o a mo e comp ehensi e unde -
s anding o in e p e abili y in ML models and emphasizes he signi icance o hese ac o s
o success ul implemen a ion in heal hca e se ings. Speci ically, his wo k con ibu es o
he exis ing li e a u e in ou ways:
1. We examine and e alua e mul iple in e p e able models om he amily o gene alized
addi i e models (GAMs) agains p e alen black-box models in an ICU se ing. We com-
123
Annals o Ope a ions Resea ch (2025) 347:1093–1132 1095
pa e h ee dis inc p edic ion a ge s: leng h-o -s ay>3 days (LOS3), leng h-o -s ay>7
days (LOS7), and mo ali y.1
2. We pe o m a compa a i e analysis o e alua e he impac o ea u e educ ion me hods
on p edic i e pe o mance, wi h pa icula emphasis on sensi i e ea u es such as gende ,
age, and e hnici y.
3. We showcase he u ili y o in e p e able models by discussing hei plausibili y wi h ou
medical expe s om di e se backg ounds.
4. To make hese models compe i i e, we also p opose mul iple ea u e enginee ing s eps
ha yield a o able esul s in compa ison o p e ious app oaches.2
Ou indings challenge he p e ailing belie ha only black-box models p o ide high p e-
dic i e pe o mance in heal hca e. We demons a e ha in e p e able models can achie e
compe i i e p edic i e pe o mance, wi h a mino dec ease o 0.2–0.9 pe cen age poin s in
a ea unde he ecei e ope a ing cha ac e is ic (AUROC) compa ed o black-box models,
while emaining ull in e p e abili y. This inding holds ue e en o pa simonious models
ha u ilize only 2.2% o he pa ien ea u es a ailable, while exhibi ing a negligible pe -
o mance d op ela i e o black-box models, anging om 0.1 o 1.0 pe cen age poin s and
a e aging a 0.5 pe cen age poin s. By showcasing he compa able accu acy o in e p e able
models e en wi h compac ea u e se s, we aim o inspi e u he esea ch and de elopmen
o in e p e able ML models in heal hca e applica ions.
The emainde o he pape is s uc u ed as ollows: Sec .2del es in o he concep ual
backg ound and p io esea ch on ML in heal hca e, legal and p ac ical equi emen s o ML
models, and in e p e able ML. Sec ion3de ails he da ase , p edic ion asks, used models, and
he expe imen s. In Sec .4, we e alua e and compa e he p oposed models agains black-box
models, and isually inspec so-called shape plo s p oduced by GAMs. Sec ion5discusses
he implica ions o ou wo k o esea ch and p ac ice, along wi h i s limi a ions. Sec ion6
concludes he pape .
2 Resea ch backg ound
2.1 Gene alized addi i e models
This s udy emphasizes he use o GAMs as a pa icula powe ul class o in e p e able models.
GAMs ha e been employed in o he high-s akes domains whe e model in e p e abili y is
essen ial (Chang e al., 2021; Zschech e al., 2022). Fundamen ally, GAMs a e ML models
ha build ela ionships be ween inpu ea u es and he a ge by summing se e al dis inc
uni a ia e non-linea mappings, called shape unc ions. Fo mally, GAMs can be exp essed
as
(x)= 1(x1)+ 2(x2)+···+ m(xm), (1)
whe e each ideno es a shape unc ion mapping he i- h inpu ea u e o he ou pu space.
As such, GAMs p eclude ea u e-in e ac ions, po en ially sac i icing model pe o mance bu
enabling comple e comp ehension o model unc ionali y. Building on his co e concep o
u ilizing nonlinea unc ions o map inpu ea u es o he ou pu space, a ious GAM e sions
1Ou e alua ion pipeline and eplica ion code can be ound he e:
h ps://gi hub.com/HBDynami e/In e p e able_ICU_p edic ions
2Fo ou ea u e enginee ing s eps, see:
h ps://gi hub.com/HB-Dynami e/mimic3-benchma ks_AoOR_da a_expo
123
1096 Annals o Ope a ions Resea ch (2025) 347:1093–1132
Fig. 1 Compa ison o he shape unc ions o a linea model and a GAM
ha e been p oposed (Lou e al., 2012; Aga wal e al., 2021; K aus e al., 2024b). A GAM
can also unc ion as a classi ica ion model by modeling he log odds o he a ge class
p obabili ies. The log odds a e he loga i hm o he odds, which a e he a io o he a ge
class p obabili ies. In a bina y classi ica ion con ex , he GAM can be exp essed as
log P(y=1)
1−P(y=1)= 1(x1)+ 2(x2)+···+ m(xm), (2)
whe e P(y=1)deno es he p obabili y o he a ge class gi en ea u es x1,...,xm,
and i(xi)a e shape unc ions. The logi unc ion maps he p obabili y space [0,1]on o
(−∞,∞), allowing he model o p edic a ge class p obabili ies. To ob ain class p obabil-
i ies, he logis ic unc ion is applied o he model ou pu .
The independence o he inpu ea u es enables a 2-dimensional isualiza ion o shape
unc ions using so-called shape plo s. Figu e1p esen s an example o a shape plo , whe e
he GAM cap u es he nonlinea ela ionship be ween body empe a u e and he p obabili y
(in log odds) o mo ali y ( ep esen ed by he solid cu e). In con as , he linea model
assumes a linea ela ionship ( ep esen ed by he s aigh line). This compa ison highligh s he
lexibili y o GAMs in modeling complex pa e ns, p o iding mo e accu a e and in e p e able
p edic ions han linea al e na i es. Fo physiological signs, o en an op imal alue exis s, wi h
de ia ions in ei he di ec ion inc easing mo ali y. E en ually, human expe s can e alua e
he es ablished ela ionships h ough his isualiza ion (Hegselmann e al., 2020).
Fu he mo e, GAMs ha e he ad an age o being in insically in e p e able (Du e al.,
2019). Tha is, hey p o ide an exac desc ip ion o hei decision logic a he han an app ox-
ima ion. This is in s a k con as o pos -hoc explana ion me hods such as Shapley addi i e
explana ions (SHAP), whe e complex ML models a e simpli ied based on ough app oxima-
ions o gain ce ain insigh s in o he models’ beha io (S iglic e al., 2020; Rudin, 2019).
Such pos -hoc explana ions ine i ably lead o a loss o in o ma ion, which c ea es he isk o
e oneous insigh s and hus a ha m ul basis o medical decision suppo (Babic e al., 2021).
The e o e, in insically in e p e able models such as GAMs o e a mo e eliable choice
because hey p o ide an undis o ed iew o he global model s uc u e, which can be easily
analyzed, adjus ed, and alida ed o sa e y and e icacy.
123

Annals o Ope a ions Resea ch (2025) 347:1093–1132 1097
2.2 Heal hca e decisions and machine lea ning
Ad anced ML models, such as boos ed decision ees and andom o es s, ha e ans o med
heal hca e analy ics by enabling ex ensi e medical da a analysis (Boh & Mema zadeh, 2020;
Saada mand e al., 2022;Hylande al.,2020). Howe e , hei complex s uc u e and black-box
beha io ha e slowed hei adop ion in high-s akes domains (Mille , 2019; Rudin, 2019). The
models’ lack o in e p e abili y makes i di icul o unde s and wha d i es hei p edic ions.
This lack o cla i y os e s skep icism among egula o s and p ac i ione s, hinde ing he
widesp ead use o hese powe ul ML echniques (Aga wal & Das, 2020).
Despi e skep icism, ML models in heal hca e p o ide nume ous bene i s, such as su -
passing human pe o mance in di e se clinical decision suppo a eas (Richens e al., 2020).
Human judgmen is limi ed by pe cep ual biases and cogni i e cons ain s, while machines
can p ocess da a mo e quickly and e icien ly. Some algo i hmic a chi ec u es e en su -
pass doc o s in p edic i e accu acy (Johnson e al., 2022). ML models can also aid essen ial
esou ce alloca ion, such as bed alloca ion o wo k scheduling, by p o iding decision-make s
wi h eal- ime in o ma ion abou he en i e pa ien popula ion (Be simas e al., 2021). This
app oach aligns wi h ope a ions esea ch p inciples, as ML models can be in eg a ed wi h
op imiza ion echniques o imp o e heal hca e decision-making (Bai e al., 2018; Johnson
e al., 2022; K aus e al., 2024a). Howe e , i is c ucial o ecognize he challenges and lim-
i a ions associa ed wi h implemen ing ML models in heal hca e se ings. These encompass
legal, p ac ical, and e hical issues, which we discuss in he ollowing sec ion.
2.3 Legal, p ac ical, and e hical equi emen s in heal hca e
The legal landscape o algo i hmic in e p e abili y is e ol ing, wi h au ho i ies u ilizing
p ima ily ecommenda ions, guidelines, and p elimina y amewo ks. In he Uni ed S a es,
an exemp ion o in e p e able medical so wa e has been in oduced (Ge ke e al., 2020)
and is o e seen by he Food and D ug Adminis a ion (FDA), which is esponsible o egu-
la ing medical de ices. In he Eu opean Union (EU), he ocus is on p omo ing algo i hmic
anspa ency. The EU en o ces a " igh o explana ion" unde he Gene al Da a P o ec ion
Regula ion (GDPR) (Pa liamen and Council o he Eu opean Union, 2016; Goodman &
Flaxman, 2017). This manda e emphasizes he impo ance o in e p e abili y o p o ec ing
sensi i e da a3and ensu ing ai and e hical ea men . Regula o y e o s may become mo e
s ingen , as he EU is de eloping he A i icial In elligence Ac (Pa liamen and Council o
he Eu opean Union, 2021), which aims o egula e ML use in high-s akes decision-making.
This legisla ion could u he emphasize he signi icance o algo i hmic in e p e abili y in
he U.S. and EU, p omo ing in e p e able and accoun able ML sys ems o ensu e e hical and
legal compliance.
F om a p ac ical pe spec i e, heal hca e decision-make s p io i ize pa ien well-being
abo e all else, and ypically do no possess ex ensi e knowledge o ML echniques. The e o e,
ML model in e p e abili y is essen ial o enable ca e gi e s o pe o m in o med decision-
making a he han blindly elying on opaque p edic ions (S iglic e al., 2020). Explana ions
should align wi h use skills and domain knowledge, educing he isk o applica ion e o s
(Coussemen & Benoi , 2021).
3"pe sonal da a e ealing acial o e hnic o igin, poli ical opinions, eligious o philosophical belie s; ade-
union membe ship; gene ic da a, biome ic da a p ocessed solely o iden i y a human being; heal h- ela ed
da a; da a conce ning a pe son’s sex li e o sexual o ien a ion" [A icle 4 (13), (14) and (15) and A icle 9 and
Reci als (51) o (56) o he GDPR].
123
1098 Annals o Ope a ions Resea ch (2025) 347:1093–1132
Rega ding e hical equi emen s, handling sensi i e da a demands special a en ion o p o-
ec indi idual p i acy and p e en disc imina o y p ac ices (Goodman & Flaxman, 2017).
The po en ially li e-al e ing consequences o heal hca e ampli y hese conce ns (Babic e
al., 2021). Inhe en ly in e p e able models, such as GAMs, add ess e hical conce ns ela ed
o sensi i e da a by p o iding insigh s in o he impac o indi idual ea u es on p edic ions
(Chang e al., 2021). This in e p e abili y acili a es he iden i ica ion and mi iga ion o po en-
ial biases, ensu ing ai and accu a e heal hca e decisions.
2.4 Achie ing model in e p e abili y
Model in e p e abili y e e s o a model’s abili y o p esen i s beha io in human-
unde s andable e ms (Doshi-Velez & Kim, 2017). The necessa y le el o in e p e abili y
depends on he use case and domain, as s akeholde s may ha e a ying expec a ions and
equi emen s. Wha is su icien o one use case may no be o ano he (Rudin, 2019).
In heal hca e, isk cha s such as he F amingham Risk Sco e o p edic ing he 10-yea isk
o de eloping ca dio ascula diseases (Wilson e al., 1998), he Simpli ied Acu e Physiology
Sco e (SAPS), and Acu e Physiology and Ch onic Heal h E alua ion (APACHE) sco es o
se e i y-o -illness classi ica ion in ICUs (Mo eno e al., 2005) a e commonly used o assess
pa ien p ognosis and in o m clinical decision-making. These isk cha s ep esen simple,
in e p e able models ha enable clinicians o quickly unde s and pa ien condi ions and make
in o med decisions.
In e p e abili y and ease o unde s anding a e c ucial ac o s when selec ing ML models
in he heal hca e domain. While mo e complex ML models can o en p o ide mo e accu a e
assessmen s o pa ien s’ condi ions, simple models such as decision ees and linea models
ha e been a o ed o hei in e p e abili y. Decision ees p o ide isual ep esen a ions
ha allow physicians o ace decision-making p ocesses (B eiman e al., 1984), making i
easie o unde s and how he model a i ed a i s p edic ions. Simila ly, linea models help
unde s and he con ibu ion o each ea u e o he p edic ions, p o iding insigh s in o he
ac o s ha in luence he model’s ou pu . By main aining in e p e abili y, hese echniques
os e us and adop ion in heal hca e se ings, whe e unde s anding he easoning behind
p edic ions is essen ial o making in o med decisions (Kundu, 2021).
GAMs ha e eme ged as powe ul ools ha s ike a balance be ween accu acy and in e -
p e abili y, making hem well-sui ed o heal hca e applica ions (Zschech e al., 2022). GAMs
combine he simplici y o linea models wi h he lexibili y o nonlinea unc ions, cap u ing
complex ela ionships be ween inpu ea u es and ou comes while p ese ing in e p e abili y
(Chang e al., 2021). The enewed in e es in GAMs has been ueled by in eg a ing ad anced
echniques like boos ing and speci ically designed neu al ne wo ks (Yang e al., 2021;Lou
e al., 2012; Aga wal e al., 2021).
Model pa simony, which e e s o he abili y o a model o desc ibe he da a using he
minimum numbe o e ms o pa ame e s, is closely ela ed o in e p e abili y. Pa simonious
models s ike a balance be ween i ing he da a well and a oiding unnecessa y complexi y
(B un on & Ku z, 2019). Thus, pa simony p omo es simplici y while e aining a high le el o
accu acy. This is achie ed h ough wo objec i es. Fi s , he model a chi ec u e should be kep
as simple as necessa y, which also allows people o unde s and i s unc ionali y. Second, he
model should be based only on a subse o selec ed ea u es, iden i ying he mos in o ma i e
ones and educing he dimensionali y o he inpu space in o de o ob ain a compac model
(James e al., 2013). GAMs in pa icula p omo e pa simony by hei a chi ec u e, as hey
a e ypically simple han mo e complex models such as neu al ne wo ks. By combining
123
Annals o Ope a ions Resea ch (2025) 347:1093–1132 1099
GAMs wi h ea u e selec ion echniques, we can ensu e he de elopmen o pa simonious
models ha a e bo h accu a e and in e p e able, simila o he isk maps commonly used in
heal hca e.
3 Resea ch app oach
This s udy aims o e alua e he pe o mance gap be ween black-box and in e p e able GAMs
in an ICU se ing. We use a well-es ablished in ensi e ca e da abase and ocus on wo com-
mon bina y classi ica ion asks: mo ali y and leng h-o -s ay p edic ion. The me hodology is
de ailed in he subsequen sec ions, co e ing he da ase , p edic ion asks, ea u e ex ac ion,
p ep ocessing s eps, models, and analyses pe o med.
3.1 MIMIC-III clinical da abase
Wi h he inc easing adop ion o heal hca e in o ma ion sys ems, hospi als s a o gene a e and
s o e la ge amoun s o da a. In pa icula , elec onic heal h eco ds ack he pa ien s’ heal h
ajec o ies combining in o ma ion abou demog aphics, physiological signs, and labo a o y
esul s. This da a po en ially allows o de i e in o med p edic ions abou he pa ien s’ heal h
(Koche u o e al., 2019).
In his s udy, we use he Medical In o ma ion Ma o In ensi e Ca e (MIMIC)-III
da abase ( 1.4), one o he mos ex ensi e heal hca e da abase ha is publicly a ailable
(Johnson e al., 2016). MIMIC-III con ains 58,976 anonymized heal h eco ds o 46,520
pa ien s admi ed o he ICU o he Be h Is ael Deaconess Medical Cen e be ween 2001 and
2012.
3.2 Mo ali y and leng h-o -s ay p edic ion
We e alua e ou models on wo common heal hca e p edic ion asks: in-hospi al-mo ali y4
and leng h-o -s ay. Mo ali y and leng h-o -s ay a e he p ima y cos d i e s in ICUs (K ame
e al., 2017) and as Ba es e al. (2014) emphasize, ea ly iden i ica ion o high- isk and high-
cos pa ien s is key o implemen ing s a egies o conse e esou ces in ca e. Consequen ly,
hese high-s akes p edic ion asks a e well sui ed o s udy he pe o mance gap be ween
in e p e able and black-box ML models.
Mo ali y. This bina y classi ica ion ask p edic s pa ien su i al o dea h based on phys-
iological da a collec ed du ing he i s ew hou s a e admission o he ICU (Ha u yunyan
e al., 2019;Wange al.,2020). The goal is o iden i y high- isk pa ien s o imely p o ision
o in ensi ied medical supe ision, ca e, and esou ce alloca ion.
Leng h-o -s ay. This ask es ima es which pa ien s will ha e longe ICU s ays and equi e
addi ional esou ces. E en ough leng h-o -s ay es ima es can signi ican ly imp o e ICU
scheduling, as long- e m pa ien s accoun o a disp opo iona e sha e o ICU esou ces
(Halpe n & Pas o es, 2010; K ame e al., 2017). The implemen a ion de ails o his p edic ion
ask a y ac oss he li e a u e, e.g., i is amed as a mul iclass classi ica ion (Ha u yunyan e
al., 2019) o eg ession p oblem (Pu usho ham e al., 2018). Fo imp o ed comp ehensibili y,
we ollow Wang e al. (2020) and di ide he ask in o wo bina y classi ica ion asks: leng h-
4We p edic in-hospi al-mo ali y by de e mining whe he pa ien s will die du ing hei hospi al s ay o su i e
o discha ge. Fo eadabili y, we use "mo ali y" as a synonym o in-hospi al-mo ali y in his pape .
123
1100 Annals o Ope a ions Resea ch (2025) 347:1093–1132
Table 1 Desc ip i e s a is ics o
he a ge ea u es: mo ali y,
Leng h-o -s ay3, Leng h-o -s ay7
- s a i ied by gende
Gende To al
Female Male
Mo ali y
Ali e 13,724 17,764 31,488
Dead 1701 1939 3640
LOS
<3 9394 12,179 21,573
>3 6031 7524 13,555
<7 13,376 17,087 30,463
>7 2049 2616 4665
To al 15,425 19,703 35,128
o -s ay>3 days (LOS3) and leng h-o -s ay>7 days (LOS7), p o iding be e pe o mance
compa abili y wi h mo ali y classi ica ion. The same ea u e se s a e used o p edic leng h-
o -s ay and mo ali y, po en ially enabling decision-make s o gain insigh in o expec ed
expendi u es a he pa ien , wa d, and hospi al le els (Halpe n & Pas o es, 2010).
Table 1p esen s desc ip i e s a is ics o he a ge ea u es, mo ali y, LOS3, and LOS7,
showing a signi ican imbalance in hei dis ibu ion. Fo ins ance, mo ali y has a la ge
numbe o ali e pa ien s (31,488 [89.6%]) compa ed o deceased pa ien s (3640 [10.4%]).
Simila ly, mos pa ien s ha e a leng h-o -s ay below he espec i e h esholds o LOS3 and
LOS7.
3.3 Fea u e ex ac ion, p ep ocessing, and e alua ion s a egy
In his sec ion, we desc ibe he p ocess o ea u e ex ac ion, p ep ocessing, and he e alua ion
s a egy o ou ML models. Ou goal is o c ea e a clean and consis en da abase o analysis
and compa ison o he pe o mance gap be ween in e p e able and black-box models.
Fea u e ex ac ion. Rega ding ea u e ex ac ion, we p io i ize ep oducibili y by p ima -
ily elying on he widely-used MIMIC-III benchma k sui e c ea ed by Ha u yunyan e al.
(2019). To c ea e a pa ien -speci ic ime se ies o physiological measu emen s, we u ilize he
co esponding sc ip s wi h he ollowing modi ica ions o he o iginal app oach:
•As sugges ed by Wang e al. (2020) and Pu usho ham e al. (2018), we educe he con-
side ed ime pe iod om he i s 48h o he i s 24h o ICU s ay.
•We emo e implausible alues om he ime se ies, ollowing he app oach o Hegsel-
mann e al. (2020), de ailed in Appendix A.
•The o en missing o al sco e o he Glasgow Coma Scale is ecalcula ed as he sum o
he sub sco es (Teasdale & Jenne , 1974) whene e possible, and subsequen ly he sub
sco es a e emo ed om u he e alua ions.
•Th ee sensi i e ea u es a e in en ionally (see Sec .2.3) included o explo e hei in luence
in de ail: gende , age, and e hnici y.
A e hese s eps, we ob ain a coho o 35,128 pa ien s, displayed in Table 2wi h a pa icula
ocus on pa ien s’ sensi i e ea u es.
To ex ac meaning ul s a ic ea u es om hese ime se ies, we compu e six sample
s a is ics (mean, s anda d de ia ion, minimum, maximum, skewness, and numbe o mea-
su emen s) o se en di e en subpe iods ( ull ime se ies and subse s ep esen ing he i s
123
Annals o Ope a ions Resea ch (2025) 347:1093–1132 1107
o he models in bo h AUROC and AUPRC ac oss all asks and achie ing he highes a e age
anking.
EBM eme ges as he op pe o me wi hin in e p e able models, demons a ing i s com-
pe i i eness wi h black-box models. The pe o mance gap be ween he op black-box model
(XGB) and he op GAM (EBM) is minimal, wi h di e ences anging om 0.2 o 0.9 pe -
cen age poin s in AUROC sco es and 1.0 o 2.8 pe cen age poin s in AUPRC sco es ac oss
asks. No ably, EBM no only anks highes among in e p e able models bu also ou pe o ms
he second black-box model, RF, in all cases excep LOS3. This obse a ion highligh s he
po en ial o in e p e able models, indica ing ha hey can achie e pe o mance compa able
o mo e complex black-box models.
Compa ing ou esul s wi h o he benchma k s udies on he MIMIC-III da ase , ou
AUROC sco es a e consis en wi h he li e a u e. Fo ins ance, Wang e al. (2020) epo
an AUROC sco e o 88.7 o LR and 89.7 o RF on he mo ali y ask, while ou LR and
RF models achie e AUROC sco es o 85.3 and 86.3, espec i ely. Simila ly, Ha u yunyan
e al. (2019) epo an AUROC sco e o 84.8 o he same ask, di e ing by p edic ing on
he i s 48h o pa ien da a, whe eas bo h ou s udy and Wang e al. (2020) p edic on he
i s 24h. Addi ionally, Hegselmann e al. (2020) epo ed esul s closely esembling ou s o
p edic ing mo ali y using LR, including an EBM model which exhibi ed compa able pe o -
mance wi h an AUROC sco e o 87.2, aligning wi h ou alue o 87.1. Fo he leng h-o -s ay
asks, ou esul s sligh ly ou pe o med hose epo ed by Wang e al. (2020). Addi ionally,
di e en p ep ocessing s a egies among a ious s udies, such as hose by Pu usho ham e
al. (2018), lead o a ia ions in he inclusion o exclusion o speci ic a iables.
In conclusion, he esul s in Table 5p o ide a comp ehensi e unde s anding o he
s eng hs and weaknesses o a ious models using he ull da ase . The na ow pe o mance
gap be ween he op black-box and in e p e able models encou ages u he explo a ion
o in e p e able models in ope a ions esea ch, pa icula ly in con ex s whe e model in e -
p e abili y is c ucial o decision-making.
4.2 Sequen ial o wa d loa ing selec ion
Figu e2displays he esul s o he sequen ial o wa d loa ing selec ion app oach using
logis ic eg ession. The dashed ho izon al lines indica e he model pe o mances when u i-
lizing 462 ea u es, excluding he sensi i e ea u es gende , age, and e hnici y (D462-None).
The esul s unco e h ee insigh s. Fi s , i is ema kable ha a single ea u e can p edic
mo ali y, LOS3, and LOS7 wi h AUROC alues anging om 69.6 o 75.2. Second, only
14 ea u es a e needed o achie e a logis ic eg ession model ha pe o ms jus 1.4 o 4.6
pe cen age poin s below he ull model ained on 462 ea u es. Thi d, o all h ee p edic-
ion asks, a pa simonious linea model ained on a subse o ea u es, excluding sensi i e
ea u es (gende , age, and e hnici y), can ou pe o m he model ained on 462 ea u es, also
excluding hese sensi i e ea u es, in e ms o c oss- alida ion pe o mance. This can be
seen by he do ed line, ep esen ing he pe o mance o he pa simonious model, c ossing
he dashed ho izon al line, which ep esen s he pe o mance o he model ained on 462
ea u es, excluding sensi i e ea u es.
4.3 Model pe o mance on compac ea u e se s
We now examine he pe o mance o di e en in e p e able and black-box models on compac
ea u e se s ob ained ia manual selec ion and SFFS. The esul s a e p esen ed in Tables 6and
123

1108 Annals o Ope a ions Resea ch (2025) 347:1093–1132
Table 5 Compa ison o model pe o mance on all ea u es
Mo ali y LOS>3Days LOS>7Days
AUROC AUPRC AUROC AUPRC AUROC AUPRC
RF 86.3 (0.8) 50.4 (2.0) 78.1 (0.6) 70.6 (0.9) 81.5 (1.3) 42.0 (1.9)
XGB 88.0 (0.5) 53.6 (1.6) 78.8 (0.7) 71.6 (0.8) 82.3 (1.4) 42.1 (1.7)
LR 85.3 (0.8) 45.9 (2.0) 76.4 (0.4) 66.9 (1.0) 80.6 (0.6) 38.2 (0.9)
DT 78.0 (0.7) 34.9 (1.0) 74.0 (0.6) 65.2 (0.7) 78.0 (0.7) 34.9 (0.9)
EBM 87.1 (0.6) 50.8 (1.6) 77.9 (0.7) 69.8 (0.8) 82.1 (0.9) 41.1 (1.5)
GAM-Splines 84.6 (0.8) 48.6 (1.4) 77.1 (0.5) 68.4 (0.6) 81.0(0.5) 38.8 (0.9)
IGANN 85.8 (0.5) 46.7 (1.1) 76.9 (0.6) 67.8 (0.8) 80.7 (0.5) 38.1 (0.7)
0.9 2.8 0.9 1.8 0.2 1.0
Sensi i e ea u es gende , age, and e hnici y a e included. deno es he di e ence be ween he bes -pe o ming black-box model and he bes -pe o ming in e p e able model
123
Annals o Ope a ions Resea ch (2025) 347:1093–1132 1109
Fig. 2 5-Fold sequen ial o wa d loa ing selec ion using op imized logis ic eg ession as es ima o . The dashed
ho izon al lines ep esen he model pe o mances when using 462 ea u es, excluding he sensi i e ea u es
gende , age, and e hnici y (D462-None)
7, compa ing he pe o mance o models using only 11 (excluding sensi i e ea u es) and 14
ea u es (including sensi i e ea u es). The numbe s in pa en heses indica e he pe o mance
imp o emen s o de e io a ions compa ed o models ained on he ull da ase including
sensi i e ea u es (D465-None-Sens).
Table 6shows ha he pe o mance o all models ypically declines as he numbe o
ea u es is educed o 14 (including sensi i e ea u es). Howe e , he pe o mance dec ease
is no subs an ial ela i e o he educ ion in he numbe o ea u es used in he models. Fo
ins ance, he AUROC o p edic ing mo ali y o GAMs and black-box models d opped by
1.4 o 2.7 pe cen age poin s despi e educing he ea u e se by 451 ea u es. This obse a ion
sugges s ha he ea u e educ ion did no signi ican ly impac he o e all p edic i e powe
o hese models. Simila esul s we e ob ained o o he asks.
Compa ing he pe o mance o models on da ase s wi h di e en ea u e selec ion me hods
e eals ha models gene ally pe o m be e on da ase s wi h ea u es selec ed using SFFS
han hose selec ed using he mean alues o nume ical ea u es. While SFFS appea s o
be a mo e e ec i e me hod o ea u e selec ion in ou se ing, i may come a he cos o
in e p e abili y, as ea u es based on mo e complex s a is ics (e.g., skewness and s anda d
de ia ion) a e in oduced (see Table 11 in Appendix E o ull ea u e lis s). Consequen ly,
he le el o abs ac ion equi ed o d aw conclusions abou he implica ions o a shape plo
migh be conside ably mo e complex han when using he mean (see Sec .4.4).
Table 7p esen s he pe o mance o he models on educed da ase s wi h 11 ea u es,
excluding he sensi i e ea u es. The esul s indica e ha hese models achie e compa able
pe o mance o hose ained wi h sensi i e ea u es included, demons a ing hei abili y o
main ain a sa is ac o y pe o mance le el despi e he exclusion. Addi ionally, i is e iden
ha he pe o mance o he models on he mo ali y ask pa icula ly su e s om he emo al
o sensi i e ea u es.
123
1110 Annals o Ope a ions Resea ch (2025) 347:1093–1132
Table 6 Compa ison o model pe o mance on educed da ase wi h 14 ea u es, whe e ea u es we e selec ed using mean alue o each ea u e (D14-Man-Sens) and pe o ming
SFFS (D14-Au o-Sens)
Mo ali y LOS >3Days LOS>7Days
AUROC AUPRC AUROC AUPRC AUROC AUPRC
Fea u es selec ed using he mean alues
RF 84.7 (−1.6) 45.1 (−5.3) 74.3 (−3.8) 65.1 (−5.5) 77.7 (−3.8) 34.8 (−7.2)
XGB 85.8 (−2.2) 46.9 (−6.7) 74.2 (−4.6) 65.0 (−6.6) 78.3 (−4.0) 36.0 (−6.1)
LR 80.2 (−5.1) 37.8 (−8.1) 71.5 (−4.9) 60.9 (−6.0) 75.5 (−5.1) 32.7 (−5.5)
DT 75.9 (−2.1) 31.3 (−3.6) 70.8 (−3.2) 59.8 (−5.4) 75.5 (−2.5) 30.9 (−4.0)
EBM 84.8 (−2.3) 44.6 (−6.2) 73.6 (−4.3) 63.0 (−6.8) 78.1 (−4.0) 35.0 (−6.1)
GAM-Splines 83.3 (−1.3) 42.3 (−6.3) 72.8 (−4.3) 62.2 (−6.2) 76.7 (−4.3) 33.9 (−4.9)
IGANN 83.9 (−1.9) 43.7 (−3.0) 73.0 (−3.9) 62.4 (−5.4) 77.0 (−3.7) 34.2 (−3.9)
1.0 2.3 0.7 2.1 0.2 1.0
Fea u es selec ed using sequen ial o wa d loa ing selec ion
RF 84.4 (−1.9) 45.0 (−5.4) 76.3 (−1.8) 68.7 (−1.9) 81.0 (−0.5) 39.1 (−2.9)
XGB 85.3 (−2.7) 45.7 (−7.9) 76.8 (−2.0) 69.2 (−2.4) 81.2 (−0.9) 39.1 (−3.0)
LR 83.5 (−2.3) 42.3 (−3.6) 75.0 (−1.4) 64.6 (−2.3) 76.0 (−4.6) 33.2 (−5.0)
DT 78.6 (+0.6) 34.4 (−0.5) 73.1 (−0.9) 64.0 (−1.2) 77.7 (−0.3) 35.9 (+1.0)
EBM 85.0 (−2.1) 45.4 (−5.4) 75.8 (−2.1) 69.8 (−2.8) 80.8 (−1.3) 38.2 (−2.9)
GAM-Splines 84.6 (–) 44.7 (+3.9) 75.4 (−1.7) 66.6 (−1.8) 80.3 (−0.7) 37.7 (−1.1)
IGANN 84.4 (−1.4) 44.5 (−2.2) 75.5 (−1.4) 66.6 (−1.2) 80.4 (−0.3) 38.2 (+0.1)
0.3 0.3 1.0 0.6 0.4 0.9
Numbe s in pa en heses deno e he imp o emen s (+) o de e io a ion (−) in model pe o mance compa ed o he model pe o mance ained on he ull da ase . Sensi i e ea u es
we e included. deno es he di e ence be ween he bes -pe o ming black-box model and he bes -pe o ming in e p e able model
123
Annals o Ope a ions Resea ch (2025) 347:1093–1132 1111
Table 7 Compa ison o model pe o mance on educed da ase wi h 11 ea u es (D11-Man)
Mo ali y LOS >3Days LOS>7Days
AUROC AUPRC AUROC AUPRC AUROC AUPRC
Fea u es selec ed using he mean alues
RF 83.4 (−1.3) 42.1 (−2.0) 73.9 (−0.4) 64.8 (−0.3) 78.0 (+0.3) 35.2 (+0.4)
XGB 84.3 (−1.5) 44.4 (−2.5) 73.9 (−0.3) 64.5 (−0.5) 78.1 (−0.2) 35.5 (−0.5)
LR 79.0 (−1.2) 36.0 (−1.8) 71.3 (−0.2) 60.7 (−0.2) 73.6 (−1.9) 30.9 (−1.8)
DT 76.2 (+0.3) 30.7 (−0.6) 70.8 (–) 59.8 (–) 75.4 (−0.1) 30.9 (–)
EBM 83.7 (−1.1) 42.5 (−2.1) 73.4 (−0.2) 63.0 (–) 78.0 (−0.1) 34.9 (−0.1)
GAM-Splines 82.2 (−1.1) 41.3 (−1.0) 72.8 (–) 62.2 (–) 73.2 (−3.5) 31.4 (−2.5)
IGANN 82.7 (−1.2) 41.8 (−1.9) 72.9 (−0.1) 62.5 (+0.1) 77.1 (+0.1) 34.3 (+0.1)
0.6 1.9 0.5 1.5 0.1 0.6
Fea u es selec ed using sequen ial o wa d loa ing selec ion
RF 83.9 (−0.5) 44.1 (−0.9) 76.0 (−0.3) 68.5 (−0.2) 80.7 (−0.3) 38.6 (−0.4)
XGB 84.4 (−0.9) 44.4 (−0.9) 76.5 (−0.3) 68.9 (−0.3) 80.9 (−0.3) 38.8 (+0.6)
LR 82.6 (−0.9) 41.1 (−1.2) 74.8 (−0.2) 64.6 (–) 76.0 (–) 33.2 (–)
DT 78.1 (−0.5) 33.2 (−1.2) 73.0 (−0.1) 63.9 (−0.1) 77.6 (−0.1) 35.3 (–)
EBM 83.9 (−1.1) 43.8 (−1.6) 75.7 (−0.1) 66.9 (−2.9) 80.7 (−0.1) 38.1 (−0.1)
GAM-Splines 83.6 (−1.0) 43.7 (−1.0) 75.2 (−0.2) 66.3 (−0.3) 80.2 (−0.1) 37.5 (−0.2)
IGANN 83.5 (−0.9) 43.2 (−1.3) 75.2 (−0.3) 66.4 (−0.2) 80.2 (−0.2) 37.9 (−0.3)
0.5 0.6 0.8 2.0 0.2 0.7
Sensi i e ea u es we e excluded. Numbe s in pa en heses deno e he imp o emen s (+) o de e io a ion (−) in model pe o mance compa ed o he model including sensi i e
ea u es. deno es he di e ence be ween he bes -pe o ming black-box model and he bes -pe o ming in e p e able model
123
1112 Annals o Ope a ions Resea ch (2025) 347:1093–1132
Tables 6and 7compa e he pe o mance o black-box and in e p e able models ac oss
h ee asks (mo ali y, LOS3, LOS7). The bes pe o ming black-box model, XGB, shows
sligh ly highe AUROC sco es han he bes pe o ming in e p e able model, EBM, o all
asks and bo h 11- and 14- ea u e da ase s, wi h EBM’s AUROC sco es only 0.1 o 1.0 pe -
cen age poin s lowe . This sugges s ha EBM o e s compe i i e pe o mance while p o iding
in e p e abili y. Compa ed o he second black-box model, RF, EBM e en shows a o able
esul s in h ee ou o wel e se ings and shows equal esul s in ano he h ee.
O e all, he black-box and in e p e able models showed compa able pe o mance on com-
pac ea u e se s. The esul s indica e ha GAMs a e capable o pe o ming well on compac
ea u e se s ob ained using he wo ea u e selec ion echniques. Fu he mo e, he pe o -
mance losses a e ask-dependen when he sensi i e ea u es a e excluded om he ea u e
se . A key obse a ion is ha pe o mance di e ences be ween ull and educed da ase s
ypically exceed di e ences be ween black-box and in e p e able models. This is also ue
o di e ences be ween manual and au oma ed ea u e selec ion. The pa e n con inues o
compa isons be ween da ase s wi h and wi hou sensi i e ea u es in he mo ali y asks. This
sugges s ha ea u e selec ion has a g ea e impac han he choice be ween black-box and
in e p e able models.
4.4 Assessmen o shape plo s
In his sec ion, we p esen he isual ou pu o he in e p e able models. By analyzing exam-
ples o he gene a ed shape unc ions, we discuss hei plausibili y and in e p e abili y. To
e alua e he plausibili y o he shape plo s, we compa ed he displayed e ec s o he ea u es,
when easible, wi h he e ec s sugges ed by widely used sco ing sys ems such as SAPS o
APACHE.
Addi ionally, we consul ed wi h mul iple medical expe s (MEs) and discussed he plo s
p esen ed in his sec ion o ensu e hei co ec ness and alignmen wi h es ablished medical
expe ise. Fo mo e in o ma ion abou he mee ings and he medical expe s’ backg ound, we
e e he in e es ed eade o Appendix G as well as o Appendix H, which con ains sample
quo es om he in e iews wi h he medical expe s ha illus a e how we a i ed a ou
indings.
I should be no ed ha he choice o da a ex ac ion and p ep ocessing echniques – such
as ou lie handling, impu a ion, and da a balancing – along wi h he selec ed ea u e se can
in luence he esul ing shape plo s. We ocus ou assessmen on hese plo s gene a ed wi h
he pa ame e s ou lined in Sec .3.3 and and using models ained on he educed da ase s.
Howe e in he ollowing igu es, shaped cu es illus a e he impac o nume ical ea u es,
while he impac o ca ego ical ea u es is depic ed by ba cha s. In all plo s he x-axis
ep esen s he ea u e alue, and he y-axis indica es he in luence on he a ge (in log odds),
wi h posi i e in luence o y>0 and nega i e in luence o y<0.
4.4.1 Assessmen o shape plo s om manually selec ed ea u es
Figu e3examines he impac o pa ien s’ age on h ee p edic ion asks and a ious models,
illus a ing shape plo s o he di e en GAMs and logis ic eg ession. Columns ep esen
models and ows ep esen p edic ion asks. All h ee GAMs display simila ends, sug-
ges ing da a co ela ions. Howe e , dis inc ions eme ge. Fo ins ance, EBM gene a es noisy
unc ions wi h ab up ansi ions, GAM-Splines p oduce less equen , la ge luc ua ions, and
IGANN yields smoo h cu es. The sha p jumps in EBM’s shape plo may seem con using
123

Annals o Ope a ions Resea ch (2025) 347:1093–1132 1113
Fig. 3 Shown is he e ec o he age ea u e on di e en p edic ion a ge s. Wi hin each plo , age (in yea s) is on
he x-axis and he e ec on p edic ion (in log odds) is on he y-axis (highe alues indica e a highe p obabili y
o mo ali y, LOS3 o LOS7). The g id con ains h ee ows o shape plo s, one o each p edic ion ask. The
4 columns ep esen di e en GAMs and LR. The bo om ow shows how he age ea u e is dis ibu ed in he
sample. All plo s we e c ea ed a e aining he models on he manually educed da ase (D14-Man-Sens)
(ME1, ME2) o con inuous ea u es like age bu a e sui able o iden i ying h esholds in
ea u e spaces. Also, ea u es wi h e y na ow anges, such as empe a u e, can be be e
analysed using mo e ab up ansi ions, such as EBM (ME3). Con e sely, he ex emely
smoo h cu es p oduced by IGANN a e easie o ead (ME1), bu h esholds migh emain
hidden o small, and meaning ul luc ua ions may go unno iced.
All models exhibi a nea -mono onic ela ionship be ween age and mo ali y, aligning wi h
assessmen ools like SAPS (Mo eno e al., 2005). The ela ionship be ween age and LOS3
shows a simila end. Howe e , a sha p dec ease in p obabili y occu s o ages abo e 80,
sugges ing olde pa ien s ha e sho e ICU s ays, po en ially due o ea ly dea h o ans e o
pallia i e ca e uni s. This assump ion is suppo ed by he mo e p onounced e ec o LOS7.
The linea model ails o cap u e his end e e sal be ween he inpu ea u e and he a ge ,
indica ing ha a linea assump ion may no adequa ely ep esen he ela ionship be ween
age and leng h-o -s ay. This example highligh s si ua ions whe e simple , linea models may
be less app op ia e o desc ibing ea u e e ec s and hinde in e p e abili y.
Figu e4p esen s a se o shape plo s o p edic ing mo ali y on he manually educed
da ase wi h sensi i e ea u es. All h ee GAMs consis en ly link abno mal empe a u e and
blood p essu e o highe mo ali y a es. Bo h upwa d and downwa d de ia ions a e ha m ul,
wi h he la e being pa icula ly de imen al. This is consis en wi h SAPS sco es (Mo eno
e al., 2005) and he expe ience o he medical expe s. An unexpec ed ela ionship a ises
be ween mo ali y and Glasgow Coma Scale To al (GCST) sco e: a mono onic nega i e
ela ionship is expec ed, as pa ien s wi h highe GCST sco es a e classi ied as mo e conscious.
Al hough his nega i e end is ecognizable, all GAMs show a s eep inc ease in mo ali y
p obabili y be ween 13 and 14, con adic ing medical expe s (ME1) as well as medical
123
1114 Annals o Ope a ions Resea ch (2025) 347:1093–1132
Fig. 4 Shown is he e ec o 5 (ou o 14) ea u es on pa ien mo ali y. We used he manually gene a ed
educed da ase including sensi i e ea u es. In each plo , di e en alues o he ea u es a e on he x-axis and
he e ec on mo ali y (in log odds) is on he y-axis (highe alues indica e a highe p obabili y o mo ali y).
The g id con ains 4 ows, one o each model, and 5 columns, each ep esen ing a ea u e. The bo om ow
shows he ea u e dis ibu ion in he sample. All plo s we e c ea ed a e aining he models on he educed
da ase (D14-Man-Sens)
li e a u e (Teasdale & Jenne , 1974). Fu he mo e, all models indica e highe mo ali y o
males, a disc epancy deba ed bu uncon i med in medical li e a u e (Hollinge e al., 2019).
Appendix F shows he shape plo s o he emaining ea u es.
4.4.2 Assessmen o shape plo s om au oma ically selec ed ea u es
Finally, we conside a selec ion o shape plo s o p edic ing mo ali y based on he educed
da ase gene a ed wi h SFFS, excluding sensi i e ea u es (D11-Au o). The selec ed plo s a e
shown in Fig.5. We obse e nea ly iden ical shape plo s o dias olic blood p essu e and
empe a u e, selec ed by bo h SFFS and he manual app oach. This also holds o espi a o y
a e, wi h SFFS conside ing only he ini ial 12h o ICU s ay. Fo he GCST sco e, SFFS
conside s he inal 10% o he i s 24h (i.e., he las 144min), e ealing he expec ed nega i e
mono onic co ela ion and inc easing plausibili y. Addi ionally, a GCST-based ea u e, he
s anda d de ia ion o he i s 12h, exhibi s a clea nega i e end. Howe e , in e p e ing
ea u es based on s anda d de ia ion can be challenging, as also ME4 con i ms. This nega i e
end in he ea u e e ec became mo e unde s andable a e discussions wi h medical expe s
who con i med ha i is e y common o pa ien s o be admi ed o he ICU unde anes hesia
a e a majo medical p ocedu e and hen wake up as planned (ME1, ME3). A high GCST
s anda d de ia ion indica es ha pa ien s expe ience a change be ween coma ose and ully
123
Annals o Ope a ions Resea ch (2025) 347:1093–1132 1115
Fig. 5 Shown is he e ec o 5 (ou o 11) ea u es on pa ien mo ali y. This is he educed da ase gene a ed
by SFFS wi hou he sensi i e ea u es (D11-Au o). In each plo , di e en alues o he ea u es a e on he
x-axis and he e ec on mo ali y (in log odds) is on he y-axis (highe alues indica e a highe p obabili y
o mo ali y). The g id con ains 4 ows, one o each model, and 5 columns, each ep esen ing a ea u e. The
bo om ow shows how he ea u es a e dis ibu ed in he sample
conscious du ing he i s 12h o hei ICU s ay. Howe e , he unc ion does no indica e
he di ec ion o he change. Medical expe s exp ess conce n abou including such logic in a
model (ME2, ME3).
5 Discussion
5.1 In e p e a ion o esul s
The ade-o be ween model pe o mance and in e p e abili y is c ucial in heal hca e analy -
ics and widely discussed in he ope a ions esea ch li e a u e (Yang, 2022; Be simas e al.,
2021). S akeholde s equi e a balance be ween in e p e abili y and pe o mance, ocusing
on ac ionable insigh s (Coussemen & Benoi , 2021), egula o y compliance (e.g., GDPR)
(Pa liamen and Council o he Eu opean Union, 2016), and e hical ai ness (Goodman &
Flaxman, 2017).
Ou expe imen s yield se e al impo an insigh s. Fi s , GAMs eme ge as a p omising
solu ion o add essing he pe o mance-in e p e abili y ade-o in heal hca e analy ics.
The minimal pe o mance gap be ween GAMs and leading black-box models makes hem an
a ac i e op ion o balancing p edic i e powe and in e p e abili y. This inding aligns wi h
he wo k o Chang e al. (2021) and Zschech e al. (2022), who demons a ed ha pe o mance
123
1116 Annals o Ope a ions Resea ch (2025) 347:1093–1132
penal ies associa ed wi h in e p e able models can anish en i ely in ce ain se ings o e en
u n in o pe o mance ad an ages.
Ou esul s a e consis en wi h o he benchma k s udies on he MIMIC-III da ase , as
discussed in Sec . 4.1. The AUROC sco es we ob ained o he mo ali y ask wi h LR
and RF models a e compa able o hose epo ed by Wang e al. (2020) and Ha u yunyan
e al. (2019). Fu he mo e, ou EBM model’s pe o mance closely ma ches he esul s o
Hegselmann e al. (2020) o he same ask. These compa isons alida e he obus ness o
ou indings and demons a e ha ou models’ pe o mance aligns wi h he s a e-o - he-a
in he ield. Howe e , i is impo an o no e ha di ec ly compa ing esul s ac oss s udies
can be challenging due o di e ences in p ep ocessing s a egies and a iable inclusion. Fo
ins ance, Pu usho ham e al. (2018) employed di e en p ep ocessing echniques, leading o
a ia ions in he a iables used o modeling. These di e ences highligh he need o cau ion
when making di ec compa isons and emphasize he impo ance o conside ing he speci ic
con ex and me hodology o each s udy.
Second, educing ea u es o ob ain pa simonious models o excluding sensi i e ea u es
can lead o accu acy loss, which in ou case was mo e p onounced wi h manual app oaches
han au oma ed me hods like Sequen ial Floa ing Fo wa d Selec ion. No ably, di e ences
in p edic i e pe o mance be ween ea u e selec ion me hods we e consis en ly mo e sub-
s an ial han dispa i ies be ween black-box and in e p e able models. Fo ins ance, sensi i e
ea u es like gende , age, and e hnici y showed a ying impo ance ac oss p edic ion asks.
Pa icula ly, o mo ali y p edic ion, he impac o excluding hese ea u es was much la ge
han o leng h-o -s ay p edic ion quali y.
Thi d, shape plo compa isons ac oss da ase e sions indica e ha GAMs exhibi s abil-
i y agains ea u e selec ion p ocedu es. The SFFS-based app oach, while esul ing in highe
ag eemen wi h medical e idence, includes mo e complex ea u es, complica ing in e p e-
a ion. As demons a ed in ou s udy, in e p e ing ea u es based on s anda d de ia ion is
challenging, and his complexi y inc eases o ea u es based on skewness o he numbe o
measu emen s aken. None heless, such ea u es can be s ong p edic o s; hus, when u i-
lizing ea u e selec ion me hods ocused on in e p e abili y, i is essen ial o balance bo h
p edic i e capaci y and simplici y o ea u e selec ion.
O e all, GAMs align well wi h medical knowledge, bu some plo de ails equi e u he
in es iga ion. The lea ned ela ionships emain s able despi e da ase changes due o sensi i e
ea u e exclusion o changes in ime pe iods. While GAMs a e desi able o hei accu acy and
in e p e abili y, i is c ucial o no e ha hey a e no causal models: shape plo s demons a e
p edic ion gene a ion bu do no p o ide easons o lea ned pa e ns o in e en ion e ec s.
The implica ions o hese indings will be discussed u he in he ollowing sec ions.
5.2 P ac ical implica ions
Ou s udy has se e al p ac ical implica ions o he applica ion o GAMs in medical da a
science and heal hca e se ings.
In e p e abili y and collabo a ion. We ea i m ha GAMs a e well-sui ed o medical
da a science applica ions due o hei high p edic i e pe o mance and in e p e abili y (Ca u-
ana e al., 2015; Aga wal & Das, 2020). To achie e in e p e abili y, models equi e ca e ul,
manageable, and o en ime-consuming ea u e ex ac ion and p ep ocessing s eps. Insu i-
cien p ep ocessing can lead o ambiguous shape plo s, pa icula ly in egions wi h only ew
da a poin s o when ex eme and implausible alues, such as nega i e age, occu . As a esul ,
123
Annals o Ope a ions Resea ch (2025) 347:1093–1132 1123
Table 10 G id showing all es ed pa ame e combina ions. No e ha he bes -pe o ming hype pa ame e s
a y ac oss ou asks
Model Tuning pa ame e s Tuning ange
XGB Num. es ima o s 50, 100, 200, 500, 1000, 2000
Max. dep h None, 3, 6, 9, 12
Lea ning a e 0.01, 0.1, 0.3
Random Fo es Num. es ima o s 50, 100, 200, 500, 1000
Max. dep h None, 5, 10, 20, 40
Class weigh None, balanced
Logis ic Reg ession Regula iza ion s eng h 1e−3, 1e−2,..., 1e2, 1e3
Penal y e m L1, L2
Sol e lb gs, saga
Class weigh None, balanced
Decision T ee Max. dep h None, 5, 10, 20, 40
Max. lea nodes None, 5, 10, 20, 40
Class weigh None, balanced
Spli e bes , andom
EBM Max. bins 256, 512
Ou e bags 8, 16
Inne bags 0, 4
GAM-Splines Num. splines 5, 10, 15, 20, 25
Regula iza ion s eng h Log scale: 10−3 o 104
IGANN ELM scaling pa ame e 1, 2, 5
Boos ing a e 0.025, 0.1
p e ML8package e sion 0.2.7. IGANN9can be ob ained om Gi Hub. We applied a g id
sea ch me hod o sys ema ically explo e he hype pa ame e space o each model. By es ing
di e en combina ions o hype pa ame e s, we aimed o iden i y he op imal con igu a ion
o each model o ensu e a ai compa ison. Table 10 lis s ou hype pa ame e g id.
Appendix E: Compa ison o ea u es selec ed by he au oma ed s. he
manual app oach
We employ wo ypes o ea u e selec ion, as discussed in Sec .3.4. The manually selec ed
ea u e se (D11-Man) emains he same ac oss all asks, while Sequen ial Fo wa d Fea u e
Selec ion (SFFS) inds po en ially op imal ea u e se s (D11-Au o and D14-Au o-Sens) o each
ask. Table 11 lis s he manually and SFFS-selec ed ea u es o ep oducibili y and ans-
pa ency. The ea u e names a e de i ed as ollows: an abb e ia ion o he ime se ies om
which he measu emen s we e aken (e.g., MBP = Mean blood p essu e), ollowed by he
speci ic po ion o he ime se ies conside ed, gi en as a pe cen age, including he sign indi-
ca ing whe he he (–) las o (+) i s pa o he ime se ies is used (e.g., +50% = i s 12h,
8h ps://gi hub.com/in e p e ml/in e p e .
9h ps://gi hub.com/Ma hiasK aus/igann.
123

1124 Annals o Ope a ions Resea ch (2025) 347:1093–1132
Table 11 Task-speci ic ea u es
selec ed by ei he he manual
(mean alues) o au oma ed
(SFFS) app oach
Task Selec ion me hod
D11-Man D11-Au o
Mo ali y GCST+100%mean MBP+100%min
Weigh +100%mean OS-10%mean
HR+100%mean RR+50%mean
MBP+100%mean pH+100%s d
SBP+100%mean GCST-10%mean
GLU+100%mean Temp+100%mean
RR+100%mean DBP-50%mean
DBP+100%mean GCST+50%s d
Temp+100%mean HR-25%max
OS+100%mean GLU-50%min
pH+100%mean Weigh -10%min
LOS>3 days GCST+100%mean Ph+50%s d
Weigh +100%mean GCST+100%s d
HR+100%mean DBP-10%min
MBP+100%mean GCST-25%len
SBP+100%mean RR+100%mean
GLU+100%mean GCST-50%mean
RR+100%mean OS+100%min
DBP+100%mean GCST-10%mean
Temp+100%mean pH-25%len
OS+100%mean SBP+100%min
pH+100%mean HR-10%mean
LOS>7 days GCST+100%mean GCST-25%len
Weigh +100%mean MBP+100%min
HR+100%mean OS-50%skew
MBP+100%mean GCST+100%s d
SBP+100%mean HR-25%max
GLU+100%mean pH-25%len
RR+100%mean GCST-25%mean
DBP+100%mean pH+50%s d
Temp+100%mean OS+50%mean
OS+100%mean RR+100%mean
pH+100%mean RR+100%min
-10% = las 2.4h), and an abb e ia ion indica ing which summa y s a is ic (mean, s anda d
de ia ion, minimum, maximum, skewness, and numbe o measu emen s) is used o calcula e
he ea u e (e.g., len = numbe o measu emen s).
123
Annals o Ope a ions Resea ch (2025) 347:1093–1132 1125
Appendix F: Addi ional shape plo s
This sec ion p esen s 36 addi ional shape plo s ha complemen he plo s shown in Fig. 4,
displaying he e ec o he emaining 9 (ou o 14) ea u es on pa ien mo ali y isk lea ned
by he GAMs analyzed in Sec .4.4. These plo s p o ide a mo e holis ic iew o he p edic i e
beha io o he models, allowing he eade o e iew a comple e basis o he p edic ions
and gain a be e unde s anding o he o e all model beha io .
Appendix G: Consul a ions wi h medical expe s
To assess he in e p e abili y and alignmen o he shape plo s wi h medical knowledge, we
consul ou medical expe s and discuss eigh se s o ques ions (Q0-Q7).
Du ing hese consul a ions, we i s ask he expe s abou hei medical backg ounds and
expe ience wi h ICU pa ien s o es ablish he con ex (Q0). We hen p o ide an o e iew
o ou s udy, explaining he a ionale o compa ing in e p e able and black-box ML mod-
els in ICU decision-making. Nex , we del e in o he da a basis o ou s udy, de ailing ou
use o he MIMIC-III da abase. We desc ibe ou p edic ion a ge s, mo ali y and leng h-o -
s ay, and ou line he p ocess o c ea ing a supe ised ML model o his pu pose. We seek
opinions on he ele ance o ML p edic ions, hei po en ial in eg a ion in o ICU wo k lows,
and associa ed isks and challenges (Q1). Emphasizing he impo ance o e alua ing ML-
gene a ed medical p ognoses, we ga he expe assessmen s o he ML sys em. We hen ocus
on pe cep ions o ML-based p edic ions, speci ically us in opaque ML sys ems and mea-
su es o inc ease us (Q2). Nex , we desc ibe in e p e able ML models, pa icula ly GAMs,
highligh hei anspa ency and isualiza ion me hods. We discuss he p ocess o a iable
ex ac ion o lea ning and p edic ions, add essing he comp ehensibili y o shape plo s wi h
ques ions abou hei use ulness and cla i y (Q3). We analyze speci ic ea u es o de e mine
whe he he ela ionships shown a e consis en wi h medical knowledge (Q4). We assess he
sui abili y o such a p edic ion model o in-hospi al use, conside ing he comp ehensibili y
o shape plo s, he numbe o plo s equi ed, and he impo ance o ea u e in e ac ions (Q5).
We compa e di e en shape plo s om a ious GAMs, looking o p e e ences and obse ed
di e ences (Q6). Finally, we discuss an example o a mo e complex ea u e and i s shape
plo in e p e a ion, ocusing on hei implica ions o p edic ion (Q7).
All consul a ions a e conduc ed ace- o- ace o ia ideo con e ence and a e suppo ed by
a p esen a ion. These sessions a e eco ded, ansc ibed, and analyzed o insigh s ega ding
he in e p e abili y and alignmen o he shape plo s wi h he expe s’ medical knowledge.
Table 12 p o ides basic in o ma ion abou he expe s, including hei cu en posi ions and
expe ience wi h in ensi e ca e pa ien s. The in e es ed eade can access he ansla ed e sion
o he p esen a ion used in he consul a ions.10 I is impo an o no e ha hese consul a ions
we e no conduc ed in English, and he s a emen s and p esen a ion we e ansla ed o his
pape . Addi ionally, he consul a ions did no include a comple e analysis o he GAMs bu
a he aimed o gauge he ini ial eac ions o medical expe s o he shape plo s and hei
expec ed alue in a p ac ical con ex .
10 Fo he ansla ed e sion o he p esen a ion see doi.o g/10.17605/OSF.IO/2WP6F.
123
1126 Annals o Ope a ions Resea ch (2025) 347:1093–1132
Fig. 7 E ec o 9 (ou o 14) ea u es on pa ien mo ali y, comple ing he shape plo s p esen ed in Fig.4.
Models we e ained on he manually educed da ase (D11-Man)
123
Annals o Ope a ions Resea ch (2025) 347:1093–1132 1127
Table 12 Summa y o medical expe ience
ID Cu en posi ion Expe ience wi h ICU pa ien s
ME1 Residen physician,
in e nal medicine
- Responsible o ICU pa ien s du ing anspo s
- 8 yea s as pa amedic
ME2 Residen su geon,
pedia ic su ge y
- Wo ks in neona al ICU
- P o ides in ensi e su gical ca e o p ema u e in an s
- Daily pedia ic ICU ca e
- 5.5 yea s as pedia ic su geon a uni e si y hospi al
ME3 Chie physician a
anes he ic ICU
- Focused on ICU pa ien s o 13 yea s
- Responsible o 30-bed anes he ic ICU
- 18 yea s as anes hesiologis
ME4 Physician in p i a e
medical o ice
- Supe ised di e en ICUs (15 yea s in o al)
- 30 yea s as anes hesiologis
Appendix H: Di ec quo es om medical expe s
In his sec ion, we p esen some di ec quo es om he in e iews wi h he medical expe s
ha explain how we a i ed a ou conclusions om he consul a ions wi h hem. These
quo es allow he eade o ge an un il e ed imp ession om he medical expe s’ pe spec i e.
•The di e en opinions o he medical expe s on he applicabili y o he shape plo s o
he di e en GAM models in he con ex o in ensi e ca e medicine.
ME1: "Well, I mean, his uns eadiness in he [EBM] g aph pe haps sugges s an accu acy
ha is ul ima ely no he e a e all. [...] Tha ’s why I wouldn’ mind a la e , sligh ly
a e aged g aph."
ME2: "The impo an hing is he end [...] no whe he he e is a sligh educ ion in
[...] mo ali y isk [a a age o ] 35 o wha e e . [...] as a doc o wo king clinically a he
bedside o an in ensi e ca e uni , I would hink, no, I don’ need i ."
ME3: "Fo hings like empe a u e and pH, whe e I ha e a ela i ely na ow ange, I
hink a mo e p ecise ep esen a ion [ e e s o he EBM] is pe haps mo e p ac ical."
•A medical expe ’s pe plexi y ega ding he inc eased mo ali y p obabili y o pa ien s
wi h a GCST o 14.
ME1: "I do no qui e unde s and he ou lie a 14."
•Commen s on he shape plo s o blood p essu e and empe a u e.
ME2: "So low blood p essu e is always wo se han e y high blood p essu e. Unless you
ge complica ions om high blood p essu e, [...] like a ce eb al hemo hage."
ME1: "Especially in he lowe empe a u e anges, he middle g aph [GAM-Splines] has
a peak a abou 35 deg ees, which I can’ qui e wo k ou whe e i comes om. Yes, as I
said, I al eady know he one on he le [EBM]. I don’ qui e unde s and why he pla eau
is in he lowe empe a u e anges. F om my poin o iew, I hink he one on he igh
[IGANN] is he mos plausible."
•Quo es on he di icul y o in e p e ing a shape plo ha is based on he s anda d de ia ion
o a ea u e a he han he mean o ha ea u e.
123
1128 Annals o Ope a ions Resea ch (2025) 347:1093–1132
ME4: "I ind ha eally di icul . Yes, so he i s hing ha is i i a ing is ha he Glasgow
Coma Scale [...], ha you don’ s a wi h 15, bu as a clinician you end o look a whe he
i ’s 3 o [...] 15, so a i s I would hink, well, I ind ha di icul and I would ha e o ask
again, wha do you mean by a iabili y in he a iable."
•Ideas on how o explain he nega i e mono onic end be ween an inc eased a iabili y
o GCST and a lowe p edic ion o mo ali y isk.
ME1: "The case wi h a high a iabili y, which would be p o ec i e, is ha a pa ien comes
o he ICU a e an ope a ion, is s ill en ila ed, hen pe haps a GCST o h ee is en e ed
on admission. And hen, o cou se, [ he pa ien ] will be allowed o wake up as quickly
as possible in he nex hou s. I he condi ion allows i . Then, o cou se, he may show a
s ong imp o emen . O cou se, his is also a case ha occu s equen ly in ICUs, i.e.,
se e al imes a day, whe e pa ien s come in and wake up a e an ope a ion. I could jus
imagine ha his he eason o he sys em o see his high a iabili y as p o ec i e."
ME3: "So he mo e common case is, o cou se, ha I admi a pa ien om he OR who
is s ill unde anes hesia, has a GCS o 3, hen I wake him up and he has a GCS o 15, so
ha ’s why he end is simply mo e equen ."
•Quo es om medical expe s exp essing conce n abou inco po a ing he lea ned ela-
ionship be ween GCST s anda d de ia ion and mo ali y in o a po en ial ICU decision
suppo sys em.
ME2: "I ind i di icul because i can go bo h ways. So i could be ha you come up
wi h a scale o 3 and hen i ’s 15 o he o he way a ound."
ME3: "The e is also he case ha I admi a pa ien wi h a GCS o 15 and now he has a
ce eb al hemo hage and hen 3 hou s la e has a GCS o 3."
•E hical implica ions o ICU mo ali y p edic ion ha eme ged du ing he consul a ions.
ME3: "The e is de ini ely an e hical componen behind his, which is no insigni ican
in in ensi e ca e medicine [...] wha do I do wi h his pe cen age [ isk o dea h]? [...]
he algo i hm ells me ha he pa ien has a 10% p obabili y o su i al [...], wha is he
conclusion hen? Is he conclusion: he is going o die wi h a chance o 90% anyway, so
we s op he he apy. O do we say: He has a 10% chance, so le ’s y e e y hing we can,
o imp o e his 10% pe haps by aking some measu es [...] so o he mo ali y isk, he
doc o p o iding he he apy is no he ideal a ge g oup."
ME4: "So he e a e si ua ions whe e you say: is i s ill wo h i ? To pu i blun ly.
When hey’ e jus o ally sick and you say you’ll y any hing [...] hen you can possibly
p olonging su e . [desc ibes example case in de ail] And igh a he beginning, i you
had aken all hese pa ame e s in o accoun , you may wouldn’ ha e had o do ha ."
•Commen s on he impo ance o in eg a ing in e ac ion e ms in o p edic i e models.
ME4: "Physiological sys ems always in e ac and he a iables a e all in e dependen .
So he e is ac ually no single a iable ha is no connec ed o any hing else, and o ha
ex en , he in e ac ion is impo an ."
Au ho Con ibu ions L.B., J.R., M.K., and P.Z. designed he s udy. L.B. and J.R. w o e he manusc ip . In
his p ocess, M.K. and P.Z. p o ided de ailed eedback and we e in ol ed in designing he o e all s uc u e.
123

Annals o Ope a ions Resea ch (2025) 347:1093–1132 1129
L.B. and J.R. ex ac ed and analyzed he da a. All au ho s c i ically e iewed he manusc ip and con ibu ed
o he in e p e a ion o he esul s. L.B. and J.R. a e he gua an o s o his wo k and, as such, had ull access
o all he da a in he s udy and ake esponsibili y o he in eg i y o he da a and he accu acy o he da a
analysis. All au ho s app o ed he inal d a o he manusc ip .
Funding Open Access unding enabled and o ganized by P ojek DEAL. J.R., P.Z., and M.K. acknowl-
edge unding om he Fede al Minis y o Educa ion and Resea ch (BMBF) on “Whi e-Box-AI” (G an
01IS22080). L.B. and P.Z. acknowledge unding om he Fede al Minis y o Educa ion and Resea ch (BMBF)
on "AddICh on" (G an 16SV8995). This wo k was suppo ed by an Academic Ha dwa e G an p o ided by
N idia.
Da a a ailibili y s a emen This s udy u ilizes he MIMIC-III da ase , a publicly a ailable and de-iden i ied
heal h- ela ed da abase con aining comp ehensi e in o ma ion on ICU pa ien s om Be h Is ael Deaconess
Medical Cen e (2001–2012). Access o he da ase equi es comple ion o he CITI "Da a o Specimens Only
Resea ch" cou se and a signed da a use ag eemen . Fo ins uc ions on ob aining access o he MIMIC-III
da ase , please isi he o icial websi e.
Rele an links: MIMIC-III: h ps://doi.o g/10.13026/C2XW26 P ojec websi e: h ps://mimic.physione .o g.
Access ins uc ions: h ps://mimic.mi .edu/ge ings a ed/access/.
Code A ailabili y As desc ibed abo e, webased ou ea u e ex ac ion on he MIMIC-III benchma k de eloped
by Ha u yunyan e al. (2019). We o ked hei o iginal Gi hub eposi o y and modi ied i o sui ou needs.
* Ou o k is a ailable he e: h ps://gi hub.com/HB-Dynami e/mimic3-benchma ks_AoOR_da a_expo .
* Fo ou expe imen s, we c ea ed a sepa a e eposi o y, a ailable he e: h ps://gi hub.com/HB-Dynami e/
In e p e able_ICU_p edic ions.
Decla a ions
Con lic o in e es The au ho s ha e no con lic o in e es o decla e ha a e ele an o he con en o his
a icle.
Open Access This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional License, which
pe mi s use, sha ing, adap a ion, dis ibu ion and ep oduc ion in any medium o o ma , as long as you gi e
app op ia e c edi o he o iginal au ho (s) and he sou ce, p o ide a link o he C ea i e Commons licence,
and indica e i changes we e made. The images o o he hi d pa y ma e ial in his a icle a e included in he
a icle’s C ea i e Commons licence, unless indica ed o he wise in a c edi line o he ma e ial. I ma e ial is
no included in he a icle’s C ea i e Commons licence and you in ended use is no pe mi ed by s a u o y
egula ion o exceeds he pe mi ed use, you will need o ob ain pe mission di ec ly om he copy igh holde .
To iew a copy o his licence, isi h p://c ea i ecommons.o g/licenses/by/4.0/.
Re e ences
Aga wal, N., & Das, S. (2020). In e p e able machine lea ning ools: A su ey. In 2020 IEEE symposium
se ies on compu a ional in elligence (SSCI), pp. 1528–1534.
Aga wal, R., Melnick, L., F oss , N., Zhang, X., Lenge ich, B., Ca uana, R., Hin on, G. E. (2021). Neu al
addi i e models: In e p e able machine lea ning wi h neu al ne s. In Ad ances in neu al in o ma ion
p ocessing sys ems ( ol. 34, pp. 4699–4711).
Babic, B., Ge ke, S., E geniou, T., & Cohen, I. G. (2021). Bewa e explana ions om AI in heal h ca e. Science,
373(6552), 284–286.
Bai, J., Fügene , A., Schoen elde , J., & B unne , J. O. (2018). Ope a ions esea ch in in ensi e ca e uni
managemen : A li e a u e e iew. Heal h Ca e Managemen Science, 21(1), 1–24.
Ba es, D. W., Sa ia, S., Ohno-Machado, L., Shah, A., & Escoba , G. (2014). Big da a in heal h ca e: Using
analy ics o iden i y and manage high- isk and high-cos pa ien s. Heal h A ai s, 33(7), 1123–1131.
Be simas, D., Pauphile , J., S e ens, J., & Tandon, M. (2021). P edic ing inpa ien low a a majo hospi al
using in e p e able analy ics. Manu ac u ing & Se ice Ope a ions Managemen .
Boh , A., & Mema zadeh, K. (2020). The ise o a i icial in elligence in heal hca e applica ions. A i icial
In elligence in Heal hca e, pp. 25–60.
123
1130 Annals o Ope a ions Resea ch (2025) 347:1093–1132
B eiman, L. (2001). Random o es s. Machine Lea ning, 45(1), 5–32.
B eiman, L., F iedman, J., S one, C. J., & Olshen, R. A. (1984). Classi ica ion and eg ession ees. London:
Rou ledge.
B un on, S. L., & Ku z, J. N. (2019). Da a-d i en science and enginee ing: Machine lea ning, dynamical
sys ems, and con ol. Camb idge: Camb idge Uni e si y P ess.
Ca uana, R., Lou, Y., Geh ke, J., Koch, P., S u m, M., Elhadad, N. (2015). In elligible models o heal hca e:
P edic ing pneumonia isk and hospi al 30-day eadmission. In P oceedings o he 21 h ACM SIGKDD
in e na ional con e ence on knowledge disco e y and da a mining (pp. 1721–1730). New Yo k, NY.
Chang, C- H., Tan, S., Lenge ich, B., Goldenbe g, A., Ca uana, R. (2021). How in e p e able and us wo hy
a e GAMs? In P oceedings o he 27 h ACM SIGKDD con e ence on knowledge disco e y and da a
mining (pp. 95–105). New Yo k, NY.
Chen, T., & Gues in, C. (2016). XGBoos : A scalable ee boos ing sys em. In P oceedings o he 22nd ACM
SIGKDD in e na ional con e ence on knowledge disco e y and da a mining (pp. 785–794). New Yo k,
NY.
Coussemen , K., & Benoi , D. F. (2021). In e p e able da a science o decision making. Decision Suppo
Sys ems, 150, 113664.
Da is, J., & Goad ich, M. (2006). The ela ionship be ween p ecision- ecall and oc cu es. In P oceedings
o he 23 d in e na ional con e ence on machine lea ning (pp. 233–240). New Yo k, NY.
Doshi-Velez, F., & Kim, B. (2017). Towa ds a igo ous science o in e p e able machine lea ning. a Xi
p ep in a Xi :1702.08608
Du, M., Liu, N., & Hu, X. (2019). Techniques o in e p e able machine lea ning. Communica ions o he
ACM, 63(1), 68–77.
Ge ke, S., Minssen, T., Cohen, G. (2020). E hical and legal challenges o a i icial in elligence-d i en heal h-
ca e. A i icial in elligence in heal hca e (pp. 295–336). Else ie .
Goodman, B., & Flaxman, S. (2017). Eu opean union egula ions on algo i hmic decision-making and a “Righ
o Explana ion”. AI Magazine, 38(3), 50–57.
Gunning, D., & Aha, D. (2019). DARPA’s explainable a i icial in elligence (XAI) p og am. AI Magazine,
40(2), 44–58.
Halpe n, N. A., & Pas o es, S. M. (2010). C i ical ca e medicine in he uni ed s a es 2000–2005: An analysis
o bed numbe s, occupancy a es, paye mix, and cos s. C i ical Ca e Medicine, 38(1), 65–71.
Ha u yunyan, H., Khacha ian, H., Kale, D. C., S eeg, G. V., & Gals yan, A. (2019). Mul i ask lea ning and
benchma king wi h clinical ime se ies da a. Scien i ic Da a, 6(1), 96.
Has ie, T., & Tibshi ani, R. (1987). Gene alized addi i e models: Some applica ions. Jou nal o he Ame ican
S a is ical Associa ion,82(398)
Hegselmann, S., Volke , T., Ohlenbu g, H., Go schalk, A., Dugas, M., E me , C. (2020). An e alua ion o
he doc o -in e p e abili y o gene alized addi i e models wi h in e ac ions. F. Doshi-Velez e al. (Eds.),
P oceedings o he 5 h machine lea ning o heal hca e con e ence ( ol. 126, pp. 46–79). PMLR.
Hollinge , A., Gaya , E., Félio , E., Paugam-Bu z, C., & Fou nie , M- C., Du an eau, J. o he s,. (2019). Gende
and su i al o c i ically ill pa ien s: Resul s om he og-icu s udy. Annals o In ensi e Ca e,9, 1–8.
Huang, G- B., Zhu, Q- Y., Siew, C- K. (2006). Ex eme lea ning machine: Theo y and applica ions. Neu o-
compu ing,70(1–3), 489–501.
Hyland, S. L., Fal ys, M., Hüse , M., Lyu, X., Gumbsch, T., Es eban, C., & Me z, T. M. (2020). Ea ly p edic ion
o ci cula o y ailu e in he in ensi e ca e uni using machine lea ning. Na u e Medicine, 26(3), 364–373.
James, G., Wi en, D., Has ie, T., Tibshi ani, R. (2013). An in oduc ion o s a is ical lea ning ( ol. 112).
Sp inge .
Johnson, A., Polla d, T., Ma k, R. (2016). MIMIC-III clinical da abase. PhysioNe .
Johnson, A. E., Polla d, T. J., Shen, L., Lehman, L. W. H., Feng, M., Ghassemi, M., & Ma k, R. G. (2016).
Mimic-iii, a eely accessible c i ical ca e da abase. Scien i ic Da a,3(1), 1–9.
Johnson, M., Albiz i, A., & Simsek, S. (2022). A i icial in elligence in heal hca e ope a ions o enhance
ea men ou comes: A amewo k o p edic lung cance p ognosis. Annals o Ope a ions Resea ch,
308(1), 275–305.
Koche u o , A., Pa dalos, P. M., & Ka aki siou, A. (2019). Massi e da ase s and machine lea ning o com-
pu a ional biomedicine: ends and challenges. Annals o Ope a ions Resea ch, 276(1), 5–34.
K ame , A. A., Das a, J. F., & Kane-Gill, S. L. (2017). The impac o mo ali y on o al cos s wi hin he ICU.
C i ical Ca e Medicine, 45(9), 1457–1463.
K aus, M., Feue iegel, S., & Saa -Tsechansky, M. (2024a). Da a-d i en alloca ion o p e en i e ca e wi h
applica ion o diabe es melli us ype ii. Manu ac u ing & Se ice Ope a ions Managemen , 26(1), 137–
153.
K aus, M., Tsche nu e , D., Weinzie l, S., & Zschech, P. (2024b). In e p e able gene alized addi i e neu al
ne wo ks. Eu opean Jou nal o Ope a ional Resea ch, 317(2), 303–316.
123
Annals o Ope a ions Resea ch (2025) 347:1093–1132 1131
Kundu, S. (2021). AI in medicine mus be explainable. Na u e Medicine, 27(8), 1328–1328.
Lenge ich, B., Tan, S., Chang, C- H., Hooke , G., Ca uana, R. (2020). Pu i ying in e ac ion e ec s wi h he
unc ional ano a: An e icien algo i hm o eco e ing iden i iable addi i e models. In S. Chiappa and
R. Caland a (Eds.), P oceedings o he wen y hi d in e na ional con e ence on a i icial in elligence
and s a is ics (Vol. 108, pp. 2402–2412). PMLR.
Lou, Y., Ca uana, R., Geh ke, J. (2012). In elligible models o classi ica ion and eg ession. In P oceedings
o he 18 h ACM SIGKDD in e na ional con e ence on Knowledge disco e y and da a mining-KDD ’12
(p.150). Beijing, China.
Lou, Y., Ca uana, R., Geh ke, J., Hooke , G. (2013). Accu a e in elligible models wi h pai wise in e ac ions.
In P oceedings o he 19 h ACM SIGKDD in e na ional con e ence on Knowledge disco e y and da a
mining-KDD ’13 (pp. 623–631). New Yo k, NY.
Malik, M. M., Abdallah, S., & Ala’ aj, M. (2018). Da a mining and p edic i e analy ics applica ions o he
deli e y o heal hca e se ices: A sys ema ic li e a u e e iew. Annals o Ope a ions Resea ch, 270(1–2),
287–312.
Mille , T. (2019). Explana ion in a i icial in elligence: Insigh s om he social sciences. A i icial In elligence,
267, 1–38.
Mo eno, R. P., Me ni z, P. G. H., Almeida, E., Jo dan, B., Baue , P., Campos, R. A., SAPS 3 In es iga o s (2005).
SAPS 3— om e alua ion o he pa ien o e alua ion o he in ensi e ca e uni . Pa 2: De elopmen o a
p ognos ic model o hospi al mo ali y a ICU admission. In ensi e Ca e Medicine,31(10), 1345–1355,
Nai , V., & Hin on, G. E. (2010). Rec i ied linea uni s imp o e es ic ed Bol zmann machines. In P oceedings
o he 27 h in e na ional con e ence on machine lea ning (icml-10) (pp. 807–814).
Obe meye , Z., Powe s, B., Vogeli, C., & Mullaina han, S. (2019). Dissec ing acial bias in an algo i hm used
o manage he heal h o popula ions. Science, 366(6464), 447–453.
Pa liamen and Council o he Eu opean Union (2016). Regula ion (EU) 2016/679 o he Eu opean pa liamen
and o he council o 27 Ap il 2016 on he p o ec ion o na u al pe sons wi h ega d o he p ocessing o
pe sonal da a and on he ee mo emen o such da a, and epealing di ec i e 95/46/ec (Gene al Da a
P o ec ion Regula ion).
Pa liamen and Council o he Eu opean Union (2021). P oposal o a egula ion o he Eu opean pa liamen
and o he council laying down ha monised ules on a i icial in elligence (A i icial In elligence Ac )
and amending ce ain union legisla i e ac s.
Pudil, P., No o iˇco á, J., & Ki le , J. (1994). Floa ing sea ch me hods in ea u e selec ion. Pa e n Recogni ion
Le e s, 15(11), 1119–1125.
Pu usho ham, S., Meng, C., Che, Z., & Liu, Y. (2018). Benchma king deep lea ning models on la ge heal hca e
da ase s. Jou nal o Biomedical In o ma ics, 83, 112–134.
Rajpu ka , P., Chen, E., Bane jee, O., & Topol, E. J. (2022). AI in heal h and medicine. Na u e Medicine,
28(1), 31–38.
Richens, J. G., Lee, C. M., & Joh i, S. (2020). Imp o ing he accu acy o medical diagnosis wi h causal
machine lea ning. Na u e Communica ions, 11(1), 3923.
Ronca olo, F., Boi in, A., & Denis, J- L., Hébe , R., Lehoux, P. (2017). Wha do we know abou he needs
and challenges o heal h sys ems? A scoping e iew o he in e na ional li e a u e. BMC Heal h Se ices
Resea ch,17, 1–18.
Rudin, C. (2019). S op explaining black box machine lea ning models o high s akes decisions and use
in e p e able models ins ead. Na u e Machine In elligence, 1(5), 206–215.
Saada mand, S., Salimi a d, K., Mohammadi, R., Kuipe , A., Ma zban, M., Fa hadi, A. (2022). Using machine
lea ning in p edic ion o icu admission, mo ali y, and leng h o s ay in he ea ly s age o admission o
co id-19 pa ien s. Annals o Ope a ions Resea ch, pp. 1–29.
Si a aman, V., Bukowski, L.A., Le in, J., Kahn, J.M., Pe e , A. (2023). Igno e, us , o nego ia e: Unde -
s anding clinician accep ance o ai-based ea men ecommenda ions in heal h ca e. In P oceedings o
he 2023 chi con e ence on human ac o s in compu ing sys ems (pp. 1–18).
S ekho en, D. J., & Bühlmann, P. (2012). Miss o es -non-pa ame ic missing alue impu a ion o mixed- ype
da a. Bioin o ma ics, 28(1), 112–118.
S iglic, G., Kocbek, P., Fijacko, N., Zi nik, M., Ve be , K., & Cila , L. (2020). In e p e abili y o machine
lea ning-based p edic ion models in heal hca e. WIREs Da a Mining and Knowledge Disco e y, 10(5),
e1379.
Teasdale, G., & Jenne , B. (1974). Assessmen o coma and impai ed consciousness: A p ac ical scale. The
Lance , 304(7872), 81–84.
Topuz, K., Une , H., Oz ekin, A., & Yildi im, M. B. (2018). P edic ing pedia ic clinic no-shows: A decision
analy ic amewo k using elas ic ne and Bayesian belie ne wo k. Annals o Ope a ions Resea ch, 263(1),
479–499.
123
1132 Annals o Ope a ions Resea ch (2025) 347:1093–1132
Vyas, D. A., Eisens ein, L. G., & Jones, D. S. (2020). Hidden in plain sigh — econside ing he use o ace
co ec ion in clinical algo i hms. New England Jou nal o Medicine, 383(9), 874–882.
Wang, S., McDe mo , M.B.A., Chauhan, G., Ghassemi, M., Hughes, M.C., Naumann, T. (2020). MIMIC-
ex ac : A da a ex ac ion, p ep ocessing, and ep esen a ion pipeline o MIMIC-III. In P oceedings o
he ACM con e ence on heal h, in e ence, and lea ning (pp. 222–235). New Yo k, NY.
Wilson, P. W., D’Agos ino, R. B., Le y, D., Belange , A. M., Silbe sha z, H., & Kannel, W. B. (1998). P edic ion
o co ona y hea disease using isk ac o ca ego ies. Ci cula ion, 97(18), 1837–1847.
Yang, C. C. (2022). Explainable a i icial in elligence o p edic i e modeling in heal hca e. Jou nal o Heal h-
ca e In o ma ics Resea ch, 6(2), 228–239.
Yang, Z., Zhang, A., & Sudjian o, A. (2021). GAMI-ne : An explainable neu al ne wo k based on gene alized
addi i e models wi h s uc u ed in e ac ions. Pa e n Recogni ion, 120, 180–192.
Yu, L., & Liu, H. (2004). E icien ea u e selec ion ia analysis o ele ance and edundancy. The Jou nal o
Machine Lea ning Resea ch, 5, 1205–1224.
Zschech, P., Weinzie l, S., Hambaue , N., Zilke , S., K aus, M. (2022). GAM(e) change o no ? An e alua ion
o in e p e able machine lea ning models based on addi i e model cons ain s. In P oceedings o he 30 h
Eu opean con e ence on in o ma ion sys ems (ECIS). Timisoa a, Romania.
Publishe ’s No e Sp inge Na u e emains neu al wi h ega d o ju isdic ional claims in published maps and
ins i u ional a ilia ions.
123