Käppel, Ma in; Acke mann, La s; Jablonski, S e an; Hä l, Simon
A icle — Published Ve sion
Explaining ans o me -based nex ac i i y p edic ion by
using a en ion sco es
P ocess Science
P o ided in Coope a ion wi h:
Sp inge Na u e
Sugges ed Ci a ion: Käppel, Ma in; Acke mann, La s; Jablonski, S e an; Hä l, Simon (2025) :
Explaining ans o me -based nex ac i i y p edic ion by using a en ion sco es, P ocess Science,
ISSN 2948-2178, Sp inge In e na ional Publishing, Cham, Vol. 2, Iss. 1,
h ps://doi.o g/10.1007/s44311-025-00018-4
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/323686
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h p://c ea i ecommons.o g/licenses/by/4.0/
Open Access
© The Au ho (s) 2025. Open Access This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional License, which pe mi s
use, sha ing, adap a ion, dis ibu ion and ep oduc ion in any medium o o ma , as long as you gi e app op ia e c edi o he o iginal
au ho (s) and he sou ce, p o ide a link o he C ea i e Commons licence, and indica e i changes we e made. The images o o he hi d
pa y ma e ial in his a icle a e included in he a icle’s C ea i e Commons licence, unless indica ed o he wise in a c edi line o he ma e-
ial. I ma e ial is no included in he a icle’s C ea i e Commons licence and you in ended use is no pe mi ed by s a u o y egula ion o
exceeds he pe mi ed use, you will need o ob ain pe mission di ec ly om he copy igh holde . To iew a copy o his licence, isi h p://
c ea i eco mmons. o g/ licen ses/ by/4. 0/.
RESEARCH
Käppele al. P ocess Science (2025) 2:11
h ps://doi.o g/10.1007/s44311-025-00018-4
P ocess Science
Explaining ans o me -based nex ac i i y
p edic ion byusing a en ion sco es
Ma in Käppel1*, La s Acke mann2, S e an Jablonski3 and Simon Hä l3
Abs ac
P edic i e business p ocess moni o ing aims o enhance p ocess execu ion by p o id-
ing eal- ime p edic ions abou he u u e e olu ion o a p ocess ins ance. In ecen
yea s, se e al deep lea ning app oaches ha e been es ablished as s a e o he a
o a ious p edic i e asks, including hose based on he ans o me a chi ec u e.
The ans o me a chi ec u e is equipped wi h a powe ul a en ion mechanism
ha assigns a en ion-based impo ance sco es o each inpu elemen , guiding
he model o ocus on he mos ele an pa s o he sequence, ega dless o hei
posi ion. This capabili y leads o mo e accu a e and con ex ually g ounded p edic ions.
Howe e , like mos deep lea ning models, ans o me s la gely ope a e as a black box,
making i challenging o ace how speci ic ea u es in luence he model’s p edic ions.
In his pape , we conduc a se ies o expe imen s o examine he ole o a en ion
sco es in a ans o me -based nex ac i i y p edic ion model. Speci ically, we in es i-
ga e whe he hese sco es p o ide meaning ul explana ions o he model’s decisions.
Ou indings e eal ha a en ion sco es can indeed se e as e ec i e explana ions.
Building on hese insigh s, we p opose wo no el, global, g aph-based explana ion
app oaches ha illus a e he model’s unde s anding o he p ocess’s con ol low. Ou
e alua ion using a ious me ics on bo h eal-wo ld and syn he ic e en logs demon-
s a es ha hese explaine s e ec i ely cap u e he model’s decision-making p ocess.
By imp o ing in e p e abili y, hese insigh s no only enhance p ocess pa icipan s’
con idence in p edic i e models bu also o e a aluable ounda ion o e ining
model pe o mance. Fu he mo e, ou in es iga ion in o he eliabili y o a en ion
sco es o e s aluable insigh s in o how ans o me models encapsula e empo al
and sequen ial dependencies in p edic ion asks.
Keywo ds: P edic i e p ocess moni o ing, T ans o me , A en ion mechanism,
Explainabili y
In oduc ion
In ecen yea s, p edic i e business p ocess moni o ing(Maggi e al. 2014; G igo i e al.
2004) has expe ienced ema kable g ow h, d i en by ad ances in a i icial in elligence
(Weinzie l e al. 2024), and has es ablished i sel as a sub ield in p ocess mining (Di
F ancescoma ino and Ghidini 2022). In con as o pos -mo em analyses o e en da a,
which ocus on pas and cu en e en s, p edic i e business p ocess moni o ing p o ides
un ime suppo o he execu ion o a business p ocess by making a ious p edic ions
*Co espondence:
ma [email p o ec ed]
1 Chai o Digi al Indus ial
Se ice Sys ems, F ied ich-
Alexande -Uni e si y E langen-
Nu embe g, Fü he S . 248,
90429 Nu embe g, Ge many
2 Depa men o Compu e
Science, Ho Uni e si y
o Applied Sciences,
Al ed-Goppel-Pla z 1, 95028 Ho ,
Ge many
3 Ins i u e o Compu e
Science, Uni e si y o Bay eu h,
Uni e si ä ss aße 30,
95447 Bay eu h, Ge many
Page 2 o 42
Käppele al. P ocess Science (2025) 2:11
abou he u u e e olu ion o he p ocess ins ance (Di F ancescoma ino and Ghidini
2022). This includes, among o he s, beha io - ela ed p edic ions (e.g., nex ac i i y (Pas-
quadibisceglie e al. 2019; Cama go e al. 2019; E e mann e al. 2017)), ou come- ela ed
p edic ions (e.g., ou come (K a sch e al. 2021)), and ime- ela ed p edic ions (e.g.,
emaining ime (Ve enich e al. 2019)).
F om a business pe spec i e, ea ly knowledge abou he u u e o a p ocess ins ance
o e s signi ican ad an ages, as i pe mi s imp o ed esou ce and ime planning, be e
p epa a ion o upcoming s eps, and ea ly iden i ica ion o po en ial p oblems (Má quez-
Chamo o e al. 2017; Maggi e al. 2014). The la e , o example, allows p ocess pa -
icipan s o ake imely co ec i e ac ions o mi iga e isks (Di F ancescoma ino and
Ghidini 2022).
Typically, p edic i e business p ocess moni o ing app oaches in ol e cons uc ing
p edic i e models based on his o ical e en log da a cap u ed by in o ma ion sys ems
(G igo i e al. 2004). These models a e hen applied o ongoing p ocess ins ances o
gene a e aluable p edic i e insigh s (Maggi e al. 2014). In ecen yea s, deep lea ning
app oaches ha e been es ablished as s a e o he a o p edic i e asks o mainly wo
easons: Fi s , hey ha e p o en o ou pe o m adi ional machine lea ning models like
decision ees o suppo ec o machines o di e en a ge s (e.g., nex ac i i y (Meh-
diye e al. 2020), ou come (K a sch e al. 2021), emaining ime (Ve enich e al. 2019))
in e ms o accu acy and ea liness o p edic ion. Second, hey a e comple ely da a-d i en
so ha hey no longe equi e an explici ep esen a ion o he unde lying p ocess model
(Sende o ich e al. 2019). As a consequence, a ious kinds o deep lea ning a chi ec u es
ha e been employed, among o he s Con olu ional Neu al Ne wo ks (Pasquadibisceglie
e al. 2019), Long Sho Te m Memo y Neu al Ne wo ks (LSTM) (E e mann e al. 2017;
Cama go e al. 2019), o mo e ecen ly ans o me a chi ec u es (Bukhsh e al. 2021).
Despi e hei s ong p edic i e pe o mance, deep lea ning models la gely ope a e
as black boxes, limi ing insigh s in o hei easoning and decision-making p ocesses
(Nau a e al. 2023). This lack o anspa ency is a signi ican ac o in why people dis-
us hese models and, hence, ma ks a majo obs acle o hei usage in p ac ice (Ca -
alho e al. 2019). The ield o explainable a i icial in elligence (XAI) aims o add ess
his issue by de eloping echniques (so-called explaine s) ha explain he decisions o
machine lea ning models (Ca alho e al. 2019; Nau a e al. 2023). In p edic i e business
p ocess moni o ing bo h ask-agnos ic explana ion echniques and echniques ailo ed
owa ds he speci ic needs o p edic i e business p ocess moni o ing a e applied (Wein-
zie l e al. 2020). In gene al, explaine s a e ca ego ized as ei he local o global based
on hei explana ion scope. While local explaine s aim o explain he p edic ion o a
single p ocess ins ance, global explaine s seek o ind an explana ion o unco e how a
model makes decisions ac oss a collec ion o ins ances (e.g., an e en log). Hence, global
app oaches p o ide insigh s in o how a model makes decisions in gene al, whe eas local
app oaches ocus on explaining single p edic ions. P e ious esea ch in p edic i e busi-
ness p ocess moni o ing has p edominan ly concen a ed on de eloping local explain-
e s, so he de elopmen o p ocess-speci ic global explaine s is s ill in i s in ancy.
Hence, he hypo hesis p oposed in he landma k wo k o E e mann e al. (2017) – ha
deep lea ning models o nex ac i i y p edic ion inhe en ly lea n he unde lying p o-
cess s uc u e – emains la gely un es ed. Peepe ko n e al. in es iga ed his hypo hesis
Page 3 o 42
Käppele al. P ocess Science (2025) 2:11
speci ically o LSTMs, inding ha hese models some imes “s uggle o lea n p ocess
model s uc u e”(Peepe ko n e al. 2023). Ne e heless, examining his hypo hesis o
o he deep lea ning a chi ec u es emains a aluable esea ch di ec ion (Peepe ko n
e al. 2023), especially gi en ha LSTMs o en ace challenges wi h long sequences, as
hey end o pay less a en ion o elemen s appea ing ea lie in he sequence. The ans-
o me a chi ec u e (Vaswani e al. 2017) add esses his issue wi h a powe ul a en ion
mechanism, which assigns impo ance sco es (so-called a en ion sco es) o each ele-
men o he inpu sequence, guiding he model o ocus on he mos ele an pa s o
he sequence, ega dless o hei posi ion (Bukhsh e al. 2021; Vaswani e al. 2017). This
a chi ec u e has led o new s a e-o - he-a app oaches ac oss mul iple esea ch ields
(Vaswani e al. 2017) (e.g., la ge language models such as GPT-4 (OpenAI e al. 2024)
and BERT (De lin e al. 2019)).
As his a chi ec u e also demons a es s ong pe o mance in p edic i e business p o-
cess moni o ing by e ec i ely p edic ing he nex ac i i y (Bukhsh e al. 2021), his pape
explo es he ollowing esea ch ques ion: Can he a en ion sco es wi hin he ans o me
a chi ec u e p o ide insigh s in o whe he he ained p edic ion model has de eloped an
unde s anding o a p ocess’s con ol low?
This a icle is an ex ended and e ised e sion o a p e iously published con e ence
pape (Käppel e al. 2024). In Käppel e al. (2024), we conduc ed ini ial expe imen s o
examine whe he he a o emen ioned a en ion sco es could se e as a solid basis o
de eloping XAI app oaches. We also p oposed wo no el, global, ans o me -speci ic
explana ion app oaches and conduc ed a quan i a i e e alua ion using eal-wo ld e en
logs. This ex ended e sion expands on ou p e ious wo k by p esen ing addi ional
expe imen s o gain deepe insigh s in o he eliabili y o a en ion sco es. Fu he mo e,
we ex end ou e alua ion o include syn he ically gene a ed e en logs om p ocess
models, enabling a di ec compa ison be ween ex ac ed explana ion ules and co e-
sponding p ocess models. We also p esen an embodimen o his app oach as a so -
wa e ool and conduc a quali a i e analysis o he ex ac ed explana ions.
The es o he pape is o ganized as ollows: Backg oundsec ion in oduces basic e -
minology and he undamen als o he ans o me a chi ec u e. A e posi ioning ou
wo k wi h espec o ela ed esea ch (Rela ed wo ksec ion), we conduc a se ies o
expe imen s o in es iga e whe he a en ion sco es can se e as an explana ion (P e-
s udy: he ele ance o a en ion sco essec ion). Building on hese indings, we p esen
he p oposed global explaine s (Explana ion app oachessec ion). In E alua ionsec ion,
we pe o m a quali a i e and quan i a i e e alua ion o ou app oach on bo h eal-wo ld
e en logs and syn he ically gene a ed e en logs. Concluding ema kssec ion discusses
po en ial limi a ions and implica ions o heo y and p ac ice, while Conclusion and
u u e wo ksec ion p o ides di ec ions o u u e esea ch.
Backg ound
In his sec ion, we i s in oduce key concep s and no a ions om he ield o p ocess
mining ha a e essen ial o unde s anding he emainde o he pape . Nex , we explain
he unc ionali y o he ans o me a chi ec u e, wi h a s ong ocus on i s a en ion
mechanism, which plays a pi o al ole in he p oposed explana ion app oaches.
Page 4 o 42
Käppele al. P ocess Science (2025) 2:11
E en logs andnex ac i i y p edic ion
A business p ocess is a sequence o ac i i ies and decisions ca ied ou o deli e a
aluable ou come o he cus ome (Dumas e al. 2018). Each execu ion o such a busi-
ness p ocess is called a p ocess ins ance o a case( an de Aals 2016)). Mode n IT
sys ems eco d and s o e in o ma ion abou p ocess execu ions in he o m o e en
logs – se s o imes amped e en s ha include a ious e en a ibu es encapsula ing
in o ma ion abou he execu ion o ac i i ies ( an de Aals 2016)). In he ollowing,
we deno e he se o ac i i ies o a business p ocess as
A
and de ine an e en o mally
as ollows:
De ini ion 1 An e en is as a uple
e=(a,c, ,d1, ..., dm)
, whe e
a∈A
is he execu ed
ac i i y, c is a case iden i ie indica ing he p ocess ins ance o which he e en belongs,
is he imes amp o execu ion, and
d1, ..., dm
ep esen he da a payload, i.e., op ional
e en a ibu es ela ed wi h he execu ion o ac i i y a.
Thus, a a minimum, an e en con ains he ollowing e en a ibu es: a case iden i ie ,
he execu ed ac i i y, and he imes amp o execu ion ( an de Aals 2016)). Acco dingly,
we use unc ions
πa(e)
,
πc(e)
,
π (e)
, and
πd1(e),...,πdm(e)
o access he ac i i y, case
iden i ie , imes amp, and he da a payload o an e en e( an de Aals 2016)).
All e en s belonging o he same p ocess ins ance can be empo ally o de ed by
hei imes amp in o a so-called ace( an de Aals 2016)):
De ini ion 2 A ace is a non-emp y, ini e sequence o e en s
σ=�e1,...,en�
such
ha , o
1≤i<j≤n
, he ollowing condi ions hold:
• all e en s a e o de ed acco ding o hei imes amp (i.e.,
π (ej)≥π (ei)
) and
• all e en s belong o he same p ocess ins ance (i.e.,
πc(ej)=πc(ei)
).
The leng h o a ace, deno ed by
|σ|
, e e s o he numbe o e en s wi hin ha ace.
Based on his de ini ion, we can de ine an e en log as ollows ( an de Aals 2016)):
De ini ion 3 An e en log L is a se o aces o he same business p ocess. The size o
he e en log, deno ed by
|L|
, is de ined as he numbe o aces con ained in L.
To ep esen p ocess ins ances a di e en poin s in ime, we u ilize he p e ixes o
a ace.
De ini ion 4 Le
σ=�e1,...,en�
be a ace and
∈{1, ...,n−1}
. The p e ix o a
ace
σ
o leng h is de ined as a unc ion hd ha e u ns he i s e en s o
σ
, i.e.,
hd(σ , )=�e1,...,e �
.
Each p e ix hus consis s o a sequence o consecu i e e en s om he s a o he
ace up o a speci ic poin , cap u ing he p ocess execu ion up o ha momen .
A nex ac i i y p edic ion model ecei es a eco d o a unning p ocess ins ance
(i.e., a p e ix) as inpu and p edic s he mos likely subsequen ac i i y. We he e o e
de ine nex ac i i y p edic ion o mally as a unc ion (Rama-Manei o e al. 2021):
Page 5 o 42
Käppele al. P ocess Science (2025) 2:11
De ini ion 5 Le
σ=�e1,...,e �
be a p e ix o leng h . Nex ac i i y p edic ion is
de ined as a unc ion
ha p edic s o he ac i i y o he nex e en
e +1
, which is no
ye known.
Thus, a nex ac i i y p edic ion app oach aims o ain a p edic ion model ha app ox-
ima es his unc ion
using a gi en e en log. Mos p edic i e business p ocess moni-
o ing app oaches employ machine lea ning algo i hms o lea n he unc ion
. In his
pape , we do no de ine a new a chi ec u e bu ins ead use he ans o me a chi ec u e
p oposed by Bukhsh e al. (2021). Since his a chi ec u e conside s solely he ac i i y
e en a ibu e and igno es he emaining e en a ibu es o a p e ix, we can – wi hou
loss o gene ali y – ep esen aces and p e ixes simply as sequences o ac i i ies. Thus,
we deno e a ace o a p e ix as ollows:
σ=�πa(e1),...,πa(en)�
.
T ans o me a chi ec u e anda en ion mechanism
The ans o me a chi ec u e was o iginally in oduced in Vaswani e al. (2017) as a
sequence- o-sequence model buil on an encode -decode a chi ec u e equipped wi h
a obus a en ion mechanism. Sequence- o-sequence modeling e e s o asks ha con-
e an inpu sequence in o an ou pu sequence o po en ially di e en leng hs, such as
language ansla ion (Zhao e al. 2023). Since i s in oduc ion, he ans o me a chi-
ec u e has also been adap ed o sequence- o- ec o asks, whe e an inpu sequence
is p ocessed o ou pu a ixed-size ec o . Such adap a ions a e pa icula ly ele an o
classi ica ion o eg ession asks, including nex ac i i y p edic ion o emaining ime
p edic ion (Bukhsh e al. 2021). This adap a ion is achie ed by omi ing he decode
pa o he a chi ec u e, which is why his a ian is also called an encode -only ans-
o me (De lin e al. 2019).
In his wo k, we ocus on desc ibing his modi ied ans o me a chi ec u e as applied
o nex ac i i y p edic ion. Speci ically, we employ he p ocess ans o me a chi ec u e
ou lined in Bukhsh e al. (2021) (see Fig.1). The ollowing sec ions desc ibe he key com-
ponen s o he a chi ec u e ele an o his s udy: he inpu , he a en ion mechanism,
and he ou pu o he ans o me . Fo de ails on o he aspec s o he a chi ec u e, we
e e he eade o Vaswani e al. (2017) and Bukhsh e al. (2021).
Inpu The ans o me ecei es a sequence o ac i i ies
σ=(a1,...,an)
as inpu . This
inpu is hen embedded in o a high-dimensional space o dimension
dm∈N>0
1 by con-
e ing each elemen in he sequence in o a so-called embedding ec o (Vaswani e al.
2017). As a esul , o each elemen
ai
o he sequence, wo embedding ec o s o dimen-
sion
dm
a e gene a ed: a ec o ep esen ing he elemen i sel (so-called inpu embed-
ding)
xi
in
∈R
1×dm
and a ec o ep esen ing he posi ion (so-called posi ion embed-
ding)
xi
pos
∈R
1×dm
. Following common p ac ice in ans o me implemen a ions, hese
embedding ec o s a e ep esen ed as ow ec o s.
Because he ans o me p ocesses he en i e inpu sequence in pa allel and does no
inhe en ly cap u e posi ional in o ma ion, posi ion embeddings a e essen ial o a oid
losing he o de o he elemen s. Bo h ec o s a e hen summed componen -wise,
1 In he a chi ec u e used in his pape
dm
is se o 36 (Bukhsh e al. 2021)
Page 6 o 42
Käppele al. P ocess Science (2025) 2:11
esul ing in a con inuous ep esen a ion o he inpu elemen ha cap u es posi ional,
seman ic, and syn ac ic p ope ies (Vaswani e al. 2017). This esul ing ec o , deno ed
as
xi∈
R
1×dm
, se es as inpu o he ans o me block (Vaswani e al. 2017). No a-
bly, hese embeddings a e au oma ically lea ned du ing aining, allowing he model o
dynamically adap o he seman ic and posi ional cha ac e is ics o he inpu da a.
T ans o me block The hea o he ans o me a chi ec u e, and cen al o i s capa-
bili ies, is he mul i-head sel -a en ion mechanism. This specialized o m o a en ion
signi ican ly enhances he model’s abili y o cap u e ela ionships wi hin sequences. In
gene al, an a en ion mechanism allows deep lea ning models o dynamically ocus on
di e en pa s o an inpu sequence when gene a ing ou pu . The co e idea behind a en-
ion is o assign a ying sco es o di e en elemen s in he inpu , indica ing how much
ocus he model should pay hem when pe o ming a ask. Hence, i allows o p io i ize
mos ele an in o ma ion leading o mo e accu a e and con ex ual ou pu . A en ion
mechanism can be in eg a ed in o a ious deep lea ning a chi ec u es. In he ans-
o me a chi ec u e in oduced in Vaswani e al. (2017), a e ined e sion o he a en ion
mechanism, known as sel -a en ion is p oposed. This mechanism enables he model o
a end o all elemen s wi hin he inpu sequence simul aneously, enhancing i s abili y o
cap u e bo h local and global dependencies.
Sel -a en ion mechanism: In he sel -a en ion mechanism, he embedding ec o
xi
is u ilized in h ee dis inc ways: as a que y ec o
qi
, a key ec o
ki
, and a alue ec o
i
.
Fig. 1 O e iew abou he ans o me a chi ec u e p oposed in Bukhsh e al. (2021)
Page 7 o 42
Käppele al. P ocess Science (2025) 2:11
These ec o s a e ob ained by mul iplying he embedding ec o
xi
wi h he p ojec ions
ma ices
WQ
∈R
d
m
×d
k ,
WK∈
R
d
m
×dk
, and
WV∈
R
d
m
×d
:
Because
WQ
,
WK
, and
WV
ans o m
xi
in a lowe -dimensional space, hey a e e e ed
o as p ojec ions. These ma ices a e au oma ically lea ned du ing aining. Acco ding o
Vaswani e al. (2017)
dk
and
d
a e ypically se o he same alue. In o de o enhance
compu a ional e iciency, hese ec o s a e packed ow-wise in o ma ices
Q∈
R
n×dk
,
K∈Rn×d
k , and
V∈Rn×d
(Vaswani e al. 2017).
We hen employ Q, K, and V o calcula e an a en ion sco e ha e lec s he ela i e
impo ance o each elemen (que y) in ela ion o he o he s in he inpu sequence. Spe-
ci ically, each elemen in he inpu sequence is compa ed o he cu en ly conside ed ele-
men by compu ing he ma ix p oduc
QKT∈
R
n×n
. The alues o he esul ing
n×n
ma ix a e scaled by di iding by d
k
and hen no malized by applying a ow-wise so -
max ope a ion:
This no maliza ion mi iga es he isk o anishing g adien s and imp o es aining e i-
ciency (Vaswani e al. 2017). The esul ing ma ix
Mσ
is called he a en ion sco e ma ix
o simply he a en ion ma ix2. No ably, he dimension o he a en ion sco e ma ix
depends on he leng h o he inpu sequence, esul ing in a en ion sco e ma ices o
a ying sizes o inpu sequences o di e en leng hs.
Finally, each alue ec o
i
(con ained in V) is mul iplied wi h he a en ion sco es:
This s ep is in ended o educe he impac o alue ec o s ha ecei e e y low a en-
ion sco es. The esul ing ma ix
A σ
is he ou pu o he sel -a en ion mechanism.
Mul i-Head A en ion: To cap u e di e se ela ionships and pa e ns be ween he
elemen s in he inpu sequence, we use h independen , pa allel a en ion mechanisms,
e e ed o as a en ion heads o heads. These heads a e hen combined in o a so-called
mul i-head a en ion(Vaswani e al. 2017):
wi h
headj
σ=A j
σ
.
In his o mula, he a en ion sco e ma ices om each a en ion head j a e conca e-
na ed (i.e., he ma ices a e a anged side by side) esul ing in a
n×hd
ma ix. This con-
ca ena ed ma ix is hen mul iplied by a lea ned weigh ma ix
WO∈
R
hd
×
dm
o p ojec
he esul back o he ini ial dimension
dm
. In he esul o he mul i-head a en ion, each
q
i=xi·WQ∈R
1×dk
k
i=xi·WK∈R1×d
k
i=xi·WV∈
R1×d
M
σ=so max
QKT
d
k.
A σ=Mσ·V∈
R
n×d
.
Mul iHeadA en ionσ
=conca (head
1
σ
,...,head
h
σ
)W
O
∈R
n×dm
2 Please no e ha
QKT
is o en e e ed o as a en ion sco e ma ix in he li e a u e.
Page 8 o 42
Käppele al. P ocess Science (2025) 2:11
ow ep esen s an en iched con ex ec o o an elemen o he inpu sequence, while
each column ep esen s a speci ic ea u e ac oss all elemen s.
To ensu e ha each head does no equi e he ull dimension
dm
, he dimensions
dk
and
d
a e se o
d
m
h
. This adjus men keeps he o al compu a ional cos compa able o
ha o a single-head a en ion mechanism. The p ocess ans o me p esen ed in Bukhsh
e al. (2021) employs ou a en ion heads, esul ing in
dk=d =9
.
The co e idea behind he mul i-head a en ion mechanism is ha each a en ion head
can po en ially lea n di e en ela ionships o ypes o dependencies be ween he ele-
men s o he inpu sequence. Because each head ope a es independen ly wi h i s own
weigh ma ices, i can ocus on dis inc pa e ns and dependencies. By calcula ing
mul iple a en ion heads in pa allel, he model can cap u e mo e di e se and complex
ela ionships wi hin he sequence han i would be possible wi h a single a en ion
mechanism.
In e p e a ion o he a en ion sco e ma ices: Fig.2 depic s an example o he
a en ion sco e ma ices o all heads ob ained o a p e ix o a eal-wo ld e en log.
In e p e ing hese a en ion sco e ma ices is essen ial o unde s anding he explana-
ion app oaches and is he e o e desc ibed in de ail. All a en ion sco e ma ices can be
in e p e ed iden ically, ega dless o he head. Each a en ion sco e ma ix illus a es
how he elemen s in he inpu sequence ela e o one ano he . In hese ma ices, ows
and columns co espond o elemen s o he inpu sequence. While he ows ep esen
hese elemen s as que ies, he columns ep esen hem as keys. By examining a ow, we
can assess he impo ance he ans o me model assigns o each o he elemen in he
inpu sequence om he pe spec i e o he elemen ep esen ed by ha ow. In con as ,
inspec ing a column e eals he le el o a en ion o he elemen s in he inpu sequence
assign o he key elemen associa ed wi h ha column. The en ies o he ma ix indica e
Fig. 2 A en ion sco es o all heads o he p e ix
A_SUB, A_PAR, W_A h, W_A h, A_PRE, W_Com
o he
BPIC12 e en log
Page 15 o 42
Käppele al. P ocess Science (2025) 2:11
Be o e calcula ing he common ea u e impo ance, we ex ac a di ec ly ollows g aph
(DFG) om he gi en e en log. The DFG is hen used o selec ac i i ies ha can be
used o eplace ac i i ies in he aces in a ealis ic way. Realis ic means ha i is likely
ha he modi ied ace ep esen s a alid p ocess execu ion.
Calcula ion o common ea u e impo ance: Fo calcula ing he common ea u e
impo ance
com
o a ace
σ
, we p oceed as ollows: We i s c ea e a collec ion o mod-
i ied e sions o he o iginal ace
σ
. The e o e, we i e a e o e all e en s in
σ
o gene a e
o each e en a se o modi ied aces in which he pa icula e en
ei
has ei he been
masked ou o i s associa ed ac i i y has been eplaced by ano he ac i i y (o sequence
o ac i i ies). In he case o a eplacemen , we selec he eplacemen ac i i y in depend-
ency o he posi ion o
ei
wi h he help o he DFG:
• I
ei
is he i s e en o
σ
, we iden i y all possible s a ac i i ies o he p ocess in
he DFG. Then we eplace
ei
wi h each possible s a ac i i y o ge se e al modi ied
aces.
• I
ei
is he las e en in he ace, we eplace i s ac i i y wi h all ac i i ies ha can suc-
ceed he p eceding ac i i y
πa(ei−1)
acco ding o he DFG.
• Fo any o he posi ion (i.e, i
ei
is nei he he i s no he las e en ), we subs i u e
ei
wi h all possible pa hs in he DFG ha connec he ac i i y o he p eceding e en ,
πa(ei−1)
, o he ac i i y o he subsequen e en ,
πa(ei+1)
.
The union
LM
o all modi ied e sions o
σ
ob ained ac oss all a ia ions can hen be
used o compu e common ea u e impo ance alues pe ac i i y. The e o e, we ini ially
se he common ea u e impo ance o each ac i i y o ze o. Fo each modi ied ace
σm∈LM
, we i s de e mine he eplaced ac i i y and compu e hen he cosine simila i y
o he p edic ion ec o ob ained o
σm
and he p edic ion ec o o he o iginal ace
σ
. This simila i y alue is hen added o he common ea u e impo ance o he eplaced
ac i i y. I is impo an o no e ha we add his sco e ega dless whe he he p edic ion
o
σ
and
σm
di e s, because we a e in e es ed in in luence o he ac i i y on he p edic-
ion. The la ge he di e ence o he p edic ion ec o , he highe is he impac . A e
e alua ing all modi ied aces, we compu e he mean o all simila i y alues belonging o
he same eplaced ac i i y. Finally, we no malize he mean alues so hey sum up o 1.0 o
ob ain a p obabili y dis ibu ion o e he ac i i ies. Please no e, ha his ea u e impo -
ance is independen o he a en ion sco es and only ela es ea u es and p edic ion.
Calcula ion o a en ion sco e ea u e impo ance: Fo calcula ing he a en ion sco e
ea u e impo ance
a
, we ely on he same collec ion o modi ied aces
LM
as be o e.
I he p edic ion o he o iginal ace and a modi ied ace a e simila (i.e., hei cosine
simila i y is lowe han
δsim
), we de e mine he a en ion sco es pe ac i i y om he
ans o me s’ a en ion sco e ma ices. A e inspec ing all modi ied e sions, we com-
pu e he mean o he cumula ed a en ion sco es pe ac i i y. I is c ucial ha we only
conside modi ied aces wi h simila p edic ions o he o iginal ace, because we wan
o iden i y, which ac i i ies a e impo an o a pa icula p edic ion.
Finally, we calcula e Kendall’s au ank co ela ion
τ
(Kendall 1938) be ween common
ea u e impo ance and a en ion sco e ea u e impo ance. Conce ning in e p e abili y
Kendall’s au is scaled in o he con inuous ange
[−1, 1]
, whe e
−1
indica es a pe ec
Page 16 o 42
Käppele al. P ocess Science (2025) 2:11
nega i e co ela ion, 0 indica es no co ela ion, and
+1
indica es a pe ec posi i e co -
ela ion. Hence, a low co ela ion means, ha he ela i e impo ance o he ea u es is
no su icien ly e lec ed in he a en ion sco es. Thus, he explana ions de i ed om he
a en ion sco es would no use he same easoning as he ans o me o be explained.
Hence, o eliable a en ion sco es la ge alues a e p e e able.
The esul s o he co ela ion o common and a en ion sco e ea u e impo ance a e
shown in Fig.3. Fo he majo i y o he e en logs he median co ela ion is sligh ly la ge
han 0.5. Following he guidelines p oposed in Akoğlu (2018) a e which alues
τ<0.3
,
0.3 ≤τ≤0.5
,
τ>0.5
indica e a weak, mode a e, and s ong co ela ion espec i ely,
we obse e a mode a e o s ong associa ion. Ne e heless, o some o he e en logs,
he co ela ion is close o ze o, indica ing no co ela ion. The la e applies o BPIC12,
BPIC12-WC, and he Sepsis e en log. In summa y, he co ela ion be ween he com-
mon ea u e impo ance and he a en ion sco e ea u e impo ance is o en signi ican ,
gi ing a i s indica ion o eliable a en ion sco es.
Expe imen 2 – A en ion Mechanism Pa ame e Manipula ion The nex expe imen
examines he necessi y o he a en ion mechanism, adhe ing o he expe imen al se up
ou lined in Wieg e e and Pin e (2019). To his end, we i s ain exac ly one baseline
model
Mb
wi hou any modi ica ions in he aining p ocess. Then we ain mul iple
manipula ed ans o me models whe e we ei he andomly ini ialize he a en ion
pa ame e s (i.e., ma ices
WQ
,
WK
,
WV
, and
WO
) o eeze he a en ion pa ame e o
uni o m alues du ing aining. The la e means ha he a en ion sco e mechanism is
comple ely elimina ed, while he i s means ha he aining p ocess p obably ends wi h
di e en a en ion sco es. Bo h modi ied aining p ocedu es a e epea ed i e imes o
mi iga e he in luence o change, each ini ia ed wi h unique andom seeds o ensu e a -
iabili y ac oss he ials.
Fig. 3 Resul s o he Kendall Tau co ela ion o common and a en ion ea u e impo ance
Page 17 o 42
Käppele al. P ocess Science (2025) 2:11
Subsequen ly, each manipula ed ans o me
Mm
(bo h he andomly ini ialized and
he ozen) is compa ed agains he baseline
Mb
. To his end, we send each es sample
h ough all ained models (i.e., he baseline model and he manipula ed ans o me s)
and ex ac each ime he co esponding a en ion sco e dis ibu ion and he p edic-
ion ec o . The a en ion sco e dis ibu ions and p edic ion ec o s de i ed om
Mb
a e hen compa ed o hose ob ained om he co esponding
Mm
using JSD and TVD,
espec i ely. To agg ega e he alues, we compu e he a e age JSD and TVD alues
ac oss all es samples o each model.
The a ionale behind his expe imen al design is o assess whe he a ying a en ion
sco e dis ibu ions (indica ed by highe JSD alues) lead o in a ian p edic ions (indi-
ca ed by lowe TVD alues) (Wieg e e and Pin e 2019; Jain and Wallace 2019). Thus,
i iden ical p edic ions can be achie ed wi h subs an ially di e en a en ion sco es, his
would imply ha he a en ion mechanism ba ely a ec s he decision p ocess o he
model. Hence, he a en ion sco es would ha e only minimal explana o y alue.
To acili a e in e p e a ion, we plo he mean JSD agains he mean TVD achie ed o
each model (see Fig.4). Models posi ioned u he o he igh and lowe in he plo s
ep esen cases wi h less eliable a en ion sco es. No ably, he majo i y o he e en
logs, ega dless o whe he hey a e eal-wo ld o syn he ically gene a ed, a e posi ioned
in egions indica ing eliable a en ion sco es. In cases whe e he models end o be on
he igh side (Complex Model, Looped And, BPIC12, and BPIC12-W) he TVD alues
a e signi ican ly abo e ze o. Fo Sequence and LDD, howe e , TVD alues emain close
o ze o, sugges ing ha p edic ions a e only minimally a ec ed by changes in a en ion
sco es. Due o he simplici y o Sequence and LDD his obse a ion is no su p ising,
since o mos o he es samples he nex ac i i y is ob ious, so ha he model akes
o he op ions ba ely in o accoun . A he same ime, bo h Sequence and LDD a e also
on he le side, indica ing only mino a ia ions in he a en ion sco es. O e all, hese
esul s sugges ha he a en ion sco es can be us ed.
To u he in es iga e he s abili y o a en ion sco es, we analyze he maximum a en-
ion alue in a dis ibu ion, i.e.,
max(ασ)
, in ela ion o JSD. The mo i a ion o his
analysis is he hypo hesis ha high maximum a en ion alues indica e a model’s s ong
ocus on a pa icula inpu elemen . I his ocus is consis en ac oss di e en model ini-
ializa ions, i could sugges ha s ong peaks in he a en ion dis ibu ion align wi h
s able and meaning ul model beha io . Con e sely, i high maximum a en ion alues
a e associa ed wi h high JSD alues, i would imply ha e en seemingly con iden a en-
ion peaks may be un eliable and a y subs an ially ac oss di e en model ins ances. By
examining his ela ionship, we aim o assess whe he s ong a en ion peaks can se e
as obus explana o y indica o s.
Following Jain and Wallace (2019), we conduc a binning p ocedu e in which all a en-
ion sco e dis ibu ions o he samples ega dless o he model a e ca ego ized based on
i s maximum a en ion alue. By combining esul s om mul iple models, he analysis
ensu es ha obse ed pa e ns a e obus and no a i ac s o speci ic model ini ializa-
ions. Speci ically, each a en ion sco e dis ibu ion is assigned o one o ou dis inc
bins, each de ined by a nume ical ange o maximum a en ion alues. Th ough he bin-
ning, we aim o examine how a ying deg ees ( ep esen ed by he anges o he bins) o
selec i e ocus co ela e wi h shi s in a en ion dis ibu ions.
Page 18 o 42
Käppele al. P ocess Science (2025) 2:11
Figu e5 isualize he dis ibu ion o JSD di e gences (compa ed o he base model) as
a unc ion o he maximum a en ion alue ac oss all models in o m o a iolin plo . The
heigh o each iolin is scaled acco ding o he numbe o ins ances in each bin, indica -
ing how o en JSD alues lie in ce ain anges. Thus, a wide a ea in he iolin means ha
many JSD alues a e concen a ed in his a ea. The iolins can be in e p e ed as ollows:
A igh -weigh ed iolin means ha he JSD alues in his bin end o be high. This indi-
ca es ha he a en ion alues o his bin a e suscep ible o manipula ion. In con as ,
a le -weigh ed iolin (wide le , wi h low JSD alues) means ha he JSD alues in his
bin a e p edominan ly low. This indica es ha he a en ion sco es o his bin a e obus
and less manipulable.
The plo s e eal ha mos a en ion sco e dis ibu ions all wi hin ei he he i s o
las bin, wi h minimal ep esen a ion in he second and hi d bins. This pa e n sugges s
Fig. 4 JSD s. TVD plo s. Rec angles = models wi h ozen weigh s, iangles = seeded base models. JSD
can only ake alues be ween 0 and
log(2)=0.693
. The ho izon al line a 0.1 se es as e e ence alue o
compa e he p ocess-speci ic esul s, wi h esul s ob ained on gene al machine lea ning da ase s. The alue
s ems om Wieg e e and Pin e (2019), whe e all TVD alues we e lowe han 0.1
Page 19 o 42
Käppele al. P ocess Science (2025) 2:11
ha he model ends o eac o ce ain elemen s ei he e y s ongly o ha dly a all.
Since he middle bins (second and hi d bins) a e only minimally ep esen ed, he model
a ely shows mode a e a en ion o an elemen .
I a en ion sco e dis ibu ions all in o he las bin (0.00–0.25), he e is o en a igh -
wa d endency. The i s bin (0.75–1.00), on he o he hand, shows a mo e he e ogene-
ous pic u e, wi h some da ase s ending o he le (BPIC13_CP, BPIC12_O, BPIC13-I,
BPIC12_WC, Sequence) and wo da ase s (Complex, Looped And) showing a s ong
igh wa d endency. The emaining logs a e mo e cen alized. O pa icula in e es a e
Fig. 5 All JSD s. maximum alue in he a en ion sco e dis ibu ion iolin plo s (scaled by coun ). The heigh
is scaled by he coun o ins ances pe bin
Page 20 o 42
Käppele al. P ocess Science (2025) 2:11
cases whe e he las bin shows a igh wa d endency while he i s bin shows a le wa d
endency (e.g., BPIC12_W, Sequence, BPIC13_CP). In hese cases, ins ances wi h low
a en ion sco es a e mo e suscep ible o manipula ion han hose wi h highe a en ion
sco es.
O e all, his binning app oach shows ha he in e p e abili y o a en ion sco es can
a y wi hin a da ase and ha pa icula maximum a en ion h esholds may se e as
indica o s o model obus ness. The de ailed analysis also suppo s he ini ial indings
om he i s pa o he expe imen ha he a en ion sco es a e p edominan ly us -
wo hy, sugges ing hey a e sui able o explana o y pu poses.
Expe imen 3 – A en ion Sco e Masking In a hi d expe imen , we in es iga e
how pa icula a en ion alues a ec he p edic ion (see Fig.6). In de ail, we wan o
check whe he he a en ion sco es a e closely linked wi h he in o ma ion in he inpu
sequence. The e o e, we examine whe he emo ing an elemen om he inpu sequence
has he same e ec in e ms o p edic ion as emo ing he elemen ’s a en ion sco es.
To do his, we once mask elemen s in he inpu sequence and once he co esponding
a en ion sco es in he heads’ a en ion sco e ma ices and compa e he model ou pu s
ia TVD. In he inpu sequence, we mask elemen s by eplacing hem wi h a padding
symbol (_), indica ing o he model ha he e is no elemen . In con as , masking he
a en ion sco es is achie ed by se ing all alues in he ele an ows and columns o he
a en ion sco e ma ices o ze o. Figu e6 illus a es his concep ual di e ence be ween
masking he inpu p e ix and he a en ion sco es o he heads’ a en ion sco e ma ices.
We pe o m his p ocedu e o all es samples, whe eby each elemen in he sample
is masked one ime. Fo each o he masked samples, we ob ain wo p edic ion ec-
o s: a p edic ion ec o
pm
o he masked sample and a p edic ion ec o
pam
o he
unmasked sample, whe e we masked he a en ion sco es. We hen compa e
pm
and
pam
by calcula ing hei TVD.
Low TVD alues sugges ha masking an elemen in he inpu sequence has a simila
impac on he p edic ion as masking i wi hin he a en ion sco e ma ices. F om his,
Fig. 6 The o me masks e en s (he e ac i i y B) di ec ly in he p e ix. The la e only masks ows and columns
co esponding o he e en in he a en ion sco e ma ices. P edic ions may be di e en
Page 21 o 42
Käppele al. P ocess Science (2025) 2:11
i can be concluded ha pa icula a en ion sco es ha e ele ance and can se e as an
explana ion. We obse e ha he his og ams o TVD alues (see Fig.7) show a s ong
le skew, indica ing ha masking in he inpu p e ix leads o almos iden ical p edic ions
as masking in he a en ion sco e ma ices. Mino de ia ions om his beha io appea
only in he his og ams o he Sequence and LDD e en log.
Conclusion The o e all esul o his se ies o expe imen s is ha he a en ion sco es
possess good eliabili y. Only e y simple e en logs such as Sequence o LDD (bo h
Fig. 7 Va ia ion be ween p edic ions TVD be ween masked elemen s in he p e ix and only masked
a en ion sco es ma ix
Page 22 o 42
Käppele al. P ocess Science (2025) 2:11
syn he ically gene a ed) de ia e sligh ly om his obse a ion, al hough bo h s ill ha e
su icien us wo hiness. This inding ha he a en ion sco es appea less eliable o
e y simple da ase s is also consis en wi h he esul s in he s udy o Wieg e e and
Pin e (2019), whe e he same obse a ion was made o da ase s om o he domains.
The eason o his anomaly lies in he simplici y o he unde lying ask so ha he
model does no equi e he powe ul a en ion mechanism a all. Based on he esul s
ob ained, i is jus i ied o build explana ion app oaches wi h a en ion sco es as a pi o al
componen .
The e o e, especially he esul s om Expe imen 2 a e bene icial, o ge ing deepe
insigh s how s ong he a en ion sco es a e p onounced. Mos o he ime he a en-
ion sco e dis ibu ions all ei he in he i s bin (maximum a en ion sco e e y high)
o he las bin (maximum a en ion sco e e y low) bu ba ely in he second o hi d
bin. Hence, we ha e ei he a leas a e y high a en ion sco e ( i s bin) o a lo o small
a en ion sco e (las bin). This obse a ion helps us o de ine h esholds, when o con-
side an elemen in he inpu sequence as ele an o no .
Explana ion app oaches
In his sec ion, we in oduce wo global ans o me -speci ic explana ion app oaches.
Each explaine ecei es as inpu a ained ans o me p edic ion model and a se o p e-
ixes L o be explained. Bo h explaine s cons uc an in e p e able di ec ed g aph ha
isualizes he con ol- low o he p ocess, se ing as an indica o o he ex en o which
he p edic ion model cap u es he con ol- low o he p ocess. Because ou p elimina y
s udy in P e-s udy: he ele ance o a en ion sco essec ion p o ides s ong e idence
ha a en ion sco es a e eliable o nex ac i i y p edic ion, ou explana ion app oaches
le e age hem as a c ucial componen .
Explaine 1: backwa d explaine
The i s explaine (so-called Backwa d Explaine ) gene a es explana ions o indi idual
p e ixes and subsequen ly in eg a es hese in o an o e a ching explana ion o all p e-
ixes in L. This p ocedu e is explained in he emainde o his sec ion and in ol es c e-
a ing a local g aph
Gσ
o each p e ix
σ∈L
, which is hen me ged di ec ly in o he global
g aph G.
C ea ing a local g aph An essen ial s ep in cons uc ing a local g aph
Gσ
o a p e ix
σ
is o u ilize he heads’ a en ion sco es o iden i y ele an ac i i ies ha a e decisi e o
he p edic ion
pσ
. To do his we p oceed as ollows: An ac i i y is conside ed ele an o
he p edic ion
pσ
i i s agg ega ed a en ion sco e ac oss mul iple modi ied e sions o
σ
is su icien ly high. To gene a e hese modi ied e sions o
σ
, we apply andomly modi-
ica ion ope a ions. Each modi ied p e ix
σm
is hen passed o he ans o me o ob ain
p edic ion ec o
pσm
and he a en ion sco e ma ices
Mi
σm
o a head i.
To check whe he he p edic ion o
σm
is s ill close o he p edic ion o
σ
, we compu e
he cosine simila i y be ween
pσm
and
pσ
and compa e i o a p ede ined h eshold
δsim
.
Page 23 o 42
Käppele al. P ocess Science (2025) 2:11
I
σm
sa is ies his condi ion, he a en ion sco es o each elemen j in
σm
a e agg ega ed
ac oss he di e en heads as ollows:
In his equa ion a en ion sco e ma ices M
i
σm
om each head a e i s summed com-
ponen -wise, and hen he j h column is summed. This yields an agg ega ed a en ion
sco e ec o
ησm=(S1, ..., S|σm|)
o each modi ica ion
σm
con aining a o al a en ion
sco e o each e en . This p ocedu e is illus a ed in Fig.8.
Nex , he o al a en ion sco es a e u he agg ega ed pe ac i i y o a ec o
ψσ
by
summing up he sco es o all e en s belonging o a ce ain ac i i y. The ec o
ψσ
is
hen no malized o ange wi hin [0, 1]. We deno e he o al a en ion sco e o ac i -
i y a as
ψσ(a)
. Finally, we il e ac i i ies based on a h eshold
δa
, keeping only
ac i i ies wi h
ψσ(a)>δ
a
o ge only hose ac i i ies wi h pa icula ly high o al
a en ion sco es. Thus, he se o ele an ac i i ies o inpu sequence
σ
is de ined as
A ={a∈A|ψσ(a)>δ
a }
. As a de aul alue we se
δa
o 0.1. This il e ing is essen-
ial o exclude i ele an elemen s ha may ha e low bu non-ze o a en ion sco es.
Wi hou his s ep, nea ly all ac i i ies would be classi ied as ele an .
Analogously, we iden i y he mos likely nex ac i i ies, deno ed as
P
, om he p e-
dic ion ec o
pσ
, by selec ing only hose ac i i ies a wi h p edic ion sco es exceeding
an a-p io i de ined h eshold
δp ed
, i.e.,
p
σ
(a)>δ
p ed
. As a de aul alue o
δp ed
we
used 0.1. We do no limi he selec ion o a ixed numbe o ac i i ies om
pσ
, because
depending on he p ocess he con idence o he model a ies: o some p e ixes, i may
indica e ha only one ac i i y is p obable (e.g., in case o a s ic sequence), while in o h-
e s (e.g., a an AND spli ), mul iple ac i i ies appea iable o be execu ed nex . Building
on his we can o mally de ine local g aph as ollows:
De ini ion 8 Le
σ
be an inpu sequence,
A
he se o ele an ac i i ies iden i ied in
σ
, and
P
he se o likely nex ac i i ies o
σ
. A local g aph o
σ
is a uple
Gσ=(V,E)
wi h
•
V=A ∪P
being he se o nodes and
•
E={(s, )|s∈A , ∈P }
being he se o edges.
S
j=
|σ
m
|
n=1
n
i=1
Mi
σm
nj
.
Fig. 8 De e mining agg ega ed a en ion sco es o each e en in he p e ix, assuming ha
M
possesses
wo heads
Page 24 o 42
Käppele al. P ocess Science (2025) 2:11
Thus, o cons uc ing he local g aph
Gσ
, we use all elemen s in
A
and
P
as nodes
and es ablish edges be ween all nodes om
A
o all nodes om
P
.
Le us illus a e he p ocedu e on he example depic ed in Fig.9. Gi en he p e-
ix
σ1=�B,A,C,B,E�
, he agg ega ed a en ion sco es highligh ac i i ies B and C
as ele an o he p edic ion, i.e.,
A ={B,C}
. Addi ionally, he p edic ion ec-
o
pσ1
sugges s ha ei he ac i i y B o D is likely o occu nex , i.e.,
P ={B,D}
.
Acco ding o De ini ion 8 we ge nodes
V=A ∪P ={B,C,D}
and edges
V={(C,B),(B,B),(B,D),(C,D)}
. This g aph can hen be in e p e ed as ollows: o exe-
cu ing ac i i y D, i is c ucial ha ac i i ies B and C ha e been execu ed be o e.
Me ging local g aphs Each local g aph
Gσ
is di ec ly in eg a ed in o he global
g aph G. This in eg a ion in ol es adding he e ices and edges o
Gσ
o G
i hey a e no ye included. To illus a e his s ep, suppose we ha e, in addi-
ion o he local g aph
Gσ1
om ou example in Fig. 9, a second local g aph
Gσ2=({A,C},{(A,C)})
. The esul ing g aph a e me ging
Gσ1
and
Gσ2
would hen be
G=({A,B,C,D},{(C,B),(B,B),(B,D),(C,D),(A,C)})
.
A e me ging a local g aph in o G, we apply a p uning s ep o emo e undesi ed
sho cu s ha a e unlikely o occu in he unde lying p ocess. Such sho cu s can a ise i
an ac i i y in he on pa o
σ
ecei es high a en ion o he p edic ion, bu his ac i -
i y is only indi ec ly equi ed o he p edic ed ac i i y, in ha sense ha his ac i i y
and he p edic ed ac i i y a e no di ec ly sequen ial. To b ing he global g aph close
o a p ocess model, we apply as heu is ic, ha we emo e such sho cu s, ha lead om
an ac i i y lying long back in he p e ix o an ac i i y ha occu s la e in he p e ix. Fo -
mally, we emo e edge
(u, )∈E
i he e exis edges
(u,an)∈E
and
(an, )∈E
, o an
ac i i y
an
. Fig u e10 illus a es his s ep on a sho example. Please no e, since we gene -
ally emo e sho cu s he e is a isk, ha we may emo e a desi ed sho cu . Since he
idea is ha we analyze all p e ixes in an e en log, he in en ion was ha , di ec connec-
ions a e su icien .
Explaine 2: a en ion explo a ion explaine
The Backwa d Explaine has a key limi a ion: i di ec ly de ines he edges o bo h he
local and global g aph when analyzing a p e ix. Once an edge is inse ed, i emains in
Fig. 9 Example o Backwa dExplaine . Rele an ac i i ies and likely nex ac i i ies a e highligh ed bold aced
in ed colo in he inpu p e ix and p edic ion ec o
Page 31 o 42
Käppele al. P ocess Science (2025) 2:11
Below, we p o ide a b ie o e iew o he in en behind he me ics and how hey can
be compu ed. Fo u he de ails we would like o e e o Nau a e al. (2023):
• Co ec ness: This me ic e alua es he u h ulness o an explana ion (Nau a e al.
2023). In he li e a u e, he co ec ness is measu ed in wo di e en ways: (i) by com-
pa ing he explana ions o g ound- u h explana ions, o (ii) by calcula ing he co -
ela ion be ween he explaine ’s ea u e impo ance and he model’s ea u e impo -
ance. Because g ound- u h explana ions a e no a ailable in ou se ing, we ely on
he second op ion. The calcula ion o he ea u e impo ance o explaine and model
is iden ical o hose used in he i s expe imen o ou p e-s udy. Thus, his me ic
anges om
[−1, 1]
, wi h highe alues indica ing ha he explaine p ima ily uses
he co ec ea u e o i s explana ions.
• Comple eness: This me ic measu es how closely he explana ions coincide wi h he
p edic ions o he black box model (Nau a e al. 2023). To calcula e comple eness, we
ea he mos likely nex ac i i ies
P
o he ans o me as g ound u h and he expla-
na ions ( o be concise he igh -hand sides) as p edic ions o be e alua ed. Hence, we
e alua e he explaine analogously o a machine lea ning model by compa ing i s p e-
dic ions wi h g ound u h da a. Because he p edic ions a e ep esen ed by one o mo e
ac i i ies his esul s in a mul i-label con usion ma ix. F om his ma ix, we de i e me -
ics such as p ecision, ecall, and F1-sco e. Highe alues indica e be e comple eness,
meaning ha he ou pu o he explaine ma ches he p edic ions o he ans o me .
• Consis ency: The consis ency me ic judges he de e minism o an explaine , i.e.,
whe he an explaine p o ides he same explana ions o wo dis inc models ha
yield simila ou pu s. To compu e his me ic, we i s ain an ini ial model
M1
and
hen i e addi ional models. F om hese, we selec he model wi h he g ea es di e -
ence om
M1
based on he mean di e ence be ween hei ainable weigh s. In he
ollowing, we deno e his model wi h
M2
.
Fo each es sample
σ
, we calcula e a consis ency sco e
cσ
i he p edic ions o
M1
and
M2
o
σ
a e simila (e alua ed using he cosine simila i y) and i hei ea u e
impo ances align. The consis ency sco e
cσ
is compu ed as ollows:
whe e
R1
and
R2
a e he ule se s gene a ed by he explaine o
M1
and
M2
, espec-
i ely, and he Jacca d coe icien measu es he simila i y be ween hese ule se s.
The in en ion behind his o mula is ha he di e ence be ween he simila i y o
he p edic ion ec o s (i.e.,
|1−cosineSim(p1(σ ),p2(σ ))
) and he Jacca d coe icien
should be close o ze o. O in o he wo ds: I he p edic ions a e no simila , he Jac-
ca d coe icien should no be ei he and ice e sa. The consis ency sco es is a e -
aged ac oss all es samples o lie in he in e al [0, 1]. A alue o 1 indica es op imal
consis ency, i.e., ha he explaine p o ides iden ical explana ions o models wi h
he same ou pu s.
• Con inui y: The con inui y me ic e alua es whe he an explaine p o ides simila
explana ions o sligh ly al e ed inpu s (Nau a e al. 2023). The in en behind his
me ic is, ha i a model p o ides s able, simila explana ions o sligh ly mod-
i ied inpu s, his sugges s ha he explaine could also p o ide meaning ul and
(1)
cσ=1.0 −|1−cosineSim(p1(σ ),p2(σ )) −Jaca d(R1,R2)|,
Page 32 o 42
Käppele al. P ocess Science (2025) 2:11
consis en explana ions o simila , p e iously unseen da a poin s (Nau a e al.
2023). Thus, i can be used as an indica o o he obus ness o explana ions. By
a ying he inpu , we examine he ex en o which he new explana ion de ia es
(Nau a e al. 2023). The e o e, we c ea e o each es sample
σ
a collec ion o
modi ied e sions o
σ
by applying masking and eplacing as in he p e-s udy. Fo
each modi ied sample
σm
, we compa e he co esponding p edic ion ec o
pσm
wi h he p edic ion ec o
pσ
ob ained o he o iginal sample
σ
. I hese ec o s
a e simila (i.e.,
cosineSim(pσm
,
p
σ
>δ
p ed
), we calcula e a con inui y sco e using
Equa ion1. He e, he ule se s
R1
and
R2
, ep esen he explana ions gene a ed
o
σ
and
σm
, espec i ely. The inal con inui y sco e is ob ained by a e aging he
con inui y sco es ac oss all modi ied samples. Simila o he consis ency me ic,
his sco e akes alues in he in e al [0, 1], whe eas 1 indica es pe ec con inui y
(Nau a e al. 2023).
• Con as i i y: The con as i i y me ic se es as he coun e pa o he con-
inui y me ic. While he con inui y me ic ocuses on modi ica ions ha esul
in simila p edic ions, con as i i y examines modi ica ions ha p oduce di -
e en p edic ions. The e o e, we only conside modi ied e sions
σm
o
σ
, i
cosineSim(pσm,pσ)<δ
p ed
(i.e., we in e he condi ion compa ed o he con inu-
i y me ic). Apa om his modi ica ion, he me ic is calcula ed in he same way
as he con inui y me ic. Thus, i e alua es how dis inc he explana ions o an
explaine a e o dissimila inpu s (Nau a e al. 2023). This me ic also anges om
[0, 1], wi h a alue o 1 indica ing pe ec con as i i y.
• Compac ness: The compac ness me ic measu es how concise an explana ion is.
The e o e, wo di e en aspec s can be measu ed: he numbe o ules gene a ed
by an explaine and he a e age leng h o he ules’ igh -hand sides. Due o he
limi a ions o human cogni i e capaci y, sho e ules a e p e e able, because hey
a e easie o unde s and. Consequen ly, his me ic is pa icula ly ele an in p ac-
ical applica ions. Howe e , i is impo an o no e ha his me ic a o s b e -
i y, which does no necessa ily co ela e wi h he quali y o he explana ions. An
explana ion wi h ewe o sho e ules may lack essen ial in o ma ion and, hus,
may no always be he mos comp ehensible o accu a e. Fo his eason, com-
pac ness should only be e alua ed in conjunc ion wi h o he me ics.
To apply he me ics, a ixed pe cen age (80%) o p e ixes is andomly selec ed om
each e en log o accommoda e hei a ying sizes. Me ics a e hen compu ed o
each p e ix indi idually, ollowed by a e aging hese alues. Table3 epo s hese
a e age alues along wi h hei s anda d de ia ions. In he ollowing, we abb e ia e
Backwa d Explaine as BE and A en ion Explo a ion Explaine as AE.
Discussion o he esul s We consis en ly obse e high co ec ness alues, indica ing
a clea co ela ion be ween high a en ion sco es and he ac i i ies decisi e o p edic-
ion. This obse a ion holds o bo h eal-wo ld and syn he ic e en logs. Only in he
cases o BPIC12_O and he syn he ic Looped And and Sequence e en logs do he alues
d op sligh ly. Since he co ela ion alues, wi h excep ion o hese logs, exceeds 0.5, we
obse e a mode a e o s ong co ela ion o e all. By compa ing AE and BE, we ind ha
Page 33 o 42
Käppele al. P ocess Science (2025) 2:11
he explaine s gene ally beha e simila ly. Howe e , in he cases o Sepsis, Helpdesk, and
Sequence, BE achie es signi ican ly highe alues.
A mo e a ied pa e n eme ges o he comple eness me ic when calcula ed using
he F1-sco e. Depending on he e en log, ei he ela i ely poo alues (BPIC12,
Complex Model, LDD, Sequence, Sepsis) o qui e good alues (BPIC13) a e achie ed.
Fu he analysis shows ha low F1-comple eness is p ima ily due o low ecall alues
(see ow Comple eness (Recall) in Table3). Al hough he explana ions a e o en co -
ec (as indica ed by signi ican ly highe comple eness p ecision), no all explana ions
a e cap u ed. In he esul ing g aphs (o ules), his means mos o he edges a e co -
ec (high p ecision), bu some edges a e missing (low ecall).
Howe e , he e a e no able di e ences be ween he wo explaine s. The AE explaine
consis en ly ou pe o ms he BE in p ecision comple eness. A simila end is seen o
ecall comple eness, whe e AE pe o ms nea ly iden ical o BE o BPIC12, BPIC12_0,
BPIC12_WC, and Complex Model bu ou pe o ms BE in BPIC12_W, BPIC13_CP,
BPIC13_I, Sepsis, and Looped And. Only o Helpdesk, Sequence, and LDD a e small
di e ences (3 o 5%) obse ed in a o o BE. These indings sugges ha AE’s expla-
na ions align mo e closely wi h he p edic ions o he model. These obse a ions will
also be isible in he quali a i e e alua ion in Quali a i e e alua ionsec ion.
Fo he consis ency me ic, we obse e ex eme di e ences bo h be ween he e en
logs and be ween he explaine s. While some e en logs show nea ly pe ec consis -
ency o each explaine (AE on BPIC13_CP and BE on Looped And), o he s exhibi
e y poo consis ency close o ze o (BPIC13_I o he BE and Looped And o AE).
The ex eme diame al beha io o BE and AE in he case o he Looped And is as on-
ishing. An explana ion o his beha io is ha he a en ion sco es o he wo models
selec ed o he calcula ion o he consis ency me ic a e e y di e en . Ne e heless,
on some e en logs he models come o he same p edic ion, since o he pa s o he
a chi ec u e signi ican con ibu e o he p edic ion. Howe e , he ele an ac i i ies
(
A
) de i ed om he a en ion sco es di e undamen ally, which a ec s he s uc-
u e o he global g aph and hus he pe o mance o he explaine s. Since i is no a
gene al pa e n, i is a s ong indica o ha he e a e da ase speci ic easons o ha .
Aside om hese ou lie s, bo h explaine s ypically show medioc e consis ency sco es
a ound 0.5 ac oss mos e en logs.
Fo he con inui y me ic, he AE explaine nea ly always achie es high alues, ypi-
cally abo e 0.85. Only o he syn he ic Sequence and LDD e en logs do we obse e
lowe alues (0.57 and 0.78, espec i ely). In con as , he BE explaine pe o ms wo se,
showing mos ly lowe alues o BPIC12 (0.71), BPIC12_W (0.81), BPIC13_CP (0.49),
BPIC13_I (0.19), Sepsis (0.57), Helpdesk (0.73), Complex Model (0.72), LDD (0.57),
and Sequence (0.78). O e all, we can conclude ha AE is a highly obus explaine ,
p o iding meaning ul explana ions e en o p e iously unseen da a. Al hough he BE
canno compe e hese alues mos o he ime, i s ill achie es ela i ely high con inu-
i y sco es a ound 0.7. When e alua ing he coun e pa , i.e., he con as i i y me ic,
we ind consis en ly low alues ac oss all e en logs, wi h bo h explaine s pe o m-
ing simila ly. The combina ion o high con inui y alues and ela i e low con as i i y
sco es sugges s ha he changes in he inpu ha e ha dly any e ec on he p edic ion.
Page 34 o 42
Käppele al. P ocess Science (2025) 2:11
Table 3 Resul s o A en ion Explo a ion Explaine (AE) and Backwa d Explaine (BE); cells: mean ( i s ), s d. de ia ion (second); bold aced = bes alues; NaN = no alue could
be compu ed (in case o s d he e was some imes no a iance due o iden ical esul s, in case o he co ec ness alue i occu s i ule se s o a p e ix and i s modi ica ions a e
iden ical)
Me ic Explaine BPIC12 BPIC12_O BPIC12_W BPIC12_WC BPIC13_CP BPIC13_I Sepsis Helpdesk Complex Model LDD Looped And Sequence
Co ec ness AE 0.65
±
0.15 0.48±0.22 0.65±0.21 0.56±0.18 0.60±0.28 0.57
±
0.25 0.59±0.17 0.73±0.18 0.51±0.12 0.68±0.22 0.31±0.36 0.17±0.25
BE 0.63±0.20 0.48±0.22 0.65±0.21 0.62
±
0.18 0.75±0.18 NaN±NaN 0.86
±
0.13 0.91
±
0.08 0.59±0.15 0.68±0.21 NaN±NaN 0.32
±
0.18
Comple eness
(P ecision) AE 0.18±0.08 0.25
±
0.07 0.49
±
0.14 0.68
±
0.35 0.87
±
0.14 0.82
±
0.06 0.54
±
0.13 0.50±0.15 0.30
±
0.07 0.10±0.00 0.31
±
0.01 0.11±Nan
BE 0.20
±
0.09 0.11±0.03 0.21±0.23 0.42±0.25 0.74±0.23 0.35±0.05 0.50±0.19 0.54
±
0.15 0.19±0.03 0.30±0.00 0.00±0.00 0.32
±
NaN
Comple eness
(Recall) AE 0.05
±
0.00 0.20
±
0.01 0.22
±
0.03 0.23±0.06 0.79
±
0.15 0.77
±
0.08 0.11
±
0.01 0.25±0.03 0.10±0.00 0.14±0.00 0.25
±
0.01 0.11± NaN
BE 0.04±0.01 0.18±0.03 0.09±0.05 0.25
±
0.06 0.46±0.11 0.26±0.04 0.04±0.02 0.30
±
0.04 0.11
±
0.01 0.17
±
0.00 0.05±0.02 0.14
±
NaN
Comple eness
(F1-sco e) AE 0.07
±
0.01 0.21
±
0.02 0.27
±
0.04 0.28±0.09 0.79
±
0.15 0.77
±
0.08 0.15
±
0.02 0.30±0.04 0.12
±
0.01 0.12±0.00 0.27
±
0.01 0.10± NaN
BE 0.06±0.01 0.13±0.02 0.10±0.08 0.29
±
0.10 0.50±0.14 0.29±0.04 0.04±0.02 0.36
±
0.05 0.10±0.01 0.18
±
0.00 0.01±0.00 0.15
±
NaN
Consis ency AE 0.19±0.06 0.31±0.05 0.73
±
0.08 0.54
±
0.07 0.99
±
0.09 0.18
±
0.03 0.10±0.02 0.09±0.05 0.25
±
0.06 0.27±0.15 0.01±0.01 0.42±0.21
BE 0.46
±
0.07 0.65
±
0.07 0.66±0.13 0.39±0.08 0.59±0.18 0.01±0.02 0.13
±
0.03 0.11
±
0.09 0.02±0.03 0.45
±
0.22 0.99
±
0.01 0.53
±
0.20
Con inui y AE 0.93
±
0.05 0.95±0.07 0.92
±
0.06 0.89±0.05 0.98
±
0.10 0.99
±
0.02 0.92
±
0.13 0.85
±
0.19 0.88
±
0.21 0.57±0.31 0.93±0.24 0.78±0.25
BE 0.71±0.05 0.95±0.07 0.81±0.13 0.90
±
0.06 0.49±0.13 0.19±0.24 0.57±0.09 0.73±0.15 0.72±0.19 0.57±0.32 0.93±0.24 0.78±0.25
Con as i i y AE 0.26±0.25 0.07±0.18 0.16±0.26 0.02±0.09 0.05±0.19 0.03±0.12 0.13±0.27 0.20
±
0.30 0.30±0.15 0.42±0.31 0.40±0.21 0.32±0.34
BE 0.45
±
0.33 0.07±0.18 0.19
±
0.30 0.03
±
0.13 0.05±0.20 0.04
±
0.18 0.17
±
0.33 0.08±0.22 0.58
±
0.17 0.42±0.31 0.40±0.21 0.33
±
0.30
Compac ness
(#Rules) AE 6.32
±
1.11 4.61±0.55 4.05±0.78 3.19
±
0.60 1.88
±
0.34 1.99±0.09 4.55
±
0.85 2.45
±
0.79 4.19
±
0.98 3.75
±
1.91 2.00
±
0.00 5.08
±
2.02
BE 13.04±2.03 4.61±0.55 3.28
±
0.57 3.97±0.89 2.16±0.53 1.38
±
0.49 7.72±1.42 3.32±0.94 7.47±2.03 3.94±2.17 4.00±0.00 7.08±3.34
Compac ness (Mean
leng h) AE 4.12±0.77 2.21
±
0.09 1.86±0.25 1.89±0.15 2.09±0.35 2.00±0.09 2.76±0.42 2.55±0.62 3.60±0.64 1.89
±
1.13 4.00±0.00 2.06±0.40
BE 1.66
±
0.20 2.56±0.06 0.90
±
0.14 1.48
±
0.09 0.60
±
0.20 0.19
±
0.24 0.83
±
0.16 1.99
±
0.39 2.01
±
0.53 2.76±0.74 1.00
±
0.00 1.96
±
0.65
Page 35 o 42
Käppele al. P ocess Science (2025) 2:11
This is essen ially due o he ac ha he ans o me p ima ily lea ns he mos e-
quen execu ion pa hs, as i is limi ed o he e en ac i i y a ibu e. As a esul , i
ends o p o ide “s anda d” p edic ions e en in cases o signi ican modi ica ions in
he inpu p e ix. This obse a ion is also in line wi h p io expe imen s conduc ed in
Käppel e al. (2021), which indica e ha p edic ion models usually ocus on equen
execu ion a ian s and ea excep ional cases as s anda d a ian s.
Wi h ega d o he compac ness me ic, we obse e ha he BE explaine ends o gen-
e a e mo e ules, al hough hey usually emain wi hin a ange ha does no hinde in e -
p e abili y ( ewe han i e ules). Excep ions can be no ed o BPIC12 (13.04), Sepsis
(7.72), Complex Model (7.47), and Sequence (7.08), whe e BE p oduces mo e ules com-
pa ed o AE. When examining he mean leng h o he igh -hand sides o he ules, we
obse e ha he AE explaine consis en ly p oduces longe ules. Howe e , hese leng hs
emain below 5, which s ill enables human in e p e abili y. In con as , he Backwa d
Explaine explaine ypically gene a es ules wi h mean leng hs a ound 1. When com-
bined wi h i s pe o mance in o he me ics, pa icula ly comple eness and con inui y,
we see his as an indica o ha BE’s ules a e po en ially oo sho o adequa ely cap u e
he p ocess beha io .
In summa y, he pe o mance o bo h explaine s is nea ly iden ical when compa ing
eal-wo ld and syn he ic e en logs. Howe e , no able di e ences exis be ween he wo
explaine s. Based on he mos ele an me ics o explaine quali y – co ec and com-
ple eness – we conside he AE explaine o be he supe io choice.
Quali a i e e alua ion
To p o ide a clea e and in ui i e unde s anding o he explaine s’ pe o mance and
weaknesses, we discuss exempla ily hei ou pu o he syn he ic Complex Model e en
log (c . Fig.12). Because i is he mos complica ed p ocess model we used o he gen-
e a ion o he syn he ic e en logs i o e s he bes insigh s in o he e ec i eness o he
explaine s. Fo c ea ing he explana ion g aphs, we used all p e ixes om he es log and
s a wi h a minimum p e ix leng h o one. Including all p e ixes is essen ial o cap u e
he ull p ocess beha io , as his allows us o ace he s ep-by-s ep de elopmen o a
p ocess ins ance.
Fo be e compa abili y, he nodes in bo h explana ion g aphs we e manually aligned
simila ly o he ac i i ies in he p ocess model. A i s glance, he BE explana ion g aph
shows signi ican ly mo e edges han he AE g aph, e lec ing he esul s o he compac -
ness me ic. Al hough he e a e signi ican ly mo e edges in he g aph o he BE, he com-
ple eness ecall alues o bo h explaine s a e nea ly iden ical (0.10 s. 0.11). Howe e ,
he AE ou pe o ms he BE wi h ega d o comple eness p ecision (0.30 s. 0.19). Hence,
his example con i ms he inding o he quan i a i e e alua ion ha he AE inds ewe
inco ec edges. Ne e heless, bo h g aphs con ain e oneous ansi ions. Fo example,
he AE g aph con ains a e lexi e edge a he node o ac i i y A and an unwan ed sho -
cu om A o J. The la e is also con ained in he BE g aph.
None heless, simila i ies o he p ocess model a e e iden in bo h explana ion g aphs.
The ini ial (A-B-C) and he subsequen AND ga eway a e ac i i y D a e clea ly ecog-
nizable. While bo h explaine s co ec ly iden i y ha ac i i ies H, E, and P can ollow
D, he AE g aph does no co e all possible ollow-up sequences, whe eas he BE allows
Page 36 o 42
Käppele al. P ocess Science (2025) 2:11
mo e lexible p ocess execu ions. A simila pa e n occu s a he second AND ga eway
(a e ac i i y B), whe e he AE g aph iden i ies a ious op ions bu ends in dead ends,
while he BE g aph iden i ies signi ican ly mo e ansi ions.
Concluding ema ks
Based on he esul s o he quan i a i e and quali a i e e alua ion, we can answe ou
esea ch ques ion as ollows: he ans o me ’s a en ion sco es a e e ec i e o check-
ing and isualizing i s p ocess unde s anding. Al hough he ans o me model p o-
duces some inco ec p edic ions – p ima ily because i elies solely on he ac i i y e en
a ibu e o i s p edic ions – ce ain pa e ns in he con ol low, such as loops o XOR-
ga eways, canno always be p edic ed co ec ly. Ne e heless, we we e able o obse e
hese pa e ns in he explana ion g aph. The eason o ha is, ha he ans o me con-
side s he di e en op ions ela ed wi h hese pa e ns, paying hem a ce ain p obabili y,
al hough he canno p edic he co ec one due o i s limi ed in o ma ion.
In he es o his sec ion, we conclude he wo k wi h some heo e ical and p ac ical
implica ions. Addi ionally, we discuss po en ial limi a ions o ou app oach.
Theo e ical and P ac ical Implica ions Ou app oach o e s aluable oppo uni ies o
ad ancing ans o me -based p edic ions o he nex ac i i y. In he ollowing, we ou -
line he p ima y implica ions o ou wo k o bo h heo y and p ac ice in he ield o
p ocess mining and especially o he sub ield o nex ac i i y p edic ion.
Fig. 12 Resul ing explana ion g aphs o Complex Model E en Log. Top: The unde lying eal p ocess model.
Le : A en ion Explo e gene a ed g aph. Righ : Backwa d Explaine gene a ed g aph
Page 37 o 42
Käppele al. P ocess Science (2025) 2:11
Ou wo k has h ee main heo e ical implica ions. Fi s , ou s udy enhances he he-
o e ical unde s anding o how ans o me models cap u e and in e p e p ocess s uc-
u es, highligh ing he ela ionship be ween model in e nals (i.e., a en ion sco es) and
eco ded p ocess beha io in o m o aces. F om a mo e gene al poin o iew, i deli e s
insigh s in o how ans o me models encapsula e empo al and sequen ial dependencies
in p edic ion asks. Second, he global explana o y capabili ies o he p oposed explain-
e s con ibu e o assessing he o e all p ocess unde s anding o he p edic ion model and
ensu e consis ency in explana ions. Unlike many local explaine s, which may yield con-
adic o y explana ions ac oss di e en p ocess ins ances, ou app oach p o ides a holis-
ic iew o he p ocess logic, os e ing cohe ence ac oss explana ions. Thi d, ou app oach
has he po en ial o lay he ounda ion o a new class o p ocess disco e y echniques,
which ex ac p ocess wo k lows based on p edic i e logic. As a consequence, a p edic-
ion model would become an ac i e componen in p ocess analysis and ( e)design.
F om a p ac ical s andpoin , ou wo k has wo main implica ions. Fi s , ou app oach
enhances anspa ency and us in p edic ion models. By making he decision-making p o-
cess o he model mo e anspa en and in e p e able, p ocess pa icipan s gain a clea e
unde s anding o he p edic ions and ha e mo e con idence in he model’s p edic ions.
This is pa icula ly impo an o p ocesses, whe e he p edic ions di ec ly impac business
ope a ions o s a egic decisions as well as o p ocesses in c i ical con ex s, whe e a p edic-
ion may ha e d as ic consequences (e.g., heal hca e sys ems). Second, he insigh s gained
h ough he explaine s can se e as a p omising s a ing poin o model imp o emen
and debugging. Especially in cases o e oneous p edic ions, he explaine s p o ide use ul
insigh s ha can guide he model imp o emen , ei he h ough e aining o by p o iding
addi ional aining da a o add ess si ua ions whe e he model cu en ly s uggles. Mo eo-
e , by analyzing he ou pu ed g aph s uc u e, p ac i ione s can iden i y po en ial weak-
nesses o a eas o e inemen . This also allows o some deg ee o an icipa e po en ial u u e
beha io o he p edic ion model, e en o si ua ions ha we e no obse ed un il now.
Limi a ions The e a e some limi a ions in ou wo k. Fi s , ou expe imen s in bo h
p e-s udy and e alua ion a e limi ed o eigh eal-wo ld e en logs and ou syn he ically
gene a ed ones. Al hough he eal-wo ld e en logs s em om di e se domains and ha e
he e ogeneous cha ac e is ics, i is possible ha he ob ained esul s would a y wi h
di e en e en logs. None heless, ou indings p o ide encou aging e idence ha ou
explana ion app oach can handle common da a quali y issues in eal-wo ld logs, such as
noise o inconsis encies.
In ou explana ion app oaches, we agg ega e he a en ion sco es ac oss all ou a en-
ion heads, o e ing a holis ic pe spec i e on he en i e a en ion mechanism bu wi hou
dis inguishing he indi idual con ibu ions o indi idual heads. Thus, i emains unex-
plo ed wha pa icula heads lea n o how hei in e p e a ions migh di e . Howe e ,
as he objec i e o ou s udy was o explo e, whe he a en ion sco es as a whole gi e
insigh s in o wha he p edic ion model has lea ned, we conside ed i app op ia e o
i s ocus on he agg ega ed a en ion mechanism. While agg ega ing may lead o losing
ine-g ained in o ma ion, his choice p ima ily a ec s he de ail le el o he explana ion
a he han he o e all use ulness o he explaine s.
Page 38 o 42
Käppele al. P ocess Science (2025) 2:11
Ano he limi a ion is ha ou explana ion app oach elies solely on a en ion
sco es as model in e nals. Al hough hese sco es play a c ucial ole in he a chi ec-
u e, as e i ied in he p e-s udy, o he componen s and laye s o he ans o me
a chi ec u e also in luence he p edic ion. Cu en ly, he impac o hese compo-
nen s is widely unexamined. Howe e , his law is mi iga ed by he ac ha he sub-
sequen laye s p ima ily p ocess he a en ion sco es u he , so ha hey can be s ill
conside ed as he cen al elemen in he model’s decision-making p ocess.
Ou explaine s employ a ious h esholds, such as hose de e mining he p edic-
ion simila i y o he ele ance o an a en ion sco e. These h esholds a ec he
explana ion quali y because hey con ol he addi ion o emo al o edges in he
explana ion g aph. Fo he a en ion sco e h esholds, his issue is somewha alle-
ia ed, as he p e-s udy e eals ha a en ion sco es a e ypically ei he e y high
o e y low, allowing se ing a h eshold wi h minimal in o ma ion loss. Howe e ,
con igu ing he p edic ion h esholds is mo e challenging, as he ideal h eshold
can a y depending on he pa icula p ocess and i s execu ion s a e. In ou s udy,
we chose a uni e sal h eshold expec ed o pe o m easonably well ac oss di e en
p ocesses. Howe e , he h eshold would be ideally adap ed o he speci ic p ocess
cha ac e is ics. Fo example, a lowe h eshold is sui able o lexible p ocesses wi h
nume ous decision poin s and al e na i e execu ion pa hs, while a highe h esh-
old is app op ia e o mo e sequen ial p ocesses. Gi en ha in p ac ice, p ocesses
a e o en a mix u e o lexible and la gely sequen ial pa s, a dynamic adjus men o
h esholds depending on he cu en execu ion s a e would p o ide he bes esul .
Conclusion and u u e wo k
This pape in es iga es whe he a ained ans o me model o nex ac i i y p e-
dic ion, based on he a chi ec u e p oposed in Bukhsh e al. (2021), has gained an
unde s anding o he con ol- low o he unde lying p ocess. Gi en ha a en ion
sco es a e a he hea o he ans o me a chi ec u e and a key ac o o i s e ec-
i eness, we ho oughly in es iga ed hei eliabili y o explana ion pu poses. Ou
expe imen s p o ide s ong e idence ha a en ion sco es o e aluable insigh s
in o he decision-making o he p edic ion model, e ec i ely suppo ing he assess-
men and isualiza ion o i s p ocess unde s anding. To his end, we de eloped
wo ans o me -speci ic global explainable a i icial in elligence app oaches based
en i ely on a en ion sco es, which c ea e g aph s uc u es esembling p ocess mod-
els. These g aphs illus a e how a en ion sco es o pa icula ac i i ies a e linked
o p edic ions. We e alua ed bo h explaine s using es ablished quan i a i e me ics,
e ealing ha a en ion sco e based explaine s hold subs an ial po en ial. Howe e ,
because he p edic ion models only inco po a e con ol- low in o ma ion, he p o-
cess canno be ully lea ned. In u u e esea ch, we plan o imp o e he explaine s
by analyzing he dis inc ela ionships lea ned by indi idual a en ion heads mo e in
de ail. Mo eo e , we aim o eplace he cu en masking and modi ica ion ope a ions
o de e mining he ele ance o e en s wi h mo e sophis ica ed echniques ailo ed
o he speci ic needs o p ocess da a. Addi ionally, we plan o ex end ou explana ion
app oach o ans o me a chi ec u es ha inco po a e addi ional ea u es, such as
esou ce o ime in o ma ion.
Page 39 o 42
Käppele al. P ocess Science (2025) 2:11
Appendix1: Hype pa ame e con igu a ion
The ollowing able lis he hype pa ame e con igu a ion used o he aining o he
ans o me models.
Hype pa ame e Value
Epochs 10
Ba ch size 12
Lea ning a e 0.001
GPU usage T ue
Numbe o heads 4
Embedding Dimension 36
Dimension o he eed o wa d ne wo k 64
Appendix2: P ocess models
In he ollowing he p ocess models used o gene a ing he syn he ic e en logs a e depic ed:
Complex Model
Fig. 13 BPMN model o he Complex Model e en log
LDD
Fig. 14 BPMN model o he Long Dis ance Dependency e en log
Page 40 o 42
Käppele al. P ocess Science (2025) 2:11
Looped And
Fig. 15 BPMN model o he Looped And e en log
Sequence
Fig. 16 BPMN model o he Sequence e en log
Au ho s’ con ibu ions
All au ho s con ibu ed o he concep ion and he w i ing o he p io BPM pape . Fo his ex ended e sion M.K., L.A.,
and S.J. w o e he main manusc ip ex . M.K., L.A., and S.H. p epa ed he isualiza ions. M.K. and S.H. con ibu ed o
implemen a ion o he app oach’s p o o ype. M.K., L.A., and S.H. conduc ed expe imen s (including hose o he p io
BPM pape ). M.K., L.A, and S.J. e iewed he manusc ip .
Funding
No unding was ecei ed.
Da a a ailabili y
No da ase s we e gene a ed o analysed du ing he cu en s udy.
Ma e ials a ailabili y
Ou code and de ailed esul s can be ound in he supplemen a y ma e ial a h ps:// gi hub. com/ mkaep/ ans o me -
expla inabi li y.
Decla a ions
E hics app o al and consen o pa icipa e
No applicable
Compe ing in e es s
The au ho s decla e no compe ing in e es s.
Recei ed: 9 No embe 2024 Accep ed: 12 May 2025
Published: 4 June 2025
