Technical Appendix o :
F om Con lic s o E iciency: Lea ning Cons ain O de ings o Con lic
De ec ion
1. In oduc ion
This documen se es as a echnical appendix o he main
pape , ”F om Con lic s o E iciency: Lea ning Cons ain
O de ings o Con lic De ec ion.” The pu pose o his ap-
pendix is o p o ide supplemen a y ma e ial ha o e s
g ea e dep h and anspa ency ega ding ou expe imen al
me hodology and esul s.
He ein, we p esen de ailed in o ma ion on ou da a p e-
p ocessing s eps, he comple e op imal hype pa ame e con-
igu a ions disco e ed o each model and da ase , isual-
iza ions o he MLP model aining p ocess ia loss cu es,
and exhaus i e pe o mance me ics. This in o ma ion is in-
ended o suppo he indings p esen ed in he main pape
and o ensu e he ull ep oducibili y o ou wo k.
2. Da a P ep ocessing De ails
Be o e model aining, we applied a s anda dized p ep o-
cessing pipeline o each da ase o simpli y he lea ning ask
and imp o e model pe o mance. This p ocess in ol es e-
mo ing inpu cons ain s ( ea u es) wi h low a iance and
ou pu cons ain s (labels) ha ha e a cons an alue ac oss
he en i e da ase . This p ep ocessing was applied uni o mly
o bo h he Mul i-Laye Pe cep on (MLP) and he Decision
T ee / Random Fo es (DT/RF) models. Labels ha we e e-
mo ed du ing his phase we e p og amma ically e-inse ed
in o he inal p edic ions wi h hei known cons an alue o
ensu e a comple e ou pu .
A cade BusyBox B2C
Cons ain s o al 47 683 194
Remo ed Fea u es 0 0 0
Remo ed Labels 3 539 72
Table 1: Summa y o ea u es and labels emo ed du ing
p ep ocessing.
The h eshold o emo ing low- a iance ea u es was se
o 0.01. This means any inpu cons ain ha had he same
alue (e.g., always ue o always alse) o mo e han 99%
o he samples would be emo ed. Such ea u es p o ide li -
le o no in o ma ion o a machine lea ning model and can
Copy igh © 2026, Associa ion o he Ad ancemen o A i icial
In elligence (www.aaai.o g). All igh s ese ed.
be sa ely disca ded. Ac oss all h ee da ase s, no ea u es
me his emo al c i e ion.
Fo he ou pu labels, each da ase con ained cons ain s
ha we e ne e pa o any Minimal Con lic Se (MCS)
ac oss all gene a ed samples (i.e., had a cons an alue o
0). Fo he model, p edic ing a cons an is a i ial ask ha
o e s no lea ning alue. These cons an -ze o labels we e
he e o e emo ed be o e aining o allow he model o
ocus on he mo e complex, non- i ial ela ionships. As
shown in Table 1, he BusyBox da ase was he mos a -
ec ed, wi h 78% o i s labels being cons an , while A cade
was he leas a ec ed (6%). When he ained model makes a
p edic ion on new da a, i co ec ly assigns a ’0’ o hese e-
mo ed labels, ensu ing he inal p edic ed MCS is comple e
and accu a e.
3. Op imal Hype pa ame e Con igu a ions
The ollowing ables de ail he op imal hype pa ame e con-
igu a ions ound using he Op una lib a y o ou bes -
pe o ming MLP and DT/RF models on each da ase . These
con igu a ions we e selec ed based on hei pe o mance on
he alida ion se and a e p o ided he e o ensu e ep o-
ducibili y.
MLP Con igu a ions
Pa ame e Explana ions ( o Table 2:
•Loss Func ion: Fo he A cade da ase , ‘BCEWi hLog-
i sLoss‘ (Bina y C oss-En opy) was op imal. Fo he
mo e imbalanced BusyBox and B2C da ase s, ‘Focal
Loss‘ was supe io . Focal Loss is a modi ica ion o c oss-
en opy ha educes he loss assigned o well-classi ied
examples, o cing he model o ocus on ha de , misclas-
si ied examples.
•Focal Loss Gamma & Alpha: These pa ame e s a e
speci ic o Focal Loss. Gamma is he ocusing pa ame e ;
a highe gamma alue inc eases he down-weigh ing o
easy examples. Alpha is a weigh ing ac o ha balances
he impo ance o posi i e e sus nega i e examples.
•Common Pa ame e s: Ac oss all da ase s, he Adam
op imize and an ea ly s opping pa ience o 20 p o ed o
be uni e sally e ec i e. Ba ch no maliza ion was consis-
en ly bene icial, helping o s abilize and accele a e ain-
ing.
Hype pa ame e A cade BusyBox B2C
Hidden Laye s [64, 64] [128, 64] [128, 64, 32]
D opou Ra e None None None
Ac i a ion Func Leaky ReLU Leaky ReLU ReLU
Ba ch Size 512 256 256
Ba ch No maliza ion ue ue ue
Max Epochs 200 300 300
Pa ience 20 20 20
Loss Func BCEWi hLogi sLoss Focal Loss Focal Loss
Op imize Adam Adam Adam
Lea ning Ra e 0.039464 0.004512 0.001737
Weigh Decay 0.001579 3.310980 7.453246
PCA alse alse alse
Focal Loss Gamma None 3.304386 3.656835
Focal Loss Alpha None 0.409502 0.451800
Table 2: Op imal hype pa ame e s o he bes MLP model on each da ase .
Decision T ee / Random Fo es Con igu a ions
Pa ame e Explana ions ( o able 3):
•Es ima o Type: The inal model was a Random Fo -
es o he A cade da ase and a single Decision T ee o
BusyBox and B2C.
•Max Dep h: This pa ame e con ols he maximum
dep h o he ee(s) o p e en o e i ing. Fo B2C, an
unlimi ed dep h yielded he bes esul s.
•Numbe o Es ima o s: This is speci ic o Random Fo -
es and de ines he numbe o ees in he o es . A alue
o 600 was op imal o A cade.
•Common Pa ame e s: The use o ‘balanced‘ class
weigh s was a consis en and c i ical choice ac oss all
da ase s, helping he models handle he inhe en imbal-
ance in he da a whe e non-con lic cons ain s as ly ou -
numbe con lic cons ain s. The choice be ween ‘Clas-
si ie Chain‘ and ‘Mul iOu pu Classi ie ‘ a ied, sugges -
ing ha he deg ee o co ela ion be ween labels di e s
ac oss he knowledge bases.
4. MLP T aining & Valida ion Loss Cu es
The ollowing plo s illus a e he aining and alida ion loss
o e epochs o he bes -pe o ming MLP model on each
da ase . These cu es p o ide insigh in o he lea ning dy-
namics and gene aliza ion capabili y o he models.
Analysis o A cade (Figu e 1): The aining loss cu e
d ops e y apidly, indica ing ha he model quickly lea ns
he ela ionships be ween cons ain s. The ini ial gap be-
ween he aining and alida ion loss sugges s some ea ly
o e i ing, bu he cu es con e ge o a low, s able alue a -
e app oxima ely 150 epochs. The inal p oximi y o he wo
cu es shows ha he model success ully gene alized o he
unseen alida ion da a wi hou o e i ing. E en hough he
model an he ull max epochs ha was se o 200, he wo
loss cu es being a nea s aigh line in he end and close o
0, meaning model has al eady had a e y good pe o mance
and won imp o e much mo e, so u he aining is also no
necessa y.
Figu e 1: T aining and alida ion loss o he A cade da ase .
Figu e 2: T aining and alida ion loss o he B2C da ase .
Analysis o B2C (Figu e 2): The loss cu es o he B2C
da ase demons a e excep ionally as lea ning, wi h bo h
aining and alida ion loss d opping o a nea -ze o alue
wi hin he i s 25 epochs. The cu es hen o e lap almos
pe ec ly as hey con inue o slowly imp o e un il he ea ly
s opping mechanism hal s aining a ound epoch 175, which
Hype pa ame e A cade BusyBox B2C
Es ima o Type Random Fo es Decision T ee Decision T ee
Max Dep h 35 80 None (unlimi ed)
Mul i-Ou pu Type Classi ie Chain Mul iOu pu Classi ie Classi ie Chain
PCA alse alse alse
Class Weigh balanced balanced balanced
Numbe o Es ima o s 600 None None
Table 3: Op imal hype pa ame e s o he bes DT/RF model on each da ase .
is a s ong indica o ha he model is no o e i ing. Bo h
cu es ha ing loss o almos exac ly 0 means he model
has lea ned a highly obus and accu a e mapping om use
equi emen s o con lic se s o his pa icula knowledge
base.
Figu e 3: T aining and alida ion loss o he BusyBox
da ase .
Analysis o BusyBox (Figu e 3): Simila o B2C, he
model o he complex BusyBox da ase lea ns e y quickly,
wi h bo h loss cu es d opping below 0.1 a e jus 25
epochs. The alida ion loss closely acks he aining loss
h oughou he p ocess, albei a a e y sligh ly highe alue,
which is a sign o no o e i ing. The loss o nea 0 also in-
dica es ha he model is no unde i ing nei he . The ea ly
s opping mechanism was igge ed a ound epoch 220 (max
epoch o his da ase was se o 300), con i ming ha he
model had con e ged o i s op imal pe o mance on he al-
ida ion se .
5. De ailed Pe o mance Resul s
This sec ion p esen s he de ailed pe o mance me ics o
ou bes -pe o ming MLP and DT/RF models on he held-
ou es se o each knowledge base. Each es se is 10% o
he whole da ase , chosen andomly, bu in a way ha s ill
includes a leas 1 sample om each unique Minimal Con-
lic Se (MCS), o ensu e we es he model on all known
MCS.
Di ec P edic ion Pe o mance
Table 4 summa izes he pe o mance o he models when
used o di ec p edic ion o he MCS, wi hou he in ol e-
men o QUICKXPLAIN.
Me ic Explana ions:
•Exac Ma ch (di ec p edic ion): We ake di ec ly he
p edic ion o he model o a MCS and compa e i o he
g ound- u h MCS. This is a e y s ic me ic ha show
he pe cen age o pe ec ly ma ched p edic ions (any p e-
dic ion ha has e en jus 1 cons ain w ong will be con-
side ed w ong).
•F1 / MCC / MAP: S anda d mul i-label classi ica ion
me ics. A sco e o 1.0 is pe ec .
•Hamming Loss: The ac ion o inco ec ly p edic ed la-
bels (lowe is be e ).
•ROC AUC: The A ea Unde he Recei e Ope a ing
Cha ac e is ic Cu e, measu ing he model’s abili y o
dis inguish be ween posi i e and nega i e classes. A
sco e o 1.0 is pe ec .
•Cosine Simila i y: Measu es he o e lap be ween he
p edic ed and ue MCS ec o s. A alue o 1.0 indica es
a pe ec ma ch.
• Exac Ma ch (a e QX): This again es s how many imes
we can pe ec ly compu e he p e e ed MCS ha ex-
ac ly ma ches he g ound- u h MCS. Di e ence wi h he
o he ’exac ma ch’ me ic is, ha his is a e we ha e e-
o de ed he cons ain s and go a MCS ha was compu ed
by QuickXplain.
Analysis: The esul s in Table 4 a e excep ionally s ong
ac oss he boa d. The Exac Ma ch o di ec p edic ion is
abo e 92% o all models, indica ing ha hese con lic de-
ec ion p oblems can be sol ed wi h high accu acy by ML
models alone. While bo h model amilies pe o m e y well,
he simple DT/RF models o en show a sligh ad an age
in di ec p edic ion accu acy, pa icula ly on he complex
BusyBox da ase . Mo eo e , he aining ime o Decision
T ee (and also Random Fo es ) a e no iceably as e han
MLP, so hey would be mo e bene icial o use o la ge
da ase . The MLP model, howe e , achie ed nea -pe ec ion
on he B2C da ase . The high sco es ac oss all o he me ics
(F1, MCC, e c.) u he co obo a e he models’ obus ness
and eliabili y.
Run ime and Consis ency Check Imp o emen
Table 5 de ails he end- o-end sys em pe o mance, compa -
ing he ML-guided QUICKXPLAIN agains a baseline whe e
cons ain s a e o de ed andomly.
Analysis: As shown in Table 5, bo h ML-guided app oaches
p o ide a subs an ial pe o mance imp o emen o e he
Me ic A cade BusyBox B2C
MLP DT/RF* MLP DT/RF* MLP DT/RF*
Exac Ma ch (di ec p edic ion) 96.34% 97.15% 92.61% 99.01% 99.85% 97.95%
F1 0.99 0.99 0.98 0.99 0.99 0.99
MCC 0.99 0.98 0.97 0.99 0.99 0.98
MAP 0.99 0.99 0.99 0.97 0.99 0.98
Hamming Loss 0.0010 0.0007 0.0004 0.0003 0.00001 0.0007
ROC AUC 0.99 0.99 0.99 0.99 0.99 0.99
Cosine Simila i y 99.70% 99.86% 99.75% 99.83% 99.99% 99.92%
Exac Ma ch (a e QX) 99.39% 99.76% 99.49% 99.67% 99.99% 99.68%
*The DT/RF model is a Random Fo es o A cade and a Decision T ee o BusyBox and B2C.
Table 4: De ailed pe o mance me ics o MLP model’s p edic ion on he es se . Fo each me ic, he be e -pe o ming model
amily is shown in bold.
Me ic A cade BusyBox B2C
MLP DT/RF Random MLP DT/RF Random MLP DT/RF Random
Run ime (s) 0.0018 0.0045 0.0086 0.0357 0.0398 0.0667 0.0045 0.0068 0.0104
Consis ency Checks 7.115 9.565 17.272 10.859 11.165 18.887 9.565 9.936 17.929
Run ime Imp o emen * 378.9% 90.3% - 87% 67.5% - 131% 52.4% -
CC Reduc ion 58.8% 44.6% - 42.5% 40.9% - 46.7% 44.6% -
*Run ime imp o emen is calcula ed as ollow: (Baseline - Model) / Model * 100%. This means, an imp o emen o 100% is
2 imes as e , 200% is 3 imes as e , 300% is 4 imes as e , and so on.
Table 5: Run ime and Consis ency Check (CC) pe o mance compa ison. Imp o emen pe cen ages a e ela i e o he andom
o de ing baseline.
andom o de ing baseline, con i ming ha in elligen o de -
ing signi ican ly helps QUICKXPLAIN. A key inding is ha
he MLP models consis en ly esul ed in a g ea e educ ion
in he numbe o consis ency checks, and consequen ly, a
as e a e age un ime. This demons a es he MLP’s supe-
io abili y o ank he cons ain s in an o de ha is highly
e ec i e o he di ide-and-conque algo i hm, making i he
p e e ed choice when aw speed is he p ima y objec i e.
No es on Rep oducibili y
All sou ce code, ained models, and esul s a e a ailable in
h ps://doi.o g/10.5281/zenodo.16739565.
