CODECHECK ce i ica e 2025-028
h ps://doi.o g/10.5281/zenodo.17533508
I em Value
Ti le o checked publica ion Clinically In e p e able Su i al P edic ion in P ima y Bilia y
Cholangi is wi h T eeSHAP and G adien -Boos ed Model
Au ho (s) Emmanuel Pio Pas o e
Re e ence h ps://doi.o g/10.71240/lcyc.66O7260
Codechecke (s) Daniel Nüs
Da e o check 2025-11-04 14:36:30
Summa y This code was s aigh o wa d o check and he ep oduc ion is
pa ially success ul. Figu e 1, 2, and 3 could be ec ea ed wi hou
e o s in he code bu in pa wi h di e ences beyond he a ia-
ions due o e-compu a ion. Table 2 could be ep oduced wi hin
compu a ional a ia ion, bu Table 3 could no be linked o any
o he ou pu iles.
Reposi o y h ps://gi hub.com/codechecke s/su - ca -pbc
Table 1: CODECHECK summa y
Ou pu Commen Size (b)
esul s/spli _me ics.cs Unclea which able/ igu e his co e-
sponds o, possibly Table 3
3315
esul s/summa y_me ics.cs Table 2, wi h small nume ical di e -
ences.
680
igu es/ alida ion_me ics.png Figu e 1 64952
igu es/ ca _ba .png Figu e 3 79251
igu es/ eeshap_summa y.png Figu e 2, wi h some di e ences in ea-
u e names and alues.
292653
Table 2: Summa y o ou pu iles gene a ed
Summa y
This code was s aigh o wa d o check. Figu e 1, 2, and 3 could be ec ea ed wi hou e o s in he code and
in pa wi h di e ences ha a e due o andomness in he wo k low and no in con adic ing he o iginal
esul s. Table 2 could be ep oduced wi hin compu a ional a ia ion, bu Table 3 could no be linked o any
o he ou pu iles. The ep oduc ion is pa ially success ul.
1
CODECHECKER no es
The Gi Hub epo h ps://gi hub.com/emmanuel6474/su - ca -pbc is men ioned in he Code sec ion
o he a icle a h ps://doi.o g/10.71240/lcyc.66O7260. The au ho s c ea ed an inco ec ly o ma ed
codecheck.yml ile, bu i p o ides a s a ing poin o compile mani es in o ma ion:
e sion: 1
equi emen s:
-py hon<=3.12.8
-- equi emen s. x
en ypoin :
cmd: py hon
a gs:
-pbc_su _ ca .py
--- epea s
-"5"
---no_shap
ou pu s:
- esul s/spli _me ics.cs
- esul s/summa y_me ics.cs
- igu es/ alida ion_me ics.png
- igu es/ ca _ba .png
- igu es/calib a ion_ s a .png
op ional:
- igu es/ eeshap_summa y.png
Based on hese ou pu iles and o he in o ma ion, I upda ed he codecheck.yml ile in he check
o k a h ps://gi hub.com/codechecke s/su - ca -pbc. This check is based on he commi
c7433e6a dc b7 28029 ec94 566bbc75637286.
The ac ual wo k low execu ion was s aigh o wa d based on he ins uc ions in he README ile, which is
complemen ed by he in o ma ion om he Ma e ials sec ion o he pape on used ha dwa e:
“The compu a ional en i onmen is ully speci ied. Analyses we e conduc ed in Py hon 3.11 on a
wo ks a ion wi h an AMD Ryzen 7 5700X CPU. Key lib a ies included: xgboos , pandas, numpy,
sciki -lea n, li elines, ma plo lib, and shap.”
I c ea ed a local en i onmen wi h he sugges ed commands, albei based on my local Py hon e sion 3.10.12.
The ins alla ion o he pinned dependencies wo ked wi hou issues.
Code Is w i en in Py hon in a single sc ip ile pbc_su _ ca .py. The code is spli up in o many unc ions,
bu uses a lo o abb e ia ed a iable names, which makes i ha d o ollow he logic o he code as an ex e nal
e iewe . The e a e no commen s in he code, and no docs ings o he unc ions. C ucially, he README
lacks a clea desc ip ion o which ables and igu es in he pape co espond o which ou pu iles om he
code execu ion.
Then, I i s un he “Fas e un” mode as sugges ed in he README ile bu including he op ional h eeshap
igu e, wi h he ollowing ou pu (manually line-b oken o eadabili y):
. en )daniel@lap op-nues ~/gi /codecheck/li ecycle-jou nal-codechecks/48/su - ca -pbc [main]$ py hon
pbc_su _ ca .py -- epea s 5
{"da ase ":"PBC-276","sou ce":"h ps:// incen a elbundock.gi hub.io/Rda ase s/cs /su i al/pbc.cs ",
"sha256":"797ea9b6ab ec34297e 07 361a2e0b dd90c3c2de 180ad 8547ea30e75b613",
"n_ ows": 276,
" ea u es": 17,
" ows_d opped_missing": 142,
"censo ing_ a e": 0.598}
--- Re e ence SOTA (same da ase / ask;use -supplied)---
No --so a_json p o ided. Add one o p in e i ied e e ence numbe s o EXACTLY he same p o ocol.
[ un 01 seed=42] XGB-AFT C=0.887 |IBS=0.363 | au=4427.000
[ un 02 seed=43] XGB-AFT C=0.813 |IBS=0.340 | au=4256.000
[ un 03 seed=44] XGB-AFT C=0.840 |IBS=0.351 | au=4427.000
[ un 04 seed=45] XGB-AFT C=0.835 |IBS=0.373 | au=4556.000
[ un 05 seed=46] XGB-AFT C=0.850 |IBS=0.332 | au=4256.000
[in o] pe -spli me ics sa ed -> esul s/spli _me ics.cs
[in o] summa y me ics sa ed -> esul s/summa y_me ics.cs
=== FAIR COMPARISON SUMMARY (PBC-276, 25× 80/20;IBS[0, au_e ], KM on TRAIN, G>gmin)===
model C_mean C_sd C_boo _lo C_boo _hi IBS_mean IBS_sd IBS_boo _lo IBS_boo _hi gap_C_ o_bes gap_IBS_ o_bes
xgb 0.845 0.027 0.825 0.869 0.352 0.017 0.339 0.365 0.000 0.202
cox 0.837 0.027 0.814 0.858 0.150 0.041 0.124 0.186 0.008 0.000
wa 0.824 0.029 0.802 0.846 0.158 0.040 0.130 0.192 0.021 0.008
[in o] alida ion plo sa ed -> igu es/ alida ion_me ics.png
2
[in o] T eeSHAP sa ed -> igu es/ eeshap_summa y.png
=== Su -TCAV (las spli )— di ec ional e ec on � ===
[TCAV] choles asis Δ� = -342.362840 (-1283.930714,+604.287570)Δ�/SD(�)=-0.104 (-0.390,+0.184)pos_ a e=0.12
[TCAV] coagulopa hy Δ� = +22.269302 (-252.168345,+318.759151)Δ�/SD(�)= +0.007 (-0.077,+0.097)pos_ a e=0.27
[TCAV] low_albumin Δ� = -154.954063 (-515.330334,+179.620646)Δ�/SD(�)=-0.047 (-0.157,+0.055)pos_ a e=0.25
[TCAV] olde _age Δ� = -412.217567 (-693.317137,-176.420951)Δ�/SD(�)=-0.125 (-0.211,-0.054)pos_ a e=0.25
[TCAV] clinical_complica ions Δ� = -325.000553 (-1240.637570,+618.042643)Δ�/SD(�)=-0.099 (-0.377,+0.188)pos_ a e=0.35
[in o] TCAV ba sa ed -> igu es/ ca _ba .png
[in o] calib a ion plo sa ed -> igu es/calib a ion_ s a .png
=== Repo ing no es (TRIPOD-li e)===
* In e nal alida ion (25× 80/20)
* E en o in e es : dea h (s a us==2). T ansplan ea ed as censo ed.
This comple es in a ew minu es wi hou e o s, and mos expec ed ou pu iles a e c ea ed. Only
igu es/calib a ion_ s a .png is missing, e en hough he log ou pu indica es i should ha e been
c ea ed. A e consul a ion wi h he au ho (see below), he igu e is no used in he submission and is no
gene a ed due o a small sample size.
Now I am manually compa ing he c ea ed ou pu iles o he igu es and ables in he pape :
• igu es/ ca _ba .png appea s o co espond o Figu e 3 in he pape wi h ma ching plo s yle, i le,
and axis labels, hough conside able di e en da a, likely because o he limi ed epea s;
• igu es/ eeshap_summa y.png appea s o co espond o Figu e 2 in he pape wi h ma ching plo
s yle, i le, and axis labels, and he da a seems o ha e oughly simila pa e ns, hough he ea u es
a e so ed di e en ly likely because o di e en “mean absolu e SHAP alue” because o he limi ed
epea s;
• igu es/ alida ion_me ics.png appea s o co espond o Figu e 1 in he pape wi h ma ching plo
s yle, i le, and axis labels, hough conside able ewe da a poin s and di e en pa ens, likely because
o he limi ed epea s;
• esul s/spli _me ics.cs likely co esponds o one o he ables in he pape , bu i is no possible
o de e mine om he ace o he alues (due o di e en numbe o epea s) o he e y sho a iable
names which able his is;
• esul s/summa y_me ics.cs likely co esponds o one o he ables in he pape , bu i is no
possible o de e mine om he ace o he alues (due o di e en numbe o epea s) o he e y sho
a iable names which able his is;
To possibly ix he disc epancies, I e- an he code wi h he de aul con igu a ion, and his also comple ed
wi hin a ew minu es (manually line-b oken o eadabili y):
(. en ) daniel@lap op-nues ~/gi /codecheck/li ecycle-jou nal-codechecks/48/su - ca -pbc [main]$ py hon pbc_su _ ca .py
{"da ase ": "PBC-276", "sou ce": "h ps:// incen a elbundock.gi hub.io/Rda ase s/cs /su i al/pbc.cs ",
"sha256": "797ea9b6ab ec34297e 07 361a2e0b dd90c3c2de 180ad 8547ea30e75b613",
"n_ ows": 276,
" ea u es": 17,
" ows_d opped_missing": 142,
"censo ing_ a e": 0.598}
--- Re e ence SOTA (same da ase / ask; use -supplied) ---
No --so a_json p o ided. Add one o p in e i ied e e ence numbe s o EXACTLY he same p o ocol.
[ un 01 seed=42] XGB-AFT C=0.887 | IBS=0.363 | au=4427.000
[ un 02 seed=43] XGB-AFT C=0.813 | IBS=0.340 | au=4256.000
[ un 03 seed=44] XGB-AFT C=0.840 | IBS=0.351 | au=4427.000
[ un 04 seed=45] XGB-AFT C=0.835 | IBS=0.373 | au=4556.000
[ un 05 seed=46] XGB-AFT C=0.850 | IBS=0.332 | au=4256.000
[ un 06 seed=47] XGB-AFT C=0.840 | IBS=0.350 | au=4509.000
[ un 07 seed=48] XGB-AFT C=0.745 | IBS=0.384 | au=4500.000
[ un 08 seed=49] XGB-AFT C=0.817 | IBS=0.342 | au=4256.000
[ un 09 seed=50] XGB-AFT C=0.817 | IBS=0.319 | au=3933.000
[ un 10 seed=51] XGB-AFT C=0.908 | IBS=0.346 | au=4256.000
[ un 11 seed=52] XGB-AFT C=0.906 | IBS=0.353 | au=4256.000
[ un 12 seed=53] XGB-AFT C=0.813 | IBS=0.368 | au=4556.000
[ un 13 seed=54] XGB-AFT C=0.879 | IBS=0.374 | au=4523.000
[ un 14 seed=55] XGB-AFT C=0.816 | IBS=0.321 | au=4191.000
[ un 15 seed=56] XGB-AFT C=0.848 | IBS=0.387 | au=4509.000
[ un 16 seed=57] XGB-AFT C=0.844 | IBS=0.363 | au=4556.000
[ un 17 seed=58] XGB-AFT C=0.864 | IBS=0.366 | au=4556.000
[ un 18 seed=59] XGB-AFT C=0.793 | IBS=0.365 | au=4427.000
[ un 19 seed=60] XGB-AFT C=0.889 | IBS=0.363 | au=4523.000
[ un 20 seed=61] XGB-AFT C=0.813 | IBS=0.355 | au=4365.000
[ un 21 seed=62] XGB-AFT C=0.861 | IBS=0.361 | au=4556.000
[ un 22 seed=63] XGB-AFT C=0.805 | IBS=0.383 | au=4500.000
3
[ un 23 seed=64] XGB-AFT C=0.798 | IBS=0.354 | au=4256.000
[ un 24 seed=65] XGB-AFT C=0.824 | IBS=0.342 | au=4365.000
[ un 25 seed=66] XGB-AFT C=0.807 | IBS=0.361 | au=4556.000
[in o] pe -spli me ics sa ed -> esul s/spli _me ics.cs
[in o] summa y me ics sa ed -> esul s/summa y_me ics.cs
=== FAIR COMPARISON SUMMARY (PBC-276, 25× 80/20; IBS[0, au_e ], KM on TRAIN, G>gmin) ===
model C_mean C_sd C_boo _lo C_boo _hi IBS_mean IBS_sd IBS_boo _lo IBS_boo _hi gap_C_ o_bes gap_IBS_ o_bes
xgb 0.837 0.038 0.822 0.851 0.357 0.018 0.349 0.363 0.000 0.207
cox 0.829 0.034 0.815 0.842 0.149 0.039 0.136 0.164 0.007 0.000
wa 0.818 0.037 0.803 0.832 0.155 0.036 0.143 0.169 0.019 0.006
[in o] alida ion plo sa ed -> igu es/ alida ion_me ics.png
[in o] T eeSHAP sa ed -> igu es/ eeshap_summa y.png
=== Su -TCAV (las spli ) — di ec ional e ec on � ===
[TCAV] choles asis Δ� = -569.489481 (-1644.770115,+433.051415) Δ�/SD(�) = -0.179 (-0.516,+0.136) pos_ a e=0.12
[TCAV] coagulopa hy Δ� = -309.125045 (-529.181518,-100.936478) Δ�/SD(�) = -0.097 (-0.166,-0.032) pos_ a e=0.26
[TCAV] low_albumin Δ� = -583.421913 (-932.153546,-238.641145) Δ�/SD(�) = -0.183 (-0.293,-0.075) pos_ a e=0.25
[TCAV] olde _age Δ� = -536.163012 (-969.125221,-139.228449) Δ�/SD(�) = -0.168 (-0.304,-0.044) pos_ a e=0.25
[TCAV] clinical_complica ions Δ� = -713.050014 (-1648.918126,+228.438344) Δ�/SD(�) = -0.224 (-0.518,+0.072) pos_ a e=0.34
[in o] TCAV ba sa ed -> igu es/ ca _ba .png
[in o] calib a ion plo sa ed -> igu es/calib a ion_ s a .png
=== Repo ing no es (TRIPOD-li e) ===
* In e nal alida ion (25× 80/20)
* E en o in e es : dea h (s a us==2). T ansplan ea ed as censo ed.
summa y_me ics.cs now has alues ha i Table 2 ela i ely closely, so ha is success ully ep oduced
wi hin compu a ional a ia ion. I s ill canno link spli _me ics.cs o Table 3 based on he alues .
eeshap_summa y.png now looks much close o he o iginal Figu e 2, hough he e a e s ill some cu ious
di e ence, such as “s age_2.0” bein missing in my ep oduc ion, bu he ep oduc ion ha ing a “spide s_1.0”
ha is missing om he o iginal.
To summa ize, while he code uns wi hou e o s and p oduces mos o he expec ed ou pu iles, bu hey
canno be pe ec ly ma ched o he o iginal esul s. Some o hese di e ences may go beyond o be expec ed
nume ical di e ences om my pe spec i e.
A e consul a ion wi h he au ho (see commen on Gi Hub), he au ho claims he de ia ions a e small
and expec ed due o s ochas ic na u e o he wo k low.
The di e ences you no ed a e all in he same di ec ion as in he pape , a y only sligh ly in
magni ude, and he 95% con idence in e als s ill o e lap comple ely. These a e well wi hin he
ange expec ed o s ochas ic models and ha e no impac on he in e p e a ion o conclusions.
In he SHAP summa y plo s, he ea u e names do no ac ually change in a meaning ul way.
The small di e ences isible nea he bo om o he igu e in ol e ea u es wi h e y low impac ,
close o s a is ical noise, so i is na u al ha hey may mo e sligh ly in o ou o he impo ance
anking. The o e all clus e s o poin s a e isually and nume ically e y simila , and he main
ea u es a he op o he anking a y only minimally because hey a e much mo e signi ican .
I ha e no eason o ques ion his s a emen , ye would in i e opical e iewe s o assess he in e p e a ion
o he di e ences independen ly.
Recommenda ions
I sugges o he au ho s o conside he ollowing sugges ions o hei nex publica ion o wo k low:
• Clea ly name ou pu iles in a way ha makes i easy o link hem o igu es and ables in he pape
(e.g. “ igu e_2_ eeshap_summa y.png” ins ead o “ eeshap_summa y.png”);
• T y o se seeds o o he andom numbe gene a ion ini ializa ions in a way ha allows exac ep o-
duc ion o esul s;
4
Mani es iles
spli _me ics.cs
Summa y s a is ics o abula da a:
-- Da a Summa y ------------------------
Values
Name ead.cs (pa h)
Numbe o ows 25
Numbe o columns 11
_______________________
Column ype equency:
logical 2
nume ic 9
________________________
G oup a iables None
-- Va iable ype: logical ----------------------------------
skim_ a iable n_missing comple e_ a e mean coun
1 c_ s 25 0 NaN ": "
2 ibs_ s 25 0 NaN ": "
-- Va iable ype: nume ic ----------------------------------
skim_ a iable n_missing comple e_ a e mean sd
1 un 0 1 13 7.36
2 seed 0 1 54 7.36
3 au 0 1 4403. 160.
4 c_xgb 0 1 0.837 0.0384
5 ibs_xgb 0 1 0.357 0.0178
6 c_cox 0 1 0.829 0.0343
7 ibs_cox 0 1 0.149 0.0385
8 c_wa 0 1 0.818 0.0368
9 ibs_wa 0 1 0.155 0.0356
p0 p25 p50 p75 p100 his
1 1 7 13 19 25 �����
2 42 48 54 60 66 �����
3 3933 4256 4427 4523 4556 �����
4 0.745 0.813 0.835 0.861 0.908 �����
5 0.319 0.346 0.361 0.366 0.387 �����
6 0.759 0.808 0.839 0.851 0.884 �����
7 0.102 0.124 0.136 0.160 0.240 �����
8 0.744 0.794 0.808 0.849 0.879 �����
9 0.103 0.132 0.147 0.165 0.233 �����
5
summa y_me ics.cs
Summa y s a is ics o abula da a:
-- Da a Summa y ------------------------
Values
Name ead.cs (pa h)
Numbe o ows 3
Numbe o columns 11
_______________________
Column ype equency:
cha ac e 1
nume ic 10
________________________
G oup a iables None
-- Va iable ype: cha ac e --------------------------------
skim_ a iable n_missing comple e_ a e min max emp y
1 model 0 1 3 4 0
n_unique whi espace
1 3 0
-- Va iable ype: nume ic ----------------------------------
skim_ a iable n_missing comple e_ a e mean sd
1 C_mean 0 1 0.828 0.00944
2 C_sd 0 1 0.0365 0.00206
3 C_boo _lo 0 1 0.814 0.00961
4 C_boo _hi 0 1 0.842 0.00960
5 IBS_mean 0 1 0.220 0.118
6 IBS_sd 0 1 0.0306 0.0112
7 IBS_boo _lo 0 1 0.209 0.121
8 IBS_boo _hi 0 1 0.232 0.114
9 gap_C_ o_bes 0 1 0.00866 0.00944
10 gap_IBS_ o_bes 0 1 0.0710 0.118
p0 p25 p50 p75 p100 his
1 0.818 0.824 0.829 0.833 0.837 �����
2 0.0343 0.0355 0.0368 0.0376 0.0384 �����
3 0.803 0.809 0.815 0.819 0.822 �����
4 0.832 0.837 0.842 0.847 0.851 �����
5 0.149 0.152 0.155 0.256 0.357 �����
6 0.0178 0.0267 0.0356 0.0371 0.0385 �����
7 0.136 0.139 0.143 0.246 0.349 �����
8 0.164 0.166 0.169 0.266 0.363 �����
9 0 0.00363 0.00725 0.0130 0.0187 �����
10 0 0.00285 0.00571 0.106 0.207 �����
6
alida ion_me ics.png
Commen : Figu e 1
7
ca _ba .png
Commen : Figu e 3
8
eeshap_summa y.png
Commen : Figu e 2, wi h some di e ences in ea u e names and alues.
9
