On he impo ance o
compu a ional
ep oducibili y in
os e ing Open and FAIR
Science
Tibo Šimko
CERN
The ep oducibili y p oblem
2
3
I ha e di e s imes in cases, whe e he
Expe imen s seem’d like o be hough
s ange, o o be dis us ed, se down
se e al T ials o he same hing, ha
hey migh mu ually suppo and con i m
one ano he .
– Robe Boyle (1627-1691)
Robe Boyle by Johann Ke seboom (1689)
h ps://en.wikipedia.o g/wiki/Robe _Boyle
4
Su ey o 1,576 esea che s on ep oducibili y
M. Bake (2016) h ps://doi.o g/10.1038/533452a
Hal o scien is s canno ep oduce hei own esul s
Rep oduce? Wha ’s in a name?
5
6
The Tu ing Way model
h ps://book. he- u ing-way.o g/ ep oducible- esea ch/o e iew/o e iew-de ini ions.h ml
7
Rep oducible? Replicable? Repea able?
H.E. Plesse (2018) h ps://doi.o g/10.3389/ nin .2017.00076
8
F om “ ep oducible” o “ eusable” analyses
C. Diaconu, U. Schwicke a h (2025) CERN Cou ie
Expe imen al pa icle physics da a a e being analysed decades a e da a aking
DPHEP (2012) h ps://a xi .o g/abs/1205.4667
9
Expe imen al physics is done in la ge collabo a ions
CMS collabo a ion: o e
4000 pa icle physicis s,
enginee s, compu e
scien is s, echnicians
and s uden s om
a ound 240 ins i u es
and uni e si ies om
mo e han 50 coun ies.
h ps://cms.ce n/collabo a ion
P oducing obus / eusable code is expensi e
16
P oduc
(solid p og am)
Sys em P oduc
(solid and eusable)
P og am
(indi idual)
Sys em
( eusable componen s)
3x
3x
I pays o de elop eusable code i you euse i a leas h ice.
F. B ooks (1975)
17
Su ey o 1008 esea che s a he NIPS con e ence
V. S odden (2010) h ps://dx.doi.o g/10.2139/ss n.1550193
Resea che s mos ly wo y abou ime; less so abou ideas being scooped
18
Resea che s a e mo e likely o euse da a han code
V. S odden (2010) h ps://dx.doi.o g/10.2139/ss n.1550193
P ese e o euse
19
20
P ese e- o- euse: 1. Da a
CERN Open Da a po al h ps://openda a.ce n
T us ed digi al eposi o ies can p ese e da a beyond expe imen li e imes
21
P ese e- o- euse: 2. Code
T us ed digi al eposi o ies can p ese e code beyond e sion con ol li e imes
h ps://guides.gi hub.com/ac i i ies/ci able-code
22
● So wa e changes (F eesu e 4.3.1, 4.5.0, 5.0.0): 8.8±6.6% ( olume); 2.8±1.3% ( hickness)
● Ope a ing sys em changes (macOS 10.5, 10.6): “abou ac o wo smalle ”
23
P ese e- o- euse: 3. Compu ing en i onmen
h ps://hub.docke .com/u/a las h ps://hub.docke .com/u/cmssw
Con aine echnology helps o encapsula e he o iginal compu ing en i onmen
24
P ese e- o- euse: 3. Compu ing en i onmen
Compu ing en i onmen s may in e ac wi h o he un ime se ices such as
da abases; hese need “s a e encapsula ion” oo in o de o allow u u e euse
Condi ion da abase snapsho s o CMS open da a
25
P ese e- o- euse: 4. Compu a ional wo k lows
Decla a i e wo k low languages can exp ess complex compu a ional wo kl ows
CWL Snakemake Yadage
32
“P ep oducible” science
P. S a k (2018) h ps://doi.o g/10.1038/d41586-018-05256-0
33
Con inuous analyses
D i ing p ep oducibili y ia “con inuous in eg a ion” o analyses
T. Šimko e al (2021) h ps://doi.o g/10.3389/ da a.2021.661501
34
Con inuous euse
Pe iodical execu ion o da a usage examples helps o ca ch oubles ea ly
M. Donadoni e al (2021) h ps://doi.o g/10.5281/zenodo.10263203
Scena io: Wo kspace con en
When he wo k low is inished
Then he wo kspace should con ain "nje s.png"
Scena io: Wo kspace size
When he wo k low is inished
Then he wo kspace size should be less han 75 MiB
Scena io: Log con en
When he wo k low is inished
Then he job logs o he s ep "skimming" should con ain
"E en has good muons: pass=36921"
Scena io: Run du a ion
When he wo k low is inished
Then he wo k low un du a ion should be less han 25 minu es
"adap able so wa e
examples [a e] he mos
e icien way o pass on he
knowledge needed o
esea ch-le el s udies on
hese da a" — CMS
A holis ic poin o iew
35
36
Funde s: Is he g an money well spen ?
A. Mulla d (2022) h ps://doi.o g/10.1038/d41573-022-00012-6
37
Publishe s: The aud is g owing
R. Richa dson e al (2025) h ps://doi.o g/10.1073/pnas.2420092122
38
In e na ional Commi ee o Fu u e Accele a o s (ICFA)
S. Campana e al (2025) h ps://a xi .o g/abs/2508.18892 h ps://ic a-da a-bes -p ac ices-demo.app.ce n.ch/
Bes p ac ices add essing a la ge a ie y o s akeholde s
Conclusions
39
40
● Da a + Code + En i onmen + Wo k low → Reusable Analyses
● Technological challenges: la ge con aine s, complex wo k lows
● Sociological challenges: ca ing ou ime in publish-o -pe ish cul u e
● Close collabo a ion be ween esea che s and compu e scien is s
● D i ing u u e eusabili y h ough ea ly p ep oducibili y
● Syne gies ac oss scien i ic disciplines (as onomy, li e sciences, physics)
Conclusions
→ See also Clemens Lange’s alk his a e noon “Nudging Scien is s in o adop ing Open
Science P ac ices” h ps://indico.ce n.ch/e en /1484392/con ibu ions/6523967/
h ps://openda a.ce n h ps://www. eana.io
