Los in Replica ion - A Jou ney h ough he Scien i ic Replica ion C isis
Building a Cul u e o Rep oducibili y/Replica ion
Leona do Egidi (legidi@uni s.i )
Dipa imen o di Scienze Economiche, Aziendali, Ma ema iche e
S a is iche "B uno de Fine i" (DEAMS)
Uni e si à degli S udi di T ies e
28 No embe 2025
Los in Replica ion
A Jou ney h ough he Scien i ic Replica ion C isis
Building a Cul u e o Rep oducibili y/Replica ion
Leona do Egidi
Uni e si y o T ies e, [email p o ec ed]
Mas e in Da a Managemen e Cu a ion (MDMC), A ea Science Pa k
28 No embe 2025
L.Egidi (UniTS) (MDMC) Los in Replica ion 1 / 25
Ou line
1Rep oducible Wo k low
2A Mini P e- egis e ed Expe imen (wi h Py hon no ebook)
3Take-home messages
L.Egidi (UniTS) (MDMC) Los in Replica ion 2 / 25
Rep oducible?
L.Egidi (UniTS) (MDMC) Los in Replica ion 3 / 25
Wha Is a Rep oducible Wo k low?
A ep oducible wo k low is no jus “ unning he code again.” I is a s uc u ed
way o o ganizing:
da a (how hey we e collec ed, cleaned, s o ed)
code (sc ip s, e sioning, execu ion o de )
documen a ion (decisions, assump ions, en i onmen )
ou pu s ( igu es, ables, epo s)
Gelman’s iew:
“Good wo k low is no abou pe ec ion. I is abou educing he numbe
o hings ha can go w ong.”
L.Egidi (UniTS) (MDMC) Los in Replica ion 4 / 25
Rep oducibili y s Replicabili y
ACM/NASEM de ini ions:
Rep oducibili y: Using he same da a and he same code, ano he
esea che ob ains he same esul s. (→compu a ional in eg i y)
Replicabili y: Collec ing new da a and ob aining esul s consis en wi h he
o iginal s udy. (→scien i ic c edibili y)
Robus ness: Resul s do no change unde easonable al e na i e models,
p io s, o assump ions.
Take-home message: “Rep oducibili y is he minimum. Replicabili y is he
goal.”
L.Egidi (UniTS) (MDMC) Los in Replica ion 5 / 25
FAIR P inciples o Da a
Mode n esea ch ollows he FAIR p inciples:
1Findable — me ada a, iden i ie s, sea chable eposi o ies
2Accessible — open o ma s, clea access condi ions
3In e ope able — s anda dized ocabula ies, common o ma s
4Reusable — licenses, documen a ion, p o enance
Why i ma e s o ep oducibili y: Da a canno be ep oduced i hey canno
be ound, unde s ood, o eused.
L.Egidi (UniTS) (MDMC) Los in Replica ion 6 / 25
Tools o Rep oducible Wo k lows
Ve sion Con ol: Gi , Gi Hub, Gi Lab — ack e e y change, ensu e
anspa ency.
Execu able Documen s:
R Ma kdown / Qua o
Jupy e No ebooks
Py hon sc ip s + en / i ualen
P ojec S uc u e: Clea olde o ganiza ion: da a/,sc ip s/,ou pu /,
docs/.
Ad ice: “S a simple, a oid magic, documen e e y hing.”
L.Egidi (UniTS) (MDMC) Los in Replica ion 7 / 25
OSF: The Open Science F amewo k
OSF p o ides:
p e- egis a ion o s udies
e sion-con olled s o age o da a and code
anspa ency in wo k lows
easy sha ing wi h collabo a o s and e iewe s
Why is his impo an ? A anspa en wo k low p e en s acciden al p-hacking,
selec i e epo ing, and encou ages be e science.
Le ’s ake a look a one o he examples.
L.Egidi (UniTS) (MDMC) Los in Replica ion 8 / 25
F om O iginal S udy o Replica ion A emp
O iginal Expe imen :
n= 60, andom assignmen o ea men /con ol.
Da a gene a ed: y= 2 + 0.35 ×g oup +noise
T ue e ec simula ed as δ= 0.35 (small bu nonze o).
F equen is es ima e + Bayesian pos e io on he e ec .
Replica ion S udy:
P e- egis e ed: n= 100.
Same da a-gene a ing p ocess, new sample.
Analysis: same model, same p io s, same wo k low.
Wha you can do:
1Re- un he no ebook (wi h ixed seed).
2Compa e o iginal s eplica ion e ec sizes.
3Check whe he “ eplica ion success” c i e ia a e me .
L.Egidi (UniTS) (MDMC) Los in Replica ion 15 / 25
Bayesian Syn hesis & Rep oducible Wo k low
Bayesian hie a chical model:
Combines bo h da ase s while allowing o unce ain y.
Es ima es a common unde lying “ ue e ec ”.
P oduces pos e io in e als o he eplica ion.
Wo k low p inciples highligh ed in he no ebook:
Fixed andom seed ⇒de e minis ic esul s.
T anspa en code: e e y s ep documen ed.
En i onmen eco ded (package e sions, OS).
Modula s uc u e: da a, model, esul s sepa a ed.
Lesson: “Good wo k low educes he numbe o hings ha can go w ong.”
L.Egidi (UniTS) (MDMC) Los in Replica ion 16 / 25
F equen is /Bayesian o iginal es ima ion
F equen is analysis:p- alue = 0.0824, es ima ed e ec is 0.4696,
CI: = (−0.0618,1.001).
Bayesian analysis: pos e io median o he e ec is 0.4696, CI is
na owe han he equen is one.
g oup
0.00 0.25 0.50 0.75 1.00
L.Egidi (UniTS) (MDMC) Los in Replica ion 17 / 25
F equen is /Bayesian eplica ion es ima ion
F equen is analysis:p- alue = 0.292, es ima ed e ec is 0.2339,
CI: = (−0.2075,0.6753).
Bayesian analysis: pos e io median o he e ec is 0.3, CI is
na owe han he equen is one.
g oup
0.0 0.2 0.4 0.6
L.Egidi (UniTS) (MDMC) Los in Replica ion 18 / 25
Final c i e ia o eplica ion in he expe imen
Well-posed and anspa en c i e ia:
C i e ion 1: is he sign o he e ec equal ac oss he o iginal and he
eplica ed expe imen s? ⇒Yes
C i e ion 2: Is he p oduc be ween he pos e io mean o he e ec
in he o iginal expe imen and ha in he eplica ed expe imen
g ea e han ze o? ⇒Yes
C i e ion 3: is he absolu e alue di e ence be ween he o iginal
expe imen and he eplica ed expe imen lowe han a gi en
h eshold, say 0.25? ⇒Yes
C i e ion 4: a e bo h he CI in e als o he e ec no con aining
ze o in bo h expe imen s? ⇒No
Commen : he e is no a unique de ini ion o eplica ion success! You,
he esea che , a e esponsible o his inal claim (see he pape Consonni
and Egidi (2025))
L.Egidi (UniTS) (MDMC) Los in Replica ion 19 / 25
Cen e o Rep oducible Science
Why I s a ed wo king on his ield:
Cen e o Rep oducible Science in Zu ich lead by P o . Leonha d Held
L.Egidi (UniTS) (MDMC) Los in Replica ion 20 / 25
Open p ojec om my ne wo k
Join wo k wi h Leo Held, Samuel Pawel, Robe o Mac `ı Dema ino.
Mix u e p io s o eplica ion s udies (unde e iew)
L.Egidi (UniTS) (MDMC) Los in Replica ion 21 / 25
And ew’s blog
Many ideas o his cou se come om his amazing blog, o which I am a
con ibu o :
L.Egidi (UniTS) (MDMC) Los in Replica ion 22 / 25
Take-home Messages
1. The eplica ion c isis is eal, bu sol able.
2. Rep oducible wo k lows a e he ounda ion o c edible science.
3. Openness (da a, code, me hods) is a cul u al shi , no a echnical one.
4. Bayesian me hods o e p incipled ways o in eg a e o iginal and eplica ion
e idence.
L.Egidi (UniTS) (MDMC) Los in Replica ion 23 / 25
A Rep oducible Wo k low a a Glance
Da a Collec ion
( aw da a, me ada a)
Da a Cleaning
(code, logs, sc ip s)
Analysis
(models, diagnos ics)
Resul s
( igu es, ables)
Repo
(R Ma kdown/Qua o)
A chi e
(OSF / Gi Hub)
A wo k low is ep oducible when da a, code, and documen a ion low oge he .
L.Egidi (UniTS) (MDMC) Los in Replica ion 24 / 25
