Replica ion: "I packed my eplica ion package
and in i I pu …"
Resea ch Da a Managemen Semina s: Replica ion
D . Phil Kolbe 1
Agenda
•Rep oducibili y and eplicabili y
•Reasons and incen i es o ep oducibili y and eplicabili y I
•Requi emen s o ep oducibili y and eplicabili y
•Reasons and incen i es o ep oducibili y and eplicabili y II
•Replica ion package
Resea ch Da a Managemen Semina s: Replica ion 2
Rep oducibili y and eplicabili y
3
Ma ix de ining ep oducible esea ch ( om The Tu ing Way)
Reasons and incen i es o
ep oducibili y and eplicabili y I
•Replica ion c isis
•T anspa ency and us in science and esea ch
•E o De ec ion and Debugging
•“S anding on he shoulde s o Gian s”
•P o essional Repu a ion
•Fos e s coope a ion/collabo a ion
•And mo e la e …
Resea ch Da a Managemen Semina s: Replica ion 4
Reasons and incen i es I: Replica ion
c isis
•Many published scien i ic indings canno be eplica ed o ep oduced
•Wide a ie y o ields a ec ed
•Causes: P-Hacking, Publica ion Bias, Low S a is ical Powe , Lack o
T anspa ency
•Consequences: E osion o us , was ed esou ces and ba ie s o scien i ic
p og ess
•Solu ions: open da a, p e- egis a ion o s udies, eplica ion s udies and
highe s anda ds o publica ion
Resea ch Da a Managemen Semina s: Replica ion 5
Reasons and incen i es I: “S anding on
he shoulde s o Gian s”
•Highligh s how scien i ic p og ess builds on p io disco e ies
•Reliable, eplicable esea ch allows scien is s o us , check and build
upon he indings o o he s
•Wi h alida ed esul s, esea che s can ocus on ex ending knowledge
a he han edoing pas wo k
•Reduces duplica ion o e o and edi ec s esou ces o new, inno a i e
esea ch a he han e isi ing un eliable s udies
•O he wise, you' e jus s anding on a pile o was epape
Resea ch Da a Managemen Semina s: Replica ion 6
Requi emen s o ep oducibili y and
eplicabili y
Wha do we need o ep oducibili y and eplicabili y?
Key equi emen s:
•Da a, Da a Access and Documen a ion
•Folde and File S uc u e
•Analysis Documen a ion
•Publica ion o code and da a
Resea ch Da a Managemen Semina s: Replica ion 7
Requi emen s o ep oducibili y and
eplicabili y: da a, da a access and
documen a ion
Da a access desc ip ion:
•Whe e is he da a loca ed? → DOI, websi e, RDC
•Who c ea ed he da a? → esea ch ins i u ion, public au ho i y, company
•Who can access he da a? → public use ile, scien i ic use ile,
con iden ial da a
•How can he da a be accessed? → open access, egis a ion, applica ion
o use, cos s
Resea ch Da a Managemen Semina s: Replica ion 8
Requi emen s o ep oducibili y and eplicabili y:
da a, da a access and documen a ion
9
ID
age
Q1
Q2
…
1
18
2
-
…
2
20
-
999
1,4
3
19
0
2,4
4
19
xxx
3,6
5
199
5
2.3
6
20
1
7
19
3
2,1
Do you unde s and his
da a se , and can you
use i ? Wha p oblems
could he e be?
Requi emen s o ep oducibili y and
eplicabili y: Publica ion o code and da a
•only published manusc ip s, p og am codes and da a a e accessible o
hi d pa ies and ensu e anspa en and e i iable esea ch
• ypical non-subjec -speci ic op ions:
–h ps://zenodo.o g/: Pe eco d limi a ion (50GB / 100 iles), wi h 200 GB excep ion
o one eco d
–h ps://os .io/: capaci y o p i a e p ojec s and componen s u ilizing OSF S o age o
5 GB and public p ojec s and componen s o 50 GB
–h ps://da a e se.ha a d.edu/: deposi iles o up o 2.5GB, and s o e up o 1TB o
da a
•check o subjec -speci ic op ions o use he ins i u ional eposi o y
MADATA
Resea ch Da a Managemen Semina s: Replica ion 16
Requi emen s o ep oducibili y and
eplicabili y: Publica ion o code and da a
•File o ma s o sha ing → Non-p op ie a y, open s anda d,
uncomp essed, unenc yp ed
17
Plain ex
x
doc
Tabula /s uc u ed da a
cs , xml
xls, d a, sa
Images
i , i
gi
Video
mp4
wm
Fo ma ed ex
PDF/A, docx, xml
doc, pp , PDF
Audio
wa , mp3
O e iew:
•h ps://kos -
ceco.ch/cms/kad_main_de.
h ml
•h ps://www.lz .n w/da ei
o ma e/
Requi emen s o ep oducibili y and
eplicabili y: FAIR Da a
Resea ch Da a Managemen Semina s: Replica ion 18
Gi Book Bo , Lambe Helle , da awomanHUB, mcancellie i, Bianca K ame , Tony Ross-Hellaue , ilabas ida, heleneb , Ped o Fe nandes,
& Jon Tennan . (2018). Open Science T aining Handbook (1.1). Zenodo. h ps://doi.o g/10.5281/zenodo.1212538
Replica ion package
19
Ma ix de ining ep oducible esea ch ( om The Tu ing Way)
Replica ion
package
Reasons and incen i es o
ep oducibili y and eplicabili y II: mo e
easons and incen i es!
Policies o esea ch ins i u ions:
Example: Code o Good Resea ch P ac ice, Uni e si y o Mannheim (2023)
•“S udies mus be eplicable. The e o e, publica ions mus include a
comple e and de ailed desc ip ion o he me hods o da a collec ion, he
da a and so wa e used, he s a is ic analysis and he esul s in o de o
allow o e i ica ion o he esul s h ough eplica ion.”
Resea ch Da a Managemen Semina s: Replica ion 20
Sou ce: h ps://www.uni-mannheim.de/en/ esea ch/ou - esea ch/good- esea ch-p ac ice/
Reasons and incen i es o
ep oducibili y and eplicabili y II: mo e
easons and incen i es!
Policies o jou nals
•Mo e and mo e jou nals
and publishe s a e
eques ing ha aw da a
and code be made a ailable
when he a icle is
published
Resea ch Da a Managemen Semina s: Replica ion 21
Sou ce: Vlaeminck (2021, p. 11)
Reasons and incen i es o
ep oducibili y and eplicabili y II: mo e
easons and incen i es!
Policies o jou nals
Resea ch Da a Managemen Semina s: Replica ion 22
Sou ce: Vlaeminck (2021, p. 13)
Replica ion package: Componen s o a
(gene ic) eplica ion package
I packed my eplica ion package and in i I pu …
•Da a
•Code/Analysis Sc ip s
•Me ada a
•Readme ile
•Licensing
Resea ch Da a Managemen Semina s: Replica ion 23
Replica ion package: Componen s o a
(gene ic) eplica ion package
•Da a
–Raw da a
–(Mos ly) no p ocessed da a
•Code/Analysis Sc ip s
–Sc ip s o code iles (e.g., Py hon, R, S a a, SPSS, MATLAB) ha we e used o c ea e and
p ocess he da ase as well as analyze he da a
•Me ada a
–De ailed desc ip ions o he da a iles, a iables, and he s uc u e o da ase s
(codebooks)
–In o ma ion on how he da a was collec ed, any p ep ocessing, and how a iables we e
cons uc ed
–In o ma ion on p o enance, such as p ojec in o ma ion, consen o ms and e hics
documen a ion, i applicable
24
Replica ion package: Componen s o a
(gene ic) eplica ion package
•Readme ile
–Ins uc ions o unning he analysis, explaining wha each ile does, how o execu e he
sc ip s, and how o in e p e he esul s.
–A b ie summa y o how o ep oduce he analysis s ep-by-s ep.
•Licensing
–A clea s a emen abou he use igh s o he eplica ion package, speci ying unde which
e ms o he s can euse he da a and sc ip s.
–CC0: Wo ks unde a CC0 license can be eely used, modi ied, and sha ed wi hou a ibu ion
–CC BY: Allows copying, sha ing, and adap ing a wo k, including comme cially, wi h c edi o
he o iginal au ho
–CC BY-NC: Allows copying, sha ing, and adap ing a wo k o noncomme cial use wi h c edi
o he o iginal au ho (howe e NC is no ideal and should be used wi h cau ion)
–CC is no designed o so wa e! Use GNU Gene al Public License o o he sui able licenses
25
Replica ion package: Summa y
Resea ch Da a Managemen Semina s: Replica ion 32
Checklis :
Da a
Code/Analysis Sc ip s
Me ada a
Read
me ile
Licensing
Thank you e y much o you a en ion!
Resea ch Da a Cen e a he Uni e si y Lib a y:
o schungsda[email p o ec ed]
D . Phil Kolbe:
phil.k[email p o ec ed]
33
Resea ch Da a Managemen Semina s: Replica ion
Sou ces
Cold Sp ing Ha bo Labo a o y Lib a y (n.d.): Resea ch Da a Managemen : Folde S uc u e. LibGuides. Re ie ed No embe 04,
2025, om h ps://cshl.libguides.com/c.php?g=1420116&p=10531416
Gi Book Bo , Lambe Helle , da awomanHUB, mcancellie i, Bianca K ame , Tony Ross-Hellaue , ilabas ida, heleneb , Ped o
Fe nandes, & Jon Tennan . (2018). Open Science T aining Handbook (1.1). Zenodo. h ps://doi.o g/10.5281/zenodo.1212538
Schenk, Pa ick, Vanessa A. Mülle , Luca Keise (2024): Social S a us and he Mo al Accep ance o A i icial In elligence. Sociological
Science 11:989-1016. DOI: 10.15195/ 11.a36
Schenk, P., Mülle , V., Keise , L., & Abend, G. (2024): Reposi o y Social S a us and he Mo al Accep ance o A i icial In elligence
[Da a se ]. Uni e si y o Luce ne. h ps://doi.o g/10.5281/zenodo.13850549
Sena e o he Uni e si y o Mannheim (2023): Code o Good Resea ch P ac ice a he Uni e si y o Mannheim. Re ie ed
No embe 04,2025, om h ps://www.uni-mannheim.de/en/ esea ch/ou - esea ch/good- esea ch-p ac ice/
The Tu ing Way Communi y (2024): O e iew o Rep oducible Resea ch. De ini ions. Re ie ed No embe 04, 2025, om
h ps://book. he- u ing-way.o g/ ep oducible- esea ch/o e iew/o e iew-de ini ions/
Vlaeminck, S en (2021) : Dawning o a New Age? Economics Jou nals’ Da a Policies on he Tes Bench, LIBER Qua e ly: The Jou nal
o he Associa ion o Eu opean Resea ch Lib a ies, ISSN 2213-056X, LIBER (Associa ion o Eu opean Resea ch Lib a ies), Den
Haag, Vol. 31, Iss. 1, pp. 1–29,h ps://doi.o g/10.53377/lq.10940
34
Resea ch Da a Managemen Semina s: Replica ion
