Fink, Lukas; Ma cus, Jan
A icle — Published Ve sion
Replica ion code a ailabili y o e ime and ac oss ields:
E idence om he Ge man Socio‐Economic Panel
Economic Inqui y
P o ided in Coope a ion wi h:
John Wiley & Sons
Sugges ed Ci a ion: Fink, Lukas; Ma cus, Jan (2024) : Replica ion code a ailabili y o e ime and
ac oss ields: E idence om he Ge man Socio‐Economic Panel, Economic Inqui y, ISSN 1465-7295,
Wiley, Hoboken, NJ, Vol. 63, Iss. 2, pp. 357-386,
h ps://doi.o g/10.1111/ecin.13267
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/319366
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h p://c ea i ecommons.o g/licenses/by-nc/4.0/
Recei ed: 31 Janua y 2024
-
Accep ed: 16 Oc obe 2024
DOI: 10.1111/ecin.13267
ORIGINAL ARTICLE
Replica ion code a ailabili y o e ime and ac oss ields:
E idence om he Ge man Socio‐Economic Panel
Lukas Fink
1
|Jan Ma cus
1,2,3
1
School o Business and Economics, F eie
Uni e si ä Be lin, Be lin, Ge many
2
Be lin School o Economics, Be lin,
Ge many
3
IZA Bonn, Bonn, Ge many
Co espondence
Jan Ma cus, School o Business and
Economics, F eie Uni e si ä Be lin,
Ga ys . 21, Be lin 14195, Ge many.
Email: [email p o ec ed]
Abs ac
P o iding eplica ion code is an inexpensi e way o acili a e ep oducibili y.
Howe e , li le is known abou he ex en o eplica ion code p o ision.
The e o e, we examine he a ailabili y o eplica ion code o o e 2500 pee ‐
e iewed a icles based on he Ge man Socio‐Economic Panel (SOEP), one o
he mos widely used da ase s in economics and o he social sciences. We ind
ha only 6% o SOEP‐based s udies ha e code a ailable, bu ha his p o-
po ion has inc eased sha ply o e ime. We p o ide e idence ha he inc ease
in code p o ision is d i en by echnological ad ances, indi idual esea che
ini ia i es, and jou nal policies.
KEYWORDS
code a ailabili y, jou nal policies, eplica ion code, ep oducibili y, SOEP
JEL CLASSIFICATION
A12, C80, C88
1
|
INTRODUCTION
The ep oducibili y o scien i ic esul s is an essen ial pa o he scien i ic me hod. I ensu es ha esea ch can be
independen ly e i ied and p omo es us in scien i ic esul s. Rep oducibili y is pa icula ly impo an in economics
because he esul s o economic s udies a e used o in o m policy decisions and, hence, may ha e di ec consequences
o he li es o many indi iduals. A s udy is compu a ionally ep oducible, when o he esea che s, using he same da a
and code, ge he same esul s as he o iginal s udy (Pé ignon e al., 2023). Hence, compu a ional ep oducibili y de-
pends on wo c ucial ing edien s: he a ailabili y o da a and he a ailabili y o code, ha is, compu e ins uc ions in
s a is ical so wa e such as S a a, R, o Ma lab.
The p ac ice o making code and da a a ailable has nume ous ad an ages o he scien i ic communi y, in addi ion
o allowing o an easy ep oduc ion o he published esul s. Fi s , i enables o he s o build upon he wo k, os e ing
u he in es iga ion and a mo e comp ehensi e unde s anding o he subjec ma e . Second, e o s in he code can be
Abb e ia ions: DIW, Ge man Ins i u e o Economic Resea ch; JIF, jou nal impac ac o ; OSF, Open Science F amewo k; SOEP, Ge man Socio‐
Economic Panel; TOP, T anspa ency and Openness P omo ion.
Managing Edi o : Fa asa Bokha i
This is an open access a icle unde he e ms o he C ea i e Commons A ibu ion‐NonComme cial License, which pe mi s use, dis ibu ion and ep oduc ion in any
medium, p o ided he o iginal wo k is p ope ly ci ed and is no used o comme cial pu poses.
© 2024 The Au ho (s). Economic Inqui y published by Wiley Pe iodicals LLC on behal o Wes e n Economic Associa ion In e na ional.
Economic Inqui y. 2025;63:357–386. wileyonlinelib a y.com/jou nal/ecin
-
357
mo e easily ound and co ec ed, s eng hening he sel ‐co ec ion ea u e o he scien i ic sys em.
1
Thi d, i acili a es
knowledge dissemina ion as o he s can gain deepe insigh s in o he applied me hods and coding p ac ices om he
p o ided code. Fou h, i migh se e as a de e en agains aud and ques ionable esea ch p ac ices like p‐hacking, as
esea che s know ha hei code migh be exposed o sc u iny. Fi h, i can se e as a signal ha he esea che s did no
engage in ques ionable esea ch p ac ices. Six h, while ep oducibili y is o en seen as a a he low ba and “a minimum
c i e ion o scien i ic c edibili y” (Fiša e al., 2024, p. 1343), i is an impo an basis o checking whe he esul s a e
obus o al e na i e me hods and analy ical choices, and whe he esul s can be eplica ed using di e en da a.
The e o e, inc easing he anspa ency o esea ch by making code publicly a ailable could help o e come he cu en
lack o such eplica ion exe cises (Ankel‐Pe e s e al., 2023).
Despi e he ad an ages, i is unclea o wha ex en economis s and esea che s om o he social sciences make da a
and code a ailable. We aim o con ibu e o illing his gap by examining he ex en o code p o ision. Hence, we ocus on
he ing edien o ep oducibili y ha is a guably easie o achie e: Basically, e e y empi ical esea che can sha e code,
bu no e e yone can sha e da a, as esea che s a e equen ly p ohibi ed om sha ing analyzed da a due o legal and
con iden iali y easons. This is especially ele an in economics, whe e seconda y analysis o da a collec ed by o he s, such
as go e nmen al o ganiza ions o gene al in e es su eys, is common and only he p ima y da a p oduce ha e he igh o
edis ibu e he da a. Fu he , p o iding code is compa a i ely inexpensi e in e ms o bo h ime and money o e-
sea che s, as i ypically akes less ime han sha ing da a.
2
Fo hese easons, we ocus on a se ing whe e da a a e a ailable
o esea che s wo ldwide a no cos , making he p o ision o code he c i ical ac o o ep oducibili y.
3
To be e unde s and he ex en o code sha ing and i s de elopmen o e ime, his me ascien i ic s udy ocuses on
code a ailabili y o publica ions based on one o he mos widely used da ase s in economics and o he social sciences,
he Ge man Socio‐Economic Panel (SOEP). Fo his pu pose, we conduc ed an in ensi e sea ch o code a ailabili y o
all SOEP‐based publica ions in pee ‐ e iewed jou nals in he 1985–2021 pe iod. We checked he jou nal websi e o
each o he 2518 a icles, he ac ual a icles, he websi es o 2615 dis inc au ho s who w o e he a icles, and speci ic
online eposi o ies. The collec ion o his a iable is a majo achie emen o his pape . We use his in o ma ion o
add ess se e al esea ch ques ions: Wha ac o s co ela e wi h code p o ision? How has code p o ision e ol ed o e
ime? Is code p o ision ela ed o quali y me ics, such as jou nal impac ac o s and ci a ions? Wha ac o s d i e he
de elopmen o code a ailabili y o e ime? Fo he la e esea ch ques ion, we ga he ed in o ma ion on jou nal
policies on code sha ing om he T anspa ency and Openness P omo ion (TOP) amewo k (Nosek e al., 2015), jou nal
websi es, and a su ey among edi o s o almos 100 jou nals in di e en disciplines.
The SOEP is pa icula ly use ul o s udying code p o ision o se e al easons. Fi s , he SOEP da a a e widely used
by he scien i ic communi y. The SOEP has abou 3500 use s wo ldwide (Goebel e al., 2019), and mo e han 2500 pee ‐
e iewed, SOEP‐based jou nal a icles ha e been published. Second, he SOEP da a a e no only employed by econo-
mis s bu also by esea che s om se e al o he disciplines (in pa icula , sociology, psychology, poli ical science,
demog aphy), allowing us o examine code‐sha ing beha io in di e en disciplines. Thi d, he SOEP g oup p o ides a
lis o all publica ions based on he SOEP da a, which allows us o clea ly de ine ou a ge popula ion. Fou h, he
SOEP makes i possible o s udy de elopmen s in code p o ision o e many yea s as i was es ablished 40 yea s ago.
Fi h, he e is a clea sepa a ion o he da a collec o (a ieldwo k agency on behal o he SOEP g oup a DIW Be lin)
om he da a analys . This means ha he aw SOEP da a can be seen as exogenous o he esea che s analyzing he
da a and ha s a egic decisions in he da a collec ion p ocess a e o li le ele ance. Six h, he SOEP da a can be
downloaded and used ee o cha ge by esea che s a ound he wo ld.
4
This is a majo ad an age o he SOEP (and
simila su ey da a) o s udies on eplicabili y and ep oducibili y, in pa icula compa ed o adminis a i e da a,
which o en can only be analyzed in es ic ed access en i onmen s and o en only a e paymen o da a access.
Ou empi ical esul s show ha he sha e o pee ‐ e iewed SOEP s udies wi h code a ailable is only 6%. Examining
co ela es o code a ailabili y, we ind ha single‐au ho ed a icles and a icles w i en in languages o he han English
a e less likely o p o ide code. In addi ion, code a ailabili y is posi i ely co ela ed wi h highe jou nal impac ac o s
and mo e ci a ions. We also see ha code p o ision is less common o s udies in economics jou nals in ou sample.
When we con ol o yea o publica ion, he co ela ions o code a ailabili y wi h a icle language and single au ho ship
disappea , and he di e ence be ween economics and o he disciplines becomes smalle , sugges ing ha pa s o hese
obse ed co ela ions a e d i en by ime e ec s.
We also documen ha he sha e o SOEP s udies ha p o ide code has inc eased sha ply since he ea ly 2010s,
eaching a sha e o 17.8% in 2021. We a gue ha he inc ease in code p o ision is d i en by a mix o echnological
ad ances, indi idual esea che awa eness, and jou nal policies. Technological ad ances, including he widesp ead use
o indi idual esea che websi es, he es ablishmen o jou nal websi es, and he c ea ion o specialized online
358
-
FINK and MARCUS
eposi o ies, ha e educed he inancial and logis ical ba ie s o code sha ing. We also p esen sugges i e e idence ha
op‐down ini ia i es, such as jou nal policies on code sha ing, ha e played an impo an ole in inc easing code
a ailabili y o e ime. Mo eo e , he ac ha a subs an ial ac ion o a icles only p o ide in o ma ion abou code
a ailabili y on he au ho s' websi es bu no on he jou nal websi e o in he a icle sugges s ha indi idual esea che s'
awa eness o he impo ance o code sha ing also plays an impo an ole. I also highligh s he need o sea ch he
au ho s' websi es o ge a comp ehensi e o e iew o code a ailabili y.
Ou s udy con ibu es o he li e a u e on esea ch anspa ency and open science p ac ices by p o iding he i s
analysis o he de elopmen o code p o ision o e ime and ac oss ields. We pa icula ly con ibu e o wo s ands o
li e a u e in his ield. The i s is he s a e o da a and code‐sha ing p ac ices and ac o s ha a e associa ed wi h i .
P e ious s udies show ha many esea che s a o da a and code sha ing (e.g., Bake , 2016; Ch is ensen e al., 2020;
Feche e al., 2015; Fe guson e al., 2023). Howe e , ew esea che s ha e ac ually sha ed da a o code (Ch is ensen
e al., 2020). While mos s udies ocus exclusi ely on he p o ision o da a (e.g., Alsheikh‐Ali e al., 2011; Feche
e al., 2015; Ghe ghina & Ka sanidou, 2013; Zenk‐Möl gen e al., 2018), some s udies examine bo h da a and code
p o ision oge he . Collia d e al. (2023) ind ha less han 5% o pape s published be ween 2010 and 2020 in one
leading jou nal in inance, he Jou nal o Financial Economics, p o ide some da a o code on he jou nal's websi e.
Ge le e al. (2018) examine 203 empi ical a icles ha we e published in nine leading economics jou nals in he las
h ee issues as o May 2016 and did no con ain es ic ed da a. They ind ha 72% o hese a icles con ain he es i-
ma ion code, bu only 42% con ain he da a manipula ion code ha u ns he aw da a in o he es ima ion da a.
Ha dwicke e al. (2020) su ey 156 empi ical a icles published be ween 2014 and 2017 ac oss he social sciences and
ind ha code is accessible o only one a icle, while Wallach e al. (2018) ind ha none o 104 empi ical biomedical
a icles published be ween 2015 and 2017 men ioned he p o ision o code. Key (2016) s udies he a ailabili y o da a
and code in empi ical a icles published in six leading poli ical science jou nals in 2013/14. 58% o he 494 a icles
con ain bo h da a and code, and a u he 2% o a icles p o ide code bu no da a, gi ing a o al o 60% o a icles
con aining eplica ion code, wi h conside able a ia ion ac oss jou nals. Key (2016) iden i ies jou nal policies as an
impo an p edic o o code and da a a ailabili y. Ge le e al. (2018) examine he websi es o 67 jou nals om di e en
disciplines and ind ha mos o he op poli ical science and economics jou nals equi e au ho s o p o ide eplica ion
code and da a be o e publica ion, while jou nals in sociology and psychology a ely do.
The second s and ela es o he de elopmen o esea ch anspa ency p ac ices o e ime. He e, a gene al inding
in he li e a u e is ha di e en p ac ices o esea ch anspa ency a e on he ise ( o an o e iew on he de elopmen
in economics, see Miguel, 2021). Fo ins ance, in a e ospec i e su ey, he p opo ion o au ho s who epo ed using
open science p ac ices (including p e‐ egis a ion and public sha ing o da a, code, o s udy ins umen s) inc eased o e
ime in bo h economics (Swanson e al., 2020) and o he disciplines (Ch is ensen e al., 2020; Fe guson e al., 2023).
McCabe and Snyde (2014) and Muelle ‐Lange and Wa (2018) ake ad an age o he ac ha some jou nals ha e
swi ched o (hyb id) open access o s udy he e ec o (hyb id) open access on ci a ions. Miguel (2021) p esen s e idence
ha p e‐ egis a ion and p e‐analysis plans a e inc easingly used in expe imen al economics and ha mo e and mo e
s udies egis e e e y yea a he Randomized Con ol T ial Regis y o he Ame ican Economic Associa ion.
5
Mo eo e ,
mo e and mo e jou nals in di e en disciplines allow o “p e‐ esul s e iew,” also known as “ egis e ed epo s,” whe e
a esea ch p ojec is e e eed based on i s esea ch plan and me hodology be o e he esea ch is conduc ed and esul s
a e ob ained (Ha dwicke & Ioannidis, 2018; Miguel, 2021).
Fu he , jou nals ha e implemen ed a ious policies o os e esea ch anspa ency. Fo ins ance, se e al jou nals
in oduced edi o ial s a emen s unde sco ing he impo ance o dissemina ing indings wi h non‐signi ican o null e-
sul s, and he e is some e idence ha hese s a emen s inc eased he sha e o esul s ha a e no s a is ically signi ican
(Blanco‐Pe ez & B odeu , 2020). Addi ionally, many jou nals implemen ed explici code and da a sha ing policies, and
se e al s udies highligh he impo ance o jou nal policies o da a and code p o ision as well as o ep oducibili y
(Ch is ensen e al., 2019; Fiša e al., 2024; Ghe ghina & Ka sanidou, 2013; Muelle ‐Lange e al., 2019; Zenk‐Möl gen
e al., 2018). Fo hese easons, we pu a special emphasis on jou nals' code‐sha ing policies in ou empi ical analyses.
Ou s udy con ibu es o he p e ious li e a u e by ocusing speci ically on code p o ision. While se e al s udies look
a da a sha ing, i is also impo an o s udy code sha ing, he o he acili a o o ep oducibili y, as e e y empi ical
esea che can sha e code bu no necessa ily da a. Ou ocus on publica ions based on he same eely a ailable da a
allows us o compa e code a ailabili y ac oss ields wi hou he dis o ions ha a ise when he use o p op ie a y
da ase s a ies ac oss ields, and when code sha ing beha io is in luenced by he abili y o sha e he da a. In com-
pa ison, he exis ing li e a u e p ima ily examines he a ailabili y o da a and code wi hin a selec ion o jou nals in a
speci ic ield (B odeu , Cook, & Neisse , 2024; Collia d e al., 2023; Key, 2016; Rainey e al., 2024; Wallach e al., 2018).
FINK and MARCUS
-
359
We also con ibu e by applying a b oade sea ch app oach as we do no only look a he jou nal websi es o ep o-
duc ion ma e ial bu also a he au ho s' websi es. Mo eo e , we examine a a he long ime ho izon spanning almos
40 yea s and ocus no only on op jou nals bu conside a much b oade ange o jou nals.
The manusc ip is o ganized as ollows: Sec ion 2desc ibes ou da a and a iables and p esen s desc ip i e s a is ics.
Sec ion 3 epo s empi ical esul s on he s a e o eplica ion code a ailabili y, i s co ela es, and i s e olu ion o e ime,
as well as how code a ailabili y ela es o jou nal impac ac o s and a icle ci a ions. Sec ion 4concludes and p o ides
an ou look o u u e esea ch.
2
|
DATA
2.1
|
Basic da a: SOEP and SOEPli
The Ge man SOEP is one o he la ges and longes ‐ unning household panel su eys in he wo ld: The i s in e iews
we e conduc ed abou 40 yea s ago (in 1984), and cu en ly, abou 30,000 indi iduals in abou 15,000 households a e
in e iewed annually. The SOEP co e s a wide ange o opics, including ques ions on employmen , income, heal h,
educa ion, demog aphy, income, housing, li e sa is ac ion, a i udes, alues, and pe sonali y. Fo all hese easons, i is
no su p ising ha he SOEP is highly ega ded by economis s and esea che s om o he social science disciplines:
SOEP has mo e han 3500 use s wo ldwide, and in he las decade, mo e han 100 pee ‐ e iewed a icles ha e been
published each yea using SOEP da a ‐ and be ween 250 and 300 o he publica ions (including wo king pape s, books,
policy epo s, e c.) (Goebel e al., 2019).
The SOEP g oup a he Ge man Ins i u e o Economic Resea ch (DIW Be lin) main ains a da abase ha aims o
co e all publica ions based on SOEP da a. This da abase, SOEPli (2022), is ou main da ase . I con ains bibliog aphic
in o ma ion abou each publica ion, including i le, ype o publica ion, yea o publica ion, language, jou nal (i
applicable), digi al objec iden i ie (DOI, i a ailable), and he names o he au ho s. Ou e sion o SOEPli is om
June 2022, and we conside SOEP‐based publica ions up o he yea 2021. We ocus ou analyses on pee ‐ e iewed
a icles published in jou nals lis ed in one o he Cla i a e Analy ics ci a ion indices, ha is, he Science Ci a ion In-
dex, he Social Science Ci a ion Index, he A s and Humani ies Ci a ion Index, and he Eme ging Sou ces Ci a ion
Index.
6
We do no include wo king pape s o wo easons. Fi s , we wan o a oid double coun ing since many wo king
pape s end up as pee ‐ e iewed a icles. Second, au ho s may be mo e eluc an o sha e wo king pape code o ea o
in ellec ual p ope y he . In o al, ou da abase consis s o 2518 dis inc pee ‐ e iewed publica ions.
2.2
|
Da a on code a ailabili y
We hen checked each publica ion o publicly a ailable eplica ion code. This was done in ou di e en ways: (i) we
examined he jou nal webpage o each a icle o code a ailabili y; (ii) We e iewed all 2518 a icles o e e ences o
publicly a ailable code
7
; (iii) We sea ched he au ho s' webpages o code a ailabili y
8
; (i ) We sea ched speci ic online
eposi o ies o code a ailabili y.
9
I code was a ailable, we e i ied ha he code is accessible by downloading and
opening i . Ou esea ch ocuses on code ha is publicly a ailable. We did no con ac he au ho s o he a icles.
10
2.3
|
Addi ional da a
We in eg a ed addi ional in o ma ion a he a icle and jou nal le el in o ou da ase . A icle‐le el in o ma ion includes
ci a ions in Google Schola , ob ained h ough web sc aping using he Py hon package Schola ly (Cholewiak
e al., 2021).
11
Jou nal‐le el in o ma ion e e s o jou nal impac ac o (JIF) me ics as well as he p ima y discipline o he
jou nal. We collec ed he 2‐yea and 5‐yea JIF me ics since 1997 om Cla i a e's Mas e Jou nal Lis (2022). The JIF is
me ged by jou nal name and yea o publica ion.
12
To assign he p ima y discipline o each jou nal, we use he clas-
si ica ion o Science‐Me ix (2024), which ca ego izes jou nals in o 174 mu ually exclusi e sub ields, nes ed wi hin 20
mu ually exclusi e ields (A chambaul e al., 2011). We ha e g ouped he sub ields in o i e ca ego ies: Economics,
Sociology, Psychology, O he Social Sciences, and Heal h & O he Sciences.
13
Appendix Table A1 p o ides, o each o
ou i e disciplina y ca ego ies, he names o he jou nals wi h he mos a icles in ou da ase . We me ged he jou nal
in o ma ion om Science‐Me ix based on he In e na ional S anda d Se ial Numbe o he jou nal.
14
360
-
FINK and MARCUS
2.4
|
In o ma ion on jou nal policies
Addi ionally, we included in o ma ion abou jou nals' policies on eplica ion code, ollowing he classi ica ion o he
TOP amewo k (Nosek e al., 2015). A he lowes le el (Le el 0), a jou nal ei he encou ages code sha ing o does no
speci ically men ion i . Mo ing up o Le el 1, a code a ailabili y s a emen is equi ed in he a icle, indica ing whe he
he code is accessible and, i so, whe e i can be ound. A Le el 2, manda o y code sha ing is implemen ed, equi ing he
code o be pos ed o a us ed eposi o y, wi h any excep ions o be iden i ied when he a icle is submi ed. The highes
le el (Le el 3) in ol es a ep oducibili y check, equi ing ha he code be pos ed o a us ed eposi o y and ha epo ed
analyses be independen ly ep oduced be o e publica ion.
The TOP Fac o Da abase (2023) p o ides he cu en le el o eplica ion code policy o 164 o he 478 jou nals
(34%) in ou sample.
15
We sea ched he websi es o he o he 315 jou nals o in o ma ion on code‐sha ing policies.
16
This sea ch esul ed in he cu en s a e o code‐sha ing policies (as o No embe 2023). Howe e , we a e pa ic-
ula ly in e es ed in he his o ical de elopmen o hese policies. We, he e o e, w o e o edi o s o jou nals wi h a code‐
sha ing policy le el o a leas 1 (code a ailabili y s a emen ) o ask (i) whe he ou assessmen was co ec and (ii) when
his pa icula policy (and p e ious code policies) had been adop ed. This included he edi o s o 94 jou nals in ou
SOEP‐based sample. We ecei ed esponses om 67 jou nals (71%) on he cu en s a us o he eplica ion code policy,
and 62 jou nals (67%) p o ided in o ma ion on when he policy was adop ed.
17
Fo he 32 jou nals om which we did
no ecei e adop ion da es, we ied o ob ain he da es using he Wayback Machine (2024) o e ace when he policies
we e i s on he jou nal's websi es. In his way, we ob ained da es o 27 o he 32 jou nals.
18
2.5
|
Desc ip i e s a is ics
Table 1p o ides desc ip i e s a is ics o ge a be e idea o he 2518 SOEP‐based a icles in ou da ase . Economics is
he discipline wi h he mos a icles in ou sample (45%), ollowed by sociology (16%), o he social sciences (15%), heal h
and o he sciences (14%), and psychology (11%). While ou da ase includes publica ions om 1985 o 2021, he numbe
o SOEP‐based publica ions pe yea inc eased s ongly o e ime, om 12 in 1990 (o which 3 in economics) o 26 in
2000 (15 in economics), 118 in 2010 (68 in economics), and 176 in 2020 (52 in economics).
TABLE 1Desc ip i e s a is ics.
Mean SD Min Max
Economics 0.45 0.50 0.00 1.00
Sociology 0.16 0.36 0.00 1.00
Psychology 0.11 0.31 0.00 1.00
O he social sciences 0.15 0.36 0.00 1.00
Heal h and o he sciences 0.14 0.34 0.00 1.00
Publica ion yea 2011.91 7.38 1985.00 2021.00
Jou nal impac ac o (JIF) 1.86 2.20 0.04 40.14
5 Yea JIF 2.60 2.58 0.09 43.77
Ci a ion coun om google schola 106.19 242.25 0.00 4359.00
Language o a icle is English 0.87 0.33 0.00 1.00
Single au ho 0.28 0.45 0.00 1.00
Two au ho s 0.38 0.49 0.00 1.00
Th ee au ho s 0.21 0.41 0.00 1.00
Fou o mo e au ho s 0.13 0.33 0.00 1.00
No e: The able displays mean, s anda d de ia ion, minimum and maximum o di e en a iables in ou sample o 2518 SOEP‐based publica ions.
Sou ce: Own calcula ions, based on SOEPli (2022), Google Schola (2023), and jou nal impac ac o s om Mas e Jou nal Lis (2022).
FINK and MARCUS
-
361
While he a e age jou nal impac ac o is 1.86 (2.60 o he 5‐yea impac ac o ), his is a highly skewed a iable
wi h a maximum o 40 o e all (Na u e in 2016) and a maximum o 9.6 in economics (Jou nal o Poli ical Economy in
2021). Ano he skewed a iable is he numbe o ci a ions, which anges om 0 o 4359, wi h an a e age o 106. Thi y
a icles ha e mo e han 1000 ci a ions, while abou 430 a icles ha e less han 10 ci a ions (including 90 wi h no ci-
a ions). The as majo i y o a icles a e w i en in English (87%). The emaining a icles a e w i en in Ge man (319
a icles; 13%) and F ench (2 a icles; less han 1%). All economics a icles in ou sample a e in English. Abou 28% a e
single‐au ho a icles, while mos a icles ha e wo au ho s (38%). Abou 13% o he a icles ha e ou o mo e au ho s.
The 2518 a icles a e published in 478 di e en jou nals, comp ising 153 economics jou nals and 325 jou nals om
o he disciplines. Th ee hund ed wel e jou nals (65%) ha e a mos wo a icles in ou da ase , while 14% o jou nals
con ibu e 10 o mo e a icles. The 10 mos equen jou nals in ou da ase accoun o 26% o he a icles in ou
sample. Appendix Table A1 lis s he jou nals wi h he mos a icles in ou da ase . While p e ious li e a u e has ocused
mainly on he op jou nals in economics, i is e iden ha we conside a much b oade ange o jou nals. The mos
common economics jou nal in ou da ase is “Labo Economics” (79 a icles), bu ou da ase also includes economics
jou nals anked highe , including 20 a icles om Top‐5 jou nals. As well as jou nals anked lowe . Simila ly, a wide
ange o high o low‐ anked jou nals is included o o he disciplines. Thus, ou esul s do no p ima ily speak o he
si ua ion a he op, bu p o ide a b oade iew o he s a e o eplica ion code a ailabili y.
3
|
RESULTS
This sec ion begins by looking a he cu en s a e o eplica ion code a ailabili y and i s co ela es in Sec ion 3.1. The
ollowing Sec ion 3.2 desc ibes ends in code a ailabili y o e ime and discusses possible explana ions o he obse ed
ends. Sec ion 3.3 examines se e al ac o s ela ed o code a ailabili y simul aneously, while Sec ion 3.4 explo es he
ela ionship be ween code a ailabili y and quali y me ics. Finally, Sec ion 3.5 discusses esul s ela ed o a mo e
igo ous measu e o code a ailabili y.
3.1
|
Replica ion code a ailabili y and i s co ela es
Ac oss all disciplines, we ound eplica ion code o 151 o he 2518 SOEP‐based publica ions (6%). Figu e 1shows he
a ailabili y o code by discipline: We ob ained eplica ion code o 43 o he 1132 SOEP‐based a icles published in
economics jou nals (3.8%) and o 108 o he 1386 a icles published in jou nals o o he disciplines (7.8%). The sha e o
a icles wi h a ailable code is lowe in economics han in he o he ou disciplina y ca ego ies in ou SOEP‐based
sample. Psychology has he highes sha e o a icles wi h a ailable code (almos 10%), ollowed by o he social sci-
ences (9%).
We also ind ha a icles w i en by a single au ho a e less likely o con ain code—and his is ue in bo h eco-
nomics and o he disciplines (see Appendix Figu e A1). One possible eason o his obse a ion is ha in mul i‐au ho
collabo a ions, i is mo e likely ha a leas one au ho has a pe sonal websi e and places impo ance on ep oducibili y.
Ano he explana ion could be ha single‐au ho a icles a e less likely o be published in jou nals wi h s ic e code‐
sha ing policies. A u he explana ion is ha he impo ance o ep oducibili y becomes mo e appa en in mul i‐
au ho collabo a ions. In such collabo a ions, ensu ing ha all au ho s ob ain iden ical esul s by unning he code
on hei espec i e compu e s becomes a mo e s aigh o wa d ask, unde sco ing he impo ance o ep oducibili y.
This would also align wi h he obse a ion o He be e al. (2021) ha a icles wi h mo e au ho s end o ha e be e
code documen a ion. Fu he mo e, code is mo e likely o be a ailable i he a icle is w i en in English (see Figu e A2
in he Appendix). Sec ion 3.3 examines whe he hese bi a ia e co ela ions also hold when con olling o o he ac o s,
including publica ion yea dummies and indica o s o he jou nal's code‐sha ing policy.
Nex , we look mo e closely a he a icles ha p o ide code. The majo i y o a icles wi h code p o ided use S a a
(87%), a p op ie a y so wa e, ollowed by R wi h 21% (see Appendix Figu e A3).
19
SPSS, MATLAB, MPLUS, and SAS
each accoun o less han 4% o s udies wi h code. These p opo ions a e oughly compa able o he igu es om he
2018 SOEP use su ey, which asked esponden s abou he s a is ical packages hey use o analyze SOEP da a: Abou
77% use S a a, 24% use R, 23% use SPSS, 11% use o he (B i zke & Schupp, 2019).
20
Simila ly, in a collec ion o mo e
han 8000 economic a icles wi h eplica ion packages, K anz (2023) inds ha S a a is he mos commonly used
362
-
FINK and MARCUS
so wa e package (71.6%), ollowed by Ma lab (24.5%) and R (9.8%). Vilhube (2020) epo s simila sha es o a icles in
he jou nals o he Ame ican Economic Associa ion.
In bo h economics and o he disciplines, mos s udies ha p o ide code use he jou nal websi e o p o ide in o -
ma ion abou code a ailabili y (see Figu e 2).
21
Howe e , o a sizable p opo ion o s udies, code a ailabili y is only
disclosed ia he au ho s' websi es (30% in economics and 18% in o he disciplines), highligh ing he impo ance o also
checking au ho s' websi es o code a ailabili y: Looking only a jou nal websi es would lead o a subs an ial unde -
es ima ion o code a ailabili y. In gene al, only a ew s udies dissemina e he code exclusi ely ia online eposi o ies,
wi hou e e ing o he a ailabili y o he code ia he jou nal o he au ho s' websi es (1 a icle in economics, 4 a icles
in o he disciplines). I is also in e es ing o see ha i seems less common in economics o disclose he a ailabili y o he
code bo h h ough he jou nal and he au ho s' websi es (5% s. 32%). Economics also di e s om o he disciplines in
e ms o whe e he code p o ided is ac ually s o ed (see Figu e A4 in he Appendix). While o he majo i y o s udies in
economics, code can be downloaded di ec ly om au ho o jou nal websi es, in o he disciplines, 49% o s udies wi h
code ely on he eposi o y o he Open Science F amewo k (OSF), compa ed o less han 5% in economics.
22
3.2
|
De elopmen s in code a ailabili y o e ime
Figu e 3displays how he a ailabili y o eplica ion code has changed o e he yea s. I s a s wi h he i s SOEP‐based
pee ‐ e iewed publica ions in 1985 (1989 in economics) and goes up o 2021. The igu e shows se e al s iking pa e ns.
Fi s , by 2012 code is a ailable o only i e s udies in economics (0.85%) and h ee (0.55%) in o he disciplines. In
economics, he i s publica ion wi h a ailable code is om 2007 (Alesina & Fuchs‐Schündeln, 2007), while o e all, he
ea lies code we ob ained is om 1995 (Rend el e al., 1995). He e he SPSS code was p in ed as an appendix o he pape
i sel , highligh ing he challenges esea che s aced in sha ing code be o e he widesp ead use o he In e ne . The i s
digi al code we ound is om 1998 (Kohle , 1998). Second, he igu e e eals ha om 2012 onwa d, he annual sha e o
publica ions wi h code inc eased s ongly o bo h economics and o he disciplines. The inc ease in economics and
o he disciplines om less han 2% in 2012 o 17% and 18% in 2021 sugges s a huge change in code‐sha ing beha io . In
economics, he bigges g ow h happened a e 2019, while in he o he disciplines he s onges inc ease occu ed a ew
yea s ea lie . Economics was lagging behind, bu is ca ching up in ecen yea s.
The b eakdown o he non‐economics ca ego y by discipline e eals ha he inc ease in code p o ision is a gene al
phenomenon and no d i en by a single discipline (see Figu e A5 in he Appendix). Un il a ound 2015, he sha e o
FIGURE 1 Publica ions wi h publicly a ailable code by discipline. This igu e shows he sha e o SOEP‐based publica ions wi h
publicly a ailable eplica ion code among all SOEP‐based publica ions in he espec i e discipline. Sou ce: Own calcula ions, based on
SOEPli (2022) and own esea ch on code a ailabili y.
FINK and MARCUS
-
363
publica ions wi h code was simila ac oss disciplines. Howe e , a e 2015, he inc ease in sociology, psychology, and
o he social sciences was ea lie and s onge han in economics.
Nex , we look a how he mode o code a ailabili y disclosu e and he loca ion o s o age ha e changed o e ime.
The p opo ion o s udies we ound in o ma ion on code a ailabili y h ough he jou nal inc eased s ongly o e he
FIGURE 2 Publica ions wi h publicly a ailable code by mode o disclosu e. This igu e displays he dis ibu ion o me hods used o
in o m o he s abou he a ailabili y o code among all SOEP‐based publica ions wi h publicly a ailable eplica ion code. The le panel
ela es o he si ua ion in economics, while he igh panel ela es o all o he disciplines. Sou ce: Own calcula ions, based on SOEPli (2022)
and own esea ch on code a ailabili y.
FIGURE 3 Publica ions wi h publicly a ailable code o e ime. This igu e shows he sha e o SOEP‐based publica ions wi h publicly
a ailable eplica ion code among all SOEP‐based publica ions in he espec i e yea and discipline. Sou ce: Own calcula ions, based on
SOEPli (2022) and own esea ch on code a ailabili y.
364
-
FINK and MARCUS
om 2007 (12 yea s la e han he i s publica ion wi h any eplica ion code), and he i s publica ion ou side o
economics ha p o ides code acco ding o he s ic e de ini ion is om 2012 (Kohle e al., 2012).
While none o he s udies p o iding code me he s ic e de ini ion in he pe iod be o e 2007, he p opo ion o
s udies wi h code mee ing he s ic e de ini ion emained ela i ely cons an o e ime (75% in 2007–2011%, 76.7% in
2012–2016%, 75.7% in 2017–2021). This sugges s ha code quali y, as measu ed by ou c ude measu e, has no imp o ed
much o e ime. Howe e , i also sugges s ha he inc eased a e o code p o ision o e ime did no come a he
expense o he quali y o he code p o ided. The Online Appendix epea s all ou analyses, his ime using he s ic e
de ini ion o code a ailabili y. Gene ally, he pa e ns a e e y simila o hose wi h ou main de ini ion o code
a ailabili y.
4
|
CONCLUSION
In his amous a icle “Le 's ake he con ou o econome ics” Edwa d Leame (1983) hea ily c i icized he s a e o
empi ical economic esea ch, culmina ing in he quo e “ha dly anyone akes anyone else's da a analysis se iously.” I
nobody akes da a analysis o o he esea che se ious, i is a se ious p oblem o a p o ession ha s ongly elies on
empi ical esul s and calls o e idence‐based policy‐making. Howe e , Ang is and Pischke (2010) a gue ha he
Leame c i ique is no longe jus i ied in empi ical mic oeconomics due o he commi men o igo ous esea ch designs
o iden i ying causal e ec s. This is wha hey label he c edibili y e olu ion in hei ap ly i led a icle “The c edibili y
e olu ion in empi ical economics: How be e esea ch design is aking he con ou o econome ics.” Howe e , ecen
s udies by, o example, Hun ing on‐Klein e al. (2021), B eznau e al. (2022), and Menk eld e al. (2024) highligh a
disconce ing phenomenon: esea che s d awing dispa a e conclusions when analyzing he same da a o answe
iden ical ques ions, e en when hey apply igo ous esea ch designs o he iden i ica ion o causal e ec s. This is due o
he many decisions ha esea che s mus make du ing da a collec ion, p epa a ion, and analysis, also known as he
esea che 's deg ee o eedom. I sugges s ha elying solely on c edible esea ch designs does no ully add ess he
linge ing conce ns oiced by Leame (1983). Making da a and code used in he empi ical analysis publicly a ailable is
one way o make he esea che 's deg ee o eedom mo e anspa en and o examine how c ucial i is o a gi en
esea ch ques ion. Ha ing access o da a and code would allow o he s o check all he de ails o he empi ical analyses
and un addi ional sensi i i y checks.
Agains his backd op, his s udy looks a how o en esea che s sha e he compu e code hey use o hei
empi ical analyses, ocusing on publica ions ha use he SOEP da ase —a widely‐used da a sou ce in economics and
o he disciplines. We sea ched ex ensi ely o ind ou i he code was a ailable o all 2518 a icles using SOEP in pee ‐
e iewed jou nals. We checked he jou nal websi es, he a icles hemsel es, he websi es o he au ho s, and speci ic
online eposi o ies.
The analysis shows ha only 6% o SOEP s udies ha e accessible eplica ion code, a low p opo ion, pa icula ly
conside ing ha code p o ision is c ucial o he ep oducibili y o scien i ic esul s and associa ed wi h li le cos s o
he esea che s. Howe e , he e is a posi i e end as he sha e o s udies wi h a ailable code inc eased s ongly o e
ime in bo h economics and o he disciplines. We a gue ha his inc ease is d i en by a mix u e o h ee ac o s:
Technological ad ances (including websi es o au ho s and jou nals as well as he es ablishmen o speci ic e-
posi o ies), op‐down ini ia i es o jou nals, o example, manda ing code sha ing, and bo om‐up ini ia i es o indi-
idual esea che s, who pos code on hei indi idual websi es. Addi ionally, we ind ha s udies wi h accessible code
end o be published in jou nals wi h highe impac ac o s and ecei e mo e ci a ions, emphasizing he po en ial o
code sha ing as a quali y signal in academic esea ch.
Ou analysis can se e as a i s s ep o u he analyses. We ske ch i e o such u he analyses. Fi s , we examine
whe he s udies p o ide eplica ion code, bu we do no s udy whe he he code ac ually allowed he esul s o be
ep oduced. We ake a i s s ep in his di ec ion, by applying he s ic e de ini ion o code a ailabili y whe e he code
mus load aw SOEP da a. Howe e , i is beyond he scope o his pape o analyze he ep oducibili y o all SOEP
pape s wi h code p o ided. Second, ano he po en ial a enue o u u e esea ch is o go mo e in he di ec ion o
causali y. While we documen co ela ions be ween jou nal policies and code a ailabili y, u u e esea ch could e al-
ua e he impac o code‐sha ing policies on code a ailabili y wi h mo e igo ous me hods o causal in e ence. Simila ly,
u u e s udies could look a he e ec s o code p o ision on ci a ion me ics.
32
Thi d, while in he in oduc ion we a gue
ha code p o ision migh wo k as a de e en agains aud and ques ionable esea ch p ac ices like p‐hacking, u he
analyses could s udy whe he his is ac ually he case.
33
Fou h, u he esea ch could iden i y he speci ic ba ie s ha
FINK and MARCUS
-
371
p e en au ho s om adop ing he habi o making code a ailable, and ways o o e come hese ba ie s (e.g., jou nals
p o iding assis ance in he o m o ins uc ions o ideos on how o p epa e eplica ion packages).
Fi h, u u e s udies could also examine code a ailabili y beyond he SOEP. A possible app oach would be o
in es iga e he de elopmen o code a ailabili y in a icles published in leading jou nals ac oss a ious disciplines.
Howe e , his sampling me hod may no o e a comp ehensi e o e iew o code sha ing as i concen a es solely on he
mos p es igious jou nals. Fu he , in many cases he p o ision o code will no be enough o allow o ep oducibili y as
da a a e missing and he e a e subs an ial di e ences be ween disciplines ega ding he p opo ion o da a ha a e
sha eable wi h he esea ch communi y. While publica ions based on he SOEP a e no a andom sample o s udies in
economics o o he disciplines, a complimen a y s a egy would be o d aw andom samples om publica ions in
di e en disciplines. Howe e , his p ocedu e p esen s se e al challenges. Fi s ly, i is necessa y o e i y whe he he
s udy is an empi ical one and he a es o empi ical s udies di e be ween disciplines. Secondly, he disciplines a y in
e ms o he p opo ion o publica ions based on p ima y o seconda y da a analyses. Thi dly, i is unclea wha he
popula ion is and how o cons uc a sampling ame. By ocusing on he SOEP, we a oid many o hese issues as he
SOEP le els he playing ield as he da ase is publicly a ailable by esea che s a ound he wo ld. In his se ing code
a ailabili y is he mos c ucial ing edien o ep oducibili y.
P o iding eplica ion code will no comple ely sol e all p oblems o ques ionable esea ch p ac ices and scien i ic
malp ac ice. Howe e , i will mi iga e some o hese p oblems and make i easie o new s udies o build on p e ious
ones. In addi ion, code p o ision is a ela i ely inexpensi e measu e: Indi idual esea che s can pa icipa e olun a ily,
and jou nals can adop explici code‐sha ing policies.
Fo jou nals, he cos o implemen ing a Le el‐1 code‐sha ing policy (code a ailabili y s a emen ) is ela i ely low.
Jou nals simply need o upda e hei web pages o explici ly men ion he policy and ensu e ha each accep ed pape
includes a code a ailabili y s a emen . The cos o implemen ing a Le el‐2 policy (manda o y code p o ision) is only
sligh ly highe , as jou nals mus en o ce his policy by e i ying ha he code is pos ed in a us ed eposi o y and ha
he link p o ided is unc ional. A po en ial downside o implemen ing an explici code‐sha ing policy could be ha
ewe esea che s would submi hei wo k o such jou nals. Howe e , we belie e ha his is unlikely because he cos s
o indi idual esea che s a e ela i ely low o bo h Le el‐1 and Le el‐2 policies. Mo eo e , i he policy de e s e-
sea che s who migh engage in audulen beha io , i could be bene icial o he jou nal o ecei e ewe bu highe
quali y submissions. F om a pu e ep oducibili y pe spec i e, a Le el‐3 policy ( ep oducibili y check) is desi able.
Howe e , his is by a he mos expensi e code‐sha ing policy. Ve i ying ha he code allows o ep oducibili y is
cos ly, and many jou nals employ da a edi o s o pe o m hese checks, a conside able expense. In ou su ey, edi o s
om se e al jou nals indica ed ha while hey would like o implemen a Le el‐3 policy, i is simply no inancially
easible. While we belie e ha only selec op‐ ie jou nals can a o d comp ehensi e ep oducibili y checks, all jou nals
can (and should) equi e au ho s o accep ed manusc ip s o make hei code publicly a ailable.
We conclude his pape by a guing o wha we call he C edibili y Re olu ion 2.0. This new pa adigm con ends ha
he me e ocus on esea ch designs is no enough o add ess he c i icisms o Leame (1983). Ins ead, i calls o a
undamen al shi owa d sha ing code and, i possible, da a as he new no mal. In ou iew, he c edibili y lies no only
in he design o he esea ch, bu also in he abili y o pee s o sc u inize e e y de ail, acili a ing collec i e alida ion o
esul s h ough ep oducibili y and anspa ency. P o iding eplica ion code is a low‐cos measu e in his new pa a-
digm, and basically any empi ical esea che can engage in i , pa ing he way o mo e us wo hy empi ical esea ch.
ACKNOWLEDGMENTS
We hank Toman Ba sbai, Ma kus Jän i, and Michael K asnicka o help ul commen s and eedback. We a e also
indeb ed o Niema El Mou i, Ashwin Fe nandez, and Alexande G awe o ou s anding esea ch assis ance. We a e
g a e ul o U a Rahmann o he SOEP g oup o p o iding us wi h he SOEPli da abase and o answe ing ou ques ions
abou SOEPli . A big hank you also goes o all he jou nal edi o s who ook he ime o answe ou ques ions abou he
implemen a ion o speci ic code‐sha ing policies.
Open Access unding enabled and o ganized by P ojek DEAL.
DATA AVAILABILITY STATEMENT
The code and da a o ep oduce all he ables and igu es in he pape a e openly a ailable in he openICPSR eposi o y
a h ps://doi.o g/10.3886/E209316V2 (Fink & Ma cus, 2024).
372
-
FINK and MARCUS
ORCID
Lukas Fink h ps://o cid.o g/0009-0002-8786-0988
Jan Ma cus h ps://o cid.o g/0000-0001-9407-6660
ENDNOTES
1
Fo example, Foo e and Goe z (2008) ound a coding e o in Donohue and Le i (2001)'s s udy on legalized abo ion and c ime, while
He ndon e al. (2013) ound a coding e o in an Excel sp eadshee in Reinha and Rogo (2010)'s o iginal s udy on high public deb and
low g ow h a es. Al hough hese examples o coding e o s ecei ed a lo o a en ion, (unin en ional) coding e o s o en ha e only
mino consequences o he conclusions o an a icle (Lau ina ichyu e e al., 2022).
2
Fo da a sha ing, esea che s may need o label a iables and ca e ully conside which a iables should no be eleased. This may be o
educe he isk o de‐anonymizing speci ic obse a ions o because hey plan o publish ano he pape using he same da a and a iables
ha ha e no been used.
3
One migh conside including all de ails o he empi ical analyses in he a icle as an al e na i e o code p o ision. Howe e , jou nals ha e
limi ed space, and e en he mos ca e ul esea che s ind i challenging o include all he impo an de ails o hei analysis in he a icle
(K ähme e al., 2023). Mo eo e , ou own expe ience in ying o ep oduce esul s o publica ions using only he p o ided da a and
in o ma ion p o ided in he a icle ells us ha his is no only ime‐consuming, bu in mos cases i is impossible o ep oduce he esul s
due o he complexi y o mos empi ical analyses and he many decisions esea che s ha e o ake.
4
Mo e speci ically, esea che s a uni e si ies and esea ch ins i u ions can use he da a o esea ch and eaching pu poses by signing a
da a dis ibu ion con ac , whe ein hey ag ee o comply wi h he da a secu i y and p i acy equi emen s o he Eu opean Gene al Da a
P o ec ion Regula ion. Da a p o ec ion ules di e inside and ou side he EU. Wi hin he EU, use s ha e ull access o he SOEP co e
da ase , ou side he EU, hey ha e access o a 95% sample o his da ase . The co e SOEP da a con ain in o ma ion on he ede al s a e.
Mo e de ailed egional da a a e a ailable ia emo e o on‐si e access. See SOEP G oup (2024) and Vilhube (2023) o mo e in o ma ion.
5
The pu pose o p e‐ egis a ions and p e‐analysis plans is o enhance con idence in a s udy's indings by educing esea che s' deg ee o
eedom and he capaci y o choose esul s selec i ely. While in economics, p e‐ egis a ion e e s o he p ocess o egis e ing and
desc ibing he basic cha ac e is ics o a s udy (usually) be o e i is conduc ed, p e‐analysis plans addi ionally desc ibe in de ail how he
analysis will be pe o med (Miguel, 2021). In economics, p e‐ egis a ion o en comes wi hou a p e‐analysis plan, while in o he disci-
plines, p e‐ egis a ion, by de ini ion, includes a p e‐analysis plan (B odeu , Cook, Ha ley, & Heyes, 2024). The empi ical s udy by
B odeu , Cook, Ha ley, and Heyes (2024) shows ha p e‐ egis a ion educes p‐hacking and publica ion bias only when accompanied by
a p e‐analysis plan.
6
In ou sample, based on he i s ca ego y o Cla i a e's Mas e Jou nal Lis , he Social Science Ci a ion Index accoun s o he majo i y o
a icles (86%), while he Science Ci a ion Index makes up ano he 10%.
7
In pa icula , s uden assis an s e iewed he acknowledgmen s and sea ched he pape o key e ms such as “code,” “ eplica ion,”
“syn ax,” “sc ip ” and “S a a.”
8
Mo e speci ically, ou s uden assis an s checked he pe sonal webpages o , i no ound, he ins i u ional webpages o he i s ou
au ho s o code. This esul s in 2615 dis inc au ho s. We only conside he i s ou au ho s as mo e han ou au ho s a e p esen in
only 6% o s udies. Among hese, a ew s udies ha e a la ge numbe o au ho s, including h ee pape s wi h close o 100 au ho s.
9
These include he GESIS da a a chi e, Ha a d Da a e se, Open Science F amewo k, openICPSR, and Zenodo. We used he sea ch e ms
“SOEP,” “GSOEP,” “Socio‐Economic Panel,” and “Sozio‐oekonomisches Panel.”
10
K ähme e al. (2023) ook a di e en app oach, con ac ing au ho s who had published wi h da a om he Eu opean Social Su ey and
asking i hey would sha e hei code. 37.5% o success ully con ac ed au ho s p o ided code.
11
Ci a ions a e as o No embe 28, 2023.
12
Since JIF in o ma ion is only a ailable om 1997 onwa ds, and some jou nals we e included only in la e yea s, we impu e he impac
ac o o ea lie yea s based on he i s yea a ailable ( his applies o 11% o he a icles in ou samples). We include a missing dummy
equal o one o hese a icles in ou eg essions whe e JIF is he dependen a iable o included as a con ol a iable. In addi ion, 2‐yea
impac ac o s a e no a ailable o some jou nals ha ha e only ecen ly been included in he ci a ion indices. This applies o less han 4%
o he a icles in ou sample. We exclude hese a icles in he eg ession analyses wi h JIF as he dependen a iable o as a con ol
a iable.
13
Speci ically, we ha e assigned he ollowing sub ields o “Economics & Business” o he discipline o “Economics”: “Economics,”
“Econome ics,” “Economic Theo y,” “Finance,” “Ag icul u al Economics & Policy,” “De elopmen S udies,” and “Indus ial Rela ions,”
as well as he sub ield “Heal h Policy & Se ices” o he ield “Public Heal h & Heal h Se ices” (since in ou sample his consis s mainly
o a icles in he Jou nal o Heal h Economics and Heal h Economics). The sub ield “Economics” accoun s o mo e han 84% o he a icles
in he discipline “Economics” in ou sample. We assigned o “Sociology” (discipline), he sub ield “Sociology,” o “Psychology” all
sub ields o he ield “Psychology & Cogni i e Sciences” as well as he sub ield “Psychia y,” o “O he Social Sciences” he ield “Social
Sciences” (excep he sub ield ‘Sociology’) as well as all emaining sub ields o he ield “Economics & Business.” Jou nals in all o he
ields and sub ields we e assigned o he esidual discipline “Heal h and o he sciences.” We decided no o u he subdi ide he dis-
ciplines o ha e a leas 10% o he obse a ions in each ca ego y.
FINK and MARCUS
-
373
14
Me ging was no success ul o se en jou nals (39 a icles) in ou da ase . Fo hese se en jou nals, we impu ed he discipline based on he
i s ca ego y o Cla i a e's Mas e Jou nal Lis (2022). In addi ion, Science‐Me ix does no p o ide a jou nal‐based classi ica ion o 15
mul idisciplina y jou nals in ou da ase (65 a icles, including o example, PLOS ONE, Economics & Human Biology, PNAS, and
Na u e). Again, we assign hese jou nals based on he i s ca ego y o Cla i a e's Mas e Jou nal Lis .
15
The TOP Fac o Da abase has a communi y‐based app oach whe e edi o s and egis e ed use s can submi a jou nal e alua ion, which
Cen e o Open Science s a e iews be o e publishing he sco es ( o mo e in o ma ion, see COS, 2024). Fo all 164 jou nals, he las
upda e o he code‐sha ing policy was in 2021 o la e . The co e age o he jou nals in he TOP Fac o Da abase di e s subs an ially ac oss
disciplines: The TOP Fac o Da abase co e s 69% o psychology jou nals o ou sample, 38% o heal h and o he sciences, 32% o sociology,
27% o economics, and 26% o o he social science jou nals.
16
We ocused on code sha ing policies and conside ed da a sha ing policies only when code, syn ax iles, o compu e p og ams we e
explici ly men ioned as pa o “da a.”
17
We also double checked wi h he da es p o ided in B odeu , Cook, and Neisse (2024) and Ch is ensen and Miguel (2018), which ocused
on da a a ailabili y policies, o 25 and 11 economics jou nals espec i ely. In gene al, we ook he in o ma ion p o ided by jou nal
websi es and edi o s o g an ed and did no check he ex en o which hese policies a e ac ually en o ced (see also he discussion in
Collia d e al., 2023).
18
As a esul , we a e missing he adop ion da es o i e jou nals, co esponding o 68 a icles in ou da ase . We exclude hese 68 a icles
om he analyses ha ely on he implemen a ion da es o code‐sha ing policies.
19
In economics, code is only p o ided in S a a (95%), Ma lab (9%) and R (7%) in ou sample. The pe cen ages add up o mo e han 100%
because some a icles ely on mo e han one so wa e p og am.
20
The sha es a e simila in he 2017 SOEP use su ey wi h 76% S a a, 31% R, 19% SPSS, 10% o he (B i zke & Schupp, 2019).
21
We also conside code ound h ough e e ences in he a icles as ha ing been dissemina ed h ough he jou nal because he a icles a e
accessible h ough he jou nal websi e.
22
The OSF is a ne wo k o esea che s o documen hei esea ch p ojec s, o example, by egis e ing analysis plans and sha ing hei
ma e ial. I also allows a DOI o be assigned o he code p o ided, making i possible o keep he code pe manen ly a ailable.
23
In line wi h hese indings, OSF use s and usage ha e inc eased exponen ially since i s incep ion in 2012 (see Nosek e al., 2022).
24
Miguel (2021, 196) highligh s he impo ance o hese online eposi o ies, saying ha hey “ha e been so success ul ha i is easy o o ge
wha an impo an inno a ion he p o essional cu a ion, s o age, and managemen o esea ch da a and code has been.”
25
The Jou nal o Poli ical Economy implemen ed a simila Le el‐2 policy in 2005 and he Re iew o Economic S udies in 2006. This was
ollowed by he B ookings Pape s on Economic Ac i i y in 2007, he Canadian Jou nal o Economics in 2008, and he Ame ican Economic
Jou nals in 2009.
26
While he uppe panels o Figu e 4show ha some economic jou nals a e a he on ‐ unne s in e ms o code‐sha ing policies, he i s
jou nal wi h an explici code‐sha ing policy was he S a a Jou nal. The jou nal, due o i s ocus on so wa e code, equi es since he i s
issue in 2001 ha so wa e code is made a ailable h ough he jou nal's websi e.
27
The Biome ical Jou nal was, in 2009, he i s jou nal in ou sample o implemen a policy o independen ly ep oducing he esul s o
accep ed manusc ip s p io o publica ion (Ho ne e al., 2016). Howe e , code sha ing is no manda o y: “ he Edi o in Chie no i ies he
au ho s …al eady du ing he e iew p ocess o he jou nal's RR [ ep oducible esea ch] policy and s ongly encou ages he submission o
code and da a o make he a icle ep oducible” (Ho ne e al., 2016, p. 418).
28
The i h a icle is published in he Jou nal o Applied Econome ics, which also s ongly emphasizes he eplicabili y o esul s by o he
esea che s. While he jou nal has a manda o y da a sha ing policy (i he da a a e no con iden ial), code sha ing is only encou aged bu
no manda o y acco ding o he jou nal websi e (Jou nal o Applied Econome ics, 2024).
29
We also exclude he 68 a icles published in he i e jou nals, o which we could no ob ain he implemen a ion da es.
30
Mo e speci ically, we es ima e equa ions o he ollowing o m:
yp¼αþx0
pβþX
2021
¼1986
δ ⋅1yea p¼ �þεp;ð1Þ
whe e ypis a bina y a iable indica ing whe he he code is publicly a ailable o publica ion p.xpis he ec o o explana o y a iables
(e.g., indica o s o he jou nal’s discipline); δ ep esen s he coe icien s o a se o dummy a iables co esponding o each publica ion
yea . Finally, εpis he e o e m. We use he e oscedas ici y‐ obus s anda d e o s h oughou ou analysis, as he a iance o εpmay no
be cons an ac oss di e en alues o he explana o y a iables.
31
S udies published in ea ly yea s ha e mo e ime o accumula e ci a ions. The e o e, Figu e 7con ols o yea o publica ion dummies.
32
Going in his di ec ion, o example, Ch is ensen e al. (2019) p o ide some e idence ha da a sha ing inc eases ci a ions when using
changes in da a sha ing policies as ins umen al a iables.
374
-
FINK and MARCUS
33
Fo ins ance, B odeu , Cook, and Neisse (2024) ind e idence ha da a‐sha ing policies do no educe p‐hacking and publica ion bias,
cau ioning agains being oo op imis ic ega ding he posi i e e ec s o manda o y esea ch anspa ency ini ia i es.
REFERENCES
Alesina, A. & Fuchs‐Schündeln, N. (2007) Good‐bye Lenin (o no ?): he e ec o communism on people’s p e e ences. The Ame ican
Economic Re iew, 97(4), 1507–1528. A ailable om: h ps://doi.o g/10.1257/ae .97.4.1507
Alsheikh‐Ali, A.A., Qu eshi, W., Al‐Mallah, M.H. & Ioannidis, J.P. (2011) Public a ailabili y o published esea ch da a in high‐impac
jou nals. PLoS One, 6(9), e24357. A ailable om: h ps://doi.o g/10.1371/jou nal.pone.0024357
Ang is , J.D. & Pischke, J.‐S. (2010) The c edibili y e olu ion in empi ical economics: how be e esea ch design is aking he con ou o
econome ics. The Jou nal o Economic Pe spec i es, 24(2), 3–30. A ailable om: h ps://doi.o g/10.1257/jep.24.2.3
Ankel‐Pe e s, J., Fiala, N. & Neubaue , F. (2023) Do economis s eplica e? Jou nal o Economic Beha io & O ganiza ion, 212, 219–232.
A ailable om: h ps://doi.o g/10.1016/j.jebo.2023.05.009
A chambaul , É., Beauchesne, O.H. & Ca uso, J. (2011) Towa ds a mul ilingual, comp ehensi e and open scien i ic jou nal on ology. In:
P oceedings o he 13 h in e na ional con e ence o he in e na ional socie y o scien ome ics and in o me ics, olume 13, pp. 66–77.
Du ban, Sou h A ica.
Bake , M. (2016) 1,500 scien is s li he lid on ep oducibili y. Na u e, 533(26), 353–366. A ailable om: h ps://doi.o g/10.1038/533452a
Be nanke, B.S. (2004) Edi o ial s a emen . The Ame ican Economic Re iew, 94(1), 404.
Blanco‐Pe ez, C. & B odeu , A. (2020) Publica ion bias and edi o ial s a emen on nega i e indings. The Economic Jou nal, 130(629), 1226–
1247. A ailable om: h ps://doi.o g/10.1093/ej/ueaa011
B eznau, N., Rinke, E.M., Wu ke, A., Nguyen, H.H., Adem, M., Ad iaans, J., e al. (2022) Obse ing many esea che s using he same da a
and hypo hesis e eals a hidden uni e se o unce ain y. P oceedings o he Na ional Academy o Sciences, 119(44), e2203150119.
A ailable om: h ps://doi.o g/10.1073/pnas.2203150119
B i zke, J. & Schupp, J. (Eds.) (2019) SOEP Wa e Repo 2018. Be lin. A ailable om: h ps://www.diw.de/documen s/publika ionen/73/
diw_01.c.632608.de/wa e_ epo _2018.pd
B odeu , A., Cook, N. & Neisse , C. (2024) P‐hacking, da a ype and da a‐sha ing policy. The Economic Jou nal, 134(659), 985–1018. A ailable
om: h ps://doi.o g/10.1093/ej/uead104
B odeu , A., Cook, N.M., Ha ley, J.S. & Heyes, A. (2024) Do p e‐ egis a ion and p e‐analysis plans educe p‐hacking and publica ion bias?
E idence om 15,992 es s a is ics and sugges ions o imp o emen . Jou nal o Poli ical Economy: Mic oeconomics, o hcoming, 2(3),
527–561. A ailable om: h ps://doi.o g/10.1086/730455
Came e , C.F., D ebe , A., Fo sell, E., Ho, T.‐H., Hube , J., Johannesson, M., e al. (2016) E alua ing eplicabili y o labo a o y expe imen s in
economics. Science, 351(6280), 1433–1436. A ailable om: h ps://doi.o g/10.1126/science.aa 0918
Came e , C.F., D ebe , A., Holzmeis e , F., Ho, T.‐H., Hube , J., Johannesson, M., e al. (2018) E alua ing he eplicabili y o social science
expe imen s in Na u e and Science be ween 2010 and 2015. Na u e Human Beha iou , 2(9), 637–644. A ailable om: h ps://doi.o g/10.
1038/s41562‐018‐0399‐z
Chang, A.C. & Li, P. (2017) A p eanalysis plan o eplica e six y economics esea ch pape s ha wo ked hal o he ime. Ame ican Economic
Re iew, Pape s & P oceedings, 107(5), 60–64. A ailable om: h ps://doi.o g/10.1257/ae .p20171034
Cholewiak, S.A., Ipei o is, P., Sil a, V. & Kannawadi, A. (2021) Schola ly: simple access o Google Schola au ho s and ci a ion using Py hon.
A ailable om: h ps://gi hub.com/schola ly‐py hon‐package/schola ly
Ch is ensen, G., Da oe, A., Miguel, E., Moo e, D.A. & Rose, A.K. (2019) A s udy o he impac o da a sha ing on a icle ci a ions using
jou nal policies as a na u al expe imen . PLoS One, 14(12), e0225883. A ailable om: h ps://doi.o g/10.1371/jou nal.pone.0225883
Ch is ensen, G. & Miguel, E. (2018) T anspa ency, ep oducibili y, and he c edibili y o economics esea ch. Jou nal o Economic Li e a u e,
56(3), 920–980. A ailable om: h ps://doi.o g/10.1257/jel.20171350
Ch is ensen, G., Wang, Z., Paluck, E.L., Swanson, N., Bi ke, D., Miguel, E., e al. (2020) Open science p ac ices a e on he ise: he s a e o social
science (3s) su ey. Wo king Pape Se ies. Cen e o E ec i e Global Ac ion. (No. 106).
Collia d, J.‐E., Hu lin, C. & Pé ignon, C. (2023) The economics o esea ch ep oducibili y. HEC Pa is Resea ch Pape , FIN‐2019‐1345
(Decembe 20, 2023).
COS (2024) New measu e a es quali y o esea ch jou nals’ policies o p omo e anspa ency and ep oducibili y. A ailable om: h ps://
www.cos.io/abou /news/new‐measu e‐ a es‐quali y‐ esea ch‐jou nals‐policies‐p omo e‐ anspa ency‐and‐ ep oducibili y [Accessed 23
May 2024].
Donohue, J.J.I. & Le i , S.D. (2001) The impac o legalized abo ion on c ime. Qua e ly Jou nal o Economics, 116(2), 379–420. A ailable
om: h ps://doi.o g/10.1162/00335530151144050
Eubank, N. (2016) Lessons om a decade o eplica ions a he Qua e ly Jou nal o Poli ical Science. PS: Poli ical Science and Poli ics, 49(2),
273–276. A ailable om: h ps://doi.o g/10.1017/s1049096516000196
Feche , B., F iesike, S. & Hebing, M. (2015) Wha d i es academic da a sha ing? PLoS One, 10(2), 1–25. A ailable om: h ps://doi.o g/10.
1371/jou nal.pone.0118053
Fe guson, J., Li man, R., Ch is ensen, G., Paluck, E.L., Swanson, N., Wang, Z., e al. (2023) Su ey o open science p ac ices and a i udes in
he social sciences. Na u e Communica ions, 14(1), 5401. A ailable om: h ps://doi.o g/10.1038/s41467‐023‐41111‐1
FINK and MARCUS
-
375
Fink, L. & Ma cus, J. (2024) ECIN eplica ion package o “Replica ion code a ailabili y o e ime and ac oss ields: e idence om he Ge man
Socio‐Economic Panel”. Ann A bo , MI: In e ‐uni e si y Conso ium o Poli ical and Social Resea ch [dis ibu o ], 2024‐09‐25.
A ailable om: h ps://doi.o g/10.3886/E209316V2
Fiša , M., G eine , B., Hube , C., Ka ok, E. & Ozkes, A.I. & Managemen Science Rep oducibili y Collabo a ion (2024) Rep oducibili y in
managemen science. Managemen Science, 70(3), 1343–1356. A ailable om: h ps://doi.o g/10.1287/mnsc.2023.03556
Foo e, C.L. & Goe z, C.F. (2008) The impac o legalized abo ion on c ime: commen . Qua e ly Jou nal o Economics, 123(1), 407–423.
A ailable om: h ps://doi.o g/10.1162/qjec.2008.123.1.407
Ge le , P., Galiani, S. & Rome o, M. (2018) How o make eplica ion he no m. Na u e, 554(7693), 417–419. A ailable om: h ps://doi.o g/
10.1038/d41586‐018‐02108‐9
Ghe ghina, S. & Ka sanidou, A. (2013) Da a a ailabili y in poli ical science jou nals. Eu opean Poli ical Science, 12(3), 333–349. A ailable
om: h ps://doi.o g/10.1057/eps.2013.8
Goebel, J., G abka, M.M., Liebig, S., K oh, M., Rich e , D., Sch öde , C., e al. (2019) The Ge man Socio‐Economic Panel (SOEP). Jah buche
ü Na ionalokonomie und S a is ik, 239(2), 345–360. A ailable om: h ps://doi.o g/10.1515/jbns ‐2018‐0022
Google Schola (2023) Google Schola da abase. A ailable om: h ps://schola .google.com [Accessed 28 No embe 2023].
Ha dwicke, T.E. & Ioannidis, J.P. (2018) Mapping he uni e se o egis e ed epo s. Na u e Human Beha iou , 2(11), 793–796. A ailable
om: h ps://doi.o g/10.1038/s41562‐018‐0444‐y
Ha dwicke, T.E., Wallach, J.D., Kidwell, M.C., Bendixen, T., C üwell, S. & Ioannidis, J.P. (2020) An empi ical assessmen o anspa ency
and ep oducibili y‐ ela ed esea ch p ac ices in he social sciences (2014–2017). Royal Socie y Open Science, 7(2), 190806. A ailable
om: h ps://doi.o g/10.1098/ sos.190806
He be , S., Kingi, H., S anchi, F. & Vilhube , L. (2021) The ep oducibili y o economics esea ch: a case s udy. Banque de F ance Wo king
Pape , (853).
He ndon, T., Ash, M. & Pollin, R. (2013) Does high public deb consis en ly s i le economic g ow h? A c i ique o Reinha and Rogo .
Camb idge Jou nal o Economics, 38(2), 257–279. A ailable om: h ps://doi.o g/10.1093/cje/be 075
Ho ne , B., Schmid, M. & Edle , L. (2016) Rep oducible esea ch in s a is ics: a e iew and guidelines o he Biome ical Jou nal. Biome ical
Jou nal, 58(2), 416–427. A ailable om: h ps://doi.o g/10.1002/bimj.201500156
Hun ing on‐Klein, N., A enas, A., Beam, E., Be oni, M., Bloem, J.R., Bu li, P., e al. (2021) The in luence o hidden esea che decisions in
applied mic oeconomics. Economic Inqui y, 59(3), 944–960. A ailable om: h ps://doi.o g/10.1111/ecin.12992
Jou nal o Applied Econome ics (2024) Jou nal o Applied Econome ics ‐ Au ho Guidelines. A ailable om: h ps://onlinelib a y.wiley.
com/page/jou nal/10991255/homepage/ o au ho s.h ml [Accessed 24 May 2024].
Key, E.M. (2016) How a e we doing? Da a access and eplica ion in poli ical science. PS: Poli ical Science & Poli ics, 49(2), 268–272. A ailable
om: h ps://doi.o g/10.1017/s1049096516000184
Kohle , U. (1998) Zu A ak i i ä de G ünen bei äl e en Wähle n. Kölne Zei sch i ü Soziologie und Sozialpsychologie, 50, 536–559.
Kohle , U., Ehle , M., G ell, B., Heisig, J.P., Radenacke , A. & Wö z, M. (2012) Ve a mungs isiken nach k i ischen Lebense eignissen in
Deu schland und den USA. KZ SS Kölne Zei sch i ü Soziologie und Sozialpsychologie, 64(2), 223–245. A ailable om: h ps://doi.o g/
10.1007/s11577‐012‐0164‐7
K ähme , D., Schäch ele, L. & Schneck, A. (2023) Ca e o sha e? Expe imen al e idence on code sha ing beha io in he social sciences. PLoS
One, 18(8), e0289380. A ailable om: h ps://doi.o g/10.1371/jou nal.pone.0289380
K anz, S. (2023) Usage sha es o p og amming languages in economics esea ch. A ailable om: h ps://sk anz.gi hub.io/ /2023/12/29/
FindingEconomicA icles7.h ml
Lau ina ichyu e, A., Yada , H. & Vasish h, S. (2022) Sha e he code, no jus he da a: a case s udy o he ep oducibili y o a icles published
in he jou nal o memo y and language unde he open da a policy. Jou nal o Memo y and Language, 125, 104332. A ailable om:
h ps://doi.o g/10.1016/j.jml.2022.104332
Leame , E.E. (1983) Le ’s ake he con ou o econome ics. The Ame ican Economic Re iew, 73(1), 31–43.
Mas e Jou nal Lis . (2022) Web o Science mas e jou nal lis by Cla i a e. A ailable om: h ps://mjl.cla i a e.com [Accessed 10 Oc obe
2022].
McCabe, M.J. & Snyde , C.M. (2014) Iden i ying he e ec o open access on ci a ions using a panel o science jou nals. Economic Inqui y,
52(4), 1284–1300. A ailable om: h ps://doi.o g/10.1111/ecin.12064
McCullough, B.D., McGea y, K.A. & Ha ison, T.D. (2006) Lessons om he JMCB a chi e. Jou nal o Money, C edi , and Banking, 38(4),
1093–1107. A ailable om: h ps://doi.o g/10.1353/mcb.2006.0061
Menk eld, A.J., D ebe , A., Holzmeis e , F., Hube , J., Johanneson, M., Ki chle , M. e al. (2024) Non‐s anda d e o s. The Jou nal o
Finance, 79(3), 2339–2390.
Miguel, E. (2021) E idence on esea ch anspa ency in economics. The Jou nal o Economic Pe spec i es, 35(3), 193–214. A ailable om:
h ps://doi.o g/10.1257/jep.35.3.193
Muelle ‐Lange , F., Feche , B., Ha ho , D. & Wagne , G.G. (2019) Replica ion s udies in economics–how many and which pape s a e chosen
o eplica ion, and why? Resea ch Policy, 48(1), 62–83. A ailable om: h ps://doi.o g/10.1016/j. espol.2018.07.019
Muelle ‐Lange , F. & Wa , R. (2018) How many mo e ci es is a 3,000 open access ee buying you? Empi ical e idence om a na u al
expe imen . Economic Inqui y, 56(2), 931–954. A ailable om: h ps://doi.o g/10.1111/ecin.12545
376
-
FINK and MARCUS
Nosek, B.A., Al e , G., Banks, G.C., Bo sboom, D., Bowman, S.D., B eckle , S.J., e al. (2015) P omo ing an open esea ch cul u e. Science,
348(6242), 1422–1425. A ailable om: h ps://doi.o g/10.1126/science.aab2374
Nosek, B.A. & E ing on, T.M. (2017) Rep oducibili y in cance biology: making sense o eplica ions. Eli e, 6, e23383. A ailable om:
h ps://doi.o g/10.7554/eli e.23383
Nosek, B.A., Ha dwicke, T.E., Moshon z, H., Alla d, A., Co ke , K.S., D ebe , A., e al. (2022) Replicabili y, obus ness, and ep oducibili y in
psychological science. Annual Re iew o Psychology, 73(1), 719–748. A ailable om: h ps://doi.o g/10.1146/annu e ‐psych‐020821‐
114157
Open Science Collabo a ion. (2015) Es ima ing he ep oducibili y o psychological science. Science, 349(6251), aac4716. A ailable om:
h ps://doi.o g/10.1126/science.aac4716
Pé ignon, C., Akmansoy, O., Hu lin, C., D ebe , A., Holzmeis e , F., Hube , J., e al. (2023) Compu a ional ep oducibili y in inance: e -
idence om 1,000 es s. HEC Pa is Resea ch Pape , (FIN‐2022‐1467).
Rainey, C., Roe, H., Wang, Q. & Zhou, H. (2024) Da a and code a ailabili y in poli ical science publica ions om 1995 o 2022. SocA Xi
P ep in .
Reinha , C.M. & Rogo , K.S. (2010) G ow h in a ime o deb . The Ame ican Economic Re iew, 100(2), 573–578. A ailable om: h ps://doi.
o g/10.1257/ae .100.2.573
Rend el, U., Wagne , G.G. & F ick, J.R. (1995) Eine S a egie zu Kon olle on Längsschni gewich ungen in Panele hebungen: Das Beispiel
des Sozio‐Oekonomischen Panels (SOEP). Allgemeines S a is isches A chi , 79(3), 252–277.
Rome o, F. (2019) Philosophy o science and he eplicabili y c isis. Philosophy Compass, 14(11), e12633. A ailable om: h ps://doi.o g/10.
1111/phc3.12633
Science‐Me ix. (2024) Science‐Me ix classi ica ion. A ailable om: h ps://science‐me ix.com/classi ica ion/ [Accessed 8 Janua y 2024].
SOEP G oup. (2024) Da a access. A ailable om: h ps://www.diw.de/en/diw_01.c.601584.en/da a_access.h ml
SOEPli . (2022) SOEPli da abase. A ailable om: h ps://www.diw.de/en/diw_01.c.789503.en/publica ions_based_on_soep_da a__soepli .
h ml [Accessed 1 June 2022].
Swanson, N., Ch is ensen, G., Li man, R., Bi ke, D., Miguel, E., Paluck, E.L., e al. (2020) Resea ch anspa ency is on he ise in economics.
AEA Pape s and P oceedings, 110, 61–65. A ailable om: h ps://doi.o g/10.1257/pandp.20201077
TOP Fac o Da abase. (2023) TOP Fac o da abase. A ailable om: h ps:// op ac o .o g/ [Accessed 30 No embe 2023].
Vilhube , L. (2020) Rep oducibili y and eplicabili y in economics. Ha a d Da a Science Re iew, 2(4), 1–39. A ailable om: h ps://doi.o g/
10.1162/99608 92.4 6b9e67
Vilhube , L. (2023) Rep oducibili y and anspa ency e sus p i acy and con iden iali y: e lec ions om a da a edi o . Jou nal o Econo-
me ics, 235(2), 2285–2294. A ailable om: h ps://doi.o g/10.1016/j.jeconom.2023.05.001
Vlaeminck, S. & He mann, L.‐K. (2015) Da a policies and da a a chi es: a new pa adigm o academic publishing in economic sciences? In
Schmid , B. & Dob e a, M. (Eds.) New A enues o Elec onic Publishing in he Age o In ini e Collec ions and Ci izen Science. P oceedings
o he 19 h In e na ional Con e ence on Elec onic Publishing. Ams e dam: IOS P ess, pp. 145–155.
Wallach, J.D., Boyack, K.W. & Ioannidis, J.P. (2018) Rep oducible esea ch p ac ices, anspa ency, and open access da a in he biomedical
li e a u e, 2015–2017. PLoS Biology, 16(11), e2006930. A ailable om: h ps://doi.o g/10.1371/jou nal.pbio.2006930
Wayback Machine (2024) In e ne A chi e ‐ Wayback Machine. A ailable om: h ps://a chi e.o g/web/ [Accessed 15 Janua y 2024].
Zenk‐Möl gen, W., Akdeniz, E., Ka sanidou, A., Naßho en, V. & Balaban, E. (2018) Fac o s in luencing he da a sha ing beha io o e-
sea che s in sociology and poli ical science. Jou nal o Documen a ion, 74(5), 1053–1073. A ailable om: h ps://doi.o g/10.1108/jd‐09‐
2017‐0126
SUPPORTING INFORMATION
Addi ional suppo ing in o ma ion can be ound online in he Suppo ing In o ma ion sec ion a he end o his a icle.
How o ci e his a icle: Fink, L. & Ma cus, J. (2025) Replica ion code a ailabili y o e ime and ac oss ields:
e idence om he Ge man Socio‐Economic Panel. Economic Inqui y, 63(2), 357–386. A ailable om: h ps://doi.
o g/10.1111/ecin.13267
FINK and MARCUS
-
377
APPENDIX A
TABLE A1 Mos equen jou nals in ou sample.
Economics
Labo Economics 79
Jah büche ü Na ionalökonomie und S a is ik 62
Re iew o Income and Weal h 49
Economics Le e s 48
Jou nal o Popula ion Economics 44
Jou nal o Economic Beha io & O ganiza ion 43
Heal h Economics 30
Ge man Economic Re iew 29
Applied Economics 26
Empi ical Economics 25
Economic Jou nal 24
Jou nal o Heal h Economics 22
Eu opean Economic Re iew 21
Jou nal o Human Resou ces 21
Sociology
Kölne Zei sch i u Soziologie und Sozialpsychologie 98
Eu opean Sociological Re iew 94
Zei sch i ü Soziologie 67
Soziale Wel 17
Resea ch in Social S a i ica ion and Mobili y 16
Psychology
Social Indica o s Resea ch 76
Jou nal o Happiness S udies 31
Jou nal o Pe sonali y and Social Psychology 20
Psychology and Aging 20
Jou nal o Resea ch in Pe sonali y 14
O he Social Sciences
Jou nal o Ma iage and Family 29
Zei sch i ü Familien o schung 26
Jou nal o Eu opean Social Policy 18
Small Business Economics 18
Demog aphy 17
Heal h & O he Sciences
AS A‐Ad ances in S a is ical Analysis 35
PLOS ONE 24
378
-
FINK and MARCUS
TABLE A1 (Con inued)
Gesundhei swesen 19
Social Science & Medicine 19
Jou nals o Ge on ology Se ies B‐Psychological Sciences 13
No e: The able lis s, o each o ou i e disciplina y ca ego ies, he names o he jou nals wi h he mos a icles in ou da ase and he co esponding numbe
o a icles.
Sou ce: Own coun ing, based on SOEPli (2022).
FIGURE A1 Code a ailabili y: Single au ho e sus mul iple au ho s. This igu e shows he sha e o SOEP‐based publica ions wi h
publicly a ailable eplica ion code, sepa a ely o a icles w i en by a single au ho and a icles wi h mul iple au ho s. The le panel ela es
o economics, while he igh panel ela es o all o he disciplines. Sou ce: Own calcula ions, based on SOEPli (2022) and own esea ch on
code a ailabili y.
FINK and MARCUS
-
379
FIGURE A2 Code a ailabili y: Language o he a icle. This igu e shows he sha e o SOEP‐based publica ions wi h publicly
a ailable eplica ion code, sepa a ely o a icles in English and in all o he languages (mos ly Ge man). The le panel ela es o economics,
while he igh panel ela es o all o he disciplines. Sou ce: Own calcula ions, based on SOEPli (2022) and own esea ch on code
a ailabili y.
FIGURE A3 So wa e used in he eplica ion packages. This igu e shows he dis ibu ion o s a is ical so wa e p og ams used in
he a ailable code packages in ou sample. The le panel ela es o economics, while he igh panel ela es o all o he disciplines. No e ha he
sha es in each panel do no sum up o 1 as some imes mo e han one s a is ical so wa e is used. Sou ce: Own calcula ions, based on
SOEPli (2022) and own esea ch on code a ailabili y.
380
-
FINK and MARCUS
