Rep oducibili y and Open Science in
Economics
La s Vilhube a
D awing on my expe ience as he Ame ican Economic Associa ion’s da a edi o ,
I examine he cu en s a e o open science in economics as acili a ed by and
ela ed o ep oducibili y. I ouch on he ension be ween accessibili y, sha ing,
and p ese a ion. The guiding heme is he accessibili y o he key ing edien s o
schola ship : manusc ip s, da a, so wa e, and he necessa y echnology o combine
he la e wo in o de o p oduce knowledge. I analyze how economic esea ch
balances openness wi h necessa y es ic ions, pa icula ly ega ding adminis a i e
and con iden ial da a. I a gue ha a la ge deg ee o openness is ne e heless
p esen , wi h ex ensi e ne wo ks ha include housands o esea che s suppo ing
collabo a i e science. I a gue ha esou ce cons ain s, such as so wa e licensing
cos s and compu a ional esou ce equi emen s, pose simila challenges. I illus a e
conc e e bene i s o open science in he economics li e a u e, using ecen a icles. I
w ap up by discussing he s a e o access o scien i ic a icles in economics.
Mo s clés : sciences ou e es; ep oduc ibili é; accès à esou ces
Keywo ds: open science; ep oducibili y; essou ce access
Classi ica ion JEL : A11; A13; B40.
a. Co nell Uni e si y, la s. ilhube @co nell.edu
1
Rep oducibili y and Open Science in Economics
1. INTRODUCTION
As a g adua e s uden in economics a Uni e si é de Mon éal, eading he
economics li e a u e was easy. While he main uni e si y lib a y had all he
ele an subsc ip ions, ou depa men lib a ian, Fe hy Mili, would popula e he
lib a y o he economics depa men wi h mul i-hued ows o wo king pape s.
Mili was also one o he key c ea o s o wha was ini ially known as Wo king
Pape s in Economics (
WoPEc
) and BibEc ( o p in ed wo king pape s) (C uz
e al. [2000]; K ichel [1997]; K ichel e Zimme mann [2009]), popula ing he
la e since 1993 (Bá iz‐Lazo e al. [2012], p. 450). The o e all ne wo k, known
as Resea ch Pape s in Economics (
RePEc
), was bo n con empo aneously wi h
he mo e widely known a Xi (Ginspa g [2011]) and he mo e cen alized Social
Science Resea ch Ne wo k (
SSRN
) (Social Science Resea ch Ne wo k [2025]).
While elec onic wo king pape s we e mos ly ee in hose days ( he e was no way
o pay o hem), Mili’s wo k consis ed o sending ou pos age-paid en elopes
o all o he a ious economics depa men s ha we e publishing he wo king
pape s, and hen ca aloging he incoming p in ed ma e ials elec onically, o
public consump ion. In he end, in o ma ion abou he exis ence o he wo king
pape s was eely a ailable, bu access o (p in ed) wo king pape s s ill equi ed a
small ee o co e he cos o shipping.
I also expe ienced he openness o code sha ing, wi h code samples by p o-
minen au ho s being a ailable o g adua e s uden s, hough disco e y was much
mo e di icul a he ime. The S a is ical So wa e Componen s (
SSC
), p ima ily
bu no exclusi ely o STATA packages, appea ed in 1998 (Cox [2010]; Cox e
Jenkins [2022]), p o iding a con enien and open way o ca alog, dis ibu e, and
p o ide open access o addi ional S a a unc ionali y. 1
Da a sha ing was ha de , o cou se, wi h lo s o loppies
2
being exchanged, bu
also he use o depa men al FTP
3
se e s (Da id Ca d’s collec ion o da a, o he
NBER’s), o e en he eplica ion a chi e o he Jou nal o Applied Econome ics,
ins an ia ed a Queens Uni e si y in 1994 unde he long- unning guidance o
James McKinnon.
4
On he o he hand, adminis a i e da a, such as he F ench
adminis a i e da a used in AKM equi ed a el o Pa is, si ing in a oom wi hou
windows a an assigned ime, and yping he code in o he sys em ha had access.
5
When da a we e a ailable, compu ing was s aigh o wa d : You logged on o
he uni e si y’s big compu e , unning some a ian o Unix, and used wha e e
1
. This kind o unc ionali y was inspi ed by simila unc ionali y a ailable o o he so wa e,
like CPAN o Pe l and CRAN o R, bu no o mos s a is ical so wa e used by economis s.
2. h ps://en.wikipedia.o g/wiki/Floppy_disk
3. h ps://en.wikipedia.o g/wiki/File_T ans e _P o ocol
4
. The JAE a chi e was mig a ed o he ZBW’s a chi es in 2022 and can now be ound a
h ps:
//jou nalda a.zbw.eu/jou nals/jae
, bu legacy iles a e s ill isible as o 2024 a
h p:
//qed.econ.queensu.ca/jae/legacy.h ml.
5
. Fo non-local au ho s, his mean a eling o ex ended ime pe iods as a Ph.D. s uden
(Ma golis) o spending a sabba ical in Pa is (Abowd), nei he o which is a cheap endea o . Bo h we e,
and s ill a e, enjoyable, hough.
2Re ue économique – ol. , no, , p. 1-33
La s Vilhube
so wa e was a ailable. So wa e licenses we e paid by he uni e si y, as was he
compu ing ha dwa e i sel . Lap ops powe ul (and ligh !) enough o do ac ual
wo k we e only hen eme ging.
In 2025, he e a e conce ned discussions abou he cos o publishing academic
a icles, o accessing hose same academic a icles, o he e e inc easing use o
adminis a i e da a (Ca d e al. [2010a,b]; Che y [2012]; Eina e al. [2014]) ha
would appea o be hidden behind insu moun able access es ic ions, he use o
“p op ie a y so wa e”, and he inc easing use o la ge compu ing in as uc u e,
all o which would seem o be es ic ing access o he basic elemen s o conduc ing
esea ch in economics. In his a icle, I will d aw on my expe ience wo king on
many aspec s o inc easing access o da a and ma e ials o all kinds, in pa icula
my ecen expe ience as he inaugu al da a edi o o he Ame ican Economic
Associa ion (
AEA
) (Du lo e al. [2018]), o pain a pic u e o economics in an
e a o open science. How di e en a e ma e s in p ac ice now compa ed o
ha ea ly iew o he ield o economics, back when I was a g adua e s uden ?
In his a icle, I will discuss he cu en s a e o open science in economics as
acili a ed by and ela ed o ep oducibili y. I will ouch on he ension be ween
accessibili y, sha ing, and p ese a ion, and some o he app oaches ha a e being
implemen ed, some imes en a i ely, in economics, and some imes elsewhe e.
My iew will be biased - I am an ac i e pa icipan in his space, p ima ily ia
my cu en appoin men as da a (and ep oducibili y) edi o o he Ame ican
Economic Associa ion, bu also as a pas pa icipan in ne wo ks ha ha e and
os e access, and a esea che and edi o in he space o disclosu e limi a ion.
The guiding heme will be he accessibili y o he key ing edien s o scho-
la ship : manusc ip s (o mo e gene ally, documen s), da a, so wa e, and he
necessa y echnology o combine he la e wo in o de o p oduce knowledge
as published in manusc ip s. My ocus will be on he la e h ee, hough I will
p o ide some obse a ions abou schola ly publishing in he las sec ion. In he
conclusion, I will iden i y a ew a eas whe e he e is (con inued) mo emen
owa ds g ea e openness.
2. CONCEPTS
In o de o w i e abou “Open Science,” a de ini ion is needed. Open science is a
su p isingly di icul e m o de ine p ecisely, and mul iple o e lapping de ini ion
a e commonly e e enced. UNESCO [2022] sees ou componen s o open science :
open scien i ic knowledge (publica ions, da a, code, and eaching ma e ials “openly
a ailable, accessible and eusable o e e yone”), open science in as uc u es
(which encompasses bo h physical in as uc u e such as ins umen s and labo a-
o ies, as well as i ual componen s such as open access publica ion pla o ms),
science communica ion (knowledge ansla ion), and b oad engagemen beyond
he bounda ies o he academy. I also ecognizes he limi a ions o such access in
a ca ea :
Re ue économique – ol. , no, , p. 1-33 3
Rep oducibili y and Open Science in Economics
... human igh s, secu i y, pe sonal p i acy, ... In such cases, i may
s ill be possible o sha e he exis ence o such in o ma ion o sha e i
among ce ain use s who mee de ined access c i e ia.
The Open Knowledge Founda ion (Open Knowledge Founda ion [2024], OKF)
de ines “open” as (my emphasis)
... anyone can eely access, use, modi y, and sha e o any pu pose
(subjec , a mos , o equi emen s ha p ese e p o enance and open-
ness).
Less b oadly, Vicen e-Saez e al. [2018] iden i y a consensus ha de ines open
science as “ anspa en and accessible knowledge ha is sha ed and de eloped
h ough collabo a i e ne wo ks.”
In his a icle, I will ocus on he wha UNESCO 2022 calls open science
“knowledge” and will b ie ly discuss “in as uc u es.” I will highligh how some
elemen s ha e been qui e widesp ead in economics o some ime. I will y o
iden i y limi s o ully open accessibili y, some o which a e in insic o he na u e
o he esea ch conduc ed in economics, and desc ibe how widesp ead such limi-
a ions may be. In pa icula , I will highligh how hose access es ic ions a e no ,
as many hink, an impedimen o open science, in he sense ha a o emen ioned
“collabo a i e ne wo ks” can s ill access hese esou ces.
A key ambigui y will a ise in how big such ne wo ks need o be in o de o
be conside ed “open.” Conside he ealized size o se e al ele an ne wo ks
in economics. The Na ional Bu eau o Economic Resea ch (
NBER
) de ines i s
a ilia ed schola s as a ne wo k :
𝑛 = 1804
as o Janua y 2025, p ima ily in
No h Ame ica (Na ional Bu eau o Economic Resea ch [2025]). Howe e , in
2024, a o al o 2,966 au ho s published 1,223 NBER wo king pape s. J-PAL has
app oxima ely
𝑛 = 1725
a ilia es a 120 uni e si ies on all popula ed con inen s
(Abdul La i Jameel Po e y Ac ion Lab [2025]). Be ween 2001 and June 2024,
he e had been
𝑛 = 2023
esea che s on p ojec s ha used con iden ial U.S. Census
Bu eau in he Fede al S a is ical Resea ch Da a Cen e s (
FSRDC
) (U.S. Census
Bu eau [2024]). In an a e age yea (2013-2023),
𝑛 = 1238
s uden s g adua e om
a U.S. uni e si y wi h a Ph.D. in economics (Na ional Science Founda ion [2024],
Table 1-5). In he app oxima ely 30 yea s since incep ion o
RePEc
,
𝑛 = 526
au ho s om 52 ins i u ions in No way ha e published a pape lis ed on
RePEc
(p esumably in economics) (IDEAS//RePEc [2025]). All o hese a e measu ed
ac oss di e en spa io- empo al dimensions. A e hey la ge? Con ex and pu pose
ma e . Some may in e sec . How many U.S. g adua e s uden s in he pas 10 yea s
ha e published an NBER wo king pape and a e now a a No wegian ins i u ion?
I is ha de o measu e he ac ual key c i e ion : How many people can po en ially
en e he ne wo k, become pa o i , o , as will be he key use o his concep
la e , how many people can po en ially access da a o ce ain ypes, accessible ia
some so o ne wo k. The ac ual size, ela i e o he numbe o po en ial en an s,
is only a p oxy o ha , and o en, a de ailed analysis o en y c i e ia is necessa y.
Fo ins ance, wha does i ake o en e he ne wo k o esea ch da a cen e s (in
he US, in F ance, in Canada, e c.) in o de o be able o access he same da a
as o he s. Finally, i may be ele an o measu e how di e se hese ne wo ks a e.
4Re ue économique – ol. , no, , p. 1-33
La s Vilhube
This ma e s o he abili y o inside s o a ce ain ne wo k o c i icize each o he .
Fo his, size is no a good measu e — e en 𝑛 = 2 may be su icien .
Jou nals play a key ole in his space, and will be an impo an backg ound
o my discussion and expe iences, possibly also my biases. Mos op economics
jou nals ha e a da a (and possibly code) a ailabili y policy.
6
The AEA’s policy was
i s implemen ed in 2005 (Ame ican Economic Associa ion [2005]; Be nanke
[2004]). While he ocus o ea ly a ailabili y policies was on he da a, he code
o en came along o he ide, albei no always in i s mos comple e o m. I am he
AEA
’s inaugu al da a edi o , appoin ed in 2018 (Du lo e al. [2018]). The AEA
implemen ed a new policy in 2019 (Ame ican Economic Associa ion [2019]).
Many o he economics jou nals appoin ed da a edi o s a ound ha ime, and
mul iple jou nals coo dina ed on a common policy co e called Da a and Code
A ailabili y S anda d (
DCAS
) (Ko en e al. [2022]), and e ised hei policies o
align wi h
DCAS
, see Ame ican Economic Associa ion [2024] o he AEA.
7
A
key pa o hese newe policies was inc eased p e-publica ion moni o ing o he
con en o eplica ion packages (Ch is ian e al. [2018]; Du lo e al. [2018]).
One way o s a o mo e away om a bina y pe spec i e o access is o conside
ime as summa y me ic ha cap u es wha is needed in o de o access gene ic
esou ces, whe he da a, manusc ip s, o compu ing esou ces. Time migh be
needed o w i e an applica ion o access da a, o ime migh be needed in o de
o ob ain access o la ge-scale compu ing esou ces. Time migh be needed in
o de o ob ain g an unding ha allows o pu chase such esou ces. I choose
ime, a he han money, as he me ic, since i migh appea o be sligh ly mo e
egali a ian, gi en ha much o science has ( heo e ical) access o subsidies and
g an s. In he o he dimension, he numbe o people who ha e some p obabili y
o accessing he esou ce ( he size o he ne wo k) can be aken as an app oxima e
measu e o openness, ega dless o how in e es ing o aluable hey migh conside
he da a o be. Figu e 1, aken om Vilhube [2023], se es o illus a e his idea,
o access o da a, wi h a ious ins i u ions ha acili a e ha access mapped ou
in o he space o ime s. size o ne wo k. I will e u n o his h oughou he
discussion.
3. DATA ACCESS
One subcomponen o open science, and locus o much a en ion h oughou
he li e a u e in he social sciences, a e “open da a.” This in p inciple easy - why
should he da a used in esea ch no be open? Howe e , he a ious ca ea s ha
6
. Fo a e iew o he his o y o da a and code a ailabili y policies in economics, see Vlaeminck
[2021].
7
. These ini ia i es a e no es ic ed o economics, o cou se. Poli ical science (Basile e al. [2023];
Da a Access & Resea ch T anspa ency (DA-RT) [2014]; Jacoby [2015]), sociology (Sociological
Science [2018]; Weeden [2023]), and gene al ini ia i es like he T anspa ency and Openness P omo ion
(TOP) guidelines (Nosek e al. [2015]).
Re ue économique – ol. , no, , p. 1-33 5
Rep oducibili y and Open Science in Economics
Figu e 1 – Concep ual ade-o be ween numbe o indi iduals accessing da a,
and ime equi ed o do so. Figu e i s published in Vilhube [2023].
policies and p inciples include a e impo an o ecognize. (Open Knowledge
Founda ion [2024], OKF) men ions “ equi emen s ha p ese e p o enance and
openness,” which does no ake in o accoun p i acy. UNESCO [2022] does no e
“human igh s, secu i y, pe sonal p i acy.” On he o he hand, e en much da a
ha is a ailable o almos anybody on he In e ne may no ac ually be “open.”
Conside he S&P 500, iewed in newspape and many websi es (e.g. S&P Dow
Jones Indices LLC [2025]), is no “open da a” because i does no allow o ee
e-use. OKF de ines “open da a” as equi ing machine eadabili y, absence o
licensing cha ges, and ee e-use, bu does no manda e a ailabili y ia download
on he in e ne , absence o all ees, no absence o any echnical measu es, such
as a equi emen o egis e and ag ee o abide by hese ules (Open Knowledge
Founda ion [2024]).
I will discuss wo sub-a eas wi hin his space : Seconda y da a use, and p ima y
da a gene a ion. Much o economic esea ch uses da a collec ed by o he s, such
as su ey o ganiza ions and na ional s a is ical o ices, bu also p i a e company
da a and a ious adminis a i e da a sou ces (“o ganic da a”, G o es [2011a,b]).
P ima y da a gene a ion is mo e equen in beha io al and de elopmen econo-
mics.
Seconda y da a : The da a p oduced by he Uni ed S a es go e nmen a e in
he public domain (i.e., wi hou any es ic ions on usage o a ibu ion), which
makes i “open” in he abo e sense (Copy igh Ac o 1976, Wikipedia [2025]).
Many coun ies ha e swi ched hei go e nmen da a o de aul o open da a
(S a is ics Canada [2012]; UK Go e nmen [2014]). Howe e , many well known
“public-use” da a a e no always “open” : IPUMS has a edis ibu ion es ic ion
6Re ue économique – ol. , no, , p. 1-33
La s Vilhube
(encapsula ed in “Te ms o Use”, no a license), and some geog aphic da a by
in e na ional s a is ical o ices emain unde mo e s ingen licensing equi emen s
in o he coun ies (e.g., Uni ed Kingdom).
Many well-known su ey o ganiza ions impose edis ibu ion and usage es-
ic ions ha a e no consis en wi h he OKF de ini ion. Many such edis ibu ion
es ic ions apply o da ase s wi h mo e de ailed pe sonal in o ma ion collec ed
h ough su eys, and a e mean o ensu e con inued compliance wi h e hical ules
o beha io , o en wi h in o med consen ag eemen s by su ey pa icipan s and
local p i acy laws. No able examples include PSID (Ins i u e o Social Resea ch
[2024]), Wo ld Value Su ey (Hae p e e al. [2024]), Demog aphic and Heal h
Su eys (
DHS
) P og am (DHS P og am [2024]), Ge man Socio-Economic Panel
(Goebel, G abka, Liebig, K oh e al. [2019]; Goebel, G abka, Liebig, Sch öde
e al. [2024]), all o which ha e b oad (cos - ee) usage, condi ional on egis a ion
and compliance wi h usage es ic ions. 8Table 1 shows a ew examples.
Tableau 1 – Example es ic ions
Da ase Va ian Usage. es ic ions.and.jus i ica ion
DHS
No edis ibu ion and access by unau ho ized indi iduals. To ensu e ha p o ide mee s
hos -coun y ag eemen s, such as compliance wi h usage by eal people wi h legi ima e
esea ch pu poses.
WVS Excep o join WVS-EVS iles
The da a can be used eely o non-comme cial pu poses such as esea ch, publica ion,
eaching. Da a edis ibu ion is p ohibi ed : publica ion o he o iginal WVS da ase s a o he
online pla o ms is agains he WVSA Cons i u ion.
PSID Public da a
Use he da a in he PSID da a se s only o scien i ic esea ch and agg ega e s a is ical
epo ing; Make no a emp s o iden i y s udy pa icipan s;
PSID Res ic ed da a
In o de o sa egua d he con iden iali y o esponden s a he highes le el, some da a a e
p o ided only unde condi ions o a es ic ed use con ac , including human subjec s e iew
and da a secu i y plan.
SOEP Eu opean e sion
No edis ibu ion and access by unau ho ized indi iduals. Compliance wi h Ge man Fede al
Da a P o ec ion Ac .
SOEP In e na ional e sion Same as Eu opean e sion, bu 95% sample, and ins i u ional ag eemen equi ed.
The s eps needed o ob ain access ange om click- h ough ag eemen s o
acknowledge compliance wi h CC-BY licenses (consis en wi h open access de i-
ni ions) o he need o w i e a pa ag aph abou he pu pose on how he da a is o
be used (maybe consis en ), o a ious commi men s o no edis ibu e he da a,
all while being cos - ee. The i s h ee ows in Table 1 apply some a ian o
he la e : While anybody can ob ain access, he es ic ion on edis ibu ion is
no consis en wi h de ini ions o open access. Ye he e a e no impedimen s o
ac ually using he da a in esea ch.
The las h ee ows o Table 1, howe e , go u he . Resea che s using SOEP
da a mus sa is y ce ain geog aphical equi emen s, such as p esence o he e-
sea che in he coun ies co e ed by Gene al Da a P o ec ion Regula ion (
GDPR
),
8. I come back o he case o he DHS P og am in Conclusion.
Re ue économique – ol. , no, , p. 1-33 7
Rep oducibili y and Open Science in Economics
o he wise hey can only access a modi ied e sion o he da a. Simila es ic ions
apply o he es ic ed-use da a o many o he su eys, o ins ance, he US-based
Na ional Longi udinal Su ey o You h (
NLSY
). Use s o es ic ed PSID da a
mus comply wi h e en mo e s ingen ules : secu e app o al om hei ins i-
u ion’s e hics boa d, and use o secu e compu ing en i onmen s. Clea ly, hese
es ic ions will inhibi b oade use o he da a pe se. Ye hese es ic ions a e
no globally e y es ic i e : The e a e in Economics alone 1238 new US-based
esea che s being gi en he abili y o eques access o geo- es ic ed NLSY da a
e e y yea : newly min ed Ph.Ds, as no ed ea lie . A simila numbe in he EU
likely ob ain he igh o access he geo- es ic ed SOEP da a as well. Despi e
he es ic ions on he use o SOEP da a, he e a e cu en ly 6,443 pape s ha
in some ashion ha e used he da a.
9
The PSID bibliog aphy
10
lis s o e 1,300
disse a ions and o e 5,300 a icles. In he space depic ed in Figu e 1, access is
e y much owa ds he le o he igu e. 11.
O ganic da a : Howe e , mos economis s, when asked abou da a subjec o
es ic ions, will hink o “p op ie a y da a,” a e m o en applied o any da a ha
may be subjec o es ic ions o use and e-use. Access o mos adminis a i e
da a is ypically no “open” in he sense o OKF, bu a e hey open enough, gi en
he p i acy conce ns ha a e a ached o hese da a? Many ha e a gued ha
access is no b oad enough (Ca d e al. [2010b]; Eina e al. [2014]), while
acknowledging he di icul y o add essing p i acy and secu i y o he da a a
scale. J. M. Abowd e I. M. Schmu e [2018] discuss he challenge o making
he choice o be ween accu acy o (public) s a is ics and da a, and he p i acy
loss inhe en in doing so. Naga aj, Shea s e al. [2020] a gue ha mo e openness
imp o es scien i ic p og ess, and Naga aj e T anche o [2023] s udy his in he
con ex o he US sys em o p o iding access (
FSRDC
). They no e ha 4% o
US-based empi ical au ho s ha e had some access o he FSRDC sys em. I am
no awa e o simila s udies o o he coun ies, such as F ance ( he equi alen
sys em is he Cen e d’accès sécu isé aux données (
CASD
), Gadouche [2019])
o Canada (Cu ie e al. [2015]). In absolu e e ms, hese ne wo ks hos se e al
hund ed esea che s e e y yea . Fo he 781 p ojec s using Census Bu eau da a in
he
FSRDC
, 2,084 esea che s ha e had access o con iden ial da a be ween 1998
9
. Sou ce :
h ps://www.diw.de/en/diw_01.c.789503.en/publica ions_based_on_
soep_da a__soepli .h ml accessed on 2025-02-08.
10
. Sou ce :
h ps://psidonline.is .umich.edu/publica ions/Bibliog aphy/
sea ch.aspx accessed on 2025-02-08
11
. In ac , au ho s some imes o ge o abide by he ules o hese da ase s. As AEA Da a Edi o , I
ge no i ied ia “ ake-down eques s” om da a p o ide s 12-15 imes pe yea , including in 2024 om
he PSID, and ha e pos ed in o ma ion on how o achie e compliance, a leas in some cases, a
h ps:
//aeada aedi o .gi hub.io/pos s/2024-11-01-psid- eques s
. Mos o he cases a ec
pape s published p io o my enu e, because I do ale au ho s o da a use ag eemen iola ions ha I
am awa e o . Ul ima ely, howe e , i is he au ho s’ obliga ion o emain complian wi h such da a use
ag eemen s.
8Re ue économique – ol. , no, , p. 1-33
La s Vilhube
and 2024.
12
Naga aj e T anche o [2023] men ion 861 pape s in scien i ic jou nals.
Simila numbe s can be ob ained o he F ench (6841 esea che s om 1109
ins i u ions on 1797 p ojec s wi h 417 publica ions)
13
and Canadian (2201 ac i e
esea che s om 42 uni e si ies as o Janua y 2025, wi h 3245 pape s published
be ween 2000 and May 2024)
14
ne wo ks.
15
The numbe o publica ions om
access o hese ne wo ks is smalle in absolu e e ms hen hose om PSID and
SOEP, hough likely highe in impac (Naga aj e T anche o [2023]). 16
P ima y da a collec ion : The discussion o choices made by su ey o gani-
za ions should in p inciple be applicable when smalle eams o economis s, no
en i e su ey o ganiza ions, do he p ima y da a collec ion. Simila o su ey
o ganiza ions, such eams ha e o balance he p i acy o hei esponden s wi h
he bene i s o open science, in pa icula he b oade knowledge o be gained
om open access o he da a. Many, so i would seem, p o ide much o he da a
in eplica ion packages, subjec o de-iden i ica ion (see Bja ke u e al. [2021];
Koppe e al. [2020], o examples), hough ypically no wi h s onge disclosu e
a oidance measu es simila o hose employed by s a is ical agencies and la ge
su ey ins i u ions ( o a b ie discussion o he issues and one possible solu ion,
see Mukhe jee e al. [2023]). Fo esea ch eams, e hics boa ds and ins i u ional
e iew boa ds (
IRB
s) ha e a ole o play (G an e al. [2019]), wi h some a guing
e y s ongly ha g ea e a ailabili y o o he s ( hough no blind publica ion o
all da a) is equi ed in o de o maximize he socie al bene i s ha a e he quid
p o quo o he esponden s’ consen o hei p i acy being in aded (G an e al.
[2019]; Meye [2018]). Making such da a as b oadly a ailable, while espec ing
he p i acy o esponden s, is p ecisely wha open access o such da a p omises,
modulo app op ia e access es ic ions o da a use ag eemen s simila o hose
ou lined in Table 1. In gene al, howe e , p ima y da a collec ions do no ha e
access o obus hi d-pa y sys ems ha would allow o access simila o he
access equi ed by PSID and simila o ganiza ions, si ua ed be ween no access and
ully public access. Thus, while access may be eques ed in ad-hoc ashion ia he
o iginal au ho s, his is known o be augh wi h p oblems (Gabelica e al. [2022];
Wa son [2022]). An ideal scena io would see esea che s deposi he da a hey
collec ed in hi d-pa y eposi o ies, which hen handle issues such as e i ying
e hics app o al and secu e access mechanisms. Some ull-se ice eposi o ies,
such as In e -uni e si y Conso ium o Poli ical and Social Resea ch (
ICPSR
)
12
. Own calcula ions based on Census Bu eau da a, see
h ps://labo dynamicsins i u e.
gi hub.io/ s dc-ex e nal-census-p ojec s/.
13
. F om
h ps://www.casd.eu/
and
h ps://www.casd.eu/
ou es-les-publica ions/lis e/0/20/ as o May 2025.
14. P o ided by G an Gibson, CRDCN, on Feb ua y 10, 2025.
15
. The Naga aj e T anche o [2023] numbe only includes pape s published by economis s. O he
numbe s a e coun s o esea che s and publica ions in all disciplines, in non-pee - e iewed publica ions,
and include non-economis s.
16
. Fo an analysis o code a ailabili y o e ime o SOEP-based publica ions, see Fink e al.
[2025].
Re ue économique – ol. , no, , p. 1-33 9
Rep oducibili y and Open Science in Economics
ins i u ions, including in low- and middle-income coun iess (
LMIC
s), may well
no ha e he unds o pu chase p op ie a y so wa e, bu access o compu e s may
be equally cons aining. The empla e README eques s in o ma ion on he ype
o compu e ha was used by he o iginal esea che , o p o ide a benchma k o
u u e e-use s. Acqui ing access o su icien memo y ( andom access memo y
(
RAM
)), s o age, and use o e ime o hose esou ces can be expensi e, e en
when en ing such esou ces in cloud en i onmen s (which e y ew esea che s
appea o be doing). T adi ionally, ha access may be embedded wi hin a single
pu chased compu e , which may ha e (in 2024) a ound 32GB o RAM, 1-2 TB o
s o age, and ha e 4-12 compu e co es a ailable exclusi ely o he owne . Mo e
complex analyses may equi e access o sha ed compu e clus e s (using hund eds
o housands o compu e co es), e y la ge s o age a ays (measu ed in he wo-
o h ee-digi TB ange), and may equi e up o 1024 GB o RAM. Cu ing edge
analyses may equi e specialized chips, such as one o mo e g aphical p ocessing
uni s (
GPU
s), o e en a clus e o
GPU
s. I ha e obse ed analyses ha may un
da a cleaning o da a acquisi ion p ocesses o mon hs a a ime.
The as majo i y o a icles published in economics jou nals usually equi e no
mo e compu ing esou ces han a mode n lap op p o ides, in all he dimensions
enume a ed in he p e ious pa ag aph. In ac , a o mal quan i a i e measu emen
o esou ce usage in economics a icles is su p isingly ha d o ob ain, as mos
esea che s a e no e y good a epo ing he esou ces hey ha e used o conduc
hei esea ch. In pa , his is because measu ing such usage is non- i ial, bu o
a la ge ex en , I pos ula e ha his is because mos esea ch ins i u ions p o ide
such esou ces o hei esea che s in a “con enien way,” and esea che s conduc
esea ch wi hin hose cons ain s. Mo e impo an ly, howe e , i sugges s an
impo an cons ain on how “open” access can be o some i no all economics
esea ch.
Some newe esea ch equi es as ly di e en ypes o esou ces. S udies using
aw sa elli e da a may equi e mo e han 10TB o da a s o age (Khachiyan, Thomas,
Zhou, Hanson, Cloninge , Rosing e A. Khandelwal [2022]; Khachiyan, Thomas,
Zhou, Hanson, Cloninge , Rosing e A. K. Khandelwal [2022]), may need mo e
han 20,000 compu e hou s on a cloud p o ide (Rudik [2020a,b]), o he use o
one (Dell [2024, 2025]) o dozens (Khachiyan, Thomas, Zhou, Hanson, Cloninge ,
Rosing e A. Khandelwal [2022])
GPU
s.
24
Access o he code and da a o he
pape s men ioned is open : The AEA- ela ed eplica ion packages o hese a -
icles a e licensed unde a s anda d C ea i e Commons A ibu ion (
CC-BY
)
license. Some o he da a no included in he Dell [2025] eplica ion package is
on Hugging ace, also unde a
CC-BY
license (Silcock e al. [2024]). Open access
o sa elli e da a is one o he canonical examples o he bene i s o open access
(Naga aj, Shea s e al. [2020]). In hese cases, he compu a ional esou ces may
es ic he bene i s o he open access o da a and code.
24
. As o Janua y 2025, he ype o
GPU
used by Dell [2025], cos s be ween USD 4500 and USD
7700, o be ween 2 and 7 imes as much as a s anda d lap op.
16 Re ue économique – ol. , no, , p. 1-33
La s Vilhube
A e such compu a ional cons ain s a p oblem? No consis en analysis exis s
ha co ela es esou ce equi emen s o academic ou comes such as ci a ions,
p ima ily because i is e y ha d o measu e consis en ly he esou ce equi emen s
o economic a icles. The e y small sample in he p e ious pa ag aph may se e
o illus a e his, bu wi hou con ols o scien i ic me i , is pu ely an indica o .
Silcock e al. [2024] had been downloaded 98 imes in Decembe 2024, six mon hs
a e he a Xi pape associa ed wi h i was published (Silcock e al. [2024]).
Rudik [2020a] has had 1432 iews, 124 downloads o eplica ion package, as he
manusc ip (Rudik [2020b]) has 15 ci a ions. The eplica ion package Khachiyan,
Thomas, Zhou, Hanson, Cloninge , Rosing e A. Khandelwal [2022] has 2124
iews, 175 downloads, while he manusc ip (Khachiyan, Thomas, Zhou, Hanson,
Cloninge , Rosing e A. K. Khandelwal [2022]) has 5 ci a ions (all as o Janua y
2025). Fo compa ison, he a e age a icle in one o he
AEA
’s jou nals has 908
iews and 106 downloads (Vilhube [2025], Table 4).
6. THE BENEFITS OF OPEN SCIENCE IN ECONOMICS
T he discussion abou bene i s o open science o en cen e s a ound da a a ai-
labili y. The Wo ld Bank, in i s annual Wo ld De elopmen Repo , iden i ies da a
a ailabili y — o esea ch, o comme ce, o educa ion — as a key con ibu o ,
and highligh s ha many LMIC con inue o ha e impedimen s o he eliable
p o ision o open access da a (Wo ld Bank [2021], pg. 62). The ( heo e ical)
op imal le el o da a a ailabili y in e sec s wi h p i acy, making he op imal le el
o da a a ailabili y a non- i ial balance be ween public and p i a e bene i s, and
p i a e cos s (J. M. Abowd e I. M. Schmu e [2018]; J. M. Abowd, I. M. Schmu e
e al. [2019]; Acquis i e al. [2016]; Duch-B own e al. [2017]). The mo e ecen
discussions (and cou cases) su ounding he use o da a in he aining o la ge
language models ha e only e-emphasized his ension (Pane ie i [2025]).
A di e en h ead in he discussion b ings up no ma i e easons o anspa ency,
o ins ance a ound Me onian (Me on [1942]) no ms o openness (see Miguel
[2021], o an o e iew). Openness a all s ages o esea ch may ac as a mode a o
o publica ion bias (B odeu e al. [2016]; Miguel [2021])
Some o he bene i s o open science ha e been measu ed indi ec ly, h ough
inc eased ci a ions, say. Some ecen s udies ind some ad an ages o s udies
wi h linked (openly a ailable) da a (Ch is ensen e al. [2019]; Cola izza e al.
[2020]; Piwowa e al. [2013]).
25
The economics li e a u e has emphasized he
bene i s o (balanced) open science. Naga aj, Shea s e al. [2020] discuss he
canonical example o imp o ed da a access h ough ( ee) public-use da a in he
con ex o sa elli e image y. Pa en s can be use ully in es iga ed, since hey a e
bo h openly iewable bu also access-limi ing by hei e y na u e. Economis s
25
. Some o hese s udies need o be aken wi h a g ain o sal , since hey ypically measu e whe he
da a is e e enced, no whe he i is ac ually openly a ailable.
Re ue économique – ol. , no, , p. 1-33 17
Rep oducibili y and Open Science in Economics
Figu e 5 – Ex ac s om Fe guson e al. [2023, Figu es 1 and 7], eused unde
CC-BY 4.0.
ha e looked a how es ic i eness, du a ion, and ype o pa en s a ec scien i ic
p og ess. In gene al, es ic ions educe social wel a e (Mu ay e al. [2016];
Williams [2013]), e en enabling an i-compe i i e beha io ha di ec ly iden i y
wel a e loss (Xie e al. [2020]). The much b oade applicabili y o open science
p ac ices is also much mo e ecen , and as o ye , ha d o measu e. Ne e heless, i
appea s o be widely accep ed in economics, as e idenced by e y s ong posi i e
a i udes documen ed in Fe guson e al. [2023], as selec i ely depic ed in Figu e 5.
The op le pa o Figu e 5 shows beha io (black) and opinions (colo scale) in
ega ds o da a sha ing among economis s, wi h mo e han 50% o economis s
ha ing sha ed da a, bu o e 90% being e y much o mode a ely ( he wo g een
colo s) in a o o sha ing da a. The igh panel illus a es he e olu ion o e ime,
depic ing he p opo ion o social scien is s in he ou disciplines (economics,
poli ical science, psychology, and sociology) who had adop ed an open science
p ac ice as o a pa icula da e, showing again he apid inc ease in ac i e da a
sha ing amongs economis s om a ound 60% in 2011 o he 2020 numbe o
o e 90%.
I wan o add o his discussion o bene i s h ee conc e e case s udies ha
ad ance science in economics, while elying hea ily on he openness o p io
esea ch. The h ee a icles in ques ion le e age he open a ailabili y o code and
da a, wi h licenses ha allow o e-use, o imp o e econome ic me hods o
u u e esea che s. The i s pape elies on he empi ical ecompu a ion o p io
pape s o assess a heo e ically ambiguous po en ial bias in in e ence. The second
pape also ocuses on in e ence, and uses ac ual da a om p e ious pape s o
simula e he ele ance o he impac . Bo h hen p o ide new so wa e (R and S a a
packages), unde open sou ce licenses, o “ ix” he p oblem o u u e s udies.
18 Re ue économique – ol. , no, , p. 1-33
La s Vilhube
The hi d s udy selec s s udies again whe e da a a e a ailable, and le e ages he
cen alized a ailabili y o such eplica ion packages o selec he s udies in hei
sample.
Ro h [2022] uses 12 p e iously published pape s o assess whe he he usual
es s o p e-exis ing di e ences in ends when using di e ence-in-di e ences
me hods a e p ope ly con olling o powe , and he empi ical impac on sub-
sequen in e ence. The heo e ical bias is ambiguous, so an empi ical e alua ion is
necessa y. The s udy bo h le e ages he open a ailabili y o ma e ials in economic
jou nals, bu also illus a es he limi a ions imposed by impe ec adhe ence o
openness. To wi , Ro h w i es ha he sea ched o
“ he ph ase “e en s udy” in pape s published in he Ame ican Economic
Re iew, Ame ican Economic Jou nal : Applied Economics, and
Ame ican Economic Jou nal : Economic Policy be ween 2014 and
June 2018 ... The sea ch e u ned 70 o al pape s ha include a igu e
ha he au ho s desc ibe as an e en -s udy plo .”
bu con inues o hen be limi ed by lack o da a in he majo i y o cases :
“I exclude 43 pape s o which da a o eplica e he main e en -s udy
plo we e una ailable. (Ro h [2022], pg. 307)” 26
While i emains unclea whe he he excluded pape s a e non-complian wi h
he AEA’s policy a he ime, o whe he hey ha e legi ima e easons no o p o ide
he da a (Ro h does no p o ide he aw esul o his sea ch), he pape is able o
make an impo an me hodological poin (723 / 415 ci a ions as o Janua y 2025,
pe Google Schola / OpenAlex) because i is able o ully ecompu e he esul s
in p e ious pape s, apply new es s and me hodologies, and come o meaning ul
ecommenda ions and ools — Ro h p o ides an (open sou ce) R package o
implemen his me hodology.
Chaisema in e Rami ez-Cuella [2024] use open access in o ma ion on RCTs
(AEA egis y) o ind 15 RCTs o a pa icula ype (clus e ed pai ed o small
s a a), o which 4 ha e publicly a ailable da a (and ep oducible a i ac s). They
hen p o ide esul s bo h on simula ions using hese da a, and in pa icula , e-
es ima e he eg essions used in hose s udies and apply hei p oposed solu ion,
showing ha he numbe o signi ican e ec s is educed by one- hi d. In o he
wo k (Chaisema in e D’Haul œuille [2020, 2024]), he esul s om a ious o he
pape s a e also ecompu ed o empi ically demons a e he ele ance o he p o-
posed me hods, and so wa e packages (e.g. Chaisema in, D’Haul oeuille e al.
[2025]) a e de eloped and made openly a ailable.
27
Chaisema in e D’Haul œuille
[2020] has been ci ed be ween 2,600 and 4,600 imes (OA, GS).
Finally, Goldsmi h-Pinkham e al. [2024] in es iga e con amina ion bias (“each
ea men ’s e ec a e con amina ed by noncon ex a e ages o he e ec s o o he
26. He also excludes ano he 15 pape s o easons no ela ed o da a a ailabili y.
27
. No e ha as o Janua y 2025, many o he packages do no ha e an explici open sou ce license
— o any license — applied, a common ea u e o economis s wo king in he open sou ce wo ld. My
p esump ion is ha hey simply assume ha e e ybody knows ha he code is openly a ailable.
Re ue économique – ol. , no, , p. 1-33 19
Rep oducibili y and Open Science in Economics
ea men s”). To do so, hey sea ch he cen alized eposi o y o AEA eplica ion
packages
28
o iden i y packages ha , c ucially, con ain da a. They hen ep oduce
one o each o iginal pape ’s speci ica ions, conduc se e al es s, and conclude
ha “economically and s a is ically meaning ul con amina ion bias [is p esen ] in
wo o he h ee obse a ional s udies while showing no e idence o bias in any
o he expe imen al s udies.” (pg. 4046). Goldsmi h-Pinkham e al. [2024] has
been ci ed be ween 65 and 118 (OA, GS) imes. 29
In each o hese examples, he abili y o access p io da a, code, and in o ma ion
is c i ical o imp o ing u u e scien i ic p og ess. In some cases, i emains limi ed
by bo h his o ical and una oidable limi a ions on openness, as well as scaling
limi a ion based on absence o “push bu on” ep oducibili y.
7. OPEN INFRASTRUCTURE : PUBLICATIONS
The challenges o openly accessible w i en schola ship a e mani old, wi h he
cu en ocus on “Plan S”, mas e publica ion ag eemen s, and in he US, simila
e o s unde he monike o he “Nelson memo” (B aina d [2024]; B aina d e
Kaise [2022]). I no e ha he economics p o ession has a e y long his o y o
making much o he w i en knowledge a ailable a e y low cos ia wo king
pape s (Vilhube [2020b]), wi h he i s wo king pape s a he epu able NBER
wo king pape se ies going back o 1973 (Welch [1973]).
O e he pas eigh yea s, I ha e o ha e had h ee edi o ial appoin men s. I am
he Da a Edi o o he jou nals o he Ame ican Economic Associa ion (
AEA
)
(Du lo e al. [2018]), a column edi o o he open access Ha a d Da a Science
Re iew (
HDSR
) (Vilhube , I. Schmu e e al. [2023]), and un il Janua y 2025,
he join execu i e edi o o he open access and mul i-disciplina y Jou nal o
P i acy and Con iden iali y (
JPC
), o which I con inue o manage he publica ion
in as uc u e (J. Abowd e al. [2025]). I will use each o hese o highligh a
pa icula pa e n in b oadening access o publica ions, wi hou any claim o
gene ali y.
The
AEA
is a no - o -p o i o ganiza ion, as a e many o he lea ned socie ies.
I sel -publishes eigh jou nals, plus he p oceedings o he annual con e ence, wi-
hou elying on a comme cial publishing house. Depending on he measu emen ,
h ee o hese publica ions a e in he op en jou nals in economics (Mogs ad
e al. [2022]). I s publica ion cos s accoun o abou hal o i s o e all ope a ing
expenses, and a e only pa ially o se by di ec ly a ibu able subsc ip ion and
membe ship ees (Che y Bekae , LLP [2024]). In ac , 6 o he op 10 jou -
nals in economics (Mogs ad e al. [2022]) a e published by socie ies (JEL, JEP,
28
. “These s udies we e iden i ied by a sys ema ic sea ch o pape s in he AEA Da a and Code
Reposi o y.” [pg. 4043]
29
. Google Schola ci a ions a e di ec ly epo ed om a iew o he a icle’s page on Google
Schola , and may include ci a ions o mul iple e sions. OpenAlex ci a ions a e he sum o ci a ions o
all eco ded wo ks on OA wi h he same i le and by he same au ho s.
20 Re ue économique – ol. , no, , p. 1-33
La s Vilhube
Econome ica, AER, Res ud, JOLE), some o which ha e as sole o p ima y
pu pose he publica ion o he jou nal.
30
A u he h ee jou nals a e p ima ily
associa ed wi h economics depa men s (QJE, JPE, RESTAT), which a guably
may no be d i en by pu e p o i . The sole ou lie in he op en is he Jou nal o
Financial Economics (
JFE
), which is owned by Else ie , a big comme cial publi-
she . I should be no ed ha he Eu opean Economics Associa ion (
EEA
) se e ed
i s ela ionship wi h Else ie in 2003 o i s o icial jou nal, c ea ing a jou nal ha
is ully owned by he associa ion, adding o he lis o socie y-owned jou nals in
economics (Ti ole e al. [2003]). Access o hese jou nals is gene ally s ill on a
subsc ip ion basis (JEP is he excep ion, being ee o ead), bu gi en he p ima ily
no - o -p o i o ganiza ion o i s owne s, pe sonal subsc ip ions (o en ia socie y
membe ship) a e qui e low, compa ed o jou nals in many o he sciences. Fo
ins ance, as o 2024, a pe sonal subsc ip ion o he Re iew o Economic S udies
is $156 o €141 pe annum; a yea ly membe ship o he
AEA
, p o iding access
o he se en subsc ip ion jou nals and he p oceedings is $25 o s uden s and
esea che s in low-income coun ies, and $150 a he highes pe sonal income
ie . As ou lined ea lie o he AEA, hese subsc ip ion ees co e only a small
ac ion o he p oduc ion cos s. Ne e heless, e en hese (a guably low) cos s do
no sa is y “Plan S” o “Nelson memo” equi emen s, which equi e no access
cos o he end consume , and in he case o “Plan S”, also equi e a libe al license
allowing o e-use. 31
In e es ingly in he con ex o he p e ious sec ions, all o he socie y-owned
jou nals in he p e ious pa ag aph ha e appoin ed da a ( ep oducibili y) edi o s.
32
F om 2018 o Janua y 2025, I was he execu i e edi o o he
JPC
, an open access
mul i-disciplina y jou nal, ha ing aken o e he jou nal om S ephen Fienbe g
(Vilhube [2018]).
33
The jou nal does no cha ge submission ees, bu in 2025, o
co e cos s, i s a ed cha ging publica ion ees. I emains ee o ead. A icles
de aul o a C ea i e Commons A ibu ion-NonComme cial-NoDe i a i es (CC-
BY-NC-ND) license, hough au ho s a e allowed o choose a mo e libe al license,
o ins ance o comply wi h “Plan S” (which does no allow o he “no-de i a i es”
pa ). As execu i e edi o , I was esponsible o all aspec s o unning he jou nal,
no jus inding e e ees o he a icles ha I am esponsible o . The jou nal
is made a ailable h ough open-sou ce so wa e called Open Jou nal Sys em,
hos ed by i s c ea o s a Simon F ase Uni e si y’s Public Knowledge P ojec ,
p ese ed ia indus y-s anda d mechanisms (CLOCKSS, a non-p o i ) in case
30
. JOLE is a bi o an ou lie , in ha one becomes a membe o he Socie y o Labo Economis s
by subsc ibing o he jou nal, a he han he o he way a ound.
31
. “The au ho (s) [...] g an (s) [...] a ee, i e ocable, wo ldwide, igh o access o, and a license
o copy, use, dis ibu e, ansmi and display he wo k publicly and o make and dis ibu e de i a i e
wo ks, [...] subjec o p ope a ibu ion o au ho ship” (Max-Planck-Gesellscha [2023]).
32
. Two addi ional socie ies, no p e iously men ioned, also employ da a edi o s : he Canadian
Economics Associa ion (CEA) and he Wes e n Economics Associa ion In e na ional (WEAI).
33
. Fienbe g, oge he wi h Cyn hia Dwo k and Alan Ka , ounded he jou nal in 2009 (J. M.
Abowd, Nissim e al. [2009]). Fienbe g passed away in 2016 (Sla ko ić e al. [2018]). In 2023, I
ec ui ed Rachel Cummings o join ly manage he jou nal.
Re ue économique – ol. , no, , p. 1-33 21
Rep oducibili y and Open Science in Economics
he jou nal e e needs o shu down, indexed in a a ie y o academic indexes,
including ia assignmen o
DOI
. Copy-edi ing is done h ough a mix u e o
p o essional copy-edi o s and olun ee wo k by edi o s and boa d membe s. Un il
2024, all edi o s, including mysel , we e unpaid, and e e ees a e, like in much o
he publishing indus y, unpaid olun ee s. Ye I did pay bills, o each o he abo e
componen s o a p ope ly managed, indexed, and p ese ed academic jou nal
— and p o essional copy-edi o s and uni e si y s a do no wo k o ee. I am
hus qui e awa e o he absolu e minimum cos o unning a (small) jou nal. O e
he yea s, unding has come om a a ie y o chai ed p o esso ships a Ca negie
Mellon (Fienbe g), Co nell (Abowd), and Ha a d (Dwo k). In o de o make
such unding mo e obus , a non-p o i socie y was c ea ed o be e and mo e
obus ly s uc u e he unding si ua ion (J. M. Abowd, Dwo k e al. [2024]), and
publica ion ees we e implemen ed – a s ep back, o some ex en , on he p io
“comple e” open access model. Time will ell i his will s abilize he unding
si ua ion, while main aining he ounda ional commi men o open access. O he s,
in pa icula Sociological Science, ha e shown ha i is easible o sus ainably
publish high-quali y esea ch
8. CONCLUSION
I ha e desc ibed in his a icle how open da a and code a e in he academic
li e a u e in economics, and how access o so wa e and ha dwa e esou ces can
be limi ing ac o s. Almos all code is openly accessible in op jou nals. The as
majo i y o da a is accessible wi h li le o no e o , and a la ge p opo ion o he
emaining es ic ed da a can be accessed by ne wo ks ha include housands o
esea che s. I p o ide a ew conc e e examples whe e he openness o he da a and
code a ailable allows o he s o di ec ly build on p io esul s. B oade assessmen s
o he bene i s o open access a e mo e di icul o measu e, in pa because he
igh con ols a e ha d o cons uc , in pa because he communi y is s ill only
s a ing o lea n how o echnically le e age he openness in a la ge-scale ashion.
Ne e heless, access o ne wo ks o da a access and inancing emains one o
he key wo ies. I am a egula pa icipan in discussions wi hin h ee esea ch
ne wo ks ha p o ide access o es iced da a (
FSRDC
, Canadian Resea ch Da a
Cen e Ne wo k (
CRDCN
), and
CASD
). Co e discussions cen e a ound equi y
and access, and how o balance hose c i e ia while p ese ing he p i acy o he
esponden s o which hese ne wo ks ac as cu a o s.
One unde -app ecia ed aspec o open access is ha i enables pe sis ence.
Fo any da a ha is subjec o a ga ekeeping mechanism, howe e objec i e,
impa ial, and ligh weigh i may be a he p esen ime, such mechanisms can and
do disappea . E e y one o he ne wo ks ha I men ion abo e has a manda e o
wo k wi hin budge cons ain s, and hose budge s a e de e mined undamen ally
by ex e nal o ces, ypically go e nmen -based unding agencies. A pa icula
s iking example, as o he w i ing o his a icle, is access o da a om he USAID-
unded
DHS
p og am. Da a access o DHS da a was classi ied as ‘mode a ely
easy o ob ain’ (as pe Table 2) un il Feb ua y 2025, as i ook only a day o wo
22 Re ue économique – ol. , no, , p. 1-33
La s Vilhube
o ob ain access subjec o a ligh weigh da a use ag eemen . My eam a he AEA
egula ly wen h ough he p ocess o ob ain da a used by o he esea che s, a
p ocess ha was easy o na iga e e en o an unde g adua e esea che on my
eam. In Feb ua y 2025, he second T ump adminis a ion shu down USAID and
“paused” he DHS p og am, wi h e y li le no ice, and no ecou se. While he
DHS p og am sys em was s ill accessible o hose wi h p io access pe mission in
la e Feb ua y 2025, no u he access eques s we e accep ed. Tha u ns i in o a
‘ e y di icul o ob ain’ da ase .
34
These e en s a e a no e o cau ion ha any kind
o edis ibu ion es ic ion may e y well nega i ely impac u u e a ailabili y o
he esea ch communi y. Whe eas jou nals can subsc ibe o mechanisms ha allow
pas issues o emain accessible e en when he jou nal is shu e ed,
35
and open
access so wa e can be p ese ed by communi ies wi h an in e es in con inued
use,
36
no such mechanism exis s o mos es ic ed-access da a. While open code
may ensu e we can ecompu e, and ad ances in compu a ional in as uc u e will
b ing he cu en cu ing edge in o he space o e e yday-accessibili y, da a ha
a e no open can and will disappea . Tha is conce ning.
Annexe
Tableau A1 – Access ca ego ies and whe he da a can be sha ed p i a ely
Access es ic ions No Yes To al Pe cen
No n/a 50
Ve y Easy o Ob ain 11 7 18 (38.89%)
Mode a ely Easy o Ob ain 3 5 8 (62.50%)
Mode a ely Di icul o Ob ain 16 9 25 (36.00%)
Ve y Di icul o Ob ain 9 5 14 (35.71%)
Any es ic ion 39 26 65 (40.00%)
*Pe cen ages a e calcula ed as he numbe o ’Yes’ di ided by he
o al numbe o esponses. An a icle can ha e mul iple ca ego ies
o da a; he sum o esponses is he e o e highe han he numbe
o a icles.
34
. As o Ma ch 2025, e o s a e unde way o p ese e he DHS p og am da a, o ins ance ia
IPUMS, and o esusci a e bo h he access o his o ical da a, as well con inue unding new su eys.
35. See CLOCKSS.
36. See .i. Da a Rescue P ojec .
Re ue économique – ol. , no, , p. 1-33 23
Rep oducibili y and Open Science in Economics
In e ing S a a and R usage
S a a usage is in e ed om downloads o S a a packages om he SSC web
se e , which is he sole o icial loca ion o ob ain hese packages. P i a e mi o s
may exis , and no all S a a packages a e ins alled om SSC - bo h he S a a
Jou nal and Gi hub a e likely o be signi ican sou ces. Da a a e ob ained om
log iles om he SSC web se e , p o ided by Ki Baum. R usage is in e ed
om downloads o R bina ies ( o Windows and MacOS). While his is likely o
be less equen han S a a packages, ela i e pa e ns a e o in e es . The e is no
easy way o ob ain he ull lis o downloads o all packages, o he han cycling
h ough se e al housand such packages. The da a s em om one o dozens o
Comp ehensi e R A chi e Ne wo k (
CRAN
) mi o s, hough his one, managed
by Posi PBC, is he i s one lis ed in he lis o mi o s ha a e o e ed o use s.
Da a in bo h cases is o Feb ua y 2025.
As a i s s ep, hese downloads we e mapped o coun ies, and hen agg ega ed
by egions (Table A2 o S a a, Table A3 o R). Mapped on o a wo ld map,
cus om_ egion coun ies Regional Downloads F ac ion
1 China 2 5438811 50.79
2 No h Ame ica 3 3509194 32.77
3 Eu ope 40 912383 8.52
4 Res o Asia 45 521557 4.87
5 A ica 51 168663 1.57
6 La in Ame ica & Ca ibbean 28 127078 1.19
7 Aus alia 1 26703 0.25
8 Res o Oceania 5 4763 0.04
Tableau A2 – Regional S a is ics o S a a Downloads
cus om_ egion coun ies Regional Downloads F ac ion
1 No h Ame ica 3 438562 66.62
2 Eu ope 50 83367 12.66
3 Res o Asia 48 72724 11.05
4 La in Ame ica & Ca ibbean 42 25316 3.85
5 China 1 18641 2.83
6 A ica 51 10124 1.54
7 Aus alia 1 8166 1.24
8 Res o Oceania 11 1406 0.21
9 O he 1 9 0.00
Tableau A3 – Regional S a is ics o R Downloads
his p o ides a p e y pic u e, hough no e y in o ma i e (Figu e A1). The da a
sugges ha downloads om China a e no ully cap u ed by he Posi -managed
mi o , possibly because o peculia i ies o he Chinese in e ne in as uc u e.
While his is possible o o he coun ies as well, i is no ully de ec able. I ha e
24 Re ue économique – ol. , no, , p. 1-33
La s Vilhube
Figu e A1 – Downloads o S a a packages and R so wa e, by egion, wi hou
China. Sou ce : published da a by Ki Baum and Posi PBC, own calcula ions.
Re ue économique – ol. , no, , p. 1-33 25
Rep oducibili y and Open Science in Economics
Ti ole J. e al. [ma s 2003], « Edi o ial », Jou nal o he Eu opean Economic
Associa ion, 1 (1), p. iii-i .
U.S. Census Bu eau [2024], uscensusbu eau/ s dc-ex e nal-census-p ojec s a
2625c2169 2d1b22cac262b90 7a 87b7e969d6b, Gi hub.
UK Go e nmen [2014], Open Go e nmen Licence o public sec o in o ma ion
V3.
UNESCO [no . 2022], Unde s anding open science, UNESCO.
Vicen e-Saez R. e Ma inez-Fuen es C. [juill. 2018], « Open Science now : A sys-
ema ic li e a u e e iew o an in eg a ed de ini ion », en, Jou nal o Business
Resea ch, 88, p. 428-436.
Vilhube L. [déc. 2018], « Relaunching he Jou nal o P i acy and
Con iden iali y », en, Jou nal o P i acy and Con iden iali y, 8 (1), Numbe : 1.
– [mai 2020a], Mig a ing his o ical AEA supplemen s, Co nell Uni e si y.
–
[déc. 2020b], « Rep oducibili y and Replicabili y in Economics », Ha a d
Da a Science Re iew, 2 (4).
–
[juin 2023], « Rep oducibili y and anspa ency e sus p i acy and con iden-
iali y : Re lec ions om a da a edi o », en, Jou nal o Econome ics, 235 (2),
Published online, p. 2285-2294.
–
[mai 2025], « Repo o he AEA Da a Edi o », en, AEA Pape s and
P oceedings.
Vilhube L., Connolly M. e al. [no . 2022], A empla e README o social
science eplica ion packages, Publishe : Zenodo, Zenodo.
Vilhube L., Schmu e I., e al. [July 2023], “Rein o cing Rep oducibili y and
Replicabili y: An In oduc ion”, Ha a d Da a Science Re iew, 5 (3).
Vilhube L., Tu i o J. e Welch K. [mai 2020], « Repo by he AEA Da a Edi o »,
en, AEA Pape s and P oceedings, 110, p. 764-75.
Vlaeminck S. [aoû 2021], « Dawning o a new age? Economics jou nals’ da a po-
licies on he es bench », en, LIBER Qua e ly : The Jou nal o he Associa ion
o Eu opean Resea ch Lib a ies, 31 (1), p. 1-29.
Wa son C. [juin 2022], « Many esea che s say hey’ll sha e da a — bu don’ »,
en, Na u e, 606 (7916), Bandie a_ab es : a Cg_ ype : News Publishe : Na u e
Publishing G oup Subjec _ e m : Funding, E hics, p. 853-853.
Weeden K. A. [oc . 2023], « C isis? Wha C isis? Sociology’s Slow P og ess
Towa d Scien i ic T anspa ency », en, Ha a d Da a Science Re iew, 5 (4).
Welch F. [juin 1973], Educa ion, In o ma ion, and E iciency, Na ional Bu eau o
Economic Resea ch.
Wikipedia [jan. 2025], Copy igh s a us o wo ks by he ede al go e nmen o
he Uni ed S a es, en, Page Ve sion ID : 1270780906.
Williams H. L. [ é . 2013], « In ellec ual P ope y Righ s and Inno a ion :
E idence om he Human Genome », en, Jou nal o Poli ical Economy, 121 (1),
p. 1-27.
Wo ld Bank [2021], Wo ld De elopmen Repo 2021 : Da a o Be e Li es, EN,
Wo ld Bank.
Xie J. e Ge akos J. [mai 2020], « The An icompe i i e E ec s o Common
Owne ship : The Case o Pa ag aph IV Gene ic En y », en, AEA Pape s and
P oceedings, 110, p. 569-572.
32 Re ue économique – ol. , no, , p. 1-33
Zillow [2021], Zillow’s Assesso and Real Es a e Da abase, en-US.
