Rojas, Ana Ma ía; Sca ascini, Ca los G.
Wo king Pape
Debiasing policymake s: The ole o beha io al economics
aining
IDB Wo king Pape Se ies, No. IDB-WP-1595
P o ided in Coope a ion wi h:
In e -Ame ican De elopmen Bank (IDB), Washing on, DC
Sugges ed Ci a ion: Rojas, Ana Ma ía; Sca ascini, Ca los G. (2024) : Debiasing policymake s: The
ole o beha io al economics aining, IDB Wo king Pape Se ies, No. IDB-WP-1595, In e -Ame ican
De elopmen Bank (IDB), Washing on, DC,
h ps://doi.o g/10.18235/0012888
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/299407
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h ps://c ea i ecommons.o g/licenses/by/3.0/igo/
Debiasing Policymake s:
The Role o Beha io al Economics T aining
A
na Ma ía Rojas M.
Ca los Sca ascini
WORKING PAPER No IDB-WP-1595
In e -
A
me ican De elopmen Bank
Depa men o Resea ch and Chie Economis
Ap il 2024
* Wo ld Bank
** In e -Ame ican De elopmen Bank
Debiasing Policymake s:
The Role o Beha io al Economics T aining
A
na Ma ía Rojas M.*
Ca los Sca ascini**
In e -
A
me ican De elopmen Bank
Depa men o Resea ch and Chie Economis
Ap il 2024
Ca aloging-in-Publica ion da a p o ided by he
In e -Ame ican De elopmen Bank
Felipe He e a Lib a y
Rojas Méndez, Ana Ma ía.
Debiasing policymake s: he ole o beha io al economics aining / Ana Ma ía
Rojas M., Ca los Sca ascini.
p. cm. — (IDB Wo king Pape Se ies ; 1595)
Includes bibliog aphical e e ences.
1. Public heal h-Decision making -La in Ame ica. 2. Public heal h-Decision
making-Ca ibbean A ea. 3. Economics-Psychological aspec s-La in Ame ica.
4. Economics-Psychological aspec s-Ca ibbean A ea. I. Sca ascini, Ca los G.,
1971- II. In e -Ame ican De elopmen Bank. Depa men o Resea ch and
Chie Economis . III. Ti le. IV. Se ies.
IDB-WP-1595
h p://www.iadb.o g
Copy igh © 2024 In e -Ame ican De elopmen Bank ("IDB"). This wo k is subjec o a C ea i e
Commons license CC BY 3.0 IGO (h ps://c ea i ecommons.o g/licenses/by/3.0/igo/legalcode). The
e ms and condi ions indica ed in he URL link mus be me and he espec i e ecogni ion mus be
g an ed o he IDB.
Fu he o sec ion 8 o he abo e license, any media ion ela ing o dispu es a ising unde such license
shall be conduc ed in acco dance wi h he WIPO Media ion Rules. Any dispu e ela ed o he use o
he wo ks o he IDB ha canno be se led amicably shall be submi ed o a bi a ion pu suan o he
Uni ed Na ions Commission on In e na ional T ade Law (UNCITRAL) ules. The use o he IDB's name
o any pu pose o he han o a ibu ion, and he use o IDB's logo shall be subjec o a sepa a e
w i en license ag eemen be ween he IDB and he use and is no au ho ized as pa o his license.
No e ha he URL link includes e ms and condi ions ha a e an in eg al pa o his license.
The opinions exp essed in his wo k a e hose o he au ho s and do no necessa ily e lec he iews o
he In e -Ame ican De elopmen Bank, i s Boa d o Di ec o s, o he coun ies hey ep esen .
Abs ac
Beha io al biases o en lead o subop imal decisions, a ulne abili y ha ex ends
o policymake s who ope a e unde condi ions o a igue, s ess, and ime cons ain s
and wi h signi ican implica ions o public wel a e. While beha io al economics o -
e s s a egies like de aul adjus men s o mi iga e decision-making cos s, deploying
hese policy in e en ions is no always easible. Thus, enhancing he quali y o pol-
icy decision-making is c ucial, and e idence sugges s ha a ge ed aining can boos
job pe o mance among policymake s. This s udy e alua es he impac o a beha -
io al aining cou se on policy decision-making h ough a andomized expe imen and
a su ey es ha inco po a es p oblem-sol ing and decision-making asks among ap-
p oxima ely 25,000 pa icipan s en olled in he cou se. Ou indings e eal a signi ican
imp o emen in he ea ed g oup, wi h esponses a e aging 0.6 s anda d de ia ions
be e han hose in he con ol g oup. Gi en he inc easing p e alence o such cou ses,
his pape unde sco es he po en ial o beha io al aining in imp o ing policy deci-
sions and ad oca es o u he esea ch h ough addi ional expe imen al s udies.
JEL classi ica ions: H83, Z18
Keywo ds: Expe imen al design, Beha io al economics, T aining, Public policy,
Go e nmen o icials
The au ho s a e g a e ul o Ka ina Ma quez Gue a and And ´es Ba i˜nas o supe b esea ch assis ance,
he many beha io al specialis s a he IDB who wo ked wi h us du ing di e en s ages o he p ojec ,
o Indhi a Rami ez, Josue Mendoza, he eam a eMBeD-WB, and o he LAER Special Issue wo kshop
pa icipan s o hei commen s and sugges ions. We a e also indeb ed o he Knowledge, Inno a ion, and
Communica ion Sec o Depa men eam ha wo ks wi h us in he main enance o he online cou se and
collec s he da a. The in o ma ion and opinions p esen ed he ein a e en i ely hose o he au ho s, and
no endo semen by he In e -Ame ican De elopmen Bank (IDB), i s Boa d o Execu i e Di ec o s, o he
coun ies hey ep esen is exp essed o implied. IDB managemen had no in ol emen in he s udy design,
analysis, o in e p e a ion o he da a, in he w i ing o he epo , o in he decision o submi he a icle
o publica ion.
1 In oduc ion
A as li e a u e om psychology and economics has shown ha indi iduals end o ha e
nons anda d p e e ences (e.g., social p e e ences), nons anda d belie s (e.g., o e con idence),
and nons anda d decision-making (e.g., aming and limi ed a en ion) (DellaVigna, 2009).
Policymake s a e no excep ion. Resea ch has consis en ly shown ha policy p o essionals a e
suscep ible o nons anda d belie s and decision-making aps. O e con idence, o example,
has been obse ed in he judgmen s o physicians, clinical psychologis s, lawye s, nego ia o s,
enginee s, banke s, and secu i y analys s (Be ne and G abe , 2008; G i in and T e sky,
1992; Ko acs e al., 2020; Lambe e al., 2012; Sand oni and Squin ani, 2004; S a k and
Sachau, 2016). Policy p o essionals a e u he a ec ed by aming ou comes as losses o
gains and by con i ma ion bias (Banu i e al., 2019).
These biases can ha e eal implica ions. Fo example, U.S. judges’ opinions a e signi -
ican ly in luenced by he poli ical composi ion o judicial panels (Suns ein, 2006), and he
empo al o de o ulings may a ec he ou comes (Danzige e al., 2011). In he case o
heal hca e, biases a e likely o in luence diagnosis and make ea men decisions and le els
o ca e dependen on pa ien cha ac e is ics (Fi zGe ald and Hu s , 2017). In educa ion,
eache s’ unconscious biases and p e e ences ela ed o s uden s’ gende , ace, sexual o ien-
a ion, socio-economic backg ound, o o he aspec s o iden i y can a ec lea ning ou comes
and pe pe ua e inequali ies in he class oom (Fa an Be an e al., 2021).
Becoming awa e o ou sys ema ic e o s may help co ec hem (Fa an Be an e al.,
2021). The e a e ways o educe o e con idence (B ookins e al., 2014) and o he biases.
Making indi iduals e lec on hei choices and p o iding in o ma ion abou ac ual pe o -
mance and he isks en ailed by w ong choices helps. Fo example, once NBA e e ees a e
made awa e o hei implici p e e ences, hei a o i ism bias disappea s (Pope e al., 2018).
This is pa icula ly ele an in he con ex o policymaking, whe e biased judgmen can ha e
signi ican wel a e consequences (Ca e a a e al., 2023).
Could aining help? A me a-analy ic e iew o managemen aining p og ams ound
ha hose ocused on human esou ces, so skills, ma ke ing, and inance and accoun ing,
especially when o ganized by local o ganiza ions, end o esul in be e i m pe o mance
(Busso e al., 2023). In he case o public se an s, some ypes o aining ha e been ound
o be e ec i e, a leas in he sho un. T aining p og ams o hospi al manage s posi i ely
a ec ed manage ial skills, knowledge, and compe encies (Ra aghi e al., 2021). T aining
police o ice s in in es iga ion echniques and so skills inc eased he sa is ac ion o c ime
ic ims (Bane jee e al., 2012) and educed some ypes o c imes (Ga cia e al., 2013).
2
In his pape , we es whe he a beha io al economics (BE) online cou se o public
o icials has an e ec on hei decision-making p ocess owa d public policy issues, including
igh ing he COVID-19 pandemic. We also es whe he he cou se imp o es hei p oblem-
sol ing skills.
The expe imen ook place in he con ex o he online beha io al cou se p o ided by
he In e -Ame ican De elopmen Bank (IDB) on i s lea ning pla o m. We andomized
he indi iduals en olled in 16 edi ions o he Spanish-language e sion o he cou se in o
ea men and con ol g oups (abou 25,000 indi iduals.) The con ol g oup was asked o
sol e p oblems in a six-ques ion ques ionnai e be o e s a ing he cou se, and he ea men
g oup did so a he end o he cou se.
Resul s indica e ha he cou se had a posi i e e ec on imp o ing p oblem-sol ing and
decision-making. When conside ing he o e all sco e, ea ed indi iduals sco ed 0.6 s anda d
de ia ions highe han he con ol g oup. In e ms o speci ic ques ions, he impac was
be ween 0 and an inc ease o 34 pe cen age poin s.
The esul s a e obus o a se ies o es s ha exploi he ac ha he con ol g oup
ook he es be o e and a e he cou se, as well as he ollo e na u e o he di e en
edi ions o he cou se. Rega ding mechanisms, we added o he su ey a ques ion (no
conside ed in he o e all sco e calcula ion) ha was co e ed in he lec u es and in-cou se
es s. Pa icipan s sco ed highe on ha one han on he o he ques ions (40.8 pe cen age
poin s), which p o ides some pa ial e idence ha he e ec s happened because o lea ning.
This s udy complemen s nascen bu s ill scan esea ch showing ha debiasing ain-
ing can signi ican ly imp o e decision-making, wi h bo h sho - e m and long- e m e ec s
(Mo ewedge e al., 2015; Sellie e al., 2019). While p e ious s udies ha e wo ked wi h a
dedica ed sample o lab o s uden pa icipan s wa ching a ideo o a case s udy, we e al-
ua e he impac o a mul i-week-long cou se designed o policymake s ha was impa ed
o e se e al yea s. I also complemen s a li e a u e ha e alua es he e ec i eness o online
lea ning ools (C is ia and Vlaicu, 2023). He e, we show ha online cou ses can imp o e
lea ning ou comes and decision-making abili ies. Finally, he pape complemen s he as
li e a u e on beha io al science by showing ha aining cou ses could be an addi ional ool
a ailable o be e decision-making. This s udy could se e as he s epping s one o ex-
pe imen s ha es p oblem-sol ing skills mo e b oadly and gene a e incen i es o u he
eplica ion s udies using he mul iple cou ses on beha io al science a ailable.
3
2 The Expe imen
2.1 The IDB Cou se on Beha io al Economics
The IDB p o ides online educa ion aimed a policymake s in La in Ame ica and he Ca ibbean.1
In 2020, he IDB launched he i s online cou se in Beha io al Economics o e ed in Span-
ish.2The cou se is in e ac i e, sel -paced, and applied o public policy design. I is o e ed a
no cos and a ge s La in Ame ican policymake s. Mo e han 14,000 indi iduals egis e ed
o pa icipa e, and by he end o 2023, he numbe had climbed o mo e han 25,000. The
Po uguese and English e sions we e launched du ing he second semes e o 2020.
The cou se is di ided in o ou modules wi h an app oxima e wo kload o 4-5 hou s pe
week.. I was designed o be comple ed wi hin a ou -week ime pe iod, bu pa icipan s a e
allowed o inish he cou se in up o six weeks. The i s wo modules co e he main concep s
o he ield (main biases and beha io al insigh s) and explana ions o how hese di e om he
no ions o he s anda d economic model. Fo example, module 1 includes 10 ac i i ies ha
ake be ween 3 o 30 minu es each o comple e. Ac i i y 1 p o ides an in oduc ion o “How
good a e we a making decisions?” Ac i i y 2 desc ibes wha beha io al science is. Ac i i y
3 p o ides an o e iew o he ield and applica ions o beha io al economics. Ac i i y 4
eaches abou examples o non-s anda d p e e ences, ac i i y 5 abou non-s anda d belie s,
and ac i i y 6 abou he ac o s ha a ec in o ma ion p ocessing. Ac i i ies 7 o 9 deal
wi h he main e ms used in he ield, how go e nmen s use beha io al insigh s, and he ole
o beha io al economics in he design and execu ion o public policies. Ac i i y 10 is he
lea ning assessmen o he module. The hi d module ocuses on applied cases in se e al
sec o s, wi h a special ocus on ax compliance and heal h, wo a eas in which he IDB has
buil a b oade po olio. S a ing wi h session 3, a speci ic sec ion on COVID-19 was added.
The e ised lea ning guide, wi h a ull desc ip ion o he con en s o he cou se, is p o ided
in he Online Appendix.
The eaching me hodology consis s o p o iding e e ence ma e ials such as ideos, in e -
ac i e p esen a ions, and eadings and ca ying ou ac i i ies and exe cises using eal case
examples om La in Ame ica, he Ca ibbean, and o he pa s o he wo ld. A e each
module, pa icipan knowledge is es ed. The e a e i e lea ning assessmen s o es s du ing
1By 2020, he IDB o e ed mo e han 200 online cou ses in de elopmen e ec i eness, in eg a ion and
ade, p ojec managemen , social and en i onmen al isks managemen , wa e and clima e change, and
o he s. The ull ca alog o cou ses is a ailable a h ps://cu sos.iadb.o g/en/indes/p og amas?lang=en
2Fo con ex , he cou se’s i s i e edi ions o sessions we e launched in Spanish on Feb ua y 18, Ma ch
17, May 19, July 28, and Oc obe 6, 2020.
4
he cou se: Modules 1 and 2 each con ibu e 20% o he o al assessmen . Module 3 consis s
o wo assessmen s, one o he ax compliance sec ion and one o he heal h sec ion, each
con ibu ing 15%. The lea ning assessmen o Module 4 is weigh ed a 30%.3Al hough
comple ing each ques ionnai e is manda o y in o de o mo e on o he nex module, passing
i is no a p e equisi e o ad ancing in he cou se. The passing sco e o each assessmen
and o he o e all cou se is a leas 80 pe cen o he o al sco e, and he inal sco e is
calcula ed based on he weigh s assigned o each ques ionnai e. Those who inish he cou se
a e awa ded a ce i ica e o comple ion (see example in he Online Appendix), and hey can
also sha e digi al badges on social media.
2.2 Expe imen Design
To e alua e he impac o he cou se, we andomized hose who egis e ed o each one o
he sessions in Spanish. Once indi iduals egis e o a cou se, hey a e di ided in o wo
g oups ( ea men and con ol) and hen assigned o i ual class ooms o up o 100 people
(each class oom is o med by indi iduals om he same g oup: ea ed o con ol.)4
Be o e s a ing he cou se, s uden s ecei e a ques ionnai e wi h basic demog aphic ques-
ions (coun y o o igin, sex, academic deg ee, e c.)5. Those indi iduals in he con ol g oup
also ecei e a su ey es ha includes 5 ques ions ( i s wo sessions) o 6 ques ions (begin-
ning wi h session 3).6E e ybody ecei ed he same su ey es a he end o he cou se.
The ques ions included in he es we e o wo ypes: i) cogni i e skills asks: a cogni i e
illusion (“ iangles”), a compu a ion o compound in e es (“lo e y”—only in sessions 1 and
2), and an expec ed alue ques ion (“disease”); and ii) public policy ques ions ha es ed he
indi idual knowledge o beha io al insigh s. One o hese ques ions (“ eache s’ incen i es”)
was explici ly conside ed in he se o ma e ials p o ided du ing he cou se; he e o e, i ac s
as a alida ion exe cise.
The ques ions included in he su ey (in he o de hey a e p esen ed o he indi iduals)
a e he ollowing ( igh answe s in bold ace):
3In he i s wo sessions, he lea ning assessmen o Module 3 was weigh ed a 20%, and Module 4 was
weigh ed a 40%.
4The pu pose o he class ooms is o p o ide he oppo uni y o in e ac ion in i ual cha s. These cha s
a e no supe ised o moni o ed.
5This in o ma ion is a ailable only o sessions 1 o 5, o hose who chose o comple e he ques ionnai e.
6The changes in he ques ionnai e esponded o he in oduc ion o COVID-19 ma e ial in he cou se;
one o he o iginal ques ions was eplaced o a oid ex ending he su ey oo much.
5
Table 2: T ea men E ec s (all cou ses pooled)
Tes T iangles: Disease: Child Anemia: Lo e y COVID-19: COVID-19: Teache s
z-sco e Reasoning Exp Value SocNo m & Loss A Beh In e Social Dis ancing Incen i es
(1) (2) (3) (4) (5) (6) (7) (8)
T ea men 0.600*** 0.030** -0.024* 0.340*** 0.012 0.147*** 0.290*** 0.408***
(0.027) (0.012) (0.014) (0.014) (0.032) (0.014) (0.014) (0.014)
Cons an -0.121*** 0.429*** 0.571*** 0.242*** 0.717*** 0.544*** 0.278*** 0.442***
(0.032) (0.018) (0.021) (0.020) (0.028) (0.016) (0.016) (0.018)
Obse a ions 5655 5655 5655 5655 864 4791 4791 5655
Clus e s 247 247 247 247 32 215 215 247
Cou se FE Yes Yes Yes Yes Yes Yes Yes Yes
Adj R-squa ed 0.086 0.008 0.005 0.132 0.003 0.038 0.107 0.196
No es: each ow shows he eg ession coe icien s and he s anda d e o in pa en hesis co esponding o an OLS eg ession,
including session ixed e ec s. S anda d e o s a e clus e ed a he session le el. *** p<0.01, ** p<0.05, * p<0.1. Di e ences
in he numbe o obse a ions ac oss columns because COVID ques ions we e included s a ing in Session 3 when he Lo e y
ques ion was elimina ed.
Figu e 1: Dis ibu ion o Co ec Answe s
0 1 2 3 4 5
Con ol
0 1 2 3 4 5
T ea men
Tes sco e dis ibu ion
12
Figu e 2: Co ec and Inco ec Answe s pe Ques ion and G oup
0% 20% 40% 60% 80% 100%
Pe cen
Inco ec Co ec
Ques ion 1 - iangles
0% 20% 40% 60% 80% 100%
Pe cen
Inco ec Co ec
Ques ion 2 - Lo e y
0% 20% 40% 60% 80% 100%
Pe cen
Inco ec Co ec
Ques ion 3 - Teache s
0% 20% 40% 60% 80% 100%
Pe cen
Inco ec Co ec
Ques ion 4 - Disease
Con ol T ea men
0% 20% 40% 60% 80% 100%
Pe cen
Inco ec Co ec
Ques ion 5 - Anemic
0% 20% 40% 60% 80% 100%
Pe cen
Inco ec Co ec
Ques ion 2 - COVID 1
0% 20% 40% 60% 80% 100%
Pe cen
Inco ec Co ec
Ques ion 2 - COVID 2
Con ol T ea men
13
We ound no signi ican he e ogeneous e ec s7. Nei he gende , academic deg ee, no
expe ience in hei job a he ime o he cou se had any di e en ial e ec . This is impo an ,
as i shows ha e e ybody bene i ed equally om he cou se.
4 Mechanism and Robus ness
Can we be su e ha he di e ences be ween he ea men and con ol g oups come om
he cou se? In o de o p o ide some e idence in his di ec ion, we ha e pe o med h ee
exe cises. Fi s , we in oduced one ques ion in he es ha was also pa o he es s wi hin
he cou se. The di e ence be ween he con ol and ea men g oups in his ques ion is
highe han o any o he es ques ions (see column 8 in Table 2.)
Second, because he indi iduals in he con ol g oup ook he ques ionnai e be o e and
a e he cou se, we can e alua e i he quali y o hei answe s imp o ed. As shown in Table
3, he indi iduals in he con ol g oup sco ed much highe in he es a e ha ing aken
he cou se han be o e he cou se. In pa icula , he o e all sco e imp o es by 0.6 s anda d
de ia ions.
Table 3: Con ol g oup: Di e ences be ween be o e and a e he cou se (all cou ses pooled)
Tes T iangles: Disease: Child Anemia: Lo e y COVID-19: COVID-19: Teache s
z-sco e Reasoning Exp Value SocNo m & Loss A Beh In e Social Dis ancing Incen i es
(1) (2) (3) (4) (5) (6) (7) (8)
A e aking he cou se 0.616*** 0.114*** -0.029*** 0.319*** 0.036 0.191*** 0.253*** 0.382***
(0.020) (0.008) (0.010) (0.014) (0.029) (0.011) (0.011) (0.014)
Cons an -0.308*** 0.424*** 0.550*** 0.277*** 0.719*** 0.541*** 0.295*** 0.447***
(0.033) (0.018) (0.020) (0.017) (0.029) (0.018) (0.017) (0.016)
Obse a ions 2933 2933 2933 2933 478 2455 2455 2933
Clus e s 124 124 124 124 17 107 107 124
Cou se FE Yes Yes Yes Yes Yes Yes Yes Yes
Adj R-squa ed 0.253 0.059 0.003 0.202 0.005 0.110 0.155 0.296
No es: each ow shows he eg ession coe icien s and he s anda d e o in pa en hesis co esponding o an OLS eg ession,
including session ixed e ec s. S anda d e o s a e clus e ed a he session le el. *** p<0.01, ** p<0.05, * p<0.1. i e ences in
he numbe o obse a ions ac oss columns because COVID ques ions we e included s a ing in Session 3 when he Lo e y
ques ion was elimina ed.
Thi d, one po en ial issue wi h he cu en analysis is ha we a e compa ing indi iduals
who ook he su ey es a di e en poin s; ha is, he con ol akes he es a ew weeks
ea lie han he ea men does. In o de o con ol o ha , we exploi he ecu ing na u e
o he cou ses and compa e g oups o people who ook he su ey es a app oxima ely he
7This analysis exclusi ely ocuses on sessions 1 o 5, as hey a e he only sessions o which he e is a ailable
in o ma ion on he a iables used o he e ogenei y analyses. Ne e heless, he a ailable in o ma ion pe ains
solely o indi iduals who chose o comple e he demog aphic ques ionnai e.
14
same ime, e en hough hey belong o di e en cou se sessions (coho s). Tha way, we can
compa e ea ed indi iduals in session 1 wi h con ol indi iduals in session 2 (who e en ually
inished hei cou se), ea ed in session 2 wi h con ols om session 3, and so on.8Resul s
a e shown in Table 4. Each column compa es he esul s om he ea ed in session wi h
he con ol g oup (who hen wen on o inish hei cou se) in session + 1. The esul s a e
e y simila o hose p esen ed so a . Those who ook he cou se answe ed be ween 0.2 and
0.9 s anda d de ia ions be e han hose who had no aken and inished he cou se ye .
Table 4: T ea men E ec Ac oss Cou ses
Cou se 1 T Cou se 2 T Cou se 3 T Cou se 4 T Cou se 5 T Cou se 6 T Cou se 7 T
Cou se 2 C Cou se 3 C Cou se 4 C Cou se 5 C Cou se 6 C Cou se 7 C Cou se 8 C
Tes z-sco e
T ea men 0.511*** 0.210*** 0.562*** 0.621*** 0.652*** 0.857*** 0.628***
(0.097) (0.053) (0.061) (0.060) (0.161) (0.096) (0.181)
Obse a ions 383 665 942 941 582 207 250
Clus e s 15 25 35 35 24 8 11
Cou se 8 T Cou se 11 T Cou se 12 T Cou se 13 T Cou se 14 T Cou se 15 T Cou se 16 T
Cou se 9 C Cou se 12 C Cou se 13 C Cou se 14 C Cou se 15 C Cou se 16 C Cou se 17 C
Tes z-sco e
T ea men 0.525*** 0.741*** 0.682*** 0.460*** 0.742*** 0.516*** 0.312**
(0.104) (0.118) (0.126) (0.138) (0.145) (0.162) (0.128)
Obse a ions 177 137 193 244 119 218 215
Clus e s 9 8 10 13 9 18 19
No es: each ow shows he eg ession coe icien s and he s anda d e o in pa en hesis co esponding o an OLS eg ession.
S anda d e o s a e obus . *** p<0.01, ** p<0.05, * p<0.1. Sou ce: Au ho s’ calcula ions
As a inal analysis, we an a obus ness exe cise by e alua ing whe he he e we e any
di e ences be ween he ea men and he con ol g oup a e bo h had aken he cou se. We
p esen he esul s in Table 5. As can be obse ed, di e ences a e small, and hey luc ua e
in e ms o he sign. The ea men g oup sco ed highe han he con ol g oup a e he
cou se in only 3 o hem. O e all, i is he opposi e e ec seems o domina e acco ding o
he composi e sco e: he con ol g oup sco ed highe han he ea men g oup. This esul
8Fo e e ence, in he i s yea , he sessions s a ed on Feb ua y 18, Ma ch 17, May 19, July 28, and
Oc obe 6, 2020
15
could be expec ed gi en ha he con ol g oup had al eady aken he su ey es be o e he
cou se, which may ha e led o some lea ning e en hough hey did no ecei e eedback.
Table 5: Ex-pos : Di e ences be ween T ea men and Con ol a e cou se (all cou ses
pooled)
Tes T iangles: Disease: Child Anemia: Lo e y COVID-19: COVID-19: Teache s
z-sco e Reasoning Exp Value SocNo m & Loss A Beh In e Social Dis ancing Incen i es
(1) (2) (3) (4) (5) (6) (7) (8)
T ea men -0.048* -0.083*** 0.006 0.026* -0.021 -0.045*** 0.038** 0.029***
(0.027) (0.011) (0.013) (0.014) (0.029) (0.014) (0.014) (0.011)
Cons an 0.024 0.563*** 0.571*** 0.654*** 0.760*** 0.732*** 0.551*** 0.844***
(0.030) (0.015) (0.016) (0.015) (0.024) (0.014) (0.016) (0.012)
Obse a ions 5655 5655 5655 5655 864 4791 4791 5655
Clus e s 247 247 247 247 32 215 215 247
Cou se FE Yes Yes Yes Yes Yes Yes Yes Yes
Adj R-squa ed 0.001 0.013 0.005 0.040 0.001 0.017 0.017 0.097
No es: each ow shows he eg ession coe icien s and he s anda d e o in pa en hesis co esponding o an OLS eg ession.
S anda d e o s a e obus . *** p<0.01, ** p<0.05, * p<0.1. Sou ce: Au ho s’ calcula ions
While his se o exe cises does no o e ull e idence on he mechanism, hey sugges
ha he e seems o be no andomiza ion bias. I does no appea ha he ea ed we e be e
han he con ol in answe ing ques ions o easons o he aking he cou se (e.g., he passage
o ime). Mo eo e , people do no seem o be lea ning abou he igh answe s independen ly
o he cou se, and he e is no selec ion e ec a ising om he di e en coho s: hose who
ook he cou se answe be e han hose in hei same coho bu also be e han hose in
u u e coho s who ake he su ey es a app oxima ely he same ime.
5 Conclusions
Beha io al biases lead o subop imal decisions, and policymake s a e no exemp om hem.
Beha io al biases end o ha e a la ge e ec when indi iduals a e i ed, ha e high s ess, o
ha e sho e imes o decide. This is usually he en i onmen in which policymake s ha e o
make decisions ha can ha e la ge wel a e consequences. Beha io al economics has p o ided
ways o educe he cos o some decisions, such as changing de aul s. S ill, es ic ing he
policy space is no always possible. Finding ways o imp o e policymaking is he e o e o
i s -o de impo ance.
T aining cou ses o policymake s ha e been shown o be e ec i e in inc easing job pe -
o mance. This pape es s whe he a beha io al cou se could imp o e policy decisions. We
show sugges i e e idence ha i does. O cou se, hese esul s should no be aken as he
16
ul ima e p oo . Fo example, es ing policymake s egula ly a e hey ook he cou se and
based on eal decisions is needed. This pape could be a s epping s one in ha di ec ion.
17
Re e ences
Bane jee, A., Cha opadhyay, R., Du lo, E., Kenis on, D., and Singh, N. (2012). Imp o ing
police pe o mance in ajas han, india: Expe imen al e idence on incen i es, manage ial
au onomy and aining. Technical epo , Na ional Bu eau o Economic Resea ch.
Banu i, S., De con, S., and Gau i, V. (2019). Biased policy p o essionals. The Wo ld Bank
Economic Re iew, 33(2):310–327.
Be ne , E. S. and G abe , M. L. (2008). O e con idence as a cause o diagnos ic e o in
medicine. The Ame ican Jou nal o Medicine, 121(5):S2–S23.
B ookins, P., Lucas, A., and Ry kin, D. (2014). Reducing wi hin-g oup o e con idence
h ough g oup iden i y and be ween-g oup con idence judgmen s. Jou nal o Economic
Psychology, 44:1–12.
Busso, M., Pa k, K., and I azoque, N. (2023). The e ec i eness o managemen aining
p og ams: A me a-analy ic e iew.
Ca e a a, F., Dominguez, P., and Sca ascini, C. (2023). O e con idence and gun p e -
e ences: How beha io al biases a ec you sa e y. Wo king Pape . In e -Ame ican De-
elopmen Bank. A ailable a h ps://publica ions.iadb.o g/en/o e con idence-and-gun-
p e e ences-how-beha io al-biases-a ec -you -sa e y.
C is ia, J. and Vlaicu, R. (2023). Digi alizing public se ices oppo uni ies o La in Ame ica
and he Ca ibbean. Washign on, DC: In e -Ame ican De elopmen Bank.
Danzige , S., Le a , J., and A naim-Pesso, L. (2011). Ex aneous ac o s in judicial deci-
sions. P oceedings o he Na ional Academy o Sciences, 108(17):6889–6892.
DellaVigna, S. (2009). Psychology and economics: E idence om he ield. Jou nal o
Economic li e a u e, 47(2):315–72.
Fa an Be an, M. G., Holla, A., and Vakis, R. (2021). Poo expec a ions.
Fi zGe ald, C. and Hu s , S. (2017). Implici bias in heal hca e p o essionals: a sys ema ic
e iew. BMC medical e hics, 18(1):19.
F ede ick, S. (2005). Cogni i e e lec ion and decision making. Jou nal o Economic pe -
spec i es, 19(4):25–42.
F ye J , R. G., Le i , S. D., Lis , J., and Sado , S. (2012). Enhancing he e icacy o
eache incen i es h ough loss a e sion: A ield expe imen . Technical epo , Na ional
Bu eau o Economic Resea ch.
Ga cia, J., Mejia, D., and O ega, D. (2013). Police e o m, aining and c ime: expe imen al
e idence om colombia’s plan cuad an es. Documen o CEDE, (2013-04).
18
G i in, D. and T e sky, A. (1992). The weighing o e idence and he de e minan s o
con idence. Cogni i e Psychology, 24:411–435.
Kahneman, D., Ba -Hillel, M., e al. (2020). Commen : Laplace and cogni i e illusions.
S a is ical Science, 35(2):171–172.
Ko acs, R. J., Laga de, M., and Cai ns, J. (2020). O e con iden heal h wo ke s p o ide
lowe quali y heal hca e. Jou nal o Economic Psychology, 76:102213.
Lambe , J., Bessi`e e, V., and N’Goala, G. (2012). Does expe ise in luence he impac o
o e con idence on judgmen , alua ion and in es men decision? Jou nal o Economic
Psychology, 33(6):1115–1128.
Mo ewedge, C. K., Yoon, H., Scopelli i, I., Symbo ski, C. W., Ko is, J. H., and Kassam,
K. S. (2015). Debiasing decisions: Imp o ed decision making wi h a single aining in e -
en ion. Policy Insigh s om he Beha io al and B ain Sciences, 2(1):129–140.
Pope, D. G., P ice, J., and Wol e s, J. (2018). Awa eness educes acial bias. Managemen
Science, 64(11):4988–4995.
Ra aghi, H., Bey an and, T., Mannion, R., Alijanzadeh, M., A yankhesal, A., and Belo geo ,
V. D. (2021). E ec i eness o aining and educa ional p og ams o hospi al manage s: a
sys ema ic e iew. Heal h se ices managemen esea ch, 34(2):113–126.
Sand oni, A. and Squin ani, F. (2004). A su ey on o e con idence, insu ance and sel -
assessmen aining p og ams. Wo king Pape , Uni e si y o Wa wick.
Sellie , A.-L., Scopelli i, I., and Mo ewedge, C. K. (2019). Debiasing aining imp o es
decision making in he ield. Psychological science, 30(9):1371–1379.
S a k, E. and Sachau, D. (2016). Lake wobegon’s guns: O e es ima ing ou gun- ela ed
compe ences. Jou nal o Social and Poli ical Psychology, 4(1):8–23.
Suns ein, C. R. (2006). In o opia: How many minds p oduce knowledge. Ox o d Uni e si y
P ess.
T e sky, A. and Kahneman, D. (1981). The aming o decisions and he psychology o
choice. science, 211(4481):453–458.
Vandenb oucke, A. R., Fah en o , J. J., Slig e, I. G., and Lamme, V. A. (2014). Seeing wi h-
ou knowing: Neu al signa u es o pe cep ual in e ence in he absence o epo . Jou nal
o cogni i e neu oscience, 26(5):955–969.
19
BEHAVIORAL ECONOMICS
FOR BETTER PUBLIC POLICIES
Lea ning Guide
A Online Appendix
A.1 Cou se Lea ning Guide
The cou se websi e link in English is: h ps://indes i ual.iadb.o g/en ol/index.php?id=1960
The ull cou se lea ning guide is also p o ided below.
BEHAVIORAL ECONOMICS
FOR BETTER PUBLIC POLICIES
Pg. 2
CONTENTS
TARGETS AND OBJECTIVES .........................................................................................................................
COURSE PACE AND METHODOLOGY ................................................................................................
"NETIQUETTE" RULES FOR FORUM PARTICIPANTS ...................................................................................
OBJECTIVES OF THE MODULES ..................................................................................................................
EVALUATION ...............................................................................................................................................
PASS POLICY ………………………………………………………………………………………….
CERTIFICATION ...........................................................................................................................................
DIGITAL BADGES .........................................................................................................................................
COURSE POLICIES .......................................................................................................................................
WORK PLAN ................................................................................................................................................
CREDITS .......................................................................................................................................................
21
BEHAVIORAL ECONOMICS
FOR BETTER PUBLIC POLICIES
Pg. 9
Module 3 - Applied cases (week 3)
6 hou s
☐Ac i i y 1: Read he “Tax Compliance” sec ion
10 min
☐Ac i i y 2: S udy he “Belie s, ba ie s, and examples o solu ions"
lesson
35 min
☐Ac i i y 3: S udy he “P e e ences, ba ie s, and examples o
solu ions" lesson
35 min
☐Ac i i y 4: S udy he “In o ma ion p ocessing, ba ie s and nudges”
lesson
30 min
☐Ac i i y 5: Pa icipa e in he "Should we shame ax e ade s?" o um
25 min
☐Ac i i y 6: Read “Conclusions"
5 min
☐Ac i i y 7: B owse he “Takeaways o ax compliance" in e ac i e
summa y
15 min
☐Ac i i y 8: Take he lea ning assessmen o he sec ion on ax
compliance
25 min
☐Ac i i y 9: Read he “Heal h" sec ion and wa ch he ideo
15 min
☐Ac i i y 10: S udy he “F equen biases in a pa ien 's decisions”
lesson
40 min
☐Ac i i y 11: S udy he “Nudges o o e come he ba ie s p esen ed"
lesson
40 min
☐Ac i i y 12: S udy he “Beha io al Economics can help igh COVID-
19” lesson
15 min
☐Ac i i y 13: Pa icipa e in he “The e hics o heal h nudges - whe e is
he limi ?” o um
25 min
☐Ac i i y 14: Read “Conclusions”
5 min
☐Ac i i y 15: B owse he “Takeaways on pa ien s’ decisions"
in e ac i e summa y
15 min
☐Ac i i y 16: Take he lea ning assessmen o he sec ion on heal h
25 min
Module 4: F om heo y o p ac ice: An in e ac i e game
(week 4) 3 hou s
☐Ac i i y 1: Wa ch he “Can beha io al economics help imp o e
accina ion a es?" ideo
5 min
☐Ac i i y 2: Pa icipa e in he in e ac i e game
120 min
☐Ac i i y 3: Take he lea ning assessmen o Module 4
40 min
☐Ac i i y 4: Wa ch he “Cou se closing” ideo
5 min
28
BEHAVIORAL ECONOMICS
FOR BETTER PUBLIC POLICIES
Pg. 10
CREDITS
This cou se was de eloped by IDB’s Resea ch Depa men and he Knowledge, Inno a ion and
Communica ion Sec o , unde he coo dina ion o i s Beha io al Economics G oup. The ollowing IDB s a
pa icipa ed in he p epa a ion o hese con en s:
• Ca los Sca ascini, Nina Rapopo , Ana Ma ía Rojas y C is ina Pa illi - Resea ch Depa men
• Flo encia Lopez Boo and Nicolás Ajzenman - Social Sec o
• Ca los Ge a do Molina and Fe nanda Came a - Knowledge, Inno a ion and Communica ion
Sec o
29
A.2 Ce i ica e o Comple ion
30
A.3 Addi ional Tables
Analysis a Cou se Session Le el
Table A1: Numbe o s uden s in each cou se session
Numbe o s uden s
Regis e ed in he cou se Finished he cou se
Cou se 1 2453 720 29.35 %
Cou se 2 650 148 22.77 %
Cou se 3 4126 1147 27.80 %
Cou se 4 2712 788 29.06 %
Cou se 5 4257 1041 24.45 %
Cou se 6 381 138 36.22 %
Cou se 7 1142 283 24.78 %
Cou se 8 1023 236 23.07 %
Cou se 9 1535 111 7.23 %
Cou se 11 982 168 17.11 %
Cou se 12 635 106 16.69 %
Cou se 13 1552 319 20.55 %
Cou se 14 894 125 13.98 %
Cou se 15 1004 119 11.85 %
Cou se 16 1275 151 11.84 %
Cou se 17 568 64 11.27 %
To al 25189 5664 100 %
31
Table A2: T ea men E ec
Cou se 1 Cou se 2 Cou se 3 Cou se 4 Cou se 5 Cou se 6 Cou se 7 Cou se 8
Tes z-sco e
T ea men 0.645*** 0.088 0.581*** 0.599*** 0.672*** 0.757** 0.765*** 0.459***
(0.064) (0.093) (0.058) (0.060) (0.067) (0.181) (0.165) (0.140)
T iangles: Reasoning
T ea men 0.082** 0.113** -0.032 -0.010 0.064* 0.035 0.074 -0.018
(0.034) (0.045) (0.028) (0.031) (0.032) (0.023) (0.047) (0.050)
Disease: Expec ed Value
T ea men 0.083** 0.095 -0.036 -0.033 -0.031 -0.078 0.000 -0.055
(0.036) (0.068) (0.031) (0.031) (0.034) (0.099) (0.089) (0.059)
Child Anemia: Social No m and Loss A e sion
T ea men 0.454*** -0.087* 0.346*** 0.367*** 0.320*** 0.395*** 0.414*** 0.318***
(0.032) (0.041) (0.030) (0.044) (0.027) (0.045) (0.079) (0.058)
Lo e y
T ea men 0.025 -0.046
(0.037) (0.041)
COVID-19: Beha io al in e en ions
T ea men 0.147*** 0.158*** 0.192*** 0.239** 0.094 0.097
(0.028) (0.034) (0.027) (0.069) (0.068) (0.060)
COVID-19: Social Dis ancing
T ea men 0.296*** 0.290*** 0.265*** 0.276* 0.384*** 0.229**
(0.029) (0.030) (0.027) (0.095) (0.081) (0.078)
Teache s Incen i es
T ea men 0.440*** -0.239*** 0.452*** 0.422*** 0.417*** 0.603*** 0.350*** 0.357***
(0.031) (0.057) (0.023) (0.029) (0.033) (0.089) (0.074) (0.062)
Obse a ions 717 147 1147 788 1041 138 283 236
Clus e s 24 8 42 28 43 4 12 11
No es: each ow shows he eg ession coe icien s and he s anda d e o in pa en hesis co esponding
o an OLS eg ession. S anda d e o s a e obus . *** p<0.01, ** p<0.05, * p<0.1.
Sou ce: Au ho s’ calcula ions
32
Table A2: T ea men E ec
Cou se 9 Cou se 11 Cou se 12 Cou se 13 Cou se 14 Cou se 15 Cou se 16 Cou se 17
Tes z-sco e
T ea men 0.722*** 0.718*** 0.852*** 0.484*** 0.538*** 0.596*** 0.382** 0.134
(0.166) (0.121) (0.092) (0.150) (0.114) (0.162) (0.168) (0.077)
T iangles: Reasoning
T ea men 0.116 0.118 -0.049 0.042 -0.089** 0.037 0.100 -0.008
(0.062) (0.066) (0.056) (0.065) (0.037) (0.070) (0.068) (0.117)
Disease: Expec ed Value
T ea men -0.116 0.059 -0.091* -0.097 0.124 -0.020 -0.212*** -0.369***
(0.078) (0.064) (0.042) (0.058) (0.100) (0.097) (0.065) (0.060)
Child Anemia: Social No m and Loss A e sion
T ea men 0.404*** 0.341*** 0.541*** 0.214*** 0.196* 0.389*** 0.157* 0.332***
(0.050) (0.085) (0.059) (0.048) (0.099) (0.086) (0.077) (0.030)
COVID-19: Beha io al in e en ions
T ea men 0.029 0.120** 0.276** 0.111 0.081 0.084 0.146* -0.025
(0.105) (0.046) (0.103) (0.067) (0.056) (0.093) (0.069) (0.121)
COVID-19: Social Dis ancing
T ea men 0.422*** 0.248*** 0.311** 0.315*** 0.357*** 0.214** 0.254*** 0.235
(0.056) (0.060) (0.096) (0.047) (0.070) (0.076) (0.044) (0.147)
Teache s Incen i es
T ea men 0.371*** 0.330*** 0.576*** 0.425*** 0.350*** 0.477*** 0.409*** 0.389***
(0.075) (0.045) (0.062) (0.055) (0.094) (0.048) (0.075) (0.066)
Obse a ions 111 168 106 319 120 119 151 64
Clus e s 6 10 6 15 9 10 13 6
No es: each ow shows he eg ession coe icien s and he s anda d e o in pa en hesis co esponding o an OLS eg ession.
S anda d e o s a e obus . *** p<0.01, ** p<0.05, * p<0.1. Sou ce: Au ho s’ calcula ions
33
