scieee Science in your language
[en] (orig)

Recent Events and the Coding of Cross-National Indicators

Abstract

Much research in political science relies on datasets produced by human coders. Many variables included in these datasets are not based on observable facts but rather require a considerable level of human judgment. This project studies the extent to which this judgment is affected by availability bias and how it influences the retrospective coding of historic cases. The analysis uses coder-level data from the V-Dem project, one of the few datasets collecting and releasing codings tagged with timestamps when they were produced. The results show that recent dramatic events in a country just prior to the coding have a small, but visible impact on coder ratings, but primarily for those variables that are directly related to the observed events. The magnitude of this effect, however, is small. This alleviates concerns that prominent events in world politics around the time of coding significantly affect the reliability of cross-national indicators.

Read accessible full text

Recent Events and the Coding of Cross-National Indicators

Author: Weidmann, Nils B.
Year: 2024
DOI: 10.1177/00104140231193006
Source: https://kops.uni-konstanz.de/bitstreams/ad7c5f0a-9156-4333-affe-5ac5dcbdde09/download
A icle
Compa a i e Poli ical S udies
2024, Vol. 57(6) 921–937
© The Au ho (s) 2023
A icle euse guidelines:
sagepub.com/jou nals-pe missions
DOI: 10.1177/00104140231193006
jou nals.sagepub.com/home/cps
Recen E en s and he
Coding o C oss-Na ional
Indica o s
Nils B. Weidmann
1

Abs ac
Much esea ch in poli ical science elies on da ase s p oduced by human
code s. Many a iables included in hese da ase s a e no based on ob-
se able ac s bu a he equi e a conside able le el o human judgmen .
This p ojec s udies he ex en o which his judgmen is a ec ed by
a ailabili y bias and how i influences he e ospec i e coding o his o ic
cases. The analysis uses code -le el da a om he V-Dem p ojec , one o
he ew da ase s collec ing and eleasing codings agged wi h imes amps
when hey we e p oduced. The esul s show ha ecen d ama ic e en s
in a coun y jus p io o he coding ha e a small, bu isible impac on
code a ings, bu p ima ily o hose a iables ha a e di ec ly ela ed o
he obse ed e en s. The magni ude o his e ec , howe e , is small. This
alle ia es conce ns ha p ominen e en s in wo ld poli ics a ound he
ime o coding significan ly a ec he eliabili y o c oss-na ional
indica o s.
Keywo ds
quan i a i e me hods, democ a iza ion and egime change, poli ical
egimes
1
Depa men o Poli ics and Public Adminis a ion, Uni e si y o Kons anz, Ge many
Co esponding Au ho :
Nils B. Weidmann, Depa men o Poli ics and Public Adminis a ion, Uni e si y o Kons anz,
Uni e si ¨
a ss , 10, Kons anz 78457, Ge many.
Email: nils.weidmann@uni-kons anz.de
Da a A ailabili y S a emen included a he end o he a icle
Kons anze Online-Publika ions-Sys em (KOPS)
URL: h p://nbn- esol ing.de/u n:nbn:de:bsz:352-2-1g7cxjpn7ioi44
Much empi ical wo k in poli ical science, and in pa icula in he compa a i e
s udy o poli ical egimes, elies on da a p oduced by humans. Using human-
coded c oss-na ional da ase s, compa a i e poli ical scien is s can s udy he
sp ead (o decline) o democ a ic egimes (Lüh mann & Lindbe g, 2019),
moni o he s a e o human igh s wo ldwide (Fa iss, 2014), o gauge he
ex en o media eedom ac oss a global sample o coun ies (Kellam & S ein,
2016). Wi hou human-coded c oss-na ional da ase s, much esea ch in
compa a i e poli ics, democ a iza ion, and de elopmen would be impossible
o conduc . Many o he a iables included in hese da ase s cons i u e
subjec i e assessmen s made by he code s. Fo example, when coding he
co up ac i i ies o he legisla u e ( a iable 2lgc p ) o he “Va ie ies o
Democ acy”da ase (V-Dem, Coppedge e al., 2020), code s need o decide
how one could possibly ecognize co up ion o membe s o he legisla u e in
e e yday poli ics, and how hese obse ed ou comes ansla e in o he di -
e en le els o he coding scale, which anges om “commonly” o “ne e , o
ha dly e e .”These codings cons i u e di ficul decisions o he code s: The e
a e ew i any p ecise guidelines (i) which empi ical ac s o ake in o accoun
o he assessmen and (ii) how o ansla e hem in o quan i a i e sco es,
lea ing bo h decisions up o he code . Addi ional complexi y a ises due o
ac ha he coding o many indica o s is his o ic and o en imes goes back
se e al yea s i no decades. In essence, he coding o hese indica o s is a
decision p ocess wi h a conside able le el o unce ain y.
Gi en ha a coding p ocess o his kind p o ides ew guidelines o code s,
i is likely o be a ec ed by di e en biases in human decision making. Recen
wo k is inc easingly ying o unde s and hese cogni i e p ocesses. Colgan
(2019), o example, a gues ha human-coded da ase s in IR may ha e an
“Ame ican”bias, since hey a e p oduced by code s holding Ame ican alues
and iews o he in e na ional sys em. A non e al. (2023) howe e , examining
quan i a i e human igh s sco es p oduced by humans, find no e idence o
bias. So a he s onges c i ique o human bias in poli ical a ings has been
pu o wa d by Li le and Meng (2023), who a gue ha findings o democ a ic
decline a e he esul o changes in human pe cep ions bu no in poli ical
ins i u ions. Compa ing subjec i e and objec i e democ acy indica o s, hey
show ha democ a ic backsliding can be de ec ed in he o me bu no he
la e .
This pape examines ye ano he ype o bias ha could a ec human
code s. Gi en ha coding decisions o his o ic cases a e pa icula ly di ficul ,
i is easonable o assume ha code s d aw on pa icula cogni i e sho cu s.
The a ailabili y bias is one o he mos esea ched among hem. I means ha
when con on ed wi h a di ficul decision-making ask, humans use eadily
a ailable and closely ela ed in o ma ion o sol e i . In he coding o his o ic,
c oss-na ional a iables, his could mean ha human code s a e influenced by
ecen d ama ic e en s in he coun y in ques ion, e en i he coding applies o
922 Compa a i e Poli ical S udies 57(6)
a ime pe iod many yea s ago. Using ime-s amped code -le el da a om he
V-Dem p ojec , he analysis es s how hese ecen e en s a ec code a ings.
The esul s show ha he e is some e idence o an a ailabili y bias in code
a ings, bu he magni ude emains gene ally small and is unlikely o a ec
esea ch done wi h hese indica o s in a majo way.
Cogni i e Biases in Human Coding
In compa a i e poli ical science esea ch, many human-coded da ase s exis
ha co e a a ie y o poli ical a iables. A (non-exhaus i e) lis shows ha
hese da a co e many “so ” a iables such as he ule o law (Wo ld Jus ice
P ojec , 2020), condi ions o independen media (In e na ional Resea ch &
Exchanges Boa d, 2016), p ess eedom (Repo e s wi hou Bo de s, 2020),
poli ical igh s and ci il libe ies (F eedom House, 2020a), ins i u ional
cha ac e is ics o coun ies (Be ho, 2012), and, o cou se, democ acy
(Ma shall e al., 2018;Economis In elligence Uni , 2020;Coppedge e al.,
2020). P oducing hese coun y a ings is a di ficul ask o human code s.
Some equi e comple ely subjec i e assessmen s, such as, o example,
V-Dem’s a iable cap u ing he deg ee o a “ igo ous and impa ial public
adminis a ion.”
1
O he s a e a leas in p inciple based on ac ual ou comes o
e en s, bu i is e y di ficul o code s o collec all he in o ma ion and/o
agg ega e hem o a single coun y a ing acco ding o clea and well-specified
coding ules. Fo example, V-Dem codes co up ion in he legisla u e on a 0–
4 scale. The ac s suppo ing his coding may exis , bu hey a e unlikely o be
a ailable o he code .
2
Decisions unde unce ain y ha e been he subjec o esea ch o a long
ime, mos ly in psychology and ela ed disciplines. T e sky and Kahneman
(1974) desc ibe h ee heu is ics ha humans ha e been ound o employ when
hey ha e insu ficien knowledge as a basis o a decision. One o hem is he
well-known “a ailabili y heu is ic.”A ailabili y means ha when making a
decision, humans d aw on hose pieces o in o ma ion ha a e mo e easily
a ailable o hem. In his pape , we explo e a ailabili y on a empo al di-
mension. When coding a iables o a pa icula coun y o a ime pe iod ha
has long passed, i is well possible ha code s a e influenced by ecen ,
d ama ic e en s in he coun y in ques ion.
Wha d ama ic e en s a e mos likely o a ec coding decisions? Exis ing
esea ch has p oduced conclusi e e idence o a “nega i i y bias”in news,
whe e eade s a e mos likely o pe cei e and emembe nega i e e en s
(So oka e al., 2019). In he c oss-na ional s udy o democ acy, whe e code s
a e coun ies along a no ma i e dimension be ween “closed au oc acy”and
“libe al democ acy,” hese nega i e shi s co espond o se backs away om
democ acy, which can mani es hemsel es in mili a y coups o he iolen
ep ession o he opposi ion. Consequen ly, i code s a e a ec ed by
Weidmann 923
a ailabili y bias, hei e ospec i e a ings o a coun y should be educed i a
d ama ic nega i e e en immedia ely p ecedes he coding.
A sho example se es o illus a e his. Imagine wo code s a e he le el
o a ep ession in a gi en coun y, en yea s ago. This is he ypical coding ask
when p oducing c oss-na ional da ase s. The fi s code p oduces his a ing in
yea , he second code a yea la e a + 1. In be ween and + 1, he coun y
expe iences a la ge wa e o p o es ha is iolen ly ep essed by he go -
e nmen , wi h se e al casual ies. A ailabili y bias a ises i hese ecen e en s
influence he second code ’s a ing such ha i is lowe han he fi s code ’s
a ing, despi e he ac ha bo h code s a e he same his o ic case—bu hey
do so a di e en poin s in ime.
I code s o compa a i e da ase s a e a ec ed by he a ailabili y heu is ic,
he e a e di e en ways in which his can happen. In pa icula , we dis inguish
which coding decisions he a ailable in o ma ion (= ecen e en s) will be used
o . In a “na ow” e sion o a ailabili y bias, code s use his a ailable in-
o ma ion o decisions ha a ec he same ype o phenomenon hey a e
supposed o code. The abo e example illus a es his: I a coun y ecen ly
expe ienced a d ama ic e en ha clea ly shows a high le el o ep ession
(such as a iolen c ackdown agains he opposi ion), code s will use his
in o ma ion when a ing he same ou come— he le el o iolen go e nmen
ep ession in his coun y. This na ow e sion o a ailabili y bias he e o e
sugges s ha ecen d ama ic e en s o go e nmen al ep ession a ec he
e ospec i e coding o ep ession- ela ed a iables.
Howe e , a ailabili y bias can also play ou in a b oade sense. Ra he han
influencing he coding o only hose a iables ha a e di ec ly ela ed o
obse ed d ama ic e en s, hese e en s could also a ec o he a iables. Fo
example, ha ing obse ed go e nmen iolence agains p o es e s, code s
may implici ly downg ade hei no ma i e assessmen o a egime and
he e o e assess his egime no jus as ep essi e bu also as “co up ,”
“clien elis ic,”“undemoc a ic,”and “illibe al.”I his b oade e sion o
a ailabili y holds, we should see an e ec o ecen d ama ic e en s also on a
b oad ange o no ma i e a iables, whe e code s change hei e ospec i e
assessmen o hese coun ies and a e hem as mo e illibe al a e hese e en s.
In he ollowing analysis, we pu he na ow and he b oad e sion o
a ailabili y bias o an empi ical es .
Resea ch Design
The empi ical analysis uses ime-s amped code a ings o a iables ela ed o
democ acy and democ a ic ins i u ions. In he ollowing, we desc ibe he da a
sou ce and he esea ch design o he analysis.
924 Compa a i e Poli ical S udies 57(6)
V-Dem Code -le el Da a
The empi ical analysis elies on he code -le el da a om Ve sion 10 o he
V-Dem p ojec (Coppedge e al., 2020), a la ge da a collec ion e o ha
cap u es many di e en aspec s o “democ acy”and is one o he leading
da ase s in he c oss-na ional analysis o egimes. V-Dem dis inguishes be-
ween ac ual (Types A and B) and mo e subjec i e (Type C) a iables
(Coppedge e al., 2020, p. 28). The la e cons i u e V-Dem’s key con ibu ion
and a e he main ocus o his p ojec . Type C a iables cap u e subjec i e
a ings o poli ical a iables, p oduced by a numbe o code s wi h pa icula
expe ise abou a coun y o egion. Each o hese code s answe s a se o
ques ions abou he coun y and ime pe iod hey ha e been assigned. Answe s
o he ques ions a e eco ded using di e en scales; ew ques ions ha e a
bina y esponse (yes/no), while mos o he s ha e an o dinal scale.
The V-Dem code -le el da ase con ains he indi idual a ings p oduced by
he code s be o e hey a e p ocessed and agg ega ed u he . In he e sion
made a ailable o his esea ch, each coding has a ime s amp associa ed o i ,
indica ing he yea in which i was p oduced. V-Dem codings a e usually
collec ed in Janua y o co e he p e ious (and o en imes also ea lie ) yea s.
To maximize he amoun o da a o his analysis while e aining compa a-
bili y be ween a iables, only a iables wi h an o dinal scale o 0–4 ( he mos
common scale in V-Dem) we e selec ed.
3
Low alues o hese a iables
indica e illibe al poli ical p ac ices, and high alues co espond o libe al-
democ a ic ea u es o ou comes. The abo emen ioned example o he
2lgc p a iable co e ing co up ion in he legisla u e is an example o his.
Figu e 1 illus a es he s uc u e o he code -le el da a om V-Dem. On he
imeline along he x-axis, he e is he ac ual case o be coded (he e, he
2lgc p a iable o Egyp in 2016). This a iable is hen coded by h ee
code s in ea ly 2018 and wo code s in 2019. We use he e m “coding ime” o
deno e he ime (yea ) in which a a iable is coded. In he example, 2lgc p
has wo coding imes: 2018 and 2019.
Figu e 1. Da a s uc u e o he V-Dem code -le el da a. A a iable ( 2lgc p )is
coded o a pa icula case (Egyp in 2016) a wo di e en coding imes (2018 and
2019).
Weidmann 925

Uni o Obse a ion
In almos all cases, a pa icula a iable o a gi en case is only coded once by
a pa icula code , which is why we canno analyze changes wi hin he same
code ’s a ings o e ime. Fo ha eason, all a ings o a gi en a iable and
case a e a e aged by coding ime; in he abo e example, his would gi e us
wo alues o 2lgc p in Egyp 2016: a coding ime 2018, he a e age o
code s 14, 16, and 20; and a coding ime 2019, he a e age o code s 34 and
41. O en imes, he e is only a single code a ing o a pa icula coding ime.
4
Fo he analysis, we make use o he ac ha in he V-Dem p ojec , codings
o pa icula coun y-yea s a e o en imes gene a ed a di e en poin s in
ime. These epea ed obse a ions allow us o examine changes in he a e age
coding decisions o e ime, be ween he di e en coding imes. Fo he
analysis, we analyze pai s o codings made in consecu i e yea s, in o he
wo ds, whe e coding ime 1 and coding ime 2 a e exac ly one yea apa .
While o he compa isons a e possible (e.g., we analyze changes in code
a ings be ween a gi en yea and h ee yea s la e ), his app oach allows us o
be e a ibu e coding changes o he e en s ha happened in be ween he wo
coding imes. Fo all pai s p o ided in he code -le el da a, we examine how
shi s in he FH sco e in be ween he wo coding imes a ec downg ades in
he coding.
Measu ing D ama ic E en s
The pu pose o he empi ical analysis is o es whe he ecen d ama ic
poli ical de elopmen s in a coun y be ween coding ime1 and coding ime2
a ec code s’subjec i e a ings o pas cases. We s a by using di e en e en
da ase s o cap u e specific e en s ha indica e a wo sening and inc easingly
illibe al poli ical si ua ion. The fi s o hese indica o s is he occu ence o
e en s wi h iolen ep ession o p o es in he coded coun y. This a iable is
coded om he “Mass Mobiliza ion Da ase ”by Cla k and Regan (2021),
selec ing p o es e en s whe e he go e nmen esponded wi h “killings”o
“shoo ings.”The second e en -based p edic o is he numbe o coups o coup
a emp s, coded om he “Cline Cen e Coup D’´
e a P ojec Da ase ”(Pey on
e al., 2020). Bo h ypes o e en s a e usually associa ed wi h high le els o
poli ical iolence o supp ess poli ical opposi ion o o ous a democ a ic
go e nmen .
These e en -based p edic o s, howe e , co e only a small subse o
de elopmen s ha indica e whe he a coun y is becoming inc easingly il-
libe al. In ac , he e is a b oad ange o e en s ha indica e he de e io a ion o
democ acy and could he e o e influence code s’ e ospec i e assessmen .
Fo example, he s uggle su ounding Poland’s cons i u ional cou in
2015–2016 was a clea and isible indica o o democ a ic decline bu is
926 Compa a i e Poli ical S udies 57(6)
ob iously no cap u ed by he wo e en da ase s. Ra he han expanding he
se o e en -based p edic o s (which may no e en be easible due o limi ed
da a a ailabili y), we use an agg ega e indica o ha cap u es a b oad ange o
shi s away om libe al democ acy: he well-known F eedom House
“F eedom in he Wo ld” a ing (FH hence o h). The FH sco es quan i y a
coun y’s le el o poli ical igh s and ci il libe ies wi h an annual sco e
be ween 0 and 100 (F eedom House, 2020a). This sco e is compu ed as he
sum o se e al cons i uen indica o s, cap u ing he elec o al p ocess, poli ical
plu alism and pa icipa ion, he unc ioning o go e nmen , he eedom o
exp ession and belie , associa ional and o ganiza ional igh s, he ule o law,
as well as pe sonal au onomy and indi idual igh s.
In many cases, F eedom House a ings change slowly. These changes a e
unlikely o be isible o code s and he e o e no expec ed o influence hei
coding as hey do no cons i u e “d ama ic”e en s. We he e o e selec majo
changes om F eedom House in wo ways. Fi s , we selec cases whe e a
coun y was downg aded by a leas 5 poin s in he FH scale wi hin a single
yea , o cap u e majo poli ical shi s away om democ acy. These majo
shi s happen a ely. Since we can only exploi a ia ion in he coding imes,
we can only use changes du ing he yea s 2016–2020. Du ing his ime, only
abou 3% o he coun y-yea s expe ienced FH d ops o 5 o mo e. A second
way o iden i y majo downg ades is by using he h ee FH s a us ca ego ies.
FH classifies a coun y as “ ee,”“pa ly ee,”o “no ee,”depending on he
alues o he unde lying poli ical igh s and ci il libe ies sco es (F eedom
House, 2020b). A s a us downg ade happens i a coun y d ops om “ ee” o
“pa ly ee”o om “pa ly ee” o “no ee.”These s a us downg ades a e
majo e en s ha a e discussed in he annual FH news elease, so hey a e
likely o be isible o he code s. In he analysis below, we es whe he om
any coding yea o he nex , a d op in he FH sco e o a leas 5 poin s o a
s a us downg ade leads o an adjus men o he code a ings downwa d.
Resul s
In he empi ical analysis, we assess he ex en o which ecen , d ama ic e en s
in a coun y a ec he code s’ e ospec i e a ings. To do so, we p oceed in a
s epwise ashion. We s a wi h a pooled analysis using se s o V-Dem
a iables, be o e es ing each a iable independen ly. The analysis ocuses
on he change in he code a ings be ween wo consecu i e yea s. Mo e
p ecisely, he dependen a iable is he di e ence o he a e age V-Dem code
a ing be ween coding ime1 and coding ime2. O e all, code assessmen s a e
ela i ely s able. Ou o he 117,817 obse a ions in he main da ase , in
31,482 cases (abou 27%) he coding emains unchanged, while upg ades
(44,288, abou 37%) and downg ades (42,047, abou 36%) occu almos a he
same a e. The dis ibu ion o changes in code a ings om one yea o he
Weidmann 927
nex is isualized in Figu e 2. I shows ha while he as majo i y o a ings
do no change (la ge peak a he cen e ), downg ades and upg ades a e simila
in magni ude and almos pe ec ly balanced. Appendix A2 p esen s summa y
s a is ics a he le el o he a iables, dis inguishing be ween a ia ion be-
ween he indi idual cases ( he s anda d de ia ion o he a e age code a ings
o each coun y/yea ) and he a e age a ia ion wi hin a case ( he mean ac oss
he s anda d de ia ions o he code a ings o each coun y/yea ).
Fo he analysis, we use OLS models wi h he fi s di e ence o he code
a ings ( he a ing a coding ime2 minus he a ing a coding ime1) as he
dependen a iable. The pooled models include fixed e ec s o he di e en
V-Dem a iables, since he scaling o each o hem could en ail pa icula ly
low o high changes om yea o yea . In addi ion, he models clus e he
s anda d e o s a he le el o cases (coun y/yea s), since hese obse a ions
a e subjec o he same poli ical de elopmen s and he e o e no independen .
Changes in Code Ra ings
In line wi h he heo e ical discussion abo e, we fi s es a na ow e sion o
he a ailabili y heu is ic, whe e code s a e influenced by ecen e en s in he
coun y o be coded bu use his in o ma ion only o code a iables ha a e
closely ela ed o hese e en s. The fi s eg ession uses he e en -based
indica o s o d ama ic iolen e en s ( ep essed p o es and coups) o see i
hese e en s a ec he coding o V-Dem a iables ela ed o iolence and
ep ession. In V-Dem, he e a e fi e o hese a iables; hese include 2cl o
( eedom om o u e), 2clkill ( eedom om poli ical killings), 2cs ep ss
(go e nmen al ep ession o ci il socie y o ganiza ions), 2cs lg ep (go -
e nmen al ep ession o eligious o ganiza ions), and 2meha j n (physical
ha assmen o jou nalis s).
The fi s wo models in Table 1 es how he iolen ep ession o p o es
(Model 1) and he occu ence o a coup (Model 2) in be ween he wo coding
imes a ec he changes in he code a ings. In line wi h ou expec a ion, bo h
Figu e 2. Dis ibu ion o di e ences in he code a ings.
928 Compa a i e Poli ical S udies 57(6)
ecei e a nega i e e ec , which indica es ha he occu ence o hese e en s is
associa ed wi h a d op in he code a ings. The magni ude o he e ec is
lowe o ep essed p o es , whe e an e en o his kind is associa ed wi h a
d op in he code assessmen s by only abou .15 on he 0–4 scale. The e ec o
a coup is conside able, indica ing ha e ospec i e a ings o ep ession-
ela ed V-Dem a iables a e on a e age .6 poin s lowe on he 0–4 scale i a
coun y expe ienced a coup.
In Models 3–6inTable 1, we expand he analysis such ha i includes all
V-Dem a iables as well as he al e na i e indica o s o d ama ic e en s based
on F eedom House. Models 3 and 4 es ima e he impac o he e en -based
p edic o s ( ep essed p o es and coups) on changes in he a ings o all
a iables, Models 5 and 6 use he occu ence o an FH downg ade o a leas
5 poin s (Model 5) o a s a us downg ade (Model 6) as p edic o s. The co-
e ficien s in hese models a e much smalle h oughou . Only ep essed p o es
is significan , bu again in he di ec ion we hypo hesized. The es ima e shows
ha he occu ence o physical ep ession agains p o es e s leads o a
Table 1. E ec o Recen E en s in Coded Coun y on Changes in Code Ra ings.
OLS Models Wi h Va iable FEs and S anda d E o s Clus e ed by Case (Coun y/
Yea ).
Dependen a iable
ΔCode a ing
Rep ession- ela ed a iables All a iables
(1) (2) (3) (4) (5) (6)
Rep essed
p o es
.148***
(.030)
.075***
(.016)
Coup o coup
a emp
.616***
(.079)
.072
(.046)
FH downg ade
≥4
.023
(.028)
FH s a us
downg ade
.083
(.059)
V-Dem a iable
FEs
Yes Yes Yes Yes Yes Yes
Clus e ed SEs
(c /yea )
Yes Yes Yes Yes Yes Yes
Obse a ions 13,431 13,431 117,817 117,817 117,817 117,817
Adjus ed R2 .008 .008 .002 .002 .002 .002
No e.*p< .05; **p< .01; ***p< .001.
Weidmann 929
Da a A ailabili y S a emen
Replica ion ma e ials and code can be ound in he Compa a i e Poli ical S udies
Da a e se a h ps://doi.o g/10.7910/DVN/FKQI6M.
Supplemen al Ma e ial
Supplemen al ma e ial o his a icle is a ailable online.
No es
1. Va iable 2cl spc , ques ion: “A e public o ficials igo ous and impa ial in he
pe o mance o hei du ies?”(Coppedge e al., 2020, 164).
2. Va iable 2lgc p , ques ion: “Do membe s o he legisla u e abuse hei posi ion o
financial gain?”(Coppedge e al., 2020, 138).
3. To ensu e compa ibili y wi h o he annual codings, a iables ha do no ollow an
annual coding pa e n (e.g., hose ela ed o elec ions) we e also excluded. We also
exclude a iables ha do no cons i u e no ma i e assessmen s, such as he ques ion
abou he dominan chambe in bicame al legisla u es. See he appendix o he
comple e lis o a iables included.
4. This esul s in (non-sys ema ic) measu emen e o , which is unlikely o bias he
esul s we ob ain om he agg ega ed codings.
Re e ences
A non, D., Haschke, P., & Pa k, B. (2023). The igh accoun ing o w ongs: Examining
empo al changes o human igh s moni o ing and epo ing. B i ish Jou nal o
Poli ical Science,53(1), 163–182. h ps://doi.o g/10.1017/s0007123421000661
Be ho, F. (2012). “P esen a ion o he ins i u ional p ofiles da abase 2012.”h p://
www.cepii. /ins i u ions/doc/IPD_2012_cahie s-2013-03_EN.pd
Cla k, D., & Regan, P. (2021). Mass mobiliza ion p o es da a. Ha a d Da a e se.
h ps://massmobiliza ion.gi hub.io
Colgan, J. D. (2019). Ame ican bias in global secu i y s udies da a. Jou nal o Global
Secu i y S udies,4(3), 358–371. h ps://doi.o g/10.1093/jogss/ogz030
Coppedge, M., Ge ing, J., Lindbe g, S. I., Jan, T., Al man, D., Be nha d, M., S e en Fish, M.,
Glynn, A., Allen, H., Anna, L., Ma qua d , K. L., Kelly, M. M., Pax on, P., Pems ein, D.,
Sigman, R., S a on, J., Wilson, S., Co nell, A., Alizada, N., & Medziho sky, J. (2020).
V-dem da ase V10 codebook. Va ie ies o democ acy (V-dem) p ojec .
Economis In elligence Uni . (2020). “Democ acy index 2019: A yea o democ a ic
se backs and popula p o es .”Economis In elligence Uni . h ps://www.eiu.
com/ opic/democ acy-index
Fa iss, C. J. (2014). Respec o human igh s has imp o ed o e ime: Modeling he
changing s anda d o accoun abili y. Ame ican Poli ical Science Re iew,108(2),
297–318. h ps://doi.o g/10.1017/s0003055414000070
F eedom House. (2020a). F eedom House Coun y Ra ings.h p://www. eedomhouse.o g.
936 Compa a i e Poli ical S udies 57(6)

F eedom House. (2020b). F eedom in he wo ld 2020 me hodology. Me hodo-
logical Documen a ion. h ps:// eedomhouse.o g/ epo s/ eedom-wo ld/
eedom-wo ld- esea ch-me hodology
In e na ional Resea ch & ExchangesBoa d. (2016). Media sus ainabili y index (MSI).
h ps://www.i ex.o g/ esou ce/media-sus ainabili y-index-msi
Kellam, M., & S ein, E. A. (2016). Silencing c i ics: Why and how p esiden s es ic
media eedom in democ acies. Compa a i e Poli ical S udies,49(1), 36–77.
h ps://doi.o g/10.1177/0010414015592644
Li le, A., & Meng, A. (2023). Subjec i e and objec i e measu emen o democ a ic
backsliding. SSRN. h ps://www.ss n.com/abs ac =4327307
Lüh mann, A., & Lindbe g, S. I. (2019). A hi d wa e o au oc a iza ion is he e: Wha
is new abou i ? Democ a iza ion,26(7), 1095–1113. h ps://doi.o g/10.1080/
13510347.2019.1582029
Ma shall, M. G., Ted, R. G., & Kei h, J. (2018). Poli y IV p ojec : Poli ical egime
cha ac e is ics and ansi ions, 1800-2018. Technical Repo . h ps://www.
sys emicpeace.o g/insc /p4manual 2018.pd
Pems ein, D., Ma qua d , K. M., Tzelgo , E., Wang, Y., Medziho sky, J., K usell, J.,
Mi i, F., & Johannes, V. R. (2020). The V-dem measu emen model: La en
a iable analysis o c oss-na ional and c oss- empo al expe -coded da a.
Technical epo V-Dem P ojec . h ps:// -dem.ne /s a ic/websi e/files/wp/
wp_21_5 h.pd .
Pey on, B., Joseph, B., Shalmon, D., Ma in, M., & Bonagu o, J. (2020). Cline cen e
coup D’´
e a p ojec da ase cline cen e o ad anced social esea ch.
V.2.0.0 No embe 16. Uni e si y o Illinois U bana-Champaign. h ps://doi.o g/
10.13012/B2IDB-9651987_V3
Repo e s wi hou Bo de s (2020). Wo ld p ess eedom index.h ps:// s .o g/en/ anking.
So oka, S., Fou nie , P., & Ni , L. (2019). C oss-na ional e idence o a nega i i y bias
in psychophysiological eac ions o news. P oceedings o he Na ional Academy
o Sciences o he Uni ed S a es o Ame ica,116(38), 18888–18892. h ps://doi.
o g/10.1073/pnas.1908369116
T e sky, A., & Kahneman, D. (1974). Judgmen unde unce ain y: Heu is ics and biases.
Science,185(4157), 1124–1131. h ps://doi.o g/10.1126/science.185.4157.1124
Wo ld Jus ice P ojec . (2020). Rule o law index. Wo ld Jus ice P ojec . h ps://
wo ldjus icep ojec .o g/ou -wo k/ esea ch-and-da a/wjp- ule-law-index-2020
Au ho Biog aphy
Nils Weidmann is P o esso o Poli ical Science and head o he “Com-
munica ion, Ne wo ks and Con en ion”Resea ch G oup a he Uni e si y o
Kons anz. His esea ch in e es s include poli ical p o es and iolen conflic ,
he poli ical impac s o new communica ion echnology, and poli ical
me hodology.
Weidmann 937