Voter information campaigns and political accountability: Cumulative findings from a preregistered meta-analysis of coordinated trials

Dunning, Thad et al. Article — Published Version Voter information campaigns and political accountability: Cumulative findings from a preregistered meta-analysis of coordinated trials Science Advances Provided in Cooperation with: WZB Berlin Social Science Center Suggested Citation: Dunning, Thad et al. (2019) : Voter information campaigns and political accountability: Cumulative findings from a preregistered meta-analysis of coordinated trials, Science Advances, ISSN 2375-2548, American Association for the Advancement of Science, Washington, DC, Vol. 5, Iss. 7, pp. 1-10, https://doi.org/10.1126/sciadv.aaw2612 This Version is available at: https://hdl.handle.net/10419/209749 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ advances.sciencemag.org/cgi/content/full/5/7/eaaw2612/DC1 Supplementary Materials for Voter information campaigns and political accountability: Cumulative findings from a preregistered meta-analysis of coordinated trials Thad Dunning*, Guy Grossman, Macartan Humphreys, Susan D. Hyde, Craig McIntosh, Gareth Nellis, Claire L. Adida, Eric Arias, Clara Bicalho, Taylor C. Boas, Mark T. Buntaine, Simon Chauchard, Anirvan Chowdhury, Jessica Gottlieb, F. Daniel Hidalgo, Marcus Holmlund, Ryan Jablonski, Eric Kramon, Horacio Larreguy, Malte Lierl, John Marshall, Gwyneth McClendon, Marcus A. Melo, Daniel L. Nielson, Paula M. Pickering, Melina R. Platas, Pablo Querubín, Pia Raffler, Neelanjan Sircar *Corresponding author. Email: thad.dunn[email protected] Published 3 July 2019, Sci. Adv. 5, eaaw2612 (2019) DOI: 10.1126/sciadv.aaw2612 This PDF file includes: Section S1. Study design materials and methods Section S2. Primary analysis: Robustness and reliability of results Section S3. Secondary analysis: A Bayesian approach Section S4. Possible explanations for the null findings Section S5. Effects of publicly disseminated information Table S1. Individual study designs. Table S2. Descriptive statistics for sample of good news. Table S3. Descriptive statistics for sample of bad news. Table S4. Balance of covariates. Table S5. Effect of information, conditional on distance between information and priors, on vote choice, and turnout. Table S6. Deviations from MPAP and study PAPs in the meta-analysis. Table S7. Differential attrition. Table S8. Manipulation check: Effect of treatment on correct recollection, pooling good and bad news (unregistered analysis). Table S9. Manipulation check: Absolute difference between posterior and prior beliefs for pooled good and bad news (unregistered analysis). Table S10. Effect of information on perception of importance of politician effort and honesty. Table S11. Effect of information and source credibility on evaluation of politician effort and honesty (unregistered analysis). Table S12. Relationship between evaluation of politician effort and honesty with vote choice (unregistered analysis). Table S13. Effect of bad news on politician backlash. Table S14. Additional hypotheses and results. Table S15. Effect of moderators on incumbent vote choice. Table S16. Effect of information and context heterogeneity on incumbent vote choice. Table S17. Effect of information and electoral competition on vote choice. Table S18. Effect of information and intervention-specific heterogeneity on vote choice. Table S19. Interaction analysis: Effect of good news on incumbent vote choice. Table S20. Interaction analysis: Effect of bad news on incumbent vote choice. Table S21. Private versus public information: Effect of good news on incumbent vote choice. Table S22. Private versus public information: Effect of bad news on incumbent vote choice. Fig. S1. Benin—Graphical representation of provided information. Fig. S2. Brazil—Flyers distributed to voters. Fig. S3. Burkina Faso—Flashcard illustrations of municipal performance indicators. Fig. S4. Mexico—Example of benchmarked leaflet in Ecatepec de Morelos, México. Fig. S5. Uganda 1—Candidate answering questions during a recording session and candidate as seen in video. Fig. S6. Power analysis of minimal detectable effects, computed using Monte Carlo simulation. Fig. S7. Bayesian meta-analysis: Vote choice. Fig. S8. Bayesian meta-analysis: Turnout. 1. Study design and methodsSection S 1.1 Meta Pre-Analysis Plan (MPAP) We first reproduce the meta pre-analysis plan, which was filed on March 9, 2015 at http://egap.org/ registration/736, prior to the fielding of interventions and before collection of baseline data. Note that we have corrected minor spelling mistakes and made minor editorial changes, including bringing footnotes into the text, to conform with Science Advances journal formatting requirements. We otherwise present the MPAP as filed. Political Information and Electoral Choices: A Meta-Preanalysis Plan (MPAP) Thad Dunning#8 Guy Grossman#8 Macartan Humphreys#8 Susan Hyde#8 Craig McIntosh#8 Claire Adida#1 Eric Arias#2 Taylor Boas#4 Mark Buntaine#7 Sarah Bush#7 Simon Chauchard#3 Jessica Gottlieb#1 F. Daniel Hidalgo#4 Marcus E. Holmlund#5 Ryan Jablonski#7 Eric Kramon#1 Horacio Larreguy#2 Malte Lierl#5 Gwnyeth McClendon#1 John Marshall,#2 Dan Nielson#7 Melina Platas Izama#6 Pablo Querubin#2 Pia Raffler#6 Neelanjan Sircar#3. March 9, 2015 Abstract We describe our plan for a meta analysis of a collection of seven studies on the impact of information on voting behavior in developing countries. The seven studies are being conducted simultaneously by seven separate research teams under a single “Metaketa” grant round administered by EGAP and University of California, Berkeley’s Center on the Politics of Development. This analysis plan has been produced before launch of any of the seven projects and provides the analysis for the joint assessment of results from the studies. Individual studies have separate pre-analysis plans with greater detail, registered prior to the launch of each study. Author annotations are: # 1 Benin study, #2 Mexico study, #3 India study, #4 Brazil study, #5 Burkina Faso study, #6 Uganda 1 study, #7 Uganda 2 study, #8 the Metaketa committee. We have many people to thank for generous thoughts and comments on this project including Jaclyn Leaver, Abigail Long, Betsy Paluck, Ryan Moore, Ana de la O, Don Green, Richard Sedlmayr, and participants at EGAP 13. The Metaketa is funded by an anonymous donor. Contents 1 Introduction 1 2 Interventions and Motivation 1 2.1 PrimaryInterventionArm................................. 1 2.2 Secondary Intervention Arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.3 Additionalvariations.................................... 3 3 Hypotheses 4 3.1 PrimaryHypotheses .................................... 4 3.2 Hypotheses on Secondary Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.3 Hypotheses on Intermediate Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.4 Hypotheses on Substitution Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.5 Context Specific Heterogeneous Effects . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.6 Intervention Specific Heterogeneous Effects . . . . . . . . . . . . . . . . . . . . . . . 6 4 Measurement 6 4.1 Outcomemeasures ..................................... 6 4.1.1 Votechoice ..................................... 6 4.1.2 Turnout....................................... 7 4.1.3 Intermediateoutcomes............................... 7 4.2 Priors on Treatment Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.3 ControlsandModerators ................................. 8 4.3.1 Individuallevelitems ............................... 8 4.3.2 Treatmentlevelitems ............................... 9 4.3.3 Election (race) level features . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.3.4 CountryLeveldata................................. 10 4.3.5 ManipulationChecks................................ 10 5 Analysis details 10 5.1 MainAnalysis........................................ 11 5.2 Analysis of Heterogeneous Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5.3 Adjustment for multiple comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5.4 Contingencies........................................ 14 5.4.1 Non-Compliance .................................. 14 5.4.2 Attrition ...................................... 14 5.4.3 Missing data on control variables . . . . . . . . . . . . . . . . . . . . . . . . . 14 6 Additional (secondary) analysis 14 6.1 Randomization checks and balance tests . . . . . . . . . . . . . . . . . . . . . . . . . 14 6.2 Disaggregatedanalyses................................... 14 6.3 Controls........................................... 14 6.4 Possible additional analysis of official data . . . . . . . . . . . . . . . . . . . . . . . . 15 6.5 Bayesian hierarchical analysis model . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 6.6 Exploratoryanalysis .................................... 16 6.7 Learningaboutlearning .................................. 16 7 Ethics 17 8 Caveats 17 1 Introduction In this document we describe the research and analysis strategy for an EGAP “Metaketa” on information and accountability. Metaketas are integrated research programs in which multiple teams of researchers work on coordinated projects in parallel to generate generalizable answers to major questions of scholarly and policy importance. The core pillars of the Metaketa approach are: 1. Major themes: Metaketas focus on major questions of scholarly and policy relevance with a focus on consolidation of knowledge rather than on innovation. 2. Strong designs: all studies employ randomized interventions to identify causal effects. 3. Collaboration and competition: teams work on parallel coordinated projects and collaborate on design and on both measurement and estimation strategies in order to allow for informed comparisons across study contexts. 4. Comparable interventions and measures: differences in findings should be attributable primarily to contextual factors and not to differences in research design or measurement. 5. Analytic transparency: all studies share a commitment to analytic transparency including design registration, open and replicable data and materials, and third-party analysis prior to publication. 6. Formal synthesis: aggregation of results of the studies is achieved though pre-specified meta-analysis and via integrated publication platform to avoid publication bias. The Information and Accountability Metaketa was launched in Fall 2013 and will run until Spring 2018. Its key objective is to implement a series of integrated experimental projects that assess the role of information in promoting political accountability in developing countries. This Metaketa is being administered by the Center on the Politics of Development at the University of California, Berkeley. This first registration document (dated: March 15, 2015) has been posted publicly to the EGAP registry prior to the administration of treatment in any of seven projects taking part in this Metaketa. 2 Interventions and Motivation Civil society groups and social scientists commonly emphasize the need for high quality public information on the performance of politicians as an informed electorate is at the heart of liberal theories of democratic practice. The extent to which performance information in effect make a difference in institutionally weak environments is, however, an open question. Specifically when does such information lead to the rewarding of good performance candidates at the polls and when are voting decisions dominated by nonperformance criteria such as ethnic ties and clientelistic relations? The studies in this project address the above questions by examining a set of interventions that provide subjects with information about key actions of incumbent political representatives. We assess the effects of providing this information on vote choice and turnout, given prior information available to voters. 2.1 Primary Intervention Arm Each of the seven projects has at least two treatment arms. The first arm is an informational intervention focused explicitly on the performance of politicians. While the specific political office (e.g., mayor or member of parliament), the type of performance information provided, and the medium for communicating the information vary somewhat across studies, the interventions are designed to be as similar as possible to each other; they are also similar to several previous informational interventions in research on political accountability. Most importantly, each intervention is designed to allow voters to update their beliefs about the performance of the politicians positively or negatively in light of the information. The extent to which such updating actually takes place will play a key role in comparing the impact of the performance information across contexts. A very brief description of the primary informational treatment, T1, in each study is included in Table 1 below. We summarize the interventions here; for more details, see the pre-analysis plans for each individual study. •In Benin, researchers provide information to respondents on indices of legislative performance of deputies in the National Assembly. Videos featuring bar graphs highlight the performance of the legislator responsible for each commune and present this information relative to other legislators in the department (a local average) and the country (national average). •In Mexico, researchers provide information in advance of municipal elections on corruption (measured as the share of total resources that are used in an unauthorized manner) or on misuse of public funds (the share of resources that have benefited non-poor individuals from funds that are explicitly earmarked to poor constituents). •In India, researchers provide information on criminal backgrounds of candidates in state assembly races. Publicly available information, culled from India’s Election Commission, will be disseminated in a door-to-door campaign across 18 randomly selected polling booths within 25 electoral constituencies in the Indian state of Bihar. •In Brazil, researchers will distribute information about general government corruption in mayoral races. In partnership with the Accounts Court in the northeastern state of Pernambuco, the research team will provide voters with information on incumbent malfeasance via report cards and oral communication, drawing on publicly available data from annual auditing reports. •In Burkina Faso, researchers provide information on the performance of municipal governments with respect to national targets for public service delivery. After pilot tests, it will be determined whether this information will be presented in the form of relative performance rankings of the municipalities within a region, or in the form of scores that indicate a municipality’s performance relative to normative targets. •In Uganda (study 1), researchers provide information on service delivery in Parliamentary constituencies using scorecards. •In Uganda (study 2), researchers will use text-messaging (SMS) to provide information on service delivery in district government races. Specifically the researchers will disseminate information on local government budget allocations, as well as comparative quality of public services (roads, water supply, and solid waste). 2.2 Secondary Intervention Arm The studies include second arms that test conditions under which the provision of information might be more or less effective. As part of the second arm, studies assess the effects of variation in the message content (absolute or relative information), the type of messenger (surveyor vs. community elites), and delivery method (providing information collectively vs. individually to groups of voters). Many studies compare a public treatment which may generate common knowledge of the intervention to a private baseline. We refer to these secondary interventions as T2: •In Benin, researchers use a 2×2 factorial design plus pure control. One dimension of the factorial design concerns whether the information is provided in a public or private fashion. In the public condition, the informational video will be screened in a public location; a random sample of villagers will be invited to the film screening. In the private condition, the same video will be shown to randomly sampled individual in households in one-to-one interactions. The other dimension crosses the presence or absence of a civics message highlighting the implications of poor legislator performance for voter welfare. •In Mexico, researchers use a 2×2 factorial design plus control. Similarly to the Benin study, one treatment group will receive information about municipal-level corruption and misuse of funds only privately (via fliers) whereas another will receive this information in a public manner (using cars with megaphones). The other cross-cutting dimension concerns whether or not citizens also receive benchmark information about the state average. •In India, this study examines the causal effect of the information “messenger”. In one treatment group, surveyors will distribute a flyer and summarize the information included in the flyer in face-to-face interactions. In the second treatment group, locally influential individuals will be contracted to disseminate the exact same information in a similar manner. •The Brazil study explores the effect of varying the saliency of the information communicated to voters. Specifically, for the alternative arm, the researchers will provide information on mayoral compliance with a highly salient crop insurance program, which allows testing for the importance of providing information on policies directly relevant to voters’ lives. •In Burkina Faso, the alternative arm includes a personal invitation to a municipal council meeting. Here first-hand experience with the municipal decision process is expected to make the political information disseminated as part of the main (common) arm more salient to citizens. •In Uganda (study 1), researchers will provide information via screenings of structured debates of parliamentary candidates. The researchers plan to exploit an additional source of variation: intravs. inter-party competition (i.e. primaries of the ruling party vs general election). The idea is to explore whether performance information is more likely to have a bite in primary settings when the impact of partisanship on vote choice is minimized. •In Uganda (study 2), researchers will vary the saturation of the level of information. Because treatments in the second arm differ across studies by design, we will not conduct pooled analysis or formal comparison of the effects of many of these treatments. However, our final report and publications will present estimates of the effects of the second arm in each study, both in absolute terms and relative to the first arm in each project. In addition, we will compare the pooled effects of private vs. public treatments, as a way to assess whether the generation of common knowledge may strengthen the effects of informational interventions. These analyses may provide important hypotheses for further studies to assess rigorously, for example, through Metaketas in which promising secondary arms in our set of studies are tested as primary (common) arms. 2.3 Additional variations We inform respondents in surveys (including those assigned to control groups) that they may be provided with information on candidate quality, and we seek consent to participate. While this enhances subject autonomy, it also risks creating Hawthorne-type biases. To assess this possibility, a set of studies will also employ a variation, T3, that randomly varies the consent script among control units (though consent for measurement is sought in all cases). 4.3.3 Election (race) level features M25 competitiveness. This measure will vary across systems. •For candidates elected through single-member/first-past-the-post elections, this is 1 minus the margin of victory of the incumbent 1-(vote share - vote share of runner up) (historical data from the electoral commission). •For proportional representation (closed list) systems, a candidate ranked in position k of a party that received mseats out of n, is accorded competitiveness score of 1 −(1 + m−k)/n. Thus individuals positioned 1,2,3 in a party that received 3 out of 7 seats have competitiveness scores 4/7, 5/7, 6/7 respectively. •For proportional representation (open list) systems, this is the difference in raw votes of the incumbent and the vote share of the candidate who received the largest number of votes and did not receive a seat A general measure of free and fairness will be made by averaging standardized versions of the following two measures: M26 secretballot: Voter confidence in the secret ballot (baseline) •How likely do you think it is that powerful people can find out how you vote, even though there is supposed to be a secret ballot in this country? (1) Not at all likely (2) Not very likely (3) Somewhat likely (4) Very likely M27 freeandfair: Voter beliefs that the election will be free and fair in constituency (baseline) •How likely do you think it is that the counting of votes in this election will be fair (1) Not at all likely (2) Not very likely (3) Somewhat likely (4) Very likely 4.3.4 Country Level data M28 freepress. Freedom House measure of freedom of the press M29 democ. Polity measure of democratic strength 4.3.5 Manipulation Checks Manipulation checks data is also gathered which can be used to assess whether treatment groups absorbed the treatment (i.e., did the individual understand the information?); whether control groups learned more about representatives between baseline and the election; and whether there was informational spillovers between treated and control units. M30 check At endline, data should be gathered from treatment and control groups about the performance of representatives using the same approach as used for Measure M9. (Here we recognize that voters could have absorbed the information and yet posteriors over candidates on the dimension of the information may not have budged—perhaps because voters filter the information through partisan lenses.) 5 Analysis details In this section we describe the primary empirical strategy that will be used to test the above set of hypotheses across studies. The most straightforward way to combine results across the seven studies pools units into one large study group and estimates treatment effects, as one would do in a large experiment in which treatment assignment is blocked. For this analysis we proceed as if blocking is implemented at the country level. From one perspective, this approach involves weak assumptions. The study group in the large experiment is not conceived as a random sample from a larger population. This follows from the design of the studies: in most of the seven projects, individuals in the study groups are not themselves random samples, and the study sites (countries and locations within countries) are also not random draws from a well defined population of possible sites. From another perspective, pooling does imply that we can treat interventions and outcome measures as sufficiently comparable that an overall average treatment effect (say, the effect on vote choice of exposure to “good news”) is meaningful. Creating such comparability is the goal of the Metaketa initiative, but in practice the information that is provided in different projects differs quite substantially, even when focusing explicitly the primary information arm. We account for this heterogeneity partially by formally examining the effects of heterogeneity in our analysis. 5.1 Main Analysis Since expected effects derive from new information rather than any information, the core estimates need to take account of both the content of the information and prior beliefs. Let Pij denote the prior beliefs of voter iregarding some politically relevant attribute of politician jand let Qjdenote the information provided to the treatment group about politician jon that attribute, measured on the same scale. Let ˆ Qjdenote the median value of Qjin a polity (or, for teams using local comparison groups, the median in the relevant comparison group). Define L+as the set of treatment subjects for whom Qj> Pij or Qj=Pij and Qj≥ˆ Qj. These are subjects that receive good news — either the information provided exceeds priors or the information confirms positive priors. Let L−denote the remaining subjects. Let N+ ij denote the difference Qj−Pij, defined for all subjects in L+and standardized by the mean and standard deviation of Qj−Pij in the L+group in each country (or relevant locality). N+ ij is therefore a standardized measure of “good news” with mean 0 and standard deviation of 1. Let N− ij denote the same quantity but for all subjects receiving bad news. Then the two core estimating equations are E(Yij|i∈L+) = β0+β1N+ ij +β2Ti+β3TiN+ ij + k X j=1 (νkZk i+ψkZk iTi) (1) E(Yij|i∈L−) = γ0+γ1N− ij +γ2Ti+γ3TiN− ij + k X j=1 (νkZk i+ψkZk iTi) (2) where Z1, Z2, ..., Zkare prespecified covariates, also standardized to have a 0 mean. Here β2is the average treatment effect of information for all voters receiving good news; γ2is the average treatment effect of information for all voters receiving bad news. Recall that according to H1a and H1b we expect β2>0 and γ2<0. Note that models 1 and 2 assume that potential outcomes (e.g. vote choice or turnout after good news, bad news, or no news) are fixed and may differ from individual to individual; the only random element in the above models is assignment to the treatment condition Ti(given priors, which by definition are determined before treatment assignment). In addition to reporting these as our primary results we will report the results for the analogous specification without covariates. We will also report the mean value of Yij by treatment condition for both sets of individuals (those in L+and those in L−), i.e., without conditioning on N+ ij or N− ij . Estimation is conducted using OLS, clustering standard errors on politicians (j) and adding fixed effects for constituencies. If treatment assignment is blocked within projects, and treatment assignment probabilities vary across blocks, analysis will account for the blocking, e.g. by the weighting of block-specific effects (or fixed effects for blocks when appropriate). For analysis of aggregate data with clustered assignment, variables are aggregated to their cluster means (where cluster is the level of treatment assignment) or standard errors are clustered at this level. If no uniform weights are used, inverse propensity weights will be employed. 5.2 Analysis of Heterogeneous Effects Following from the main estimating equations, for a covariate Xij the heterogeneous effect of positive and negative information will be estimated through interaction analysis. Note that we again do not pool since we expect heterogeneous effects to work differently for good news and bad news, as is the case if a covariate is associated with stronger or weaker effects. E(Yij|i∈L+) = β0+β1N+ ij +β2Ti+β3TiN+ ij +β4Xi+β5TiXi+Pk j=1(νkZk i+ψkZk iTi)eq.het1(3) E(Yij|i∈L−) = γ0+γ1N− ij +γ2Ti+γ3TiN− ij +γ4Xi+γ5TiXi+Pk j=1(νkZk i+ψkZk iTi)(4) Where Xis the variable of interest (which we assume is not included in the set of other covariates Z). The heterogeneous effects of the impact of positive information, for average news levels, are given by β5and the heterogeneous effects of negative information are given by γ5. Note that we do not include a triple interaction between T, X and N+/N−in these analyses. For H12 we can combine data and estimate more simply: E(Yij) = δ0+δ1(Qj−Pij) + δ2Ti+δ3Ti(Qj−Pij) (5) Under H12 we expect δ3>0. Note that our measures of Qj−Pij are largely ordinal not interval; and estimating a linear marginal effect of the gap may not be meaningful if the marginal effect is not in fact linear. Perhaps more importantly, we do not manipulate priors in our experiments, and we lack an identification strategy that would allow us to make strong causal claims about the effects of such a gap. Such caveats should be born in mind, yet we believe it is valuable to assess H12 with the tools at our disposal. The mapping between hypotheses (section 3) and measures (section 4) is outlined in Table 2. Where controls are: •for individual level specifications: {M14, M15, M16, M17, M18, M19, M20, M21, M22, M26, M27} •for cluster level specifications: averages of {M15, M17, M18, M19, M20, M21, M22, M26, M27} 5.3 Adjustment for multiple comparisons We handle multiple comparisons concerns in two ways. First note that most tests are conducted using pairs of analyses—e.g. the (positive) effect of good news on voting and the (negative) effect of bad news. For each of these pairs of analyses, in addition to the simple pvalues reported for each regression, we will calculate a pvalue for the pair of regressions which will be given by the probability that both the coefficients would be as large (in absolute value) as they are under the sharp null of no effect of exposure to information (good or bad) for any unit. (We calculate this pvalue using randomization inference. Let f(b) denote a bivariate distribution of coefficients b1, b2generated under the sharp null, and let b∗= (b∗ 1, b∗ 2) Family # Abbreviated Hypothesis Y X Interact’n Controls Subset Spec’n Primary H1a Good news effects M1 T1 zM10=1 Eq1 (1) H1b Bad news effects M1 T1 zM10=0 Eq2 Secondary H2a Turnout (Good news) M3 T1 zM10=1 Eq 1 (2) H2b Turnout (Bad news) M3 T1 zM10=0 Eq 2 H4 Candidate effort M5 T1 ! M10=1 Eq1 H4 Candidate effort M5 T1 ! M10=0 Eq2 Mediators H3 Candidate integrity M6 T1 ! M10=1 Eq1 (3) H3 Candidate integrity M6 T1 ! M10=0 Eq2 H5 Candidate responses M8 T1 ! M10=0 Eq2 Substitution H6 Non coethnics M1 T1 M15 ! M10=1 Eq3 (4) H6 Non coethnics M1 T1 M15 ! M10=0 Eq4 H7 Partisanship M1 T1 M19 ! M10=1 Eq3 H7 Partisanship M1 T1 M19 ! M10=0 Eq4 H8 Clientelism M1 T1 M22 ! M10=1 Eq3 H8 Clientelism M1 T1 M22 ! M10=0 Eq4 Context H9 Informational environment M1 T1 M11 ! M10=1 Eq3 (5) H9 Informational environment M1 T1 M11 ! M10=0 Eq4 H10 Competitive elections M1 T1 M25 ! M10=1 Eq3 H10 Competitive elections M1 T1 M25 ! M10=0 Eq4 H11 Free and fair elections M1 T1 M26+M27 ! M10=1 Eq3 H11 Free and fair elections M1 T1 M26+M27 ! M10=0 Eq4 Design H12 Information content M1 T1 ! All Eq5 (6) H13 Information welfare relevant M1 T1 M23 ! M10=1 Eq3 H13 Information welfare relevant M1 T1 M23 ! M10=0 Eq4 H14 Credible Information M1 T1 M24 ! M10=1 Eq3 H14 Credible Information M1 T1 M24 ! M10=0 Eq4 H15 Public Channels M1 T1 T2 ! M10=1 Eq3 H15 Public Channels M1 T1 T2 ! M10=0 Eq4 H16 Hawthorne M1 T1 T3 ! M10=1 Eq3 H16 Hawthorne M1 T1 T3 ! M10=0 Eq4 Here, zindicates that we will present results with and without controls; see subsections 5.1 and 5.3. denote the estimated coefficients. Then the pvalue of interest is given by R1(min(|b|)≥min(|b∗|))× 1(max(|b|)≥max(|b∗|))f(b)db, where 1is an indicator function.) Second, for each of our six families of hypothesis, we will present tests using both nominal p-values and tests that employ a false discovery rate (FDR) correction to control the Type-1 error rate. We will control the FDR at level 0.05. Thus, for a given randomization with m(null) hypotheses and massociated p-values, we order the realized nominal p-values from smallest to largest, p(1) ≤p(2) ≤. . . ≤p(m). Let kbe the largest ifor which p(i)≤i m0.05 Then, we reject all H(i)for i= 1,2, . . . , k, where H(i)is the null hypothesis corresponding to p(i). Note that FDR corrections will be implemented using the estimated pvalues from pairs of tests. Thus for example if in a family there are three pairs of tests, then the FDR correction will be applied using three pvalues, one extracted from each pair. We consider as families of tests those outlined in Table 2. For example, for the primary hypotheses and outcomes, we consider good news effects and bad news effects on vote choice (with and without controls); for the primary hypotheses and secondary outcomes, we consider good news and bad news effects on turnout (with and without controls). MPAP Table 2: Specifications, Hypotheses and Measures 5.4 Contingencies 5.4.1 Non-Compliance Studies will analyze subjects according to their treatment assignation under the intended design, and the primary analysis will ignore non-compliance or failure to treat due to logistical mishaps. 5.4.2 Attrition If there is attrition for entire blocks containing four or more treatment and control units (for example if entire studies fail to complete or if regions within countries become inaccessible) these blocks will be dropped from analysis without adjustment unless there is substantive reason to believe the attrition is due to treatment status. Studies will test for two forms of attrition. First, are levels of attrition different across treatment and control groups? Second, are the correlates of attrition differential between the treatment and control? The former test will be conducted by comparing mean attrition in treatment and control groups, and reporting t-test statistics. The second test will be conducted regressing an attrition indicator on the interactions of treatment and the core baseline control measures specified above and reporting the F-statistic for all of the interacted variables. Data from studies that find no evidence for problematic attrition from these two tests will be analyzed ignoring attrition. If differential attrition is detected, Lee bounding techniques will be used to provide estimates of the magnitude of bias that could have resulted from differential attrition, from problematic studies, as well as testing whether the core findings of the study are robust to the observed rate of differential attrition. 5.4.3 Missing data on control variables If there is missing data on control variables, missing data will be imputed using block mean values for the lowest block for which data is available. 6 Additional (secondary) analysis In addition to the core analyses descibed above we will undertake a set of secondary analyses. 6.1 Randomization checks and balance tests Using the full set of baseline covariates described in this document we will report study-by-study Fstatistics for the hypothesis that all covariates are orthogonal to treatment. In addition we will report balance for all covariates in terms of the country-specific standard deviation of these covariates. 6.2 Disaggregated analyses In addition to the core metanalysis described here we will present the same analyses but conducted on all of the individual studies separately. 6.3 Controls Versions of the core tests described in Table 3 but without the use of any covariates will also be reported. 6.4 Possible additional analysis of official data For many studies official data on turnout and voting at the group level may become available. At this stage the granularity of this data is not known and, pending other official data, there is uncertainty about the polling station level dosage of interventions administered by the different studies. Official data has the advantage of being free of reporting biases (at least when elections are free and fair), but has the disadvantage of providing a noisy measure in cases with low dosage. The decision to include polling station areas for analysis using official data will be made as follows. Polling stations will be ordered, 1,2, . . . , k, . . . , n in terms of treatment intensity (share of registered voters exposed to treatment T1) within each study (separately for the good news and bad news groups). Then, for each kthe power to identify an effect as large as the estimated effect from the individual level analysis will be assessed, given an analysis including areas with density as large as kor greater. The largest group of polling station areas that collectively yield power of 50% or more will be included in this analysis. Note that with low dosages this set may be empty. For any included sets the analysis will assess the effect of treatment as follows: Define Dhas the share of cluster h(polling station area) individuals that would get treated if the unit were in treatment (dosage). Let Ddenote the (country specific) mean of D. Let D0=D−D denote Dnormalized to have a 0 mean. Then conditional on the polling station receiving good news (based on average values of Qi−Pij) estimate yh=β0+β1N+ j+β2Th+β3ThN+ h+β4ThD0 h+β5D0 h+ k X j=1 (νkZk i+ψkZk iTi) + h(6) where yhis the vote share for the incumbent, This the treatment status of the cluster, N+ jis the cluster average of N+ ij , normalized again to have 0 mean across clusters, the Zvariables are cluster level controls, and his an error term. Here β2is the estimated treatment effect for a unit with average dosage. β1/Dis the estimated individual level treatment effect (under the assumption of no spillovers), generated from the polling station level data. The analogous expression holds for bad news poling station areas. In implementing this analysis we are conscious of the risk of ecological biases since the good news assessment is defined based on a group average but treatment effects may be drived by different individuals. As robustness check we plan to supplement this analysis with the same analysis but not conditioning on Qonly and not Pij. Good news areas for that analysis will be areas with performance equal to or above the median. 6.5 Bayesian hierarchical analysis model A second analysis will employ Bayesian, multi-level meta-analysis techniques to allow for learning across cases and probe the sources of variation across cases. This approach requires stronger assumptions than the primary analysis but allows one to reassess the most likely estimates for each case in light of learning from other cases. The simplest approach, drawing on a canonical model, is of the following form. Say there are n1jtreated units and n0jcontrol units in study j. Let m1jand m0jdenote the number of votes for the incumbent among treated and control units in study jrespectively. Then the data model is mij ∼Bin(nij, pij for i∈ {0,1}(7) This captures simply the idea that the number of votes in favor of the incumbent is a draw from a binomial distribution with a given number of voters and a given probability of supporting the incumbent in each arm of each study. Working on the logit scale we define parameters: β1j=1 2(logit(p1j) + logit(p0j)) (8) β2j= logit(p1j)−logit(p0j) (9) These correspond to the average support for the incumbent and the treatment effect of the informational intervention, respectively. We are interested especially in β2jwhich corresponds to the average treatment effect in each study, on the log-odds scale. Our priors on the collection of pairs (β1j, β2j) is given by a product of bivariate normal distributions with parameters αand Λ: p(β|α, Λ) = 7 Y j=1 N β1j β2jα1 α2,Λ(10) Here α2is of particular interest corresponding to the population analogue of β2j. For hyperpriors we assume uninformative uniform priors over α1, α2,Λ11,Λ22 and the correlation Λ12/(Λ11Λ22)1 2. The quantities we extract are the treatment effects for each study (with credibility intervals) as well as the posteriors on α1,α2. In addition to this simple model we will report results from a second hierarchical logistic model that allows for systematic individual and study level variation in the same manner assumed in the core specification but allowing country level covariates to enter at the country level and cluster and individual level covariates enter at those levels. As with the core model, inverse propensity weights are included when non-uniform assignment propensities are employed. Again from this model study level average treatment effects will be estimated along with population parameters. 6.6 Exploratory analysis In addition to the core tests described above, the analysis will engage in more exploratory analyses to assess how treatments altered the decisions voters took (using measure M7) as well as the comparability of effects across sites. For the latter analysis the country level treatment effects will be compared in light of the effects of treatment on mediators — that is, we will seek to report the shift in voting outcomes for units of treatment scaled in terms of the effects of treatment on mediators. 6.7 Learning about learning One of the key tests of the usefulness of the Metaketa initiative is the extent to which the research and the policy communities learn from the aggregation of the coordinated studies. At the end of this Metaketa, we will gather a set of policymakers and academics, randomly divide them into samples, provide a briefing on the design of all studies, and then elicit prior beliefs about the effects of all studies. For treatment samples, we provide each with results from a random set of 5 of the studies, and incentivize them to provide updated expectations of results from the remaining studies. Some treatment samples will be encouraged (or required) to use predictive models while others will rely on subjective assessment and subject-matter knowledge. From this we expect to learn how results from some studies affect general beliefs, whether they make beliefs more accurate and how subjective inferences across studies fares relative to out-of-sample assessments of fitted models. The full analysis strategy for this component will be developed at a later stage. 7 Ethics All projects in the Metaketa will abide by a common set of principles above and beyond minimal requirements (i.e. securing formal IRB approvals, avoiding conflicts of interest, and ensuring all interventions do not violate local laws): •The egap principles on research transparency http://egap.org/resources/egap-statement-ofprinciples/ •Protect staff: Do not put research staff in harm’s way. •Informed consent: Subjects that are individually exposed to treatments will know that information they receive is provided as part of a research project. Core project data will be publicly available in primary languages at http://egap.org/research/Metaketa/ •Partnership with local civil society or governmental actors to ensure appropriateness of information •Non-partisan interventions: Only non-partisan information will be provided where by nonpartisan we mean that (1) it is coming from a non-partisan source; (2) it reveals information about performance of incumbents (candidates) regardless of their party. •Approval from the relevant electoral commission when appropriate The studies in general will not seek consent from individual politicians even though these may be affected by the interventions. The principle is that any information provided is information that exists in the political system that voters can choose to act upon or not and that this information is provided with consent, in a non-partisan way, without deception, and in cooperation with local groups, where appropriate. 8 Caveats We are conscious of a number of limitations of this research design which will be relevant for interpretation of some results. Most important are: 1. Although we are in the good position of being able to assess comparable interventions in multiple sites, these sites are not themselves random draws from a population of sites. They reflect case level features such as the timing of elections and the feasibility of doing research as well as research team features such as researcher connections to these sites. 2. Although the information that is provided in different areas share many features they also differ in systematic ways (see discussion above). 3. Although there is reasonable statistical power in individual studies and in pooled analyses; power is weak for assessing some heterogeneous effects, especially those operating at the country level. 4. By design, with information provided to voters and treatment status not assigned at the politician level or made known to politicians, the effects estimated are partial equilibrium effects. 5. Although we gather data on the information available to voters prior to administration of treatment (in all studies with a baseline survey), we do not know what information voters receive between baseline and the vote. Thus estimates should be interpreted as intent-totreat estimates even when treatment is delivered to all treatment units (and only those). Manipulation checks can be used to assess the extent to which treated and control units change beliefs between baseline and endline. 1.2 Study Designs Table 1 summarizes key information on the individual study designs. Further designs details are available at http://egap.org/metaketa/metaketa-information-and-accountability, where links are provided to the pre-analysis plans for the individual studies. Table S1. Individual study designs. Country Uganda 1 Uganda 2 Brazil Mexico Benin Burkina Faso India (as planned) Location(s) within country Central and Eastern Uganda Countrywide Pernambuco province States of Guanajuato, M´exico, San Luis Potos´ı, and Quer´etaro Countrywide Sahel, Centre-Nord, Plateau Central, Centre-Est, CentreSud, and Cascades regions Bihar state Governmental and nongovernmental partners Innovations for Poverty Action Twaweza Tribunal de Contas do Estado de Pernambuco (TCE-PE) Qu´e Funciona para el Desarrollo (QFD); Borde Politico Centre d’´ Etude et de Promotion de la D´emocratie Innovations for Poverty Action (IPA); Programme d’appui aux collectivit´es territoriales (PACT) Sunai Consultancy Intervention Information content Candidates answering six questions on their policy positions and backgrounds, including: (a) their priority policy-area for the constituency; (b) their position on whether additional administrative districts should be created; (c) their position on the legal consequences for candidates convicted of vote buying; (d) their qualifications for running for office; (e) the personal characteristic they believe best prepares them for office; and (f) their past achievements Information on local budget irregularities based on reports from the Office of the Auditor General: percentage of unaccounted-for funds in a citizen’s district compared with unaccounted-for funds in other districts; also, examples of unaccounted-for funds when the district ranked below the national median on budget management, and examples of public projects that were managed well when district budgets had fewer irregularities than the national median Information about whether municipal accounts were recently approved or rejected by TCE-PE auditors Information on percentage of unauthorized/ misallocated spending (relative to municipalities governed by opposition parties in the same state) based on information from the Federal Auditor’s Office (ASF) Information on legislative performance of incumbent (relative to department and national averages), including: (a) rate of attendance at legislative sessions; (b) rate of posing questions during legislative sessions; (c) rate of attendance in committees; (d) productivity of committee work; information compiled into three indices Information on municipal government performance (relative to other municipalities in the region) in areas of primary education, primary health care, water, sanitation, and administrative services Information the criminal backgrounds of candidates for state assembly elections: number of criminal cases faced, and information on the average number of criminal cases faced by candidates in other constituencies in the same subdivision of the state Mode of information delivery Video displayed on tablet and presented to respondent by enumerator SMS Flyer during baseline survey, with enumerators providing oral summaries Flyer Video displayed on tablet and presented to respondent by enumerator in one of eight local languages Flashcard presentation by enumerators, plus oral summaries Flyers provided to respondents and explained (for 6 minutes) by survey enumerators; respondents asked to keep flyer visible in their household until the election Timing of information delivery 1–2 weeks before election One week before election 2–3 weeks before election Between 3 weeks and four days before elections Between 7 weeks and 2.5 weeks before election 1–3 weeks before election One month before election Alternative arm intervention Public screenings of candidate videos in village meetings Information on quality of local public goods and services (education, health, roads, water) Information on municipality’s performance in National Literacy Examination Information publicly provided via loudspeakers, alongside common arm treatment (a) Civics lesson on importance of legislative performance; (b) public provision of information in village meetings, with varying dosages Personal invitation to attend a municipal council/special delegation meeting Common arm information on politicians’ criminal backgrounds delivered by influential local leaders rather than survey enumerators Measurement 2 3 4 5 6 7 8 Definition of P (prior beliefs about substance of the information provided) Respondents’ prior beliefs about incumbents’ policy positions and personal attributes, plus the importance (i.e. weight) they attach to each policy/attribute Respondents’ prior beliefs about budget management in their district relative to other districts (5-point scale: much worse, a little worse, no prior belief, a little better, much better) Respondents’ prior beliefs on whether their municipalities’ accounts were accepted or rejected None in primary analysis; but in robustness test, impute priors (measured at endline) from control precincts in same randomization block Legislative performance index compared to department and nation (4point scale) How voters rank their own municipality relative to other municipalities in the region with respect to the quality of public services under the previously elected municipal government N/A Burkina Faso Fig. S3. Burkina Faso—Flashcard illustrations of municipal performance indicators. Top panel: Provision of school latrines. Bottom panel: Provision/maintenance of water points. Mexico INFORMACIÓN IMPORTANTE! ¡BORDE ES UNA ASOCIACIÓN CIVIL SIN FINES PARTIDISTAS Y TE TRAEMOS La información de este volante está basada en los reportes oficiales de la Auditoria Superior de la Federación que puedes encontrar en: www.asf.gob.mx Cualquier inquietud contáctanos al 52 08 01 88 o en [email protected] Visita www.borde.mx/2015 para ver más datos y los documentos originales. AGUA POTABLE DRENAJE CAMINOS LUZ ESCUELAS CLÍNICAS VIVIENDA AGUA POTABLE DRENAJE CAMINOS LUZ ESCUELAS CLÍNICAS VIVIENDA AGUA POTABLE DRENAJE CAMINOS LUZ ESCUELAS CLÍNICAS VIVIENDA MUNICIPIOS DE TU ESTADO GOBERNADOS POR OTROS PARTIDOS GASTARON EN PROMEDIO 9% EN COSAS QUE NO DEBEN GASTÓ COMO NO DEBE 45 PARTIDO QUE GOBIERNA ECATEPEC OTROS PARTIDOS EN TU ESTADO 9 EL DINERO DEL FISM, FONDO DE INFRAESTRUCTURA SOCIAL MUNICIPAL, DEBE GASTARSE EN OBRAS DE INFRAESTRUCTURA EN 2013, EL PARTIDO QUE GOBIERNA ECATEPEC RECIBIÓ 146.3 MILLONES DE PESOS DEL FISM Y GASTÓ 45% EN COSAS QUE NO DEBE LOS GASTOS QUE NO SEAN EN OBRAS DE INFRAESTRUCTURA DEBEN SER 0% ¡COMPAREMOS CON LOS GASTOS DE OTROS PARTIDOS! ¡PIÉNSALO! EL ¡COMPÁRTELO! EL VOTO DEPENDE DE TI 7DE JUNIO Fig. S4. Mexico—Example of benchmarked leaflet in Ecatepec de Morelos, M´exico. Uganda 1 Fig. S5. Uganda 1—Candidate answering questions during a recording session and candidate as seen in video. Uganda 2 The following are examples of text messages sent via mobile phones to voters: •“The Auditor General conducts yearly audits to record instances where LC Vs could not satisfactorily explain how its money has been spent.” •“Unexplained spending is often an indicator of mismanagement, fraud or poor quality services.” •“Your LC V did much worse than most other LC Vs in the recent audit.” •“In your LC V, the auditor found issues with 120 million UGX from its budget of 19 billion UGX. This is much worse than in other districts.” •“This means that 6.3 out of 1000 UGX in your LC V budget had issues. In most LC Vs, 2.2 out of 1000 UGX had issues. Your LC V did much worse than average.” •“One reason your LC V did much worse than average is that payments of 98 million UGX were made without proper documentation.” •“Another reason your LC V did much worse than average is that a bid for borehole construction included unexplained expenditures.” 1.4 Descriptive Statistics Table 2 Descriptive statistics for sample of good news Statistic N Mean St. Dev. Min Max Nij 19,400 1.650 1.156 0.000 4.000 Voter turnout 16,037 0.801 0.399 0 1 Effort 13,237 2.396 0.944 1 4 Dishonesty 13,756 2.458 1.209 1 5 Backlash 2,157 0.232 0.309 0.000 1.000 Age 20,020 35.461 12.729 17 99 Co-ethnicity 17,382 0.665 0.472 0.000 1.000 Education 20,033 7.191 4.108 0.000 20.000 Wealth 19,903 2.772 1.091 −2.317 5.000 Co-Partisanship 16,550 1.155 1.797 0 9 Voted in past election 20,015 0.827 0.378 0.000 1.000 Secret ballot 19,788 2.098 1.420 1.000 5.000 Free and fair elections 19,144 3.344 1.506 1.000 5.000 Note: ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001. S . . Table 3 Descriptive statistics for sample of bad news Statistic N Mean St. Dev. Min Max Nij 19,191 −1.077 1.031 −4.000 0.000 Voter turnout 15,597 0.798 0.401 0 1 Effort 12,761 2.597 0.938 1 4 Dishonesty 13,589 2.481 1.239 1 5 Backlash 2,339 0.147 0.192 0.000 1.000 Age 19,584 37.370 13.345 18 92 Co-ethnicity 16,749 0.815 0.388 0.000 1.000 Education 19,604 6.657 3.968 0.000 20.000 Wealth 19,260 2.890 1.057 −2.805 5.000 Co-Partisanship 17,002 1.151 1.675 0 9 Voted in past election 19,568 0.862 0.345 0.000 1.000 Secret ballot 19,281 2.401 1.407 1.000 5.000 Free and fair elections 18,966 3.567 1.442 1.000 5.000 Note: ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001. S . . 1.5 Balance Tests Table S4. Balance of covariates. Baseline covariate Control mean Treat mean d-stat ˆ β1ˆ β2N Prior 1.35 1.38 0.03 0.05 0.02* 20617 (1.26) (1.28) (0.01) (0.01) Good news 0.48 0.48 0 -0.1*** -0.01*** 23803 (0.5) (0.5) (0.02) (0.01) Gender 0.43 0.42 -0.02 0.01* -0.02*** 23998 (0.5) (0.49) (0.01) (0.01) Age 39.56 39.62 0 0*** 0*** 23917 (14.96) (14.96) (0) (0) Co-ethnicity 0.65 0.63 -0.03 0*** -0.03*** 19391 (0.48) (0.48) (0.01) (0.01) Education 5.45 5.43 0 0*** 0*** 23960 (4.79) (4.71) (0) (0) Wealth 2.42 2.41 -0.01 0.02* 0.01* 23693 (1.44) (1.42) (0.01) (0) Co-Partisanship 3.64 3.6 -0.01 0.06 0*** 20025 (2.81) (2.78) (0) (0) Voted in past election 0.78 0.77 -0.01 0.07 0.17 23892 (0.42) (0.42) (0.01) (0.01) Voted incumbent past election 0.66 0.66 0 0.22 0.03* 19869 (0.47) (0.47) (0.01) (0.01) Clientelism 1.99 1.96 -0.02 -0.04*** 0*** 22911 (1.41) (1.41) (0) (0) Salience of information 0.52 0.54 0.03 -0.04*** 0*** 20143 (0.5) (0.5) (0.01) (0.01) Credibility of information 0.41 0.43 0.05 -0.02*** -0.01*** 21415 (0.49) (0.5) (0.01) (0.01) Pr(χ2) 0.1 Note: Results show the control and treatment means for each of the pre-treatment covariates. Means and standard deviations are weighted by block share of non-missing observations. d-stat is calculated as the difference between treatment and control means normalized by one standard deviation of the control mean. ˆ β1(ˆ β2) is the coefficient in a regression of vote choice (turnout) on each covariate separately, in the control sample. As with main specification, we include randomization block fixed effects and standard errors clustered at the level of treatment assignment. We also show the probability of rejecting the null that none of the covariates is predictive of treatment. All regressions include block fixed effects, standard errors clustered at the level of assignment and inverse propensity weights, and all countries are weighted equally. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001. 1.6 Power Analysis Calculating the power of our meta-analysis is somewhat difficult since there are many blocks and clusters of unequal size, complex assignment schemes—different in different studies—and complex estimation involving inverse propensity score weights, country weights, and clustered standard errors; moreover, the average effects of interest are averages over heterogeneous effects that depend upon our specification of good news and bad news groups. Off-the-shelf power calculators are not able to deliver estimates of power for designs like this. Nevertheless, power calculations are possible using an ex-post simulation approach, at least conditional on a model of the data-generating process. We implement this approach using the DeclareDesign package, in which we formally declare our data structure, our conjectured data generating process, our assignment schemes, our estimands, and our estimation strategy. We then use Monte Carlo simulations to “run” the design many times and assess statistical power—that is, the fraction of runs in which we reject a false null hypothesis—conditional on different conjectures about the size of the true effect. We note that a bonus of this approach is that we can check that our estimates are unbiased, given our design. This is a nontrivial question since the estimation strategy had to be tailored to match different assignment strategies used in different sites. Moreover, unbiasedness is not guaranteed given heterogeneous sized clusters in some studies. The results from the “diagnosis” of this design suggest no bias concerns. The most important feature of the power analysis involves the specification of a data-generating process. We assume that an individual in block band cluster cwill vote for the incumbent with probability p0 bc, where p0 bc is drawn from a distribution centered on the observed block level share supporting the incumbent in the control group, with a variance that produces an intra-cluster correlation coefficient (conditional on block, b) approximately equal to the observed correlation in that group. For any stipulated effect δwe assume that individuals support the incumbent in the treatment condition with probability p1 bc p1 bc =φφ−1(p0 bc) + δNi(1) where φis the standard normal density and φ−1its inverse. The approach here then assumes that treatment induces a constant effect (conditional on the value of Ni) on a latent support variable that determines the propensity to support the incumbent. For instance, for δ= 1 an individual that supports an incumbent with probability p0=.5 in control and for whom Ni= 1, would support them with probability 0.84 in treatment (i.e. φ(0 + 1)). In practice, a probit-type approach is employed, in which an individual has a normally distributed shock eiand votes for the incumbent if eifalls below φ−1(pt) for condition t; this ensures that in realizations individuals with positive effects have non-negative changes in their votes. Note that for any specified δ, different individuals have heterogeneous effects that depend upon the propensity in their control condition and their own value of Ni. Given all these different propensities across all individuals, the estimand of interest is the average difference in voting propensity, across studies, for individuals in treatment and control. To calculate power, we consider a range of possible δs and for each one calculate the implied estimand and the probability that our estimate of that estimand will be statistically significant. Results are presented in Figure 6. We see that power for different average effects depends on the outcomes of interest. For the electoral support outcomes, we hit 80 percent power for average treatment effects of around 5 percentage points; for the turnout quantities, we would hit power of 80 percent with effects of around 4 percentage points. In other words, to register a statistically significant result on our primary outcome in 80 percent of repeated hypothetical experiments, the interventions would have to change the vote choice of 5 out of every 100 voters. Together with the tightness of our observed confidence intervals, we see these results as evidence that null results were not forgone conclusions. ● ● ● ● ●● 0.00 0.02 0.04 0.06 0.08 0.0 0.2 0.4 0.6 0.8 1.0 (a) Vote for incumbent, good news ATE (Absolute value) power ● ● ● ● ● ● 0.00 0.02 0.04 0.06 0.08 0.0 0.2 0.4 0.6 0.8 1.0 (b) Vote for incumbent, bad news ATE (Absolute value) power ● ● ● ●●● 0.00 0.02 0.04 0.06 0.08 0.0 0.2 0.4 0.6 0.8 1.0 (c) Turnout, good news ATE (Absolute value) power ● ● ● ●● ● 0.00 0.02 0.04 0.06 0.08 0.0 0.2 0.4 0.6 0.8 1.0 (d) Turnout, bad news ATE (Absolute value) power Fig. S6. Power analysis of minimal detectable effects, computed using Monte Carlo simulation. The horizontal axis varies the conjectured average treatment effect, while the vertical axis shows statistical power: the probability of rejecting the null hypothesis at α= 0.05. Robustness and reliability of resultsSection S2. Primary analysis: 2.1 Additional Test on Average Effects Across Cases Table S5. Effect of information, conditional on distance between information and priors, on vote choice, and urnout Vote Choice Turnout Vote Choice Turnout Good News Bad News Good News Bad News Overall (1) (2) (3) (4) (5) (6) Treatment 0.0002 −0.003 0.002 0.018 0.003 0.017∗ (0.015) (0.015) (0.013) (0.012) (0.010) (0.008) Nij −0.015 −0.049∗∗∗ 0.002 0.011 −0.050∗∗∗ 0.010 (0.016) (0.015) (0.014) (0.013) (0.012) (0.011) Treatment * Nij −0.012 −0.001 −0.002 0.0001 −0.002 −0.002 (0.020) (0.020) (0.019) (0.015) (0.012) (0.011) Control mean 0.355 0.398 0.843 0.835 0.368 0.837 RI p-value 0.994 0.848 0.89 0.18 0.81 0.057 Joint RI p-value 0.972 0.309 Covariates Yes Yes Yes Yes Yes Yes Observations 13,190 12,531 14,494 13,148 25,814 27,731 R20.298 0.281 0.200 0.160 0.274 0.165 Note: “Vote choice” indicates support for the incumbent candidate or party. Standard errors are clustered at the level of treatment assignment. Pooled results exclude non-contested seats and include vote choice for LCV councilors as well as chairs in the Uganda 2 study (see Table 6, below, for further explanation). This means each respondent in the Uganda 2 study enters twice, and we cluster the standard errors at the individual level. We include randomization block fixed effects and a full set of covariate-treatment interactions. Control mean is the weighted and unadjusted average in the control group. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001 t . ● ● ● ● ● ● Effects of Good News on Vote Choice Posterior on Effect Sizes (95% credibility) µ = 0.01 τ = 0.02 −0.15 −0.10 −0.05 0.00 0.05 0.10 0.15 Uganda 2 Uganda 1 Mexico Burkina Faso Brazil Benin Overall ● ● ● ● ● ● ● ● ● ● ● ● ● ● Effects of Bad News on Vote Choice Posterior on Effect Sizes (95% credibility) µ = 0 τ = 0.02 −0.15 −0.10 −0.05 0.00 0.05 0.10 0.15 Uganda 2 Uganda 1 Mexico Burkina Faso Brazil Benin Overall ● ● ● ● ● ● ● ● Fig 7 Bayesian eta-analysis: Vote hoice. The solid dots and lines show the estimates from the Bayesian model; the top row shows the overall meta-estimate of µand τ. The white dots show the original frequentist estimates: in many cases shrinkage can be observed, especially in cases that have effects that are more imprecisely estimated. . S . m c ● ● ● ● ● ● Effects of Good News on Turnout Posterior on Effect Sizes (95% credibility) µ = 0 τ = 0.03 −0.15 −0.10 −0.05 0.00 0.05 0.10 0.15 Uganda 2 Uganda 1 Mexico Burkina Faso Brazil Benin Overall ● ● ● ● ● ● ● ● ● ● ● ● ● ● Effects of Bad News on Turnout Posterior on Effect Sizes (95% credibility) µ = 0.02 τ = 0.01 −0.15 −0.10 −0.05 0.00 0.05 0.10 0.15 Uganda 2 Uganda 1 Mexico Burkina Faso Brazil Benin Overall ● ● ● ● ● ● ● ● Fig. S8. Bayesian meta-analysis: Turnout. The solid dots and lines show the estimates from the Bayesian model; the top row shows the overall meta-estimate of µand τ. The white dots show the original frequentist estimates: in many cases shrinkage can be observed, especially in cases that have effects that are more imprecisely estimated. Possible xplanations for the ull indingsSection S4. 4.1 Voter Updating 4.1.1 Manipulation Check Table S8. Manipulation check: Effect of treatment on correct recollection, pooling good and bad news (unregistered analysis). Correct Recollection Overall Benin Brazil Mexico Uganda 1 Uganda 2 (1) (2) (3) (4) (5) (6) Treatment 0.072∗∗∗ 0.050 0.038 0.149∗∗∗ 0.119∗∗∗ −0.0001 (0.015) (0.059) (0.021) (0.015) (0.035) (0.008) Covariates No No No No No No Observations 16,173 897 1,677 2,089 750 10,760 R20.320 0.276 0.378 0.137 0.035 0.205 Notes: The table reports results on manipulation checks across studies, using recollection or accuracy tests at endline that were specific to the content of each study’s interventions (MPAP measure M30). The dependent variable, correct recollection, is dichotomized in each study using the following measures: Benin: whether correctly recalled the relative performance of incumbent in plenary and committee work; Brazil: whether correctly recalled whether municipal account was accepted or rejected; Mexico: identification of content of the flyer; Uganda 1: index consisting of knowledge of MP responsibilities, MP priorities for constituency, and identities of contesting candidates. Individuals with an index equal to or greater than 1.5 on a 0-3 scale were coded as correct recalls; Uganda 2: whether correctly recalled relative financial accountability relative to other districts. We include randomization block fixed effects. Standard errors are clustered at the level of treatment assignment. ∗p<0.05; ∗∗p<0.01; ∗∗∗p<0.001. e n f Table S9. Manipulation check: Absolute difference between posterior and prior beliefs for pooled good and bad news unregistered analysis Absolute difference between posterior and prior beliefs Overall Benin Brazil Uganda 2 (1) (2) (3) (4) Treatment 0.006 0.063 −0.003 −0.023 (0.025) (0.089) (0.022) (0.023) Covariates No No No No Observations 12,704 389 1,677 10,638 R20.241 0.176 0.358 0.111 Notes: The table reports differences between beliefs about politician performance after (MPAP measure M30) and prior to treatment (MPAP measure M9). Posterior beliefs are measured using recollection tests at endline specific to the content of each study’s intervention. Burkina Faso is excluded because their recollection measure was collected among treated subjects only. Mexico is excluded from results because the study does not contain pre-treatment measures of subjects beliefs. Uganda 1 is not included because the M30 measure is an aggregate measure of subjects’ political knowledge and cannot be directly compared with the scale used for measuring priors. We include randomization block fixed effects. Standard errors are clustered at the level of treatment assignment. ∗p<0.05; ∗∗p<0.01; ∗∗∗p<0.001. ( ). 4.1.2 Perceptions Table 10 Effect of information on perception of importance of politician effort and honesty Effort Dishonesty Good News Bad News Good News Bad News (1) (2) (3) (4) Treatment effect −0.014 −0.051 −0.053 0.099 (0.046) (0.051) (0.047) (0.098) Control mean 2.449 2.7 2.755 2.724 RI p-value 0.8 0.466 0.36 0.756 Joint RI p-value 0.507 0.292 Covariates No No No No Observations 7,039 5,963 7,278 6,755 R20.253 0.294 0.300 0.231 Note: The table reports the effect of the treatment on voters’ perception of how hard-working (MPAP measure M5) and dishonest (MPAP measure M6) the incumbent politician is. We pool Benin, Burkina Faso, Uganda 1, and Uganda 2 in columns (1) and (2), and Benin, Burkina Faso, Mexico, and Uganda 2 in columns (3) and (4). MPAP measures M5 (effort) and M6 (dishonesty). Regressions include randomization block fixed effects; standard errors are clustered at the level of treatment assignment. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001 S . . Table S11. Effect of information and source credibility on evaluation of politician effort and honesty (unregistered analysis). Dependent variable: Effort Dishonesty Good News Bad News Good News Bad News (1) (2) (3) (4) Treatment −0.034 −0.088 −0.037 −0.037 (0.079) (0.090) (0.085) (0.085) Credible Source −0.051 −0.010 −0.022 −0.022 (0.079) (0.081) (0.064) (0.064) Treatment * Credible Source 0.033 0.070 0.010 0.010 (0.095) (0.105) (0.093) (0.093) Control mean 2.451 2.703 2.75 2.75 RI p-values 0.725 0.516 0.72 0.72 Joint RI p-value 0.476 0.72 Covariates No No No No Observations 6,436 5,406 6,483 6,483 R20.261 0.293 0.329 0.329 Note: The table reports the effects of information and the credibility of the information source on voter’s perception of how hard-working (MPAP measure M5) and dishonest (MPAP measure M6) the incumbent politician is. We pool Benin, Burkina Faso, Uganda 1, and Uganda 2 in columns (1) and (2), and Benin, Burkina Faso, Mexico, and Uganda 2 in columns (3) and (4). Regressions include randomization block fixed effects; standard errors are clustered at the level of treatment assignment. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001 4.1.3 Association of Perceptions and Electoral Support Table S12. Relationship between evaluation of politician effort and honesty with vote choice (unregistered analysis). Incumbent vote choice Good news Bad news (1) (2) (3) (4) Effort 0.052∗∗∗ 0.066∗∗∗ (0.006) (0.006) Dishonesty −0.054∗∗∗ −0.026∗∗∗ (0.005) (0.005) Covariates No No No No Observations 11,040 11,452 10,190 10,943 R20.229 0.217 0.282 0.266 Note: The table reports the effects of information and the credibility of the information source on voter’s perception of how hard-working (MPAP measure M5) and dishonest (MPAP measure M6) the incumbent politician is. We pool Benin, Burkina Faso, Uganda 1, and Uganda 2 in columns (1) and (3), and Benin, Burkina Faso, Mexico, and Uganda 2 in columns (2) and (4). Results exclude non-contested seats and include vote choice for LCV councilors as well as chairs in the Uganda 2 study. Regressions include randomization block fixed effects; standard errors are clustered at the level of treatment assignment. ∗ p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001 4.2 Politician Response Table 13 Effect of bad news on politician backlash. Politician response / backlash Overall Benin Mexico (1) (2) (3) Treatment effect 0.069∗0.068 0.070∗∗∗ (0.028) (0.057) (0.010) Control mean 0.108 0.068 0.146 RI p-value 0.089 0.438 0 Covariates No No No Observations 2,052 702 1,350 R20.623 0.504 0.848 Note: The table reports on whether the treatment led to the incumbent party or candidate campaigning on dimensions of the dissemminated information (MPAP measure M8). Backlash was measured for studies with clustered assignment. Regressions include randomization block fixed effects; standard errors are clustered at the level of treatment assignment. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001 S . 4.3 Learning from Variation We pre-registered a number of additional hypotheses is the MPAP. We list these in Table 14. Subsequent tables report the results of the additional analysis. Table S14. Additional hypotheses and results. MPAP hypothesis Prediction Moderator measure News subgroup Evidence for interaction Substitution effects: Ethnicity, partisanship, or clientelist relations could provide heuristic substitutes for information H6: Non-coethnics Good news effects more positive for incumbent’s non-coethnics M15 Good No (Table 15) H6: Non-coethnics Bad news effects more negative for incumbent’s non-coethnics M15 Bad No (Table 15) H7: Partisanship Good news effects more positive for voters with weaker partisan identities M19 Good No (Table 15) H7: Partisanship Bad news effects more negative for voters with weaker partisan identities M19 Bad No (Table 15) H8: Clientelism Good news effects more positive for voters who have not received clientelistic benefits M22 Good No (Table 15) H8: Clientelism Bad news effects more negative for voters who have not received clientelistic benefits M22 Bad No (Table 15) Context-specific heterogeneity: Information will have greater impact among voters with less exposure to information in the pre-treatment period, and in competitive, free and fair elections H9: Informational environment Good news effects are more positive in low information environments M11 Good No (Table 16) H9: Informational environment Bad news effects are more negative in low information environments M11 Bad No (Table 16) H10: Competitive elections Good news effects are more positive where electoral competition is greater M25 Good No (Table 17) H10: Competitive elections Bad news effects are more negative where electoral competition is greater M25 Bad No (Table 17) H11: Free and fair elections Good news effects are more positive where elections are believed to be free and fair M26/M27 Good No (Table 16) H11: Free and fair elections Bad news effects are more negative where elections are believed to be free and fair M26/M27 Bad No (Table 16) Intervention-specific heterogeneity H12: Information content Information effects—both positive and negative—are stronger when the gap between voters’ prior beliefs about candidates and the information provided is larger Nij All No (Table 18) H13: Information welfare salient Good news effects are more positive the more the information relates directly to individual welfare M23 Good No (Table 18) H13: Information welfare salient Bad news effects are more negative the more the information relates directly to individual welfare M23 Bad No (Table 18) H14: Credible source Good news effects are more positive the more reliable and credible is the information source M24 Good No (Table 18) H14: Credible source Bad news effects are more negative the more reliable and credible is the information source M24 Bad No (Table 18) Covariate-treatment interactions in MPAP equations (3) and (4) Demographics No (Tables 19 and 20) Table S18. Effect of information and intervention-specific heterogen ity on vote choice. Incumbent vote choice Good news Bad news Good news Bad news Good news Bad news (1) (2) (3) (4) (5) (6) Treatment 0.001 −0.010 0.025 −0.022 −0.017 −0.013 (0.016) (0.016) (0.024) (0.036) (0.021) (0.023) Nij −0.027 −0.053∗∗∗ (0.016) (0.014) Treatment * Nij −0.006 −0.006 (0.020) (0.019) Information salient −0.016 −0.041 (0.029) (0.035) Treatment * Information salient −0.015 0.053 (0.034) (0.042) Credible source −0.007 0.005 (0.028) (0.027) Treatment * Credible source 0.036 0.020 (0.030) (0.031) Control mean 0.356 0.398 0.355 0.435 0.363 0.385 RI p-values 0.956 0.592 0.314 0.61 0.452 0.628 Joint RI p-value 0.779 0.232 0.347 Covariates No No No No No No Observations 13,274 12,563 12,343 10,587 12,354 11,407 R20.275 0.249 0.265 0.221 0.260 0.240 Note: The table reports results of the effect of information and (a) the gap between priors and information (MPAP measure Nij ), (b) salience of information (MPAP measure M23) and (c) credibility of information source on voters’ decision to vote for the incumbent. Columns 1, 2, 5 and 6 pool observations from all studies while Columns 3 and 4 pool Benin, Brazil, Uganda 1 and Uganda 2. Results exclude non-contested seats and include vote choice for LCV councilors as well as chairs in the Uganda 2 study. Regressions include randomization block fixed effects; standard errors are clustered at the level of treatment assignment. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001 e 4.3.4 Heterogeneity by Demographics Table S19. Interaction analysis: Effect of good news on incumbent vote choice. Incumbent vote choice, good news ALL BEN BRZ BF MEX UG 1 UG 2 (1) (2) (3) (4) (5) (6) (7) Treatment 0.0002 −0.033 0.007 0.004 −0.036 0.048 0.009 (0.015) (0.065) (0.030) (0.049) (0.031) (0.033) (0.012) Nij −0.015 0.008 −0.016 −0.052∗∗ −0.010 (0.016) (0.062) (0.000) (0.039) (0.000) (0.018) (0.009) Treatment * Nij 0.001 0.131 −0.026 −0.028 0.034 −0.025 −0.003 (0.008) (0.077) (0.023) (0.056) (0.018) (0.013) (0.006) Age −0.001 −0.008 0.001 0.002 −0.003 0.003∗0.002∗∗ (0.001) (0.006) (0.002) (0.003) (0.002) (0.002) (0.001) Treatment * Age −0.005 −0.140 −0.060∗−0.083∗0.054∗∗ −0.009 0.004 (0.008) (0.072) (0.024) (0.037) (0.020) (0.015) (0.006) Education −0.002 −0.009 0.010 0.002 −0.003 −0.011 −0.002 (0.003) (0.010) (0.007) (0.020) (0.008) (0.007) (0.003) Treatment * Education −0.012 −0.037 −0.018 0.035 −0.013 (0.020) (0.068) (0.000) (0.050) (0.000) (0.027) (0.012) Wealth 0.022 0.038 0.061 −0.007 0.033 0.041 0.016 (0.013) (0.049) (0.039) (0.039) (0.034) (0.027) (0.009) Treatment * Wealth 0.001 0.014∗−0.004 0.001 0.004 −0.005∗0.0004 (0.001) (0.007) (0.003) (0.005) (0.003) (0.002) (0.001) Voted previously 0.053 −0.070 0.073 0.096 0.185∗∗∗ −0.157∗∗ 0.057∗ (0.027) (0.068) (0.079) (0.085) (0.048) (0.057) (0.025) Treatment * Voted previously 0.008 0.025 −0.010 −0.033 0.009 0.019∗−0.003 (0.004) (0.016) (0.008) (0.026) (0.010) (0.009) (0.003) Supported incumbent 0.191∗∗∗ 0.004 0.293∗∗∗ 0.242 0.308∗∗∗ 0.178∗∗ 0.111∗∗∗ (0.028) (0.107) (0.058) (0.147) (0.049) (0.055) (0.024) Treatment * Supported incumbent −0.041∗−0.134 0.030 0.036 −0.079 −0.129∗∗ 0.003 (0.018) (0.086) (0.052) (0.052) (0.046) (0.041) (0.012) Clientelism −0.040∗∗∗ −0.096 −0.073∗∗∗ 0.007 −0.054∗−0.019 −0.006 (0.010) (0.077) (0.021) (0.086) (0.026) (0.018) (0.006) Treatment * Clientelism −0.030 0.149 0.085 −0.053 −0.156∗0.026 0.041 (0.039) (0.133) (0.110) (0.125) (0.071) (0.086) (0.034) Credible source −0.022 −0.123 0.025 −0.089 −0.008 −0.052 −0.0001 (0.033) (0.172) (0.112) (0.081) (0.065) (0.049) (0.032) Treatment * Credible source −0.030 −0.129 0.092 −0.006 0.112 −0.109 −0.002 (0.041) (0.110) (0.073) (0.197) (0.093) (0.075) (0.033) Secret ballot 0.016 0.143 −0.016 0.100 0.042 0.009 0.007 (0.013) (0.112) (0.027) (0.123) (0.035) (0.023) (0.009) Treatment * Secret ballot 0.058 0.283 −0.042 0.052 0.120 0.074 0.011 (0.044) (0.247) (0.137) (0.124) (0.086) (0.068) (0.043) Free, fair election −0.002 −0.121 0.012 0.003 −0.036 0.040∗−0.004 (0.011) (0.083) (0.031) (0.076) (0.028) (0.019) (0.008) Treatment * free, fair election 0.019 0.022 0.020 0.117∗−0.016 0.030 0.008 (0.012) (0.082) (0.026) (0.050) (0.032) (0.022) (0.008) Covariates Yes Yes Yes Yes Yes Yes Yes Observations 13,190 214 859 389 725 456 10,547 R20.298 0.360 0.484 0.392 0.224 0.177 0.240 Note: The table presents results from fitting MPAP equation (3). Standard errors are clustered at the level of treatment assignment. Results in columns (1) and (7) include vote choice for LCV councilors as well as chairs in the Uganda 2 study (see Buntaine et al., Chapter 8). This means each respondent in the Uganda 2 study enters twice, and we cluster the standard errors at the individual level. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001 Table S20. Interaction analysis: Effect of bad news on incumbent vote choice. Incumbent vote choice, bad news ALL BEN BRZ BF MEX UG 1 UG 2 (1) (2) (3) (4) (5) (6) (7) Treatment −0.003 −0.079 −0.022 0.037 −0.013 0.010 −0.006 (0.015) (0.086) (0.030) (0.028) (0.018) (0.053) (0.012) Nij −0.049∗∗∗ −0.091 −0.100∗∗∗ −0.005 −0.036 −0.002 (0.015) (0.048) (0.028) (0.026) (0.000) (0.036) (0.009) Treatment * Nij −0.001 −0.117 −0.004 0.018 0.047∗∗∗ −0.015 −0.002 (0.011) (0.094) (0.021) (0.033) (0.013) (0.028) (0.006) Age 0.0004 −0.004 0.00003 0.002 0.0003 0.002 0.001 (0.001) (0.004) (0.002) (0.002) (0.001) (0.003) (0.001) Treatment * Age 0.0002 −0.015 0.001 −0.030 0.012 0.019 −0.003 (0.011) (0.072) (0.026) (0.020) (0.013) (0.032) (0.007) Education −0.003 −0.006 −0.001 −0.006 −0.010∗0.0004 −0.003 (0.003) (0.010) (0.005) (0.008) (0.004) (0.012) (0.003) Treatment * Education −0.001 0.090 −0.063 −0.004 −0.002 −0.003 (0.020) (0.070) (0.034) (0.031) (0.000) (0.053) (0.012) Wealth 0.037∗0.021 0.012 0.001 0.037 0.089 0.013 (0.015) (0.093) (0.041) (0.023) (0.020) (0.045) (0.009) Treatment * Wealth −0.00005 0.003 −0.001 0.001 0.0002 −0.001 −0.001 (0.001) (0.008) (0.002) (0.002) (0.001) (0.004) (0.001) Voted previously 0.036 0.063 −0.053 0.083 0.123∗∗ −0.138 0.075∗ (0.037) (0.282) (0.089) (0.047) (0.038) (0.103) (0.029) Treatment * Voted previously 0.001 −0.013 0.006 0.006 −0.005 −0.003 0.002 (0.005) (0.018) (0.007) (0.012) (0.006) (0.017) (0.004) Supported incumbent 0.190∗∗∗ −0.021 0.282∗∗∗ 0.249∗∗∗ 0.465∗∗∗ 0.204∗0.065∗ (0.046) (0.165) (0.049) (0.067) (0.035) (0.090) (0.030) Treatment * Supported incumbent −0.025 0.025 0.027 −0.018 0.015 −0.105 −0.024 (0.020) (0.104) (0.054) (0.031) (0.033) (0.061) (0.012) Clientelism −0.032∗∗ −0.011 −0.086∗∗∗ 0.019 −0.012 −0.020 0.005 (0.010) (0.139) (0.019) (0.055) (0.016) (0.026) (0.007) Treatment * Clientelism −0.023 −0.276 0.186 −0.042 0.030 −0.094 0.027 (0.046) (0.313) (0.105) (0.066) (0.052) (0.145) (0.041) Credible source −0.012 −0.055 0.015 −0.042 0.027 −0.025 0.002 (0.034) (0.205) (0.075) (0.051) (0.040) (0.083) (0.036) Treatment * Credible source 0.016 −0.104 −0.052 0.133 −0.090 0.073 0.011 (0.055) (0.236) (0.068) (0.086) (0.058) (0.116) (0.042) Secret ballot −0.008 −0.032 0.026 0.027 −0.056∗−0.013 −0.007 (0.014) (0.144) (0.025) (0.076) (0.023) (0.037) (0.009) Treatment * Secret ballot 0.049 0.283 0.029 0.095 0.012 −0.015 0.056 (0.047) (0.486) (0.114) (0.081) (0.056) (0.115) (0.051) Free, fair election 0.016 −0.001 0.043 0.013 −0.033 0.033 −0.003 (0.014) (0.211) (0.027) (0.043) (0.018) (0.037) (0.009) Treatment * free, fair election −0.004 0.154 −0.018 0.003 −0.015 −0.039 0.010 (0.014) (0.150) (0.029) (0.029) (0.024) (0.045) (0.010) Covariates Yes Yes Yes Yes Yes Yes Yes Observations 12,531 181 818 911 1,215 294 9,112 R20.281 0.306 0.420 0.311 0.296 0.208 0.278 Note: The table presents results from fitting MPAP equation (4). Standard errors are clustered at the level of treatment assignment. Results in columns (1) and (7) include vote choice for LCV councilors as well as chairs in the Uganda 2 study (see Buntaine et al., Chapter 8). This means each respondent in the Uganda 2 study enters twice, and we cluster the standard errors at the individual level. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001 Effects of ublicly isseminated nformationSection S5. Table S21. Private versus public information: Effect of good news on incumbent vote choice. Incumbent vote choice, good news Overall Benin Mexico Uganda 1 (1) (2) (3) (4) Private information −0.008 0.012 −0.029 0.008 (0.023) (0.044) (0.043) (0.027) Public information 0.055∗0.146∗∗ −0.002 0.019 (0.022) (0.047) (0.041) (0.023) Control mean 0.356 0.439 0.498 0.186 F-test p-value 0.018 0.006 0.598 0.708 Covariates No No No No Observations 2,962 776 784 1,402 R20.192 0.189 0.088 0.068 Note: The table reports results of the effect of good news about the incumbent on vote choice, depending on whether voters received this information in private or public settings. We pool Benin, Mexico, and Uganda 1. Regressions include randomization block fixed effects and standard errors are clustered at the level of treatment assignment. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001 p d i Table S22. Private versus public information: Effect of bad news on incumbent vote choice. Incumbent vote choice, bad news Overall Benin Mexico Uganda 1 (1) (2) (3) (4) Private information −0.027 −0.012 −0.036 −0.035 (0.030) (0.074) (0.030) (0.042) Public information 0.009 0.006 0.015 0.009 (0.026) (0.069) (0.032) (0.032) Control mean 0.441 0.535 0.383 0.426 F-test p-value 0.018 0.006 0.598 0.708 Covariates No No No No Observations 2,909 601 1,309 999 R20.178 0.241 0.102 0.153 Note: The table reports results of the effect of bad news about the incumbent on vote choice, depending on whether voters received this information in private or public settings. We pool Benin, Mexico, and Uganda 1. Regressions include randomization block fixed effects and standard errors are clustered at the level of treatment assignment. ∗p < 0.05; ∗∗ p < 0.01; ∗∗∗ p < 0.001