Convergent and external validity of risk preference elicitation methods: Evidence from Viet Nam

Vu, Trang Thu; Munro, Alistair Working Paper Convergent and external validity of risk preference elicitation methods: Evidence from Viet Nam ADBI Working Paper, No. 1433 Provided in Cooperation with: Asian Development Bank Institute (ADBI), Tokyo Suggested Citation: Vu, Trang Thu; Munro, Alistair (2024) : Convergent and external validity of risk preference elicitation methods: Evidence from Viet Nam, ADBI Working Paper, No. 1433, Asian Development Bank Institute (ADBI), Tokyo, https://doi.org/10.56506/FCYT4789 This Version is available at: https://hdl.handle.net/10419/296825 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-nc-nd/3.0/igo/ ADBI Working Paper Series CONVERGENT AND EXTERNAL VALIDITY OF RISK PREFERENCE ELICITATION METHODS: EVIDENCE FROM VIET NAM Trang Thu Vu and Alistair Munro No. 1433 February 2024 Asian Development Bank Institute The Working Paper series is a continuation of the formerly named Discussion Paper series; the numbering of the papers continued without interruption or change. ADBI’s working papers reflect initial ideas on a topic and are posted online for discussion. Some working papers may develop into other forms of publication. The Asian Development Bank refers to “China” as the People’s Republic of China. Suggested citation: Trang, T. V. and A. Munro. 2024. Convergent and External Validity of Risk Preference Elicitation Methods: Evidence from Viet Nam. ADBI Working Paper 1433. Tokyo: Asian Development Bank Institute. Available: https://doi.org/10.56506/FCYT4789 Please contact the authors for information about this paper. Email: [email protected], [email protected] Trang Thu Vu is a research associate at the Asian Development Bank Institute, Tokyo, Japan. Alistair Munro is a professor of microeconomics at the National Graduate Institute for Policy Studies, Tokyo, Japan. The views expressed in this paper are the views of the author and do not necessarily reflect the views or policies of ADBI, ADB, its Board of Directors, or the governments they represent. ADBI does not guarantee the accuracy of the data included in this paper and accepts no responsibility for any consequences of their use. Terminology used may not necessarily be consistent with ADB official terms. Discussion papers are subject to formal revision and correction before they are finalized and considered published. Asian Development Bank Institute Kasumigaseki Building, 8th Floor 3-2-5 Kasumigaseki, Chiyoda-ku Tokyo 100-6008, Japan Tel: +81-3-3593-5500 Fax: +81-3-3593-5571 URL: www.adbi.org E-mail: [email protected] © 2024 Asian Development Bank Institute ADBI Working Paper 1433 Trang and Munro Abstract In this study, we add to the body of evidence on the reliability of risk preference measurements using evidence from a survey and experiment in rural Viet Nam. We conducted a field survey and experiment with a random sample of 350 households. Subjects face various incentivized elicitation methods, including multiple price lists and GneezyPotters-style tasks as well as non-incentivized tasks and general attitude questions about willingness to take on risk. Most elicitation methods provide evidence that respondents are, on average, risk-averse. Respondents appear less risk-averse in the self-assessment method than with other methods. Therefore, comparing risk preferences elicited from the survey and experiments should be done with caution. Unlike other studies on supporting the use of self-assessment of risk attitude in surveys such as Dohmen et al. (2011), we find that self-assessment, both in general and in specific contexts, has limited validity as it has the smallest or no relation with other measures. This finding could reflect the differences between developed and developing countries. Lastly, the multiple price list and loss–gain measures are stronger at predicting behaviors in experiments and predicting risky behaviors than other elicitation measures. Keywords: risk preferences, experiment, validity JEL Classification: D90, O10 ADBI Working Paper 1433 Trang and Munro Contents 1. INTRODUCTION .......................................................................................................... 1 2. SURVEY AND EXPERIMENTAL DESIGN .................................................................. 2 2.1 Research Area ................................................................................................. 2 2.2 Survey and Experimental Design ..................................................................... 3 2.3 Data Description ............................................................................................... 3 3. METHODS TO ELICIT RISK PREFERENCES ............................................................ 4 3.1 Self-assessment Willingness to Take Risks ..................................................... 4 3.2 Survey Question: Hypothetical Lottery ............................................................. 6 3.3 Multiple Price List ............................................................................................. 6 3.4 Loss–Gain Task ............................................................................................... 7 3.5 Investment Task ............................................................................................... 9 4. VALIDITY TESTS AND FINDINGS .............................................................................. 9 4.1 Internal Consistency ....................................................................................... 10 4.2 Correlation Among Elicitation Methods .......................................................... 12 4.3 Experimental Validity of Elicitation Measures ................................................. 13 4.4 Validity of Risk Preference Measures in Relation to Risky Behaviors ............ 15 5. DISCUSSIONS AND CONCLUSIONS ....................................................................... 17 REFERENCES ...................................................................................................................... 19 APPENDIX ............................................................................................................................. 23 ADBI Working Paper 1433 Trang and Munro 1 1. INTRODUCTION Risk is inherent in economic decision-making and differences in risk preferences across individuals or households account for differences in behaviors across a wide number of domains, including savings, investment, and health protection. The measure of risk preferences is critical for policy prescriptions in determining the appropriate level of risk reduction and in helping people, especially the poor, the vulnerable, and marginalized groups, to cope with shocks in daily life. Therefore, figuring out ways to accurately measure this important parameter can shed light on the sources of differences in individual preferences and their role in fundamental economic choices. Economists and psychologists have developed a variety of methodologies to elicit individual risk attitudes. In general, methods for assessing risk preferences can be categorized into two primary groups: incentivized (or experimental) methods involving real financial implications, and hypothetical measures. In hypothetical measures, subjects make choices among risky options, but they do not receive actual payoffs based on their choices; alternatively, they express their own perceived level of risk tolerance through self-rating questions on their attitude towards risk. While nonincentivized questions are generally deprecated by economists, hypothetical measures are easier and less costly to implement on a larger scale, which can be important if evidence on risk preferences a subsidiary part of a larger survey exercise is. For example, in the specific case of Viet Nam, the large-scale and nationwide Viet Nam Access to Resources Household Survey (VARHS) has recently incorporated hypothetical questions designed to measure risk attitudes. The well-known work of Dohmen et al. (2011) for Germany and Hardeweg, Menkhoff, and Waibel (2013) for Thailand has provided evidence that hypothetical questions can be reliable in particular circumstances, but it is not clear whether such circumstances include rural families in developing countries. Indeed, the original source of skepticism towards hypothetical questions and much of the impetus towards incentivized questions came from the careful work of Binswanger (1980) in low-income agricultural areas of India. Thus, to extend the evidence on the reliability of hypothetical and attitude questions, we conduct a series of parallel tasks amongst farmers in southern Viet Nam. More specifically, we focus on five elicitation methods commonly used in the literature, namely: (i) self-assessment survey questions; (ii) lottery tasks (hypothetical settings); (iii) loss–gain tasks (hypothetical or incentivized settings); (iv) multiple price list (MPL) tasks (hypothetical or incentivized settings); and (v) incentivized investment tasks. We view reliability as having two important components: consistency across different elicitation methods and the ability of each elicitation method to predict actual individual or household risk-taking behaviors. Given this, we focus on five sub-research questions: • Do the subjects understand the questions? • Are the responses consistent among subjects across elicitation methods? • Are elicitation methods significantly correlated with each other? ADBI Working Paper 1433 Trang and Munro 2 • Do the responses given in the hypothetical measures predict actual risk-taking behavior in the incentivized measures? • Does risk preference, from each elicitation method, predict observed individual and household behaviors? To answer the above research questions, we conduct a field survey and an experiment with a random sample of 350 households. The hypothetical elicitation methods contain a set of self-assessment questions that are adopted from Dohmen et al. (2011) and a set of hypothetical questions taken from the Viet Nam Access to Resources Household Survey (VARHS). The experimental methods include three tasks, two of which are modified from the equivalent hypothetical questions in the VARHS and an incentivized investment task. Thus, our major contribution is the wider range of elicitation tasks that we use compared to other studies, which when combined with data on risk lifestyles enables us to probe more carefully for consistency and predictive validity. To preview the main results, most of the participants have no difficulty in understanding the elicitation tasks. Meanwhile, most elicitation methods, except for the selfassessment method, provide evidence that respondents are, on average, risk-averse. In addition, the degrees of risk aversion are slightly lower in the MPL than in the investment task.1 Hence, when comparing risk preferences derived from survey or hypothetical and experimental methods, caution is advised. Results from an internal consistency test show that in the MPL task, 75% of subjects are consistent or nearly consistent when making a choice between a hypothetical and an experimental situation. However, many more people (more than half of the sample) show inconsistent responses between experiment and hypothetical questions for loss aversion. Meanwhile, the strongest correlation is between questions that have the same design such as the MPL and loss–gain tasks. The investment task also shows a strong association with other methods like MPL and loss–gain. In contrast to Dohmen et al. (2011), for example, we find that self-assessment, both in general and in specific contexts, has limited validity as it has the smallest or no relation with other measures. This finding is in line with Binswanger (1980) and could reflect the differences between developed and developing countries. The rest of the study proceeds as follows. Section 2 begins by laying out the research design and describes elicitation methods used for this study. Section 3 analyzes the results of internal consistency. Section 4 presents and discusses the findings of experimental and behavioral relevance validity tests. Section 5 concludes the study. 2. SURVEY AND EXPERIMENTAL DESIGN 2.1 Research Area The field survey and experiment were conducted in rural areas of two provinces, Kien Giang and Long An, located in the Mekong Delta region of southern Viet Nam. Kien Giang is known for fishing, shrimp growing, and rice farming with nearly 90% of its population living in rural areas. Long An is situated in an advantageous position in the Southern Key Economic Region of Viet Nam. It serves as a bridge between the big city—Ho Chi Minh City in the north—and 12 provinces in the Mekong Delta in the 1 In the MPL, the mean Constant Relative Risk Aversion (CRRA) is 1.12 (SD 1.07) for the hypothetical setting and 1.09 (SD 1.13) for the experimental setting while the mean midpoint of the CRRA interval in the investment task is 2.51 (SD 1.76). ADBI Working Paper 1433 Trang and Munro 3 south. Due to its low-lying geography, Long An has some areas that are subject to flooding during the rainy season and is susceptible to sea level rises caused by climate change. In recent years, the two provinces have experienced major shocks such as saltwater intrusion (Kien Giang) and flooding (Long An). The main economic activities in Long An are rice production and growing crops. The two provinces share similar geographical and economic characteristics and are suitable places to examine the impact of shocks on the daily decisions of people and their attitude towards risk. 2.2 Survey and Experimental Design We conducted a field survey and an experiment from January to May 2019 with a random sample of 350 households. Twenty-five households were interviewed in each of six rural villages in two communes in Kien Giang province and 25 households in each of eight rural villages in two communes in Long A province. The households were randomly chosen from a complete population list of the villages by systematic sampling.2 One month before the real survey and experiments, we provided training for enumerators and implemented a pilot survey. In each household, we interviewed a household representative member face-to-face. The interview lasted about 1.5 hours and comprised two main parts: survey and experiment parts. The first part was the survey part consisting of detailed demographic information, hypothetical elicitation questions, and risk perceptions. After completing the survey part, subjects participated in an experiment. The experiment part included three main tasks with some similar elicitation methods to those in the survey part. However, subjects were paid in this section depending on their choice. To prevent a spillover effect in the thinking process, the time gap between when subjects answered the hypothetical questions and the experimental questions was about 45 minutes. To help subject comprehension, the enumerators read the questions aloud and used examples, pictures, and red and black tokens to explain about 50:50 probability. As for the implementation of the payout, before starting the interview, subjects were informed that after they had completed both the survey and experiment parts, they would receive a fixed participation fee of VND90,000. In addition, they might lose or gain some amount of money aside from the participation fee depending on their choice in the experiment. After a participant had completed the experiment, subjects pulled a chip from a bag to determine which question became relevant for that participant’s payoff. 2.3 Data Description Table 1 presents key summary statistics of the sample. More than 70% of the participants are household heads. The average age is 48.3 years old, and the average number of years of schooling is six. Females account for about 30% of the sample and 97% of sampled individuals are married. On average, a household has four members. Nearly 40%, 30%, and 10% of participants report that they smoke, drink, and play a lottery very often, respectively. 2 Systematic sampling is a probability method in which researchers select members of the population at a regular interval determined in advance. In our case, the commune leaders provided us with a list of household heads in each village in alphabetical order. We decided to sample every 20th or 30th household in each village. ADBI Working Paper 1433 Trang and Munro 4 Table 1: Descriptive Statistics (N = 350) Mean Household head 0.71 Age 48.38 Male 0.71 Years of schooling 6.07 Household size 4.70 Number of children 2.86 Married 0.97 Kinh (Ethnic Vietnamese) 0.94 No religion 0.83 Dependency ratio 0.49 Household income (log) 18.39 Average consumption per month (log) 15.52 Agriculture land, acre 26.55 House ownership (= 1) 0.98 Lottery 0.09 Smoking 0.37 Drinking 0.28 Note: Playing lottery, smoking, and drinking = often doing the activities. The average survey and experimental earning for the three tasks was VND196,242 (about USD19 at the time), equivalent to about six to nine days’ wages for casual unskilled labor such as harvesting and construction work. 3. METHODS TO ELICIT RISK PREFERENCES Overall, five methods were used: survey questions, lottery task (lowand high-stake), loss–gain task, multiple price list, and investment task. Notably, among these methods, the study utilized a set of hypothetical elicitation questions from the Viet Nam Access to Resources Household Survey (VARHS).3 A detailed description of each elicitation method is presented below: 3.1 Self-assessment Willingness to Take Risks The survey questions are adopted from Dohmen et al. (2011). They measure the subject’s willingness to take risks in general and in some specific activities such as agriculture, healthcare, and investment in the education of children. Participants look at a Likert scale with integers ranging from zero (= completely unwilling to take risks) to 10 (= completely willing to take risks) and select the integer that best matches their own willingness to take risks. 3 The VARHS is a longitudinal household survey constructed biannually by the University of Copenhagen in collaboration with the Central Institute for Economic Management, the Institute for Labor Studies and Social Affairs, and the Institute of Policy and Strategy for Agriculture and Rural Development for rural areas of 12 provinces in Viet Nam. The survey includes questions to measure risk preferences. In particular, hypothetical lottery and loss–gain tasks are included in three waves: 2010, 2012, and 2014. A hypothetical multiple price list was added to the questionnaire in 2016 and 2018. ADBI Working Paper 1433 Trang and Munro 11 (IC) Inconsistent in risk aversion: Subjects who respond contrast between two tasks. For instance, they are highly risk-averse (r > 2.91) in the experimental MPL while being much less risk-averse (r < 0) in the hypothetical MPL. These subjects are concentrated in the top-right and bottom-left corners of the table. They account for nearly 25% of the sample. Internal Consistency Between MPL and Loss–Gain Tasks In Section 3.4, we obtained the loss aversion interval for each individual in each hypothetical and experimental loss–gain task by using their equivalent risk preferences from hypothetical and experimental MPL tasks, respectively. If an individual has the same risk preferences in both hypothetical and experimental MPL tasks, they may have the same loss aversion interval. If the risk preferences are different in both cases, we would like to examine whether their loss aversion intervals overlap in both cases. Figure 2 shows the distribution of individual responses between numbers of accepted lotteries in the loss–gain task and their associated risk preference in the MPL task. The left side of the figure is for the hypothetical situation while the right side of the figure is for the experimental case. The X-axis shows numbers of accepted lotteries in the loss–gain task, while the Y-axis shows the proportion of subjects, and the Z-axis shows the risk preference parameter (r) in the MPL. Figure 2: Percentage Responses from MPL and Loss–Gain Tasks Na* = Always choose safe option A in MPL; 98* = Irrational answers. The figure shows a relatively clear pattern in which a more risk-averse person, who has a higher value of the risk aversion parameter (r) in the MPL task (shown along the Z-axis), has lower numbers of accepted lotteries. In other words, more risk-averse people are less likely to agree to participate in the lottery. Subjects who always choose the safe option in the MPL task mostly reject or accept only one or two lotteries as observed in the light blue column: 22% of the participants in the experiment and nearly 20% of them in the hypothetical task. Similarly, the trend is reversed when subjects are less risk-averse (r is smaller than 1) in the MPL task. They tend to have higher numbers of accepted lotteries and their distributions skew to the right of the figure. ADBI Working Paper 1433 Trang and Munro 12 Internal Consistency Between Hypothetical and Experimental Loss–Gain Tasks A subject is considered to be consistent between the two loss–gain tasks when their two loss intervals from hypothetical and experimental cases overlap. And if the intervals do not overlap, they are inconsistent in revealing their loss preferences. After excluding irrational participants and participants who always choose the safe option in the MPL tasks,10 we categorize the participants by looking closely at their two loss aversion intervals: • (C_L) Consistent in loss aversion: Subjects whose two loss aversion intervals overlap. • (NC_L) Nearly consistent in loss aversion: Subjects have the two loss aversion intervals that are next to each other or have one common point. • (IC_L) Inconsistent in loss aversion: Subjects have two loss aversion intervals that do not overlap and are not next to each other. Depending on the gap between the two loss aversion intervals, we have different degrees of inconsistency: Subjects are very inconsistent if the gap is very big or the two intervals are very far away from each other, particularly if the gap is larger than or equal to 1; individuals are inconsistent if the gap is from 0.12 to 1; and individuals are slightly inconsistent if the gap is smaller than 0.12. In general, 60% of subjects are inconsistent in loss aversion while 40% are consistent or nearly consistent in loss aversion. 4.2 Correlation Among Elicitation Methods Table 6 presents correlation among elicitation methods.11 Correlations are larger and more statistically significant among subjects who answered tasks that have a similar design rationality. Specifically, for the lottery, MPL, and loss–gain tasks, the association is substantially high and significant. The degree of correlation in most cases is more than half. For instance, there is a highly significant and positive correlation between the lottery2 and the lottery20 with a magnitude of 0.86. In the MPL task, the correlation between hypothetical and experimental cases is 0.52, while in the loss–gain task, the correlation between hypothetical and experimental ones is relatively low (0.38). The observed correlation might be explained by the fact that people might not perceive or feel the loss in the hypothetical case as clearly as in the experimental case. In the loss–gain experiment, the respondents are aware that the loss would be deducted from their endowment (in this case, the participation fee) and so they might take a longer time to think and consider before making a final decision in the loss–gain experiment. 10 In which there are 21 irrational subjects who have multiple or reverse switching and 107 subjects who always choose the safe option in the MPL tasks, so their risk parameter interval is unidentified (referred to as “NA” in the MPL tasks). In the end, 222 subjects have two specific intervals of loss aversion parameters. 11 We do not include the self-reported questions in the specific domain as we find no significant correlations between most self-assessment methods and other elicitation methods. Also, the lottery tasks, both low-stake (lottery2) and high-stake (lottery20), do not have a significant connection with most of the methods. ADBI Working Paper 1433 Trang and Munro 13 Table 6: Correlations Between the Elicitation Methods (N = 329) Hypothetical Tasks Hypothetical Tasks Incentivized Tasks WTTR Lottery2 Lottery20 MPL Loss– Gain MPL Loss– Gain Amount Invested WTTR in general 1.00 –0.06 –0.02 –0.10 0.01 –0.04 0.04 0.07 Lottery2 (low-stake) –0.06 1.00 0.86 –0.07 0.24 –0.04 0.12 0.12 Lottery20 (high-stake) –0.02 0.86 1.00 –0.08 0.22 –0.05 0.13 0.09 Hypothetical MPL –0.10 –0.07 –0.08 1.00 –0.39 0.49 –0.38 –0.34 Hypothetical loss–gain 0.01 0.24 0.22 –0.39 1.00 –0.42 0.34 0.28 Incentivized Tasks MPL –0.04 –0.04 –0.05 0.49 –0.42 1.00 –0.56 –0.48 Loss–gain 0.04 0.12 0.13 –0.38 0.34 –0.56 1.00 0.48 Amount invested 0.07 0.12 0.09 –0.34 0.28 –0.48 0.48 1.00 Note: Spearman correlations reported. Inconsistent responses are excluded. A higher score in WTTR in general reflects more willingness to take risks; lottery and investment tasks are measured in terms of amount invested; the MPL task is measured in terms of numbers of safe options chosen; the loss–gain task is measured in terms of numbers of accepted lotteries. Correlations above 0.20 are in bold. *** significant at 1%, ** significant at 5%, and * significant at 10%. The correlation between the MPL and the loss–gain task is also high and significant. Negative signs show that safer options chosen in the MPL task are associated with fewer accepted lotteries in the loss–gain one. The strongest association is between the experimental MPL and loss–gain tasks (0.66). In the investment task, the invested amount has strong and significant connections with other responses in the loss–gain and MPL tasks. The strongest relation is with the experimental loss–gain task (0.52) and the experimental MPL task (–0.53). A negative relationship between the amount invested and the experimental MPL task indicates that a higher amount invested from the investment game is linked with fewer safe options chosen in the incentivized MPL. In other words, people who have more numbers of safe options chosen in the experimental MPL task or are more risk-averse also tend to invest less in the investment scenario. To conclude, in the validity test of internal consistency, the strongest correlation is between hypothetical and experimental tasks with the same design, for instance, the MPL and loss–gain tasks, the MPL and loss–gain tasks. The investment scenario also shows a strong association with other methods like MPL and loss–gain. Selfassessment and hypothetical lottery tasks have the smallest or no relation with other measures. A possible explanation for his may be that people’s perception of “risk” in the self-assessment is quite different from the “risk” in the MPL or loss–gain tasks. In the latter, the risk is only defined by the two choices (50:50) and by monetary gains/losses, while the term “risk” in the self-rating questions people would perceive as being much more complex than what the MPL/loss–gain tasks would capture. 4.3 Experimental Validity of Elicitation Measures In this section, we examine the experimental validity of elicitation measures in two ways. We explore whether responses from hypothetical tasks can predict actual responses in the experimental tasks. For example, we want to study whether greater willingness to take risks in general and in specific contexts is closely connected with more choices of risky options in the experimental multiple price list. If this is the case, using hypothetical or survey tasks can be a time-efficient and cost-saving substitution for experiments. The following equation expresses the relationship of behaviors between elicitation methods: ADBI Working Paper 1433 Trang and Munro 14 Responses from experimental tasks = α + β*(Responses from hypothetical tasks) + controls + ɛ Table 7 reports the coefficient estimate based on a separate regression of the respective experimental elicitation method on a particular hypothetical risk measure with a set of controls. The results show that the self-report survey measure and the lottery tasks (both lowand high-stake) show no significant relation to all three experimental measures. Table 7: Validity of Experimental Relevance (1) (2) (3) Experimental MPL Experimental Loss–Gain Investment Willingness to take risk In general context –0.0694 0.0593 0.108 (0.0720) (0.0778) (0.0936) In agricultural activities –0.0776 0.101 0.127 (0.0696) (0.0655) (0.0798) Lottery2 0.0427 –0.0227 0.0206 (0.0329) (0.0294) (0.0433) Lottery20 0.00518 –0.00474 –0.00145 (0.00488) (0.00540) (0.00557) Hypothetical MPL 0.551*** 0.0254 –0.559*** (0.112) (0.0999) (0.0992) Hypothetical loss–gain –0.389*** –0.0837 0.399*** (0.0831) (0.0745) (0.0849) Controls Yes Yes Yes Observations 259 257 350 Note: Interval regression coefficient estimates. Each row reports coefficient estimate based on a separate regression of the particular risk measure and a set of controls. The set of controls includes age, gender, education, ethnicity, household consumption (log), household size, numbers of children, and dummies for enumerator. Multiple price list is measured in terms of numbers of safe options chosen. Loss–gain task is measured in terms of numbers of accepted risky options. Investment task is measured in terms of amount willing to invest from 0 to 100. Willingness to take risk is measured on a scale from 0 to 10; a higher score corresponds with higher willingness to take risk. Lottery tasks are measured in terms of the amount one is willing to pay for the lotteries. Details of the regressions are presented in the Appendix. Robust standard errors are reported in brackets below the coefficient estimates in each regression. *** p < 0.01, ** p < 0.05, * p < 0.1. On the other hand, hypothetical MPL and hypothetical loss–gain tasks are the most significantly relevant predictors in predicting actual risk-taking behaviors in the experimental multiple price list and investment tasks. The coefficients are significant at any conventional level, indicating that the responses given in the hypothetical measures do predict behaviors in the experiments. The sign of coefficients is also as expected. For instance, in column (1), the negative sign of coefficient (–0.389) indicates that when a subject accepted more risky options in the hypothetical loss–gain tasks, they also tended to choose more risky options in the experimental MPL. In general, this section confirms the validity of the experimental relevance of hypothetical MPL and loss–gain tasks in comparison to other methods such as self-assessment and lottery tasks. ADBI Working Paper 1433 Trang and Munro 15 4.4 Validity of Risk Preference Measures in Relation to Risky Behaviors In this section, we examine the validity of the elicitation measures with respect to several real-life behaviors that are vital for the subjects’ livelihood. The relationship between them is expressed in the following equation: Risky behaviors = α + β*(Responses from elicitation methods) + controls + ɛ, where risky behaviors include health-related behaviors such as smoking and drinking. Other risky behaviors are migration and major changes that a household has made since 2010 to manage farming and livelihoods. When asked about how risky the participants think migration is in comparison to not moving, more than 90% of the respondents stated that migration is riskier than staying in one place. The survey area is located in the Vietnamese Mekong Delta – a rural lowland area that is exposed to severe conditions and climate change such as drought, salinization, and flooding (Trinh and Munro 2023). More than 80% of the sampled households were affected by severe drought and salinity intrusion during 2015–2016. Respondents were asked about major changes they had made to manage farming and livelihood over the last ten years. 12 There are five major changes: adjusting planting calendar (50%); crop/livestock/aquaculture diversification or changing varieties (78%); investing in irrigation (38%); finding other nonfarm activities for income (22%); and moving to other provinces or cities for working and living (3%). Each change inherits higher risks and more other uncertainties than other changes: for instance, trying new seeds in crop diversification, and moving to new places. Therefore, more changes inherently increase risks and uncertainties, and so it requires more willingness to take risks when adopting more adjustments. In our sample, among those we surveyed, 7% didn’t make any changes. Around 60% made one or two changes, while 30% made at least three changes. We then examine whether risk-averse subjects (measured by each elicitation method) are more likely to adopt fewer than three major changes. The results in Figure 3 show that risk preferences measured by the MPL (both hypothetical and experimental settings) and by the loss–gain task (experiment) are significantly related to the implementation of at least three major changes. A one-standard-deviation increase in the numbers of safe options chosen in the hypothetical MPL task is associated with a nearly 6% increase in the probability of making at least three major changes or an 18% increase over the mean. In addition, Figure 3 shows that risk preferences from the MPL task and the hypothetical loss–gain task significantly predict the propensity to migrate. A one-standard-deviation increase in accepted risky options in the loss–gain task is associated with about a 4% increase in the probability of migration. Smoking is used in many studies as a risky health behavior. Furthermore, smoking has been used as a proxy for risk preferences where there are no direct measures of risk attitude (Dohmen et al. 2011). The corresponding variable is equal to 1 if the subject smokes. For smoking and drinking, willingness to take risks in the domain of health and more risky options chosen in the loss–gain tasks have a stronger and highly significant association with smoking and drinking as shown by the larger marginal effect (Figure 4). 12 The original question was: “In the past 10 years, has your family made any big adjustments or changes in agricultural activities and living?” ADBI Working Paper 1433 Trang and Munro 16 Figure 1: Risky Behavior Validation Note: Separate logit regression models are estimated for each behavior as outcome and each elicitation method as control variable of interest. The confidence interval is 90%. Behavior outcomes are binary. All risk measures are standardized. Reported coefficients are probit marginal effect estimates, evaluated as the means of independent variables. Therefore, the coefficients show the impact of a one-standard-deviation change in the corresponding measure of risk preferences. Other controls include gender, age, education, household affected by climate change in 2015, household land, household consumption (log), number of children, household size, and dummies for the enumerator. Robust standard errors allow for clustering at the village level. Figure 2: Health-Related Behaviors: Predictive Validity Note: Separate logit regression models are estimated for each behavior as outcome and each elicitation method as control variable of interest. The confidence interval is 90%. Behavior outcomes are binary. All risk measures are standardized. Reported coefficients are probit marginal effect estimates, evaluated as the means of independent variables. Therefore, the coefficients show the impact of a one-standard-deviation change in the corresponding measure of risk preferences. Other controls include gender, age, education, household land, household consumption (log), number of children, household size, and dummies for the enumerator. Robust standard errors allow for clustering at the village level. ADBI Working Paper 1433 Trang and Munro 17 A one-standard-deviation increases in willingness to take risks in healthcare is associated with a 7% increase in the probability of being a smoker and a 5% increase in the propensity to drink. Given a sample mean of 37%, this translates into an 18.8% increase for smoking. The finding about smoking is similar to that of Dohmen et al. (2011), who show that a one-standard-deviation increase in willingness to take risks in healthcare increases the likelihood of being a smoker in Germany by 20% of the mean. 5. DISCUSSIONS AND CONCLUSIONS This study examines the validity of various elicitation methods in the context of rural areas in Viet Nam. We conducted a field survey and an experiment with 350 households. The elicitation methods include four hypothetical and three experimental tasks, in which we utilize a set of hypothetical questions from a Vietnamese household survey. We provide a more comprehensive validity test of elicitation methods than other existing studies (e.g., Nielse, Keil, and Zeller 2013; Dohmen et al. 2011). This study is also the first to investigate the validity of hypothetical elicitation questions in a household survey in Viet Nam. Most of the participants have no difficulty in understanding the elicitation tasks. Most elicitation methods, except for the self-assessment method, provide evidence that respondents are, on average, risk-averse. This finding supports other studies in Viet Nam (e.g., Tanaka, Camerer, and Nguyen 2010; Nielsen, Keil, and Zeller 2013). Respondents appear less risk-averse in the self-assessment method than in other methods such as in the MPL and investment tasks. In addition, the degrees of risk aversion are slightly lower in the MPL than in the investment task.13 Hence, when comparing risk preferences derived from survey or hypothetical and experimental methods, caution is advised. Discrepancies may arise due to the broad context brought by surveys, despite their simplicity, ease of use, and cost-effectiveness. Characteristics of each type of elicitation may have also contributed to the difference in the perception of “risk.”14 Notably, people would perceive the term “risk” in the self-assessment as being much broader and more complex than what MPL/loss–gain tasks would capture. In comparison, the “risk” in the MPL or loss–gain tasks is only defined by the two choices (50:50) and by monetary gains/losses. In addition, survey methods often lack a clear theoretical background, limiting their usefulness in estimating utility function parameters, thereby restricting their applicability in structural modeling.15 As usually happens with self-report questions, this may be biased due to framing effects because about 20% of total subjects selected the middle category.16 Our findings show that MPL and loss–gain tasks perform best amongst the methods in this context. Self-assessment, in general and in most specific contexts (except for self-assessment in healthcare), has limited validity since it has the smallest or no relation with other elicitation measures and risky behaviors. This finding is similar to those reported by Nielsen, Keil, and Zeller (2013) for Vietnamese farmers, Lönnqvist et al. (2015) for German students, Ding, Hartog, and Sun (2014) for Chinese students, 13 In the MPL, the mean CRRA is 1.12 (SD 1.07) for the hypothetical setting and 1.09 (SD 1.13) for the experimental setting, while the mean midpoint of the CRRA interval in the investment task is 2.51 (SD 1.76). 14 The respondents were given a clear definition of risk and risk taking at the beginning of the survey and experiment. 15 Eckel, C. C. 2019. Measuring Individual Risk Preferences. IZA World of Labor: https://wol.iza.org/ articles/measuring-individual-risk-preferences/long. 16 As proposed by Nielsen, Keil, and Zeller (2013), the self-assessment task can be rescaled, such as from 0 to 9, to avoid an easily identifiable middle category. ADBI Working Paper 1433 Trang and Munro 18 Bauer, Chytilová, and Miguel (2020) for Kenyan farmers, and Binswanger (1980) for Indian farmers17 but differ from other studies that support the use of self-assessment of risk attitude in surveys, such as Dohmen et al. (2011) and Hardeweg , Menkhoff, and Waibel (2013). A possible explanation for the opposite result might come from differences in the study context. Subjects in Dohmen et al. (2011) are German adults from a range of backgrounds, while subjects in our study are Vietnamese farmers. They may have different life experiences, living and working environments, and personal traits, so their perception of risks and interview behavior may differ. Understanding whether the differences between our and Binswanger’s (1980) results, on the one hand, and those in Dohmen et al. (2011), on the other, are driven by differences between developing and industrialized countries requires further research from a wider range of cultures and population subgroups. 17 Nielsen, Keil, and Zeller (2013) find that the correlation between self-assessment scale and multiple price list is weak (0.19). Ding, Hartog, and Sun (2014) found a low association among the hypothetical lottery question, the self-assessment question, and an experimental lottery. ADBI Working Paper 1433 Trang and Munro 19 REFERENCES Anderson, L. R., and J. M. Mellor. 2009. Are Risk Preferences Stable? Comparing an Experimental Measure with a Validated Survey-Based Measure. Journal of Risk and Uncertainty 39(2): 137–160. https://doi.org/10.1007/s11166-009-9075-z. Armin, Falk, A. Becker, T. J. Dohmen, D. Huffman, and U. Sunde. 2016. The Preference Survey Module: A Validated Instrument for Measuring Risk, Time, and Social Preferences. SSRN Electronic Journal. https://doi.org/10.2139/ ssrn.2725874. Baetschmann, G., A. Ballantyne, K. E. Staub, and R. Winkelmann. 2020. Feologit: A New Command for Fitting Fixed-Effects Ordered Logit Models. Stata Journal 20(2): 253–275. https://doi.org/10.1177/1536867X20930984. Bauer, M., J. Chytilová, and E. Miguel, 2020. Using Survey Questions to Measure Preferences: Lessons from an Experimental Validation in Kenya. European Economic Review, 127: 103493. Bezabih, M., and M. Sarr, 2012. Risk Preferences and Environmental Uncertainty: Implications for Crop Diversification Decisions in Ethiopia. Environmental and Resource Economics Volume 53: 483–505. Binswanger, H. P. 1980. Attitudes toward Risk: Experimental Measurement in Rural India. American Journal of Agricultural Economics 62(3): 395–407. Blattman, C., J. Jamison, T. Koroknay-Palicz, K. Rodrigues, and M. Sheridan. 2016. Measuring the Measurement Error: A Method to Qualitatively Validate Survey Data. Journal of Development Economics 120: 99–112. https://doi.org/10.1016/ j.jdeveco.2016.01.005. Bohm, P., J. Lindén, and J. Sonnegård. 1997. Eliciting Reservation Prices: Becker– DeGroot–Marschak Mechanisms vs. Markets. The Economic Journal 107(443): 1079–1089. Brauw, A. de, and P. Eozenou. 2014. Measuring Risk Attitudes among Mozambican Farmers. Journal of Development Economics 111: 61–74. https://doi.org/ 10.1016/j.jdeveco.2014.08.002. Burfurd, I., and T. Wilkening. 2018. Experimental Guidance for Eliciting Beliefs with the Stochastic Becker–DeGroot–Marschak Mechanism. Journal of the Economic Science Association 4(1): 15–28. Charness, G., and A. Viceisza. 2016. Three Risk-Elicitation Methods in the Field: Evidence from Rural Senegal. Review of Behavioral Economics 3(2): 145–171. https://doi.org/10.1561/105.00000046. Charness, G., and Gneezy, U. (2010). Portfolio choice and risk attitudes: An experiment. Economic inquiry, 48(1), 133–146. Charness, G., T. Garcia, T. Offerman, and M. C. Villeval. 2020. Do Measures of Risk Attitude in the Laboratory Predict Behavior under Risk in and Outside of the Laboratory? Journal of Risk and Uncertainty 60(2): 99–123. https://doi.org/10.1007/s11166-020-09325-6. Charness, G., U. Gneezy, and A. Imas. 2013. Experimental Methods: Eliciting Risk Preferences. Journal of Economic Behavior and Organization 87: 43–51. https://doi.org/10.1016/j.jebo.2012.12.023. ADBI Working Paper 1433 Trang and Munro 20 Clarke, D., J. P. Romano, and M. Wolf. 2020. The Romano–Wolf Multiple-Hypothesis Correction in Stata. The Stata Journal 20(4): 812–843. Cleave, B. L., N. Nikiforakis, and R. Slonim. 2013. Is there Selection Bias in Laboratory Experiments? The Case of Social and Risk Preferences. Experimental Economics 16(3): 372–382. Cooper, D. J., and K. J. Saral. 2013. Entrepreneurship and Team Participation: An Experimental Study. European Economic Review 59: 126–140. Corr, P., and A. Plagnol. 2018. Behavioral Economics: The Basics. Routledge. Crosetto, P., and A. Filippin. 2016. A Theoretical and Experimental Appraisal of Four Risk Elicitation Methods. Experimental Economics 19(3): 613–641. Dasgupta, U., S. Mani, S. Sharma, and S. Singhal. 2019. Internal and External Validity: Comparing Two Simple Risk Elicitation Tasks. Journal of Behavioral and Experimental Economics 81: 39–46. Dave, C., C. C. Eckel, C. A. Johnson, and C. Rojas. 2010. Eliciting Risk Preferences: When Is Simple Better? Journal of Risk and Uncertainty 41(3): 219–243. Ding, X., J. Hartog, and Y. Sun. 2014. Can We Measure Individual Risk Attitudes in a Survey? Forschungsinstitut Institute for the Study of Labor, May. Dohmen, T., A. Falk, D. Huffman, and U. Sunde. 2018. On the Relationship between Cognitive Ability and Risk Preference. Journal of Economic Perspectives 32(2): 1151–34. Dohmen, T. et al. 2011. Individual Risk Attitudes: Measurement, Determinants, and Behavioral Consequences. Journal of the European Economic Association 9(3): 522–550. https://doi.org/10.1111/j.1542-4774.2011.01015.x. Drost, E. A. 2011. Validity and Reliability in Social Science Research. Education Research and Perspectives 38(1): 105–123. Einav, L., A. Finkelstein, I. Pascu, and M. R. Cullen. 2012. How General Are Risk Preferences? Choices under Uncertainty in Different Domains. American Economic Review 102(6): 2606–2638. Falk, A., A. Becker, T. Dohmen, B. Enke, D. Huffman, and U. Sunde. 2018. Global Evidence on Economic Preferences. The Quarterly Journal of Economics 133(4): 1645–1692. Fehr, E., and L. Goette. 2007. Do Workers Work more if Wages Are High? Evidence from a Randomized Field Experiment. American Economic Review 97(1): 298–317. Gächter, S., E. J. Johnson , and A. Herrmann. 2022. Individual-level Loss Aversion in Riskless and Risky Choices. Theory and Decision 92(3): 599–624. Gal, D., and D. D. Rucker. 2018. The Loss of Loss Aversion: Will It Loom Larger than Its Gain? Journal of Consumer Psychology 28(3): 497–516. Gloede, O., L. Menkhoff, and H. Waibel. 2015. Shocks, Individual Risk Attitude, and Vulnerability to Poverty among Rural Households in Thailand and Vietnam. World Development 71: 54–78. https://doi.org/10.1016/j.worlddev.2013.11.005. Gneezy, U., and J. Potters. 1997. An Experiment on Risk Taking and Evaluation Periods. Quarterly Journal of Economics 112(2): 631–645. https://doi.org/ 10.1162/003355397555217.