Product representations in conjoint analysis in an LMIC setting: Comparing attribute valuation when three-dimensional physical prototypes are shown versus two-dimensional renderings

Coulentianos, Marianna J. et al. Article Product representations in conjoint analysis in an LMIC setting: Comparing attribute valuation when three-dimensional physical prototypes are shown versus two-dimensional renderings Development Engineering Provided in Cooperation with: Elsevier Suggested Citation: Coulentianos, Marianna J. et al. (2021) : Product representations in conjoint analysis in an LMIC setting: Comparing attribute valuation when three-dimensional physical prototypes are shown versus two-dimensional renderings, Development Engineering, ISSN 2352-7285, Elsevier, Amsterdam, Vol. 6, pp. 1-13, https://doi.org/10.1016/j.deveng.2021.100063 This Version is available at: https://hdl.handle.net/10419/242320 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ Development Engineering 6 (2021) 100063 Available online 26 May 2021 2352-7285/© 2021 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). Product representations in conjoint analysis in an LMIC setting: Comparing attribute valuation when three-dimensional physical prototypes are shown versus two-dimensional renderings Marianna J. Coulentianos a , * , Mojtaba Arezoomand a , Suzanne Chou a , Jesse Austin-Breneman a , Achyuta Adhvaryu a , Kowit Nambunmee b , c , Richard Neitzel a , Kathleen H. Sienko a a University of Michigan, Ann Arbor, United States b School of Health Science, Mae Fah Luang University, Chiang Rai, Thailand c Urban Safety Innovation Research Group (USIR), Mae Fah Luang University, Chiang Rai, Thailand ARTICLE INFO Keywords: Conjoint experiment Product representation Renderings Physical prototypes Lowand Middle-Income Countries ABSTRACT Conjoint experiments (CEs) provide designers with insights into consumer preferences and are one of several user-based design approaches aimed at meeting users’ needs. Traditional CEs require participants to evaluate products based on two-dimensional (2D) visual representations or written lists of attributes. Evidence suggests that product representations can affect how participants perceive attributes, an effect that might be exacerbated in a Lowand Middle-Income Country setting where CEs have seldom been studied. This study examined how physical three-dimensional (3D) prototypes and 2D renderings with written specifications of attribute profiles generated differences in estimated utilities of a CE about a hypothetical new tool for electronic-waste recycling, among workers in North-Eastern Thailand. Two independent CEs were performed with each representation form. Ninety participants across both experiments each ranked three sets of five alternative tool concept solutions from most to least preferred. The results of the conjoint analysis guided the design of a tool optimized for user preferences, which was then distributed to half of the sample through a Becker-DeGroot-Marschak auction experiment. One month after the auction, participants completed an endline survey. The results point toward potential differences in relative importance of different product attributes based on product representation. Price was found to have no significant impact on the valuation of tools in either experiment. The differences in relative importance of product attributes may have been explained by the limitations of 2D renderings for conveying sizes. Further research is needed to understand the impact of product representation on preferences in this context. We recommend careful consideration for product representations – specifically, how well the representations convey all product attributes being evaluated – in CEs. Using a combination of 2D renderings and 3D product features might have satisfied both the speed and low-cost advantages of renderings while enabling participants to have a better sense of product features. 1. Introduction Many products developed for Lowand Middle-Income Countries (LMICs) with demonstrated benefits still retain very low adoption rates even when distributed for free. Examples can be found for water filtration schemes (Berry et al., 2020), cook stoves (Levine et al., 2018; Mobarak et al. n.d.), and bed nets (Yukich et al., 2017). Some studies have investigated and documented the reasons for low adoption, a main reason being the lack of good contextual design (Chavan et al., 2009). Gathering reliable user data in developing settings may be challenging because of the lack of infrastructure usually relied on in developed countries such as receipts, web traffic, and household economic surveys (Kroll et al., 2014). In response, designers are developing diverse methods to understand consumer preferences and estimate demand * Corresponding author. E-mail addresses: [email protected] (M.J. Coulentianos), [email protected] (M. Arezoomand), [email protected] (S. Chou), [email protected] (J. AustinBreneman), [email protected] (A. Adhvaryu), [email protected] (K. Nambunmee), [email protected] (R. Neitzel), [email protected] (K.H. Sienko). Contents lists available at ScienceDirect Development Engineering journal homepage: www.elsevier.com/locate/deveng https://doi.org/10.1016/j.deveng.2021.100063 Received 28 September 2020; Received in revised form 12 April 2021; Accepted 10 May 2021 Development Engineering 6 (2021) 100063 2 curves in developing settings (Kroll et al., 2014). Prominent methods to model consumer behavior are discrete choice experiments (DCEs) and conjoint experiments (CEs). While there is overlap between these two methods, in this paper we discuss rank-based (or rating-based) CEs, different from traditional choice-based methods in DCEs. CEs provide a means of investigating relative preferences (trade-offs) across attributes of goods or services and are widely applied in marketing research (Green and Srinivasan, 1978; Wittink et al., 1990). Marketers have traditionally employed such methods to assess consumer trade-offs for product features in High-Income Countries (HICs) (Green et al., 2004; Green and Srinivasan, 1990). In addition, CEs have been commonly used in transportation, psychology, environmental valuation, municipal planning and others (Hope and Garrod, 2004; Scarpa et al., 2003). In a CE experiment, consumers are presented with alternative profiles with varying attribute levels and are asked to choose their preferred profile. Conjoint analysis assumes that consumers make choices based on the sum of utilities derived from specific attribute levels of a product or service. The goal of conjoint analysis is to estimate the utilities for each attribute (Green and Srinivasan, 1978), which enable designers to make design decisions about product attributes to include. CE is increasingly being used in LMICs. However, conjoint-based studies have mainly been studied in HIC settings; there have been very few studies reported of CEs applied in LMIC settings (Mangham et al., 2009). In LMICs, CEs have been used in agriculture (Kamuanga et al., 2002; Yesuf et al., 2005), for clean water initiatives (Hope and Garrod, 2004), and in the transport and tourism sectors (Baidu-Forson et al., 1997; Tiwari and Kawakami, 2001). The use of CEs in LMICs has also been reported for health policy and planning questions, where it appears to be of growing interest (Chomitz et al., 1998; Hanson et al., 2005). Moreover, Baltussen and Niessen (2006), argued that choice experiments, as a technique for undertaking multi-attribute analysis, should be used more routinely to guide resource allocation decisions in the field of global health (Baltussen and Niessen, 2006). However, very few studies consider the population characteristics in which the CE is being performed and the impact that may have on the outcomes of using different CE designs (He et al., 2012). We cannot assume that the research on product representation is transferrable to an LMIC context, where familiarity with survey methods and local contextual factors could influence the outcomes. Hence, studying CE methods in LMIC contexts is needed. In addition, there are concerns over general CE validity, whether carried out in LMICs or not, since the outcomes of CE rely on participants being able to respond according to their true preferences (Orzechowski et al., 2005). The parameters of CE, such as the response format (Boyle et al., 2001), the attributes and levels included (Zhang et al., 2015), and the order of presentation of attributes (Kjær et al., 2006), have been shown to affect participants’ revealed preferences during a CE. For example, the analytical hierarchy process has been shown to be inadequate as a field method in one LMIC setting (Chou et al., 2020). Product representation is one such parameter that has been shown to affect participant preferences (Sylcott et al., 2016). Product representation concerns the way in which the attribute levels in a CE are presented. Typical representations of products or services in CE consist of verbal descriptions of the attribute levels, presented as a list, which might be complex and hard to understand (Orzechowski et al., 2005). Verbal descriptions could lead to misinterpretations, especially in a cross-cultural setting, where designers from HICs are conducting CEs in LMICs (Meyer and Rosenzweig, 2016). The use of pictures have been recommended when conducting CEs in LMICs (Meyer and Rosenzweig, 2016). However, using images could lead participants to focus on aspects of the product that are irrelevant to the CE (Orzechowski et al., 2005). Research in engineering design has also shown that a prototype form can impact the feedback received by stakeholders in other methods such as usability testing (Reyes et al., 2017) and interviews in an LMIC setting (Deininger et al., 2019). Hence, prior research suggests that the importance of product representation might be exacerbated in LMIC settings, which is why we believe this analysis is pertinent in this setting. In this study, we proposed to examine the impact on estimated product attributes of conducting a CE with 3D physical prototypes versus 2D renderings with verbal specifications in an LMIC setting. This study contributes to the limited literature on the effect of product representation on CE outcomes and on the use of CEs in LMIC settings. 2. Background 2.1. Product representation in CEs The impact of various product representation in conjoint analysis on the valuation of product features has been studied across product types. One might think that the ideal product representation would be a highfidelity physical model. For example, Dominique-Ferreira et al. (2012), used real water bottles to conduct a CE on bottle preferences (Dominique-Ferreira et al., 2012). However, creating physical products for a CE comes at a high cost in time, money, space, and logistics because of the number of product variations that need to be created (Tovares et al., 2014). Hence, creating physical prototypes for all product variations is often infeasible. Because of the high cost associated with creating physical models for every feature level of most products, various studies examined the impact of product representation on the outcomes of CEs (Tovares et al., 2014), with the goal of understanding what product representations can lead to reliable results at an affordable implementation price. Much of the research on product representation in CE has examined the impact of product representation on aesthetics evaluation (Kelly and Papalambros, n.d.; Orsborn et al., 2009; Reid et al., 2010; Tseng et al., 2012). Some consensus exists around visual CE, where objects are represented with 2D images, as a way to accurately describe product aesthetic preferences while “effectively addressing the limitations of physical prototyping, focus groups, and traditional conjoint [with verbal descriptions]" (Tovares et al., 2014). However, research has also shown that introducing images in a CE may lead participants to evaluate ‘accidental details,’ a by-product of introducing imagery that carries more information than listing features and levels (Jansen et al., 2009; Sylcott et al., 2016). For example, showing physical prototypes that were not the final product led to a lower performing utility model due to the low-fidelity nature of the prototype regarding functional attributes, even when participants were asked to disregard those and concentrate on aesthetic evaluation (Sylcott et al., 2016). Vriens et al. (1998), concluded that pictorial representations do improve participants’ understanding of the attribute levels being tested as compared to verbal representations. However, verbal representations seem to make it easier for participants to make choices (Vriens et al., 1998). The use of rendering software to produce photorealistic images of products for CE comes at a higher cost than using verbal descriptions (Vriens et al., 1998) and might introduce bias into the attribute evaluation by participants. Some studies have investigated experiential CEs, where participants experience part of the product they are evaluating, for example, through virtual reality (Tovares et al., 2014). CEs have also been used earlier in a design process to elicit customer preferences for experiences, by evaluating storyboard scenarios later translated in product features (Kim et al., 2017). Other novel product representations include short videos (Intille et al., 2002), and a multimedia online buying environment meant to increase realism (Urban et al., 1996). 2.2. Product representation in CEs in LMICs The challenges of conducting CEs in cross-cultural LMIC setting relate to different cultural or language settings, low levels of literacy, and the novelty of market research techniques (Chou et al., 2020; Mangham et al., 2009). The literature on CEs in LMICs suggests that M.J. Coulentianos et al. Development Engineering 6 (2021) 100063 3 participants can state their preferences on health service provision and areas for policy reform (Baltussen and Niessen, 2006; Chomitz et al., 1998; Hanson et al., 2005; Mangham et al., 2009; McPake and Mensah, 2008). The results also suggest that the preferences are reasoned and deliberate. Hence, CEs seem to be a sensible choice of methodology for consumer preferences data collection in LMICs. When designing in LMICs, different cultural and language settings, low levels of literacy, and the novelty of market research techniques (Hope and Garrod, 2004) are reasons to hypothesize that the product representation may lead to misunderstandings and miscommunications between designers, marketers, and users, which impact feature evaluation. Meyer and Rosenzweig (2016) presented tools to use when conducting CEs in developing countries and recommend translating attributes into images (Meyer and Rosenzweig, 2016). Indeed, prototypes have been shown to be powerful communication tools and can aid stakeholders in understanding the concepts and ideas of the designer and to actively participate in the design process (Lauff et al., 2020). Hence, showing prototypes in CEs could increase the mutual understanding between designer and user. 2.3. Gap There is a gap in understanding the effect of 2D versus 3D product representation on CE results. In addition, there is a gap in understanding the effect of product representation on CE results when conducting CEs in a cross-cultural LMIC settings. This paper describes an experimental study that examined the effects of 2D versus 3D product representations on preferences of electronic-waste workers for a cutting tool, conducted in rural, north-eastern Thailand. 3. Methods This study aimed to answer the following research question: What is the effect of product representation on the estimated utilities of product attributes, when conducting a CE in a cross-cultural LMIC setting? 3.1. Study design A CE was conducted to better understand electronic-waste (e-waste) recycling workers’ preferences for features of a new cutting tool. E-waste recycling involves the dismantling of various electronic components such as refrigerators, fans, washing machines, and televisions to retrieve and sell various materials including steel, copper, aluminum, plastic, PCB, screen, and cables. The informal e-waste sector is less regulated (Perkins et al., 2014) and the rate of worker injury is much higher than in formal sectors (Arain, 2019). Multiple stakeholder engagement activities revealed increased risk when workers dismantle stators, depicted in Fig. 1. Figs. 1 and 2 also illustrate tools used by participants. E-waste workers in our sample bought their own tools and maintained them by regularly sharpening them. Hence, they were regularly making purchase choices and evaluating the tradeoffs in their choices for tools and were therefore a good population for a choice experiment to reveal tool preferences. The preferences revealed through the CE then led to the design of an optimized tool, which was manufactured and distributed to half the sample through an auction experiment. A total of 105 participants conducted a baseline survey and 83 participants conducted the 3D CE (i.e., with physical prototypes). Both activities were conducted during a field visit in August 2019. A subset (15) of the participants were not available to conduct the CE with physical prototypes at that time due to work travels and instead conducted the 2D CE (i.e., with paper prototypes) during a following field visit in November 2019. It was common for workers to hold multiple jobs in addition to e-waste recycling. Hence, it was common for workers to travel for their other work, such as agricultural work. Table 2 of the results section shows that both groups were balanced on e-waste as participants’ main job in the baseline survey (59% of 3D CE participants against 67% of 2D CE participants). Furthermore, we tried to limit any effects of the passage of time between the 3D CE conducted in August and the 2D CE conducted in November. We interviewed participants at their e-waste workplace, so they all had been working on e-waste that day which provided some consistency of context (versus interviewing a participant while they were doing farm work). To the best of our knowledge, there was no seasonality difference in the e-waste work. No prototypes of the tools were introduced to the community between August and November. While participants of the 2D conjoint could have talked to participants of the 3D conjoint about the experiment, no pictures had been taken so we expect that the amount of information exchanged was limited and would rather bias both samples towards similar results. 3.2. Attribute selection The research team conducted a field trip prior to August 2019, during which informal interviews were conducted with workers and feedback was gathered on early tool designs. Feedback from nine ewaste workers suggested a novel hand tool was of interest and attributes were determined based on designs and functions of preferred existing tools to dismantle motors: chisels and blades. Table 1 summarizes the attributes of the tool and their respective levels. The attributes were selected to represent the major design choices that would have a large impact on usability (mainly impacted by handle position, blade length), safety (guard), durability (blade thickness), and price (blade length, blade thickness, guard). The number of attributes and levels resulted in 24 different tool designs at 4 different prices for a Fig. 1. E-waste worker dismantling a stator with a blade and hammer (left). Set of typical tools used to dismantle E-waste (right). M.J. Coulentianos et al. Development Engineering 6 (2021) 100063 4 total of 96 possible alternatives. A rank-order design was chosen, where participants were presented with a subset of five knives and were asked to rank-order the different alternatives. Participants were presented with a total of three sets of five knives. This CE design was chosen to gather more information in a short amount of time, given the field constraints. The sets of knives were randomly generated. 3.3. Estimation procedure Given that participants’ utility functions are not directly observable, we indirectly estimated aggregate utilities by observing participants’ ranks when presented with sets of five tools. The model results in an estimation of the influence of the product attributes on participant choices. We assumed that participants could rank possible alternatives in order of preference and follow a logical process of choosing options that were more desirable. To analyze the data, we fit a rank order logit model also known as the exploded logit model (Punj and Staelin, 1978), using the cmrologit function in Stata (Stata Statistical Software, 2019). This model is appropriate for the data because it uses rank ordered alternatives, it generalizes a version of McFadden’s choice model in the case where alternatives vary for each participant, which is the case of our data (each participant saw different random sets of tools), and data from a same participant are linked together by a case ID variable. 3.4. Testing the product representation effect Two methods of representation of the attribute levels were developed: 3D physical prototypes and 2D renderings made from Computer Aided Design (CAD) models, to compare the effect of representation on stakeholder preferences. The 3D prototypes were built using materials from a home-improvement store. The 2D renderings were displayed on a packaging sleeve that mimicked the current blade purchased by participants and provided the specifications of the tool at the bottom of the package rendering, in the same format as the benchmark tool. Examples of the two trial set-ups are shown in Fig. 3 and a close-up of the price representations is shown in Fig. 4. We refer to the CE conducted with 3D physical prototypes as the 3D CE and we refer to the CE conducted with 2D renderings as the 2D CE. To study the effect of product representation on attribute valuation in the analysis of the CEs, we included interaction variables where all attributes were multiplied by a dummy variable (equal to 1 if the product representation is 2D; else 0). The statistical significance of interaction variable coefficients, interpreted as utilities, would signify that the product representation impacted the valuation of that attribute. Because our sampling method was non-random due to field constraints, we first examined the demographics of the two groups (3D CE and 2D CE, results presented in 4.1). Because we found an imbalance of the samples related to age, we conducted a sensitivity analysis for which we created a nearest neighbor matching for our 2D participants based on the following normalized baseline characteristics: Age (yrs.), Average monthly household income (kTHB), Number of people per household (person), Worker (binary), Gender (binary), Education secondary or higher (binary), E-waste as a main job (binary). We present the results of the sensitivity analysis in part 4.4. 3.5. Hypotheses We expected that the relative weighting of attributes would be affected by the different representations. Here, we formulated two specific hypotheses regarding the change in attribute weighting. Fig. 2. Close-up of typical tools used to dismantle e-waste (from left to right: blade, chisel, knife). Table 1 Tool attributes and respective levels. Prices in Thai currency, US$ 1 =THB 30.34. Attribute Description Levels and coding Expected sign of coefficient Price Purchase price Continuous variable in THB (100, 200, 300, 400) Negative Handle position Handle positioned at the top (mimicking a chisel design) or side of the blade (mimicking a knife design) Top =0 Side =1 Positive Blade length Length of the cutting blade: short 4′′ , medium 7′′ , and long 9′′ BL1 Medium =1 {Short, Long} ={0,0} Positive BL2 Long =1 {Short, Medium =0} = {0,0} Positive Blade thickness Thickness of the cutting blade Thin (0.8 mm) =0 Thick (3 mm) =1 Positive Guard Presence or absence of a hand guard to both protect from hammer hits and reduce vibrations Absent =0 Present =1 Positive M.J. Coulentianos et al. Development Engineering 6 (2021) 100063 5 H1. The weighting of the blade length and blade thickness attributes would decrease relative to the other attributes in the 2D CE. Indeed, 2D renderings were less effective at communicating size (de Beer et al., 2009) and blade length and thickness might therefore have been significantly less tangible in a rendering than in a 3D prototype representation. We hypothesized that participants would struggle to evaluate the different lengths and thicknesses accurately when shown a 2D rendering. H2. The weighting of the price attribute would increase in the 2D CE. The price attribute is more accurately represented in the 2D rendering, as it mimics the representation of the price of one of the blades that was used as a benchmark because it was frequently bought and used by participants (benchmark blade shown in Fig. 5). 3.6. Auction experiment and endline survey Based on the results of the 3D CE, an ‘optimal’ tool, which included all highest-ranking attributes (Fig. 6), was designed and manufactured locally. We conducted a Becker-DeGroot-Marschak (BDM) auction experiment to elicit participants’ WTP for the ‘optimal’ tool during the November field visit. A BDM auction experiment aims to elicit participants’ WTP for a product through a system of bids and random price draws. During the BDM experiment, participants stated their bid for the tool (that is the highest amount they were willing to pay for the tool). We then drew a random price. If the random price was greater than the participant’s bid, the participant did not purchase the tool. If the random price was lower than the participant’s bid, the participant purchased the product at the draw price rather than at their initial bid. The participants’ utility maximizing strategy is to bid their true maximum WTP, because the stated WTP does not affect the price paid, only the probability of purchasing the tool. Participants used play money provided by the research team and leftover playmoney money could be spent to buy household goods directly at the study location. During the auction experiment, 32 participants received the ‘optimal’ tool. None of the participants were aware of the opportunity to receive the tool when conducting the CEs. One month later, we conducted an endline survey to measure participants’ preferences for the tool, among other outcomes. In the survey, we asked a subset of multiple-choice questions about tool preferences to both participants who did and did not receive the tool, the visual aids for these questions are included in Appendix 1. These multiple-choice questions were based on design questions that remained after receiving some qualitative feedback during the auction experiment and in conversations with the manufacturer. The questions enabled us to further study how preferences evolved after using the tool for some time. A summary of the preference-related questions asked during the baseline, along with the answers to the questions, are included in Table 5 in the results. 4. Results 4.1. Participant demographics Table 2 displays a summary of participant demographics. The full Fig. 3. Pictures of the trial set-up for the 3D CE (left) and 2D CE (right), each presenting five alternative for participants to choose from. Fig. 4. Examples of price representations in the 3D CE (left, 300 บาท Thai Baht) and in the 2D CE (right, 100 บาท Thai Baht). M.J. Coulentianos et al. Development Engineering 6 (2021) 100063 6 sample of 98 participants was made up of 42 workers (i.e., participants who were employed in an e-waste firm), and 56 owners (i.e., participants who owned and operated their own e-waste business). A total of 54 participants were male and the average age of participants was 46 years (st.dev. 11). Participants’ average monthly household income was kTHB 8.7 (st.dev. 15). Only two respondents had never attended school, and 41% had attended secondary school or higher. A majority (61%) of participants stated that e-waste recycling was their main job. The average family size was 4.7 people (st.dev. 2.1). Participants available during the first field trip (August 2019) were assigned to the 3D CE, participants who were not available during the first field trip but were available during the second field trip (November 2019) were assigned to the 2D CE. We evaluated sample differences based on the available information collected in the baseline survey, including demographics and tool usage to account for potential selfselection of participants whose main job was not e-waste (hence, they were working elsewhere during the first field trip and could not be found) or other non-explicit reasons. The groups were balanced on all measures across participant groups except for age. Table 3 reports the CE regression results. The estimates of utilities for the different attributes for 3D CE participants and 2D CE participants are reported in Part A, columns 1 and 2, respectively. To study the statistical significance of the utility estimate differences, we report the coefficients of the product representation binary variable interacted with all Fig. 5. Benchmark blade packaging. Fig. 6. Optimal knife design (handle in side position, 9 inch blade, thick blade, guard). Table 2 Group summary statistics by CE design (3D and 2D). 3D CE (N =83 participants) 2D CE (N =15 participants) T-test p-value Age (yrs.) 47 (st.dev. 10) 40 (st.dev. 11) 0.018** Average monthly household income (kTHB) 10.6 (4.0) 9.2 (4.5) 0.32 Number of people per household (person) 4.7 (2.1) 4.8 (1.8) 0.89 Chi square pvalue Workers 35 (42%) 7 (47%) 0.75 Men 46 (55) 8 (53) 0.88 Education secondary or higher 33 (40) 6 (40) 0.99 E-waste as a main job 49 (59) 10 (67) 0.58 Tool usage Blade 60 (72) 11 (73) 0.46 Chisel 70 (84) 9 (90) 0.25 Knife 61 (73) 6 (75) 0.23 *p <0.1, **p <0.05, ***p <0.01, ****p <0.001. Table notes: Income was winsorized at the kTHB2 and kTHB15 levels, meaning income levels reported as below kTHB2 and above kTHB15 were counted as kTHB2 and kTHB15, due to multiple choice format of the question which asked participants to indicate their income bracket. Where there were missing responses, we calculated the statistics on the available data only. M.J. Coulentianos et al. Development Engineering 6 (2021) 100063 7 attributes, in Part B. of Column 1. The coefficients can be interpreted as utilities for each attribute level. 4.2. 3D CE The results of the 3D CE indicate that participants saw the most value in the handle in the side position (coefficient =1.3). Hence, if presented with two alternatives, a participant would choose a tool with the handle in the side position 79% of the time, with all other attributes being equal. A breakdown of probabilities for each attribute is given in Table 4. In order of importance, a thicker blade (utility =0.86, 3 mm compared to 0.8 mm), a longer blade (utility =0.61 for a 9-inch blade), a guard (utility =0.56), and finally a medium blade (utility =0.34 for a 7inch blade), were all attractive attributes for participants in the 3D CE. Price was not found to have a statistically significant effect on participants’ preferences. 4.3. Comparing 3D and 2D product representation in CE results Looking at the attribute valuation of the 2D CE, we found that the order of importance of attributes had changed as compared to the 3D CE results. While the handle in the side position was still the most heavily weighted attribute (utility =2.5), the guard came in as second most weighted attribute (coefficient =0.94). The 9-inch blade and 7-inch blade lengths followed with coefficients of 0.68 and 0.49 respectively. The attribute of blade thickness had the lowest coefficient before price (coefficient =0.15). Lastly, the utility for price, while not statistically significant, was negative. The difference in probabilities associated with each attribute between 3D and 2D CEs is reported in Table 4. For example, with all other attributes being equal, a participant of the 3D CE would pick the knife with a guard 64% of the time, while a participant of the 2D CE would pick the knife with the guard 72% of the time. Furthermore, we found statistically significant coefficients of interaction variables (Table 3, Part B), namely ‘2D * Handle position’ (p < 0.01) and ‘2D * Blade Thickness’ (p <0.01). These results implied that the product representation influenced the respondents’ valuation of attributes. For blade thickness, the negative coefficient −0.71 signified that the 2D product representation decreases the relative importance of a thick blade by a factor of 1/6. In the case of the handle position, the positive coefficient 1.2 signified that the 2D product representation increased the relative importance of the handle in the side position by a factor of 1.9. A visual representation of the attribute utilities is included in Fig. 7. 4.4. Sensitivity analysis To assess the sensitivity of our model, we conducted nearest neighbor matching to select 3D CE participants that most closely resembled the 2D CE participants, based on the following normalized baseline characteristics: Age (yrs.), Average monthly household income (kTHB), Number of people per household (person), Worker (binary), Gender (binary), Education secondary or higher (binary), E-waste as a main job (binary). We included the two nearest neighbors (NN) which resulted in a sample of 21 neighbors (because of overlap) from the 3D CE participant sample, which we name NN 3D. The regression results are included in Table 3 (Column 3). The demographic comparison of the NN 3D CE subset and the 2D CE participants is included in Appendix 2. While the two groups were balanced on all baseline measures across participant groups, the two groups still lacked complete overlap of age, as seen in Fig. 8. Hence, we ran the regression on a data subset where all participants under the age of 20 and over the age of 60 were removed. These results are also included in Table 3 (Column 4). The regressions ran to assess sensitivity of the results to changes in the datasets show that our main results, that participant preferences were different for the 2D CE versus the 3D CE, remained constant. 4.5. One-month post-auction preferences We evaluated the aggregated participant preferences one month after the auction experiment (Table 5). We found that participants who had used the tool had significantly different responses than those who Table 3 Regression results for the multinomial logit model with all data. A. Attributes Coefficients (utilities) Sensitivity analysis 3D (1) 2D (2) NN 3D (3) 21–59 year old (4) Handle in the side position 1.3**** (0.14) 2.5**** (0.34) 2.0**** (0.34) 1.3**** (0.15) Thick blade 0.86**** (0.14) 0.15 (0.17) 0.79*** (0.16) 0.84**** (0.16) Blade length 9 7 0.61**** (0.16) 0.34*** (0.12) 0.68*** (0.25) 0.49 (0.27) 0.55 (0.29) 0.30 (0.25) 0.57*** (0.18) 0.35** (0.13) Guard present 0.56**** (0.094) 0.94*** (0.30) 0.20 (0.16) 0.49**** (0.10) Price (continuous) 0.00022 (0.00039) −0.00062 (0.0013) 0.00019 (0.00096) 0.00022 (0.00045) B. Difference between 3D and 2D results 2D*Handle in the side position 1.2*** (0.37) 0.52 (0.49) 1.2*** (0.42) 2D*Thick Blade −0.71*** (0.22) −0.64* (0.29) −0.60* (0.24) 2D*Blade length 2D*9 2D*7 0.067 (0.29) 0.14 (0.29) 0.13 (0.38) 0.19 (0.37) 0.073 (0.32) 0.26 (0.32) 2D*Guard present 0.56 (0.32) 0.75* (0.34) 0.37 (0.34) 2D*Price −0.00084 (0.0014) −0.00081 (0.0017) −0.00084 (0.0017) Observations 1446 530 1,16 Cases 288 106 230 Respondents 98 (83 + 15) 36 (21 + 15) 82 (69 +13) Log-pseudo-likelihood −1152 −383.9 −926.1 *p <0.1, **p <0.05, ***p <0.01, ****p <0.001. Table notes: Part A reports the attribute utilities. Column 1 presents the results of the 3D CE; Column 2 presents the results of the 2D CE. Part B reports the interaction terms between the attributes and a 2D product representation dummy; hence these results are the differences between the 3D and 2D estimated utilities. A subset of participants did not complete the full three sets of rank ordering when conducting the 3D CE. Hence, the number of observations does not equal the expected 98*15 =1470 observations. Results are clustered at the participant level. Columns 3 and 4 present the results of the sensitivity analysis where the regression was re-run on two data subsets: Column 3 presents the results of the regression on the nearest neighbor matching of 21 3D CE participants to the 15 2D CE participants; Column 4 presents the results of the regression on the dataset where all participants below 20 and above 60 years old were removed. Table 4 Probabilities of choosing a tool with a specific attribute, all other attributes being equal, based on 3D CE results. (in %) 3D CE 2D CE Handle in the side position 79 92 Thick blade 70 54 Long versus short blade 65 66 Long versus medium blade 57 55 Medium versus short blade 59 62 Guard 64 72 M.J. Coulentianos et al. Development Engineering 6 (2021) 100063 8 had not used the tool for one of the three preference questions asked in the endline survey. Indeed, while we failed to reject the null hypothesis that preferences for blade length were different, we found that participants who had received the tool during the auction preferred a thin and long blade, while participants who had not received the tool preferred a thick and short blade at significantly higher rates. In addition, the preferred width of the blade was also statistically different between participants who had received the tool and those who had not. We anticipated such differences based on qualitative evaluation of a video of an e-waste worker (not part of the study sample) who tested the tools before they were distributed in the BDM experiment. We observed that the blade was not wide enough compared to the width of the motor which prevented the worker to cut the motor from a single side. Rather the worker had to turn to motor around to cut from both sides. Furthermore, the blade seemed very thick compared to the space where a blade is typically inserted in a motor for dismantling. 5. Discussion This paper investigated the methodological question of what product representation to use for CEs when eliciting participant preferences in an LMIC setting. Specifically, we investigated the differences in attribute valuation when 2D renderings were shown versus when 3D prototypes were shown. We found that the representation impacted participants’ relative weighting of the attributes significantly. We further found that price had no statistically significant effect on participants’ preferences. 5.1. H1: the weighting of the blade length and blade thickness attributes would decrease relative to the other attributes in the 2D CE The first hypothesis was supported in part by our findings. The weight of the blade thickness attribute did indeed decrease, but we failed to reject the null hypothesis that the attribute weights were different for blade length (both 9-inch and 7-inch blades). If a designer was deciding which attributes to include in the tool based on priorities, they would have designed different tools if they used 3D versus 2D prototype form: based on 3D CE results, the two attributes with the highest utility were the handle in side position and a thick blade; based on the 2D CE results, the two highest ranked attributes were handle in side position and the presence of a guard. The difference in relative importance of blade thickness may have been explained by the limitations of 2D renderings for conveying sizes (de Beer et al., 2009). Indeed, blade thickness was mainly displayed through the shading on the sharpened side of the blade, which might not have communicated the thickness appropriately. Instead, participants may have evaluated the size of the sharpened area rather than the thickness of the blade (as stated by three participants during 2D CE). Furthermore, the blade thickness specifications were given at the bottom of the rendering, but small measures such as 0.8 mm and 3 mm might have been harder to imagine than the larger measures associated with blade length (9-inch, 7-inch, 4-inch). Blade length differences might also have been more apparent because of the use of empty space in the renderings. Furthermore, the order of importance was different in the 2D CE compared to the 3D CE results for multiple attributes. For example, the Fig. 7. Attribute utilities and rankings. The statistical significance of attribute valuation differences between 2D and 3D CE results are indicated with black arrows. Fig. 8. Age distributions of all participants (2D and 3D CE) and of participants in the 2D CE. Table 5 Preference-related endline questions. Question Responses All participants (N = 105) Participants who received the tool (N =54) Participants who did not receive the tool (N =51) Chi-2 p value What tool would you prefer? Thin and long 45 (43%) 33 (61%) 13 (25%) p < 0.001**** Thick and short 42 (40) 21 (39) 38 (75) Which blade length would you prefer? Short 1 (1) 0 (0) 1 (2) p =0.771 Medium 51 (49) 32 (59) 20 (39) Long 27 (26) 15 (28) 12 (24) Extra-long 8 (8) 6 (11) 2 (4) How wide would you like the blade? Small 40 (38) 28 (52) 13 (25) p <0.01*** Medium 36 (34) 15 (28) 21 (41) Large 11 (10) 10 (19) 1 (2) *p <0.1, **p <0.05, ***p <0.01, ****p <0.001. M.J. Coulentianos et al.