The Taguchi approach to large-scale experimental designs: A powerful and efficient tool for advancing marketing theory and practice

Moffett, Jordan W.; Fennell, Patrick; Harmeling, Colleen M.; Sheehan, Daniel; Bleier, Alexander Article — Published Version The Taguchi approach to large-scale experimental designs: A powerful and efficient tool for advancing marketing theory and practice Journal of the Academy of Marketing Science Provided in Cooperation with: Springer Nature Suggested Citation: Moffett, Jordan W.; Fennell, Patrick; Harmeling, Colleen M.; Sheehan, Daniel; Bleier, Alexander (2024) : The Taguchi approach to large-scale experimental designs: A powerful and efficient tool for advancing marketing theory and practice, Journal of the Academy of Marketing Science, ISSN 1552-7824, Springer US, New York, NY, Vol. 53, Iss. 3, pp. 949-954, https://doi.org/10.1007/s11747-024-01059-0 This Version is available at: https://hdl.handle.net/10419/323655 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by/4.0/ Vol.:(0123456789) Journal of the Academy of Marketing Science (2025) 53:949–954 https://doi.org/10.1007/s11747-024-01059-0 METHODOLOGICAL PAPER The Taguchi approach tolarge‑scale experimental designs: Apowerful andefficient tool foradvancing marketing theory andpractice JordanW.Moffett1· PatrickFennell2· ColleenM.Harmeling3 · DanielSheehan1· AlexanderBleier4 Received: 1 January 2024 / Accepted: 27 September 2024 / Published online: 5 November 2024 © The Authors 2024, corrected publication 2024 Abstract Current research often relies on narrowly focused experimental methods that address just a few independent variables or correlational designs, despite calls for future research to take big-picture perspectives that offer real-world applicability and causal evidence. This disparity likely reflects the constraints imposed by the need for extensive resources to conduct broad, causal examinations. To bridge this gap, the current article presents the Taguchi approach to large-scale experimental design, which remains notably underutilized in marketing research despite being well-established in other fields. Its effectiveness stems from the robust catalog of experimental design rubrics that can incorporate many different independent variables systematically and efficiently. The causal and efficient experimental option for broad scopes of investigation embraces the embeddedness of independent variables and thus can help build marketing theory and advance practice. This article details the fundamentals of the Taguchi approach, its relative advantages, and a three-step implementation process. Keywords Experimental design· Hypothesis testing· Taguchi methodology· Fractional factorial design· Marketing theory The modern marketing landscape is complex. Yet traditional academic research designs tend to abstract reality to pragmatically reduce its complexity. For example, experiments offer strong evidence of causality, but 98% of experimental studies published in top marketing journals in 2023 manipulated only one to three independent variables.1 This narrow focus offers high internal validity to test a limited set of hypotheses but often fails to account for the complex web of factors influencing real-world outcomes, reducing external validity. Further, sterile lab settings require omitting or holding constant influential variables, which risks over- or under-estimating focal effects occurring outside of experimental vacuums and obscuring recommendations for consumers, firms, and society. Thus, current scientific conventions and overreliance on narrow research scopes may hinder theoretical and practical advancements (Kindermann etal., 2024). As a complementary method, we present the Taguchi approach—a causal and efficient experimental option for broad scopes of investigation that embraces the embeddedness of independent variables in real-world environments. It allows researchers to pragmatically and precisely study complex concepts while establishing causality, a feat often challenging to accomplish with traditional techniques. Jordan W. Moffett and Patrick Fennell contributed equally. Mark Houston served as editor for this article. * Colleen M. Harmeling [email protected] Jordan W. Moffett [email protected] Patrick Fennell [email protected] Daniel Sheehan [email protected] Alexander Bleier [email protected] 1 Gatton College ofBusiness andEconomics, University ofKentucky, Lexington, KY40506, USA 2 Coles College ofBusiness, Kennesaw State University, Kennesaw, GA30144, USA 3 Dr. Persis E. Rockwood School ofMarketing, College ofBusiness, Florida State University, Tallahassee, FL32306, USA 4 Frankfurt School ofFinance & Management, 60322FrankfurtamMain, Germany 1 We reviewed research published in the Journal of Consumer Research, Journal of Marketing, Journal of Marketing Research, Journal of the Academy of Marketing Science, and Marketing Science. We also found that 97% of experimental studies used singlefactor or full-factorial designs and 3% used a conjoint or fractional factorial approach. 950 Journal of the Academy of Marketing Science (2025) 53:949–954 Panel A: Panel B: Stimuli Number IndependentVariables and Manipulation Levels – Stimulus #1 Stimulus #12 Panel C: – Roy (1990) and Minitab’s interactive design tool walks researchers through adapting Taguchi’s catalog designs to Level1 Level2 A Absent Present B SolidPatterned C RectangleCircle D OrganicNatural E Passivevoice Active voice F Absent Present G 4oz. 11 oz. H Absent Present I Absent Present J Absent Present K Absent Present Manipulations Independent Variables Stimuli Number IndependentVariables and Manipulation Levels Fig. 1 Product package design illustrative example 951Journal of the Academy of Marketing Science (2025) 53:949–954 Fundamentals andadvantages oftheTaguchi approach inmarketing research The Taguchi approach originated in engineering, where it was developed to isolate and test independent variables within highly complex systems and ultimately identify optimal configurations while ensuring resource efficiency (Taguchi, 1986). Agriculture and medical research also use this approach to tackle questions related to product quality control, variety selection, and process optimization. Despite its suitability for broad-scope investigations, Taguchi-designed studies have only recently emerged in marketing (e.g., Bleier etal., 2019). The power of Taguchi’s approach lies in its unique, accessible, and robust catalog of experimental design rubrics, which serve as templates for researchers. These rubrics specify the minimum required number and specific configurations of experimental conditions needed to estimate the main effects of numerous independent variables on a dependent variable, requiring only a fraction of the conditions needed in full-factorial designs. The rubrics are fractional factorial designs that prescribe the manipulation levels for each independent variable for each condition and are orthogonally balanced so that (1) each manipulation level of each independent variable and (2) each possible ordered pair of independent variables at each level appear the same number of times in the overall design. This carefully constructed balance enables efficient and reliable estimations of main effects and a select number of interaction effects while reducing impacts of non-theorized interaction effects (Roy, 1990). This approach can be applied to various contexts (e.g., social media communications, sales training, loyalty programs, food consumption).2 As an illustration, consider product packaging design a marketing problem with 11 coexisting independent variables (e.g., logo shape, size) that might affect a dependent variable, such as purchase intentions (Fig.1, Panel A). With this example, we discuss four advantages of the Taguchi approach, highlighting benefits over other methods (e.g., full-factorial experiments, conjoint analyses, observational data analyses, meta-analyses). First, the Taguchi approach offers significant resource savings compared with other approaches. Full-factorial experiments are best suited for investigating the effects of a small set of independent variables; Taguchi-designed studies are especially adept for broad investigations due to their reduced number of experimental conditions. In our example, Taguchi’s catalog specifies that the main effects of 11 package design elements manipulated at two levels can be investigated with 12 experimental conditions (e.g., L12 design rubric; Fig.1, Panel B). This requires access to roughly 2,400 participants, equating to costs of around $6,000.3 In contrast, a full-factorial, between-subjects experiment would require 2,048 experimental conditions (211), access to over 400,000 participants—2.5 times the number available on leading platforms—and a budget exceeding $1,000,000. Because each participant is only exposed to one stimulus, a Taguchi-designed study is also economical in terms of participant attention and stamina, especially relative to traditional conjoint approaches that use within-subjects designs and expose participants to multiple stimuli (e.g., 12 product package designs). Current recommendations on conjoint studies even limit the number of independent variables explored to six or seven. Thus, the resource efficiencies of the Taguchi approach allow theory and intuition to drive the selection of variables with reduced biases related to participant practice and fatigue, enabling investigative scopes that were previously difficult to achieve without compromising causality. Second, the Taguchi approach enhances researchers’ abilities to demonstrate causality by testing maximally diverse, experimentally manipulated configurations with random participant assignment. In contrast, secondary data sets are limited to observable variables collected in the past, which are further constrained by the norms and conventions of the specific context (e.g., packaging norms within a particular industry or firm and what the firm deemed worthy of tracking). Therefore, the configuration diversity achievable through the Taguchi approach can help researchers uncover relationships that might remain hidden in secondary datasets due to these constraints. Taguchi-designed studies are thus adept at modeling real-world complexity. Manipulating co-occurring drivers of the dependent variable, rather than holding them constant across conditions (as is the case in traditional experiments), may provide more accurate estimates of each independent variable’s contribution to variance in the dependent variable. Taguchi-designed studies meet the stringent conditions required to establish causality and support the investigation of the impacts of many independent variables in tandem, rather than in isolation through single or sequential narrow, full-factorial studies. This allows researchers to substantiate broad constructs (e.g., package design experience) while isolating the independent variables that drive observed changes (e.g., package design elements). Third, researchers can causally investigate select, theorized two-way interactions without generating non-theorized two-way or higher-order interaction effects. Taguchi’s 2 For additional examples of Taguchi-designed experiments, see https:// tinyu rl. com/ bdff4 f5c. 3 The sample size is based on 200 participants per experimental condition, with costs at $2 per participant plus a 25% Prolific or 40% MTurk fee. 952 Journal of the Academy of Marketing Science (2025) 53:949–954 designs are carefully balanced so that the impacts of nontheorized interaction effects (e.g., 3-way, 5-way, etc.) are uniformly distributed across all conditions. The theorized main and interaction effects thus can be clearly identified and estimated. Considering Taguchi’s catalog,4 researchers can explore up to 20 of the 55 possible two-way interactions among the 11 package design elements in our example, using anywhere from 16 to 32 experimental conditions (e.g., L16 catalog design allows up to 4 interactions, and L32 allows up to 20). A full-factorial design of this size would generate 55 two-way interaction effects and 1,981 higherorder interactions. These, typically not theorized, higherorder interactions can alter and complicate the interpretation of the main effects. Further, traditional conjoint studies do not typically accommodate any interactions, because they require both more sophisticated modeling techniques and more data for estimations. Fourth, with the Taguchi approach, researchers can consolidate thought while being forward-looking. Designing novel broad-scope investigations is feasible using the Taguchi approach, considering its manageable data collection and respondent requirements. This grants researchers the opportunity to consolidate diverse theoretical perspectives to build theory using previously unconsidered independent variables and relationships. Although meta-analyses can consolidate archival thought, they require mature domains with sufficient previously published research. Alternatively, the Taguchi approach can be applied in emerging and novel domains because it entails new data collection. For our product packaging example, researchers could investigate different sets of independent variables that various theories prioritize, like cognitive or affective information processing (e.g., a founder narrative, active versus passive description) or sensing (e.g., graphics, logo shape), in a single study. They also could investigate independent variables that have not been collected or tested before (e.g., the presence of a QR code). Implementing theTaguchi approach toexperimental design The Taguchi approach to experimental design can be implemented in the lab or the field, with a systematic three-step process. Before beginning this process, researchers will identify their dependent variable(s) of interest along with a potentially large set of independent variables and their manipulation levels by reviewing relevant theory and practice, as well as qualitative research (e.g., expert or customer interviews). The independent variables must be nonnested and controllable so that they can be manipulated independently of one another in the experiment (e.g., graphic colorfulness would be nested within the presence [versus absence] of graphics). Step 1: Select theappropriate Taguchi experimental design rubric Considering the number of independent variables identified for investigation (e.g., 11) and their manipulation levels (e.g., 2), researchers select an experimental design rubric from Taguchi’s catalog. This catalog features various rubrics that accommodate anywhere from 4 (e.g., L4) to 26 (e.g., L54) different independent variables, each with 2–8 manipulation levels, thereby supporting a wide variety of research contexts across different domains. These rubrics specify the number of required stimuli (e.g., product package configurations). In our product packaging scenario, the L12 design allows the simultaneous investigation of the 11 package design elements at two different manipulation levels with only 12 experimental stimuli. Further, these rubrics tell researchers how to configure each stimulus, indicating the manipulation level for each independent variable in each stimulus (Fig.1, Panel B). These rubrics are orthogonally balanced, meaning that each manipulation level of each package design element and each possible ordered pair of elements at each level appears the same number of times in the overall design. For instance, logo shape Level 1 (rectangle) appears 6 times, and Level 2 (circle) also appears 6 times. Logo shape Level 1 (rectangle) with background Level 2 (patterned) appears 6 times, and the reverse appears 6 times (circle logo shape with solid background). Tools for design selection are available in the Minitab statistical software, JMP by SAS, or XLSTAT.5 Researchers can also investigate a limited set of two-way interaction effects between the manipulated variables, with careful consideration of the independent variables and their column assignments in the rubric (Roy, 1990; see Minitab’s interactive design tool). In our example, the theory could predict interactions between the background (e.g., solid, patterned) and graphics (e.g., present, absent), as well as the background and logo shape (e.g., rectangle, circle). With the L16 rubric, we can investigate the main effects of the 11 package design elements and these two interaction effects with 16 experimental conditions (Compare Fig.1, Panel B and Panel C). 4 You can find Taguchi’s catalog on Minitab’s website: https:// www. minit ab. com. 5 https:// www. minit ab. com; https:// www. jmp. com; https:// www. xlstat. com 953Journal of the Academy of Marketing Science (2025) 53:949–954 Step 2: Execute andanalyze theTaguchi‑designed experiment After selecting the appropriate design rubric, stimuli are designed and produced accordingly (e.g., 12 stimuli; Fig.1, Panel B), then randomly presented to participants in a between-subjects design. After the experiment, analyses of variance can provide an initial determination of the relative impact of each independent variable on the dependent variable (Roy, 1990). In our example, we determine the relative importance of each of the 11 package design variables for increasing purchase intentions. Researchers might also perform Tukey’s honestly significant difference post hoc test for multiple comparisons. Researchers can apply other analysis methods, depending on their conceptual models (e.g., structural equation modeling to test parallel mediation or second-stage moderation; Bleier etal., 2019). Step 3: Follow upwithaverification study Taguchi-designed experiments provide comprehensive, variable-level examinations of a phenomenon but should be verified with a single-factor configuration-level study (Roy, 1990). Findings from the Taguchi-designed study will identify which independent variables significantly affect the dependent variable and the direction of these effects. Researchers should then validate these results by comparing a stimulus configuration containing one or all the identified influential variables at their optimal levels (i.e., optimal condition) against a baseline configuration with different levels (i.e., suboptimal condition). Results that yield similar and predicted changes in the dependent variable corroborate the conclusions of the Taguchi study. For example, consider that our illustrative Taguchidesigned study revealed that a circle logo shape (Level 2), small package size (Level 1), organic origin claim (Level 1), and QR code (Level 2) are the significant variables and manipulation levels that produce the greatest purchase intentions. The optimal configuration condition would feature a stimulus with these variables and levels. The comparison alternative would be any suboptimal configuration, which researchers can select according to their goals and context. Researchers could use a stimulus that mimics current practices or a stimulus with the same four significant variables but at their sub-optimum levels: a rectangle logo shape (Level 1), large package size (Level 2), natural origin claim (Level 2), and no QR code (Level 1). Non-significant variables from the Taguchi study could be held constant. This simple design validates the Taguchi study findings and provides additional confidence that the results are not affected by unanticipated relationships among the independent variables. Researchers can also use additional studies to further understand the phenomenon (e.g., explore mediators and moderators, extend generalizability to other situations and contexts). Extension oftheTaguchi approach toexperimental design: Signal‑to‑noise ratios Due to its origins in engineering and emphasis on firm-con- trollable factors, Taguchi-designed experiments can assess and minimize sensitivity to variations in noise. Noise factors are difficult or expensive to control in practice but can be experimentally manipulated (e.g., shelf placement, customers’ perceived time pressure while shopping) and are expected to influence the dependent variable of interest. In the social sciences, many factors can be controlled, and between-subjects designs with participant randomization are often used; since variation and noise should be randomly distributed across conditions, including them may not be necessary. Still, some researchers may include noise factors in their experimental designs to ensure the findings are robust across factors beyond their research scope or to address interaction effects between the focal independent variables and noise factors. Researchers then estimate signal-to-noise ratios, which indicate the relative effects of independent variables across varying levels of noise. Stronger effects in the presence of noise suggest more robust effects. For example, researchers might aim to design the most effective product packaging to stimulate purchase intentions regardless of retail shelf placement. By conducting a Taguchi-designed study in the lab or the field multiple times across different shelf locations, robust effects can be identified. Conclusion andimplications In considering the role of science in everyday life, it is important to recognize the colloquial use of the evidence presented in research. Such evidence is frequently construed as objective facts and reflections of reality that demand attention, warrant consideration, and ultimately catalyze change. Yet, the vacuum that science often creates out of necessity in the pursuit of causal evidence stands in stark contrast to the complex, multifaceted realities influencing any given outcome. This discrepancy can lead to errors when applying findings. The Taguchi approach is a powerful experimental tool that pragmatically emulates real-world complexity in its broad experimental designs. By integrating diverse, theoretically-relevant independent variables— whether previously identified in literature or practice, or novel variables yet to be collected or studied—the Taguchi approach not only has the potential to consolidate existing perspectives but also to introduce new ones. Its benefits, 954 Journal of the Academy of Marketing Science (2025) 53:949–954 however, go beyond pragmatism and consolidation. Narrow scopes of investigation often accompany a single or limited set of hypotheses, which can lead to researcher attachment, an endowment effect, or confirmation bias—all potential threats to the field. In defense of rigorous scientific methods, Platt (1964, p. 350) offers a clear antidote: the “method of multiple hypotheses …[which] differs from the simple working hypothesis in that it distributes the effort and divides the affections.” The Taguchi approach supports this by encouraging the friendly co-existence of multiple competing hypotheses and providing a feasible means to test them simultaneously. It thereby serves as a powerful tool for facilitating scientific discovery and advancement. Acknowledgements We want to acknowledge and thank Dr. Natalie Chisam for her friendly reviews and feedback. Authors' contributions All authors contributed to the conceptualization. Funding Not applicable. Data availability Not applicable. Declarations Ethical approval Not applicable. Competing interest Not applicable. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. References Bleier, A., Harmeling, C. M., & Palmatier, R. W. (2019). Creating effective online customer experiences. Journal of Marketing, 83(2), 98–119. Kindermann, B., Wentzel, D., Antons, D., & Salge, T. O. (2024). Conceptual contributions in marketing scholarship: Patterns, mechanisms, and rebalancing options. Journal of Marketing, 88(3), 29–49. Platt, J. R. (1964). Strong inference. Science, 146(3642), 347–353. Roy, R. (1990). A primer on the Taguchi method. Society of Manufacturing Engineers. Taguchi, G. (1986). Introduction to quality engineering: designing quality into products and processes. Asian Productivity Organization. Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.