The “CEO in context” technique revisited: A replication and extension of Hambrick and Quigley (2014)

Keller, Tobias; Glaum, Martin; Bausch, Andreas; Bunz, Thorsten Article — Published Version The “CEO in context” technique revisited: A replication and extension of Hambrick and Quigley (2014) Strategic Management Journal Provided in Cooperation with: John Wiley & Sons Suggested Citation: Keller, Tobias; Glaum, Martin; Bausch, Andreas; Bunz, Thorsten (2022) : The “CEO in context” technique revisited: A replication and extension of Hambrick and Quigley (2014), Strategic Management Journal, ISSN 1097-0266, John Wiley & Sons, Ltd., Chichester, UK, Vol. 44, Iss. 4, pp. 1111-1138, https://doi.org/10.1002/smj.3453 This Version is available at: https://hdl.handle.net/10419/287826 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by-nc-nd/4.0/ RESEARCH ARTICLE The “CEO in context”technique revisited: A replication and extension of Hambrick and Quigley (2014) Tobias Keller 1 | Martin Glaum 1 | Andreas Bausch 2 | Thorsten Bunz 2 1 WHU –Otto Beisheim School of Management, Vallendar, Germany 2 Justus-Liebig-Universität Giessen, Gießen, Germany Correspondence Martin Glaum, WHU –Otto Beisheim School of Management, Burgplatz 2, 56179 Vallendar, Germany. Email: [email protected];www. whu.edu/facct. Abstract Research Summary: Hambrick and Quigley's (2014) “CEO in context”(CiC) technique leads to a much larger CEO effect than traditional ANOVA or multilevel modeling. We replicate H&Q's study, apply their CiC technique to a much more comprehensive U.S. sample, and assess the sensitivity of the model findings to variations in method and data. We generally confirm H&Q's finding of a high CEO effect, but find a smaller industry effect and a larger firm effect in our much larger sample. Applying the CiC technique with adjusted R 2 s has only a moderate impact on year, industry, and firm effects, but markedly reduces the CEO effect. We also document that CiC model findings are sensitive to sample characteristics, namely firm size and CEO tenure. Managerial Summary: Hambrick and Quigley (2014) introduced a new method to analyze the influence of CEOs on firm performance. The study's empirical analysis focused on large U.S. firms. We replicate the original study and extend it to a much larger, comprehensive sample of U.S. firms that is composed of 33,996 firm-year observations, compared to 4,866 Received: 27 September 2015 Revised: 31 July 2022 Accepted: 18 August 2022 Published on: 19 September 2022 DOI: 10.1002/smj.3453 This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made. © 2022 The Authors. Strategic Management Journal published by John Wiley & Sons Ltd. Strat Mgmt J. 2023;44:1111–1138. wileyonlinelibrary.com/journal/smj 1111 firm-years in the original study. Controlling for the number of variables used in the estimations, the model attributes about a third of the total variance of firm performance (ROA) to the CEO. Further analyses show that the model findings differ for firms of different size and CEO tenure; the larger the firms and the longer the CEO tenures, the smaller tends to be the percentage of variance explained by the CEO. KEYWORDS CEO effect, CEOs, firm performance, managerial discretion, variance partitioning 1|INTRODUCTION The impact of top executives on organizational outcomes has been a topic of scientific inquiry for decades (e.g., Crossland & Hambrick, 2007; Hambrick & Finkelstein, 1987; Lieberson & O'Connor, 1972). Some theorists have argued that top executives have a considerable impact on organizational outcomes (Child, 1972; Hambrick & Mason, 1984). In contrast, others have suggested that the impact of executives is negligible because their actions are greatly constrained by external pressures (DiMaggio & Powell, 1983; Hannan & Freeman, 1977). Various scholars have attempted to analyze the influence of executives empirically by partitioning total variance in firm performance and attributing it to different levels: the industry, the firm, and, finally, the CEO. Earlier studies apply sequential ANOVA (e.g., Lieberson & O'Connor, 1972); later work mostly relies on multilevel modeling (e.g., Crossland & Hambrick, 2011). More recently, Hambrick and Quigley (2014; hereafter H&Q) proposed the “CEO in context,”or CiC, technique to more accurately contextualize the impact of CEOs on firm performance. A key criticism of customary variance partitioning methods is that the common use of industry and firm indicators (or grand averages on those levels) leads to misspecifications. To capture these contextual influences more accurately, H&Q introduce benchmark variables that aim to separate the effects of the CEO from those of the firm or industry. Based on data for large U.S. firms for the years 1992–2011, their study yields estimates for the CEO effect that are much higher than those of most previous studies using ANOVA or multilevel modeling. In this article, we replicate H&Q's (2014) original work; extend the study to a much larger, comprehensive sample of U.S. stock-listed firms; and analyze the sensitivity of the study's inferences to an important change in its method, the use of adjusted R 2 , and to two important sample characteristics, firm size and CEO tenure. Our work responds to a call for replication studies in strategic management (Bettis, Ethiraj, Gambardella, Helfat, & Mitchell, 2016; Hubbard, Vetter, & Little, 1998). Revisiting and testing the generalizability of extant findings is particularly relevant for CEO effect studies, for several reasons: the importance of top executives is one of the central questions in strategic management; the question continues to attract controversy (e.g., Fitza, 2014,2017; Quigley & Graffin, 2017); and empirical results vary with measurement techniques and samples. We focus on H&Q (2014) because of the study's importance. It 1112 KELLER ET AL. introduces a sophisticated and arguably superior technique to isolate the CEO's impact on firm performance, and it generates a high CEO effect estimate, which, according to the authors, is “much more in line with what would be expected from accepted theory about CEO influence on performance”(p. 475). We start our work by providing what Bettis, Helfat, and Shaver (2016) call a “narrow replication”of H&Q's study (also see Tsang & Kwan, 1999), using the same data and the same research design as H&Q (2014). Subsequently, we proceed with what Bettis et al. call “quasireplications”—that is, we assess the sensitivity and robustness of H&Q's findings to variations in the method and the data. More specifically, H&Q focused on a sample of large firms, “roughly the 1,500 largest U.S. corporations”(Hambrick & Quigley, 2014, p. 480). This raises the question of whether their findings also hold for a broader sample that is more representative of the economy as a whole—more specifically, a sample that also includes smaller firms, which tend to have a higher performance variance than larger firms (for a similar line of argument see Chang & Sing, 2000). Thus, following the guidelines developed by Bettis, Helfat, and Shaver (2016), we step by step enlarge H&Q's sample and analyze the effects on the model estimations. Our largest sample, which exploits as completely as possible the currently available data on CEOs of stock-listed firms in the U.S., comprises 33,996 firm-year observations, 1,983 unique firms, and 5,191 CEOs, as compared to H&Q's 4,866 firm-years, 315 firms, and 830 CEOs. Our sample also spans a longer time horizon (50 years vs. H&Q's 20 years), includes more industries, and reflects more diversity in firm size and profitability. In the second part of our investigation, building on contributions by Fitza (2014) and Quigley and Graffin (2017), we assess the sensitivity of the CiC model to the use of adjusted R 2 . Fitza (2014) argues that the commonly applied variance decomposition methods overstate the CEO effect because part of the variance in performance that is attributed to the CEO is effectively random. In a reply to Fitza (2014), Quigley and Graffin (2017) argue, inter alia, that the “random chance”element of variance decomposition can be accounted for by basing inferences on adjusted R 2 , instead of unadjusted R 2 . Thus, while H&Q (2014) introduced their CiC method with unadjusted R 2 s, in the second part of our empirical analysis we investigate in detail how using adjusted R 2 s affects the findings from their model. In the third part of our investigation, we examine the impact of firm size and CEO tenure on the CiC model findings, following H&Q's own suggestion to assess “when and where CEOs matter most (and least)”(Hambrick & Quigley, 2014, p. 488). We do this mainly by splitting our total sample into quartiles for each of the two characteristics and applying the CiC method to the resulting subsamples. This part of our analysis benefits from our comprehensive dataset, which includes a large and highly diverse set of firms, and a large number of CEOs, among them many CEOs with long tenures. Our findings can be summarized as follows. First, while we generally confirm H&Q's main finding of a high CEO effect, extending H&Q's original sample does affect the contextual effects of the CiC model. More specifically, H&Q's sample focused on the largest firms in the U.S. stock market. When we add data for smaller and more volatile firms, the CEO effect remains high in all steps of our analysis, but the industry effects are very small and the firm-specific effect becomes more important than in Hambrick and Quigley (2014). Second, applying H&Q's CiC model with adjusted R 2 s instead of unadjusted R 2 s does not have a strong impact on the estimates for the year, industry, and firm effects. In contrast, the CEO effect is markedly lower with adjusted R 2 s, because of the large number of CEO indicator variables that are introduced in the last stage of the CiC model. However, comparisons of year, KELLER ET AL.1113 industry, firm, and CEO effects across samples are not materially affected by the adjustment of R 2 s, because the adjustment leaves differences across samples largely intact. Third, we document that the CiC model findings are sensitive to sample characteristics, namely firm size and CEO tenure. More concretely, the CEO effect is larger, and both firm and industry effects are smaller, in smaller companies than in larger ones. As regards tenure, our investigation reveals that CEO tenure is negatively related to both the CEO and the firm effects. Some of the revealed associations between the sample characteristics and the model findings are conceptually founded and intuitive. Other associations are less intuitive and may be driven by technical aspects of the CiC method, for example, the size-weighting of the industry benchmarks. 2|STUDIES INVESTIGATING THE CEO EFFECT Over the past decades, numerous studies have attempted to empirically gauge the CEO's influence on firm performance, by using variance partition methods that attribute the total variance of a dependent variable (usually return on assets [ROA]) to factors such as the year, the industry, the firm, and the CEO. We provide an overview of the most important CEO effect studies in Table S1. 1 In an early paper, Lieberson and O'Connor (1972) find that the CEO accounts for 14.5% of the variance in performance. However, subsequent studies have generated very diverse findings, with CEO effect estimates ranging from about 2% (Bertrand & Schoar, 2003) to more than 40% (Weiner & Mahoney, 1981). Possible reasons for the diverse results could be the different time periods and samples of the studies, but also the different estimation methods (analysis of variance [ANOVA], maximum likelihood estimation, or multilevel modeling [MLM]). Hambrick and Quigley (2014) criticize another common aspect of traditional CEO effect studies, the use of simple indicator (“dummy”) variables to measure the contextual effects of industries and firms: Nominal indicators of context do not specify the pertinent, proximal conditions in which individual CEOs are located, which […] causes substantial blurring of contextual effects and CEO effects. The use of nominal predictors is especially problematic because it treats some of the CEO's own impact as part of the context in which he or she is operating, thus systematically underestimating overall CEO influence. (Hambrick & Quigley, 2014, p. 474). As we explain in detail below, H&Q instead introduce year-specific benchmarks for the performance of firms' industries and CEO-specific variables for firms' “inherited”health and performance. Applying their refined model to a sample of large U.S. firms for the years from 1992 to 2011 yields a CEO effect of 38.5%, considerably higher than their results from sequential ANOVA (16%) or MLM (20%). In the first part of our own empirical analysis, we replicate the original H&Q study and then investigate whether their findings also hold in a much broader sample that also includes medium-sized and smaller firms. 1 In a related stream of research, starting with Schmalensee (1985) and Rumelt (1991), studies using variance decomposition investigate the relative importance of industry, business line, and corporate effects. For an overview of earlier studies, see Bowman and Helfat (2001); for more recent studies, see McGahan and Porter (2002), Adner and Helfat (2003), Misangyi, Elms, Greckhamer, and Lepine (2006), and Guo (2017). 1114 KELLER ET AL. In a critical assessment of the CEO effect literature published in parallel to Hambrick and Quigley (2014), Fitza (2014) argues that part of the variance in performance that variance-decomposition studies attribute to the CEO is in fact random. As Fitza further points out, the CEO effect is the more inflated the shorter the CEOs' tenures, and the lower the number of individual CEOs in a sample. In a reply, Quigley and Graffin (2017) argue that the “random chance”element of variance decomposition can be controlled bybasinginferencesonadjustedR 2 , instead of unadjusted R 2 . 2 We take account of this methodological debate by carefully examining, in the second part of our empirical investigation, how using adjusted R 2 s affects the findings from H&Q's CiC model. Over the years, several authors have argued that researchers should not content themselves with the question of whether CEOs, in general, have an impact on firm performance, but should also investigate under which sets of conditions CEOs matter more, or less (e.g., Fitza, 2017; Hambrick & Quigley, 2014; Quigley & Graffin, 2017; Wasserman, Anand, & Nohria, 2010). For example, Lieberson and O'Connor (1972) and Wasserman et al. (2010) point out that the magnitude of the CEO effect differs across industries; Crossland and Hambrick (2007,2011) identify country differences; and the findings of Quigley and Hambrick (2015) suggest that the importance of CEOs has increased over time. In the third part of our own empirical investigation, we examine more closely how firm size and CEO tenure moderate the CiC model findings. We focus on these two factors because Hambrick and Quigley (2014) limited their original study to large U.S. firms and because tenure is closely connected to the CEO effect and plays an important role in the argumentation of Fitza (2014). 3|METHOD Our study replicates and extends H&Q's 2014 study. H&Q define a set of nested equations to analyze firm performance and to isolate the portion of its variance that is attributable to the CEO. As in most studies on the CEO effect, the performance variable is ROA, calculated as net income divided by total assets for each firm-year. Like previous researchers, in a first step H&Q use indicator variables for calendar years to capture macroeconomic effects [see Equation (1) below]. Previous studies also used indicator variables to measure the influence of the industry. It is a key insight of H&Q that this approach is imperfect because the firm itself contributes to the mean performance in its industry,andbecauseitassumes that the industry influence on performance is constant over the time horizon of the study. Therefore, H&Q replace the industry dummies with an industry benchmark, calculated as the sizeweighted mean ROA per industry in a given year, excluding the focal firm (and hence its CEO). 3 Similar arguments apply at the firm level. That is, just as the firm itself contributes to the mean performance in its industry, each CEO contributes to the firm's average performance across the study's data panel. In addition, H&Q point out that using firm indicator variables for the entire time span of a study not only assumes constancy of the firm effect but also implies that CEOs are assessed against an average firm performance that includes the years beyond their tenures (Hambrick & Quigley, 2014, p. 479). To address these issues, H&Q replace the firm dummies with two variables meant to account for the condition of the company at the start of each CEO's tenure: “inherited 2 More recently, Fitza (2017) suggests that CEO effect estimates in variance decomposition studies can be affected not only by random noise, but also by the autocorrelation of firm performance. 3 In accordance with H&Q, when calculating the industry benchmarks we include all firms in Compustat for which the necessary basic data (i.e., net income, total assets) are available. KELLER ET AL.1115 profitability,”proxied by the mean ROA for the 2 years preceding the CEO's tenure, and “inherited health,”measured as the company's market-to-book ratio (MTB) divided by the median MTB ratio of the industry excluding the focal firm, at the start of the CEO's tenure. In the fourth and last of the regression equations, in accord with previous studies, H&Q (2014) use indicator variables for individual CEOs to capture their aggregate effect. The following equations summarize Hambrick and Quigley's (2014) CiC model: ROAyiko =XDYEARy+ε1 yiko ð1Þ ROAyiko =XDYEARy+I–BENCHyik +ε2 yiko ð2Þ ROAyiko =XDYEARy+I–BENCHyik +INPROFko +INHEALTHko +ε3 yiko ð3Þ ε3 yiko =XDCEOo+ε4 yiko,ð4Þ where subscripts y,i,k, and oindicate the year, industry, firm, and CEO, respectively. The year effect is estimated as the R 2 from Equation (1). The industry and firm effects are estimated as the incremental R 2 sofEquations(2)and(3) respectively. The CEO effect is calculated in two steps 4 : first, by calculating the R 2 from Equation (4), which indicates how much of the remaining unexplained performance variance in Equation (3) is explained by the CEO indicator variables; and second, by multiplying this R 2 value by the percentage of unexplained variance from Equation (3), which yields the incremental total variance explained by the CEO indicator variables. 5 In the first part of our own analysis, we “narrowly”replicate and then extend H&Q's 2014 study by progressively enlarging their sample in five steps, which we describe in detail in Section 4. As we enlarge the sample, we compare year, industry, firm, and CEO effects across the steps. We use Fisher's z-transformation to estimate 95% confidence intervals for all effect estimates, and in our discussion, we focus on step-to-step changes that are clearly statistically distinguishable because the confidence intervals of the two effects do not overlap. 6 4 According to H&Q (2014, p. 482), “using the residuals from Equation (3) as the dependent variable rather than adding the CEO dummy variables as further explanatory variables to Equation (3), assures that the fixed effect coefficients for individual CEOs […] can be meaningfully interpreted as each CEO's net effect after completely controlling for contextual factors.”They further explain, “adding the CEO dummies to the full model generates the same amount of variance explained, or R-squared, but yields less stable estimates for the individual CEOs.” 5 We follow H&Q (2014) and winsorize continuous variables in all regression estimations at the top and bottom 2.5 percentiles. To take account of potential serial correlation in the panel data, H&Q use generalized estimating equations (GEE). Their specific GEE model assumes a normal distribution and a first-order autoregressive structure for the within-panel correlation (for details of GEE model estimation, see Hardin & Hilbe, 2013). In our own empirical investigation, we mainly use OLS estimation. The reason for this is that a major part of our analyses is concerned with the use of adjusted R 2 , and we are not aware of an established method to compute adjusted R 2 s for GEE. 6 We are grateful to one of the reviewers for suggesting the use of Fisher's z-transformation. Thus, following Quigley and Hambrick (2015), we interpret year, industry, firm, and CEO effects as partial R 2 s and use Fisher's z-transformation to estimate confidence intervals. We use the Stata command corrcii with the fisher option to calculate Fisher's z (see Cox, 2008). In line with general SMJ policy, we use the confidence intervals descriptively and not to identify specific cutoff levels of statistical significance. Focusing on confidence intervals that do not overlap is a conservative approach, because some differences in R 2 s across steps could be “statistically significant”in the traditional sense even when confidence intervals overlap (for details see Schenker & Gentleman, 2001). For detailed discussions on the estimation of confidence intervals of R 2 , see Olkin and Finn (1995) and Zou (2007). 1116 KELLER ET AL. In the second part of our analysis, building on the work of Fitza (2014) and Quigley and Graffin (2017), we investigate how using adjusted, rather than unadjusted, R 2 s affects the model's findings. In the third and final part of our analysis, we investigate how sensitive the H&Q model is to important sample characteristics, namely firm size and CEO tenure. We do this mainly by splitting our sample into quartiles and running the model separately for the resulting subsamples. We again use confidence intervals based on Fisher's z-transformation to assess differences between the quartiles. 4|DATA The starting point of our analysis is H&Q's data on CEOs of large stock-listed U.S. corporations for the years 1992 to 2011 from the Execucomp database, matched to key financial data for these firms from the Compustat database: 4,866 firm-year observations, pertaining to 830 CEOs and 315 unique firms in 44 industries (4-digit SIC), with at least four firms in each industry. 7 We follow H&Q (2014) and exclude financial institutions, public sector entities, and unclassified firms, as well as firms with only one CEO and CEOs who stayed only 1 year or less. 8 We then extend the sample in steps. In Step 1, we focus on the same firms as H&Q (2014) and use the longest time series for which we can find data in Execucomp and Compustat (1971–2019). In Step 2, we add data for all further available firms in the same industries as in H&Q (2014), and in Step 3 we extend the sample to additional industries, that is, we make full use of the available data in Execucomp and Compustat. 9 In Step 4, we broaden our sample further by adding data from the Refinitiv Eikon database (formerly ThompsonReuters Eikon), which provides both CEO data and financial data (i.e., it includes the formerly separate databases Datastream and Worldscope). At this stage, the sample has 29,318 firm-year observations and comprises data for 4,515 CEOs, 1,725 unique firms, and 164 industries. While we thus have markedly expanded our sample in comparison to H&Q (2014), we are still constrained by requiring at least four benchmark firms in each industry, measured at the four-digit SIC level, the finest available industry classification. 10 We could attain a still broader 7 We thank Tim Quigley for having made available to us the final dataset of H&Q's (2014) study as well as his Stata code for estimating the variance partitioning. 8 In line with H&Q (2014), we use the Execucomp variable “COPEROL”to identify the CEOs. 9 Here and in what follows, in order for firms to be included in our sample we require them to have nonnegative equity and at least US$ 1 m in assets. 10 A factor limiting the estimation of industry effects in variance decomposition studies in general is the quality of the data, that is, the firms’industry classification provided in databanks such as Compustat and Refinitiv Eikon. Like H&Q (2014) and previous researchers, we assign all firms in our samples to the industries of their respective primary product segments. However, the assignment of firms to industries can be complicated and may lead to errors in the databases (Jacobs & O'Neill, 2003; Kahle & Walkling, 1996). Moreover, firms’product portfolios change over time, so using static SIC industry affiliations, as is common in variance partitioning studies, is problematic, especially over longer horizons. Also problematic is the industry classification of multisegment firms, where the data providers assign “primary codes” to indicate the firms’major areas of activities. For all of these reasons, industry effect estimates from variance partitioning studies need to be interpreted with care. KELLER ET AL.1117 representation of firms and CEOs if we conducted the analysis on the broader three-digit SIC level instead. 11 Thus, in an intermediate step, we examine whether the CiC model estimations are sensitive to the granularity of the industry categorization by estimating the model for the threedigit SIC level, using the same firm-year observations as in Step 4. Finding that the estimation results are largely the same as those based on four-digit SIC industry benchmarks, in the fifth and last step we include all firms for which CEO and financial data are available and for which we can find at least four firms in the same industry, defined at SIC level 3. The resulting sample, the largest possible sample to which we can apply the H&Q (2014) CiC model, has 33,996 firm-year observations, with 5,191 individual CEOs for 1,983 firms in 140 SIC level-3 industries. 5|FINDINGS 5.1 |Part 1: Narrow replication of H&Q (2014) and stepwise sample extensions The findings of the first part of our analysis are summarized in Table 1. The first column of Table 1presents H&Q's (2014) original estimation. The next two columns present our narrow replication of their estimation, and the subsequent columns present the findings from quasireplications in which we extend our sample step by step. Panel A describes the samples; Panel B shows the estimation results for the performance variance components. We present point estimates for all year, industry, firm, and CEO effects, and we present confidence intervals based on Fisher's z-transformations for all OLS estimations. Panel C presents descriptive statistics on sales, total assets, ROA, and CEO tenure for the samples. We begin by replicating H&Q's original estimation as closely as possible, using H&Q's set of sample firms and method (GEE), but our own data and calculations (Stata code). As the results in column 2 of Table 1document, our estimation yields nearly the same results as those of H&Q—the industry effect is 7.22% (vs. 6.9% in H&Q [2014]), the firm effect is 11.51% (vs. 12.1%), and the CEO effect is 39.01% (vs. 38.5%). The minor differences between our point estimates and those of H&Q may be related to the industry benchmarks, that is, I–BENCHyik in Equations (2) and (3). To calculate these benchmarks, H&Q use all firms available in Compustat, and we follow their procedure. But it is likely that the set of firms covered by Compustat, and the industry affiliation of some firms, have changed over time. 12 In the second column, we provide the findings from OLS estimations with the same data, that is, the original H&Q sample. These findings are very similar to those with GEE. In fact, the firm effect is now slightly higher than with GEE, and the CEO effect slightly lower, bringing the estimates yet closer to the original H&Q (2014) results. Having thus established that there is 11 Weiner (2005) has shown that the degree of homogeneity of firms belonging to the same industry according to Compustat or Datastream Worldscope (now Refinitiv Eikon) is very similar for the four-digit and the three-digit SIC classification levels. Weiner argues that narrow industry classifications, such as the four-digit SIC level, lead to small populations, so that results can easily be biased through outliers. Broader industry definitions do not suffer as much from outliers but tend to become inhomogeneous. Examining this trade-off further, Weiner concludes that the threedigit SIC level yields the most accurate value predictions. For a more formal treatment of the trade-off between the efficiency gains from pooling and the possible bias resulting from heterogeneity see Wang, Zhang, and Paap (2019). 12 Using Tim Quigley's final dataset and his Stata code yields exactly the results reported by H&Q (2014). Furthermore, we also get exactly the same results when we use his dataset and our code. 1118 KELLER ET AL. be very small. 23 The same holds for the next step, which adds two proxy variables for firms’ inherited profitability and health: the impact of adjusting R 2 on the firm effect will also be small. The situation is different in Equation (4), where the residuals from Equation (3) are regressed on a set of (o−1) CEO dummy variables. Here, the number of variables is very high in relation to the sample sizes. Hence, adjusting R 2 will have a pronounced impact on the CEO effect. In Table 2and Figure 1, we present the CiC model findings for H&Q's original (2014) sample and our five sample extensions, using both unadjusted R 2 s (Panel A) and adjusted R 2 s (Panel B). 24 As shown in Table 1, we present point estimates and confidence intervals for the effects, using Fisher's z-transformation. Panel C of Table 2presents the differences between the point estimates. Figure 1graphically summarizes the main results from Table 2, presenting the confidence intervals for the CiC effect estimates based on unadjusted R 2 s next to those for estimates based on adjusted R 2 s. The four effects of the CiC model are presented in rows, with the unadjusted effect estimates on the left and the adjusted effect estimates on the right. When we compare the results based on adjusted R 2 s with those calculated with unadjusted R 2 s, the differences are consistent with our reasoning above. We focus first on the point estimates. The year effects based on adjusted R 2 s are somewhat lower than those based on unadjusted R 2 s, but the differences are small, at least in absolute terms. As one would expect, the differences between the adjusted and unadjusted year effects get smaller as we expand the sample in the later steps by adding more observations for an (almost) constant number of years. The estimates for industry and firm effects are very similar for unadjusted and adjusted R 2 s (mean differences across samples <0.1%). 25 In contrast, and predictably, the CEO effect based on adjusted R 2 s is markedly lower than the estimates based on unadjusted R 2 s—the estimates now range between 27.81% (Step 2) and 34.02% (Step 4), whereas those based on unadjusted R 2 are between 34.92% (Step 1) and 40.72% (Step 4). On average, across the H&Q (2014) sample and our sample extensions, the CEO effect estimates based on adjusted R 2 s are 6.99 percentage points lower than those based on unadjusted R 2 s. An important insight, particularly apparent in Figure 1, is that the R 2 adjustment leaves the differences between effect estimates across sample extensions (i.e., our five steps) largely intact: the estimates are effectively shifted downwards in parallel. Hence, whenever confidence intervals do not overlap between subsamples with unadjusted R 2 s, they also do not overlap for 23 One could argue that I-BENCH, while technically only one variable, is similar to a set of industry benchmarks, that is, industry-year dummy variables or industry-year ROA means. This is because excluding the focal firm from the yearly ROA mean leads to values that are firm-year-specific, but nonetheless very similar within a given industry-year. This holds especially in large industries, where the removal of one observation does not materially affect the mean. The more the ex-focal-firm benchmark correlates with the benchmark including the firm, the more it is comparable to a set of industry-year indicators. To the best of our knowledge, there is no guidance in the literature on how to adjust R 2 in such a context, and we therefore use the “standard”approach of considering I-BENCH as a single regressor. However, if IBENCH were interpreted as a set of indicators, introducing it into the regression equation would use up a higher number of degrees of freedom, and the adjustment of R 2 would have a greater impact on the industry effect. 24 The effect estimates based on unadjusted R 2 in Table 2, Panel A, are identical to those presented in Table 1.We include them here again to ease direct comparisons with the results based on adjusted R 2 s. 25 As appears in Panel C of Table 2, most of the differences between the industry and firm estimates based on unadjusted R 2 s and those based on adjusted R 2 s are negative. However, these differences are all very close to zero in absolute terms. Furthermore, the estimation of the industry and the firm effects follows the estimation of the year effects, and the R 2 - adjustment reduces the year effects more strongly, leaving more variance potentially to be explained in the subsequent stages of the CiC model. The resulting increase in the industry and firm effects appears to overcompensate for the very small adjustment of the partial R 2 s. KELLER ET AL.1125 FIGURE 1 Legend on next page. 1126 KELLER ET AL. subsamples with adjusted R 2 s. This is the case for the industry and firm effects in Steps 2 and 4, and for the CEO effect in Step 4. To sum up, CEO effect estimates are inflated when they are based on unadjusted R 2 s. Thus, in line with Quigley and Graffin (2017), the H&Q (2014) CiC model should be used only with adjusted R 2 s. As our analysis shows, doing so markedly lowers the CEO effect, but does not strongly alter the year, industry, and firm effects, nor the comparisons of effects across samples. 5.3 |Part 3: Sensitivity of H&Q (2014) model findings to sample characteristics In the third and last part of our analyses, using only adjusted R 2 s, we conduct additional tests to examine in detail the sensitivity of the H&Q (2014) CiC method to two sample characteristics, firm size, and CEO tenure. This analysis also responds to a call, expressed several times in the CEO effect literature, to investigate in more detail under what conditions and circumstances CEOs matter more, or less (e.g., Fitza, 2017; Hambrick & Quigley, 2014; Quigley & Graffin, 2017). We examine firm size because H&Q (2014) focused on large U.S. corporations, and in the stepwise sample enlargements in the first part of our empirical analyses we saw that adding smaller firms to H&Q's sample had a marked impact on the CiC estimations. And we investigate CEO tenure because it is closely connected to the central object of our investigation, the CEO effect, and because CEO tenure plays a central role in the argumentation of Fitza (2014). We conduct the following tests on our total sample, the one comprising 33,996 firm-year observations. 5.3.1 | Firm size To examine the impact of firm size more systematically, we divide our largest sample into quartiles, using sales as a size proxy. 26 Figure 2presents CiC estimation findings for the four quartiles; we also present the estimation results and descriptive statistics for the quartiles in Table S2. The graph documents that firm size has nontrivial implications for the CiC model effects. Most striking is the impact on the industry effect—the performance of small firms tends to be idiosyncratic, and consequently, in the first quartile there is practically no industry effect at all (and even the year effect is significantly smaller than in the other size classes). As firms become larger, their performance gets more similar to the performance of their respective industries; that is, the industry effect estimates increase monotonically from one size quartile to the next, FIGURE 1 Confidence intervals for year, industry, firm, and CEO effect estimates from the CiC model, based on OLS estimation and unadjusted R 2 s (left column) and adjusted R 2 s (right column), for the original H&Q (2014) sample and step-by-step sample extensions. The “handlebars”represent 95% confidence intervals, based on Fisher's z-transformations, for the effect estimates from the CiC model estimations, that is, the lower bar indicates the lower value of the confidence interval, and the upper bar indicates the upper value. The graphs use different scalings on the vertical axes. Descriptive statistics of the various samples are presented in Table 1 26 The results are qualitatively very similar if we use total assets instead. KELLER ET AL.1127 and in the fourth size quartile the industry effect explains a little more than 10 % of the firms' performance variance (10.83%). 27 A technical reason that helps to explain the pronounced impact of firm size on the industry effect is that H&Q use size-weighted ROAs as industry benchmarks. In this analysis, we split our sample according to size and then compare the firms in each quartile with the complete, “unsplit”industry benchmarks, that is, the entire population of firms in the same industry available in Compustat and Refinitiv Eikon. Obviously, the size-weighted benchmarks explain the performance of larger firms better than that of smaller firms. The quartile regressions presented in Figure 2thus bring to light a specific aspect of the FIGURE 2 CiC model estimations for quartile subsamples based on firm size (sales). The figure presents results from H&Q CiC model estimations for subsamples—quartiles based on firm size, as measured by sales revenues—of our largest sample (n=33,996). The estimation findings for the smallest firms are presented on the left, and the estimation findings for the largest firms are on the right. The bars represent the estimated year, industry, firm, and CEO effects as well as the remaining unexplained variance from the CiC model. All effects are calculated using OLS estimation and adjusted R 2 s. The estimation results and descriptive statistics for the various subsamples are summarized in Table S2 27 Here, and generally in the third part of our analyses, we focus on cross-quartile differences for which the confidence intervals derived from Fisher's z-transformations do not overlap. For example, the confidence intervals indicate that the industry effect estimates are clearly different from quartile to quartile. Analogously, the firm effects in quartiles one, two, and three are different from those in quartiles two, three, and four, respectively. And the CEO effect in the first quartile is different from that in the second. The CEO effects in the second and third quartiles are not different from each other, but the estimator in the third quartile is again different from that in the fourth. The confidence intervals for all effect estimates in all quartiles are reported in Table S2. 1128 KELLER ET AL. CiC method, because, in the “normal”application of the method in the total sample, smaller sample firms are also compared to size-weighted industry benchmarks. Figure 2further reveals that the firm effect is more pronounced for very small (11.58%) and very large firms (13.07%) and less pronounced for medium-sized ones (7.44% and 10.17%). It seems that inherited performance and health persist longer in small and large firms than in medium-sized firms. Finally, the CEO effect decreases monotonically as firm size increases. It is 35.75% for the smallest size quartile and only 25.79% for the largest quartile—10% points smaller in the largest quartile. It is intuitively obvious, and in line with previous studies (Finkelstein & Hambrick, 1990), that CEOs have relatively less discretion and thus less impact on firm performance as firms get larger. 28 5.3.2 | CEO tenure We present two sets of analyses to examine the impact of CEO tenure on the H&Q (2014) CiC model findings. First, we again split our sample into quartiles, this time by the firms' maximum CEO tenures. 29 The findings are summarized in Figure 3. 30 Second, to provide yet deeper insights we run a series of 24 CiC model estimations in which we successively increase the minimum tenure required for including a CEO in the samples. The “effect lines”resulting from these estimations are presented in Figure 4. The two figures document that CEO tenure profoundly influences the estimation results. Figure 3shows that in firms with shorter CEO tenures, that is, firms in the first and the second quartile, the industry effect does not explain firm performance at all. That is, the performance of these firms is quite idiosyncratic, dissimilar to that of their industry peers. 31 At the same time, firms with shorter CEO tenures have a pronounced firm effect. In fact, the firm effect is high in the first three quartiles, at 19.57% in the first quartile, 17.45% in the second, and 19.84% in the third quartile, before dropping to only 9.11% in the fourth quartile. Figure 3further indicates that the CEO effect declines monotonically with increasing CEO tenure. It is 34.68% in the first quartile and only 26.42% in the fourth quartile. 32 28 We also note that firm size is related to CEO tenure. The average tenure increases monotonically with size; mean (median) tenure is only 5.31 (4) years in the first size quartile and 7.17 (6) years in the fourth quartile (Table S2). The length of CEOs’tenures may itself be related to the strength of the CEO effect; we examine this relation below. More generally, we acknowledge that the univariate analysis in this part of our study does not allow us to isolate the “pure” impact of a given firm characteristic (e.g., size) on the CiC model effects, by holding all other factors constant. 29 As in the previous investigation on firm size, we split the sample into quartiles based on a firm characteristic (here, maximum tenure) and assign all observations for a given firm to the same quartile. Thus, the number of firms is roughly the same in all four quartiles. However, in the present investigation, the numbers of firm-year observations are of course higher in the quartiles with longer maximum tenures. 30 For an overview of estimation results and descriptive statistics, see Table S3. 31 Accordingly, the standard deviation of ROA is negatively related to CEO tenure; it is 46.9% in the first quartile and only 12.9% in the fourth quartile; see Table S3 for details; also see Brookman and Thistle (2009) on the relation between CEO tenure and firm characteristics. 32 These numbers gauge the importance of the CEO relative to the variance of ROA in the respective subsamples. In absolute terms, CEOs “matter”much more in the first quartile because the performance of firms in this quartile is much more heterogeneous. Among the firms in the first quartile, CEOs explain 36.36% of a standard deviation of ROA of 46.9% age points, whereas in the fourth quartile CEOs explain 26.34% of a standard deviation of ROA of only 12.9% points. KELLER ET AL.1129 The effect lines in Figure 4show in greater detail that year (brown line) and industry effect (turquoise line) estimates increase steadily with longer CEO tenures, albeit on low absolute levels. The firm effect (green line) and the CEO effect (red line) decrease as we increase the tenure requirement, until minimum tenures of 13 and 11 years, respectively. The negative association between CEO tenure and the firm effect may be at least partly due to a technical reason. The CiC model proxies for the firm effect, inherited profitability (INPROF ko ), and inherited health (INHEALTH ko ), are CEO-specific and get “updated”with each new CEO. While these proxies yield greater insight than the simple firm dummies used in traditional variance decomposition studies, relying on them implies that the firm itself affects performance over the entire course of the CEO's tenure solely through inheritance. Put differently, changes on the firm level that affect performance are attributed entirely to the CEO; the rest of the organization is assumed not to matter at all. It should also be noted that, in general, with increasing tenures, it becomes increasingly difficult to disentangle statistically the firm effect and CEO effect. Turning to the CEO effect, one might expect that CEOs with long tenures influence their firms' performance more profoundly. Figure 4indicates the opposite: the CiC model estimates of the CEO effect become smaller and smaller as we impose higher and higher minimum tenure restrictions. There are several possible explanations for this phenomenon. First, over time CEOs FIGURE 3 CiC model estimations for quartile subsamplesbasedonlengthofCEOtenure(maximumCEO tenures). The figure presents results from H&Q CiC model estimations for subsamples—quartiles based on the length of the CEOs' tenures, as measured by firms' maximum CEO tenures—of our largest sample (n=33,996). The estimation findings for firms with the shortest CEO tenures are presented on the left, and the findings for firms with the longest CEO tenures are on the right. The bars represent the estimated year, industry, firm, and CEO effects as well as the remaining unexplained variance from the CiC model. All effects are calculated using OLS estimation and unadjusted R 2 s. The estimation results and descriptive statistics for the various subsamples are summarized in Table S3 1130 KELLER ET AL. may lose some of their energy and become less adaptable to changing environments (e.g., Miller, 1991; on the dynamics of managerial capabilities, also see Adner & Helfat, 2003; Helfat & Martin, 2015). 33 Second, firms with longer tenures may operate in stable markets, in which firms may need (and allow) minimal CEO discretion (Hambrick & Abrahamson, 1995). 34 Third, the association between CEO tenure and the strength of the CEO effect may also be driven by other variables that are related to both tenure and the CEO effect—for example, firm size, which increases, and firm performance, which improves, over the CEO tenure quartiles FIGURE 4 CiC model effect lines, conditional on CEO minimum tenure. The figure presents estimation results from a series of 24 CiC model estimations in which we successively increase the minimum tenure that we require for the inclusion of the CEOs in the respective samples. The brown line represents the results for the year effects, the turquoise line the industry effect, the green line the firm effect, and the red line the CEO effect. The gray line at the top of the figure shows the remaining unexplained variance. All of the effect estimates are based on OLS estimations and adjusted R 2 s. The points on the very left of the lines are based on a minimum tenure of 2 years, that is, the same requirement as in our main analysis. As we move along the lines to the right, we increase the minimum tenure to 3, 4, and more years, and eliminate all CEOs with shorter tenures, until we reach a minimum tenure of 25 years. The thickness of the effect line indicates the size of the sample 33 Also see Simsek (2007) on the relation between CEO tenure and top management's risk-taking propensity. 34 Accordingly, we find that industry discretion scores from Hambrick and Abrahamson (1995) and Finkelstein, Hambrick, and Cannella (2009) decrease monotonically over the four quartiles. KELLER ET AL.1131 (Table S3). 35 Fourth, the use of CEO dummies assumes a constant CEO effect over the whole tenure. More precisely, the dummies measure a time-invariant difference in ROA for a given CEO compared to other, earlier, and later, CEOs at the helm of the same firm. But the ROA differential might not be constant. For example, during a crisis a CEO may incur write-offs and restructuring losses 36 ; if the turnaround is successful, earnings recover in later periods. Thus, a strong CEO action may lead, in the CiC model (as well as in other variance decomposition models) to a rather weak CEO effect. Given that the two most potent effects in the CiC model, the firm effect and the CEO effect, lose explanatory power with higher minimum tenures, it is clear that the unexplained residual variance must increase. In Figure 4, the residual variance (represented by the gray line) is 45.49% at the very left, that is, in our full sample with the two-year minimum tenure, and it increases to more than 60% with minimum tenures of 11 or more years. Finally, Figure 4reveals that the relations between CEO tenure and year, industry, firm, and CEO effects become less stable on the right-hand side of the graph, with sudden kinks and reversals. One reason is that the ever-stricter tenure requirements lead to changes in the industry composition of the sample, which in turn can provoke sudden changes in effect estimates. 37 Another reason is that the size of the sample decreases towards the right-hand side, and removing further firms from the sample thus can have a stronger impact on the overall result. 6|DISCUSSION In the following, we summarize and discuss the main findings from our empirical analyses. (1) Our replication generally confirms H&Q's main finding of a high CEO effect on firm performance. We confirm the high CEO effect in H&Q's original sample and in our stepwise sample extensions. We again document strong CEO effects, albeit shifted downwards to a somewhat lower absolute level, when we reestimate the CiC model using adjusted R 2 s (33.04% for the largest sample, see Table 2). And we also generally find pronounced CEO effects when we split our sample by firm size or maximum CEO tenure, still using adjusted R 2 s. The last results in our analyses suggest that a lower-bound estimate for the CEO effect, among U.S. CEOs with long tenures, may be 25% (Figures 3and 4). While lower than the 38.5% originally reported by H&Q, our estimates are corrected for the number of variables used in the estimation equations, and they are still larger than the CEO effects reported in most previous studies that used traditional ANOVA or MLM variance partitioning methods in most previous studies that used traditional ANOVA or MLM variance partitioning methods (see Table S1 in the online appendix). 35 CEO tenure may also vary across industries. Hence, the firms in the four quartiles may differ not only in CEO tenure, but also in industry composition and perhaps other related characteristics. 36 See, for example, Strong and Meyer (1987). Guan, Wright, and Leikam (2005) find evidence that suggests that incoming CEOs often actively depress earnings by engaging in “big bath”earnings management. 37 Our full sample comprises firms from 140 industries (SIC 3). When we impose minimum CEO tenure requirements, up to a minimum of 6 years the number of industries within the sample remains constant. With minimum tenures between 7 and 10 years, the sample still comprises 139 industries, that is, only one industry drops out. However, at minimum tenures of 12 (15) [18] years, only 124 (104) [81] industries are left, and the drop-out of several industries in each of these steps can have a marked impact on the estimates. It should be noted that the sample sizes, in terms of firm-year observations, remain relatively large even at higher minimum tenure levels. For example, at a minimum CEO tenure of 16 years the sample is made up of 5,001 firm-year observations, more than in H&Q's original (2014) study. 1132 KELLER ET AL. While the CiC method consistently generates high CEO estimates, the industry effect estimates in our sample extensions and in most of the sample splits are lower (and firm effect estimates are higher) than those found by H&Q (2014). 38 There may be several reasons for this. First, our total sample comprises not only large firms, as did the original H&Q (2014) sample, but also medium-sized and smaller firms, whose performance tends to be more idiosyncratic. With the addition of the smaller firms to the sample, the within-industry variance becomes larger relative to the between-industry variance, rendering the industry effects less useful. Second, the H&Q CiC model uses size-weighted industry means as benchmarks. This was justified in H&Q's original study, with its focus on the largest U.S. firms, but may not be appropriate for studies targeting samples that also include medium-sized and smaller firms. Third, unlike the simple industry indicator variables in traditional variance partitioning studies, the CiC model's industry benchmarks (I-BENCH) by construction exclude the focal company and, in this sense, generate “out-of-sample”estimates, for which it is much harder to achieve high R 2 scores than for pure within-sample explanation. The weak industry effect leaves most of the performance variance unexplained in Equation (2) of the CiC model, allowing stronger explanatory power for the firm-specific variables INPROF and INHEALTH in Equation (3). (2) The CEO effect is markedly lower with adjusted R 2 s. The reason is that Equation (4)of the CiC model includes a large number of CEO dummy variables—in H&Q's original study, 829. Given the sample size of 4,866 firm-year observations, there are only 5.87 observations for each additional regression parameter to be estimated. Similarly, in our largest sample, with 5,191 CEOs and a sample size of 33,996 firm-year observations, there are 6.55 observations for each additional parameter estimate. Rules of thumb in the econometrics literature typically require 20, 15, or at least 10 observations per predictor in linear regression estimations. 39 In other words, while both H&Q's sample and our own may appear large in absolute terms, they are actually not, in relation to the number of model parameters to be estimated. In technical terms, the number of degrees of freedom in the estimation of Equation (4) of the CiC model is quite low (the same holds for the estimation of the CEO effect in other variance decomposition studies), and as a result, the CEO effect is likely to be overestimated because the model artificially describes part of the random error in the data. Because of this “overfitting”the unadjusted R 2 s from the estimation do not reflect the model's “true”predictive ability in the general population. 40 To correct for the overfitting, in line with Quigley and Graffin (2017) the H&Q (2014) CiC model should be used with adjusted R 2 s. 41 38 In H&Q's (2014) sample, the industry effect was 6.9%. In our largest sample it is only 0.37%. Earlier CEO studies, as well as studies focusing on industry, parent company, and business unit effects (e.g., Bowman & Helfat, 2001; McGahan & Porter, 2002; Short, Ketchen, Palmer, & Hult, 2007), mostly arrive at industry effects between 10% and 20%. 39 See, for example, Harrell (2015, pp. 72–74). In a more detailed analysis, Green (1991) suggests that regression analysis should be based, at a minimum, on a base sample of 50 observations plus eight additional observations per explanatory variable. Also see Austin and Steyerberg (2015) and Jenkins and Quintana-Ascencio (2020). 40 At first glance, one could perhaps argue that our dataset comes close enough to the general population of CEOs of U.S. stock-listed companies during our sample period so that there might be no need to make inferences; instead, it might suffice to interpret the observed effects. However, such an argument would miss the point that, even if we could indeed observe all CEOs of all U.S. firms during a given sample period, the unadjusted R 2 s from estimations of the CiC model, while reflecting statistical associations within this sample, will always be inflated by random noise and thus cannot be interpreted as measuring the effects of CEOs on firm performance. Furthermore, the observed unadjusted R 2 s are not predictive for new data generated by firms in years after the sample period, or for firms and CEOs in countries outside the U.S. Thus, it may be useful to interpret our sample as being drawn from a hypothetical infinite “superpopulation”(see Lavrakas, 2008). 41 Alternatively, one could assess the model fit in the population also by using out-of-sample prediction or cross validation; see, for example, Harrell (2015). KELLER ET AL.1133 (3) The CiC model findings are sensitive to the sample characteristics firm size and CEO tenure. Some of the associations between these characteristics and the model results are conceptually founded and intuitively plausible. For example, CEO effects are larger, and firm effects are smaller, in smaller (and more volatile) companies. In other cases, our analyses have revealed associations that are not immediately obvious. One example is the link between firm size and the strength of the industry effect, a link that, inter alia, likely reflects the use of sizeweighted industry means as benchmarks. Another example is the negative relation between CEO tenure and the strength of the CEO effect. One could perhaps argue that longer CEO tenures allow for a more precise estimation and that the lower CEO effect estimates in samples with longer CEO tenures are therefore more accurate. On the other hand, the median CEO tenure in our most comprehensive sample is 5 years. Restricting the sample to long tenures would thus ignore a large part of the population and would potentially bias the findings, to the extent that short CEO tenures indeed differ systematically from longer ones. As we explain above, focusing on long CEO tenures also raises another issue, the assumed constancy of the estimated effects. The CiC method generates average, “fixed”effects; in this sense, like most other variance partitioning models, it is “cross-sectional in design and static in its logic”(Henderson, Miller, & Hambrick, 2006, p. 447). However, research suggests that CEOs arrive in their positions well equipped and learn even more in the early phases of their tenures, but later become “tired, enshrined and stale”(Miller, 1991, p. 41). Moreover, the speed of this obsolescence depends on the dynamism of the industry (Henderson et al., 2006). The CiC model and most other variance decomposition models mask these temporal effects, as well as possible industry differences. 42 7|CONCLUSION Using a refined and arguably superior method to partition firm performance variance and to isolate the CEO's impact on firm performance, and using data for large U.S. firms for the years 1992–2011, Hambrick and Quigley (2014) estimated the CEO effect at 38.5%, much higher than most previous estimates based on traditional ANOVA or multilevel modeling. In the present paper, we replicate H&Q's study, apply their CiC technique to a much more comprehensive U. S. sample, and assess the sensitivity and robustness of their findings to variations in the method and the data. We generally confirm H&Q's finding of a high CEO effect, but find a smaller industry effect and a larger firm effect in our larger sample. We show that applying the CiC technique using adjusted R 2 s changes year, industry, and firm effects only moderately, but markedly reduces the CEO effect. And we document that the CiC technique is sensitive to sample characteristics, namely firm size and CEO tenure. While our replication study speaks directly to the validity and generalizability of H&Q's (2014) argument, we also more generally add to the discourse on CEO effects and the importance of top managers, and we hope that our findings will provide impulses for future research. For example, further research could revisit the question “when and where CEOs matter most 42 Guo (2017) employs longitudinal multilevel modeling to analyze stable and dynamic parts of performance variance, albeit for business unit, corporation, industry, and year effects, not for CEO effects. 1134 KELLER ET AL.