The Pre-Eminence of Theory versus the European CVAR Perspective in Macroeconometric Modeling

Spanos, Aris Article The Pre-Eminence of Theory versus the European CVAR Perspective in Macroeconometric Modeling Economics: The Open-Access, Open-Assessment E-Journal Provided in Cooperation with: Kiel Institute for the World Economy – Leibniz Center for Research on Global Economic Challenges Suggested Citation: Spanos, Aris (2009) : The Pre-Eminence of Theory versus the European CVAR Perspective in Macroeconometric Modeling, Economics: The Open-Access, Open-Assessment EJournal, ISSN 1864-6042, Kiel Institute for the World Economy (IfW), Kiel, Vol. 3, Iss. 2009-10, pp. 1-14, https://doi.org/10.5018/economics-ejournal.ja.2009-10 This Version is available at: https://hdl.handle.net/10419/27530 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by-nc/2.0/de/deed.en Vol. 3, 2009-10| April 7, 2009 | http://www.economics-ejournal.org/economics/journalarticles/2009-10 The Pre-Eminence of Theory versus the European CVAR Perspective in Macroeconometric Modeling Aris Spanos Virginia Tech, USA Abstract The primary aim of the paper is to place current methodological discussions in macroeconometric modeling contrasting the ‘theory first’ versus the ‘data first’ perspectives in the context of a broader methodological framework with a view to constructively appraise them. In particular, the paper focuses on Colander’s argument in his paper “Economists, Incentives, Judgement, and the European CVAR Approach to Macroeconometrics” contrasting two different perspectives in Europe and the US that are currently dominating empirical macroeconometric modeling and delves deeper into their methodological/philosophical underpinnings. It is argued that the key to establishing a constructive dialogue between them is provided by a better understanding of the role of data in modern statistical inference, and how that relates to the centuries old issue of the realisticness of economic theories. Special issue “Using Econometrics for Assessing Economic Models” JEL: B4, C1, C3 Keywords: Econometric methodology; ‘general-to-specific’; pre-eminence of theory; cointegrated VAR; statistical adequacy; realisticness of a theory; statistical model; actual versus nominal error probabilities Correspondence Aris Spanos, Department of Economics, Virginia Tech, 3019 Pamplin Hall, Blacksburg VA, 24061, USA; email: [email protected] Many thanks are due to Kevin Hoover and Katarina Juselius for several valuable comments and suggestions. © Author(s) 2009. Licensed under a Creative Commons License - Attribution-NonCommercial 2.0 Germany Economics: The Open-Access, Open-Assessment E-Journal 1 1 Introduction Colander (2009) (this volume) compares and contrasts two alternative perspectives in empirical macroeconomics, and attempts to explain the extent of their in‡uence on the discipline in terms of the incentive scheme perpetrated on the profession by US dominated journals. In broad terms his argument is that the European perspective, based primarily on the ‘general-to-speci…c’Cointegrated Vector AutoRegressive (CVAR) approach, has been largely ignored by US dominated journals because it places observation before theory and requires researcher judgment to be part of the analysis”. In contrast, the editorial boards of these journals have manifested a strong preference for the ‘theory …rst’perspective, currently dominated by ‘Dynamic Stochastic General Equilibrium’(DSGE) models, where data play only a subordinate role in ‘quantifying’these models. As a result, young researchers operating in a ‘publish or perish’environment would naturally avoid the European perspective because it requires hard work and judicious judgment in data modeling without any obvious professional payo¤. Instead, it is rational for empirical macroeconomists to opt for the US perspective where one only needs to demonstrate technical dexterity in solving/approximating and calibrating DSGE models. Hence, the current dominance of the DSGE in empirical macro-modeling has very little to do with the superior attributes of that perspective on either substantive or empirical grounds. Colander’s incentive-based diagnosis, although broadly right-minded, does not go far enough to bring out the deeper methodological issues and the rationale underlying the two perspectives. For instance, his analysis does not explain why the US dominated journals have adopted the ‘theory …rst’perspective in the …rst place, or why the European perspective places observation before theory, as he claims, knowing that such a perspective will not lead to publications in prestigious journals. Indeed, his ‘theory …rst’ vs. ‘data …rst’ is overly simplistic and invariably misleading because neither side will consider it as adequately characterizing their respective thesis. The US perspective is better described as a ‘Pre-Eminence of Theory’(PET) standpoint, where the data are assigned a subordinate role broadly described as ‘quantifying theories presumed adequate’. In contrast, the European ‘general-tospeci…c’ CVAR perspective attempts to give data a more substantial role in the theory-data confrontation and is more accurately described as endeavoring to accomplish the goals a¤orded by sound practices of frequentist statistical methods in learning from data. Colander’s description of the European perspective requiring ‘researcher judgment’gives the misleading impression that he refers to subjective judgments and skills in statistical modeling. This is misleading because any judgement/skill/claim that can be appraised independently by other researchers is not subjective in the same sense as one’s choice a prior distribution re‡ecting personal beliefs that nobody can question. A crucial component of Johansen’s (2007) call for assessing the premises of inference has nothing subjective about it, and the judgment/skills one needs concern the proper implementation of the Fisher-Neyman-Pearson (F-N-P) model-based statistical induction; see Cox and Hinkley (1974). In particular, he raises the question of validating the statistical premises to secure the reliability of the resulting inferences. www.economics-ejournal.org 2Economics: The Open-Access, Open-Assessment E-Journal 2 The Methodological Underpinnings of the Two Perspectives 2.1 The Pre-Eminence of Theory (PET) Perspective Why does the pre-eminence of theory (PET) perspective currently dominate US empirical macroeconomic modeling? The short answer is that, arguably, ‘it represents the status quo’with a long history in economics going back to Ricardo (1817). A case can be made that the PET perspective has dominated economic modeling for the last two centuries; see Spanos (2009a). The conventional wisdom underlying this perspective is that one builds simple idealized models which capture certain key aspects of the phenomenon of interest, and uses such models to gain insight concerning alternative economic policies. The role of the data is only subordinate in the sense that it can help to instantiate such models by quantifying them. Mill (1844) articulated an early temperate form of this perspective by arguing that causal mechanisms underlying economic phenomena are too complicated –they involve too many contributing factors –to be disentangled using observational data. This is in contrast to physical phenomena whose underlying causal mechanisms are not as complicated – they involve only a few dominating factors – and the use of experimental data can help to untangle them by ‘controlling’ the ‘disturbing’ factors. Hence, economic theories can only establish general tendencies and not precise enough implications whose validity can be assessed using observational data. These tendencies are framed in terms of the primary causal contributing factors with the rest of the numerous (potential) disturbing factors relegated to ceteris paribus clauses whose appropriateness cannot, in general, be assessed using observational data. This means that empirical evidence contrary to the implications of a theory can always be explained away as due to counteracting disturbing factors. Hence, Mill (1844) rendered theory testing via observational data impossible, and attributed to the data the auxiliary role of investigating the ceteris paribus clauses in order to shed light on the disturbing factors which prevent the establishment of the tendencies predicted by the theory in question. Marshall (1891) largely retained Mill’s methodological stance concerning the predominance of theory over data in economic theorizing despite paying lip-service to the importance of data in economic modeling. Robbins (1935) reverted to Cairnes’ (1888) more extreme version that pronounced data, more or less, irrelevant for appraising the truth of deductively established propositions. Indeed, both of them went as far as to claim that the deductive nature of economic theories bestows upon them a superior status than even physical theories because it is ultimately based on ‘self-evident truths’derived by ‘introspection’; according to Robbins (1935), p. 105: “In Economics, . . . , the ultimate constituents of our fundamental generalizations are known to us by immediate acquaintance. In the natural sciences they are known only inferentially. There is much less reason to doubt the counterpart in reality of the assumption of individual preferences than that of the assumption of the electron.” Robbins was well aware of the developments in statistics during the early 20th century, but dismissed their pertinence to theory appraisal in economics on the basis of the argument that such techniques are only applicable to data which can be conwww.economics-ejournal.org Economics: The Open-Access, Open-Assessment E-Journal 3 sidered as ‘random samples’from a static population. Unfortunately, this argument, stemming from sheer ignorance concerning the applicability and relevance of modern statistical methods, lingers on to this day (see Mirowski, 1994). Robbins1was not just dismissive of any attempts to use data for theory appraisal, he jested at early attempts to quantify demand curves using an example of a ‘Dr Blank investigating the demand for herrings’; see ibid., p. 107. In modern times, echoes of that extreme version of the PET perspective can be found in Kydland and Prescott (1991): "The issue of how con…dent we are in the econometric answer is a subtle one which cannot be resolved by computing some measure of how well the model economy mimics historical data. The degree of con…dence in the answer depends on the con…dence that is placed in the economic theory being used." (ibid., p. 171) Indeed, the theory being appraised should be the …nal arbiter: “The model economy which better …ts the data is not the one used. Rather currently established theory dictates which one is used." (ibid., p. 174). The great puzzle is that Kydland and Prescott never tell us how the ‘currently established theory’was instituted and whether anything could ever count against it. During the 19th and 20th centuries one can …nd much less extreme versions of the PET perspective where data is assigned, in principle, a less subordinate role in theory appraisal. Indeed, there is no shortage of eminent economists paying lip-service to the role of the data in economic modeling, but there is a crucial disconnect between the rhetoric and the practice; with enough perseverance one would be able to …nd remarks, even by the most extreme adherents to the PET standpoint, that would allude to the ‘important’role of the data in economic theorizing! What was missing from economic modeling was an appropriate modeling framework in the context of which the theory-data confrontation can be properly applied without compromising the credibleness of either source of information. This lack of an appropriate framework is most apparent in the extensive literature initiated by Friedman (1953) concerning the realisticness of economic theories, as well as the notable methodological exchanges between Keynes and Tinbergen and Koopmans and Vinning; see Spanos (2006a). The primary di¤erence between the 19th and the later part of the 20th century is that the developments in statistical inference, associated with the Fisher-NeymanPearson (F-N-P) model-based approach that culminated in the 1930s, helped to shed illuminating light on the role of data in empirical modeling in ways which were unknown to Mill or Marshall. Unfortunately for economics, some of the key elements of the F-N-P statistical perspective, including the importance of statistical model validation, never made it into modern econometrics, primarily because the Cowles Commission literature solidi…ed the PET perspective in econometric modeling; see Spanos (2006a). 1Ironically, Robbins lived long enough to regret his claims concerning “the limited predictive value of time series and suchlike statistical material”: “This part of the book, more than any other, re‡ects the circumstances in which it was written. It is a reaction –doubtless overdone –against the ridiculous claims of the institutionalists and the cruder econometricians.”(Robbins, 1971, p. 149). www.economics-ejournal.org 4Economics: The Open-Access, Open-Assessment E-Journal A strong case can be made (see Spanos, 2009a) that the numerous attempts to redress the balance and give data a more substantial role in theory testing were frustrated by several challenging methodological/philosophical problems bedeviling empirical modeling in economics since Ricardo (1917), the most crucial being: (MP1) the huge gap between economic theories and the available observational data, (MP2) the issue of assessing when a model ‘accounts for the regularities in the data’, (MP3) relating statistical inferences to substantive claims, hypotheses or theories. These same problems are currently entangling the discussion between these two perspectives rendering any dialogue between them almost impossible. Due primarily to problem (MP1), early attempts to give data a more substantive role focused on data-driven models implicitly assuming that their theoretical concepts and the available data largely coincide, and relying on goodness-of-…t measures, like the R2, to assess (MP2). These attempts had disastrous consequences for empirical modeling in economics because they inadvertently contributed to the forti…cation of the PET perspective for a variety of reasons. (C1) Unreliability. Data-driven correlation, linear regression, factor analysis and principal component analysis, relying on goodness-of-…t, have been notoriously unreliable when applied to observational data, especially in the social sciences. (C2) Statistical spuriousness. The arbitrariness of goodness-of-…t measures created a strong impression that one can ‘forge’signi…cant correlations (or regression coe¢ cients) at will, if one was prepared to persevere long enough ‘mining’the data. This (mistaken) impression is almost universal among philosophers and social scientists, including economists. (C3) Misplaced role for substantive information. The impression in C2 has led to widely held (but erroneous) belief that substantive subject matter (theory) information provides the only safeguard against statistical spuriousness. Exploiting the confusions created by (C1)-(C3), the PET perspective consolidated its dominance on economic modeling and persistently charged any alternative perspective that took the data seriously, including the European CVAR approach, as yet another form of ‘measurement without theory’, ‘data-mining’and ‘hunting’ for statistical signi…cance and the like. Admonitions and rebukes concerning the devastating e¤ects of invoking invalid assumptions by Campos et al (2005), Johansen (2007) and Juselius and Franchi (2007) do not resonate well with the advocates of the PET perspective because they sound like a sermon they have heard many times before. To them these admonitions sound like a well-rehearsed complaint concerning the unrealisticness of their structural models. Indeed, numerous critics of the PET perspective have articulated the unrealisticness argument over and over again during the last two centuries, beginning with Malthus (1836) who criticized the Ricardian method as based on ‘premature generalization’which occasions “an unwillingness to bring their theories to the test of experience.”(ibid, p. 8). Nevertheless, modern advocates of the PET perspective, often invoking the authority of Friedman (1953), counter that such unrealisticness is inevitable, since all models are idealizations and not faithful descriptions of reality. The abstraction/idealization argument is right-headed and perfectly legitimate at the level of the theory, but adherents of the PET perspective do not seem to appreciate the www.economics-ejournal.org Economics: The Open-Access, Open-Assessment E-Journal 5 fact that if their implicit inductive premises are invalid –vis-a-vis the data –any inferences based on such premises will be highly misleading. Indeed, in light of (C1)-(C3), the PET advocates feel that they can ignore the statistical misspeci…cation issue and argue instead that what matters is the extent to which such models ‘shed light’on the phenomenon of interest and help in formulating e¤ective economic policies. What they do not seem to realize is that any assessment concerning the sign, magnitude and signi…cance of estimated coe¢ cients, however informal, constitutes an inference whose credibility is completely undermined when the estimated model is statistically misspeci…ed; an insight from the F-N-P model-based statistical induction. 2.2 The European CVAR Perspective The European CVAR perspective has its roots in the London School of Economics (LSE) ‘general-to-speci…c’econometrics tradition (see Sargan, 1964, Hendry, 2000), and can be best understood as an attempt to redress the balance between theory and data by avoiding both extreme positions: theory-driven vs. data-driven modeling. Having re‡ected on this perspective for several years, I feel that the best way to describe this European perspective is in terms of a threefold objective (aims/aspires): (A1) to give data ‘a voice of its own’, independent of any economic theory, (A2) to reliably constrain economic theorizing using the data, and (A3) avoid ‘foisting’the theory onto the data at the outset because it precludes any genuine theory testing. In light of the huge gap between theory and data, objective (A3) renders the European CVAR perspective vulnerable to charges of ‘data-mining’ because any attempt to take the data seriously forces one to begin the modeling with a largely data-driven model like the Autoregressive Distributed Lag (ADL) and VAR models; see Hendry (1995). Indeed, the methodological problems (MP1)-(MP3) and the misleading impressions created by (C1)-(C3), have contributed signi…cantly to a genuine lack of communication between the two sides, rendering any constructive dialogue between them almost impossible. For the PET advocates the European CVAR approach is another form of data-based modeling which ignores the theory, despite their declarations to the contrary, and is highly vulnerable to problems (C1)- (C3). Worse, the aims (A1)-(A3) make little sense because for them theory is the only source of legitimate information for modeling purposes. The key to unraveling the tangled arguments separating the two perspective is provided by distinguishing between statistical adequacy and the realisticness of the structural model in question. A closer examination of the ‘testing assumptions’ criticism raised by the European CVAR approach (see Johansen, 2007, Juselius and Franchi, 2007), reveals that it has two separate components one of which concerns the proper application of statistical inference and the other has to do with the empirical adequacy of the structural model vis-a-vis the data in question. The …rst component is concerned with the validity of the probabilistic assumptions comprising the inductive premises for inference. It’s only the second component that relates to the centuries old realisticness criticism (see Maki, 2000). Hence, the advocates of www.economics-ejournal.org 6Economics: The Open-Access, Open-Assessment E-Journal the PET perspective cannot de‡ect or sidestep the statistical inadequacy criticism by invoking their arguments against the realisticness of a theory criticism; the two issues are fundamentally di¤erent. For a proper understanding of these two components and their respective roles one needs a methodological framework where these and related issues are clearly brought out. A framework that can be used to elucidate the strengths and weaknesses of both perspectives and provide the basis for a constructive dialogue between them. The same framework should also o¤er suggestions on how one might be able to address the methodological problems (C1)-(C3) mentioned above, as well as accommodate the threefold objective (A1)-(A3) of the European perspective. 3 An All-Encompassing Methodological Framework Spanos (1986), p. 17, proposed an all-encompassing methodological framework (Figure 1), devised to enable the modeler to bridge the gap between theory and data using a sequence of interconnected models with a view to delineate and probe for the potential errors at di¤erent stages of modeling; see Mayo (1996) for a similar proposal. The key to unraveling the testing of assumptions argument is provided by drawing a clear distinction between substantive and statistical assumptions because their respective validity has very di¤erent implications for inference. The substantive assumptions pertain to the realisticness issue, but the statistical assumptions pertain to the (statistical) reliability of inference. This is because when any of the statistical assumptions are invalid for data Z0, inferences based on the estimated model are often unreliable because the nominal and actual error probabilities are likely to be di¤erent. The surest way to lead an inference astray is to apply a :05 signi…cance test when the actual type I error is closer to 1:0; see Spanos and McGuirk (2001). The crucial problem in econometric modeling is that foisting the substantive information on the data by estimating the structural model M'(z)directly, is invariably an injudicious strategy because statistical speci…cation errors are likely to undermine the prospect of reliably evaluating the relevant errors for primary inferences. When modeling with observational data, the estimated Mb '(z)is often both statistically and substantively inadequate, and one has no way to delineate the two; is the theory wrong or are the (implicit) inductive premises invalid for data Z0? To avert this impenetrable quandary, the modeling framework in Figure 1 distinguishes, ab initio, between statistical and substantive information and then allows for bridging the gap between them by a sequence of interconnecting models which enable one to delineate and probe for the potential errors at di¤erent stages of modeling. From the theory side, the substantive information is initially encapsulated by a theory model and then modi…ed into a structural one M'(z)to render it estimable with data z0:From the data side, the statistical information is distilled by a statistical www.economics-ejournal.org Economics: The Open-Access, Open-Assessment E-Journal 7 model M(z)whose parameterization is chosen with a view to render M'(z)a reparametrization/restriction thereof. Stochastic phenomenon of interest Theory Theory model Data Structural model Statistical model Specification Estimation Misspecification testing Respecification Statistically Adequate model Identification Statistical Analysis Empirical model Figure 1: An Empirical Modeling Framework Distinguishing between substantive and statistical assumptions is not as straight forward as it might seem at …rst sight. The problem can be seen in Ireland (2004) where the assumptions invoked: (1) all structural parameters are constant over time, (2) total factor productivity is driving the system, (4) log output, consumption, and capital are trend-stationary, (5) labor is stationary, (6) labor augmented technological progress follows a linear trend which in‡uences the other variables identically, (7) the observable variables follow a VAR(1) process, (8) the errors are NIID, constitute a mixture of substantive and statistical assumptions; see Juselius and Franchi (2007). The initial separation depends on having a clear-cut distinction between a structural M'(z)and a statistical model M(z)where the former is viewed as an estimable form of a theory model (hence, built on substantive information) in view of the available data Z0, and the latter as a purely probabilistic construal whose structure depends solely on the statistical information contained in the data Z0:=(zt; t=1;2; :::; n); see Spanos (1986). The latter is accomplished by viewing the statistical model as a particular parameterization of a generic vector stochastic process fZt; t2Ngwhose probabilistic structure is chosen so as to render data Z0a ‘truly typical realization’of this process. The particular parameterization of fZt; t2Ngis selected so as to enable one to embed the structural model in its context. www.economics-ejournal.org 14 Economics: The Open-Access, Open-Assessment E-Journal Spanos, A. (2005), “Misspecification, Robustness and the Reliability of Inference: the simple t-test in the presence of Markov dependence,” Working Paper, Virginia Tech. Spanos, A. (2006a), “Econometrics in retrospect and prospect,” In New Palgrave Handbook of Econometrics, vol. 1, ed. T. C. Mills and K. Patterson, Macmillan, London. Spanos, A. (2006b), “Revisiting the omitted variables argument: substantive vs. statistical adequacy,” Journal of Economic Methodology, 13: 179–218. Spanos, A. (2008), “Statistics and Economics,” pp. 1129-1162 in the New Palgrave Dictionary of Economics, 2nd edition, Eds. S. N. Durlauf and L. E. Blume. Palgrave Macmillan, London. Spanos, A. (2009a), “Theory Testing in Economics and the Error Statistical Perspective,” forthcoming in Error and Inference, D. G. Mayo and A. Spanos (2009b), Cambridge University Press, Cambridge. Spanos, A. (2009b), “Philosophy of Econometrics,” forthcoming in the Handbook of the Philosophy of Science, edited by D. Gabbay, P. Thagard, and J. Woods, Elsevier, North Holland. Spanos, A. and McGuirk, A. (2001), “The model specification problem from a probabilistic reduction perspective,” Journal of the American Agricultural Association, 83: 1168–76. www.economics-ejournal.org Please note: You are most sincerely encouraged to participate in the open assessment of this article. You can do so by either rating the article on a scale from 1 (bad) to 5 (excellent) or by posting your comments. Please go to: www.economics-ejournal.org/economics/journalarticles/2009-10 The Editor © Author(s) 2009. Licensed under a Creative Commons License - Attribution-NonCommercial 2.0 Germany