Ridge regularized estimation of VAR models for inference

Ballarin, Giovanni Article — Published Version Ridge regularized estimation of VAR models for inference Journal of Time Series Analysis Provided in Cooperation with: John Wiley & Sons Suggested Citation: Ballarin, Giovanni (2024) : Ridge regularized estimation of VAR models for inference, Journal of Time Series Analysis, ISSN 1467-9892, John Wiley & Sons, Ltd, Oxford, UK, Vol. 46, Iss. 2, pp. 235-257, https://doi.org/10.1111/jtsa.12737 This Version is available at: https://hdl.handle.net/10419/319273 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by-nc/4.0/ JOURNAL OF TIME SERIES ANALYSIS J. Time Ser. Anal. 46: 235–257 (2025) Published online 18 February 2024 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1111/jtsa.12737 SPECIAL ISSUE ARTICLE RIDGE REGULARIZED ESTIMATION OF VAR MODELS FOR INFERENCE GIOVANNI BALLARIN Department of Economics, University of Mannheim, Mannheim, Germany Ridge regression is a popular method for dense least squares regularization. In this article, ridge regression is studied in the context of VAR model estimation and inference. The implications of anisotropic penalization are discussed, and a comparison is made with Bayesian ridge-type estimators. The asymptotic distribution and the properties of cross-validation techniques are analyzed. Finally, the estimation of impulse response functions is evaluated with Monte Carlo simulations and ridge regression is compared with a number of similar and competing methods. Received 17 July 2023; Accepted 30 January 2024 Keywords: Impulse responses; inference; ridge regularization; vector autoregression. JEL. C32; C51; C52. MOS subject classification: 60G10; 60G25; 62F12; 62J07; 62M10; 62M20; 62P20. 1. INTRODUCTION While the idea of using ridge regression for vector autoregressive model estimation dates back to Hamilton (1994), there seems to be no complete analysis of its properties and asymptotic theory in the literature. This article fills this gap by analyzing the geometric and distributional properties of ridge in a VAR estimation framework, discussing its comparison to well-known Bayesian approaches and deriving the validity of cross-validation as a selection procedure for the ridge penalty. First, I show that the shrinkage induced by the ridge estimator, while intuitive in the setting of an isotropic penalty, produces complex effects when estimating a VAR model with a more flexible penalization scheme. This implies that the benefits of the bias-variance trade-off (Hastie, 2020) may be hard to gauge a priori. I provide a tractable example where ridge can yield estimates that have higher autoregressive dependence than the least squares solution. To better understand how different ridge penalization strategies can be designed, I also make a comparison with Bayesian VAR estimators commonly used in macroeconometric practice. Second, I generalize the analysis of Fu and Knight (2000) and prove the consistency and asymptotic normality of the ridge estimator, a result that seems to be missing in the literature. For standard inference, the ridge penalty should either be negligible in the limit or its centering converge in probability to the true parameter vector. In both these cases, there is no asymptotic bias and no reduction in variance. Alternatively, in settings where a researcher is willing to assume that a subset of the VAR parameters features small coefficients, one can achieve an asymptotic reduction of variance by correctly tuning the ridge penalty matrix. I further derive the properties of cross-validation, which is a popular approach in practical applications to tune penalized estimators (Hastie et al.,2009; Bergmeir et al.,2018). More specifically, I show that cross-validation is able to select penalties that are asymptotically valid ∗Correspondence to: Giovanni Ballarin, Department of Economics, University of Mannheim, L7, 3-5, Mannheim, 68131, Germany. Email: giov[email protected] © 2024 The Authors. Journal of Time Series Analysis published by John Wiley & Sons Ltd. This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes. 236 G. BALLARIN for inference. In passing, I also prove that, in an autoregressive setup, the time dependence of regressors has an exponentially small effect on in-sample prediction error evaluation. Lastly, I use Monte Carlo simulations to study the performance of the different ridge approaches discussed, focusing on impulse response inference. I consider two exercises: one is based on a three-variable VARMA(1,1) data generating process from Kilian and Kim (2011); the other is a VAR(5) model estimated in levels from a set of seven macroeconomic series, following Giannone et al. (2015). The finding is that ridge can lead to improvements over unregularized methods in impulse response confidence interval length, while Bayesian estimators show the best overall performance due to the underlying flexibility of their priors. 1.1. Related Literature This article does not discuss the high-dimensional setting, where the number of regressors grows together with the sample size. Some important work has been done in this direction already. Dobriban and Wager (2018)derivean explicit expression for the predictive risk of ridge regression assuming a high-dimensional random effects model. Other works in this vein are Liu and Dobriban (2020); Patil et al. (2021) and Hastie et al. (2022), which are mostly focused on penalty selection by cross-validation, as well as structural features of ridge. Generally speaking, the complexity of analyzing ridge regression in high dimensions is a challenge to precisely understanding its practical implications. As I show below, in the context of finite-dimensional VARs, asymptotic inference demands that the ridge penalty becomes asymptotically negligible at appropriate rates. Thus, a challenge is understanding in what way high-dimensional time series problems would benefit from the use of ridge. This question is beyond the scope of this article. In the time series forecasting literature, ridge regression is commonly used for prediction. I provide a partial list of contributions in this direction. Inoue and Kilian (2008) use ridge regularization for forecasting U.S. consumer price inflation and argue that it compares favorably with bagging techniques; De Mol et al. (2008) use a Bayesian VAR with posterior mean equivalent to a ridge regression in forecasting; Ghosh et al. (2019) again study the Bayesian ridge, this time, however, in the high-dimensional context; Goulet Coulombe et al. (2022), Fuleky (2020), Babii et al. (2021), and Medeiros et al. (2021) compare LASSO, ridge and other machine learning techniques for forecasting with large economic datasets. Fuleky (2020) gives a textbook treatment of penalized time series estimation, including ridge, but does not discuss inference. The ridge penalty is considered within a more general mixed 𝓁1-𝓁2penalization setting in Smeekes and Wijler (2018), who study the performance and robustness of penalized estimates for constructing forecasts. Regarding inference, Li et al. (2024) provided a general exploration of shrinkage procedures in the context of structural impulse response estimation. Very recently, Cavaliere et al. (2023) suggested a methodology for inference on ridge-type estimators that relies on bootstrapping. Finally, shrinkage of autoregressive models to constrained submodels was discussed by Hansen (2016b) in a more general setting. Finally, various estimation problems can either be cast as or augmented with ridge-type regressions. Goulet Coulombe (2023) shows that the estimation of VARs with time-varying parameters can be written as ridge regression. Plagborg-Møller (2016) and Barnichon and Brownlees (2019) both use ridge to derive smoothed local projection impulse response functions. 1.2. Outline Section 2provides a discussion of the ridge penalty and the ridge VAR estimator. In Section 3I deal with the properties of ridge-induced shrinkage in the autoregressive coefficients. I discuss the connections between frequentist and Bayesian ridge for VAR models within Section 4. Section 5develops the asymptotic theory and inference result in the case where there is no asymptotic shrinkage. This includes studying the property of cross-validation under dependence. Section 6provides inference and CV results in a setting where some shrinkage of a subset of parameters is possible. Section 7presents Monte Carlo simulations focused on impulse response estimation. Section 8concludes. Finally, the Data S1 Supplementary Appendix contains more detailed derivations, as well as all proofs, additional tables and further information on simulations. wileyonlinelibrary.com/journal/jtsa © 2024 The Authors. J. Time Ser. Anal. 46: 235–257 (2025) Journal of Time Series Analysis published by John Wiley & Sons Ltd. DOI: 10.1111/jtsa.12737 RIDGE REGULARIZED ESTIMATION OF VAR MODELS 237 1.3. Notation Define R+to be the set of strictly positive real numbers. Vectors v∈RNand matrices A∈RN×Mare always denoted with lower and uppercase letters respectively. Throughout, I will use IMto represent the identity matrix of dimension M. For any vector v∈RN,‖v‖is the Euclidean norm. For any matrix A∈RN×M,‖A‖is the spectral norm unless stated otherwise; ‖A‖max =maxi,j|||aij|||is the maximal entry norm; ‖A‖F=(tr{A′A})−1∕2is the Frobenius norm; vec(⋅)is the vectorization operator and ⊗is the Kronecker product (Lütkepohl, 2005). If a vector represents a vectorized matrix, then it will be written in bold, that is, for A∈RN×MI write vec(A)=a∈RNM. Let Λ=diag{𝜆1,…,𝜆K2p},𝜆i>0foralli=1,…,K2p. To give the partial ordering of diagonal positive semi-definite penalization matrices, let Λ1=diag{𝜆1,j}K2p j=1and Λ2=diag{𝜆2,j}K2p j=1. I write Λ1≺Λ2if 𝜆1,i<𝜆 2,i for all i=1,…,K2p;Λ1⪯Λ 2if 𝜆1,i≤𝜆2,ifor all iand ∃j∈1,…,K2psuch that 𝜆1,j<𝜆 2,j. Symbols P −−→and d −−→are used to indicate convergence in probability and convergence in distribution respectively. 2. RIDGE REGULARIZED VAR ESTIMATION Let yt=(y1t,…,yKt)′be a K-dimensional vector autoregressive process with lag length p≥1 and parametrization yt=𝜈t+A1yt−1+A2yt−2+···+Apyt−p+ut,(1) where ut=(u1t,…,uKt)′is additive noise such that utare i.i.d. with E[uit]=0andVar[ut]=Σ u,and𝜈t is a deterministic trend. For simplicity, in the remainder I shall assume that 𝜈t=0sothatythas no trend component – equivalently, ytis a detrended series. For a given sample size Tdefine Y=(y1,…,yT)∈RK×T,zt=(y′ t,y′ t−1,…,y′ t−p+1)′∈RKp,Z= (z0,…,zT−1)∈RKp×T,B=(A1,…,Ap)∈RK×Kp,U=(u1,…,uT)∈RK×T, and vectorized counterparts y=vec(Y),𝜷=vec(B)and u=vec(U). Accordingly, Y=BZ +Uand y=(Z′⊗IK)𝜷+u, where Σu=IK⊗Σu. Importantly, throughout this article, I will assume that the cross-sectional dimension K remains fixed. Ridge regularization is a modification of the least squares objective by the addition of a term dependent on the Euclidean norm of the coefficient vector. The isotropic Ridge-regularized Least Squares (RLS) estimator is therefore defined as  𝜷R(𝜆)∶=arg min 𝜷 1 T‖‖y−(Z′⊗IK)𝜷‖‖2+𝜆‖𝜷‖2, where 𝜆>0 is the scalar regularization parameter or regularizer. When 𝜆‖𝜷‖2is replaced with quadratic form 𝜷′Λ𝜷for a positive definite matrix Λthe above is often termed Tikhonov regularization. To avoid confusion, I shall also refer to it as ‘ridge’, since in what follows Λwill always be assumed diagonal. As Λdoes not, in general, penalize all coefficients equally, it can be used to construct an anisotropic ridge estimator. By solving the normal equations (see Supplementary Appendix A.1), the RLS estimator with positive semi-definite regularization matrix Λ∈RK2p×K2pisshowntobe  𝜷R(Λ) = (ZZ′ T⊗IK+Λ )−1(Z⊗IK)y T. When a centering vector 𝜷0≠0isincludedinpenalty(𝜷−𝜷0)′Λ(𝜷−𝜷0), the RLS estimator becomes  𝜷R(Λ,𝜷0)=(ZZ′ T⊗IK+Λ )−1((Z⊗IK)y T+Λ𝜷0).(2) J. Time Ser. Anal. 46: 235–257 (2025) © 2024 The Authors. wileyonlinelibrary.com/journal/jtsa DOI: 10.1111/jtsa.12737 Journal of Time Series Analysis published by John Wiley & Sons Ltd. 238 G. BALLARIN In the context of multi-variate estimation, one has to make a further distinction between two related types of ridge estimators. I let  BR(Λ,𝜷0)be the de-vectorized coefficient estimator obtained from reshaping  𝜷R(Λ,𝜷0)to a K×Kp matrix. But one can also consider the matrix RLS estimator  BR mat(ΛKp,B0)given by  BR mat(ΛKp,B0)=T−1(Y+B0ΛKp)Z′(T−1ZZ′+Λ Kp)−1, where ΛKp =diag{𝜆1,…,𝜆Kp}and B0is a centering matrix. The distinction is important because the vectorized and matrix RLS estimators in general need not coincide. As discussed in Supplementary Appendix A.2, BR(Λ,𝜷0) allows for more general penalty structures compared to  BR mat(ΛKp,B0). I, therefore, focus on the former rather than the latter. Remark 1. Equation (2) implies that  𝜷R(Λ,𝜷0)and, therefore,  𝜷R(Λ) provide simultaneous estimation of all the coefficients in 𝜷. However, by analogy with ordinary least squares (OLS) VAR estimation, one may also consider an equation-by-equation ridge regression (ebe-RLS) scheme. For k=1,…,K,letyk=(Z′⊗ IK)𝜷k+ukbe the autoregressive equation for the kth series of yt. Then, we can define the kth equation RLS estimator to be  𝜷R k(Λ,𝜷0)=(ZZ′ T⊗IK+Λ k)−1((Z⊗IK)yk T+Λ𝜷0k), where 0 ⪯Λ kand 𝜷0kare the kth equation regularizer and centering respectively. Notice that the ebe-RLS approach allows, by construction, to penalize the estimates for one component differently than for another, and the two can be independently chosen. This provides a higher degree of freedom than the one afforded by, for example, the anisotropic lag-adapted scheme proposed in Section 3.2 or the Bayesian schemes of Section 4. However, implementing ebe-RLS in applications inherently implies that data-driven tuning of Λkwill be significantly more computationally intensive – with costs growing linearly in K. Due to this higher complexity, in both theoretical derivation and simulations below, I will focus on studying the properties of the simultaneous RLS estimator. Remark 2. Further regarding ebe-RLS, another way to approach estimation is through the recursive form of the VAR model. Let Σu=P−1DP−1′,whereP−1is an unitriangular matrix and Da diagonal matrix, so that we may write Y=GZ − PY +D−1∕2E, where G=PB, P=P−IKand noise term Ehas identity covariance matrix. Estimation can now be performed in ebe-RLS fashion, and matrices P,Band Dare recovered (Hausman, 1983). Notice, however, that in this framework the ordering of variables plays a role, since it also determines the decomposition of Σu. Indeed, even if a penalization scheme is fixed, permuting the entries may yield different penalized estimates for P,Band D,sothat both slope and covariance parameter estimates are different, implying (structural) IRF estimates will also differ. However, note that this issue is somewhat mirrored in a recursive shock identification approach: after estimation,  Σuis Cholesky decomposed to identify the shocks’ rotation, and the ordering of variables is key and must be economically justified. 3. SHRINKAGE Here, I discuss both the isotropic ridge penalty, i.e., the ‘standard’ ridge approach, and an anisotropic penalty that is better adapted to the VAR setting. An important result is that, even in simple setups with only two variables, the shrinkage induced by ridge can either increase or reduce bias, as well as the stability of autoregressive estimates. wileyonlinelibrary.com/journal/jtsa © 2024 The Authors. J. Time Ser. Anal. 46: 235–257 (2025) Journal of Time Series Analysis published by John Wiley & Sons Ltd. DOI: 10.1111/jtsa.12737 RIDGE REGULARIZED ESTIMATION OF VAR MODELS 239 Throughout this section, I consider fixed design matrices and the focus will be on the geometric properties of ridge. 3.1. Isotropic Penalty The most common way to perform a ridge regression is through isotropic regularization, that is, Λ=𝜆Ifor some scalar 𝜆≥0. Isotropic ridge has been extensively studied, see for example, the comprehensive review of Hastie (2020). With regard to shrinkage, an isotropic ridge penalty can be readily studied. Proposition 1. Let Z∈RM×T,Y∈RTfor T>Mbe regression matrices. For 𝜆•>𝜆>0 and isotropic RLS estimator  𝛽R(𝜆)∶=(T−1ZZ′+𝜆IM)−1(T−1ZY)it holds ‖‖‖ 𝛽R(𝜆•)‖‖‖<‖‖‖ 𝛽R(𝜆)‖‖‖. Proof. Using the full singular-value decomposition (SVD), decompose T−1∕2Z=UDV′∈RM×Twhere Uis M×Morthogonal, Dis M×Tdiagonal and Vis T×Torthogonal. Write  𝛽R(𝜆•)=(T−1ZZ′+𝜆•IM)−1(T−1ZY) =(UDV′VDU′+𝜆•IM)−1UDV′(T−1∕2Y) =U(D2+𝜆•IM)−1DV′(T−1∕2Y) =U(D2+𝜆•IM)−1(D2+𝜆IM)(D2+𝜆IM)−1DV′(T−1∕2Y) =[U(D2+𝜆•IM)−1(D2+𝜆IM)U′] 𝛽R(𝜆). Since D2=diag{𝜎2 j}M j=1, the term within brackets is Udiag{(𝜎2 j+𝜆)∕(𝜎2 j+𝜆•)}M j=1U′. Moreover, because the spectral norm is unitary invariant and 𝜆1>𝜆 2, it follows that ‖‖U(D2+𝜆•IM)−1(D2+𝜆IM)U′‖‖=‖‖‖diag{(𝜎2 j+𝜆)∕(𝜎2 j+𝜆•)}M j=1‖‖‖<1. Finally, by the sub-multiplicative property it holds ‖‖‖ 𝛽R(𝜆•)‖‖‖≤‖‖U(D2+𝜆1IM)−1(D2+𝜆IM)U′‖‖⋅‖‖‖ 𝛽R(𝜆)‖‖‖<‖‖‖ 𝛽R(𝜆)‖‖‖, as claimed. ◾ Proposition 1and its proof expose the main ingredients of ridge regression. From the SVD of T−1∕2Zused above, it is clear that ridge regularization acts uniformly along the orthogonal directions that are the columns of V.The improvement in the conditioning of the inverse comes from all diagonal factors [(D2+𝜆•IM)−1D]j=𝜎j∕(𝜎2 j+𝜆•) being well-defined, even when 𝜎j=0 (as is the case in systems with collinear regressors). However, directly applying isotropic ridge to vector autoregressive models is not necessarily the most effective estimation approach. Stable VAR models show decay in the absolute size of coefficients over lags. Thus, it is reasonable to choose a more general ridge penalty that can accommodate lag decay. 3.2. Lag-Adapted Penalty I now consider a different form for Λthat is of interest when applying ridge specifically to a VAR model. Define family (p)of lag-adapted ridge penalty matrices as (p)={diag{𝜆1,…,𝜆p}⊗IK2|𝜆i∈R+,i=1,…,p}, J. Time Ser. Anal. 46: 235–257 (2025) © 2024 The Authors. wileyonlinelibrary.com/journal/jtsa DOI: 10.1111/jtsa.12737 Journal of Time Series Analysis published by John Wiley & Sons Ltd. 240 G. BALLARIN where each 𝜆iintuitively implies a different penalty for the elements of each coefficient matrix Ai,i=1,…,p.1 The family (p)allows imposing a ridge penalty that is coherent with the lag dimension of an autoregressive model. It is parametrized by pdistinct penalty factors, meaning that the penalization is anisotropic. Proposition 2. Let Z∈RKp×T,y∈RKT for T>Kp be multi-variate VAR regression matrices. Given subset ⊆{1,…,p}of cardinality s=||,forΛ(p)∈(p)define  𝜷R(Λ(p))[]as the vector of sK2coefficient estimates located at indexes 1 +K2(j−1),…,K2jfor j∈.Letc={1,…,p}⧵be the complement of . a. If 𝜆1≥𝜆2,then‖‖‖ 𝜷R(𝜆1IK2p)[]‖‖‖≤‖‖‖ 𝜷R(𝜆2IK2p)[]‖‖‖for any ⊂{1,…,K2p}. The inequality is strict when 𝜆1>𝜆 2. b. Let  𝜷LS []be the least squares estimator of the autoregressive model with only the lags indexed by included and zeros as coefficients for the lags indexed by c. Similarly, let Λ(p) []be the subset of diagonal elements in Λ(p)penalizing the lags in .Then lim Λ(p) []→0 Λ(p) [c]→∞  𝜷R(Λ(p))= 𝜷LS [], where Λ(p) []→0andΛ(p) [c]→∞are to be intended as the element-wise convergence. Proposition 2shows that the limiting geometry of a lag-adapted ridge estimator is thus identical to that of a least squares regression run on the subset specified by . By controlling the size of coefficients {𝜆1,…,𝜆p} it is therefore possible to obtain pseudo-model-selection. However, in the next section, I show that anisotropic penalization produces complex effects on the model’s coefficient estimates. 3.3. Illustration of Anisotropic Penalization Here, I aim to illustrate the effects of a lag-adapted ridge penalty on VAR coefficients estimates using a particular example. This further helps motivate and contextualize the results of the simulation exercises provided in Section 7. More generally, before moving on to the discussion of more sophisticated forms of ridge regression, it is important to gain some intuition regrading the properties of anisotropic penalization, which I highlight with the help of a simple bivariate VAR(2) model. Note that, since ridge operates along principal components, there is no immediate relationship between a specific subset of the estimated coefficients and a given diagonal block of Λ(p). With regard to autoregressive modeling, three effects are of interest: the shrinkage of coefficient matrices Airelative to the choice of 𝜆i; the entity of the bias introduced by shrinkage, and the impact of shrinkage on the persistence of the estimated model. To showcase these effects, I consider the VAR(2) model yt=A1yt−1+A2yt−2+ut,ut∼i.i.d. (0,Σu), where A1=[0.80.1 −0.10.7],A2=[0.1−0.2 −0.10.1],Σu=[0.30 05 ]. 1Note that with a lag-adapted penalty it is also possible to directly use the matrix ridge estimator since the penalty for  𝜷Ris given by diag{𝜆1,…,𝜆p}⊗IK2=(diag{𝜆1,…,𝜆p}⊗IK)⊗IK, see Supplementary Appendix A.2. Importantly, this kind of structure is minimal in terms of modeling the relative size of coefficients within each coefficient matrix Ai. If economic theory or intuition provides information about the effects of one specific variable and lag on another – say, the contemporaneous effect of the first series on the second series is zero – more structure can be integrated into the ridge penalty matrix. This would mean, however, that different ridge estimator forms are not equivalent. wileyonlinelibrary.com/journal/jtsa © 2024 The Authors. J. Time Ser. Anal. 46: 235–257 (2025) Journal of Time Series Analysis published by John Wiley & Sons Ltd. DOI: 10.1111/jtsa.12737 RIDGE REGULARIZED ESTIMATION OF VAR MODELS 241 Figure 1. Shrinkage of coefficients estimate in Frobenius norm (a); bias induced by shrinkage (b); change in stability of estimated VAR model at different levels of penalization, measured by the absolute value of the largest companion form eigenvalue (c) A single sample of length T=200 is drawn, demeaned and used to estimate coefficients A1and A2. The VAR(2) model is fitted using the lag-adapted ridge estimator  BR(Λ(2)),whereΛ(2)=diag{𝜆1,𝜆2}⊗I2. Note that  BR(Λ(2)) can be partitioned into estimates ÂR 1(Λ(2))and ÂR 2(Λ(2))for the respective parameter matrices. 3.3.1. Shrinkage To illustrate shrinkage, I consider the restricted case of 𝜆1∈[10−2,106]and 𝜆2=0. The ridge estimator is computed for varying 𝜆1over a logarithmically spaced grid. Figure 1(a) shows that ‖‖‖ BR(Λ(2))‖‖‖F≈‖‖‖ BLS‖‖‖Ffor 𝜆1≈0, but as the penalty increases ‖‖‖ÂR 1(Λ(2))‖‖‖Fdecreases while ‖‖‖ÂR 2(Λ(2))‖‖‖Fgrows. The resulting behavior of ‖‖‖ BR(Λ(2))‖‖‖Fis non-monotonic in 𝜆1, although indeed ‖‖‖ BR(Λ(2))‖‖‖F<‖‖‖ BLS‖‖‖Fin the limit 𝜆1→∞. This effect is J. Time Ser. Anal. 46: 235–257 (2025) © 2024 The Authors. wileyonlinelibrary.com/journal/jtsa DOI: 10.1111/jtsa.12737 Journal of Time Series Analysis published by John Wiley & Sons Ltd. 242 G. BALLARIN due to the model selection properties of lag-adapted ridge, and the resulting omitted variable bias. Therefore, in practice, it is not generally true that anisotropic ridge induces monotonic shrinkage of estimates. 3.4. Bias Since ridge bias is hard to study theoretically, I use a simulation with the same setup of Figure 1(a), this time with 𝜆1,𝜆2∈[10−2,104]. The grid is logarithmic with 150 points. Figure 1(b) presents a level plot of the sup-norm ridge bias ‖‖‖ BR(Λ(2))−B‖‖‖∞given multiple combinations of 𝜆1and 𝜆2. While there can be gains compared to the least squares estimator  BLS, they are modest. Moreover, level curves of the bias surface show that gains concentrate in a very thin region of the parameter space. Consequently, one may imagine that, in practice, any (data-driven) ridge penalty selection criterion is unlikely to yield bias improvement over least squares. Yet, in large VAR models with many lags, the reduction in variance of the ridge estimator often yields improvements over un-regularized procedures (Li et al.,2024). However, the bias-variance trade-off in ridge is not a free-lunch when performing inference. Pratt (1961) showed that it is not possible to produce a test (equivalently, a CI procedure) which is valid uniformly over the parameter space and yields meaningfully smaller confidence intervals than any other valid method. 3.5. Stability To study the stability of ridge VAR estimates, I reuse the results of the bias simulation above. Let Abe the companion matrix of [A1,A2],and  ARthe companion matrix of estimates [ÂR 1(Λ(2)),ÂR 2(Λ(2))]. For all combinations (𝜆1,𝜆2), I compute the largest eigenvalue 𝜔1( AR)of  AR. Note that if |𝜔1( A)|<1, then the estimated VAR(2) is stable (Lütkepohl, 2005). Figure 1(c) presents the level sets for the surface of maximal eigenvalue moduli, and for comparison |𝜔1( BLS)|is shown at the origin.2While along the main diagonal there is a clear decrease in |𝜔1( AR)| as isotropic penalization increases, when 𝜆1is large and 𝜆2≪1 (or vice versa) the maximal eigenvalue increases instead. Therefore, an estimate of a VAR model obtained with anisotropic ridge may be closer to unit root than the least squares estimate. 4. BAYESIAN AND FREQUENTIST RIDGE So far, I have discussed standard ridge penalization schemes. Here, I study the posterior mean of Bayesian VAR (BVAR) priors commonly applied in the macroeconometrics literature. I show that such posteriors are in fact specific GLS formulations of the ridge estimator. This comparison highlights that ridge can be seen as a way to embed prior knowledge into the least squares estimation procedure by means of centering and rescaling coefficient estimates. 4.1. Litterman–Minnesota Priors In Bayesian time series modeling, the so-called Minnesota or Litterman prior has found great success (Litterman, 1986). For stationary processes which one believes to have reasonably small dependence, a zero-mean normal prior can be put on the VAR parameters, with non-zero prior variance. Assuming that the covariance matrix of errors Σuis known, the Litterman–Minnesota has posterior mean 𝜷|Σu=[V−1 𝜷+(ZZ′⊗Σ−1 u)]−1(Z⊗Σ−1 u)y,(3) 2If Λ(2)→0, then by continuity of eigenvalues it follows that |𝜔1( AR)|→|𝜔1( ALS)|, see Supplementary Appendix A.3. wileyonlinelibrary.com/journal/jtsa © 2024 The Authors. J. Time Ser. Anal. 46: 235–257 (2025) Journal of Time Series Analysis published by John Wiley & Sons Ltd. DOI: 10.1111/jtsa.12737 RIDGE REGULARIZED ESTIMATION OF VAR MODELS 249 (i) 𝜷=(𝜷′ 1,𝜷′ 2)′where 𝜷1∈RK2(p−n)and 𝜷2=T−(1∕2+𝛿)b2for 𝛿>0, b2∈RK2nfixed. (ii) Λ=diag{(L′ 1,L′ 2)′}where L1∈RK2(p−n) +and L2∈RK2n +. (iii) L1=oP(T−1∕2)and L2 P −−→L2as T→∞. (iv) 𝜷0=0. Let ΓΛ=Γ+Λwhere Λ≽0isgivenby(8). Then, results (a)-(c) hold and (d′′)√T( 𝜷R(Λ,𝜷0)−𝜷)d −−→(0,Γ−1 ΛΓΓ −1 Λ⊗Σu). It is easy to see that indeed the term Γ−1 ΛΓΓ −1 Λin Theorem 6is weakly smaller than Γ−1in the positive-definite sense. Note that Γ−1 ΛΓΓ −1 Λ≺Γ−1⟺(Γ + Λ)−1Γ≺Γ−1(Γ + Λ) ⟺IK2p−(Γ+Λ)−1Λ≺IK2p+Γ −1Λ ⟺0⪯((Γ+Λ)−1+Γ −1)Λ. The last inequality is true by definition of Λ. Shrinkage gains are concentrated at the components that have non-zero asymptotic shrinkage, i.e. those penalized by L2. Remark 6. A key point in the application of Theorem 6is identification of 𝜷1and 𝜷2. In practice, one may then proceed in two ways. As discussed in Section 4, one can see the ridge approach as a frequentist ‘counterpart’ to implementing a Bayesian prior. Therefore, the researcher may split 𝜷into subsets of small and large parameters based on economic intuition, domain knowledge or preliminary information. Alternatively, in the following section, I show that cross-validation is able to automatically tune Λappropriately. Finally, it is immediate to generalize the argument of Theorem 6to the case where 𝜷is not split into subsets based on the relative size of coefficients, but rather a non-zero, partially consistent centering sequence 𝜷0is used. Corollary 2. Consider the setup of Theorem 6, where now assumptions (i) and (iv) are replaced by (i′)𝜷=(𝜷′ 1,𝜷′ 2)′where 𝜷1∈RK2(p−n)and 𝜷2∈RK2nare fixed. (iv ′)𝜷0=(𝜷′ 01,𝜷′ 02)′where 𝜷01 ∈RK2(p−n)is such that 𝜷01 ≠𝜷1,and𝜷02 =𝜷2+T−(1∕2+𝛿)b2for 𝛿>0, b2∈RK2nfixed. Then, results (a)-(c) and (d′′) still hold. 6.1. Cross-validation with Partitioned Coefficients One can use the same approach applied to derive Theorem 5to show that cross-validating the RLS estimator with Err( 𝜷R ⧫(Λ)) is also asymptotically valid under partitioning. Corollary 3. Consider the setup of Theorem 6and assume that the assumptions of Theorem 5are met. It holds [Λ1,⧫0 0Λ2,⧫]∶= arg min Λ∈𝜆 Err( 𝜷R ⧫(Λ))=[op(T−1∕2)0 0oP(1)] Moreover, any Λ2,⧫such that 0 ⪯Λ 2,⧫⪯𝜆Iis asymptotically valid. In theory, one would like to be able to quantify the gains obtained in the asymptotic shrinkage setup of Theorem 6 compared to the standard setting of Theorems 1and 2, particularly when using cross-validation. Unfortunately, it is J. Time Ser. Anal. 46: 235–257 (2025) © 2024 The Authors. wileyonlinelibrary.com/journal/jtsa DOI: 10.1111/jtsa.12737 Journal of Time Series Analysis published by John Wiley & Sons Ltd. 250 G. BALLARIN in general hard to study the cross-validation error loss even in setups without dependence. Stephenson et al. (2021), in fact, show that the ridge leave-one-out CV loss is not generally convex. This suggests that studying the behavior of CV when penalizing with a diagonal anisotropic Λcan be a very complex task in a finite sample setup. 7. SIMULATIONS To study the performance of ridge-regularized estimators, I now perform simulation exercises focused on impulse response functions (IRFs). Throughout the experiments I will consider structural impulse responses, and I assume that identification can be obtained in a recursive way (Kilian and Lütkepohl, 2017), which is a widely used approach for structural shock identification in macroeconometrics. I consider two setups: 1. The three-variable VARMA(1,1) design of Kilian and Kim (2011), representing a small-scale macro model. I term this setup ‘A’. 2. A VAR(5) model in levels, using the model specification of Giannone et al. (2015) with the dataset of Hansen (2016b) consisting of K=7 variables in levels.8I term this setup ‘B’. For the ease of exposition, in the discussion I will tabulate results only for three variables – real GDP, investment and federal funds rate – but complete tables can be found in Supplementary Appendix D.5. The specification of Kilian and Kim (2011) has already been extensively used in the literature as a benchmark to gauge the basic properties of inference methods. On the other hand, the estimation task of Giannone et al. (2015) involves more variables and a higher degree of persistence. This setting is useful to evaluate the effects of ridge shrinkage when applied to realistic macroeconomic questions. It is also a suitable test bench to compare Bayesian methods with frequentist ridge. 7.1. Estimators For frequentist methods, I include both  𝜷Rand  𝜷RGLS ridge estimators as well as the local projection estimator of Jordà (2005). For Bayesian methods, I implement both the Minnesota prior approach of Ba´ nbura et al. (2010) with stationary prior and the hierarchical prior BVAR of Giannone et al. (2015).9The full list of method I consider is given in Table I. To make methods comparable, I have extended the ridge estimators to include an intercept in the regression. A precise discussion regarding the tuning of penalties and hyperparameters of all methods can be found in Supplementary Appendix D. 7.2. Pointwise MSE The first two simulation designs explore the MSE performance of ridge-type estimators versus alternatives. Let 𝜃km(h)be the horizon hstructural IRF for variable kgiven a unit shock from variable m. To compute the MSE for each k,define MSEk(h)∶= K ∑ m=1 E[( 𝜃km(h)−𝜃km(h))2], which is the total MSE for the kth variable over all possible structural shocks. In simulations, I use Breplications to estimate the expectation. All MSEs are normalized by the mean squared error of the least squares estimator. 8The dataset is supplied by the author at https://users.ssc.wisc.edu/~bhansen/progs/var.html. While the data provided by Hansen (2016b) includes releases until 2016, I do not include more recent quarterly data since this is a simulation exercise. Moreover, due to the effects of the COVID-19 global pandemic, an extended sample would likely only add data released until Q4 2019 due to overwhelming concerns of a break point. 9To estimate hierarchical prior BVARs I rely on the original MATLAB implementation provided by Giannone et al. (2015) on the authors’ website at http://faculty.wcas.northwestern.edu/gep575/GLPreplicationWeb.zip. wileyonlinelibrary.com/journal/jtsa © 2024 The Authors. J. Time Ser. Anal. 46: 235–257 (2025) Journal of Time Series Analysis published by John Wiley & Sons Ltd. DOI: 10.1111/jtsa.12737 RIDGE REGULARIZED ESTIMATION OF VAR MODELS 251 Table I. List of estimation methods Type Name Description Frequentist LS Least squares estimator RIDGE Ridge estimator, CV penalty RIDGE-GLS GLS ridge estimator, CV penalty RIDGE-AS Ridge estimator with asymptotic shrinkage, CV penalty LP Local projections with Newey-West covariance estimate Bayesian BVAR-CV Litterman-Minnesota Bayesian VAR, CV tightness prior H-BVAR Hierarchical Bayesian VAR of Giannone et al. (2015) Table II. MSE relative to OLS – Setup A Variable Method h=1h=4h=8h=12 h=16 h=20 h=24 Investment RIDGE 0.97 0.74 0.64 0.64 0.65 0.63 0.60 growth RIDGE-GLS 5.16 0.89 0.55 0.47 0.44 0.41 0.38 LP 1.00 1.05 1.13 1.52 2.15 3.20 4.87 BVAR-CV 1.55 0.84 0.70 0.70 0.71 0.70 0.66 H-BVAR 1.80 0.66 0.53 0.52 0.54 0.53 0.50 Deflator RIDGE 0.93 0.78 0.69 0.68 0.67 0.64 0.59 RIDGE-GLS 2.43 0.83 0.59 0.52 0.48 0.44 0.40 LP 1.00 1.05 1.13 1.44 1.99 2.90 4.47 BVAR-CV 1.03 0.89 0.74 0.73 0.73 0.70 0.66 H-BVAR 1.01 0.70 0.58 0.56 0.55 0.53 0.50 Paper rate RIDGE 0.94 0.76 0.66 0.66 0.66 0.64 0.60 RIDGE-GLS 1.80 0.87 0.59 0.52 0.47 0.43 0.39 LP 1.00 1.05 1.13 1.46 1.99 2.86 4.31 BVAR-CV 0.87 0.87 0.74 0.73 0.73 0.71 0.66 H-BVAR 0.81 0.69 0.57 0.55 0.56 0.54 0.51 7.2.1. Setup A A time series of length T=200 is generated a number B=10,000 of times for replication. All VAR estimators are computed using p=10 lags, while LPs include q=10 regression lags. Table II shows relative MSEs for this design. It is important to notice that, in this situation, GLS ridge has remarkably low performance at horizon h=1 compared to other methods. The primary issue is that Σufeatures strong correlation between components, and thus the diagonal lag-adapted structure does not shrink along the appropriate directions. This is much less prominent as the horizon increases due to the fact that impulse responses eventually decay to zero, since the underlying VARMA DGP is stationary. While there is no clear ranking, the MSE of the baseline ridge VAR estimator is in between those of the BVAR and hierarchical BVAR approaches. The degrading quality of local projection estimates are mainly due to the smaller samples available in regressions at each increasing horizon (Kilian and Kim, 2011). This behavior is one of the prime reasons behind the development of LP shrinkage estimators, like that proposed in Plagborg-Møller (2016) or the SLP estimator of Barnichon and Brownlees (2019). 7.2.2. Setup B Using the data of Hansen (2016b), I estimate and simulate a stationary but highly persistent VAR(5) model using the same sample size and number of replications as Setup A. For all methods, p=5 lags are used, so that VAR estimators are correctly specified. The results can be found in Table III. In this setup, unlike in the previous experiment, one can clearly notice that impulse responses computed via cross-validated ridge show increasing MSE as horizon hgrows. There are two main reasons behind this behavior. First, the chosen setup features a very persistent data generating process, as the largest root of the underlying VAR model is 0.9945. This means that the true IRFs J. Time Ser. Anal. 46: 235–257 (2025) © 2024 The Authors. wileyonlinelibrary.com/journal/jtsa DOI: 10.1111/jtsa.12737 Journal of Time Series Analysis published by John Wiley & Sons Ltd. 252 G. BALLARIN Table III. MSE relative to OLS – Setup B Variable Method h=1h=4h=8h=12 h=16 h=20 h=24 Real GDP RIDGE 1.11 1.08 1.16 1.06 0.90 0.89 0.94 RIDGE-GLS 1.16 1.00 0.99 1.00 0.93 0.93 0.95 LP 1.00 1.14 1.37 1.52 1.72 1.98 2.24 BVAR-CV 0.90 0.87 1.04 1.01 0.92 0.92 0.98 H-BVAR 0.83 0.62 0.78 0.73 0.62 0.62 0.68 Investment RIDGE 1.49 1.27 1.17 0.99 0.70 0.73 1.61 RIDGE-GLS 1.34 1.14 1.02 1.02 0.86 0.82 0.86 LP 1.00 1.15 1.40 1.63 2.03 2.76 3.59 BVAR-CV 1.51 1.01 0.97 0.97 0.93 1.08 1.24 H-BVAR 1.06 0.68 0.69 0.66 0.63 0.87 1.14 Fed funds RIDGE 2.17 1.21 0.96 0.93 1.03 4.00 53.18 rate RIDGE-GLS 1.21 1.04 0.90 0.93 0.90 0.88 0.91 LP 1.00 1.18 1.51 1.71 1.97 2.44 2.99 BVAR-CV 0.92 0.94 0.91 0.90 0.86 0.87 0.92 H-BVAR 0.75 0.77 1.32 1.38 1.25 1.15 1.20 revert to zero only over long horizons, while lag-adapted ridge estimates yields models with lower persistence and thus flatter impulse responses. Second, the dataset from Hansen (2016b) is not normalized, and the included series have markedly heterogenous variances. Since GLS ridge shrinks along covariance-rotated data, shrinkage is adjusted according to each series variance, unlike that baseline ridge estimator  𝜷R. The MSE for the Fed Fund Rate impulse responses shows that the pointwise difference between baseline and GLS ridge can be severe for long horizon IRFs when the DGP is highly persistent. On short horizons, Bayesian estimators perform on par or better than baseline least squares estimates, while at longer horizons differences are less stark. It is, however, clear that the hierarchical prior BVAR of Giannone et al. (2015) shows the overall best results. As in the previous setup, local projections show degrading performance at larger horizons. Remark 7. The comparison between methods in both Setup A and Setup B is largely consistent with the findings of Li et al. (2024), who make extensive computational simulations by simulating from synthetic DGPs. They provide a comprehensive treatment of the question of which model – VAR or LP – is best suited for IRF inference in a given scenario in terms of bias-variance trade-off. They show that a key balance of bias vs. variance exists between LP and VAR estimates of impulse responses: LPs tend to have low bias, due to their flexibility, but they also feature large variance at higher horizons. Their results allow one to better understand the trade-offs at play in Tables II and III. In particular, it is clear that ridge shrinkage is beneficial at short horizons only if the penalization scheme is well-adapted to the DGP at hand. Otherwise, as is the case for RIDGE and RIDGE-GLS methods, the induced bias can be such that ridge MSEs surpass that of OLS estimates. One also finds that the medium and long horizons MSE gains over LPs are more pronounced in cases of moderate dependence, but in the case of the Federal Funds Rate IRFs in Setup B zero-centered RIDGE estimates thoroughly mistake long-term behavior. 7.3. Confidence Intervals I now try and evaluate whether ridge shrinkage has a negative impact on inference. There have also been recent contributions directly aimed at studying shrinkage effects. Using the same simulation setups as in the previous section, I investigate coverage and size properties of pointwise CIs constructed using the methods in Table I.All confidence intervals are constructed with nominal 90% level coverage. In this set of simulations, I swap GLS ridge for the asymptotic shrinkage ridge estimator,  𝜷R as, see Section 6, since the latter allows for a partially non-negligible penalization in the limit. To implement  𝜷R as, one needs to wileyonlinelibrary.com/journal/jtsa © 2024 The Authors. J. Time Ser. Anal. 46: 235–257 (2025) Journal of Time Series Analysis published by John Wiley & Sons Ltd. DOI: 10.1111/jtsa.12737 RIDGE REGULARIZED ESTIMATION OF VAR MODELS 253 Table IV. Impulse response inference – SetupA–CIcoverage Variable Method h=1h=4h=8h=12 h=16 h=20 h=24 Investment LS 0.88 0.88 0.87 0.88 0.91 0.93 0.94 growth RIDGE 0.90 0.92 0.94 0.93 0.94 0.95 0.95 RIDGE-AS 0.90 0.92 0.88 0.88 0.88 0.89 0.89 LP 0.88 0.97 0.99 0.99 0.99 0.99 0.99 BVAR-CV 0.77 0.88 0.88 0.90 0.92 0.94 0.96 H-BVAR 0.72 0.89 0.89 0.92 0.93 0.95 0.96 Deflator LS 0.88 0.87 0.86 0.88 0.91 0.92 0.94 RIDGE 0.91 0.92 0.93 0.92 0.93 0.94 0.95 RIDGE-AS 0.91 0.91 0.88 0.88 0.87 0.87 0.88 LP 0.88 0.97 0.99 0.99 0.99 0.99 1.00 BVAR-CV 0.80 0.86 0.88 0.91 0.93 0.94 0.96 H-BVAR 0.84 0.88 0.90 0.92 0.94 0.95 0.97 Paper rate LS 0.87 0.86 0.86 0.88 0.90 0.92 0.94 RIDGE 0.90 0.91 0.93 0.93 0.93 0.94 0.95 RIDGE-AS 0.89 0.90 0.89 0.88 0.88 0.88 0.88 LP 0.87 0.97 0.99 0.99 0.99 0.99 0.99 BVAR-CV 0.82 0.84 0.87 0.90 0.92 0.93 0.95 H-BVAR 0.88 0.88 0.90 0.92 0.93 0.95 0.96 choose a partition of 𝜷which identifies asymptotically negligible coefficient. To do this, I split 𝜷by lag and penalize all coefficients with lag orders greater than a given threshold p, such that 1 <p<p.InsetupA,I choose p=6, while in setup B I set p=3. In Bayesian methods, including the cross-validated Minnesota BVAR, I construct high-probability intervals by drawing from the posterior. Comparison between frequentist CIs and Bayesian posterior densities is not generally valid, because they are not analogous concepts. Therefore, the discussion below is intended to highlight differences in structure between ridge approaches. 7.3.1. Setup A Simulations with the DGP of Kilian and Kim (2011), presented in Tables IV and V, highlight some of the advantages of applying ridge when performing inference. Focusing on estimator  𝜷R, it is clear that CI coverage is in fact higher than the intervals obtained by least squares estimation in all situations. At impact, ridge CIs are larger than the LS baseline, but they shrink as horizons increase. Thus, is IRFs revert relatively quickly to zero, ridge can effectively reduce length while preserving coverage. As discussed in Section 3, these gains are inherently local to the DGP – shrinkage to zero at deep lags embodies correct prior knowledge of a weakly persistent process. For Bayesian estimators, one can note that quantile intervals at small horizons tend to be shorter compared to least squares and ridge methods. 7.3.2. Setup B The effects of ridge shrinkage on a DGP with high persistence are much more severe, as shown in Tables VI and VII. Focusing on frequentist ridge, one can observe that close to impact (h=1) ridge has similar or even higher coverage than other methods for real GDP10 However, as the IRF horizon grows, shrinkage often leads to severe undercoverage, with asymptotic shrinkage estimator  𝜷R as giving the worst results. In comparison, Bayesian methods are much more reliable at all horizons, although the only estimator that can consistently improve on the benchmark least squares VAR CIs is the hierarchical prior BVAR of Giannone et al. (2015). The reason behind this is simple enough: the implementation of the Minnesota-prior BVAR I have used here has a white noise prior on all variables, which in this case is far from the truth. Indeed, Ba´ nbura et al. (2010) implement the same BVAR by tuning the 10 This also is the case with consumption and compensation, see also Tables 9 and 10 in Supplementary Appendix D.5. J. Time Ser. Anal. 46: 235–257 (2025) © 2024 The Authors. wileyonlinelibrary.com/journal/jtsa DOI: 10.1111/jtsa.12737 Journal of Time Series Analysis published by John Wiley & Sons Ltd. 254 G. BALLARIN Table V. Impulse response inference – SetupA–CIlength Variable Method h=1h=4h=8h=12 h=16 h=20 h=24 Investment LS 2.99 5.11 5.78 5.35 4.79 4.17 3.56 growth RIDGE 3.13 5.20 5.82 5.17 4.48 3.78 3.09 RIDGE-AS 3.11 5.15 4.84 4.33 3.70 3.06 2.48 LP 2.99 7.50 10.97 12.89 13.99 14.55 14.70 BVAR-CV 2.84 4.48 4.70 4.38 3.99 3.56 3.11 H-BVAR 2.71 4.20 4.50 4.29 3.96 3.56 3.13 Deflator LS 1.19 1.92 2.23 2.14 1.94 1.71 1.46 RIDGE 1.24 1.97 2.25 2.09 1.84 1.54 1.26 RIDGE-AS 1.24 1.95 1.95 1.78 1.52 1.25 1.01 LP 1.19 3.03 4.56 5.42 5.90 6.14 6.21 BVAR-CV 1.03 1.69 1.87 1.80 1.67 1.50 1.31 H-BVAR 1.01 1.64 1.83 1.79 1.67 1.51 1.33 Paper rate LS 0.97 1.42 1.64 1.57 1.44 1.27 1.09 RIDGE 1.01 1.44 1.65 1.53 1.36 1.16 0.95 RIDGE-AS 1.01 1.43 1.42 1.31 1.13 0.94 0.77 LP 0.97 2.19 3.28 3.90 4.26 4.43 4.48 BVAR-CV 0.84 1.22 1.35 1.30 1.21 1.09 0.96 H-BVAR 0.85 1.21 1.34 1.31 1.22 1.10 0.97 Table VI. Impulse response inference – Setup B: CI coverage Variable Method h=1h=4h=8h=12 h=16 h=20 h=24 Real GDP LS 0.87 0.81 0.75 0.72 0.71 0.72 0.73 RIDGE 0.90 0.79 0.66 0.62 0.65 0.68 0.68 RIDGE-AS 0.89 0.72 0.61 0.58 0.61 0.65 0.65 LP 0.87 0.93 0.94 0.94 0.93 0.93 0.91 BVAR-CV 0.70 0.71 0.63 0.64 0.71 0.75 0.76 H-BVAR 0.84 0.86 0.76 0.76 0.83 0.88 0.88 Investment LS 0.87 0.82 0.76 0.73 0.75 0.82 0.87 RIDGE 0.85 0.79 0.65 0.62 0.73 0.80 0.81 RIDGE-AS 0.82 0.69 0.59 0.57 0.68 0.77 0.77 LP 0.87 0.94 0.94 0.95 0.94 0.94 0.94 BVAR-CV 0.70 0.73 0.67 0.71 0.77 0.81 0.83 H-BVAR 0.80 0.86 0.81 0.82 0.87 0.88 0.88 Fed funds LS 0.85 0.83 0.80 0.78 0.77 0.79 0.80 rate RIDGE 0.79 0.77 0.74 0.68 0.68 0.72 0.72 RIDGE-AS 0.78 0.66 0.68 0.64 0.64 0.68 0.69 LP 0.85 0.94 0.96 0.96 0.95 0.94 0.93 BVAR-CV 0.76 0.72 0.76 0.77 0.77 0.81 0.83 H-BVAR 0.87 0.86 0.74 0.73 0.78 0.84 0.87 wileyonlinelibrary.com/journal/jtsa © 2024 The Authors. J. Time Ser. Anal. 46: 235–257 (2025) Journal of Time Series Analysis published by John Wiley & Sons Ltd. DOI: 10.1111/jtsa.12737 RIDGE REGULARIZED ESTIMATION OF VAR MODELS 255 Table VII. Impulse response inference – Setup B: CI length (rescaled ×100) Variable Method h=1h=4h=8h=12 h=16 h=20 h=24 Real GDP LS 0.71 1.56 2.07 2.31 2.32 2.24 2.15 RIDGE 0.79 1.56 1.85 1.95 1.92 1.85 1.77 RIDGE-AS 0.74 1.31 1.65 1.76 1.75 1.70 1.64 LP 0.71 2.42 4.21 5.40 5.90 5.91 5.70 BVAR-CV 0.53 1.23 1.74 2.00 2.10 2.13 2.15 H-BVAR 0.58 1.36 1.87 2.16 2.32 2.44 2.55 Investment LS 3.38 6.65 7.89 7.89 7.31 6.69 6.18 RIDGE 3.79 6.81 6.93 6.46 5.79 5.19 4.73 RIDGE-AS 3.59 5.57 6.11 5.77 5.21 4.72 4.34 LP 3.37 10.16 16.00 18.85 19.06 18.22 17.23 BVAR-CV 2.64 5.26 6.59 6.91 6.78 6.57 6.38 H-BVAR 2.89 5.74 7.08 7.54 7.63 7.60 7.58 Fed funds LS 0.25 0.39 0.43 0.43 0.41 0.38 0.35 rate RIDGE 0.29 0.39 0.37 0.36 0.33 0.30 0.29 RIDGE-AS 0.27 0.31 0.33 0.32 0.30 0.28 0.27 LP 0.25 0.59 0.88 1.01 1.05 1.03 0.98 BVAR-CV 0.21 0.31 0.36 0.37 0.36 0.35 0.34 H-BVAR 0.23 0.36 0.42 0.44 0.45 0.45 0.46 prior to a random walk for very persistent variables in their applications. In this sense, the cross-validated BVAR considered – which is assumed centered at zero – is really the flip-side of ridge estimators. Therefore, the addition of a prior on the mean of the autoregressive parameters as done by Giannone et al. (2015) is a key element to perform shrinkage in high persistence setups in a way that does not systematically undermine asymptotic inference on impulse responses. 8. CONCLUSION In this article, I have studied ridge regression and its application to vector autoregressive model estimation in detail. This appears to be the first work that provides a thorough analysis of ridge penalization in the context of time series data, including geometric as well as asymptotic properties. I have also derived results on the validity of cross-validation as a method to select the penalty intensity in practice, and I have shown that CV produces asymptotically valid penalization rates. Finally, I have compared both frequentist and Bayesian ridge formulation in simulations aimed at quantifying the applicability of ridge for impulse response inference. The key takeaway of this article is that ridge penalization is a useful approach to VAR estimation as long as the chosen penalty structure is well-adapted to the model’s structure. Bayesian ridge posteriors are especially flexible, with hierarchical priors also allowing shrinkage toward non-zero coefficient vectors. However, it is important to note that the Bayesian approach also permits the researcher to specify uninformative priors, so that the influence of the priors’ hyperparameters is less pronounced. This is not the case in frequentist ridge, cf. including an explicit non-zero centering vector. However, prior knowledge or a pre-estimation procedure may be available to the researcher, so that ridge can be effectively implemented without the need to implement aBVAR. To conclude, there are still avenues of research regarding ridge which would be interesting to develop. First and foremost, the high-dimensional setup, for which, however, it seems non-trivial to find a domain of applicability, as discussed in the introduction. Second, a more in-depth analysis of cross-validation, especially in the multi-variate case, would be extremely valuable. Moreover, both the latter and former topics should be jointly addressed in the context of mild cross-sectional dimension growth, i.e., K→∞such that K∕T→𝜌∈(0,1), which is comparable to factor model setups. J. Time Ser. Anal. 46: 235–257 (2025) © 2024 The Authors. wileyonlinelibrary.com/journal/jtsa DOI: 10.1111/jtsa.12737 Journal of Time Series Analysis published by John Wiley & Sons Ltd. 256 G. BALLARIN ACKNOWLEDGEMENTS I am grateful for the comments and suggestions from Lyudmila Grigoryeva, So Jin Lee, Thomasz Olma, Oliver Pfäuti and Mikkel Plagborg-Møller, and the seminar participants at the University of Mannheim, the HKMetrics Workshop and the Young Researchers Workshop on Big and Smart Data Analysis in Finance. I am especially thankful to Claudia Noack for pointing out an important error in a previous version of this article, as well as Jonas Krampe and Carsten Trenkler for their insightful discussions which helped develop this article significantly. Lastly, I wish to thank Peter C. B. Phillips, Atsushi Inoue and many other colleagues for the suggestion to consider adding a formal analysis of cross-validation in the article. The author acknowledges support by the state of Baden–Württemberg through bwHPC. Open Access funding enabled and organized by Projekt DEAL. CONFLICT OF INTEREST STATEMENT The author reports that there are no competing interests to declare. DATA AVAILABILITY STATEMENT The data that support the findings of this study are openly available in GitHub at https://github.com/giob1994 /ridge_var. SUPPORTING INFORMATION Additional Supporting Information may be found online in the supporting information tab for this article. REFERENCES Babii A, Ghysels E, Striaukas J. 2021. Machine learning time series regressions with an application to nowcasting. Journal of Business & Economic Statistics 40(3):1–23. Ba´ nbura M, Giannone D, Reichlin L. 2010. Large Bayesian vector auto regressions. Journal of Applied Econometrics 25(1):71–92. Barnichon R, Brownlees C. 2019. Impulse response estimation by smooth local projections. The Review of Economics and Statistics 101(3):522–530. Bauwens L, Chevillon G, Laurent S. 2023. We modeled long memory with just one lag! Journal of Econometrics 236(1):105467. Bergmeir C, Hyndman RJ, Koo B. 2018. A note on the validity of cross-validation for evaluating autoregressive time series prediction. Computational Statistics & Data Analysis 120:70–83. Boubacar Mainassara Y, Francq C. 2011. Estimating structural VARMA models with uncorrelated but non-independent error terms. Journal of Multivariate Analysis 102(3):496–505. Brockwell PJ, Davis RA. 1991. Time Series: Theory and Methods New York: Springer Science +Business Media. Burman P, Chow E, Nolan D. 1994. A cross-validatory method for dependent data. Biometrika 81(2):351–358. Cavaliere G, Gonçalves S, Nielsen MØ, Zanelli E. 2023. Bootstrap inference in the presence of bias. Journal of the American Statistical Association 1–11. https://doi.org/10.1080/01621459.2023.2284980 Davidson J. 1994. Stochastic Limit Theory: An Introduction for Econometricians New York: Oxford University Press. De Mol C, Giannone D, Reichlin L. 2008. Forecasting using a large number of predictors: Is Bayesian shrinkage a valid alternative to principal components? Journal of Econometrics 146(2):318–328. Dobriban E, Wager S. 2018. High-dimensional asymptotics of prediction: ridge regression and classification. Annals of Statistics 46(1):247–279. Fu W, Knight K. 2000. Asymptotics for lasso-type estimators. The Annals of Statistics 28(5):1356–1378. Fuleky P (ed.). 2020. Macroeconomic Forecasting in the Era of Big Data: Theory and Practice.Advanced Studies in Theoretical and Applied Econometrics, Vol. 52 Springer International Publishing, Cham. Ghosh S, Khare K, Michailidis G. 2019. High-dimensional posterior consistency in bayesian vector autoregressive models. Journal of the American Statistical Association 114(526):735–748. Giannone D, Lenza M, Primiceri GE. 2015. Prior selection for vector autoregressions. The Review of Economics and Statistics 97(2):436–451. Goulet Coulombe P. 2023. Time-varying parameters as ridge regressions. Working Paper. wileyonlinelibrary.com/journal/jtsa © 2024 The Authors. J. Time Ser. Anal. 46: 235–257 (2025) Journal of Time Series Analysis published by John Wiley & Sons Ltd. DOI: 10.1111/jtsa.12737 RIDGE REGULARIZED ESTIMATION OF VAR MODELS 257 Goulet Coulombe P, Leroux M, Stevanovic D, Surprenant S. 2022. How is machine learning useful for macroeconomic forecasting? Journal of Applied Econometrics 37(5):920–964. Hamilton JD. 1994. Time Series Analysis Princeton, NJ: Princeton University Press. Hansen BE. 2016a. Efficient shrinkage in parametric models. Journal of Econometrics 190(1):115–132. Hansen BE. 2016b. Stein combination shrinkage for vector autoregressions. Working Paper. Hastie T. 2020. Ridge Regularization: an essential concept in data science. Technometrics 62(4):426–433. Hastie T, Tibshirani R, Friedman JH, Friedman JH. 2009. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, second ed. New York: Springer. Hastie T, Montanari A, Rosset S, Tibshirani RJ. 2022. Surprises in high-dimensional ridgeless least squares interpolation. The Annals of Statistics 50(2):949–986. Hausman JA. 1983. Chapter 7: specification and estimation of simultaneous equation models.InHandbook of Econometrics, Vol. 1, Amsterdam: Elsevier; 391–448. Heaps SE. 2023. Enforcing stationarity through the prior in vector autoregressions. Journal of Computational and Graphical Statistics 32(1):74–83. Huber F, Koop G. 2023. Subspace shrinkage in conjugate Bayesian vector autoregressions. Journal of Applied Econometrics 38(4):556–576. Inoue A, Kilian L. 2008. How useful is bagging in forecasting economic time series? a case study of u.s. consumer price inflation. Journal of the American Statistical Association 103(482):511–522. Jordà Ò. 2005. Estimation and inference of impulse responses by local projections. American Economic Review 95(1):161–182. Kadiyala KR, Karlsson S. 1997. Numerical methods for estimation and inference in bayesian var-models. Journal of Applied Econometrics 12(2):99–132. Kilian L, Kim YJ. 2011. How reliable are local projection estimators of impulse responses? Review of Economics and Statistics 93(4):1460–1466. Kilian L, Lütkepohl H. 2017. Structural Vector Autoregressive Analysis Cambridge: Cambridge University Press. Li D, Plagborg-Møller M, Wolf CK. 2024. Local projections vs. vars: lessons from thousands of DGPs. Working Paper. Litterman RB. 1986. Forecasting with Bayesian vector autoregressions five years of experience. Working Papers No.274, Federal Reserve Bank of Minneapolis. Liu S, Dobriban E. 2020. Ridge Regression: Structure, Cross-Validation, and Sketching. In International Conference on Learning Representations. Lütkepohl H. 1990. Asymptotic distributions of impulse response functions and forecast error variance decompositions of vector autoregressive models. The Review of Economics and Statistics 72(1):116–125. Lütkepohl H. 2005. New Introduction to Multiple Time Series Analysis Heidelberg: Springer Berlin. Medeiros MC, Vasconcelos GFR, Veiga Á, Zilberman E. 2021. Forecasting inflation in a data-rich environment: the benefits of machine learning methods. Journal of Business & Economic Statistics 39(1):98–119. Mikusheva A. 2007. Uniform inference in autoregressive models. Econometrica 75(5):1411–1452. Mikusheva A. 2012. One-dimensional inference in autoregressive models with the potential presence of a unit root. Econometrica 80(1):173–212. Park JY, Phillips PC. 1988. Statistical inference in regressions with integrated processes: part 1. Econometric Theory 4(3):468–497. Park JY, Phillips PC. 1989. Statistical inference in regressions with integrated processes: part 2. Econometric Theory 5(1):95–131. Patil P, Wei Y, Rinaldo A, Tibshirani R. 2021. Uniform consistency of cross-validation estimators for high-dimensional ridge regression. In Banerjee A, & Fukumizu, K (Eds.), Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR; 3178–3186. Pesavento E, Rossi B. 2006. Small-sample confidence intervals for multivariate impulse response functions at long horizons. Journal of Applied Econometrics 21(8):1135–1155. Phillips PCB. 1988. Regression theory for near-integrated time series. Econometrica 56(5):1021–1043. Plagborg-Møller M. 2016. Essays in Macroeconometrics. PhD thesis, Harvard University. Pratt JW. 1961. Length of confidence intervals. Journal of the American Statistical Association 56(295):549–567. Sims CA, Stock JH, Watson MW. 1990. Inference in linear time series models with some unit roots. Econometrica: Journal of the Econometric Society 58(1):113–144. Smeekes S, Wijler E. 2018. Macroeconomic forecasting using penalized regression methods. International Journal of Forecasting 34(3):408–430. Stephenson W, Frangella Z, Udell M, Broderick T. 2021. Can we globally optimize cross-validation loss? Quasiconvexity in ridge regression. In Ranzato M, Beygelzimer A, Dauphin Y, Liang PS & Wortman Vaughan J. Advances in Neural Information Processing Systems, Vol. 34, Curran Associates, Inc; 24352–24364. J. Time Ser. Anal. 46: 235–257 (2025) © 2024 The Authors. wileyonlinelibrary.com/journal/jtsa DOI: 10.1111/jtsa.12737 Journal of Time Series Analysis published by John Wiley & Sons Ltd.