Nonparametric regression estimation for multivariate null recurrent processes

Cai, Biqing; Tjøstheim, Dag Article Nonparametric regression estimation for multivariate null recurrent processes Econometrics Provided in Cooperation with: MDPI – Multidisciplinary Digital Publishing Institute, Basel Suggested Citation: Cai, Biqing; Tjøstheim, Dag (2015) : Nonparametric regression estimation for multivariate null recurrent processes, Econometrics, ISSN 2225-1146, MDPI, Basel, Vol. 3, Iss. 2, pp. 265-288, https://doi.org/10.3390/econometrics3020265 This Version is available at: https://hdl.handle.net/10419/171827 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by/4.0/ Econometrics 2015,3, 265-288; doi:10.3390/econometrics03020265 OPEN ACCESS econometrics ISSN 2225-1146 www.mdpi.com/journal/econometrics Article Nonparametric Regression Estimation for Multivariate Null Recurrent Processes Biqing Cai and Dag Tjøstheim * Department of Mathematics, University of Bergen, 5020 Bergen, Norway; E-Mail: [email protected].no *Author to whom correspondence should be addressed; E-Mail: [email protected].no; Tel.: +47-55582885. Academic Editor: Timo Teräsvirta Received: 26 November 2014 / Accepted: 2 April 2015 / Published: 14 April 2015 Abstract: This paper discusses nonparametric kernel regression with the regressor being ad-dimensional β-null recurrent process in presence of conditional heteroscedasticity. We show that the mean function estimator is consistent with convergence rate pn(T)hd, where n(T)is the number of regenerations for a β-null recurrent process and the limiting distribution (with proper normalization) is normal. Furthermore, we show that the two-step estimator for the volatility function is consistent. The finite sample performance of the estimate is quite reasonable when the leave-one-out cross validation method is used for bandwidth selection. We apply the proposed method to study the relationship of Federal funds rate with 3-month and 5-year T-bill rates and discover the existence of nonlinearity of the relationship. Furthermore, the in-sample and out-of-sample performance of the nonparametric model is far better than the linear model. Keywords: β-null recurrent; cointegration; conditional heteroscedasticity; Markov chain; nonparametric regression JEL classifications: C13; C14; C22; E43 Econometrics 2015,3266 1. Introduction The interplay of nonlinearity and nonstationarity has been an important topic in recent developments of econometrics. Karlsen and Tjøstheim [1] and Karlsen et al. [2], respectively, discuss the asymptotics for nonparametric estimation of autoregression and cointegrating regression when the regressor is a β-null recurrent Markov process. Using different data generating assumptions (i.e., the regressor is a unit root process with innovations being a linear process), Wang and Phillips [3,4] discuss asymptotics for nonparametric estimation for nonlinear cointegrating regression models. The two frameworks have their own advantages and drawbacks. The β-null recurrence framework generalizes the unit root framework by incorporating more kinds of processes than the unit root process although it encounters some other restrictions. For example, the processes need to be Markov. For more discussion of linkage and difference of these two frameworks, we refer to [5]. The papers [1–4] focus on nonparametric estimation when the regressor is a univariate process. The issue of estimation when the regressor is a multivariate process has received less attention. As argued by Park and Phillips [6], the difficulty of extending the theory from a univariate regressor to a multivariate regressor is due to the fact that the recurrence property of a higher dimensional random walk process is different from the one dimensional random walk. Dong et al. [7] provide an intuitive example showing that the nonparametric estimate, when the regressor is a bivariate independent random walk, is not consistent. One way to avoid this problem is to use semi-parametric models rather than nonparametric models. For example, Schienle [8] considers an additive model rather than a pure nonparametric model to avoid this problem while in [9] a partial linear model is considered. However, nonparametric estimation is still possible in the multivariate case when the regressors are not independent random walks. Gao and Phillips [10] provide the theory of nonparametric estimation for multivariate regressors when one regressor is a unit root process and the other regressors are stationary processes. The reason why the model setup of [10] works in nonparametric estimation while the two-dimensional random walk does not is due to the fact that a one-dimensional random walk together with a multi-dimensional positive recurrent process form a 1/2-null recurrent system while the two dimensional random walk process is null recurrent but not β-null recurrent for any β∈(0,1). As discussed in [1], the β-null recurrence property plays a vital role to guarantee validity of nonparametric estimates. In this paper, we introduce the theory of nonparametric estimation for a multivariate β-null recurrent system. The multivariate β-null recurrent processes include but are not restricted to the case of [10]. For example, our theory can cover the case where two regressors are both random walks but at the same time are cointegrated which is not covered by [10]. This will be discussed in more detail in Section 2. The cointegrated case is of importance in economics because it is well known that many macroeconomic time series are nonstationary but cointegrated such that they are driven by a common stochastic trend. Furthermore, in this paper, we use different mathematical techniques compared to [10]. In their paper, the technique of local time approximation for partial sums of functionals of unit root process is used, while in our paper, we use the Markov chain null recurrence framework. It is well known that for the nonparametric kernel estimation for a d-dimensional stationary process, the convergence rate is √Thdwith hbeing the bandwidth. In our model, the convergence rate is pn(T)hd, where n(T)is the number of regenerations of the β-null recurrent process which Econometrics 2015,3267 is discussed in more detail in [1] (see also Appendix A). The difference of these two rates is due to the fact that for β-null recurrent processes, the number of observations in a small set (we refer to p. 376 in [1] for the definition and discussion of the small set) is OP(TβLs(T)) rather than OP(T)(in the stationary case), where Ls(.)denotes a function slowly varying at infinity (cf. p. 6, [11]). Furthermore, unlike [2], which assumes the regression error has constant variance, we allow existence of conditional heteroscedasticity. This is important for economic or financial time series modeling because many of these series are regarded to contain conditional heteroscedasticity (cf. [12–14]). In [15,16], estimation of conditional variance functions in autoregressive and regression models are discussed when the data are stationary. Wang and Wang [17] discuss nonparametric estimation of conditional mean and variance function when the regressor is a unit root process. Our paper is different from [17] in two ways: first, in our model, the regressor is multivariate rather than univariate; second, we employ the Markov β-null recurrence technique which is different from the local time approximation technique used in [17]. The rest of this paper is organized as follows. In Section 2, we introduce the model and the nonparametric estimate; in Section 3, the asymptotic properties for the estimator will be provided; in Section 4, Monte Carlo simulations will be conducted to examine the finite sample performance of the estimator; in Section 5, we apply the method to estimate the relationship of Federal funds rate with short term and long term T-bill rates; in Section 6, concluding remarks are made. To make this paper self-contained, we provide some basic notations and theory of Markov processes, especially the β-null recurrent processes in Appendix A. The mathematical proofs are contained in Appendix B. Throughout this paper, all limits are taken “as T→ ∞” where Tis the sample size, →ddenotes weak convergence, →pdenotes convergence in probability. OP(.)means stochastic order same as, oP(.) means stochastic order less than. 2. Model and Estimation We are going to discuss estimation for the following model yt=g(xt) + εt,(1) where t= 1, . . . , T,{xt}={x1,t, . . . , xd,t}τis a d-dimensional β-null recurrent process (see Appendix A for a precise definition), εt=σ(xt)etwith {et}being a positive recurrent process and σ(.)being a positive function. When the data are stationary, the model (1) has been widely studied, see, e.g., [15,16] (with univariate regressor) and [18] (with multivariate regressor). Recently, Wang and Wang [17] study the estimation of model (1) with {xt}being a univariate unit root process. As we have mentioned in the introduction, our paper has important differences from their paper. The examples of univariate β-null recurrent processes with β= 1/2include the random walk process with the innovation having second moment (cf. [19]); the threshold unit root model (cf. [20]) with arbitrary behavior in a compact set and unit root behavior outside the compact set. Moreover, under some regularity conditions, it has been shown that several multivariate Markov processes are β-null recurrent. For example, when d= 2, the following models of {x1,t, x2,t}are 1/2-null recurrent: Econometrics 2015,3268 •Example 1:{x1,t}is a 1/2-null recurrent process and {x2,t}is a positive recurrent process independent of {x1,t}. This is proved by Lemma 3.1 of [2]. In fact, the independence assumption of {x1,t}and {x2,t}can be relaxed to asymptotic independence. We refer to Example 4.1 of [2] for more discussion of this. •Example 2:{x1,t}and {x2,t}are both unit root processes and cointegrated. More specifically xt=Axt−1+et, where {et}is a bivariate i.i.d. process satisfying some regularity conditions, Ais a 2×2 matrix having one eigenvalue equal to 1 and the other eigenvalue with absolute value less than 1. Myklebust et al. [5] show that in this model, {xt}is 1/2-null recurrent. •Example 3:{xt}can be threshold cointegrated process. Consider the model xt=Axt−1I(xt−1∈Cc) + Bxt−1I(xt−1∈C) + et, where Cis a compact set in R2,Ais a 2×2matrix having one eigenvalue equal to 1 and the other eigenvalue with absolute value less than 1, Bis an arbitrary matrix and {et}is a bivariate i.i.d. process satisfying some regularity conditions. Cai et al. [21] prove that {xt}is 1/2-null recurrent under this model setup. •Example 4:{xt}is generated from x2,t =f(x1,t) + ut, with {x1,t}being a 1/2-recurrent process and {ut}being an i.i.d. sequence and independent of {x1,t}. This is the nonlinear cointegration type model of [2]. Remark 1: The cases discussed can be extended to dimension higher than 2. For example, Myklebust et al. [5] show that a d-dimensional VAR(1) model is 1/2-null recurrent if the autoregressive matrix has one eigenvalue equal to 1 and the other eigenvalues with absolute values less than 1. By Theorem 2 of [5], it is shown that one-to-one transformation of a β-null recurrent process is also a β-null recurrent process. Remark 2: We can see that the model of [10] is related to Example 1. Our methodology can be applied to other models listed above. We propose to estimate the functional form g(x)at (x1,··· , xd)by the conventional local constant method through minimizing 1 T T X t=1 (yt−α)2K(xt−x h)over α=g(x1,··· , xd),(2) where K(xt−x h) = Πd i=1ki(xi,t−xi h)with ki(.)being univariate kernel functions and hbeing the bandwidth parameter1. 1In this paper, we use same bandwidth for different regressors. This is a little bit restrictive in practice. We leave the case of different choices of bandwidths for different regressors to future research. Econometrics 2015,3269 Equation (2) implies the resulting estimate is given by bg(x) = PT t=1 K(xt−x h)yt PT t=1 K(xt−x h).(3) So that we have bg(x)−g(x) = PT t=1 K(xt−x h)(g(xt)−g(x)) PT t=1 K(xt−x h)+PT t=1 K(xt−x h)εt PT t=1 K(xt−x h)(4) ≡I1+I2, where I1and I2are respectively the bias and variance terms for the nonparametric estimate. 3. Asymptotic Theory To study the asymptotics for the estimate (3), we need to introduce some technical assumptions. A.1. {xt}={x1,t,··· , xd,t}is a d-dimensional Harris β-null recurrent Markov chain. Let πs(.)be an invariant measure of the recurrent process admitting a locally Lipschitz continuous density ps(.) which is locally bounded. σ(.)is a locally bounded, positive and Lipschitz continuous function such that for a vector x, there exists a constant Csuch that when yis in a neighbourhood of x, we have |σ(y)−σ(x)|< C kx−yk, where k.kis the Euclidean norm. {et}is an i.i.d. sequence independent of {xt}with E(e1) = 0,E(e2 1)=1and E(|e1|4+δ)<∞for some δ > 0. A.2. For any given x,g(x)is twice continuously differentiable and the second order partial derivatives are locally bounded and Lipschitz continuous, i.e.,|∂2g(x) ∂xixj−∂2g(y) ∂xixj|< C kx−ykwhen yis in a neighbourhood of x,i= 1,··· , d and j= 1,··· , d. A.3. For i= 1,··· , d, each kiis a symmetric, nonnegative and bounded probability density function with compact support. Furthermore, the support of the kernel functions are small sets. A.4. h→0as T→ ∞,u(T)hd→ ∞ as T→ ∞, and h2(pu(T)hd)→0as T→ ∞, where u(T) = TβLs(T), see Appendix A (cf. [1]), which is associated with n(T), where n(T)is the number of regenerations for the null recurrent Markov chain. Remark: Assumption A.1 restricts the regressors to be a β-null recurrent system with some of the examples having been given in Section 2. The assumption on the error term is quite common in the literature, see, e.g., [18]. As shown in Lemma 3.1 of [2], the compound process {xt, et}is also a β-null recurrent process. It is possible to relax the assumption on {et}such that endogeneity and autocorrelation are involved by applying some results in [2]. For the current paper, we use this assumption for illustrative purpose. Assumptions A.2 and A.3 are often used in nonparametric kernel estimation problems. We assume the support of the kernel function to be compact and a small set as in [1]. The bandwidth restriction in Assumption A.4 ensures that the nonparametric estimator is consistent and the estimation bias converges to 0 in probability. To derive the asymptotic theory for the nonparametric estimator, we need the following three lemmas. Econometrics 2015,3270 Lemma 1. Under assumptions A.1–A.4, 1 n(T)hd T X t=1 K(xt−x h)→pps(x). Remark. Lemma 1 shows that the denominator of the nonparametric estimator converges to an invariant density of the null recurrent process which is different from the positive recurrent case where the denominator converges to the probability density of the process. In Example 1 of Section 2, ps(x) = ps1(x1)×ps2(x2), where ps1(x1)is an invariant density of {x1,t}and ps2(x2)is the (unique) invariant stationary density of {x2,t}. Lemma 2. Under assumptions A.1–A.4, 1 pn(T)hd T X t=1 K(xt−x h)εt→dN(0, ps(x)σ2(x)ZK2(u)du), where N(., .)denotes a normal variable and RK2(u)du = ΠiRk2 i(v)dv. Lemma 3. Under assumptions A.1–A.4, 1 n(T)hd T X t=1 K(xt−x h)(g(xt)−g(x)) = OP(h2). After proving Lemmas 1–3, we can derive the asymptotic distribution for bg(x)−g(x). We have the following theorem Theorem 1. Under assumptions A.1–A.4, pn(T)hd(bg(x)−g(x)) →dN(0, p−1 s(x)σ2(x)ZK2(u)du). Remark 1. In this theorem, stochastic normalization is used. As suggested by Equation (A.3) in Appendix A, we also have pu(T)hd(bg(x)−g(x)) →dMN(0, M−1 β(1)p−1 s(x)σ2(x)ZK2(u)du), where MN(.)denotes a mixed-normal variable. See also the discussion in [1]. Remark 2. Combining this theorem and Lemma 1, we have v u u tT X t=1 K(xt−x h)(bg(x)−g(x)) →dN(0, σ2(x)ZK2(u)du). Econometrics 2015,3271 This self-normalization quantity is the same as that used in [2]. RK2(u)du is known when a specific kernel function is used2. So that for statistical inference purpose or construction of confidence band, we need to estimate σ2(x). In [15,16], different methods for the variance estimation are proposed. Similar to [4] and [17], we estimate this quantity by a localized version of the usual residual-based method, i.e., bσ2(x) = T P t=1 (yt−bg(x))2K(xt−x hσ) T P t=1 K(xt−x hσ) ,(5) where hσis another bandwidth. This is a two-step estimator which corresponds to local constant regression of the square of the residuals on the regressors. To investigate Equation (5), we impose the following assumption on hσ. A.5. hσsatisfies same restrictions as hin Assumption A.4. Then we have following theorem. Theorem 2. Under assumptions A.1–A.5, bσ2(x)→pσ2(x). Remark 1. Theorem 2 shows that the estimator is consistent. In this paper, we focus on the mean function estimation. The investigation of more efficient estimation of σ2(x)(cf. [16]) is left for future research. Remark 2. Combining Theorem 1 and Theorem 2, and using the Slutsky theorem, we have sPT t=1 K(xt−x h) bσ2(x)RK2(u)du(bg(x)−g(x)) →dN(0,1).(6) Thus we can construct the 95% confidence interval for the mean function estimator as bg(x)±1.96 ×sbσ2(x)RK2(u)du PT t=1 K(xt−x h).(7) 4. Monte Carlo Simulation In this section, we conduct Monte Carlo simulations to evaluate the finite sample performance of the nonparametric estimator. We focus on the case where d= 2. The performance of nonparametric estimation is evaluated by the root mean squared error (RMSE), which is defined by: RMSE =v u u t1 N 1 T T X t=1 N X n=1 (bg(x1,tn, x2,tn)−g(x1,tn, x2,tn))2,(8) 2For the Quartic kernel used in the Monte Carlo simulation, it is equal to (5 7)d. Econometrics 2015,3272 where xi,tnwith i= 1 or 2 denotes the observation at time-tin the n-th replication with total replication number N= 1000. For both variables, the Quartic kernel (i.e.,k(u) = 15 16(1 −u2)21(|u| ≤ 1)) is used. It is well known that the kernel selection plays little role in performance of nonparametric estimation. Bandwidth selection plays an important role in the performance of nonparametric estimates such that a large bandwidth may lead to large bias but small variance and vice versa. From the theoretical analysis, we know that the variance of the estimator is of order (TβLs(T)hd)−1and the square of bias is of order h4. So that the optimal bandwidth minimizing the mean square error (MSE) for the estimator should be of order (T−βL−1 s(T)) 1 4+d. In the simulation, we concentrate on the case β= 1/2and d= 2, so that we report the simulation results with bandwidth being cT−1/12 with some pre-specified values of c. We also report the results when h=h∗, with h∗chosen by the leave-one-out cross validation method, which is widely used when the data is positive recurrent (cf. p. 50, [22]). From the simulation results below, we can find that the method performs quite well even when the regressors are null recurrent. Specifically, we consider following models: Model 1: {x1,t} ∼ i.i.d.N(0,1), x2,t =x2,t−1+ut, with {ut} ∼ i.i.d.N(0,1) yt=g(x1,t, x2,t) + εt, where {εt} ∼ i.i.d.N(0,1) and is independent of {x1,t}and {x2,t}. In this model, {x1,t}is i.i.d., {x2,t}is a random walk and {xt}is a 1/2-null recurrent process because it is a special case of Example 1 in Section 2 (it is also a special case of [10]). We let g(x1, x2) = x1+1 1+x2 2or g(x1, x2) = x1×1 1+x2 2. The results for Model 1 are summarized in Table 1. Table 1. RMSEs for Model 1. Functional Form c T = 200 T = 400 T = 800 1 0.3707 0.2996 0.2427 2 0.3046 0.2678 0.2357 g(x1, x2) = x1+1 1+x2 2 3 0.4104 0.3777 0.3454 4 0.5490 0.5131 0.4770 h∗0.3183 0.2629 0.2192 1 0.3636 0.2915 0.2360 2 0.2456 0.2071 0.1765 g(x1, x2) = x1×1 1+x2 2 3 0.2295 0.2010 0.1750 4 0.2425 0.2111 0.1818 h∗0.2471 0.2054 0.1744 From Table 1, we can see that because of the trade-off between bias and variance, either a too large or a too small bandwidth will make the estimator less precise by the RMSE criterion. The bandwidth selected Econometrics 2015,3279 In summary, the pairwise exploratory phase suggests that the relationship of FF and TB3m and TB5y may not be linear. The nonlinearity may be due to transaction cost as suggested by [28] or the policy interventions as suggested by [30]. Next, we estimate our model (1) by estimating gnonparametrically. We compare the in-sample mean square error of the nonparametric model with the linear model. Define the following MSE MSE =1 T T X t=1 (FFt−d FFt)2, where d FF is the fitted value from the nonparametric model or the linear model (regress FF on TB3m, TB5yand a constant). The MSE for the nonparametric model is 0.0995 while the MSE for the linear model is 0.2802. It can be seen that the nonparametric model outperforms the linear model. However, it is well known that comparison based on the in-sample performance as done above is sensitive to outliers and data mining, see, e.g., [31]. Empirical evidence of out-of-sample forecast performance is generally regarded as more trustworthy. In addition, out-of-sample forecasts can better reflect the information available to forecasters in “real time” (cf. [32]). As emphasized by [33], out-of-sample forecast is the “ultimate test of forecasting model”. We study the out-of-sample forecasts of the nonparametric model and linear model by comparing their one-step-ahead forecasting performance. Specifically, we define the out-of-sample MSE (OMSE) as OMSE =1 m T X t=T−m+1 (FFt−g FFt)2, where g FFtis the fitted value at (TB3mt, TB5yt)with the model estimated using the observations up to time t−16. Furthermore, out-of-sample size mis taken to be [1/10T],[1/15T],[1/20T], where Tis the full sample size. The reason for choosing relatively small proportion of out-of-sample evaluation period is mainly due to the fact that for the nonparametric forecast, because of the nature of local estimate, a too small proportion of in-sample size may make the forecast impossible if the observation of the period we are going to forecast is an “outlier” such that we do not have enough observations in the neighborhood. The results for the out-of-sample forecasts are reported Table 5. Table 5. OMSE Comparison. Model m= [1/10 T] m= [1/15 T] m= [1/20 T] Linear model 0.1018 0.0671 0.0519 Nonparametric model 0.0021 0.0022 0.0015 From Table 5, we can see that the nonparametric model consistently outperforms the linear model by the out-of-sample evaluation. It is also interesting that the OMSE is smaller than the in-sample 6I.e., we use recursive (expanding) windows in estimation. Compared with the rolling-window forecasting which uses fixed in-sample size, the recursive window utilizes all the data when one forecasts for the next period, thus may increase the precision of the in-sample estimation, which in turn, leads to better out-of-sample forecasting. Econometrics 2015,3280 MSE. It is possible because the volatility increases with regressors (see Figure 5 below) and our out-of-sample forecasting periods are the periods where the interest rates are very low as is seen from Figure 2. The comparison of OMSE provides additional strong evidence that there exists nonlinearity in the relationship. 3.7 3.8 3.9 4 4.1 4.2 4.3 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24 TB3m (a) 7.7 7.8 7.9 8 8.1 8.2 8.3 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 TB3m (b) Figure 5. In (a) we estimate the shape of the variance with TB3m when TB5y = 4; in (b) we estimate the shape of the variance with TB3m when TB5y = 8. Similarly, we can estimate the conditional variance function using Equation (5) (with the same bandwidth as that used for the mean function estimation). Figure 5 studies the variance conditional on TB3m when TB5y is fixed. When TB5y is 4, we study shape of the variance with TB3m within [3.7 4.3] and when TB5y is 8, we study the relationship with TB3m within [7.7 8.3]. Figure 5 indicates that there exists conditional heteroscedasticity7. The conditional heteroscedasticity of FF is also found in [34], where a univariate model with ARCH-type (cf. [12]) volatility function is used. Chan et al. [35] study different models for the short term interest rate encompassed in the following stochastic differential equation (SDE): drt= (α+βrt)dt +σrγ tdWt,(9) where rtis the (spot) interest rate, Wtis the standard Brownian motion, α,β,σand γare some constants. When γ= 0, Equation (9) is the Vasicek [36] model and when γ= 1/2, Equation (9) is the famous Cox-Ingersoll-Ross (CIR, [37]) model. We can see that in [36], the process is conditional homoscedastic and when γ6= 0, there exists conditional heteroscedasticity. The empirical findings of [35] suggest that γis significantly larger than 0, so that the volatility increases with the interest rate. Our model setup is regression rather than autoregression. However, the interest rates are cointegrated and move in the same direction, so that the finding of [35] implies the volatility in our model also increases with the regressors. This is consistent with the empirical results in our paper. 7We can also find that the volatility with TB3m fixed is not constant. To save space, we don’t report the results here. Econometrics 2015,3281 6. Conclusions In this paper, we establish the asymptotic theory of nonparametric estimation for multivariate β-null recurrent processes in presence of heteroscedasticity. The Monte Carlo simulation results suggest that the nonparametric estimator performs well using bandwidth selected by the cross validation method. The application to the relationship of Federal funds rate with short term and long term T-bill rates indicates the existence of nonlinearity. In the current paper, the variance function estimation is not discussed in details as for the mean function estimation. This is left for future research. In empirical applications, because of different ranges of different regressors especially in this multivariate nonstationary case, different bandwidths may be used for different regressors to make the estimate more efficient (cf. [18]). Acknowledgements We thank the editor and three anonymous referees for invaluable feedback which greatly improved the structure of the paper. Author Contributions Biqing Cai is the main author of this paper and both authors contribute to this paper. Appendix A. Some Markov Theory To make this paper self-contained, in this Appendix, we introduce necessary notions of Markov chain properties. They are fundamental for deriving the asymptotic theory in this paper. We refer to [1,2] and [38] for more comprehensive treatments. Let {Xt, t ≥0}be a φ-irreducible Markov chain on the state space (E, E)with transition probability P. This means that for any set A∈ E with φ(A)>0, we have ∞ P t=1 Pt(x, A)>0for all x∈E. In this paper, we take E⊆Rd. We further assume that the φ-irreducible Markov chain {Xt}is Harris recurrent. Definition 1. A Markov chain {Xt}is Harris recurrent if, given a neighborhood Nxof xwith φ(Nx)>0, {Xt}returns to Nxwith probability one, for any x∈E. The Harries recurrent chain is positive recurrent if there exists an invariant probability measure such that {Xt}is strictly stationary and is null recurrent otherwise. The Harris recurrence allows one to construct a split chain, which decomposes the partial sum of functions of {Xt}into blocks of i.i.d. parts and two asymptotically negligible remaining parts. Let τkbe the regeneration times, Tthe number of observations and n(T)the number of regenerations as in [1]8. 8They use the notation T(n)instead of n(T). Econometrics 2015,3282 For the process {G(Xt) : t≥0}, defining Uk=                τ0 P t=0 G(Xt), k = 0 τk P t=τk−1+1 G(Xt),1≤k≤n(T), T P t=τn(T)+1 G(Xt), k =n(T)+1 where G(·)is a real function defined on Rd, then we have Sn(G) = T X t=0 G(Xt) = U0+ n(T) X k=1 Uk+Un(T)+1.(A.1) From [38], we know that {Uk, k ≥1}is a sequence of i.i.d. random variables, and U0and Un(T)+1 converge to zero almost surely (a.s.) when they are divided by the number of regenerations n(T)(using Lemma 3.2 in [1]). The general Harris recurrence only yields stochastic rates of convergence in asymptotic theory of the nonparametric estimators, where distribution and size of the number of regenerations n(T)have no a priori known structure but fully depend on the underlying process {Xt}. To obtain a specific rate of n(T)in our asymptotic theory for the null recurrent process, we next impose some restrictions on the tail behavior of the distribution of the recurrence times of the Markov chain. Definition 2. A Markov chain {Xt}is β-null recurrent if there exist a small nonnegative function f, an initial measure λ, a constant β∈(0,1), and a slowly varying function Lf(·)such that Eλ( T X t=1 f(Xt)) ∼1 Γ(1 + β)TβLf(T),(A.2) as T→ ∞, where Eλstands for the expectation with initial distribution λand Γ(1 + β)is the Gamma function with parameter 1 + β. The definition of a small function fin the above definition can be found in some existing literature (cf. p. 15, [38]). Assuming β-null recurrence restricts the tail behavior of the recurrence time of the process to be a regularly varying function. In fact, for all small functions f, by Lemma 3.1 in [1], we can find an Ls(·)such that Equation (A.2) holds for the β-null recurrent Markov chain with Lf(·) = πs(f)Ls(·), where πsis an invariant measure of the Markov chain {Xt},πs(f) = Rf(x)πs(dx)and sis the small function in the minorization inequality (3.4) of [1]. Letting Ls(T) = Lf(T)/(πs(f)) and following the argument in [1], we may show that the regeneration number n(T)of the β-null recurrent Markov chain {Xt}has the following asymptotic distribution n(T) TβLs(T)→dMβ(1),(A.3) where Mβ(1) is the Mittag-Leffler distribution with parameter β(cf. [39]). Since n(T)< T a.s. for the null recurrent case by Equation (A.3), the rates of convergence for the nonparametric kernel estimators are slower than those for the stationary time series case (cf. [2]). We also denote u(T) = TβLs(T), which is used in the main text of this paper. In Section 2 of this paper, some typical examples of multivariate β-null recurrent processes are provided. Econometrics 2015,3283 B. Mathematical Proofs Proof of Lemma 1. According to the split chain technique (cf. (A.1)), we have 1 n(T)hd T X t=1 K(xt−x h) = 1 n(T)hd{U0(Kh) + n(T) X k=1 Uk((Kh)) + Un(T)+1(Kh)}, where Kh= Πd i=1ki(xi,t−xi h). We have for k= 1,··· , n(T) E(Uk(Kh)) = πs(Kh) = Zk1(y1−x1 h)···kd(yd−xd h)ps(y1,··· , yd)dy. Denoted y1−x1 h=z1,···,yd−xd h=zd, then Zk1(y1−x1 h)···kd(yd−xd h)ps(y1,··· , yd)dy =hdZk1(z1)···kd(zd)ps(x1+z1h, ··· , xd+zdh)dz =hdps(x) + o(hd). And similar to the proof of Lemma 3.2 of [1], we have Pλ(|U0(Kh)|<∞) = 1 and Pλ(|Un(T)+1(Kh)|<∞) = 1, where λis an arbitrary initial measure. Then Lemma 1 follows from the weak law of large numbers because n(T)hd→ ∞as T→ ∞. Proof of Lemma 2. We have 1 pn(T)hd T X t=1 K(xt−x h)εt=1 pn(T)hd T X t=1 K(xt−x h)σ(xt)et =1 pn(T)hd T X t=1 K(xt−x h)σ(x)et+1 pn(T)hd T X t=1 K(xt−x h)[σ(xt)−σ(x)]et ≡I1+I2.(B.1) It follows from Theorem 3.5 of [2] (it is easy to show that the conditions of the theorem hold under our assumptions A.1–A.4) that I1→dN(0, ps(x)σ2(x)ZK2(u)du).(B.2) For the term I2, by the i.i.d. assumption on {et} E[ T X t=1 K(xt−x h)[σ(xt)−σ(x)]et]2=E T X t=1 K2(xt−x h)[σ(xt)−σ(x)]2 ≤C2h2E T X t=1 K2(xt−x h) = o(u(T)hd) Econometrics 2015,3284 by Assumption A.1 on σ(.). So that by the Markov inequality I2=oP(1).(B.3) Combining the results (B.1) (B.2) and (B.3), we have proved this lemma. Proof of Lemma 3. Using the same split chain technique as in the proof of Lemma 1, T X t=1 K(xt−x h)(g(xt)−g(x)) = U0(Lh) + n(T) X k=1 Uk(Lh) + Un(T)+1(Lh), with Lh=K(xt−x h)(g(xt)−g(x)). We have E(Uk(Lh)) = πs(Lh) = ZK(y−x h)(g(y)−g(x))ps(y)dy. Using the change of variables =hdZk1(z1)···kd(zd)(g(x1+z1h, ··· , xd+zdh)−g(x1,··· , xd))ps(x1+z1h, ··· , xd+zdh)dz and by Taylor expansion =hdZk1(z1)···kd(zd)(g0 1(x1,··· , xd)z1h+···+g0 d(x1,··· , xd)zdh +1/2 d X i=1 d X j=1 g00 ij(˜x1,··· ,˜xd)zizjh2)ps(x1+z1h, ··· , xd+zdh)dz), where ˜xiis a real number between xiand xi,t. So that E(Uk(Lh)) = O(h2+d) and E(1 n(T)hd n(T) X k=1 Uk(Lh)) = O(h2).(B.4) Moreover, we have E(U2 k(Lh)) = ZK2(y−x h)(g(y)−g(x))2ps(y)dy =O(h2d+2) and E(1 n(T)hd n(T) X k=1 Uk(Lh))2=O(1 u(T)h2d×h2d+2).(B.5) By Equations (B.4) and (B.5) and the Markov inequality 1 n(T)hd n(T) X k=1 Uk(Lh) = OP(h2) + OP(h pn(T)) = OP(h2)(B.6) by Assumption A.4. Econometrics 2015,3285 Using the fact that |U0(Lh)|=| τ0 X t=1 K(xt−x h)(g(xt)−g(x))| ≤ τ0 X t=1 |K(xt−x h)(g(xt)−g(x))| and by the definition of τ0, E( τ0 X t=1 |K(xt−x h)(g(xt)−g(x))|)≤E( τ1 X t=τ0|K(xt−x h)(g(xt)−g(x))|) =ZK(y−x h)|g(y)−g(x)|ps(y)dy =O(hd+1). So that by the Markov inequality 1 n(T)hdU0(Lh) = OP(h n(T)) = oP(h2)(B.7) by Assumption A.4. Similarly, we have 1 n(T)hdUn(T)+1(Lh) = oP(h2).(B.8) Lemma 3 holds because of Equations (B.6)–(B.8). Proof of Theorem 1. We have bg(x)−g(x) = PT t=1 K(xt−x h)(g(xt)−g(x)) PT t=1 K(xt−x h)+PT t=1 K(xt−x h)εt PT t=1 K(xt−x h) ≡I1+I2. So that Theorem 1 follows from Lemmas 1–3 and the Slutsky theorem. Proof of Theorem 2. We have bσ2(x) = T P t=1 (yt−bg(x))2K(xt−x hσ) T P t=1 K(xt−x hσ)≡B1+B2+B3 B0 , where B0= T P t=1 K(xt−x hσ),B1= T P t=1 K(xt−x hσ)ε2 t,B2= T P t=1 K(xt−x hσ)(g(xt)−bg(x))2and B3= 2 T P t=1 K(xt−x hσ)(g(xt)−bg(x))εt. We have B1 B0 = T P t=1 K(xt−x hσ)σ2(x) + T P t=1 K(xt−x hσ)(ε2 t−σ2(xt)) + T P t=1 K(xt−x hσ)(σ(xt)2−σ2(x)) T P t=1 K(xt−x hσ) Econometrics 2015,3286 =σ2(x) + OP((u(T)hd σ)−1/2) + OP(hσ) = σ2(x) + oP(1) by Lemma 2 and Assumption A.1 on σ(.). Thus Theorem 2 holds if we can prove B2 B0=oP(1) and B3 B0=oP(1). Using Taylor expansion, we have B2= T X t=1 K(xt−x hσ )(g(xt)−g(x) + g(x)−bg(x))2 ≡B21 +B22 +B23, where B21 = T P t=1 K(xt−x hσ)(g(xt)−g(x))2,B22 = T P t=1 K(xt−x hσ)(g(x)−bg(x))2, and B23 = 2 T P t=1 K(xt−x hσ)(g(xt)−g(x))(g(x)−bg(x)). By Lemma 3, we have B21 B0 =oP(1). And by Theorem 1, B22 B0 = (g(x)−bg(x))2=oP(1). By the Cauchy-Schwarz inequality, B2 23 ≤B2 21B2 22, which implies B23 B0 =oP(1). Again by the Cauchy-Schwarz inequality, B2 3≤B2 1B2 2. Thus B3 B0 =oP(1). So that bσ2(x)→pσ2(x). Conflicts of Interest The authors declare no conflict of interest. References 1. Karlsen, H.A.; Tjøstheim, D. Nonparametric estimation in null recurrent time series. Ann. Statist. 2001,29, 372–416. 2. Karlsen, H.A.; Myklebust, T.; Tjøstheim, D. Nonparametric estimation in a nonlinear cointegration type model. Ann. Statist. 2007,35, 252–299. Econometrics 2015,3287 3. Wang, Q.; Phillips, P.C.B. Asymptotic theory for local time density and nonparametric cointegrating regression. Econometr. Theory 2009a,25, 710–738. 4. Wang, Q.; Phillips, P.C.B. Structural nonparametric cointegrating regression. Econometrica 2009b, 77, 1901–1948. 5. Myklebust, T.; Karlsen, H.A.; Tjøstheim, D. Null recurrent unit root processes. Econometr. Theory 2012,28, 1–41. 6. Park, J.; Phillips, P.C.B. Nonlinear regression with integrated time series. Econometrica 2001,69, 117–161. 7. Dong, C.; Gao, J.; Tjøstheim, D.; Yin, J. Specification Testing for Nonlinear Multivariate Cointegrating Regressions; Monash University working paper; Monash University: Melbourne, Australia, 2014. 8. Schienle, M. Nonparametric Nonstationary Regression; University of Mannheim working paper; University of Mannheim: Mannheim, Germany, 2008. 9. Chen, J.; Gao, J.; Li, D. Estimation in semi-parametric regression with non-stationary regressors. Bernoulli 2012,18, 678–702. 10. Gao, J.; Phillips, P.C.B. Functional Coefficient Nonstationary Regression with Nonand Semiparametric Cointegration; Monash University working paper; Monash University: Melbourne, Australia, 2013. 11. Bingham, N.H.; Goldie, C.M.; Teugels, J.L. Regular Variation; Cambridge University Press: Cambridge, UK, 1989. 12. Engle, R.F. Autoregressive conditional heteroscedasticity with estimates of the variance of U.K. inflation. Econometrica 1982,50, 987–1008. 13. Bollerslev, T. Generalized autoregressive conditional hetoroskedasticity. J. Econometr. 1986,31, 307–327. 14. Bollerslev, T.; Chou, R.Y.; Kroner, K.F. ARCH modeling in finance: A review of the theory and empirical evidence. J. Econometr. 1992,52, 5–59. 15. Härdle, W.; Tsybakov, A. Local polynomial estimators of the volatility function in nonparametric autoregression. J. Econometr. 1997,81, 223–242. 16. Fan, J.; Yao, Q. Efficient estimation of conditional variance functions in stochastic regression. Biometrika 1998,85, 645–660. 17. Wang, Q.; Wang, Y. Nonparametric cointegrating regression with NNH errors. Econometr. Theory 2013,29, 1–27. 18. Yang, L.; Tschernig, R. Multivariate bandwidth selection for local linear regression. J. R. Statist. Soc.: Ser. B 1999,61, 793–815. 19. Kallianpur, G.; Robbins, H. The sequence of sums of random variables. Duke Math. J. 1954,21, 285–307. 20. Gao, J.; Tjøstheim, D.; Yin, J. Estimation in threshold autoregressive models with a stationary and a unit root regime. J. Econometr. 2013,172, 1–13. 21. Cai, B.; Gao, J.; Tjøstheim, D. A New Class of Bivariate Threshold Cointegration Models; University of Bergen working paper; University of Bergen: Bergen, Norway, 2014. Econometrics 2015,3288 22. Pagan, A.; Ullah, A. Nonparametric Econometrics; Cambridge University Press: Cambridge, UK, 1999. 23. Rudebusch, G.D. Federal reserve interest rate, targeting rational expectations, and the term structure. J. Monet. Econ. 1995,35, 245–274. 24. Anderson, T.G.; Lund, J. Estimating continuous-time stochastic volatility models of the short-term interest rate. J. Econometr. 1997,77, 343–377. 25. Shiller, R.J.; Campbell, J.Y.; Schoenholtz, K.L.; Weiss, L. Forward rates and future policy: Interpreting the term structure of interest rates. Brook. Papers Econ. Activ. 1983,1, 173–223. 26. Rudebusch, G.D. Term structure evidence on interest rate smoothing and monetary policy inertia. J. Monet. Econ. 2002,49, 1161–1187. 27. Sarno, L.; Thornton, D.L. The dynamic relationship between the federal funds rate and the Treasure bill rate: An empirical investigation. J. Bank. Finan. 2003,27, 1079–1110. 28. Anderson, H.M. Transaction costs and non-linear adjustment towards equilibrium in the US treasure bill market. Oxford Bull. Econ. Statist. 1997,59, 465–484. 29. Clarida, R.H.; Sarno, L.; Taylor, M.P.; Valenter, G. The role of asymmetries and regime shifts in the term structure of interest rates. J. Bus.2006,79, 1193–1224. 30. Balke, N.S.; Fomby, T.B. Threshold cointegration. Int. Econ. Rev. 1997,38, 627–645. 31. White, H. A reality check for data snooping. Econometrica 2000,68, 1097–1126. 32. Diebold, F.X.; Rudubusch, G. Forecasting output with the composite leading index: A real-time analysis. J. Am. Statist. Assoc.1991,86, 603–610. 33. Stock, J.; Watson, M. Why has U.S. inflation become harder to forecast? J. Money Credit Bank. 2007,39, 3–33. 34. Hamilton, J. The daily market for Federal funds. J. Polit. Econ.1996,104, 26–56. 35. Chan, K.C.; Karolyi, G.A.; Longstaff, F.A.; Sanders, A.B. An empirical comparison of alternative models of the short-term interest rate. J. Finan. 1992,47, 1209–1227. 36. Vasicek, O. An equilibrium characterization of the term structure. J. Finan. Econ. 1977,5, 177–188. 37. Cox, J.C.; Ingersoll, J.E.; Ross, S.A. A theory of term structure of interest rates. Econometrica 1985,53, 385–407. 38. Nummelin, E. General Irreducible Markov Chains and Non-negative Operators; Cambridge University Press: Cambridge, UK, 1984. 39. Kasahara, Y. Limit theorems for Lévy processes and Poisson point processes and their applications to Brownian excursions. J. Math. Kyoto Univ. 1984,24, 521–538. c 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).