On invariant density estimation for ergodic diffusion processes

Statistics & Operations Research Transactions SORT 28 (2) July-December 2004, 111-124 Statistics & Operations Research Transactions On Invariant Density Estimation for Ergodic Diffusion Processes Yu. A. Kutoyants∗ Universit´e du Maine Abstract We present a review of several results concerning invariant density estimation by observations of ergodic diffusion process and some related problems. In every problem we propose a lower minimax bound on the risks of all estimators and then we construct an asymptotically efficient estimator. MSC: 62M05, 62G07, 62G20 Keywords: ergodic diffusion, density estimation, large sample theory, efficient estimation 1 Introduction Suppose that we observe a trajectory XT={Xt,0≤t≤T}of the diffusion process dXt=S(Xt)dt+σ(Xt)dWt,X0,t≥0 (1) where the trend coefficient S(·)is an unknown function and the diffusion coefficient σ(·)2is a known positive function. We assume that these functions are such that the equation (1) has a unique weak solution (see, e.g. Durret (1996). Moreover, we suppose that the following conditions are fulfilled. This contribution has been presented in the Barcelona Conference on Asymptotic Statistics, which took place in Bellaterra (Barcelona, 2003). ∗Address for correspondence: Laboratoire de Statistique & Processus, Universit´ e du Maine, 72085 Le Mans, Cedex 9, FRANCE. Received: November 2003 Accepted: June 2004 112 On Invariant Density Estimation for Ergodic Diffusion Processes Conditions A0: Zx 0exp(−2Zy 0 S(v) σ(v)2dv)dy→ ±∞,as x→ ±∞ and G(S)=Z∞ −∞ 1 σ(x)2exp(2Zx 0 S(v) σ(v)2dv)dx<∞. By these conditions the solution of equation (1) has ergodic properties, with the invariant density fS(x)=1 G(S)σ(x)2exp(2Zx 0 S(v) σ(v)2dv),(2) i.e., for any function h(·)such that ES|h(ξ)|<∞the law of large numbers 1 TZT 0h(Xt)dt−→ ESh(ξ)p.s. holds. We denote by ξa random variable with density function fS(·). We consider the problem of estimation of the invariant density by observations XT and study the properties of its estimators in the asymptotic of large samples T→ ∞. The initial value X0is supposed to have the same density function fS(·), so the observed process is stationary and the observed value Xtis a random variable with density fS(·). We recall that in i.i.d. case the density of one observation entirely defines the distribution of the whole sample, but in the case of continuous time stochastic processes the distribution of the observed trajectory is defined by all finite dimensional distributions. Hence the density of one observation does not identify the model. For an ergodic diffusion process this density nevertheless identifies the whole model, because the model is entirely defined by the trend (unknown) and diffusion (known) coefficients and having invariant density we can write the trend coefficient as S(x)=³σ(x)2fS(x)´′ 2fS(x).(3) The problem of invariant density estimation was considered by many authors (see, e.g., Nguen (1979), Delecroix (1980), Castellana and Leadbetter (1986), Bosq (1998), van Zanten (2001) et al. ). In particular, Castellana and Leadbetter (1986) showed that for any stationary process with one and two dimensional densities f(y),f(τ, y,z) under condition : CL. The functions f(y),f(τ, y,z), τ > 0 are continuous at point x and |f(τ, y,z)−f(y)f(z)|≤ψ(τ)∈ L1(R+),(4) Yu. A. Kutoyants 113 this density f(y)can be estimated with the parametric rate √T, i.e., let ˆ fT(x)be a kernel type estimator ˆ fT(x)=1 TϕTZT 0KÃXt−x ϕT!dt(5) where ϕT→0,TϕT→ ∞ and the kernel K(·)satisfies the usual properties, then lim T→∞TE³ˆ fT(x)−f(x)´2=A(x). Here A(x)=2Z∞ 0hf(τ, x,x)−f(x)2idτ. The condition CL can be verified for ergodic diffusion processes (see Veretennikov (1999) for sufficient conditions). Note that this verification requires much more regularity from the coefficients S(·)and σ(·), than we really need in this estimation problem. In this work we propose several asymptotically efficient estimators of the density function without supposing that the condition CL is fulfilled. 2 Lower bound We start with the minimax lower bound on the risks of all estimators. This bound was established for a wide class of loss functions (see Kutoyants (1997b), (1998)) but for simplicity of exposition we consider quadratic loss functions only. Fix some S∗(·) and and δ > 0 and introduce the set Vδ={S(·) : sup x∈R|S(x)−S∗(x)| ≤ δ}. The role of Fisher information in our problem plays the quantity If(S,x)=       4fS(x)2ES      χ{ξ>x}−FS(ξ) σ(ξ)fS(ξ)      2       −1 where FS(·) is the distribution function of the invariant law. Hence FS(ξ) is a uniform [0,1] random variable. We have the following result. Theorem 1 Let supS∈VδG(S)<∞,and If(S∗,x)>0, then for all estimators ¯ fT(x) lim δ→0lim T→∞ sup S(·)∈Vδ TES³¯ fT(x)−fS(x)´2≥If(S∗,x)−1. 114 On Invariant Density Estimation for Ergodic Diffusion Processes As usual in this type of problem (see Ibragimov and Khasminskii (1981), Chapter 4) the proof is based on the estimate sup S(·)∈Vδ ES³¯ fT(x)−fS(x)´2≥sup ϑ∈Θδ Eϑ³¯ fT(x)−fϑ(x)´2 where the parametric sub-model corresponds to the trend coefficient S(ϑ, x)=S∗(x)+ (ϑ−ϑ0)ψ(x)σ(x)2∈Vδ,with the function ψ(·)from the class K=(ψ(·) : ES∗Zx ξ ψ(v)dv=¡2fS∗(x)¢−1). For this parametric family with Fisher information Iψwe obtain a Hajek-Le Cam minimax bound and then choose the least favourable family (with minimal Fisher information) as follows inf ψ∈K Iψ=If(S∗,x). Note that for ψ(·)∈ K we have fϑ(x)=ϑ+o(1)as δ→0. The details of the proof can be found in Kutoyants (1997d), (1998), (2003). This lower bound allows us define the asymptotically efficient estimator ϑ∗ Tby the following equality: lim δ→0lim T→∞ sup S(·)∈Vδ TES³f∗ T(x)−fS(x)´2=If(S∗,x)−1. 3 Asymptotically efficient estimators We consider below three type of estimators: local time, unbiased and kernel type . Local time estimator Recall that local time ΛT(x) of the diffusion process (1) is defined by the formula ΛT(x)=lim ε↓0 meas{t:|Xt−x| ≤ ε, 0≤t≤T} 4ε and it admits the representation (see Karatzas and Shreve (1991)) 2ΛT(x)=|XT−x|−|X0−x|+ZT 0sgn(x−Xt)dXt Yu. A. Kutoyants 115 Local time estimator of the density is defined by the equality f◦ T(x)=2ΛT(x) σ(x)2T. We study its asymptotic behavior under the following conditions: U. The law of large numbers PS−lim T→∞ 4fS(x)2 TZT 0      χ{Xt>x}−FS(Xt) σ(Xt)fS(Xt)      2 dt=If(S,x)−1 is uniform on S(·)∈Vδ. B. sup S(·)∈Vδ          ES¯¯¯¯¯¯¯ χ{ξ>x}−FS(ξ) σ(ξ)fS(ξ)¯¯¯¯¯¯¯ 2 +ES¯¯¯¯¯¯¯Zξ 0 χ{v>x}−FS(v) σ(v)2fS(v)dv¯¯¯¯¯¯¯ 2         <∞. Theorem 2 Let the conditions U,Bbe fulfilled and If(S,x)be continuous on Vδ, then the estimator f◦ T(x)is unbiased: ESf◦ T(x)=fS(x),asymptotically normal: LSn√T³f◦ T(x)−fS(x)´o⇒ N ³0,If(S,x)−1´ and is asymptotically efficient. The proof follows directly from the representation √T³f◦ T(x)−fS(x)´=2f(x) √TZXT X0 χ{v>x}−F(v) σ(v)2f(v)dv −2f(x) √TZT 0 χ{Xt>x}−F(Xt) σ(Xt)f(Xt)dWt(6) and the central limit theorem for stochastic integrals. Unbiased estimator Let us introduce the estimator of the density function f∗ T(x)=1 TZT 0Rx(Xt)dXt+1 TZT 0Nx(Xt)dt 116 On Invariant Density Estimation for Ergodic Diffusion Processes where h(·)∈ C′(R)and Rx(y)= 2χ{y<x}h(y) σ(x)2h(x),Nx(y)= χ{y<x}h′(y)σ(y)2 σ(x)2h(x). Then it is easy to see that ESf∗ T(x)=fS(x) and LSn√T³f∗ T(x)−fS(x)´o⇒ N ³0,If(S,x)−1´. Thedetailed proof(with conditions likeU,B) can befound in Kutoyants (1998), (2003). It is based on the representation √T³f∗ T(x)−fS(x)´=√T³f◦ T(x)−fS(x)´+o(1). In particular, if σ(x)≡1 and h(x)=x3, then for x,0 f∗ T(x)=2 Tx3ZT 0χ{Xt<x}X3 tdXt+3 Tx3ZT 0χ{Xt<x}X2 tdt is unbiased and asymptotically efficient estimator of the density. Kernel type estimator Let us introduce the kernel type estimator ˆ fT(x)=1 √TZT 0K³√T(Xt−x)´dt where the kernel K(·) is a bounded function with compact support [A,B] and ZB AK(u) du=1,ZB Au K(u) du=0. To study this estimator we need to suppose that the function fS(x)is continuously differentiable (the function S(x)is continuous and σ(x)2is continuously differentiable). Then we obtain the representation √T³ˆ fT(x)−f(x)´=√T¡f◦ T(x)−f(x)¢+o(1) and the asymptotic normality of this estimator LSn√T³ˆ fT(x)−fS(x)´o⇒ N ³0,If(S,x)−1´ Yu. A. Kutoyants 117 follows from the asymptotic normality of the local time estimator. The detailed proof with exact conditions can be found in Kutoyants (1998), (2003). Using similar arguments we obtain its asymptotic efficiency. It is clear that A(x)=If(S,x)−1. The consistent estimation of the quantity If(S,x)−1is proposed in Dehay and Kutoyants (2004). 4 Semiparametric estimation Let us consider the problem of parameter ϑS=ESR(ξ)S(ξ)+ESN(ξ) estimation by observations (1) (with unknown S(·)and known σ(·)2). Here R(·)and N(·)are known functions. We will see later that the problem of invariant density estimation is a particular case of this problem. Introduce the Fisher information Iϑ(S)=         ES       R(ξ)σ(ξ)2fS(ξ)+2MS(ξ) σ(ξ)fS(ξ)       2         −1 where MS(y)=Eµ·FS(y)−χ{ξ<y}¸£R(ξ)S(ξ)+N(ξ)¤¶. The first result is the lower bound similar to that of Theorem 1. Theorem 3 Let supS∈VδG(S)<∞and Iϑ(S∗)>0,then for all estimators ¯ ϑT lim δ→0lim T→∞ sup S(·)∈Vδ TES³¯ ϑT−ϑS´2≥Iϑ(S∗)−1. The proof can be found in Kutoyants (1997c) and (2003). By direct calculation we verify that the empirical estimator ˜ ϑT=1 TZT 0R(Xt)dXt+1 TZT 0N(Xt)dt (under moments conditions) is asymptotically normal: LSn√T³˜ ϑT−ϑS´o⇒ N ³0,Iϑ(S)−1´. 118 On Invariant Density Estimation for Ergodic Diffusion Processes and asymptotically efficient: lim δ→0lim T→∞ sup S(·)∈Vδ TES³˜ ϑT−ϑS´2=Iϑ(S∗)−1. Let us consider three different choices of the functions R(·)and N(·). •Distribution function estimation. Let us put R(y)=0 and N(y)=χ{y<x}, then ϑS=FS(x). Hence the empirical distribution function •˜ ϑT=ˆ FT(x)=1 TZT 0χ{Xt<x}dt is asymptotically efficient estimator of the invariant distribution function Kutoyants (1997a). •Density estimation. Let us put R(y)=sgn(x−y) σ(x)2and N(y)=0, then ϑS=fS(x) and •˜ ϑT=¯ fT(x)=1 Tσ(x)2ZT 0sgn(x−Xt)dXt. Therefore, ¯ fT(x)is an asymptotically efficient estimator of the density. •Moments estimation. Let us put R(y)=0 and N(y)=yk, then ϑS=ESξkand the empirical moment •˜ ϑT=1 TZT 0Xk tdt is asymptotically efficient estimator of the moments of ergodic diffusion process. 5 Integral type risk Let us consider integral type quadratic risk R³¯ fT,fS´=ESZ(¯ fT(x)−fS(x))2dx and denote by Rf(S)=ZIf(S∗,x)−1dx the limit value of this risk for local time estimator. Yu. A. Kutoyants 119 Introduce the condition sup S(·)∈Vδ         ZES¯¯¯¯¯¯¯ χ{ξ>x}−FS(ξ) σ(ξ)fS(ξ)¯¯¯¯¯¯¯ 2 dx+ +ZES¯¯¯¯¯¯¯Zξ 0 χ{v>x}−FS(v) σ(v)2fS(v)dv¯¯¯¯¯¯¯ 2 dx         <∞.(7) Theorem 4 Let the condition (7) be fulfilled, then lim δ→0lim T→∞inf ¯ fT sup S(·)∈Vδ TR³¯ fT,fS´=Rf(S∗). Note that this theorem contains two results. The first one is the lower bound for all estimators lim δ→0lim T→∞ sup S(·)∈Vδ TR³¯ fT,fS´≥ Rf(S∗).(8) The upper bound we obtain with the help of local time estimator. Slight modification of the conditions allows obtain the same limit for the risks of unbiased and kernel type estimators. Therefore, all these estimators are asymptotically efficient in the sense of the bound (8). The proofs can be found in Kutoyants (2003). Note that Negri (2001) establishes the asymptotic efficiency of the local time estimator for the loss function with uniform metric, i.e., for ESℓ³supx√T¯¯¯¯ fT(x)−fS(x)¯¯¯´. 6 Second order efficiency Having so many asymptotically efficient estimators we seek now the second order efficient one. Let us study the quantity ³TR³¯ fT,fS´−Rf(S)´. Note that for LTE T1 2³TR³f◦ T,fS´−Rf(S)´→Q,0. It can be shown that if the function S(·)is k−1 times differentiable then for certain kernel type estimators ˆ fT(·) T1 2k−1³TR³ˆ fT,fS´−Rf(S)´→ −P<0. To answer the question why the rate T1 2k−1is better than T1 2we write the last expression as TR³ˆ fT,fS´=Rf(S)−P T−1 2k−1(1+o(1))