Intrinsic geometry on the class of probability densities and exponential families

Publ. Mat. 51 (2007), 309–332 INTRINSIC GEOMETRY ON THE CLASS OF PROBABILITY DENSITIES AND EXPONENTIAL FAMILIES Henryk Gzyl and L´ azaro Recht Abstract We present a way of thinking of exponential families as geodesic surfaces in the class of positive functions considered as a (multiplicative) sub-group G+of the group Gof all invertible elements in the algebra Aof all complex bounded functions defined on a measurable space. For that we have to study a natural geometry on that algebra. The class Dof densities with respect to a given measure will happen to be representatives of equivalence classes defining a projective space in A. The natural geometry is defined by an intrinsic group action which allows us to think of the class of positive, invertible functions G+as a homogeneous space. Also, the parallel transport in G+and Dwill be given by the original group action. Besides studying some relationships among these constructions, we examine some Riemannian geometries and provide a geometric interpretation of Pinsker’s and other classical inequalities. Also we provide a geometric reinterpretation of some relationships between polynomial sequences of convolution type, probability distributions on Nin terms of geodesics in the Banach space ℓ1(α). 1. Introduction and preliminaries Exponential families of probability densities appear at least in three different ways: On one hand, they appear as parametric distributions for which unbiased estimators achieve their maximum efficiency (or minimum variance). Even though the result is a textbook matter, see [LM] or [W] for example, let us briefly describe it following the presentation in [S] (which together with [GP] are a good staring point for a fertile interaction between probability, physics and geometry): Let 2000 Mathematics Subject Classification. Primary: 46L05, 53C05, 53C56, 62B01, 60B99, 60E05; Secondary: 53C30, 51M05, 55M, 62A25, 22E, 33E. Key words. Exponential families, projective geometry, parallel transport, sequences of convolution type. 310 H. Gzyl, L. Recht T= (T1,...,Tk) be an unbiased estimator of Θ = (θ1,...,θk), unknown parameters of a family Π = {p(x, Θ)}describing the probability densities of an (S, S)-valued random variable Xwith respect to a given probability measure m(dx). If we put Vij =EΘ[(Ti−θi)(Tj−θj)], and under the usual regularity assumptions we set Gij =EΘh∂ln p(x,Θ) ∂θi ∂ln p(x,Θ) ∂θji, then in matrix terms the Cramer-Rao inequality asserts that (V−G−1) is positive, and when V G =I, then the minimum variance or maximum efficiency is achieved. In this case, this amounts to saying that p(x, Θ) = c(Θ)−1exp{hΘ,Ti}. The second way in which exponential families appear is related to the notion of sufficiency. To cite an old result on the relation of exponential families and sufficient statistics, consider the following result by Brown, [Br], improving on a result by Dynkin: Theorem 1.1. Let Π = {p(x, Θ)}be a family of densities on an interval Isuch that every p(x, Θ) is bounded away from zero and continously differentiable on I. Suppose that there is a nontrivial sufficient statistics Tfor Θon the basis of n-observations. Then Πis a k-parameter exponential family with k < n. For much more about this and properties of exponential families, their use in statistics and information theory, some classical references are the volumes by Barndorff-Nielsen [B], Kullback [K] and Vajda [V]. As we shall mention later on, this old, important theorem is recalled here for two reasons: on one hand it brings forth the relationship between exponential families and sufficiency, and on the other, it makes the special class of densities we deal with (namely the invertible elements in the algebra Adefined below), appear as the natural class to consider. The third road to exponential families comes from the solution of moment (or generalized moment) problems by information theoretic or maxentropic methods. To state it as simple as possible, consider the problem of finding a density p(x) on the measure space (S, S, m) such that the expected value of an RK-valued random variable Xis preassigned to be Ep[X] = RX(x)p(x)m(dx) = µ, where µ∈RKis a preassigned vector. The problem is not trivial when K= 1 only when m(S) = ∞. This problem seems to have been first solved by Esscher in [Es], although it was during the 1950’s that the idea was systematized into a variational method to lay down the foundations of statistical mechanics by Jaynes in [J]. The interplay between geometry and probability has proceeded along two mayor directions. One described in [Ef], [A] and the collection Geometry on the Class of Probability Densities 311 [ABKLR], which goes from a geometry induced on the space of parameters to statistical properties of the parameterized family of densities, and the other which can be traced back from [GP] and [PRo], where the authors examine a geometric (manifold) structure on the class of probability densities of probabilities equivalent to a given one. Regretfully there does not seem to be an easy connection between these approaches and the one we develop here. Here we shall examine a completely different geometric structure on the space of probabilities, that is, an approach that does not bear any relationship to that described in the above mentioned references. For us, probability distributions with density with respect to a given measure (or probability) will be described by representatives of a projective structure on the class of positive, invertible elements in a special complex Banach algebra, namely the Banach algebra of complex, measurable functions defined on a given measure space. To make this note self contained, instead of referring the reader to [GR] where the basics were outlined, we shall again recall some results obtained by Corach, Porta and Recht in [CPR], [PR1] and [PR2], from which we draw freely. Our case is simpler that the theory developed there, because all the Banach algebras with which we deal with here are commutative. This is done in Section 2. In that section we obtain some properties of the geodesics in the space of positive invertible functions and we examine two Riemannian structures on that space. There we provide a geometric way of understanding Pinsker’s inequalities. In Section 3 we consider two different Riemannian structures on the class of probability densities thought of as representatives of rays in the class of positive invertible elements of the algebra, that is, as representatives of a natural equivalence relation. The geometry of that projective space is transported onto the class of densities which will be representatives of the rays. In Section 4 we provide the geometrical characterization of exponential families in terms of exponential surfaces in the intrinsic geometry. A shortcoming in the above presentation, is that the C∗-algebra to which it readily applies is the algebra of bounded, measurable functions, and for probabilistic applications one is interested in more general algebras. In Section 5 we explore the geometry on the space ℓ1(α), which is a convolution algebra, but not a C∗-algebra. A motivation for looking into this example is that it is the simplest example of convolution algebra which contains the measures on a countable set. Surprisingly enough, a variety of connections between probability and polynomial convolution sequences aquire a new meaning in the geometric setup that we propose. 312 H. Gzyl, L. Recht 2. The fundamental C∗algebra and its properties In this section we recall the basic facts about the geometry on a commutative C∗algebra Awith a unit. The basic model we should have in mind is A={f:S→C| kfk= esssup(|f|)<∞}, where (S, S, m) is a given measure space. We shall assume that the measure mis either a probability or a finite measure. We will define a special connection, describe its geodesics and the parallel transport along them, as well as the resulting geometries on the positive elements of the algebra, as well as the basic properties of the projective spaces in the set of positive invertible elements in A. In this algebra the set of invertible (with respect to the usual pointwise multiplication as the product on the algebra) vectors G={g∈ A | g−1exists}is a (commutative) group and the class G+⊂Gdenotes the class of positive invertible elements. Also, it is standard result that Gis an open set in Aand that the inversion operation is continuously differentiable. This allows us to provide Gwith a manifold structured modeled on Athought of as a Banach algebra and that the Lie algebra of G, i.e. the tangent space to Gat 1, is A. 2.1. The basic reductive homogeneous space. We want to think of G+both as a homogeneous space and as the base space for a principal bundle. For that, we first need an action of G on G+. We define Lg:G+−→ G+;Lg(a) = (g∗)−1ag−1,∀a∈G+, for any g∈G. Since the product is commutative, Lg(a) = |g|−2a. An intuitive way of understanding that mapping is to realize that every a∈ G+defines a scalar product on Ha≡L2(am) by hX, Y ia=RXY a dm. Now we may interpret the group action as an isometry Ha→ HLg(a). This action is transitive and it is easy to see that G+≃G/Ia,where Iais defined a few lines below. There is a well known way to define connections in this setup, see Chapters 10 and 11 of [KN] for example. To define the principal bundle, fix some a∈G+and define the projection operator πa:G−→ G+by means of πa(g) = Lg(a) and notice right away that the isotropy group of adefined by Ia={g∈G|πa(g) = a}={g∈G|g∗g= 1}, verifies that π−1 a(a) = Ia. Moreover the fiber π−1 a(b) over b∈G+is given by Iahfor appropriate h∈G. Notice that when Ais a function algebra, Iais the class of functions taking value in the circle. Geometry on the Class of Probability Densities 313 We mentioned that Gis an open subset in A, its tangent space at 1 is A, i.e. (T G)1=A, and it is easy to see that (T Ia)1=Va={0} ⊕ iAs. The group action can be used to move this splitting around, and the existence of such splitting is equivalent to the existence of connections. The derivative (Dπa)1(X) of πaat 1 in the direction of X∈ A is easy to compute, and it is given by (Dπa)1(X) = −a(X+X∗). Clearly (Dπa): A −→ (T G+)a≡ As⊕ {0}. We shall define the horizontal space at aby Ha≡ {X∈ A | (a)−1X∗a=X}={X∈ A | X∗=X}=As⊕ {0} and we have the obvious splitting A=Ha⊕Va. Not only that, the map (Dπa)1is invertible from the left. That is, there exists a mapping (actually a section of the bundle (G, G+, πa)), κa: (T G+)a→(T G)1, given by κa(z)≡ −a−1 2z such that (T G+)a κa −→ (T G)1 (Dπa) −→ (T G+)ais the identity mapping. Definition 2.1. Define the A-valued 1-form κ:G+→L(T G+,As) by κb=κa◦(DL)−1 g, where Lg(a) = b. Comment. From now on we shall use the shorthand and ˜ Lgfor the tangent mapping DLgand ˜πafor the tangent Dπa, etc. The mapping κis called the structure 1-form of the homogeneous space G+.All the geometry on G+comes from κ. The basic properties of these mappings are contained in the simple Lemma 2.1. With the notations above, and if Lg(a) = b, we have (i) πb=Lg◦πa, (ii) or in differential form: ˜πb=˜ Lg◦˜πa, (iii) κis equivariant, that is, κb◦˜ Lg=κa, and (iv) ˜πb◦κb: (T G+)b→(T G+)b is the identity mapping. 314 H. Gzyl, L. Recht 2.2. Geodesics in G+. Let us now recall the basic construction leading to the definition of geodesics in G+. We begin with Definition 2.2. Let a(t) be a continuously differentiable curve in G+ such that a(0) = a. Let γ(t) be the solution to the transport equation (1) ˙γ(t) = κa(t)(˙a(t))γ(t), γ(0) = 1. The curve γ(t) is called the horizontal lift of a(t). Proposition 2.1. With the notations introduced above we have: (i) πa(γ(t)) = a(t)and (ii) ˙γ(t)∈ As=κa((T G+)a). Proof: Just note that integrating (1) we obtain γ(t) = a(o) a(t)1/2. Definition 2.3. Let a(t) be a continuously differentiable curve in G+ such that a(0) = a, and let Y(t) be a vector field defined along a(t). The covariant derivative of Y(t) along a(t) is defined to be (2) D˙a(t)Y(t) = πa(t)d dtκa(t)(Y(t)). Observe now that if a(t) is a twice differentiable curve in G+, by considering Y(t) = ˙a(t) in (2) above, a simple computation yields: (3) D˙a(t)˙a(t) = −˙a2(t) a(t)+ ¨a(t) and if, as usual, we say that a(t) is a geodesic whenever D˙a(t)˙a(t) = 0, we readily obtain that (4) a(t) = a(0)etX,where X= ln ˙a(0) a(0). Note as well that the geodesic which goes from a0to a1, both in G+, in one time unit is rapidly obtained from (4) to be given by (5) a(t) = a1−t 0at 1. Comments. Clearly we also have X= ln a1 a0.To justify the name of the curves (5) we have to verify that they minimize some distance function. To begin with, at a∈G+define the following norm. For x∈(T G+)a set kxka=ka−1xk, where k · k is the norm in the Banach algebra. Let now a(t) for t∈[0,1], be a continuously differentiable curve. Define its length by l(a) = Z1 0 k˙a(t)ka(t)dt. Geometry on the Class of Probability Densities 315 Define now for given a0, a1∈G+ (6) d(a0, a1) = inf{l(c)|ccontinuously differentiable curve joining a0to a1}. It so happens that the curve (5) is the minimizer of (6), and therefore κis the connection in the Finsler geometry associated to d(a0, a1).We have Theorem 2.1. Let a0, a1∈G+and denote by ga0,a1(t)the curve (5) and let c(t)be any other continuously differentiable curve joining a0to a1 in a unit of time. Then l(c)≥l(ga0,a1). Proof: Consider l(c) = Z1 0 k˙c(t)kc(t)dt =Z1 0 kc−1(t)˙c(t)kdt ≥   Z1 0 dln c(t)dt    =    ln a1 a0    =d(a0, a1). To close this section note that going from a0to a1along a geodesic can be realized by means of the group action: a1=a0eX=Lg(a0),with g=e−X/2, which prompts the following definition for vector fields along curves in G+. Definition 2.4. Let Y(t) be a vector field defined along a geodesic curve a(t)∈G+.We say that Yis parallel if Y(t) = Lg(t)(Y(0)) for every twhere g(t) is a group element taking a0onto a1. Let us examine the basics of a Riemannian geometry on G+. 2.3. Two Riemannian structures on G+. In this section we examine two different Riemannian structures on G+. With respect to the first one, the geodesic transport is our old group action, which turns out to be self dual. With respect to the other, there are two different parallel transports in duality. We also provide a geometric interpretation for Pinsker’s inequality and a converse to it. Definition 2.5. We define the scalar product on the tangent bundle T G+as follows: On the tangent space (T G+)1=Asto G+at 1, we can define the scalar product hX, Y i1≡Em[XY ] which can be made equivariant by setting hX, Y ia=Em[a−1Xa−1Y]. 316 H. Gzyl, L. Recht To begin with, we have the simple Lemma 2.2. The parallel transport in Definition 2.5, is self dual with respect to this scalar product, that is, if Lg(1) = a,Xa=Lg(X1)and Ya=Lg(Y1), (7) hXa, Yaia=hX1, Y1i1. Comment. Duality of different parallel transports is studied in [A], [ABKLR] and [S]. Here we provide a parallel transport leading to a connection that is self dual. We can define the squared distance along curves for this Riemannian metric by Definition 2.6. Let c(t) be a continuously differentiable curve in G+ joining c(0) to c(1). The (squared) distance along cis defined by (8) d1(c(0), c(1)) = Z1 0 h˙c(t),˙c(t)i(c)(t)dt. What is interesting and remarkable at this stage is that Proposition 2.2. The geodesics of (8) are the geodesics of the connection in G+and are given by (5). Proof: Left for the reader. Just notice that the Euler-Lagrange equation d dt ∂L ∂˙c=∂L ∂c is easy to obtain in the commutative case. Here L(c) is given by the right hand side of (8). The solution is as claimed. A simple computation shows that along a geodesic a(t) = a0etX with X= ln a1 a0, the distance is as given in Theorem 2.1, i.e., (9) d1(a0, a1) = kXk=    ln a1 a0    . We shall now introduce a different Riemannian structure on G+. Definition 2.7. Let us now define the scalar product in (T G+)aby setting (X, Y )a≡Em[a−1X Y ]. The analogue of Lemma 2.2 is the following Lemma 2.3. Let Xa=Lg(X1) = T+ 1,a(X1)where a=Lg(1) and let Y=T− 1,a(Y), that is, the parallel transport from the ambient space in which parallel transport is assumed to be the identity mapping. Then (T+ 1,a(X1), T − 1,a(Y))a=Em[a−1aX, Y ] = (X, Y )1 that is, the geodesic transport and the identical parallel transport are dual with respect to the (·,·)product on T G+. Geometry on the Class of Probability Densities 317 We also have Proposition 2.3. Let c(t)be a differentiable curve in G+joining a0 to a1in a unit of time. Define d2(a0, a1) = Z1 0 (˙c(t),˙c(t))c(t)dt. Then the geodesic in that metric is a∗(t) = a1/2 0+t(a1/2 1−a1/2 0)2. Proof: Notice to begin with the Euler-Lagrange equations determining the geodesic are now 2 ¨c c=˙c c2, which can be integrated to yield the claimed result. Next a simple application of Jensen’s inequality yields that for any other continuously differentiable curve ˆcwith the same initial and final points (10) Z1 0 (˙a∗(t),˙a∗(t))a∗(t)≡4Em(a1/2 1−a1/2 0)2≤Z1 0 (˙ ˆc(t),˙ ˆc(t))ˆc(t), thus concluding the proof. Comment. Actually (10) connects us to Corollary 2.1. Let now c(t) = a0etX with X= ln a1 a0. Then (10) implies that (11) Ema1−a0ln a1 a0≥4Em(a1/2 1−a1/2 0)2. Now, when Em[a0] = Em[a1] = 1, we obtain 4Em(a1/2 1−a1/2 0)2≥ Em[(a1−a0)]2after a simple application of the Cauchy-Schwarz inequality, which turns (11) into a simple extension of the famous Pinsker’s inequality. We thus obtain a totally geometric proof of that famous inequality. To obtain a kind of converse to that inequality using our geometric setup, compute the distance d1(a0, a1) along the geodesic a∗(t) = a1/2 0+ t(a1/2 1−a1/2 0)2for d2to obtain Proposition 2.4. With the notations introduced above the following inequality holds (12)     ln a1 a0    ≤ (a1/2 1−a1/2 0) . 324 H. Gzyl, L. Recht Theorem 5.1. Consider a convolution algebra Aof functions such that a) the only non zero elements fin the algebra that solve f(t+s) = f(t) + f(t)are f(t) = ct, with c∈C, b) the only non zero elements fin the algebra that solve f(t+s) = f(t)f(t)are f(t) = eat with a∈C. Then {fn(t) : n≥0}is a sequence of convolution type if and only if there exists a∈Cand a sequence {gn∈ A}, with g06= 0 such that (16) fn(t) = eat n X 0 gk∗ ntk k!. In the geometric setup an important role is played by (the connected component of) the group Gof all invertible (with respect to the convolution product) elements of A=ℓ1(α). Let us introduce the notation E={ex|x∈ A}, where ex≡Pkxk∗, then Theorem 5.2. With the notations introduced above, G=E. The proof of this result and the next are detailed in [dB]. In terms of the generic automorphisms introduced above, we also have Theorem 5.3. With the notations introduced above G=E=(x∈ A | Λz(x) = X n xnzn6= 0,∀ |z| ≤ eρ). We mention in passing that for |z| ≤ eρ, the sets Mz={x∈ A | Λz(x) = 0}are maximal ideals in A, and we have the mapping Λ: A −→ A(eρ)x−→ Λ•(x) where A(r) = {g:{|z| ≤ r} → C|ganalytic for |z|< r and continuous for |z| ≤ r}. 5.2. Geometry on G+. The constructions developed in Section 2 above have to be transported to this example with care. This is due to the fact that in this case positivity is now relative to the convolution product, that is, a∈ℓ(a) is positive if there exits b∈ℓ(α) such that a=b∗¯ b. And it is important to keep in mind that positivity does not imply pointwise positivity. Consider b= (1,−2,0,0,...), then b∗b= (1,−4,4,0,...) which is positive but not pointwise positive. As above, an action is defined for g∈Gand a∈G+by Lg(a) = ¯g−1∗a∗g−1= (¯g∗g)−1∗a, Geometry on the Class of Probability Densities 325 and it is clear that the group Gacts on G+transitively. The isotropy group of a∈G+is defined by Ia={g∈G|Lg(a) = a}, and clearly is independent of aand we have Lemma 5.4. With the notations above we have I1= exp(ℓa 1(α)), where ℓa 1(α) = {X∈ℓ1(α)|¯ X+X= 0} the antisymmetric elements in ℓ1(α). Comment. Since Gis open in Aits tangent space at 1 is A, and when we want to think of the elements of Aas tangent vectors, we shall denote them by capitals. Also, for a1, a2∈G+, clearly g= (a−1 1∗a2)1/2∗u with u∈I1satisfies Lg(a1) = a2. Proof: Commutativity implies that I1={g∈G|¯g∗g=1}. Since G=Ewe know that g=eX∗for some X. Since eX∗∗e¯ X∗=e0we are through. Lemma 5.5. With the notations introduced above, the following identities hold. G+=G/I,(T G)1=ℓ1(α),I ≡ (T I)1=ℓa 1(α)and for any a∈G+,(G+)a=ℓs 1(α) = {X∈ A | X=¯ X}. Proof: Let us verify the third assertion. Let u(t) be a smooth curve in I1 such that u(0) = 1 and ˙u(0)=X. Differentiate both sides of ¯u(t)∗u(t)=1 at t= 0. 5.2.1. Connection and geodesics on G+. Again we direct the reader to Section 2 above or to [KN], [CPR], or [GR] for full details. Once we know that G+is a homogeneous bundle, the procedure to define a geometry on it is standard. For a∈G+,the bundle map πa:G→G+is again πa(g) = Lg(a). Notice now that A= (T G)1=ℓ1(α) = ℓs 1(α) + ℓa 1(α) = (T G+)a⊕ I. This splitting can be transported everywhere by means of the group action. Here (T G+)awill play the role of the horizontal space and Iwill be the vertical space at every point. The 1-form connection κa: (T G+)a:→(T G)+ is now κa(X) = −1 2a−1∗X. Is easy to verify that if Dπadenotes de derivative (tangent of push forward map) of πa. Then, Dπa◦κa: (T G+)a→(T G+)ais the identity mapping. Again, the lifting of a continuously differentiable curve a(t) in G+ with a(0) = ais the curve g(t) = (a(0) ∗a(t)−1)1/2in G. Once we have this lifting, we can use the group action to define parallel transport and geodesics as in Section 2. Adapting what we did to this case we have Proposition 5.1. The geodesics in G+are the curves a(t) = a(0)∗etX∗ with X= ˙a(0) ∗a(0)−1. 326 H. Gzyl, L. Recht Proof: It is easy to verify that the equation for the geodesics ¨a(t)− a(t)−1∗˙a2(t) = 0, and then it is simpler to verify that a(0)∗etX∗satisfies the equation. Comments. For the sake of emphasis, etX∗is the element in Awith components (etX∗)n= n X k=0 tkXk∗ n/k! As in Section 2, to justify the calling a(0) ∗etX∗a geodesic, we should verify that it minimizes some distance function. To begin with, at a∈G+ define the following norm. For X∈(T G+)aset kXkα,a =ka−1Xkα, where k·kαis the norm in the Banach algebra. Let now a(t) for t∈[0,1], be a continuously differentiable curve. Define its length by l(a) = Z1 0 k˙a(t)kα,a(t)dt. Define now for given a0, a1∈G+ (17) d(a0, a1) = inf{l(c)|ccontinuously differentiable curve joining a0to a1}. It so happens that the curve a(0)∗etX∗with X= ˙a(0)a−1(0) = lna(1)∗ a(0)−1is the minimizer of (17), and therefore κis the connection in the Finsler geometry associated to d(a0, a1).We have Theorem 5.4. Let a0, a1∈G+and denote by ga0,a1(t)the curve a(0) ∗ etX∗and let c(t)be any other continuously differentiable curve joining a0 to a1in a unit of time. Then l(c)≥l(ga0,a1). Proof: As in Section 2, consider l(c) = Z1 0 k˙c(t)kα,c(t)dt =Z1 0 kc−1(t)∗˙c(t)kdt ≥   Z1 0 dln c(t)dt   α =    ln a1 a0   α =d(a0, a1). To close this section note that going from a0to a1along a geodesic can be realized by means of the group action: a1=a0∗eX∗=Lg(a0),with g=e−X/2∗, and a vector field Ydefined along a geodesic curve a(t)∈G+is parallel if Y(t) = Lg(t)(Y(0)) for every twhere g(t) is a group element taking a0 onto a1. Geometry on the Class of Probability Densities 327 5.3. Sequences of convolution type and geodesics in G+. Let us begin by recalling two results from [dB]. Theorem 5.5. Let q(t)≡ {qn(t) : n≥0}be a sequence of polynomials of convolution type, with coefficient sequence {gn:n≥0} (i.e. qn(t) = Pn k=0 gk∗ ntk/k!), and let {αn}be as above. Then the following are equivalent: a) {gn} ∈ ℓ1(α), b) there exists M > 0such that kq(t)k1,α ≤e|t|Mfor all t∈C, c) limt↓0kq(t)k1,α = 1, d) lim supt↓0kq(t)k1,α <2, e) there exist δ > 0and t0∈(0, δ)such that q(t)∈ℓ1(α)for all t∈ (0, δ)and Ψ(t0, z)≡Λz(q(t0)) 6= 0 if |z|=eρ, f) there exists t0∈Csuch that q(t0)∈ℓ1(α)and Λz(q(t0)) 6= 0 if |z| ≤ eρ, g) there exists t0∈Csuch that q(t0)∈ℓ1(α)and q(−t0)∈ℓ1(α). Moreover, if one of these conditions holds, then Pnqn(t)zn/n! = etg(z) where g(z) = Pngnznand both series converge in A(eρ)and all t∈C. From parts (a) and (b) we can safely begin manipulating our polynomial sequences without worrying much about convergence issues. And of interest here is other result from [dB]. Theorem 5.6. Let q(t)≡ {qn(t) : n≥0}be a sequence of polynomials of convolution type with coefficient sequence {gn|n≥0}. Then Λz(q(t)) = X n qn(t)zn=etg(z) where g(z) = Pngnzn. Also q0= 1 as well as qn(0) = 0 for all n≥1. Let us now go backwards. Consider a geodesic a(t) = etX∗in G+, (see Proposition 5.1) with X∈ℓ1(α) that joins a(0) = 1 to a(1) = eX. We have Lemma 5.6. With the notations above, the family a(t) = {an(t)|n≥ 0}is of convolution type with coefficient sequence {Xn|n≥0}. Proof: The claim follows from identifying the n-th coordinate of a(t+ s) = a(t)∗a(s). Lemma 5.7. With the notations introduced above, we also have Λz(etX∗) = etΛz(X). 328 H. Gzyl, L. Recht Proof: Just a simple computation. As in Section 4, we can define a k-dimensional geodesic surface in G+ by Definition 5.2. A geodesic surface generated by X= (X1,...,Xk) is a mapping Θ: Rk→ A given by t= (t1,...,tx)→eht,Xi, where ht, Xi=PitiXi. Comment. The following lemma is a variation on the theme of the previous lemma, describes how cross-sequences are related to geodesic surfaces in G+. Lemma 5.8. Let {an(t1, t2)|n≥0}be the sequence of polynomials determined by the geodesic surface, then for any (t1, t2)and (s1, s2) an(t1+s1, t2+s2) = n X k=0 ak(t1, s1)an−k(t2, s2). Proof: For the reader. Let us now consider a special class of sequences: Definition 5.3. We shall say that a differentiable curve in w:R→ℓ1(α) is a Scheffer sequence with generating sequence g∈ℓ1(α), whenever its Gelfand transform Λz(w(t)) satisfies (18) d dtΛz(w(t)) = Λz(g)Λz(w(T)). Proposition 5.2. Let q∈ℓ1(α)be the convolution sequence generated by g∈ℓ1(α). If w(t)is the convolution sequence satisfying (18), then a) wn(t+s) = Pn k=0 wk(t)qn−k(s), and in particular b) wn(t) = Pn k=0 wk(0)qn−k(t). The proof is for the reader. The geometric way to understand Scheffer sequences as geodesics is contained in Lemma 5.9. Let w0∈G+and X∈ℓs 1(α)be a tangent vector to G+, then the geodesic w(t)≡w0∗etX∗through w0is a Scheffer sequence. Proof: Just observe that Λz(w(t)) satisfies (18). Geometry on the Class of Probability Densities 329 5.4. Geodesics in Dand exponential families. We shall now consider the class of “densities” with respect to αdefined by D=(q∈G+|Eα(q)≡ ∞ X n=0 qnαn= 1). As with G+, here we differ radically from what we did above in Sections 2–4, since now positivity does not mean sequential positivity, and Dis richer than a collection of densities on N. Nevertheless, it still can be thought of as a class of representatives for P+, the projective space arising from the equivalence relation on G+defined by a1∼a2⇔ ∃ r∈(0,∞) such that a1=ra2. One can regard P+as a homogeneous reductive structure, and then try to pass on this structure on to D. But this is not necessarily possible. Instead, one can simply project the connection on G+directly on to D. This is carried out in detail in Sections 3 and 4, where it is shown that curves ρ(t) = ρ1−t 0ρt 1 Em[ρ1−t 0ρt 1] are geodesics. Actually, what changes from there to here is the nature of the product in the algebra, but in an abstract setting, things are similar. Let us briefly sketch the results. To go from G+to Dwe consider the projection Φ: G+−→ D;a−→ Φ(a) = a Eα(q). Comment. Notice that this mapping is constant along rays {ra |r > 0}, thus it makes sense to think of Das representatives for P+. Notice as well that the tangent mapping ˜ Φ≡DΦ: (T G+)a→(TD)ρ, with ρ= Φ(a) is given by (19) X−→ ˜ Φ(X) = ρX −ρEα[ρX]. Definition 5.4. With the notations introduced above, if ρ(t) is a curve in Dwith tangent Xand Y(t) is a vector field along ρ(t), tangent to D, its covariant derivative along Xis defined by ˜ DX(Y) = ˜ Φ(DX(Y)) = ρDX(Y)−ρEα[DX(Y)] where DX(Y) = d dt Lg(t)−1(Y(t))t=0. The following were proved in Section 3. The difference being that we cannot talk of true densities anymore. 330 H. Gzyl, L. Recht Theorem 5.7. Let q(t)=q(0)∗etX be a geodesic in G+passing trough q(0) at t= 0 with speed X. Then Φ(q(t)) is a geodesic in D(i.e., it satisfies ˜ D˙ρ(t)( ˙ρ(t)) = 0). Corollary 5.1. Let ρ0and ρ1be in D, then ρ(t) = ρt∗ 0∗ρ(1−t)∗ 1 Eα[ρt∗ 0∗ρ(1−t)∗ 1] is a geodesic going from ρ0to ρ1in a unit of time. Theorem 5.8. Let Ψ: t∈Rk→ρ(t)∈Dbe a continuously differentiable mapping such that Xi=∂Ψ(t) ∂tiare linearly independent in ℓ1(α). The family Π≡ {ρ(t)|t∈Rk}is exponential if and only if, for any pair t1and t2, the curve s→Ψ(st1+(1−s)t2)is a geodesic joining ρ(t1) to ρ(t2)in D. In this case Ψ(t) = Φ(eht,Xi). Comment. Notice that as stated, the result may not be applicable to probability densities. For that to be happen, we would have to add that ρ(t) is pointwise positive. How do the pointwise sequences sit in G+is an open question. Acknowledgements. We want to thank the referee for her/his comments, which contributed to clarify the presentation. References [A] S.-I. Amari,“Differential-geometrical methods in statistics”, Lecture Notes in Statistics 28, Springer-Verlag, New York, 1985. [ABKLR] S.-I. Amari, O. E. Barndorff-Nielsen, R. E. Kass, S. L. Lauritzen and C. R. Rao,“Differential geometry in statistical inference”, Institute of Mathematical Statistics Lecture Notes-Monograph Series 10, Institute of Mathematical Statistics, Hayward, CA, 1987. [B] O. E. Barndorff-Nielsen,“Information and exponential families in statistical theory”, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Ltd., Chichester, 1978. [BCR] C. Berg, J. P. R. Christensen and P. Ressel,“Harmonic analysis on semigroups”, Theory of positive definite and related functions, Graduate Texts in Mathematics 100, Springer-Verlag, New York, 1984. Geometry on the Class of Probability Densities 331 [Br] L. Brown, Sufficient statistics in the case of independent random variables, Ann. Math. Statist 35 (1964), 1456–1474. [dB] A. Di Bucchianico,“Probabilistic and analytical aspects of the umbral calculus”, CWI Tract 119, Stichting Mathematisch Centrum, Centrum voor Wiskunde en Informatica, Amsterdam, 1997. [CPR] G. Corach, H. Porta and L. Recht, The geometry of the space of selfadjoint invertible elements in a C∗-algebra, Integral Equations Operator Theory 16(3) (1993), 333–359. [Ef] B. Efron, The geometry of exponential families, Ann. Statist. 6(2) (1978), 362–376. [Es] F. Esscher, On the probability function in the collective theory of risk, Scand. Actuar. J. 15 (1932), 175–195. [GRS] I. Gelfand, D. Raikov and G. Shilov,“Commutative normed rings”, Translated from the Russian, with a supplementary chapter, Chelsea Publishing Co., New York, 1964. [GP] P. Gibilisco and G. Pistone, Connections on non-parametric statistical manifolds by Orlicz space geometry, Infin. Dimens. Anal. Quantum Probab. Relat. Top. 1(2) (1998), 325–347. [GR] H. Gzyl and L. Recht, A geometry in the space of probabilities II: Projective spaces and exponential families, Rev. Mat. Iberoamericana (to appear). [J] E. T. Jaynes, Information theory and statistical mechanics, Phys. Rev. (2) 106 (1957), 620–630. [KN] S. Kobayashi and K. Nomizu,“Foundations of differential geometry”, Vol. II, Interscience Tracts in Pure and Applied Mathematics 15, Vol. II, Interscience Publishers John Wiley & Sons, Inc., New York-London-Sydney, 1969. [K] S. Kullback,“Information theory and statistics”, 2nd. revised edition, Dover Publications, Inc., Mineola, NY, 1968. [LM] R. J. Larsen and M. L. Marx,“An Introduction to Mathematical Statistics and its Applications”, Prentice-Hall, Englewood Cliffs, N.J., 2006. [PRo] G. Pistone and M. P. Rogantin, The exponential statistical manifold: mean parameters, orthogonality and space transformations, Bernoulli 5(4) (1999), 721–760. [PS] G. Pistone and C. Sempi, An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one, Ann. Statist. 23(5) (1995), 1543–1561. 332 H. Gzyl, L. Recht [PR1] H. Porta and L. Recht, Conditional expectations and operator decompositions, Ann. Global Anal. Geom. 12(4) (1994), 335–339. [PR2] H. Porta and L. Recht, Exponential sets and their geometric motions, J. Geom. Anal. 6(2) (1996), 277–285. [S] R. F. Streater, Classical and quantum info-manifolds, Analytical study of quantum information and related fields (Japanese) (Kyoto, 2000), S¯urikaisekikenky¯usho K¯oky¯uroku 1196 (2001), 32–51. [V] I. Vajda,“Theory of statistical inference and information”, Theory and Decision Library B 11, Kluwer Academic Publishers, Dordrecht, 1989. [W] D. Williams,“Weighing the odds”, A course in probability and statistics, Cambridge University Press, Cambridge, 2001. Henryk Gzyl: Centro de Finanzas Iesa San Bernardino Caracas 1010 Venezuela Departamento de C´omputo Cient´ıfico y Estad´ıstica Universidad S´ımon Bol´ıvar Caracas Venezuela E-mail address:[email protected] L´azaro Recht: Departamento de Matem´aticas Universidad Sim´on Bol´ıvar Apartado Postal 89000 Caracas 1080-A Venezuela E-mail address:[email protected] Primera versi´o rebuda el 31 de maig de 2006, darrera versi´o rebuda el 17 de gener de 2007.