Existence and uniqueness of solutions to the Bellman equation in stochastic dynamic programming

Rincón-Zapatero, Juan Pablo Article Existence and uniqueness of solutions to the Bellman equation in stochastic dynamic programming Theoretical Economics Provided in Cooperation with: The Econometric Society Suggested Citation: Rincón-Zapatero, Juan Pablo (2024) : Existence and uniqueness of solutions to the Bellman equation in stochastic dynamic programming, Theoretical Economics, ISSN 1555-7561, The Econometric Society, New Haven, CT, Vol. 19, Iss. 3, pp. 1223-1260, https://doi.org/10.3982/TE5161 This Version is available at: https://hdl.handle.net/10419/320265 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-nc/4.0/ Theoretical Economics 19 (2024), 1223–1260 1555-7561/20241223 Existence and uniqueness of solutions to the Bellman equation in stochastic dynamic programming Juan Pablo Rincón-Zapatero Departamento de Economía, Universidad Carlos III de Madrid In this paper, we develop a framework to analyze stochastic dynamic optimization problems in discrete time. We obtain new results about the existence and uniqueness of solutions to the Bellman equation through a notion of Banach contractions that generalizes known results for Banach and local contractions. We apply the results obtained to an endogenous growth model and compare our approach with other well-known methods, such as the weighted contraction method, countable local contractions, and the Q-transform. Keywords. Stochastic dynamic programming, Bellman equation, contraction mapping, weighted contraction, local contraction, Q-transform, endogenous growth. JEL classification. C61, E21. 1. Introduction Stochastic dynamic programming incorporates uncertain events into a suitable framework to find optimal policies. A useful approach for showing the existence of optimal stationary plans is to prove that the dynamic programming equation admits a unique solution—the value function—in a suitable space of functions. See Blackwell (1965), Maitra (1968), Furukawa (1972), Bertsekas and Shreve (1978), Stokey, Lucas, and Prescott (1989), Hernández-Lerma and Lasserre (1999), or Bäuerle and Rieder (2011), where this problem is analyzed in detail. Also, there is a large amount of literature that applies stochastic dynamic programming to economics. Brock and Mirman (1972), Mirman and Zilcha (1975), Donaldson and Mehra (1983), Danthine and Donaldson (1981), Majumdar, Mitra, and Nyarko (1989), Hopenhayn and Prescott (1992), or Mitra (1998)are only a few of the many relevant papers that have contributed to developing this field of research. Olson and Roy (2006) make a review of the contributions to the stochastic optimal growth model. Many dynamic programs have both unbounded rewards and unbounded shocks, which cannot be handled by the theory initiated by Blackwell (1965), based on the properties of monotonicity and discount of the dynamic programming operator. In general, Juan Pablo Rincón-Zapatero: [email protected] This paper is based on my working papers Rincón-Zapatero (2019,2022). I acknowledge of five referees of this and of another journal for their insightful comments, which significantly improved the exposition, leading to the current version. Support from the Spanish Ministerio de Economía y Competitividad, grants PID2020-117354GB-I00, CEX2021-001181-M, MICIN/AEI/10.13039/501100011033, and Comunidad de Madrid (Spain), grant EPUC3M11 (V PRICIT), are gratefully acknowledged. ©2024 The Author. Licensed under the Creative Commons Attribution-NonCommercial License 4.0. Available at https://econtheory.org.https://doi.org/10.3982/TE5161 1224 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) a problem with unbounded utility cannot be transformed into an equivalent bounded problem, since the optimal policies of both models will differ due to the dynamic structure. Also, imposing artificial bounds to deal with a compact shock space may be incompatible with modeling uncertainty by means of a first-order stochastic process. Think, for instance, of the simple random walk. It takes every integer with positive probability. Weighted contractions, (Wessels (1977), Stokey, Lucas, and Prescott (1989), Boyd (1990), Hernández-Lerma and Lasserre (1999), or Bäuerle and Rieder (2011)), countable local contractions, (Matkowski and Nowak (2011), Ja´ skiewicz and Nowak (2011), Balbus, Reffett, and Wozny (2018)) and the recent Q-transform due to Ma, Stachurski, and Toda (2022) are useful approaches to deal with stochastic programs with unbounded rewards and shocks. The weighted norm approach needs to identify a suitable bounding function. This is not immediate in some models. Our paper provides sufficient conditions, which do not need a bounding function. In fact, we show a one-sector optimal growth model with linear technology and strictly increasing and concave utility function, which does not admit a bounding function, but our approach applies. Countable local contractions1impose, roughly speaking, that the conditional probability measures defined by the transition kernel have bounded support. This is due to the need of constructing a countable family of increasing compact sets covering the state space. We dispense with this assumption. The same growth model described above serves to show that this method gives a more restricted condition to the discount factor. The Q-transform consists in taking conditional expectations at both sides of the Bellman equation that, in some models, converts an unbounded dynamic program into a bounded one. This is a similar idea to what we do to obtain the companion operator parameter L(see Proposition 3below) but the purpose is different, as we work with the original Bellman operator. The Q-transform deals with a transformed operator. For unbounded from above rewards, the Q-transform is the same for that weighted contraction, but for unbounded from below rewards it may take advantage of the averaging operation to obtain a bounded program. We present a quadratic example where it is not possible to apply the Q-transform, nor the countable contraction approach, but our results show that the Bellman equation defines a contraction mapping.2 Our aim is to develop a new framework to study programs with unbounded rewards and/or unbounded shocks by extending the local contraction method developed in Rincón-Zapatero and Rodríguez-Palmero (2003,2009)andMartins da Rocha and Vailakis (2010) for deterministic programs to the stochastic setting, while preserving the 1The local contraction approach generalizes the Banach contraction principle for function spaces whose topology is defined by a family of seminorms. Had˘ zi´ c and Stankovi´ c(1970) is one of the first papers dealing with this extension. Rincón-Zapatero and Rodríguez-Palmero (2003,2007,2009), independently, introduced different hypotheses and applied the results to the deterministic Bellman and Koopman’s equations. Martins da Rocha and Vailakis (2010) extended the theory to the case of an uncountable family of seminorms. 2It is worth noting that the results in Ma, Stachurski, and Toda (2022) and Ja´ skiewicz and Nowak (2011) may be applied to unbounded from below programs where the utility function may take the value −∞ on the state space; this is beyond the scope of this paper. Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1225 monotonicity of the Bellman operator. To this end, we define a suitable space of functions and a suitable family of seminorms. The seminorms combine the usual supremum norm in the endogenous variables with an L1norm in the exogenous variables, and define a complete space of functions—a Carathéodory function space. To work within this framework, we need to extend the notion of contraction mapping by considering the contraction parameter(s) in the local contraction definition as an operator acting on the family of seminorms. This operator is what we call the companion operator associated with the contraction mapping.3Thetheorywedevelopis quite general and could be applied to other equilibrium problems in economics beyond stochastic dynamic programming. Our framework allows us to relax continuity of the period utility function with respect to the exogenous variable. This is important since Feller continuity of the Markov chain is not enough to preserve continuity when the space of shocks is not compact. Also, the L1type norm defined on shocks makes it possible to obtain less restrictive bounds on the discount factor than with other known methods, as it is demonstrated in the paper. Our approach is designed for problems where utility functions may be unbounded from below, but not taking the value −∞ on the state space. This is restrictive, as it takes out of consideration important problems in economics. Nevertheless, we still get new insights in better-behaved models.4 The paper is organized as follows. Section 2develops a theory of contraction mappings on topological spaces whose topology is given by a family of pseudometrics that makes it Hausdorff and sequentially complete. The contraction parameter is given by an operator acting on pseudometrics. Section 3applies the results of Section 2to the stochastic dynamic programming equation for models with shocks driven by an exogenous Markov chain. The main assumption used to obtain our results states that today’s conditional expectation of the utility function is bounded by the present value of tomorrow’s conditional expectation, in such a way that the resulting infinite sum of all expected values is finite. In Section 4, we study a model of endogenous growth, allowing for correlated and unbounded shocks. Section 5makes a comparison of our results with those obtained with weighted contractions, countable local contractions, and the Q-transform addressed above. Section 6concludes. Appendices Aand Bcontain the proofs not in the main text of Sections 2and 3, respectively. Appendix Cprovides a pure currency model where the value function is discontinuous with respect to the shock variable, showing in a simple economic model the well-known fact that Feller continuity 3This idea is not new. Kozlov, Thim, and Turesson (2010) developed a fixed-point theorem in locally convex spaces whose topology is given by a family of seminorms. However, the results obtained depend on the companion contraction parameter operator being linear, precluding application to the dynamic programming equation, since it genuinely demands a nonlinear companion contraction parameter, due to the presence of a maximization operation in the definition of the Bellman equation. 4To adapt our results to this class of models, it may be promising to work with pseudometrics, instead of seminorms, as in Rincón-Zapatero and Rodríguez-Palmero (2003); this paper does not explore this issue. Also, the theory we develop applies only to problems where uncertainty is exogenous. However, to extend the approach to models where actions affect uncertainty is possible, and in fact Rincón-Zapatero (2022) drafts how it could be done. 1226 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) of the Markov chain is not enough to preserve continuity when the shock space is not compact. 2. A general class of Banach contractions Let (E,D)be a topological space, where Eis a set whose topology is generated by a saturated family of pseudometrics D={da}a∈A,withAan arbitrary index set. Since the family Dis saturated, the topology it generates is Hausdorff.5We suppose that (E,D) is sequentially complete: if {xn}is a sequence in E, which is Cauchy with respect to all da∈D,thatis,ifda(xn,xm)→0asn,m→∞, then there is x∈Esuch that da(xn,x)→0 as n→∞for all a∈A. Given a sequentially complete subset F⊆E, we study the existence and uniqueness of a fixed point of a mapping T:F→E. Let RAbe the set of functions d:A→R+and let RA +be the nonnegative cone of RA. On this set, we consider the order it generates, that is, for two elements d,d∈RA +,we say that d≤dif and only if d(a)≤d(a)for all a∈A. The family Dcan be embedded into RA +, since that, for x,y∈Egiven, the mapping a→ da(x,y)defines a function in RA +,thatwedenotedx,y(a):=da(x,y). In general, for a given subset F⊆E,weletD(F) be the set of functions in RA +, which are generated by pairs x,y∈F,thatis, D(F):=d:A→R+:d=dx,yfor some x,y∈F. Definition 1. Let F⊆E. The mapping T:F→Eis an L-local contraction on Fwith contraction operator parameter (COP) L, if there are a set C⊆RA +such that D(F)⊆C, and an operator L:C→RA +,suchthat da(Tx,Ty)≤Ldx,y(a), for all x,y∈Fand for all a∈A. Note that the inequality above can be rewritten dTx,Ty ≤Ldx,y, that is, as an order relation in the space RA +. The definition of L-contractions for mappings T:F−→ E, not imposing T:F−→ F, will facilitate the definition of the COP parameter Lof the Bellman operator in Section 3. Of course, the property T:F−→ Fis fundamental for Theorem 2below, and will be checked carefully in Section 3. The following two examples show that the operator Lis a generalization of the concept of contraction parameter of a (local) contraction mapping. 5Apseudometricd:E×E→R+is a function satisfying d(x,y)≥0, d(x,x)=0, d(x,y)=d(y,x), and d(x,z)≤d(x,y)+d(y,z)for any x,y,z∈E,butd(x,y)=0 does not imply x=y. The family Dof pseudometricsissaturatedifda(x,y)=0foralla∈Aimplies x=y. Sometimes, the pseudometrics are defined through seminorms pa,a∈A,byda(x,y)=pa(x−y),wherenowEis a real vector space. A seminorm is afunctionp:E→R+that satisfies all the axioms to be a norm, except that p(x)=0 does not imply that x is the null vector of E. If the family of seminorms is saturated, then the topology defined by the family is Hausdorff and the space Eis a locally convex space. See Willard (1970) for further details. Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1227 Example (Banach Contractions). In the classical Banach’s theorem, Eis endowed with a complete metric d, so the index set Ais a singleton, D={d},andTis a contraction of constant parameter β,with0<β<1: d(Tx,Ty)≤βd(x,y),foranyx,y∈E.TheCOPis L=βI,whereIis the identity map in R+. A generalization of the Banach contraction concept is provided in Wong (1968), where it is considered T:E−→ Efor which there is a function L:R+−→ R+satisfying d(Tx,Ty)≤Ld(x,y),(1) for all x,y∈E. Note that our definition is an extension of this concept to topological spaces whose topology is given by a family of pseudometrics. ♦ Example (k-Local Contractions). Suppose that A=Nis countable. In Rincón-Zapatero and Rodríguez-Palmero (2003,2007), we introduced the concept of k-local contraction in the study of the deterministic Bellman and Koopmans equations, respectively. A klocal contraction on F,k=0, 1, 2, is a mapping T:F⊆E−→ Esatisfying dj(Tx,Ty)≤βjdj+k(x,y) forsomefixedsequenceofnumbers{βj}j∈Nwith 0 <β j<1, and for all x,y∈F.Ifwelet s=RNbe the set of real sequences and s+be the subset of sof nonnegative sequences, then the COP associated with Tis the linear operator L:s+−→ s+acting on sequences given by L(d1,d2,,dj,)=(β1d1+k,β2d2+k,,βjdj+k,), where k≥0isfixed. Suppose that Ais uncountable and let a mapping α:A−→ A.Martins da Rocha and Vailakis (2010) worked with the following generalization of the countable class above: T:E−→ Eis an α-local contraction if there exists a function β:A−→ [0, 1)such that da(Tx,Ty)≤β(a)dα(a)(x,y). The COP Lacts on functions d:A−→ R+by translation in the independent variable by α, and a multiplication by β,thatis,(Ld)(a)=β(a)d(α(a)).ItturnsoutthatLis also a linear mapping, as in the countable case above. ♦ In what follows, we use the standard notation for successive iterations of the operators Tand L. For instance, L0is the identity operator on C,L1=L, and for t≥2, Lt=L◦Lt−1.WeimposetoC,L,andTthe Assumptions (I) to (VI) listed below. The Assumptions (I) to (V) concern the behavior of Lon the set C. Assumption (VI) links directly the operators Tand L. (I) D(F)⊆C(hence the null function 0∈C). For all d,d∈C, the sum d+d∈C,and any bounded subset of Cis countable chain complete.6Moreover, if d∈C,d∈RA +, and d≤d,thend∈C. 6AsubsetS⊆Cis bounded with respect to the order inherited from RAif there is d∈Csuch that d≤d for all d∈S. The bounded subset Sis countably chain complete if for any countably chain d1≤d2≤···dt≤ ··· in S,supt∈Ndt∈S. 1228 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) (II) L(C)⊆C;L0=0. (III) Lis monotone: for all d,d∈Cwith d≤d,Ld ≤Ld. (IV) Lis subadditive:for any d,d∈C, Ld+d≤Ld +Ld. (V) Lis upper semicontinuous sup-preserving:7for any bounded countable chain in C,d1≤d2≤···≤dt≤···, Lsup t dt≤sup t Ldt. (VI) There are x0∈Fand r0∈Cwith da(x0,Tx0)≤r0(a)and R0(a):= ∞  t=0 Ltr0(a)<∞, for all a∈A. Since Ltr0∈C,forallt=0, 1, , and the countable chain {r0,r0+Lr0,,r0+Lr0+ ···+Ltr0,}is bounded in Cby (VI), R0is in Cby Assumption (I). For F⊆E,x0∈F,andm∈RA +, let the set VF(x0,m)=x∈F:da(x0,x)≤m(a),∀a∈A.(2) When Eis a metric space, that is, when Ais a singleton, the pseudometric is a metric, and VF(x0,m)is simply the intersection with Foftheclosedballcenteredatx0and radius m. Let N0:={0}∪Nand set J:=A×N0. Consider next the family of pseudometrics :=(δj)j∈Jon Fwhere δa,t(x,y)=Ltdx,y a. Assumptions (I)–(IV) imply that δjis a pseudometric while a straightforward computation shows that δa,t(Tx,Ty)=Ltda(Tx,Ty)≤Lt+1da(x,y))=δa,t+1(Tx,Ty). Now let the map r:J→Jbe defined by r(a,t)=(a,t+1).ThenTis a local contraction with respect to (,r).8 7For instance, the sup-preserving property, L(suptdt)=suptLdt, plays a prominent role in the fixedpoint theorem of Kantorovich–Tarski. In our context, it can be weakened to a kind of upper semicontinuity. 8This construction was shown to the author by a referee. It allows us to apply directly Martins da Rocha and Vailakis (2010, Theorem 2.1) to obtain existence and uniqueness of the fixed point. However, in some cases, the family could be not saturated or not defining a sequentially complete topology. An example within the dynamic programming class is as follows. Suppose a deterministic problem with X=R+and (x)={x+1}. Then, for all compact sets Kof Xand all functions p=pf,withfcontinuous, Lp(K)= βmaxx∈Kp((x))=βpK+1and Ltp(K)=βtpK+t,whereK+t={x+t:x∈K},forallt=1, 2, It is clear Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1229 Theorem 1. Let (E,D)be a Hausdorff and sequentially complete topological space. Let T:F→Fbe an L-local contraction on the sequentially complete subset F⊆Eand let x0∈Fbe such that (I)–(VI) hold true. Suppose that the family of pseudometrics defined above is saturated and the space (E,)is complete. Then there is a unique fixed point x∗∈VF(x0,R0)of T, which is the limit of any iterating sequence yt+1=Tyt,t=0, 1, 2, , where y0=x∈VF(x0,R0)is arbitrary. Proof. The result follows from Martins da Rocha and Vailakis (2010, Theorem 2.1), with K=VF(x0,R0). See Lemma 2in Appendix A. The next result is a corollary to the above theorem that provides conditions for the uniqueness of the fixed point in Fand not only in VF(x0,R0). When Tis indeed an L-local contraction on the whole E, this result provides global uniqueness of the fixed point on E. Corollary 1. Let (E,D)be a Hausdorff, sequentially complete space. Let T:F→Fbe an L-local contraction on the sequentially complete subset F⊆Eand let x0∈Fbe such that (I)–(VI) hold true. Suppose that for all x∈Fthere exists r0∈Csatisfying (VI) such that x∈VF(x0,R0),whereR0=∞ t=0Ltr0. Then there is a unique fixed point of Tin F and convergence to the fixed point of successive iterations of Tis attained from any x∈F. Next, we establish a useful sufficient condition for (VI). Note that the Bellman operator satisfies the extra condition imposed on L. Proposition 1. Let (E,D)be a Hausdorff and sequentially complete topological space. Let T:F−→ Fbe an L-local contraction on F⊆E,withCOPLsatisfying (I) to (V) and L(αd)≤αLd, for all d∈C, for all α∈[0, 1].Letx0∈F, for which there is t0∈{0, 1, 2, }, s∈C,andθ∈[0, 1)such that Lt0d0≤sand Ls ≤θs, where d0(a)=da(x0,Tx0).Then(VI)holdswithr0=d0. 3. Stochastic dynamic programming and Bellman equation Consider a dynamic programming model (X,Z,,Q,U,β),whereX×Zis the set of possible states of the system, is a correspondence that assigns a nonempty set (x,z)of feasible actions to each state (x,z),andQis the transition function, which associates a conditional probability distribution Q(z,·)on Zto each z∈Z.Hence, the law of motion is assumed to be a first-order Markov process, which could be degenerate, giving rise to a deterministic model. We will use indistinctly the notation that {δK,t}is not a saturated family of seminorms. Supposing that fand gare continuous functions on R+, such that f= gin [0, 1]and f=gin (1, ∞);yet,δK,t(f−g)=βtpK+t(f−g)=0forallKand all t=1, 2,  A direct approach, without the use of ,isshowninRincón-Zapatero (2022). The applications we study in further sections have both saturated and complete. 1230 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) Qz(·)=Q(z,·); the function Uis the one-period return function, defined on the graph of ,={(x,y,z):(x,z)∈X×Z,y∈(x,z)},andβis a discount factor. Starting at some state (x0,z0), the agent chooses an action x1∈(x0,z0), obtaining areturnofU(x0,x1,z0)and the system moves to the next state (x1,z1),whichisdrawn according to the probability distribution Q(z0,·). Iteration of this process yields a random sequence (x0,z0,x1,z1,)and a total discounted return ∞ t=0βtU(xt,xt+1,zt).A history of length tis zt=(z0,z1,,zt).LetZtbe the set of all histories of length t,and let Zt=Z×···×Z(ttimes), where Zis the Borel σ-algebra of Z. A (feasible) plan πis a constant value π0∈Xand a sequence of measurable functions πt:Zt−→ X,suchthat πt(zt)∈(πt−1(zt−1),zt),forallt=1, 2, .Denoteby(x0,z0)the set of all feasible plans starting at the state (x0,z0). Any feasible plan π∈(x0,z0), along with the transition function Q, defines a distribution Pπ,(x0,z0)on all possible futures of the system {(xt,zt)}∞ t=1, as well as the expected total discounted utility u(π,x0,z0)=Eπ,(x0,z0)∞  t=0 βtU(xt,xt+1,zt). The expectation Eπ,(x0,z0)is taken with respect to the distribution Pπ,(x0,z0).Theproblem is then to find a plan π∈(x0,z0)such that u(π,(x0,z0)) ≥u(π,(x0,z0)) for all π∈(x0,z0),forall(x0,z0)∈X×Z. The value function of the problem is v(x0,z0)= supπ∈(x0,z0)u(π,(x0,z0)). Consider the functional equation corresponding to the above dynamic programming problem as stated in Stokey, Lucas, and Prescott (1989). For x∈X,z∈Z, v(x,z)=sup y∈(x,z)U(x,y,z)+βZ vy,zQz,dz.(3) A solution of the Bellman equation satisfying additional assumptions is the value function of the infinite programming problem. This is the content of Theorem 2below, whose proof needs the notion of the probability measure μtdefined on the sequence space of shocks (Zt,Zt)for finite t=1, 2, ,where Zt,Zt=(Z×···×Z,Z×···×Z)( ttimes) and where Zis defined in (B1) below. For any rectangle B=A1×···×At∈Zt,μtis defined by μt(z0,B)=A1 ···At−1At Qzt−1(dzt)Qzt−2(dzt−1)···Qz0(dz1), and by the Hahn extension theorems, μt(z0,·)has a unique extension to a probability measure on all of Zt. We omit the details, which can be found in Stokey, Lucas, and Prescott (1989), Section 8.2, whose presentation we follow closely. Defining the Bellman operator in a suitable function space E,suchthatforf∈E, (Tf )(x,z)=sup y∈(x,z)U(x,y,z)+βZ fy,zQz,dz, Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1237 ≤Z β ∞  s=t lsπ0(z0),z1Qz0(dz1) ≤ ∞  s=t+1 ls(x0,z0), again by the monotone convergence theorem, and where we haveused Fubini’s theorem and the induction hypothesis. This and (9) imply (iii), since the series w0converges. Thus, v∗is the value function. The claims about ∗are immediate from the theorem of the maximum of Bergé and the measurable maximum theorem; see Aliprantis and Border (1999). The following result provides a sufficient condition for (B6). Proposition 4. Let Assumptions (B1) to (B5) hold. Suppose that there is l0∈L1(Z; C(X)) with |ψ|≤l,α≥0such that αβ < 1,and Z max y∈(x,z)l0y,zQzdz≤αl0(x,z), for all x∈X,z∈Z. Then (B6) holds, with R0(K,z)=pK,z(l0)/(1−αβ). Proof. Choose lt=(αβ)tl0,fort=0, 1, Then βZ max y∈(x,z)lty,zQzdz=β(αβ)tZ max y∈(x,z)l0y,zQzdz ≤(αβ)t+1l0(x,z)=lt+1(x0,z0). Hence, w(x0,z0)=l0(x0,z0)/(1−αβ )and R0(K,z)=pK,z(l0)/(1−αβ)for K∈Kand z∈Z. 3.1 Sharper estimates Many interesting problems have utility functions, which are unbounded from below.11 In this case, the estimates given in Assumption (B6) are not efficient. A generalization of (B6) allows us to construct sharper estimates in the form of an orderinterval of functions that is mapped into itself by the operator T. (B6’) There are two collections of functions kt,lt:X×Z→R,withkt,lt∈L1(Z; C(X)),forallt=0, 1, , satisfying that for all x∈X,allz∈Z,thereexists y(x,z)∈(x,z)such that k0(x,z)≤minUx,y(x,z),z,0 ; 11We refer here to problems where the utility function is not bounded from below, but never takes the value −∞. 1238 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) kt+1(x,z)≤βZ kty(x,z),zQzdz; l0(x,z)≥maxψ(x,z),0 ; lt+1(x,z)≥Z max y∈(x,z)lty,zQzdz,allt=1, 2, ; and the series ut(x,z):= ∞  s=t ks(x,z)and wt(x,z):= ∞  s=t ls(x,z) are unconditionally convergent, for all t=0, 1, 2,  Let Iu0,w0={f∈L1(Z;C(X)) :u0≤f≤w0}. The following result states Theorem 2 in the more restricted space of functions VIu0,w0, thus providing a method to get the contraction property of the Bellman operator when Uis unbounded from below but never takes the value −∞ on . In the following theorem, we let lt=max{|kt|,|lt|}for all t=0, 1, and  R0(K,z)=∞ t=0pK,z( lt)for all K∈Kand z∈Z. Theorem 3. Suppose that (B1)–(B5) and (B6’) hold. Then Theorem 2holds with VIu0,w0(0,  R0), replacing V(0,  R0). Proof. By Lemma 7in Appendix B,Tis a self-map on Iu0,w0.Noticethat|ψ(x,z)|≤  l0(x,z). Hence, (B6) holds true for  ltby the definition of  ltand Theorem 2applies in VIu0,w0(0,  R0). In what follows, to simplify the exposition, we introduce the following notation: for afunctionf∈Ca(X×Z),  fx,z,z=max y∈(x,z)fy,z. (10) 4. Application to endogenous growth Endogenous growth models have become fundamental to understand economic growth. Several contributions consider an unbounded shock space, as in Stachurski (2002), Kamihigashi (2007), Matkowski and Nowak (2011), or Bäuerle and Ja´ skiewicz (2018). I consider here the stochastic endogenous growth model studied in Jones, Manuelli, Siu, and Stacchetti (2005), which is described as follows. The preferences of the agent over random consumption sequences are given by max E ∞  t=0 βtc1−σ tυ(t) 1−σ, (11) subject to ct+kt+1+ht+1≤ztAkα t(ntht)1−α+(1−δk)kt+(1−δh)ht, (12) Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1239 t+nt≤1, (13) ct,kt,ht,t,nt≥0 (14) for all t=0, 1, ,withk0and h0given. Here, {zt}is a Markov stochastic process with transition probability Qz(·)and Z=(0, ∞);ctis consumption; tis leisure; ntis hours spent working; ktand htare the stock of physical and human capital, respectively; δk and δhare the depreciation rates on physical and human capital, respectively; and υis a continuous function on (0, 1], strictly increasing. The usual nonnegativity constraints on consumption, investment, leisure, and hours worked apply. If we let k=kt+1,h= ht+1,andk=kt,h=ht,c=ct,n=nt,and=t, then the feasible correspondence is (k,h,z)=k,h:Therearec,n,such that (12)–(14)hold , and the utility function is U(c,)=c1−συ()/(1−σ). Regarding the function υ,weconsider υ()=ψ(1−σ). The endogenous state space is X=[0, ∞)×[0, ∞)and the family of compact sets Kis formed by compact sets in the product space R+×R+.TheMarkov chain is given by the log-log process lnzt+1=ρln zt+lnwt+1, (15) with ρ≥0andwherethew’s are i.i.d., with support in W⊆(0, ∞).Letμbe the distribution measure of the w’s. Note that ρ=0 corresponds to shocks ztthat are i.i.d. Jones et al. (2005) suppose that zt=exp(ζt−(1/2)σ2 /(1−ρ2)),whereζt+1=ρζt+t+1and the ’s are i.i.d., normal with mean 0 and variance σ2 . This corresponds to (15)with wt+1=exp(t+1−(1/2)σ2 /(1+ρ)).Wedonotrestrictto be normally distributed. To shorten notation, we will use the following definitions in this subsection: γ=αα(1−α)1−α, δ=min{δk,δh}, ν=(1−δ)1−σ, E= ∞  s=0 Ewρs(1−σ). In this section, convergence means convergence with respect to the seminorms pK,z,K∈K,z∈Z. We only assume that the expectation Ewis finite. Proposition 5. Consider the endogenous growth model described in (11)–(15)with0≤ σ<1and 0≤ρ<1.If βAγ(Ew)1/(1−ρ)1−σ+ν<1, (16) then the associated Bellman equation admits a unique solution, v∗, in the set V(0, R0) defined in (2)givenby V(0, R0)=f∈L1Z;C(X):pK,z(f)≤R0(K,z),∀(K,z)∈K×Z, 1240 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) where for K∈Kand z∈Z,R0(K,z)=pK,z(∞ t=0lt), and the family {lt}∞ t=0is given by (17) and (21) in the proof below. Moreover, v∗is the value function vand Tnv0converge to v as n→∞for all initial guesses v0∈V(0, R0). Finally, the optimal policies are continuous functions of (k,h). Proof. We check all the hypotheses of Theorem 2. It is clear that (B1)–(B5) are fulfilled. We focus on (B6) and define g(k,h,z)=Azkαh1−α+(1−δ)(k+h). Since 0 ≤σ<1, both Uand υare bounded from below by zero, and υis bounded above by 1. By the definition of δ,wehavezAkα(nh)1−α+(1−δk)k+(1−δh)h≤g(h,k,z).Then ψ(k,h,z)=max (k,h,c,n,)∈(k,h,z)u(c,)≤1 1−σg(k,h,z)1−σ≡l0(k,h,z). (17) According to (10), let  l0k,h,z,z=max (k,h,c,n,)∈(k,h,z)l0k,h,z. Let us determine a bound for l0(k,h,z,z). To this end, consider the Lagrange problem max l0k,h,z, (18) such that k+h≤g(k,h,z), (19) k,h≥0, (20) and notice that its feasible set is larger than (k,h,z). The constraint is binding at the optimal solution, which is k=αg(k,h,z),h=(1−α)g(k,h,z). Substituting this into the objective function of (20), we find its optimal value, which is Azαα(1−α)1−α+(1−δ)1−σl0(k,h,z)≤A1−σz1−σγ1−σ+νl0(k,h,z), where the inequality is due to the function c→ c1−σbeing subadditive, as it is concave and null at zero. Computing the conditional expectation of the right-hand side of the above inequality, we have Z l0k,h,z,zQzdz≤A1−σzρ(1−σ)Ew1−σγ1−σ+νl0(k,h,z). Thus, we define l1(k,h,z)=β(A1−σzρ(1−σ)Ew1−σγ1−σ+ν)l0(k,h,z). To calculate  l1(k,h,z,z), we note that we face the same Lagrange problem (20) above, modulo the “constant” factor β(A1−σ(z)ρ(1−σ)Ew1−σγ1−σ+ν). Thus, we will find  l1k,h,z,z≤A1−σzρ(1−σ)Ew1−σγ1−σ+νl1(k,h,z) and then Z l1k,h,z,zQzdz≤A1−σzρ2(1−σ)Ew1−σEwρ(1−σ)γ1−σ+νl1(k,h,z). Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1241 Now define l2(k,h,z)=β(A1−σzρ2(1−σ)Ew1−σEwρ(1−σ)γ1−σ+ν)l1(k,h,z). Using induction,itcanbeprovedexactlythesameasforthecasest=1andt=2, that the family of functions {lt}∞ t=0given by lt+1(k,h,z)=βA1−σzρt+1(1−σ)Ew1−σEwρ(1−σ)···Ewρt(1−σ)γ1−σ+νlt(k,h,z), (21) for all t=0, 1, , satisfies the inequalities demanded in (B6). Regarding the series w0(k,h,z)=∞ t=0lt(k,h,z), note that the ratio lt+1(k,h,z) lt(k,h,z)=βA1−σzρt+1(1−σ)Ew1−σEwρ(1−σ)···Ewρt(1−σ)γ1−σ+ν, converges to βA1−σEγ1−σ+νas t→∞, which is smaller than one by the assumption of the theorem, since by Jensen’s inequality E≤(Ew)(1−σ)/(1−ρ).Thus,bytheratio test, the series converges pointwise. It is easy to see that inequality (16) guarantees unconditional convergence as well, thus (B6) is fulfilled. To complete the proof, Theorem 2shows that the optimal policy correspondence is nonempty and upper hemi– continuous. The convex maximum theorem of Bergé assures that the value function is concave and since the utility function is strictly concave, the optimal policies are unique, and so continuous with respect to (k,h). 5. Comparison with other approaches In this section, we compare our results with those obtained by other methods: the already classical weighted norm approach, the one based on countable local contractions, and the recent Q-transform. In each case, we give a brief description of the method, and then we work through model examples showing the differences with our method. A word of caution is needed here. As said in the Introduction, we study dynamic problems where the actions do not influence the evolution of uncertainty; it is exogenous. The present paper’s aim is to take advantage of the special structure of the state space of the kinds of problems that we analyze to obtain further insights. 5.1 The weighted norm approach In the weighted contraction approach (see Boyd (1990), Becker and Boyd (1997)forthe deterministic case, and Hernández-Lerma and Lasserre (1999), Ja´ skiewicz and Nowak (2011)andBäuerle and Rieder (2011) for the stochastic case), it is postulated that the existence of a continuous function ϕ:X×Z:−→ R++, called the bounding or weighing function, such that there exist nonnegative constants Mand αsuch that for all x∈X, z∈Z,y∈(x,z), (W1) |U(x,y,z)|≤Mϕ(x,z)and (W2) Zϕ(y,z)Qz(dz)≤αϕ(x,z). Given such a function ϕ, the following Banach space is considered: Cϕ=f:X×Z−→ Rcontinuous : sup (x,z)∈X×Zf(x,z) ϕ(x,z)<∞, where the norm is defined by fϕ=sup(x,z)∈X×Z(|f(x,z)|/ϕ(x,z)). 1242 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) Consider the following result that can be found in Hernández-Lerma and Lasserre (1999), Section 8.3, or in Bäuerle and Rieder (2011), Theorem WSN, p. 208. Theorem 4. Let ϕbe a continuous function satisfying (W1)–(W2) above, such that αβ < 1. (22) If (WC1) is nonempty and continuous and (x,z)is compact valued for all (x,z)∈ X×Z; (WC2) Uis continuous; (WC3) (x,z)−→ Zϕ(x,z)Qz(dz)is continuous; then (a) the value function is the unique continuous solution to the Bellman equation in Cϕ, (b) value function iteration converges from any initial f∈Cϕ, (c) at least one optimal policy exists, and (d) that policy maximizes the right-hand side of the Bellman equation. The conditions of Theorem 4are stricter than those of Theorem 2. To see this, we can take l0=ϕin Proposition 4above to construct the family {lt}∞ t=0such that Theorem 2applies. The opposite is not true, that is, Theorem 2is not contained in Theorem 4.First, Theorem 2does not impose continuity in both variables (x,z),asrequiredin(WC3). We show in Appendix Ca simple pure currency model where the value function is discontinuous with respect to the exogenous variable and where no bounding function ϕ may satisfy (WC3). Second, even if (WC3) is fulfilled, the uniqueness of the solution to the Bellman equation is given in a larger space, since Ca(X×Z)contains Cϕ(X×Z), for any bounding function ϕ. Third and last, more importantly beyond the issues of continuity just discussed, there are bounded from below models for which a bounding function cannot exist but Theorem 2is applicable. Since that, in principle, there are infinitely many candidates for bounding functions, to find a model with returns bounded from below, for which there is not a suitable bounding function, that is not an easy task. We will use the result below, which states a necessary condition for the existence of a bounding function ϕ. For a function fdepending on the variables (x,z), remind the notation  f(x,z,z)= maxy∈(x,z)∈X×Zf(y,z). Also, define 0 0=1. Proposition 6. Let there be a dynamic programming problem (X,Z,,Q,U,β)for which there is a continuous bounding function ϕsatisfying the conditions of Theorem 4. Define the family of functions {lt}∞ t=0,withl0∈Cϕ(X×Z)and lt+1=βZ ltQz. Then there is a constant Msuch that ∞  t=0t+r rlt≤M (1−αβ)r+1ϕ,for all r=0, 1,  (23) Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1243 Proof. We often eliminate the arguments (x,z)in what follows to simplify notation. Note that l1=βZ l0Qz≤MβZϕQz≤M(αβ)ϕfor some constant Mand αβ < 1, since l0∈Cϕ(X×Z)and ϕis a suitable bounding function. By induction, lt≤M(αβ)tϕ,for all t.Hence,w0=∞ t=0lt≤Mϕ/(1−αβ)is finite and Z w0Qz≤M (1−αβ)ZϕQz≤Mα (1−αβ)ϕ. Let w1=∞ s=1ls. Then, as in the proof of Theorem 2,wehave Z w0Qz=Z ∞  t=0 ltQz= ∞  t=0Z ltQz=1 β ∞  t=0 lt+1=1 βw1. Thus,wehaveobtainedw1(x,z)≤Mϕ(x,z)αβ/(1−αβ). In general, the following inequality holds: wt(x,z)≤M(αβ)t 1−αβ ϕ(x,z),forallt=0, 1, , (24) where wt=∞ s=tls. To show this, we use the principle of induction. The cases t=0, 1 have just been proved. Suppose that it is true for t.Then Z wtQz≤M(αβ)t 1−αβ ZϕQz≤M(αβ)t 1−αβ αϕ, and on the other hand, as in the computation above, Z wtQz=Z ∞  s=t lsQz= ∞  s=tZ lsQz=1 β ∞  s=t ls+1=1 βwt+1, thus we get the inequality sought. Adding (24)fromt=0tot=∞,weobtain 12 ∞  t=0 (t+1)t(x,z)≤M (1−αβ)2ϕ(x,z)<∞. (25) From (25), replacing xby xand zby zand integrating with respect to Qzin both sides of the inequality and using the properties of ϕ,wehave Z ∞  t=0 (t+1) ltQz= ∞  t=0 (t+1)Z ltQz=1 β ∞  t=0 (t+1)lt+1≤Mα (1−αβ)2ϕ. Thus, ∞ t=0(t+1)lt+1≤Mϕαβ/(1−αβ)2,whichis(23)withr=1. Repeating the scheme above ssteps, we get ∞ t=0(t+1)lt+s≤Mϕ(αβ)s/(1−αβ)2. Adding in sagain as in (25), we obtain ∞  t=0t+2 2lt= ∞  t=0 (t+2)(t+1) 2lt≤M (1−αβ)3ϕ. 12w0+w1+w2+···=(l0+l1+l2+···)+(l1+l2+···)+(l2+···)+···=l0+2l1+3l2+···. 1244 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) By induction, and using the same arguments as for the cases r=1andr=2, it can be proved that for any r≥0, ∞  t=0t+r rlt≤M (1−αβ)r+1ϕ. This result will be used to show that the weighted norm approach cannot be applied to the following simple growth model with a linear technology, multiplicative shocks, and a strictly increasing, strictly concave and bounded from below felicity function (but our approach works). Example. Consider an optimal growth model, which the Bellman equation is v(k,z)=max k∈[0,zk]Uk,k,z+βZ vk,zQzdz, where k∈[0, ∞),Z={1, g},withg>1, Qz:=Qis given by Q(g)=p>0, Q(1)=q>0, with p+q=1, and the utility function is U(k,k,z)=u(zk −k),where u(c)=1+c 3+ln2(1+c)−1 3. The function uis nonnegative, continuous, unbounded, strictly increasing, and strictly concave on [0, ∞),withu(0)=0.13 The discount factor is taken to be β≤1/(gp +q)<1. Clearly, any solution of the Bellman equation has v(0, z)=0, for all k,z.Letk>0andz∈{1, g}.Withaviewtouse the necessary condition in Proposition 6,letustake0(k,z)=maxk∈[0,zk]U(k,k,z)= u(zk). It is clear that l0≤Mϕ for M=1, since l0=ψ≤ϕby definition of the bounding function ϕ.Let 1(k,z)=βZ max k∈[0,zk]0k,zQdz=βpu(gzk)+qu(zk), 2(k,z)=βZ max k∈[0,zk]1k,zQdz=β2p2ug2zk +pqu(gzk)+qpu(gzk)+q2u(zk) . . . In general, t(k,z)=βtt s=0t spsqt−su(gszk),forallt, which follows by induction. We have ∞  t=0 lt(k,z)= ∞  t=0 t  s=0t s(βp )s(βq)t−sugszk= ∞  s=0 (βp )sugszk∞  t=st s(βq)t−s, 13Letting c>0, denote x=ln (1+c)>0. Then u(c)=(3+x2−2x)/(3+x2)2>0 and u(c)=−2e−x((x− 1)3+2)/(3+x2)3<0. Also, note that uis unbounded but u(c)→0asc→∞. For another example where there is no suitable bounding function ϕwhatever the value of the discount factor β,seeRincón-Zapatero (2022). Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1245 where the change of order summation is admissible since the double series is of positive terms. Now ∞  t=st s(βq)t−s=1 s! ∞  t=s t(t−1)···(t−s+1)(βq)t−s. The series at the right-hand side is the value of the sth derivative of the power series ∞ t=0xt=1/(1−x), evaluated at x=βq;hence,∞ t=st(t−1)···(t−s+1)(βq)t−sequals (ds/dxs)(1/(1−x))|x=βq =s!/(1−βq)s+1.Thus, ∞  t=0 lt(k,z)=1 (1−βq) ∞  s=0βp 1−βqs ugszk. (26) Let us see that this series converges for all k>0andz∈Z.Notethat ugszk=1 3+ln21+gszk+gszk 3+ln21+gszk−1 3 Thus, the series (26) decomposes into the sum of three series. The first and the third series are 1 (1−βq) ∞  s=0βp 1−βqs1 3+ln21+gszkand −1 3(1−βq) ∞  s=0βp 1−βqs , respectively, which are convergent, since βp < 1−βq (use the ratio test), and the series inthemiddleis 1 (1−βq) ∞  s=0βpg 1−βqszk 3+ln21+gszk. Obviously, this series converges when β<1/(gp +q).Whenβ=1/(gp +q),(βpg )/(1− qβ)=1, and the series reduces to (1−qβ)−1zk∞ s=01/(3+ln2(1+gszk)), which is convergent.14 A similar and straightforward argument shows that the series ∞ t=0pK,z(lt) converges for all compact set Kof [0, ∞)and z∈Z.15 Hence, all conditions of Theorem 2are fulfilled and this growth model admits a solution. Now we argue by contradiction, assuming that a bounding function ϕsatisfying the assumptions of Theorem 4 14Since lim s→∞ 3+ln21+gszk 3+ln2gszk=1, by the limit comparison test, the series has the same character than ∞  s=0 1 3+ln2gszk= ∞  s=0 1 3+slng+ln (zk)2, which is convergent for all k>0 and z∈Z. 15Since pK,z(lt)=Zmaxk∈Klt(k,z)Q(dz)and uis increasing, the problematic part in ∞ t=0pK,z(lt), which is (1−qβ)−1za∞ t=01/(3+ln2(1+gtza)), is also convergent, where a=maxKand K= {0}are a compact set of [0, ∞).IfthecompactsetisK={0},thenpK,z(lt)=0. 1246 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) exists when β=1/(gp +q). Then, by Proposition 6, there are constants Mand α≥0 such that αβ < 1and∞ t=0t+r rlt(k,z)≤αβM ϕ(k,z)/(1−αβ)2<∞,forallk>0, z∈Z, for all r≥0. This is clearly impossible, for instance, for r=1, the left-hand series is ∞  t=0 (t+1)lt(k,z)=zk (1−qβ) ∞  t=0 t+1 3+ln21+gtzk, which diverges, attaining a contradiction.16 Thus, a function ϕsatisfying the assumptions of Theorem 4cannot exist. This example can be extended to u(c)=(1+c)/(b+lna(1+c)) −1/b,witha>1 and b≥max(1+a,(1−a)(1−a)). These inequalities guarantee that uis strictly increasing and strictly concave. The series (26) converges since a>1 with the same condition for β,β≤1/(gp +q),buttheseries ∞  t=0t+r rlt+1(k,z)=zk r!(1−qβ) ∞  t=0 (t+r)(t+r−1)···(t+1) b+lna1+gtzk diverges for all positive integers r≥a−1, since the numerator is a polynomial of degree rand the series then has the same character than ∞ t=11/ta−r, which is convergent if and only if a−r>1. ♦ 5.2 Countable local contractions Matkowski and Nowak (2011)andJa´ skiewicz and Nowak (2011)extendthe(countable) local contraction approach developed in Rincón-Zapatero and Rodríguez-Palmero (2003) from the deterministic case to the stochastic case.17 In this section, we describe the method and point out some of the features that may restrict its applicability to stochastic dynamic programs where the Markov chain is exogenous. The following are the assumptions in Ja´ skiewicz and Nowak (2011,Section3),and adapted to our setting: (W) The technological correspondence is upper semicontinuous, (x,z)is compact for each (x,z)∈X×Z,U:−→ R∪{−∞}is upper semicontinuous and 16Note that t+1 3+ln21+gtzk 1 t+1 =(t+1)2 3+tlng+ln (zk)2 3+tlng+ln (zk)2 3+ln21+gtzk→1 ln2g·1, as t→∞; thus, by the limit comparison test, the series has the same character than the harmonic series ∞ t=01/(t+1). 17Our approach can be considered a way to extend the (uncountable) local contraction method in Martins da Rocha and Vailakis (2010) from the deterministic to a stochastic setting. The words “countable” and “uncountable” refer to the cardinality of the family of seminorms used to describe the topology of the function space where a given operator is defined. Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1253 (ii) Since pK,z(f)<∞for all K∈Kand all z∈Z,and(x,z)is a compact set for any x∈X,z∈Z,thenZ |f|(y,z)Qz(dz)=p(x,z),z(f)<∞. Obviously, the same is true for  f. Proof of Lemma 1. We organize the proof in several previous lemmas. Lemma 4. Let Assumptions (B1) to (B6) to hold. Then: (i) ∞ t=0Ltpl0<∞; (ii) R0[]∈L1(Z;C(X)) and pl0+LR0≤R0. Proof.Givenx∈Xand z∈Z,βplt[](x,z)=βZmaxy∈(x,z)lt(y,z)Qz(dz)≤lt+1(x, z);hence,plt[]∈L1(Z,C(X)) and then Lplt(K,z)=βpK,z(plt[]) ≤pK,z(lt+1),forall t=0, 1, Thus, Ltpl0≤Lt−1pl1≤···≤plt.By(B6),theseries∞ t=0plt(K,z)converges for all K∈Kand z∈Z,thus∞ t=0Ltpl0converges. To conclude the proof, by the triangle inequality pK,zpl0[]+···+plt[]≤pK,zpl0[]+···+pK,zpl0[] ≤(pK,z(l1)+···+pK,z(lt+1). Letting t→∞and adding pK,z(ψ)to both sides of the above inequality, we have pK,z(ψ)+pK,z(R0[]) ≤R0(K,z),showingatthesametimethatR0[]∈L1(Z;C(X)). Lemma 5. Let Assumptions (B1) to (B6) hold. Then f∈V(0, R0)implies Tf ∈L1(Z; C(X)). Proof.Letf∈L1(Z;C(X)).Weusethenotationfxand fzintroduced above at the beginning of this section. The function fxis Borel measurable for all x∈Xand Qzintegrable for any z∈Z.Thus,fxcan be written as the difference of two positive, Qz-integrable functions, fx=f+ x−f− x,wheref+ x=max(fx,0)and f− x=max(−fx,0). Applying Theorem 8.1 in Stokey, Lucas, and Prescott (1989), both Mf + xand Mf − xare Borel measurable. Since (Mf )x=M(fx)=M(f+ x)−M(f− x),(Mf )xis measurable for any x∈X. To see that (Mf )zis continuous, consider a sequence {xn}in Xthat converges to x∈X. Then the sequence and its limit form the compact set K={xn}∪{x}.Let fn:=fxn,forn≥1. For all z∈Z,fn(z)→fx(z)as n→∞, since fis continuous in x. Moreover, |fz|≤supx∈K|fz(x)|,andz→ supx∈K|fz(x)|is Qz-integrable by definition of L1(Z;C(X)), thus by the Lebesgue dominated convergence theorem, (Mf )(xn,z)=Z fnzQzdz→Z fxzQzdz=(Mf )(x,z), and thus (Mf )zis continuous. Hence, Mf is a Carathéodory function, and thus U(x,y,z)+βMf(y,z)is continuous in (x,y)for all z, and it is Borel measurable in z for all (x,y). By the Bergé maximum theorem, the function Tf is thus continuous in x 1254 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) for all z, and by the measurable maximum theorem, it is Borel measurable for any x.In short, the function (x,z)→ Tf(x,z)=max y∈(x,z)U(x,y,z)+βMf(y,z), is a Carathéodory function. Moreover, if f∈Fand x∈X,z∈Z, Tf(x,z)≤max y∈(x,z)U(x,y,z)+βmax y∈(x,z)Z max y∈(x,z)fy,zQzdz ≤l0(x,z)+βZ max y∈(x,z)wy,zQzdz ≤l0(x,z)+βp(x,z),z(f). Since (x,z)∈K,forf∈V(0, R0),wehavep(x,z),z(f)≤R0[](x,z). By Lemma 4, pK,z(l0)+βpK,z(R0[]) ≤R0(K,z).Hence,pK,z(Tf )≤R0(K,z). Thisprovesthat Tf ∈V(0, R0), and hence that Tf ∈L1(Z,C(X)). Lemma 6. Let Assumptions (B1) to (B6) hold. Then D(V(0, R0))⊆C. Proof. Since f∈V(0, R0),pf≤R0, and hence we can take c=1. Also, pf∈Ca(X×Z), since pf[](x,z)=Zmaxy∈(x,z)|f(y,z)|Qz(dz)is continuous in xand Borel measurable in z, by Lemma 3.Moreover,pf[]≤R0[]implies pK,z(pf[]) ≤pK,z(R0[]) ≤ 1 βR0(K,z), by Lemma 4.Hence,pf[]∈L1(Z,C(X)). Now we are in position to prove Lemma 1. First, let us see that L:C−→ C.Letp∈C; by the definition of the operator Land Lemma 4, Lp(K,z)=βpK,zp[]≤βpK,zcR0[]≤cR1(K,z)≤cR0(K,z), and so, Lp[]≤cR0[]and Lp[]∈L1(Z,C(X)). Second, we prove that the Assumptions (I) to (VI) are fulfilled. Regarding (I), note that p+q∈Cif p,q∈C, trivially, as well it is also immediate that if p∈Cand p≤p,thenp∈C. On the other hand, if a countable chain of partial sums p0,p0+p1,p0+p1+p2, is bounded by an element Pin C, then the infinite sum, p:=∞ n=0pn, is well-defined and p≤P≤cR0for some constant c. Moreover, since p[]≤cR0[]and R0[]∈L1(Z;C(X)) by Lemma 4, the monotone convergence theorem implies that p[](x,·)is Qz-integrable for all z∈Z and all x∈X. On the other hand, each function pi[](·,z)is continuous in x,forall i=1, 2, . By the Weierstrass M-test, the function p[](·,z)is also continuous in x for all z∈Z. These two observations imply that p[]∈L1(Z;C(X)). (II) is trivial and (III) holds, since the integral is monotone, and regarding (IV), it holds true, since for all p,q∈C,pK,z(p[]+q[]) ≤pK,z(p[]) +pK,z(q[]) by definition of the seminorms pK,z, and hence L(p+q)(K,z)=pK,zp[]+q[] ≤pK,zp[]+pK,zq[] =Lp(K,z)+Lq(K,z). Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1255 Lis clearly sup-preserving in Cby the monotone convergence theorem; hence, (V) also holds. Finally, (VI) is implied by Lemma 4and Lemma 5. Let the two families of functions {ut}∞ t=0and {lt}∞ t=0defined in (B6’). Let Iu0,w0={f∈ L1(Z;C(X)):u0≤f≤w0}. Lemma 7. T:Iu0,w0−→ Iu0,w0. Proof. We prove that for all x∈X,z∈Z,andt=0, 1,  βZ uty(x,z),zQzdz≥ut+1(x,z) and βZ max y∈(x,z)wty,zQzdz≤wt+1(x,z). In fact, βZ uty(x,z),zQzdz=Z s≥t ksy(x,z),zQzdz =β s≥tZ ksy(x,z),zQzdz =β s≥t ks+1(x,z) =ut+1(x,z), where the exchange of integral and summatory is due to the monotone convergence theorem. In the same way, βZ max y∈(x,z)wty,zQzdz≤βZ max y∈(x,z) s≥t lsy,zQzdz ≤ s≥t βZ max y∈(x,z)lsy,zQzdz ≤ s≥t ls+1(x,z) = s≥t+1 ls(x,z) =wt+1(x,z). Let f∈Iu0,w0.ThatTf is in L1(Z,C(X)) is proved as in Lemma 5above. Let us show first that Tf ≥u0.Forx∈Xand z∈Zto simplify notation, let y=y(x,z).Then Tf(x,z)≥U(x,y0,z)|+βZ |fy,z|Qzdz 1256 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) ≥k0(x,z)+βZ u0y,zQzdz ≥k0(x,z)+βu1(x,z) =u0(x,z). Also, Tf(x,z)≤max y∈(x,z)U(x,y,z)+βmax y∈(x,z)Z max y∈(x,z)fy,zQzdz ≤l0(x,z)+βZ max y∈(x,z)fy,zQzdz ≤l0(x,z)+βZ max y∈(x,z)w0y,zQzdz ≤l0(x,z)+w1(x,z) =w0(x,z). Appendix C: Continuity of the Markov operator The issue of continuity of the value function in the unbounded case (and unbounded space of shocks) is not an easy one. The translation of Stokey, Lucas, and Prescott (1989), Lemma 12.14 to this case is not straightforward, even if the Markov chain is strong Feller continuous. Recall that Qhas the weak (strong) Feller property if Mmaps bounded continuous functions (resp., bounded measurable functions) on Zinto bounded continuous functions. To see these kinds of problems that may emerge for unbounded functions, consider the following example, adapted from Stoyanov (2013). Let Z=[0, ∞) and let the transition function Q:Z×Z−→ Rbe defined as follows: Q(z,B)=⎧ ⎨ ⎩ δ0(B),ifz=0; B dFzz,ifz>0, (28) where δ0is the Dirac measure at the point 0, that is, δ0(B)=1if0∈Band δ0(B)=0 otherwise, and for 0 <z≤1, Fzz=⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ 0, if z=0; zz2+1−z,if0<z ≤1 z; 1, if z>1 z. Finally, for z≥1, Q(z,B)=λ(B∩[0, 1]),whereλdenotes the Lebesgue measure of R. Note that, for 0 <z<1, Fzis a distribution function; it is nondecreasing, continuous except at 0, where the right-sided limit exists, 0 ≤Fz≤1, and dFz=zz2+1−z−0z=0+1 z 0z2dz=1−z+z=1. Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1257 Moreover, it is clear that Q(·,B)is Borel measurable. Thus, Qis a transition function. Let f(y,z)=f(z)be independent of yand continuous in z.ThenMf is well-defined in this particular example and depends only on z,with(Mf )(0)=f(z)Q(0, dz)=f(0). For 0 <z<1, we have (Mf )(z)=fzQz,dz=fzdFzz =f(0)zz2+1−z−0z=0+1 z 0fzz2dz=f(0)(1−z)+z21 z 0fzdz. Since fis continuous, Mf is continuous for 0 <z<1. For z≥1, Mf is constant and given by (Mf )(z)=fzQz,dz=[0,1] fzdz. Now, if fis bounded, there is k>0suchthat−k≤f≤k;hence,−kz ≤z21 z 0f(z)dz≤ kz,andthusz21 z 0f(z)dztends to 0 as z→0+.Hence,Mf (z)→f(0)=Mf (0),when z→0+,andthusMf is continuous at 0. On the other hand, f(0)(1−z)+z21 z 0f(z)dz tends to 1 0f(z)dz=Mf (1)as z→1−,andthusMf is continuous at 1. Thus, Mf is continuous, and hence Qis strong Feller continuous. However, considering the unbounded function g(z)=z,wehaveMg(0)=g(0)=0andMg(z)=1/2forz>0; thus, Mg is discontinuous at 0. Consider now the following simple pure currency model with linear utility, where agents’ preferences are subject to random shocks; see Stokey, Lucas, and Prescott (1989) for further details about this model. Let the utility the utility u(c,z)=(1+z)cdepend on consumption cand shock z,andlet(m)=[0, m+y],wherem≥0, y>0isaconstant, X=R+,Z=[0, ∞], and let a discount factor βsuch that β<2/3. The dynamic programming equation is v(m,z)=max m∈[0,m+y](1+z)m+y−m+β[0,∞) vm,zQzdz. The random shocks are assumed to be governed by the Markov chain Qdescribed in (28). We are simply interested in showing that the value function is not jointly continuous in (m,z). It is easily checked that v(m,z)=⎧ ⎪ ⎨ ⎪ ⎩ m+y+yβ 1−β,ifz=0; (1+z)(m+y)+3 2yβ 1−β,ifz>0, is a solution in the class Ca(R+×R+), which coincides with the value function, and it is not continuous in z. To prove that (B6) is fulfilled, take l0(m,z)=ψ(m,z)=(1+z)(m+y).Now,noticing that ZzQz(dz)=1/2 and recalling that Q0is the Dirac measure at 0, it is easy to 1258 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) compute where m=m+y, Z 0m,zQzdz=βm+2y,ifz=0; (3/2)(m+2y),ifz>0. Thus, l1(m,z)=β(m+2y),ifz=0,andl1(m,z)=β(3/2)(m+2y),forz>0. In general, lt(m,z)=βt(3/2)(m+(t+1)y),forallt≥1. Clearly, ∞ t=0ltis unconditionally convergent for all β<1, thus Assumption (B6) holds. Given β,ϕ(m,z)=(1+az)(m+y),with a≥1isaboundofU(m,m,z). However, Zϕ(m,z)Q0(dz)=ϕ(m,0)=m+yand Zϕ(m,z)Qz(dz)=(1+(a/2))(m+by ),ifz>0. Thus, (m,z)→ Zϕ(m,z)Qz(dz)is discontinuous at z=0 and then (WC3) in Theorem 4is not fulfilled. Any other majorant function of the selected ϕwill suffer the same problem. References Aliprantis, Charalambos D. and Kim Border (1999), Infinite Dimensional Analysis,second edition. Springer Verlag, Heidelberg. [1237,1252] Álvarez, Fernando and Nancy L. Stokey (1998), “Dynamic programming with homogeneous functions.” J. Econ. Theory, 82, 167–189. [1231] Balbus, Łukasz, Kevin Reffett, and Łukasz P. Wozny (2018), “On uniqueness of time– consistent Markov policies for quasi-hyperbolic consumers under uncertainty.” J. Econ. Theory, 176, 293–310. [1224] Bäuerle, Nicole and Anna Ja´ skiewicz (2018), “Stochastic optimal growth model with risk sensitive preferences.” J. Econ Theory, 173, 181–200. [1238] Bäuerle, Nicole and Ulrich Rieder (2011), Markov Decision Processes With Applications to Finance. Springer Verlag, Berlin. [1223,1224,1231,1232,1241,1242] Becker, Robert A. and John H. Boyd III (1997), Capital Theory, Equilibrium Analysis and Recursive Utility. Blackwell, London. [1241] Bertsekas, Dimitri P. and Steven E. Shreve (1978), Stochastic Optimal Control: The Discrete-Time Case.AcademicPressInc.,NewYork.[1223] Binder, Michael and Mohammad H. Pesaran (1999), “Stochastic growth models and their econometric implications.” J. Econ. Growth, 4, 139–183. [1231] Blackwell, David (1965), “Discounted dynamic programming.” Ann. Appl. Stat., 36, 226– 235. [1223,1233] Bloise, Gaetano, Cuong Le Van and Yiannis Vailakis (2024), “Do not blame Bellman: It is Koopmans’ fault.” Econometrica, 92, 111–140. [1251] Bloise, Gaetano and Yiannis Vailakis (2018), “Convex dynamic programming with (bounded) recursive utility.” J. Econ. Theory, 173, 118–141. [1251] Boyd, John H. III (1990), “Recursive utility and the Ramsey problem.” J. Econ. Theory, 50, 326–345. [1224,1231,1232,1241] Theoretical Economics 19 (2024) Existence and uniqueness of solutions 1259 Brock, William A. and Leonard J. Mirman (1972), “Optimal economic growth and uncertainty: The discounted case.” J. Econ. Theory, 4, 479–513. [1223] Danthine, Jean P. and John B. Donaldson (1981), “Stochastic properties of fast vs. slow growing economies.” Econometrica, 49, 1007–1033. [1223] Donaldson, John B. and Rajnish Mehra (1983), “Stochastic growth with correlated production shocks.” J. Econ. Theory, 29, 282–312. [1223] Furukawa, Nagata (1972), “Markovian decision processes with compact action spaces.” Ann. Math. Statist., 43, 1612–1622. [1223] Had˘ zi´ c, Olga and Bogoljub Stankovi´ c (1970), “Some theorems on the fixed points in locally convex spaces.” Publications de L’institut Mathématique, 24, 9–19. [1224] Hernández-Lerma, Onésimo and Jean B. Lasserre (1999), Further Topics on DiscreteTime Markov Control Processes. Springer, New York. [1223,1224,1231,1232,1241,1242] Hopenhayn, Hugo A. and Edward C. Prescott (1992), “Stochastic monotonicity and stationary distributions for dynamic economies.” Econometrica, 1387–1406. [1223] Ja´ skiewicz, Anna and Andrzej S. Nowak (2011), “Discounted dynamic programming with unbounded returns: Application to economic models.” J. Math. Anal. Appl., 378, 450– 462. [1224,1231,1241,1246,1247,1250,1251] Jones, Larry E., Rodolfo E. Manuelli, Henry E. Siu, and Ennio Stacchetti (2005), “Fluctuations in convex models of endogenous growth, I: Growth effects.” Rev. Econ. Dynam.,8, 780–804. [1238,1239] Kamihigashi, Takashi (2007), “Stochastic optimal growth with bounded or unbounded utility and with bounded or unbounded shocks.” J. Math. Econ., 43, 477–500. [1238] Kozlov, Vladimir, Johan Thim, and Bengt O. Turesson (2010), “A fixed point theorem in locally convex spaces.” Collect. Math., 61, 223–239. [1225] Ma, Qingyin and John Stachurski (2019), “Optimal timing of decisions: A general theory based on continuation values.” J. Econ. Dyn. Control., 101, 62–81. [1231] Ma, Qingyin, John Stachurski, and Alexis A. Toda (2022), “Unbounded dynamic programming via the Q-transform.” J. Math. Econ., 100, 102652. [1224,1248,1251] Maitra, Ashok (1968), “Discounted dynamic programming on compact metric spaces.” Sankhya Ser. A, 30, 211–216. [1223] Majumdar, Mukul, Tapan Mitra and Yaw Nyarko (1989), “Dynamic optimization under uncertainty: Non-convex feasible set.” In Joan Robinson and Modern Economic Theory (George R. Feiwel, ed.), 545–590, Macmillan Press, London. [1223] Marinacci, Massimo and Luigi Montrucchio (2010), “Unique solutions for stochastic recursive utilities.” J. Econ. Theory, 145, 1776–1804. [1251] Martins da Rocha, Victor F. and Yiannis Vailakis (2010), “Existence and uniqueness of a fixed point for local contractions.” Econometrica, 78, 1127–1141. [1224,1227,1228,1229, 1246,1251] 1260 Juan Pablo Rincón-Zapatero Theoretical Economics 19 (2024) Matkowski, Janusz and Andrzej S. Nowak (2011), “On discounted dynamic programming with unbounded returns.” Econ. Theory, 46, 455–474. [1224,1231,1238,1246] Mirman, Leonard J. and Itzhak Zilcha (1975), “On optimal growth under uncertainty.” J. Econ. Theory, 11, 329–339. [1223] Mitra, Kaushik (1998), “On capital accumulation paths in a neoclassical stochastic growth model.” Econ. Theory, 11, 457–464. [1223] Olson, Lars J. and Santanu Roy (2006), “Theory of stochastic optimal economic growth.” In Handbook on Optimal Growth 1, 297–335, Springer-Verlag, Berlin, Heidelberg. [1223] Rincón-Zapatero, Juan P. (2019), “Existence and uniqueness of solutions to the stochastic Bellman equation with unbounded shock.” arXiv:1907.07343 [econ.TH]. [1223] Rincón-Zapatero, Juan P. (2022), “Existence and uniqueness of solutions to the Bellman equation in stochastic dynamic programming.” UC3M Working paper 2022, https: //www.eco.uc3m.es/~jrincon/files/SDP_2021.pdf.[1223,1225,1229,1232,1244,1251] Rincón-Zapatero, Juan P. and Carlos Rodríguez-Palmero (2003), “Existence and uniqueness of solutions to the Bellman equation in the unbounded case.” Econometrica, 71, 1519–1556. [1224,1225,1227,1231,1246,1251] Rincón-Zapatero, Juan P. and Carlos Rodríguez-Palmero (2007), “Recursive utility with unbounded aggregators.” Econ. Theory, 33, 381–391. [1224,1227] Rincón-Zapatero, Juan P. and Carlos Rodríguez-Palmero (2009), “Corrigendum to “Existence and uniqueness of solutions to the Bellman equation in the unbounded case”.” Econometrica, 77, 317–3186. [1224] Stachurski, John (2002), “Stochastic optimal growth with unbounded shock.” J. Econ. Theory, 106, 40–65. [1231,1238] Stokey, Nancy L., Robert E. Lucas, and Edward C. Prescott (1989), Recursive Methods in Economic Dynamics.Harvard University Press, Cambridge, MA. [1223,1224,1230,1231, 1232,1234,1235,1248,1253,1256,1257] Stoyanov, Jordan M. (2013), Counterexamples in Probability. Dover Publications, Mineola, New York. [1256] Wessels, Jaap (1977), “Markov programming by successive approximations with respect to weighted supremum norms.” J. Math. Anal. Appl., 58, 326–335. [1224] Willard, Stephen (1970), General Topology. Addison Wesley, Reading, Massachusetts. [1226] Wong, James S. W. (1968), “Two extensions of the Banach contraction mapping principle.” J. Math. Anal. Appl., 22, 438–443. [1227] Co-editor Florian Scheuer handled this manuscript. Manuscript received 24 December, 2021; final version accepted 26 September, 2023; available online 16 October, 2023.