Hamilton-Jacobi conditions for a class of impulsive control problems

HAMILTON-JACOBI CONDITIONS FOR A CLASS OF IMPULSIVE CONTROL PROBLEMS A. C. Matos1, F. Lobo Pereira1 1Instituto de Sistemas e Rob´ otica – Porto Faculdade de Engenharia da Universidade do Porto R. Dr. Roberto Frias, 4200-465 Porto, Portugal e-mail: {anibal,flp}@fe.up.pt Keywords: impulsive control, Hamilton-Jacobi conditions Abstract This work presents optimality conditions of Hamilton-Jacobi type for a classe of vector-valued impulsive control optimal problems. The dynamics are defined by a measure driven differential inclusion and the vector fields associated with the singular term do not satisfy the so called Frobenius condition. The concept of verification function for the class of problems addressed here is presented. Besides some regularity hypotheses, verifications functions satisfy a set of Hamilton-Jacobi type conditions, as well as a given boundary condition. It is shown that the existence of a verification function is a necessary and sufficient condition for the optimality of a feasible trajectory (in the sense of proper solution). It is also shown that the value function of the family of problems parametrized by the initial date is a verification function, with some extra properties, and results relating subgradients of the value function and multipliers of necessary conditions of the Maximum Principle are presented, too. 1 Introduction Impulsive control problems arise in a variety of application areas such as finance, mechanics, resources management, and space navigation, (see, for example, [4], [5], [8], [12], and [14]) whose solutions might involve discontinuous trajectories, motivating a significant research effort on the so-called Impulsive Control Problem (for a selected set of references see [17] and references therein). Impulsive control problems with vector valued control measures have been addressed by a number of authors, namely [18], [2], [9], and [11], by imposing the Frobenius condition on the vector fields associated with the singular term. This very strict assumption ensures an unique jump endpoint, x(t+), once specified the value of the state variable at t−and the measure dµ(t). In [3], this commutativity assumption is lifted by noting that a certain quotient control system, obtained by an appropriate nonlinear local change of coordinates in the state space, is an impulsive one satisfying the above mentioned commutative hypothesis. In this article, we present optimality conditions of Hamilton-Jacobi type for a class of impulsive control problems, like the one of [17]. These problems have dynamics described by impulsive differential inclusions driven by vector valued measures, without commutativity assumptions on the vector field associated to the singular term. The article is organized as follows. In section 2 we present the class of addressed problems. Section 3 is devoted to the definition of feasible trajectories for the impulsive dynamics under study. In section 4 we defined the value function for the class of addressed problems and derived some of its properties. In section 5 we present optimality condition of HamiltonJacobi type. Finally, in section 6 we provide a result relating generalized gradients of the value function and multipliers of the maximum principle principle for this class of problems. 2 Statement of the Problem We will consider the following optimal control problem: (P) min h(x(1)) subject to: dx(t)∈F(t, x(t))dt+ +G(t, x(t))µ(dt), t ∈[0,1], (1) µ∈ K,(2) x(0) = x0.(3) Here, F: [0,1] ×Rn⇒Rnis a multifunction and G: [0,1]×Rn⇒Rn×qis a vector of multifunctions. Function h:Rn→Rdefines the cost, that depends only on the final state, and x0∈Rnis the initial state. Kis the subset of the regular measure space C∗([0,1]; Rq)formed by the measures µsatisfying µ(A)∈Kfor all Borel sets A⊂[0,1], where K⊂Rqis a positive, convex, closed, and pointed cone. The set Kis also represented by C∗([0,1]; K). 3 Solution Concept In this work we do not assume any commutativity property of the vector fields generated by the columns of the multifunction G. Therefore, the concept of feasible trajectory for the impulsive differential inclusion (1) has to be properly defined. One of the traditional approaches to the study of impulsive control problems consists on performing a change of independent variable, in such a way as to assure the absolute continuity of trajectories, as functions of the new independent variable. This approach, originally developed in [18], has been followed in several works, such as [19], [20] or [17]. Such approach is usually called reparametrization of the independent variable and will also be used here. The construction of the new independent variable considered in this work is somehow similar to the one used in [17]. Take a real interval [a, b]a a measure µ∈ C∗([a, b]; K). Define the non-negative scalar measure ¯µas the total variation measure associated to µ, i.e., ¯µ(dt) = Pq i=1 µi(dt), since Kis a positive cone. Here, µi,i= 1,...,q, denote the components of the measure µ. Also define M: [a, b]→Kand η: [a, b]→Rby M(t) = (R[a,t]µ(ds)∀t∈]a, b] 0if t=a, η(t) = t−a+ q X i=1 Mi(t), Consider also the function θ: [0, η(b)] →[a, b]defined by θ(s) = sup{t∈[a, b] : η(t)≤s}.(4) Consider {tj}N j=1 (where N∈N0∪ {+∞}) is an enumeration of the atoms of ¯µand, for each j,Sj= θ−1({tj}). The function η, defined above, is called the reparametrization associated to the measure µ. Every function (θ, γ) : [0, η(b)] →R×Rqwhere θis as above and γsatisfies γ(s) = (M(θ(s)), s ∈[0, η(b)] \SjSj, M(t− j) + Rs η(t− j)vj(σ)dσ, ∃j:s∈Sj, where for each j,vjis a map from Sjto Ksatisfying q X i=1 vji(σ) = 1, Sj−q.s., ZSj vj(σ)dσ =µ({tj}), is called a graph completion of the measure µ. The definition and results presented below follow [19] and [17]. To simplify the notation we consider the time interval to be [0,1], being obvious that all definitions and results have readily extensions for any other time interval. Definition 1 (Reparametrized trajectory) Take x0∈Rna measure µ∈C∗([0,1]; K). Consider a given graph completion (θ, γ)of µ. The function z∈AC([0, η(1)]; Rn)is said to be reparametrized trajectory of (1), corresponding to the initial condition x0and to the graph completion (θ, γ)of µ, if z(0) = x0and ˙z(s)∈F(θ(s), z(s)) ˙ θ(s) + G(θ(s), z(s)) ˙γ(s),(5) s∈[0, η(1)] −a.e.. Here, ηdenotes the reparametrization of the measure µ. Inclusion (5) means that there exist f∈L1([0, η(1)]; Rn)and g∈ L1([0, η(1)]; Rn×q)such that f˙ θand g˙γare integrable and ˙z(s) = f(s)˙ θ(s) + g(s)˙γ(s), s ∈[0, η(1)] −a.e., f(s)∈F(θ(s), z(s)), s ∈[0, η(1)] −a.e., g(s)∈G(θ(s), z(s)), s ∈[0, η(1)] −a.e.. The definition of of admissible trajectory is given by the robust solution concept defined below, which is a direct extension to the vector valued measures case of the one introduced in [19]. Differently from the definition of robust solution presented in [17], our definition is independent of the cost function of the underlying optimization problem. Our definition of robust solution has a semigroup property, which is crucial for the dynamic programming results derived here. Definition 2 (Robust solution) Let x0∈Rnand µ∈C∗([0,1]; K). The function x∈ BV +([0,1]; Rq)is a robust solution of the impulsive differential inclusion (1), corresponding to the initial condition x0and to the measure µ, if the exist f: [0,1] →Rn, Lebesgue integrable, g: [0,1] → Rn,¯µintegrable, such that, for all t∈]0,1], x(t) = x0+Zt 0 f(τ)dτ +Z[0,t] g(τ)¯µ(dτ), and x(0) = x0, f(t)∈F(t, x(t)),[0,1] −a.e., g(t)∈˜ G(t, x(t−); µ({t})),¯µ−a.e., where ¯µis the total variation measure associated to µand the multifunction ˜ G: [0,1] ×Rn×K⇒Rn is defined by ˜ G(t, z;α) = ({ψw(t) : ψ∈G(t, z)},if |α|1= 0, Y(t, z, α),if |α|1>0, where Y(t, z, α) =ny(|α|1)−y(0) |α|1 : ˙y(s)∈G(t, y(s))˙γ(s), (0,˙γ(s)) ∈K1, s ∈[0,|α|1]−a.e., y(0) = z, γ(0) = 0, γ(|α|1) = αo. Here, |α|1=Pq i=1 αi,w: [0,1] →Rqis the RadonNikodym derivative of µwith respect to ¯µ, and K1is the subset of R1+qdefined by K1={(r0, r)∈[0,1] ×K:r0+ q X i=1 ri= 1}. The following proposition, see [19] (theorem 4.1) and also [17], establishes an equivalence relationship between robust solutions e reparametrized trajectories. Proposition 1 Suppose Fand Gtake values closed sets, Fis Lebesgue×Borel measurable and Gis Borel measurable. Take any x0∈Rnand µ∈ C∗([0,1]; K), and let be ηthe reparametrization function for µ. Let xbe a robust solution of (1), corresponding to x0and µ. Then, there exists z, reparametrized trajectory zfor (1), such that x(t) = z(η(t)),∀t∈[0,1].(6) Let zbe a reparametrized trajectory for (1), corresponding to x0and to a graph completion (θ, γ)of µ. Then, there exists x, robust solution for (1), such that (6) holds. Let xbe a robust solution and za reparametrized trajectory of (1) such that (6) holds. Then, kxkT V ≤ kzkT V . 4 Value Function In this section we defined the value function for the class of addressed problems and derive some of its properties. Several works, such as [15], [16], and [1], concerning the application of dynamic programming concepts to impulsive control problems are available in the literature. In almost all of them, the families of auxiliar problems, and, therefore, the associated value functions, depend not only on the initial time and state, but also on an additional parameter that defines the total variation allowed for the singular part of the dynamics. It is shown that such value functions satisfy what can be called dynamic programming principles in the form of generalized Hamilton-Jacobi conditions. Nevertheless, that additional parameter is somewhat artificial and is mainly intended to assure by construction the well posedness of the family of auxiliar problems, in the sense that the optimal value of each of those problems in bounded from below. In such formulations the value function also satisfies quite general continuity properties. On the contrary, in this work we consider a less restrictive definition of the value function, by considering only its dependance on the initial data (time and state). In this case, as shown below, we derive some elementary properties of the value function and also give sufficient conditions for its Lipschitz continuity. For each (s, y)∈[0,1] ×Rnthe impulsive control problem (Ps,y)is defined by: (Ps,y) min h(x(1)) subject to: dx(t)∈F(t, x(t))dt+ +G(t, x(t))µ(dt), t ∈[s, 1],(7) µ∈C∗([s, 1]; K),(8) x(s) = y. (9) For all these problems, the function h, the multifunctions Fand G, as well as the cone Kare the same as those for problem (P). The concepts of feasible process and of solution for each of these problems are the “natural” extensions of those considered for (P). In the following, X(s, y)denotes the set of feasible processes for (Ps,y). The value function V: [0,1] ×Rn→R∪ {−∞} considered here will be defined as V(s, y) = inf (x,µ)∈X(s,y)h(x(1)). The next lemma asserts that feasible processes of the impulsive differential inclusion (according to our definition) satisfy a semigroup property. Lemma 1 Take two intervals [a, b]and [b, c]in [0,1] andlet(x1, µ1)and(x2, µ2)befeasibleprocesses for the impulsive differential inclusion in [a, b]and [b, c], respectively, and such that x1(b) = x2(b). Then, (x, µ)∈BV +([a, c]; Rn)×C∗([a, c]; K), defined by x(t) =      x1(t),if t∈[a, b[, x2(b+),if t=b, x2(t),if t∈]b, c], µ(A) = µ1(A∩[a, b]) + µ2(A∩[b, c]),∀Aof Borel is a feasible process in the interval [a, c]. The following result states elementary properties of the value function. Lemma 2 For all y∈Rnone has V(1, y)≤g(y). Let (s, y)∈[0,1] ×Rnand let (x, µ)be feasible for (Ps,y). Then, for any t∈[s, 1] one has V(s, y)≤ V(t, x(t)). Moreover, if (x, µ)is optimal for (Ps,y) then V(s, y) = V(t, x(t)) for any t∈[s, 1]. The first assertion of this lemma is easily obtained by considering problems of the form (P1,y), while the second one is a consequence of the semigroup property stated in lemma 1. The next result concerns the approximation of arcs by reparametrized trajectories for the impulsive control system. This result resembles the Fillipov approximation theorem, derived for absolutely continuous control systems, see (Aubin, Cellina 1984). Theorem 1 Let F:R×Rn⇒Rnand G: R×Rn⇒Rn×qbe such that F(·, x)and G(·, x) are Lebesgue measurable ∀x∈IRn, and F(t, ·)and G(t, ·)are Lipschitz continuous of rank kfor all t∈R. Take a > 0and consider two Lipschitz continuous functions θ: [0, a]→Rand γ: [0, a]→Rq, satisfying (˙ θ(s),˙γ(s)) ∈K1, s ∈[0, a]−a.e.. Take also z∈AC([0, a]; Rn)such that ˙z(s) = φ(s)˙ θ(s) + ψ(s)˙γ(s), s ∈[0, a]−a.e. where φ∈L1([0, a]; IRn)and ψ∈L1([0, a]; Rn×q). Let p∈L1([0, a]; R)be such that dF(θ(s),z(s))(φ(s)) ≤p(s)˙ θ(s)6= 0 −a.e. dG(θ(s),z(s))(ψ(s)) ≤p(s) ˙γ(s)6= 0 −a.e. and take x0∈Rn. Let λ: [0, a]→Rbe defined as λ(s) = |z(a)−x0|ek(s−a)+Zs a ek(s−σ)p(σ)dσ. Then there exist x∈AC([0, a]; Rn),fand g, measurable selections of F(θ(s), x(s)) and G(θ(s), x(s)), respectively, satisfying ˙x(s) = f(s)˙ θ(s) + g(s)˙γ(s), s ∈[0, a]−a.e., x(0) = x0, and such that |x(s)−z(s)| ≤ λ(s),∀s∈[0, a] |˙x(s)−˙z(s)| ≤ kλ(s) + p(s), s ∈[0, a]−a.e. |f(s)−φ(s)| ≤ kλ(s) + p(s),˙ θ(s)6= 0 −a.e. |g(s)−ψ(s)| ≤ kλ(s) + p(s),˙γ(s)6= 0 −a.e.. The following theorem presents sufficient conditions for the Lipschitz continuity of the value function. Its proof is based on theorem 1. Its proof relies heavily on theorem 1. Theorem 2 Assume the following hypotheses. (i) his Lipschitz continuous of constant khin Rn; (ii) Fis continuous, and for each t∈[0,1] F(t, ·) is Lipschitz continuous of constant k; (iii) Gis Lipschitz continuous of constant k; (iv) for each r > 0there is a constant ko(r)∈R, such that there exists an optimal solution (x, µ) for (Ps,y)satisfying kµk ≤ ko(r), for all ysuch that |y|≤r, and for all s∈[0,1]. Then, the value function V´ e locally Lipschitz continuous. 5 Optimality Conditions The statement of Hamilton-Jacobi type optimality conditions for dynamic optimization problems is usually connected to the concept of verification function ([10], [6], [7], or [13], present such kind of results for problems with absolutely continuous trajectories). Here we extend such results to the class of impulsive control problems addressed. Definition 3 (Verification function) A function W: [0,1] ×Rn→Ris a verification function for problem (P)when 1. Wis locally Lipschitz continuous; 2. W(1, x)≤h(x),∀x∈Rn; 3. for all (t, x)∈]0,1[ ×Rn max (r0,r)∈K1 f∈F(t,x) g∈G(t,x) {W0((t, x); −(r0, fr0+gr))} ≤ 0, (10) and for all (t, x)∈[0,1] ×Rn max r∈K g∈G(t,x) {W0 (t)(x;−gr)} ≤ 0.(11) In this definition, and throughout this work, W(t)denotes, for each t∈[0,1], the function from Rnto Rdefined by W(t)(x) = W(t, x), for each x∈Rn, where W: [0,1] ×Rn→R. Monotonicity properties along feasible trajectories permit us to obtain the optimality conditions stated in the following result. Theorem 3 Let (z, ν)be a feasible process for (P) and let Wbe a verification function such that h(z(1)) = W(0, x0). Then, (z, ν)is optimal for (P). The next theorem gives further properties for locally Lipschitz continuous value functions. Theorem 4 Suppose the value function Vis locally Lipschitz continuous on [0,1] ×Rn. Then, for all (s, y)∈]0,1[ ×Rn max (r0,r)∈K1 f∈F(s,y) g∈G(s,y) {V0((s, y); −(r0, fr0+gr))}= 0, for all (s, y)∈[0,1] ×Rn max r∈K g∈G(s,y) {V0 (s)(y;−gr)} ≤ 0, and for all y∈Rn, max{max r∈K g∈G(1,y) {V0 (1)(y;−gr)}, V (1, y)−h(y)}= 0. This results shows that the value function, when continuous, is a verification function with a boundary condition. 6 Multipliers and Gradients of the Value Function In this section we present a result relating multipliers of necessary conditions of optimality for problem (P)and the generalized gradient of the value function Vdefined above. Definition 4 Let (x, µ)be a feasible process for (P), corresponding to the graph completion (θ, γ). The element p∈BV +([0,1]; Rn), is called a multiplier if 1. (x, p)is a robust solution of (−dp(t), dx(t)) ∈∂HF(t, x(t), p(t))dt+ +∂HG(t, x(t), p(t))µ(dt) in the interval [0,1], corresponding to the graph completion (θ, γ), 2. and the following conditions are satisfied σK(HG(t, x(t), p(t))) ≤0,∀t∈[0,1], σK(HG(t, x(t), p(t))) ≥0, µ −a.e.. In this definition, HFdenotes the hamiltonian function associated to F,HGdenotes the vector composed by the hamiltonians of the multifunctions defining G, and, for each r∈Rq,σK(r) = supk∈K{k·r}. The next proposition (theorem 3.1 of [17]) gives necessary conditions of optimality for problem (P). Proposition 2 Suppose the data defining problem (P)satisfies the following hypotheses: (H1) his locally Lipschitz continuous; (H2) Fis Lipschitz measurable; (H3) for each tthe multifunction x7→ G(t, x)is Lipschitz continuous with a constant not depending on t, and for each xthe multifunction t7→ G(t, x)is continuous; (H4) Fand Gare multifunctions with closed graphs and taking as values convex sets. Let (x, µ)be an optimal solution for (P). Then, there exists a multiplier psuch that −p(1) ∈ ∂Lh(x(1). The next results relates multipliers of the necessary conditions of optimality to the generalized gradient of the value function. Theorem 5 Suppose the above hypotheses on the data defining (P)are satisfied. Suppose also that the value function of the family of problems (Ps,y)is locally Lipschitz continuous. Let (x, µ)be an optimal process for (P). Then, there exists a multiplier of the necessary conditions of optimality psuch that −p(0) ∈∂V(0)(x0) (h(t),−p(t)) ∈∂V (t, x(t)),∀t∈]0,1[ , where h(t) = HF(t, x(t), p(t)). References [1] E. N. Barron, R. Jensen, and J. L. Menaldi, Optimal control and differential games with measures, Nonlinear Analysis, Theory, Methods & Applications 21 (1993), no. 4, 241–268. [2] A. Bressan and F. Rampazzo, Impulsive control systems with commutative vector fields, Journal of Optimization Theory and Applications 71 (1991), no. 1, 67–83. [3] , Impulsive control systems without commutativity assumptions, Journal of Optimization Theory and Applications 81 (1994), no. 3, 435–457. [4] B. Brogliato, Nonsmooth impact mechanics: Models, dynamics and control, Lect. Notes in Control and Inform. Sci., vol. 220, SpringerVerlag, 1996. [5] C. Clark, F. Clarke, and G. Munro, the optimal exploitation of renewable stocks, Econometrica 47 (1979), 25–47. [6] F. H. Clarke, Optimization and nonsmooth analysis, Addison-Wesley, New York, 1983. [7] F. H. Clarke and R. B. Vinter, Local optimality conditions and lipschitzian solutions to the Hamilton-Jacobi equation, SIAM Journal on Control and Optimization 21 (1983), no. 6, 856–870. [8] G. DalMaso and F. Rampazzo, On systems of ordinary differential equations with measures as controls, Differential and Integral Equations 4(1991), 739–765. [9] V. Dykhta and O. Sumsonuk, A maximum principle for optimal impulsive processes and applications, Proceedings of the European Control Conference (1997). [10] W. H. Fleming and R. W. Rishel, Deterministic and stochastic optimal control, SpringerVerlag, New York, 1975. [11] G. Kolokolnikova, A variational maximum principle for discontinuous trajectories of unbounded asymptotically linear control systems, Diff. Equations 33 (1997), 1633–1640. [12] D. Lawden, Optimal trajectories for space navigation, Butterworth, London, 1963. [13] P. D. Loewen, Optimal control via nonsmooth analysis, American Mathematical Society, Montr´ eal, 1993. [14] J. P. Marec, Optimal state trajectories, Elsevier, 1979. [15] M. Motta and F. Rampazzo, Dynamic programming for impulsive systems driven by ordinary and impulsive controls, SIAM Journal on Control and Optimization 34 (1996), no. 1, 199– 225. [16] , The value function of a slow growth control problem with state constraints, Journal of Mathematical Systems, Estimation, and Control 7(1997), no. 3, 1–20. [17] F. L. Pereira and G. N. Silva, Necessary conditions of optimality for vector-valued impulsive control problems, Systems & Control Letters 40 (2000), 205–215. [18] R. W. Rishel, An extended pontryagin principle for control systems whose laws contain measures, SIAM Journal on Control 3(1965), no. 2, 191–205. [19] G. N. Silva and R. B. Vinter, Measure differential inclusions, Journal of Mathematical Analysis and Applications 202 (1996), 727–746. [20] ,Necessary conditions for optimal impulsive control problems, SIAM Journal on Control and Optimization 35 (1997), no. 6, 1829–1846.