Optimal control with asymptotic stability constraints

DINCON 2006 Brazilian Conference on Dynamics, Control and Their Applications May 22 -⢠A¸S 26, 2006, Guaratinguetà ˛a, SP, Brazil OPTIMAL CONTROL WITH ASYMPTOTIC STABILITY CONSTRAINTS Fernando Lobo Pereira1and Geraldo Nunes Silva2 1FEUP - Faculdade de Engenharia da Universidade do Porto, Porto, Portugal, [email protected] 2IBILCE, Universidade Estadual de S. Paulo, S. José do Rio Preto, Brasil, gsilv[email protected] Abstract: In this article, we address the infinite horizon problem of optimizing a given performance criterion by choosing control strategies whose trajectories are asymptotically stable. In a first stage, we state and discuss sufficient conditions of optimality conditions in the form of an Hamilton-Jacobi-Bellman equation, and, based on them. Then, we present necessary conditions of optimality in the form of a maximum principle and show how it can be derived from an auxiliary optimal control problem with mixed constraints. Keywords: Optimal control, asymptotic stability, infinite horizon. 1. INTRODUCTION In this article, we discuss optimality conditions for an infinite horizon optimal control problem whose control processes must satisfy the usual constraints and be such that the corresponding trajectories converge asymptotically to an equilibrium point in a given target set. In other words, we consider the problem P∞(τ, z)Minimize g(ξ)(1) subject to            ˙x(t) = f(x(t), u(t)),a.a. t≥τ x(τ) = z u∈ U x(t)→ξas t→ ∞ ξ∈S⊂IRn. (2) Here, S⊂IRnis a closed set containing at least on equilibrium point ξwhich is also a decision variable, g:IRn→IR and f:IRn×IRm→IRnare given functions, and U:= {u∈L∞[τ, ∞) : u(t)∈Ωa.a. t≥τ} is the set of control strategies where Ωis a compact set in IRm. Notice that the value of the optimal cost depends not only on the equilibrium point, but also on the specific trajectory reaching it. This is substantially different from the problem usually understood in the control literature by optimal stabilization which is, in fact, time-optimal stabilization, i.e., finding a control that steers the state of the system to the origin in minimum time. In the next section, we will discuss sufficient conditions of optimality for this problem in the form of a generalized Hamilton-Jacobi-Bellman equation which can be regarded as a version of the ones derived for conventional problems in [13] and based on which an algorithm for synthesis of a feedback control strategy was presented in [10]. Then, necessary conditions of optimality are presented for a variant of problem P∞(τ, z)where the stabilizing constraints are incorporated via mixed inequality constraints. There has been a significant demand of results for this problem. A small sample of optimal stabilization application problems include micro-electro-mechanical (MEMS) control systems [3], economic systems under a variety of constraints and assumptions, [1, 12], rigid body mechanical systems [6], biological, medical, and health care systems [7], to name just a few. This contrasts with what appears to be a small body of results available for the general nonlinear dynamic optimization framework addressing the pertinent issues. See for example, [9] for a very specific problem and approach. The problem of stabilizing general dynamic nonlinear control systems has been receiving a considerable attention in the control literature, [4, 5, 11] and references cited therein. It has also emerged the important role of dynamic optimization andmethodsofnonsmoothanalysistoderive stability results, see [4, 5]. However, to the best of our knowledge, no results have been derived for optimal control problems where control strategies are restricted to the subset of stabilizing ones. 2. OPTIMALITY CONDITIONS OF HAMILTONJACOBI-BELLMAN TYPE First, we point out that, by x¯u(t)→ξas t→ ∞ where x¯uis the trajectory associated to the control function some ¯u(·)∈U, we mean lim t→∞ Zt τ eγskx(s)−ξkds < ∞, for some γ > 0. Now, we present a number of preliminary concepts and results needed in order to state the optimality conditions of this section. Let H: [0,∞)×IRn×IRn→IR be the Hamiltonian function for this problem defined by H(t, x, η) := sup v∈Ω {hη, f(x, v)i}.(3) We say that a continuous function φ: [τ, ∞)×IRn→IR is a viscosity solution to the Hamilton-Jacobi-Bellman equa- tion if φt(t, x)− H(t, x, −φx) = 0,∀(t, x)∈[τ, ∞)×IRn, wherever ∇tw(t, x)− H(t, x,−∇xw(t, x))½≤0∀(t, x)∈A− φ−w ≥0∀(t, x)∈A+ φ−w for any C1function w:IR ×IRn→IR. Here A+ φ−wand A− φ−wdenote, respectively, the argmax and the argmin of the function (φ−w)(·,·)in [0,∞)×IRn. This solution concept satisfies the uniqueness and nonsmoothness requirements of the generalized solution to the HJB equation, but a characterization of an extended valued, lower semicontinuous solution is needed when endpoint state constraints are present. So, we will adopt the solution concept based on the notion of proximal sub-gradient and proximal super-gradient (see [5, 13] for the corresponding definitions). Definition. A lower semicontinuous function v: [τ, ∞)× IRn→IR∪{+∞} is a proximal solution to the HJB equation if ∀(t, x)∈[τ, ∞)×IRn, such that ∂Pv(t, x)6=∅, η0− H(t, x, −η) = 0,∀(η0, η)∈∂Pv(t, x),(4) where ∂Pvdenotes the proximal sub-gradient of the function v.There are well known results in the literature providing a characterization of the value function, V:IR ×IRn→IR, for an optimal control problem, defined for our problem by V(τ, z) := Inf{P∞(τ, z)} as a generalized lower semicontinuous solution to the HJB equation (see for example Theorem 12.3.7 in [13]). Such a result was derived for the infinite time horizon in [2]. Since invariance type results provide more detailed information on optimal control processes than this characterization of the value function, we proceed with the definition and properties of verification functions. Now, we extend the concept of local verification function for this new problem formulation and provide conditions under which the existence of a verification function for a reference process (¯x, ¯ ξ, ¯u)is necessary and sufficient for its optimality. Let ¯xbe an admissible arc of problem P∞(τ, z). Let T(¯x, ²)be a tube centered at ¯xdefined by T(¯x, ²) := {(t, x)∈[τ, ∞)×IRn:kx−¯x(t)k ≤ ²}. Definition. A function φ:T(¯x, ²)→IR ∪+∞is a lower semicontinuous local verification function for (¯x, ¯ ξ, ¯u)if φis lower semicontinuous and the following conditions are satisfied. 1. For all (t, x)∈int T(¯x, ²)such that ∂Pφ(t, x)6=∅, η0+ min u∈Ω{hη, f(x, u)i} ≥ 0, for all (η0, η)∈∂Pφ(t, x). 2. For all ξ∈Sand admissible control process (x, u), lim inf t→∞ φ(t, ξ)≤g(ξ). 3. For all ξ∈S∩[¯ ξ+²B], lim inf t↑∞,ξ0→ξ φ(t, ξ 0) = lim inf t↑∞ φ(t, ξ). 4. φ(τ, z) = g(¯ ξ). The following assumptions on the data of the problem are required for the conditions of optimality stated below. H1) fis continuous and locally Lipschitz in x. H2) There exists c > 0such that f(t, x, u)∈c(1 + kxk)B, ∀(t, x)∈[0,∞)×IRn. H3) The set f(t, x, Ω) is convex-valued for all (t, x)∈ [0,∞)×IRn H4) The set Ωis compact. H5) gis lower semicontinuous. Theorem. Let (¯x, ¯ ξ, ¯u)be an admissible process of problem P∞(τ, z). We have the following: 1. If there exists a lower semicontinuous local verification function for (¯x, ¯ ξ, ¯u), then this control process is a strong local minimizer for P∞(τ, z). 2. Conversely, if (¯x, ¯ ξ, ¯u)is a strong local minimizer of P∞(τ, z), then there exists a lower semicontinuous local verification function for (¯x, ¯ ξ, ¯u). The proof is a slight modification of a similar result for finite time interval problems in [13]. Our approach consists in considering a family of auxiliary optimal control problems where this asymptotic convergence constraint gives rise to a penalization term added to the cost function of the original problem, i.e., we consider the problem: Pl ∞(τ, z)Minimize g(ξ) + Z∞ τ+l eγtkx(t)−ξkdt subject to ˙x(t) = f(x(t), u(t)),a.a. t≥τ x(τ) = z u∈ U ξ∈S⊂IRn. Note that we should have Z∞ τ+l eγtkx(t)−ξkdt →0 as l→ ∞, thus recovering the original optimization problem without the explicit constraint. Then, we show how to construct an (almost) optimal feedback control for problem Pl ∞(τ, ξ). This framework also allows us to construct stabilizing optimal feedback controls. Here, discuss briefly an algorithm for feedback control synthesis for problem P∞(τ, z)that, essentially, is a version of the procedure in [13] modified in order to force the state to reach the target set S. A partition π={tk}of [τ, ∞)is a countably, strictly increasing sequence tksuch that ti> tj, whenever i>j,tk→ ∞ as k→ ∞. The diameter of π, denoted by hπ, is defined by sup k≥0 {∆k}, where ∆k=tk+1 − tk. Let us assume that τ= 0. Let φbe a given local verification function as defined in the previous section and let x∈IRnbe a given state. Define U(x) := {u∈Ω : hNP S(pS(x)), f(x, u)i ≤ 0} where pS(x)is the proximal point of xat S. Let us start with x(0) = x0. Then, an approximating optimal control process is constructed recursively by computing a piecewise constant control function given, for each k= 0,1, . . . by ¯uπ k∈arg max u∈U(xπ(tπ k))nφ³tπ k, xπ(tπ k) + ∆kf(xπ(tπ k), u)´} and the corresponding trajectory is obtained by integrating the dynamics differential equation with the boundary condition given by the last value of the state variable in the previous time subinterval of the partition. Namely, xπ(t)is defined on [tπ k, tπ k+1)as the solution of ˙x(t) = f(t, x(t),¯uπ k)a.e.t∈(tπ k, tπ k+1], with initial value x(tπ k)given by the value of the state variable in the previous interval. We have the following main result of this work. Theorem. Assume that (H1) −(H5) hold. Let φbe a lower semicontinuous solution to the Hamilton-JacobiBellman equation. Take (xπ, uπ), the control process obtained by the recursive procedure described above. Then, xπ has a cluster point1with respect to the topology of uniform convergence on compact intervals, and, associated with such a point x(·), there is a pair, control u(·)and limit point ξ, such that (x(·), ξ, u(·)) is an optimal process of P∞(0, x0). 3. NECESSARY CONDITIONS OF OPTIMALITY In this section, we derive necessary conditions for a variant of problem P∞(τ, z)where the minimum rate of the asymptotic convergence of the trajectory a mixed inequality constraint of the form h(x, u) := xTf(x, u) kxk2+γ≤0 where γis a given positive number. 1A cluster point of a given sequence is a point to which there is a convergent subsequence. Let us fix τ= 0 and z=x0, and consider the following optimal control problem (P)Minimize g(ξ)(5) subject to ˙x(t) = f(x(t), u(t)) L−a.e. (6) x(0) = x0(7) x(t)→ξ∈S(8) h(x(t), u(t)) ≤0∀t≥0(9) u(t)∈Ω∀t≥0.(10) Obviously, it is implicit that u∗is such that x∗(t)→ξ∗∈ Sas t→ ∞. In order to state the necessary conditions of optimality, we consider the pseudo-Hamiltonian (or Pontryagin function) defined by H(x, p, q, u) := pTf(x, u) + qh(x, u), and assume the following set hypotheses on the data of our problem: H1) The functions g,fand hare locally Lipschitz continuous in xuniformly w.r.t. all other variables. H2) The functions fand hare Borel measurable w.r.t. the control variable. H3) The sets S∈IRnand Ω∈IRmare closed and bounded. H4) There is at least one equilibrium point in S. H5) The set {f(x, u), h(x, u) + v) : u∈Ω, v ≥0} is convex ∀x∈IRn. H6) There exists δ > 0such that inf{h(x, u) : u∈Ω} ≤ −δ. Remark that a generalization of H6) to vector-valued mixed constraints, i.e., h:IRn×IRm→IRkis: ∃δ > 0 such that δBk⊂ {h(x, u) + v:u∈Ω, v ≥0}. Here Bkis the open unit ball in IRkcentered at the origin. Theorem. Let (x∗, u∗)be an optimal control process for problem (P). Then, there exists an absolutely continuous function p: [0,∞)→IRn, a L1function q: [0,∞)→IR, and a number λ≥0satisfying: −˙p(t)∈co∂xH(x∗(t), p(t), q(t), u∗(t)) a.e. (11) lim s→∞ p(s)∈ −λ∂xg(ξ∗)−NS(ξ∗)(12) ½q(t)≤0a.e. and q(t)h(x∗(t), u∗(t)) = 0 a.e. (13) u∗(t)maximizes a.e. the mapping v→H(x∗(t), p(t), q(t), λ, v)on Ω.(14) Here, NS(ξ)and ∂f(ξ)are, respectively, the normal cone to the set Sand the generalized gradient of fat ξ, both in the sense of Clarke (see [5]). Now, we outline the proof which essentially consists in extending the main result (more specifically, corollary 3.2) in [8] to infinite horizon. We consider the following steps: a) The infinite horizon is regarded as the limit of the conditions for finite time for the problem (PT). Given an optimal control process for the infinite time horizon, its truncation to some finite interval [0, T ]for Tsufficiently large is proved to be an almost minimizer of the auxiliary finite time optimal control problem. b) Then, after showing that the requirements of Ekeland’s variational principle hold, we write down the necessary conditions of optimality proved in [8] for another convenient auxiliary optimal control problem approximating the original one and whose optimal control process is known. c) Finally, limits are extracted in order to get the stated conditions. 4. CONCLUSION We presented and discussed an infinite time horizon control optimization problem in which a given objective functional is optimized by choosing control strategies which ensure the stabilization of the dynamic control system within a given target set. We provided a dynamic programming based algorithm which yields a control process defined in a feedback form that approximates the optimal process. The method proposed here is modification of previous construct in [10] for a simpler problem with neither target nor stability constraints and addressing a finite time interval. We also present necessary conditions of optimality in the form of a maximum principle for an optimal control process satisfying a prescribed minimum rate for the asymptotic convergence towards the optimal equilibrium point in a given target set. ACKNOWLEDGMENTS Both authors are partially supported by research grants of the research project “Aplicações da Teoria do Controlo Impulsional e de Visão Computacional para Sistemas Robóticos Autonomos" funded within the Convénio FCT-CAPES framework. REFERENCES [1] K. Adam, “Optimal stabilization policy when the private sector has information processing constraints", No ¯ 81, Centre for Studies in Economics and Finance, Dept of Economic Sciences - University of Salerno, 2002. [2] J. Baumeister, A. Leitao and G. N. Silva, “On the value function for control problems with infinite horizon", IMA Journal of Mathematical Control and Information, submitted in 2005 [3] J. G. Chase, and S. Bhashyam, “Optimal stabilization of plate buckling", Smart Materials and Structures,8, 1999, pp. 204–211. [4] F. Clarke, Y. Ledyaev, E. Sontag and N. Subbotin (1997), “Asymptotic controllability implies feedback stabilization", IEEE Trans. Automat. Control,42, 1997, pp. 1394–1407. [5] F. Clarke, F., Y. Ledyaev, R. Stern and P. Wolenski, Nonsmooth analysis and control theory, SpringerVerlag, New York, 1998. [6] A. El-Gohary, “Optimal stabilization of a rigid body motion using rotors system", Appl. Math. and Computation,136, 2003, pp. 229–239. [7] C. R. Gomez, and J. McLaughlin, “New method for optimal stabilization of the transcranial doppler transducer during monitoring: normal controls and patient application", J. Neuroimaging,1, 1991, pp. 91–94. [8] M. R. Pinho, R. B. Vinter and H. Zheng, “A maximum principle for optimal control problems with mixed constraints", IMA J. Math. Control Information,18, 2001, pp. 189–205. [9] C. Prieur, and E. Trelat, “Robust optimal stabilization of the Brockett integrator via a hybrid feedback", Proceedings of MTNS, Leuven, Belgium, 2004. [10] J. Rowland, and R. B. Vinter, “Construction of optimal feedback controls", Systems and Control Letters, 16, 1991, pp. 337–367. [11] E. Sontag, Mathematical Control Theory, SpringerVerlag, New York, 1998. [12] R. Suescun, “Optimal commodity price stabilization over the business cycle", Procs IEEE Conference on Decision and Control, Banco de la Republica. Bogota, Colombia, 1998. [13] R. B. Vinter, Optimal Control: F oundations and Applications, Birkhauser, Boston, 2000.