DINCON 2006
B azilian Con e ence on Dynamics, Con ol and Thei Applica ions
May 22 -â˘
A¸S 26, 2006, Gua a ingue à ˛a, SP, B azil
OPTIMAL CONTROL WITH ASYMPTOTIC STABILITY CONSTRAINTS
Fe nando Lobo Pe ei a1and Ge aldo Nunes Sil a2
1FEUP - Faculdade de Engenha ia da Uni e sidade do Po o, Po o, Po ugal, [email p o ec ed]
2IBILCE, Uni e sidade Es adual de S. Paulo, S. José do Rio P e o, B asil, gsil [email p o ec ed]
Abs ac : In his a icle, we add ess he in ini e ho izon
p oblem o op imizing a gi en pe o mance c i e ion by
choosing con ol s a egies whose ajec o ies a e asymp-
o ically s able. In a i s s age, we s a e and discuss su -
icien condi ions o op imali y condi ions in he o m o
an Hamil on-Jacobi-Bellman equa ion, and, based on hem.
Then, we p esen necessa y condi ions o op imali y in he
o m o a maximum p inciple and show how i can be de-
i ed om an auxilia y op imal con ol p oblem wi h mixed
cons ain s.
Keywo ds: Op imal con ol, asymp o ic s abili y, in ini e
ho izon.
1. INTRODUCTION
In his a icle, we discuss op imali y condi ions o
an in ini e ho izon op imal con ol p oblem whose con ol
p ocesses mus sa is y he usual cons ain s and be such ha
he co esponding ajec o ies con e ge asymp o ically o an
equilib ium poin in a gi en a ge se . In o he wo ds, we
conside he p oblem
P∞(τ, z)Minimize g(ξ)(1)
subjec o 










˙x( ) = (x( ), u( )),a.a. ≥τ
x(τ) = z
u∈ U
x( )→ξas → ∞
ξ∈S⊂IRn.
(2)
He e, S⊂IRnis a closed se con aining a leas on equilib-
ium poin ξwhich is also a decision a iable, g:IRn→IR
and :IRn×IRm→IRna e gi en unc ions, and
U:= {u∈L∞[τ, ∞) : u( )∈Ωa.a. ≥τ}
is he se o con ol s a egies whe e Ωis a compac se in
IRm.
No ice ha he alue o he op imal cos depends no only
on he equilib ium poin , bu also on he speci ic ajec o y
eaching i . This is subs an ially di e en om he p oblem
usually unde s ood in he con ol li e a u e by op imal s a-
biliza ion which is, in ac , ime-op imal s abiliza ion, i.e.,
inding a con ol ha s ee s he s a e o he sys em o he o i-
gin in minimum ime.
In he nex sec ion, we will discuss su icien condi ions
o op imali y o his p oblem in he o m o a gene alized
Hamil on-Jacobi-Bellman equa ion which can be ega ded
as a e sion o he ones de i ed o con en ional p oblems
in [13] and based on which an algo i hm o syn hesis o a
eedback con ol s a egy was p esen ed in [10]. Then, nec-
essa y condi ions o op imali y a e p esen ed o a a ian o
p oblem P∞(τ, z)whe e he s abilizing cons ain s a e in-
co po a ed ia mixed inequali y cons ain s.
The e has been a signi ican demand o esul s o his
p oblem. A small sample o op imal s abiliza ion applica ion
p oblems include mic o-elec o-mechanical (MEMS) con ol
sys ems [3], economic sys ems unde a a ie y o cons ain s
and assump ions, [1, 12], igid body mechanical sys ems [6],
biological, medical, and heal h ca e sys ems [7], o name jus
a ew.
This con as s wi h wha appea s o be a small body o
esul s a ailable o he gene al nonlinea dynamic op imiza-
ion amewo k add essing he pe inen issues. See o ex-
ample, [9] o a e y speci ic p oblem and app oach. The
p oblem o s abilizing gene al dynamic nonlinea con ol
sys ems has been ecei ing a conside able a en ion in he
con ol li e a u e, [4, 5, 11] and e e ences ci ed he ein. I
has also eme ged he impo an ole o dynamic op imiza ion
andme hodso nonsmoo hanalysis ode i e s abili y esul s,
see [4, 5]. Howe e , o he bes o ou knowledge, no esul s
ha e been de i ed o op imal con ol p oblems whe e con-
ol s a egies a e es ic ed o he subse o s abilizing ones.
2. OPTIMALITY CONDITIONS OF HAMILTON-
JACOBI-BELLMAN TYPE
Fi s , we poin ou ha , by x¯u( )→ξas → ∞ whe e
x¯uis he ajec o y associa ed o he con ol unc ion some
¯u(·)∈U, we mean
lim
→∞ Z
τ
eγskx(s)−ξkds < ∞,
o some γ > 0.
Now, we p esen a numbe o p elimina y concep s and
esul s needed in o de o s a e he op imali y condi ions o
his sec ion. Le H: [0,∞)×IRn×IRn→IR be he Hamil-
onian unc ion o his p oblem de ined by
H( , x, η) := sup
∈Ω
{hη, (x, )i}.(3)
We say ha a con inuous unc ion φ: [τ, ∞)×IRn→IR
is a iscosi y solu ion o he Hamil on-Jacobi-Bellman equa-
ion i
φ ( , x)− H( , x, −φx) = 0,∀( , x)∈[τ, ∞)×IRn,
whe e e
∇ w( , x)− H( , x,−∇xw( , x))½≤0∀( , x)∈A−
φ−w
≥0∀( , x)∈A+
φ−w
o any C1 unc ion w:IR ×IRn→IR. He e A+
φ−wand
A−
φ−wdeno e, espec i ely, he a gmax and he a gmin o he
unc ion (φ−w)(·,·)in [0,∞)×IRn.
This solu ion concep sa is ies he uniqueness and non-
smoo hness equi emen s o he gene alized solu ion o he
HJB equa ion, bu a cha ac e iza ion o an ex ended alued,
lowe semicon inuous solu ion is needed when endpoin s a e
cons ain s a e p esen . So, we will adop he solu ion con-
cep based on he no ion o p oximal sub-g adien and p ox-
imal supe -g adien (see [5, 13] o he co esponding de in-
i ions).
De ini ion. A lowe semicon inuous unc ion : [τ, ∞)×
IRn→IR∪{+∞} is a p oximal solu ion o he HJB equa ion
i ∀( , x)∈[τ, ∞)×IRn, such ha ∂P ( , x)6=∅,
η0− H( , x, −η) = 0,∀(η0, η)∈∂P ( , x),(4)
whe e ∂P deno es he p oximal sub-g adien o he unc ion
.The e a e well known esul s in he li e a u e p o iding a
cha ac e iza ion o he alue unc ion, V:IR ×IRn→IR,
o an op imal con ol p oblem, de ined o ou p oblem by
V(τ, z) := In {P∞(τ, z)}
as a gene alized lowe semicon inuous solu ion o he HJB
equa ion (see o example Theo em 12.3.7 in [13]). Such a
esul was de i ed o he in ini e ime ho izon in [2].
Since in a iance ype esul s p o ide mo e de ailed in o -
ma ion on op imal con ol p ocesses han his cha ac e iza-
ion o he alue unc ion, we p oceed wi h he de ini ion and
p ope ies o e i ica ion unc ions.
Now, we ex end he concep o local e i ica ion unc ion
o his new p oblem o mula ion and p o ide condi ions un-
de which he exis ence o a e i ica ion unc ion o a e e -
ence p ocess (¯x, ¯
ξ, ¯u)is necessa y and su icien o i s op i-
mali y.
Le ¯xbe an admissible a c o p oblem P∞(τ, z). Le
T(¯x, ²)be a ube cen e ed a ¯xde ined by
T(¯x, ²) := {( , x)∈[τ, ∞)×IRn:kx−¯x( )k ≤ ²}.
De ini ion. A unc ion φ:T(¯x, ²)→IR ∪+∞is a lowe
semicon inuous local e i ica ion unc ion o (¯x, ¯
ξ, ¯u)i φis
lowe semicon inuous and he ollowing condi ions a e sa is-
ied.
1. Fo all ( , x)∈in T(¯x, ²)such ha ∂Pφ( , x)6=∅,
η0+ min
u∈Ω{hη, (x, u)i} ≥ 0,
o all (η0, η)∈∂Pφ( , x).
2. Fo all ξ∈Sand admissible con ol p ocess (x, u),
lim in
→∞ φ( , ξ)≤g(ξ).
3. Fo all ξ∈S∩[¯
ξ+²B],
lim in
↑∞,ξ0→ξ
φ( , ξ
0) = lim in
↑∞ φ( , ξ).
4. φ(τ, z) = g(¯
ξ).
The ollowing assump ions on he da a o he p oblem a e
equi ed o he condi ions o op imali y s a ed below.
H1) is con inuous and locally Lipschi z in x.
H2) The e exis s c > 0such ha
( , x, u)∈c(1 + kxk)B, ∀( , x)∈[0,∞)×IRn.
H3) The se ( , x, Ω) is con ex- alued o all ( , x)∈
[0,∞)×IRn
H4) The se Ωis compac .
H5) gis lowe semicon inuous.
Theo em. Le (¯x, ¯
ξ, ¯u)be an admissible p ocess o p oblem
P∞(τ, z). We ha e he ollowing:
1. I he e exis s a lowe semicon inuous local e i ica-
ion unc ion o (¯x, ¯
ξ, ¯u), hen his con ol p ocess is
a s ong local minimize o P∞(τ, z).
2. Con e sely, i (¯x, ¯
ξ, ¯u)is a s ong local minimize o
P∞(τ, z), hen he e exis s a lowe semicon inuous lo-
cal e i ica ion unc ion o (¯x, ¯
ξ, ¯u).
The p oo is a sligh modi ica ion o a simila esul o
ini e ime in e al p oblems in [13]. Ou app oach consis s
in conside ing a amily o auxilia y op imal con ol p oblems
whe e his asymp o ic con e gence cons ain gi es ise o a
penaliza ion e m added o he cos unc ion o he o iginal
p oblem, i.e., we conside he p oblem:
Pl
∞(τ, z)Minimize g(ξ) + Z∞
τ+l
eγ kx( )−ξkd
subjec o ˙x( ) = (x( ), u( )),a.a. ≥τ
x(τ) = z
u∈ U
ξ∈S⊂IRn.
No e ha we should ha e
Z∞
τ+l
eγ kx( )−ξkd →0
as l→ ∞, hus eco e ing he o iginal op imiza ion p ob-
lem wi hou he explici cons ain . Then, we show how o
cons uc an (almos ) op imal eedback con ol o p oblem
Pl
∞(τ, ξ). This amewo k also allows us o cons uc s abi-
lizing op imal eedback con ols.
He e, discuss b ie ly an algo i hm o eedback con ol
syn hesis o p oblem P∞(τ, z) ha , essen ially, is a e sion
o he p ocedu e in [13] modi ied in o de o o ce he s a e
o each he a ge se S. A pa i ion π={ k}o [τ, ∞)is a
coun ably, s ic ly inc easing sequence ksuch ha i> j,
whene e i>j, k→ ∞ as k→ ∞. The diame e o π,
deno ed by hπ, is de ined by sup
k≥0
{∆k}, whe e ∆k= k+1 −
k. Le us assume ha τ= 0.
Le φbe a gi en local e i ica ion unc ion as de ined in
he p e ious sec ion and le x∈IRnbe a gi en s a e. De ine
U(x) := {u∈Ω : hNP
S(pS(x)), (x, u)i ≤ 0}
whe e pS(x)is he p oximal poin o xa S.
Le us s a wi h x(0) = x0. Then, an app oxima ing
op imal con ol p ocess is cons uc ed ecu si ely by com-
pu ing a piecewise cons an con ol unc ion gi en, o each
k= 0,1, . . . by
¯uπ
k∈a g max
u∈U(xπ( π
k))nφ³ π
k, xπ( π
k) + ∆k (xπ( π
k), u)´}
and he co esponding ajec o y is ob ained by in eg a ing
he dynamics di e en ial equa ion wi h he bounda y condi-
ion gi en by he las alue o he s a e a iable in he p e-
ious ime subin e al o he pa i ion. Namely, xπ( )is de-
ined on [ π
k, π
k+1)as he solu ion o
˙x( ) = ( , x( ),¯uπ
k)a.e. ∈( π
k, π
k+1],
wi h ini ial alue x( π
k)gi en by he alue o he s a e
a iable in he p e ious in e al.
We ha e he ollowing main esul o his wo k.
Theo em. Assume ha (H1) −(H5) hold. Le φbe
a lowe semicon inuous solu ion o he Hamil on-Jacobi-
Bellman equa ion. Take (xπ, uπ), he con ol p ocess ob-
ained by he ecu si e p ocedu e desc ibed abo e. Then, xπ
has a clus e poin 1wi h espec o he opology o uni o m
con e gence on compac in e als, and, associa ed wi h such
a poin x(·), he e is a pai , con ol u(·)and limi poin ξ,
such ha (x(·), ξ, u(·)) is an op imal p ocess o P∞(0, x0).
3. NECESSARY CONDITIONS OF OPTIMALITY
In his sec ion, we de i e necessa y condi ions o a a i-
an o p oblem P∞(τ, z)whe e he minimum a e o he as-
ymp o ic con e gence o he ajec o y a mixed inequali y
cons ain o he o m
h(x, u) := xT (x, u)
kxk2+γ≤0
whe e γis a gi en posi i e numbe .
1A clus e poin o a gi en sequence is a poin o which he e is a con-
e gen subsequence.
Le us ix τ= 0 and z=x0, and conside he ollowing
op imal con ol p oblem
(P)Minimize g(ξ)(5)
subjec o ˙x( ) = (x( ), u( )) L−a.e. (6)
x(0) = x0(7)
x( )→ξ∈S(8)
h(x( ), u( )) ≤0∀ ≥0(9)
u( )∈Ω∀ ≥0.(10)
Ob iously, i is implici ha u∗is such ha x∗( )→ξ∗∈
Sas → ∞.
In o de o s a e he necessa y condi ions o op imali y,
we conside he pseudo-Hamil onian (o Pon yagin unc-
ion) de ined by
H(x, p, q, u) := pT (x, u) + qh(x, u),
and assume he ollowing se hypo heses on he da a o ou
p oblem:
H1) The unc ions g, and ha e locally Lipschi z con inu-
ous in xuni o mly w. . . all o he a iables.
H2) The unc ions and ha e Bo el measu able w. . . he
con ol a iable.
H3) The se s S∈IRnand Ω∈IRma e closed and bounded.
H4) The e is a leas one equilib ium poin in S.
H5) The se
{ (x, u), h(x, u) + ) : u∈Ω, ≥0}
is con ex ∀x∈IRn.
H6) The e exis s δ > 0such ha
in {h(x, u) : u∈Ω} ≤ −δ.
Rema k ha a gene aliza ion o H6) o ec o - alued
mixed cons ain s, i.e., h:IRn×IRm→IRkis: ∃δ > 0
such ha
δBk⊂ {h(x, u) + :u∈Ω, ≥0}.
He e Bkis he open uni ball in IRkcen e ed a he o igin.
Theo em. Le (x∗, u∗)be an op imal con ol p ocess o
p oblem (P).
Then, he e exis s an absolu ely con inuous unc ion p:
[0,∞)→IRn, a L1 unc ion q: [0,∞)→IR, and a numbe
λ≥0sa is ying:
−˙p( )∈co∂xH(x∗( ), p( ), q( ), u∗( )) a.e. (11)
lim
s→∞
p(s)∈ −λ∂xg(ξ∗)−NS(ξ∗)(12)
½q( )≤0a.e. and
q( )h(x∗( ), u∗( )) = 0 a.e. (13)
u∗( )maximizes a.e. he mapping
→H(x∗( ), p( ), q( ), λ, )on Ω.(14)
He e, NS(ξ)and ∂ (ξ)a e, espec i ely, he no mal cone
o he se Sand he gene alized g adien o a ξ, bo h in he
sense o Cla ke (see [5]).
Now, we ou line he p oo which essen ially consis s in
ex ending he main esul (mo e speci ically, co olla y 3.2)
in [8] o in ini e ho izon. We conside he ollowing s eps:
a) The in ini e ho izon is ega ded as he limi o he con-
di ions o ini e ime o he p oblem (PT). Gi en an
op imal con ol p ocess o he in ini e ime ho izon, i s
unca ion o some ini e in e al [0, T ] o Tsu icien ly
la ge is p o ed o be an almos minimize o he auxil-
ia y ini e ime op imal con ol p oblem.
b) Then, a e showing ha he equi emen s o Ekeland’s
a ia ional p inciple hold, we w i e down he necessa y
condi ions o op imali y p o ed in [8] o ano he con e-
nien auxilia y op imal con ol p oblem app oxima ing
he o iginal one and whose op imal con ol p ocess is
known.
c) Finally, limi s a e ex ac ed in o de o ge he s a ed
condi ions.
4. CONCLUSION
We p esen ed and discussed an in ini e ime ho izon con-
ol op imiza ion p oblem in which a gi en objec i e unc-
ional is op imized by choosing con ol s a egies which en-
su e he s abiliza ion o he dynamic con ol sys em wi hin
a gi en a ge se . We p o ided a dynamic p og amming
based algo i hm which yields a con ol p ocess de ined in a
eedback o m ha app oxima es he op imal p ocess. The
me hod p oposed he e is modi ica ion o p e ious cons uc
in [10] o a simple p oblem wi h nei he a ge no s abil-
i y cons ain s and add essing a ini e ime in e al. We also
p esen necessa y condi ions o op imali y in he o m o a
maximum p inciple o an op imal con ol p ocess sa is ying
a p esc ibed minimum a e o he asymp o ic con e gence
owa ds he op imal equilib ium poin in a gi en a ge se .
ACKNOWLEDGMENTS
Bo h au ho s a e pa ially suppo ed by esea ch g an s o
he esea ch p ojec “Aplicações da Teo ia do Con olo Im-
pulsional e de Visão Compu acional pa a Sis emas Robó i-
cos Au onomos" unded wi hin he Con énio FCT-CAPES
amewo k.
REFERENCES
[1] K. Adam, “Op imal s abiliza ion policy when he p i-
a e sec o has in o ma ion p ocessing cons ain s", No
¯
81, Cen e o S udies in Economics and Finance, Dep
o Economic Sciences - Uni e si y o Sale no, 2002.
[2] J. Baumeis e , A. Lei ao and G. N. Sil a, “On he alue
unc ion o con ol p oblems wi h in ini e ho izon",
IMA Jou nal o Ma hema ical Con ol and In o ma ion,
submi ed in 2005
[3] J. G. Chase, and S. Bhashyam, “Op imal s abiliza ion
o pla e buckling", Sma Ma e ials and S uc u es,8,
1999, pp. 204–211.
[4] F. Cla ke, Y. Ledyae , E. Son ag and N. Subbo in
(1997), “Asymp o ic con ollabili y implies eedback
s abiliza ion", IEEE T ans. Au oma . Con ol,42, 1997,
pp. 1394–1407.
[5] F. Cla ke, F., Y. Ledyae , R. S e n and P. Wolen-
ski, Nonsmoo h analysis and con ol heo y, Sp inge -
Ve lag, New Yo k, 1998.
[6] A. El-Goha y, “Op imal s abiliza ion o a igid body
mo ion using o o s sys em", Appl. Ma h. and Compu-
a ion,136, 2003, pp. 229–239.
[7] C. R. Gomez, and J. McLaughlin, “New me hod o
op imal s abiliza ion o he ansc anial dopple ans-
duce du ing moni o ing: no mal con ols and pa ien
applica ion", J. Neu oimaging,1, 1991, pp. 91–94.
[8] M. R. Pinho, R. B. Vin e and H. Zheng, “A maximum
p inciple o op imal con ol p oblems wi h mixed con-
s ain s", IMA J. Ma h. Con ol In o ma ion,18, 2001,
pp. 189–205.
[9] C. P ieu , and E. T ela , “Robus op imal s abiliza ion
o he B ocke in eg a o ia a hyb id eedback", P o-
ceedings o MTNS, Leu en, Belgium, 2004.
[10] J. Rowland, and R. B. Vin e , “Cons uc ion o op i-
mal eedback con ols", Sys ems and Con ol Le e s,
16, 1991, pp. 337–367.
[11] E. Son ag, Ma hema ical Con ol Theo y, Sp inge -
Ve lag, New Yo k, 1998.
[12] R. Suescun, “Op imal commodi y p ice s abiliza ion
o e he business cycle", P ocs IEEE Con e ence on
Decision and Con ol, Banco de la Republica. Bogo a,
Colombia, 1998.
[13] R. B. Vin e , Op imal Con ol: F ounda ions and Ap-
plica ions, Bi khause , Bos on, 2000.