Discrete Mechanics and Optimal Control
Von der Fakult¨at f¨ur Elektrotechnik,
Informatik und Mathematik
der Universit¨at Paderborn
zur Erlangung des akademischen Grades
Doktor der Naturwissenschaften
– Dr. rer. nat. –
genehmigte Dissertation
von
Sina Ober-Bl¨obaum
Paderborn, 2008
Gutachter: Prof. Dr. Michael Dellnitz
Prof. Dr. Oliver Junge
Prof. Dr. Jerrold E. Marsden
Tag der m¨undlichen Pr¨ufung: 13.03.2008
THE FUNDAMENTAL VARIATIONAL PRINCIPLE
Namely, because the shape of the whole universe is the
most perfect and, in fact, designed by the wisest creator,
nothing in all the world will occur in which no maximum or
minimum rule is somehow shining forth...
Leonhard Euler (1744)
iii
Acknowledgements
To begin, I would like to thank my advisor Prof. Dr. Michael Dellnitz for his
guidance, support, and motivation, and, in particular, for the great freedom he
has given me during my PhD research.
I would also like to thank Prof. Dr. Oliver Junge from the Technische Univer-
sit¨at M¨unchen for his extensive supervision over the last years.
Professor Jerrold E. Marsden from the California Institute of Technology has
provided valuable guidance during my studies. He introduced me to the area
of discrete variational mechanics and provided interesting ideas for application.
Mutual visits and discussions contributed to the progress of my research as well.
Special thanks goes to my co-workers Dr. Sigrid Leyendecker, Dr. Kathrin
Padberg and Mirko Hessel-von Molo for many interesting and enlightening dis-
cussions, exciting joint work, and their constant support.
I gratefully acknowledge support under the DFG-sponsored research project
SFB 376 on Massively Parallel Computation.
My colleages in Paderborn participated in lengthy discusssions on scientific
and unscientific topics and I appreciate their technical and administrative sup-
port, including Alessandro Dell’Aere, Olaf Bonorden, Tanja B¨urger, Sebastian
Hage-Packh¨auser, Marianne Kalle, Stefan Klus, Anna-Lena Meyer, Marcus Post,
Dr. Robert Preis, Marcel Schwalb, Stefan Sertl, Bianca Thiere, Julia Timmer-
mann, and Katrin Witting.
I am very grateful to all my friends, primarly Oliver Krimmer and Farina
Schneider for proof-reading the manuscript. Special thanks goes to Kay Klobe-
danz, who has been a great support and motivator over the last years. For support
during the last months I would especially like to thank Helene Waßmann.
Most of all, I thank my family. In particular, the constant support and en-
couragement from my parents Ingetraud and Hartmut and my brother Mark has
accompanied me through many years of studies.
Finally, I thank Andreas Kohlos - for a number of things.
iv
Abstract
The optimal control of physical processes is of crucial importance in all modern
technological sciences. In general, one is interested in prescribing the motion of
a dynamical system in such a way that a certain optimality criterion is achieved.
Typical challenges are the determination of a time-minimal path in vehicle dy-
namics, an energy-efficient trajectory in space mission design, or optimal motion
sequences in robotics and biomechanics.
In order to solve optimal control problems for mechanical systems, this thesis
links the theory of optimal control with concepts from variational mechanics.
The application of discrete variational principles allows for the construction of an
optimization algorithm that enables the discrete solution to inherit characteristic
structural properties from the continuous problem.
The numerical performance of the developed method and its relationship to
other existing optimal control methods are investigated. This is done by means of
theoretical considerations as well as with the help of numerical examples arising
in problems from trajectory planning and space mission design.
The development of efficient approaches for exploiting the mechanical system’s
structures reduce for example the computational effort. In addition, the optimal
control framework is extended to mechanical systems with constraints in multi-
body dynamics and applied to robotical and biomechanical problems.
v
Zusammenfassung
Die optimale Steuerung physikalischer Prozesse ist in allen modernen techno-
logischen Wissenschaften von wichtiger Bedeutung. Das Ziel ist es, die Bewe-
gung eines dynamischen Systems so vorzuschreiben, dass ein bestimmtes Op-
timlit¨atskriterium erreicht wird. Typische Anwendungen sind die Bestimmung
zeitoptimaler Wege in der Fahrzeugdynamik, energieeffizienter Trajektorien von
Raumfahrtmissionen oder optimaler Bewegungsabl¨aufe in der Robotik und der
Biomechanik.
Diese Arbeit vereint die Theorie der optimalen Steuerung mit den Konzepten
der Variationsmechanik, um Steuerungsprobleme mechanischer Systeme zu l¨osen.
Die Anwendung diskreter Variationsprinzipien erm¨oglicht es, einen Optimierungs-
algorithmus zu konstruieren, dessen L¨osung charakteristische strukturelle Eigen-
schaften des kontinuierlichen Problems erbt.
Die numerische Effizienz der entwickelten Methode, sowie Vergleiche und Re-
lationen zu existierenden optimalen Steuerungsmethoden, werden sowohl anhand
theoretischer Betrachtungen als auch anhand numerischer Beispiele untersucht.
Die Entwicklung effizienter Ans¨atze zur Ausnutzung der speziellen Struktur
des mechanischen Systems reduziert beispielsweise den rechnerischen Aufwand.
Abschließend wird die vorgestellte Methode dahingehend erweitert, dass sie sich
auf mechanische Systeme mit Zwangsbedingungen in der Mehrk¨orperdynamik
anwenden l¨asst. Dabei werden Probleme aus der Robotik und der Biomechanik
behandelt.
vii
Contents
1 Introduction 1
2 Optimal control 13
2.1 Optimal control problem . . . . . . . . . . . . . . . . . . . . . . . 13
2.1.1 Problem formulation . . . . . . . . . . . . . . . . . . . . . 13
2.1.2 Necessary conditions for optimality . . . . . . . . . . . . . 15
2.2 Solution methods for optimal control problems . . . . . . . . . . . 16
2.2.1 Indirect methods . . . . . . . . . . . . . . . . . . . . . . . 17
2.2.2 Directmethods ........................ 18
2.3 Solution methods for nonlinear constrained optimization problems 20
2.3.1 Local optimality conditions . . . . . . . . . . . . . . . . . 20
2.3.2 Sequential Quadratic Programming (SQP) . . . . . . . . . 21
2.4 Discussion of direct methods . . . . . . . . . . . . . . . . . . . . . 23
3 Variational mechanics 25
3.1 Lagrangian mechanics . . . . . . . . . . . . . . . . . . . . . . . . 26
3.1.1 Basic definitions and concepts . . . . . . . . . . . . . . . . 26
3.1.2 Discrete Lagrangian mechanics . . . . . . . . . . . . . . . 29
3.2 Hamiltonian mechanics . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2.1 Basic definitions and concepts . . . . . . . . . . . . . . . . 32
3.2.2 Discrete Hamiltonian mechanics . . . . . . . . . . . . . . . 35
3.3 Forcingandcontrol.......................... 37
3.3.1 Forced Lagrangian systems . . . . . . . . . . . . . . . . . . 37
3.3.2 Forced Hamiltonian systems . . . . . . . . . . . . . . . . . 38
3.3.3 Legendre transform with forces . . . . . . . . . . . . . . . 39
3.3.4 Noether’s theorem with forcing . . . . . . . . . . . . . . . 40
3.3.5 Discrete variational mechanics with control forces . . . . . 40
3.3.6 Discrete Legendre transforms with forces . . . . . . . . . . 42
3.3.7 Discrete Noether’s theorem with forcing . . . . . . . . . . 43
ix
4 Discrete mechanics and optimal control (DMOC) 45
4.1 Optimal control of a mechanical system . . . . . . . . . . . . . . . 45
4.1.1 Lagrangian optimal control problem . . . . . . . . . . . . . 46
4.1.2 Hamiltonian optimal control problem . . . . . . . . . . . . 47
4.1.3 Transformation to Mayer form . . . . . . . . . . . . . . . . 48
4.2 Optimal control of a discrete mechanical system . . . . . . . . . . 49
4.2.1 Discrete Optimal Control Problem . . . . . . . . . . . . . 51
4.2.2 Transformation to Mayer form . . . . . . . . . . . . . . . . 52
4.2.3 Fixed boundary conditions . . . . . . . . . . . . . . . . . . 53
4.3 Correspondence between discrete and continuous optimal control
problem ................................ 55
4.3.1 Exact discrete Lagrangian and forcing . . . . . . . . . . . 55
4.3.2 Order of consistency . . . . . . . . . . . . . . . . . . . . . 58
4.3.3 Discrete problem as direct solution method . . . . . . . . . 62
4.4 High-order discretization . . . . . . . . . . . . . . . . . . . . . . . 64
4.4.1 Quadrature approximation . . . . . . . . . . . . . . . . . . 64
4.4.2 High-order discrete optimal control problem . . . . . . . . 67
4.4.3 Correspondence to Runge-Kutta discretizations . . . . . . 69
4.5 Adjointsystem ............................ 76
4.5.1 Continuous setting . . . . . . . . . . . . . . . . . . . . . . 76
4.5.2 Discrete setting . . . . . . . . . . . . . . . . . . . . . . . . 79
4.5.3 The transformed adjoint system . . . . . . . . . . . . . . . 82
4.6 Convergence.............................. 85
5 Implementation, applications and extension 89
5.1 Implementation............................ 89
5.2 Comparison to existing methods . . . . . . . . . . . . . . . . . . . 91
5.2.1 Low thrust orbital transfer . . . . . . . . . . . . . . . . . . 91
5.2.2 Two-link manipulator . . . . . . . . . . . . . . . . . . . . . 94
5.3 Application: Trajectory planning . . . . . . . . . . . . . . . . . . 99
5.3.1 A group of hovercraft . . . . . . . . . . . . . . . . . . . . . 99
5.3.2 Perfect underwater glider . . . . . . . . . . . . . . . . . . . 102
5.4 Application: Optimal control of multi-body systems . . . . . . . . 104
5.4.1 Thefallingcat ........................ 104
5.4.2 A gymnast (three-link mechanism) . . . . . . . . . . . . . 109
5.5 Reconfiguration of formation flying spacecraft – a decentralized
approach................................ 116
6 Optimal control of constrained mechanical systems in multi-body
dynamics 127
6.1 Constrained dynamics and optimal control . . . . . . . . . . . . . 128
x
6.1.1 Optimization problem . . . . . . . . . . . . . . . . . . . . 128
6.1.2 Constrained Lagrange-d’Alembert principle . . . . . . . . . 129
6.1.3 Null space method . . . . . . . . . . . . . . . . . . . . . . 129
6.1.4 Reparametrization . . . . . . . . . . . . . . . . . . . . . . 130
6.2 Constrained discrete dynamics and optimal control . . . . . . . . 130
6.2.1 Discrete constrained Lagrange-d’Alembert principle . . . . 130
6.2.2 Discrete null space method . . . . . . . . . . . . . . . . . . 131
6.2.3 Nodal reparametrization . . . . . . . . . . . . . . . . . . . 132
6.2.4 Boundary conditions . . . . . . . . . . . . . . . . . . . . . 133
6.2.5 Discrete constrained optimization problem . . . . . . . . . 135
6.3 Optimal control for rigid body dynamics . . . . . . . . . . . . . . 135
6.3.1 Constrained formulation of rigid body dynamics . . . . . . 135
6.3.2 Actuation of the rigid body . . . . . . . . . . . . . . . . . 138
6.3.3 Kinematic pairs . . . . . . . . . . . . . . . . . . . . . . . . 139
6.4 Applications.............................. 143
6.4.1 Optimal control of a rigid body with rotors . . . . . . . . . 143
6.4.2 Biomechanics: The optimal pitch . . . . . . . . . . . . . . 144
7 Conclusions and outlook 151
A Definitions 157
B Adjoint system 167
C Convergence proof 171
xi
List of Figures
1.1 The calculus of variations in mechanics and optimal control theory 5
1.2 The discrete calculus of variations in discrete mechanics and opti-
mizationtheory............................ 8
1.3 Optimal control for mechanical systems: the order of variation and
discretization for deriving the necessary optimality conditions . . 9
3.1 Left and right discrete forces . . . . . . . . . . . . . . . . . . . . . 42
3.2 Correspondence between the forced discrete Lagrangian and the
forced discrete Hamiltonian map . . . . . . . . . . . . . . . . . . . 43
4.1 Correspondence between the exact discrete Lagrangian and forces
and the continuous forced Hamiltonian flow . . . . . . . . . . . . 57
4.2 Comparison of solution strategies for optimal control problems:
standard direct methods and DMOC . . . . . . . . . . . . . . . . 62
4.3 Correspondence of indirect and direct methods in optimal control
theory ................................. 85
5.1 Low thrust orbital transfer: approximated cost . . . . . . . . . . . 93
5.2 Low thrust orbital transfer: difference of force and change in an-
gularmomentum ........................... 95
5.3 Low thrust orbital transfer: accuracy of final point condition . . . 96
5.4 Two-link manipulator: model . . . . . . . . . . . . . . . . . . . . 96
5.5 Two-link manipulator: objective function value and momentum-
forceconsistency ........................... 98
5.6 Two-link manipulator: convergence rates . . . . . . . . . . . . . . 99
5.7 Hovercraft:model........................... 100
5.8 Hovercraft: optimal trajectories in phase space . . . . . . . . . . . 102
5.9 Underwater glider: time and effort optimal . . . . . . . . . . . . . 105
5.10 Underwater glider: energy, work and objective values . . . . . . . 106
5.11Fallingcat:model........................... 107
5.12 Falling cat: optimal solution . . . . . . . . . . . . . . . . . . . . . 110
5.13 Falling cat: energy and angular momentum . . . . . . . . . . . . . 111
xiii
5.14 Gymnast: modeled as three-link mechanism . . . . . . . . . . . . 111
5.15 Gymnast: optimal solution . . . . . . . . . . . . . . . . . . . . . . 114
5.16 Gymnast: snapshots of optimal motion sequence . . . . . . . . . . 115
5.17 Dynamical model for formation flying spacecraft: the circular re-
stricted three body problem . . . . . . . . . . . . . . . . . . . . . 117
5.18 Hierarchical formulation of the optimal control of formation flying
spacecraft ............................... 120
5.19 Formation of six spacecraft . . . . . . . . . . . . . . . . . . . . . . 123
5.20 Formation of 30 spacecraft . . . . . . . . . . . . . . . . . . . . . . 124
6.1 Discrete redundant and generalized forces . . . . . . . . . . . . . . 133
6.2 Configuration of a rigid body with respect to an orthonormal frame
{eI}fixedinspace .......................... 136
6.3 Rigid body with three rotors: configuration . . . . . . . . . . . . . 144
6.4 Rigid body with three rotors: torque . . . . . . . . . . . . . . . . 145
6.5 Rigid body with three rotors: energy and angular momentum . . 145
6.6 Rigid body with two rotors: configuration . . . . . . . . . . . . . 146
6.7 Rigid body with two rotors: torque . . . . . . . . . . . . . . . . . 146
6.8 Rigid body with two rotors: energy and angular momentum . . . 147
6.9 Model for a pitcher’s arm . . . . . . . . . . . . . . . . . . . . . . . 148
6.10 Pitcher: snapshots of the optimal motion sequence. . . . . . . . . 150
xiv
Nomenclature
In Chapters 2 to 5 we use a slighly different notation as in Chapter 6. This is due
to the development of a uniform methodology consisting of two different existing
methods with partly the same notation for different objects. Therefore, besides
an overview of the used symbols in Chapters 2 to 5 we will give a list of symbols
that are in additional or in different use in Chapter 6.
List of Acronyms
AD Automatic Differentiation
CRTBP Circular Restricted Three Body Problem
DMOC Discrete Mechanics and Optimal Control
HOCP Hamiltonian Optimal Control Problem
KKT Karush-Kuhn-Tucker
LOCP Lagrangian Optimal Control Problem
NLP Nonlinear Programming
OCP Optimal Control Problem
PMP Pontryagin Maximum Principle
QP Quadratic Programming
SQP Sequential Quadratic Programming
List of Symbols
Qreal configuration manifold
TQ tangent bundle
T∗Qcotangent bundle
Ucontrol manifold
¨
Qsecond order submanifold
¨
Qddiscrete second order submanifold
Cpath space
Cddiscrete path space
xv
Dcontrol path space
Dddiscrete control path space
qconfiguration vector
˙qvelocity vector
pmomentum vector
ucontrol parameter (Chapters 2 to 5)
λLagrange multiplier
LLagrangian of the mechanical system
Lddiscrete Lagrangian
LE
dexact discrete Lagrangian
HHamiltonian of the mechanical system
Kkinetic energy
Vpotential energy
Gaction map
Gddiscrete action map
XLLagrangian vector field
XLddiscrete Lagrangian evolution operator
FLLagrangian flow
FLddiscrete Lagrangian map
XHHamiltonian vector field
FHHamiltonian flow
˜
FLddiscrete Hamiltonian map
ΘLLagrangian one-form
Θ±
Lddiscrete Lagrangian one-form
ΩLLagrangian symplectic form
ΩLddiscrete Lagrangian symplectic form
Θ canonical one-form
Ω canonical two-form
Jmomentum map
JLLagrangian momentum map
JLddiscrete Lagrangian momentum map
JHHamiltonian momentum map
FLfiber derivative (Legendre transform)
F±Lddiscrete fiber derivatives (discrete Legendre transforms)
FHfiber derivative of the Hamiltonian
DELLEuler-Lagrange equations
DDELLdiscrete Euler-Lagrange equations
xvi
fLLagrangian force
fLC Lagrangian control force
fHHamiltonian force
fHC Hamiltonian control force
f±
dleft and right discrete force
f±
Cdleft and right discrete Lagrangian control force
fE±
Cdleft and right exact discrete Lagrangian control force
Jobjective functional
Ccost function of the objective functional (Chapters 2 to 5)
Φ final condition of the objective functional (Mayer term)
LLagrangian of the optimal control system
˜
LLagrangian of the constrained optimization problem
HHamiltonian of the optimal control system
hpath constraint
rfinal point constraint
∇Jacobian
Dkpartial derivatives with respect to the k-th argument
dexterior derivative
iinterior product
ttime
htime step
Tfinal time
GLie group
gLie algebra
φaction of a Lie group
τQnatural projection
πQcanonical projection
Ckset of k-times continuously differentiable functions
F(M) set of continuously differentiable real-valued functions on M
Lp(Rn) Lebesgue space of measurable functions x: [0, T]→Rn
with |x(·)|pintegrable
Wm,p(Rn) Sobolev space consisting of vector-valued measurable functions
x: [0, T]→Rnwhith j-th derivative in Lpfor all 0 ≤j≤m
Ba(x) closed ball centered at xwith radius a
xvii
Chapter 6
Cconstraint manifold
Wspace of generalized forces
ϕplacement of center of mass
˙ϕtranslational velocity of placement of center of mass
{dI}director triad
{˙
dI}director velocities
ωangular velocity
%joint location with respect to body-fixed director triad
uϕincremental displacement of center of mass
θincremental rotation
fredundant force
fϕforce applied to the center of mass
fIforce applied to director dI
τgeneralized force
τϕtranslational force
τθrotational torque
DJacobian
gholonomic constraints
gddiscrete holonomic constraints
Gconstraint Jacobian
Gddiscrete constraint Jacobian
Pnull space matrix
Bcost function of the objective functional
xviii
Chapter 1
Introduction
The optimization and control of physical processes is of crucial importance in all
modern technological sciences. In particular, optimal control theory is a mathe-
matical optimization method for deriving control policies such that a certain
optimality criterion is achieved. Rather than describing its observed behavior,
one is interested in prescribing the motion of a dynamical system. This means
that the laws that govern the system’s behavior must contain variables whose val-
ues can be changed by someone acting outside and independently of the system
itself. Thus, in addition to state variables that define precisely what the system
is doing at time t, we have control variables or parameters that can be used to
modify the subsequent behavior of the system.
In mechanical engineering, one is often interested in steering or guiding a
mechanical system from an initial to a final state under the influence of control
forces such that a given quantity, for example control effort or maneuver time is
minimal. Typical challenges arise in vehicle dynamics, space mission design, or
robotics in determining the time-minimal path of a vehicle, an energy-efficient
trajectory of a satellite in space, or in optimizing the motion of a robot.
Obviously, in contrast to machines, nature behaves in general in an optimal
way. On the one hand, the theory and application of optimal control helps cap-
turing and understanding certain biological processes, such as the motion of a
falling cat that flips in an efficient way to land on its feet. On the other hand,
it can be used to optimize the behavior of biomechanical models of, for example
the human body. In this way, the simulation of optimal control processes may
support the development of prostheses and implants in modern medical surgery.
Another area of growing interest is the optimization of movement in sports. The
knowledge gained from optimal control simulations may help to improve indivi-
dual techniques or even suggest the development of new techniques (as has been
observed during the last decades, for example for high and ski jumping).
In economies, the determination of optimal financing and investment strate-
1
gies (for example, finding an optimal capital structure or optimal mix of funds,
or optimal portfolio choice) for corporations and the economy is important for an
efficient allocation of resources.
Having mentioned just a few fields of application, one can see that optimal
control theory obviously has a high relevance to modern developments in science,
industry and commerce.
As a consequence of nonlinearities present in even the simplest models of in-
terest, an analytical solution to the optimal control system is rarely feasible. This
necessitates numerical methods that approximate the solution of the mathema-
tical model. For the numerical treatment, discretization techniques are required.
A variety of solution strategies exist differing in manner and point in time of the
discretization, which results in different approximation behaviors and properties.
Naturally, one seeks realistic approximations that share the relevant properties
of the analytical solution.
For the simulation of mechanical systems in particular it is desirable to pre-
serve certain qualitative properties and structures such as energy, momentum, and
symplecticity, or, in the case of forced systems, the change in energy and momen-
tum due to the actuation by control forces. Time-stepping schemes which inherit
the (conservation) properties of the continuous mechanical system are referred
to as mechanical integrators. In addition to the benefit of good energy behavior,
the conservation of momentum maps or the symplectic form along the discrete
solution enhances its veritableness since its qualitative and structural charac-
teristics are transferred to the discrete solution. Energy-momentum conserving
schemes relying on a direct discretization of the ODEs, have been widely investi-
gated; see for example [9, 34, 83, 82, 133, 141, 142, 143, 145]. Based on concepts
from discrete variational mechanics, namely the discretization of the variational
formulation behind the ODEs, symplectic-momentum integrators have been de-
rived for example by [56, 73, 91, 93, 113, 132]; see [140, 144] for a discussion on
energy-momentum and symplectic schemes.
In order to solve optimal control problems for mechanical systems, this thesis
links two important areas of research:
optimal control and variational mechanics.
The motivation for combining these fields of investigation is twofold. Besides the
aim of preserving certain properties of the mechanical system for the approxi-
mated optimal solution, optimal control theory and variational mechanics have
their common origin in the calculus of variations - the theory of the optimization
of integrals. In mechanics, the concept of calculus of variations is the principle
of stationary action which, when applied to the action of a mechanical system
can be used to obtain the equations of motion for that system. This principle
led to the development of the Lagrangian and Hamiltonian formulations of clas-
sical mechanics. In optimal control theory the concept of calculus of variations
2
bears fundamental results in order to derive optimality conditions for the control
system under consideration.
In addition to the importance in continuous mechanics and control theory,
discrete calculus of variations and the corresponding discrete variational princi-
ples play an important role in constructing efficient numerical methodologies for
the simulation of mechanical systems and for optimizing dynamical systems.
Due to their common origin in theory as well as in computational approaches,
the combination of these two fields of research provides interesting insight into
theoretical and computational issues and provides profitable approaches to spe-
cific problems.
Optimal control and variational mechanics The theory of optimal control
is an area developed since the 1950s in response to American and Russian efforts
to explore the solar system. The mathematical problems of space navigation
include optimization problems. One wishes to construct trajectories along which
a space vehicle, controlled by a small rocket motor, will reach its destination in
minimum time or using the minimum amount of fuel. These new problems were
not solvable by the methods available at that time and the theory had to be
extended to meet new challenges.
In contrast to the theory of optimal control, classical mechanics is one of the
oldest and largest subjects in science and technology. It deals with the dynamics
of particles, rigid bodies, continuous media (fluid, plasma, and solid mechanics),
as well as other fields of physics (such as electromagnetism, gravity, etc.). The
problem of computing the dynamics of general mechanical systems arose already
in the work of Galilei (published 1638). Newton’s Principia (1687,[122]) allowed
reducing the analysis of the motion of free mass points (the mass points being
planets such as Mars or Jupiter) to the solution of differential equations. However,
many important ideas in mechanics are based on variational principles and are
accredited to Euler, Lagrange and Hamilton. The notion that the laws of nature
act in such a way as to extremize some function facilitates the description of
motions of more complicated systems such as rigid bodies or bodies attached to
each other by rods or springs. We consider the variational problem of extremizing
the action integral
T
Z0
L(q(t),˙q(t)) dt
among all curves q(t) that connect two given points q(0) = q0and q(T) = qT.
Here, the Lagrangian Lconsists of the difference between kinetic energy K=
K(q, ˙q) (which is often of the form 1
2˙qTM(q) ˙qwhere M(q) is symmetric and
positive definite) and potential energy V=V(q). In fact, assuming q(t) to be
3
extremal and considering a variation q(t) + εδq(t) with the same endpoints, that
is with δq(0) = δq(T) = 0 and using partial integration yields
0 =
T
Z0∂L
∂q δq +∂L
∂˙qδ˙qdt=
T
Z0∂L
∂q −d
dt
∂L
∂˙qδq dt.
This leads to the differential equations
d
dt ∂L
∂˙q=∂L
∂q ,(1.1)
which constitute the Lagrangian equations or Euler-Lagrange equations of the
system. The (numerical or analytical) integration of these equations allows one
to predict the motion of any such system from given initial values.
Lagrange (1736-1813, [84, 85]) himself did not recognize the equations of mo-
tion as being equivalent to a variational principle. This was observed only a
few decades later by Hamilton (1805-1865) and is therefore known as Hamilton’s
principle.
A special property of the solutions of the equations of motion in Lagrangian
or Hamiltonian dynamics is the conservation of first integrals. Under certain
suppositions, the energy, the momentum maps related to the system’s symmetries
and the symplectic form remain unchanged along these solutions. This is for
example stated within Noether’s theorem (see Section 3).
If one wishes to talk about control theory for mechanical systems then the
natural “inputs” for these systems are forces. We may think of the Euler-Lagrange
equations for a Lagrangian Las representing the“natural” motion of the system,
that is the motion of the system in the absence of external interaction. A force,
on the other hand is an external interaction. The Lagrange-d’Alembert principle
states how a force fshould appear in the Euler-Lagrange equations. Given a
Lagrangian Land a force field f, the Lagrange-d’Alembert principle for a curve
q(t) in Qis
δ
T
Z0
L(q(t),˙q(t)) dt+
T
Z0
f(q(t),˙q(t)) ·δq(t) dt= 0,
for a given variation δq (vanishing at the end points). Thus, the condition for a
stationary curve takes the form of the standard Euler-Lagrange equations with
forces: d
dt ∂L
∂˙q−∂L
∂q =f(q, ˙q).(1.2)
4
These are a generalization of Lagrange’s equations for mechanical systems with
forcing. For forced systems, the forced version of Noether’s theorem (see Section
3) still gives a relation between the evolution of first integrals over time and the
amount of the applied forces changing the quantities that have been conserved in
the unforced case.
From a purely mathematical point of view, optimal control problems are also
variants of a class of problems of the calculus of variations. As in Hamilton’s
principle, the problem is to minimize an integral (the objective functional) which
is now subject to constraints describing the dynamical behavior of the under-
lying system. These constraints also determine the set of admissible variations
and are typically differential equations or, for mechanical systems, given by the
Lagrange-d’Alembert principle. The constraints on these system’s dynamics can
be adjoined to the objective functional by introducing a time-varying Lagrange
multiplier vector λ, whose elements are called the costates (or adjoints) of the sys-
tem. The optimal control law that minimizes the augmented objective functional
can be derived using the Pontryagin maximum principle (PMP) (a necessary
condition). This condition - as a generalization of the Euler-Lagrange equation -
forms the fundamental theorem of optimal control theory (see Chapter 2). The
resulting boundary value system is also known as state-costate(adjoint) system.
In Figure 1.1 the analogy of the role of the calculus of variations in both
mechanics and optimal control theory is depicted.
Lagrangian system Optimal control system
Calculus of variations
Hamilton’s principle Pontryagin maximum principle
Euler-Lagrange equations necessary optimality conditions
(state-costate system)
Variational principle
Figure 1.1: The calculus of variations in mechanics and optimal control theory.
5
Discrete optimal control and discrete variational mechanics The theory
of discrete variational mechanics has its roots in the optimal control literature
of the 1960s, see for example [29, 61, 65]. In [29] Cadzow developed a discrete
calculus of variations theory in the following way: A function is introduced whose
values depend on a sequence of numbers. This sequence can be thought of as a
discrete time sequence of numbers which evolve with discrete time. The problem
of finding the discrete time sequence minimizing this function requests that the
optimal sequence must satisfy a second-order difference equation. This set of
necessary conditions is called the discrete Euler equation because of its similarity
to the Euler equation of classical calculus of variations.
Equivalent to the application of variational techniques in the optimization of
continuous dynamical systems to derive necessary optimality conditions (Pon-
tryagin maximum principle), discrete variational techniques provide necessary
optimality conditions for discrete control systems. This procedure has been re-
ferred to as the discrete maximum principle ([61]).
The numerical application of discrete calculus of variations to optimal control
problems is known as direct solution method.Direct and indirect methods for
the numerical solution of optimal control problems exist (see Section 2.2 for an
overview and references). The indirect methods are based on the explicit expres-
sion of the necessary optimality conditions that are derived via the Pontryagin
maximum principle. In contrast, direct methods transform the optimal control
problem into a finite dimensional nonlinear optimization problem by a finite di-
mensional parametrization of the controls only or of both, states and controls (see
Section 2.2.2). For example, one uses discrete time-stepping schemes to derive a
discrete description of the dynamical system under consideration. The discrete
equations then serve as equality constraints for the optimization problem. A
solution of the necessary conditions for the resulting constrained optimization
problem known as Karush-Kuhn-Tucker equations is in accordance with a critical
point of the discrete variational principle applied to the discrete objective func-
tion augmented by adjoining the discrete dynamical equations as constraints.
Due to new generations of computers with rapidly growing capacities and the
development of new and efficient techniques for the solution of large constrained
optimization problems, the direct methods have become more popular since the
1970s.
The theory of discrete variational mechanics describes a variational approach
to discrete mechanics and mechanical integrators. The simple but important idea
in discrete mechanics is the following: Consider a mechanical system with con-
figuration manifold Q. The velocity phase space is then TQ and the Lagrangian
is a map L:TQ →R. In discrete mechanics, the starting point is to replace
TQ with Q×Qand we intuitively regard two nearby points q0and q1as being
the discrete analogue of a velocity vector. Now consider a discrete Lagrangian
6
Ld(q0, q1), which we think of as approximating the action integral along the curve
segment between q0and q1. For a discrete curve of points {qk}N
k=0 we calculate
the discrete action along this sequence by summing the discrete Lagrangian on
each adjacent pair. Computing variations of this action with the fixed boundary
points q0and qNgives a discrete version of Hamilton’s principle and results in
the discrete Euler-Lagrange equations.
Analogous to the continuous case, conservation of discrete energy, discrete
momentum maps related to the discrete system’s symmetries and the discrete
symplectic form can be shown. This is due to the discretization of the variational
structure of the mechanical system directly. Early work on discrete mechanics
was often independently done by [30, 89, 90, 100, 103, 104, 105]. In this work,
the point of the discrete action sum, the discrete Euler-Lagrange equations and
the discrete Noether’s theorem were clearly understood. The variational view
of discrete mechanics and its numerical implementation is further developed in
[158, 159] and then extended in [18, 19, 72, 73, 109, 110].
In considering a discrete Lagrangian system as an approximation of a given
continuous system, the discrete system is an integrator for the continuous system.
Since discrete Lagrangian maps preserve the symplectic structure, they are, re-
garded as integrators, necessarily symplectic. This route of a variational approach
to symplectic-momentum integrators has been taken by Suris [147], MacKay [102]
and in a series of papers by Marsden and coauthors, see the review by Marsden
and West [113] and references therein. In [113] a detailed derivation and inves-
tigation of these variational integrators for conservative as well as for forced and
constrained systems is given.
In Figure 1.2 the analogy of applying the discrete calculus of variations to
mechanical and optimal control systems is depicted.
Combining optimal control and variational mechanics This thesis con-
cerns the optimal control of dynamical systems whose behavior can be described
by the Lagrange-d’Alembert principle. To numerically solve this kind of prob-
lem, we make use of discrete calculus of variations only, that means we apply
the discrete variational principle on two layers. On the one hand we use it for
the description of the mechanical system under consideration, and on the other
hand for the derivation of necessary optimality conditions for the optimal control
problem.
In Figure 1.3 the different possibilities arranging discretization and variations
for deriving necessary optimality conditions are illustrated. The left branch is
based on variations on the continuous level only (variation, variation, discretiza-
tion), and corresponds to the application of an indirect solution method. The
middle branch consisting of a mixture of variations on the continuous and the dis-
7
Discrete Lagrangian system Constrained optimization
problem
Discrete calculus of variations
Discrete Hamilton’s principle Discrete PMP
Discrete Euler-Lagrange
equations
(e. g. variational integrators)
Karush-Kuhn-Tucker equations
Discrete variational principle
Figure 1.2: The discrete calculus of variations in discrete mechanics and opti-
mization theory.
crete stage (variation, discretization, discrete variation) denotes the strategy of
a direct solution method. The approach depicted in the right branch (discretiza-
tion, discrete variation, discrete variation), as presented in this thesis comprises
the calculus of variations on the discrete level only. This method is denoted by
DMOC, standing for Discrete Mechanics and Optimal Control.
The application of discrete variational principles already on the dynamical
level (namely the discretization of the Lagrange-d’Alembert principle) leads to
structure-preserving time-stepping equations which serve as equality constraints
for the resulting finite dimensional nonlinear optimization problem. The benefits
of variational integrators are handed down to the optimal control context. For
example, in the presence of symmetry groups in the continuous dynamical system,
also along the discrete trajectory the change in momentum maps is consistent
with the control forces. Choosing the objective function to represent the control
effort, which has to be minimized is only meaningful if the system responds
exactly according to the control forces.
Related work A survey of different methods for the optimal control of dy-
namical systems described by ordinary differential equations is given in Section
2.2. However, to our knowledge, DMOC is the first approach to solutions of
optimal control problems involving the concept of discrete mechanics to derive
8
objective functional +
Lagrange-d’Alembert principle
variation
(mechanics)
objective functional +
Euler-Lagrange equations
discretization
variation
(OCP)
discrete objective
function + dis-
cretized differential
equation
Pontryagin
maximum principle
Karush-Kuhn-
Tucker
equations
discretized
Pontryagin
maximum principle
discretization
discrete objective function +
discrete Lagrange-
d’Alembert principle
discrete objective function +
discrete Euler-
Lagrange equations
Karush-Kuhn-
Tucker
equations
discrete
variation
(mechanics)
discrete
variation
(OCP)
discretization
discrete
variation
(OCP)
DMOC
indirect direct
Figure 1.3: Optimal control for mechanical systems: the order of variation and
discretization for deriving the necessary optimality conditions.
9
structure-preserving schemes for the resulting optimization algorithm.
Since our first formulations and applications to space mission design and for-
mation flying ([67, 69, 68]), DMOC has been applied for example to problems
from robotics and biomechanics ([74, 78, 79, 114, 124, 134]) and to image ana-
lysis ([116]). From the theoretical point of view, considering the development
of variational integrators, extensions of DMOC to mechanical systems with non-
holonomic constraints or to systems with symmetries are quite natural and have
already been analyzed in [78, 79]. Further extensions are currently under in-
vestigation, for example DMOC for hybrid systems [124] and for constrained
multi-body dynamics (see Chapter 6 and [98, 99]).
DMOC related approaches are presented in [87, 88]. The authors discretize
the dynamics by a Lie group variational integrator. Rather than solving the
resulting optimization problem numerically, they construct the discrete necessary
optimality conditions via the discrete variational principle and solve the resulting
discrete boundary value problem (the discrete state and adjoint system). The
method is applied to the optimal control of a rigid body and to the computation
of attitude maneuvers of a rigid spacecraft.
Main issues and outline of this work
The primary objective of this work is the development of a unified variational
framework for the optimal control of mechanical systems in a continuous and a
discrete setting. Based on discrete variational principles on all levels, the discrete
optimal control problem yields a strategy for the efficient solution of these kind
of problems.
In addition to the theoretical and numerical investigation of this method, we
(i) give a focused survey of optimal control theory and existing methods and
on discrete variational mechanics, (ii) present numerical results of a variety of
different problems arising in different areas of application, (iii) develop efficient
computational approaches for exploiting system structures to reduce for example
the computational effort, (iv) extend the optimal control framework to mechanical
systems with constraints in multi-body dynamics.
Moreover, the numerical performance of the developed method and its rela-
tionship to other existing optimal control methods are investigated both through
theoretical considerations as well as with the help of numerical examples.
The outline of this thesis is as follows:
Chapter 2 and Chapter 3 comprise the basic principles of the two areas of
research linked within this work. In Chapter 2 we review the relevant con-
cepts from classical optimal control theory, ranging from the problem formulation
10
to a discussion of different solution methodologies to optimal control problems.
We introduce the Pontryagin maximum principle and the Karush-Kuhn-Tucker
equations providing the necessary conditions of optimality for optimal control
problems and for constrained optimization problems, respectively. Both kinds of
problems are linked via the application of direct solution methods transforming
the optimal control problem into a constrained optimization problem. We give a
brief overview of numerical solution strategies and refer to corresponding software
packages.
In Chapter 3 we introduce the relevant concepts from classical variational
mechanics and discrete variational mechanics. Following the work of [113], we
give basic definitions and concepts for both Lagrangian mechanics as well as
for Hamiltonian mechanics. In this context, we introduce Hamilton’s principle,
Noether’s theorem concerning preservation properties, and the Legendre trans-
form linking Lagrangian and Hamiltonian mechanics. We transfer these concepts
to the discrete mechanics framework. In the second part of Chapter 3, we focus on
the Lagrangian and Hamiltonian description of control forces for the established
framework of variational mechanics. Definitions and concepts of the variational
principle, the Legendre transform, and Noether’s theorem are readopted for the
forced case in both the continuous as well as the discrete setting.
The main part of this work - the description of our optimal control method
DMOC - is presented in Chapter 4 linking optimal control theory and variational
mechanics to set up a variational formulation for a special class of problems: opti-
mal control problems for mechanical systems. Again, we develop a continuous and
a discrete framework for Lagrangian and Hamiltonian optimal control problems.
We link both frameworks viewing the discrete problem as an approximation of the
continuous one. The application of discrete variational principles for a discrete
description of the dynamical system leads to structure-preserving time-stepping
equations. Here, the special benefits of variational integrators are handed down
to the optimal control context. These time-stepping equations serve as equality
constraints for the resulting finite dimensional nonlinear optimization problem,
therefore the described procedure can be categorized as a direct solution method
as introduced in Chapter 2.
Based on quadrature approximation we construct optimal control problem
approximations of higher order and show how to compute the order of consis-
tency with the help of a comparison of continuous and discrete Lagrangians and
forces. In arranging our formulation in the context of existing concepts and me-
thodologies for optimal control problems, we show the equivalence of the discrete
Lagrangian optimal control problems to those resulting from Runge-Kutta dis-
cretizations of the corresponding Hamiltonian system. This equivalence allows
us to construct and compare the adjoint systems of the continuous and the dis-
crete Lagrangian optimal control problem. In this way, one of our main results
11
is related to the order of approximation of the adjoint system of the discrete
optimal control problem to that of the continuous one. With the help of this
approximation result, we show that the solution of the discrete Lagrangian opti-
mal control system converges to the continuous solution of the original optimal
control problem. The proof strategy is based on existing convergence results of
optimal control problems discretized via Runge-Kutta methods ([41, 53]).
Chapter 5 gives a detailed description of implementation issues of DMOC as
well as a short overview of existing routines for the optimization and differentia-
tion of the resulting problem. Furthermore, a variety of numerical examples and
results demonstrating the capability of DMOC to solve optimal control problems
are presented. Applications range from trajectory planning problems of single
and multiple vehicles in formation to the optimal control of multi-body dynam-
ics. Here, a variety of different formulations for objective functionals, boundary
conditions and path constraints are used. We numerically verify the preservation
and convergence properties of DMOC and the benefits of using DMOC compared
to other standard methods to the solution of optimal control problems. Finally,
we develop and investigate a spatial decentralized approach to a problem in space
mission design: the efficient reconfiguration of formation flying spacecraft.
In Chapter 6 we extend the developed framework for the optimal control of
mechanical systems to constrained mechanical systems in multi-body dynamics.
In this context, the multi-body system is formulated as a constrained system,
that is each body is viewed as a constrained continuum, described in terms of
redundant coordinates subject to holonomic constraints. The couplings between
the bodies are characterized via external holonomic constraints describing the
kinematic conditions arising from the specific joint connections. Within the vari-
ational framework of Lagrangian mechanics the configuration constraints are en-
forced adjoining them to the Lagrangian with Lagrange multipliers. With the
help of the null space method (see [7]), we reduce the number of unknowns (con-
figurations and torques at the time nodes) and the dynamic constraints to the
minimal possible number. The combination with DMOC leads to an optimiza-
tion algorithm that inherits both the conservation properties from the constrained
scheme as well as the benefit of exact constraint fulfillment, correct computation
of the change in momentum maps and good energy behavior. We apply the de-
veloped methodology to the optimal control of two multi-body systems arising in
robotics and biomechanics.
The thesis concludes with a summary of the results and a discussion of open
problems and possible future directions.
The Appendix provides a collection of definitions and relevant notions. More-
over, it contains details concerning the derivation of the transformed adjoint
system introduced in Section 4.5 and a detailed proof of the convergence result
stated in Section 4.6 based on the proof in [41, 53].
12
Chapter 2
Optimal control
In this chapter we give an introduction into optimal control theory beginning with
the general problem formulation and the necessary optimality conditions derived
by the Pontryagin maximum principle. We give an overview of existing solution
methods for optimal control problems falling into two kinds of approaches - indi-
rect and direct methods - where the focus will be on the latter one. The solution
methods are thoroughly compared and their advantages and disadvantages are
discussed. Here, we mainly follow the work of [17].
2.1 Optimal control problem
The aim of optimal control is to guide or steer certain processes, arising in nature
and engineering, such that a given quantity, for example control effort or ma-
neuver time is minimal. To be more precise, a given objective functional has to
be optimized by taking into account the dynamics of the process described by a
dynamical system. In this section we give a mathematical formulation of optimal
control problems and introduce necessary optimality conditions for this kind of
problems.
2.1.1 Problem formulation
Assume that on the time interval I= [0, T], a dynamical system is given by a
differential equation of the form
˙x(t) = f(x(t), u(t)) (2.1)
with the state function x:I→Rnx, the control function u:I→Rnuand
f:Rnx×Rnu→Rnxcontinuously differentiable. Assume that the system has to
13
be steered within the time interval Ifrom an initial to a final state given by the
final point constraint r:Rnx→Rnr,0≤nr≤nx,
r(x(T)) = 0,(2.2)
and the trajectories x(t) and u(t) are bounded by the path constraint defined by
h:Rnx×Rnu→Rnh,
h(x(t), u(t)) ≥0.(2.3)
At the same time, a given objective functional
J(x, u) = ZT
0
C(x(t), u(t)) dt(2.4)
with C:Rnx×Rnu→Rcontinuously differentiable and a final condition defined
by Φ : Rnx→R,
Φ(x(T)),(2.5)
the Mayer term of the objective, have to be minimized. The sum of (2.4) and
(2.5) is called the Bolza type objective functional. An optimal control problem
can now be formulated as
Problem 2.1.1 (Optimal Control Problem (OCP))
min
x(·),u(·),(T)J(x, u) = ZT
0
C(x(t), u(t)) dt+ Φ(x(T)) (2.6a)
subject to
˙x(t) = f(x(t), u(t)),(2.6b)
x(0) = x0,(2.6c)
0≤h(x(t), u(t)),(2.6d)
0 = r(x(T)).(2.6e)
The interval length Tmay either be fixed, or appear as a degree of freedom in
the optimization problem.
Remark 2.1.2 More general, additional algebraic constraints can be included as
well. Then we obtain a differential algebraic system instead of the ordinary dif-
ferential equation (2.6b). As the optimal control of differential algebraic systems
is not the main focus in this thesis we restrict ourselves to the problem formu-
lation above. In Chapter 6 optimal control problems with additional algebraic
constraints arising from constrained multi-body dynamics are considered.
14
Definition 2.1.3 A pair (x(·), u(·)) is admissible (or feasible), if the constraints
(2.6b)-(2.6e) are fulfilled. The set consisting of all admissible (feasible) pairs is the
admissible (feasible) set of Problem 2.1.1. An admissible (feasible) pair (x∗, u∗)
is an optimal solution of Problem 2.1.1, if
J(x∗, u∗)≤J(x, u)
for all admissible (feasible) pairs (x, u). The admissible (feasible) pair (x∗, u∗) is
alocal optimal solution, if there exists a neighborhood Bδ((x∗, u∗)), δ > 0, such
that
J(x∗, u∗)≤J(x, u)
for all admissible (feasible) pairs (x, u)∈Bδ((x∗, u∗)). The function x∗(t) is called
(locally) optimal trajectory, and the function u∗(t) is the (locally) optimal control.
2.1.2 Necessary conditions for optimality
In this section, we introduce necessary conditions for the optimality of a solution
(x∗(t), u∗(t)) to Problem 2.1.1. We restrict ourselves to the case of optimal con-
trol problems with the controls constrained to the (nonempty) pointwise control
constraint set U={u(t)∈Rnu|h(u(t)) ≥0}(also called the set of pointwise
admissible controls) and fixed final time T. Therefore, (2.6) reduces to
min
x(·),u(·)J(x, u) = ZT
0
C(x(t), u(t)) dt+ Φ(x(T)) (2.7a)
subject to
˙x(t) = f(x(t), u(t)),(2.7b)
x(0) = x0,(2.7c)
0 = r(x(T)),(2.7d)
u(t)∈U. (2.7e)
Necessary conditions for optimality of solution trajectories η(t)=(x∗(t), u∗(t)),
t∈Ican be derived based on variations of an augmented cost function, the
Lagrangian of the control system.
Definition 2.1.4 The Lagrangian of the optimal control system (2.7) is a func-
tion L:Rnx×Rnu×Rnxdefined by
L(η, λ) = ZT
0
C(x(t), u(t)) + λT(t)·( ˙x−f(x(t), u(t))) dt+ Φ(x(T)),(2.8)
15
where the variables λi, i = 1, . . . , nx, are the adjoint variables or the costates. A
point (η∗(t), λ∗(t)) is a saddle point of (2.8), if for all η(t) and λ(t) it holds
L(η(t), λ∗(t)) ≤ L(η∗(t), λ∗(t)) ≤ L(η∗(t), λ(t)).
The function H:Rnx×Rnu×Rnx→Rdefined by
H(x, u, λ) := −C(x, u) + λT·f(x, u),(2.9)
is called the Hamiltonian of the optimal control problem.
To motivate Theorem 2.1.5, we associate a local solution of (2.7) with a saddle
point of the Lagrangian L. Thus, by setting variations of Lwith respect to η
and λto zero, the resulting Euler-Lagrange equations (see Chapter 3) provide
necessary optimality condition for the optimal control problem (2.7). This result
is stated in the following Theorem.
Theorem 2.1.5 (Pontryagin Maximum Principle) Let (x∗, u∗)be an opti-
mal solution of (2.7). Then, there exists a piecewise continuous differentiable
function λ: [0, T]→Rnxand a vector α∈Rnrsuch that
H(x∗(t), u∗(t), λ(t)) = max
u(t)∈UH(x(t), u(t), λ(t)) for all t∈[0, T],(2.10a)
˙x∗(t) = ∇λH(x∗(t), u∗(t), λ(t)), x∗(0) = x0,(2.10b)
˙
λ(t) = −∇xH(x∗(t), u∗(t), λ(t)),(2.10c)
λ(T) = ∇xΦ(x∗(T)) − ∇xr(x∗(T)) α. (2.10d)
Proof: A general proof requires more technical details as the intuitive derivation
via the calculus of variations and can be found, for example in [127].
Early developments of the maximum principle have been carried out by Pon-
tryagin et al. [127], Isaacs [63], and Hestenes [59]. The approach has been ex-
tended to handle general constraints (2.6d) on the control and state variables
(for an overview see, for example Hartl, Sethi, and Vickson [58]). The Pontrya-
gin maximum principle plays an important role in the construction of solution
methods (see Section 2.2.1) and to analyze their convergence properties (see Sec-
tion 4.6).
2.2 Solution methods for optimal control prob-
lems
In this section, solution methods for optimal control problems of the form of
Problem 2.1.1 are introduced and discussed (an overview of existing solution
16
methods and a more detailed description can be found, for example in [17]). The
numerical treatment of optimal control problems can be mainly divided into two
branches: the indirect and the direct methods.
Indirect methods use the explicit expression of the necessary conditions for
the optimal control problem, derived via the Pontryagin maximum principle.
Instead, direct methods transform the problem into a finite dimensional nonlinear
optimization problem by a finite dimensional parametrization of the controls only
or of both, states and controls.
2.2.1 Indirect methods
The necessary conditions for optimality of solution trajectories x∗(t) and u∗(t), t ∈
Igiven by the Pontryagin maximum principle can be formulated as a boundary
value problem
x∗(0) = x0,
0 = r(x∗(T)),
λ∗(T) = ∇xΦ(x∗(T)) − ∇xr(x∗(T)) α, (2.11)
˙x∗(t) = ∇λH(x∗(t), u∗(t), λ∗(t)),
˙
λ∗(t) = −∇xH(x∗(t), u∗(t), λ∗(t)),
with α∈Rnrbeing the Lagrange multipliers for the end point constraints. The
optimal controls are obtained by a pointwise maximization of the Hamiltonian:
u∗(t) = arg max
u(t)∈UH(x∗(t), u∗(t), λ∗(t)).(2.12)
Gradient methods can now be used to iteratively improve an approximation of the
optimal control by maximizing the Hamiltonian subject to the boundary value
problem (2.11) (Cauchy [31]; Kelley [77]; Tolle [152]; Bryson and Ho [27]; Miele
[117]; Chernousko and Luybushin [33]). In each iteration step, the differential
equation (2.6b) is numerically integrated forward in time while the adjoint dif-
ferential equations are integrated backwards in time.
On the other hand multiple shooting and collocation can be used to solve
the resulting multipoint boundary value problem (MPBVP) derived from the
necessary conditions of optimality. Numerical multiple shooting methods have
been developed by Fox [47], Keller [76], Bulirsch [28], Deuflhard [36], Bock [20],
and Hiltmann [60], collocation methods by for example Dickmanns and Well [37],
B¨ar [5] and Ascher et al. [2]. For an introduction into multiple shooting we refer
to Ascher et al. [3] or Stoer and Bulirsch [146].
17
For the practical use of indirect methods significant knowledge and experience
in the theory and the problem under consideration is required. Proper formula-
tions of the necessary conditions in a numerically suitable way must be derived.
In addition, changes in the problem formulation (for example by a modification
of the model equations) are difficult to include in the solution procedure. Fur-
thermore, indirect methods are sensitive with respect to initial guesses for the
adjoints and controls. Therefore, carefully chosen suitable initial guesses of the
state and adjoint trajectories must be provided.
These practical drawbacks of indirect methods motivate the introduction of a
second class of numerical techniques finding solutions to optimal control problems,
the class of direct methods.
2.2.2 Direct methods
The basic idea of direct methods for the solution of optimal control problems is to
transcribe the original infinite dimensional problem (2.6) into a finite dimensional
Nonlinear Programming (NLP) problem (Kraft [80]; Bock and Plitt [21]; Biegler
[16]; Betts [11, 12]; Polak [126]; von Stryk and Bulirsch [156]). This approach
has been pushed by the progress in nonlinear optimization (Han [57]; Powell
[128]; Barclay, Gill, and Rosen [6]; Betts [11]). Two basically different solution
strategies for the reformulated problem exist (see Pytlak, [129], for a survey):
(i) Sequential simulation and optimization: In every iteration step of the opti-
mization method, the model equations are solved “exactly” by a numerical
integration method for the current guess of control parameters.
(ii) Simultaneous simulation and optimization: The discretized differential equa-
tions enter the transcribed optimization method as nonlinear constraints
that can be violated during the optimization procedure. At the solution,
however, they have to be satisfied.
In the following, we briefly sketch three different strategies of direct methods:
single shooting (a pure sequential approach), collocation methods (a pure simulta-
neous approach) and multiple shooting (a mixture of sequential and simultaneous
approach).
Direct single shooting In the direct single shooting method (for example
Kraft [80]) the control function is approximated by a finite dimensional parame-
trization. In each iteration step, the initial value problem (2.6b), (2.6c) deter-
mined by a guess for the control parameter vector, can be solved for the solution
trajectory x(t) depending on the control parameters only. By substituting this
trajectory into the objective functional one obtains a cost function depending
18
on the control parameters only. This has to be minimized subject to path con-
straints, which have been discretized on a given time grid and subject to the final
point constraint. Here, all equations are formulated depending on the control
parameters only.
Direct multiple shooting The direct multiple shooting method (Bock and
Plitt [21]) is based on the same idea as the single shooting approach, but addi-
tionally, the time interval Iis divided into subintervals. On each subinterval a
corresponding initial value problem has to be solved. To guarantee continuity of
the solution trajectories over the interval boundaries, the states at the boundaries
of each interval are introduced as extra variables (node values). For these nodes
we enforce, that the solution trajectory of the previous interval matches the ini-
tial value of the following. This strategy results in an optimization problem that
depends on the control parameters and the node values only. The problem is re-
stricted by the discretized path constraints and the matching conditions between
the intervals as inequality and equality constraints, respectively.
Direct collocation In the direct collocation method (for example von Stryk
[153]), both state and control variables are approximated by piecewise defined
functions on a given time grid. Within each collocation interval these functions
are chosen as parameter dependent polynomials. The polynomials’ coefficients
are determined by enforcing continuity or higher order differentiability of the ap-
proximating functions at the boundaries of the subintervals. In order to formulate
a nonlinear optimization problem, the model equations and the continuous con-
straints are explicitly discretized. For example the model equations are only to
be satisfied at the collocation points within each subinterval and the inequality
constraints on a second grid within the time interval I. Substituting the ap-
proximating functions into the objective functional leads to the formulation of an
optimization problem that is restricted by discrete path constraints and discrete
model equations.
Other discrete approximation techniques are based for example on the use of
Runge-Kutta discretizations ([53]), finite differences ([106]), or Galerkin methods
([43]).
Remark 2.2.1 In Section 2.4 we give a short comparison of the direct methods
introduced above as well as a classification of the optimal control method DMOC
within these concepts.
19
2.3 Solution methods for nonlinear constrained
optimization problems
In this section, methods for the solution of nonlinear optimization problems are
presented. These problems arise from the discretization of optimal control prob-
lems by direct transcription methods as described in Section 2.2.2.
Let ξbe the set of parameters introduced by the discretization of an infinite
dimensional optimal control problem. Then, the nonlinear programming problem
to be solved is minξϕ(ξ)
subject to a(ξ) = 0, b(ξ)≥0,(2.13)
where ϕ:Rn→R, a :Rn→Rm,and b:Rn→Rpare continuously differen-
tiable.
2.3.1 Local optimality conditions
First of all, we review some conditions, which allow to decide whether a point ξ∗
is a (local) solution of an optimization problem.
Afeasible point is a point ξ∈Rnthat satisfies a(ξ) = 0 and b(ξ)≥0. A local
minimum of (2.13) is a feasible point ξ∗which has the property that ϕ(ξ∗)≤ϕ(ξ)
for all feasible points ξin a neighborhood of ξ∗. A strict local minimum satisfies
ϕ(ξ∗)< ϕ(ξ) for all neighboring feasible points ξ6=ξ∗.
Active inequality constraints bact(ξ) at a feasible point ξare those components
bj(ξ) of b(ξ) with bj(ξ) = 0. We subsume the equality constraints and the active
inequalities at a point ξ(known as active set) in a combined vector function of
active constraints as
˜a(ξ) := a(ξ)
bact(ξ).
Note that the active set may be different at different feasible points.
Regular points are feasible points ξthat satisfy the condition that the Jacobian
of the active constraints, ∇˜a(ξ)T, has full rank, that means that all rows of ∇˜a(ξ)T
are linearly independent.
To investigate local optimality in the presence of constraints, we introduce
the Lagrangian multiplier vectors λ∈Rmand µ∈Rp, that are also called adjoint
variables, and we define the Lagrangian function ˜
Lby
˜
L(ξ, λ, µ) := ϕ(ξ)−λTa(ξ)−µTb(ξ).(2.14)
We now state a variant of the Karush-Kuhn-Tucker necessary conditions for local
optimality of a point ξ∗. These conditions have been derived first by Karush in
20
1939 ([75]) and independently by Kuhn and Tucker in 1951 ([81]). For brevity,
we restrict our attention to regular points only.
Theorem 2.3.1 (Karush-Kuhn-Tucker conditions (KKT)) If a regular
point ξ∗∈Rnis a local optimum of the NLP problem (2.13), then there exist
unique Lagrange multiplier vectors λ∗∈Rmand µ∗∈Rpsuch that the triple
(ξ∗, λ∗, µ∗)satisfies the following necessary conditions:
∇ξ˜
L(ξ∗, λ∗, µ∗) = 0,
a(ξ∗) = 0,
b(ξ∗)≥0,
µ∗≥0,
µ∗
jbj(ξ∗) = 0, j = 1, . . . , p.
Proof: See for example [64].
A triple (ξ∗, λ∗, µ∗) that satisfies the Karush-Kuhn-Tucker conditions is called
aKarush-Kuhn-Tucker point (KKT point).
2.3.2 Sequential Quadratic Programming (SQP)
Sequential quadratic programming (SQP) (Han [57]; Powell [128]) is a very effi-
cient iterative method to find a KKT point (ξ∗, λ∗, µ∗) of the NLP problem (2.13).
The basic idea of SQP is to model (2.13) at a given approximate solution ξk, by
a quadratic programming (QP) subproblem derived from a quadratic approxima-
tion of the Lagrangian of the NLP problem subject to the linearized constraints.
Then the solution of this subproblem is used to construct a better approximation
ξk+1, where the QP problem is assumed to reflect in some way the local proper-
ties of the original problem (for a detailed description see for example Barclay,
Gill, and Rosen [6]; Gill, Murray, and Saunders [48]; Boggs and Tolle [22]). This
process is iterated to create a sequence of approximations that will converge to a
solution ξ∗.
To establish global convergence for constrained optimization algorithms, a
way of measuring progress towards a solution is needed. For SQP this is done by
constructing a merit function, a reduction in which implies that an acceptable
step has been taken.
For the class of SQP methods considered, the vector of optimization variables
ξk∈Rnitself and the vector of multipliers vk:= (λ, µ)k∈Rm+pare changed
from (the major SQP) iteration number kto iteration number k+ 1 by
ξk+1
vk+1 =ξk
vk+αkdk
uk−vk, k = 0,1,2,...,
21
where the search direction (dk, uk) is obtained as a solution of a nearly constrained
quadratic problem resulting from a quadratic approximation of the Lagrangian
˜
L(ξ, λ, µ) = ϕ(ξ)−
m
X
i=1
λiai(ξ)−
p
X
j=1
µjbj(ξ), λ ∈Rm, µ ∈Rp:
min
d
1
2dTAkdT+∇ϕ(ξk)Td
subject to ∇ai(ξk)Td+ai(ξk) = 0, i = 1, . . . , m,
∇bj(ξk)Td+bj(ξk)≥0, j = 1, . . . , p.
(2.15)
Usually, Akis a positive definite approximation of the Hessian Hkof the La-
grangian ˜
L(ξk, λk, µk). The search direction dkis the solution of the QP problem
(2.15) and ukis the corresponding multiplier. The quadratic (sub-)problem (2.15)
itself is solved by an iterative method (usually, an active set strategy or an inte-
rior point method is employed). The success of SQP depends on the existence of
a rapid and accurate algorithm for solving quadratic programs.
The step size αk∈Ris obtained by an (approximate) one-dimensional mini-
mization of a merit function (line search)
ψrξ
v+αd
u−v
with respect to α. A suitable merit function is, for example the Lagrangian
augmented by penalty terms.
Existing SQP methods differ mainly by the choice of the step length αk, and
the choice of the approximation of the Hessian matrix Ak. The iterates (ξk, vk)
form a sequence that is expected to converge towards a KKT point (ξ∗, v∗) of
the NLP problem. In practice, the iterations are stopped when a prespecified
convergence criterion is fulfilled.
Remark 2.3.2 SQP methods are only guaranteed to find a local solution of
the NLP problem. Algorithms for finding the global solution are of an entirely
different flavor.
A widely used and robust general-purpose line search based SQP method is
NPSOL (Gill, Murray, Saunders, and Wright [49]), which is suitable for small
to medium sized problems. The treatment of optimal control problems via di-
rect transcription methods usually results in large scale problems with sparsity
properties, that is the Jacobians of objective and constraint function are sparse
matrices. This structure can be exploited to solve large quadratic programming
problems more efficiently. For example, the sparse SQP method SNOPT is a
successor of NPSOL and an efficient and robust, general-purpose SQP method
for large-scale problems (Gill, Murray, and Saunders [48]).
22
2.4 Discussion of direct methods
In this section we briefly compare different direct methods. Some advantages
and disadvantages of their application are discussed. After an overview of corre-
sponding software tools, this chapter concludes with a brief classification of the
optimal control method DMOC developed within this thesis.
Comparison For direct single shooting in each major SQP iteration an initial
value problem is numerically solved with high solution accuracy. Consequently,
the resulting NLP problem remains small and the dynamic model is fulfilled dur-
ing all SQP iterations, such that in time critical cases, a premature stop yields
a physical trajectory, which however may still violate state and endpoint con-
straints. Similarly, in the direct multiple shooting method, the underlying initial
value problems are numerically solved within prescribed accuracy in each SQP
iteration. However, the continuity of the system trajectory is only fulfilled after
successful termination of the SQP solution procedure. Although the resulting
NLP problem is larger than for single shooting, its structure can be exploited to
yield even faster convergence.
The direct collocation method leads to potentially faster computations com-
pared to shooting techniques, as the ODE simulation and the control optimization
problems are solved simultaneously. For efficient computations, the sparsity of
the problem can be exploited at all levels.
For highly unstable systems (that is, initial value problems with a strong de-
pendence on the initial values) the direct shooting optimization algorithm inherits
the ill-conditioning of the initial value problem, even if the optimization problem
itself is well-conditioned. In contrast, multiple shooting and collocation are able
to optimize such systems much better.
Additionally, the use of collocation methods enables a reliable estimation of
the adjoint variables on the entire state variable discretization grid [153].
Software The single shooting algorithm can, for example be found in soft-
ware packages gOPT (Process Systems Enterprise, [101]), DYNOPT (Abel et al.
[1]), OPTISIM (Engl et al. [42]). An implementation of the multiple shooting
method is found, for example within the optimal control package MUSCOD-II
(Leineweber, [92]), or in Petzold et al. [125]). Collocation algorithms have been
implemented by Betts and Huffmann [15] (SOCS), Cervantes and Biegler [32],
Schulz [136] (OCPRSQP), and von Stryk [155] (DIRCOL).1
1OCPRSQP uses a partially reduced SQP method. DIRCOL employs SNOPT (Gill et al.
[48]). SNOPT approximates the Hessian of the NLP Lagrangian by limited-memory quasi-
Newton updates and uses a reduced Hessian algorithm for solving the QP subproblems. The
23
DMOC Arranging the optimal control method DMOC within the context of
the presented methodologies, it mostly resembles a direct transcription method
and would therefore be classified within the category of simultaneous methods.
However, the derivation is of a different flavor as we will present in Chapter 4.
Differences and similarities to existing simultaneous methods will be discussed in
Sections 4.3.3 and 4.4.3 and numerically compared in Section 5.2. .
null-space matrix of the working set in each iteration is obtained from a sparse LU factorization.
In the code SOCS of Betts and Frank [14] a Schur-complement QP method is implemented
instead of a reduced-Hessian QP method.
24
Chapter 3
Variational mechanics
During the progress of classical mechanics two main branches developed: Lagran-
gian and Hamiltonian mechanics. The Lagrangian formulation of mechanics can
be based on the observation that there are variational principles behind the fun-
damental laws of force balance as given by Newton’s law F=ma. That means
that the laws of nature act in such a way as to extremize some functional known
as action. The Hamiltonian formalism focuses on the observation of energy and
can be embedded into a certain geometrical structure. The Lagrangian forma-
lism relies on the coordinate’s position and velocity in the phase space TQ. The
Hamiltonian formalism instead considers the position and the momentum as in-
dependent coordinates in the phase space T∗Q, which is the cotangent bundle to
the configuration manifold Q. Fortunately, in many cases these formulations are
equivalent, as we shall see within this chapter. However, since the Hamiltonian
perspective is advantageous over the Lagrangian in some situations and disad-
vantageous in others, we introduce the main concepts of variational mechanics
from both, the Lagrangian and the Hamiltonian point of view. Starting with the
continuous formulation, we present the corresponding discrete formulation for
each geometric structure. For both, the continuous and the discrete framework,
we introduce the most important properties that are relevant for the focus of this
work.
Our formulations are mainly based on the work of Marsden and West in [113].
They used the concept of discrete variational mechanics to derive variational
integrators simulating initial value problems in dynamical mechanics. This thesis
aims a description of optimal control systems. Therefore, the last section in this
chapter gives an extensive formulation of control forces that are included to the
existing concepts of variational mechanics with dissipative forces.
For a detailed description of discrete variational mechanics for Lagrangian and
Hamiltonian systems we refer to [73, 86, 94, 95, 113, 158]. In [113] an overview
of references and the history concerning variational mechanics and variational
25
integrators is given. In addition, we refer to Marsden and Ratiu [111] for historical
remarks and background on geometric mechanics.
3.1 Lagrangian mechanics
3.1.1 Basic definitions and concepts
Consider an n-dimensional configuration manifold Q(see A.1) with local coordi-
nates q= (q1, . . . , qn), with associated state space given by the tangent bundle
TQ (see A.3), and a Lagrangian L:TQ →R. Given a time interval [0, T], we
define the path space to be
C(Q) = C([0, T], Q) = {q: [0, T]→Q|qis a C2curve}
and the action map G:C → Rto be
G(q) =
T
Z0
L(q(t),˙q(t)) dt. (3.1)
The tangent space TqC(Q) to C(Q) at the point q(see A.3) is the set of C2maps
vq: [0, T]→TQ such that τQ◦vq=q, where τQ:TQ →Qis the natural
projection (see A.4). We define the second order submanifold of T(TQ) (see A.2)
to be ¨
Q={w∈T(TQ)|TτQ(w) = τTQ(w)} ⊂ T(TQ)
where τTQ :T(TQ)→TQ and τQ:TQ →Qare the natural projections. ¨
Q
is the set of second order derivatives d2q/dt2(0) of curves q:R→Q, which are
elements of the form ((q, ˙q),( ˙q, ¨q)) ∈T(TQ).
To describe the dynamics of a mechanical system given by a Lagrangian Lon
the tangent bundle TQ, we have the following theorem.
Theorem 3.1.1 (Hamilton’s principle) Given a CkLagrangian L,k≥2,
there exists a unique Ck−2mapping DELL:¨
Q→T∗Qand a unique Ck−1one-
form ΘLon TQ (see A.8), such that, for all variations δq ∈TqC(Q)of q(t)(see
A.27), we have
dG(q)·δq =
T
Z0
DELL(¨q)·δq dt+ ΘL( ˙q)·ˆ
δqT
0,(3.2)
with dGthe differential of G(see A.7), and where
ˆ
δq =q(t),∂q
∂t (t),δq(t),∂δq
∂t (t).
26
The mapping DELLis called the Euler-Lagrange map and has the coordinate
expression
(DELL)i=∂L
∂qi−d
dt
∂L
∂˙qi.
The one-form ΘLis called the Lagrangian one-form and in coordinates is given
by
ΘL=∂L
∂˙qidqi.(3.3)
Proof: Computing the variations of the action map gives
dG(q)·δq =
T
Z0∂L
∂qiδqi+∂L
∂˙qi
d
dtδqidt
=
T
Z0∂L
∂qi−d
dt
∂L
∂˙qiδqidt+∂L
∂˙qiδqiT
0
,
using integration by parts. The terms of this expression can be identified as the
Euler-Lagrange map and the Lagrangian one-form.
Lagrangian vector fields and flows
The Lagrangian vector field XL:TQ →T(TQ) is a second order vector field on
TQ (see A.6) satisfying
DELL◦XL= 0 (3.4)
and the Lagrangian flow FL:TQ ×R→TQ is the flow of XL(see A.6). For the
map FLat some fixed time twe shall write Ft
L:TQ →TQ.1
A curve q∈ C(Q) is said to be a solution of the Euler-Lagrange equations if the
first term on the right-hand side of (3.2) vanishes for all variations δq ∈TqC(Q).
This is equivalent to (q, ˙q) being an integral curve of XL(see A.6), and means
that qmust satisfy the Euler-Lagrange equations
∂L
∂qi(q, ˙q)−d
dt∂L
∂˙qi(q, ˙q)= 0 (3.5)
for all t∈(0, T).
1For a regular (see Section 3.2.1) Lagrangian L, these objects exist and are unique.
27
Properties of Lagrangian flows In the following we summarize some well
known properties of Lagrangian flows, which play an important role in the discrete
formulation as well.
First of all, Lagrangian flows are symplectic, that is the Lagrangian symplectic
form ΩL=dΘL, given in coordinates by
ΩL(q, ˙q) = ∂2L
∂qi∂˙qjdqi∧dqj+∂2L
∂˙qi∂˙qjd˙qi∧dqj
is preserved under the Lagrangian flow, as
(FT
L)∗(ΩL) = ΩL,
where (FT
L)∗(ΩL) is the pullback of the Lagrangian symplectic form ΩLby the
map FT
L(see A.9).
Another key property of Lagrangian flows is its behavior with respect to group
actions. Assume a Lie group G(see A.14) with Lie algebra g(see A.16) acts on
Qby the (left or right) action φ:G×Q→Q(see A.18). Consider the tangent
lift (see A.5) of this action to φTQ :G×TQ given by φTQ
g(vq) = T(φg)·vq, which
is
φTQ(g, (q, ˙q)) = φi(g, q),∂φi
∂qj(g, q) ˙qj.
For ξ∈gthe infinitesimal generators (see A.19) ξQ:Q→TQ and ξT Q :TQ →
T(TQ) are defined by
ξQ(q) = d
dg(φg(q)) ·ξ, ξTQ(vq) = d
dgφTQ
g(vq)·ξ,
and the Lagrangian momentum map JL:TQ →g∗(see A.24) is defined to be
JL(vq)·ξ= ΘL·ξTQ(vq).
If the Lagrangian is invariant under the lift of the action, that is we have L◦φTQ
g=
Lfor all g∈G(we also say, the group action is a symmetry of the Lagrangian), the
Lagrangian momentum map is preserved of the Lagrangian flow. More formally
it holds
Theorem 3.1.2 (Noether’s theorem) Consider a Lagrangian system L:TQ →
Rwhich is invariant under the lift of the (left or right) action φ:G×Q→Q,
that is L◦φTQ
g=Lfor all g∈G. Then the corresponding Lagrangian momentum
map JL:TQ →g∗is a conserved quantity of the flow, such that JL◦Ft
L=JL
for all times t.
28
Proofs for the symplecticity and the preservation of the momentum map of La-
grangian flows can be found for example in [113].
Examples 3.1.3 (Momentum maps) (i) Let Q=Rnand let G=Rnop-
erate on Qby translation, that is φ:G×Q→Qis given by φ(g, q) = g+q.
Then the momentum map (see A.22) can be calculated as J(ξ)(qj, pj) =
N
P
j=1
pj·ξ, that is J(qj, pj) =
N
P
j=1
pjis the total linear momentum.
(ii) For the Lie group of proper rotations G=SO(3) acting on the configuration
space Q=R3via φ(A, q) = A·q, the momentum map is the angular
momentum J(q, p) = q×p.
3.1.2 Discrete Lagrangian mechanics
Again we consider a configuration manifold Q, but now we define the discrete
state space to be Q×Q. That means that rather than taking a position qand
velocity ˙q, we now consider two positions q0and q1and a time step h∈R.
These positions should be thought of as being two points on a curve at time h
apart, such that q0≈q(0) and q1≈q(h). Q×Qcontains the same amount of
information as (is locally isomorphic to) TQ. A discrete Lagrangian is a function
Ld:Q×Q×R→R, which we think of as approximating the action integral along
the curve segment between q0and q1. In the following chapters we neglect the h
dependence except where it is important and consider the discrete Lagrangian as
a function Ld:Q×Q→R.
We construct the increasing sequence of times {tk=kh |k= 0, . . . , N} ⊂ R
from the time step h, and define the discrete path space to be
Cd(Q) = Cd({tk}N
k=0, Q) = {qd:{tk}N
k=0 →Q}.
We will identify a discrete trajectory qd∈ Cd(Q) with its image qd={qk}N
k=0,
where qk=qd(tk). The discrete action map Gd:Cd(Q)→Ralong this sequence
is calculated by summing the discrete Lagrangian on each adjacent pair and
defined by
Gd(qd) =
N−1
X
k=0
Ld(qk, qk+1).
As the discrete path space Cdis isomorphic to Q× · · · × Q(N+ 1 copies), it can
be given a smooth product manifold structure. The discrete action Gdinherits
the smoothness of the discrete Lagrangian Ld.
The tangent space TqdCd(Q) to Cd(Q) at qdis the set of maps vqd:{tk}N
k=0 →
TQ such that τq◦vqd=qd, which we will denote by vqd={(qk, vk)}N
k=0.
29
The discrete object corresponding to T(TQ) is the set (Q×Q)×(Q×Q).
With the projection operator πand the translation operator σdefined as
π: ((q0, q1),(q0
0, q0
1)) 7→ (q0, q1)
σ: ((q0, q1),(q0
0, q0
1)) 7→ (q0
0, q0
1),
the discrete second order submanifold of (Q×Q)×(Q×Q) is defined to be
¨
Qd={wd∈(Q×Q)×(Q×Q)|π1◦σ(wd) = π2◦π(wd)},
which has the same information content as (is locally isomorphic to) ¨
Q. Con-
cretely, the discrete second order submanifold is the set of pairs of the form
((q0, q1),(q1, q2)).
Now, analogously to the continuous setting, the discrete version of Hamilton’s
principle describes the dynamics of the discrete mechanical system determined
by a discrete Lagrangian Ldon Q×Q.
Theorem 3.1.4 (Discrete Hamilton’s principle) Given a Ckdiscrete Lagran-
gian Ld,k≥1, there exists a unique Ck−1mapping DDELLd:¨
Qd→T∗Qand
unique Ck−1one-forms Θ+
Ldand Θ−
Ldon Q×Q, such that, for all variations
δqd∈TqdC(Q)of qd, we have
dGd(qd)·δqd=
N−1
X
k=1
DDELLd((qk−1, qk),(qk, qk+1)) ·δqk
+Θ+
Ld(qN−1, qN)·(δqN−1, δqN)−Θ−
Ld(q0, q1)·(δq0, δq1).(3.6)
The mapping DDELLdis called the discrete Euler-Lagrange map and has coordi-
nate expression
DDELLd((qk−1, qk),(qk, qk+1)) = D2Ld(qk−1, qk) + D1Ld(qk, qk+1).
The one-forms are Θ+
Ldand Θ−
Ldare called the discrete Lagrangian one-forms and
in coordinates are
Θ+
Ld(q0, q1) = D2Ld(q0, q1)dq1=∂Ld
∂qi
1
dqi
1,(3.7a)
Θ−
Ld(q0, q1) = −D1Ld(q0, q1)dq0=−∂Ld
∂qi
0
dqi
0.(3.7b)
Proof: Computing the derivative of the discrete action map gives
dGd(qd)·δqd=
N−1
P
k=0
[D1Ld(qk, qk+1)·δqk+D2Ld(qk, qk+1)·δqk+1]
=
N−1
P
k=1
[D1Ld(qk, qk+1) + D2Ld(qk−1, qk)] ·δqk
+D1Ld(q0, q1)·δq0+D2Ld(qN−1, qN)·δqN
30
using a discrete integration by parts (rearrangement of the summation). Identify-
ing the terms with the discrete Euler-Lagrange map and the discrete Lagrangian
one-forms now gives the desired result.
Note that unlike the continuous case, in the discrete case there are two one-
forms that arise from the boundary terms. Observe, however, that dLd= Θ+
Ld−
Θ−
Ldand so using d2= 0 (see A.11) shows that
dΘ+
Ld=dΘ−
Ld.
Thus, as the continuous situation, there is only a single discrete two-form, which
is important for symplecticity as reflected below.
Discrete Lagrangian evolution operator and mappings
The discrete Lagrangian evolution operator XLdplays the same role as the con-
tinuous Lagrangian vector field, and is defined to be the map XLd:Q×Q→
(Q×Q)×(Q×Q) satisfying π◦XLd=id and
DDELLd◦XLd= 0.
The discrete object corresponding to the Lagrangian flow is the discrete Lagran-
gian map FLd:Q×Q→Q×Qdefined by FLd=σ◦XLd. Since XLdis of second
order which corresponds to the requirement that XLd(Q×Q)⊂¨
Qd, it has the
form XLd: (q0, q1)7→ (q0, q1, q1, q2), and so the corresponding Lagrangian map
will be FLd: (q0, q1)7→ (q1, q2).2
A discrete path qd∈ Cd(Q) is said to be a solution of the discrete Euler-
Lagrange equations if the first term on the right-hand side of (3.6) vanishes for all
variations δqd∈TqdCd(Q). This means that the points {qk}satisfy FLd(qk−1, qk) =
(qk, qk+1) or, equivalently, that they satisfy the discrete Euler-Lagrange equations
D2Ld(qk−1, qk) + D1Ld(qk, qk+1) = 0,for all k= 1, . . . , N −1.(3.8)
Properties of discrete Lagrangian maps One can show that discrete La-
grangian maps inherit the properties we summarized for continuous Lagrangian
flows. That means the discrete Lagrangian symplectic form ΩLd=dΘ+
Ld=dΘ−
Ld,
with coordinate expression
ΩLd(q0, q1) = ∂2Ld
∂qi
0∂qj
1
dqi
0∧dqj
1
2For a regular discrete Lagrangian Ld(see Section 3.2.2), these objects are well-defined and
the discrete Lagrangian map is invertible.
31
is preserved under the discrete Lagrangian map as
(FLd)∗(ΩLd) = ΩLd.
We say that FLdis discretely symplectic.
There exists a discrete analogue of Noether’s theorem which states, that mo-
mentum maps of symmetries are constants of the motion. To see this, we in-
troduce the action lift to Q×Qby the product φQ×Q
q(q0, q1)=(φg(q0), φg(q1)),
which has an infinitesimal generator ξQ×Q:Q×Q→T(Q×Q) given by
ξQ×Q(q0, q1) = (ξQ(q0), ξQ(q1)),
where ξQ:Q→TQ is the infinitesimal generator of the action. The two discrete
Lagrangian momentum maps are
J+
Ld(q0, q1)·ξ= Θ+
Ld·ξQ×Q(q0, q1),
J−
Ld(q0, q1)·ξ= Θ−
Ld·ξQ×Q(q0, q1),
or alternatively written as
J+
Ld(q0, q1)·ξ=hD2Ld(q0, q1), ξQ(q1)i,
J−
Ld(q0, q1)·ξ=h−D1Ld(q0, q1), ξQ(q0)i.
J+
Ldand J−
Ldare equal in the case of a discrete Lagrangian that is invariant
under the lifted action, that is Ld◦φQ×Q
g=Ldholds for all g∈G. Then,
JLd:Q×Q→g∗is the unique single discrete Lagrangian momentum map.
Theorem 3.1.5 (Discrete Noether’s theorem) Consider a given discrete La-
grangian system Ld:Q×Q→Rwhich is invariant under the lift of the (left or
right) action φ:G×Q→Q. Then the corresponding discrete Lagrangian mo-
mentum map JLd:Q×Q→g∗is a conserved quantity of the discrete Lagrangian
map FLd:Q×Q→Q×Q, such that JLd◦FLd=JLd.
In [113] proofs for the discrete symplecticity and the preservation of discrete
momentum maps by discrete Lagrangian maps are presented.
3.2 Hamiltonian mechanics
3.2.1 Basic definitions and concepts
Consider an n-dimensional configuration manifold Q, and define the phase space
to be the cotangent bundle T∗Q(see A.7). The Hamiltonian is a function H:
32
T∗Q→R. We will take local coordinates on T∗Qto be (q, p) with q= (q1, . . . , qn)
and p= (p1, . . . , pn).
Define the canonical one-form Θ on T∗Qby
Θ(pq)·upq=hpq, TπQ·upqi,(3.9)
where pq∈T∗Q,upq∈Tpq(T∗Q), πQ:T∗Q→Qis the canonical projection,
TπQ:T(T∗Q)→TQ is the tangent map of πQand h·,·i denotes the natural
pairing between vectors and covectors. In coordinates, we have Θ(q, p) = pidqi.
The canonical two-form Ω on T∗Qis defined to be
Ω = −dΘ,
which has the coordinate expression Ω(q, p) = dqi∧dpi.
Given now a Hamiltonian H, define the corresponding Hamiltonian vector
field XHto be the unique vector field on T∗Qsatisfying
iXHΩ = dH, (3.10)
where idenotes the interior product (see A.10). Writing XH= (Xq, Xp) in
coordinates, we see that the above expression is
−Xpidqi+Xqidpi=∂H
∂qidqi+∂H
∂pi
dpi,
which gives the familiar Hamilton’s equations for the components of XH, namely
Xqi(q, p) = ∂H
∂pi
(q, p),(3.11a)
Xpi(q, p) = −∂H
∂qi(q, p).(3.11b)
The Hamiltonian flow FH:T∗Q×R→T∗Qis the flow of the Hamiltonian vector
field XH. Note that, unlike the Lagrangian situation, the Hamiltonian vector field
XHand the flow map FHare always well-defined for any Hamiltonian.
For any fixed t∈R, the flow map Ft
H:T∗Q→T∗Qis symplectic, that is
(Ft
H)∗Ω = Ω holds.
Legendre transform
To relate Lagrangian mechanics to Hamiltonian mechanics we define the Legendre
transform or fiber derivative FL:TQ →T∗Qby
FL(vq)·wq=d
d=0
L(vq+wq),
33
where vq, wq∈TqQ, and which has coordinate form
FL: (q, ˙q)7→ (q, p) = q, ∂L
∂˙q(q, ˙q).
If the fiber derivative of Lis a local isomorphism then we say that Lis regular, and
if it is a global isomorphism then Lis said to be hyperregular. We will generally
assume that we are working with hyperregular Lagrangians.
The fiber derivative of a Hamiltonian is the map FH:T∗Q→TQ defined by
αq·FH(βq) = d
d=0
H(βq+αq),
where αq, βq∈T∗
qQ, and which in coordinates is
FH: (q, p)7→ (q, ˙q) = q, ∂H
∂p (q, p).
Similarly to the situation for Lagrangians, we say that His regular if FHis a
local isomorphism, and that His hyperregular if FHis a global isomorphism.
The canonical one- and two-forms and the Hamiltonian momentum maps
(see A.23) are related to the Lagrangian one- and two-forms and the Lagrangian
momentum maps by pullback under the fiber derivative, such that
ΘL= (FL)∗Θ,ΩL= (FL)∗Ω,and JL= (FL)∗JH.
If we additionally relate the Hamiltonian to the Lagrangian by
H(q, p) = FL(q, ˙q)·˙q−L(q, ˙q),(3.12)
where (q, p) and (q, ˙q) are related by the Legendre transform, then the Hamil-
tonian and Lagrangian vector fields and their associated flow maps will also be
related by pullback
XL= (FL)∗XH;Ft
L= (FL)−1◦Ft
H◦FL.
In coordinates this means that Hamilton’s equations (3.11) are equivalent to the
Euler-Lagrange equations (3.5). To see this, we compute the derivatives of (3.12)
34
to give
∂H
∂q (q, p) = p·∂˙q
∂q −∂L
∂q (q, ˙q)−∂L
∂˙q(q, ˙q)∂˙q
∂q
=−∂L
∂q (q, ˙q)
=−d
dt∂L
∂˙q(q, ˙q)
=−˙p,
∂H
∂p (q, p) = ˙q+p·∂˙q
∂p −∂L
∂˙q(q, ˙q)∂˙q
∂p
= ˙q,
where p=FL(q, ˙q) defines ˙qas a function of (q, p).
A calculation similar to the above also shows that if Lis hyperregular and H
is defined by (3.12), then His also hyperregular and the fiber derivatives satisfy
FH= (FL)−1. The converse statement also holds (see [111] for more details).
3.2.2 Discrete Hamiltonian mechanics
Discrete Legendre transforms
Just as the standard Legendre transform maps the Lagrangian state space TQ
to the Hamiltonian phase space T∗Q, we can define discrete Legendre transforms
or discrete fiber derivatives F+Ld,F−Ld:Q×Q→T∗Q, which map the discrete
state space Q×Qto T∗Q. These are given by
F+Ld(q0, q1)·δq1=D2Ld(q0, q1)·δq1,
F−Ld(q0, q1)·δq0=−D1Ld(q0, q1)·δq0,
which can be written as
F+Ld: (q0, q1)7→ (q1, p1) = (q1, D2Ld(q0, q1)),
F−Ld: (q0, q1)7→ (q0, p0) = (q0,−D1Ld(q0, q1)).
We say that Ldis regular if both discrete fiber derivatives are local isomorphisms
(for nearby q0and q1). In general we assume working with regular discrete La-
grangians. If both discrete fiber derivatives are global isomorphisms we say that
Ldis hyperregular.
The canonical one- and two-forms and Hamiltonian momentum maps are re-
lated to the discrete Lagrangian forms and discrete momentum maps by pullback
by the discrete fiber derivatives, such that
Θ±
Ld= (F±Ld)∗Θ,ΩLd= (F±Ld)∗Ω,and J±
Ld= (F±Ld)∗JH.
35
Momentum matching
By introducing the notation
p+
k,k+1 =p+(qk, qk+1) = F+Ld(qk, qk+1),
p−
k,k+1 =p−(qk, qk+1) = F−Ld(qk, qk+1),
for the momentum at the two endpoints of each interval [k, k+1] the discrete fiber
derivatives permit a new interpretation of the discrete Euler-Lagrange equations
D2Ld(qk−1, qk) = −D1Ld(qk, qk+1),
which can be written as
F+Ld(qk−1, qk+1) = F−Ld(qk, qk+1),(3.13)
or simply
p+
k−1,k =p−
k,k+1.
That is, the discrete Euler-Lagrange equations are enforcing the condition that
the momentum at time kshould be the same when being evaluated from the
lower interval [k−1, k] or the upper interval [k, k + 1]. This means that along a
solution curve there is a unique momentum at each time k, which is denoted by
pk=p+
k−1,k =p−
k,k+1.
A discrete trajectory {qk}N
k=0 in Qcan thus also be regarded as either a trajectory
{(qk, qk+1)}N−1
k=0 in Q×Qor, equivalently, as a trajectory {(qk, pk)}N
k=0 in T∗Q.
Note that (3.13) can be written as
F+Ld=F−Ld◦FLd.(3.14)
Discrete Hamiltonian maps
Using the discrete fiber derivatives also enables us to push the discrete Lagrangian
map FLd:Q×Q→Q×Qforward to T∗Q. We define the discrete Hamiltonian
map ˜
FLd:T∗Q→T∗Qby ˜
FLd=F±Ld◦FLd◦(F±Ld)−1with coordinate expression
˜
FLd: (q0, p0)7→ (q1, p1), where
p0=−D1Ld(q0, q1),(3.15a)
p1=D2Ld(q0, q1).(3.15b)
As the discrete Lagrangian map preserves the discrete symplectic form and dis-
crete momentum maps on Q×Q, the discrete Hamiltonian map will preserve the
pushforwards of these structures. As we saw above, however, these are simply
the canonical symplectic form and canonical momentum maps on T∗Q, and so
the discrete Hamiltonian map is symplectic and momentum-preserving.
36
3.3 Forcing and control
Our aim is to optimally control Lagrangian and Hamiltonian systems. Thus,
in this section we extend the discrete variational framework to include external
forcing resulting from dissipation and friction or control forces and loading on
mechanical systems.
In contrast to [73] and [113], we consider forces depending on control param-
eters from the beginning. Therefore, we extend the definitions for Lagrangian
forces in [113] to Lagrangian control forces.
3.3.1 Forced Lagrangian systems
As defined in [113], a Lagrangian force is a fiber-preserving map fL:TQ →T∗Q
over the identity idQ(see A.25), which we write in coordinates as
fL: (q, ˙q)7→ (q, fL(q, ˙q)).
This kind of forces results for example from dissipation and friction or from
configuration- and velocity-dependent but time-independent loading on mechani-
cal systems. To define time-dependent control forces for Lagrangian systems, we
introduce a control manifold U⊂Rmand for a given interval I= [0, T] we define
the control path space to be
D(U) = D([0, T], U) = {u: [0, T]→U|u∈L∞},
with u(t)∈Ualso called the control parameter.Lp(Rm) denotes the usual
Lebesgue space of measurable functions x: [0, T]→Rmwith |x(·)|pintegrable,
equipped with its standard norm
kxkLp=
T
Z0
|x(t)|pdt
1
p
,
where |·|is the Euclidean norm. p=∞corresponds to the space of essentially
bounded, measurable functions equipped with the essential supremum norm.
With this notation we define a Lagrangian control force as a map fLC :TQ×U→
T∗Q, which is given in coordinates as
fLC : (q, ˙q, u)7→ (q, fLC(q, ˙q, u)).
We interpret a Lagrangian control force as a parameter-dependent Lagrangian
force, that is a parameter-dependent fiber-preserving map fL(u) : TQ →T∗Q
over the identity idQ, which we write in coordinates as
fL(u) : (q, ˙q)7→ (q, fL(u)(q, ˙q)).
37
Here and in the sequel, whenever we denote fLC(q, ˙q, u) as one-form on TQ, we
mean the parameter-dependent horizontal one-form fL(u)(q, ˙q) on TQ induced by
the parameter-dependent fiber-preserving map fL(u) (see A.26 and Prop. A.1).
Remark 3.3.1 Note that the definition of a Lagrangian control force also in-
cludes forces that are independent on the control parameter. Thus in the follow-
ing we restrict ourselves to a formulation with control forces which gives us the
opportunity to include friction or dissipative forces as well.
Given a control force, we modify Hamilton’s principle, seeking stationary points
of the action, to the Lagrange-d’Alembert principle, which seeks curves q∈ C(Q)
satisfying
δ
T
Z0
L(q(t),˙q(t)) dt+
T
Z0
fLC(q(t),˙q(t), u(t)) ·δq(t) dt= 0,(3.16)
where δrepresents variations vanishing at the endpoints. The second integral in
(3.16) is the virtual work acting on the mechanical system via the force fLC. Using
integration by parts shows that this is equivalent to the forced Euler-Lagrange
equations, which have the coordinate expression
∂L
∂q (q, ˙q)−d
dt∂L
∂˙q(q, ˙q)+fLC(q, ˙q, u) = 0.(3.17)
Note that these are the same as the standard Euler-Lagrange equations (3.5)
with the forcing term added and implicitly define the forced Lagrangian flow
FL(u) : TQ ×R→TQ.
3.3.2 Forced Hamiltonian systems
AHamiltonian control force is a map fHC :T∗Q×U→T∗Qidentified by a
parameter-dependent fiber preserving map fH(u) : T∗Q→T∗Qover the identity.
Given such a control force, we define the corresponding horizontal one-form (see
A.26 and Prop. A.1) f0
H(u) on T∗Qby
f0
H(u)(pq)·wpq=hfH(u)(pq), TπQ·wpqi,
where πQ:T∗Q→Qis the projection. This expression is reminiscent of definition
(3.9) of the canonical one-form Θ on T∗Q, and in coordinates it reads f0
H(u)(q, p)·
(δq, δp) = fHC(q, p, u)·δq, thus the one-form is clearly horizontal.
The forced Hamiltonian vector field XHis now defined by the equation
iXHΩ = dH−f0
H(u)
38
and in coordinates this gives the well-known forced Hamilton’s equations
Xq(q, p) = ∂H
∂p (q, p),(3.18a)
Xp(q, p, u) = −∂H
∂q (q, p) + fHC(q, p, u),(3.18b)
which are the same as the standard Hamilton’s equations (3.11) in coordinates
with the forcing term added to the momentum equation. This defines the forced
Hamiltonian flow FH(u) : T∗Q×R→T∗Qof the forced Hamiltonian vector field
XH= (Xq, Xp).
3.3.3 Legendre transform with forces
Given a Lagrangian L, we can take the standard Legendre transform FL:TQ →
T∗Qand relate Hamiltonian and Langrangian control forces by
fL(u) = fH(u)◦FL.
If we also have a Hamiltonian Hrelated to Lby the Legendre transform according
to (3.12), then the forced Euler-Lagrange equations and the forced Hamilton’s
equations are equivalent. That is, if XLand XHare the forced Lagrangian and
Hamiltonian vector fields, respectively, then (FL)∗(XH) = XL. To see this, we
compute
∂H
∂q (q, p) = p·∂˙q
∂q −∂L
∂q (q, ˙q)−∂L
∂˙q(q, ˙q)∂˙q
∂q
=−∂L
∂q (q, ˙q)
=−d
dt∂L
∂˙q(q, ˙q)+fLC(q, ˙q, u)
=−˙p+fH(u)◦FL(q, ˙q),
=−˙p+fHC (p, q, u),
∂H
∂p (q, p) = ˙q+p·∂˙q
∂p −∂L
∂˙q(q, ˙q)∂˙q
∂p
= ˙q,
where p=FL(q, ˙q) defines ˙qas a function of (q, p).
39
3.3.4 Noether’s theorem with forcing
We now consider the effect of forcing on the evolution of momentum maps that
arise from symmetries of the Lagrangian. In [113] it is shown that the evolution
of the momentum map from time 0 to time Tis given by the relation
(JL◦FT
L(u))(q(0),˙q(0)) −JL(q(0),˙q(0))·ξ=
T
Z0
fLC(q(t),˙q(t), u(t))·ξQ(q(t)) dt,
(3.19)
where ξ∈gand ξQ:Q→TQ is the infinitesimal generator (see A.19). Equation
(3.19) shows, that forcing will generally alter the momentum map. However, in
the special case that the forcing is orthogonal to the group action, the above
relation shows that Noether’s theorem will still hold.
Theorem 3.3.2 (Forced Noether’s theorem) Consider a Lagrangian system
L:TQ →Rwith control forcing fLC :TQ ×U→T∗Qand a symmetry action
φ:G×Q→Qsuch that hfLC(q, ˙q, u), ξQ(q)i= 0 for all (q, ˙q)∈TQ,u∈Uand
all ξ∈g. Then the Lagrangian momentum map JL:TQ →g∗will be preserved
by the flow, such that JL◦Ft
L(u) = JLfor all t.
3.3.5 Discrete variational mechanics with control forces
To complete the discrete setting for forced mechanical systems, in this section
we present a discrete formulation of the control forces introduced in the previous
section. Since the control parameter function u:I→Uhas no geometric
interpretation, we have to find an appropriate discrete formulation to identify a
discrete structure for the Lagrangian control force.
Discrete Lagrangian control forces
In Section 3.1.2 we replaced the path space by a discrete path space defined on
the time grid {tk}N
k=0 as qd(tk) = qk. For the replacement of the control path space
by a discrete one we introduce a new time grid ∆˜
t. This time grid is generated
via an increasing sequence of intermediate control points c={cl|0≤cl≤1, l =
1, . . . , s}as ∆˜
t={tkl |k∈ {0, . . . , N −1}, l ∈ {1, . . . , s}}, where tkl =tk+clh.
With this notation the discrete control path space is defined to be
Dd(U) = Dd(∆˜
t, U) = {ud: ∆˜
t→U}.
We define the intermediate control samples ukon [tk, tk+1] as uk= (uk1, . . . , uks)∈
Usto be the values of the control parameters guiding the system from qk=qd(tk)
to qk+1 =qd(tk+1), where ukl =ud(tkl) for l∈ {1, . . . , s}.
40
The set of the discrete controls Uscan be viewed as a finite dimensional
subspace of the control path space D([0, h], U).
Examples 3.3.3 a) For s= 1 and c= 0.5 we obtain just one intermediate
control parameter per interval that approximates the control parameter on
[tk, tk+1] as uk=ud(tk+h/2) ∈U.
b) For s= 2 and c={0,1}the control parameter is approximated via two
discrete control parameters at the left and right boundary of each interval,
respectively, as uk= (ud(tk), ud(tk+1)) ∈U×U.
With this interpretation of the discrete control path space, we take two discrete
Lagrangian control forces f+
Cd, f−
Cd:Q×Q×Us→T∗Q, given in coordinates as
f+
Cd(qk, qk+1, uk) = (qk+1, f+
Cd(qk, qk+1, uk)),(3.20)
f−
Cd(qk, qk+1, uk) = (qk, f−
Cd(qk, qk+1, uk)),(3.21)
also called left and right discrete forces.3Analogously to the continuous case, we
interpret the two discrete Lagrangian control forces as two parameter-dependent
discrete fiber-preserving Lagrangian forces f±
d(uk) : Q×Q→T∗Qin the sense
that πQ◦f±
d=π±
Qwith f±
d(uk)(qk, qk+1) = f±
Cd(qk, qk+1, uk) and with the pro-
jection operators π+
Q:Q×Q→Q, (qk, qk+1)7→ qk+1 and π−
Q:Q×Q→
Q, (qk, qk+1)7→ qk. We combine the two discrete control forces to give a sin-
gle one-form fd(uk) : Q×Q→T∗(Q×Q) defined by
fd(uk)(qk, qk+1)·(δqk, δqk+1) = f+
d(uk)(qk, qk+1)·δqk+1 +f−
d(uk)(qk, qk+1)·δqk,
(3.22)
and we define
fCd(qk, qk+1, uk) = fd(uk)(qk, qk+1).
Remark 3.3.4 To simplify the notation we denote the left and right discrete
forces by f±
k:= f±
Cd(qk, qk+1, uk), respectively, and the combination of both by
fk:= fCd(qk, qk+1, uk).
Corresponding to Figure 3.1 we interpret the left discrete force f+
k−1as the force
resulting from the continuous control force acting during the time span [tk−1, tk]
on the configuration node qk. The right discrete force f−
kis the force acting on
qkresulting from the continuous control force during the time span [tk, tk+1].
3Observe, that the discrete control force is now dependent on the discrete control path.
41
qk-1
q
q
fk-1
+k
f+
k
f-
k
k
k+1
uu
f-
k-1
k-1
Figure 3.1: Left and right discrete forces.
Discrete Lagrange-d’Alembert principle
As with discrete Lagrangians, the discrete control forces also depend on the time
step h, which is important when relating discrete and continuous mechanics.
Given such forces, we modify the discrete Hamilton’s principle, following [73], to
the discrete Lagrange-d’Alembert principle, which seeks discrete curves {qk}N
k=0
that satisfy
δ
N−1
X
k=0
Ld(qk, qk+1) +
N−1
X
k=0 f−
Cd(qk, qk+1, uk)·δqk+f+
Cd(qk, qk+1, uk)·δqk+1= 0
(3.23)
for all variations {δqk}N
k=0 vanishing at the endpoints. This is equivalent to the
forced discrete Euler-Lagrange equations
D2Ld(qk−1, qk)+D1Ld(qk, qk+1)+f+
Cd(qk−1, qk, uk−1)+f−
Cd(qk, qk+1, uk) = 0,(3.24)
which are the same as the standard discrete Euler-Lagrange equations (3.8) with
the discrete forces added. These implicitly define the forced discrete Lagrangian
map FLd(uk−1, uk) : Q×Q→Q×Q.
3.3.6 Discrete Legendre transforms with forces
Although in the continuous case we used the standard Legendre transform for
systems with forcing, in the discrete case it is necessary to take the forced discrete
Legendre transforms to be
Ff+Ld(u0) : (q0, q1)7→ (q1, p1) = (q1, D2Ld(q0, q1) + f+
Cd(q0, q1, u0)),(3.25a)
Ff−Ld(u0) : (q0, q1)7→ (q0, p0) = (q0,−D1Ld(q0, q1)−f−
Cd(q0, q1, u0)).(3.25b)
42
Using these definitions and the forced discrete Euler-Lagrange equations (3.24),
we can see that the corresponding forced discrete Hamiltonian map ˜
FLd(u0) =
Ff±Ld(u1)◦FLd(u0, u1)◦(Ff±Ld)−1(u0) is given by the map ˜
FLd(u0) : (q0, p0)7→
(q1, p1), where
p0=−D1Ld(q0, q1)−f−
Cd(q0, q1, u0),(3.26a)
p1=D2Ld(q0, q1) + f+
Cd(q0, q1, u0),(3.26b)
which is the same as the standard discrete Hamiltonian map (3.15) with the
discrete forces added.
Figure 3.2 shows that the following two definitions of the forced discrete
Hamiltonian map
˜
FLd(u0) = Ff±Ld(u1)◦FLd(u0, u1)◦(Ff±Ld)−1(u0),(3.27a)
˜
FLd(u0) = Ff+Ld(u0)◦(Ff−Ld)−1(u0),(3.27b)
are equivalent with coordinate expression (3.26). Thus from expression (3.27b)
and Figure 3.2, it becomes clear, that the forced discrete Hamiltonian map that
maps (q0, p0) to (q1, p1), just depends on u0.
(q0, q1) (q1, q2)
(q0, p0) (q1, p1) (q2, p2)
FLd(u0, u1)
Ff−Ld(u0)Ff+Ld(u0)Ff−Ld(u1)Ff+Ld(u1)
˜
FLd(u0)˜
FLd(u1)
Figure 3.2: Correspondence between the forced discrete Lagrangian and the
forced discrete Hamiltonian map.
3.3.7 Discrete Noether’s theorem with forcing
As for the unforced case, we can formulate a discrete version of the forced
Noether’s theorem (for the derivation see for example [113]). Therefor, the dis-
43
crete momentum map in presence of forcing is defined as
Jf+
Ld(q0, q1)·ξ=hFf+Ld(u0)(q0, q1), ξQ(q1)i,
Jf−
Ld(q0, q1)·ξ=hFf−Ld(u0)(q0, q1), ξQ(q0).
The evolution of the discrete momentum map is described by
hJf+
Ld◦FN−1
Ld(ud)−Jf−
Ldi(q0, q1)·ξ=
N−1
X
k=0
fCd(qk, qk+1, uk)·ξQ×Q(qk, qk+1).(3.28)
Again, in the case that the forcing is orthogonal to the group action we have the
unique momentum map Jf
Ld:Q×Q→g∗and it holds:
Theorem 3.3.5 (Forced discrete Noether’s theorem) Consider a discrete
Lagrangian system Ld:Q×Q→Rwith discrete control forces f+
Cd, f−
Cd:Q×Q×
Us→T∗Qand a symmetry action φ:G×Q→Qsuch that hfCd, ξQ×Qi= 0 for
all ξ∈g. Then the discrete Lagrangian momentum map Jf
Ld:Q×Q→g∗will
be preserved by the discrete Lagrangian evolution map, such that Jf
Ld◦FLd=Jf
Ld.
44
Chapter 4
Discrete mechanics and optimal
control (DMOC)
This chapter comprises the main part of this work: Having introduced the basics
of optimal control and variational mechanics in Chapters 2 and 3, we combine
both concepts to build up a setting for the optimal control of a continuous and a
discrete mechanical system in Sections 4.1 and 4.2, respectively. Furthermore, in
Section 4.3, we compare both problems and show that the discrete formulation
results in a direct solution method as it has been introduced in Chapter 2. The
remainder of this chapter aims at showing the convergence of the discrete opti-
mal solution of the discrete optimal control system to the continuous solution of
the original optimal control problem. To this end, in Section 4.4, we first give
an appropriate formulation for the optimal control of high-order discretized me-
chanical systems. We show the equivalence of these problems to those resulting
from Runge-Kutta discretizations of the corresponding Hamiltonian system. This
equivalence allows us to construct and compare the adjoint systems of the con-
tinuous and the discrete optimal control problem in Section 4.5. In this context,
one of our main results is stated that concerns the order of approximation of the
adjoint system of the discrete to the continuous optimal control problem. To-
gether with existing convergence results of optimal control problems discretized
via Runge-Kutta methods (see for example [41, 53]) this leads directly to the
convergence formulation of DMOC in Section 4.6.
4.1 Optimal control of a mechanical system
On the configuration space Qwe consider a mechanical system described by a
regular Lagrangian L:TQ →R. Additionally, there acts a Lagrangian control
force on the system defined by the map fLC :TQ ×U→T∗Qwith fLC :
45
(q, ˙q, u)7→ (q, fLC(q, ˙q, u)) and u:I→U, t 7→ u(t) the time-dependent control
parameter. Note, that the Lagrangian control force can imply both, dissipative
forces within the mechanical system and external control forces resulting from
actuators steering the system.
4.1.1 Lagrangian optimal control problem
Consider now the following optimal control problem: During a time interval I=
[0, T] the mechanical system described by the Lagrangian Lis to be moved on a
curve q(t)∈Qfrom an initial state (q(0),˙q(0)) = (q0,˙q0)∈TQ to a final state.
The motion is influenced via a Lagrangian control force fLC with chosen control
parameter u(t) such that a given objective functional
J(q, ˙q, u) = ZT
0
C(q(t),˙q(t), u(t)) dt+ Φ(q(T),˙q(T)) (4.1)
is minimized with J:TQ ×U→R, the cost function C:TQ ×U→Rand
the final condition (Mayer term) Φ : TQ →Rcontinuously differentiable. In this
way, the final state is given by the final time constraint r(q(T),˙q(T), qT,˙qT)=0
with r:TQ ×TQ →Rnr, where (qT,˙qT)∈TQ is a fixed value for the desired
final state.
At the same time, the motion q(t) of the system is to satisfy the Lagrange-
d’Alembert principle, which requires that
δZT
0
L(q(t),˙q(t)) dt+ZT
0
fLC(q(t),˙q(t), u(t)) ·δq(t) dt= 0 (4.2)
for all variations δq with δq(0) = δq(T) = 0. Additionally, there are constraints
on states and control parameters to be fulfilled given by the path constraints
h:TQ ×U→Rnh, h(q(t),˙q(t), u(t)) ≥0.
The optimal control problem for a Lagrangian system can now be formulated
as follows
Problem 4.1.1 (Lagrangian optimal control problem (LOCP))
min
q(·),˙q(·),u(·),(T)J(q, ˙q, u) = ZT
0
C(q(t),˙q(t), u(t)) dt+ Φ(q(T),˙q(T)) (4.3a)
subject to
δZT
0
L(q(t),˙q(t)) dt+ZT
0
fLC(q(t),˙q(t), u(t)) ·δq(t) dt= 0,(4.3b)
46
q(0) = q0,˙q(0) = ˙q0,(4.3c)
h(q(t),˙q(t), u(t)) ≥0,(4.3d)
r(q(T),˙q(T), qT,˙qT) = 0.(4.3e)
As in Chapter 2 the final time Tmay either be fixed, or appear as degree of
freedom in the optimization problem.
Definition 4.1.2 A triple (q(·),˙q(·), u(·)) is admissible (or feasible), if the con-
straints (4.3b)-(4.3e) are fulfilled. The set consisting of all admissible (feasible)
triples is the admissible (feasible) set of Problem 2.1.1. An admissible (feasible)
triple (q∗,˙q∗, u∗) is an optimal solution of Problem 4.1.1, if
J(q∗,˙q∗, u∗)≤J(q, ˙q, u)
for all admissible (feasible) triples (q, ˙q, u). The admissible (feasible) triple
(q∗,˙q∗, u∗) is a local optimal solution, if there exists a neighborhood Bδ((q∗,˙q∗, u∗)),
δ > 0, such that
J(q∗,˙q∗, u∗)≤J(q, ˙q, u)
for all admissible (feasible) triples (q, ˙q, u)∈Bδ((q∗,˙q∗, u∗)). The function q∗(t) is
called (locally) optimal trajectory, and the function u∗(t) is the (locally) optimal
control.
4.1.2 Hamiltonian optimal control problem
For describing the optimal control problem within the Hamiltonian framework
we use the Hamiltonian formulation for the system dynamics. This is equivalent
to the Lagrangian formulation as we have seen in Section 3.2.
In addition, we need equivalent formulations for the objective functional, the
boundary conditions and the path constraints. Altogether, these conditions de-
termine the admissible set for configuration and velocity in the tangent space. To
this end we are searching for constraints on the configuration and the momentum
level that determine the corresponding admissible set in the cotangent space.
Assume, the feasible set determined via a constraint g:TQ →Rngon TQ is
given by
R={(q, ˙q)∈TQ |g(q, ˙q)≥0}.
By defining
˜g:T∗Q→Rng,(q, p)7→ g◦(FL)−1(q, p)
using the standard Legendre transform FL:TQ →T∗Q
FL: (q, ˙q)7→ (q, p) = (q, D2L(q, ˙q)),
47
we correspondingly get the feasible set in the cotangent space as
˜
R={(q, p)∈T∗Q|˜g(q, p)≥0}.
We apply the Legendre transform to the final condition Φ, the initial velocity
constraint, the boundary conditions, and the path constraints. Then, the optimal
control problem in the Hamiltonian formulation reads as follows:
Problem 4.1.3 (Hamiltonian optimal control problem (HOCP))
min
q(·),p(·),u(·),(T)J(q, p, u) = ZT
0
˜
C(q(t), p(t), u(t)) dt+˜
Φ(q(T), p(T)) (4.4a)
subject to
˙q(t) = ∇pH(q(t), p(t)), q(0) = q0,(4.4b)
˙p(t) = −∇qH(q(t), p(t)) + fHC(q(t), p(t), u(t)), p(0) = p0,(4.4c)
0≤˜
h(q(t), p(t), u(t)),(4.4d)
0 = ˜r(q(T), p(T), qT, pT).(4.4e)
with ˜
C=C◦((FL)−1(q, p), u),˜
Φ = Φ ◦(FL)−1(q, p),˜
h=h◦((FL)−1(q, p), u),
˜r=r◦(FL)−1(q, p),(FL)−1(qT, pT)with (qT, pT) = FL(qT,˙qT) and p(0) =
D2L(q(0),˙q(0)), p0=D2L(q0,˙q0).
Remark 4.1.4 By defining x(t) = (q(t), p(t)), this optimal control formulation
is equivalent to the formulation in (2.6) introduced in Section 2.1.1.
4.1.3 Transformation to Mayer form
Within the previous sections we introduced optimal control problems in the Bolza
form, where the objective functional consists of the final point constraint and a
cost functional of integral form. For error analysis (see Section 4.5), it is useful
to transform the optimal control problem into the Mayer form, that means the
objective functional consists of the final point constraint only, thus, reduces to
an objective function.
To construct an optimal control problem of Mayer type we define for the
Lagrangian optimal control problem a new state variable as
z(t) := Zt
0
C(q(t),˙q(t), u(t)) dt, 0≤t≤T.
Analogously, we define
y(t) := Zt
0
˜
C(q(t), p(t), u(t)) dt, 0≤t≤T,
48
for the Hamiltonian optimal control problem. By extension of the state space
from TQ to TQ ×R, and from T∗Qto T∗Q×R, respectively, the new objective
function of Mayer type reads as
J(q, ˙q, z) = z(T) + Φ(q(T),˙q(T))
and
J(q, p, y) = y(T) + ˜
Φ(q(T), p(T)),
respectively. Additionally, we obtain an extended set of equations describing the
extended dynamical system, namely an ordinary differential equation with an
initial value describing the evolution of the new variable as
˙z(t) = C(q(t),˙q(t), u(t)), z(0) = 0,(4.5a)
and
˙y(t) = ˜
C(q(t), p(t), u(t)), y(0) = 0,(4.5b)
respectively.
4.2 Optimal control of a discrete mechanical sys-
tem
For the numerical solution we need a discretized version of Problem 4.1.1. To this
end we formulate an optimal control problem for the discrete mechanical system
described by discrete variational mechanics introduced in Chapter 3. In Section
4.3 we show how the optimal control problem for the continuous and the discrete
mechanical system are related.
To obtain a discrete formulation, we replace each expression in (4.3) by its
discrete counterpart in terms of discrete variational mechanics. As described in
Chapter 3, we replace the state space TQ of the system by Q×Qand a path
q: [0, T]→Qby a discrete path qd:{0, h, 2h, . . . , Nh =T} → Q,N∈N, with
qk=qd(kh). Analogously, the continuous control path u: [0, T]→Uis replaced
by a discrete control path ud: ∆˜
t→U(writing uk={ud(kh +clh)}s
l=1 ∈Usfor
cl∈[0,1], l = 1, . . . , s).
Discrete Lagrange-d’Alembert principle Based on this discretization, the
action integral in (4.2) is approximated on a time slice [kh, (k+1)h] by the discrete
Lagrangian Ld:Q×Q→Rdefined in Section 3.1.2,
Ld(qk, qk+1)≈Z(k+1)h
kh
L(q(t),˙q(t)) dt,
49
and likewise the virtual work by the left and right discrete forces, defined in
Section 3.3.5, as
f−
k·δqk+f+
k·δqk+1 ≈Z(k+1)h
kh
fLC(q(t),˙q(t), u(t)) ·δq(t) dt,
where f−
k, f+
k∈T∗Q.
As introduced in Section 3.1.2 the discrete version of the Lagrange-d’Alembert
principle (4.2) requires one to find discrete paths {qk}N
k=0 such that for all varia-
tions {δqk}N
k=0 with δq0=δqN= 0, one has
δ
N−1
X
k=0
Ld(qk, qk+1) +
N−1
X
k=0 f−
k·δqk+f+
k·δqk+1= 0.(4.6)
or, equivalently, the forced discrete Euler-Lagrange equations
D2Ld(qk−1, qk) + D1Ld(qk, qk+1) + f+
k−1+f−
k= 0,(4.7)
where k= 1, . . . , N −1.
Boundary conditions In the next step, we need to incorporate the boundary
conditions q(0) = q0,˙q(0) = ˙q0and r(q(T),˙q(T), qT,˙qT) = 0 into the discrete
description. Those on the configuration level can be used as constraints in a
straightforward way as q0=q0. However, since in the present formulation veloci-
ties are approximated in a time interval [tk, tk+1] (as opposed to an approximation
at the time nodes), the velocity conditions have to be transformed to conditions
on the conjugate momenta. These are defined at each and every time node using
the discrete Legendre transform. The presence of forces at the time nodes has to
be incorporated into that transformation leading to the forced discrete Legendre
transforms Ff−Ldand Ff+Lddefined in (3.25). Using the standard Legendre
transform FL:TQ →T∗Q, (q, ˙q)7→ (q, p)=(q, D2L(q, ˙q)) leads to the discrete
initial constraint on the conjugate momentum
D2L(q0,˙q0) + D1Ld(q0, q1) + f−
Cd(q0, q1, u0) = 0.
As shown in the previous section, we can transform the boundary condition from
a formulation with configuration and velocity to a formulation with configuration
and conjugate momentum. Thus, instead of considering a discrete version of the
final time constraint ron TQ we use a discrete version of the final time constraint
˜ron T∗Q. We define the discrete boundary condition on the configuration level
to be
rd:Q×Q×Us×TQ →Rnr,
50
rd(qN−1, qN, uN−1, qT,˙qT) = ˜rFf+Ld(qN−1, qN, uN−1),FL(qT,˙qT),
with (qN, pN) = Ff+Ld(qN−1, qN, uN−1) and (qT, pT) = FL(qT,˙qT), that is pN=
D2Ld(qN−1, qN) + f+
Cd(qN−1, qN, uN−1) and pT=D2L(qT,˙qT).
Remark 4.2.1 For the simple final velocity constraint r(q(T),˙q(T), qT,˙qT) =
˙q(T)−˙qT, we obtain for the transformed condition on the momentum level
˜r(q(T), p(T), qT, pT) = p(T)−pTthe discrete constraint
−D2L(qT,˙qT) + D2Ld(qN−1, qN) + f+
Cd(qN−1, qN, uN−1) = 0.
Discrete path constraints Opposed to the final time constraint we appro-
ximate the path constraint in (4.3d) on each time interval [tk, tk+1] rather than
at each time node. Thus, we maintain the formulation on the velocity level and
replace the continuous path constraint via a discrete path constraint defined by
hd:Q×Q×Us→Rsnh,(qk, qk+1, uk)7→ hd(qk, qk+1, uk), k = 0, . . . , N −1,
where hd(qk, qk+1, uk)≥0∈Rsnh, k = 0, . . . , N−1, replaces h(q(t),˙q(t), u(t)) ≥0
on snodes within the interval [tk, tk+1].
Discrete objective function Similar to the Lagrangian we approximate the
objective functional in (4.1) on the time slice [kh, (k+ 1)h] by
Cd(qk, qk+1, uk)≈Z(k+1)h
kh
C(q(t),˙q(t), u(t)) dt.
Analogously to the final time constraint, we approximate the final condition via
a discrete version Φd:Q×Q×Us→Ryielding the discrete objective function
Jd(qd, ud) =
N−1
X
k=0
Cd(qk, qk+1, uk)+Φd(qN−1, qN, uN−1).
4.2.1 Discrete Optimal Control Problem
To summarize, after performing the above discretization steps, one is faced with
the following discrete optimal control problem.
Problem 4.2.2 (Discrete Lagrangian optimal control problem)
min
qd,ud,(h)Jd(qd, ud) =
N−1
X
k=0
Cd(qk, qk+1, uk)+Φd(qN−1, qN, uN−1) (4.8a)
51
subject to
δ
N−1
X
k=0
Ld(qk, qk+1) +
N−1
X
k=0 f−
k·δqk+f+
k·δqk+1= 0,(4.8b)
q0=q0, D2L(q0,˙q0) + D1Ld(q0, q1) + f−
0= 0,(4.8c)
hd(qk, qk+1, uk)≥0k= 0, . . . , N −1,(4.8d)
rd(qN−1, qN, uN−1, qT,˙qT) = 0.(4.8e)
This problem is equivalent to
min
qd,ud,(h)Jd(qd, ud) =
N−1
X
k=0
Cd(qk, qk+1, uk)+Φd(qN−1, qN, uN−1) (4.9a)
subject to
q0=q0,(4.9b)
D2L(q0,˙q0) + D1Ld(q0, q1) + f−
0= 0,(4.9c)
D2Ld(qk−1, qk) + D1Ld(qk, qk+1) + f+
k−1+f−
k= 0, k = 1, . . . , N −1,(4.9d)
hd(qk, qk+1, uk)≥0, k = 0, . . . , N −1,(4.9e)
rd(qN−1, qN, uN−1, qT,˙qT) = 0.(4.9f)
Recall that the f±
kare dependent on uk∈Us. To incorporate a free final time
Tas in the continuous setting, the step size happears as a degree of freedom
within the optimization problem. However, in the following formulations and
considerations we restrict ourselves to the case of fixed final time Tand thus
fixed step size h.
4.2.2 Transformation to Mayer form
As in the continuous setting the discrete optimal control problem (4.9) can be
transformed into a problem in Mayer form, that is the objective function consists
of a final condition only. The transformation is performed on the discrete level
to keep the Lagrangian structure of the original problem.
We introduce extra variables zd={zl}N
l=0 such that
z0= 0,
zl=
l−1
X
k=0
Cd(qk, qk+1, uk), l = 1, . . . , N,
and reformulate the discrete optimal control problem (4.9) into a problem of
Mayer type as
52
min
qd,ud,zd
˜
Φd(qd, ud, zN) = zN+ Φd(qN−1, qN, uN−1) (4.10a)
subject to
q0=q0,(4.10b)
z0= 0,(4.10c)
D2L(q0,˙q0) + D1Ld(q0, q1) + f−
0= 0,(4.10d)
D2Ld(qk−1, qk) + D1Ld(qk, qk+1) + f+
k−1+f−
k= 0, k = 1, . . . , N −1,(4.10e)
hd(qk, qk+1, uk)≥0, k = 0, . . . , N −1,(4.10f)
rd(qN−1, qN, uN−1, qT,˙qT) = 0,(4.10g)
zk+1 −zk−Cd(qk, qk+1, uk) = 0, k = 0, . . . , N −1.(4.10h)
Thus equations (4.10c) and (4.10h) provide the corresponding discretization for
the additional equation of motion (4.5a) resulting from the Mayer transformation
of the Lagrangian optimal control problem on the continuous level.
4.2.3 Fixed boundary conditions
Consider the special case of an optimal control problem with fixed initial and
final configuration and velocities and without path constraints as
min
q(·),˙q(·),u(·)J(q, ˙q, u) = ZT
0
C(q(t),˙q(t), u(t)) dt(4.11a)
subject to
δZT
0
L(q(t),˙q(t)) dt+ZT
0
fLC(q(t),˙q(t), u(t)) ·δq(t) dt= 0,(4.11b)
q(0) = q0,˙q(0) = ˙q0, q(T) = qT,˙q(T) = ˙qT.(4.11c)
A straightforward way to derive initial and final constraints for the conjugate
momenta rather than for the velocities from the variational principle directly is
stated in the following proposition:
Proposition 4.2.3 With (q0, p0) = FL(q0,˙q0)and (qT, pT) = FL(qN,˙qN)equa-
tions (4.11b) and (4.11c) are equivalent to the following principle with free initial
and final variation and with augmented Lagrangian
0 = δZT
0
L(q(t),˙q(t)) dt+p0(q(0) −q0)−pT(q(T)−qT)
+ZT
0
fLC(q(t),˙q(t), u(t)) ·δq(t) dt= 0.(4.12)
53
Proof: Variations of (4.11b) with respect to qand zero initial and final variation
δq(0) = δq(T) = 0 together with (4.11c) yield
d
dt
∂
∂˙qL(q(t),˙q(t)) −∂
∂qL(q(t),˙q(t)) = fLC(q(t),˙q(t), u(t)),(4.13a)
q(0) = q0,˙q(0) = ˙q0, q(T) = qT,˙q(T) = ˙qT.(4.13b)
On the other hand variations of (4.12) with respect to qand λ= (p0, pT) with
free initial and final variation lead to
d
dt
∂
∂˙qL(q(t),˙q(t)) −∂
∂qL(q(t),˙q(t)) = fLC(q(t),˙q(t), u(t)),(4.14a)
∂
∂˙qL(q(t),˙q(t))t=0
=pT,(4.14b)
∂
∂˙qL(q(t),˙q(t))t=T
=p0,(4.14c)
q(0) = q0, q(T) = qT.(4.14d)
The Legendre transform applied to the velocity boundary equations in (4.13b)
gives the corresponding momenta boundary equations (4.14b) and (4.14c) .
On the discrete level we derive the optimal control problem for fixed initial
and final configurations and velocities in an equivalent way. Thus, we consider
the discrete principle with discrete augmented Lagrangian
δ N−1
X
k=0
Ld(qk, qk+1) + p0(q0−q0)−pT(qN−qT)
!+
N−1
X
k=0 f−
k·δqk+f+
k·δqk+1= 0,
(4.15)
which, with free initial and final variation δq0and δqN, respectively, is equivalent
to
δ
N−1
X
k=0
Ld(qk, qk+1) +
N−1
X
k=0 f−
k·δqk+f+
k·δqk+1= 0,(4.16a)
q0=q0, p0+D1Ld(q0, q1) + f−
0= 0,(4.16b)
qN=qT,−pT+D2Ld(qN−1, qN) + f+
N−1= 0,(4.16c)
where the second equations in (4.16b) and (4.16c) are exactly the discrete initial
and final velocity constraints derived in Remark 4.2.1 with p0=D2L(q0,˙q0) and
pT=D2L(qT,˙qT).
54
Remark 4.2.4 This derivation of the discrete initial and final conditions di-
rectly gives the same formulation that we found before by first transforming the
boundary condition on the momentum level T∗Qand then formulating the cor-
responding discrete constraints on Q×Q×Us.
4.3 Correspondence between discrete and con-
tinuous optimal control problem
In this section, we relate the continuous and the discrete optimal control problem
to specify certain properties that the discrete problem inherits from the continu-
ous one.
First, along the lines of [113], we define expressions for the discrete mechanical
objects that exactly reflect the continuous mechanical objects. Based on the
exact discrete expressions, we determine the order of consistency concerning the
difference between the continuous and the discrete mechanical system. Finally, we
give an interpretation of the discrete optimal control problem as an approximation
of the continuous optimal control problem. We classify the discrete problem as
a standard solution strategy for Lagrangian optimal control problems as it is
presented in Chapter 2.
4.3.1 Exact discrete Lagrangian and forcing
Along the lines of [113] we define a particular choice of discrete Lagrangian and
discrete forces which give an exact correspondence between discrete and contin-
uous systems.
Given a regular Lagrangian L:TQ →Rand a Lagrangian control force
fLC :TQ×U→T∗Q, we define the exact discrete Lagrangian LE
d:Q×Q×R→R
and the exact discrete control forces fE+
Cd, fE−
Cd:Q×Q× D([0, h], U)×R→T∗Q
to be
LE
d(q0, q1, h) =
h
Z0
L(q(t),˙q(t)) dt, (4.17)
fE+
Cd(q0, q1, u|[0,h], h) =
h
Z0
fLC(q(t),˙q(t), u(t)) ·∂q(t)
∂q1
dt, (4.18)
fE−
Cd(q0, q1, u|[0,h], h) =
h
Z0
fLC(q(t),˙q(t), u(t)) ·∂q(t)
∂q0
dt, (4.19)
55
where q: [0, h]→Qis the solution of the forced Euler-Lagrange equations
(3.17) with control function u: [0, h]→Ufor Land fLC satisfying the boundary
conditions q(0) = q0and q(h) = q1. Observe, that the exact discrete control forces
now depend on the control path space, rather than on the control parameter as
for the continuous control forces. Consequently, the exact forced discrete Legendre
transforms are given by
Ff+LE
d(q0, q1, u|[0,h], h) = (q1, D2LE
d(q0, q1, h) + fE+
Cd(q0, q1, u|[0,h], h)),
Ff−LE
d(q0, q1, u|[0,h], h) = (q0,−D1LE
d(q0, q1, h)−fE−
Cd(q0, q1, u|[0,h], h)).
The next lemma is based on a result in [113] (Lemma 1.6.2) extended to the
presence of control forces and establishes a special relationship between the Le-
gendre transforms of a regular Lagrangian and its corresponding exact discrete
Lagrangian. This also proves that exact discrete Lagrangians are automatically
regular.
Lemma 4.3.1 A regular Lagrangian Land the corresponding exact discrete La-
grangian LE
dhave Legendre transforms related by
Ff+LE
d(q0, q1, u|[0,h], h) = FL(q0,1(h),˙q0,1(h)),
Ff−LE
d(q0, q1, u|[0,h], h) = FL(q0,1(0),˙q0,1(0)),
for sufficiently small hand close q0, q1∈Q.
Proof: Analogous to the proof in [113] (Lemma 1.6.2) for the unforced case we
begin with Ff−LE
dand compute
Ff−LE
d(q0, q1, u|[0,h], h) = −
h
Z0∂L
∂q ·∂q0,1
∂q0
+∂L
∂˙q·∂˙q0,1
∂q0dt−
h
Z0
fLC ·∂q0,1
∂q0
dt
=−
h
Z0∂L
∂q −d
dt
∂L
∂˙q−fLC·∂q0,1
∂q0
dt−∂L
∂˙q·∂q0,1
∂q0h
0
,
using integration by parts. Since q0,1(t) is a solution of the forced Euler-Lagrange
equations the first term is zero. With q0,1(0) = q0and q0,1(h) = q1for the second
term we get ∂q0,1
∂q0
(0) = id and ∂q0,1
∂q0
(h) = 0.
Substituting these into the above expression for Ff−LE
dnow gives
Ff−LE
d(q0, q1, u|[0,h], h) = ∂L
∂˙q(q0,1(0),˙q0,1(0)),
56
which is simply the definition of FL(q0,1(0),˙q0,1(0)).
The result for Ff+LE
dcan be established by a similar computation.
Combining this result with the diagram in Figure 3.2 gives the commutative
diagram shown in Figure 4.1 for the exact discrete Lagrangian and forces. The
diagram also clarifies the following observation, that was already proved in [113]
(Theorem 1.6.3) for unforced systems and can now be established for the forced
case as well: Consider the pushforward of both, the continuous Lagrangian and
forces and their exact discrete Lagrangian and discrete forces to T∗Q, yielding a
forced Hamiltonian system with Hamiltonian Hand a forced discrete Hamiltonian
map ˜
FLE
d(uk), respectively. Then, for a sufficiently small time step h∈R, the
forced Hamiltonian flow map equals the pushforward discrete Lagrangian map:
Fh
H(u|[0,h]) = ˜
FLE
d(u|[0,h]).
(q0, q1) (q1, q2)
(q0, p0) (q1, p1) (q2, p2)
(q0,˙q0) (q1,˙q1) (q2,˙q2)
FLE
d(u|[0,2h])
Ff−LE
d(u|[0,h])Ff+LE
d(u|[0,h])
Ff−LE
d(u|[h,2h])Ff+LE
d(u|[h,2h])
˜
FLE
d(u|[0,h]) = Fh
H(u|[0,h])˜
FLE
d(u|[h,2h]) = Fh
H(u|[h,2h])
FLFLFL
Fh
L(u|[0,h])Fh
L(u|[h,2h])
Figure 4.1: Correspondence between the exact discrete Lagrangian and forces
and the continuous forced Hamiltonian flow.
57
4.3.2 Order of consistency
In the previous section we observed that the exact discrete Lagrangian and forces
generate a forced discrete Hamiltonian map that exactly equals the forced Hamil-
tonian flow of the continuous system. Since we are interested in using discrete
mechanics to reformulate optimal control problems, we generally do not assume
that Ldand Lor Hare related by (4.17). Moreover, the exact discrete Lagrangian
and exact discrete forces are generally not computable. In this section we deter-
mine the error we obtain by using discrete approximations for the Lagrangian
and the control forces.
Forward error analysis is concerned with the difference between an exact tra-
jectory given by the Hamiltonian flow FH:T∗Q×R→T∗Qof a given Hamilto-
nian vector field XHand a discrete trajectory determined by a numerical method
F:T∗Q×R→T∗Qwhich approximates the Hamiltonian flow. The consistency
of the numerical method Fis described by its deviation from the flow FH: An
integrator Fof XHis said to be (consistent) of order rif there exist an open set
V⊂T∗Qand constants Cl>0 and hl>0 such that
kF(q, p, h)−FH(q, p, h)k ≤ Clhr+1
for all (q, p)∈Vand h≤hl. The expression on the left-hand side of this
inequality is known as the local error, and if a method is at least of order 1 then
it is said to be consistent. The integrator Fof XHis said to be convergent of
order rif there exist an open set V⊂T∗Qand constants Cg>0, hg>0 and
Tg>0 such that
k(F)N(q, p, h)−FH(q, p, T)k ≤ Cghr,
where h=T/N, for all (q, p)∈V,h≤hgand T≤Tg. The expression on
the left-hand side is the global error and indicates the total effect of all local
errors “propagated” by the flow. For one-step methods used as integrators for a
mechanical system, convergence follows from a local error bound on the method
and a Lipschitz bound on XH. The convergence of solution trajectories of a
discrete optimal control problem to solution trajectories of the continuous optimal
control problem needs some more analysis and is treated in Sections 4.5 and 4.6.
Rather than considering how closely the trajectory of Fmatches the exact
trajectory given by FH, we use the concepts variational error analysis as intro-
duced in [113]. In this context we consider how closely a discrete Lagrangian
matches the exact discrete Lagrangian given by the action. For forced systems,
we additionally take into account how closely the discrete forces match the exact
discrete forces. As was stated in the last section, if the discrete Lagrangian is
equal to the action and the discrete forces are given by (4.18) and (4.19), then
the corresponding forced discrete Hamiltonian map ˜
FLd(uk) will exactly equal the
58
forced flow FH(u). Usually, this is just an approximation, therefore we define the
local variational error as follows (see [113]).
Definition 4.3.2 (Order of consistency) A given discrete Lagrangian Ldis
of order r, if there exist an open subset Vv⊂TQ with compact closure and
constants Cv>0 and hv>0 such that
kLd(q(0), q(h), h)−LE
d(q(0), q(h), h)k ≤ Cvhr+1 (4.20)
for all solutions (q(t), u(t)) of the forced Euler-Lagrange equations with initial
condition (q0,˙q0)∈Vvand for all h≤hv. Analogously, a given discrete force f±
Cd
is of order r, if there exist an open subset Vw⊂TQ with compact closure and
constants Cw>0 and hw>0 such that
kf±
Cd(q(0), q(h),{u(cih)}s
i=1, h)−fE±
Cd(q(0), q(h), u|[0,h], h)k ≤ Cwhr+1 (4.21)
for all solutions (q(t), u(t)) of the forced Euler-Lagrange equations with initial
condition (q0,˙q0)∈Vwfor all h≤hw. The discrete Legendre transforms F+Ld
and F−Ldof a discrete Lagrangian Ldare of order rif there exist an open subset
Vf⊂TQ with compact closure and constants Cf>0 and hf>0 such that
kF±Ld(q(0), q(h),{u(cih)}s
i=1, h)−F±LE
d(q(0), q(h), u|[0,h], h)k ≤ Cfhr+1 (4.22)
for all solutions (q(t), u(t)) of the forced Euler-Lagrange equations with initial
condition (q0,˙q0)∈Vfand for all h≤hf.
To give a relationship between the orders of a discrete Lagrangian, discrete
forces, the forced discrete Legendre transforms, and their forced discrete Hamil-
tonian maps, we first have to introduce what we mean by equivalence of discrete
Lagrangians: L1
dis equivalent to L2
dif their discrete Hamiltonian maps are equal.
For the forced case, we say analogously, that the discrete pair (L1
d, f1
Cd) is equi-
valent to the discrete pair (L2
d, f2
Cd) if their forced discrete Hamiltonian maps are
equal, such that ˜
FL1
d=˜
FL2
d. With ˜
FL1
d=Ff+L1
d◦(Ff−L1
d)−1, it follows that if
(L1
d, f1
Cd) and (L2
d, f2
Cd) are equivalent, then their forced discrete Legendre trans-
forms are equal. Thus, equivalent pairs of discrete Lagrangians and control forces
generate the same integrators.
The following theorem is an extended version of the unforced case stated in
[113] (Theorem 2.3.1).
Theorem 4.3.3 Given a regular Lagrangian L, a Lagrangian control force fLC
and a corresponding Hamiltonian Hwith Hamiltonian control force fHC, the fol-
lowing statements are equivalent for a discrete Lagrangian Ldand the discrete
forces f±
Cd:
59
(i) the forced discrete Hamiltonian map for (Ld, f±
Cd)is of order r,
(ii) the forced discrete Legendre transforms of (Ld, f±
Cd)are of order r,
(iii) (Ld, f±
Cd)is equivalent to a pair of discrete Lagrangian and discrete forces,
both of order r.
Proof: Considering the proof of the unforced version in [113] (Theorem 2.3.1),
this proof is straightforward by taking into account the discrete Lagrangian con-
trol forces.
Variational order calculation
Given a discrete Lagrangian and discrete forces, their order can be calculated by
expanding the expressions for Ld(q(0), q(h), h) and f±
Cdin a Taylor series in hand
comparing these to the same expansions for the exact Lagrangian and the exact
forces, respectively. If the series agree up to rterms, then the discrete Lagrangian
is of order r. Analogously, the discrete forces are of order r, if for both expansions
the first rterms are identical.
The first few terms of the expansion of the exact discrete Lagrangian are given
by
LE
d(q(0), q(h), h) = hL(q0,˙q0) + 1
2h2∂L
∂q (q0,˙q0)·˙q0+∂L
∂˙q(q0,˙q0)·¨q0+O(h3),
(4.23)
where q0=q(0),˙q0= ˙q(0) and so forth. Higher derivatives of q(t) are determined
by the Euler-Lagrange equations. In the same way we evaluate the first few terms
of the expansion of the exact discrete control forces and obtain
fE−
Cd(q(0), q(h), u|[0,h], h) = hfLC(q0,˙q0, u0) + 1
2h2fLC(q0,˙q0, u0)·∂˙q0
∂q
+∂fLC
∂q (q0,˙q0, u0)·˙q0+∂fLC
∂˙q(q0,˙q0, u0)·¨q0+∂fLC
∂u (q0,˙q0, u0)·˙u0
+O(h3),(4.24)
with q0=q(0), u0=u(0),˙q0= ˙q(0),˙u0= ˙u(0) and so forth and analogously,
fE+
Cd(q(0), q(h), u|[0,h], h) = hfLC(qh,˙qh, uh)−1
2h2fLC(qh,˙qh, uh)·∂˙qh
∂q
+∂fLC
∂q (qh,˙qh, uh)·˙qh+∂fLC
∂˙q(qh,˙qh, uh)·¨qh+∂fLC
∂u (qh,˙qh, uh)·˙uh
+O(h3),(4.25)
with qh=q(h), uh=u(h),˙qh= ˙q(h),˙uh= ˙u(h) and so forth.
60
Example 4.3.4 Given a discrete Lagrangian
Lα
d(q0, q1, h) = hL (1 −α)q0+αq1,q1−q0
h
and discrete control forces
fα−
Cd(q0, q1, u0, h) =
hfLC (1 −α)q0+αq1,q1−q0
h, u((1 −α)t0+αt1)·∂((1 −α)q0+αq1)
∂q0
=
(1 −α)hfLC (1 −α)q0+αq1,q1−q0
h, u((1 −α)t0+αt1)
fα+
Cd(q0, q1, u0, h) =
hfLC (1 −α)q0+αq1,q1−q0
h, u((1 −α)t0+αt1)·∂((1 −α)q0+αq1)
∂q1
=
αhfLC (1 −α)q0+αq1,q1−q0
h, u((1 −α)t0+αt1)
for some parameter α∈[0,1] and calculating the expansions in hgives
Lα
d(q(0), q(h), h) = hL(q, ˙q) + 1
2h22α∂L
∂q (q, ˙q)·˙q+∂L
∂˙q(q, ˙q)·¨q+O(h3),
and
fα−
Cd(q(0), q(h),{u(cih)}s
i=1, h) = hfLC(q, ˙q, u) + 1
2h22αfLC(q, ˙q, u)·∂˙q
∂q
+2α∂fLC
∂q (q, ˙q, u)·˙q+∂fLC
∂˙q(q, ˙q, u)·¨q+ 2α∂fLC
∂u (q, ˙q, u)·˙u
+O(h3),
fα+
Cd(q(0), q(h),{u(cih)}s
i=1, h) = hfLC(q, ˙q, u)−1
2h22(1 −α)fLC(q, ˙q, u)·∂˙q
∂q
+2(1 −α)∂fLC
∂q (q, ˙q, u)·˙q+∂fLC
∂˙q(q, ˙q, u)·¨q+ 2(1 −α)∂fLC
∂u (q, ˙q, u)·˙u
+O(h3).
Comparing these to the expansions (4.23), (4.24) and (4.25) for the exact discrete
Lagrangian and the exact discrete control forces shows that the method is second
order if and only if α= 1/2.
61
4.3.3 Discrete problem as direct solution method
Within the discrete context, all relevant objects are approximated by discrete
expressions. Therefore, we obtain a finite dimensional optimization problem re-
stricted by the forced discrete Euler-Lagrange equations, the boundary condi-
tions, and the path constraints. Thus, by understanding the discrete optimal
control problem as a discrete approximation of the continuous Lagrangian opti-
mal control problem 4.1.1, a direct solution method is provided as described in
Chapter 2. The main difference here is, that rather than discretizing the differen-
tial equations arising from the Lagrange-d’Alembert principle, we discretize one
step earlier. Namely, we derive the discrete equations for the optimization prob-
lem by considering the discrete Lagrange-d’Alembert principle directly. In Figure
4.2 we present schematically the different strategies of standard direct methods
and DMOC.
objective functional +
Lagrange d’Alembert principle
variation
objective functional +
Euler-Lagrange equations
num. discretization
discrete objective function +
discretized differential
equation
geom. discretization
discrete objective function +
discrete Lagrange-d’Alembert
principle
discrete objective function +
discrete Euler-Lagrange
equations
variation
finite differences,
multiple shooting,
collocation, ...
DMOC
Figure 4.2: Comparison of solution strategies for optimal control problems: stan-
dard direct methods and DMOC.
Our approach derived via the concept of discrete mechanics leads to a special
discretization of the system equations based on variational integrators, which are
dealt with in detail in [113]. Thus, the discrete optimal control problem inherits
62
special properties exhibited by variational integrators. In the following, we specify
particular important properties and phenomena of variational integrators and try
to translate their meaning into the optimal control context.
Preservation of momentum maps If the discrete system, obtained by apply-
ing variational integration to a mechanical system, inherits the same symmetry
groups as the continuous system, the corresponding discrete momentum maps are
preserved (see Theorem 3.1.5). For the forced case the same statement holds, if
the forcing is orthogonal to the group action (see Theorem 3.3.5).
On the one hand, this means for the optimal control problem, that if the con-
trol force is orthogonal to the group action, our discretization leads to a discrete
system, for which the corresponding momentum map is preserved. An example
is given in Section 5.4.1. On the other hand, in the case of the forcing not being
orthogonal to the group action, the forced discrete Noether’s theorem provides
an exact coherence between the change in angular momentum and the applied
control force via
[Jf+
Ld◦FN−1
Ld(ud)−Jf−
Ld](q0, q1)·ξ=
N−1
X
k=0
fCd(qk, qk+1, uk)·ξQ×Q(qk, qk+1),
(see Section 5.2 for examples).
Conservation of modified energy As shown in Section 3.1.2 variational inte-
grators are symplectic, which implies that a certain modified energy is conserved
(see for example [55]). This is an important property if the long time behavior of
dynamical systems is considered. For the case of the optimal control of systems
with long maneuver time such as low thrust space missions, it would therefore be
interesting to investigate the relation between a modified energy and the virtual
work. However, this has not been considered within this thesis.
Implementation Rather than using a configuration-momentum implementa-
tion of variational integrators as proposed in [113], we stay on Q×Q. That means
we just determine the optimal trajectory for the configuration and the control
forces and reconstruct the corresponding momenta and velocities via the forced
discrete Legendre transforms. This yields computational savings. A more detailed
description of the computational savings compared to standard discretizations for
optimal control problems is given in Remark 4.4.4.
63
4.4 High-order discretization
As shown in the previous section, we have to construct discrete Lagrangians and
discrete forces which accurately approximate the action integral and the virtual
work to design high-order discretizations.
In this section, we first show how to create such discretizations in practice. An
appropriate way is to use polynomial approximations to the trajectories and nu-
merical quadrature to approximate the integrals. We formulate the corresponding
discrete optimal control problem and show the equivalence to the optimization
problem resulting by a special form of Runge-Kutta discretization for Hamilto-
nian dynamics.
This equivalence is important for following sections concerning error analysis
and convergence, since much is known about optimal control problems discretized
by Runge-Kutta methods (for example Hager et al. [39, 40, 41, 53, 54]).
4.4.1 Quadrature approximation
We approximate the space of trajectories C([0, h], Q) = {q: [0, h]→Q|q(0) =
q0, q(h) = q1}by a finite-dimensional approximation Cs([0, h], Q)⊂ C([0, h], Q) of
the trajectory space given by
Cs([0, h], Q) = {q∈ C([0, h], Q)|q∈Πs},
with Πsthe space of polynomials of degree s. Analogously, we approximate the
control trajectory by a polynomial of degree mand define the corresponding
control space as
Dm([0, h], U) = {u∈ D([0, h], U)|u∈Πm},
with D([0, h], U) = {u: [0, h]→U}the original control space. Given con-
trol times 0 = d0< d1<· · · < ds−1< ds= 1 and control points q0
0=
q0, q1
0, q2
0, . . . , qs−1
0, qs
0=q1the degree spolynomial qd(t;qν
0, h) which passes through
each qν
0at time dνh, that is, qd(dνh) = qν
0for ν= 0, . . . , s, is uniquely defined.
Analogously, we choose a set of interior points u0
0, . . . , um
0that represents a para-
metrization of the space of polynomials of degree mmapping [0, h] to U.
Now we approximate the action integral with numerical quadrature to give
an approximate action Gs:C([0, h], Q)→Rby
Gs(qd(t;qν
0, h)) = h
s
X
i=1
biL(qd(cih),˙qd(cih)) ,(4.26)
where ci∈[0,1], i = 1, . . . , s are a set of quadrature points and bia set of order
weights and define the multipoint discrete Lagrangian as
Ld(q0
0, q1
0, . . . , qs
0, h) = Gs(qd(t;qν
0, h)),(4.27)
64
and the appropriate discrete action sum over the entire trajectory to be
Gd({(qk=q0
k, q1
k, . . . , qs
k=qk+1)}N−1
k=0 ) =
N−1
X
k=0
Ld(q0
k, q1
k, . . . , qs
k, h).
Similar to the action integral we approximate the virtual work for one time in-
terval Ws:C([0, h], Q)× D([0, h], U)→Rby
Ws(qd(t;qν
0, h), ud(t;uη
0, h)) = h
s
X
i=1
bifLC(qd(cih),˙qd(cih), ud(cih)) ·δqd(cih)
and define the appropriate discrete virtual work over the entire trajectory to be
Wd({(qk=q0
k, q1
k, . . . , qs
k=qk+1)}N−1
k=0 ,{(u0
k, . . . , um
k)}N−1
k=0 ) =
N−1
X
k=0
h
s
X
i=1
bifLC(qd(tk+cih),˙qd(tk+cih), ud(tk+cih)) ·δqd(tk+cih).
We define the multipoint discrete forces as follows
fν+
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h) = h
s
X
i=1
bifLC(tk+cih)∂qd(tk+cih)
∂qν
k
,
fν−
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h) = h
s
X
i=1
bifLC(tk+cih)∂qd(tk+cih)
∂qν
k
,
with fLC(tk+cih) = fLC(qd(tk+cih),˙qd(tk+cih), ud(tk+cih)). To simplify the
notation we denote the left and right discrete forces fν±
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h)
by fν±
k.
With δqd(tk+cih) = Ps
ν=0
∂qd(tk+cih)
∂qν
kδqν
kand requiring that the discrete La-
grange d’Alembert principle holds for Gdand Wd,Gdleads to the extended set
of discrete Euler-Lagrange equations
Dν+1Ld(q0
k, . . . , qs
k, h) + fν±
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h) = 0, ν = 1, . . . , s −1
(4.28a)
Ds+1Ld(q0
k, . . . , qs
k, h) + D1Ld(q0
k+1, . . . , qs
k+1, h)+
fs+
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h) + f0−
Cd(q0
k+1, . . . , qs
k+1, u0
k+1, . . . , um
k+1, h) = 0.
(4.28b)
65
Following [113] the (s+ 1)-point discrete Lagrangian (4.27) can be used to derive
a standard two-point discrete Lagrangian by taking
Ld(qk, qk+1, h) = ext(q1
k,...,qs−1
k)Ld(qk=q0
k, q1
k, . . . , qs
k=qk+1, h),
where extLdmeans that Ldshould be evaluated at extreme or critical values of
q1
k, . . . , qs−1
k, that means evaluated on the trajectory within the step which solves
(4.28a). We define the left and right discrete forces to be
f+
Cd(qk, qk+1, uk, h) =
fs+
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h) +
s−1
X
ν=1
fν+
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h)·∂qν
k
∂qk+1
,
(4.29a)
f−
Cd(qk, qk+1, uk, h) =
f0−
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h) +
s−1
X
ν=1
fν−
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h)·∂qν
k
∂qk
,
(4.29b)
and combine them to a single one-form as
fCd(qk, qk+1, uk, h)·(δqk, δqk+1) = f+
Cd(qk, qk+1, uk, h)·δqk+1+f−
Cd(qk, qk+1, uk, h)·δqk.
Note that the forced discrete Euler-Lagrange equations with these definitions
satisfy
D1Ld(qk, qk+1, h)+D2Ld(qk−1, qk, h)+f+
Cd(qk−1, qk, uk−1, h)+f−
Cd(qk, qk+1, uk, h)=
D1Ld(qk, q1
k, . . . , qs−1
k, qk+1, h) +
s−1
X
ν=1
Dν+1Ld(qk, q1
k, . . . , qs−1
k, qk+1, h)·∂qν
k
∂qk
+
Ds+1Ld(qk−1, q1
k−1, . . . , qs−1
k−1, qk, h)+
s−1
X
ν=1
Dν+1Ld(qk−1, q1
k−1, . . . , qs−1
k−1, qk, h)·∂qν
k−1
∂qk
+fs+
Cd(q0
k−1, . . . , qs
k−1, u0
k−1, . . . , um
k−1, h)
+
s−1
X
ν=1
fν+
Cd(q0
k−1, . . . , qs
k−1, u0
k−1, . . . , um
k−1, h)·∂qν
k−1
∂qk
+f0−
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h) +
s−1
X
ν=1
fν−
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h)·∂qν
k
∂qk
=
66
D1Ld(qk, q1
k, . . . , qs−1
k, qk+1, h) + Ds+1Ld(qk−1, q1
k−1, . . . , qs−1
k−1, qk, h)
+fs+
Cd(q0
k−1, . . . , qs
k−1, u0
k−1, . . . , um
k−1, h) + f0−
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h)
+
s−1
X
ν=1 Dν+1Ld(qk, q1
k, . . . , qs−1
k, qk+1, h) + fν−
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h)·∂qν
k
∂qk
+
s−1
X
ν=1 Dν+1Ld(qk−1, q1
k−1, . . . , qs−1
k−1, qk, h)
+fν+
Cd(q0
k−1, . . . , qs
k−1, u0
k−1, . . . , um
k−1, h)·∂qν
k−1
∂qk
=
D1Ld(qk, q1
k, . . . , qs−1
k, qk+1, h) + Ds+1Ld(qk−1, q1
k−1, . . . , qs−1
k−1, qk, h)
+fs+
Cd(q0
k−1, . . . , qs
k−1, u0
k−1, . . . , um
k−1, h) + f0−
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k, h) = 0,
and thus are equivalent to the Euler-Lagrange equations (4.28).
4.4.2 High-order discrete optimal control problem
We now formulate the optimal control problem of high-order discretizations ob-
tained via the quadrature approximation described above.
In the previous section we have shown, how to discretize the Lagrange-
d’Alembert principle via a quadrature approximation of the Lagrangian and the
virtual work. However, we still have to find appropriate discrete high-order ver-
sions for the objective function, the boundary conditions, and the path con-
straints.
Boundary conditions In a straightforward manner compared to Section 4.2,
we set up the boundary conditions on the configuration level as q0=q0. For
the constraint on the velocity level we use the discrete version of the correspond-
ing continuous constraint on the conjugate momentum and define the discrete
multipoint boundary condition as
¯rd(q0
N−1, . . . , qs
N−1, u0
N−1, . . . , um
N−1, qT,˙qT) = ˜r(qN, pN,FL(qT,˙qT)),
where the final momentum pNis given by the discrete Legendre transform as pN=
Ds+1Ld(q0
N−1, . . . , qs
N−1, h)+fs+
Cd(q0
N−1, . . . , qs
N−1, u0
N−1, . . . , um
N−1, h). To trace the
discrete multipoint boundary condition back to the discrete boundary condition
defined in Section 4.2 we define
rd(qN−1, qN, uN−1, qT,˙qT) =
¯rd(q0
N−1, . . . , qs
N−1, u0
N−1, . . . , um
N−1, qT,˙qT),
67
where uN−1={ud(tN−1+cih;u0
N−1, . . . , um
N−1, h)}s
i=1 is the sequence of control
parameters guiding the system from qN−1to qNas defined in Section 3.3.5.
Discrete path constraints As before, we replace the path constraints h(qd(tk+
cih;qν
k, h),˙qd(tk+cih;qν
k, h), ud(tk+cih;uη
k, h)) ≥0 on each substep of each time
interval [tk+cih, tk+ci+1h], k = 0, . . . , N −1, i = 1, . . . , s −1 by the discrete
path constraint
¯
hd(q0
k, . . . , qs
k, u0
k, . . . , um
k)≥0,
k= 0, . . . , N −1, i = 1, . . . , s. With hddefined as
hd(qk, qk+1, uk) = ¯
hd(q0
k, . . . , qs
k, u0
k, . . . , um
k)
we obtain the original form of the discrete path constraints defined in Section 4.2.
Discrete objective function Analogous to the discrete Lagrangian we appro-
ximate the objective functional RT
0C(q(t),˙q(t), u(t)) dtwith the same quadrature
rule as
¯
Cd(q0
0, . . . , qs
0, u0
0, . . . , um
0) = h
s
X
i=1
biC(qd(cih),˙qd(cih), ud(cih))
and the final condition analogous to the boundary condition as
¯
Φd(q0
N−1, . . . , qs
N−1, u0
N−1, . . . , us
N−1).
This results to the discrete objective function
Jd(qd, ud) =
N−1
X
k=0
¯
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k) + ¯
Φd(q0
N−1, . . . , qs
N−1, u0
N−1, . . . , us
N−1).
With Cddefined as
Cd(qk, qk+1, uk) = ¯
Cd(q0
k, . . . , qs
k, u0
k, . . . , um
k),
and Φdbeing defined as
Φd(qN−1, qN, uN−1) = ¯
Φd(q0
N−1, . . . , qs
N−1, u0
N−1, . . . , um
N−1),
we obtain the original expression for the discrete objective function
Jd(qd, ud) =
N−1
X
k=0
Cd(qk, qk+1, uk)+Φd(qN−1, qN, uN−1).
68
Optimal control problem of high-order The optimal control problem of
high-order discretizations obtained via quadrature approximation can now be
formulated in the same way as Problem 4.2.2
min
qd,ud
Jd(qd, ud) =
N−1
X
k=0
Cd(qk, qk+1, uk)+Φd(qN−1, qN, uN) (4.30a)
subject to
q0=q0,(4.30b)
D2L(q0,˙q0) + D1Ld(q0, q1) + f−
0= 0,(4.30c)
D2Ld(qk−1, qk) + D1Ld(qk, qk+1) + f+
k−1+f−
k= 0, k = 1, . . . , N −1,(4.30d)
hd(qk, qk+1, uk)≥0, k = 0, . . . , N −1,(4.30e)
rd(qN−1, qN, uN, qT,˙qT) = 0.(4.30f)
Within the formulation above, the extended Euler-Lagrange equations (4.28a)
have to be satisfied and represent extra conditions for the resulting optimization
problem.
Examples 4.4.1 (a) Let the configuration trajectory q(t) be approximated by
straight lines, that is a polynomial of order s= 1 as qd(t) = qk+t−tk
h(qk+1 −
qk), for t∈[tk, tk+1], and the control u(t) by a polynomial of order m= 0
as ud(t) = u((tk+tk+1)/2) on [tk, tk+1]. Choose quadrature points c=1
2
and weights b= 1 to obtain a midpoint quadrature approximation as Ld=
hL(qk+qk+1
2,qk+1−qk
h, u(tk+h/2)).
(b) By taking quadrature points cias done in the Gauss-Legendre quadrature,
which is the highest-order quadrature for a given number of points, and by
enforcing c0= 0 and cs= 1, we obtain the Lobatto quadrature.
4.4.3 Correspondence to Runge-Kutta discretizations
In the following we show, that the discrete optimal control problem derived via
the discrete Lagrange-d’Alembert principle is equivalent to the optimization prob-
lem resulting from a symplectic partitioned Runge-Kutta discretization for the
Hamiltonian dynamics.
First, we introduce symplectic partitioned Runge-Kutta methods. We propose
corresponding expressions for the objective function, the boundary conditions,
and the path constraints. Using these expressions we formulate the optimal con-
trol problem from the Hamiltonian point of view discretized via a Runge-Kutta
map.
69
Symplectic partitioned Runge-Kutta methods
As shown in [113] (Theorem 2.6.1), the discrete Hamiltonian map generated by
the discrete Lagrangian is a symplectic partitioned Runge-Kutta method. A
similar statement is true for discrete Hamiltonian maps with forces. The resulting
method is still a partitioned Runge-Kutta method, but no longer symplectic in
the original sense since the symplectic form is not preserved anymore due to the
presence of control forces. However, we still denote it as a symplectic method
having in mind that the symplectic form is preserved only in absence of external
control forces.
A partitioned Runge-Kutta method for the regular forced Lagrangian system
Land fLC is a map T∗Q×Us×R→T∗Qspecified by the coefficients bi, aij,˜
bi,˜aij
for i= 1, . . . , s, and defined by (q0, p0, u0)7→ (q1, p1) for
q1=q0+h
s
X
j=1
bj˙
Qj,
Qi=q0+h
s
X
j=1
aij ˙
Qj,
Pi=∂L
∂˙q(Qi,˙
Qi),
p1=p0+h
s
X
j=1
˜
bj˙
Pj,(4.31a)
Pi=p0+h
s
X
j=1
˜aij ˙
Pj,(4.31b)
˙
Pi=∂L
∂q (Qi,˙
Qi) + fLC(Qi,˙
Qi, Ui),(4.31c)
i= 1, . . . , s, where the points (Qi, Pi) are known as the internal stages and Uiare
the control samples given by Ui=u0i=ud(t0+cih). For aij = ˜aij and bi=˜
bi
the partitioned Runge-Kutta method is a Runge-Kutta method.
The method is symplectic (that is, it preserves the canonical symplectic form
Ω on T∗Qin absence of external forces) if the coefficients satisfy
bi˜aij +˜
bjaji =bi˜
bj,
bi=˜
bi,
i, j = 1, . . . , s, (4.32a)
i= 1, . . . , s. (4.32b)
Since discrete Lagrangian maps are symplectic, we can assume that we have
conditions satisfying (4.32) and write a discrete Lagrangian and discrete La-
grangian control forces that generate the corresponding symplectic partitioned
Runge-Kutta method. Given points (q0, q1)∈Q×Q, we can regard (4.31) as
implicitly defining p0, p1, Qi, Pi,˙
Qiand ˙
Pifor i= 1, . . . , s. Taking these to be
defined as functions of (q0, q1), we construct a discrete Lagrangian
Ld(q0, q1, h) = h
s
X
i=1
biL(Qi,˙
Qi),(4.33)
70
and left and right discrete forces as
f+
Cd(q0, q1, u0, h) = h
s
X
i=1
bifLC(Qi,˙
Qi, Ui)·∂Qi
∂q1
,(4.34a)
f−
Cd(q0, q1, u0, h) = h
s
X
i=1
bifLC(Qi,˙
Qi, Ui)·∂Qi
∂q0
.(4.34b)
Based on the lines of [113] (Theorem 2.6.1) we show, that the corresponding
forced discrete Hamiltonian map is exactly the map (q0, p0, u0)7→ (q1, p1) which
is the symplectic partitioned Runge-Kutta method for the forced Hamiltonian
system (3.18).
Theorem 4.4.2 The discrete Hamiltonian map generated by the discrete La-
grangian (4.33) together with the discrete forces (4.34) is a symplectic partitioned
Runge-Kutta method.
Proof: We need to check that the equations (3.26) are satisfied. It holds
∂Ld
∂q0
(q0, q1) + f−
Cd(q0, q1, u0)
=h
s
X
i=1
bi"∂L
∂q ·∂Qi
∂q0
+∂L
∂˙q·∂˙
Qi
∂q0#+h
s
X
i=1
bifLC ·∂Qi
∂q0
=h
s
X
i=1
bi"˙
Pi−fLC·∂Qi
∂q0
+Pi·∂˙
Qi
∂q0
+fLC ·∂Qi
∂q0#
=h
s
X
i=1
bi"˙
Pi·∂Qi
∂q0
+Pi·∂˙
Qi
∂q0#,
using the definitions for Piand ˙
Piin (4.31). Differentiating the definition for Qi
in (4.31b) with respect to q0and substituting the resultant expression and the
definition of Piin (4.31b) gives
∂Ld
∂q0
(q0, q1) + f−
Cd(q0, q1, u0)
=h
s
X
i=1
bi"˙
Pi· I+h
s
X
j=1
aij
∂˙
Qj
∂q0!+ p0+h
s
X
j=1
˜aij ˙
Pj!·∂˙
Qi
∂q0#
=h
s
X
i=1
bi"˙
Pi+p0·∂˙
Qi
∂q0#+h2
s
X
i=1
s
X
j=1
(bi˜aij +bjaji)˙
Pj·∂˙
Qi
∂q0
,
71
and with the symplecticity identities (4.32) we obtain
∂Ld
∂q0
(q0, q1) + f−
Cd(q0, q1, u0)
=p0·"h
s
X
i=1
bi
∂˙
Qi
∂q0#+h
s
X
i=1
bi˙
Pi+h
s
X
j=1
bj˙
Pj"h
s
X
i=1
bi
∂˙
Qi
∂q0#
=−p0,
where we have differentiated the expression for q1in (4.31a) with respect to q0to
obtain the identity
h
s
X
i=1
bi
∂˙
Qi
∂q0
=−I.
This establishes the satisfaction of (3.26a). Analogously, we can show that (3.26b)
holds, and therefore the discrete Hamiltonian map ˜
FLdgenerated by the discrete
Lagrangian (4.33) together with the discrete forces (4.34) is indeed the symplectic
partitioned Runge-Kutta method for the forced Hamiltonian system (3.18).
Optimal control and Runge-Kutta discretizations
In this section, we carry forward the results of the previous section into the
context of optimal control problems. To formulate the optimal control problem
for the discrete system in terms of Runge-Kutta discretizations, we have to give
appropriate expressions for the boundary conditions, the path constraints, and
the objective functional.
Concerning the dynamics of the mechanical system, we have already seen that
the discretization obtained via the discrete Lagrange-d’Alembert principle can be
rewritten as a Runge-Kutta scheme for the corresponding mechanical system in
terms of (p, q) as in (4.31). Since we consider regular Lagrangians and regular
Hamiltonians, we reformulate (4.31c) with the help of the Hamiltonian as
˙
Qki =∂H
∂p (Qki, Pki),
˙
Pki =−∂H
∂q (Qki, Pki) + fHC(Qki, Pki, Uki),
where the additional index kdenotes the dependence of the intermediate state
and control variables on the time interval [tk, tk+1].
72
Boundary conditions Due to a formulation of the optimal control problem
within the Hamiltonian framework, we use the same formulation for the boundary
constraint as the one for the continuous Hamiltonian optimal control problem
4.1.3 evaluated at (qN, pN), which reads
˜r(qN, pN, qT, pT) = 0.
Discrete path constraints Again we use the formulation of the Hamiltonian
optimal control problem 4.1.3 and enforce the path constraints to hold in each
time step (Qi, Pi, Ui) as
˜
h(Qki, Pki, Uki)≥0, k = 0, . . . , N −1, i = 1, . . . , s.
Discrete objective function We construct the discrete objective function in
the same way as the discrete Lagrangian (4.33). However, corresponding to the
Hamiltonian formulation evaluated in each substep (Qi, Pi, Ui), it now reads
Jd(qd, pd, ud) =
N−1
X
k=0
h
s
X
i=1
bi˜
C(Qki, Pki, Uki) + ˜
Φ(qN, pN),
where the final constraint holds for the node corresponding to the final time T.
Combining all terms, the discrete optimal control problem from the Hamilto-
nian point of view reads
min
qd,pd,ud
J(qd, pd, ud) =
N−1
X
k=0
h
s
X
i=1
bi˜
C(Qki, Pki, Uki) + ˜
Φ(qN, pN) (4.35a)
subject to
qk+1 =qk+h
s
X
j=1
bj˙
Qkj, Qki =qk+h
s
X
j=1
aij ˙
Qkj, q0=q0,(4.35b)
pk+1 =pk+h
s
X
j=1
˜
bj˙
Pkj, Pki =pk+h
s
X
j=1
˜aij ˙
Pkj, p0=p0,(4.35c)
˙
Qki =∂H
∂p (Qki, Pki),˙
Pki =−∂H
∂q (Qki, Pki) + fHC (Qki, Pki, Uki),(4.35d)
0≤˜
h(Qki, Pki, Uki), k = 0, . . . , N −1, i = 1, . . . , s, (4.35e)
0 = ˜r(qN, pN, qT, pT).(4.35f)
73
Problem (4.35) is the finite dimensional optimization problem resulting from the
Hamiltonian optimal control problem 4.1.3 that is discretized via a symplectic
partitioned Runge-Kutta scheme.
Equivalence With Qki =qd(tk+cih;qν
k, h) and ˙
Qki = ˙qd(tk+cih;qν
k, h) and
since with (4.31c) (Qki, Pki) = FL(Qki,˙
Qki) holds the equivalence of problem
(4.30) and problem (4.35) can be shown: each discrete optimal control problem
derived via the discrete Lagrange-d’Alembert principle (4.30) can be reformulated
as an optimal control problem derived via a Runge-Kutta discretization for the
Hamiltonian dynamics (4.35). In particular, according to Theorem 4.4.2, for a
given discrete Lagrangian (4.26) and discrete forces defined in (4.29) we always
find a symplectic partitioned Runge-Kutta map describing the discrete Hamil-
tonian mechanical system. On the other hand, for each symplectic partitioned
Runge-Kutta map there exist a discrete Lagrangian and discrete forces genera-
ting this map. Additionally, with the results of Section 4.1.2, the feasible sets of
the intermediate points (Qki,˙
Qki) and (Qki, Pki) defined by the boundary condi-
tions and path constraints in tangent and cotangent space, respectively, are equal
under the Legendre transform.
Mayer formulation Similar as for the discrete Lagrangian optimal control
problem, (4.35) can be transformed into an optimal control problem of Mayer type
as follows: Analogous to Section 4.2.2, we introduce extra variables yd={yl}N
l=0
as
y0= 0,
yl=
l−1
X
k=0
h
s
X
i=1
bi˜
C(Qki, Pki, Uki), l = 1, . . . , N,
yielding the discrete optimal control problem of Mayer type as
min
qd,pd,ud,yd
¯
Φ(qN, pN, yN) = yN+˜
Φ(qN, pN) (4.36a)
subject to
qk+1 =qk+h
s
X
j=1
bj˙
Qkj, Qki =qk+h
s
X
j=1
aij ˙
Qkj, q0=q0,(4.36b)
pk+1 =pk+h
s
X
j=1
˜
bj˙
Pkj, Pki =pk+h
s
X
j=1
˜aij ˙
Pkj, p0=p0,(4.36c)
yk+1 =yk+h
s
X
i=1
bi˜
C(Qki, Pki, Uki), y0= 0,(4.36d)
74
˙
Qi=∂H
∂p (Qki, Pki),˙
Pki =−∂H
∂q (Qki, Pki) + fHC(Qki, Pki, Uki),(4.36e)
0≤˜
h(Qki, Pki, Uki), k = 0, . . . , N −1, i = 1, . . . , s, (4.36f)
0 = ˜r(qN, pN, qT, pT).(4.36g)
We obtain exactly the same problem by discretizing the continuous Hamiltonian
optimal control problem of Mayer type (as introduced in Section 4.1.3) with the
same partitioned Runge-Kutta discretization for the extended system of differen-
tial equations
˙
˜q(t) = ν(˜q(t),˜p(t), u(t)),˜q(0) = ˜q0,
˙
˜p(t) = η(˜q(t),˜p(t), u(t)),˜p(0) = ˜p0,
with ˜q= (q, y), ˜p=p,ν(˜q(t),˜p(t), u(t)) = (∇pH(q(t), p(t)),˜
C(q(t), p(t), u(t))),
η(˜q(t),˜p(t), u(t)) = −∇pH(q(t), p(t)) + fHC(q(t), p(t), u(t)), ˜q0= (q0,0) and ˜p0=
p0.
Examples 4.4.3 (a) With b= 1 and a=1
2we obtain the implicit midpoint
rule as a symplectic Runge-Kutta scheme, that is the partitioned scheme
reduces to a standard one-stage Runge-Kutta scheme. The resulting dis-
cretization is equivalent to the discretization derived via the Lagrangian
approach with midpoint quadrature as introduced in Example 4.4.1(a).
(b) The standard Lobatto IIIA-IIIB partitioned Runge-Kutta method is ob-
tained by using the Lobatto quadrature for the discrete Lagrangian as ex-
plained in Example 4.4.1(b).
Remark 4.4.4 The formulations of the discrete optimal control problem on the
one hand based on the discrete Lagrange-d’Alembert principle and on the other
hand based on a Runge-Kutta discretization of the Hamiltonian dynamics are
equivalent, in the sense that the same discrete solution set is described. Note
however, that the Lagrangian formulation needs less discrete variables and less
constraints for the optimization problem, as it is formulated on the configuration
space only, rather than on the space consisting of configurations and momenta:
For q∈Rnand Nintervals of discretization we obtain with the Lagrangian ap-
proach (Ns + 1)nunknown configurations qν
k, k = 0, . . . , N −1, ν = 0, . . . , s
with qs
k=q0
k+1, k = 1, . . . , N −1 and nN(s−1) + n(N−1) extended discrete
Euler-Lagrange equations, so altogether, (Ns −1)nconstraints for the optimiza-
tion problem excluding boundary conditions. The Runge-Kutta approach for
the Hamiltonian system yields 2(Ns+1)nunknown configurations and momenta
and 2nN + 2nN(s−1) = 2nNs equality constraints. Thus, via the Runge-Kutta
75
approach we obtain twice as many state variables and (Ns + 1)nmore equal-
ity constraints such that the resulting optimization problem is numerically more
expensive. Comparisons concerning the computational effort are presented in
Section 5.2.2.
4.5 Adjoint system
The adjoint system provides necessary optimality conditions for the optimal con-
trol problem. In the continuous case the adjoint system is derived by the Pon-
tryagin maximum principle. The KKT equations provide the adjoint system for
the discrete optimal control problem. In this section we derive a transformed
adjoint system of the discrete optimal control problem. This adjoint system is
then compared to the continuous one and we determine the corresponding order
of approximation.
All derivations are based on an analysis by Hager. In [53] he derives a trans-
formed adjoint system using standard Runge-Kutta discretizations of the opti-
mal control problem. He identifies constraints on the Runge-Kutta coefficients
to determine the order of approximation of the adjoint scheme. In this way,
the resulting optimality system, after some change of variables, is a partitioned
Runge-Kutta scheme and Hager computes the order conditions for order up to 4.
In [23] Bonnans et al. extend this approach to the order conditions for order up
to 7.
By using the same strategy as Hager in [53], we show, that the DMOC dis-
cretization leads to a discretization of the same order for the adjoint system as
for the state system. Therefore no additional order conditions on the coefficients
determining the corresponding discretization scheme are necessary.
In the following we ignore path and control constraints and restrict ourselves
to the case of unconstrained optimal control problems.
4.5.1 Continuous setting
First of all, instead of considering the Lagrangian optimal control problem 4.1.1,
we analyze the Hamiltonian optimal control problem that is formulated in an
equivalent way. For simplicity, we restrict ourselves to the case without final
constraint and without path constraints.
Problem 4.5.1
min
q,p,u Φ(q(T), p(T)) (4.37a)
76
subject to
˙q(t) = ν(q(t), p(t)),for all t∈[0, T],(4.37b)
˙p(t) = η(q(t), p(t), u(t)),for all t∈[0, T],(4.37c)
q(0) = q0, p(0) = p0,(4.37d)
u(t)∈U, q, p ∈W1,∞, u ∈L∞,(4.37e)
with ν:Rn×Rn→Rn, ν(q(t), p(t)) = ∇pH(q(t), p(t)), η:Rn×Rn×Rm→
Rn, η(q(t), p(t), u(t)) = −∇qH(q(t), p(t)) + fHC(q(t), p(t), u(t)) and U⊂Rmis
closed and convex. Since we want to neglect constraints on the control function,
we assume U=Rmfor the following analysis.
Wm,p(Rn) is the Sobolev space consisting of vector-valued measurable functions
q: [0, T]→Rnwhose j-th derivative lies in Lpfor all 0 ≤j≤mwith the norm
kqkWm,p =
m
X
j=0
kq(j)kLp.
Ba(x) is the closed ball centered at xwith radius a.
Remark 4.5.2 In the case of objective functionals of Bolza type, we transform
the optimal control problem into the Mayer form and consider an extended sys-
tem ˙
˜q(t) = ˜ν(˜q(t), p(t), u(t)) with ˜q= (q, y) and ˜ν:Rn×Rn×Rm→Rn+1,
˜ν(˜q(t), p(t), u(t)) = (∇pH(q(t), p(t)),˜
C(q(t), p(t), u(t))). However, within this sec-
tion we restrict ourselves to the formulation of Problem 4.5.1, having in mind,
that a Bolza problem can be analyzed in a straightforward way.
For brevity, we introduce the notation x(t)=(q(t), p(t)), x0= (q0, p0) and
define
˜
f(x(t), u(t)) = ν(q(t), p(t))
η(q(t), p(t), u(t)) ,Φ(x(T)) = Φ(q(T), p(T)),
such that (4.37b) - (4.37d) reads as
˙x(t) = ˜
f(x(t), u(t)) for all t∈[0, T], x(0) = x0.
In the following we will need both kinds of notation.
Along the lines of [53], we now present the assumptions that are employed
in the analysis of DMOC discretizations of Problem 4.5.1. First, a smoothness
assumptions is required to ensure regularity of the solution and the problem
functions. Second, we enforce a growth condition that allows for having a unique
solution for the control function of the optimal control problem.
77
Assumption 4.5.3 (Smoothness) For some integer κ≥2, Problem 4.5.1 has
a local solution (x∗, u∗) which lies in Wκ,∞×Wκ−1,∞. There exists an open set
Ω⊂R2n×Rmand ρ > 0 such that Bρ(x∗(t), u∗(t)) ⊂Ω for every t∈[0, T].
The first κderivatives of νand ηare Lipschitz continuous in Ω, and the first κ
derivatives of Φ are Lipschitz continuous in Bρ(x∗(T)).
Under this assumption there exits an associated Lagrange multiplier
ψ∗= (ψq,∗, ψp,∗)∈Wκ,∞for which the following form of the first-order opti-
mality conditions derived via the Pontryagin maximum principle is satisfied at
(x∗, ψ∗, u∗):
˙q(t) = ν(q(t), p(t)) for all t∈[0, T], q(0) = q0,(4.38a)
˙p(t) = η(q(t), p(t), u(t)) for all t∈[0, T], p(0) = p0,(4.38b)
˙
ψq(t) = −∇qH(q(t), p(t), ψq(t), ψp(t), u(t)) for all t∈[0, T],(4.38c)
ψq(T) = ∇qΦ(q(T), p(T)),(4.38d)
˙
ψp(t) = −∇pH(q(t), p(t), ψq(t), ψp(t), u(t)) for all t∈[0, T],(4.38e)
ψp(T) = ∇pΦ(q(T), p(T)),(4.38f)
u(t)∈U, ∇uH(q(t), p(t), ψq(t), ψp(t), u(t)) = 0 for all t∈[0, T],(4.38g)
with Hthe Hamiltonian defined by
H(q(t), p(t), ψq(t), ψp(t), u(t)) = ψqν(q, p) + ψpη(q, p, u),(4.39)
where ψqand ψpare row vectors in Rn. For brevity, we define
H(x(t), ψ(t), u(t)) := ψ˜
f(x, u),(4.40)
such that (4.38c) - (4.38f) reads as
˙
ψ(t) = −∇xH(x(t), ψ(t), u(t)) for all t∈[0, T],
ψ(T) = ∇xΦ(x(T)).
To formulate the second-order sufficient optimality conditions along the lines of
[53], we define the following matrices:
A(t) = ∇x˜
f(x∗(t), u∗(t)), B(t) = ∇u˜
f(x∗(t), u∗(t)), V =∇2Φ(x∗(T)),
P(t) = ∇xxH(x∗(t), ψ∗(t), u∗(t)), R(t) = ∇uuH(x∗(t), ψ∗(t), u∗(t)),
S(t) = ∇xuH(x∗(t), ψ∗(t), u∗(t)).
Let Bbe the quadratic form defined by
B(x, u) = 1
2x(T)TV x(T) + hx, Pxi+hu, Rui+ 2hx, Sui,
where h·,·i denotes the usual L2inner product.
78
Assumption 4.5.4 (Coercivity) There exists a constant α > 0 such that
B(x, u)≥αkuk2
L2for all (x, u)∈ M,
where M={(x, u) : x∈W1,2, u ∈L2,˙x=Ax +Bu,
x(0) = 0, u(t)∈Rma. e. t∈[0, T]}.
Coercivity is a strong form of a second-order sufficient optimality condition in
the sense that it implies not only strict local optimality, but also Lipschitzian
dependence of the solution and multipliers on parameters (see for example [38]).
By coercivity and smoothness the control uniqueness property holds (as shown
in [40] and [52]). That means the Hamiltonian has a locally unique minimizer in
the control, such that there exists a locally unique solution denoted by u(x, ψ)
depending Lipschitz continuously on xand ψ.
In the case where the control is uniquely determined by (x, ψ) through mini-
mization of the Hamiltonian, let φ= (φq, φp) denote the function defined by
φq(q, p, ψq, ψp) = −∇qH(q, p, u, ψq, ψp)|u=u(q,p,ψq,ψp),
φp(q, p, ψq, ψp) = −∇pH(q, p, u, ψq, ψp)|u=u(q,p,ψq,ψp).
Additionally, let η(q, p, ψq, ψp) denote the function η(q, p, u(ψq, ψp)). By sub-
stituting in (4.38) the control obtained by solving (4.38g) for u(t) in terms of
(x(t), ψ(t)), we obtain the following two-point boundary-value problem:
˙q(t) = ν(q(t), p(t)), q(0) = q0,(4.41a)
˙p(t) = η(q(t), p(t), ψq(t), ψp(t)), p(0) = p0,(4.41b)
˙
ψq(t) = φq(q(t), p(t), ψq(t), ψp(t)), ψq(T) = ∇qΦ(q(T), p(T)),(4.41c)
˙
ψp(t) = φp(q(t), p(t), ψq(t), ψp(t)), ψp(T) = ∇pΦ(q(T), p(T)),(4.41d)
or, equivalently,
˙x(t) = ˜
f(x(t), ψ(t)), x(0) = x0,
˙
ψ(t) = φ(x(t), ψ(t)), ψ(T) = ∇xΦ(x(T)).
4.5.2 Discrete setting
In Section 4.4.3 it has been shown that the discretization of the mechanical system
obtained by using the discrete Lagrange d’Alembert principle is equivalent to a
discretization obtained by using a s-stage symplectic partitioned Runge-Kutta
time-stepping scheme, that is
79
q0
k=
s
X
i=1
bi˙
Qki, q0=q0,
Qki =qk+h
s
X
j=1
aq
ij ˙
Qkj,
p0
k=
s
X
i=1
bi˙
Pki, p0=p0,
Pki =pk+h
s
X
j=1
ap
ij ˙
Pkj,
1≤i≤s, 0≤k≤N−1, with biap
ij +bjaq
ji =bibjto ensure the symplecticity
of the Runge-Kutta scheme. Prime denotes, in this discrete context, the forward
divided differences:
q0
k=qk+1 −qk
h.
Qkj,˙
Qkj,Pkj and ˙
Pkj are the intermediate state variables on the interval [tk, tk+1].
With
˙
Qki =∇pH(Qki, Pki),
˙
Pki =−∇qH(Qki, Pki) + fHC(Qki, Pki, Uki),
where Uki are the intermediate control variables on [tk, tk+1], we can reformulate
the discrete Lagrangian optimal control problem 4.2.2 and obtain an equivalent
problem formulation as follows:
min
qd,pd,ud
C(qN, pN) (4.43a)
subject to
q0
k=
s
X
i=1
biν(Qki, Pki), q0=q0,(4.43b)
Qki =qk+h
s
X
j=1
aq
ijν(Qkj, Pkj),(4.43c)
p0
k=
s
X
i=1
biη(Qki, Pki, Uki), p0=p0,(4.43d)
Pki =pk+h
s
X
j=1
ap
ijη(Qkj, Pkj, Ukj),(4.43e)
1≤i≤s, 0≤k≤N−1.
80
For small enough h, and for xk= (qk, pk) near x∗(tk) = (q∗(tk), p∗(tk)) and
Ukj,1≤j≤s, near u∗(tk), the intermediate variables Qki and Pki in (4.43c) and
(4.43e) are uniquely determined. This follows from smoothness and the implicit
function theorem as for standard Runge-Kutta discretization as stated in [53] (see
for example [46]).
Let νh:R2n×Rsm →Rn,ηh:R2n×Rsm →Rnand ˜
fh:R2n×Rsm →R2n
be defined by
νh(q, p, u) =
s
X
i=1
biν(Qi(q, p, u), Pi(q, p, u)),
ηh(q, p, u) =
s
X
i=1
biη(Qi(q, p, u), Pi(q, p, u), Ui) and
˜
fh(x, u) = νh(q, p, u)
ηh(q, p, u).
In other words,
νh(q, p, u) =
s
X
i=1
biν(Qi, Pi),
ηh(q, p, u) =
s
X
i=1
biη(Qi, Pi, Ui),
where y= (Q, P) is the solution of (4.43c) and (4.43e) given by the state unique-
ness property and u= (U1, U2, . . . , Us)∈Rsm. The corresponding discrete Hamil-
tonian Hh:R2n×R2n×Rsm →Ris defined to be
Hh(p, q, ψq, ψp, u) = ψqνh(q, p, u) + ψpηh(q, p, u).
For brevity, we also use the notation
Hh(x, ψ, u) = ψ˜
fh(q, p, u).
We consider the following version of the first-order necessary optimality conditions
associated with (4.43) (Karush-Kuhn-Tucker conditions):
q0
k=νh(qk, pk, uk), q0=q0,(4.44a)
p0
k=ηh(qk, pk, uk), p0=p0,(4.44b)
(ψq
k)0=−∇qHh(qk, pk, ψq
k+1, ψp
k+1, uk), ψq
N=∇qΦ(qN, pN),(4.44c)
(ψp
k)0=−∇pHh(qk, pk, ψq
k+1, ψp
k+1, uk), ψp
N=∇pΦ(qN, pN),(4.44d)
Uki ∈U, ∇UiHh(qk, pk, ψq
k+1, ψp
k+1, uk) = 0,1≤i≤s, (4.44e)
81
where ψq
k, ψp
k∈Rn, 0 ≤k≤N−1. Here, uk∈Rms is the entire discrete control
vector at time level k:
uk= (Uk1, Uk2, . . . , Uks)∈Rms.
For ψk= (ψq
k, ψp
k)∈R2n, (4.44) reads
x0
k=˜
fh(xk, uk), x0=x0,(4.45a)
ψ0
k=−∇xHh(xk, ψk+1, uk), ψN=∇Φ(xN),(4.45b)
Uki ∈U, ∇UiHh(xk, ψk+1, uk) = 0,1≤i≤s. (4.45c)
4.5.3 The transformed adjoint system
To rewrite the first order conditions (4.44) in a way that is better suited for the
analysis and computation, we now use the same strategy as Hager in [53]. We just
describe the essential steps in this section. A detailed explanation of the trans-
formation steps is presented in Appendix B. In contrast to Hager’s formulation,
we already start with a partitioned Runge-Kutta scheme for the state equations.
However, we can use the same transformation as in [53] applied to the scheme for
both state variables qand p, respectively, leading to two symplectic partitioned
Runge-Kutta schemes for two state-costate equations. Due to the symplecticity
of the discretization, we obtain altogether one extended symplectic partitioned
scheme for the state-adjoint system. This scheme is of the same order as the state
scheme that discretizes the Hamiltonian system.
To reformulate the adjoint system, Hager’s and therefore our transformation
strategy can be summarized as follows: First of all we introduce multipliers for
the intermediate equations (4.43c) and (4.43e) in addition to the multipliers for
equations (4.43b) and (4.43d) and formulate the corresponding KKT equations.
After some transformation and change of variables (see Appendix B) we obtain
82
ψq
k=ψq
k+1 +h
s
X
i=1
bi(χq
ki∇qν(Qki, Pki) + χp
ki∇qη(Qki, Pki, Uki)) ,
ψq
N=∇qΦ(qN, pN),(4.46a)
ψp
k=ψp
k+1 +h
s
X
i=1
bi(χq
ki∇pν(Qki, Pki) + χp
ki∇pη(Qki, Pki, Uki)) ,
ψp
N=∇pΦ(qN, pN),(4.46b)
χq
ki =ψq
k+1 +h
s
X
j=1
bjaq
ji
biχq
kj∇qν(Qkj, Pkj) + χp
kj∇qη(Qkj, Pkj, Ukj),(4.46c)
χp
ki =ψp
k+1 +h
s
X
j=1
bjap
ji
biχq
kj∇pν(Qkj, Pkj) + χp
kj∇pη(Qkj, Pkj, Ukj),(4.46d)
Uki ∈U, −χp
ki∇uη(Qki, Pki, Uki) = 0,(4.46e)
1≤i≤sand 0 ≤k≤N−1.
The conditions (4.46a) - (4.46d) are in essence a partitioned Runge-Kutta
scheme applied to the continuous adjoint equations (4.38c) - (4.38f).
Along the lines of [53] it holds, that the transformed first-order system (4.46)
(for hsufficiently small and (xk, Ukj)∈Bβ(x∗(tk), u∗(tk)) for each jand k) and
the Karush-Kuhn-Tucker equations (4.44) are equivalent if bj>0 for each j(see
Appendix B). So far, the costate equations (4.46a) - (4.46d) march backwards
in time. Therefore, we reverse the order of time. Furthermore, we apply again
the control uniqueness property to get rid of the control variables from the state
multiplier equations. The transformed adjoint system now reads as
qk+1 =qk+h
s
X
i=1
biν(Qki, Pki), q0=q0,(4.47a)
pk+1 =pk+h
s
X
i=1
biη(Qki, Pki, χq
ki, χp
ki), p0=p0,(4.47b)
ψq
k+1 =ψq
k+h
s
X
i=1
biφq(Qki, Pki, χq
ki, χp
ki), ψq
N=∇qΦ(qN, pN),(4.47c)
ψp
k+1 =ψp
k+h
s
X
i=1
biφp(Qki, Pki, χq
ki, χp
ki), ψp
N=∇pΦ(qN, pN),(4.47d)
83
Qki =qk+h
s
X
j=1
aq
ijν(Qkj, Pkj),(4.47e)
Pki =pk+h
s
X
j=1
ap
ijη(Qkj, Pkj, χq
kj, χp
kj),(4.47f)
χq
ki =ψq
k+h
s
X
j=1
¯aq
ijφq(Qkj, Pkj, χq
kj, χp
kj),(4.47g)
χp
ki =ψp
k+h
s
X
j=1
¯ap
ijφp(Qkj, Pkj, χq
kj, χp
kj),(4.47h)
¯aq
ij =bibj−bjaq
ji
bi
,¯ap
ij =bibj−bjap
ji
bi
.(4.47i)
Since u(x, ψ) depends Lipschitz continuously on xnear x∗(t) and ψnear ψ∗(t)
for any t∈[0, T], we have, analogous to the state uniqueness property, uniquely
determined intermediate costates, such that there exists a locally unique solution
(Qki, Pki, χq
ki, χp
ki),1≤i≤sof (4.47e) - (4.47h).
The scheme (4.47) can be viewed as a discretization of the two-point boundary-
value problem (4.41). To ensure a desired order of approximation for this two-
point boundary-value problem, Hager derives in [53] constraints on the coefficients
of the Runge-Kutta scheme via Taylor expansion. However, in case of the DMOC
discretizations, we obtain the following result concerning the order of consistency.
Theorem 4.5.5 (Order of consistency) If the symplectic partitioned Runge-
Kutta discretization of the state system is of order κand bi>0for each i, then
the scheme for the adjoint system is again a symplectic partitioned Runge-Kutta
scheme of the same order (in particular we obtain the same schemes for (q, p)
and (ψp, ψq)).
Proof: With the symplecticity condition ap
ij =bibj−bjaq
ji
biit holds
¯aq
ij =ap
ij,
¯ap
ij =aq
ij.
With Theorem 4.3.3 and 4.4.2 it follows immediately:
Lemma 4.5.6 Let a Lagrangian optimal control problem with regular Lagrangian
Land Lagrangian control force fLC be given. If the discrete Lagrangian Ld(4.33)
and the discrete control force f±
Cd(4.34) with bi>0for each iare both of order
κ, then the corresponding adjoint scheme is also of order κ.
84
Remark 4.5.7 From the equations (4.47) we see that the discretization of the
dynamics by a Runge-Kutta map always leads to a symplectic partitioned Runge-
Kutta scheme for the state-adjoint system. In general, a direct solution method
can be viewed as taking discrete variations of the objective functional that is
augmented by the system’s differential equation of motion. This procedure yields
a variational integration of the neccessary optimality conditions. This makes the
resulting scheme symplectic. Direct and indirect methods are therefore linked by
the symplectic integration of the state-adjoint system as illustrated in Figure 4.3.
Optimal control problem
direct methodindirect method
KKT equations
state-adjoint
system symplectic integration
Figure 4.3: Correspondence of indirect and direct methods in optimal control
theory.
4.6 Convergence
In this chapter we show convergence of DMOC, that is solutions of the discrete
optimal control problem (4.9a) - (4.9d) converge to solutions of the Lagrangian
optimal control problem (4.3a) - (4.3c). For example, approximating the optimal
control problem by a discrete Lagrangian and discrete forces of second order we
show that if the functions defining the control problem are smooth enough and
a coercivity condition holds, the error in the discrete approximation is O(h) if
the optimal control is Lipschitz continuous, o(h) if the derivative of the optimal
control is Riemann integrable, and O(h2) if the derivative of the optimal control
has bounded variation.
In recent years, it has become clear that standard convergence theorems fre-
quently employed in the analysis of differential equations are not applicable to
discretizations of optimal control problems. For example, Hager [53] has shown
85
that Runge-Kutta methods converging in a purely differential equation setting
can diverge in an optimal control setting. On the other hand Betts et al. [13]
demonstrate that Runge-Kutta methods that do not converge for the simulation
of differential equations can converge if applied to optimal control problems.
Therefore, during the last years, convergence of optimal control problems has
been investigated for a variety of different transcription methods and modifica-
tions. Schwartz and Polak [137] consider a nonlinear optimal control problem
with control and endpoint constraints and analyze consistency of explicit Runge-
Kutta approximations. Convergence is proved for the global solution of the dis-
crete problem to the global solution of the continuous problem. Malanowski,
B¨uskens and Maurer [107] analyze Euler discretizations of a nonlinear problem
with mixed control and state constraints. Reddien [131] discusses collocation
at Gauss points and derives convergence rates for unconstrained optimal control
problems, assuming positive definiteness of the Hamiltonian for all points (func-
tions) in a bounded neighborhood of the true solution. Kameswaran and Biegler
[70] present convergence results along with an adjoint estimation procedure for
direct transcription of unconstrained optimal control problems using collocation
at Radau points. In [71] they prove convergence and convergence rates in the
presence of final-time equality constraints.
As in Section 4.5, we restrict ourselves to the case without path and control
constraints and without final point constraint. Our convergence statement is
based on that of Hager in [41] and [53], who proved convergence for standard
Runge-Kutta discretizations. Therefore, in this section, we give only a brief
summary the proof strategy. In Appendix C a more detailed explanation of the
single steps is presented.
The proof strategy is as follows: First, the necessary optimality conditions for
the continuous and the discrete optimal control problem are derived. Secondly,
we have to show convergence of solutions of the resulting discrete scheme to
solutions of the corresponding boundary value problem. Preparatory work for
this step was already done in the previous section. There we derived the adjoint
system and determined the order of consistency. A standard strategy to show
convergence of a discrete scheme is to prove consistency and stability. This is
abstractly stated within a result by Hager (see for example [39]), which Hager’s
and our convergence proof are based on:
Proposition 4.6.1 Let Xbe a Banach space and let Ybe a linear normed space
with the norm in both spaces denoted by k · k. Let F:X 7→ 2Ybe a set-valued
map, let L:X 7→ Y be a bounded, linear operator, and let T:X 7→ Y with
Tcontinuously Frech´et differentiable in Br(w∗)for some w∗∈ X and r > 0.
Suppose that the following conditions hold for some δ∈ Y and scalars , λ and
σ > 0:
86
(P1) T(w∗) + δ∈ F(w∗).
(P2) k∇T (w)− Lk ≤ for all w∈Br(w∗).
(P3) The map (F −L)−1is single-valued and Lipschitz continuous in Bσ(π), π =
(T − L)(w∗), with Lipschitz constant λ.
If λ < 1,r ≤σ, and kδk ≤ (1 −λ)r/λ, then there exists a unique w∈Br(w∗)
such that T(w)∈ F(w). Moreover, we have the estimate
kw−w∗k ≤ λ
1−λkδk.(4.48)
Proof: See for example [54].
Proposition 4.6.1 says roughly
Consistency + Stability ⇒Convergence,
where consistency corresponds to assumption (P1) and the bounds on the norm
of δ, stability corresponds to assumption (P3) and the bound on the Lipschitz
constant λfor the linearization, and convergence is stated in (4.48).
The main result is formulated in terms of the averaged modulus of smoothness
of the optimal control (see [53]). If J⊂Ris an interval and v:J→Rn, let
ω(v, J;t, h) denote the modulus of continuity:
ω(v, J;t, h) = sup{|v(s1)−v(s2)|:s1, s2∈[t−h/2, t +h/2] ∩J}.(4.49)
The averaged modulus of smoothness τof vover [0, T] is the integral of the
modulus of continuity:
τ(v;h) =
T
Z0
ω(v, [0, T]; t, h) dt.
It is shown in [139] that limh→0τ(v;h) = 0 if and only if vis Riemann integrable,
and τ(v;h)≤ch with constant cif vhas bounded variation. Our main result
based on the convergence theorem in [53] (Theorem 2.1) is stated below in the
context of unconstrained Lagrangian optimal control problems.
Theorem 4.6.2 (Convergence) If smoothness and coercivity hold, the discrete
Lagrangian Ld(4.33) and the discrete control force f±
Cd(4.34) are of order κwith
bi>0for each i, and U=Rm, then for all sufficiently small h, there exists
a strict local minimizer (xh, uh)of the discrete optimal control problem (4.9a) -
87
(4.9d) and an associated adjoint variable ψhsatisfying (4.45a) and (4.45b) such
that
max
0≤k≤Nxh
k−x∗(tk)+ψh
k−ψ∗(tk)+u(xh
k, ψh
k)−u∗(tk)
≤chκ−1h+τdκ−1
dtκ−1u∗;h,(4.50)
where u(xh
k, ψh
k)is a local minimizer of the Hamiltonian (4.40) corresponding to
x=xkand ψ=ψk. Furthermore, (x∗(t), u∗(t)) = (q∗(t), p∗(t), u∗(t)) is the
optimal solution of the Lagrangian optimal control problem (4.3a) - (4.3c).
Remark 4.6.3 Note that the estimate for the error in the discrete control in
(4.50) is expressed in terms of u(xh
k, ψh
k) not uk. This is due to the fact, that we
derive the estimate via the transformed adjoint system with removed control due
to the control uniqueness property. In [41] the estimate is proved in terms of uk
for Runge-Kutta discretization of second order.
Proof of Theorem 4.6.2 With Theorem 4.3.3 and 4.4.2 we know that a dis-
crete Langrangian and discrete forces both of order κlead to a symplectic par-
titioned Runge-Kutta discretization of order κfor the optimal control system.
Because of smoothness and coercivity we can build up a discrete adjoint scheme
for the optimal control problem with eliminated control that approximates the
continuous adjoint scheme with order κ(see Lemma 4.5.6). This leads in Hager’s
notation to a Runge-Kutta scheme of order κfor optimal control, and therefore
Hager’s convergence result for standard Runge-Kutta schemes in [53] (Theorem
2.1) is directly applicable (see Appendix C).
Remark 4.6.4 Theorem 4.6.2 can be extended to optimal control problems with
constraints on the control function u(t) as it was done in [53] for Runge-Kutta
discretizations of order 2.
88
Chapter 5
Implementation, applications and
extension
In this chapter, we first explain in Section 5.1 how to implement DMOC ef-
ficiently using existing routines for the optimization and differentiation of the
problem. Additionally, we demonstrate the capability of DMOC to solve optimal
control problems resulting from real applications. In Section 5.2 we start with
simple systems to numerically verify the preservation and convergence properties
of DMOC. We demonstrate the benefits of using DMOC compared to other stan-
dard methods for the solution of optimal control problems. In Sections 5.3 and
5.4 we consider more challenging problems from trajectory planning and multi-
body systems. Here, we use a variety of different formulations for the objective
functionals, the boundary conditions, and the path constraints. In Section 5.5
we present an application from space mission design to introduce a spatial decen-
tralized approach to the solution of the corresponding optimal control problem.
5.1 Implementation
As a balance between accuracy and efficiency we employ the midpoint rule for ap-
proximating the relevant integrals for the example computations in the following
section, that is we set
Cd(qk, qk+1, uk) = hC qk+1 +qk
2,qk+1 −qk
h, uk+1/2,
Ld(qk, qk+1) = hL qk+1 +qk
2,qk+1 −qk
h,
89
k= 0, . . . , N −1, as well as
Z(k+1)h
kh
fLC(q(t),˙q(t), u(t)) ·δq(t) dt≈
hfLC qk+1 +qk
2,qk+1 −qk
h, uk+1/2·δqk+1 +δqk
2=
h
2fLC qk+1 +qk
2,qk+1 −qk
h, uk+1/2·δqk
+h
2fLC qk+1 +qk
2,qk+1 −qk
h, uk+1/2·δqk+1.
Here, f−
k=f+
k=h
2fLC qk+1+qk
2,qk+1−qk
h, uk+1/2are used as the left and right
discrete forces with qk=q(tk) and uk+1/2=utk+tk+1
2.
SQP Method We solve the resulting finite dimensional constrained optimiza-
tion problem by a standard SQP method (see Section 2.3.2) as implemented for
example in the routine fmincon of MATLAB. For more complex problems with
higher dimensions we use the routine nag opt nlp sparse of the NAG library1.
SQP is a local optimization method. That means if the problem has more than
one local optimum, different guesses for initializing the algorithm can lead to
different optimal solutions. This can be observed for almost all our example
computations.
Automatic Differentiation To compute an optimal solution of the optimiza-
tion problem the SQP method makes use of the first and second derivatives of
the constraints and the objective function. In the case where no derivatives are
provided, the used algorithms approximate those by finite differences. This ap-
proximation is time-consuming and round-off errors in the discretization process
occur leading to worse convergence behavior of the algorithm.
To avoid these drawbacks we make use of the concept of Automatic Differenti-
ation (AD) (see [51, 130, 160] for a basic introduction), a method to numerically
evaluate the derivative of a function specified by a computer program. AD ex-
ploits the fact that any computer program that implements a vector function
y=F(x) (generally) can be decomposed into a sequence of elementary assign-
ments, any one of which may be trivially differentiated by a simple table lookup.
These elemental partial derivatives, evaluated at a particular argument, are com-
bined in accordance with the chain rule from differential calculus to yield informa-
tion on the derivative of F(such as gradients, tangents, and the Jacobian matrix).
1www.nag.com
90
This process yields (to numerical accuracy) exact derivatives. Because symbolic
transformations occur only at the most basic level, AD avoids the computational
problems inherent to complex symbolic computation.
In particular, for some of our optimal control problems, we implemented the
package ADOL-C (Automatic Differentiation by OverLoading in C++, [157])
that has been written primarily for the evaluation of gradient vectors (rows or
columns of Jacobians). It turns out that the optimization process performs in a
faster and more robust way when providing the derivatives by automatic differ-
entiation rather than via finite differences.
5.2 Comparison to existing methods
In this section, we apply DMOC and other standard methods to the solution of
two optimal control problems: the low thrust orbital transfer and the optimal
control of a two-link manipulator. In this way, we compare different numeri-
cal properties and results indicating the quality of performance of the different
algorithms.
5.2.1 Low thrust orbital transfer
In this application we investigate the problem of optimally transferring a satellite
with a continuously acting propulsion system from one circular orbit around the
Earth to another one.
Consider a satellite with mass mwhich moves in the gravitational field of the
Earth (mass M). The satellite is to be transferred from one circular orbit to one
in the same plane with a larger radius, while the number of revolutions around
the Earth during the transfer process is fixed. In 2d-polar coordinates q= (r, ϕ),
the Lagrangian of the system has the form
L(q, ˙q) = 1
2m( ˙r2+r2˙ϕ2) + γMm
r,
where γdenotes the gravitational constant. Assume that the propulsion system
continuously exhibits a force uin the direction of motion of the satellite, such
that the corresponding Lagrangian control force is given by
fLC =0
r u .
Boundary conditions Assume further that the satellite initially moves on a
circular orbit of radius r0. Let (r(0), ϕ(0)) = (r0,0) be its position at t= 0, then
91
its initial velocity is given by ˙r(0) = 0 and ˙ϕ(0) = pγM/(r0)3. Using its thruster,
the satellite is required to reach the point (rT,0) at time T=d(T0+TT)/2 and,
without any further control input, to continue to move on the circle with radius
rT. Here, dis a prescribed number of revolutions around the Earth and T0
and TTare the orbital periods of the initial and the final circle, respectively.
Thus, the boundary values at t=Tare given by (r(T), ϕ(T)) = (rT,0) and
( ˙r(T),˙ϕ(T)) = (0,pγM/(rT)3).
Objective functional During this transfer, our goal is to minimize the control
effort, correspondingly the objective functional is given by
J(q, ˙q, u) = ZT
0
u(t)2dt.
Results The computations are performed with the following parameter values:
m= 100, M = 6 ·1024,
γ= 6.673 ·10−26, r0= 5, rT= 6,
T0= 2πp(r0)3/(γM), TT= 2πp(rT)3/(γM).
We compare DMOC to a simple finite difference approach, where the dynamical
constraints are discretized by applying a one-step method to the associated ordi-
nary differential equations of the system (the forced Euler-Lagrange equations).
For demonstration purposes, the forward Euler scheme is used, in which case the
constraints read
xk+1 −xk−h F(xk, uk) = 0, k = 0, . . . , N −1,
(where xk= (qk,˙qk) and Fdenotes the vector field of the forced Euler-Lagrange
equations), as well as the midpoint rule, yielding the constraints
xk+1 −xk−h F xk+1 +xk
2,uk+1 +uk
2= 0,
k= 0, . . . , N −1.
We consider d= 1 and d= 2 revolutions around the Earth and solve the
problem for various N. In Figure 5.1 the dependence of the resulting cost on
Nfor all methods as well as for d= 1 (top) and d= 2 (bottom) is shown. It
is intriguing to see that the cost is almost constant for the variational approach
(DMOC), even for very large step sizes, whereas the cost of the Euler-based
method seems to converge towards this “benchmark value” for larger N. The
midpoint rule performs, as one might have expected, almost equally well as the
92
10 15 20 25 30 35 40 45 50
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22
0.24
number of nodes
objective function value
Euler
DMOC
5 10 15 20 25 30 35 40 45 50
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
number of nodes
objective function value
midpoint rule
DMOC
a) b)
10 20 30 40 50 60 70 80 90 100
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
number of nodes
objective function value
Euler
DMOC
10 20 30 40 50 60 70 80 90 100
0.0245
0.025
0.0255
0.026
0.0265
0.027
0.0275
0.028
0.0285
number of nodes
objective function value
midpoint rule
DMOC
c) d)
Figure 5.1: Variational approach (DMOC) vs. a finite difference based discretiza-
tion (Euler’s scheme and midpoint rule): approximated cost of the orbital transfer
in dependence of the number Nof discretization points for one [a) and b)] and
two [c) and d)] revolutions around the Earth.
93
DMOC discretization. As a second numerical test we investigate how well the
balance between the change in angular momentum and the amount of the control
force is preserved. Due to the invariance of the Lagrangian under the rotation
ϕ, the angular momentum of the satellite is preserved in the absence of exter-
nal forces (as stated in Noether’s theorem). However, in the presence of control
forces, equation (3.19) gives a relation between the forces and the evolution of the
angular momentum. In Figure 5.2, we compare the amount of the acting force
with the change in angular momentum in each time interval. For the solution
resulting from DMOC, the change in angular momentum equals exactly the sum
of the applied control force (to numerical accuracy). The Euler scheme and the
midpoint rule fail to capture the change in angular momentum accurately. These
results are consistent with the well-known conservation properties of variational
integrators. However, note that for the DMOC discretization, the resulting op-
timization problem is only half as large as for the finite difference schemes. As
a third numerical test, we investigate how well the computed open loop control
performs for “the real solution”. To this end, the forced Euler-Lagrange equa-
tions are integrated using the classical fourth order Runge-Kutta scheme with
very small constant step size h= 10−3, where the computed control values are
interpolated by a cubic spline. In Figure 5.3 the deviation of the resulting final
state (at t=T) from the requested one is shown. Again, DMOC and the mid-
point rule perform similar in the order of accuracy, as they are schemes of the
same order. As accepted the Euler scheme lacks accuracy, especially for larger
step sizes.
5.2.2 Two-link manipulator
As a second numerical example we consider the optimal control of a two-link ma-
nipulator. We compare solutions obtained by applying DMOC to those resulting
from a collocation method of the same order. To this end, we apply the colloca-
tion method to the Hamiltonian system, expressed in the coordinates qand pand
to the system formulated in state space with the coordinates qand ˙q, denoted by
v.
Model The two-link manipulator (see Figure 5.4) consists of two coupled planar
rigid bodies with mass mi, length liand moment of inertia Ji,i= 1,2, respec-
tively. For i∈1,2, we let θidenote the orientation of the ith link measured
counterclockwise from the positive horizontal axis. If we suppose one end of the
first link to be fixed in an inertial reference frame, the configuration of the system
is specified by q= (θ1, θ2).
94
5 10 15 20 25 30 35 40 45 50
0
5
10
15
20
25
30
35
40
number of nodes
difference of force and change in angular momentum
Euler
DMOC
5 10 15 20 25 30 35 40 45 50
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
0.045
0.05
number of nodes
difference of force and change in angular momentum
midpoint rule
DMOC
a) b)
10 20 30 40 50 60 70 80 90 100
0
0.5
1
1.5
2
2.5
3
3.5
number of nodes
difference of force and change in angular momentum
Euler
DMOC
10 20 30 40 50 60 70 80 90 100
0
0.005
0.01
0.015
0.02
0.025
0.03
number of nodes
difference of force and change in angular momentum
midpoint rule
DMOC
c) d)
Figure 5.2: Variational approach (DMOC) vs. a finite difference based discretiza-
tion (Euler’s scheme and midpoint rule): difference of force and change in angular
momentum of the orbital transfer in dependence on the number Nof discretiza-
tion points for one [a) and b)] and two [c) and d)] revolutions around the Earth.
95
5 10 15 20 25 30 35 40 45 50
0
1
2
3
4
5
6
7
8
number of nodes
deviation of final state
Euler
midpoint rule
DMOC
Figure 5.3: Comparison of the accuracy of the computed open loop control for
DMOC, the Euler-based approach, and the midpoint rule: Deviation of the actual
final state of the satellite from the requested one in dependence of the number of
discretization points.
θ1
θ2
τ2
τ1
Figure 5.4: Model of a two-link manipulator.
96
The Lagrangian is given via the kinetic and potential energy
K(q, ˙q) = 1
8(m1+ 4m2)l2
1˙
θ2
1+1
8m2l2
2˙
θ2
2
+1
2m2l1l2cos (θ1−θ2)˙
θ1˙
θ2+1
2J1˙
θ2
1+1
2J2˙
θ2
2
and
V(q) = 1
2m1gl1sin θ1+m2gl1sin θ1+1
2m2gl2θ2,
with the gravitational acceleration g. Control torques τ1, τ2are applied at the
base of the first link and at the joint between the two links. This leads to the
Lagrangian control force
fLC(τ1, τ2) = τ1−τ2
τ2.
Boundary conditions The two-link manipulator is to be steered from the
stable equilibrium point q0= (−π
2,−π
2) with zero angular velocity ˙q0= (0,0) to
the instable equilibrium point qT= (π
2,π
2) with velocity ˙qT= (0,0).
Objective functional For the motion of the manipulator the control effort is
to be minimized as
J(τ1, τ2) =
T
Z0
1
2(τ2
1(t) + τ2
2(t)) dt
with a final time T= 1.
Results As for the orbital transfer in Section 5.2.1 we solve the problem for
various N. In Figure 5.5 the dependence on Nof a) the resulting cost and b)
the difference of the amount of force (including the control and the gravitational
force) and the change in angular momentum for all methods is shown. The collo-
cation method of order 2 corresponds to a discretization via the implicit midpoint
rule. Thus, the optimization problem resulting from DMOC is equivalent to that
obtained by applying collocation to the Hamiltonian formulation of the system as
demonstrated in Example 4.4.3 a). Obviously, we obtain exactly equal solutions
by applying both methods. The midpoint rule applied to the system formu-
lated on tangent space performs, as one might have expected, equally well with
respect to the objective value evolution depending on the step size. However,
it does not reflect the momentum-force consistency as good as the other me-
thods as shown in Figure 5.5. In Figure 5.6 the convergence rates are depicted.
97
3 4 5 6 7 8 9
81
82
83
84
85
86
87
log2(N=2k)
objective value
DMOC
collocation (q,p)
collocation (q,v)
3 4 5 6 7 8 9
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
log2(N=2k)
difference of force and change in angular momentum
DMOC
collocation (q,p)
collocation (q,v)
a) b)
Figure 5.5: Comparison of the accuracy of the computed open loop control for
DMOC and a collocation approach: a) approximated cost b) difference of force
and change in angular momentum in dependence on the number of discretization
points.
Table 5.1: Comparison of CPU times in seconds
number of nodes 8 16 32 64 128 256 512
CPU time (DMOC) 0.02 0.05 0.1 0.18 0.71 4.26 45.22
CPU time (collocation (q, p)) 0.03 0.08 0.22 0.82 2.64 7.69 67.71
CPU time (collocation (q, v)) 0.04 0.18 0.24 2.01 2.31 16.34 89.29
Here, a reference trajectory is computed with N= 512 discretizations points and
time step h= 1.9·10−3. The error in the configuration and control parame-
ter of the discrete solution with respect to the reference solution is computed as
maxk=0,...,N |q(tk)−qref(tk)|and maxk=0,...,N |u(tk)−uref(tk)|, respectively, where
|·|is the Euclidean norm. For all three methods the convergence rate for the
configuration and control trajectory is O(h2), as expected for a scheme of second
order accuracy.
However, DMOC performs much faster than the collocation method (compare
the CPU times in Table 5.1). This is due to the formulation on the discrete
tangent space Q×Qthat yields an optimization problem with less variables and
less constraints as stated in Remark 4.4.4.
98
123456
1
2
3
4
5
6
7
8
9
10
!log(h)
!log(max(q!qref))
2
1
DMOC
collocation (q,p)
collocation (q,v)
123456
!1
0
1
2
3
4
5
6
7
!log(h)
!log(max(u!uref))
2
1
DMOC
collocation (q,p)
collocation (q,v)
a) b)
Figure 5.6: Comparison of the convergence rates in dependence of the time step
hfor DMOC and a collocation approach. a) Error of configuration trajectory. b)
Error of control trajectory.
5.3 Application: Trajectory planning
Trajectory planning represents a huge application area for optimal control the-
ory. In this section, we apply DMOC to two typical problems from this field. We
consider for both applications an additional specific feature concerning the prob-
lem formulation. Rather than fixing the final state via a final point constraint,
we formulate a target manifold for a group of hovercraft that allows the system
to adopt a final state with certain degrees of freedom. As a second example we
consider the motion of a perfect underwater glider. Here, we treat the maneuver
time (the final time of the movement) as an additional free variable that has
to be optimized. In addition, we formulate an objective functional consisting
of the weighted sum of two different objectives and investigate the problem in
dependence on the weighting of both objectives.
5.3.1 A group of hovercraft
We consider a group of (identical) hovercraft which, starting from an arbitrary
initial state, are required to attain a given final formation with minimal control
effort. The final formation is defined by a set of equality constraints on the final
configurations of the individual hovercraft and a fixed final velocity.
Model The configuration of a hovercraft is described by three degrees of free-
dom: its position (x, y)∈R2and its orientation θ∈S1, that is, its configuration
99
r
x
y
u1
u2
θ
Figure 5.7: Hovercraft model.
manifold is Q=R2×S1. It is actuated by a control force u= (u1, u2), applied
at a distance rfrom the center of mass (see Figure 5.7). The force u1acts in the
direction of motion of the body, while u2acts orthogonally to it. The system is
underactuated, but still configuration controllable [123], that means each point
in configuration space can be reached by applying suitably chosen forces u1(t)
and u2(t).
The Lagrangian of the system consists only of its kinetic energy,
L(q, ˙q) = 1
2(m˙x2+m˙y2+J˙
θ2),
where q= (x, y, θ), mis the mass of the hovercraft and Jits moment of inertia.
The Lagrangian control forces acting in x-, y- and θ-direction resulting from u1
and u2are
fLC(θ(t), u(t)) =
cos θ(t)u1(t)−sin θ(t)u2(t)
sin θ(t)u1(t) + cos θ(t)u2(t)
−ru2(t)
.
We denote by qi= (xi, yi, θi) the configuration of the i-th hovercraft and by
ui= (ui
1, ui
2) the corresponding forces.
100
Objective functional and boundary conditions The goal is to minimize
the control effort while, at the same time, attaining the desired final formation.
As a suitable objective functional for each hovercraft we choose a measure of the
control effort
J(qi, ui) = ZT
0ui
1(t)2+ui
2(t)2dt, (5.1)
while the objective functional for the entire group is given by the sum of these.
The final configuration of the group has certain degrees of freedom: In the
case of three hovercraft the final formation has an overall rotational degree of
freedom; it is determined by the following conditions:
(a) a fixed final orientation ϕiof each hovercraft:
θi(T) = ϕi, i = 1,2,3,
(b) equal distances dbetween the final positions:
(xi(T)−xj(T))2+ (yi(T)−yj(T))2=d2,1≤i, j ≤3, i 6=j;
(c) the center M= (Mx, My) of the formation is prescribed:
(x1(T) + x2(T) + x3(T))/3 = Mx,
(y1(T) + y2(T) + y3(T))/3 = My,
(d) fixed final velocities:
˙xi=vi
x,˙yi=vi
y,˙
θi=vi
θ, i = 1,2,3,
where vi
x, vi
yand vi
θare given.
The boundary conditions for the case of a group of six hovercraft are determined
analogously, that is, the craft are required to form a regular hexagon.
Results The group of three hovercraft starts from an initial configuration along
a line, the group of six craft from a random initial configuration and with zero
initial velocity for each craft, respectively. The group has to optimally arrange
into an equilateral triangle, or into a hexagon, respectively. Using different initial
guesses different orders of arrangement on the target manifold are adopted by
the group. To obtain the solution with the lowest costs we perform several runs
of the optimization with a variety of different initial guesses. In Figure 5.8 the
configuration trajectories for a formation of three and six hovercraft, respectively,
are illustrated. As the rotational and the translational motion is coupled via the
influence of the control force u, the hovercraft have to perform on large curves to
reach the desired final orientation ϕi.
101
Figure 5.8: Left: Optimal rearrangement of a group of three hovercraft from an
initial configuration along a line into an equilateral triangle. Right: Optimal
rearrangement of a group of six hovercraft from a random initial configuration
into a hexagon.
5.3.2 Perfect underwater glider
As an application with free final time we consider a class of Autonomous Un-
derwater Vehicles (AUVs) known as gliders. AUVs are becoming increasingly
popular for collecting scientific data in the ocean because they are low cost and
highly sustainable (see for example [4, 45, 135]). However, it is desirable to mi-
nimize the amount of energy the gliders use for transport in order to keep the
gliders autonomously operational for the greatest amount of time. Therefore, it
is advantageous to make use of ocean currents to help propel the gliders around
the ocean for sustainable missions (see for example [62]).
The problem considered here is to find an optimal trajectory of a glider that
needs to move from one location to another within a prescribed current. Here,
the glider is assumed to be actuated by a gyroscopic force which implies that
the relative forward speed of the glider is constant. However, the orientation of
the glider cannot change instantly and the control force induces the change in
the orientation of the glider. In addition to the minimization of the amount of
control effort, our goal is to identify trajectories that are also time-optimal, such
that the glider needs as little time as possible to reach the final destination.
Dynamic model As in [161] the glider is modeled as a pointmass in R2and
actuated by a gyroscopic force acting orthogonal to the relative velocity between
fluid and body given by
Fgyr(q(t),˙q(t), u(t), t) = −mu(t) ( ˙y(t)−Vy(t))
mu(t) ( ˙x(t)−Vx(t)) (5.2)
102
with the glider mass m, the configuration q(t) = (x(t), y(t)), the absolute glider
velocity ˙q(t) = ( ˙x(t),˙y(t)), the current velocity field V(t)=(Vx(t), Vy(t)), and
the control input u(t)∈Rrepresenting the change in the orientation.
By introducing ˙qrel(t) = ( ˙q(t)−V(t)) as relative velocity, the Lagrangian in
the body fixed frame consists of the kinetic energy of the relative motion of the
glider, therefore
L(qrel(t),˙qrel(t)) = 1
2˙qrel(t)0M˙qrel(t) (5.3)
with constant Mass matrix M=m0
0mand qrel(t) = q(t). The correspond-
ing gyroscopic force acting on the system is then given by
fLC(qrel(t),˙qrel(t), u(t)) = −mu(t) ˙qrel,y(t)
mu(t) ˙qrel,x(t).
The resulting Euler-Lagrange equations read as
¨x(t) = −u(t)( ˙y(t)−Vy(t)) + ˙
Vx(t),
¨y(t) = u(t)( ˙x(t)−Vx(t)) + ˙
Vy(t).
Objective functional The perfect glider has to be steered within the time
span [0, T] with free final time Tfrom an initial configuration q(0) = q0to a final
one q(T) = qT, optimally with respect to the objective function
J=w1ZT
0
kf(qrel(t),˙qrel(t), u(t))k2dt+w2T.
So, the aim is to minimize the control effort and duration time at once by using a
weighted objective function of both goals with weights w1>0 and w2>0. In the
discrete setting we model the free final time by a variable step size hthat acts as
an additional optimization variable bounded as 0 < h ≤hmax to ensure positive
step size and solutions of desired accuracy. For a fixed number of discretization
points the final time is then given by T= (N−1)h.
Boundary conditions As initial constraint we assume a prescribed initial con-
figuration qrel(0) = (10,0) and an initial relative velocity as ˙qrel(0) = (−10,−10).
The final configuration is given by qrel(T) = (15,2), while the final relative ve-
locity is free with same magnitude as the initial one, as the control force only
influences the orientation, rather than the magnitude of the relative velocity.
103
Results For weights w1, w2∈(0,1) with w1+w2= 1 we compute the corre-
sponding optimal solutions. As a first example, the current velocity of the fluid is
assumed to be zero. In a second computation, we impose a current velocity with
configuration dependent x- and zero y-component as V= (x, 0). Figure 5.9 shows
the trajectories of the configuration and the controls for both problems. For both
cases the initial velocity is directed away from the destination. Therefore, for
the case of zero fluid velocity, the gyroscopic control force enforces the glider
performing a circular motion starting in direction of the initial velocity towards
the destination as depicted in Figure 5.9. For increasing weight of the final time
objective and decreasing weight of the control effort objective, the circular motion
becomes smaller and the corresponding control parameter adopts higher values
as shown in Figure 5.9 b). Concerning the behavior dependent on the weights,
we observe the same behavior for nonzero fluid velocity (see Figure 5.9 c) and
d)). However, due to the fluid velocity in x-direction, the gilder moves along
different trajectories to reach the desired final location as depicted in Figure 5.9
c). Since the gyroscopic force is acting orthogonal to the relative velocity of the
glider, that means the work resulting from the control force is zero, the energy
of the system should be constant. Figure 5.10 shows that DMOC nicely captures
the preservation of constant energy a) and zero work b). In Figure 5.10 c) the
solutions for different weights are shown. As expected, the control effort increases
for decreasing maneuver time. However, the fluid velocity under consideration
does not help propel the glider towards the desired final location: For the same
maneuver time the control effort for a glider moving in zero current velocity is
lower compared to the solutions for a glider moving in a current with velocity
V= (x, 0).
5.4 Application: Optimal control of multi-body
systems
In this section, we apply DMOC to simple multi-body systems. To this end,
we use a formulation in generalized coordinates as opposed to the constrained
formulation of a multi-body system introduced in Chapter 6.
5.4.1 The falling cat
A nice example to demonstrate the structure-preserving properties of DMOC is
the falling cat that performs from an inverted orientation an overall rotation of
π, landing back on its feet in an optimal way. For this the cat flips itself side up,
even though its angular momentum is zero. It does this by changing its shape.
104
!5 0 5 10 15 20 25
!40
!35
!30
!25
!20
!15
!10
!5
0
5
x
y
0 1 2 3 4 5 6 7
0
1
2
3
4
5
6
7
8
9
10
t
control paramter u
a) b)
6 8 10 12 14 16
!5
0
5
10
15
20
25
30
x
y
0 1 2 3 4 5 6 7
!10
!8
!6
!4
!2
0
2
4
6
8
t
control paramter u
c) d)
Figure 5.9: Optimal trajectories of the motion of an underwater glider in de-
pendence of the weights w1and w2for the maneuver time and the control ef-
fort in the objective function. Top: The glider moves in zero current velocity
V= (0,0). a) Optimal trajectories in configuration space, b) optimal control
parameter. Bottom: The glider moves in current velocity V= (x, 0). c) Op-
timal trajectories in configuration space, d) optimal control parameter. Red:
w1= 0.9, w2= 0.1. Blue: w1= 0.8, w2= 0.2. Black: w1= 0.7, w2= 0.3. Cyan:
w1= 0.6, w2= 0.4. Mangenta: w1= 0.5, w2= 0.5. Yellow: w1= 0.4, w2= 0.6.
Green: w1= 0.3, w2= 0.7. Red: w1= 0.2, w2= 0.8. Blue: w1= 0.1, w2= 0.9.
105
0 1 2 3 4 5 6
99.999
99.9992
99.9994
99.9996
99.9998
100
100.0002
100.0004
100.0006
100.0008
100.001
t
energy
0123456
!1
!0.8
!0.6
!0.4
!0.2
0
0.2
0.4
0.6
0.8
1x 10!5
t
virtual work
a) b)
02468
0
1000
2000
3000
4000
5000
6000
7000
T
control effort
without current velocity
with current velocity
c)
Figure 5.10: a) The total energy of the underwater glider is constant. This corre-
sponds to the b) zero work that is done by the control force since the gyroscopic
force is acting orthogonal to the relative velocity of the glider. c) Objective values
(maneuver time and control effort) in dependence on the weights in the objective
function.
106
Remark 5.4.1 In 1991, Montgomery investigated the associated optimal control
problem, which resulted in Montgomery’s falling cat theorem [118, 119, 120, 121]
that relates the optimal reorientation of the falling cat to the dynamics of particles
in a Yang-Mills field.
Model As in [120], the falling cat is modeled as two identical cylinders (which
can be thought of as the upper torso and the lower torso), which are attached at
a joint in the center of the cat’s body. The reference configuration for this system
is shown in Figure 5.11.
x
y
θf
θb
ψ
2
Figure 5.11: Falling cat model.
Each cylinder is allowed to rotate around its symmetric axis by an angle θf
for the front body half, and θbfor the back body half. When θb=θf= 0, the cat
is oriented upright and facing along the x-axis. The cat is also allowed to bend
at its center joint. The parameter ψdescribes the angle between the symmetric
axes of the two cylinders. The cat is completely straight when ψ=π, and the cat
is completely folded onto itself when ψ= 0. Additionally, the system as a whole
is given an orientation g∈SO(3), which represents the orientation about the
center of mass between the line containing the system’s center of mass and the
hinge point, and the y-axis in the reference configuration shown in Figure 5.11.
Each cylinder has mass mand radius a. The distance from the joint to each of
the individual cylinder’s center of mass is denoted by r. The cylinders’ moment
of inertia tensors have the form I= diag(J1, J1, J3). Finally, torques are applied
independently to the internal variables θf,θband ψto affect the dynamics of the
107
cat and ultimately enable the desired flipping motion.2
We restrict ourselves to the no-twist model, in which the cat is allowed to
bend in the direction ψ, and twist the front and back halves of its body in equal
proportions; that is, θ=θf=θb. This assumption reduces the global orientation
of the system to a subset g∈S1⊂SO(3).
Assuming that the cat is in free fall, we can treat the system as an articulated
body rotating about its center of mass, with the Lagrangian equalling the total
system kinetic energy consisting of linear and rotational kinetic energy (for a
derivation of the Lagrangian we refer to [115]):
L(q, ˙q) = Klin(q, ˙q) + Krot(q, ˙q)
=J3˙
θ2+1
2(J1+J3)˙g2+1
8(2J1+mr2)˙
ψ2
+1
84(J1−J3)˙g2+mr2˙
ψ2cos ψ−2˙g˙
θJ3sin ψ
2,
with q= (θ, ψ, g). The control force as described above reads
fLC(τ1, τ2) =
τ1
τ2
0
.
Since the Lagrangian is invariant under rotations around its center of mass (that
is φ(g, q) = g·qrepresents the group action that is a symmetry of the Lagran-
gian) and the control force is orthogonal to that group action (there is no torque
influencing directly this degree of freedom), we know from the forced Noether’s
theorem that the angular momentum
∂L
∂˙g= ˙g(J1+J3+ (J1−J3) cos ψ)−2˙
θJ3sin ψ
2
should be zero during the entire motion of the cat.
Boundary conditions We must specify boundary conditions consistent with
the motion of a falling cat. The simplest choice of these parameters is to specify
conditions which correspond to a perfectly straight inverted and correctly oriented
cat
(θ0, ψ0, g0) = (0, π, π) and (θT, ψT, gT) = (0, π, 0).
Additionally, the cat is assumed to start and end with zero angular velocities,
that is
(˙
θ0,˙
ψ0,˙g0) = (0,0,0) and ( ˙
θT,˙
ψT,˙gT) = (0,0,0).
2This dynamical model stands in contrast to Montgomery’s kinematic model, in which the
angular rates of the internal variables are directly controlled.
108
Objective functional We attempt to minimize the control effort required by
the cat to perform the flip, using
J(τ) = 1
2ZT
0τ1(t)2+τ2(t)2dt,
where τ= (τ1, τ2) is the control torque associated with ˙
θand ˙
ψ.
Results One fascinating result that we can demonstrate immediately with this
simple no-twist system is that the DMOC solution actually mimics the behavior
of real cats. Figure 5.12 a) contains image captures of our model at different time
intervals, and these images have a striking resemblance to the image captures
of real live falling cats. In Figure 5.12 b) and c) the orientation angles and
the corresponding control torques are shown, respectively. In Figure 5.13 a) the
energy behavior with zero initial and final kinetic energy is depicted. As stated
above, the cat’s total angular momentum is supposed to be zero, since the cat can
just adjust its orientation due to shape changes. The momentum preservation is
nicely captured by DMOC to numerical accuracy of the algorithm as shown in
Figure 5.13 b).
5.4.2 A gymnast (three-link mechanism)
A further application of the optimal control of a mechanical system is the deter-
mination of optimal motion sequences in sports. As a simple example we consider
a gymnast performing a giant swing around a bar. After one giant we want the
gymnast to have maximal energy to perform the next giant, that means the aim
is to maximize the kinetic energy in the handstand position.
Model description The gymnast is modeled as a multi-body system consisting
of three rigid bodies, the arms, the torso, and the legs, coupled by revolute joints.
Since the gymnast is assumed to stay in one plane during his performance, we
consider a planar model with mass miand length liof body i(see Figure 5.14).
The bar is located at the coordinate origin, θ1is the angle between the vertical
and the arms, θ2determines the angle between arms and torso, and θ3describes
the bending in the hip. Here, we assume that arms and legs are outstretched
during the whole motion. Furthermore, we assume that the gymnast is able to
control the shoulder and the hip muscles to create a desired motion. We model
the influence of the muscle forces by two external control torques acting on the
system: τ1influences the motion of the torso relative to the arms. The second
torque, τ2, enables the gymnast to move his legs relative to the torso.
109
a)
0 2 4 $ % 10
!3
!2
!1
0
1
2
3
4
t
conf-gurat-on
!
"
g
b)
0 2 4 6 8 10
!1.5
!1
!0.5
0
0.5
1
1.5
t
torque
!1
!2
c)
Figure 5.12: Optimal solution trajectory of the falling cat. a) Image captures
of the falling cat: comparison of a real cat with the computed solution. b)
Configuration (rotation θaround its body half’s symmetric axis, bending angle
ψ, overall orientation g) over time. c) Control torques over time.
110
0 2 4 6 8 10
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
t
total energy
0 2 4 6 8 10
!1
!0.8
!0.6
!0.4
!0.2
0
0.2
0.4
0.6
0.8
1x 10!5
t
angular momentum
a) b)
Figure 5.13: a) Energy behavior of the falling cat. b) The total angular mo-
mentum of the cat is zero since it adjusts its orientation due to shape changes.
θ1
θ2
τ2
τ1
θ3
x
y
Figure 5.14: A gymnast performing a swing giant modeled as three-link mecha-
nism.
111
The Lagrangian consisting of kinetic and potential energy of each single rigid
body with gthe gravitational acceleration reads as
L(θ, ˙
θ) = K(θ, ˙
θ)−V(θ) =
1
2m2l2
1+1
6m1l2
1+1
6m2l2
2+1
2m2l1l2cos θ2+1
3m3l2
1+1
2m3l2
2
+1
6m3l2
3+m3l1l2cos θ2+1
2m3l1l3cos (θ1+θ2) + 1
2m3l2l3cos θ3˙
θ2
1
+1
6m2l2
2+m3l2
2+1
3m3l2
3+m3l1l2cos θ2
+1
2m3l1l3cos (θ1+θ2) + m3l2l3cos θ3˙
θ1˙
θ2
+1
3m3l2
3+1
2m3l1l3cos (θ1+θ2) + 1
2m3l2l3cos θ3˙
θ1˙
θ3
+1
6m2l2
2+1
2m3l2
2+1
2m3l2l3cos θ3˙
θ2
2
+1
3m3l2
3+1
2m3l2l3cos θ3˙
θ2˙
θ3+1
6m3l2
3˙
θ2
3
+1
2m1+m2+m3l1gcos θ1+1
2m2+m3l2gcos (θ1+θ2)
+1
2m3l3gcos (θ1+θ2+θ3).
The control torques τ1and τ2yield the virtual work
W(τ, θ) =
T
Z0
τ(t)·δθ(t) dt,
with θ= (θ1, θ2, θ3) and τ= (0, τ1, τ2).
Boundary conditions, objective function, and path constraints The
gymnast starts in the handstand position, θ0= (π, 0,0) with initial angular ve-
locity ω0. We now determine the optimal trajectories (θ, ˙
θ), the optimal control
force τand the optimal duration time Tfor one rotation around the bar, such
that the kinetic energy in the final configuration θ(T) = θT=θ0is maximized,
so the objective function reads as
J(θ, ˙
θ) = K(θ(T),˙
θ(T)).
112
To restrict to realistic motions we need to incorporate bounds on the configuration
variables as θl
i≤θi≤θu
i, i = 1,2,3 that are in accordance with the human body’s
anatomy. In addition, we have bounds on the torques τl
i≤τi≤τu
i, i = 1,2,
representing the possible amount of torque that a muscle force can generate in a
specific joint.
Results The computations are performed with an initial angular velocity ω0=
(1,1,−2) and with bounds on the joint angles and the control torques as θl=
(−∞,−π
12 ,−π
12 ), θu= (∞,11
12 π, 5
6π) and τl= (−60,−70), τu= (60,70). In
Figure 5.15 and 5.16 one particular locally optimal solution is shown. Although
we use a simple model without taking into account the effect of the muscles the
results nicely approximate the motion of a real giant swing:
In general, two phases in performing a motion exist: the hyperextension and
the contraction phase. A posture is called hyperextended if the extension of
a bodily joint is beyond its normal range of motion. This condition effects an
initial tension and allows to build up more muscle strength. The creation of
the necessary muscle strength is the preparation for the contraction phase, also
known as active phase: Here, the muscles are contracted and therefore the angles
in shoulder and hip decrease. This changes the moment of inertia of the system
and allows the gymnast to move his body to the desired position.
Figure 5.16 shows snapshots of the optimal solution for different time points.
The yellow and purple sticks demonstrate orientation and the amount of the
applied control torques in shoulder and hip joint, respectively.
The pictures 1 to 7 show the swing down while the pictures 8 to 11 correspond
to the swing up of the giant. During the swing down we observe in the pictures
2 to 5 a hyperextension of the gymnast’s body, that is followed by a contraction
phase (picture 6) to move the body from a horizontal aligned position (picture 5)
to the vertical position. Then, again a hyperextened posture (picture 7) allows a
real gymnast to build up the necessary muscle strength in continuing his motion
to the desired handstand position (last picture) during another contraction phase.
In Figure 5.15 the evolution of joint angles, applied torques, angular velocities
and energy over time are shown. The angle θ1between arms and bar increases
smoothly, that means the gymnast’s body rotates uniformly around the bar.
The small oscillations of the shoulder θ2and hip angle θ3indicate permanent
changes between a hyperextension and contraction phase that is also observed
in Figure 5.16 as well as for real giant swing motions. Since the kinetic energy
in the final configuration is maximized (K(θ(T),˙
θ(T)) = 504.6485), the control
torques reach the value of the prescribed bounds τland τu, respectively. Here,
the time of the giant swing duration is T= 2.0338 seconds.
Remark 5.4.2 As observed for the previous two applications of multi-body sys-
113
0 0.5 1 1.5 2 2.5
!1
0
1
2
3
4
5
6
7
t
!
!1
!2
!3
0 0.5 1 1.5 2 2.5
!80
!60
!40
!20
0
20
40
60
80
t
!
!1
!2
a) b)
0 0.5 1 1.5 2 2.5
!600
!400
!200
0
200
400
600
800
t
!
!1
!2
!3
0 0.5 1 1.5 2 2.5
!4000
!2000
0
2000
4000
6000
8000
10000
12000
14000
t
energy
E
K
V
c) d)
Figure 5.15: A gymnast performing a swing giant with initial angular velocity
ω0= (1,1,−2), configuration bounds θl= (−∞,−π
12 ,−π
12 ), θu= (∞,11
12 π, 5
6π),
torque bounds τl= (−60,−70), τu= (60,70), final time T= 2.0338 seconds,
and final kinetic energy K(θ(T),˙
θ(T)) = 504.6485. a) Angle between hand and
bar θ1, shoulder angle θ2and hip angle θ3over time t. b) Control torque τ1in the
shoulder and torque τ2in the hip joint over time t. c) Angular velocity ωover
time t. d) Total energy over time t.
114
t= 0.0t= 0.2034 t= 0.4068
t= 0.6101 t= 0.8135 t= 1.0169
t= 1.2203 t= 1.4237 t= 1.6271
t= 1.8304 t= 2.0338
Figure 5.16: Snapshots of the motion sequence of a gymnast’s swing giant: The
swing down and the swing up motion goes along with changes between hyperex-
tension and contraction phases as for real gymnasts that perform a giant.
115
tems, the expressions for the Lagrangians in generalized coordinates typically
become long and complicated for complex multi-body models. In Chapter 6 we
therefore propose an alternative approach to the optimal control of multi-body
systems formulated within a mechanical framework with holonomic constraints.
This novel approach has several advantages, in particular, it simplifies the mathe-
matical expressions for the Lagragian functions substantially.
5.5 Reconfiguration of formation flying space-
craft – a decentralized approach
The following application presents a decentralized approach to the optimal re-
configuration of formation flying spacecraft based on a hierarchical framework
of DMOC developed in [68]. For upcoming space missions like Darwin3, control
strategies have to be devised that enable precise formation flying of a group of
spacecraft. In light of the tight mass budget of these missions it is of great interest
to minimize the propellant consumption in performing the associated maneuvers
(see [35]).
A group of nspacecraft, viewed as one large mechanical system, is placed in
the vicinity of an L2-Halo orbit and is required to adopt a certain configuration
of the spacecraft relative to each other.
To exploit the structure of the given system, which is in fact composed of many
identical subsystems, we develop a decentralized approach for solving the given
(large) optimal control problem. We derive a hierarchical formulation of the op-
timal control problem by exploiting this structure. The hierarchical formulation
is naturally suited for a solution of the associated subproblems in parallel.
Model Each spacecraft is modeled as a rigid body with six degrees of freedom
(position and orientation), so its configuration manifold is SE(3). We assume
that each spacecraft can be controlled in this configuration space by a force-
torque pair (F, τ), acting on its center of mass.
For several reasons (consistent solar illumination characteristics, lack of dis-
turbing perturbations, relative ease of sending and retrieving spacecraft), an at-
tractive region in space for missions like Darwin is in the vicinity of a Libration
orbit around the Earth-Sun L2Lagrange point. Correspondingly, for each space-
craft the dynamical model for the motion of its center of mass is given by the
circular restricted three body problem (CRTBP) [149, 150]: Two large bodies (the
primaries, Sun and Earth in our case) with masses m1and m2rotate on circles
with common angular velocity ωaround their common center of mass. A third
3http://www.esa.int/science/darwin
116
body, the spacecraft, moves within their gravitational potential without affecting
the motion of the primaries. We neglect gravitational forces between the space-
craft. Figure 5.17 a) shows the position of the primaries and the equilibria (the
Lagrange points)L1, . . . , L5in the x-y-plane of a rotating coordinate system. In
Figure 5.17 b) we plot a family of periodic orbits (Halo orbits) in the vicinity of
the L2Lagrange point. This family has been computed by a predictor corrector
method on an initial orbit found by a shooting technique (see [66]). In a normal-
m2
m1
1−µ
µ
0
L1
L2
L3
L4
L5
x
y
0.999 1.0016 1.0042 1.0068 1.0094 1.012
−0.01
0
0.01
−12
−10
−8
−6
−4
−2
0
2
4
x 10−3
L2
x
E
y
z
a) b)
Figure 5.17: a) Rotating coordinate system: location of the primaries and the
Lagrange points [138]. b) Family of periodic orbits in the circular restricted three
body problem in the vicinity of the L2-Lagrange point [66].
ized, rotating coordinate system (see Figure 5.17 a)), the potential energy of the
spacecraft at position x= (x1, x2, x3)∈R3is given by
V(x) = −1−µ
|x−(1 −µ, 0,0)|−µ
|x−(−µ, 0,0)|,(5.4)
where µ=m1/(m1+m2) is the normalized mass. Its kinetic energy is the sum
of
Ktrans(x, ˙x) = 1
2(( ˙x1−ωx2)2+ ( ˙x2+ωx1)2+ ˙x2
3) and Krot(Ω) = 1
2ΩTJΩ,
where Ω ∈R3is the angular velocity and Jthe inertia tensor of the spacecraft.
117
The control problem
Our goal is to compute control laws (F(i)(t), τ(i)(t)), i= 1, . . . , n, for each
spacecraft, such that the group of spacecraft moves from a given initial state
(x(i), p(i),˙x(i),˙p(i))n
i=1 into a prescribed target manifold within a prescribed time
interval. Here, the unit quaternion p(i)∈R4represents the orientation of the i-th
spacecraft, and the unity constraint is incorporated as additional optimization
constraint. In our application context, the target manifold is defined by prescrib-
ing the relative positioning of the spacecraft, their common velocity, as well as a
common orientation.
Boundary conditions For their target state, we require the spacecraft to be
located in a planar regular polygonal configuration with center on a Halo orbit.
Let ν∈R3be a given unit vector representing the “line of sight” of the spacecraft.
The target manifold M⊂TSE(3)nis the set of all states (x(i), p(i),˙x(i),˙p(i))n
i=1
such that
1. all spacecraft are located in a plane with normal ν, that is
hx(i)−x(j), νi= 0, i, j = 1, . . . , n; (5.5)
2. within that plane, the spacecraft are located at equal distances on a circle
with prescribed radius and prescribed center on a Halo orbit. Let r0∈R
be a given radius and ¯x∈R3a certain point on a Halo orbit and let
ν⊥
1⊥ν⊥
2∈R3be two perpendicular unit vectors that are perpendicular to
ν. For i= 1, . . . , n we consider the vector
z(i)= [ν⊥
1ν⊥
2]T(x(i)−¯x)∈R2(5.6)
and require that
h(z(i)) = kz(i)k − r0= 0, i = 1, . . . , n (5.7)
and
k(z) = 0, z = (z(1), . . . , z(n)),(5.8)
with functions h:R2→Rand k:R2n→Rn, where the constraint
(5.7) forces each spacecraft to be at a distance r0from the center and
the constraint (5.8) guarantees an equidistant arrangement.
The idea of this formulation is not to prescribe a particular point on the cir-
cle for each spacecraft but rather to let the optimization process determine
the best possible arrangement.
118
3. all spacecraft have their “line of sight” aligned with ν. For simplicity we
impose a more restrictive condition, namely that each spacecraft is rotated
according to a prescribed unit quaternion pT,(i), that is we require that
p(i)=pT, i = 1, . . . , n; (5.9)
4. all spacecraft have the same prescribed translational velocity,
˙x(i)= ˙xT, i = 1, . . . , n,
where ˙xNwill typically be determined on basis of the Halo orbit under
consideration, and they have zero angular velocity, that is
Ω(i)= 2 ˙p(i)¯p(i)= 0, i = 1, . . . , n,
where ¯p(i)is the conjugate quaternion to p(i).
Objective functional While controlling the formation to reach the target
manifold, we would like to minimize the fuel consumption of the spacecraft. Here,
we consider the objective functional
J(F, τ) =
n
X
i=1
Ji(F(i), τ(i))
=
n
X
i=1 ZT
0
|F(i)(t)|2+|τ(i)(t)|2dt, (5.10)
where Jiis the objective functional for spacecraft iand F(t) = (F(1)(t), . . . , F(n)(t))
and τ(t) = (τ(1)(t), . . . , τ(n)(t)) denote the force and torque functions for the sys-
tem.
Remark 5.5.1 A major numerical challenge for a direct application of DMOC
are the scales of interest. These differ by a factor of around 1017: the distance
between the Sun and the Earth is of the order of 1011 m, while the distances
between the spacecraft are of the order of several 100 m and have to be kept con-
stant up to an error of 10−6m. When using standard double-precision floating
point arithmetic, rounding errors will notably influence any corresponding com-
putation. On the other hand, we are only interested in the relative positions of
the spacecraft with respect to each other. We therefore perform our computations
in a local coordinate system by linearizing the system around a Halo-orbit. The
discrete forced Euler-Lagrange equations linearized around the points (qH
k,˙qH
k),
k= 1, . . . , N, on a given Halo-orbit then provide constraints for the discrete
optimization problem (see [69] for a detailed description).
119
Decentralization
When collision avoidance concerns4are neglected, the optimal control problem
described in the previous section is “almost” decoupled in the sense that the
coupling only enters through the constraints (5.8) on the final configuration. In
this section, we show how one can exploit this fact in order to carry out the
associated computations in parallel.
Hierarchical optimal control problem The basic observation is that the
problem can be formulated as a hierarchical optimization problem, where the
outer problem relates to the correct arrangement of the final configuration and
the ninner problems determine the optimal trajectory for one spacecraft with
fixed initial and final configuration, respectively.
We parametrize the final positions of the spacecraft projected onto the pre-
scribed plane by the vector ϕ= (ϕ(1), . . . , ϕ(n)) via
z(i)=r0cos ϕ(i)
r0sin ϕ(i),(5.11)
where ϕ(i)is the angle of spacecraft i, determining the final position on a given
circle with prescribed center (see Figure 5.18). First, we want to derive the final
Inner problems
q(i)f(i)
,
Outer problem
ϕ(i)
a) b)
Figure 5.18: Hierarchical formulation of the optimal control of formation flying
spacecraft. a) The ninner problems determine the optimal trajectory for one
spacecraft with fixed initial and final configuration, respectively. b) The outer
problem relates to the correct arrangement of the final configuration.
4See [69] for incorporating collision avoidance strategies.
120
constraint (5.8) with the help of this parametrization. We define the artificial
potential G:Sn→Rby
G(ϕ) =
n
X
i,j=1,i6=j
1
kz(i)−z(j)k2.
This artificial potential is comparable to a gravitational potential that affects
attraction or repulsion, respectively, between bodies. For an equidistant arrange-
ment on the circle the resulting forces dG/dϕ acting on each spacecraft have to
be zero. Therefore, we obtain as final constraint
g(ϕ) = dG
dϕ (ϕ) = 0,
with a function g:Sn→Rn. With the parametrization (5.11) it holds by defining
a function ˜
G:R2n→R,˜
G(z) := G(ϕ)
0 = g(ϕ) = dG
dϕ (ϕ) = d˜
G
dz (z)·dz
dϕ =: k(z),(5.12)
which results in the final constraint (5.8).
With parametrization (5.11) the problem has the following hierarchical form:
Let q(i)=x(i)
p(i)∈Q=SE(3) denote the configuration and f(i)=F(i)
τ(i)the
control force of spacecraft i. By optimizing within a fixed time interval I= [0, T]
we obtain the optimal control problem
min
ϕJ(ϕ) = min
ϕ
n
X
i=1
min Ji(q(i), f(i)),
where the first minimization is constrained by g(ϕ) = 0 and the second mi-
nimization is constrained by q(i): [0, T]→Q, f(i): [0, T]→T∗Q, q(i)(0) =
q0,(i),˙q(i)(0) = ˙q0,(i), A q(i)(T) = b(ϕ(i)),˙q(i)(T) = ˙qT,(i). Furthermore, (q(i), f(i))
have to fulfill the dynamics of spacecraft i. Here, the matrix is
A=[ν⊥
1ν⊥
2]T0
0I4∈R6×7, where I4is the unit 4 ×4 matrix and the vec-
tor b(ϕ(i)) = z(i)+ [ν⊥
1ν⊥
2]T¯x
pT∈R6is defined by equations (5.6) and (5.9).
Due to the parametrization (5.11), we do not have to incorporate the constraint
(5.7).
The inner problems are uncoupled since for each spacecraft its costs have to
be minimized separately subject to fixed initial and final states and its dynamics.
121
The outer problem includes the constraint for the final configuration, that is the
coupling of the system. We use SQP for the solution of both, the inner and the
outer problems. In each step of the solution of the outer problem all ninner
problems have to be solved anew with the new boundary constraints.
Remark 5.5.2 Constraint (5.12) provides a discrete solution set of the final
configurations ϕ(i). Due to the local optimality of SQP, the sequence of spacecraft
on the circle depends on the initial guess for the optimization problem.
Equivalence of both optimal control problems In order to show the equi-
valence of the “monolithic” formulation of the optimal control problem to the
hierarchical one, we consider the following abstract optimization problem:
min
(x,ϕ)∈X×ΦJ(x, ϕ) s.t. A(x, ϕ) = 0, g(ϕ) = 0,(5.13)
where X⊂Rdx,Φ⊂Rdyare compact and J:X×Φ→R,A:X×Φ→Raand
g: Φ →Rgare continuous. Defining X(ϕ) = {x∈X|A(x, ϕ) = 0}, we see that
[
ϕ∈g−1(0)
X(ϕ)× {ϕ}={(x, ϕ)|A(x, ϕ) = 0, g(ϕ) = 0}.
Thus,
min {J(x, ϕ)|A(x, ϕ) = 0, g(ϕ) = 0}
= min
J(x, ϕ)|(x, ϕ)∈[
ϕ∈g−1(0)
X(ϕ)× {ϕ}
= min [
ϕ∈g−1(0)
{J(x, ϕ)|(x, ϕ)∈X(ϕ)× {ϕ}}
= min min {J(x, ϕ)|x∈X(ϕ)} | ϕ∈g−1(0)
= min{min{J(x, ϕ)|A(x, ϕ) = 0} | g(ϕ) = 0},
that means we finally obtain a hierarchical formulation of the problem. The
“inner problem” is given by minimizing J(x, ϕ) subject to A(x, ϕ) = 0 (for a
fixed ϕ∈Φ), while the “outer problem” is given by minimizing
ˆ
J(ϕ) = min{J(x, ϕ)|A(x, ϕ) = 0}s.t. g(ϕ) = 0.
Since in our specific application, the inner objective function, J(x, ϕ) is given by
the sum n
X
i=1
min Ji(q(i), f(i))
and since all Jiare nonnegative, the inner problem decouples into nindependent
subproblems which can be solved independently.
122
Parallelization The hierarchical structure of the problem enables a computa-
tional solution in parallel, as we are faced with nuncoupled inner problems in each
step of the solution of the outer problem. These nsubproblems are solved in n
different tasks. Our implementation uses the software package PUB (Paderborn
University BSP-Library, [24]) developed within the DFG research project CRC
376 “Massively Parallel Computation” at the University of Paderborn. In the
terminology of PUB, each step of the iteration scheme for the solution of the
outer problem represents one superstep. After each superstep, the tasks have to
communicate during the synchronization phase.
Example computations In all following computations we use N= 10 time
intervals in the time discretization of the trajectories and solve the resulting
finite-dimensional (nonlinear) optimization problem by the SQP method as imple-
mented in the routine E04UEF of the NAG-library. We use numerical derivatives
both for the cost and for the constraint functions.
First, we consider a group of six spacecraft in the vicinity of a Halo-orbit and
require the spacecraft to adopt a planar hexagonal formation with center on the
orbit. Figure 5.19 a) shows the Halo orbit (in normalized coordinates) that we
have chosen for this computation and the part of the orbit that we use for the
linearization of the problem.
0.999 1.0016 1.0042 1.0068 1.0094 1.012
0.01
0
0.01
12
10
8
6
4
2
0
2
4
•
x 10 3
L2
x1
Earth
x2
x3
−4
−2
0
2−1.5 −1−0.5 00.5 11.5
−1.2
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
x2
x1
x3
a) b)
Figure 5.19: a) L2Halo-orbit chosen for the example computations (thin line) and
the part used for the linearization (thick line). b) Initial positions (×), optimal
trajectories and final positions (◦) for a reconfiguration of six spacecraft in the
CRTBP.
Figure 5.19 b) shows (in normalized coordinates) the initial positions (×),
the optimal trajectories, as well as the final positions (◦). Initially, the group
123
!6
!4
!2
0
2
4
6
!5
0
5
!5
0
5
x2
x1
x3
0 5 10 15 20 25 30 35
0
5
10
15
20
25
number of processors
speed up
a) b)
Figure 5.20: a) Initial positions (◦), optimal trajectories and final positions (×)
for a reconfiguration of 30 spacecraft in the CRTBP in x1-x2-x3-space. b) Speed
up diagram in dependence on the number of processors for the reconfiguration of
30 spacecraft in the CRTBP.
124
Table 5.2: Speed up diagram.
number of processors 2 4 8 16 32
speed up 1.33 2.65 5.98 11.09 20.76
CPU time [104sec] 9.55 4.8 2.13 1.15 0.62
is located along a line with initial orientation p0,(i)= (cos π
2,sin π
2·(1,0,0)) for
each spacecraft i(that is a rotation of θ=πaround the x1-axis). The final
configuration is a hexagonal formation in the plane with normal n= (1,0,1) and
final orientation pT,(i)= (cos π, sin π·(0,1,0)) for each spacecraft i(that is a
rotation of θ= 2πaround the x2-axis).
As a second example we consider a group of 30 spacecraft modeled as rigid
bodies in 3-space, with the circular restricted three body problem governing their
dynamics. Figure 5.20 a) shows the initial positions (◦), the optimal trajectories,
as well as the final positions (×) in normalized coordinates. Initially, the group is
located on a grid in the x1-x2-plane with initial orientation p0,(i)= (cos π
2,sin π
2·
(1,0,0)) for each spacecraft i(that is a rotation of θ=πaround the x1-axis).
The final configuration is a circle formation in the plane with normal n= (1,0,1)
and final orientation pT,(i)= (cos π, sin π·(0,1,0)) for each spacecraft i(that is
a rotation of θ= 2πaround the x2-axis).
Figure 5.20 b) and Table 5.2 show the dependence of the computation time
on the number of processors (for only one processor the computation time is
12.75 ·104seconds). The almost linear speed up indicates the effectiveness of our
decentralized approach.
125
Chapter 6
Optimal control of constrained
mechanical systems in
multi-body dynamics
In this chapter, we formulate a framework for the optimal control of constrained
mechanical systems. Such systems occur in multi-body dynamics.1For the appli-
cations in Section 5.4 we searched for generalized coordinates for the underlying
multi-body system. These are often not easy to determine. In contrast, in this
chapter the multi-body system is formulated as a constrained system, that is
each body is viewed as a constrained continuum, described in terms of redundant
coordinates subject to holonomic constraints. The couplings between the bodies
are characterized via external holonomic constraints which describe the kinematic
conditions that arise from the specific joint connections.
To optimally control these kinds of systems, in Sections 6.1 and 6.2 we ex-
tend the variational formulation of DMOC to a constrained version: In the first
place, configuration constraints are enforced using Lagrange multipliers. Then,
to reduce the number of unknowns (configurations and torques at the time nodes)
and the dynamic constraints to the minimal possible number, the discrete null
space method method ([7]) is used, which is suitable for the accurate, robust
and efficient time integration of constrained dynamical systems (in particular for
multi-body dynamics). This procedure does not only lead to lower computational
costs for the optimization algorithm, but it also inherits the conservation proper-
ties from the constrained scheme. Furthermore, it has the advantage of circum-
venting the difficulties associated with rotational parameters ([8]). The benefit
of exact constraint fulfillment, correct computation of the change in momentum
maps and good energy behavior is guaranteed by the optimization algorithm.
1See for example [25, 154] for other methods to the optimal control of multi-body dynamics.
127
The combination of DMOC and the nullspace method has first been presented in
[99] and is currently investigated in [98].
In particular, in Section 6.3 we develop an optimal control framework for a
rigid body and for multi-body dynamics. The constrained formulation as well as
the actuation via control forces are described in more detail. Finally, in Section
6.4, we apply the developed method to the optimal control of two multi-body
systems arising in robotics and biomechanics.
Remark 6.0.3 Compared to the previous chapters, we use a slightly different
notation to be consistent with the notation in [98, 99] (see the nomenclature).
Therefore, the control parameters denoted by u(t)∈Uare represented by the
generalized independent control forces τ(t)∈W, whereas urepresents the genera-
lized independent coordinates of the system. Cdenotes the manifold determined
via the holonomic constraints. We denote the cost function of the objective
functional Jby B. Furthermore, we restrict ourselves to fixed boundary con-
ditions on the configuration and the velocity level (q(0),˙q(0)) = (q0,˙q0) and
(q(T),˙q(T)) = (qT,˙qT).
6.1 Constrained dynamics and optimal control
This section presents the derivation of the equations of motion for forced con-
strained systems which have to be fulfilled as constraints in the optimization
problem. The transformation of the differential algebraic equations by the null
space method and reparametrization, and in particular the equivalence of the
resulting equations of motion, is described in detail in [97] for unforced systems.
Consider an n-dimensional mechanical system with the time-dependent con-
figuration vector q(t)∈Qand velocity vector ˙q(t)∈Tq(t)Q, where t∈[0, T]⊂R
denotes the time and T∈N. Let the configuration be constrained by the function
g(q) = 0 ∈Rmand influenced by the force field f:W×TQ →T∗Q. Due to the
presence of constraints, the forces fare not independent. They can be calculated
in terms of the time dependent generalized control forces τ(t)∈W⊆Rn−m.
6.1.1 Optimization problem
The goal is to determine the optimal force field, such that the system is moved
from the initial state (q0,˙q0) to the final state (qT,˙qT) while the objective func-
tional
J(q, ˙q, f) = ZT
0
B(q, ˙q, f(q, ˙q, τ)) dt(6.1)
is minimized. Note that Jdenotes a function J:TQ ×T∗Q→Rrather than
J:TQ ×U→Ras in Chapter 4.
128
6.1.2 Constrained Lagrange-d’Alembert principle
Contemporaneously, the motion (q, ˙q) has to be in accordance with an equation
of motion which in the present case is based on a constrained version of the
Lagrange-d’Alembert principle (see for example [113]) requiring
δZT
0
L(q, ˙q)−gT(q)·λdt+ZT
0
f(q, ˙q, τ)·δq dt= 0 (6.2)
for all variations δq ∈TQ and δλ ∈Rmvanishing at the endpoints. The La-
grangian L:TQ →Rcomprises the kinetic energy 1
2˙qT·M·˙qincluding the
consistent mass matrix M∈Rn×nand a potential function V:Q→R. Further-
more, λ(t)∈Rmrepresents the vector of time dependent Lagrange multipliers.
The constrained Lagrange-d’Alembert principle (6.2) leads to the differential-
algebraic system of equations of motion
∂L(q, ˙q)
∂q −d
dt∂L(q, ˙q)
∂˙q−GT(q)·λ+f(q, ˙q, τ) = 0,(6.3a)
g(q) = 0,(6.3b)
where G(q) = Dg(q) denotes the Jacobian of the constraints. The vector −GT(q)·
λrepresents the constraint forces that prevent the system from deviations of the
constraint manifold
C={q∈Q|g(q) = 0}.(6.4)
6.1.3 Null space method
Assuming that the constraints are independent, for every q∈Cthe basis vectors
of TqCform an n×(n−m) matrix P(q) with corresponding linear map P(q) :
Rn−m→TqC. This matrix is called null space matrix, since
range (P(q)) = null (G(q)) = TqC. (6.5)
Thus a premultiplication of the differential equation (6.3a) by PT(q) eliminates
the constraint forces including the Lagrange multipliers from the system. The
resulting equations of motion read
PT(q)·∂L(q, ˙q)
∂q −d
dt∂L(q, ˙q)
∂˙q+f(q, ˙q, τ)= 0,
g(q)=0.
(6.6)
129
6.1.4 Reparametrization
For many applications it is possible to find a reparametrization of the constraint
manifold F:U⊆Rn−m→Cin terms of independent generalized coordinates
u∈U. Then the Jacobian DF(u) of the coordinate transformation plays the role
of a null space matrix. Since the constraints (6.3b) are fulfilled automatically
by the reparametrized configuration variable q=F(u), the system is reduced to
n−msecond order differential equations. This is the minimal possible dimension
for the present mechanical system which consists of precisely n−mconfigurational
degrees of freedom. Consequently, there are n−mindependent generalized forces
τ∈W⊆Rn−macting on the degrees of freedom. These can be calculated as
τ=∂F
∂u T·f, see for example [50].
6.2 Constrained discrete dynamics and optimal
control
Analogous steps are performed in the temporal discrete variational setting to de-
rive the forced constrained discrete Euler-Lagrange equations and their reduction
to minimal dimension. Again, these steps have been investigated in detail in [97]
for unforced systems.
As in Section 3.1.2, corresponding to the configuration manifold Q, the dis-
crete phase space is defined by Q×Qwhich is locally isomorphic to TQ. For a
constant time step h∈R, a path q: [0, T]→Qis replaced by a discrete path
qd:{0 = t0, t0+h, , . . . , t0+Nh =T} → Q,N∈N, where qk=qd(t0+kh) is
viewed as an approximation of q(t0+kh). Similarly, λk=λd(tk) approximates
the Lagrange multiplier at tk=t0+kh, while the force field fis approximated
by the two discrete forces f−
k, f+
k:W×Q→T∗Cdefined in (6.17).
6.2.1 Discrete constrained Lagrange-d’Alembert principle
According to the derivation of variational integrators for constrained dynamics in
[97], the action integral in (6.2) is approximated in a time interval [tk, tk+1] using
the discrete Lagrangian Ld:Q×Q→Rand the discrete constraint function
gd:Q→Rmvia
Ld(qk, qk+1)−1
2gT
d(qk)·λk−1
2gT
d(qk+1)·λk+1 ≈Ztk+1
tk
L(q, ˙q)−gT(q)·λdt. (6.7)
130
Among various possible choices to approximate this integral, the midpoint rule
is used for the Lagrangian, that is
Ld(qk, qk+1) = hL qk+1 +qk
2,qk+1 −qk
h(6.8)
and
gT
d(qk) = hgT(qk) (6.9)
for the constraints. Likewise, the virtual work is approximated by
f−
k·δqk+f+
k·δqk+1 ≈Ztk+1
tk
f(q, ˙q, τ)·δq dt, (6.10)
where f+
k, f−
kare the left and right discrete forces, respectively. They are specified
in (6.17).
The discrete version of the constrained Lagrange-d’Alembert principle (6.2)
requires the discrete path {qk}N
k=0 and multipliers {λk}N
k=0 to fulfill
δ
N−1
X
k=0
Ld(qk, qk+1)−1
2gT
d(qk)·λk−1
2gT
d(qk+1)·λk+1 +
N−1
X
k=0 f−
k·δqk+f+
k·δqk+1= 0
(6.11)
for all variations {δqk}N
k=0 and {δλk}N
k=0 with δq0=δqN= 0, which is equivalent
to the constrained forced discrete Euler-Lagrange equations
D2Ld(qk−1, qk) + D1Ld(qk, qk+1)−GT
d(qk)·λk+f+
k−1+f−
k= 0,(6.12a)
g(qk+1) = 0,(6.12b)
for k= 1, . . . , N −1, where Gd(qk) denotes the Jacobian of gd(qk). As in Section
3.1.2 the time-stepping scheme (6.12) has not been deduced discretizing (6.3) but
via a discrete variational principle.
6.2.2 Discrete null space method
A reduction of the time-stepping scheme (6.12) can be accomplished in analogy
to the continuous case according to the discrete null space method. In order to
eliminate the discrete constraint forces from the equations, a discrete null space
matrix fulfilling
range (P(qk)) = null (Gd(qk)) (6.13)
is employed. Analogous to (6.6), the premultiplication of (6.12) by the transposed
discrete null space matrix cancels the constraint forces from the system, that is the
131
Lagrange multipliers are eliminated from the set of unknowns and the dimension
of the system is reduced to nas
PT(qk)·D2Ld(qk−1, qk) + D1Ld(qk, qk+1) + f+
k−1+f−
k= 0,(6.14a)
g(qk+1)=0.(6.14b)
6.2.3 Nodal reparametrization
Similar to the continuous case, a reduction of the system to the minimal possible
dimension can be accomplished by a local reparametrization of the constraint
manifold in the neighborhood of the discrete configuration variable qk∈C. At
the time nodes, qkis expressed in terms of the discrete generalized coordinates
uk∈U⊆Rn−mby the map F:U⊆Rn−m×Q→C, such that the constraints
are fulfilled:
qk=F(uk, qk−1) with g(qk) = g(F(uk, qk−1)) = 0, k = 1, . . . , N. (6.15)
The discrete generalized control forces are assumed to be constant in each time
interval (see Figure 6.1). First of all, the effect of the generalized forces acting in
[tk−1, tk] and in [tk, tk+1] is transformed to the time node tkvia
τ+
k−1=h
2τk−1,(k= 1, . . . , N), τ−
k=h
2τk,(k= 0, . . . , N −1).(6.16)
Secondly, the components of the discrete force vectors f+
kand f−
kcan be calcu-
lated as
f+
k−1=fqk, τ+
k−1, f−
k=fqk, τ−
k∈T∗
qkC,
fk=f+
k+f−
k,
fd={fk}N−1
k=0 .
(6.17)
Thus f+
k−1denotes the effect of the generalized force τk−1acting in [tk−1, tk] on qk
while f−
kdenotes the effect on qkof τkacting in [tk, tk+1]. fkis the total discrete
force in [tk, tk+1] resulting from the generalized force τk.
Insertion of the nodal reparametrizations for the configuration (6.15) and the
force (6.17) into the scheme redundantizes (6.14b). The resulting scheme
PT(qk)·D2Ld(qk−1, qk) + D1Ld(qk, F(uk+1, qk)) + f+
k−1+f−
k= 0 (6.18)
has to be solved for uk+1 whereupon qk+1 is obtained from (6.15). (6.18) is
equivalent to the constrained scheme (6.12), thus it also has the key properties
of exact constraint fulfilment, symplecticity and momentum consistency, that is
any change in the value of a momentum map reflects exactly the applied forces.
When no load is present, momentum maps are conserved exactly.
132
Figure 6.1: Relation of redundant forces at tkto piecewise constant discrete
generalized forces.
6.2.4 Boundary conditions
In the next step, the boundary conditions q(0) = q0,˙q(0) = ˙q0and q(T) =
qT,˙q(T) = ˙qThave to be specified. Those on the configuration level can be used
as constraints for the optimization algorithm in a straightforward way. At t= 0
we can request u0=u0. Within this work, we use an absolute reparametrization,
that is (6.15) is changed to qk=F(uk, q0), then uN=uTis prescribed (see [98]
for the formulation with relative reparametrization). Along the lines of Section
4.2 the velocity conditions have to be transformed to conditions on the conjugate
momentum, which is defined at each and every time node using the discrete
Legendre transform. The presence of forces as well as constraint forces at the
time nodes has to be incorporated into that transformation leading to constrained
forced discrete Legendre transforms Fcf−Ld:Q×Q→T∗Qand Fcf+Ld:Q×Q→
T∗Qreading
Fcf−Ld: (qk, qk+1)7→ (qk, p−
k)
p−
k=−D1Ld(qk, qk+1) + 1
2GT
d(qk)·λk−f−
k,(6.19a)
Fcf+Ld: (qk−1, qk)7→ (qk, p+
k)
p+
k=D2Ld(qk−1, qk)−1
2GT
d(qk)·λk+f+
k−1.(6.19b)
As in the unforced case, the time-stepping scheme (6.12a) can be interpreted as
a matching of momenta p+
k−p−
k= 0 such that along the discrete trajectory,
there is a unique momentum at each time node kwhich can be denoted by pk.
However, just as the appearance of Lagrange multipliers is avoided in the dynamic
constraints for the optimization problem (6.18), their presence in the initial and
final momentum conditions complicates matters unnecessarily. Even though they
can be related to the discrete trajectory via
λk=RT
d(qk)·D1Ld(qk, qk+1) + D2Ld(qk−1, qk) + f+
k−1+f−
k,(6.20)
133
where
Rd(qk) = GT
d(qk)·Gd(qk)·GT
d(qk)−1,(6.21)
the following versions of the discrete Legendre transforms do not need the La-
grange multipliers. The projected discrete Legendre transforms QFcf−Ld:Q×
Q→ηT∗
qkCand QFcf+Ld:Q×Q→ηT∗
qkCread
Qp−
k=Q(qk)·−D1Ld(qk, qk+1)−f−
k,(6.22a)
Qp+
k=Q(qk)·D2Ld(qk−1, qk) + f+
k−1,(6.22b)
where Q(qk) is given by (see [113])
Q=In×n−GT
d·Gd·M−1·GT
d−1Gd·M−1,(6.23)
and fulfills Q(qk)·GT
d(qk) = 0n×m. Note that for the constrained discrete Legendre
transforms and for the projected discrete Legendre transforms the output is an
n-dimensional momentum vector. In the projected case, it lies in the (n−m)-
dimensional submanifold ηT∗
qkCbeing the embedding of T∗
qkCinto T∗
qkQ. Yet
another possibility is to compute an (n−m)-dimensional momentum vector by
projecting with the discrete null space matrix. The reduced discrete Legendre
transforms PFcf−Ld:Q×Q→T∗Uand PFcf+Ld:Q×Q→T∗Uare given by
Pp−
k=PT(qk)·−D1Ld(qk, qk+1)−f−
k,(6.24a)
Pp+
k=PT(qk)·D2Ld(qk−1, qk) + f+
k−1.(6.24b)
This version is most appropriate to be used as a constraint in the optimization
problem, since it yields the minimal number of independent conditions, while
conditions formulated using (6.22) are redundant and (6.19) involves the Lagrange
multipliers. Note that according to the range of the projection (6.23), Qpkfulfills
the constraints on the momentum level hd, that is
hd(qk,Qpk) = G(qk)·M−1·Qpk= 0,(6.25)
while this is not in general the case for pk. This question is superfluous for Ppk.
Prescribed initial and final velocities of course should be consistent with the
constraints on the velocity level. Using the standard continuous Legendre trans-
form FL:TC →T∗C
FL: (q, ˙q)7→ (q, p) = (q, D2L(q, ˙q)) (6.26)
yields momenta which are consistent with the constraints on the momentum
level as well. With these preliminaries, the velocity boundary conditions are
transformed to the following conditions on the momentum level p0=p0, pN=pT,
which read in detail
PT(q0)·D2L(q0,˙q0) + D1Ld(q0, q1) + f−
0= 0,
PT(qN)·−D2L(qT,˙qT) + D2Ld(qN−1, qN) + f+
N−1= 0.(6.27)
134
6.2.5 Discrete constrained optimization problem
Now the discrete optimal control problem for the constrained discrete dynamical
problem can be formulated. To begin with, we define an approximation
Bd(qk, qk+1, fk)≈Ztk+1
tk
B(q, ˙q, f(τ, q, ˙q)) dt(6.28)
of the continuous objective functional (6.1). Similar to the approximations in
(6.8) the midpoint rule is applied, that is
Bd(qk, qk+1, fk) = hB qk+1 +qk
2,qk+1 −qk
h, fk(6.29)
with the discrete forces given in (6.17). This yields the discrete objective function
Jd(qd, fd) =
N−1
X
k=0
Bd(qk, qk+1, fk),(6.30)
where the discrete configurations and forces are expressed in terms of their cor-
responding independent generalized quantities. Alternatively a new objective
function can be formulated directly in the generalized quantities
¯
Jd(ud, τd) =
N−1
X
k=0
¯
Bd(uk, uk+1, τk) (6.31)
depending on the desired interpretation of the optimization problem. In any
case, (6.30) or (6.31) has to be minimized with respect to ud, τdsubject to the
constraints
u0−u0= 0,
uN−uT= 0,
PT(q0)·D2L(q0,˙q0) + D1Ld(q0, q1) + f−
0= 0,
PT(qN)·−D2L(qT,˙qT) + D2Ld(qN−1, qN) + f+
N−1= 0,
PT(qk)·D2Ld(qk−1, qk) + D1Ld(qk, qk+1) + f+
k−1+f−
k= 0,
(6.32)
for k= 1, . . . , N −1.
6.3 Optimal control for rigid body dynamics
6.3.1 Constrained formulation of rigid body dynamics
The treatment of rigid bodies as structural elements relies on the kinematic as-
sumptions (see [10]) that the placement of a material point in the body’s config-
uration X=XIdI∈ B ⊂ R3relative to an orthonormal basis {eI}fixed in space
135
Figure 6.2: Configuration of a rigid body with respect to an orthonormal frame
{eI}fixed in space.
can be described as
x(X, t) = ϕ(t) + XIdI(t),(6.33)
see Figure 6.2 for an illustration. Here XI∈R, I = 1,2,3 represent coordinates
in the body-fixed director triad {dI}. The time-dependent configuration variable
of a rigid body
q(t) =
ϕ(t)
d1(t)
d2(t)
d3(t)
∈R12 (6.34)
consists of the placement of the center of mass ϕ∈R3and the directors dI∈
R3, I = 1,2,3 which are constrained to stay orthonormal during the motion,
representing the rigidity of the body and its orientation. Thus we work with the
embedding of the constraint manifold C=R3×SO(3) into the configuration
manifold Q=R12. These orthonormality conditions pertaining to the kinematic
assumptions of the underlying theory are termed internal constraints. There are
mint = 6 independent internal constraints for the rigid body with associated
constraint functions
gint(q) =
1
2[dT
1·d1−1]
1
2[dT
2·d2−1]
1
2[dT
3·d3−1]
dT
1·d2
dT
1·d3
dT
2·d3
.(6.35)
For simplicity, it is assumed that the axes of the body frame coincide with the
principal axes of inertia of the rigid body. Then the body’s Euler tensor with
respect to the center of mass can be related to the inertia tensor Jvia
E=1
2(trJ)I−J, (6.36)
136
where Idenotes the 3 ×3 identity matrix. The principal values of the Euler
tensor Eitogether with the body’s total mass Mϕbuild the rigid body’s constant
symmetric positive definite mass matrix
M=
MϕI0 0 0
0E1I0 0
0 0 E2I0
0 0 0 E3I
,(6.37)
where 0 denotes the 3 ×3 zero matrix. The angular momentum of the rigid body
can be computed as
L=ϕ×pϕ+dI×pI,(6.38)
where Einstein’s summation convention is used to sum over the repeated index
I.
Null space matrix This description of rigid body dynamics has been discussed
in [7, 96] where also the null space matrix
Pint(q) =
I0
0−b
d1
0−b
d2
0−b
d3
(6.39)
corresponding to the constraints (6.35) has been derived. Here badenotes the
skew-symmetric 3 ×3 matrix with corresponding axial vector a∈R3. With
the null space matrix, the independent generalized velocities of the rigid body,
namely the translational velocity ˙ϕ∈R3and the angular velocity ω∈R3being
comprised in the twist of the rigid body
t=˙ϕ
ω(6.40)
can be mapped to the redundant velocities ˙q=P(q)·t.
Nodal reparametrization When the nodal reparametrization of unknowns
is applied, the configuration of the free rigid body is specified by six unknowns
uk+1 = (uϕk+1 , θk+1)∈U⊂R3×R3, characterizing the incremental displacement
and incremental rotation, respectively. Accordingly, in the present case the nodal
reparametrization F:U→Cintroduced in (6.15) assumes the form
qk+1 =Fd(uk+1, qk) =
ϕk+uϕk+1
exp( d
θk+1)·(d1)k
exp( d
θk+1)·(d2)k
exp( d
θk+1)·(d3)k
,(6.41)
137
where Rodrigues’ formula is used to obtain a closed form expression of the expo-
nential map, see for example [111].
6.3.2 Actuation of the rigid body
The single rigid body can be actuated by generalized forces consisting of a trans-
lational force τϕ∈R3and a torque τθ∈R3. Assume that the force is not applied
in the center of mass, but in material points of the rigid body located at
%=%IdI(6.42)
away from the center of mass. This results in the force τ1
ϕto be applied in the
center of mass and a torque τ1
θas
τ1
ϕ=τϕand τ1
θ=τθ+%×τϕ.(6.43)
Then the redundant forces can be computed as
fϕ=τ1
ϕand fI=1
2τ1
θ×dI.(6.44)
For the rigid body, the transformation (6.16) of the generalized forces acting in
[tk−1, tk] and in [tk, tk+1] to the time node tkreads
τ1
ϕ
+
k−1=h
2τϕk−1and τ1
θ
+
k−1=h
2τθk−1+%k×τϕk−1,
τ1
ϕ
−
k=h
2τϕkand τ1
θ
−
k=h
2τθk+%k×τϕk.
(6.45)
Insertion into (6.17) and taking into account (6.44) yields the 12-dimensional left
and right discrete forces f+
k−1, f−
kfor the rigid body. A straightforward calculation
shows
PT(qk)·(f+
k−1+f−
k) =
τ1
ϕ
+
k−1+τ1
ϕ
−
k
τ1
θ
+
k−1+τ1
θ
−
k
.(6.46)
Thus the resulting reduced forces in (6.18) are correct and meaningful.
Proposition 6.3.1 The definition of the redundant left and right discrete forces
guarantees that the change in angular momentum along the solution trajectory qd
of (6.18) is induced only by the discrete generalized forces.
138
Proof: Computation of p+
k+1 and via p−
kthe discrete Legendre transforms (6.19a)
and (6.19b), respectively, and insertion into the definition of angular momentum
(6.38) yields
Lk+1 −Lk=
ϕk+1 ×pϕ+
k+1 +dIk+1 ×pI+
k+1 −ϕk×pϕ−
k−dIk×pI−
k=
ϕk+1 ×fϕ+
k+dIk+1 ×fI+
k−ϕk×−fϕ−
k−dIk×−fI−
k=
ϕk+1 ×τ1
ϕ
+
k+ϕk×τ1
ϕ
−
k+dIk+1 ×1
2τ1
θ
+
k×dIk+1+dIk×1
2τ1
θ
−
k×dIk=
ϕk+1 ×τ1
ϕ
+
k+ϕk×τ1
ϕ
−
k+τ1
θ
+
k+τ1
θ
−
k.
(6.47)
Remark 6.3.2 (Gravitation) The computation in (6.47) is performed for the
case in which no potential is present. In the presence of gravity of value g∈R
in the negative e3-direction, the corresponding potential reads
V(q) =
0
0
−Mϕg
0
.
.
.
0
T
·q. (6.48)
In this case, (6.47) yields
Lk+1 −Lk=ϕk+1 ×τ1
ϕ
+
k+ϕk×τ1
ϕ
−
k+τ1
θ
+
k+τ1
θ
−
k−(ϕk+1 +ϕk)×
0
0
−Mϕg
,
(6.49)
meaning that the third component of the angular momentum changes only ac-
cording to the applied forces. In particular in the absence of any external forces,
this shows that the third component of the angular momentum is exactly con-
served.
6.3.3 Kinematic pairs
In this chapter, we present the constrained formulation of the dynamics of kine-
matic pairs and the subsequent reduction of the equations of motion via the
139
discrete null space method with nodal reparametrization. Furthermore, we show
how the generalized forces of a kinematic pair act on the respective bodies.
The coupling of two neighboring links (body 1 and body 2) by a specific joint
Jyields m(J)
ext external constraints gext(q)∈Rm(J)
ext where
q=q1
q2.(6.50)
Depending on the number of external constraints m(J)
ext, the degrees of freedom of
the relative motion of one body with respect to the other is decreased from 6 to
r(J)= 6 −m(J)
ext.
Altogether, m=mint +mext constraints describing the multi-body system
and the corresponding constraint Jacobians can be combined to
g(q) = gint(q)
gext(q)∈Rm, G(q) = Gint(q)
Gext(q)∈Rm×n.(6.51)
In [96, 98] details of the external constraints caused by lower kinematic pairs and
their treatment in the framework of the discrete null space method are discussed.
With the null space matrices for kinematic pairs at hand, a generalization to
multi-body systems being composed by pairs can be performed easily, see [7, 96,
97].
Null space matrix In a kinematic pair, the motion of the second body with
respect to an axis fixed in the first body can be accounted for by introducing r(J)
joint velocities τ(J). Thus the motion of the kinematic pair can be characterized
by the independent generalized velocities ν(J)∈R6+r(J)with
ν(J)=t1
τ(J).(6.52)
In particular, introducing the 6 ×(6 + r(J)) matrix P2,(J)
ext (q), the twist of the
second body t2∈R6can be expressed as
t2,(J)=P2,(J)
ext (q)·ν(J).(6.53)
Accordingly, the twist of the kinematic pair Jcan be written in the form
t(J)=P(J)
ext (q)·ν(J)(6.54)
with the 12 ×(6 + r(J)) matrix P(J)
ext (q), which may be partitioned according to
P(J)
ext (q) = I6×606×r(J)
P2,(J)
ext (q)!.(6.55)
140
Once P(J)
ext (q) has been established, the total null space matrix pertaining to the
kinematic pair under consideration can be calculated from
P(J)(q) = Pint(q)·P(J)
ext (q) = P1
int(q1) 012×r(J)
P2
int(q2)·P2,(J)
ext (q)!.(6.56)
Finally, the 24-dimensional redundant velocity vector of the kinematic pair can
be expressed in terms of the independent generalized velocities ν(J)∈R6+r(J)via
˙q=P(J)(q)·ν(J).(6.57)
Provided that P2,(J)
ext (q) has been properly deduced from (6.53),
˙q∈null (G(J)(q)),(6.58)
and the above procedure warrants the design of viable null space matrices which
automatically satisfy the relationship
G(J)(q)·P(J)(q) = 0.(6.59)
To summarize, in order to construct a null space matrix pertaining to a specific
kinematic pair, essentially relationship (6.53) is applied to deduce the matrix
P2,(J)
ext (q). Once P2,(J)
ext (q) has been determined, the complete null space matrix
pertaining to a specific pair follows directly from (6.56).
Nodal reparametrization Corresponding to the independent generalized ve-
locities ν(J)∈R6+r(J)introduced in (6.52), the redundant coordinates q∈R24 of
each kinematic pair Jmay be expressed in terms of 6 + r(J)independent genera-
lized coordinates. Concerning the reparametrization of unknowns in the discrete
null space method, relationships of the form
qk+1 =F(J)(µ(J)
k+1, qk) (6.60)
are required, where
µ(J)
k+1 = (u1
ϕk+1 , θ1
k+1, ϑ(J)
k+1)∈R6+r(J)(6.61)
consists of a minimal number of incremental unknowns in [tk, tk+1] for a specific
kinematic pair. In (6.61), (u1
ϕk+1 , θ1
k+1)∈R3×R3are incremental displacements
and rotations, respectively, associated with the first body (see Section 6.3.1).
Furthermore, ϑ(J)
k+1 ∈Rr(J)denote incremental unknowns which characterize the
configuration of the second body relative to the axis of relative motion fixed in the
141
first body. In view of (6.50), the mapping in (6.60) may be partitioned according
to q1
k+1 =F1(u1
ϕk+1 , θ1
k+1, q1
k),
q2
k+1 =F2,(J)(µ(J)
k+1, qk).(6.62)
Here, F1(u1
ϕk+1 , θ1
k+1, q1
k) is given by (6.41). It thus remains to specify the map-
ping F2,(J)(µ(J)
k+1, qk) for each kinematic pair under consideration. Of course, the
mapping F(J)(µ(J)
k+1, qk) is required to satisfy the constraints specified by (6.51),
that is g(J)
ext(F(J)(µ(J)
k+1, qk)) = 0, for arbitrary µ(J)
k+1.
In [7, 96] details of the treatment of specific kinematic pairs Jare provided.
In essence, the present approach requires the specification of (i) the external con-
straint function g(J)
ext(q), along with the corresponding constraint Jacobian G(J)
ext(q),
and (ii) the null space matrix P2,(J)
ext (q), which is needed to set up the complete null
space matrix (6.56). Finally, (iii) the mapping F2,(J)(µ(J)
k+1, qk) is specified, which
is needed to perform the reparametrization of unknowns according to (6.60), and
allows the reduction of the discrete system of equations of motion to the minimal
possible dimension.
Actuation of a kinematic pair The actuation of kinematic pairs is twofold.
First of all, the overall motion of the pair can be influenced by applying transla-
tional forces τϕ∈R3and torques τθ∈R3to one of the bodies, say body 1. Any
resulting change in the first bodies velocities will be transferred to the second
body via the constrained equations of motion. Secondly, the relative motion of
the pair can be influenced. Actuation of the joint connection itself affects both
bodies, where according to actio equals reactio, the resulting generalized forces
on the bodies are equal, but opposite in sign. The dimension of the joint force
τ(J)is determined by the number of relative degrees of freedom r(J)permitted by
the specific joint.
The forces τα
ϕand torques τα
θon the α-th body depend on τϕ, τθand τ(J)∈
Rr(J)according to the specific joint used. The redundant forces fαon each body
can then be computed analogous to (6.44) by
fα
ϕ=τα
ϕand fα
I=1
2τα
θ×dα
I,(6.63)
and the total redundant force vector reads
f=f1
f2.(6.64)
Besides depending on the translational forces τϕk−1, τϕkand torques τθk−1, τθk, the
resulting forces and torques on body 1 in (6.45) now include the effect of the joint
actuation τ(J)
k−1
+=h
2τ(J)
k−1and τ(J)
k
−=h
2τ(J)
k.
142
Similar to (6.46), the product of the null space matrix and the redundant
forces of the kinematic pair yields
PT(qk)·(f+
k−1+f−
k) =
τ1
ϕ
+
k−1+τ1
ϕ
−
k
τ1
θ
+
k−1+τ1
θ
−
k
τ(J)
k−1
++τ(J)
k
−
.(6.65)
In [98] an overview of various lower kinematic pairs, their representation and
their actuation are given in detail. Additionally, it is proved for various specific
joints that the change in angular momentum along the solution trajectory qdof
(6.18) is induced only by the discrete generalized forces.
6.4 Applications
In this section, we demonstrate the optimal control method for two multi-body
systems arising in applications from robotics and biomechanics. Besides demon-
strating the suitability of the methodology to optimally control the motion of real
systems with high complexity, the preservation properties of the algorithm are
numerically shown.
6.4.1 Optimal control of a rigid body with rotors
Fully actuated case
Inspired by space telescopes such as the Hubble telescope, whose change in ori-
entation is induced by external spinning rotors, we analyze a multi-body system
consisting of a main body to which rotors are connected by revolute joints. The
revolute joints allow each rotor to rotate relative to the main body around an
axis through its center which is fixed in the main body. The goal is to determine
optimal torques to guide the main body into the final position uT
θ=π
14 (1,2,3),
where the absolute reparametrization qk=F(uk, q0) is used instead of (6.15)
here. The motion starts and ends at rest. The duration is 5 seconds and the
time step is h= 0.1, thus N= 50. The objective function ¯
Jd=hPN−1
k=0 ||τk||2
represents the control effort to be minimized. Due to the presence of three rotors
with non-planar axes of rotation, this problem is fully actuated.
Figure 6.3 shows the configuration of the system at t= 0,...,5 seconds. The
static frame represents the required final orientation where the axes must coincide
with the centers of the rotors as the motion ends (see last picture). The optimal
torques, which are constant in each time interval, are depicted in Figure 6.4.
143
!5
0
5
!4!20246
!2
!1
0
1
2
3
4
5
6
!5
0
5
!2024
!2
!1
0
1
2
3
4
!5
0
5
!2024
!2
!1
0
1
2
3
4
!5
0
5
!2024
!2
!1
0
1
2
3
4
!5
0
5
!2024
!2
!1
0
1
2
3
4
!5
0
5
!2024
!2
!1
0
1
2
3
4
Figure 6.3: Rigid body with three rotors: configuration at t= 0,...,5 (h= 0.1).
They yield a control effort of ¯
Jd= 2.8242 ·106. Finally Figure 6.5 illustrates
the evolution of the kinetic energy and a special attribute of the system under
consideration. It has a geometric phase which means that the motion occurs
although the total angular momentum remains zero at all times. As shown in
Figure 6.5, the algorithm is able to represent this correctly.
Underactuated case
The same rest-to-rest maneuver is investigated for the underactuated system
where one momentum wheel has been removed. Using the same time step and
the same number of time steps as for the fully actuated case, the reorientation
maneuver depicted in Figure 6.6 requires only slightly more control effort ¯
Jd=
2.9168 ·106.
Consistency of angular momentum is observable from Figure 6.8. It also
shows that the energy does not evolve as symmetrically as for the fully actuated
problem. That means that acceleration phase and breaking phase are not exactly
inverse to each other. This becomes also obvious from Figure 6.7 which shows
the evolution of the optimal generalized forces.
6.4.2 Biomechanics: The optimal pitch
Motivated by the determination of the optimal motion sequences of a gymnast
in Section 5.4.2, in this section we aim at the investigation of the optimal pitch.
144
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
!1000
!500
0
500
1000
1500
t
torque
!"1
!"2
!"3
Figure 6.4: Rigid body with three rotors: torque over time (h= 0.1).
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
0
2000
4000
6000
8000
t
energy
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
!1
!0.5
0
0.5
1x 10!7
t
angular momentum
L1
L2
L3
Figure 6.5: Rigid body with three rotors: energy and components of angular
momentum vector L=LIeIover time (h= 0.1).
145
Figure 6.6: Rigid body with two rotors: configuration at t= 0,...,5 (h= 0.1).
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
!1500
!1000
!500
0
500
1000
1500
t
torque
!"1
!"2
Figure 6.7: Rigid body with two rotors: torque over time (h= 0.1).
146
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
0
1000
2000
3000
4000
t
energy
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
!1
!0.5
0
0.5
1x 10!6
t
angular momentum
L1
L2
L3
Figure 6.8: Rigid body with two rotors: energy and components of angular mo-
mentum vector L=LIeIover time (h= 0.1).
We model the collarbone, the upper and the forearm as multi-body system (see
Figure 6.9), where the single bodies are interconnected by joints and actuated via
external control torques representing the muscle activation.
Model The first rigid body, representing the collarbone is assumed to be fixed
in the inertial frame via a revolute joint modelling the rotation of the torso
around the e3-axes. Collarbone and upper arm are connected via a spherical
joint, representing the three dimensional rotation of the shoulder. A revolute
joint serves as the elbow between upper and forearm allowing the forearm to
rotate around a prescribed axis fixed in the upper arm.
The actuations via the muscles are modeled as external torques τi, i = 1,2,3,
acting in the joints. Here, it is assumed that all degrees of freedom, that is the
rotations of the collarbone, the shoulder and the elbow, are directly steerable.
That means there is one rotational torque τs∈R3acting in the shoulder joint
and two scalar torques τcand τe∈Racting in the first revolute joint and the elbow
joint, respectively. We describe the system by five generalized joint coordinates
θ1, θ5∈S1,and θs= (θ2, θ3, θ4)∈SO(3) that constitute the degrees of freedom
actuated by the torques.
147
θ4
θ1
θ2
θ3
θ5
l1
l2
l3
collarbone
upper arm
forearm
e1
e2
e3
Figure 6.9: Model for a pitcher’s arm consisting of collarbone, upper arm, and
fore arm.
Boundary conditions, objective functional and path constraints The
pitcher is assumed to begin his motion with prescribed initial configuration and
zero velocity. Rather than prescribing final configurations for all present bodies,
we formulate a limited, but not fixed final configuration of the hand position2,
for example positive e2- and e3-position. Due to the human body’s anatomy each
joint has only a limited degree of freedom. To obtain realistic motions, each
generalized configuration variable is bounded as θl
i≤θi≤θu
i, i = 1,...,5, for
example the forearm is assumed to bend in only one direction. In addition we also
need to incorporate bounds τl
i≤τi≤τu
i, i = 1,...,5, on the control torques,
since the muscles are not able to create an arbitrary amount of strength.
The goal is to maximize the velocity of the hand in e2-direction. During the
optimization the final time is free, that means we also determine the optimal
duration of the pitch.
Numerical results Starting from an initial position of the joints as θ1=θ2=
θ3=θ4= 0, θ5=−π
4, we obtain, depending on the initial guesses for the op-
timization problem, different solutions for the optimal motion. In Figure 6.10
2Since the hand is not modeled as a separate rigid body within the system, we assume it to
be located at the endpoint of the forearm.
148
snapshots of a particular locally optimal motion are depicted with the opti-
mal final time T= 0.306. The final configuration and velocity of the hand is
qhand = (0.009,−0.662,0.363) and ˙qhand = (−3.359,−48.575,21.070). Starting
from the initial configuration shown in the first picture, the pitcher strikes out
rearwards, pulls his arm above his head, before he finally moves his arm like a
whip to obtain the necessary swing to maximize the final velocity.
This application demonstrates the capacity of the developed method for the
optimal control of constrained multi-body dynamics in biomotion. Based on
first results presented within this thesis, an analysis of different local optimal
solutions varying in the values for the objective functional, the time duration and
the applied control effort is investigated in [151].
To obtain more realistic motions, the next step is to consider more complex
and thus more realistic models that behave more realistically. For example, in-
stead of modeling the actuation of the limbs by external control torques, the
interaction of the muscles and the resulting muscle force can be modeled as well
(as also investigated in [151]). Due to the constrained formulation of multi-body
dynamics, model extensions can easily be incorporated by coupling new bodies
to the system via constraints.
149
t= 0.0t= 0.034 t= 0.068
t= 0.102 t= 0.136 t= 0.17
t= 0.204 t= 0.238 t= 0.272
t= 0.296 t= 0.306
Figure 6.10: The optimal pitch: snapshots of the motion sequence.
150
Chapter 7
Conclusions and outlook
Analogous to the variational formulation of the continuous optimal control of
mechanical systems, in this thesis a fully discrete formulation is developed. Based
on discrete variational principles on all levels the discrete optimal control problem
yields a strategy for the efficient solution of these kinds of problems. This optimal
control methodology is denoted by DMOC.
In contrast to other methods that rely on a direct integration of the associated
ordinary differential equations or on its fulfillment at certain grid points (for
example shooting, multiple shooting, or collocation methods), DMOC is based
on the discretization of the variational structure of the mechanical system. In the
context of variational integrators, the discretization of the Lagrange-d’Alembert
principle leads to structure-preserving time-stepping equations which serve as
equality constraints for the resulting finite dimensional nonlinear optimization
problem. This can be solved by standard nonlinear optimization techniques like
sequential quadratic programming.
The benefits of variational integrators are passed on to the optimal control
context. For example in the presence of symmetry groups in the continuous
dynamical system, also along the discrete trajectory the change in momentum
maps is consistent with the control forces (as also numerically shown for specific
examples). Choosing the objective functional to represent the control effort to
be minimized is only meaningful if the system responds exactly according to the
control forces.
To discretize the optimal control problem, we approximate the tangent space
via two copies of the configuration space itself, that means, the algorithm works on
the configuration level rather than on the configuration-momentum or configura-
tion-velocity level. This yields computational savings as we just determine the
optimal trajectory for the configuration and the control forces and reconstruct the
corresponding momenta and velocities via the forced discrete Legendre transform.
The computational savings are numerically verified via a CPU time comparison
151
between DMOC and a collocation approach of the same order.
Due to the symplecticity of variational integrators, the order of approximation
of the discrete adjoint dynamics is the same as for the state dynamics. This is in
contrast to other standard non-symplectic discretizations. This fact guarantees a
convergence rate of the algorithm depending on the order of discretization of the
underlying mechanical system and is numerically confirmed via the convergence
rate computation in a simple standard example.
Besides theoretical and numerical investigations of the proposed optimal con-
trol method, we developed efficient decentralized computational approaches to
exploit system structures based on a hierarchical decomposition of the problem
under consideration. This approach reduces the discrete optimal control system
to a set of several smaller identical subproblems. In this way, the computational
effort of the optimization algorithm can be decreased significantly by parallel
computations as numerically demonstrated.
Finally, we developed an efficient approach to solving problems in mechanics
with holonomic constraints occurring in multi-body dynamics. The presented
strategy enables us to treat interesting and exciting applications in robotics and
biomechanics in an accurate and computationally efficient way.
Outlook
Open problems
Although the developed method works very successfully in the examples consid-
ered within this thesis, there are still some open problems and challenges.
Evolution of conserved quantities for controlled systems When applying
symplectic integration to a conservative system, a certain modified energy of
the original system is conserved (see for example [55]). This is an important
property if the long-term behavior of dynamical systems is considered. When
forcing or dissipation is added to a Lagrangian or a Hamiltonian system, then
the symplectic form, the momentum maps and the energy are no longer preserved
by the flow. However, it is often important to be able to correctly simulate the
amount by which these quantities change over time. For the case of the optimal
control of systems with a long maneuver time, such as low thrust space missions,
it would be interesting to investigate the balance between modified energy and
virtual work. The variational framework for discrete systems with forcing given in
Section 3 offers a way in which this evolution can be studied, both for the discrete
system and for the true Lagrangian system. Concepts and ideas of variational
152
backward error analysis and variational integrators with forcing might be helpful
for theoretical investigations.
Convergence for state constraint problems From an optimal control point
of view a convergence result in the presence of state constraints would be impor-
tant and interesting. In particular, for applications from biomechanics involving
the human body, the incorporation of appropriate bounds on the joints is crucial
to guarantee realistic motions that are in accordance with the body’s anatomy.
These state constraints complicate the dynamics of the system and pose theore-
tical questions about the convergence of the used control method. On the other
hand, the application of homotopy or continuation techniques might be helpful
to improve and speed up the convergence of the optimization algorithm. The
solution of a simplified problem with relaxed or even no state constraints can be
useful to find a solution for a less simplified problem.
Future directions
Having built the appropriate variational framework for the optimal control of
mechanical systems, there are many directions one could focus on for future
investigations.
The framework could be extended to the optimal control of mechanical sys-
tems with stochastic influence or contact problems making use of the the-
ory of stochastic ([26]) and nonsmooth variational integrators ([44]), respectively.
Due to the variational formulation of DMOC, an extension towards the optimal
control of partial differential equations for the treatment of fluid and conti-
nuum mechanics might be interesting as well (see [112, 108] for basic extensions
of variational integrators to the context of PDEs).
From a computational point of view, the savings in CPU time using hierar-
chical decentralized approaches indicate worthwhile extensions of a spatial and
time hierarchical decomposition of complex networked systems.
Appropriate areas of application comprising the proposed extensions are hy-
brid optimal control systems and the optimal control of complex multi-body sys-
tems in biomechanics:
Hybrid systems Approaches from variational collision integrators might be
helpful to find a variational formulation for hybrid optimal control systems, in
particular, for systems whose discrete behavior is determined via contact condi-
tions describing instantaneous changes in velocity, acceleration, and forces. In
the second place, an efficient computational approach has to be developed to
treat the discontinuities within the optimal control problem. In general, both
153
the collision point and the collision time are a priori unknown. In principle, a
multi-phase optimal control problem with unknown phase transition times and
states is associated with each of the discrete state sequence candidates. Akin to
the decentralized method applied for the control of formation flying satellites in
Section 5.5, a decomposition approach is possible: in an inner loop multi-phase
optimal control problems are numerically solved for a given sequence of phases in
parallel whereas the order and type of phases is varied during the outer iteration.
A simplified version of this approach was already applied to the optimal control
of a compass gait biped ([124]).
Relevant applications are the actuation of systems with contact, for example
collisions of vehicle formations or walking robots. In addition, the developed
concepts and methods for the control of hybrid systems find an ideal application
in the control of biomechanical systems, such as the contact of club and ball in
optimizing a golf swing.
Complex multi-body systems in biomechanics It is of great interest to
understand the muscle activation and coordination of the human body for several
reasons, for example to construct prostheses and implants in modern medical
surgery. Another area of growing interest is the optimization of movements in
sports. The knowledge gained from optimal control simulations might help to
improve individual techniques or might even suggest the development of new
techniques (as it has been observed during the last decades, for example for
high and ski jumping). In order to optimize the human body’s movements, the
development of robust models, structure exploiting control methods, and efficient
computational techniques are in demand.
Due to the high complexity of the dynamics for an increasing number of
connected rigid bodies, it is important to identify those limbs and those parts of
the body that are important for the underlying task. The activation of muscles
depending on the configuration and velocity of the corresponding joint angle has
to be included in the model. The initial steps in that direction are presented in
[151]. Here, the motion of a pitcher’s arm is optimized, with the muscles modeled
by the Hill model; ([148]) the activation levels are treated as control parameters.
Besides the incorporation of appropriate bounds on the joints due to the human
body’s anatomy, the choice of an objective functional is important for the quality
of the optimal solution. Depending on the specific task, one or multiple objectives
have to be determined. The development of a hierarchical decomposition of the
overall system is applicable for these kinds of applications. Here, optimal solutions
for a system consisting of only the most relevant parts of the human body are
used to create solutions for more complex models including additional limbs.
Due to the constrained formulation of multi-body dynamics, model extensions
154
can easily be incorporated by coupling new bodies to the system via constraints.
This procedure is assumed to improve and speed up the convergence properties
of the optimization algorithm of the complex dynamical multi-body systems.
Convergence and optimality of solutions resulting from the hierarchical approach
should be compared to a non-hierarchical one. In addition, different models and
different objectives for different phases of the motion sequence are logical (see for
example the different phases of a long jump motion).
Furthermore, to account for the uncertainties arising from the discrepancy
between the model and reality and the difficulty of measuring model parameters
as for the muscles, a stochastic component can be included in the model.
In summary, this thesis gives initial impulses in the direction of a complete
discrete variational point of view of the optimal control in mechanics. As indi-
cated by the long list of possible future directions, it is an exciting and worthwhile
area of research where there is still a large potential for the development of new
and efficient techniques for relevant applications.
155
Appendix A
Definitions
The following standard definitions are mainly based on the representations in
[111].
Definition A.1 (Differentiable manifold) Given a set M, a chart on Mis an
open set Uin Euclidean space Rnwith coordinates (x1, . . . , xn) (more generally
Ucan be open in a Banach space) together with a one-to-one map ϕof Uonto
some subset of M,
ϕ:U→ϕ(U)⊂M.
We call Madifferentiable manifold if the following holds:
(M1) It is covered by a collection of charts, that is, every point is represented in
at least one chart.
(M2) If two charts U, U0have an overlapping image in M, then V=ϕ−1(ϕ(U)∩
ϕ0(U0)) and V0= (ϕ0)−1(ϕ(U)∩ϕ0(U0)) are open sets in Rn. Hence the
mapping ϕ0−1◦ϕ:V→V0from an open subset of Rnto a subset of Rn
is defined. The charts U, U0are called compatible if these nfunctions of n
variables ϕ0−1◦ϕare C∞.
(M3) Mhas an atlas, that is, Mcan be written as a union of compatible charts.
Two atlases are called equivalent if their union is also an atlas. One often rephrases
the definition by saying that a differentiable structure on a manifold is an equi-
valence class of atlases.
Aneighborhood of a point xin a manifold Mis the image under a map ϕ:
U→Mof a neighborhood of the representation of xin a chart U. Neighborhoods
define open sets and the open sets in Mdefine a topology, that is assumed to be
Hausdorff. A differentiable manifold Mis called an n-manifold if every chart has
domain in an n-dimensional vector space.
157
Definition A.2 (Differentiable submanifold) A subset Sof a differentiable
manifold Mis a differentiable submanifold if, for each point x∈S, there is an
admissible chart (U, ϕ) for Mwith x∈U, and such that
(i) ϕtakes its value in a product Rk×Rn−k, and
(ii) ϕ(U∩S) = ϕ(U)∩(Rk× {0}).
A chart with these properties is a submanifold chart for S.
Definition A.3 (Tangent vector, tangent space, tangent bundle) Two
curves t7→ c1and t7→ c2in an n-manifold Mare called equivalent at xif
c1(0) = c2(0) = xand (ϕ−1◦c1)0(0) = (ϕ−1◦c2)0(0)
in some chart ϕ. This definition is chart independent. A tangent vector vto a
manifold Mat a point x∈Mis an equivalence class of curves at x. Let Ube
a chart of an atlas for Mwith coordinates (x1, . . . , xn). The components of the
tangent vector vto the curve t7→ (ϕ−1◦c)(t) are defined by
vi=d
dt(ϕ−1◦c)it=0
, i = 1, . . . , n.
The set of tangent vectors to Mat xforms a vector space, called the tangent
space to Mat x, denoted by TxM. The tangent bundle of M, denoted by TM,
is the differentiable manifold whose underlying set is the disjoint union of the
tangent spaces to Mat the points x∈M, that is,
TM =[
x∈M
TxM.
Let x1, . . . , xnbe local coordinates on Mand let v1, . . . , vnbe components of a
tangent vector in this coordinate system. Then the 2nnumbers
x1, . . . , xn, v1, . . . , vngive a local coordinate system on TM.
Definition A.4 (Natural projection, fiber) The natural projection is the map
τM:TM →Mthat takes a tangent vector vto the point x∈Mat which the
vector is attached. The inverse image τ−1
M(x) of a point x∈Munder the natural
projection τMis the tangent space TxM. This space is called the fiber of the
tangent bundle over the point x∈M.
Definition A.5 (Derivative) Let f:M→Nbe a map of a manifold Mto a
manifold N.fis called differentiable (or Ck) if in local coordinates on Mand N
158
it is given by differentiable (or Ck) functions. The derivative (or tangent lift) of a
differentiable map f:M→Nat a point x∈Mis defined to be the linear map
Txf:TxM→Tf(x)N
constructed in the following way: For v∈TxM, choose a curve c:] −, [→M
with c(0) = x, and velocity vector dc/dt|t=0 =v. Then Txf·vis the velocity
vector at t= 0 of the curve f◦c:R→N, that is
Txf·v=d
dtf(c(t))t=0
.
The vector Txf·vdoes not depend on the curve cbut only on the vector v. If
Mand Nare manifolds and f:M→Nis of class Cr+1, then Tf :TM →TN
is a mapping of class Cr.
Definition A.6 (Vector field, integral curve, flow) Avector field Xon a
manifold Mis a map X:M→TM that assigns a vector X(x) at the point
x∈M, that is, τM◦X=id. An integral curve of Xwith initial condition x0
at t= 0 is a (differentiable) map c:]a, b[→Msuch that ]a, b[ is an open interval
containing 0, c(0) = x0and
c0(t) = X(c(t))
for all t∈]a, b[. The flow of Xis the collection of maps
ϕt:M→M
such that t7→ ϕt(x) is the integral curve of Xwith initial condition x.
Definition A.7 (Differential, cotangent bundle) Let Mbe a manifold and
f:M→Rbe a smooth function, we can differentiate it at any point x∈Mto
obtain a map Txf:TxM→Tf(x)R. Identifying the tangent space of Rat any
point with itself, we get the linear map df(x) : TxM→R. That is df(x)∈T∗
xM,
the dual of the vector space TxM, and reads in coordinates
df(x)·v=
n
X
i=1
∂f
∂xivi,
with v∈TxM. (We will employ the summation convention and drop the sum-
mation sign when there are repeated indices.) dfis called the differential of
f.
We can identify a basis of TxMusing the operators ∂/∂xiand write
(e1, . . . , en) = ∂
∂x1,..., ∂
∂xn
159
for this basis so that v=vi∂/∂xi.
If we replace each vector space TxMwith its dual T∗
xM, we obtain a new
2n-manifold called the cotangent bundle and denoted T∗M. The dual basis to
∂/∂xiis denoted dxi. Thus, relative to a choice of local coordinates we get the
basic formula
df(x) = ∂f
∂xidxi
for any smooth function f:M→R.
Definition A.8 (Differential form) Atwo-form Ω on Mis a function Ω(x) :
TxM×TxM→Rthat assigns to each point x∈Ma skew-symmetric bilinear
form on the tangent space TxMto Mat x. More generally, a k-form α(also
called differentiable form of degree k) on a manifold Mis a function α(x) :
TxM× · · · × TxM(kfactors) →Rthat assigns to each point x∈Ma skew-
symmetric k-multilinear map on the tangent space TxMto Mat x.
Let x1, . . . , xndenote coordinates on M, let
{e1, . . . , en}={∂/∂x1, . . . , ∂/∂xn}
be the corresponding basis for TxM, and let {e1, . . . , en}={dx1, . . . , dxn}be the
dual basis for T∗
xM. Then at each x∈M, we can write a two-form as
Ωx(v, w) = Ωij(x)viwj,where Ωij(x) = Ωx∂
∂xi,∂
∂xj,
and more generally a k-form can be written
αx(v1, . . . , vk) = αi1...ik(x)vi1
1. . . vik
k,
summed on i1, . . . , ik, with
αi1...ik(x) = αx∂
∂xi1,..., ∂
∂xik,
and where vi=vj
i∂/∂xj, with a sum on j.
Definition A.9 (Pullback, pushforward) Let ϕ:M→Nbe a C∞map from
the manifold Mto the manifold Nand αbe a k-form on N. The pullback ϕ∗α
of αby ϕis the k-form on Mgiven by
(ϕ∗α)x(v1, . . . , vk) = αϕ(x)(Txϕ·v1, . . . , Txϕ·vk).
If Yis a vector field on the manifold Nand ϕis a diffeomorphism, the pullback
ϕ∗Yis a vector field on Mdefined by
(ϕ∗Y)(x) = Txϕ−1◦Y◦ϕ.
If ϕis a diffeomorphism, the pushforward ϕ∗is defined by ϕ∗= (ϕ−1)∗.
160
Definition A.10 (Interior product) Let αbe a k-form on a manifold Mand
Xa vector field. The interior product iXα(also called the contraction of Xand
α, and written i(X)α) is the (k−1)-form
(iXα)x(v2, . . . , vk) = αx(X(x), v2, . . . , vk)
for x∈Mand (v2, . . . , vk)∈TxM.
Definition A.11 (Exterior derivative) The exterior derivative dαof a k-form
αon Mis the (k+ 1)-form on M, which is uniquely determined by the following
properties:
(i) If αis a 0-form, that is, α=f∈C∞(M), then dfis the one-form which is
the differential of f.
(ii) dαis linear in α.
(iii) dαsatisfies the product rule, that is,
d(α∧β) = dα∧β+ (−1)kα∧dβ,
where αis a k-form and βis an l-form.
(iv) d2= 0, that is, d(dα) = 0 for any k-form α.
(v) dis a local operator, that is, dα(x) only depends on αrestricted to any
open neighborhood of x. If Uis open in M, then
d(α|U) = (dα)|U.
Definition A.12 (Jacobi-Lie bracket) Let Mbe a smooth C∞manifold, f∈
F(M) (with F(M) the set of continuously differentiable real-valued functions on
M) and X, Y :M→TM two vector fields on M. Then the derivation
f7→ X[Y[f]] −Y[X[f]],
where X[f] = df·X, determines a unique vector field denoted by [X, Y ] and
called the Jacobi-Lie bracket of Xand Y.
Definition A.13 (Cotangent lift) Given two manifolds Mand Nand a dif-
feomorphism f:M→N, the cotangent lift T∗f:T∗N→T∗Mof fis defined
by
hT∗f(αs), vi=hαs,(Tf ·v)i,
where
αs∈T∗
sN, v ∈TqM, and s=f(q).
161
Definition A.14 (Lie group) ALie group is a differentiable manifold Gthat
has a group structure consistent with its manifold structure in the sense that the
group operations
µ:G×G→G; (g, h)7→ gh
I:G→G;g7→ g−1
are C∞maps. The maps Lg:G→G;h7→ gh, and Rh:G→G;g7→ gh are
called the left and right translation maps.
Definition A.15 (Left invariant vector field) A vector field Xon Gis called
left invariant if for every g∈G,L∗
gX=X, that is, if
(ThLg)X(h) = X(gh)
for every h∈G.
Definition A.16 (Lie bracket, Lie algebra) Let Vbe a vector space and [·,·] :
V×V→VaLie bracket, i. e. it is a bilinear, skew-symmetric map which fulfills
the Jacobi-identity
[A, [B, C]] + [B, [C, A]] + [C, [A, B]] = 0 for all A, B, C ∈V.
Then the pair (V, [·,·]) is called Lie algebra.
More specific, every Lie group Ginduces a Lie algebra: the vector space XL(G)
of left invariant vector fields on Gis isomorphic to the tangential space TeGto
Gat the neutral element e. Define the Lie bracket for ξ, η ∈TeGby
[ξ, η] := [Xξ, Xη](e),
where Xξ, Xηare the vector fields induced by ξand η, respectively, and [Xξ, Xη]
is the Jacobi-Lie bracket of vector fields. This makes TeGinto a Lie algebra. It
is denoted by g= Lie(G) and is called Lie algebra of G.
Definition A.17 (Exponential map) Let Gbe a Lie group. For all ξ∈g=
Lie(G) let γξ:R→Gdenote the integral curve of the left-invariant vector field
Xξon Ginduced by ξ, which is defined uniquely by claiming
Xξ(e) = ξ, γξ(0) = e, γ0
ξ(t) = Xξ(γξ(t)) for all t∈R.
The map
exp : g→G, exp(ξ) = γξ(1)
is called exponential map of the Lie algebra gin G.
162
Definition A.18 (Action of Lie groups) Let Mbe a manifold and let Gbe
a Lie group. A (left)action of a Lie group Gon Mis a smooth mapping φ:
G×M→Msuch that:
(i) φ(e, x) = xfor all x∈M, and
(ii) φ(g, φ(h, x)) = φ(gh, x) for all g, h ∈Gand x∈M.
Aright action is a map Ψ : M×G→Mthat satisfies Ψ(x, e) = xand
Ψ(Ψ(x, g), h) = Ψ(x, gh).
Definition A.19 (Infinitesimal generator) Suppose φ:G×M→Mis an
action. For ξ∈g, the map φξ:R×M→M, defined by φξ(t, x) = φ(exp tξ, x),
is an R-action on M. In other words, φexp tξ :M→Mis a flow on M. The
corresponding vector field on M, given by
ξM(x) := d
dtt=0
φexp tξ(x),
is called the infinitesimal generator of the action corresponding to ξ.
Definition A.20 (Symplectic manifold) Asymplectic manifold is a pair (P, Ω)
where Pis a manifold and Ω is a closed (weakly) nondegenerated two-form on P.
Ω is called closed if dΩ = 0, where dis the exterior derivative, and it is
called weakly nondegenerated if for z∈Pthe induced map Ω[
z:TzP→T∗
zPwith
Ω[
z(x)(y) = Ωz(x, y) is injective, i. e. let x∈TzP, if Ωz(x, y) = 0 for all y∈TzP
then x= 0. In the case of strong nondegeneracy Ω[
zis an isomorphism.
If Pis finite dimensional, weak nondegeneracy and strong degeneracy are
equivalent.
Definition A.21 (Symplectic / canonical transformation) A differentiable
map f:P1→P2between symplectic manifolds (P1,Ω1) and (P2,Ω2) is called
symplectic (or canonical transformation) if
f∗Ω2= Ω1.
That is, by definition of the pullback of a two-form
(f∗Ω2)z(x, y) = Ω2f(z)(Tzf(x), Tzf(y)) = Ω1z(x, y)
for each z∈P1and all x, y ∈TzP1, with the derivative Tzf:TzP1→Tf(z)P2.
163
Definition A.22 (Momentum map) Let a Lie algebra gact canonically (on
the left) on the symplectic manifold P. Suppose there is a linear map J:g→
F(P) such that
XJ(ξ)=ξP
for all ξ∈g. The map J:P→g∗defined by
hJ(z), ξi=J(ξ)(z)
for all ξ∈gand z∈Pis called a momentum mapping of the action.
Definition A.23 (Hamiltonian momentum map) Let the Lie algebra gact
on the left on the manifold M, such that gacts on P=T∗Mon the left by the
canonical action ξP=ξ0
M, where ξ0
Mis the cotangent lift of ξMto Pand ξ∈g.
Then, the Hamiltonian momentum map JH:P→g∗is given by
hJH(αq), ξi=hαq, ξM(q)i.
Definition A.24 (Lagrangian momentum map) Let the Lie algebra gact
on the left on the manifold Mand assume that L:TM →Ris a regular La-
grangian. Endow TM with the Lagrangian symplectic form ΩL= (FL)∗Ω, where
Ω = −dΘ is the canonical symplectic form on T∗M. Then gacts canonically on
P=TM by
ξP(vq) = d
dtt=0
Tqϕt(vq),
where ϕtis the flow of ξMand the Lagrangian momentum map JL:TM →g∗is
given by
hJL(vq), ξi=hFL(vq), ξM(q)i.
Definition A.25 (Fiber-preserving map) A diffeomorphism ϕ:T∗S→T∗Q
is a fiber-preserving map iff
f◦πQ=πS◦ϕ−1,
with πQ:T∗Q→Qand πS:T∗S→Sthe canonical projections and where
f:Q→Sis defined by f=ϕ−1|Q. For ϕ:T∗Q→T∗Qbeing a fiber-preserving
map over the identity it holds
id ◦πQ=πQ◦ϕ−1,
and therefore, we have πQ◦ϕ=πQ.
164
Definition A.26 (Vertical vector field, horizontal one-form) A vector field
Yon TQ is vertical if TτQ◦Y= 0, with τQ:TQ →Qthe canonical projection.
Such a vector field Ydefines a horizontal one-form on TQ by
∆Y=−iYΩL,
i. e., ∆Y(U) = 0 for any vertical vector field Uon TQ.
Proposition A.1 Any fiber-preserving map F:TQ →T∗Qover the identity
induces a horizontal one-form ˜
Fon TQ by
˜
F(v)·Vv=hF(v), Tvτ(Vv)i,(A.1)
where v∈TQ and Vv∈Tv(TQ). Conversely, equation (A.1) defines, for any
horizontal one-form ˜
Fa fiber-preserving map Fover the identity.
Proof: A proof can be found in [111].
Definition A.27 (Variation) Let γ: [a, b]→Mbe a C2-curve. A variation of
γis a C2-map ϑ:J×[a, b]→Mwith the properties
(i) J⊂Ris an interval for which 0 ∈int(J),
(ii) ϑ(0, t) = γ(t) for all t∈[a, b],
(iii) ϑ(s, a) = γ(a) for all s∈J, and
(iv) ϑ(s, b) = γ(b) for all s∈J.
The infinitesimal variation associated with a variation ϑis the vector field along
γgiven by
δϑ(t) = d
dss=0
ϑ(s, t)∈Tγ(t)M.
165
Appendix B
Adjoint system
This section presents in detail how to derive the transformed adjoint system (4.46)
from the discrete optimal control problem (4.43). Furthermore, its equivalence
to the adjoint system (4.44) is shown.
Suppose that a multiplier λki = (λq
ki, λp
ki) is introduced for the i-th intermedi-
ate equations (4.43c) and (4.43e) on [tk, tk+1] in addition to the multiplier ψk+1 for
equations (4.43b) and (4.43d). Taking into account these additional multipliers,
the Karush-Kuhn-Tucker equations are the following:
ψq
k−ψq
k+1 =
s
X
i=1
λq
ki, ψq
N=∇qΦ(qN, pN),(B.1a)
ψp
k−ψp
k+1 =
s
X
i=1
λp
ki, ψp
N=∇pΦ(qN, pN),(B.1b)
h bjψq
k+1 +
s
X
i=1
λq
kiaq
ij!∇qν(Qkj, Pkj)
+h bjψp
k+1 +
s
X
i=1
λp
kiap
ij!∇qη(Qkj, Pkj, Ukj) = λq
kj,(B.1c)
h bjψq
k+1 +
s
X
i=1
λq
kiaq
ij!∇pν(Qkj, Pkj)
+h bjψp
k+1 +
s
X
i=1
λp
kiap
ij!∇pη(Qkj, Pkj, Ukj) = λp
kj,(B.1d)
Ukj ∈U, − bjψp
k+1 +
s
X
i=1
ap
ijλp
ki!∇uη(Qkj, Pkj, Ukj) = 0,(B.1e)
1≤j≤sand 0 ≤k≤N−1. Here and elsewhere, the dual multipliers are
167
treated as row vectors. In the case bj>0 for each j, we reformulate the first-
order conditions in terms of the variables χkj = (χq
kj, χp
kj) defined by
χq
kj =ψq
k+1 +
s
X
i=1
aq
ij
bj
λq
ki,1≤j≤s, (B.2a)
χp
kj =ψp
k+1 +
s
X
i=1
ap
ij
bj
λp
ki,1≤j≤s. (B.2b)
With this definition, (B.1c) and (B.1d) reduce to
hbjχq
kj∇qν(Qkj, Pkj) + χp
kj∇qη(Qkj, Pkj, Ukj)=λq
kj,(B.3a)
hbjχq
kj∇pν(Qkj, Pkj) + χp
kj∇pη(Qkj, Pkj, Ukj)=λp
kj.(B.3b)
Multiplying (B.3a) by aq
ij/bi, and (B.3b) by ap
ij/bi, summing over j, and substi-
tuting from (B.2a) and (B.2b), respectively, we have
h
s
X
j=1
bjaq
ji
biχq
kj∇qν(Qkj, Pkj) + χp
kj∇qη(Qkj, Pkj, Ukj)=
s
X
j=1
aq
ji
bi
λq
kj =χq
ki −ψq
k+1,
(B.4a)
h
s
X
j=1
bjap
ji
biχq
kj∇pν(Qkj, Pkj) + χp
j∇pη(Qkj, Pkj, Ukj)=
s
X
j=1
ap
ji
bi
λp
kj =χp
ki −ψp
k+1.
(B.4b)
Summing in (B.3) over jand utilizing (B.1a) and (B.1b) gives
h
s
X
i=1
bjχq
kj∇qν(Qkj, Pkj) + χp
kj∇qη(Qkj, Pkj, Ukj)=
s
X
i=1
λq
kj =ψq
k−ψq
k+1,
(B.5a)
h
s
X
i=1
bjχq
kj∇pν(Qkj, Pkj) + χp
kj∇pη(Qkj, Pkj, Ukj)=
s
X
i=1
λp
kj =ψp
k−ψp
k+1.
(B.5b)
Finally, substituting (B.2b) in (B.1e) yields
Ukj ∈U, −bp
jχp
kj∇uη(Qkj, Pkj, Ukj) = 0,1≤j≤s. (B.6)
The positive factor bjin (B.6) can be removed and equations (B.4) - (B.6) yield
the transformed first-order system (4.46).
The remainder of this section shows the equivalence of the two transformed
adjoint systems (B.1) and (4.46), and consequently the equivalence of the adjoint
systems (4.44) and (4.46).
168
Proposition B.1 If bj>0for each j, then the first-order system (B.1) and
the transformed first-oder system (4.46) are equivalent. That is, if λq
1, . . . , λq
s,
λp
1, . . . , λp
ssatisfy (B.1), then (4.46) holds for χq
jand χp
jdefined in (B.2). Con-
versely, if χq
1,...χq
s,χp
1,...χp
ssatisfy (4.46), then (B.1) holds for λq
jand λp
jde-
fined in (B.3).
Proof: We already derived the transformed first-order conditions starting from
the original first-order conditions. Now suppose that χq
1,...χq
s,χp
1,...χp
ssatisfy
the transformed conditions (4.46). Summing over jin (B.3), and utilizing (4.46a)
and (4.46b) yields (B.1a) and (B.1b). To verify (B.1c) - (B.1e), we substitute for
λq
iand λp
iusing (B.3), respectively, to obtain
bjψq
k+1 +
s
P
i=1
aq
ijλq
i=bjψq
k+1 +h
s
P
i=1
biaq
ij (χq
i∇qν(wi, zi) + χp
i∇qη(wi, zi, uki))
=bjψq
k+1 +hbj
s
P
i=1
aq
ij bi
bj(χq
i∇qν(wi, zi) + χp
i∇qη(wi, zi, uki))
=bjχq
j
(B.7)
and
bjψp
k+1 +
s
P
i=1
ap
ijλp
i=bjψp
k+1 +h
s
P
i=1
biap
ij (χq
i∇pν(wi, zi) + χp
i∇pη(wi, zi, uki))
=bjψp
k+1 +hbj
s
P
i=1
ap
ij bi
bj(χq
i∇pν(wi, zi) + χp
i∇pη(wi, zi, uki))
=bjχp
j,
(B.8)
respectively, where the last lines come from (4.46c) and (4.46d). Multiplying (B.7)
on the right by ∇qν(wj, zj), multiplying (B.8) on the right by ∇qη(wj, zj, ukj),
adding these terms and substituting from (B.3a) gives (B.1c). Equivalently, we
obtain (B.1d). Multiplying (B.8) on the right by ∇uη(wj, zj, ukj) and utilizing
(4.46e) yields (B.1e).
With the s×sblock matrix Mwhose (i, j) block is the 2n×2nmatrix
aq
ij∇qν(wj, zj)ap
ij∇pν(wj, zj)
aq
ij∇qη(wj, zj, ukj)ap
ij∇pη(wj, zj, ukj)!,
it follows from [53] (Prop. 3.3), that the multipliers ψq
kand ψp
kobtained by
solving (4.46c) - (4.46d) and substituting into (4.46a) - (4.46b) are identical to
the multipliers obtained from (4.44c) - (4.44d). Moreover, the condition (4.46e)
involving χp
jsatisfying (4.46c) - (4.46d) is equivalent to the condition (4.44e).
169
Appendix C
Convergence proof
This section presents the proof arguments of our convergence result 4.6.2 stated
in Section 4.6.
By applying Proposition 4.6.1 we follow the same arguments as in [53]: We
utilize discrete analogues of various continuous spaces and norms. In particular,
for a sequence z0, z1, . . . , zNwhose i-th element is a vector zi∈Rn, the discrete
analogues of the Lpand L∞norms are the following:
kzkLp= N
X
i=0
h|zi|p!1
p
and kzkL∞= sup
0≤i≤N
|zi|.
With this notation, the space Xin the discrete control problem is the discrete
L∞space consisting of 3-tuples w= (x, ψ, u) where
x= (x0, x1, x2, . . . , xN), xk∈R2n,
ψ= (ψ0, ψ1, ψ2, . . . , ψN), ψk∈R2n,
u= (u0, u1, u2, . . . , uN−1), uk∈Rsm.
The mappings Tand Fof Proposition 4.6.1 are selected in the following way
(according to [53]):
T(x, ψ, u) =
x0
k−fh(xk, uk),0≤k≤N−1
ψ0
k+∇xHh(xk, ψk+1, uk),0≤k≤N−1
∇ujHh(xk, ψk+1, uk),1≤j≤s, 0≤k≤N−1
ψN− ∇C(xN)
and F(x, ψ, u) = 0.
The space Y, associated with the four components of T, is a space of 4-tuples
of finite sequences in L1×L1×L∞×R2n. The reference point w∗is the sequence
171
with elements
w∗
k= (x∗
k, ψ∗
k, u∗
k),
where x∗
k=x∗(tk), ψ∗
k=ψ∗(tk), and u∗
ki =u(y∗
ki, χ∗
ki). Here y∗
ki and χ∗
ki are the
solutions to (4.47e) - (4.47h) corresponding to x=x(tk) and ψ=ψ(tk). The
operator Lis obtained by linearizing around w∗, evaluating all variables on each
interval at the grid point to the left, and dropping terms that vanish at h= 0.
In particular, we choose
L(w) =
x0
k−Akxk−Bkukb, 0≤k≤N−1
ψ0
k+ψk+1Ak+ (qkxk+Skukb)T,0≤k≤N−1
bj(uT
kjRk+xT
kSk+ψk+1Bk),1≤j≤N−2,0≤k≤N−1
ψn+V xN
.
In [41] and [53] Hager examines each of the hypotheses of Proposition 4.6.1 for this
choice of spaces and functions. In [41] (Lemma 5.1) he shows that by smoothness,
k∇T (w)− Lk ≤ k∇T (w)− LkL∞≤c(kw−w∗k+h) (C.1)
for every w∈Bβ(w∗), where βappears in the state uniqueness property. More-
over, by smoothness, coercivity, and [41] (Lemma 6.1), the map (F − L)−1is
Lipschitz continuous with a Lipschitz constant λindependent of hfor hsuffi-
ciently small. Thus we can take σ=∞in Proposition 4.6.1.
Since we focus on the case where U=Rmand F= 0, obtaining an estimate for
kT (w∗)kis equivalent to estimating the distance from T(w) to F(w) to examine
(P1) of Proposition 4.6.1. In [53] Hager shows that the estimation is given by
kT (w∗)k ≤ chκ−1h+τ(u∗(κ−1);h).(C.2)
To complete the proof of Theorem 4.6.2, by using Proposition 4.6.1, let λbe large
enough and let ¯
hbe small enough such that the Lipschitz constant of L−1is less
than λfor all h≤¯
h. Choose small enough such that λ < 1. Choose a small r
and choose ¯
hsmaller if necessary such that c(r+¯
h)≤where cis the constant
appearing (C.1). Finally, choose ¯
hsmaller if necessary such that for the residual
bound in (C.2), we have
c¯
hκ−1¯
h+τ(u∗(κ−1);¯
h)≤(1 −λ)r/λ.
Since the hypotheses of Proposition 4.6.1 are satisfied, we conclude that for each
h≤¯
h, there exists wh= (xh, ψh, uh)∈Br(w∗) such that T(wh) = 0 and the
estimate (4.48) holds, which establishes the bounds for the state and costate vari-
ables in (4.50). The estimate in (4.50) for the error in the control follows from
the control uniqueness property and the fact that ∇uH(x∗(tk), ψ∗(tk), u∗(tk)) = 0.
Finally, by [41] (Lemma 7.2), (xh, uh) is a strict local minimizer in (4.43) for h
sufficiently small.
172
Bibliography
[1] O. Abel, A. Helbig, and W. Marquadt. DYNOPT User Manual (Release 2.4).
Lehrstuhl f¨ur Prozesstechnik, RWTH Aachen, 1999.
[2] U. Ascher, J. Christiansen, and R. D. Russell. A collocation solver for mixed order
systems of boundary value problems. Mathematics of Computation, 33:659–679,
1979.
[3] U. Ascher, R. Mattheij, and R. D. Russell. Numerical solution of Boundary Value
Problems for Differential Equations. SIAM, 1988.
[4] R. Bachmayer, N. E. Leonard, J. Graver, E. Fiorelli, P. Bhatta, and D. Paley. Un-
derwater gliders: Recent developments and future applications. In International
Symposium on Underwater Technology, Taipei, Taiwan, China, 2004.
[5] V. B¨ar. Ein Kollokationsverfahren zur numerischen L¨osung allgemeiner
Mehrpunktrandwertaufgaben mit Schalt- und Sprungbedingungen mit Anwen-
dungen in der Optimalen Steuerung und der Parameteridentifizierung. Diploma
thesis, Bonn, 1983.
[6] A. Barclay, P. E. Gill, and J. B. Rosen. SQP methods and their application
to numerical optimal control. In W. H. Schmidt, K. Heier, L. Bittner, and R.
Bulirsch, editors, Variational Calculus. Optimal Control and Application, 1998.
[7] B. Betsch and S. Leyendecker. The discrete null space method for the energy
consistent integration of constrained mechanical systems. Part II: Multibody dy-
namics. International Journal for Numerical Methods in Engineering, 4:499–552,
2006.
[8] P. Betsch, A. Menzel, and E. Stein. On the parametrization of finite rotations in
computational mechanics; A classification of concepts with application to smooth
shells. Computer Methods in Applied Mechanics and Engineering, 155:273–305,
1998.
[9] P. Betsch and P. Steinmann. Inherently energy conserving time finite elements
for classical mechanics. Computational Physics, 160:88–116, 2000.
173
[10] P. Betsch and P. Steinmann. Constrained integration of rigid body dynamics.
Computer Methods in Applied Mechanics and Engineering, 191:467–488, 2001.
[11] J. T. Betts. Survey of numerical methods for trajectory optimization. AIAA J.
Guidance, Control, and Dynamics, 21(2):193–207, 1998.
[12] J. T. Betts. Practical methods for optimal control using nonlinear programming.
SIAM, Philadelphia, PA, 2001.
[13] J. T. Betts, N. Biehn, and S. L. Campbell. Convergence of nonconvergent IRK dis-
cretizations of optimal control problems with state inequality constraints. SIAM
J. Scientific Computing, 23(6):1981–2007, 2002.
[14] J. T. Betts and P. D. Frank. A sparse nonlinear optimization algorithm. Opti-
mization Theory and Applications, 82(3):519–541, 1994.
[15] J. T. Betts and W. P. Huffmann. Mesh refinement in direct transcription methods
for optimal control. Optimal Control, Applications and Methods, 19:1–21, 1998.
[16] L. T. Biegler. Solution of dynamic optimization problems by successive quadratic
programming and orthogonal collocation. Computers and Chemical Engineering,
8(3/4):243–248, 1984.
[17] T. Binder, L. Blank, H. G. Bock, R. Bulirsch, W. Dahmen, M. Diehl, T. Kro-
nseder, W. Marquardt, J. P. Schl¨oder, and O. von Stryk. Introduction to model
based optimization of chemical processes on moving horizons. In M. Gr¨otschel,
S. O. Krumke, and J. Rambau, editors, Online Optimization of Large Scale Sys-
tems: State of the Art, pages 295–340. Springer, 2001.
[18] A. I. Bobenko and Y. B. Suris. Discrete Lagrangian reduction, discrete Euler-
Poincar´e equations, and semidirect products. Letters in Mathematical Physics,
49:79–93, 1999.
[19] A. I. Bobenko and Y. B. Suris. Discrete time Lagrangian mechanics on Lie
groups, with an application to the Lagrange top. Communication in Mathematical
Physics, 204:147–188, 1999.
[20] H. G. Bock. Numerical solutions of nonlinear multipoint boundary value problems
with applications to optimal control. ZAMM, 58(T407), 1978.
[21] H. G. Bock and K. J. Plitt. A multiple shooting algorithm for direct solution
of optimal control problems. In 9th IFAC World Congress, pages 242–247, Bu-
dapest, Hungary, 1984. Pergamon Press.
[22] P. T. Boggs and J. W. Tolle. Sequential quadratic programming. Acta Numerica,
4:1–50, 1995.
174
[23] J. F. Bonnans and J. Laurent-Varin. Computation of order conditions for sym-
plectic partitioned Runge-Kutta schemes with application to optimal control.
Numerische Mathematik, 103(1):1–10, 2006.
[24] O. Bonorden, M. Dynia, J. Gehweiler, and R. Wanka. Pub-library - user guide
and function reference. Release 8.1-pre, 2003.
[25] C. L. Bottasso and A. Croce. Optimal control of multibody systems using an en-
ergy preserving direct transcription method. Multibody System Dynamics, 12(1),
2004.
[26] N. Bou-Rabee and H. Owhadi. Stochastic variational integrators. Submitted,
2007.
[27] A. E. Bryson and Y. C. Ho. Applied Optimal Control. Hemisphere, 1975.
[28] R. Bulirsch. Die Mehrzielmethode zur numerischen L¨osung von nichtlinearen
Randwertproblemen und Aufgaben der optimalen Steuerung. Report of the Carl-
Cranz-Gesellschaft e.V., DLR, Oberpfaffenhofen, 1971.
[29] J. A. Cadzow. Discrete calculus of variations. International Journal of Control,
11:393–407, 1970.
[30] J. A. Cadzow. Discrete-Time Systems: an introduction with interdisciplinary
applications. Prentice-Hall, 1973.
[31] A. L. Cauchy. M´ethode g´en´erale pour la r´esolution syst`emes d’´equations
simultan´ees. Comptes rendus de l’Acad´emie des scinces, 25:536–538, 1847.
[32] A. Cervantes and L. T. Biegler. Large-scale DAE optimization using a simulta-
neous NLP formulation. AIChE Journal, 44(5):1038–1050, 1998.
[33] F. L. Chernousko and A. A. Luybushin. Method of successive approximations
for optimal control problems (survey paper). Optimal Control, Applications and
Methods, 3:101–114, 1982.
[34] M. Crisfield and J. Shi. An energy conserving co-rotational procedure for non-
linear dynamics with finite elements. Nonlinear Dynamics, 9:37–52, 1996.
[35] M. Dellnitz, O. Junge, A. Krishnamurthy, S. Ober-Bl¨obaum, K. Padberg, and R.
Preis. Efficient control of formation flying spacecraft. In Friedhelm Meyer auf
der Heide and Burkhard Monien, editors, New Trends in Parallel & Distributed
Computing, volume 181, pages 235–247. Heinz Nixdorf Institut Verlagsschrift-
reihe, 2006.
[36] P. Deuflhard. A modified newton method for the solution of ill-conditioned sys-
tems of nonlinear equations with application to multiple shooting. Numerische
Mathematik, 22:289–315, 1974.
175
[37] E. D. Dickmanns and K. H. Well. Approximate solution of optimal control prob-
lems using third order hermite polynomial functions. Lecture Notes in Computer
Science, 27:158–166, 1975.
[38] A. L. Dontchev and W. W. Hager. Lipschitzian stability in nonlinear control and
optimization. SIAM J. Control and Optimization, 31:569–603, 1993.
[39] A. L. Dontchev and W. W. Hager. The Euler approximation in state constrained
optimal control. Mathematics of Computation, 70:173–203, 2001.
[40] A. L. Dontchev, W. W. Hager, A. B. Poore, and B. Yang. Optimality, stability
and convergence in nonlinear control. Applied Mathematics and Optimization,
31:297–326, 1995.
[41] A. L. Dontchev, W. W. Hager, and V. M. Veliov. Second order Runge-Kutta
approximations in control constrained optimal control. SIAM J. Numerical Ana-
lysis, 38:202–226, 2000.
[42] G. Engl, A. Kr¨oner, T. Kronseder, and O. von Stryk. Numerical simulation
and optimal control of air separation plants. In H.-J. Bungartz, F. Durst, and
Chr. Zenger, editors, High Performance Scientific and Engineering Computing,
volume 8 of Lecture Notes in Computational Science and Engineering, pages 221–
231. Springer, 1999.
[43] A. Fedorova and M. Zeitlin. Wavelets in dynamics, optimal control and Galerkin
approximations. In IEEE Digital Signal Processing Workshop, pages 409–412,
Norway, 1996.
[44] R. C. Fetecau, J. E. Marsden, M. Ortiz, and M. West. Nonsmooth Lagrangian
mechanics and variational collision integrators. SIAM J. Applied Dynamical Sys-
tems, 2(3):381–416, 2003.
[45] E. Fiorelli, N. E. Leonard, P. Bhatta, D. Paley, R. Bachmayer, and D.M. Fratan-
toni. Multi-AUV control and adaptive sampling in Monterey Bay. In IEEE
Autonomous Underwater Vehicles 2004: Workshop on Multiple AUV Operations
(AUV04), Sebasco, ME, 2004.
[46] L. Flatto. Advanced Calculus. Williams & Wilkins, 1976.
[47] L. Fox. Some numerical experiments with eigenvalue problems in ordinary differ-
ential equations. In R. E. Langer, editor, Boundary Value Problems in Differential
Equations, 1960.
[48] P. E. Gill, W. Murray, and M. A. Saunders. SNOPT: An SQP algorithm for large-
scale constrained optimization. Report NA 97-2, Department of Mathematics,
University of California, San Diego, CA, USA, 1997.
176
[49] P. E. Gill, W. Murray, M. A. Saunders, and M. H. Wright. User’s guide for
NPSOL (version 5.0): a Fortran package for nonlinear programming. Numerical
Analysis Report 98-2, Department of Mathematics, University of California, San
Diego, CA, USA, 1998.
[50] H. Goldstein, C. Poole, and J. Safko. Classical Mechanics. Addison Wesley, 2002.
[51] A. Griewank. Evaluating Derivatives: Principles and Techniques of Algorithmic
Differentiation. SIAM, 2000.
[52] W. W. Hager. Convex control and dual approximations. In Constructive Ap-
proaches to Mathematical Models, pages 189–202. Academic Press, New York,
1979.
[53] W. W. Hager. Runge-Kutta methods in optimal control and the transformed
adjoint system. Numerische Mathematik, 87:247–282, 2000.
[54] W. W. Hager. Numerical analysis in optimal control. In International Series
of Numerical Mathematics, volume 139, pages 83–93. Birkh¨auser Verlag, Basel,
Switzerland, 2001.
[55] E. Hairer, C. Lubich, and G. Wanner. Geometric numerical integration, volume 31
of Springer Series in Computational Mathematics. Springer, 2002.
[56] E. Hairer, G. Wanner, and C. Lubich. Geometric Numerical Integration:
Structure- Preserving Algorithms for Ordinary Differential Equations. Springer,
2004.
[57] S. P. Han. Superlinearly convergent variable-metric algorithms for general non-
linear programming problems. Mathematical Programming, 11:263–282, 1976.
[58] R. F. Hartl, S.P. Sethi, and R.G. Vickson. A survey of the maximum principle for
optimal control problems with state constraints. SIAM Review, 37(2):181–218,
1995.
[59] M. R. Hestenes. Calculus of Variations and Optimal Control Theory. John Wiley
& Sons, 1966.
[60] P. Hiltmann. Numerische L¨osung von Mehrpunkt-Randwertproblemen und Auf-
gaben der optimalen Steuerung ¨uber endlichdimensionalen R¨aumen. PhD thesis,
Fakult¨at f¨ur Mathematik und Informatik, Technische Universit¨at M¨unchen, 1990.
[61] C. L. Hwang and L. T. Fan. A discrete version of Pontryagin’s maximum princi-
ple. Operations Research, 15:139–146, 1967.
[62] T. Inanc, S. C. Shadden, and J. E. Marsden. Optimal trajectory generation in
ocean flows. In American Control Conference, Portland, OR, USA, 2005.
177
[63] R. Isaacs. Differential Games: A Mathematical Theory with Applications to War-
fare and Pursuit, Control and Optimization. John Wiley & Sons, 1965.
[64] F. Jarre and J. Stoer. Optimierung. Springer, 2004.
[65] B. W. Jordan and E. Polak. Theroy of a class of discrete optimal control systems.
Journal of Electronics and Control, 17:697–711, 1964.
[66] O. Junge, J. Levenhagen, A. Seifried, and M. Dellnitz. Identification of halo
orbits for energy efficient formation flying. In International Symposium Formation
Flying, Toulouse, France, 2002.
[67] O. Junge, J. E. Marsden, and S. Ober-Bl¨obaum. Discrete mechanics and optimal
control. In 16th IFAC World Congress, Prague, Czech Republic, 2005.
[68] O. Junge, J. E. Marsden, and S. Ober-Bl¨obaum. Optimal reconfiguration of
formation flying spacecraft - a decentralized approach. In IEEE Conference on
Decision and Control and European Control Conference ECC, pages 5210–5215,
San Diego, CA, USA, 2006.
[69] O. Junge and S. Ober-Bl¨obaum. Optimal reconfiguration of formation flying
satellites. In IEEE Conference on Decision and Control and European Control
Conference ECC, Seville, Spain, 2005.
[70] S. Kameswaran and L. T. Biegler. Convergence rates for direct transcription of
optimal control problems using collocation at Radau points. Preprint.
[71] S. Kameswaran and L. T. Biegler. Convergence rates for direct transcription of
optimal control problems with final-time equality constraints using collocation
at Radau points. In American Control Conference, pages 165–171, Minneapolis,
Minnesota, USA, 2006.
[72] C. Kane, J. E. Marsden, and M. Ortiz. Symplectic energy-momentum integrators.
Mathematical Physics, 40:3353–3371, 1999.
[73] C. Kane, J. E. Marsden, M. Ortiz, and M. West. Variational integrators and the
Newmark algorithm for conservative and dissipative mechanical systems. Inter-
national Journal for Numerical Methods in Engineering, 49(10):1295–1325, 2000.
[74] E. Kanso and J. E. Marsden. Optimal motion of an articulated body in a per-
fect fluid. In IEEE Conference on Decision and Control and European Control
Conference ECC, Seville, Spain, 2005.
[75] W. Karush. Minima of functions of several variables with inequalities as side
constraints. Master’s thesis, Department of Mathematics, University of Chicago,
1939.
178
[76] H. J. B. Keller. Numerical Methods for Two-Point Boundary Value problems.
Waltham: Blaisdell, 1968.
[77] H. J. Kelley. Gradient theory of optimal flight paths. Journal of the American
Rocket Society, 30:947–953, 1960.
[78] M. Kobilarov, M. Desbrun, J. E. Marsden, and G. S. Sukhatme. A discrete
geometric optimal control framework for systems with symmetries. In Robotics:
Science and Systems (R:SS), Zurich, Switzerland, 2007.
[79] M. Kobilarov and G. S. Sukhatme. Optimal control using nonholonomic integra-
tors. In IEEE International Conference on Robotics and Automation (ICRA),
pages 1832–1837, Rome, Italy, 2007.
[80] D. Kraft. On converting optimal control problems into nonlinear programming
problems. In K. Schittkowsky, editor, Compuational Mathematical Programming,
volume F15 of NATO ASI series, pages 261–280. Springer, 1985.
[81] H. W. Kuhn and A. W. Tucker. Nonlinear programming. In J. Neyman, editor,
Proceedings of the Second Berkeley Symposium on Mathematical Statisics and
Probability. University of California Press, Berkeley, 1951.
[82] R. A. LaBudde and D. Greenspan. Energy and momentum conserving methods
of arbitrary order for the numerical integration of equations of motion, I. Motion
of a single particle. Numerische Mathematik, 25:323–346, 1976.
[83] R. A. LaBudde and D. Greenspan. Energy and momentum conserving methods
of arbitrary order for the numerical integration of equations of motion, II. Motion
of a system of particles. Numerische Mathematik, 26:1–16, 1976.
[84] J.-L. Lagrange. Applications de la methode expos´ee dans le m´emoire pr´ec´edent
a la solution de diff´erents probl`emes de dynamique. Oeuvres, 1:365–468, 1760.
[85] J.-L. Lagrange. M´echanique analitique. Paris, 1788.
[86] S. Lall and M. West. Discrete variational Hamiltonian mechanics. Journal of
Physics A: Mathematical and General, 39(19):5509–5519, 2006.
[87] T. Lee, N. H. McClamroch, and M. Leok. Attitude maneuvers of a rigid space-
craft in a circular orbit. In American Control Conference, pages 1742–1747,
Minneapolis, Minnesota, USA, 2006.
[88] T. Lee, N. H. McClamroch, and M. Leok. Optimal control of a rigid body using
geometrically exact computations on SE(3). In IEEE Conference on Decision and
Control and European Control Conference ECC, pages 2710–2715, San Diego, CA,
USA, 2006.
179
[89] T. D. Lee. Can time be a discrete dynamical variable? Physical Letters B,
121:217–220, 1983.
[90] T. D. Lee. Difference equations and conservation laws. Journal of Statistical
Physics, 46:843–860, 1987.
[91] B. Leimkuhler and G. Patrick. A sympletic integrator for Riemannian manifolds.
Nonlinear Science, pages 1–19, 1996.
[92] D. B. Leineweber. Efficient reduced SQP methods for the optimization of chemical
processes described by large sparse DAE models. In Fortschr.-Bericht VDI Reihe
3, Verfahrenstechnik, volume 613. VDI-Verlag, 1999.
[93] A. Lew, J. E. Marsden, M. Ortiz, and M. West. Asynchronous variational inte-
grators. Archive for Rational Mechanics and Analysis, 167:85–146, 2003.
[94] A. Lew, J. E. Marsden, M. Ortiz, and M. West. An overview of variational
integrators. In L. P. Franca, T. E. Tezduyar, and A. Masud, editors, Finite
Element Methods: 1970’s and Beyond, pages 98–115. CIMNE, 2004.
[95] A. Lew, J. E. Marsden, M. Ortiz, and M. West. Variational time integrators. In-
ternational Journal for Numerical Methods in Engineering, 60(1):153–212, 2004.
[96] S. Leyendecker. Mechanical Integrators for constrained dynamical systems in
flexible multibody dynamics. PhD thesis, University of Kaiserslautern, 2006.
[97] S. Leyendecker, J. E. Marsden, and M. Ortiz. Variational integrators for con-
strained dynamical systems. In preparation.
[98] S. Leyendecker, S. Ober-Bl¨obaum, and J. E. Marsden. Discrete mechanics and
optimal control for constrained systems (DMOCC). In preparation.
[99] S. Leyendecker, S. Ober-Bl¨obaum, J. E. Marsden, and M. Ortiz. Discrete mechan-
ics and optimal control for constrained multibody dynamics. In 6th International
Conference on Multibody Systems, Nonlinear Dynamics, and Control, ASME In-
ternational Design Engineering Technical Conferences, Las Vegas, Nevada, USA,
2007.
[100] J. D. Logan. First integrals in the discrete calculus of variation. Aequationes
Mathematicae, 9:210–220, 1973.
[101] Process Systems Enterprise Ltd. gPROMS Advanced User Guide (Release 1.8).
London, 2000.
[102] R. MacKay. Some aspects of the dynamics of Hamiltonian systems. In D. S.
Broomhead and A. Iserles, editors, The dynamics of numerics and the numerics
of dynamics, pages 137–193. Clarendon Press, Oxford, 1992.
180
[103] S. Maeda. Canonical structure and symmetries for discrete systems. Mathematica
Japonica, 25:405–420, 1980.
[104] S. Maeda. Extension of discrete Noether theorem. Mathematica Japonica, 26:85–
90, 1981.
[105] S. Maeda. Lagrangian formulation of discrete systems and concept of difference
space. Mathematica Japonica, 27:345–356, 1981.
[106] K. Malanowski. Finite difference approximations to constrained optimal control
problems. In Optimization and Optimal Control, Lecture Notes in Control and
Information Sciences, pages 243–254. Springer, 1981.
[107] K. Malanowski, C. B¨uskens, and H. Maurer. Convergence of approximations
to nonlinear optimal control problems. In A. V. Fiacco, editor, Mathematical
Programming with Data Perturbations, volume 195 of Lecture Notes in Pure and
Applied Mathematics, pages 253–284. Marcel Dekker, New York, 1979.
[108] J. E. Marsden, G. W. Patrick, and S. Shkoller. Multisymplectic geometry, varia-
tional integrators, and nonlinear PDEs. Communication in Mathematical Physics,
199:351–395, 1998.
[109] J. E. Marsden, S. Pekarsky, and S. Shkoller. Discrete Euler-Poincar´e and Lie
Poisson equations. Nonlinearity, 12:1647–1662, 1999.
[110] J. E. Marsden, S. Pekarsky, and S. Shkoller. Symmetry reduction of discrete
Lagrangian mechanics on Lie groups. Geometry and Physics, 36:140–151, 1999.
[111] J. E. Marsden and T. Ratiu. Introduction to Mechanics and Symmetry, volume 17
of Texts in Applied Mathematics. Springer, 1994.
[112] J. E. Marsden and S. Shkoller. Multisymplectic geometry, covariant Hamiltoni-
ans, and water waves. Mathematical Proceedings of the Cambridge Philosophical
Society, 125:553–575, 1999.
[113] J. E. Marsden and M. West. Discrete mechanics and variational integrators. Acta
Numerica, 10:357–514, 2001.
[114] J. Martin. Discrete mechanics and optimal control. Master’s thesis, Department
of Control and Dynamical Systems, California Institute of Technology, 2006.
[115] J. Martin, S. Ober-Bl¨obaum, S. Leyendecker, and J. E. Marsden. Applications
of DMOC: The falling cat. In preparation.
[116] R. I. McLachlan and S. Marsland. Discrete mechanics and optimal control for
image registration. In Computational Techniques and Applications Conference
(CTAC), 2006.
181
[117] A. Miele. Gradient algorithms for the optimization of dynamic systems. In C. T.
Leondes, editor, Control and Dynamic Systems, volume 60, pages 1–52, 1980.
[118] R. Montgomery. Isoholonomic problems and some applications. Communication
in Mathematical Physics, 128:565–592, 1990.
[119] R. Montgomery. Optimal control of deformable bodies and its relation to gauge
theory. In T. Ratiu, editor, The Geometry of Hamiltonian Systems, pages 403–
438. Springer-Verlag, 1991.
[120] R. Montgomery. Gauge theory of the falling cat. Fields Institute Communications,
1:193–218, 1993.
[121] R. Montgomery. A tour of subriemannian geometries, their geodesics and appli-
cations. American Mathematical Society, Mathematical Surveys and Monographs,
91, 2002.
[122] I. Newton. Philosophiae naturalis principia mathematica. Londini anno MD-
CLXXXVII, 1687.
[123] S. Ober-Bl¨obaum. Zur optimalen Kontrolle von Verbunden starrer K¨orper.
Diploma thesis, University of Paderborn, 2004.
[124] D. Pekarek, A. D. Ames, and J. E. Marsden. Discrete mechanics and optimal
control applied to the compass gait biped. In IEEE Conference on Decision and
Control and European Control Conference ECC, New Orleans, USA, 2007.
[125] L. Petzold, J. B. Rosen, P. E. Gill, L. O. Jay, and K. Park. Numerical optimal
control of parabolic PDEs using DASOPT. In Biegler, Coleman, Conn, and
Santosa, editors, Large Scale Optimization with Applications, Part II, Lecture
Notes in Computational Science and Engineering, pages 271–299. Springer, 1997.
[126] E. Polak. Optimization: Algorithms and consistent Approximations. Springer,
New York, 1997.
[127] L. S. Pontryagin, V. G. Boltyanski, R. V. Gamkrelidze, and E. F. Miscenko. The
Mathematical Theory of Optimal Processes. John Wiley & Sons, 1962.
[128] M. J. D. Powell. A fast algorithm for nonlinearly constrained optimization cal-
culations. In G. A. Watson, editor, Numerical Analysis, volume 630 of Lecture
Notes in Mathematics, pages 261–280. Springer, 1978.
[129] R. Pytlak. Numerical Methods for optimal Control Problems with State Con-
straints. Springer, 1999.
[130] L. B. Rall. Automatic Differentiation: Techniques and Applications, volume 120
of Lecture Notes in Computer Science. Springer Verlag, Berlin, 1981.
182
[131] G. W. Reddien. Collocation at Gauss points as a discretization in optimal control.
SIAM J. Control and Optimization, 17(2):298–306, 1979.
[132] S. Reich. Momentum conserving symplectic integrations. Physica D, 76(4):375–
383, 1994.
[133] S. Reich. Enhancing energy conserving methods. BIT, 36:122–134, 1995.
[134] S. D. Ross. Optimal flapping strokes for self-propulsion in a perfect fluid. In
American Control Conference, pages 4118–4122, Minneapolis, Minnesota, USA,
2006.
[135] D. L. Rudnick, R. E. Davis, C. C. Eriksen, D. M. Fratantoni, and M. J. Perry.
Underwater gliders for ocean research. Marine Technology Society Journal, 38(1),
2004.
[136] V. H. Schulz. Solving discretized optimization problems by partially reduced SQP
methods. Computing and Visualization in Science, 1:83–96, 1998.
[137] A. L. Schwartz and E. Polak. Consistent approximation for optimal control
problems based on Runge-Kutta integration. SIAM J. Control and Optimization,
34(4):1235–1269, 1996.
[138] A. Seifried. Formationsflug von Raumfahrzeugen. Diploma thesis, University of
Paderborn, 2003.
[139] B. Sendov and V. A. Popov. The averaged moduli of smoothness. John Wiley,
1988.
[140] J. Simo and O. Gonzalez. Assessment of energy-momentum and symplectic
schemes for stiff dynamical systems. In American Society of Mechanical En-
gineers, ASME Winter Annual Meeting, New Orleans, Louisiana, USA, 1993.
[141] J. Simo, D. Lewis, and J. E. Marsden. Stability of relative equilibria. Part I:
The reduced energy-momentum method. Archive for Rational Mechanics and
Analysis, 115:15–59, 1991.
[142] J. Simo and N. Tarnow. A new energy and momentum conserving algorithm for
the non-linear dynamics of shells. International Journal for Numerical Methods
in Engineering, 37:2527–2549, 1994.
[143] J. Simo, N. Tarnow, and M. Doblare. Non-linear dynamics of three-dimensional
rods: Exact energy and momentum conserving algorithms. International Journal
for Numerical Methods in Engineering, 38:1431–1473, 1995.
[144] J. Simo, N. Tarnow, and K. Wong. Exact energy-momentum conserving algo-
rithms and symplectic schemes for nonlinear dynamics. Computer Methods in
Applied Mechanics and Engineering, 100:63–116, 1992.
183
[145] J. Simo and K. Wong. Unconditionally stable algorithms for rigid body dynamics
that exactly preserve energy and momentum. International Journal for Numerical
Methods in Engineering, 31:19–52,1321–1323, 1991.
[146] J. Stoer and R. Bulirsch. Introduction into numerical analysis. 2nd ed., Springer,
1993.
[147] Y. B. Suris. Hamiltonian methods of Runge-Kutta type and their variational
interpretation. Mathematical Modelling, 2:78–87, 1990.
[148] M. Sust. Modular aufgebaute deterministische Modelle zur Beschreibung mensch-
licher Bewegungen. In R. Ballreich and W. Baumann, editors, Grundlagen der
Biomechanik des Sports. Ferdinand-Enke-Verlag, Stuttgart, 1996.
[149] V. Szebehely. Theory of orbits – The Restricted Problem of Three Bodies. Aca-
demic Press, 1967.
[150] R. Thurman and A. Worfolk. The geometry of halo orbits in the circular restricted
three-body problem. Technical Report GCG95, University of Minnesota, 1996.
[151] J. Timmermann. Die Nullraum-Methode in Kombination mit DMOC zur op-
timalen Steuerung mechanischer Systeme mit holonomen Zwangsbedingungen.
Diploma thesis, University of Paderborn, 2008.
[152] H. Tolle. Optimization Methods. Springer, 1975.
[153] O. von Stryk. Numerical solution of optimal control problems by direct colloca-
tion. In R. Bulirsch, A. Miele, J. Stoer, and K. H. Well, editors, Optimal Con-
trol - Calculus of Variation, Optimal Control Theory and Numerical Methods,
volume 111 of International Series of Numerical Mathematics, pages 129–143.
Birkh¨auser, 1993.
[154] O. von Stryk. Optimal control of multibody systems in minimal coordinates.
ZAMM, 78(3):1117–1120, 1998.
[155] O. von Stryk. Numerical hybrid optimal control and related topics. Habilitation,
Department of Mathematics, Technische Universit¨at M¨unchen, 2000.
[156] O. von Stryk and R. Bulirsch. Direct and indirect methods for trajectory opti-
mization. Annals of Operations Research, 37:357–373, 1992.
[157] A. Walther, A. Kowarz, and A. Griewank. ADOL-C: A package for the automatic
differentiation of algorithms written in C/C++. ACM TOMS, 22(2):131–167,
1996.
[158] J. M. Wendlandt and J. E. Marsden. Mechanical integrators derived from a
discrete variational principle. Physica D, 106:223–246, 1997.
184
[159] J. M. Wendlandt and J. E. Marsden. Mechanical systems with symmetry, varia-
tional principles and integration algorithms. In M. Alber, B. Hu, and J. Rosen-
thal, editors, Current and Future Directions in Applied Mathematics, pages 219–
261. Birkh¨auser, 1997.
[160] R. E. Wengert. A simple automatic derivative evaluation program. Communica-
tions of the ACM, 7(8):463–464, 1964.
[161] W. Zhang, T. Inanc, S. Ober-Bl¨obaum, and J. E. Marsden. Optimal trajectory
generation for a dynamic glider in ocean flows modeled by 3D B-Spline func-
tions. In IEEE International Conference on Robotics and Automation (ICRA),
Pasadena, CA, USA, 2008.
185
