Professional Documents
Culture Documents
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
O UTLINE
I NTRODUCTION
Preliminaries
Bibliography
O PTIMIZATION BASICS
Local Extrema
Constraints
VARIATIONAL C ALCULUS
Classical Variational Calculus
Free Terminal Time
Systems with Constraints
O PTIMAL C ONTROL
VARIATIONAL C ALCULUS
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
P RELIMINARIES
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
P RELIMINARIES
C OURSE C ONTENT
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
B IBLIOGRAPHY
B IBLIOGRAPHY
E. B. Lee and L. Markus, Foundations of Optimal Control
Theory. New York: John Wiley and Sons, Inc., 1967.
A. E. Bryson and Y.-C. Ho, Applied Optimal Control.
Waltham: Blaisdell, 1969.
S. J. Citron, Elements of Optimal Control. New York: Holt,
Rinehart and Winston, Inc., 1969.
D. E. Kirk, Optimal Control Theory: An Introduction.
Englewood Cliffs, NJ: Prentice-Hall, 1970. (now available
from Dover)
O PTIMAL C ONTROL
O PTIMIZATION BASICS
I NTRODUCTION
VARIATIONAL C ALCULUS
D EFINITION
P ROBLEM D EFINITION
We will dene the basic optimal control problem:
Given system dynamics
x = f (x, u) ,
x X Rn , u U Rm
J (u ()) = gT (x (T)) +
g (x (t) , u (t)) dt
0
R EMARK
gT is called the terminal cost and g is the running cost. The terminal
time T can be xed or free. The target set can be xed or moving.
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
D EFINITION
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
E XAMPLES
x=v
v = u,
The cost function is
|u| 1
T
dt T
J=
0
R EMARK
This is an example of a problem with a control constraint.
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
E XAMPLES
h=v
u
v = g + m
m = ku
The thrust u is used to steer the system to h = 0, v = 0. In
addition we wish to minimize the fuel used during landing, i.e.
t
J=
k udt
0
R EMARK
This problem has both control and state constraints.
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
E XAMPLES
x = Ax + Bu
We seek a feedback control that steers the system from an
arbitrary initial state x0 towards the origin in such a way as to
minimize the cost
J = xT (T) QT x (T) +
1
2T
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
E XAMPLES
x = Ax + B1 w + B2 u
z = C1 x + D11 w + D12 u
y = C2 x + D21 w + D22 u
where w is an external disturbance. The goal is to nd an output (y)
feedback synthesis such that the performance variables (process
errors) z remain close to zero. Note that w (t) can be characterized in
several ways, stochastic (the H2 problem)
zT (t) z (t) dt
J=E
zT (t)z(t)dt
J = max
w
O PTIMAL C ONTROL
2 =1
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
L OCAL E XTREMA
D EFINITIONS
Consider the scalar function
f (x) ,
x Rn
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
L OCAL E XTREMA
D EFINITIONS
D EFINITION (L OCAL M INIMA & M AXIMA )
An interior point x intD is a local minimum if there exists a
neighborhood U of x such that
f (x) f (x )
x U
It is a local maximum if
f (x) f (x )
x U
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
L OCAL E XTREMA
O PTIMAL C ONTROL
2f
(x ) < 0
x2
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
C ONSTRAINTS
H (x , )
(x) = 0
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
C ONSTRAINTS
(x1 , x2 )
(x1 , x2 )
dx1 +
dx2 = 0
x1
x2
From (1) and (2) it must be that
d (x1 , x2 ) =
f (x1 , x2 )/x1
f (x1 , x2 )/x2
=
=
(x1 , x2 )/x1
(x1 , x2 )/x2
f (x1 , x2 )
(x1 , x2 )
+
=0
x2
x2
R EMARK
Note that these are the form of the previous slide.
O PTIMAL C ONTROL
(1)
(2)
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
C ONSTRAINTS
H
1
2H
dx + dxT 2 dx T d + h.o.t.
x
2
x
but
dx = 0 dx ker
dx = d
x
x
(x)
= span ker
x
Thus, for x extremal
d =
1 T T 2 H (x )
d
d + h.o.t.
2
x2
2 H (x )
T
> 0 min
x2
2
H (x )
T
< 0 max
x2
df (x ) =
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
C ONSTRAINTS
E XAMPLE
2
2
f (x1 , x2 ) = x1 x1 + 2x2 1 ,
2
2
(x1 , x2 ) = x1 + x2 1
Interior:
(x1 , x2 , f ) = (0, 0.707, 0) (0, 0.707, 0) (0.577, 0, 0.385) (0.577, 0. 0.385)
Boundary:
(x1 , x2 , , f ) = (0.577, 0.8165, 1.155, 0.385) (0.577, 0.8165, 1.155, 0.385)
(0.577, 0.8165, 1.155, 0.385) (0.577, 0.8165, 1.155, 0.385)
(1, 0, 1, 0) (1, 0, 1, 0)
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
O PTIMAL C ONTROL
O PTIMIZATION BASICS
I NTRODUCTION
VARIATIONAL C ALCULUS
L (q (t) , q (t)) dt
J (q (t)) =
t1
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
T = 1 qT M (q) q
2
M (q) =
2
2
2 m2 ( 2 + 1 cos 2 )
1 (m1 + m2 ) + 2 m2 + 2 1 2 m2 cos 2
2
2 m2 ( 2 + 1 cos 2 )
2 m2
V (q) = m1 g (g 1 (m1 + m2 ) sin 1 + g 2 m2 sin (1 + 2 ))
m1
2
2
m2
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
J (q (t, )) =
t1
O PTIMAL C ONTROL
=0
=0
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
C ALCULUS OF VARIATIONS 2
t2
J (q (t)) =
t1
L (q (t) , q (t))
L (q (t) , q (t))
(t) +
(t) dt
q
q
L
L
d +
dt
q
q
We can apply the integration by parts formula
J
udv = uv
vdu
t2
t1
q
d
dt L(
O PTIMAL C ONTROL
(t),q (t))
L( (t),q (t))
q
q
t2
L( (t),q (t))
q
(t)
t1
(t) dt
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
C ALCULUS OF VARIATIONS 3
Now, set J (q (t)) = 0 to obtain:
the Euler-Lagrange equations
=0
dt
q
q
the transversality (boundary) conditions
L (q (t1 ) , q (t1 ))
q (t1 ) = 0,
L (q (t2 ) , q (t2 ))
q (t2 ) = 0
R EMARK
These results allow us to treat problems in which the initial and terminal times are xed
and individual components of q(t1 ) and q(t2 ) are xed or free. Other cases of interest
include: 1) the terminal time is free, and 2) the terminal time is related to the terminal
conguration, e.g., by a relation (q (t2 ) , t2 ) = 0.
O PTIMAL C ONTROL
O PTIMIZATION BASICS
I NTRODUCTION
VARIATIONAL C ALCULUS
t2 +t
J (q , q, t) =
t1
t2
t1
d
dt
L(q
t2
t2
( ),q ( ))
q (t) dt
q (t2 ) + L (q (t2 ) , q (t2 )) t
Now, we want to allow both the nal time and the end point to vary. The
actual end state is:
q2 = q (t2 + t) = q (t2 ) + q (t2 ) t
O PTIMAL C ONTROL
O PTIMIZATION BASICS
I NTRODUCTION
VARIATIONAL C ALCULUS
L(q
t2
t1
t2
d
dt
t2
( ),q ( ))
q2 + L (q (t2 ) , q (t2 ))
q (t) dt
(t2 ) t
L (q (t1 ) , q (t1 ))
q (t1 ) = 0,
L (q (t2 ) , q (t2 ))
q (t2 ) = 0
L (q (t2 ) , q (t2 ))
q (t2 ) = 0
L (q (t2 ) , q (t2 ))
O PTIMAL C ONTROL
L (q (t2 ) , q (t2 ))
q (t2 ) t = 0
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
L (q (t2 ) , q (t2 ))
L (q (t2 ) , q (t2 ))
q (t2 ) = 0
L (q (t2 ) , q (t2 ))
= 0, L (q (t2 ) , q (t2 )) = 0
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
L (u (t) , x (t)) dt
J (x (t)) =
t1
1 2
x + u2 dt
0 2
The Euler-Lagrange equations become simply
J (x (t)) =
d
dt
O PTIMAL C ONTROL
L
u
=0ux=0
x
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
E XAMPLE 1, C ONT D
Thus, we have the equations
0 1
1 0
x
u
L (x (1) , u (1))
= 0 u (1) = 0
u
x (0) = x0 ,
Consequently,
x (t)
u (t)
=e
0 1
1 0
a
b
a = x0 , b =
O PTIMAL C ONTROL
x (t)
u (t)
x0 e2 x0
1 + e2
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
O PTIMAL C ONTROL
x0 + e2 x0
1 + e2
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
O PTIMAL C ONTROL
VARIATIONAL C ALCULUS
O PTIMIZATION BASICS
I NTRODUCTION
E XAMPLE 1, C ONT D
The gures below show the optimal control and state
trajectories from the initial state x0 = 1,xf = 0, with t2 = 1 for the
xed time case. For the free time case xf = 0.01.
Free terminal state, Fixed
time:
x,u
x,u
1.0
1.0
1.0
0.5
0.5
0.5
t
0.2
0.4
0.6
0.8
1.0
1
t
0.2
0.4
0.6
0.8
1.0
0.5
0.5
0.5
1.0
1.0
R EMARK
In the free time case, with xf = 0.01, the nal time is t2 = 4.60517. With xf = 0 the nal
time is t2 = . Note that the case of free terminal state and free terminal time (not
shown) is trivial with t2 = 0.
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
L (q (t) , q (t) , t) dt
J (q (t)) =
t1
(q (t) , q (t) , t) = 0
where : Rn Rn R Rm .
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
J (q (t)) =
t1
t2
t1
d
dt Lq + T q + Lq + T q q (t) + T dt
+ Lq + T q t q2 + L + T q Lq + T q q t t2
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
dt
The differential constraints:
(q, q, t) = 0
and the boundary conditions
Lq + T q
t2
q2 = 0
O PTIMAL C ONTROL
t2
=0
O PTIMIZATION BASICS
I NTRODUCTION
E XAMPLE
Once again consider the problem
t2
x = u,
J (x (t)) =
0
1 2
x + u2 dt
2
( , x) = x2 1 u2 1 = 0 u = 1
x
The Euler-Lagrange equations now become
d
x
Lu + T u Lx + T x = 0 =
dt
2u
R EMARK
O PTIMAL C ONTROL
t
0
u2 dt = t
VARIATIONAL C ALCULUS
I NTRODUCTION
O PTIMIZATION BASICS
E XAMPLE C ONTINUED
O PTIMAL C ONTROL
VARIATIONAL C ALCULUS
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
system x = u
Euler equation =
x
2u
constraint u = 1
boundary conditions x (0) = x0 , x (1) = x1
O PTIMAL C ONTROL
I NTRODUCTION
O PTIMIZATION BASICS
VARIATIONAL C ALCULUS
system x = u
Euler equation =
x
2u
constraint u = 1
initial condition x (0) = x0
terminal condition x (1) = x1
terminal condition Lu + T u
t2
= 0 (t2 ) = 1
2
O PTIMAL C ONTROL
1
T=
2
1 20
I NTRODUCTION
VARIATIONAL C ALCULUS
O PTIMIZATION BASICS
0 =
1
2
is optimal.
1
2
0 =
3
4
0.5
0.5
0.0
x,,J
x,,J
0.0
0.5
0.5
1.0
0.0
0.2
0.4
0.6
t
O PTIMAL C ONTROL
0.8
1.0
1.0
0.0
0.2
0.4
0.6
t
0.8
1.0
O PTIMIZATION BASICS
I NTRODUCTION
VARIATIONAL C ALCULUS
system x = u
Euler equation =
x
2u
constraint u = 1
initial condition x (0) = x0
terminal condition x (t2 ) = 0
terminal condition
L + T q Lu + T u u
t2
1
2
1 2 (t2 ) = 0 (t2 ) = 1
4
(x0 0) (k = 1) (t2 = x0 ) 0 =
1
2
1 + x0
4
O PTIMAL C ONTROL
1
2
1 + x0
4