You are on page 1of 41

I NTRODUCTION

T HE O PTIMAL C ONTROL P ROBLEM

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

I NTRODUCTION TO O PTIMAL C ONTROL


S YSTEMS
Harry G. Kwatny
Department of Mechanical Engineering & Mechanics
Drexel University

O PTIMAL C ONTROL

I NTRODUCTION

T HE O PTIMAL C ONTROL P ROBLEM

O PTIMIZATION BASICS

O UTLINE
I NTRODUCTION
Preliminaries
Bibliography

T HE O PTIMAL C ONTROL P ROBLEM


Denition
Examples

O PTIMIZATION BASICS
Local Extrema
Constraints

VARIATIONAL C ALCULUS
Classical Variational Calculus
Free Terminal Time
Systems with Constraints
O PTIMAL C ONTROL

VARIATIONAL C ALCULUS

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

P RELIMINARIES

W HAT IS O PTIMAL C ONTROL ?


Optimal control is an approach to control systems design that
seeks the best possible control with respect to a performance
metric.
The theory of optimal control began to develop in the WW II
years. The main result of this period was the Wiener-Kolmogorov
theory that addresses linear SISO systems with Gaussian noise.
A more general theory began to emerge in the 1950s and 60s
In 1957 Bellman published his book on Dynamic Programming
In 1960 Kalman published his multivariable generalization of
Wiener-Kolmogorov
In 1962 Pontryagin et al published the maximal principle
In 1965 Isaacs published his book on differential games

O PTIMAL C ONTROL

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

P RELIMINARIES

C OURSE C ONTENT

Introduction & Preliminary Examples


Optimization Concepts
Reachability, Controllability & Time Optimal Control
The Maximum Principle
Principle of Optimality, Dynamic programming, & the HJB
Equation
Game Theory

O PTIMAL C ONTROL

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

B IBLIOGRAPHY

B IBLIOGRAPHY
E. B. Lee and L. Markus, Foundations of Optimal Control
Theory. New York: John Wiley and Sons, Inc., 1967.
A. E. Bryson and Y.-C. Ho, Applied Optimal Control.
Waltham: Blaisdell, 1969.
S. J. Citron, Elements of Optimal Control. New York: Holt,
Rinehart and Winston, Inc., 1969.
D. E. Kirk, Optimal Control Theory: An Introduction.
Englewood Cliffs, NJ: Prentice-Hall, 1970. (now available
from Dover)

O PTIMAL C ONTROL

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

VARIATIONAL C ALCULUS

D EFINITION

P ROBLEM D EFINITION
We will dene the basic optimal control problem:
Given system dynamics

x = f (x, u) ,

x X Rn , u U Rm

Find a control u (t) , t [0, T] that steers the system from


an initial state x (0) = x0 to a target set G and minimizes the
cost
T

J (u ()) = gT (x (T)) +

g (x (t) , u (t)) dt
0

R EMARK
gT is called the terminal cost and g is the running cost. The terminal
time T can be xed or free. The target set can be xed or moving.
O PTIMAL C ONTROL

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

D EFINITION

O PEN L OOP VS . C LOSED L OOP


If we are concerned with a single specied initial state x0 , then
we might seek the optimal control control u (t), u : R Rm that
steers the system from the initial state to the target. This is an
open loop control.
On the other hand, we might seek the optimal control as a
function of the state u (x), u : Rn Rm . This is a closed loop
control; sometimes called a synthesis.
The open loop control is sometimes easier to compute, and the
computations are sometimes performed online a method
known as model predictive control.
The closed loop control has the important advantage that it is
robust with respect to model uncertainty, and that once the
(sometimes difcult) computations are performed off-line, the
control is easily implemented online.
O PTIMAL C ONTROL

I NTRODUCTION

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

VARIATIONAL C ALCULUS

E XAMPLES

E XAMPLE A M INIMUM T IME P ROBLEM


Consider steering a unit mass, with bounded applied control
force, from an arbitrary initial position and velocity to rest at the
origin in minimum time. Specically,

x=v

v = u,
The cost function is

|u| 1
T

dt T

J=
0

R EMARK
This is an example of a problem with a control constraint.

O PTIMAL C ONTROL

I NTRODUCTION

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

VARIATIONAL C ALCULUS

E XAMPLES

E XAMPLE A M INIMUM F UEL P ROBLEM


Consider the decent of a moon lander.

h=v
u

v = g + m

m = ku
The thrust u is used to steer the system to h = 0, v = 0. In
addition we wish to minimize the fuel used during landing, i.e.
t

J=

k udt
0

Furthermore, u is constrained, 0 u c, and the state


constraint h 0 must be respected.

R EMARK
This problem has both control and state constraints.
O PTIMAL C ONTROL

I NTRODUCTION

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

VARIATIONAL C ALCULUS

E XAMPLES

E XAMPLE A L INEAR R EGULATOR P ROBLEM


Consider a system with linear dynamics

x = Ax + Bu
We seek a feedback control that steers the system from an
arbitrary initial state x0 towards the origin in such a way as to
minimize the cost
J = xT (T) QT x (T) +

1
2T

xT (t) Qx (t) + uT (t) Ru (t) dt


0

The nal time T is considered xed.

O PTIMAL C ONTROL

I NTRODUCTION

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

VARIATIONAL C ALCULUS

E XAMPLES

E XAMPLE A ROBUST S ERVO P ROBLEM


Consider a system with dynamics

x = Ax + B1 w + B2 u
z = C1 x + D11 w + D12 u
y = C2 x + D21 w + D22 u
where w is an external disturbance. The goal is to nd an output (y)
feedback synthesis such that the performance variables (process
errors) z remain close to zero. Note that w (t) can be characterized in
several ways, stochastic (the H2 problem)

zT (t) z (t) dt

J=E

or deterministic (the H problem)

zT (t)z(t)dt

J = max
w

O PTIMAL C ONTROL

2 =1

I NTRODUCTION

T HE O PTIMAL C ONTROL P ROBLEM

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

L OCAL E XTREMA

D EFINITIONS
Consider the scalar function
f (x) ,

x Rn

which is dened and smooth on a domain D Rn . We further


assume that the region D is dened by a scalar inequality
(x) 0, i.e.,
intD = {x Rn | (x) < 0 }
D = {x Rn | (x) = 0 }

O PTIMAL C ONTROL

I NTRODUCTION

T HE O PTIMAL C ONTROL P ROBLEM

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

L OCAL E XTREMA

D EFINITIONS
D EFINITION (L OCAL M INIMA & M AXIMA )
An interior point x intD is a local minimum if there exists a
neighborhood U of x such that
f (x) f (x )

x U

It is a local maximum if
f (x) f (x )

x U

Similarly, for a point x D, we use a neighborhood U of x


within D. With this modication boundary local minima and
maxima are dened as above.
O PTIMAL C ONTROL

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

L OCAL E XTREMA

O PTIMAL I NTERIOR P OINTS


Necessary conditions. A point x intD is an extremal
point (minimum or maximum) only if
f
(x ) = 0
x
Sufcient conditions. x is a minimum if
2f
(x ) > 0
x2
a maximum if

O PTIMAL C ONTROL

2f
(x ) < 0
x2

I NTRODUCTION

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

VARIATIONAL C ALCULUS

C ONSTRAINTS

O PTIMIZATION WITH CONSTRAINTS N ECESSARY


C ONDITIONS
We need to nd extremal points of f (x), with x D. i.e., nd
extremal points of f (x) subject to the constraint (x) = 0.
Consider a more general problem where there are m constraints, i.e.,
: Rn Rm .
Let be an m-dimensional constant vector (called Lagrange
multipliers) and dene the function

H (x, ) = f (x) + T (x)


Then x is an extremal point only if
H (x , )
= 0,
x

H (x , )
(x) = 0

Note there are n + m equations in n + m unknowns x,


O PTIMAL C ONTROL

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

C ONSTRAINTS

S IGNIFICANCE OF THE L AGRANGE M ULTIPLIER


Consider extremal points of f (x1 , x2 ) subject to the single constraint

(x1 , x2 ) = 0. At an extremal point (x1 , x2 ) we must have


f (x1 , x2 )
f (x1 , x2 )
dx1 +
dx2 = 0
x1
x2
but dx1 and dx2 are not independent. They satisfy

df (x1 , x2 ) =



(x1 , x2 )
(x1 , x2 )
dx1 +
dx2 = 0
x1
x2
From (1) and (2) it must be that

d (x1 , x2 ) =



f (x1 , x2 )/x1
f (x1 , x2 )/x2
=
=


(x1 , x2 )/x1
(x1 , x2 )/x2

Accordingly, (1) and (2) yield




f (x1 , x2 )
(x1 , x2 )
+
= 0,
x1
x1



f (x1 , x2 )
(x1 , x2 )
+
=0
x2
x2

R EMARK
Note that these are the form of the previous slide.
O PTIMAL C ONTROL

(1)

(2)

I NTRODUCTION

T HE O PTIMAL C ONTROL P ROBLEM

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

C ONSTRAINTS

O PTIMIZATION WITH C ONSTRAINTS S UFFICIENT


C ONDITIONS
df =

H
1
2H
dx + dxT 2 dx T d + h.o.t.
x
2
x

but

dx = 0 dx ker
dx = d
x
x
(x)
= span ker
x
Thus, for x extremal
d =

1 T T 2 H (x )
d
d + h.o.t.
2
x2
2 H (x )
T
> 0 min
x2
2

H (x )
T
< 0 max
x2

df (x ) =

O PTIMAL C ONTROL

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

C ONSTRAINTS

E XAMPLE
2
2
f (x1 , x2 ) = x1 x1 + 2x2 1 ,

2
2
(x1 , x2 ) = x1 + x2 1

Interior:
(x1 , x2 , f ) = (0, 0.707, 0) (0, 0.707, 0) (0.577, 0, 0.385) (0.577, 0. 0.385)
Boundary:
(x1 , x2 , , f ) = (0.577, 0.8165, 1.155, 0.385) (0.577, 0.8165, 1.155, 0.385)
(0.577, 0.8165, 1.155, 0.385) (0.577, 0.8165, 1.155, 0.385)
(1, 0, 1, 0) (1, 0, 1, 0)

O PTIMAL C ONTROL

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

O PTIMIZING A T IME T RAJECTORY


We are interested in steering a controllable system along a
trajectory that is optimal in some sense.
Three methods are commonly used to address such
problems:
The calculus of variations
The Pontryagin maximal Principle
The principle of optimality and dynamic programming

The calculus of variations was rst invented to characterize


the dynamical behavior of physical systems governed by a
conservation law.

O PTIMAL C ONTROL

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

C ALCULUS OF VARIATIONS : L AGRANGIAN S YSTEMS


A Lagrangian System is characterized as follows:
The system is dene in terms of a vector of conguration

coordinates, q, associated with velocities q.

The system has kinetic energy T (q, q) = qT M (q) q/2, and


potential energy V (q) from which we dene the Lagrangian

L (q, q) = T (q, q) V (q)


The system moves along a trajectory q (t), between initial
and nal times t1 , t2 in such a way as to minimize the
integral
t2

L (q (t) , q (t)) dt

J (q (t)) =
t1

O PTIMAL C ONTROL

I NTRODUCTION

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

E XAMPLES OF L AGRANGIAN S YSTEMS

T = 1 qT M (q) q
2
M (q) =
2
2
2 m2 ( 2 + 1 cos 2 )
1 (m1 + m2 ) + 2 m2 + 2 1 2 m2 cos 2
2
2 m2 ( 2 + 1 cos 2 )
2 m2
V (q) = m1 g (g 1 (m1 + m2 ) sin 1 + g 2 m2 sin (1 + 2 ))

m1
2

2
m2
O PTIMAL C ONTROL

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

C ALCULUS OF VARIATIONS 1: N ECESSARY


C ONDITIONS
A real-valued, continuously differentiable function q (t) on the
interval [t1 , t2 ] will be called admissible.
Let q (t) be an optimal admissible trajectory and q (t, ) a not
necessarily optimal trajectory, with
q (t, ) = q (t) + (t)
where > 0 is a small parameter and (t) is arbitrary.
Then
t2

J (q (t, )) =

L (q (t) + (t) , q (t) + (t)) dt

t1

An extremal of J is obtained from


J (q (t, ))
J (q (t)) =

O PTIMAL C ONTROL

=0
=0

I NTRODUCTION

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

C ALCULUS OF VARIATIONS 2
t2

J (q (t)) =
t1

L (q (t) , q (t))
L (q (t) , q (t))
(t) +

(t) dt

q
q

L
L
d +
dt

q
q
We can apply the integration by parts formula
J

udv = uv

vdu

to the rst term to obtain


J (q (t)) =

t2
t1

q
d
dt L(

O PTIMAL C ONTROL

(t),q (t))

L( (t),q (t))
q
q
t2

L( (t),q (t))
q
(t)

t1

(t) dt

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

C LASSICAL VARIATIONAL C ALCULUS

C ALCULUS OF VARIATIONS 3
Now, set J (q (t)) = 0 to obtain:
the Euler-Lagrange equations

d L (q (t) , q (t)) L (q (t) , q (t))

=0

dt
q
q
the transversality (boundary) conditions

L (q (t1 ) , q (t1 ))
q (t1 ) = 0,

L (q (t2 ) , q (t2 ))
q (t2 ) = 0

R EMARK
These results allow us to treat problems in which the initial and terminal times are xed
and individual components of q(t1 ) and q(t2 ) are xed or free. Other cases of interest
include: 1) the terminal time is free, and 2) the terminal time is related to the terminal
conguration, e.g., by a relation (q (t2 ) , t2 ) = 0.
O PTIMAL C ONTROL

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

F REE T ERMINAL T IME , 1


Consider the case of xed initial state and free terminal time. Let q (t) be the

optimal trajectory with optimal terminal time t2 . The perturbed trajectory

terminates at time t2 + t2 . Its end state is q (t2 ) + (t2 + ). The


perturbed cost is

t2 +t

J (q , q, t) =

L (q (t) + q (t) , q (t) + q (t)) dt

t1

From this we obtain:


J (q (t)) =
+

t2
t1

d
dt

L(q

L(q (t),q (t))

t2

t2

( ),q ( ))

L(q (t),q (t))


q

q (t) dt


q (t2 ) + L (q (t2 ) , q (t2 )) t

Now, we want to allow both the nal time and the end point to vary. The
actual end state is:


q2 = q (t2 + t) = q (t2 ) + q (t2 ) t

O PTIMAL C ONTROL

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

F REE T ERMINAL T IME , 2


J (q (t)) =
+

L(q

t2
t1

t2

d
dt

t2

L(q (t),q (t))

( ),q ( ))

L(q (t),q (t))


q


q2 + L (q (t2 ) , q (t2 ))

q (t) dt

L(q (t2 ),q (t2 ))

(t2 ) t

Thus, we have have the Euler-Lagrange Equations, as before, but the


transversality conditions become:
the previous conditions:

L (q (t1 ) , q (t1 ))
q (t1 ) = 0,


L (q (t2 ) , q (t2 ))

q (t2 ) = 0

plus, additional conditions:


L (q (t2 ) , q (t2 ))

q (t2 ) = 0


L (q (t2 ) , q (t2 ))

O PTIMAL C ONTROL


L (q (t2 ) , q (t2 ))

q (t2 ) t = 0

I NTRODUCTION

T HE O PTIMAL C ONTROL P ROBLEM

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

F REE T ERMINAL T IME , 3


Ordinarily, the initial state and time are xed so that the transversality
conditions require
q (t1 ) = 0
For free terminal time t is arbitrary. Thus from the transversality conditions:
1. If the terminal state is xed, q (t2 ) = 0 the terminal condition is:


L (q (t2 ) , q (t2 ))


L (q (t2 ) , q (t2 ))

q (t2 ) = 0

2. If the terminal state is free, q2 arbitrary the terminal condition is:


L (q (t2 ) , q (t2 ))


= 0, L (q (t2 ) , q (t2 )) = 0

O PTIMAL C ONTROL

I NTRODUCTION

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

E XAMPLE 1: F REE ENDPOINT, F IXED TERMINAL TIME


First note that in the elementary variational calculus, we could

simply replace q by x, and add the denition x = u, so that the


cost function becomes
t2

L (u (t) , x (t)) dt

J (x (t)) =
t1

Hence, we have a simple control problem. Now suppose x R,


t1 = 0, t2 = 1, and L = x2 + u2 /2:
1

1 2
x + u2 dt
0 2
The Euler-Lagrange equations become simply
J (x (t)) =

d
dt
O PTIMAL C ONTROL

L
u

=0ux=0
x

I NTRODUCTION

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

E XAMPLE 1, C ONT D
Thus, we have the equations

0 1
1 0

x
u

L (x (1) , u (1))
= 0 u (1) = 0
u

x (0) = x0 ,
Consequently,

x (t)
u (t)

=e

0 1
1 0

a
b

a = x0 , b =
O PTIMAL C ONTROL

x (t)
u (t)

x0 e2 x0
1 + e2

a cosh (t) + b sinh (t)


b cosh (t) + a sinh (t)

I NTRODUCTION

T HE O PTIMAL C ONTROL P ROBLEM

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

E XAMPLE 1: F IXED ENDPOINT, F IXED TERMINAL TIME


Suppose we consider a xed end point, say x (t2 ) = 0 with the
terminal time still xed at t2 = 1. Then the Euler-Lagrange
equations remain the same, but the boundary conditions
change to:
x (0) = x0 , x (1) = 0
Thus, we compute
a = x0 , b =

O PTIMAL C ONTROL

x0 + e2 x0
1 + e2

I NTRODUCTION

T HE O PTIMAL C ONTROL P ROBLEM

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

F REE T ERMINAL T IME

E XAMPLE 1: F IXED ENDPOINT, F REE TERMINAL TIME


Suppose we consider a xed end point, say x (t2 ) = xf with the
terminal time t2 free. Then the Euler-Lagrange equations
remain the same, but the boundary conditions change to:
x (0) = x0 , x (t2 ) = xf , (L Lu u)|t2 = 0
From this we compute
a = x0 , a2 = b2 , a cosh (t2 ) + b sinh (t2 ) = xf

O PTIMAL C ONTROL

VARIATIONAL C ALCULUS

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

F REE T ERMINAL T IME

E XAMPLE 1, C ONT D
The gures below show the optimal control and state
trajectories from the initial state x0 = 1,xf = 0, with t2 = 1 for the
xed time case. For the free time case xf = 0.01.
Free terminal state, Fixed
time:

Fixed terminal state,


Fixed Time:

x,u

Fixed terminal state,


Free time:
x,u

x,u

1.0

1.0

1.0

0.5

0.5

0.5
t
0.2

0.4

0.6

0.8

1.0
1

t
0.2

0.4

0.6

0.8

1.0

0.5
0.5

0.5

1.0
1.0

R EMARK
In the free time case, with xf = 0.01, the nal time is t2 = 4.60517. With xf = 0 the nal
time is t2 = . Note that the case of free terminal state and free terminal time (not
shown) is trivial with t2 = 0.

O PTIMAL C ONTROL

I NTRODUCTION

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

VARIATIONAL C ALCULUS WITH D IFFERENTIAL


C ONSTRAINTS
Systems with nonintegrable differential , i.e., nonholonomic,
constraints have been treated by variational methods. As
before, we seek extremals of the functional:
t2

L (q (t) , q (t) , t) dt

J (q (t)) =
t1

But now subject to the constraints:

(q (t) , q (t) , t) = 0
where : Rn Rn R Rm .

O PTIMAL C ONTROL

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

D IFFERENTIAL C ONSTRAINTS , C ONT D


The constraints are enforce for all time, so introduce the m
Lagrange multipliers (t) and consider the modied functional
t2

J (q (t)) =

L (q (t) , q (t) , t) + T (t) (q (t) , q (t) , t) dt

t1

We allow variations in both q and and t2 , Taking the variation


and integrating by parts yields
J =

t2
t1

d
dt Lq + T q + Lq + T q q (t) + T dt
+ Lq + T q t q2 + L + T q Lq + T q q t t2

O PTIMAL C ONTROL

I NTRODUCTION

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

N ECESSARY C ONDITIONS WITH C ONSTRAINTS


Again we have the Euler-Lagrange equations, now in the form:
d
Lq + T q Lq + T q = 0

dt
The differential constraints:

(q, q, t) = 0
and the boundary conditions
Lq + T q

t2

q2 = 0

With free terminal time, we also have


L + T q Lq + T q q

O PTIMAL C ONTROL

t2

=0

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

S YSTEMS WITH C ONSTRAINTS

E XAMPLE
Once again consider the problem
t2

x = u,

J (x (t)) =
0

1 2
x + u2 dt
2

But now, add the constraint

( , x) = x2 1 u2 1 = 0 u = 1
x
The Euler-Lagrange equations now become
d
x

Lu + T u Lx + T x = 0 =
dt
2u

R EMARK

Note that u = 1 u = 0 almost everywhere,


Also,

O PTIMAL C ONTROL

t
0

u2 dt = t

VARIATIONAL C ALCULUS

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE C ONTINUED

We now consider three cases:


xed terminal time, xed end point
xed terminal time, free end point
free terminal time xed end point

O PTIMAL C ONTROL

VARIATIONAL C ALCULUS

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE : F IXED TERMINAL T IME , F IXED E ND P OINT


Steer from x (0) = x0 to x (1) = x1 .

system x = u

Euler equation =

x
2u

constraint u = 1
boundary conditions x (0) = x0 , x (1) = x1

Assume single switch at time T


x0 + kT k(1 T) = x1 (k = 1) 0 T 1
1 + x1 x0 1 + x1 (k = 1) T = kx0 +x1
2k

O PTIMAL C ONTROL

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

O PTIMIZATION BASICS

VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE : F IXED TERMINAL T IME , F REE E ND P OINT


Steer from x (0) = x0 .

system x = u

Euler equation =

x
2u

constraint u = 1
initial condition x (0) = x0
terminal condition x (1) = x1
terminal condition Lu + T u

t2

= 0 (t2 ) = 1
2

The necessary conditions are satised with


3
1
(k = 1) 0
4
2

O PTIMAL C ONTROL

1
T=
2

1 20

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

VARIATIONAL C ALCULUS

O PTIMIZATION BASICS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE : F IXED TERMINAL T IME , F REE E ND P OINT2


1
Only the case 0 = 2 , T =

0 =

1
2

is optimal.

1
2

0 =

3
4

0.5

0.5

0.0

x,,J

x,,J

0.0

0.5

0.5

1.0
0.0

0.2

0.4

0.6
t

O PTIMAL C ONTROL

0.8

1.0

1.0
0.0

0.2

0.4

0.6
t

0.8

1.0

O PTIMIZATION BASICS

T HE O PTIMAL C ONTROL P ROBLEM

I NTRODUCTION

VARIATIONAL C ALCULUS

S YSTEMS WITH C ONSTRAINTS

E XAMPLE : F REE TERMINAL T IME , F IXED E ND P OINT


Steer from x (0) = x0 to x (t2 ) = 0

system x = u

Euler equation =

x
2u

constraint u = 1
initial condition x (0) = x0
terminal condition x (t2 ) = 0
terminal condition
L + T q Lu + T u u

t2

1
2

1 2 (t2 ) = 0 (t2 ) = 1
4

(x0 0) (k = 1) (t2 = x0 ) 0 =

1
2
1 + x0
4

(x0 0) (k = +1) (t2 = x0 ) 0 =

O PTIMAL C ONTROL

1
2
1 + x0
4

You might also like