Chapter 1

Introduction to Optimal Control

 Introduction
• Optimal control – is finding a controller that
drives a system towards a desired operating
condition while achieving a given performance
criteria
• The performance criteria is given as a cost
function J
• Most commonly used J

– (x(to),to), (x(tf),tf), is Condition at the beginning

and end of the control and
– (x(t),t) the cost in the entire interval
 Introduction
• The type of function used for ,  in the
performance function determines the type of
optimal control
• Problem is constrained optimization problem
– Constraints may be given time, state condition,
amount of control effort or energy etc
• Solution techniques depend on the type of
function used for the initial and final
conditions
 Problem formulation
• Given a system

• With initial condition

• Find a control signal u that can

– drive the system to a final state
– Fulfill state constraint
– Minimize the function
 Classification of optimal control
problems
• Optimal control problems can be classified
into various types based on the performance
function used
• These are
– Minimum time control problem
– Terminal control problem
– Minimum control effort problem
– Optimal servo mechanism
– Optimal regulator problem
 Minimum time control problem
• Objective is to minimize the time required to
drive a system from its initial state to final
state
• J or performance function is
tf


J  dt
to

• Example: optimal control of robotic

manipulator to finish a task
 Example for Minimum time control
problem
• A rocket burn trajectory is desired to minimize a
travel time between a starting point and a final
point, 10 units of distance away.
– The thrust can be between an upper limit of 1.1 and a
lower limit of -1.1.
– The initial and final velocity must be zero and the
maximum velocity can never exceed 1.7.
– It is also desirable to minimize the use of fuel to
perform the maneuver. There is a drag resistance that
is proportional to the square of the velocity and mass
is lost as the fuel is burned during thrust operations.
 Example- minimum time problem
• Since the problem is minimum time problem, the
objective functions is minimize J given by
tf


J  dt
• This is similar to to

݉݅݊݅݉݅‫ݐ ݁ݖ‬௙

• Once we determined the objective function, next

we have to determine the constraints. The first
constraint usually is the system dynamic equation
 Example-minimum time problem
• The dynamic equation of the rocket is then
ௗ௦
ௗ௧
=v

݀‫ݒ‬
= (‫ ݑ‬− 0.2‫ ݒ‬ଶ )/݉
݀‫ݐ‬
݀݉
= −0.01‫ݑ‬ଶ
݀‫ݐ‬

• When the system is implemented, it will have

additional constraints. These are the maximum
and minimum control signal limits, the maximum
and minimum velocity which the system can have
• The last constraints are the patch constraints
 Example- the minimum time control
problem
• The final problem formulation will then be
݉݅݊݅݉݅‫ݐ ݁ݖ‬௙

• Subject to ‫ݏ‬ሶ = ‫ݒ‬

‫ݒ‬ሶ = ‫ ݑ‬− 0.2‫ ݒ‬ଶ /݉
݉ሶ = −0.01‫ݑ‬ଶ
u ≤ 1.1
‫ ≥ ݑ‬−1.1
‫ ≤ ݒ‬1.7
‫ ݒ‬0 =0 ‫ݐ ݒ‬௙ = 0
‫ ݏ‬0 =0 ‫ݐ ݏ‬௙ = 0
 Terminal control problem
• Objective: is to minimize the final error value,
i.e. given a desired value, find a control signal
that can drive the system towards a point
where final value has minimum deviation from
desired value
• J has the form of
J  [ x(t f )  d ]T S [ x(t f )  d ]
– d is the desired value of the system at final time
 Example- Pick and place robotic
manipulator
• Consider a third degree of freedom robotic
manipulator shown below
݈ଵ , ݈ଶ and ݈ଷ are links of the robot
ߠଵ , ߠଶ and ߠଷ are the joints which are
driven by motors.

The objective is to move the robotic

manipulator end effector from its position to
point A with minimum terminal error by
applying torque to the motors which drive the
joints.

Moreover, each angle should not be moved more than

its limits and each motor has a maximum torque limit .
Formulate the problem as a terminal control problem
 Example- pick and place robot
• First we have to identify the objective function. As a
terminal control problem, the objective is to
minimize the terminal error. The error is the
difference between desired terminal
position/orientation and actual position/orientation
of the end effector.
• Assume the end effector desired position and
orientation at A is given by (3 pos. variable and 3
orientation var.) ‫ݔ‬஺
‫ݕ‬஺
‫ݖ‬஺
ߙ஺ =Desired position and orientation D
ߚ஺
ߛ஺
 Example- pick and place robot
• If the actual final position and orientation of
the robot is given by ‫ݔ‬ ଵ
‫ݔ‬ଶ
‫ݔ‬ଷ
‫ݐ ݔ‬௙ = ‫ݔ‬
ସ
‫ݔ‬ହ
‫଺ݔ‬
• Then the error will also be six dimensional
variable given by ‫ݐ(ݔ = ܧ‬௙ )-D
• Hence the objective function will then be
‫ݔ‬ଵ − ‫ݔ‬஺ ் ‫ݔ‬ଵ − ‫ݔ‬஺
‫ݔ‬ଶ − ‫ݕ‬஺ ‫ݔ‬ଶ − ‫ݕ‬஺
‫ݔ‬ଷ − ‫ݖ‬஺ ‫ݔ‬ଷ − ‫ݖ‬஺
݉݅݊݅݉݅‫ ݔ = ܬ ݁ݖ‬− ߙ S ‫ ݔ‬− ߙ
ସ ஺ ସ ஺
‫ݔ‬ହ − ߚ஺ ‫ݔ‬ହ − ߚ஺
‫ ଺ݔ‬− ߛ஺ ‫ ଺ݔ‬− ߛ஺
 Minimum control effort
• Objective: is to minimize the energy used to
drive a system from its initial state to a final
state
• J has the form
tf


J  U (t )T RU (t )dt
to

– R is a weighting matrix with dimension mxm

• Example: minimum fuel problem
 Example: Reformulate the minimum time control
problem of the rocket as a minimum effort problem
• In rocket control, one important objective is
minimizing the control effort of fuel
consumed.
• The minimum time problem when formulated
as energy minimization, it will be given as
Minimum effort rocket launch problem
௧ ଵ
݉݅݊݅݉݅‫׬ = ܬ ݁ݖ‬௧ ೑ ଶ ‫ݑ‬ଶ dt
೚

subject to
‫ݏ‬ሶ = ‫ݒ‬
‫ݒ‬ሶ = ‫ ݑ‬− 0.2‫ ݒ‬ଶ /݉
݉ሶ = −0.01‫ݑ‬ଶ
u ≤ 1.1
‫ ≥ ݑ‬−1.1
‫ ≤ ݒ‬1.7
‫ ݒ‬0 =0 ‫ݐ ݒ‬௙ = 0
‫ ݏ‬0 =0 ‫ݐ ݏ‬௙ = 0
 Optimal servomechanism or tracking
problem
• Objective: is to design a control signal which
minimizes the error between desired actual
path of a system
• J has the form
tf
tf
 
T
J  [ x(t )  d (t )] Q[ x(t )  d (t )]dt  e(t )T Qe(t )dt
to
to

• Example: satellite tracking, continuous path

control of a robot etc
 Optimal servomechanism contd
• The above problem can be extended to more
advanced forms
– Servomechanism with minimum control effort
tf


J  ([ x(t )  d (t )]T Q[ x(t )  d (t )]  U (t )T RU (t ))dt
to

– Servomechanism with minimum effort and

terminal control
tf


J  [ X (t f )  D (t f )]T S[ X (t f )  D(t f )]  ([ X (t )  d (t )]T Q[ x(t )  d (t )]  U (t )T RU (t ))dt
to
 Optimal regulator problem
• This is a sub problem of the optimal
servomechanism where the final point is zero
• Objective: is to drive a system towards a point
where the final state value is zero.
• The performance measure J is given by
tf


J  [ X (t f )]T S[ X (t f )]  ([ X (t )]T Q[ x(t )]  U (t )T RU (t ))dt
to
 Examples
• Consider a body of mass M moving along a
frictionless surface
u
M

x o , vo
x f ,v f

• Find a control signal u, bounded as -amax<u<amax

which can transfer the system from its initial
state to final state with minimum time.
 Examples
• Model the system
d d2
F  ma  m (v(t ))  m 2 ( x(t ))
dt dt
x&1  x2
x& 2  ( F / m)  u

• The performance measure J is then

tf


J  dt  t f
0
 Examples
• If friction of the surface us considered in the
motion of the above body, the SS model
dx d d2
F b  ma  m (v(t ))  m 2 ( x(t ))
dt dt dt
x&1  x2
F b b
x& 2   x2  u  x2
m m m
• The cost function can be taken to be the same
 Conclusion
• When considering optimal control problems,
the following four points need to be
considered
– Existence of solution
– Uniqueness
– Main features of the optimal solution
– The form of the solution