D

x0
u(t) x(t) x(t) + y(t)
B + + C +

T6: Introduction to Optimal Control

Gabriel Oliver Codina

Systems, Robotics & Vision Group
Universitat de les Illes Balears

G.Oliver, UIB

Introduction
Optimal control deals with the problem of finding a control law for a given system such that a
certain optimality criterion (performance index) is achieved.
The system which is the end result of an optimal design is not supposed merely to satisfy
some of the constrains associated with the classical control (stability, bandwidth) but it is
supposed to be the best possible system of a particular type.

In a general (nonlinear) system, the optimization problem can be stated as follows:

Consider the nonlinear state equation: x & (t) = f [ x(t ), u (t ), t ] x (t0 ) = x0
and an objective function, or cost function, to be minimized:
tf

J = L[x (t ), u (t ), t ]dt + [ x (t f ), t f ]
t0

L(x,u,t) is called the loss function that assigns a penalty to the state and input and (x,t) is the
terminal cost that characterizes the final state.
The optimization technique will try to determine the optimal state trajectory and input signal
denoted by [x*(t), u*(t)] that minimizes the performance index subject to the constraints
imposed by the state equation and the initial state.
For a particular problem, the cost function L quantifies the conflicting objectives of regulation
performance and control effort.
Necessary (in general, not sufficient) conditions that must be satisfied by the optimal solution
[x*(t), u*(t)] can be derived from variational calculus and dynamic programming tools.

G.Oliver, UIB

1
 The Linear Quadratic Regulator
The linear quadratic regulator (LQR) is a special case of optimal control problem that can
be analytically solved, which arises when the objective (cost) function is a quadratic function
of state x and input and u, and a LTI state equation is considered.
The quadratic performance index to minimize is:
tf

J=
1
2 t0
[ ] 1
xT (t )Qx(t ) + uT (t )Ru(t ) dt + xT (t f )Sx(t f )
2
thus, the loss function and the terminal cost function can be expressed as quadratic forms:

L ( x, u ) = (
1 T 1
) Q 0 x
x Qx + uT Ru = [ xT uT ]
1
( x) = xT Sx
2 2 0 R u 2
It is in general assumed that the weighting matrices Q, R and S are symmetric, with Q and S
positive semidefinite and R positive definite.
Qualitative idea: if the quadratic term involving the state are large compared to those of the
input (Q>>R), then the optimal solution will exhibit good regulation performance, that is, the
response will go fast to its desired value, but at the expense of large control energy.

A square matrix M is called symmetric if it verifies: M=MT

A real squared matrix M is called positive definite if xTMx>0, for every vector x0
A real squared matrix M is called positive definite if xTMx0, for every vector x0

G.Oliver, UIB

The Linear Quadratic Regulator

Under the stated conditions, it can be demonstrated that the optimal control law
can be expressed as a linear state feedback:
u(t ) = K (t ) x(t)
where the time-varying feedback gain matrix is given by:
K (t ) = R 1B T P(t)
and P(t) is the solution of the differential Riccati equation:
P& (t) = AT P (t ) + P (t ) A P (t )BR 1BT P (t ) + Q
with boundary condition P(tf)=S

G.Oliver, UIB

2
 Steady-State LQR Problem
In the S-S linear quadratic regulator problem (tf ), provided the pair (A,B) is
stabilizable (<0, (A-BK)), the differential Riccati equation tends to a
constant steady-state value P which characterizes the equilibrium solution to
the Riccati differential equation, which is referred to as the algebraic Riccati
equation:
AT P + P A P BR 1BT P + Q = 0

In this case, the optimal control feedback law is time invariant and given by:
u(t ) = K LQR x(t) K LQR = R 1BT P
which yields the optimal time-invariant CLoop state equation:
x& (t ) = ( A BK LQR ) x(t ) = ( A BR 1BT P )x(t ) x(0) = x 0

x0
u(t) x(t) x(t) y(t)
B + + C

-R
R-1BTKLQR

G.Oliver, UIB

Steady-State LQR Problem

The next important result can be demonstrated: If (A,B) is controllable and (A,C)
is observable, the algebraic Riccati equation has a unique solution P , which is
symmetric positive-definite, and the CLoop state-equation is asymptotically
stable.

MATLAB functions for LQR design:

Klqr=lqr(A,B,Q,R) calculates the optimal linear quadratic regulator gain matrix

Klqr that minimizes the quadratic performance index J for the control law u=-Kx
applied to the system x=Ax+Bu

G.Oliver, UIB