Linear Control Systems

Lecture # 14
Linear Quadratic Regulator (LQR)

p. 1/2

Consider the system

x = Ax + Bu

and suppose we want to design state feedback control

u = F x to stabilize the system. The design of F is a
tradeoff between the transient response and the control
effort. The optimal control approach to this design tradeoff
is to define the performance index (cost functional)
Z
J =
[xT (t)Qx(t) + uT (t)Ru(t)] dt
0

and search for the control u = F x that minimizes this index.

Q is an n n symmetric positive semidefinite matrix and R
is an m m symmetric positive definite matrix

p. 2/2

The matrix Q can be written as Q = M T M , where M is a

p n matrix, with p n. With this representation
xT Qx = xT M T M x = z T z

where z = M x can be viewed as a controlled output

Optimal Control Problem: Find u(t) = F x(t) to minimize J
subject to the constraint x = Ax + Bu
Since J is defined by an integral over [0, ), the first
question we need to address is: Under what conditions will
J exist and be finite?

p. 3/2

Write J as

J = lim J(tf )
tf

J(tf ) =

tf

[xT (t)Qx(t) + uT (t)Ru(t)] dt

J(tf ) is a monotonically increasing function of tf . Hence,

as tf , J(tf ) either converges to a finite limit or
diverges to infinity

Under what conditions will limtf J(tf ) be finite?

p. 4/2

Recall that (A, B) is stabilizable if the uncontrollable

eigenvalues of A, if any, have negative real parts
Notice that (A, B) is stabilizable if (A, B) is controllable or
Re [(A)] < 0
Definition: (A, C) is detectable if the unobservable
eigenvalues of A, if any, have negative real parts
Lemma 1 : Suppose (A, B) is stabilizable, (A, M ) is
detectable, where Q = M T M , and u(t) = F x(t). Then, J
is finite for every x(0) Rn if and only if
Re [(A + BF )] < 0

p. 5/2

Remarks:
1. The need for (A, B) to be stabilizable is clear, for
otherwise there would be no F such that
Re [(A + BF )] < 0
2. To see why detectability of (A, M ) is needed, consider
Z
x = x + u, J =
u2 (t) dt
0

A = 1, B = 1, M = 0, R = 1
(A, B) is controllable, but (A, M ) is not detectable
F = 0 u(t) = 0 J = 0

This control is clearly optimal and results in a finite J but it

does not stabilize the system because A + BF = A = 1

p. 6/2

Lemma 2: For any stabilizing control u(t) = F x(t), the

cost is given by
J = x(0)T W x(0)
where W is a symmetric positive semidefinite matrix that
satisfies the Lyapunov equation
W (A + BF ) + (A + BF )T W + Q + F T RF = 0

Remark: The control u(t) = F x(t) is stabilizing if

Re [(A + BF )] < 0

p. 7/2

Theorem: Consider the system x = Ax + Bu and the

performance index
Z
J =
[xT (t)Qx(t) + uT (t)Ru(t)] dt
0

where Q = M T M , R is symmetric and positive definite,

(A, B) is stabilizable, and (A, M ) is detectable. The
optimal stabilizing control is
u(t) = R1 B T P x

where P is the symmetric positive semidefinite solution of

the Algebraic Riccati Equation (ARE)
0 = P A + AT P + Q P BR1 BP

p. 8/2

Remarks:
1. Since the control is stabilizing,
Re [(A BR1 B T P )] < 0

2. The control is optimal among all square integrable

signals u(t), not just among u(t) = F x(t)
3. The Riccati equation can have more than one solution,
but there is only one solution that is positive semidefinite

p. 9/2

Example:
x 1 = x2 , x 2 = u
Z
[x21 + u2 ] dt, > 0
J =
0

"

"

"

0 1
0
1 0
A=
, B=
, Q=
, R=
0 0
1
0 0
" #
i
h
i
1 h
Q=
1 0 M = 1 0
0
"
#
0 1
rank[B, AB] = rank
= 2 (A, B) is controllable
1 0

p. 10/2

rank

"

M
MA

= rank

"

1 0
0 1

=2

(A, M ) is observable
0 = P A + AT P + Q P BR1 BP
P =

"

p11 p12
p12 p22
p12

p12

0 = 1 1 p212

0 = p11 1 p12 p22

0 = 2p12 1 p222
p

= , p22 = 2 p12

3/4
1/4
1

= , p22 = 2 , p11 = p12 p22 = 2

p. 11/2

P =

"

21/4
1/2
3/4
1/2

p11 > 0, det(P ) = > 0 P is positive definite

i
h

1
F = R1 B T P =
1/2
23/4

#
"
0
1
1/4
A + BF =
1/2
2

1,2 =

1/4

1
(1 j)
2

p. 12/2

Notice how F and the eigenvalues depend on

i
h

F = 1/2
21/4
1,2 =

1/4

1
(1 j)
2

What happens as you change ?

p. 13/2

Matlab Calculations:
X = lyap(A,N) solves the Lyapunov equation
AX + XAT + N = 0

To solve the equation

W (A + BF ) + (A + BF )T W + Q + F T RF = 0

use W = lyap((A+B*F)0 ,Q+F0 *R*F) ( Notice the transpose)

p. 14/2

X = are(A,S,Q)

solves the Riccati equation

XA + AT X + Q XSX = 0
S = BR1 B

[K,P,E] = lqr(A,B,Q,R)

solves the Riccati equation

P A + AT P + Q P BR1 B T P = 0
K = R1 B T P and E is a vector whose elements are the
eigenvalues of (A BR1 B T P )
Notice that F = K

p. 15/2

LQR Design
Given the system
x = Ax + Bu

where (A, B) is stabilizable, we want to design state

feedback control u = F x to stabilize the system while
meeting
Transient response specifications
Magnitude constraints on x and u

p. 16/2

Procedure:
1. Choose Q and R such that Q = M T M , with (A, M )
detectable, and R = RT > 0
2. Solve the Riccati equation
P A + AT P + Q P BR1 B T P = 0

and compute F = R1 B T P
3. Simulate the initial response of x = (A + BF )x for
different initial conditions
4. If the transient response specifications and/or the
magnitude constraints are not met, go back to step 1 and
re-choose Q and/or R

p. 17/2

Typical Choice:

q1

q2
Q=
...

qi =

tsi (ximax

)2

ri =

r1
r2

, R =

qn

rm

(uimax

...

)2

>0

tsi is the desired settling time of xi

ximax is a constraint on |xi |
uimax is a constraint on |ui |
is chosen to tradeoff regulation versus control effort

p. 18/2

Example:

0
0
1
0

A= 0
0
1 , B = 0
1
0.4 4.2 2.1

Open-loop eigenvalues are: 0.1, 1 1.7321j

Desired settling time is 4 sec.
Q = I3 ; Try R = 0.01, 0.1, and 1
R

0.01 9.7364, 0.9039 0.4593j

0.1 0.6568, 1.9548 1.0158j
1
0.2599, 1.1433 1.6840j

p. 19/2

Q = diag(1,1,1)

1.5

R=0.01
R=0.1
R=1

1.5

0.5
0
0.5
0

0.5

0.5

0.5

10

0.2

0.4

0.6

0.8

15

1.5

20

25

2.5

30

p. 20/2

R=0.1
1.5

Q =1
11
Q = 10

0.5
x2

11

0.5

0.5

0
0.5

1
0

10

15
3
0

20

p. 21/2

Q =1;R=0.1
11
Q =10;R=0.1
11
Q =10;R=0.4

1.5

1
0.5

0.5

11

0.5

1
0.5

0.2

0.4

0.6

0.8

10

15
3
0

20

p. 22/2