Filters, Cost Functions, and

Controller Structures

Robert Stengel
Optimal Control and Estimation MAE 546
Princeton University, 2013"

! Dynamic systems as low-pass filters"

! Frequency response of dynamic systems"
! Shaping system response"
! LQ regulators with output vector cost functions"
! Implicit model-following"
! Cost functions with augmented state vector"

First-Order Low-Pass
Filter

min J =
u

1
xT (t)Qx(t) + uT (t)Ru(t) dt
2 0

subject to
x(t) = Fx(t) + Gu(t)

Copyright 2013 by Robert Stengel. All rights reserved. For educational use only.!
http://www.princeton.edu/~stengel/MAE546.html!
http://www.princeton.edu/~stengel/OptConEst.html

Low-Pass Filter"
! Low-pass filter passes low frequency signals and
attenuates high-frequency signals"

x ( t ) = ax ( t ) + au ( t )
a = 0.1, 1, or 10

Response of 1st-Order Low-Pass Filters to

Step Input and Initial Condition"
x ( t ) = ax ( t ) + au ( t )
a = 0.1, 1, or 10

! Laplace transform, x(0) = 0"

x (s) =

a
u (s)
( s + a)

! Frequency response, s = j!

x ( j ) =

a
u ( j )
( j + a )

Smoothing effect on sharp changes!

Response of 1st-Order Low-Pass

Filters to Sine-Wave Inputs"

Frequency Response of
Dynamic Systems

sin ( t )

u ( t ) = sin ( 2t )

sin ( 4t )

Input!

Output!
x ( t ) = 10x ( t ) + 10u ( t )

x ( t ) = x ( t ) + u ( t )

x ( t ) = 0.1x ( t ) + 0.1u ( t )

Response of 1st-Order Low-Pass

Filters to White Noise"

Relationship of Input Frequencies

to Filter Bandwidth"
sin ( t )

u ( t ) = sin ( 2 t )

sin ( 4 t )

x ( t ) = ax ( t ) + au ( t )
a = 0.1, 1, or 10

BANDWIDTH"
Input frequency at
which magnitude =
3dB"
How do we
calculate frequency
response?"

Bode Plot Asymptotes, Departures,

and Phase Angles for 1st-Order Lags"
General shape of amplitude
ratio governed by asymptotes"
Slope of asymptotes changes
by multiples of 20 dB/dec
at poles or zeros"
Actual AR departs from
asymptotes"

x ( j ) =

( j + a )

u ( j )

2nd-Order Low-Pass Filter"

x ( t ) = 2 n x ( t ) n2 x ( t ) + n2u ( t )
! Laplace transform, I.C. = 0"

x (s) =

AR asymptotes of a real pole"

When = 0, slope = 0 dB/
dec"
When , slope = 20 dB/
dec"

Phase angle of a real,

negative pole"

n2
u (s)
s 2 + 2 n s + n2

! Frequency response, s = j!

x ( j ) =

n2
u ( j )
( j )2 + 2 n ( j ) + n2

When = 0, = 0"
When = , =45"
When -> , -> 90"

2nd-Order Step Response"

Amplitude Ratio Asymptotes and Departures of

Second-Order Bode Plots (No Zeros)"

AR asymptotes of a
pair of complex poles"
When = 0, slope
= 0 dB/dec"
When n, slope
= 40 dB/dec"

Height of resonant
peak depends on
damping ratio"

Phase Angles of Second-Order

Bode Plots (No Zeros)"

Transformation of the System

Equations"
Time-Domain System Equations!

Phase angle of a pair of

complex negative poles"
When = 0, = 0"
When = n, =90"
When -> , ->
180"

Abruptness of phase
shift depends on
damping ratio"

x (t) = F x(t) + G u(t)

y(t) = H x x(t) + H u u(t)
Laplace Transforms of System Equations!

sx(s) x(0) = F x(s) + G u(s)

x(s) = [ sI F ]

[ x(0) + G u(s)]

y(s) = H x x(s) + H u u(s)

Scalar Frequency Response

from Transfer Function Matrix"

Transfer Function Matrix"

Laplace Transform of Output Vector!

y(s)=H x x(s) + H u u(s) = H x [ sI F ]

[ x(0) + G u(s)] + H u(s)

u

1
1
= H x ( sI F ) G + H u u(s) + H x [ sI F ] x(0)

= Control Effect + Initial Condition Effect

Transfer Function Matrix relates control input to
system output "
with Hu = 0 and neglecting initial condition!

H (s) = H x [ sI F ] G
1

(r x m)

Transfer function matrix with s = j"

H (j ) = H x [ j I F ] G
1

(r x m)

yi ( s )
1
= H ij (j ) = H xi [ j I F ] G j
u j ( s )
H xi = i th row of H x
G j = j th column of G

(r x m)

Second-Order Transfer Function"

Second-order dynamic system"
! x ( t )
1
x ( t ) = #
# x2 ( t )
"

$ ! f
& = # 11
& #" f21
%

f12 $ ! x1 ( t )
&#
f22 & # x2 ( t )
%"

! y (t )
1
y (t ) = #
# y2 ( t )
"

$ ! h
& = # 11
& #" h21
%

$ ! x1 ( t )
&#
&% #" x2 ( t )

h12
h22

$ ! g
& + # 11
& #" g21
%

g12 $ ! u1 ( t )
&#
f22 & # u2 ( t )
%"

$
&
&
%

$
&
&
%

Second-order transfer function matrix"

h11 h12
H(s) = H x A ( s ) G =
h21 h22

(s f )
11
adj
f21

(s f )

11
det
f21

f12

( s f22 )
f12

( s f22 )

(n = m = r = 2)

g11

g21

g12

f22

(s)

kij s q + bq 1s q 1 + ... + b1s + b0

(s

+ an 1s

n 1

+ ... + a1s + a0

Just one element of the matrix, H(s)"

Denominator polynomial contains n roots"
Each numerator term is a polynomial with q zeros,
where q varies from term to term and n 1"

kij ( s z1 )ij ( s z2 )ij ...( s zq )ij

( j 1 )( j 2 )...( j n )

= a( )+ jb( ) AR( ) e j ( )
Frequency response is a complex function of
input frequency, "

# zeros = q!
# poles = n!

( s 1 ) ( s 2 ) ...( s n )

http://www.mathworks.com/matlabcentral/!
http://www.mathworks.com/matlabcentral/fileexchange/10183-bode-plot-with-asymptotes!

2nd-Order Pitch Rate Frequency Response, with zero"

kij ( j z1 )ij ( j z2 )ij ... j zq

MATLAB Bode Plot with asymp.m"

Substitute: s = j"

Real and imaginary parts, or"

** Amplitude ratio and phase angle **"

H ij (s) =

kij nij (s)

( r n )( n n )( n m )
= (r m) = (2 2)

Scalar Frequency Response

Function"

H ij (j ) =

Scalar Transfer Function

from uj to y"i

ij

bode.m"

asymp.m"

Desirable Open-Loop Frequency Response

Characteristics (Bode)"

High gain (amplitude)

at low frequency"

Examples of
Proportional LQ
Regulator Response

Desired response is
slowly varying"

Low gain at high

frequency"
Random errors vary
rapidly"

Crossover region is
problem-specific!

Example: Open-Loop Stable and

Unstable 2nd-Order LTI System
Response to Initial Condition"

0
1
FS =

16 1
0
1
FU =

16 +0.5

Stable Eigenvalues ="

-0.5000 + 3.9686i"
-0.5000 - 3.9686i"
"
Unstable Eigenvalues ="
0.2500 + 3.9922i"
0.2500 - 3.9922i"

Example: Stabilizing Effect of Linear-Quadratic

Regulators for Unstable 2nd-Order System"
1

min J = min x12 + x22 + ru 2 dt

u
u
2 0

x1 (t)
= c1 x1 (t) c2 x2 (t)
u(t) = c1 c2

x2 (t)

For the
unstable
system"

r = 1"
Control Gain (C) ="
0.2620 1.0857"
"
Riccati Matrix (S) ="
2.2001 0.0291"
0.0291 0.1206"
"
Closed-Loop Eigenvalues ="
-6.4061"
-2.8656"

r = 100"
Control Gain (C) ="
0.0028 0.1726"
"
Riccati Matrix (S) ="
30.7261 0.0312"
0.0312 1.9183"
"
Closed-Loop Eigenvalues ="
-0.5269 + 3.9683i"
-0.5269 - 3.9683i"

Example: Stabilizing/Filtering Effect of LQ

Regulators for the Unstable 2nd-Order System"

Example: Open-Loop Response of the Stable

2nd-Order System to Random Disturbance"

r = 1"
Control Gain (C) ="
0.2620 1.0857"
"
Riccati Matrix (S) ="
2.2001 0.0291"
0.0291 0.1206"
"
Closed-Loop Eigenvalues ="
-6.4061"
-2.8656"

r = 100"
Control Gain (C) ="
0.0028 0.1726"
"
Riccati Matrix (S) ="
30.7261 0.0312"
0.0312 1.9183"
"
Closed-Loop Eigenvalues ="
-0.5269 + 3.9683i"
-0.5269 - 3.9683i"

Eigenvalues = -1.1459, -7.8541"

Example: Disturbance Response of

Unstable System with Two LQRs"

LQ Regulators with Output

Vector Cost Functions

State Rate Can Be

Expressed as an Output
to be Minimized"

Quadratic
Weighting of the
Output"
y(t) = H x(t) + H u(t)
x

x(t) = Fx(t) + Gu(t)

1
J = yT (t)Qy y(t) dt
20

y(t) = H x x(t) + H u u(t) Fx(t) + Gu(t)

J=

1
[ Hx x(t) + Hu u(t)]T Qy [ Hx x(t) + Hu u(t)] dt
2 0

1
min J = min xT (t) uT (t)
u
u 2 0

1
min J min xT (t) uT (t)
u
u 2 0

x(t) dt
H uT Qy H x H uT Qy H u + R o u(t)

x(t)
QO
MO

dt
T
MO R O + R o u(t)

H xT Qy H x

H xT Qy H u

u(t) = uC (t) CO x ( t )

Implicit Model-Following LinearQuadratic Regulator"

Model the response of one airplane with another
using feedback control"

1
T
T
x (t) u (t)

2 0

1
xT (t) uT (t)

20

J=

1
1
yT (t)Qy y(t) dt = xT (t)Qy x(t) dt
2 0
20

x(t) dt
GT Qy F GT QyG + R o u(t)

x(t)
QSR
M SR

dt
MTSR R SR + R o u(t)

FT Q y F

FT Q y G

u(t) = uC (t) C SR x ( t )

Special case of output weighting!

Princeton Variable-Response
Research Aircraft (VRA)"

Implicit Model-Following
LQ Regulator"

Implicit ModelFollowing LQ
Regulator"

x(t) = Fx(t) + Gu(t)

x M (t) = FM x M (t)

Simulator aircraft dynamics"

x(t) = Fx(t) + Gu(t)

If simulation is successful,
x M (t) x(t)
and
x M (t) FM x(t)

Ideal aircraft dynamics"

x M (t) = FM x M (t)
Feedback control law"

u(t) = uC (t) C IMF x ( t )

Another special case of output weighting!

Implicit Model-Following
LQ Regulator"

Proportional-Derivative Control"

Cost function penalizes difference between actual and ideal

model dynamics"

1
T
J = [ x(t) x M (t)] Q M [ x(t) x M (t)] dt
20
FF T Q FF

T
(
( F FM )T QM G
1
M)
M (
M)
x(t) u(t)

20
GT Q M ( F FM )
GT Q M G + R o

x(t)
T Q IMF
M IMF
1

x(t) u(t)
dt
T
20
M IMF R IMF + R o u(t)

J=

u(t) = Cx ( t ) + uC (t)
! Derivative feedback can either quicken or slow system
response ( lead or lag ), depending on the control gain sign"

x(t) dt
u(t)

Therefore, ideal model is implicit in the optimizing feedback

control law"
u(t) = uC (t) C IMF x ( t )

! Basic LQ regulators provide proportional control"

u(t) = C P x ( t ) C D x ( t ) + uC (t)

! How can proportional-derivative

(PD) control be implemented
with an LQ regulator?"

Explicit Proportional-Derivative Control"

Implicit Proportional-Derivative Control"

u(t) = C P x ( t ) C D x ( t ) + uC (t)

Add state rate, i.e., the derivative, to a standard cost function"

Include system dynamics in the cost function"

Substitute for the derivative"

u(t) = C P x ( t ) C D [ Fx(t) + Gu(t)] + uC (t)

[ I C G ] u(t) = C
D

J=

x ( t ) C D Fx(t) + uC (t)

Structure is the same as that of proportional control"

u(t) = [ I C DG ] ( C P C D F ) x ( t ) + uC (t)
1

C PD x ( t ) + [ I C DG ] uC (t)
1

Penalty/reward for fast motions"

"

QPD
1
T
T
x (t) u (t)
2 0
MTPD

M PD x(t)

dt
R PD u(t)

Must verify guaranteed stability criteria"

"

LQ

u(t) = C PD x ( t ) + uC (t)

Inverse Problem: Given a stabilizing gain matrix, CPD, does

it minimize some (unknown) cost function? [TBD]!

Integral Compensation Can Reduce

Steady-State Errors"
! Sources of Steady-State Error"

Cost Functions with

Augmented State Vector

T
1
xT (t)Qx x(t) Fx ( t ) + Gu ( t ) Qx Fx ( t ) + Gu ( t ) + uT (t)Ru(t) dt
2 0

J=

Implement as ad hoc modification of

proportional LQ control, e.g.," C D = C P

1
xT (t)Qx x(t) xT (t)Qx x(t) + uT (t)Ru(t) dt
2 0

! Constant disturbance"
! Errors in system dynamic model"

! Selector matrix, HI,

can reduce or mix
integrals in feedback"

x(t) = Fx(t) + Gu(t)

( t ) = H x(t)
I

LQ ProportionalIntegral (PI) Control"

Integral State is Added to

the Cost Function and
the Dynamic Model"

u(t) = C B x ( t ) C I H I x ( ) d
0

C B x ( t ) C I ( t ) + uC (t)
where the integral state is
t

( t ) H I x ( ) d
0

dim ( H I ) = m n

x ( t )
define ( t )
(t )

min J =
u

1
xT (t)Qx x(t) + T ( t ) Q ( t ) + uT (t)Ru(t) dt
2 0

1 T Qx
(t)
2 0
0

subject to
= F (t) + G u(t)
(t)

0
(t) + uT (t)Ru(t) dt
Q

u(t) = C (t) + uC (t)

= C B x ( t ) C I ( t ) + uC (t)

Integral State is Added to the Cost

Function and the Dynamic Model"
u(t) = C (t) + uC (t)
= C B x ( t ) C I ( t ) + uC (t)
u(s) = C (s) + uC (s)
= C B x ( s ) C I ( s ) + uC (s)
= C B x ( s ) C I

H x x ( s )
+ uC (s)
s

C B sx ( s ) + C I H x x ( s )
+ uC (s)
s
[C s + C I Hx ] x ( s ) + u (s)
= B
C
s

u(s) =

Form of (m x n)
Bode Plots
from x to u?!

Actuator Dynamics and

Proportional-Filter LQ
Regulators

Proportional LQ Regulator: HighFrequency Control in Response to HighFrequency Disturbances"

Actuator Dynamics May Impact

System Response"

2nd-Order System Response with Perfect Actuator!

Good disturbance rejection, but may high bandwidth may be unrealistic"

Actuator Dynamics May Affect

System Response"

LQ Regulator with Actuator Dynamics"

! Cost function is minimized with re-defined
state and control vectors"

! Augment state dynamics to include actuator dynamics"

x(t) F G x(t) 0

=
+

v(t)

u(t)
0 K u(t) I

x(t)
F G
0
(t) =
; F =
; G =

u(t)
0 K
I

min J =
u

1
xT (t)Qx x(t) + uT (t)R u u(t) + v T (t)R v v(t) dt
2 0
=

! Control variable is actuator forcing function"

v(t) = u Integrator (t) = C B x ( t ) + v C (t) is sub - optimal

1 T Qx 0
T

dt

(t)
(t)
+
v
(t)R
v(t)
v
2 0

0 R u

subject to
= F (t) + G v(t)
(t)

LQ Regulator with Actuator Dynamics"

LQ Regulator with Actuator Dynamics"

= Ku(t) C A u(t) C B x ( t ) + v C (t)

u(t)

Control Force!

su(s) = Ku(s) C A u(s) C B x ( s ) + v C (s) + u(0)

v(t) = C (t) + v C (t)

= C B x ( t ) C A u(t) + v C (t)
v(s) = C B x ( s ) C A u(s) + v C (s)

Control Displacement!

[ sI + K + C ] u(s) = C x ( s ) + v (s)
u(s) = [ sI + K + C ] C x ( s ) + v (s)
A

LQ Regulator with Artificial

Actuator Dynamics"

Proportional-Filter
(PF) LQ Regulator"

! LQ control variable is derivative of actual system control"

x(t)

u(t)

F G x(t) 0
=
+

v(t)
0 0 u(t) I

x(t)
F G
0
(t) =
; F =
; G =

u(t)
0 0
I
Optimal LQ Regulator!

v(t) = u Integrator (t) = C (t) + v C (t)

v(t) = u Int (t) = C B x ( t ) C A u(t) + v C (t)
CA introduces artificial actuator dynamics!

CA provides low-pass filtering effect on the control input"

u(s) = [ sI + C A ] C B x ( s ) + v C (s)
1

Proportional-Filter LQ Regulator Reduces

High-Frequency Control Signals"
2nd-Order System Response!

Next Time:
Linear-Quadratic Control
System Design

... at the expense of decreased disturbance rejection"