You are on page 1of 105

©

Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Chapter 4
Design via Optimal Control
Techniques
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Introdcution
We will emphasize methods for solving these
tougher problems based on optimal control theory.
There are three reasons for this choice:
1. It is relatively easy to understand
2. It has been used in a myriad of applications. (Indeed, the
authors have used these methods on approximately 20
industrial applications).
3. It is a valuable precursor to other advanced methods -
e.g., Model Predictive Control, which is explained in the
next chapter.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The analysis presented in this chapter builds on the
results in state space design methods were briefly
described in the SISO context.
We recall, from that chapter, that the two key
elements were
 state estimation by an observer
 state-estimate feedback
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
State-Estimate Feedback
Consider the following MIMO state space model
having m inputs and p outputs.

By analogy with state-estimate feedback in the SISO
case (as in Chapter 7), we seek a matrix K e 
m×n
and
a matrix J e 
n×p
such that (A
o
- B
o
K) and (A
o
- JC
o
)
have their eigenvalues in the LHP. Further we will
typically require that the closed-loop poles reside in
some specified region in the left-half plane. Tools such
as MATLAB provide solutions to these problems.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The Optimal Control Problem
Consider a general nonlinear system with input u(t) e 
m
,
described in state space form by


Problem: (General optimal control problem). Find an
optimal input u
o
(t), for t e [t
o
, t
f
], such that


where v(s, u, t) and g(x(t
f
)) are nonnegative functions.

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The Linear Quadratic Regulator
(LQR)
We next apply the above general theory to the following
problem.
Problem: (The LQR problem). Consider a linear time-
invariant system having a state space model, as defined
below:


We aim to drive the initial state x
o
to the smallest possible
value as soon as possible in the interval [t
o
, t
f
], but without
spending too much control effort.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Riccati Equation
Simple application of the general conditions for
optimality leads to



where P(t) satisfies

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The above equation is known as the Continuous Time
Dynamic Riccati Equation (CTDRE). This equation
has to be solved backwards in time, to satisfy the
boundary condition:

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Quick Review of Properties
We make the following simplifying assumptions:
(i) The system (A, B) is stabilizable from u(t).
(ii) The system states are all adequately seen by the cost
function. Technically, this is stated as requiring that
(+
½
, A) be detectable.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Under these conditions, the solution to the CTDRE,
P(t), converges to a steady-state limit P
s
·
as t
f
÷ ·.
This limit has two key properties:
 P
s
·
is the only nonnegative solution of the matrix algebraic
Riccati equation
obtained by setting dP(t)/dt = 0 in


 When this steady-state value is used to generate a feedback
control law, then the resulting closed-loop system is stable.




©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
More Detailed Review of Properties
Proposition: If P(t) converges as t
f
÷ ·, then the limiting
value P
·
satisfies the following Continuous-Time Algebraic
Riccati Equation (CTARE):

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Discrete-Time Optimal Regulators
The theory for optimal quadratic regulators for
continuous-time systems can be extended in a
straightforward way to provide similar tools for
discrete-time systems. We will briefly summarize
the main results.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Consider a discrete-time system having the following
state space description:


and the cost function
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The optimal quadratic regulator is given by

where K
u
[k] is a time-varying gain, given by

where P[k] satisfies the following Discrete Time
Dynamic Riccati Equation (DTDRE).
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
This equation must also be solved backwards, subject
to the boundary condition
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The steady-state (k
f
÷ ·) version of the control law is
given by

where K
·
and P
·
satisfy the associated Discrete Time
Algebraic Riccati Equation (DTARE):


with the property that A - BK
·
has all its eigenvalues
inside the stability boundary, provided that (A, B) is
stabilizable and (A, +
½
) has no unobservable modes on
the unit circle.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Connections to Pole Assignment
Note that, under reasonable conditions, the steady-
state LQR ensures closed-loop stability.
However, the connection to the precise closed-loop
dynamics is rather indirect; it depends on the choice
of + and u.
Thus, in practice, one usually needs to perform some
trial-and-error procedure to obtain satisfactory
closed-loop dynamics.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Observer Design
Next, we turn to the problem of state estimation.
Here, we seek a matrix J e 
n×p
such that A - JC has
its eigenvalues inside the stability region.
Again, it is convenient to use quadratic optimization.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
As a first step, we note that an observer can be designed
for the pair (C, A) by simply considering an equivalent
(called dual) control problem for the pair (A, B). To
illustrate how this is done, consider the dual system with

Then, using any method for state-feedback design, we
can find a matrix K' e 
p×n
such that A' - B'K' has its
eigenvalues inside the stability region. Hence, if we
choose J = (K')
T
, then we have ensured that A - JC has
its eigenvalues inside the stability region. Thus, we have
completed the observer design.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Table 22.1: Duality between quadratic regulators and filters

Note that, using the above correspondences, one can
convert an optimal filtering problem into an optimal
control problem and vice versa.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The procedure leads to a stable state estimation of the
form

Of course, using the tricks outlined above for state-
variable feedback, one can also use transformation
techniques to ensure that the poles describing the
evolution of the observer error also end up in any
region that can be related to either the continuous- or
the discrete-time case by a rational transformation.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Kalman Filter
The Kalman filter is a special observer that has
optimal properties under certain hypotheses. In
particular, suppose that.
1) w
k
and n
k
are statistically independent
(uncorrelated in time and with each other)
2) w
k
and n
k
, have Gaussian distributions
3) The system is known exactly
The Kalman filter algorithm provides an observer
vector J that results in an optimal state estimate.

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Details of the Stochastic Model
Consider a linear stochastic system of the form


where dv(t) dw(t) are known as orthogonal
increment processes.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Since a formal treatment of stochastic differential
equations is beyond the scope of this book, it suffices
here to think of the formal notation (t), (t) as
white-noise processes with impulsive correlation:



where E{} denotes mathematical expectation and
o() is the Dirac-delta function.
w
v
) ( } ) ( ) ( { , o , ÷ = t w t w
T
Q   E
) ( } ) ( ) ( { , o , ÷ = t v t v
T
R   E
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
We can then informally write the model as



For readers familiar with the notation of spectral density
for random processes, we are simply requiring that the
spectral density for (t) and (t) be Q and R,
respectively.
w v
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
In particular, one is frequently interested in the
steady-state optimal filter obtained when A, C, Q and
R are time invariant and the filtering horizon tends to
infinity. By duality with the optimal control
problem, the steady-state filter takes the form

where

and P
s
·
is the stabilizing solution of the following
CTARE:
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
We state without proof the following facts that are
the duals of those given for the LQP.
(i) Say that the system (C, A) is detectable from y(t); and
(ii) Say that the system states are all perturbed by noise.
(Technically, this is stated as requiring that (A, Q
½
) is
stabilizable).
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Then, the optimal solution of the filtering Riccati
equation tends to a steady-state limit P
s
·
as t ÷ ·.
This limit has two key properties:
 P
s
·
is the only nonnegative solution of the matrix
algebraic Riccati Equation

obtained by setting
dP(t)
/
dt
in

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
 When this steady-state value is used to generate a
steady-state observer, then the observer has the
property that (A - J
s
·
C) is a stability matrix.

Note that this gives conditions under which a stable
filter can be designed. Placing the filter poles in
particular regions follows the same ideas as used
earlier in the case of optimal control.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Discrete-Time Optimal Quadratic
Filter
We can readily develop discrete forms for the optimal filter.
In particular, consider a discrete-time system having the
following state space description:


where w[k] e 
n
and v[k] e 
n
are uncorrelated stationary
stochastic processes, with covariances given by


where Q e 
n×p
is a symmetric nonnegative definite matrix
and R e 
n×p
is a symmetric positive definite matrix
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Consider now the following observer to estimate the
system state:

Furthermore, assume that the initial state x[0] satisfies

Then the optimal choice (in a quadratic sense) for the
observer gain sequence {J
o
[k]} is given by

where P[k] satisfies the following discrete-time dynamic
Riccati equation (DTDRE).
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22


which can be solved forward in time, subject to
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The steady-state (k ÷ ·) filter gain satisfies the
DTARE given by
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22

Discrete-Time State-Space Model
k k
k k k
x y
Bu x x
C
A
=
+ =
+1
The above state-space system is deterministic since no
noise is present.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
We can introduce uncertainty into the model by
adding noise terms





This is referred to as a stochastic state-space model.
k k k k
w u x x + + =
+
B A
1
k k k
n x y + = C
Process
noise
Measurement
noise
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
In particular, for a 3rd Order System we have:
k k k k
w u x x + + =
+
B A
1
k k k
n x y + = C
Process
noise
Measurement
noise
k
k
k k
w
w
w
u
x
x
x
x
x
x
|
|
.
|

\
|
+
|
|
.
|

\
|
+
|
|
.
|

\
|
|
|
.
|

\
|
=
|
|
.
|

\
|
+
3
2
1
3
2
1
3
2
1
33 32 31
23 22 21
13 12 11
1
3
2
1
B
B
B
A A A
A A A
A A A
( ) ) ( ) (
3
2
1
3 2 1
k
k
k
n
x
x
x
y +
|
|
.
|

\
|
= C C C
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
x
k
3
˜ y
k
B, A C
+
+
x
k
1
x
k
2
y
k
n
k
w
k
u
k
~
X
k
- state vector
A - system matrix
B - input matrix
C - output matrix
y
k
- output (PV
m
)
y
k
- noise free output (PV)
w
k
- process noise
n
k
- measurement noise
u
k
- control input (MV)
This is illustrated below:
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
We recall that a Kalman Filter is a particular type of
observer. We propose a form for this observer on the
next slide.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Observers
We are interested in constructing an optimal observer
for the following state-space model:


An observer is constructed as follows:

where J is the observer gain vector, and is the
best estimate of y
k
i.e.


k k k
k k k k
n x y
w u x x
+ =
+ + =
+
C
B A
1
) ˆ ( ˆ ˆ
1 k k k k k
y y J u x x ÷ + + =
+
B A
k

. ˆ ˆ
k k
x y C =
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Thus the observer takes the form:


This equation can also be written as:
) ˆ ( ˆ ˆ
1 k k k k k
x y J u x x C B A ÷ + + =
+
k k k k
u Jy x J x B C A + + ÷ =
+
ˆ ) ( ˆ
1
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The optimal J is referred to as the Kalman Gain (J*)
k
k k k k k
x y
y y J u x x
ˆ ˆ
) ˆ ( * ˆ ˆ
1
C
B A
=
÷ + + =
+
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Solution:
The Kalman Filter
) ˆ ( ˆ
ˆ
ˆ
*
1 k k k k k k
x y J u x x C B A ÷ + + =
+
( )
1
2 *
÷
E + =
n
T T
k k
J CPC C AP
( ) ( )
( ) ( )
2
1
2
2
*
2
* *
1
*
w
T
k n
T
k
T
k k
n
T
k
w
T
k k k k
J J J J
¿ + ¿ + ÷ =
+ + ÷ ÷ =
÷
+
¿ ¿
A CP C CP C P P A
C A P C A P
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Simple Example
Problem:
Estimate a constant from measurements y
k
corrupted by
white noise of variance 1.
Model for constant ¬ x
k+1
= x
k
; w
k
= 0
Model for the corrupted measurement ¬ y
k
= x
k
+ n
k



An initial estimate of this constant is given, but this
initial estimate has a variance of 1 around the true value.
| | ( ) 1 var
2 2
= ¿ = = ¬
n k
k
n n E
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Solution Formulation
( ) | | ( )
1 ; 0 ; 0 ; 1
1 ˆ var ˆ
2 2
0 0 0
2
0 0
= ¿ = ¿ = =
= = ÷ = ÷
n w
x x x x E
B A
P
From previous Kalman Filter equations with A = 1; B = 0;
C = 1; ¿
w
2
= 0; ¿
n
2
= 1
1
1
) ˆ ( ˆ ˆ
2
1
*
*
1
+
÷ =
+
=
÷ + =
+
+
k
k
k k
k
k
k
k k k k k
J
x y J x x
P
P
P P
P
P
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Calculate P
k
(Given P
0
= 1)
( )
;
6
1
,
5
1
,
4
1
3
1
1 2
1
1
2
1
1 1
1
1
1
5 4 3
2
1
2
2
1
1
2
1
1 2
2
0
2
0
0 1
= = =
=
+
÷ =
+
÷ =
=
+
÷ =
+
÷ =
P P P
P
P
P P
P
P
P P
etc.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Calculate the estimate given the initial estimate
and the noisy measurements y
k

k

0
ˆ x
( )
( )
( )
0 0
0 0 0
0 0
0
0
0 1
ˆ
2
1
ˆ
1 1
1
ˆ
ˆ
1
ˆ ˆ
y x
x y x
x y x x
+ =
÷
+
+ =
÷
+
+ =
P
P
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
( )
( ) ( ) ( )
( )
( )
( )
3 2 1 0 0 4
2 1 0 0 3
1 0 0
0 0 1
2
1
2
1
0 0
1 1
1
1
1 2
ˆ
5
1
ˆ
ˆ
4
1
ˆ
ˆ
3
1
ˆ
2
1
1
ˆ
2
1
ˆ
1
ˆ ˆ
y y y y x x
y y y x x
y y x
y x y y x
x y x x
+ + + + =
+ + + =
+ + =
+ ÷
+
+ + =
÷
+
+ =
P
P
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The above result (for this special problem) is intuitively
reasonable. Note that the Kalman Filter has simply
averaged the measurements and has treated the initial
estimate as an extra piece of information (like an extra
measurement). This is probably the answer you would
have guessed for estimating the constant before you
ever heard of the Kalman Filter.
The fact that the answer is heuristically reasonable in
this special case encourages us to believe that the
Kalman Filter may give a good solution in other, more
complex cases. Indeed it does !
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Summary
 Full multivariable control incorporates the interaction
dynamics rigorously and explicitly.
 The fundamental SISO synthesis result that, under mild
conditions, the nominal closed-loop poles can be assigned
arbitrarily carries over to the MIMO case.
 Equivalence of state-feedback and frequency-domain pole
placement by solving the (multivariable) Diophantine
Equation carries over as well.
 The complexities of multivariable systems, cause criterion-
based synthesis (briefly alluded to in the SISO case) to gain
additional motivation.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
 A popular family of criteria are functionals involving
quadratic forms of control error and control effort.
 For a general nonlinear formulation, the optimal solution is
characterized by a two-point boundary-value problem.
 In the linear case (the so-called linear quadratic regulator,
LQR) the general problem reduces to the solution of the
continuous-time dynamic Riccati equation, which can be
feasibly solved, leading to time-variable state feedback.
 After initial conditions decay, the optimal time-varying
solution converges to a constant state feedback, the so-
called steady-state LQR solution.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
 It is frequently sufficient to neglect the initial transient of
the strict LQR and only implement the steady-state LQR.
 The steady-state LQR is equivalent
 to a model-matching approach, where a desired complementary
sensitivity is specified and a controller is computer that matches it
as closely as possible according to some selected measure, and
 to pole placement, where a closed-loop polynomial is specified and
a controller is computed to achieve it.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
 Thus, LQR, model matching, and pole placement are
mathematically equivalent, although they do offer different
tuning parameters.
Equivalent
synthesis
techniques
Tuning parameters
LQR relative penalties on control error
versus control effort.
Model matching closed-loop complementary
sensitivity reference model and
weighted penalty on the
difference to the control loop.
Pole placement closed-loop polynomial
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
 These techniques can be extended to discrete-time systems.
 There is a very close connection to the dual problem of
filtering: inferring a state from a related (but not exactly
invertible) set of measurements.
 Optimal-filter design based on quadratic criteria leads again to
a Riccati equation.
 The filters can be synthesized and interpreted equivalently in a
 linear quadratic,
 model-matching, or
 pole-placement
framework.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
 The arguably most famous optimal-filter formulation, the
Kalman filter, can be given a stochastic or a deterministic
interpretation, depending on taste.
 The LQR does not automatically include integral action;
thus, rejection of constant or other polynomial disturbances
must be enforced via the Internal Model Principle.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Chapter 23
Model Predictive Control
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Motivation
All real world control problems are subject to constraints of
various types.
The most common constraints are actuator constraints
(amplitude and slew rate limits).
In addition, many problems also have constraints on state
variables (e.g. maximal pressures that cannot be exceeded,
minimum tank levels, etc).
In many design problems, these constraints can be ignored, at
least in the initial design phase.
However, in other problems, these constraints are an
inescapable part of the problem formulation since the system
operates near a constraint boundary.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Outline
We describe one such mechanism here based on
Model Predictive Control.
This has actually been a major success story in the
application of modern control.
More than 2,000 applications of this method have
been reported in the literature - predominantly in the
petrochemical area.
Also, the method is being increasingly used in
electromechanical control problems.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Advantages of Model Predictive
Control
The main advantages of MPC are:
 it provides a one-stop-shop for MIMO control in the
presence of constraints,
 it is one of the few methods that allows one to treat state
constraints, and
 several commercial packages are available which give
industrially robust versions of the algorithms aimed at
chemical process control.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
What is Model Predictive
Control?
Model Predictive Control is a control algorithm
based on solving an on-line optimal control problem.
A receding horizon approach is used which can be
summarized in the following steps:
(i) At time k and for the current state x(k), solve, on-line, an
open-loop optimal control problem over some future
interval taking account of the current and future
constraints.
(ii) Apply the first step in the optimal control sequence.
(iii) Repeat the procedure at time (k+1) using the current state
x(k+1).
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Turning the solution into a closed
loop policy
The solution is converted into a closed loop strategy
by using the measured value of x(k) as the current
state.
When x(k) is not directly measured then one can
obtain a closed loop policy by replacing x(k) by an
estimate provided by some form of observer.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Details for Nonlinear Model
Given a model

the MPC at event (x, k) is computed by solving a constrained
optimal control problem:

where



and U
N
is the set of U that satisfy the constraints over the
entire interval [k, k + N - 1]; i.e.

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22



together with the terminal constraint


Usually U c R
m
is convex and compact, X c R
n
is convex
and closed, and W is a set that can be appropriately selected to
achieve stability.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
In the above formulation, the model and cost
function are time invariant. Hence, one obtains a
time-invariant feedback control law.




Then, the actual control applied at time k is the first
element of this sequence, i.e.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Figure 23.10: Receding horizon control principle
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
An alternative view of Receding Horizon
optimization is shown on the next slide
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Alternative View of Receding
Horizon Control Policy
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The associated literature can be divided into four
generations as follows:
 First generation (1970’s) - used impulse or step response linear
models, quadratic cost function, and ad-hoc treatment of constraints.
 Second generation (1980’s) - linear state space models, quadratic
cost function, input and output constraints expressed as linear
inequalities, and quadratic programming used to solve the
constrained optimal control problem.
 Third generation (1990’s) - several levels of constraints (soft, hard,
ranked), mechanisms to recover from infeasible solutions.
 Fourth generation (late 1990’s) - nonlinear problems, guaranteed
stability, and robust modifications.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Stability
A remarkable property of MPC is that one can
establish stability of the resultant feedback system (at
least with full state information).
This is made possible by the fact that the value
function of the optimal control problem acts as a
Lyapunov function for the closed loop system.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Assumptions
For clarity of exposition, we make the following
simplifying assumptions:
A1: An additional constraint is placed on the final state


A2: L(x, u) is positive definite in both arguments.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Theorem: Consider the system controlled by the
receding horizon MPC algorithm and subject to a
terminal constraint. This control law renders the
resultant closed loop system globally asymptotically
stable.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Linear Models with Quadratic
Cost Function
Model:
Error:
Combined “disturbance”:
Cost:
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
It can be shown that, if the design is unconstrained,
J
0
is minimized by taking
U = -W
-1
V
where W and V are functions of A, B, C matrices.
V is also a function of the disturbances and desired
output.
Magnitude and rate constraints on both the plant
input and output can be easily expressed as linear
constraints on U of the form
LU s K

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
QP Solution
Thus the constrained problem takes the form:



This optimization is a convex problem due to the
quadratic cost and linear constraints. Also, standard
numerical procedures (called Quadratic
Programming algorithms or QP for short) are
available to solve this sub-problem.
V W
T T
U
OPT
U U U U
K LU
2 min arg + =
s
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Summary of Key Idea
 Solve receding horizontal optimal control problem subject
to constraints
k
T
k k
T
k
N
k
N N
T
N
x
U u
Ru u Qx x x P x Min V +
¿
+ =
+
O e
e
1
 Apply first step of control - move forward one step
 Stability can be established - terminal constraint of x
crucial.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Example
1 2 . 0
3
) (
2
+ +
=
s s
s G
U
ss
= 0.33
4 . 0 < u
Plant:
Desired Output:
Steady State Input:
Constraint on Input:
1 =
ss
Y
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Results: dashed line - unconstrained input
solid line - constrained input found via MPC
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Output Response: Dashed line - response when unconstrained input is
saturated and applied to plant
Solid line - response obtained from MPC algorithm
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
One Degree of Freedom Controller
A one-degree of freedom output feedback controller
is obtained by including the set point in the quantities
to be estimated. The resultant output feedback MPC
strategy is schematically depicted below.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Figure 23.14: One-degree-of-freedom MPC architecture
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Integral Action
An important observation is that the architecture
described above gives a form of integral action. In
particular y is taken to the set-point y
s
irrespective of
the true plant description (provided a steady state is
reached and provided that, in this steady state, u is
not constrained).
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Rudder Roll Stabilization of Ships
Here we present a realistic application of model
predictive control to rudder roll stabilization of ships.
It is desirable to reduce the rolling motion of ships produced
by wave action so as to prevent cargo damage and improve
crew efficiency and passenger comfort. Conventional
methods for ship roll stabilization include water tanks,
stabilization fins and bilge keels. Another alternative is to
use the rudder for roll stabilization as well as course
keeping. However, using the rudder for simultaneous course
keeping and roll reduction is non-trivial since only one
actuator is available to deal with two objectives.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Constraint
An important issue in this problem is that the rudder
mechanism is usually limited in amplitude and slew
rate. Hence this is a suitable problem for model
predictive control.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Figure 23.15: Magnitudes and conventions for ship motion
description
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
The Model
Wave disturbances are usually considered at the
output, and the complete model of the ship dynamics
including the output equation is of the form:


where only the roll and yaw are typically directly
measured i.e. y := [, ]
T
. d
wave
is the wave induced
disturbance on the output variables.
wave
d x y
x x
+ =
+ =
C
B A o 
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Model for Wave Disturbances
The wave disturbances can be characterized in terms
of their frequency spectrum. This frequency
spectrum can be simulated by using filtered white
noise. The filter used to approximate the spectrum is
usually a second order one of the form
2
0 0
2
2
) (
e çe + +
=
s s
s K
s H
w
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Control Objectives
The design objectives for rudder roll stabilization are:
 Increase the damping and reduce the roll amplitude
 Control the heading of the ship

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Details of the Model
The following model was used for the ship

This system was sampled with a zero order hold
and sampling period of 0.5.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Details of the Optimization
Criterion
Details of the model predictive control optimization
criterion were that a standard LQR cost function was
employed with


Optimization horizon, N = 20; Control horizon M = 18.
1 . 0 ;
3 0
0 90
= u
(
¸
(

¸

= +
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Estimating the System State
Since only roll, , and yaw, , were assumed to be
directly measured, then a Kalman filter was
employed to estimate the 5 system states and 2 noise
states in the composite model.
The results of applying MPC to this problem are
shown on the following slides.
We consider three cases:
(i) No constraints on the rudder
(ii) Rudder constrained to a maximum angle of 30 degrees and a
maximum slew rate of 15 degrees per second
(iii) Rudder constrained to a maximum angle of 20 degrees and
a maximum slew rate of 8 degrees per second.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Figure 23.16: Ship motion with no constraints on rudder
motion
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22

Figure 23.17: Ship motion when rudder constrained to
maximum angle of 30 degrees and maximum
slew rate of 15 degrees/sec.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Figure 23.18: Ship motion when rudder constrained to
maximum angle of 20 degrees and maximum
slew rate of 8 degrees/sec.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Observations
We see that MPC offers a satisfactory solution to this
problem. Roll motion has been significantly reduced
and, when reformulated as part of the problem
description, constraints on maximum rudder angle
and slew rate have been imposed.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Summary
 MPC provides a systematic procedure for dealing with
constraints (both input and state) in MIMO control
problems.
 It has been widely used in industry.
 Remarkable properties of the method can be established,
e.g. global asymptotic stability provided certain conditions
are satisfied (e.g. appropriate weighting on the final state).
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
 The key elements of MPC for linear systems are:
 state space (or equivalent) model,
 on-line state estimation (including disturbances),
 prediction of future states (including disturbances),
 on-line optimization of future trajectory subject to
constraints using Quadratic Programming, and
 implementation of first step of control sequence.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Embellishments
The results presented in this chapter suggest there is a
close connection between MPC and anti-windup
provided the demands made on the system are not too
severe.
Actually, recent research has shown that there exists
a non-trivial region of state space in which MPC and
anti-windup are equivalent.
The next slides illustrate this idea.
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Illustrative Example
| |, 1 0 ,
08 . 0
4 . 0
,
1 4 . 0
0 1
=
(
¸
(

¸

=
(
¸
(

¸

= C B A
This is the zero-order hold discretisation (with sampling
period T
s
- 0.4 sec.) of the double integrator
). ( ) (
), ( ) (
), ( ) (
2
1 2
1
t x t y
t x t x
t u t x
=
=
=


 Saturation level:
1 = U
 Fixed horizon: 10 = N
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Regions of State Space: A : Region where no saturation occurs
B : Region in which MPC and anti-
windup are equivalent.
-4 -3 -2 -1 0 1 2 3 4
-8
-6
-4
-2
0
2
4
6
8
x1
x
2

B
A
B
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
First Step of MPC starting from a point in state space
in which MPC ÷ anti-windup but where input
saturation occurs
-2 -1 0 1 2
-1.5
-1
-0.5
0
0.5
1
1.5
x1
x
2

0 5 10 15 20 25
-1.5
-1
-0.5
0
0.5
1
1.5
k
u
(
k
)

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Second Step - Note input still saturated
-2 -1 0 1 2
-1.5
-1
-0.5
0
0.5
1
1.5
x1
x
2

0 5 10 15 20 25
-1.5
-1
-0.5
0
0.5
1
1.5
k
u
(
k
)

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Third Step - Note input still saturated
-2 -1 0 1 2
-1.5
-1
-0.5
0
0.5
1
1.5
x1
x
2

u
(
k
)

0 5 10 15 20 25
-1.5
-1
-0.5
0
0.5
1
1.5
k
©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Fourth Step - Note that state has entered
region in which input is no longer saturated
-2 -1 0 1 2
-1.5
-1
-0.5
0
0.5
1
1.5
x1
x
2

0 5 10 15 20 25
-1.5
-1
-0.5
0
0.5
1
1.5
k
u
(
k
)

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Remainder of the response - Note input
remains unsaturated
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
-1.5
-1
-0.5
0
0.5
1
1.5
x1
x
2

0 5 10 15 20 25
-1.5
-1
-0.5
0
0.5
1
1.5
k
u
(
k
)

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Overlay the anti-windup Solution - Note that
MPC and anti-windup give the same solution
for this particular initial condition
-2 -1 0 1 2
-1.5
-1
-0.5
0
0.5
1
1.5
x1
x
2

0 5 10 15 20 25
-1.5
-1
-0.5
0
0.5
1
1.5
k
u
(
k
)

©
Goodwin, Graebe, Salgado , Prentice Hall 2000
Chapter 22
Observations
1. It is not surprising that anti-windup performs well
in some cases because it can be shown that there
is a non-trivial region of state space in which anti-
windup and MPC are equivalent.
2. However, caution is needed in interpreting this
result. In general, anti-windup is too shortsighted
and MPC will perform better.
3. Some of these issues are explored in the problems
for readers given in the book.