System Identification for MPC

Conflict of Conflux?
B. Erik Ydstie, Carnegie Mellon University

Course Objectives:
1. The McNamara Program for MPC
2. The Feldbaum Program for MPC
3. From Optimal Control to MPC to SYSID for Control and Back
4.Towards Tractable Optimization Formulations
5. State of Art (Challenging – Problem of Integration and Software)

B Erik Ydstie, CMU 1

 System Identification (SYSID) Review
Mass and Energy Balance Constraints (nonlinear)
nM VX
+nDV
dzi
=pi (z) + fi (uj , z), i = 1, ..., n
dt j=1

yk =hk (z), k = 1, ..., nP V

Model Predictive Controller
Linear (output) error model Setpoints y*
Distributed Control System (DCS)

1 Measured
e(t) = y(t) Gp (q )u(t) Control Inputs u
Outputs y

Interface Layer (SCADA)

• Capture Flowsheet structure LT

• Energy and material balances FT

- Collinearity Feed TT CT
FT

- Uncollinearity
FT
Product
Used for very large systems Cooling water return
50 + MV/DVs 100+ CVs
 Prior Information
Step-response
State Space

Data Flow Laguerre,…

Tuning Parameters,…

MPC
Control

ABB
.XML Honeywell
.TXT
Aspen
Process
Emerson

3
B. Erik Ydstie, ILS Inc.
MPC –SYSID
• Boiler Master –Turbine Master Controls (Emerson/Ovation)
• Turbine Controls for Siemens
SYSID Data from

Model estimated using output error identification (global optimality)

B. Erik Ydstie
B. Erik Ydstie
 Prior Information: System Strucure Collinearity:
SVD
• Prior structure of G
G25 RGA
• Digraph (edges in the process network) Angles
• Parametric representation for each Gij (nodes)’
• Information of collinearity structure
• Process Data
• Semi-closed loop G38
• Experiment Design (n 1)n(m 1)m
N (n, m) =
4
= 2970, 7! 0.12 deg separation
Bilinear constraints
400

Digraph 350

10 MV/DVs
Network 300

12 CVs
250

200
I/O Data
150

100

Problem: Define the Operator G that “best” 50

matches the prior information and process data.

0

Bayes Estimation Problem with Constraints

-50
0 1000 2000 3000 4000 5000 6000 7000
 Prior Information: Model Structures

Polynomials used Name

B FIR, SR
A,B ARX, Equation error,
Instrumental variables,..
A,B,C ARMAX, AML
B,F OE (Output Error,
Markov-Laguerre, Kautz,…..)

State space representations have become popular for multivariable

systems after the introduction of sub-space identification.

Halfway Conclusion:
• The components are in place for systematic SYSID
• Software is lacking
• Quite difficult to do due to non-stationary disturbances
• Theory not that easy to understand completely
• Comprehensive Software solutions not available yet.
7
B. Erik Ydstie, ILS Inc.
 The Admissibility Problem

[
rank B | AB | A2 B | × | An-1B = n]
Reachability: Any state can be éC ù
reached in a finite amount of time
êCA ú
ê ú
rank êCA2 ú = n
Observability: Any ê ú
B(q ) state can be ê× ú
y(t ) = u (t ) determined in a finite êCAn -1 ú
A(q) ë û
amount of time

• A(q) and B(q) no common factors = Observable+Controllable (Co-prime)

• A(q) and B(q) no common unstable factors = Detectable+Stabilizable

Detectable: Any unstable state is observable

Stabilizable: Any unstable state is reachable
The FIR / Markov-Laguerre Models are automatically stabilizable 8
MISO Identification 6
Excitation
N MV 1 D 5

16
1 X
MV2 A CV 3
14 u(k)u(k i) 4

12
N
k=1 MV3 A 3
Prior (blue)
10
MV7 is most excited
MV 4 A Update (red)
Data (yellow)
2
8
MV 5 A 1
6
MV 6 A
4 Cond(F) = O(1) 0

2
-1
0

-2 MV2 is least excited -2

-4
0 500 1000 1500 2000 2500 -3
0 1000 2000 3000 4000 5000 6000 7000 8000 9000

Data is persistently excited from a SISO case.

We get (Ljung, Wahlberg, Forsell) Bias and Variance:

Bias

Variance

B. Erik Ydstie, ILS Inc. 9

 Generating Multivariable Input Signals
u1 y1
System
u2 y2

Same results hold as long as PE and independent noise and disturbance sequences.
Results based law of large numbers, difficult to achieve using PRBS type excitation.
N ⇢
1 X R > 0, i = 0
Orthogonal inputs: N
u(k)u(k i) = V (N )T V (N i) = R(N ) =
0, i 6= 0
k=1

Input sequences must be independent in time wrt to the network.

44
[0.2019]
10
42

40 1
8
38
0.5
36
6
34
0
32
4
30
-0.5
10
28
0 1000 2000 3000 4000 5000 6000 7000 2 8 10

 6
4 6
8

Mass balance 0.2210 0.2206 2

2
4

0 0 0

0.2206 0.3301 1 2 3 4 5 6 7 8 9 10
constraints in the
process

B. Erik Ydstie, ILS Inc. 10

 Closed Loop Identification
Process

Closed Loop y(t) = Gc0 (q 1

)u(t) + H0c (q 1
)v(t)
System: Excitation
Gc0 (q 1
) = S0c (q 1
)G0 (q 1
)
H0c (q 1
) = S0c (q 1
)H0 (q 1
)

Issues for closed loop identification: MPC

1
• Model parameterization S0c (q 1 ) = 1 )K(q 1 )
1 + G0 (q
• Algorithm and mathematical approach
• Filters to shape bias and variance
• Excitation (complete theory for SISO, Lacking for MIMO, some progress for Networks
• Extension to multivariable case (treated very superficially in most books and papers)

Methods that may fail:

• Regression type models (equation error, instrumental variables)
• Subspace methods
• Compensation methods (direct and indirect)
• Correlation/spectral methods

Use output error methods for identification (open and closed loop)

B. Erik Ydstie, ILS Inc. 11

 Integration of SYSID with MPC:
The Decision Problem

Model and Desired Performance Objectives

Control/Plan Process Measure

Robert McNamara,1960,
(CEO Ford, US Secretary of State)

1. Defining clear business objectives (Stable/Robust Performance)

2. Developing plans to achieve the objectives (Predictive Control)
3. Systematically monitoring progress against the plan (Feedback, Filter)
4. Adapt objectives/plans as new needs and opportunities arise (Identification)

Repeated Identification -> Iterative Learning -> Adaptive Control

12
 The McNamara Program for MPC
Performance Objectives

Model and Desired Performance Objectives MPC Design/Identify/Adapt

Evaluate
MPC Process Measure
Critic
Current Practice
Predictive Model

1. Measure, evaluate and critique (Gap analysis)

2. Control strategies (Optimal Control/Model Predictive Control/Hinfinity)
3. Identification, Learning, Adaptation
a) Adapt Controllers (directly or indirectly)
b) Adapt Performance Objectives (closed loop, Q,R/move suppression)

• The Decision problem is driven by Uncertainty (more than accurate models)

• Numerous Practical and Theoretical Challenges Remain
• MPC provides a fruitful Paradigm to Study these Challenges 13
 The Feldbaum (1961) Program
1
X
min (y(t + i + 1) y(t + i + 1)⇤ )2 + ru(t + i)2
u2U,y2Y
i=1

• Optimal (Certainty Equivalent, LQ Optimal Control, 1980 to MPC)

• Caution (Robust H1 Control, 1980 ) *
• Probing (System ID / Adaptation, 1980 )

Each field well advanced, but poorly integrated

(Especially on the software side)

* T
X
min x̂(t + i)T Qx̂(t + i) + u(t + i)T Ru(t + i) + x̂(t + i)T P x̂(t + i)
u(t+i) | {z }
i=0
| {z } Terminal Cost
Finite Horizon Cost
Subject to: x̂(t + 1) = Ax̂(t) + Bu(t)
x̂(t + i)min  x̂(t + i)  x̂(t + i)max
u(t + i)min  u(t + i)  u(t + i)max

B Erik Ydstie, CMU 14

 From LQ to MPC and Back Again
Step 1: Formulate a (linear) model
x(t + 1) = Ax(t) + Buf (t) + Ke(t)
ŷ(t) = ✓T x(t) + Duf (t) + e(t)

Step 2: Split Objective in Two and use Predictions from Model

T
X1 1
X
⇤ 2
min (ŷ(t + i + 1) 2
y(t + i + 1) ) + ru(t + i) + (ŷ(t + i + 1) y(t + i + 1)⇤ )2 + ru(t + i)2
u2U,ŷ2Y | {z } | {z }
i=0 i=T
Model Predictive Control LQ Control

Step 3: Ignore last part

Step 4: Solve QP and use first control.
Step 3: Repeat Step 4 (and hope for the best) x(t + T )T PT x(t + T )

Theory for robust stability and performance

B Erik Ydstie, CMU 15

MPC and SYSID:
Learn from the Past and Control Into the Future
Step 2B: Split Objective in Three and use Past Information
N
X
min (ŷ(t i) y(t i)⇤ )2 + (✓ ✓0 )T F0 (✓ ✓0 ) Model Identified from Past Data
✓2⇥ | {z }
i=0
SYSID Bayes
1
X
min (ŷ(t + i + 1) y(t + i + 1)⇤ )2 + ru(t + i)2 Control Into the Future
u2U,ŷ2Y | {z }
i=1
Robust MPC

• Adaptive Control
• Iterative Control
• Closed Loop Identification
• Identification for Control
+++

Basic Idea: Controller works

while data is collected
✓(0) 7! ✓(t1 ) 7! ✓(t2 ), ....
B Erik Ydstie, CMU 16
SYSID and MPC - Conflict or Conflux?
Adapted from Polderman (1986)
Definition: An Identification Based Control is said to be Self-Tuning if SYSID
gives the “correct control”
Set of Identified Models : G
Set of Parameters with correct controls : H
Control and Estimation are Self Tuningif : H✓G

Exampe: LTI System: y(t) = ay(t 1) + bu(t 1)

e(t) = y(t) ŷ(t)
y
u
System
+
Model
-

Linear feedback: u(t) = K(â, b̂)y(t)

Question: Will system satisfy performance specifications when the same

control is applied to both systems?

(The question of (Roust Lagrange) stability for closed loop identification and control was addressed by 1995)
Analysis: Assume model output matches plant output

G = {â, b̂ : ay(t 1) bK(✓)x(t) = ây(t 1) b̂K(✓)x(t)}

| {z } | {z }
y(t) ŷ(t)

An infinite number of solutions. These depend on K.

Example 1: One step ahead predictive control

Solve for u(t) : y(t + 1)⇤ = ay(t) + bu(t)
⇢
â a â
u(t) = y(t) G= â, b̂ : =
1 b̂ b b̂
The admissible set
0.5

Get correct control even if

0
parameter estimates are off.
b

-0.5
Thm: Any identifier that minimizes
-1
-1 -0.5 0 0.5 1
prediction error is self tuning when used
a
with minimum variance control.
Admissibility Problem (close to singularity gives large, oscillatory controls)
(Problem of “direction”) B Erik Ydstie, CMU 18
 ⇢
The set that gives a â
H= â, b̂ : y(t)) = y(t) H✓G
correct controls b b̂

Example 2: Pole placement control (Vogel and Edgar,

Find gains so that : y(t) = a0 y(t 1), 0 < a0 < 1

⇢
a a0 â a0
â a0 H= â, b̂ : y(t)) = y(t)
u(t) = y(t) b b̂
b̂
4
H✓G
3
Thm: Any identifier that minimizes
2
prediction error is self tuning when used
1
Admissible set with pole-placement.
b

0
In this case Admissibility is more Complex as we
-1
require:
-2
Observability and Controllability
-3
-1 -0.5 0 0.5 1
a Can be expressed as Bilinear Constraints in
SYSID problem.
B Erik Ydstie, CMU It is going well so far!! 19
 Example 3: Model Predictive Control
min (ŷ(t + 1)2 + ru(t)2 ) + py(t + 1)2
u
( )
ba ˆ
ba
u(t) = G= â, b̂ : a â = (b b̂)
r + b2 r + b̂2
⇢
5 a a0 â a0
H= â, b̂ : y(t)) = y(t)
H b b̂
H 6✓ G
unless :
0
G
r = 0 and/or
H â = a, b̂ = b

-5
-1 -0.5 0 0.5 1

Thm: MPC does NOT satisfy the self-tuning principle.

B Erik Ydstie, CMU 20

Problem: Information in the Past is Not Connected to Future Information

Additional means are needed to get optimal controls for MPC.

• Persistent Excitation to Converge Controls
• More Complex Controls to Align sets G and H?
• “Intelligent” Excitation (SYSID for Control, Dual Control)

B Erik Ydstie, CMU 21

 From Feldbaum to MPC and Back
T
X 1
X
⇤ 2
min (y(t + i) y(t + i) ) + ru(t + i) + 2
(y(t + i) y(t + i)⇤ )2 + ru(t + i)2
u2U,y2Y
i=1 i=T +1
| {z } | {z }
Model Predictive Control LQ Control

T
X
min (ŷ(t + i) y(t + i)⇤ )2 + ru(t + i)2 + x(t + i)T F (t + i) 1
x(t + i) +x(t)PT x(t)
u2U,ŷ2Y | {z } | {z }
i=1
Robust CE AMPC Information Gathering

ŷ(t + i) Maximum Likelihood Estimate

F (t + i) Fisher Information Matrix
Challenges (identification using past data is the easiest):
• Solve a robust control problem on line (structured and unstructured uncertainty)
• “Back out control signals” from forward propagation of Fisher matrix
• What to do with the arrival cost

B Erik Ydstie, CMU 22

 Special Case (TA Heirung/ J Morinelly)

• Fix the transition matrix A (step-response/Kautz model)

• Solve CE (rather than robust Hinfinity) control problem (caution
related to parameter uncertainty only)
• Ignore arrival cost

Computationally Expensive and Untested

 So What are the Issues?
Data Rich – But Information Poor Systems
(Nature is not a kind adversary)

• MPC and SYSID - Conflict or Conflux?

• How to represent/parametrize the system
• How to excite the system
• How to manage changing models
• Directionality
• Complexity
• Software

24
 Conclusions
• MPC is not self tuning
• There is a “strong" inter-action between
control and identification
• Different ways to “solve the problem”
– More complex “control”
– External Excitation (setpoints/inputs)
– Identification for control
• Need to Retune Controller when model
changes
• Collinearity issue is not well understood
• Very Large scale Applications are challenging
• MPC Maintenance is still challenging

25
 Golden Opportunities
15 lines Pitt
of MATLAB CMU
4756 lines code (2014)
of assembly
code (1983)