# A Lecture on Model Predictive Control

Jay H. Lee
School of Chemical and Biomolecular Engineering Center for Process Systems Engineering Georgia Inst. of Technology

Prepared for Pan American Advanced Studies Institute Program on Process Systems Engineering

Schedule

•Lecture 1: Introduction to MPC •Lecture 2: Details of MPC Algorithm and Theory •Lecture 3: Linear Model Identification

Lecture 1
Introduction to MPC
- Motivation - History and status of industrial use of MPC - Overview of commercial packages

ui ) ui p g i ( xi . ui ) i =0 ? u0 = µ ( x0 ) HJB Eqn. Set x0 = xk (Estimated Current State) Solve the optimization problem numerically Implement solution u0 as the current move. ui ) ≥ 0 xi +1 = F ( xi . Repeat! .Key Elements of MPC • Formulation of the control problem as an (deterministic) optimization problem min ∑ φi ( xi . • • On-line optimization Receding horizon implementation (with feedback update) ˆ At t = k .

Popularity of Quadratic Objective in Control • Quadratic objective Linear State Space System Model ∑x Qx + ∑u Ru i =0 T i i i =0 T i p m−1 i xk +1 = Axk + Buk yk = Cxk • Fairly general – State regulation – Output regulation – Setpoint tracking • • • Unconstrained linear least squares problem has an analytical solution. (Kalman’s LQR) Solution is smooth with respect to the parameters Presence of inequality constraints → no analytical solution .

Heuristics PID Controllers Lead / Lag Filters Switches Min. Max Selectors If / Then Logics Sequence Logics • Regulation • Constraint handling • Local optimization • Model is not explicitly used inside the control algorithm • No clearly stated objective and constraints Other Elements • Inconsistent performance • Complex control structure • Not robust to changes and failures • Focus on the performance of a local unit .Classical Process Control t ⎛ 1 de ⎞ ⎜1 + ∫ e(t ' )dt '+τ D ⎟ p = kc ⎜ dt ⎟ ⎝ τI 0 ⎠ Ad Hoc Strategies.

Example 1: Blending System • Control rA and rB • Control q if possible • Flowrates of additives are limited Classical Solution .

u ) Controller Plant Set-point Input Output r(t) u(t) y(t) Open-Loop Optimal Control Problem ⎫ ⎧ p −1 φ ( x i . y = g ( x . u ) Path constraints Terminal constraints Model constraints . u p −1 ⎩ i = 0 ⎭ stage-wise cost terminal cost g i ( xi .Model-Based Optimal Control & x = f ( x . ui ) ≥ 0 g p (xp ) ≥ 0 & x = f ( x. u i ) + φ p (x p )⎬ min ⎨ ∑ u 0 .K . u ).

esp. stage-wise terminal • Analytical solution possible only in a few cases (LQ control) cost cost Open-Loop Optimal Control Problem g i(xi.u ) ≥ 0 • Computational i limitation for numerical solution.Model-Based Optimal Control & x = f ( x .K .u ) Model constraints . back in Path constraints p ( x p ) ≥ the ’50sgand ’60s 0 Terminal constraints & x = f (x. u ) Plant Controller Set-point Input Output r(t) u(t) Measurements y(t) • Open-loop optimal solution is not robust • Must be coupled with− on-line state / model parameter update ⎫ ⎧ p 1 φ ( x i . y = g ( x . u i ) + φ p (x p )⎬ ⎨ • Requiresuminu p − 1 solution for each updated problem on-line ⎩ ∑0 i= ⎭ 0 . u ).

Model Predictive Control (Receding Horizon Control) – At time k. solve the open-loop optimal control problem on-line with x0 = x(k) – Apply the optimal input moves u(k) = u0 – Obtain new measurements. update the state and solve the OLOCP at time k+1 with x0 = x(k+1) – Continue this at each sample time Implicitly defines the feedback law u(k) = h(x(k)) .

Analogy to Chess Playing my move his move my move Opponent (The Plant) My Move I (The Controller) The Opponent’s Move New State .

Conventional Structure Plant-Wide Optimization Unit 2 .Operational Hierarchy Before and After MPC Unit 1 .MPC Structure Global Steady-State Optimization (every day) Local Steady-State Optimization (every hour) Dynamic Constraint Control (every minute) Supervisory Dynamic Control (every minute) Basic Dynamic Control (every second) LC Unit 1 Local Optimization Unit 2 Local Optimization High/Low Select Logic PID Lead/Lag PID Model Predictive Control (MPC) SUM SUM Unit 1 Distributed Control System (PID) Unit 2 Distributed Control System (PID) FC PC TC LC FC PC TC .

Example: Blending System • Control rA and rB • Control q if possible • Flowrates of additives are limited Classical Solution p = Size of prediction window MPC: Solve at each time k p (rA (k + i | k ) − rA *)2 u ( j ).u ( j ).u ( j ) ∑ min 2 1 3 j = k . γ << 1 + (rB (k + i | k ) − rB *) 2 2 . k + p −1 i =1 (ui ) min + γ (q(k + i | k ) − q *) ≤ ui ( j ) ≤ (ui ) max . i = 1. L.L.3.

Optimization and Control .

An Exemplary Application (1) .

An Exemplary Application (2) .

Profimatics – Today’s Players: Aspen Technology. Many vendors specializing in the technology – Early Players: DMCC. Invensys.Industrial Use of MPC • • • • • Initiated at Shell Oil and other refineries during late 70s. Technology is used not only for multivariable control but for most economic operation within constraint boundaries. Chemical and pulp and paper are the next areas. Honeywell. . Setpoint. Majority of applications (67%) are in refining and petrochemicals. >4600 worldwide installations + unknown # of “in-house” installations (Result of a survey in yr 1999). ABB • • Models used are predominantly empirical models developed through plant testing. The most applied advanced control technique in the process industries.

.MPC Industry Consolidation CPC-V (late 1995) Adersa Setpoint CCI DMCC DMCC DOT Products Honeywell Honeywell Litwin (Profimatics) Neuralware Pavilion Predictive Controls Profimatics Litwin Setpoint Treiber Control MDC Technology Aspentech Neuralware Treiber Update (Late 2000) Adersa ABB DOT CCI GE Pavilion SimSci Foxboro Invensys PCL FRSI MDC Emerson Elec.

Linear MPC Vendors and Packages • Aspentech – DMCplus – DMCplus-Model • • Honeywell – Robust MPC Technology (RMPCT) Adersa – Predictive Functional Control (PFC) – Hierarchical Constraint Control (HIECON) – GLIDE (Identification package) • MDC Technology (Emerson) – SMOC (licensed from Shell) – Delta V Predict • • Predictive Control Limited (Invensys) – Connoisseur ABB – 3d MPC .

Result of a Survey in 1999 (Qin and Badgwell) .

Nonlinear MPC Vendors and Packages • Adersa – Predictive Functional Control (PFC) • Aspen Technology – Aspen Target • Continental Controls • DOT Products xk+1 = Axk + Buuk + Bv v k yk = g(xk ) = Cxk + NN(xk ) – Multivariable Control (MVC): Linear Dynamics + Static Nonlinearity – NOVA Nonlinear Controller (NLC): First Principles Model • Pavilion Technologies – Process Perfecter: Linear Dynamics + Static Nonlinearity .

Results of a Survey in 1999 for Nonlinear MPC .

Determine the relevant CV’s.Controller Design and Tuning Procedure 1. and DV’s 2. Derive a dynamic model from the plant test data 4. Configure the MPC controller and enter initial tuning parameters 5. Commission the controller and refine the tuning as needed . Conduct plant test: Vary MV’s and DV’s & record the response of CV’s 3. Test the controller off-line using closed loop simulation 6. Download the configured controller to the destination machine and test the model predictions in open-loop mode 7. MV’s.

Role of MPC in the Operational Hierarchy Determine plant-wide the optimal operating condition for the day Make fine adjustments for local units Take each local unit to the optimal condition fast but smoothly without violating constraints Multivariable Control Plant-Wide Optimization Local Optimization MPC Distributed Control System (PID) FC PC TC LC .

RMPCT introduced a dynamic optimizer recently • Linear Program (LP) for SS optimization. CV’s are ranked in priority so that SS control performance of a given CV will never be sacrificed to improve performance of lower priority CV’s.Local Optimization • A separate steady-state optimization to determine steady-state targets for the inputs and outputs. MV’s are also ranked in priority order to determine how extra degrees of freedom is used . the LP is used to enforce input and output constraints and determine optimal input and output targets for the thin and fat plant cases • The RMPCT and PFC controllers allow for both linear and quadratic terms in the SS optimization • The DMCplus controller solves a sequence of separate QPs to determine optimal input and output targets.

an input penalty. and an input rate penalty: J = ∑ j =1 e P y k+ j 2 Qj + ∑ j =0 ∆uk + j M −1 2 Sj + ∑ j =0 e M −1 u k+ j 2 Rj A vector of inputs uM is found which minimizes J subject to constraints on the inputs and outputs: u = u . an output penalty.. all of the controllers can be described (approximately) as minimizing a performance index with up to three terms..u M T 0 T 1 ( T T M−1 ) u ≤ uk ≤ u ∆u ≤ ∆u k ≤ ∆ u y ≤ yk ≤ y x k +1 = f (x k . u . u k ) y k +1 = g ( x k +1 ) + b k +1 ..Dynamic Optimization At the dynamic optimization stage.

u minM J = r ∑ P j =1 y r k+ j − yk+ j 2 Q + u M −1 − u ss 2 S subject to a funnel constraint . CV errors are minimized first. followed by MV errors • Connoisseur allows for a multi-model approach and an adaptive approach • The RMPCT algorithm defines a funnel and finds the optimal trajectory yr and input uM which minimize the following objective: y k + j .Dynamic Optimization • Most control algorithms use a single quadratic objective • The HIECON algorithm uses a sequence of separate dynamic optimizations to resolve conflicting control objectives.

Adersa’s quadratic penalty Setpoint past future past future Soft Constraint.Output Trajectories Aspen Tech’s DMC ID-COM. “Zone Control” quadratic penalty Reference trajectory Honeywell’s RMPCT quadratic penalty Zone past future past future Funnel quadratic penalty • Move suppression is necessary when reference trajectory is not used .

Output Horizon Finite horizon past future prediction horizon P Coincidence points .

Input Parameterization u Multiple moves (with blocking) control horizon u Single move (extreme blocking) u Basis function (parametrized) .

U S.Process Model Types Model Type Differential Equations State-Space Origin Linear/Nonlinear Stable/Unstable S.U .NL data S.NL physics L.U Laplace Transfer physics L Function data ARMAX/NARMAX data Convolution data (Finite Impulse or Step Response) Other (Polynomial.NL S.U physics L.NL L S.U S L. Neural Net) data L.

es non-PRBS signals • Most products use FIR.Identification Technology • Most products use PRBS-like or multiple steps test signals. ARX or step response models – Glide uses transfer function G(s) – RMPCT uses Box-Jenkins – SMOC uses state space models Glide us • Most products use least squares type parameter estimation: – prediction error or output error methods – RMPCT uses prediction error method – Glide uses a global method to estimate uncertainty • Connoisseur has adaptive capability using RLS • A few products (DMCplus. SMOC) have subspace identification metho ds available for MIMO identification • Most products have uncertainty estimate. but most products do not m ake use of the uncertainty bound in control design .

Summary • MPC is a mature technology! – Many commercial vendors with packages differing in model form. etc. preferably with some guaranteed properties – Better management of “uncertainty” • creating models with uncertainty information (e.g.. – Sound theory and experience • Challenges are – Simplifying the model development process • plant testing & system identification • nonlinear model development – State Estimation • Lack of sensors for key variables – Reducing computational complexity • approximate solutions. objective function form. stochastic model) • on-line estimation of parameters / states • “robust” solution of optimization .

FCCU Debutanizer ~20% under capacity .

TC 400 F RVP Slurry Pump Around Gasoline to blending .Debutanizer Diagram PC 190 lb Pressure PCT Pre-Heater 160 F Fan Reflux To Deethanizer From Stripper Feed TC TC Flooding Tray 20 Temp.

over design capacity. Consequences: • High RVP.usually jet flooding. giving away Octane Number • High OVHD C5. • Flooding -.Process Limitation Operation Problems: • Overloading -.especially in summer. • Lack of Overhead Fan Cooling -. causing problems at Alky. . causing very poor separation.

• Rejecting disturbance not through slurry.Control Objectives Constrained Control: • Preventing safety valve from relieving • Keep the tower from flooding • Keep RVP lower than its target. if possible. . Regulatory Control: • Regulate OVHD PCT or C5 at spec.

Real-Time Optimization Optimization Objectives: While maintaining PCT. RVP on their specifications • Minimizing energy consumed • Minimizing overhead reflux • Minimizing overhead cooling required • Minimizing overhead pressure • Maximizing separation efficiency. .

. OVHD C5 % OVHD Pressure R-PID Internal Reflux Manipulated Fan Output Reflux Pre-heater By-Pass Reboiler By-Pass MPC R-PID Pre-heater By-Pass Reboiler By-Pass Disturbances Feed Stripper Bottom Temp.MPC Configuration Controlled PCT Fan RVP Diff. Pressure Feed Temp. Tray 20 Temp.

MPC’s MV Moves Optimal Point Reached 200 180 160 140 120 100 80 60 40 20 0 0 Pressure (lb) Feed-heater by-pass Reboiler by-pass Reflux 500 1000 1500 2000 2500 3000 minute .

Feed 32 30 28 26 24 22 20 18 16 14 12 0 500 1000 1500 2000 2500 3000 Feed An indication that separation efficiency increased.Reflux vs. Reflux minute .

Reflux-to-Feed Ratio

0.54

0.52

0.5

0.48

Handling Flood
0.46 0.44

0.42

0

500

1000

1500

2000

2500

3000

minute

Handling Flood

3.2 3 2.8 2.6

Potential Flooding

Differential Pressure Across Tower

2.4 2.2 2 1.8 1.6 1.4 1.2 0 500 1000 1500 2000 2500 3000

minute

Product Spec's

180 160 140 120 100 80 60 40 20 0 0 500

Top pressure compensated temperature (PCT)

Bottom Reid Vapor Pressure (RVP)
1000 1500 2000 2500 3000

minute

Tray 20 After MPC is on 400 F RVP Temperature Gasoline to blending Tray # .MPC Operation Overview 190 lb Pressure PCT Pre-Heater 160 F Fan Top Tray Reflux Feed Tray Feed Flooding Tray 20 Temp.

1991 Date Debutanizer Bottom C4.5 4 3. N-C4 & C4=-) .5 0 1 2 3 4 5 6 7 8 9 10 MPC When RMPCT was turned on 11 12 13 14 15 16 17 April.MPC Control Results (1) C4.Total 5 % 4.5 3 2.5 2 1.Total (I-C4.5 1 0.

5 6 5.MPC Control Results (2) RVP 7.5 7 When RMPCT was turned on MPC 6. 1991 Date Debutanizer Bottom RVP Daily Average .5 5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 April.

N-C5 & C5=+) .MPC Control Results (3) C5+ Total 20 % 18 16 14 12 10 8 6 4 2 1 2 3 4 5 6 7 8 9 10 MPC When RMPCT was turned on 11 12 13 14 15 16 17 April. 1991 Date Debutanizer Top C5+ Total (I-C5.

Note: Variance on the tray-20 temperature is increased.Other Benefits • Reflux is 15% lower than before • Separation efficiency is increased • Now have room for lower RVP or PCT. if needed • Variance on PCT and RVP is reduced. and it should be!! • Energy saving (~10%) • OVHD fan maximum bound may be avoided • Flooding is eliminated. .

UT.Acknowledgment • Professor S. Austin • Dr. Joseph Lu. Joe Qin. AZ • Center for Process Systems Engineering Industrial Members . Honeywell. Phoenix.

Use of state estimation .Optimization .Infinite-horizon MPC and stability .Use of nonlinear models .Lecture 2 Details of MPC Algorithms and Theory .Impulse and step response models and the prediction equation .

Important Ingredients of an MPC Algorithm • Dynamic Model → Prediction Model – Predicted future outputs = Function of current “state” (stored in memory) + feedforward measurement + feedback measurement correction + future input adjustments • Objective and Constraints • Optimization Algorithm • Receding Horizon Implementation .

General Setup Dynamic Model State: Compact representation Of the past input record Previous State (in Memory) State Update New Input Move (Just Implemented) Current State Prediction Future Input Moves (To Be Determined) Prediction Model for Future Outputs ? Measurement Correction Feedback / Feedforward Measurements To Optimization .

Options • Model Types – Finite Impulse Response Model or Step Response Model – State-Space Model – Linear or Nonlinear • Measurement Correction – To the prediction (based on open-loop state calculation) – To the state (through state estimation) • Objective Function – Linear or Quadratic – Constrained or Unconstrained .

Prediction Model for Different Model Types • Finite Impulse Response Model • Step Response Model • State-Space Model .

y2 .L} v can be a MV (u) or a Measured DV (d) {v0 .L} . v2 .Sample-Data (Computer) Control Model relates input samples to output samples ↓ {y0 . v1 . y1 .

t.Finite Impulse Response Model (1) Assumptions: • H0 = 0: no immediate effect • The response settles back in n steps s. Hn+1 = Hn+2 = … = 0: “Finite Impulse Response” (reasonable for stable processes) .

Finite Impulse Response Model (2) Linear Model→ “Superposition Principle” y (k ) = H1v(k − 1) + L + H n v(k − n) .

Finite Impulse Response Model (3) • State x( k ) = ⎡ v ( k − 1).L . v ( k − n) ⎤ ⎣ ⎦ T T T n past input samples (includes both MVs and measured DVs) • State Update: Easy! x(k + 1) = M 0 x(k ) + Τ v(k ) { { insertion shift ⎡ v(k ) ⎤ ⎢ v(k − 1) ⎥ ⎢ ⎥ M ⎢ ⎥ ⎢v(k − n + 2)⎥ ⎢ v(k − n + 1) ⎥ ⎣ ⎦ insert ⎡ v(k − 1) ⎤ ⎢ v(k − 2) ⎥ ⎢ ⎥ M ⎢ ⎥ ⎢v(k − n + 1)⎥ ⎢ v ( k − n) ⎥ ⎣ ⎦ drop .

=d(k+p-1)) .Finite Impulse Response Model (4) • Prediction Model prediction of y Model prediction error y (k ) = H1v(k − 1) + L + H n v(k − n) = Cx (k ) Model error+Unmeasured disturbance e( k ) = y m ( k ) − y ( k ) Measured output Future input samples (to be decided) Predicted future output samples Past input samples stored in memory ⎡ yk +1|k ⎤ ⎡ u (k − 1) ⎤ ⎡ u (k ) ⎤ ⎢ M ⎥ = Ψu ⎢ M ⎥ + Ψu ⎢ ⎥ M 1 2 Dynamic ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ Matrices ⎢ yk + p|k ⎥ ⎣u (k − n)⎦ ⎣u (k + p − 1)⎦ ⎣ ⎦ (made of ⎡ d (k − 1) ⎤ ⎡ d ( k ) ⎤ ⎡e( k ) ⎤ impulse ⎥+⎢ M ⎥ response Ψ1d ⎢ M ⎥ + Ψ2d ⎢ M coefficients) ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ d ( k − n) ⎦ ⎣d (k + p − 1)⎦ ⎣e(k )⎦ Feedforward term: Past disturbance samples stored in memory Feedback Error Correction Feedforward term: new measurement (Assume d(k)=d(k=1)=….

t.Step Response Model (1) Assumptions: • S0 = 0: no immediate effect • The response settles in n steps s. Sn= Sn+1 = …= S∞: the same as the finite impulse response assumption • Relationship with the impulse response coefficients: .

L . yn −1T (k ) ⎤ ⎣ ⎦ T n future outputs assuming the input remains constant at the most recent value yi (k ) = y (k + i ) w/ ∆u (k ) = ∆u (k + 1) = L = 0 x(k) Note yn −1 (k ) = yn (k ) = L = y∞ (k ) Also y (k ) = y0 (k ) .Step Response Model (2) •State x(k ) = ⎡ y0T (k ).

Pictorial Representation of the State Update x(k+1) M1 x(k) .

Step Response Model (3) • State Update x(k + 1) = M 1 x(k ) + { shift step response S { ∆v(k ) drop ⎡ y0 (k + 1) ⎤ ⎢ y (k + 1) ⎥ 1 ⎢ ⎥ M ⎢ ⎥ ⎢ yn − 2 (k + 1)⎥ ⎢ y (k + 1) ⎥ ⎣ n −1 ⎦ S1 S2 Sn-1 Sn ⎡ y0 ( k ) ⎤ ⎢ y (k ) ⎥ 1 ⎢ ⎥ ⎢ M ⎥ ⎢ yn − 2 (k )⎥ ⎢ y (k ) ⎥ ⎣ n −1 ⎦ .

Step Response Model (4) •Prediction x(k) Effect of future input moves (to be determined) .

0.0]x(k ) Future input moves (to be decided) Dynamic Matrices (made of step response coefficients) ⎡ yk +1|k ⎤ ⎡ y1 (k ) ⎤ ⎡ ∆u (k ) ⎤ ⎢ ⎥ ⎢ ⎥ ⎥ M ⎥ = ⎢ M ⎥ + Ωu ⎢ M ⎢ ⎢ ⎥ ⎢ y k + p| k ⎥ ⎢ y p ( k ) ⎥ ⎢∆u (k + p − 1)⎥ ⎣ ⎦ ⎦ ⎣ ⎦ ⎣ ⎡ ∆d (k ) ⎤ ⎡e(k )⎤ ⎥+⎢ M ⎥ Ωd ⎢ M ⎢ ⎥ ⎢ ⎥ ⎢∆d (k + p − 1)⎥ ⎢e(k )⎥ ⎣ ⎦ ⎣ ⎦ Feedforward term: new measurement (Assume ∆d(k+1)=…= ∆ d(k+p-1)=0) Feedback Error Correction .Step Response Model (4) • Prediction Model prediction of y(k) Model prediction error Predicted future output samples e( k ) = y m ( k ) − y ( k ) The “state” stored in memory y (k ) = y0 (k ) = [1. L.

L N } Test Data . i = 1.State-Space Model (1) z (k + 1) = Az (k ) + B u u (k ) + B d d (k ) y (k ) = Cz (k ) Discretization State-Space Realization ~ ~ ~ & z = Az + B u u + B d d ~ y = Cz Linearization y ( z ) = G u ( z )u ( z ) + G d ( z )d ( z ) or y ( s ) = G ( s )u ( s ) + G d ( s )d ( s ) I/O model Identification State-Space Identification & z = f ( z. u. d ) y = g ( z) Fundamental Model {y(i). u (i).

State-Space Model (2) ⎛ z (k + 1) = Az (k ) + B u u (k ) + B d d (k ) ⎞ ⎜ ⎟ ⎜ y (k + 1) = Cz (k + 1) ⎟ ⎝ ⎠ ⎛ z (k ) = Az (k − 1) + B u u (k − 1) + B d d (k − 1) ⎞ ⎜ ⎟ ⎜ y (k ) = Cz (k ) ⎟ ⎝ ⎠ ∆z (k + 1) = A∆z (k ) + B u ∆u (k ) + B d ∆d (k ) - ∆y (k + 1) = C∆z (k + 1) → y (k + 1) = y (k ) + C A∆z (k ) + B u ∆u (k ) + B d ∆d (k ) ( ) ⎡ Bd ⎤ ⎡∆z (k + 1)⎤ ⎡ A 0⎤ ⎡∆z (k )⎤ ⎡ B u ⎤ ⎢ y (k + 1) ⎥ = ⎢CA I ⎥ ⎢ y (k ) ⎥ + ⎢ u ⎥ ∆u (k ) + ⎢ d ⎥ ∆d (k ) ⎦ ⎣CB ⎦ ⎦⎣ ⎦ ⎣ ⎣ ⎣CB ⎦ State Update ⎡∆z (k )⎤ y (k ) = [0 I ]⎢ ⎥ x(k + 1) = Φx(k ) + Γ u ∆u (k ) + Γ d ∆d (k ) ⎣ y (k ) ⎦ y ( k ) = Ξx ( k ) .

State-Space Model (3) • Prediction Model prediction of y(k) Model prediction error Predicted future output samples y (k ) = Ξ x(k ) e( k ) = y m ( k ) − y ( k ) The “state” stored in “memory” Future input moves (to be decided) Dynamic Matrix (made of step response coefficients) ⎡ yk +1|k ⎤ ⎡ΞΦ ⎤ ⎡ ∆u (k ) ⎤ ⎢ ⎥ ⎢ ⎥ ⎥ x(k ) + Ωu ⎢ M ⎥ = ⎢M M ⎢ ⎥ ⎢ ⎥ ⎢ yk + p|k ⎥ ⎢ΞΦ p ⎥ ⎢∆u (k + p − 1)⎥ ⎣ ⎦ ⎦ ⎣ ⎦ ⎣ ⎡ ∆d (k ) ⎤ ⎡e(k )⎤ ⎥+⎢ M ⎥ Ωd ⎢ M ⎢ ⎥ ⎢ ⎥ ⎢∆d (k + p − 1)⎥ ⎢e(k )⎥ ⎣ ⎦ ⎣ ⎦ Feedforward term: new measurement (Assume ∆d(k+1)=…= ∆ d(k+p-1)=0) Feedback Error Correction .

one gets the prediction equation in the form of ⎡ yk +1|k ⎤ ⎡ ∆u (k ) ⎤ ⎢ ⎥ ⎥ M ⎥ = Lx x(k ) + Ld ∆d (k ) + Le e(k ) + Lu ⎢ M ⎢ ⎢ ⎥ 14444 4444 2 3 known ≡ b ( k ) ⎢ y k + p| k ⎥ ⎢∆u (k + p − 1)⎥ ⎣ 4 ⎣4 4⎦ 14 244 ⎦ 3 123 Y (k ) ∆U ( k ) • Assumptions – Measured DV (d) remains constant at the current value of d(k) – Model prediction error (e) remains constant at the current value of e(k) .Summary • Regardless of model form.

Ramp Type Extrapolation • For Integrating Processes. Slow Dynamics e(k + i ) = e(k ) + i (e(k ) − e(k − 1)) ∆d (k ) = ∆d (k + 1) = L = ∆d (k + p − 1) e(k)-e(k-1) .

Use of State Estimation .

Measurement Correction of State Previous State Estimate (in “Memory”) State Update New Input Move (Just Implemented) Current State Estimate Prediction Future Input Moves (To Be Determined) Prediction Model for Future Outputs Measurement Correction Feedback / Feedforward Measurements State Estimate: Compact representation Of the past input and measurement record To Optimization .

g..State Update Equation x(k + 1) = Φx(k ) + Γ ∆u (k ) + Γ ∆d (k ) + K ( ym (k ) − Ξx(k )) u d • K is the update gain matrix that can be found in various ways – Pole placement: Not so effective with systems with many states (most chemical processes) – Kalman filtering: Requires a stochastic model of form x(k + 1) = Φx(k ) + Γ u ∆u (k ) + Γ d ∆d (k ) + w(k ) y ( k ) = Ξx ( k ) + ν ( k ) Can be obtained using. subspace ID White noises of known covariances Effect of unmeasured disturbances and noise . e.

Prediction Equation Contains past feedback measurement corrections ⎡ yk +1|k ⎤ ⎡ΞΦ ⎤ ⎡ ∆u (k ) ⎤ ⎢ ⎥ ⎢ ⎥ ⎥ u⎢ M ⎢ M ⎥ = ⎢M ⎥ x(k ) + Ω ⎢ ⎥ ⎢ yk + p|k ⎥ ⎢ΞΦ p ⎥ ⎢∆u (k + p − 1)⎥ ⎣ ⎦ ⎦ ⎣ ⎦ ⎣ ⎡ ∆d (k ) ⎤ ⎡e(k )⎤ ⎥+⎢ M ⎥ d⎢ Ω ⎢ M ⎥ ⎢ ⎥ ⎢∆d (k + p − 1)⎥ ⎢e(k )⎥ ⎣ ⎦ ⎣ ⎦ Additional measurement correction NOT needed here! .

What Are the Potential Advantages? • Can handle unstable processes – Integrating processes. Unmeasured inputs Measured inputs Early. run-away processes • Cross-channel measurement update – More effective update of output channels with delays or measurement problems based on other channels. Measurement Difficulties. robust update through modeled correlation y1 Process Delays. Slow Sampling y2 • Systematic handling of multi-rate measurements • Optimal extrapolation of output error and filtering of noise (based on the given stochastic system model) .

Optimization .

Objective Function • Minimization Function: Quadratic cost (as in DMC) p m −1 i =0 V (k ) = ∑ ( yk +i|k − y*) Λ ( yk +i|k − y*) + ∑ ∆u T (k + i )Λu ∆u (k + i ) T y i =1 – Consider only m input moves by assuming ∆u(k+j)=0 for j≥m – Penalize the tracking error as well as the magnitudes of adjustments • Use the prediction equation. First m columns of Lu T V (k ) = (Y (k ) − Y *) diag(Λy )(Y (k ) − Y *) + ∆U m (k )diag(Λu )∆U m (k ) T Substitute T m Y (k ) = b(k ) + Lu ∆U m (k ) m T V (k ) = ∆U (k ) H∆U m (k ) + g (k )∆U m (k ) + c(k ) constant .

Constraints

Substitute the prediction equation and rearrange to

C∆U m (k ) ≥ h(k )

Optimization Problem

∆U m ( k )

min ∆U (k ) H∆U m (k ) + g (k )∆U m (k )
T m T

such that C∆U m (k ) ≥ h(k )
• Unconstrained Solution

1 −1 ∆U m (k ) = − H g (k ) 2
• Constrained Solution
– Must be solved numerically

• Minimization of a quadratic function subject to linear constraints • Convex and therefore fundamentally tractable • Solution methods
– Active set method: Determination of the active set of constraints on the basis of the KKT condition – Interior point method: Use of barrier function to “trap” the solution inside the feasible region, Newton iteration

• Solvers
– Off-the-shelf software, e.g., QPSOL – Customization is desirable for large-scale problems

Two-Level Optimization
Steady-State Optimization (Linear Program) min L( y∞|k , u s (k ))
us ( k )

⎡ y ∞| k ⎤ Cs ⎢ ⎥ ≥ cs ( k ) ⎣u s (k )⎦ u s (k ) = u (k − 1) + ∆u (k ) + L + ∆u (k + m − 1) y∞|k = bs (k ) + Ls ∆u s (k )
Optimal Setting Values (setpoints)
Steady-State Prediction Eqn. State Feedforward Measurement Feedback Error Dynamic Prediction Eqn.

y , u (k )
* ∞| k * s

(sometimes input deviations are included in the quadratic objective function)

Use of Infinite Prediction Horizon and Stability

Use of ∞ Prediction Horizon – Why? • Stability guarantee – The optimal cost function can be shown to be the control Lyapunov function • Less parameters to tune • More consistent. e. LQG control . intuitive effect of weight parameters • Close connection with the classical optimal control methods.g..

Step Response Model Case V (k ) = ∑ ( yk +i|k − y*) Λ ( yk +i|k − y*) + ∑ ∆u T (k + i )Λu ∆u (k + i ) T y i =1 i =0 ∞ m −1 V (k ) = m + n −1 i =1 ( yk +i|k − y*) Λ ( yk +i|k − y*) + ∑ ∆u T (k + i )Λu ∆u (k + i ) ∑ T y i =0 m −1 with extra constraint yk + m + n −1|k = y * Must be at y* for the cost to be bounded .

Additional Comments • Previously. we assumed finite settling time. • Can be generalized to state-space models – More complicated procedure to turn the ∞-horizon problem into a finite horizon problem – Requires solving Lyapunov equation to get the terminal cost matrix – Also. must make sure that output constraints will be satisfied beyond the finite horizon → construction of output admissible set • Use of a sufficiently large horizon (p≈ m+ the settling time) should have a similar effect • Can we always satisfy the settling constraint? – y=y* may not be feasible due to input constraints or insufficient m → use two-level approach .

u (k ) * ∞| k * s Dynamic Optimization (∞-horizon MPC) * Constraint yk + m + n −1|k = y∞|k is guaranteed to be feasible. * * Constraint ∆u(k) + L + ∆u(k + m .1 ) = ∆us → yk + m + n-1|k = y∞|k .Two-Level Optimization Steady-State Optimization (Linear Program or Quadratic Program) Optimal Setting Values (setpoints) y .

Use of Nonlinear Model .

d (k )). u(k). u . u (k ).Difficulty (1) & x = f ( x. u (k ). d (k )) y (k ) = g ( x(k )) Orthogonal Collocation yk + 2|k = g o F (F ( x(k ). u (k ).t. u (k ). d (k ) ) + e(k ) M yk + p|k = g o F p ( x(k ). ……. d (k ) ) + e(k ) yk +1|k = g o F ( x(k ). d ) y = g ( x) Discretization? x(k + 1) = F ( x(k ). u (k + 1).r.Lu (k + p − 1). u(k+p-1) Nonlinear Program (Not so nice!) . d (k )) + e(k ) The prediction equation is nonlinear w.

e.Difficulty (2) State Estimation & x = f ( x. d ) + K ( k )( y k m (k ) − g ( x(k ) ) • Computationally more demanding steps.. u . d ) + w y = g ( x) + ν Extended Kalman Filtering k +1 x(k + 1) = ∫ f ( x . u . may not be stable • Best practical solution at the current time • Promising alternative: Moving Horizon Estimation (requires solving NLP) • Difficult to come up with an appropriate stochastic system model (no ID technique) .g. calculation of K at each time step • Based on linearization at each time step – not optimal.

Practical Algorithm EKF x(k) Dynamic Matrix based on the linearized model at the current state and input values ⎡ k +1 ⎤ ⎢ ∫ f ( x. d )⎥ Linearized Effect 4 Input Adjustments ⎢∫ of Future ⎣ k ⎦ Model integration with constant input u=u(k-1) and d=d(k) Linear prediction equation Linear or Quadratic Program . u . u . d ) ⎥ + Ω u (k ) ⎢ ⎥ ⎢ M ⎥ ⎢k ⎥ ⎢ ⎥ M ⎢ ⎥ ⎢k + p M ⎥ ⎢ ⎥ + p −1 ⎢ y k + p| k ⎥ ⎣∆u (k444 )⎦ ⎣ ⎦ ⎢ ⎥ 144 2 3 f ( x. u . d ) ⎥ ⎡ yk +1|k ⎤ ⎢k ⎥ ⎡ ∆u (k ) ⎤ ⎢y ⎥ ⎢k +2 ⎥ ⎢ ∆u (k + 1) ⎥ ⎢ k + 2|k ⎥ = ⎢ ∫ f ( x.

Additional Comments / Summary • Some refinements to the “Practical Algorithm” are possible – Use the previously calculated input trajectory (instead of the constant input) in the integration and linearization step – Iterate between integration/linearization and control input calculation • Full-blown nonlinear MPC is still computationally prohibitive in most applications .

Model structure .Lecture 3 Linear Model Identification .Data pretreatment .Error analysis .Plant testing .Parameter/model estimation .Model validation .

System Identification Building a dynamic system model using data obtained from the plant .

Why Important? • Almost all industrial MPC applications use an empirical mo del obtained through system identification • Poor model → Poor Prediction → Poor Performance • Up to 80% of time is spent on this step • Direct interaction with the plant – Cost factor. safety issues. credibility issue • Issues and decisions are sufficiently complicated that syst ematic procedures must be used .

size of perturbation) • Closed-loop or open-loop? • One-input-at-a-time or simultaneous? • How long? • Etc.Steps and Decisions Involved Plant Test Raw Data • Test signal (shape. • Outlier removal • Pre-filtering Pretreatment Conditioned Data ID Algorithm Model • Model structure • (Parameter) estimation algorithm No Validation Yes • Source of validation data • Criterion End Objective: Control .

Model Structure .

Model Structure (1) Model Process Noise Disturbance Model Output Noise Measured Outputs Inputs Plant Dynamics Σ Σ • I/O Model White noise sequence y ( k ) = G ( q )u ( k ) + 1 24 4 3 effect of inputs effect of disturbances. noise H ( q ) e( k ) 14 3 24 Models auto.and cross-correlations of the residual (not physical cause-effect) Assume wolog that H(0)=1 .

H (q) = • ARMAX (Moving average of −the noise term) q −1 − L − a q − n 1 − a1q −1 − L − an q n 1 − a1 n y (k ) = a1 y (k − 1) + L + an y (k − n) + b1u (k − 1) + L + bmu (k − m) + e(k ) + c1e(k − 1) + L + cn e(k − n) b1q −1 + L + bm q − m 1 + c1q −1 + L + cn q − n G (q) = .SISO I/O Model Structure (1) • FIR (Past inputs only) y (k ) = b1u (k − 1) + L + bmu (k − m) + e(k ) • ARX (Past inputs and outputs: “Equation Error”) G (q ) = b1q −1 + L + bm q − m . H (q) = −1 −n 1 − a1q − L − an q 1 − a1q −1 − L − an q − n . H (q ) = 1 y (k ) = a1 y (k − 1) + L + an y (k − n) + b1u (k − 1) + L + bmu (k − m) + e(k ) b1q −1 + L + bm q − m 1 G (q) = .

H (q) = 1 −1 −n 1 − a1q − L − an q • Box Jenkins (More general than ARMAX) b1q −1 + L + bm q − m 1 + c1q −1 + L + cn q − n G (q) = . H (q) = −1 −n 1 − a1q − L − an q 1 − d1q −1 − L − d n q − n .SISO I/O Model Structure (2) • Output Error (No noise model or white noise error) ~ (k ) = a ~ (k − 1) + L + a ~ (k − n) + b u (k − 1) + L + b u (k − m). y 1y ny 1 m y ( k ) = ~ ( k ) + e( k ) y b1q −1 + L + bm q − m G (q) = .

Coefficients are matrices. For example. ARX model becomes y (k ) = A1 y (k − 1) + L + An y (k − n) G (q) = I − A1q − L − An q −1 1 ( H (q) = (I − A q + B1u (k − 1) + L + Bmu (k − m) + e(k ) −1 − n −1 − L − An q − n −1 ) (B q ) 1 −1 + L + Bm q − m ) • Identifiability becomes an issue A is an sets n matrix and Bi is an n y × same G(q) – iDifferentn y × of ycoefficient matrices giving exactlynu matrixand H( q) through pole/zero cancellations → Problems in parameter estimati on → Requires special parameterization to avoid problem .MIMO I/O Model Structure • • Inputs and outputs are vectors.

and cross-correlation of the residual • Identifiability can be an issue here too – State coordinate transformation does not change the I/O relationship .State Space Model • Deterministic x(k + 1) = Ax(k ) + Bu (k ) y (k ) = Cx(k ) + e(k ) • Combined Deterministic / Stochastic Output Error Structure x(k + 1) = Ax(k ) + Bu (k ) + Ke(k ) y (k ) = Cx(k ) + e(k ) xd (k + 1) = Axd (k ) + Bu (k ) yd (k ) = Cx(k ) Effect of deterministic input ARMAX Structure + xs (k + 1) = Axs (k ) + Ke(k ) y s (k ) = Cxs (k ) + e(k ) Auto.

Parameter (Model) Estimation .

Overview Data Model Structure Selection Model Structure Parameter (Model) Estimation Model For Validation Prediction Error Method IV Method Statistical Method • MLE • Bayesian Subspace ID Method ETFE • Frequency Domain .

System ID Toolbox for MATLAB • Computational complexity depends on the model st ructure – ARX. FIR → Linear least squares – ARMAX. BJ → Nonlinear optimization • Not easy to use for identifying multivariable models . OE.Prediction Error Method • Predominant method at current time • Developed by Ljung and coworkers • Flexible – Can be applied to any model structure – Can be used in recursive form • Well developed theories and software tools – Book by Ljung.

θ )u (k ) ) 14 244 4 3 Contains at least 1 delay ( ) e(k ) = y (k ) − yk |k −1 = H −1 (q. OE.θ )u (k ) + I − H −1 (q.θ )e(k ) → yk |k −1 = G (q.Prediction Error Method • Put the model in the predictor form y (k ) = G (q.θ )u (k ) ) – ARX.θ )( y (k ) − G (q.θ ) ( y (k ) − G (q.θ )u (k ) ) • Choose the parameter values to minimize the sum of the pr ediction error for the given data ⎧1 min ⎨ θ ⎩N ∑ k =1 N ⎫ e( k ) 2 ⎬ ⎭ 2 e(k ) = H −1 (q.θ )( y (k ) − G (q. FIR → Linear least squares.θ )u (k ) + H (q. – ARMAX. BJ → Nonlinear least squares .

linear algebra Good for identifying multivariable models – No special parameterization is needed Not optimal in any sense May need a lot of data for good results May be combined with PEM – Use SS method to obtain an initial guess for PEM .Subspace Method • • • • • • • • • More recent development Dates back to the classical realization theories but rediscov ered and extended by several people Identifies a state-space model Some theories and software tools Computationally simple – Non-iterative.

Cov(e)} within some similarity transformation We are free to choose the state coordinates . B.Main Idea of the SS-ID Method (1) Assumed Form of the Underlying Plant x(k + 1) = Ax(k ) + Bu (k ) + w(k ) y (k ) = Cx(k ) + ν (k ) Innovation Form (Steady-State Kalman Filter) Equivalent to the above in I/O sense x(k + 1) = Ax(k ) + Bu (k ) + Ke(k ) y (k ) = Cx(k ) + e(k ) Identify {A. C. K.

Main Idea of the SS-ID Method (2) ⎡ y (k − 1) ⎤ ⎢ M ⎥ ⎢ ⎥ ⎢ y (k − n )⎥ ⎣ ⎦ Past outputs An upper bound of the state dimension n L1 ⎡ x1 (k ) ⎤ ⎢ M ⎥ ⎢ ⎥ ⎢ xn ( k ) ⎥ ⎣ ⎦ ⎡ C ⎤ ⎢ CA ⎥ Extended ⎢ ⎥ Observability ⎢ M ⎥ Matrix ⎢ n −1 ⎥ ⎣CA ⎦ Γo ⎡ u (k − 1) ⎤ ⎢ M ⎥ ⎢ ⎥ ⎢u (k − n )⎥ ⎣ ⎦ Past inputs L2 State (Minimum Storage) ⎡ yk |k −1 ⎤ ⎡ ek |k −1 ⎤ ⎢ ⎥ ⎢ ⎥ M ⎥ +⎢ M ⎥ ⎢ ⎢ yk + n −1|k −1 ⎥ ⎢ek + n −1|k −1 ⎥ ⎣ ⎦ ⎣ ⎦ Future Output Prediction Prediction Error ⎡ u (k ) ⎤ ⎢ ⎥ M ⎢ ⎥ ⎢u (k + n − 2)⎥ ⎣ ⎦ Future inputs .

Main Idea of the SS-ID Method (3) Y 0+ ⎡Y − ⎤ = Γo [L1 L2 ]⎢ − ⎥ + L3U 0+ + E 0+ 1 24 U 4 3 ⎣ ⎦ M • Find M through linear least squares – Consistent estimation since E0+ is independent of the regres sors – Oblique projection of data matrices • Perform SVD on M and find n as well as Γo M = [Q1 ⎡Σ n Q2 ]⎢ ⎣0 0 ⎤ ⎡ P1T ⎤ ⎥⎢ T ⎥ ≈ 0⎦ ⎣ P2 ⎦ Γo = Q1Σ1/ 2 n Some variations exist among different algorithms in terms of picking the state basis The column space of Q1 is the state space .

Main Idea of the SS-ID Method(4) • Obtain the data for x(k) ⎡Y − (k ) ⎤ x(k ) = Γo− ⎢ − ⎥ { U (k )⎦ pseudo inverse ⎣ • • Obtain the data for x(k+1) in a similar manner Obtain A.B and C through linear regression – Consistent estimation since the residual is independent of the regre ssor ⎡ x(k + 1)⎤ ⎡ A B ⎤ ⎡ x(k ) ⎤ ⎡ Ke(k )⎤ ⎢ y (k ) ⎥ = ⎢C 0 ⎥ ⎢ y (k )⎥ + ⎢ e(k ) ⎥ ⎦⎣ ⎦ ⎣4 4⎦ ⎣ ⎦ ⎣ 123 Residual • Obtain K and Cov(e) by using the residual data .

B. rather than minimization of the pre diction error for the given data .Properties • N4SID (Van Overschee and DeMoor) – – – – Kalman filter interpretation Proof of asymptotically unbiasedness of A. and C Efficient algorithm using QR factorization - • CVD (Larimore) – Founded on statistical argument – Same idea but the criterion for choosing the state basis (Q1) diff ers a bit from N4SID – based on “correlation” between past I/O data and future output data.

A → B • MOESP (Verhaegen) .Alternative Γo → C .

Error Analysis .

Variance↑ n ˆ cov(vec(GN )) ≈ (Φ u ) −T ⊗ Φ d N 144 44 2 3 Noise − to −signal ratio . pre-filtering of the data. the number of data poin ts. but not on pre-filtering – Asymptotic distribution (as n.g.by Ljung • Variance: Error due to limited availability of data – Vanishes as # of data points → ∞ – Depends on the number of parameters.Error Types • Bias: Error due to structural mismatch – Bias = the error as # of data points → ∞ – Independent of # of data points collected – Bias distribution (e.. etc. – Frequency-domain bias distribution under PEM . etc. N → ∞): • Main tradeoff – Richer structure (more parameters) → Bias↓. S/N ratio. in the frequency domain) depends on the input spectrum.

Plant Test for Data Generation .

Test Signals • Very Important – Signal-to-noise ratio → Distribution and size of the variance – Bias distribution • Popular Types – Multiple steps: Power mostly in the low-frequency region→ Good e stimation of steady-state gains (even with step disturbances) but g enerally poor estimation of high frequency dynamics – PRBS: Flat spectrum → Good estimation of the entire frequency re sponse. given the error also has a flat spectrum (often not true) – Combine steps w/ PRBS? .

Single Input Testing (SIT) • MIT gives better signal-to-noise ratio for a given t esting time • Control-relevant data generation requires MIT • MIT can be necessary for identification of highly i nteractive systems (e.g. systems with large RGA) • SIT is often preferred in practice because of the m ore predictable effect on the on-going operation ..Multi-Input Testing (MIT) vs.

Open-Loop vs. Closed-Loop Open-Loop Testing Perturbation Signal r G0 u Closed-Loop Testing d y Dither Signal r Location 1 Location 2 d y G0 _ C u .

less damaging effect on the on-going operation – Generates data that are more relevant to closed-loop c ontrol • Cons – Correlation between input perturbations and disturbanc es / noise through the feedback. – Many algorithms can fail or give problems – They give “ bias” unless the assumed noise structure is perfect .Pros and Cons of Closed-Loop Testing • Pros – Safer.

– Perturbations due to error feedback hardly contributes to variance reduction (since they are correlated to the errors) ˆ )) ≈ n (Φ r ) −T ⊗ Φ cov(vec(GN u d N Portion of the input spectrum due to the dithering Output error spectrum – The level of external perturbation signals also contribute to the siz e of bias due to the feedback-induced correlation e − − − Φ eu Φ u 1 = ( Pe Φ u 1 ) × (Φ u Φ u 1 ) 123 1 24 4 3 Noise − to − signal ratio − ˆ E{G0 − GN } = ( H 0 − H µ )Φ eu Φ u 1 Relative contribution of noise feedback to input spectrum • Specialized algorithm may be necessary to avoid bias .Important Points from Analysis • External perturbations (“dither”) are necessary.

. N } → G N ( and H N ) • Indirect Approach ˆ T I − ˆ C) ˆ ˆ yr ⎯G =⎯(⎯T⎯⎯→ G ˆ ⎯ D = { y (i ). u (i ).. r(i)... u (i ). TN ) ⎯G =⎯ ⎯⎯→ GN N N yr N ur −1 N • Two-Stage or Projection Approach ~N ˆ D = {u(i).Different Approaches to Model Identification with Closed-Loop Data • Direct Approach ˆ ˆ D N = { y (i ). N } → TN N N N yr N yr N −1 • Joint I/O Approach ˆ ˆ ˆ ˆ yr ˆ ur ˆ ⎯ T (T ) D = { y (i )..... r (i ). i = 1... N } → (TN .. i = 1.i = 1.. i = 1...N} → D1 = {u(i)} → GN ~N D2 = { y(i)} N 1 . r (i )...

Data Pretreatment .

Main Issues (1) • Time-consuming but very important • Remove outliers • Remove portions of data corresponding to u nusual disturbances or operating conditions • Filter the data – Affects bias distribution (emphasize or de-emphasize different frequency regions) – Does NOT improve the S/N ratio – often a misconcept ion .

set point changes) that can destroy the effectiveness of m any ID methods (e. ∆u(k) = u(k) − u(k-1)) – Removes trends (e.g. so low-pass filtering may be necessary .g. subspace ID) – Often used in practice – Also removes the input power in the low-frequency re gion..Main Issues (2) • Difference the data? (∆y = y(k) − y(k-1).. noise ).. (PRBS → zero input power at ω = 0) – Amplifies high-frequency parts of the data (e.g. effect of step disturbances.

Model Validation .

.g.Overview • Use fresh data different from the data used for m odel building • Various methods – Size of the prediction error – “Whiteness” of the prediction error – Cross correlation test (e. prediction error and inputs) • Good prediction with test data but poor predictio n with validation data – Sign of “overfit” – Reduce the order or use more compact structures like ARM AX (instead of ARX) .

Concluding Remarks on Linear ID • System ID is often the most expensive and difficult par t of model-based controller design • Involves many decisions that affect – Plant operation during testing – Eventual performance of the controller • Good theories and systematic tools are available • System ID can also be used for constructing monitorin g models – Subspace identification – Trend model. not a causal model → Active testing is not needed .

u j ) ≥ 0 g p (xp ) ≥ 0 x j +1 = f ( x j .K. • Uncertainty is not explicitly addressed.u p − ⎩ j = 0 ⎭ stage-wise terminal cost cost g j (x j . u j ) Path constraints Terminal constraints Model constraints • General formulation for deterministic control and scheduling problems. • Continuous and integer state / decision variables possible • In control. u j ) + φ p (x p )⎬ u0 . p=∞ case is solved typically.Deterministic Multi-Stage Optimization ⎧ p −1 ⎫ min1 ⎨∑ φ ( x j . .

g.): • Fixed parameter case solution • Computational limitation for large-size problem (e..Solution Approaches • Analytical approach: 50s-70s – Derivation of closed form optimal policy ( requires solution to HJB equation (hard!) u j = µ* (x j ) ) • Numerical approach: 80s-now – Math programming (LP.g. a lookup table) • Significantly higher computational burden – Practical solution: • Resolve the problem on-line whenever parameters are updated or constraints are violated (e. when p= ∞). MILP. – Parametric programming: • General parameter dependent solution (e. in Model Predictive Control or Reactive Scheduling)..g. . NLP. QP. etc..

• No satisfactory solution approach currently available. and other real-time decision problems in an uncertain dynamic environment. u j ) ⎬ u j =µ ( x j ⎩ j =0 ⎭ 0 <α <1 x j +1 = f h ( x j . scheduling. ω j ) Pr g ( x j .Stochastic Multi-Stage Decision Problem with Recourse ⎧∞ j ⎫ min) E ⎨∑ α φ ( x j . u j . model Chance constraint [ ] • Next “holy-grail” of control: A general form for control. u j ) ≥ 0 ≥ ζ Discount factor Markov sys. .

Limitation of Stochastic Programming Approach Simple case of 2 scenarios (↑ or↓) per stage x k+2 ↑↑ … … … … … … … … xk+1 xk uk u k +1 ↑ ↑ u ↑↑2 k+ x↑↓ k+2 u↑↓ k+2 x↓↑ k +2 Total number of decision variables = (1+2+4+…+2p-1) nu Number of branches to evaluate for each candidate decisions = 2p x↓+1 k u k +1 ↓ u ↓↑ k +2 x ↓↓ k +2 u ↓↓ k +2 .

Stochastic Programming Approach General case (S number of scenarios per stage) • Total number of decision variables = (1 + S + S2 +…+ Sp-1) nu • Number of branches to evaluate for each decision candidate = Sp • Not feasible for large S (large number of scenarios) and/or large p (large number of stages) – practically limited to two stage problems with a small number of scenarios. BUT highly limited! . • Current practical approach: Evaluate most likely branch(es) only.

u ) ) u∈U { } .Dynamic Programming (DP) • The concept of cost-to.* J ( xk ) ⇒ go – Represents future costs under (optimal) control – Parameterizes the solution as a function of state x u j = k +1 α j − k −1φ (x j . u) )} J*(x) is a solution to Bellman Equation J * ( x ) = min E φ ( x. u ) + αJ * ( f h ( x. u ) + J * ( fh ( x. µ * ( x j ) ) ∑ ∞ µ ( x ) = arg min {φ ( x.

u k ) u(k ) [ ] Discretization of entire state space ali ) ion ces s +a en J ip1(ξ ) = min E φ (ξ .Value Iteration Approach to Solving DP J * ( xk ) = min E φ ( xk . u k ) + αJ * ( f h ( xk . u ) + αJ i (ξˆ) S m u Di tion f c eo A s ur te & sampling & C ta discretization (S Valuety iteration [ ] i = i+1 .

Approximate Dynamic Programming (ADP) ∀ • Bellman equation needs to be solved x ∈ X – Curse of dimensionality! Not suitable for high dimens. – Iteratively improve the policy and cost-to-go function • Iterate over only the sampled points in the state space • Use interpolation to evaluate the cost-to-go values for nonsampled points. . sys. • Key idea of ADP – To find approximate cost-to-go function – Use simulations under known suboptimal policy to sample a very small “relevant” fraction of the states and initialize cost-to-go value table.

u ) ) u∈U [ ] Sample Avg.Approximate Dynamic Programming (ADP) Approximate Value Iteration On-line Decision ~ u = µ ( x) = arg min E φ ( x. etc.u ) + αJ i ( f ( x.u ) ) u [ ] Monte-Carlo Simulations • Closed-loop w/ suboptimal policies • MPC. Iteration on the Bellman Equation ~ J i +1 ( x) = min E φ ( x. PI. u k • Initial cost-to-go: J ( x) = ∑ α iφ i ~ J ( x) := xk → J ( xk ) .u ) + αJ ∗ ( f ( x. sampled state points Sampled State Points i ← i +1 converged solution ~ J* Approximation of Cost-to-Go • State and input trajectories: xk .