Filtering and Identification: The System Identification Cycle

Filtering and Identification
Lecture 12:
The system identification cycle
Michel Verhaegen and Jan Willem van Wingerden

1/39
Delft Center for Systems and Control

Delft University of Technology
System identification
Is there more then fitting a model to the data?
2/39
System identification
Is there more then fitting a model to the data?
General Rule: Identify the system under

conditions it will be used for!
2/39

Experiment
Start
Design
Data pre (u(k),y(k) ) Fit model

to data
processing
Model
validation
Yes
ok?
End
No
Model
structure
selection
3/39
Overview
Experiment design
Data pre-processesing
Model structure selection
Model validation
Demo acoustical duct
4/39
Experiment design
Choice of
Sampling frequency
Experiment duration
Type of input sequence
Persistency of excitation of the input
5/39
Choice of sampling frequency

Aliasing: [Shannon] All frequency components
with a frequency higher than S /2 are mirrored
across the line = 2S in the band [ 2S , 2S ].
6/39

Aliasing: [Shannon] All frequency components
with a frequency higher than S /2 are mirrored
across the line = 2S in the band [ 2S , 2S ].
if frequency band of interest equals [B , B ]
then pre-filter signals by a band-pass filter
1
B 0
6/39

Bandwith LTI system: B is the frequency at
which |G()| |G(0)|

2
7/39

Bandwith LTI system: B is the frequency at
which |G()| |G(0)|

2
qualitative info by simple transient
experiments.
Rule of tumb: based on a rough estimate of
bB
one over selects S 10b
B .
7/39
Example: qualitative selection

1
Consider G(s) =
,
1 + s
1
then B = , ( = 10)
When G(s) is unknown:

get info on B via the step
response
0.8
0.6
0.4
0.2
0
0
10
20
30
40
50
60
Step response
8/39
Example: qualitative selection

1
Consider G(s) =
,
1 + s
1
then B = , ( = 10)
When G(s) is unknown:

get info on B via the step
response
0.8
0.6
0.4
0.2
0
0
10
20
30
40
50
60
Step response
Rule: If we select about 8 or 9 samples during

the period [0, steady state], we sample at
about 10B . The time interval T between the
2
= 2 seconds.
circles- equals 10
B
8/39
Transient analysis
The use of the response to simple inputs (steps,
impulses, . . . ) to retrieve info on:
which input affects which output?
linearity of the system
dominant time constants
9/39
Transient analysis
The use of the response to simple inputs (steps,
impulses, . . . ) to retrieve info on:
which input affects which output?
linearity of the system
dominant time constants
Procedures can be devised to

determine dominant time constants (and bandwith) from step
response when the system is underdamped
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
0
10
15
9/39
Experiment duration
In PEM, the covariance matrix of the estimated
parameter vector equals
" N 1
#1
e2 1 X
Select large N .
(k)(k)T
N N
k=0
10/39
Experiment duration
In PEM, the covariance matrix of the estimated
parameter vector equals
" N 1
#1
e2 1 X
Select large N .
(k)(k)T
N N
k=0
However, slow system dynamics constrain N .

For example: with a destillation column the
collection of 1000 data points may require
testing for over a week.
This can be a lot of money!
10/39
Experiment duration
A rule of tumb: experiment duration 10 times
the largest time constant.
11/39
Experiment duration
1
Example: Consider G(s) = (1+1 s)(1+
2 s)
1 = 1 sec. and 2 = 0.01 sec.
1 rad
Sampling frequency: s = 10. 0.01
sec 160 Hz
Experiment duration: 10 1 sec.
11/39
Experiment duration
1
Example: Consider G(s) = (1+1 s)(1+
2 s)
1 = 1 sec. and 2 = 0.01 sec.
1 rad
Sampling frequency: s = 10. 0.01
sec 160 Hz
Experiment duration: 10 1 sec.
What efficiency do we gain if we could handle
multi-rate (double-rate) sampled data
sequences?
11/39
Type of input sequences

a
Step
0
1
Harmonic
c
a
Short pulse
0
1
Doublet
b
Gaussian
white noise
Frequency
sweep
a
n steps
Staircase
0
1
PRBS
0
1
PRBS: Pseudo Random Binary Sequence

12/39
Persistency of excitation
Example Consider the input u(k):
(
sin k k = 1, 2, , N
u(k) =
0
k0
Can we estimate the parameters of the following
model uniquely?
y(k) = 1 u(k) + 2 u(k 1) + 3 u(k 2)
13/39
Persistency of excitation
The sequence {u(k)}N
k=1 is persistently exciting
of order n if and only if there exists an integer N
such that the matrix:
u(0)
u(1)
u(N 1)
u(1)
u(2)
u(N
)
U1,n,N =
.
.
.
.
.
.
u(n 1) u(n) u(N + n 2)
has full column rank n.
14/39
Persistency of excitation with an ARX model
Lemma 2: Let the input-output data be generated

by the system:
Bn (q)
u(k) + v(k)
y(k) =
An (q)
with v(k) p.e. of any order, then we can uniquely
estimate the coefficients of an n-th order ARX
model provided the input u(k) is persistently exciting of order n.
15/39
Overview
Experiment
Start
Design

processing
to data
Model
validation
Yes
ok?
End
No
Model
structure
selection
16/39
Data pre-processing
Decimation
Detrending
Pre-filtering
Concatenating data sequences
17/39
Decimation
Definition: Taking every jn + 1-th sample (for
j N and n = 0, 1, ) of the original
input-output data sequences. S jS
Caution: Make use of a digital anti-aliasing filter
with cut-off 2jS ( rad
s ).
18/39
Decimation
When is decimation necessary?
The effect of having selected a too high (original)
sampling frequency may reflect into:
The poles cluster around the point z = 1:

lim ej T = 1
T 0
High-frequency disturbance in the data

(above the frequency band of interest)
19/39
Why Detrending?
Example: Consider the nonlinear system,

x(k + 1) = f x(k)
y(k) = x(k)
Linearization of f (x) = x2 in
the point (1, 1)
x(k + 1)
x2 (k)
Local coordinates x
(k)
x(k + 1)
x
(k + 1)
2x(k) 1
x
(k)
1
x(k)
-1
x(k)
-1
20/39
Detrending
A linear model approximation in a limited region
of the operation envelope:
x
e(k + 1) = Ae
x(k) + B(u(k) u) x
e(k0 ) = x0
y(k) = C x
e(k) + y
Two ways to deal with unknown offsets (u, y):
Subtracting sample means from input and

output sequences {u(k), y(k)}.
Estimating the offsets together with other

system parameters.
21/39
RPs in the frequency domain

The cost function optimized by Prediction error
methods:
2
lim JN () = E[(k) ]
N
The stochastic variant of Parseval is:

Z /T
T
2
E[(k) ] = R (0) =
()d
2 /T
22/39
Data pre-filtering
+ He:
\ = y = Gu
Given {uk , yk } and a linear filter L(q), and SGM
what system is identified using {L(q)uk , L(q)yk }?
23/39
Data pre-filtering
+ He:
\ = y = Gu
yk = G(q)uk + vk (L(q)yk ) = G(q)(L(q)uk ) + L(q)vk
23/39
Data pre-filtering
+ He:
\ = y = Gu
yk = G(q)uk + vk (L(q)yk ) = G(q)(L(q)uk ) + L(q)vk

However, JN () is modified,
1
JN () =
2
2
2
|L|
|L|
v
u
2
b
|G G|
+
d
2
2
b
b
|H|
|H|
The filter L(q) can be used a posteriori to modify the weighting

b G|2 .
term of |G
b 1 .
For example: to cancel the effect of H
23/39
Overview
Experiment
Start
Design

processing
to data
Model
validation
Yes
ok?
End
No
Model
structure
selection
24/39
Model structure selection
Delay or system dead-time estimation

shifting the data sequences and then
identifying delay-free models.
Model structure selection in subspace

identification
Illustration with acoustical duct model
25/39
Model selection in subspace ident.

Recipe: Selecting the number of (block) rows of
the Hankel matrices, that is the index s.
s>n
Therefore only a rough estimate of the system
order needs to be known.
Subspace methods provide usefull information
on the only model structure parameter, namely
the order of the state-space model.
26/39

2
10
10
The computed
singular values
10
10
10
10
15
20
LET THE DATA SPEAK FOR ITSELF

27/39

100
95
90
VAF for
different
model orders
85
80
75
70
65
60
2
VAF (y(k), yb(k, )) =
1
N
PN
2
ky(k)
y
b
(k,
)k
2
k=1
P
N
1
2
ky(k)k
2
k=1
N
10
12
100%
28/39
An illustrative example
Consider a schematic view of our acoustical duct:
Noise
Speaker
Second Speaker
Microphone
u(k)
y(k)
We determine the parameters of a sixth order
state space model [A, B, C, D] given
{u(k), y(k)}6000
k=1 with subspace identification using
s = 40.
The results are compared with PEM.
29/39
An illustrative example
Distribution of the VAF values
for 100 identification experiments
on the acoustical duct
100
100
90
53.1 %
90
100
89.5 %
90
80
80
80
70
70
70
60
60
60
50
50
50
40
40
40
30
30
30
20
20
20
10
10
10
0
0
20
40
60
80
100
0
0
20
PEM
40
SI
60
80
100
0
0
96.9 %
20
40
60
80
100
SI and PEM
30/39
Overview
Experiment
Start
Design

processing
to data
Model
validation
Yes
ok?
End
No
Model
structure
selection
31/39
Model Validation
u(k)
G(q)
+
+
v(k)
y(k)
b )
H(q,
b )
G(q,
e(k)
y(k)
b ) (H(q,
b )) such that
Objective determine G(q,
e(k) is small as possible.
32/39
Model Validation
u(k)
G(q)
+
+
v(k)
y(k)
b )
H(q,
b )
G(q,
e(k)
y(k)
b ) (H(q,
b )) such that
Objective determine G(q,
e(k) is small as possible.
What is small?
32/39
When does small not imply fitting the noise?
How to check this?
33/39
When does small not imply fitting the noise?
How to check this?

Split the available data into two sets:
one data set to estimate the models
another data set to validate the models
Validation tests:
VAF or value of cost function
33/39
What is small/What is a good model?

When it satisfies the conditions for optimality?
e(k) is a white noise signal
34/39

e(k), u(k) are statistically independent
34/39

How to check this?
34/39

How to check this?
Auto-correlation residuals
Cross-correlation residuals and inputs
34/39
Check Conditions for optimality?

White
noise
test
Auto-correlation
by
residual
Cross-correlation
residual
test
and
input
0.1
0.5
0
0.5
1
20
10
10
20
0.1
20
10
10
20
ARX
10
10
20
OE
10
10
20
0.1
0.5
0
0.5
1
20
10
10
20
0.1
20
0.1
0.5
0
0.5
1
20
10
10
20
time lag
0.1
20
ARMAX
time lag
35/39
Demo acoustical duct
Sampling rate Fs = 2000Hz
Excitation: zero-mean white noise
Number of samples N = 4000
Measure data (u, y)
36/39
Summary of Lecture 12
Experiment
Start
Design

to data
processing
Model
validation
Yes
ok?
No
Model
structure
selection
37/39
End
And there is much more . . .
Controller design by system identification
38/39
Adaptive control via parameter estimation

methods
38/39

methods
Fast algorithms for system identification
38/39

methods
Fast algorithms for system identification
Interesting applications: Adaptive Optics,

Windenergy, Automotive,
38/39
Next ...
Wednesday Jan 9, 2013: Q&A and old exam
review
39/39

Filtering and Identification: The System Identification Cycle

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Filtering and Identification: The System Identification Cycle

Uploaded by

Copyright:

Available Formats

Filtering and Identification

Michel Verhaegen and Jan Willem van Wingerden

Delft Center for Systems and Control

Delft Center for Systems and Control

General Rule: Identify the system under

Delft Center for Systems and Control

The system identification cycle

Data pre (u(k),y(k) ) Fit model

Delft Center for Systems and Control

Model structure selection

Demo acoustical duct

Delft Center for Systems and Control

Type of input sequence

Persistency of excitation of the input

Delft Center for Systems and Control

Choice of sampling frequency

Delft Center for Systems and Control

Choice of sampling frequency

Delft Center for Systems and Control

Choice of sampling frequency

which |G()| |G(0)|

Delft Center for Systems and Control

Choice of sampling frequency

which |G()| |G(0)|

Delft Center for Systems and Control

Example: qualitative selection

When G(s) is unknown:

Delft Center for Systems and Control

Example: qualitative selection

When G(s) is unknown:

Rule: If we select about 8 or 9 samples during

Delft Center for Systems and Control

which input affects which output?

linearity of the system

dominant time constants

Delft Center for Systems and Control

which input affects which output?

linearity of the system

dominant time constants

Procedures can be devised to

Delft Center for Systems and Control

Delft Center for Systems and Control

However, slow system dynamics constrain N .

Delft Center for Systems and Control

Delft Center for Systems and Control

Delft Center for Systems and Control

Delft Center for Systems and Control

Type of input sequences

PRBS: Pseudo Random Binary Sequence

Delft Center for Systems and Control

Delft Center for Systems and Control

Delft Center for Systems and Control

Persistency of excitation with an ARX model

Lemma 2: Let the input-output data be generated

Delft Center for Systems and Control

Data pre (u(k),y(k) ) Fit model

Delft Center for Systems and Control

Concatenating data sequences

Delft Center for Systems and Control

Delft Center for Systems and Control

The poles cluster around the point z = 1:

High-frequency disturbance in the data

Delft Center for Systems and Control