A short introduction to parameter identification

with Matlab worked examples

Alain Vande Wouwer

Control Department
University of Mons
Purpose of the talk : introduction to linear and nonlinear
parameter estimation

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 2 / 19

Purpose of the talk : introduction to linear and nonlinear
parameter estimation

Estimating a physical quantity θ requires a specific experiment

allowing for this quantity to ”express itself as much as possible”
(notion of sensitivity)
This experiment requires a system onto which inputs u(t ) are applied
(stimuli) and whose outputs y (t ) are collected (observations)
t is the explanatory variable (time for a dynamic experiment)
A model M is required to mathematically express the dependence of
ˆ M (t , θ, u)
the system response with respect to these factors =

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 3 / 19

Purpose of the talk : introduction to linear and nonlinear
parameter estimation

Measurements y are subjected to random perturbations (intrinsic

noise  ) which in turn will generate perturbed estimated values θ̂,
even if the model is perfect
The mathematical model may not correspond exactly to the reality of
the experiment, leading to a biased estimation θ̂

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 4 / 19

 Simple bacterial growth

ϕ
kS S → X
A variety of kinetic laws:
Monod law µ(S ) = µmax KMS+S
Contois law µ(S ) = µmax KM XS+S
Haldane law µ(S ) = µmax K S
2 /K
M +S +S I

For example:
" # " #
Ẋ 1 S
= µmax X
Ṡ −kS KM + S
with unknown model parameters that have to be estimated from
experimental data... how?

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 5 / 19

 Main ingredients of parameter estimation
A set of (informative !) experimental data (with experimental errors)
A model structure with a set of parameters
A distance between the experimental data and the model prediction
(which is function of the parameter values)
A minimization procedure (analytical, if feasible, or numerical)

A popular choice for the distance is the sum of squared errors.

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 6 / 19
 Model structure and parametrization

A lack of information in the data (extreme case: less data than

parameters) leads to parameter uncertainty and correlation (extreme
case: total undetermination)
A wrong model structure will lead to systematic errors in the
parameters
Overparametrization (a too large set of the degrees of freedom) will
lead to parameter uncertainty and/or overfitting (undesired
reproduction of experimental errors)

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 7 / 19

 Linear and nonlinear models
Linearity (in the parameters !!!) can be an interesting feature to ease the
parameter estimation procedure
a straight line
m(t ) = θ1 t + θ2
a polynomial in t
m(t ) = θ1 t 2 + θ2 t + θ3
a linear combination of functions
m(t ) = θ1 exp (t ) + θ2 cos (t ) + θ3 t 3 + θ4
Linearity can also be expressed through a transformation

S
µ(S ) = µmax
KM + S
can be transformed to

1 KM + S 1 KM 1 1
= = + = θ1 + θ2
µ(S ) µmax S µmax µmax S S
A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 8 / 19
Distance between measured data and model prediction
The most popular distance between the measured data 2(tk ) and the
model prediction m(tk , θ) is the sum of squared errors

M
X
J (θ) = (y (tk ) − m(tk , θ))2
k =1

but other distances can be used as well, for instance a weighted sum of
squared errors,

M
X
J (θ) = wk (y (tk ) − m(tk , θ))2
k =1

The weighting factors can translate the trust that the modeler has in the
data. Some common choices include wk = σ12 (probabilistic), wk = t1k
k
(forgetting factor).

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 9 / 19

Distance between measured data and model prediction

Other norms (distances) can be used as well, for instance a sum of

absolute errors

M 
X
J (θ) =
y (t ) − m(t , θ)
k k
k =1

This criterion emphasizes less the larger deviations (could be interesting if

there are outliers).
However, the derivative is discontinuous, so that it is harder to minimize
(and nonuniqueness of solution).

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 10 / 19

 Distance minimization
The minimization of J (θ) can be achieved in an analytical way in linear
least square problems. Otherwise it has to be achieved in a numerical way.
Potential problems are:
presence of local minima
lack of sensitivity in some parameter range

borrowed from http :

//orion.uwaterloo .ca / hwolkowi /henry /reports /talks .d /t09talks .d /09waterloomatlab .d /optimTipsWebinar /html /optimTipsTricksWalkthrough
A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 11 / 19
 Overall view of parameter estimation procedure

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 12 / 19

 Cost function topology

Different possible shapes of the cost function (distance)

borrowed from http : //clerc .maurice .free .fr /pso /Semi − continuousc hallenge /Semi − continuousc hallenge .htm

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 13 / 19

 Sensitivity analysis and experiment design
Compute the parametric sensitivities

∂mi 


∂θj θ∗

j

and evaluate the influence of the several parameters on the measurement

outputs as well as potential links between them.
Design experiments (selection of initial conditions and/or operating
conditions) to enhance sensitivity

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 14 / 19

 Parameter estimation: an art !

Parameter estimation is a difficult task as it combines:

Informative experimental studies
Selection of a good model structure (in the following we will assume
that this choice is well made: certainly requires a good
physical/biological inspiration !)
Selection of good parameter estimation tools (the goal of this
presentation !)
Model validation (direct and cross validation)

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 15 / 19

 An example of the importance of model structure and
validation

Identification of a kinetic model based on 11 measurements of µ for S from

0 to 500 (by steps of 50) with an additive white noise with normal
distribution, zero mean and standard deviation of 5

S
µ = 120
40 + S

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 16 / 19

 An example of the importance of model structure and
validation

Identification of a kinetic model based on 11 measurements of µ for S from

0 to 500 (by steps of 50) with an additive white noise with normal
distribution, zero mean and standard deviation of 5

S
µ = 120
40 + S

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 16 / 19

 An example of the importance of model structure and
validation
Different candidate model structures
1 Exact model structure (Monod law)

S
µ = θ1
θ2 + S
2 Same level of complexity (Tessier law)

µ = θ1 1 − e −S /θ2
 

3 Overparametrized model (polynomial ”black-box” model)

µ = θ1 + θ2 S + θ3 S 2 + θ4 S 3 + θ5 S 4 + θ6 S 5

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 17 / 19

An example of the importance of model structure and
validation

Direct Validation

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 18 / 19

An example of the importance of model structure and
validation

Direct Validation Cross Validation

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 18 / 19

 Parameter estimation: an art !

A. Vande Wouwer (UMONS) A short introduction to parameter identification UMONS 19 / 19