17th IFAC Symposium on System Identification

17th IFAC
Beijing
17th Symposium
International
IFAC Symposium on SystemCenter
Convention
on Identification
17th IFAC
Beijing Symposium
International on System Identification
SystemCenter
Convention Identification
October
Beijing
Beijing 19-21, 2015.
International
International Beijing, China
Convention
Convention Available online at www.sciencedirect.com
Center
Center
October
October 19-21, 2015. Beijing, China
October 19-21,
19-21, 2015.
2015. Beijing,
Beijing, China
China
ScienceDirect
IFAC-PapersOnLine 48-28 (2015) 415–420
Particle
Particle filtering
filtering based
based identification
identification for
for
Particle filtering
autonomous based identification
nonlinear ODE models for
⋆⋆
autonomous nonlinear ODE models
autonomous nonlinear ODE models ⋆
Jerker Nordh ∗∗ Torbjörn Wigren ∗∗ Thomas B. Schön ∗∗
Jerker Nordh ∗ Torbjörn Wigren ∗∗ ∗∗ Thomas B. Schön ∗∗ ∗∗
Jerker Nordh
Jerker Nordh Torbjörn ∗ Torbjörn Wigren
Wigren Thomas
Bo Bernhardsson ∗∗ ∗
Thomas
∗
B.
B. Schön
Schön ∗∗
Bo
Bo Bernhardsson ∗
Bo Bernhardsson
Bernhardsson ∗
∗
∗ Department of Automatic Control, Lund University, SE-221 00 Lund,
∗ Department of Automatic Control, Lund University, SE-221 00 Lund,
∗ Department of Automatic Control, Lund
Sweden
Department
Sweden
(e-mail: {jerker.nordh,
of Automatic
(e-mail: Control,
{jerker.nordh, Lund University,
University, SE-221 SE-221 00
bo.bernhardsson}@control.lth.se).
bo.bernhardsson}@control.lth.se). 00 Lund,
Lund,
Sweden
∗∗
Sweden
∗∗
(e-mail:
Department
(e-mail: {jerker.nordh,
of Information
{jerker.nordh, bo.bernhardsson}@control.lth.se).
Technology, Uppsala
bo.bernhardsson}@control.lth.se). University,
∗∗ Department
SE-751 05, of Information Technology, Uppsala University,
∗∗
Department
Department of Information
Uppsala,
of InformationSweden Technology,
Technology, Uppsala
Uppsala University,
(e-mail: {torbjorn.wigren, University,
SE-751
SE-751 05,
05, Uppsala,
Uppsala, Sweden
Sweden (e-mail:
(e-mail:
thomas.schon}@it.uu.se.). {torbjorn.wigren,
{torbjorn.wigren,
SE-751 05, Uppsala, Sweden (e-mail:
thomas.schon}@it.uu.se.). {torbjorn.wigren,
thomas.schon}@it.uu.se.).
thomas.schon}@it.uu.se.).
Abstract: This paper presents a new black-box algorithm for identification of a nonlinear
Abstract: This
This paper
paper presents
presents a
a new black-box algorithm for identification of a nonlinear
Abstract:
autonomous
Abstract:
autonomous
system
This
system
in stable
paper
in presents
stable a new
periodic
periodic new black-box
motion.
black-box
motion. The
algorithm
The particle
algorithm
particle
for
for identification
filtering
filtering
based algorithm
identification
based algorithm
of
of aamodels
nonlinear
nonlinear
models
the
the
autonomous
signal
autonomous system
as thesystem in stable
outputinofstable periodic
a continuous-time
periodic motion.
motion. The
second
The particle
order
particle filtering
ordinary
filtering based
differential
based algorithm
equation
algorithm models
(ODE).
models the
the
signal
signal
The as
as
model the
the
is output
output
selected of
of a
a
based continuous-time
continuous-time
on previous work second
second
which order
order
proves ordinary
ordinary
that a differential
differential
second order equation
equation
ODE is (ODE).
(ODE).
sufficient
signal
The as
model the output of a continuous-time second order ordinary differential equation (ODE).
The
to
The model
model
model a is
is
wide
is
selected
selected
class of
selected
based
based on
on
nonlinear
based
previous
on previous
systems
previous
work
work
workwith which
which proves
proves
periodic
which proves modesthat
that
that of
aa second
motion,order
a second
second also ODE
order
order ODE
systems
ODE
is
is sufficient
is sufficient
that are
sufficient
to
to model
model a
a wide
wide class
class of
of nonlinear
nonlinear systems
systems with
with periodic
periodic modes
modes of
of motion,
motion, also
also systems
systems that
that are
described
to model aby
described
by
widehigher order
classorder
higher
ODEs.
of nonlinear
ODEs.
Such
systems
Such
systems are
with periodic
systems are
common
commonmodes in
in
systems
ofsystems biology.
motion,biology.
also systemsThe
The that are
proposed
proposed are
described
algorithm
described by
by higher
is applied order
higher order to dataODEs.
ODEs. Such
fromSuch systems
the systems
well-known are common
Hodgkin-Huxley
are common in systems
in systems biology.
neuron
biology. The
model.
The This proposed
proposed is a
algorithm
algorithm is
is applied
applied to
to data from the well-known Hodgkin-Huxley neuron model. This is a
challenging
algorithm
challenging isproblem
applied since
problem to data
since
the from
data
the
the
the well-known
Hodgkin-Huxley
from
Hodgkin-Huxley well-knownmodelHodgkin-Huxley
model
is a fourth orderneuron
Hodgkin-Huxley
is a fourth order
model,model.
neuron
model,
but has
model.
but has
This
Thisa mode
a
is
is aa
mode
challenging
of oscillationproblem
challenging in a second
problem since the
the Hodgkin-Huxley
since order subspace. The numerical
Hodgkin-Huxley model
model is is aa experiments
fourth
fourth orderordershow model,
model, thatbut
butthe has aa mode
hasproposedmode
of
of oscillation
oscillation
algorithm does in
in a
a second
second
indeed order
order
solve the subspace.
subspace.
problem. The
The numerical
numerical experiments
experiments show
show that
that the
the proposed
proposed
of oscillation
algorithm does in a
does indeed second
indeed solve order
solve the subspace.
the problem.
problem. The numerical experiments show that the proposed
algorithm
algorithm does indeed solve the problem.
© 2015, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved.
Keywords: Autonomous system, identification, neural dynamics, nonlinear systems, oscillation,
Keywords:
Keywords:
particle Autonomous
Autonomous
filtering, system,
periodicsystem, identification,
system,identification,
phase plane. neuralneural dynamics,
dynamics, nonlinear nonlinear systems,
systems, oscillation,
oscillation,
Keywords:
particle Autonomous
filtering, periodicsystem,
system,identification,plane. neural dynamics, nonlinear systems, oscillation,
phase plane.
particle filtering, periodic system,
particle filtering, periodic system, phase plane. phase
1. INTRODUCTION from measured data alone. However, algorithms tailored
1. INTRODUCTION
1. INTRODUCTION from
from measured
measured data of alone.
secondHowever, algorithms tailored
1. INTRODUCTION for
from
for measured data
identification
identification data of
alone.
alone.
second
However,
order ODEs
However,
order ODEs
algorithms
algorithms
from
tailored
from periodic
tailored
periodic
The identification of nonlinear autonomous systems is a for for identification
dataidentification
appeared in Wigren of
of second
second order
et al.order
(2003a), ODEs
ODEs from
Manchester
from periodic
et al.
periodic
The identification
identification of nonlinear
nonlinear autonomous systems data
is aa (2011)
data appeared
appeared
and Wigren in
in Wigren
Wigren
(2014).etet al.
Aal. (2003a),
(2003a),
result Manchester
Manchester
on identifiability et
et al.
al.
The
fairly
The unexplored field.
identification of
of While a very
nonlinear autonomous
large amount
autonomous systems
systems is
of work
is a data
(2011) appeared
and Wigren in Wigren
(2014). et A al. (2003a),
result on Manchester
identifiability etthat
al.
that
fairly
fairly
has unexplored
unexplored
been field.
field.
devoted field. While
While
to theWhileanalysisa
a very
very large
large amount
amount
of autonomous of
of work
work
systems, (2011)
gives
(2011) and
conditions
and Wigren
Wigren for (2014).
when
(2014). a A
A result
second
result on
order
on identifiability
ODE is that
sufficient
identifiability that
fairly unexplored a very large amount of work gives conditions
has been
has
given
has
been
been
devoted differential
devoted
an ordinary to the
to the analysis
to the analysis
devoted differential analysis
of autonomous
of
equation autonomous systems,
(ODE), thesystems,
of autonomous inverse gives
systems,
gives
for conditions
modeling
conditions of for
for
for
when
when
aa second
second order
when oscillations
periodic a second order
is also
order
ODE
ODE
ODE
is
is sufficient
sufficient
available,
is see
sufficient
given
given an
an ordinary
ordinary differential equation
equation (ODE),
(ODE), the
the inverse
inverse for
for modeling
modeling
Wigren and of
of periodic
periodic
Söderström oscillations
oscillations
(2005) and is
is
Wigren also
also available,
available,
(2015). see
see
That
problem
given has
an ordinaryreceived much
differential less
equationattention.
(ODE), At the
the same
inverse for modeling of periodic oscillations is also available, see
problem
problem has
has received
received much
much less
less attention.
attention. At
At the
the same
same Wigren
Wigren
work provesand
and Söderström
Söderström
that in case (2005)
(2005)
the phase and
and Wigren
Wigren
plane of the (2015).
(2015).
data isThat
That
such
time
problem system
has identification
received much based
less on particle filtering
attention. At the ideas
same Wigren and Söderström (2005) and Wigren (2015). That
time system identification based
based on particle filtering ideas work
work proves
proves that
that in
in case
case the
the phase plane of the data isorder
such
time
is
time system
expanding identification
rapidly. However,
system identification based onlittle
on particle filteringhas
attention
particle filtering ideas
ideasso thatwork
that
the
proves
the
orbit
orbit
does
that
does in not
not the phase
caseintersectphase
intersect
plane
itself,
planethen
itself, then
of
of the
the
a
data
data is
a second
second
such
isorder
such
is
is
far expanding
expanding
been given rapidly.
rapidly.
to However,
However,
identification of little
little attention
attention
nonlinear has
has
autonomous so
so that the
ODE
that the
is orbit does
always
orbit does not intersect
sufficient
not intersect
for itself, then
identification.
itself, thenIn a second
a second
other order
words,
order
is expanding rapidly. However, little attention has so ODE
far been
far been The
systems.
far been The
givenpresent
given to identification
to
givenpresent
identification
paper addresses
to identification
of nonlinear
of nonlinear autonomous
autonomous
this by presenting
of nonlinear autonomous a ODE higheris
ODE is always
always
isorder
always
sufficient
sufficient
models cannot
sufficient
for
for identification.
for identification.
be uniquely identifiable.
identification.
In
In other words,
In other
other words,
words,
systems.
systems. The present paper
paper addresses
addresses this
this by
by presenting
presenting a higher
higher
aa higher order
order models
models cannot
cannot be
be uniquely
uniquely identifiable.
identifiable.
new
systems.approach
The for identification
present paper of a nonlinear
addresses this by autonomous
presenting order models cannot be uniquely
There is a vast literature on stable oscillations in bio- identifiable.
new approach
new
ODE approach
model based for identification
for identification of aa nonlinear
of
on recently developed nonlinear autonomous
autonomous
particle filtering There
new approach for identification of a nonlinear autonomous There is
logical is
isand
aa vast
a vast
chemical literature
literature
systems, on
on
on see
stable
stable e.g.oscillations
oscillations
Rapp (1987). in
in bio-
in bio-
An
ODE
ODE
methods.
ODE
model
model
model
based
based
Furthermore,
based
on
on
on
recently
recently
the paper
recently
developed
developed
presents
developed
particle
particle
particle results There
filtering
new filtering
filtering logical
logical
important and
and
vast
chemical
chemical
example
literature
is systems,
systems,
given by
stable
see
see
neuron e.g.
e.g.
oscillations
Rapp
Rapp
spiking (1987).
(1987).
(Doi et
bio-
An
An
al.,
methods.
methods.
on neural Furthermore,
Furthermore,
modeling, by the
the paper the
paper
applying presents
presents
new new results
new
algorithmresultsto logical
important andexample
chemicalis systems,
given by see e.g.spiking
neuron Rapp (1987).
(Doi et An
al.,
methods.
on neural Furthermore,
modeling, by the paper
applying presents
the new new
algorithmresultsto important
2002;
important example
Izhikevich,
example is
2003;
is given
Hodgkin
given by
by neuron
and
neuron spiking
Huxley,
spiking (Doi
1952).
(Doi et
et al.,
This
al.,
on
data
on neural modeling,
generated by the by by applying
well-known the new
applyingHodgkin-Huxley algorithmmodel. to
to 2002;
spikingIzhikevich, 2003; Hodgkin
in that itand is Huxley,
the way 1952). nerve This
dataneural
data
These
modeling,
generated
generated
constitutes bythe
by thetwo
the well-known
well-known
the new algorithm
Hodgkin-Huxley
Hodgkin-Huxley
main contributions paper. 2002;
model.
of the model.
2002;
spiking
Izhikevich,
is fundamental
Izhikevich,
is fundamental
2003;
2003; Hodgkin
Hodgkin
in that
and
and
it is
Huxley,
Huxley,
the way
1952).
1952).
nerve
This
cells
This
cells
data
These generated
constitutes by the
the well-known
two main Hodgkin-Huxley
contributions of the model.
paper. spiking
communicate,
spiking is
is fundamental
for
fundamental example in
in that
in
that the it
it is
human
is the
the way
brain.
way nerve
The
nerve cells
field
cells
These
These constitutes
constitutes the
the two
two main
main contributions
contributions of
of the
the paper.
paper. communicate, for example in the human brain. The field
As stated above, much work has been performed on the communicate, for example in the human brain. The field
communicate,
of reaction for
kinetics example
provides in the
further human
examples brain. of The
dynamic
field
As stated above, much work has been performed on the of
of reaction
reaction kinetics
kinetics provides
provides further
infurther examples
examples of
of dynamic
dynamic
As
As stated
analysis
stated ofabove,
a given
above, much
much work
nonlinear
work has
has been
autonomous
been performed
performedsystem,on
on see systems
the
the of reaction that are relevant
kinetics provides the field
further of molecular
examples of dynamicsys-
analysis
analysis of (1996).
of a given
a given nonlinear
nonlinear autonomous
autonomous system,
system, see systems
see systems
tems that are
that
biology, are
see relevant
relevant
e.g. in the
in
Ashmore the (1975).
field of
field ofSuch
molecular
molecular
kinematicsys-
sys-
e.g. Khalil
analysis of (1996).
a given The classical
nonlinear analysis
autonomous provided
system,by e.g. systems that are relevant in
see tems biology, see e.g. Ashmore (1975). Such kinematic the field of molecular sys-
e.g.
e.g. Khalil
Khalil
Poincaré (1996).
provides The
The
tools classical
classical
for analysis
analysis
prediction provided
provided
of the by
by e.g.
e.g.
existence tems
tems biology,
equations
biology, see
typically
see e.g.
e.g. Ashmore
result in
Ashmore systems(1975).
(1975). of Such
ordinary
Such kinematic
differ-
kinematic
e.g. Khalilprovides
Poincaré (1996). The tools classical analysis of
for prediction
prediction provided by e.g. equations typically result in systems of ordinary differ-
theBifurcation
existence equations typically result
rightin systems of ordinary differ-
Poincaré
of periodic
Poincaré
of periodic
provides
orbits of
provides
orbits
tools
tools
of a
for
a second order ODE.
for prediction
second order
of
of the
ODE. existence ential
existence
theBifurcation equations
ential
equations
typically
equations
with
with result
right in hand
systems
hand
sides
sides of where
ordinary
where
fractions
differ-
fractions
of periodic
analysis and orbits
similar of a
tools second
have order
also been ODE.
widely Bifurcation
applied to ential
of
ential equations
polynomials
equations in with
the
with right
states
right hand
appear.
hand sides
sides The where
many
where fractions
authors
fractions
of periodic
analysis orbits
and similar
similar of a
tools second
have also order
also ODE.
been widely
widely Bifurcation
applied to to of polynomials in theidentification
states appear. The many authors
analysis
the and
analysis of chaos, tools have
common been
in nonlinear applied
autonomous of
thatpolynomials
have dealt with in
in the states
states appear. of the The many
many authors
TheHodgkin-Huxley
analysis
the and similar
analysis of chaos,tools have also
common in been widely
nonlinear applied to of
autonomous thatpolynomials
have dealt with theidentification appear. of the authors
Hodgkin-Huxley
the
ODEs analysis
(Khalil, of chaos,
1996; Li common
et al., in
2007). nonlinear
There autonomous
are much less that
model
that have
havehave dealt
dealt with
typically
with identification
built on
identification complete of
of the
the Hodgkin-Huxley
models, including
Hodgkin-Huxley
the analysis of 1996;
chaos,Licommon in nonlinear autonomous less model have typically built
ODEs
ODEs
ODEs
(Khalil,
(Khalil,
publications
(Khalil, on1996;
1996; Li
Li et
et al.,
al., 2007).
et al., 2007).
and connections to the
2007).
There
There
There
are
are much
are much
inverse less an
less
problem,
much
model
input
model have
have typically
signal current,
typically built
builtseeon
on complete
complete
one.g. Doi et al.
complete
models,
models,
(2002);
models,
including
including
Saggar
including
publications
publications on
on and
and connections
connections to
to the
the inverse
inverse problem,
problem, an
et input
an al.
input signal current,
signalTobenkin
(2007); current, see see
see
et e.g. Doi
e.g.(2010);
al. Doi et et al. (2002);
(2002); Saggar
et Manchester
al. (2002); Saggar
et al.
i.e. the
publications identification of
on and connections an autonomous nonlinear
to the inverse ODE
problem, an input signal current, e.g. Doi al. Saggar
i.e. the
i.e. the identification
identification of an
of an autonomous
autonomous nonlinear
nonlinear ODE et
ODE et al.
al.
(2011);
et al.
(2007);
(2007);
Lankarany
(2007);
Tobenkin
Tobenkin
Tobenkin et al.et
et
et
al.
al.
(2013).
al.
(2010);
(2010);
With
(2010);
Manchester
Manchester
the exception
Manchester
et
et al.
et al.
of
al.
i.e. the identification of an autonomous nonlinear ODE (2011); Lankarany et al. (2013). With the exception of
⋆ This work was supported by the project Probabilistic modeling of (2011);
Wigren
(2011); Lankarany
(2015)
Lankarany there et
et al.
does
al. (2013).
not
(2013). seem With
With to the
have
the exception
been
exception muchof
of
⋆
⋆ This work was supported by the project Probabilistic modeling of Wigren
Wigren
work (2015)
(2015)
considering there
there
the does
does
fact not
not
that seem
seem to
to
identification have
have been
been
may much
much
not be
dynamical
This worksystems
was (Contract
supported bynumber:
the 621-2013-5524)
project Probabilistic
⋆ This work was supported by the project Probabilistic modeling of funded by
modeling the
of Wigren
work (2015) there does not seem to have been much
dynamical
Swedish
dynamical
dynamical
systems
Research (Contract
systemsCouncil.
systems (Contract
(Contract (*)number: 621-2013-5524)
are members
number:
number: of the LCCC
621-2013-5524)
621-2013-5524)
funded by
Linnaeus
funded
funded by the
by the
the
work considering
possible
work considering
if only periodic
considering
the
the fact
the fact
factdatathat
that
that is identification
identification
available.
identification
may
may not
may not be
not be
be
Swedish
Center
Swedish Research
and the
Research Council.
ELLIIT
Council. (*)
Excellence
(*) are
are members
Center
members at of the
Lund
of the LCCC Linnaeus
University.
LCCC Linnaeus possible
possible if
if only
only periodic
periodic data
data is
is available.
available.
Swedish Research Council. (*) are members
Center and the ELLIIT Excellence Center at Lund University. of the LCCC Linnaeus possible if only periodic data is available.
Center
Center and
and the
the ELLIIT
ELLIIT Excellence
Excellence Center
Center at
at Lund
Lund University.
University.
Copyright IFAC 2015
2405-8963 © 2015, 415 Hosting by Elsevier Ltd. All rights reserved.
IFAC (International Federation of Automatic Control)
Copyright
Peer review©
Copyright © IFAC
IFAC 2015
2015 415
415
Copyright ©under
IFAC responsibility
2015 of International Federation of Automatic
415Control.
10.1016/j.ifacol.2015.12.163
2015 IFAC SYSID
416
October 19-21, 2015. Beijing, China Jerker Nordh et al. / IFAC-PapersOnLine 48-28 (2015) 415–420

The particle filter was introduced more that two decades
ago as a solution to the nonlinear state estimation prob-
lem, see e.g. Doucet and Johansen (2011) for an introduc-
tion. However, when it comes to the nonlinear system iden-
tification problem in general, it is only relatively recently
that the particle filter has emerged as a really useful tool,
see Kantas et al. (2014) for a recent survey. The algorithm
presented here is an adaptation of the so-called PSAEM
algorithm introduced by Lindsten (2013). It provides a
solution to the nonlinear maximum likelihood problem by
combining the stochastic approximation expectation max-
imization algorithm of Delyon et al. (1999) with the PGAS
kernel of Lindsten et al. (2014). This improves upon the
earlier work of Wigren et al. (2003b), since it is no longer
necessary to rely on the sub-optimal extended Kalman
filter and restrictive models of the model parameters.
2. FORMULATING THE MODEL AND THE
PROBLEM
The periodic signal is modeled as the output of a second
order differential equation, and can in continuous-time
thus be represented as
xt = (pt vt )T , (1a)
 
vt
ẋt = . (1b)
f (pt , vt )
The discretized model is obtained by a Euler forward
approximation and by introducing noise acting on the
measurements and on the second state to describe the
model errors
pt+1 = pt + hvt , (2a)
vt+1 = vt + hf (pt , vt ) + wt (2b)
yt = pt + e t . (2c)
The noise is only acting on one of the states, implying
that one of the states can be marginalized. The previous
choice of Euler forward gives a model where the required
non-Markovian smoothing densities can be evaluated in
constant time, for many formulations this does not hold
and the time-complexity instead grows with the size of
the dataset. The marginalization, which will prove useful
later for the particle filter, results in the following non-
Markovian model
pt+1 = pt + h(vt−1 + hf (pt−1 , vt−1 ) + wt−1 )
 
2 pt − pt−1
= 2pt − pt−1 + h f pt−1 , + density (JSD) arbitrarily well according to
h
hwt−1 , (3a)
y t = pt + e t , (3b)
where the noise is assumed Gaussian according to
wt−1 ∼ N (0, Qw ), et ∼ N (0, R). (3c)
As suggested by Wigren et al. (2003b), the function f (p, v)
is parametrized according to
m  m
f (p, v) = aij pi v j . (3d)
i=0 j=0
Here, m is a design parameter deciding the degree of
the model used for the approximation and the indices
aij denote unknown parameters to be estimated together
with the process noise covariance Qw . Hence, the unknown
parameters to be estimated are given by
θ = {Qw a00 ... a0m ... am0 ... amm }. (4)
It has been assumed that the measurement noise covari-
ance R is known. The problem under consideration is that
of computing the maximum likelihood (ML) estimate of
the unknown parameters θ by solving
θML = argmax log pθ (y1:T ), (5)
θ
where y1:T = {y1 , . . . , yT } and pθ (y1:T ) denotes the likeli-
hood function parameterized by θ.
3. PARTICLE FILTERING FOR AUTONOMOUS
SYSTEM IDENTIFICATION
After the marginalization of the v state in model (2) the
problem becomes non-Markovian, for an introduction to
non-Markovian particle methods see e.g., Lindsten et al.
(2014) and Lindsten and Schön (2013). The remainder of
this section will go through the components required for
the algorithm and note specific design choices made to
apply the methods to the particular class of problems that
are of interest in this paper.
3.1 Expectation maximization algorithms
The expectation maximization (EM) algorithm (Dempster
et al., 1977) is an iterative algorithm to compute ML
estimates of unknown parameters (here θ) in probabilistic
models involving latent variables (here, the state trajec-
tory x1:T ). More specifically, the EM algorithm solves the
ML problem (5) by iteratively computing the so-called
intermediate quantity

Q(θ, θk ) = log pθ (x1:T , y1:T )pθk (x1:T | y1:T )dx1:T (6)
and then maximizing Q(θ, θk ) w.r.t. θ. There is now
a good understanding of how to make use of EM-type
algorithms to identify dynamical systems. The linear state
space model allows us to express everything in closed form
(Shumway and Stoffer, 1982; Gibson and Ninness, 2005).
However, when it comes to nonlinear models, like the ones
considered here, approximate methods have to be used,
see e.g. (Lindsten, 2013; Schön et al., 2011; Cappé et al.,
2005).
The sequential Monte Carlo (SMC) methods (Doucet and
Johansen, 2011) or the particle Markov chain Monte Carlo
(PMCMC) methods introduced by Andrieu et al. (2010)
can be exploited to approximate the joint smoothing
N
p(x1:T | y1:T ) = wTi δxi1:T (x1:T ). (7)
i=1
Here, xi1:T denotes the i:th sample of x1:T , the notation
x1:T is used for the collection (x1 , ..., xT ). wTi denotes the
weights given to the i:th sample and δx denotes a point-
mass distribution at x. Schön et al. (2011) used the SMC
approximation (7) to approximate the intermediate quan-
tity (6). However, there is room to make even more efficient
use of the particles in performing ML identification, by
making use of the stochastic approximation developments
within EM according to Delyon et al. (1999). In the so-
called stochastic approximation expectation maximization
(SAEM) algorithm, the intermediate quantity (6) is re-
placed by the following stochastic approximation update
k (θ) = (1 − γk )Q
Q k−1 (θ) + γk log pθ (x1:T [k], y1:T ), (8)

416
 2015 IFAC SYSID
October 19-21, 2015. Beijing, China Jerker Nordh et al. / IFAC-PapersOnLine 48-28 (2015) 415–420 417

where γk denotes the step ∞size, which is a design pa- system {xi1:T , wTi }N
i=1 that is then used to approximate the
∞
rameter that must fulfill k=1 γk = ∞ and k=1 γk2 < intermediate quantity according to
∞. Furthermore, x1:T [k] denotes a sample from the JSD N

pθk (x1:T | y1:T ). The sequence θk generated by the SAEM k (θ) = (1 − γk )Q
Q k−1 (θ) + γk wTi log pθ (xi1:T , y1:T ).
algorithm outlined above will under fairly weak assump- i=1
tions converge to a maximizer of pθ (y1:T ) (Delyon et al., (11)
1999).
Note that similarly to (8) only the reference trajectory
For the problem under consideration the recently devel- x1:T [k] could have been used, but in making use of the
oped PMCMC methods (Andrieu et al., 2010; Lindsten entire particle system the variance of the resulting state
et al., 2014) are useful to approximately generate samples estimates are reduced (Lindsten, 2013). The result is
from the JSD. This was realized by Lindsten (2013), result- provided in Algorithm 2.
ing in the so-called particle SAEM (PSAEM) algorithm,
which is used in this work. Algorithm 2 PSAEM for sys. id. of autonomous systems
1: Initialization: Set θ[0] = (Qw
0 0T ) and set x1:T [0] using an
3.2 The PGAS kernel FFBSi particle smoother. Set Q 0 = 0 and set w[0] to an empty
vector.
2: Draw x′1:T using FFBSi.
The particle Gibbs with ancestor sampling (PGAS) kernel 3: for k ≥ 1 do
was introduced by Lindsten et al. (2014). It is a procedure 4: Draw x1:T [k], wT by running Algorithm 1 using x′1:T as
very similar to the standard particle filter, save for the fact reference.
i .
that conditioning on one so-called reference trajectory x′1:T 5: Draw j with P (j = i) = wT
j
is performed. Hence, x′1:T have to be retained throughout 6: ′
Set x1:T = x1:T [k]
the sampling procedure. For a detailed derivation see Lind- 7: Set w[k] = ((1 − γk )w[k − 1] γk wT )
sten et al. (2014), where it is also shown that the PGAS 8: Compute Q k (θ) according to (11).
kernel implicitly defined via Algorithm 1 is uniformly 9: Compute θ[k] = argmax Q k (θ).
ergodic. Importantly, it also leaves the target density 10: if termination criterion is met then
p(x1:T | y1:T ) invariant for any finite number of particles 11: return {θ[k]}
N > 1 implying that the resulting state trajectory x⋆1:T 12: end if
13: end for
can be used as a sample from the JSD. The notation used
in Algorithm 1 is as follows, xt = (x1t , . . . , xN t ) denotes all Note that the initial reference trajectory x1:T [0] is obtained
the particles at time t and x1:T = (x1 , . . . , xT ) the entire
by running a so-called forward filter backward simula-
trajectories. The particles are propagated according to a
tor (FFBSi) particle smoother, see Lindsten and Schön
proposal distribution rt (xt | xt−1 , yt ). The resampling step
(2013) for details. To indicate that the trajectories were
and the propagation step of the standard particle filter has
generated at iteration k, we use x1:T [k] and analogously
been collapsed into jointly sampling the particles {xit }N i=1 for the weights. The 9th row of Algorithm 2 will for the
and the ancestor indices {ait }N i=1 independently from model (3) under consideration amount to a weighted least
w at squares problem, which is solved in Algorithm 3. For the
Mt (at , xt ) = N t rt (xt | xa1:t−1
t
, y1:t ). (9) work presented in this article the termination criterion of
w l
l=1 t
Algorithm 2 is chosen as a fixed number of iterations.
Finally, Wt denotes the weight function,
p(yt | x1:t )p(xt | x1:t−1 ) Algorithm 3 Maximizing Q
Wt (x1:t , y1:t ) = . (10) 1: For each trajectory in x1:T [k] calculate the velocity at each time
r(xt | x1:t−1 , y1:t ) (i) (i) (i)
vt = (pt+1 − pt )/h
2: For each time step and for each trajectory in x1:T [k], evaluate
Algorithm 1 PGAS kernel (i) (i)
f (pt , vt ).
1: Initialization (t = 1): Draw xi1 ∼ r1 (x1 |y1 ) for i = 1, . . . , N − 3: Find a00 ...amm via the related weighted least squares (WLS)
1 and set xN ′ i i problem.
1 = x1 . Compute w1 = W1 (x1 ) for i = 1, . . . , N .
2: for t = 2 to T do 4: Find Qw by estimating the covariance of the residuals of the
3: Draw {ait , xit } ∼ Mt (at , xt ) for i = 1, . . . , N − 1. WLS-problem.
4: Set xN
t = xt .
′ 5: Set θ[k] = {Qw a00 ... amm }.
  wi p(x′t | xi1:t−1 )
5: Draw aN N
t with P at = i ∝ N t−1l
wt−1 p(x′t | xl1:t−1 )
ait
l=1
3.4 Choosing the proposal distribution
6: Set xi1:t = {x1:t−1 , xit } for i = 1, . . . , N .
7: Compute wti = Wt (xi1:t , y1:t ) for i = 1, . . . , N . The proposal density r(xt |x1:t−1 , y1:t ) constitutes an im-
8: end for portant design choice of the particle filter that will signifi-
9: Return x1:T , wT .
cantly affect its performance. The commonly used boot-
strap particle filter amounts to making use of the dy-
namics to propose new particles, i.e. r(xt |x1:t−1 , y1:t ) =
3.3 Identifying autonomous systems using PSAEM p(xt | x1:t−1 ). However, we will make use of the measure-
ment model and the information present in the current
The PSAEM algorithm for ML identification of au- measurement to propose new particles, i.e. r(xt |x1:t−1 , y1:t )
tonomous systems now simply amounts to making use of = p(yt | xt ). This is enabled by the marginalization of the
the PGAS kernel in Algorithm 1 to generate a particle deterministic state in the model, since the dimension of

417
2015 IFAC SYSID
418
October 19-21, 2015. Beijing, China Jerker Nordh et al. / IFAC-PapersOnLine 48-28 (2015) 415–420

the state-space then matches that of the measurement. 2.5

As a possible further improvement, the proposal could
2
also include the predicted state using the model. Recently,

Parameter value
Kronander and Schön (2014) showed that the combined 1.5
use of both the dynamics and the measurements results in 1
competitive algorithms. In such a scenario the estimated
uncertainty in the model would initially be large and thus 0.5
not affect the proposal distribution significantly, but for 0
each iteration of the PSAEM algorithm the model will be
more and more accurate in predicting the future states, −0.5
and its influence on the proposal distribution would in- −1
crease accordingly.
−1.5
4. NUMERICAL ILLUSTRATIONS −2

−2.5
The performance of the proposed algorithm is illustrated
using two examples, namely the Van der Pol oscillator −3
0 5 10 15 20 25 30 35 40 45 50
in Section 4.1 and the Hodgkin-Huxley neuron model in iteration
Section 4.2 – 4.3. There is no prior knowledge of the model,
Fig. 1. Parameter convergence for the Van der Pol exam-
hence it is assumed that all the parameters aij in (3d) are
ple (12). The dashed lines are the coefficients obtained
zero. The initial covariance Qw 0 is set to a large value. from the discretization of the model, the other lines
The Python source code for the following examples can be
represent the different parameters in the identified
downloaded from Nordh (2015a), the implementation was
model. Note that while the parameters do not con-
carried out using the pyParticleEst software framework
verge to the ’true’ values, they provide a very accurate
(Nordh, 2015b, 2013).
model for predicting the signal as shown in Fig. 3.
4.1 Validation on the Van der Pol oscillator
True
Predicted
To demonstrate the validity of the proposed solution it is 4 Measurements

first applied to the classic Van der Pol oscillator (Khalil, 3

1996), which belongs to the model class defined by (3).
2
The oscillator is described by
 
vt 1
ẋt = , (12)
−pt + 2(1 − p2t )vt
v

0
T
where xt = (pt vt ) . Performing 50 iterations of Algo- −1
rithm 2 gives the parameter convergence shown in Fig. 1.
−2
Here N = 15 particles were used for the PGAS sampler,
and the model order was chosen as m = 2. The dataset −3
was sampled with h = 0.025s. For the SAEM step, the
−4
sequence γ1:K is chosen as

1 if k ≤ 5, −2 −1.5 −1 −0.5 0 0.5 1 1.5 2
γk = (13) p
(k − 5)−0.9 if k > 5.
Fig. 2. Phase-plane plot for the Van der Pol example (12).
It can be seen that the parameters converge to values close
to those obtained from the Euler forward discretization. under-modeling is therefore also assessed. Secondly, the
To analyze the behavior further the phase-plane for the new algorithm and the identification results contribute to
system is shown in Fig. 2 and the time-domain realization an enhanced understanding of spiking neurons by provid-
in Fig. 3. Here it can be seen that the identified model ing better performance compared to previous algorithms.
captures the behavior of the true continuous-time system
significantly better than the model obtained from the The Hodgkin-Huxley model formulation of Siciliano (2012)
discretization. is used, where the following ODE is given

4.2 The Hodgkin-Huxley neuron model dv 1 

= I − gna m3 h(v − Ena )
dt Cm
The well-known Hodgkin-Huxley model uses a nonlinear 
×gK n4 (v − EK ) − gl (v − El ) , (14a)
ODE to describe the dynamics of the action potentials in
a neuron. In this paper the model will be used for two dn
= αn (v)(1 − n) − βn (v)n, (14b)
purposes. First, simulated spiking neuron data is used to dt
characterize the performance of the proposed algorithm dm
= αm (v)(1 − m) − βm (v)m, (14c)
when identifying a nonlinear autonomous system. It should dt
be noted that the data does not correspond to a system dh
that is in the model set. The ability to handle nonlinear = αh (v)(1 − h) − βh (v)h. (14d)
dt

418
 2015 IFAC SYSID
October 19-21, 2015. Beijing, China Jerker Nordh et al. / IFAC-PapersOnLine 48-28 (2015) 415–420 419

2.5
True
Fig. 5 shows the same dataset in the time-domain, it
Predicted shows the true signal (without added noise) and the output
2 Discretized
predicted by the identified model when initialized with the
1.5 correct initial state. It can be seen that the model captures
the behaviour of the signal, but introduces a small error
1
in the frequency of the signal, causing the predicted and
0.5 true signal to diverge over time.
0
p

350 True
Predicted
−0.5 Measurements
300
−1
250
−1.5
200
−2

−2.5 150
0 200 400 600 800 1000 1200 1400 1600 1800 2000

v
t
100
Fig. 3. Predicted output using the true initial state and the
estimated parameters. The plot labeled ”discretized” 50
is obtained by discretization of the true continuous-
0
time model using Euler forward and using that to
predict the future output. It can been seen that the −50
discretized model diverges over time from the true
signal, clearly the identified parameter values give
−60 −40 −20 0 20 40 60
a better estimate than using the values from the p
discretization.
Fig. 4. Phase-plane for the Hodgkin-Huxley dataset. It
Here, v denotes the potential, while n, m and h relate to can be seen that the predicted model fails to capture
each type of gate of the model and their probabilities of the initial transient, but accurately captures the limit
being open, see Siciliano (2012) for details. The applied cycle.
current is denoted by I. The six rate variables are de-
scribed by the following nonlinear functions of v 60
True
Predicted

0.01(v + 50) 40
αn (v) = , (15a)
1 − e−(v+50)/10
βn (v) = 0.125e−(v+60)/80, (15b) 20
0.1(v + 35)
αm (v) = , (15c) 0
1 − e−(v+35)/10
p

βm (v) = 4.0e−0.0556(v+60) , (15d)

−20
−0.05(v+60)
αh (v) = 0.07e , (15e)
1 −40
βh (v) = . (15f)
1+ e−0.1(v+30)
The corresponding numerical values are given by Cm = −60
0.01 µF/cm2 , gN a = 1.2 mS/cm2 , EN a = 55.17 mV ,
gK = 0.36 mS/cm2 , EK = −72.14 mV , gl = 0.003 −80
mS/cm2 , and El = −49.42 mV . 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
t

4.3 Identifying the spiking mode of the Hodgkin-Huxley Fig. 5. Predicted output using the true initial state and the
model estimated parameters. The overall behaviour of the
signal is captured, but there is a small frequency error
which leads to the predicted signal slowly diverging
Using a simulated dataset of length T = 10 000 sampled
from the true signal.
with h = 0.04s, a model of the form (3) with m = 3 is
selected. Algorithm 2 was run for 200 iterations, employing
N = 20 particles in the PGAS kernel. For the SAEM step, 5. CONCLUSIONS
the sequence γ1:K is chosen as
 The new identification method successfully identified mod-
1 if k ≤ 40,
γk = (16) els for both the Van der Pol example and for the more com-
(k − 40)−0.5 if k > 40. plex Hodgkin-Huxley model. Even though the Hodgkin-
The predicted phase-plane of the identified model is shown Huxley model in general cannot be reduced to a second
in Fig. 4 along with the true phase-plane and the measure- order differential equation it is possible to identify a good
ments. second order model for its limit cycle as shown in Fig. 4.

419
2015 IFAC SYSID
420
October 19-21, 2015. Beijing, China Jerker Nordh et al. / IFAC-PapersOnLine 48-28 (2015) 415–420
The use of a Bayesian nonparametric model in place
of (3d) constitutes an interesting continuation of this
work. A first step in this direction would be to employ
the Gaussian process construction by Frigola et al. (2013,
2014).
Another interesting topic for future research is the ques-
tion of asymptotic stability of the periodic solutions of the
estimated models. Based on the selected model structure,
this problem could be approached with standard tools
from nonlinear systems theory, including invariant sets and
Poincaré maps, see (Khalil, 1996, chapter 7.3).
Furthermore the authors see no reason that the method
could not be extended to identify the noise covariance as
well, but for the work done here it was left as a design
variable.
REFERENCES
Andrieu, C., Doucet, A., and Holenstein, R. (2010). Particle Markov
chain Monte Carlo methods. Journal of the Royal Statistical
Society, Series B, 72(2), 1–33.
Ashmore, P.G. (1975). Kinetics of Oscillatory Reactions. The
Chemical Society.
Cappé, O., Moulines, E., and Rydén, T. (2005). Inference in Hidden
Markov Models. Springer Series in Statistics. Springer, New York,
USA.
Delyon, B., Lavielle, M., and Moulines, E. (1999). Convergence of
a stochastic approximation version of the EM algorithm. The
Annals of Statistics, 27(1), 94–128.
Dempster, A.P., Laird, N.M., and Rubin, D.B. (1977). Maximum
likelihood from incomplete data via the EM algorithm. Journal
of the Royal Statistical Society B, 39(1), 1–38.
Doi, S., Onada, Y., and Kumagai, S. (2002). Parameter estimation of
various Hodgkin-Huxley-type neuronal models using a gradient-
descent learning method. In Proceedings of the 41st SICE Annual
Conference, 1685–1688. Osaka, Japan.
Doucet, A. and Johansen, A.M. (2011). A tutorial on particle filtering
and smoothing: Fifteen years later. In D. Crisan and B. Rozovsky
(eds.), Nonlinear Filtering Handbook. Oxford University Press.
Frigola, R., Lindsten, F., Schön, T.B., and Rasmussen, C.E. (2013).
Bayesian inference and learning in Gaussian process state-space
models with particle MCMC. In Advances in Neural Information
Processing Systems (NIPS) 26, 3156–3164.
Frigola, R., Lindsten, F., Schön, T.B., and Rasmussen, C.E. (2014).
Identification of Gaussian process state-space models with particle
stochastic approximation EM. In Proceedings of the 18th World
Congress of the International Federation of Automatic Control
(IFAC). Cape Town, South Africa.
Gibson, S. and Ninness, B. (2005). Robust maximum-likelihood
estimation of multivariable dynamic systems. Automatica, 41(10),
1667–1682.
Hodgkin, A.L. and Huxley, A.F. (1952). A quantitative description
of membrane current and its application to conduction and
excitation in nerve. The Journal of Physiology, 117(4), 500–544.
Izhikevich, E.M. (2003). Simple models of spiking neurons. IEEE
Transactions on Neural Networks, 14(6), 1569–1572.
Kantas, N., Doucet, A., Singh, S.S., Maciejowski, J., and Chopin, N.
(2014). On particle methods for parameter estimation in state-
space models. ArXiv:1412.8695, submitted to Statistical Science.
Khalil, H.K. (1996). Nonlinear Systems. Prentice Hall, Upper Saddle
River.
Kronander, J. and Schön, T.B. (2014). Robust auxiliary particle
filters using multiple importance sampling. In Proceedings of the
IEEE Statistical Signal Processing Workshop (SSP). Gold Coast,
Australia.
Lankarany, M., Zhu, W.P., and Swamy, N.S. (2013). Parameter es-
timation of Hodgkin-Huxley neuronal model using dual extended
Kalman filter. In Proceedings of the IEEE International Sym-
420
posium on Circuits and Systems (ISCAS), 2493–2496. Beijing,
China.
Li, H.Y., Che, Y.Q., Gao, H.Q., Dong, F., and Wang, J. (2007).
Bifurcation analysis of the Hodgkin-Huxley model exposed to
external DC electric field. In Proceedings of the 22nd International
Symposium on Intelligent Control (ISIC), 271–276. Singapore.
Lindsten, F. (2013). An efficient stochastic approximation EM
algorithm using conditional particle filters. In Proceedings of the
38th International Conference on Acoustics, Speech, and Signal
Processing (ICASSP). Vancouver, Canada.
Lindsten, F., Jordan, M.I., and Schön, T.B. (2014). Particle Gibbs
with ancestor sampling. Journal of Machine Learning Research
(JMLR), 15, 2145–2184.
Lindsten, F. and Schön, T.B. (2013). Backward simulation methods
for Monte Carlo statistical inference. Foundations and Trends in
Machine Learning, 6(1), 1–143.
Manchester, I., Tobenkin, M.M., and Wang, J. (2011). Identification
of nonlinear systems with stable oscillations. In Proceedings of the
50th IEEE Conference on Decision and Control and European
Control Conference (CDC-ECC), 5792–5797. Orlando, FL, USA.
Nordh, J. (2013). pyParticleEst software framework. URL
http://www.control.lth.se/Staff/JerkerNordh/
pyparticleest.html.
Nordh, J. (2015a). Example source code for ”Particle filtering
based identification for autonomous nonlinear ode models”. URL
http://www.control.lth.se/Staff/JerkerNordh/ode-id.html.
Nordh, J. (2015b). pyParticleEst: A Python framework for particle
based estimation methods. Journal of Statistical Software. Pro-
visionally accepted.
Rapp, P. (1987). Why are so many biological systems periodic?
Progress in Neurobiology, 29(3), 261–273.
Saggar, M., Mericli, T., Andoni, S., and Miikulainen, R. (2007).
System identification for the Hodgkin-Huxley model using arti-
ficial neural networks. In Proceedings of the International Joint
Conference on Neural Networks. Orlando, FL, USA.
Schön, T.B., Wills, A., and Ninness, B. (2011). System identification
of nonlinear state-space models. Automatica, 47(1), 39–49.
Shumway, R.H. and Stoffer, D.S. (1982). An approach to time series
smoothing and forecasting using the EM algorithm. Journal of
Time Series Analysis, 3(4), 253–264.
Siciliano, R. (2012). The Hodgkin–Huxley model – its extensions,
analysis and numerics. Technical report, Department of Mathe-
matics and Statistics, McGill University, Montreal, Canada.
Tobenkin, M.M., Manchester, I., Wang, J., Megretski, A., and
Tedrake, R. (2010). Convex optimization in identification of stable
non-linear state space models. In Proceedings of the 49th IEEE
Conference on Decision and Control (CDC), 7232–7237. Atlanta,
USA.
Wigren, T. (2014). Software for recursive identification of second
order autonomous systems. Technical report, Department of
Information Technologt, Uppsala University, Uppsala, Sweden.
URL http://www.it.uu.se/research/publications/
reports/2014-014/SWAutonomous.zip.
Wigren, T. (2015). Model order and identifiability of non-linear
biological systems in stable oscillation. IEEE/ACM Transactions
on Computational Biology and Bioinformatics. Accepted for
publication.
Wigren, T., Abd-Elrady, E., and Söderström, T. (2003a). Harmonic
signal analysis with Kalman filters using periodic orbits of nonlin-
ear ODEs. In Proceedings of the IEEE International Conference
on Acoustics, Speech, and Signal Processing (ICASSP), 669–672.
Hong Kong, PRC.
Wigren, T. and Söderström, T. (2005). A second order ODE is
sufficient for modeling of many periodic systems. International
Journal of Control, 77(13), 982–986.
Wigren, T., Abd-Elrady, E., and Söderström, T. (2003b). Least
squares harmonic signal analysis using periodic orbits of ODEs.
In Proceeding of 13th IFAC symposium on system identification
(SYSID), 1584–1589. Rotterdam, The Netherlands.