This article was downloaded by: [Case Western Reserve University]

On: 13 October 2014, At: 14:43

Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office:
Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal of Control

Publication details, including instructions for authors and subscription
information:
http://www.tandfonline.com/loi/tcon20

Identification of non-linear stochastic systems

by state dependent parameter estimation
Peter C. Young , Paul McKenna & John Bruun
Published online: 08 Nov 2010.

To cite this article: Peter C. Young , Paul McKenna & John Bruun (2001) Identification of non-linear stochastic
systems by state dependent parameter estimation, International Journal of Control, 74:18, 1837-1857, DOI:
10.1080/00207170110089824

To link to this article: http://dx.doi.org/10.1080/00207170110089824

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”)
contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors
make no representations or warranties whatsoever as to the accuracy, completeness, or suitability
for any purpose of the Content. Any opinions and views expressed in this publication are the opinions
and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of
the Content should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings,
demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising
directly or indirectly in connection with, in relation to or arising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any substantial
or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or
distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can
be found at http://www.tandfonline.com/page/terms-and-conditions
 INT. J. CONTROL, 2001 , VOL. 74, NO. 18, 1837 ± 1857

Identi®cation of non-linear stochastic systems by state dependent parameter estimation

PETER C. YOUNG{*, PAUL MCKENNA{ and JOHN BRUUN{{

The paper outlines how improved estimates of time variable parameters in models of stochastic dynamic systems can be
obtained using recursive ®ltering and ®xed interval smoothing techniques, with the associated hyper-parameters opti-
mized by maximum likelihood based on prediction error decomposition. It then shows how, by exploiting special data re-
ordering and back-®tting procedures, similar recursive parameter estimation techniques can be utilized to estimate much
more rapid State Dependent Parameter (SDP) variations. In this manner, it is possible to identify and estimate a widely
applicable class of nonlinear stochastic systems, as illustrated by several examples that include simulated and real data
from chaotic processes. Finally, the paper points out that such SDP models can form the basis for new methods of signal
processing, automatic control and state estimation for nonlinear stochastic systems.
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014

1. Introduction ministic di€ erential equation models. They employed

Previous publications (e.g. Young 1988, 1989, 1998,
1999, 2000, 2001a) have discussed an approach to non-
stationary and non-linear signal processing based on the
identi®cation and estimation of stochastic models with
time variable (TVP) or state dependent (SDP) par-
ameters. Here the term `non-stationary’ is assumed to
mean that the statistical properties of the signal, as
de®ned by the parameters in an associated stochastic
model, are changing over time at a rate which is `slow’
in relation to the rates of change of the stochastic state
variables in the system under study. Although such non-
stationary systems exhibit fairly complex behaviour, this
can often be approximated well by TVP (or piece-wise
linear) models, the parameters of which can be estimated
using recursive methods of estimation in which the par-
ameters are assumed to evolve in a simple stochastic
manner (e.g. Young 1984, 1999 and the prior references
therein). On the other hand, if the changes in the par-
ameters are functions of the state or input variables (i.e.
the parameters and/or their changes actually constitute
stochastic state variables), then the system is truly non-
linear and likely to exhibit severe non-linear behaviour.
Normally, this cannot be approximated in a simple TVP
manner; in which case, recourse must be made to the
alternative, and more powerful SDP modelling methods
that are the main topic of this paper.
The extension of the TVP estimation methods to
allow for state dependency can be traced back to a little
known Conference paper by Hoberock and Kohr
(1966), who used it in connection with simulated deter-
Received 20 October 1999. Revised 16 October 2000.
* Author for correspondence. e-mail: p.young@lancaster.
ac.uk. Also, Centre for Resource and Environmental Studies,
Australian National University, Canberra, ACT 0200,
Australia.
{ Systems and Control Group, Centre for Research on
Environmental Systems and Statistics, Lancaster University,
Lancaster LA1 4YQ, UK.
{ Now with Unilever Research PLC at Port Sunlight, UK.
a continuous-time steepest descent algorithm, imple-
mented within an analog computer, to estimate the
parameters and, as a result, their method is rather
sensitive to noise on the data. As far as the ®rst author
is aware, the idea of SDP modelling, within a more
robust stochastic setting, exploiting recursive estima-
tion, was originated in Young (1969) and Mendel
(1969).
They enhanced recursive estimation performance by
assuming that the model parameters could vary because
of their dependence on the variations in other measured
variables. Young (1978) then explored these ideas within
a broader SDP setting and Priestley (1988) took them up
in a series of papers and a book on the subject. These
earlier publications do not, however, exploit the power
of recursive ®xed interval smoothing (FIS), which pro-
vides the main engine for the developments described in
the present paper (Young, 1993).
SDP estimation, as proposed here, involves the non-
parametric identi®cation of the state dependency using
recursive methods of time variable parameter estimation
which allow for rapid (state dependent) parametric
change. As we shall see, the standard methods of TVP
estimation developed previously for non-stationar y time
series analysis need to be modi®ed considerably in this
SDP setting to allow for the much more rapid temporal
changes that arise from the state dependency. This pro-
cess exploits recursive Fixed Interval Smoothing (FIS)
algorithms, combined with special data re-ordering
and `back-®tting’ procedures, to obtain estimates of
any state dependent parameter variations. These state
dependencies are estimated in the form of non-para-
metric relationships (graphs) between the estimated
rapid parameter variation and the associated state or
input variable(s). Parameterization of these non-para-
metric relationships can be accomplished in various
ways, from simple curve ®tting based on weighted
least squares methods to the use of neural/neuro-fuzzy
networks, or radial basis functions.

International Journal of Control ISSN 0020±7179 print/ISSN 1366±5820 online # 2001 Taylor & Francis Ltd
http://www.tandf.co.uk/journals
DOI: 10.1080/00207170110089824
 1838 P. C. Young et al.
Having identi®ed a structural form for the non-
linear model of the system based on the parameterized
non-linear relationships, this model is converted into a
stochastic state space form. The ®nal constant par-
ameter estimation phase of the non-linear modelling
then exploits some method of non-linear optimization,
such as least squares or Maximum Likelihood (ML)
methods of estimation based on Gaussian assumptions
for the stochastic disturbances and the application of
Prediction Error Decomposition (Schweppe, 1965). The
resulting model should then provide a parametrically
e cient representation of the stochastic, non-linear
system that has considerable potential for use in sub-
sequent signal processing, time series analysis, fore-
casting and automatic control system design. For
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
example, the methodology described here exploits recur-
sive estimation in an o€ -line manner but this sequential
processing of the data facilitates the development of
related on-line adaptive methods of signal processing,
forecasting and control.
In order to illustrate the practical application and
utility of the SDP approach, the paper also contains a
number of simulation examples, as well as a practical
study involving an analysis of electrical signals obtained
from experiments carried out on the axon of a squid.
Other simulation and practical examples cited in the
references cover a variety of application areas from the
environment through engineering to economics.
2. Identi®cation of time variable parameter and
nonlinear input-outpu t systems
A previous paper (Young 1999) has discussed the
estimation of time variable parameters in the various
kinds of `linear’ regression model. One of these,
the Dynamic{ Auto-Regressive eXogenous variables
(DARX) model is capable of modelling the input±
output behaviour of non-stationar y stochastic, dynamic
systems. While the DARX model can produce fairly
complex response characteristics, it is only when the par-
ameters are functions of the system variables, and so
vary at a rate commensurate with these variables, that
the resultant model can behave in a heavily non-linear
or even chaotic manner. We will refer to a State
Dependent Parameter (SDP) model of this type as a
State Dependent Parameter ARX (SDARX) model.
In its simplest single input, single output form, the
SDARX model equation can be written most conveni-
ently in the form
yt ˆ zTt pt ‡ et et ˆ N…0; ¼2 † …1†
where,
{ The term `dynamic’ is used here for historical reasons (see
reference) to mean a time variable parameter ARX model.
9
zTt ˆ ‰¡yt¡1 ¡ yt¡2 ¡ yt¡n ut¡¯ ut¡¯¡mŠ =
pt ˆ ‰a1 …v t †a2…v t † an …v t †b0 …v t † bm…v t †ŠT ;
…1 a†
while ai …v t †, i ˆ 1; 2; . . . ; n and bj …v t †, j ˆ 0; 1; . . . ; m
are the state dependent parameters, which are
assumed to be functions of one of the variables in
a non-minimal state vector v Tt ˆ ‰zTt UTt Š. Here
Ut ˆ ‰U1;t U2;t U r;t ŠT is a vector of other variables
that may a€ ect the relationship between these two pri-
mary variables (see Young 1969, 1993, Young and
Beven 1994). The pure time delay ¯, measured in sam-
pling intervals, is introduced to allow for any temporal
delay that may occur between the incidence of a change
in the input ut and its ®rst e€ ect on the output yt ; and et
is a zero mean, white noise input with Gaussian normal
amplitude distribution and variance ¼2 (although this
assumption is not essential to the practical application
of the resulting estimation algorithms). Finally, for con-
venience of notation, let pt be de®ned as
pt ˆ ‰ p1;t p2;t pn‡m‡1;t ŠT …1 b†
with pi;t , i ˆ 1; 2; . . . ; n ‡ m ‡ 1, relating to the ai …v t †
and bj …v t † through (1a).
A typical example of a SDARX model is the forced
logistic equation
yt ˆ ¬yt¡1 ¡ ¬y2t¡1 ‡ ut ‡ et et ˆ N…0; ¼2 † …2†
or
yt ˆ a1 … yt¡1 † yt¡1 ‡ b0 …ut † ut ‡ et …2 a†
where
a1 … yt¡1 † ˆ ¬ ¡ ¬yt¡1 b0 …ut † ˆ 1:0 8t …2 b†
As we shall see, although it is simple, this model can
exhibit rich behavioural patterns: from simple logistic
growth to chaotic response.
2.1. Time variable parameter estimation
Before considering the full SDP model (1), it is
instructive to ®rst deal with the simpler situation
where the parameters in pt are slowly variable with
time. In order to estimate these time variable par-
ameters, it is necessary to make some assumptions
about the nature of their temporal variability.
Re¯ecting the statistical setting of the analysis and refer-
ring to previous research on this topic, it seems desirable
if this is characterized in some stochastic manner.
Normally, when little is known about the nature of the
time variability, this model needs to be both simple and
¯exible. One of the simplest and most generally useful
class of stochastic, state space models involves the
assumption that the ith parameter, pi;t ,
i ˆ 1; 2; . . . ; n ‡ m ‡ 1; is de®ned by a two-dimensional
stochastic state vector xi;t ˆ ‰li;t di;t ŠT , where li;t and di;t
 Identi®cation of non-linear stochastic systems
are, respectively, the changing level and slope of the
associated TVP. This selection of a two-dimensional
state representation of the TVPs is based on practical
experience over a number of years. Initial research in the
1960s (e.g. Lee 1964, Young 1969, 1970, and many
others) tended to use a simple scalar random walk
(RW) model for the parameter variations . Later research
in the 1970s and 1980s (see Norton 1976, Jakeman and
Young 1979, 1984) showed the value of modelling not
only the level changes in the TVPs but also their rates of
change, as in the de®nition of xi;t , above.
The stochastic evolution of each xi;t (and, therefore,
each of the n ‡ m ‡ 1 parameters in pt ) is assumed to
be described by a generic Generalized Random Walk
(GRW: Jakeman and Young 1979, 1984) process
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
de®ned in the State Space (SS) terms
xi;t ˆ Fi xi;t¡1 ‡ Gi gi;t i ˆ 1; 2; . . . ; m ‡ n ‡ 1
where
µ ¶ µ ¶
¬ ­ ¯ 0
Fi ˆ ; Gi ˆ
0 ® 0 "
T
and gi;t ˆ ‰²1i;t ²2i;t Š is a 2 1, zero mean, white noise
vector that allows for stochastic variability in the
parameters and is assumed to be characterized by a
(normally diagonal) covariance matrix Q ²i . This is, of
course, a generic model formulated in this manner only
to unify various random walk-type models: it is never
used in its entirety since it is clearly over-parameterized .
As such, it comprises as special cases the Integrated
Random Walk (IRW: ¬ ˆ ­ ˆ ® ˆ " ˆ 1; ¯ ˆ 0); the
scalar Random Walk (RW: scalar but equivalent to (3)
with ­ ˆ ® ˆ " ˆ 0; ¬ ˆ ¯ ˆ 1: i.e. just the ®rst equation
in (3), see below); the intermediate case of Smoothed
Random Walk (SRW: 0 < ¬ < 1; ­ ˆ ® ˆ " ˆ 1; and
¯ ˆ 0); the ®rst order autoregressive process (AR(1):
again scalar with ­ ˆ ® ˆ " ˆ 0; 0 < ¬ < 1; ¯ ˆ 1);
and, ®nally, both the Local Linear Trend (LLT:
¬ ˆ ­ ˆ ® ˆ " ˆ ¯ ˆ 1){; and Damped Trend (DT:
¬ ˆ ­ ˆ ¯ ˆ " ˆ 1; 0 < ® < 1): see Harvey (1984,
1989). The various, normally constant, parameters in
this GRW model (¬, ­ , ®, ¯, " and the elements of
Q²i ) are often referred to as hyper-parameters . This is
to di€ erentiate them from the TVPs that are the main
object of the estimation analysis. However, the hyper-
parameters are also assumed to be unknown a priori
and need to be estimated from the data, as we shall
see in the subsequent discussion.
Note that, in the case of the RW model, i.e.
li;t ˆ li;t¡1 ‡ ²1i;t ; pi;t ˆ li;t
{ Interestingly, the GRW model with ¬ ˆ ® ˆ " ˆ 1 allows
any linear combination of RW and IRW models to be realised;
and the LLT model can be considered simply as one such
combination.
1839
each parameter can be assumed to be time-invarian t if
the variance of the white noise input ²1i;t is set to zero.
Then, the stochastic TVP setting reverts to the more
normal, constant parameter TF model situation. In
other words, if RW models with zero variance white
noise inputs are speci®ed for the model parameters,
then the recursive estimation algorithm described
below for the general stochastic TVP case will provide
recursive estimates that are identical to those obtained
with the normal recursive estimation algorithm for ARX
models with constant parameters (e.g. Young 1984). Of
course, there is some added value to the recursive sol-
ution even in this situation, since the user is provided
with the recursive estimates over the whole interval
t ˆ 1; 2; . . . ; N. These can provide additional useful
information on the model: for example, they show
how the estimates are converging and can be used (e.g.
…3†
Brown et al. 1975) to detect both the presence of poten-
tial parametric change and possible over-parameteriza -
tion (i.e. the model contains too many parameters to
provide unambiguous estimation results).
Clearly other, more general and higher order stoch-
astic processes could be used to model the stochastic
TVPs, provided such models can be identi®ed satisfac-
torily from the data. For example the higher order
IRWs (Double and Triple Integrated Random Walk
(DIRW, TIRW), etc.), the Integrated or Double
Integrated AutoRegressive (IAR, DIAR: see Young
1994) model, and even more general processes.
However, the more complex models introduce addi-
tional hyper-parameter s that would have to be well iden-
ti®ed from the data and optimized, thus introducing
potential practical di culties.
The idea of assuming that the model parameters
evolve over time as non-stationar y stochastic variables
may seem complex at ®rst sight but it is, in fact, just a
statistical device to allow for the estimation of para-
metric change. After all, the assumption of the RW
model is simply a means of introducing into the estima-
tion problem the freedom for the associated parameter
to vary at each sample in time by a small random
amount de®ned by the variance of the white noise
input ²1i;t . And the more complex GRW models in (3)
are just a way of re®ning and adding to this freedom. In
fact, it can be shown (Young and Pedregal 1998) that
the GRW assumptions on the parameter variations have
an implicit but physically interpretable e€ ect when used
with ®xed interval smoothing algorithms (see later): they
make the recursive parameter estimates, at any sample
time t, depend only on the local data in the vicinity of
…3 a†
this sample, with the selected GRW model de®ning the
local weighting e€ ect. In the case of the RW model, for
instance, this weighting e€ ect or `kernel’ has a Gaussian-
like shape that applies maximum weight to the current
data with declining weight each side. And the `band-
 1840 P. C. Young et al.

width’ of the kernel is de®ned by the ratio of the vari-
ance of the white noise input ²1i;t to the residual variance
¼2 (the Noise Variance Ratio (NVR): see later). This can
be related to the more explicit use of localized data
weighting in methods such as locally weighted kernel
regression (e.g. Holst et al. 1996), regularization
(Jakeman and Young 1979, 1984) and `wavelet’ methods
(e.g. Daubechies 1988).
Having introduced the GRW models for the par-
ameter variations, an overall SS model can then be con-
structed straightforwardl y by the aggregation of the
subsystem matrices de®ned in (3), with the `observation’
equation de®ned by the model equation (1): i.e.
¼ a result, the phase lag associated with the forward-pass
Observation equation: yt ˆ Ht xt ‡ et …i†
…4†
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
ithm, where the `®xed interval’ is the interval covered
by the total sample size N.
The reason for this two-pass approach is easy to
understand. The forward-pass ®ltering estimate of xt ,
which de®nes the estimated TVPs, can be denoted by
x
^ tjt (or simply x ^ t , for convenience) since it represents
the estimate at sample t given only the data up to and
including sampling instant t. However, under our
assumption that each of the parameters evolve stochas-
tically according to equation (3), a superior `smoothed’
estimate x ^ tjN , exists and can be generated by the FIS
algorithm, in which the estimate at t is based on all
the data over the sampling interval t ˆ 1; 2; . . . ; N. As
®ltering estimate (since it cannot anticipate any change

State equation: xt ˆ Fxt¡1 ‡ Ggt …ii† until the evidence for change in the series has been pro-
where cessed) is eliminated on the backward smoothing pass.
xt ˆ ‰xT1;t xT2;t xTn‡m‡1;t ŠT …4 a† Thus, any variation in the parameters is estimated as it
occurs, without any lag e€ ect (indeed, it may even be
If p ˆ 2…n ‡ m ‡ 1†, then F is a p p block diagonal anticipated if the smoothing e€ ect is substantial , as it
matrix with blocks de®ned by the Fi matrices in (3); G can be in high noise situations). This proves particularly
is a p p block diagonal matrix with blocks de®ned by useful in operations such as interpolation over gaps in the
the corresponding subsystem matrices Gi in (3); and gt is data, estimation and removal of individual components
a p-dimensional vector containing, in appropriate loca- from the data (signal extraction), and seasonal adjust-
tions, the white noise input vectors gi;t (`system disturb- ment (see Young and Pedregal 1998; Young et al. 1999).
ances’ in normal SS terminology) to each of the GRW In relation to the time series yt , t ˆ 1; 2; . . . ; N, the
models in (3). These white noise inputs, which provide recursive ®ltering/smoothing algorithm has the follow-
the stochastic stimulus for parametric change in the ing form.
model, are assumed to be independent of the observa-
tion noise et and have a covariance matrix Q formed 1. Forward pass recursive LS equations
from the combination of the individual covariance
Prediction:
matrices Q ²;i . Finally, Ht is a 1 p vector of the form
x
^ tjt¡1 ˆ F^
xt¡1
Ht ˆ ‰¡yt¡1 0 ¡ yt¡2 0
Ptjt¡1 ˆ FPt¡1 FT ‡ GQ r GT
¡ yt¡n 0 ut¡¯ 0 ut¡¯¡m 0Š …4 b† Correction:
that relates the scalar observation yt to the state vari- )
x ^tjt¡1 ‡ Ptjt¡1 HTt ‰1 ‡ Ht Ptjt¡1 HTt Š¡1 f yt ¡ Ht x
^t ˆ x ^ tjt¡1 g
ables de®ned by (4 a), so that it represents the model (1),
with each parameter de®ned as a GRW process. In the Pt ˆ Ptjt¡1 ¡ Ptjt¡1 HTt ‰1 ‡ Ht Ptjt¡1 HTt Š¡1 Ht Ptjt¡1
case of the scalar RW and AR(1) models, the alternate …5 a†
zeros are simply omitted.
The SS formulation in equations (4) is particularly
well suited for optimal recursive estimation in which the 2. Backward pass smoothing equations{
time variable parameters (acting as surrogate `states’ in 9
x xt‡1jN ‡ GQr GT Lt Š
^tjN ˆ F¡1 ‰^ >
>
this SS formulation) are estimated sequentially whilst >
>
Lt ˆ ‰I ¡ Pt‡1 HTt‡1 Ht‡1 ŠT >
>
working through the data in temporal order (usually >
=
termed `forward-pass ®ltering’). In the o€ -line situation, ‰FT Lt‡1 ¡ HTt‡1 f yt‡1 ¡ Ht‡1 x
^t‡1 gŠ …5 b†
where all the time series data are available for analysis, >
>
>
this ®ltering operation is accompanied by optimal recur- PtjN ˆ Pt ‡ Pt FT P¡1
t‡1jN ‰Pt‡1jN ¡ Pt‡1jt ŠP¡1 >
t‡1jt FPt >
>
>
;
sive smoothing (see e.g. Bryson and Ho 1969 and the LN ˆ 0
prior references therein; Norton 1986). Here, the
estimates obtained from the forward-pass, ®ltering { Note that here the F matrix needs to be non-singular,
algorithm are updated sequentially whilst working which is the case for the GRW type model. Also note that
an alternative FIS algorithm is available in which, at each
through the data in reverse temporal order (usually backwards recursion, the estimate x^tjN is based on an update
termed `backward-pass smoothing’) using a back- of the ®ltering estimate x
^ t (see Norton 1976, Young 1984).
wards-recursive Fixed Interval Smoothing (FIS) algor- This can be speci®ed as an alternative to (5 b).
 Identi®cation of non-linear stochastic systems
In these algorithms, the p p Noise Variance Ratio
(NVR) matrix Qr and the p p matrix Pt are de®ned as
Q Pt
Qr ˆ ; Pt ˆ
¼2 ¼2
where Pt is the error covariance matrix associated with
the state estimates which, in the present TVP context,
de®nes the estimated uncertainty in the parameters. For
simplicity, it is normally assumed that the NVR matrix
Qr is diagonal, although this is not essential. The NVR
parameters that characterize Qr (as well as any other
unknown hyper-parameter s in the SS model (4)) are
unknown prior to the analysis and clearly need to be
estimated on the basis of the time series data yt and ut
before the ®ltering and smoothing algorithms can be
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
utilized. The optimization of these hyper-parameter s is
discussed in the next sub-section.
2.2. Maximum likelihood (ML) optimization of hyper-
parameters
The approach to ML optimization based on
Prediction Error Decomposition (PED) derives originally
from the work of Schweppe (1965), who showed how to
generate likelihood functions for Gaussian signals using
the Kalman ®lter (see also Bryson and Ho 1969, p. 389).
Its importance in the present context was probably ®rst
recognized by Harvey (1981) and Kitagawa (1981) in
their development of `unobserved component’ forecast-
ing models. Since then, it has become one of the two
standard approaches to the problem (the other being
the Expectation and Minimization (EM) algorithm:
Dempster et al. 1977).
With given initial values for the hyper-parameters ,
the Kalman ®lter algorithm will yield the one-step-ahea d
prediction errors (also termed the `innovations’ or
`recursive residuals’) "t , where
" t ˆ y t ¡ Ht x
^tjt¡1 t ˆ 1; 2; . . . ; N
If the ®rst p observations are regarded as ®xed, the log-
likelihood function of yp‡1 ; . . . ; yN can be de®ned in
terms of the standard `regression’ form of prediction
error decomposition, i.e.
¡…N ¡ p† 1 latest selection of hyper-paramete r estimates made by
log L ˆ log …2º† ¡ log …¼2 †
2 2
1 XN
¡ log …1 ‡ H t Ptjt¡1 H Tt †
2 tˆp‡1
1 X N
"2t
¡ …7†
2¼2 tˆp‡1 1 ‡ Ht Ptjt¡1 HTt
where it can be shown that ¼2 …1 ‡ Ht Ptjt¡1 HTt † is the
variance of "t , so that the last term in (7) is based on
the sum of squares of the normalized one-step-ahea d
1841
prediction errors. Now the ML estimate of ¼2 , con-
ditional on the hyper-parameters , is given by
…5 c† 1 X N
"2t
¼^2 ˆ …8†
N ¡ p tˆp‡1 1 ‡ Ht Ptjt¡1 HTt
so that it can be estimated in this manner and `concen-
trated out’ of the expression (7) by substituting (8)
into (7), to yield the following expression for the `con-
centrated likelihood’
N ¡p
log …Lc † ˆ ¡ log …2º ‡ 1†
2
1 X N
¡ log …1 ‡ Ht Ptjt¡1 H Tt †
2 tˆp‡1
N¡p
¡ log …¼^2 † …9†
2
which needs to be maximized with respect to the
unknown hyper-parameter s in order to obtain their
ML estimates.
Since (9) is non-linear in the hyper-parameters , the
likelihood maximization needs to be carried out numeri-
cally. Consequently, it is more convenient to remove the
constant term (since it will play no part in the optimiza-
tion) and consider
1 X N
log …Lc † ˆ ¡ log …1 ‡ H t Ptjt¡1 H Tt †
2 tˆp‡1
N¡p
‡ log …¼^2 † …10†
2
which can then be minimized if it is multiplied by
¡1. This minimization is accomplished by initiating
the optimization with the hyper-paramete r estimates
…6† either selected by the user or set to some default values
(in both cases, ensuring that the resulting optimization
does not converge on a local minimum). The recursive
TVP estimation algorithm is used repeatedly to generate
the one step ahead prediction errors "t and, thence,
the log-likelihood value in (10) associated with the
the optimization algorithm. The optimization algorithm
then adjusts its selection of hyper-paramete r estimates
in order to converge on those estimates that minimize
(10). Typical methods that can be used for numerical
optimization are the `fmins’ and `fminu’ functions
available in the Matlab software system, or their
equivalents, although more complex and e cient
procedures are available. Further details of this and
alternative ML optimization procedures are given, for
example, in Harvey (1989) and Harvey and Peters
(1990).
 1842 P. C. Young et al.
2.3. Full state dependent parameter estimation
Since the parameter vector pt is potentially state-
dependent, it may vary at a rate commensurate with
the temporal variations in yt , ut and Ui;t , and so it
cannot be assumed that the simple GRW model (3)
is appropriate to describe the parametric variation
over time. At ®rst sight, it would appear that the stoch-
astic state space model should include prior informa-
tion on the nature of the parameter variation if the
TVP estimation methodology discussed in previous
sections is to work satisfactorily. Fortunately, it is
possible to remove this requirement if we resort to
the rather unusual procedure, at least within a time
series context, of sorting the data in a non-temporal
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
order. Then, if the ordering is chosen so that the SDP
variations associated with the sorted series are
smoother and less rapid, it is more likely that a
simple GRW process can be utilized to describe their
evolution.
For example, if the time series are sorted in some
common `ascending order of magnitude’ manner (i.e.
the sort operation in Matlab), then the rapid natural
variations in yt and ut are e€ ectively eliminated from
the data and replaced, in the sorted data space, by
much smoother and less rapid variations. And if the
SDPs are, indeed, related to these variables, then
they will be similarly a€ ected by the sorting. Follow-
ing FIS estimation, however, these SDP estimates
can be `unsorted’ (a trivial unsort operation to reverse
Matlab’s sort) and their true, rapid variation will
become apparent. Of course, the nature of the sorting
will a€ ect the estimation and it seems likely that there
will be an optimum sorting which results in minimum
variance estimates. However, such optimum sorting will
naturally depend upon the nature of the state depen-
dency and its de®nition would require some sort of itera-
tive estimation procedure. In practical terms, therefore,
the common ascending order sorting and un-sorting
operations seem the most straightforward and will be
utilized here.
One obvious requirement with this new approach to
SDP estimation is that the sorting of data, prior to FIS
estimation, must be common to all of the variables in the
relationship (1). If a single, ascending order strategy is
selected, therefore, it is necessary to decide upon which
variable in the model the sorting should be based. The
simplest strategy is to sort according to the ascending
order of the `dependent’ variable yt . Depending upon
the nature of each SDP in the vector pt , however, a
single variable sorting strategy, such of this, may not
produce satisfactory results. If this is the case, then a
more complicated, but still straightforward, `back®tting’
procedure can be exploited. Here, each parameter is
estimated in turn, based on the `modi®ed dependent
variable’ (MDV) series obtained by subtracting all the
other terms on the right-hand side of (1) from yt , using
the values of the other parameter estimates from the
previous iteration. At each such back®tting iteration,
the sorting can then be based on the single variable
associated with the current SDP being estimated.
Since the SDP estimates resulting from this back®t-
ting algorithm are themselves time series, it will be noted
that the algorithm constitutes a special form of non-
parametric estimation and, as such, can be compared
with other non-parametri c methods, such as the
Generalized Additive Modelling (GAM) approach of
Hastie and Tibshirani (1996). However, in both concep-
tual and algorithmic terms, the SDP approach described
here is signi®cantly di€ erent from this earlier approach
and seems more appropriate to the estimation of non-
linear, stochastic, dynamic models. Moreover, the recur-
sive methodology, on which SDP estimation is based, is
couched in optimal maximum likelihood terms that
seem more elegant and ¯exible than the scatter-plot
smoothing procedures used by Hastie, Tibshirani and
others.
The back®tting algorithm for the SDARX model (1)
takes the following form.
Back®tting algorithm for SDP models
Step 1. Assume that FIS estimation has yielded prior
TVP estimates p^0i;tjN ; i ˆ 1; 2; . . . ; m ‡ n ‡ 1 of
the SDPs.{
Step 2. Iterate: i ˆ 1; 2; . . . ; m ‡ n ‡ 1; k ˆ 1; 2; . . . ; kc
P
(i) form the MDV yit ˆ yt ¡ j6ˆi zj;t p^kj;tjN ;
(ii) sort{ both yit and zi;t according to the
ascending order of zi;t ;
(iii) obtain an FIS estimate p^ki;tjN of pi;t in the
MDV relationship yit ˆ pi;t zi;t .
Step 3. Continue Step 2 (each time forming the MDV
and then sorting according to the current right-
hand side variable zi;t , prior to FIS estimation),
until iteration kc , when the individual SDPs
(which are each time-series of length N) have
not changed signi®cantly according to some
chosen criterion. The smoothing hyper-par-
ameters required for FIS estimation at each
stage are optimized by Maximum Likelihood
(ML), as explained earlier in } 2.2 and discussed
further below.
{ As a default, these can be simply the constant least
squares parameter estimates, since the convergence of the
back®tting procedure is not too sensitive to the prior
estimates, provided they are reasonable.
{ Depending on the nature of the state dependency, sorting
may need to be with respect to another variable in v t .
 Identi®cation of non-linear stochastic systems 1843
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014

Figure 1. Simulated forced logistic growth model. Upper panel: low noise level (10%); lower panel: high noise level (100%). Noisy
data (circles); noise free data (full line); noise (‡1.0) above.

Note that the ML optimization can be carried out
in various ways: after every complete iteration (each
involving m ‡ n ‡ 1 FIS operations) until convergence
is achieved; only at the initial complete iteration, with
the hyper-parameter s maintained at these values for the
rest of the back®tting; or just on the ®rst two iterations.
The latter choice seems most satisfactory in practice,
since very little improvement in convergence occurs if
optimization is continued after this stage. Normally,
convergence is completed after only a few iterations,
although it can be more lengthy in some circumstances
(see Conclusions } 4).
Simulation example 1: As a simulation example of
SDARX modelling, consider the forced logistic growth
equation (2), mentioned previously, with ¬ ˆ 2:0 and
a zero mean, random input signal ut . The unforced
equivalent of this model (i.e. ut ˆ 0 8t: see Simulation
example 2) is, in fact, the example used by the ®rst
author in what appears to be the ®rst publication on
stochastic SDP modelling (Young 1978) although, at
that time, the more powerful SDP estimation algor-
ithms described in this paper had not been developed
and simple recursive parameter estimation (®ltering)
was utilized.
The estimation is based on 1000 samples of ut and yt
under two di€ erent noise conditions, 200-sampl e seg-
ments of which are presented in ®gure 1. Here the per-
centage noise is de®ned in terms of the standard
deviation of the noise et in relation to the standard devi-
ation of noise-free output. The upper panel shows the
low noise (10%) situation, with the noisy data (circles)
compared with the noise-free data (full line) and the
noise et plotted above (‡1.0). The lower panel is similar
but the noise level is now at the much higher 100% level.
In fact, if the noise is de®ned in terms of di€ erence
between the measured, noisy output and the noise-free
output, then the noise level is 110%.

HP Optimization WLS Estimation

Noise NVR…a1 † NVR…b0 † ¬^1 ¬

^2 ­^0 R2T

Low 0.452 2:5 10¡17 1.996 71.993 1.003 0.999

(0.000 016) (0.000 032) (0.006)
High 0.0165 2:5 10¡17 1.959 71.908 1.016 0.998
(0.00 023) (0.000 46) (0.064)

Table 1 a. Hyper-parameter and ®nal WLS estimation results.

 1844 P. C. Young et al.
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014

Figure 2. Simulated forced logistic growth model. Upper panels: low noise level (10%); lower panels: high noise level (100%). Left
panels: FIS estimate of feedback state dependent parameter a1 … yt¡1 †. Right panels: FIS estimate of input (time-invariant)
parameter b0 . True values shown as dashed lines and standard error bands shown as dotted lines.

The initial SDP estimation results are shown in the
left side of Table 1 a, using a RW model for each SDP.
Here, the optimum NVR values are obtained by ML
optimization at the ®rst back®tting iteration and they
are maintained at these values for the subsequent two
iterations required for convergence. Note how, in both
low and high noise situations, the optimized
NVR…b0 † ˆ 2:5 10¡17 is insigni®cantly di€ erent from
zero, illustrating how the optimization has, quite objec-
tively, identi®ed that b0 is constant. In contrast the opti-
mized NVRs for the `lag’ parameter a1 … yt¡1 †, are 0.452
and 0.0165, respectively, so indicating the presence of
state dependency and allowing for its estimation.
The FIS estimation results, revealing the estimated
nature of the state dependency, are shown in ®gures 2
and 3. In all the plots, the true functions are plotted as
dashed lines and estimation standard error bands are
given by the dotted lines. Figure 2 provides plots of
the estimated SDP variation against the associated
non-minimal state variable, in each case. The left-hand
panel shows the estimated relationship between a^1;tjN ˆ
a^1 … yt¡1 j N† and yt¡1 ; while the right-hand panel
shows the estimated b^0;tjN ˆ b^0 …ut j N† as a function
of ut . As expected, these correspond to those used in
the simulated model: a^1 … yt¡1 j N† is a clear linear func-
tion of yt¡1 with slope and intercept close to 72 and 2,
respectively, in correspondenc e with equation (2 b);
while b^0 …ut j N† is constant for all t, at an estimated
value close to unity. Figure 3 presents the associated
estimates of the non-linear functions: f^1 … yt¡1 † ˆ
a^1 … yt¡1 j N† yt¡1 and f^2 …ut † ˆ b^0 …ut j N† ut plotted as
functions of their associated states.
The e€ ectiveness of the sorting procedure in facilitat-
ing state dependent parameter estimation is illustrated in
®gure 4. This compares the SDP estimate a^1 … yt¡1 †
plotted in the sorted data space, as obtained from the
back®tting algorithm (above), with the same estimate
plotted in the natural data space (below). It is clear
that the direct estimation of this SDP, based on the
time series processed in the natural (temporal ) order,
would produce poor results without access to a priori
information on the nature of the parameter variation.
For example, when this is attempted in the low
noise case, the ML optimized NVR values are
very small, the resulting parameter estimates
hardly change and the explanation of the data is
very poor. And even if we assume prior knowledge
(e.g. set NVR…a1 † ˆ 100; NVR…b0 † ˆ 0), the model
explains the data better but the associated SDP plots
are very noisy indeed, with no clear indication of
the nature of the non-linearity. In contrast, by
sorting the data in ascending order of magnitude, the
sequential variations in the SDP are now slow and
smooth enough to be well estimated by the FIS algor-
ithm, under the assumption that the parameter varies as
a RW process.
 Identi®cation of non-linear stochastic systems 1845
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014

Figure 3. Simulated forced logistic growth model. Upper panels: low noise level (10%); lower panels: high noise level (100%). Left
panels: FIS estimate of feedback non-linearity f1 … yt¡1 †. Right panels: FIS estimate of input (linear) function f2 …ut † ˆ b0 ut .
True values shown as dashed lines and standard error bands shown as dotted lines.

Figure 4. Simulated forced logistic growth model: comparison of the SDP estimate a^1 … yt¡1 † plotted in the sorted data space
(above) with the same estimate plotted in the natural, un-sorted, temporal order (below).
 1846
Noise ¬
^1 ¬
^2 ­^0
Low 1.999 72.011 1.002
(10%) (0.006) (0.013) (0.006)
High 1.988 72.031 1.059
(100%) (0.064) (0.132) (0.060)
Table 1 b. Final least squares estimation results.
Given the non-parametric estimation results shown
in ®gures 2 and 3, it is clear that, even in the high noise
case, the forced logistic growth equation, with three
unknown parameters (¬1 ; ¬2 and ­ 0 ), i.e.
yt ˆ ¬1 yt¡1 ¡ ¬2 y2t¡1 ‡ ­ 0 ut ‡ et
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
provides a parsimonious representation of the data. It is
straightforward, therefore, to obtain estimates of these
parameters by simply ®tting appropriate parameterized
functions to the estimated state dependent relationships
using ordinary least squares or WLS estimation (see
Young 1993; Young and Beven 1994). The WLS esti-
mates obtained in this manner are listed at the right-
hand side of Table 1 a, where R2T is the coe cient of
determination based on the simulation model residuals.
Although these estimates appear somewhat biased in
the high noise case, the resulting deterministic model
output, obtained from equation (11) with et ˆ 0, is
very similar to the noise free output of the actual system.
Having identi®ed a parametric form for the model,
based on the non-parametri c SDP results, however,
improved ®nal estimates of the model parameters can
be obtained using linear estimation based directly on the
measured data (the non-linear model (11) is linear in the
parameters). As might be expected, the resulting esti-
mates, as shown in Table 1 b, are now very good.
2.4. The problem of errors-in-variable s estimation
As in the constant parameter situation, the TVP and
SDP estimates are `asymptotically’ biased away from
their true values if the assumed, and rather special, sig-
nal topology of the SDARX model (1) is violated. Here,
we use `asymptotic’ in a loose sense, since concepts such
as asymptotic bias are not strictly applicable in the TVP
and SDP situations. However, the presence of measure-
ment noise results in parameter estimates which are per-
manently biased away from the true parameter values in
a systematic manner redolent of asymptotic bias in the
constant parameter case. For example, if yt in (1) is
measured in the presence of noise ¹t , in addition to the
`system’ noise et , then this kind of troublesome bias can
appear on the parameter estimates.
In the slow TVP situation, it is possible to obviate
this `errors-in-variables ’ problem by either extending the
model to include additional noise terms (e.g. Norton
1986); or utilizing a recently developed approach based
P. C. Young et al.
R2T
on Instrumental Variable (IV) techniques (Young 2000).
Although limited to input±output systems, the latter IV
0.999 procedure is simpler and more e€ ective because it does
not require simultaneous estimation of the non-station-
0.994 ary noise model parameters. It is also e€ ective in the
SDP situation, but only if the non-linear system does
not exhibit chaotic behaviour.
When considering the practical implications of
any errors-in-variable s bias on the estimates of the
parameters in the SDARX model, it is important to
stress that SDP estimation is being suggested here
primarily as a method of identifying the non-linear
model structure and not necessarily as an end in itself.
…11† Moreover, the bias can be quite small even in quite high
noise situations. For instance, consider the following
extended version of the model (2) with an additional
tanh non-linearity on the input
xt ˆ 2:0xt¡1 ¡ 2:0x2t¡1 ‡ 0:01…tanh …ut †
‡ 0:01ut † ut ˆ U…0; 3:2†
y t ˆ xt ‡ ¹ t ¹t ˆ N…0; 3 10¡5 †
Here, the errors-in-variable s noise on xt is quite high
(70% by standard deviation) but, despite this, both the
lag SDP and the input non-linearity, as shown in
®gure 5, are estimated well, so providing useful informa-
tion on the location and nature of the non-linearities in
this case.
3. Identi®cation of purely stochastic non-linear systems
A special example of the SDP model (1) is the
following State Dependent parameter Auto-Regressive
(SDAR) model
yt ˆ zTt pt ‡ et …12†
where
9
zTt ˆ ‰¡yt¡1 ¡ yt¡2 ¡ yt¡n Š =
…12 a†
pt ˆ ‰a1 …v t † a2 …v t † an …v t †ŠT ;
Clearly, the same SDARX estimation methods dis-
cussed previously can be applied to this model, a simple
example of which is the chaotic version of the logistic
equation. Typical simulation results for this model are
discussed below.
Simulation example 2: In order to consider further
the e€ ects of errors-in-variable s measurement noise, as
discussed in } 2.4 for input±output systems{, the
{ Note that, in this purely stochastic situation, it is not
possible to utilize the IV approach mentioned in } 2.4 to
obviate bias since this requires the presence of an input
signal to generate the instrumental variables.
 Identi®cation of non-linear stochastic systems 1847
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014

Figure 5. SDP estimation results for a system with severe input and lag non-linearities, showing the relative insensitivity, in this
case, to a high level of errors-in-variables noise (70% by standard deviation).

following Noisy SDAR (NSDAR) version of the
chaotic logistic growth model (cf. equations (2)) will
be used in this example:
) The most notable feature of the results in ®gure 6 is
xt ˆ 4:0xt¡1 ¡ 4:0x2t¡1 ‡ et et ˆ N…0; 0:0064†
yt ˆ xt ‡ ¹t ¹t ˆ N…0; 0:0012†
…13 a†
or
yt ˆ a1 … yt¡1 † yt¡1 ‡ ±t a1 … yt¡1 † ˆ 4:0 ¡ 4:0yt¡1
…13 b†
This model generates chaotic behaviour (see later, ®gure
7) and the noise ±t is a complex non-linear function of et ,
¹t and yt . The noise level is about 10% (by standard
deviation). As expected, without this measurement
noise present, SDP estimation is straightforward with
excellent, low variance SDP estimates that identify the
nature of the non-linearity without any di culty.
Even with the measurement noise, the estimation
results are reasonable. Since there is only one SDP,
a1 … yt¡1 †, ML optimization of the associated scalar
NVR quickly yields Qr ˆ 0:000016. The subsequent
SDP estimation results are illustrated in ®gure 6,
where the top left-hand panel shows the FIS estimated
non-linear function with its characteristic quadratic
shape; while the top right-hand panel shows the
estimated state dependency of the SDP estimate
a^1 … yt¡1 j N†. In both plots, the true relationship is
shown as a dashed line, while the estimated standard
error bounds are shown dotted.
the larger errors in the SDP estimate at low values of
yt¡1 . Since the measurement noise variance is constant,
this is the region where the measurement noise is having
its most deleterious biasing e€ ect. Even with these
errors, however, the WLS estimates of the parameters
(3:884…0:111† and ¡3:857…0:135†, respectively) are quite
close to the true values (4 and ¡4). The results are better
still, however, if the noise is itself made state dependent,
in the sense that it is set proportional to the signal level
(i.e. ¹tsd ˆ ‰xt =max …xt †Š ¹t ). This is a common situation
with real data and it signi®cantly reduces the noise e€ ect
at low signal levels where the bias is largest. As a result,
the WLS estimates are improved to 3:960…0:013† and
¡3:946…0:016†, respectively. Indeed, in this state depen-
dent noise situation, the results are still good even if the
variance of ¹t is doubled, so that the noise level on the
data is visibly quite large, as shown in ®gure 7. The
resulting SDP estimation results are presented in the
lower panels of ®gure 6, which identify clearly that the
data were generated by the logistic model with chaos-
inducing parameter values.
Simulation example 3: The cosine map model (e.g.
Zhan-Qian and Smith 1998) takes the form
 1848 P. C. Young et al.
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014

Figure 6. SDP estimation of the simulated chaotic logistic model. Left panels: estimated non-linear functions with ordinary
(upper) and state-dependent (lower) noise. Right panels: estimated state dependent parameters with ordinary (upper) and
state-dependent (lower) noise.

Figure 7. Simulated chaotic logistic model. Noisy (full line) and noise-free (dashed line) output, with the state-dependent noise
(‡1.2) shown above.
 Identi®cation of non-linear stochastic systems
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
Figure 8. Simulated cosine map model. Upper panel: measured output. Lower panel: phase plane (embedding) plot of yt vs yt¡1
(noise-free response shown as a full line).
yt ˆ cos …2:8yt¡1 † ‡ 0:3yt¡2 ‡ et et ˆ N…0; 0:01†
…14†
and a typical 2000 sample simulation of the model is
presented in ®gure 8, which shows the time response in
the upper panel and the phase plane … yt yt¡1 † plot in
the lower panel. In the latter graph, the noise free
response is shown as a full thick line with the noisy
response plotted as dots.
This a typical stochastic model that exhibits under-
lying chaotic response characteristics. It provides a test-
ing example for the SDP approach, however, because it
is not in the assumed a ne form (Co 1996): in particu-
lar, the SDP term a1 … yt¡1 † yt¡1 in the most appropriate
SDAR model
yt ˆ a1 … yt¡1 † yt¡1 ‡ a2 … yt¡2 † yt¡2 ‡ et et ˆ N…0; ¼2 †
…14 a†
is not able to represent the equivalent term cos …2:8yt¡1 †
in (14) exactly, since cos …2:8yt¡1 †=yt¡1 has a singularity
at yt¡1 ˆ 0. Despite this di culty, SDAR estimation
yields excellent results, as illustrated in ®gures 9±11.
These results were obtained with the ML optimized
NVR matrix Q r ˆ diag ‰0:0057 1:28 10¡7 Š and it is
clear, yet again, that the optimization has successfully
identi®ed that the potential state dependency resides in
the ®rst lag parameter a1;t ˆ a1 … yt¡1 †, while the second
1849
lag parameter a2;t ˆ a2 … yt¡2 † is e€ ectively time-
invariant.
Figure 9 shows the FIS estimate of the cosine non-
linearity subsequent to the convergence of the back®t-
ting procedure, which took six iterations in this case.
Except for the region around the singularity at
yt¡1 ˆ 0 (see below), the estimation is very good. The
associated a^2 … yt¡2 j N† ˆ 0:291…0:0038†, for all t, is
estimated as being time-invariant despite the fact that
the NVR, Qr …2; 2† ˆ 1:28 10¡7 , is not too small in this
case. Figure 10 compares the actual phase plane plot for
the data used in the estimation (left-hand panel) with a
similar plot based on data from a typical random reali-
zation of the SDAR model (right-hand panel). Similar
agreement is found in both the time plots of the two
series and the histograms.
Figure 11 compares the FIS estimate a^1 … yt¡1 j N†,
plotted as a function of yt¡1 , with the theoretical
function given by a1 … yt¡1 † ˆ cos …2:8yt¡1 †=yt¡1 (which
migrates to 1 at the point yt¡1 ˆ 0). Over the most
important region ¡0:1 > yt¡1 > 0:1, the estimate is very
accurate. Not surprisingly, it becomes inaccurate close
to the singularity, but the algorithm is robust enough to
handle this well, without impairing the SDP estimates
elsewhere. And the overall cosine shape of the non-
linear function is clearly estimated accurately, as we
see in ®gure 9.
 1850 P. C. Young et al.
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014

Figure 9. Simulated cosine map model. Comparison of FIS estimated and actual cosine non-linearity.

Figure 10. Simulated cosine map model. Comparison of the phase-plane plot for the data used in the estimation (left panel) with
the phase-plane plot obtained from a random realization of the estimated SDAR model.
 Identi®cation of non-linear stochastic systems
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
Figure 11. Simulated cosine map model. Comparison of the FIS estimate a^1 … yt¡1 j N† as a function of yt¡1 , with the theoretical
function given by a1 … yt¡1 † ˆ cos …2:8yt¡1 †=yt¡1 .
Finally, on the basis of the above SDP results, the
form of the non-linear equation is identi®ed correctly as:
yt ˆ ¬ cos …­ yt¡1 † ‡ ®yt¡2 ‡ et …14 b†
and the optimized ML estimates of the, now con-
stant, parameters ¬, ­ and ® in this model are
^ ˆ 0:998…0:004†, ­^ ˆ 2:797…0:004† and ®^ ˆ 0:303
¬
…0:003†.
Real example: analysis of squid data: This example is
based on the analysis of the signal shown in ®gure 12,
which was obtained by Kazu Aihara and Gen Matsu-
moto from experiments on the giant axon of a squid
(Mees et al. 1992). The signal comprises voltage meas-
urements made from a micro-pipette inserted into a
giant axon. Squid are used for such experiments be-
cause they have large diameter axons: this is because
the nerves are not myelinated (insulated), so they need
to have large diameter to reduce ion leakage and so
maintain the transmission speed. The experiment is
done in vitro (i.e. the nerves are chopped out of the
squid), with the membrane voltage clamped. This is
normally referred to as the forced response. However,
the forcing is periodic and the response is not: in ef-
fect, ®gure 12 shows a putative chaotic response to a
periodic signal. It is analysed here, however, as a
purely stochastic, unforced dynamic process. Only the
1851
®rst order SDAR model results will be discussed here:
additional results based on the estimation of a second
order SDAR model are presented in Young (2001 a).
The simplest SDAR model that could produce the
dynamics shown in ®gure 12 is the following ®rst order
SDAR
yt ˆ a1 … yt¡1 † yt¡1 ‡ et …15†
The FIS estimate of the SDP is shown in ®gure 13 and
the associated non-linear function in ®gure 14. These
results were obtained with the single NVR hyper-par-
ameter optimized at NVR…a1 † ˆ 5:01 106 , under the
assumption of an RW model for the SDP parameter
variation (an IRW assumption provides a slightly
more smoothed estimate but does not make any signi®-
cant di€ erence). The standard error band is shown as
the dashed lines on both plots. This model explains
92.2% of the squid data (i.e. the coe cient of determi-
nation based on the model residuals R2 ˆ 0:922).
Figure 15 provides a qualitative but discerning com-
parison of the model behaviour and the squid data
(shown in the lower panel): here, the upper panel
shows a random realization of the model (15) generated
by a Simulink simulation model using a look-up table
for the non-linearity based on the SDP estimation
results. It is clear that the general nature of the response
visually matches the actual squid signal rather well and
 1852 P. C. Young et al.
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014

Figure 12. Squid example: signal obtained from experiments on the giant axon.

Figure 13. Squid example: FIS estimate of the state dependent parameter a1 … yt¡1 †.

this is con®rmed by the similarity in the statistical prop-
erties of the two signals.
The estimated non-parametric non-linearity can be
parameterized in various ways. Here, a nine-function
RBF model, designed using the Matlab Neural
Network Toolbox function solverb, is able to reproduce
the non-linearity with an R2 ˆ 0:997, and the Simulink
simulated behaviour of the resulting parameterized
model closely resembles that of both the non-parametric
model and the squid data.
 Identi®cation of non-linear stochastic systems 1853
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014

Figure 14. Squid example: FIS estimate of non-linearity f1 … yt¡1 † ˆ a1 … yt¡1 † yt¡1 .

Figure 15. Squid example: qualitative comparison of a random realization of the SDAR model behaviour (upper plot) and the
squid data (lower panel).

Note ®nally, that the dots shown in ®gure 14 repre- SDP approach to modelling non-linear systems and
sent the phase-plane or embedding graph of the squid existing methods used by non-linear systems theorists,
data, with yt plotted versus yt¡1 . This is interesting where smooth curves are often ®tted to the embedding
because it draws a comparison between the present graphs in order to model the non-linear system (see e.g.
 1854 P. C. Young et al.
Mees 1991, 1993). The advantage of the SDP approach
is that the smoothing is optimal in a ML sense and is
carried out within a stochastic, dynamic systems setting.
Also, the smoothing is applied to the estimation of the
SDP parameter, rather than the non-linear function as a
whole, and so it provides a more ¯exible and informa-
tive result.
For instance, the SDP in the model (15) can be con-
sidered, in an approximate sense, as the changing eigen-
value of a ®rst order, discrete-time dynamic system (we
might refer to this as a virtual-eigenvalue) and this pro-
vides some additional insight into the nature of the
system. By reference to ®gure 13, for example, we see
that, for yt less than about 7120, this eigenvalue is
approximately constant at a value of about 0.75,
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
suggesting that the underlying behaviour over this part
of the state space is quite stable, with a (virtual) time
constant of approximatel y 3.5 sampling intervals.
Between values of about 7120 and 7105, however,
the eigenvalue increases steadily to unity, where the
system is at a point of neutral stability (i.e. the system
is acting transiently as an integrator). Thereafter, for
values greater than 7105, the eigenvalue rises sharply
to around 1.5 and the underlying system is clearly
exponentially unstable. However, if we now consider
the temporal variation of the SDP, it is clear that,
every time that the SDP exceeds unity, the instability
of the system drives it immediately back to a location
where the SDP is in the region where the eigenvalue is
about 0.75 again and the system is stabilized. This is also
well illustrated by a stacked plot of the changing impulse
response associated with the AR model de®ned by the
SDAR model at each instant of time instant.
So, to conclude, it would appear that the ®rst-order
SDAR model provides a reasonable mechanism for
characterizing the behaviour of the electrical activity in
the axon of the squid and that, although the second-
order SDAR model provides a slightly better explana-
tion of the squid data (see Young 2001 a), this does not
seem su cient to justify the increased complexity of the
model.
4. SDP state estimation and automatic control
It is clearly possible to develop state estimation and
control system design methods based on the new class of
SDP non-linear models discussed in this paper. These
can be considered in parametric or non-parametric
form (the latter providing a new way of considering
control and estimation system design).
The combined KF and FIS algorithms, when used in
a more conventional state estimation setting, can func-
tion with system matrices characterized by time, or even
state dependent parameters. This extends their range of
applicability to a considerable extent. Thus, the SDP
models described here can provide the basis for rather
novel non-parametric or state dependent parameter
KF±FIS design, with implications for both modelling
and non-linear optimization based on prediction error
decomposition. For example, the blow¯y population
modelling example in Young (2000) has shown how a
SDP implementation of the KF can be used as a vehicle
in the ®nal stage of non-linear model parameter optimi-
zation based on prediction error decomposition.
In the case of control system design, Young (1996)
adumbrated a SDP approach to non-linear control
system design by showing how a SDP version of the
Proportional-Integral-Plu s (PIP) controller (see Taylor
et al. 2000 and the prior references therein) could be
used to successfully control the chaotic logistic growth
equation (13 a). In more recent research (McCabe et al.
2001), we have provided a theoretical basis for such
SDP-PIP control system design and shown how it can
be applied to a variety of non-linear systems, including
the non-linear model of a hydraulic servo with asym-
metric limiting and deadzone non-linearities discussed
elswhere in this Special Issue (see Hu et al. 2001;
Young 2001 b).
5. Conclusions
Recursive estimation has a long and rich history:
from its beginnings in Gauss’s original derivation of
recursive least squares (Gauss 1823; see Appendix 2 of
Young 1984), through its re-discovery by Plackett (1950)
and Kalman’s seminal work on stochastic state estima-
tion (Kalman 1960), to the burgeoning of research on
recursive estimation in a whole range of di€ erent aca-
demic disciplines between 1960 and the present. In the
last ten years, however, the advent of fast computers and
the desire of theorists to extend the boundaries of time
series analysis has led to an explosion of research on
Monte Carlo-based numerical methods, from either
classical (e.g. Durbin and Koopmans 1999) or
Bayesian (e.g. Ruanaidh and Fitzgerald 1996,
Gamerman 1997) perspectives.
The motivation of this more recent research is clearly
to extend the `Gaussian’ methods of standard recursive
estimation to non-Gaussian and non-linear time series,
using models in which the stochastic inputs are non-
Gaussian. But models of non-Gaussian and non-linear
processes do not necessarily require the assumption of
non-Gaussian inputs. As we have seen in this paper, a
fairly wide class of non-Gaussian and non-linear time
series can be represented by time variable (TVP) and
state dependent (SDP) parameter, non-linear, stochastic
models with Gaussian inputs. When it is possible (and
the methods do seem quite widely applicable), this is
clearly advantageous , since it allows for the use of well
tried and robust algorithms that are computationally
 Identi®cation of non-linear stochastic systems
much less demanding than even the `classical’ non-
Gaussian methods (see Durbin and Koopmans 1999).
The fact that the FIS algorithm can function well as
a non-parametri c estimator means that it provides a
powerful, recursive alternative to other, more conven-
tional, methods of smoothing, such as regularization,
smoothing splines, kernel smoothers and locally
weighted kernel regression (see e.g. the practical ex-
ample in Young and Pedregal 1998). It also provides a
non-parametric method for transforming random vari-
ables (e.g. Gaussian to non-Gaussian) , or identifying the
nature of a parametric transform between random vari-
ables.
Finally, it is clear that the simulated and real
examples presented in the paper, combined with those
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
discussed in other cited references, demonstrate the
e cacy of the proposed SDP approach to modelling
for a fairly wide and practically useful class of non-
linear stochastic systems. However, the proposed tech-
nique is relatively new and it raises a variety of interest-
ing theoretical questions and possibilities for extending
the approach to an even richer class of non-linear stoch-
astic systems. For example:
. How can the approach be extended to handle mul-
tivariable state dependencies, where the SDPs may
be functions of several state variables? One
approach currently being developed by the ®rst
author is to model the SDPs with neuro-fuzzy
functions of several variables (e.g. Hu et al.
2001) but with the structure of the non-linear
system identi®ed ®rst from the data using the
methodology described in the present paper (see
the discussion in Young 2001 b).
. What is the best method of handling the estima-
tion bias that occurs when the proposed SDP
modelling approach is applied in situations
where errors-in-variables problems occur? An
instrumental variable (IV) method, such as that
mentioned in } 2.4, can obviate such problems in
the case of well-behaved non-linear input±output
models. But alternative approaches will be
required in the case of purely stochastic non-linear
systems without inputs and/or non-linear systems
that exhibit sensitive, chaotic behaviour.
. Although no convergence problems have been
encountered so far in the evaluation of the pro-
posed SDP estimation procedure, what conditions
are required for convergence of the back®tting
procedure? Hastie and Tibshirani (1996) use a
similar back®tting procedure for estimation of
their Generalized Additive Model (GAM). It
needs to be established whether their conclusions
as regards convergence (which are not entirely per-
suasive, in any case) are applicable to the models
1855
and back®tting procedure described in this paper.
Unlike the GAM, for instance, the non-linear
functions in the SDP models are factorized into
the product of the SDP and the model variable;
and the SDP is estimated by optimal FIS smooth-
ing (rather than the more conventional scatter-
plot smoothing used by Hastie and Tibshirani).
. The back®tting procedure does not provide com-
plete covariance information on the SDP esti-
mates. Could this be distorting the standard
errors on the estimates? In more general terms,
what are the full theoretical statistical properties
of the SDP estimates obtained by back®tting?
. Finally, what are the identi®ability conditions on
the SDP models? It is clear that problems analo-
gous to collinearity in constant parameter model
estimation can occur and that back®tting conver-
gence may be a€ ected by such problems. Also, in
the case of input±output models, the nature of the
input signals will a€ ect the identi®ability of the
model parameters. It is necessary to explore
these factors further and establish what other fac-
tors may a€ ect the identi®ability of the model.
Regardless of the answers to these questions, how-
ever, the SDP approach to the identi®cation of non-
linearities in stochastic systems appears to hold great
promise. In contrast to other approaches, such as neural
networks and NARMAX models, for example, it
attempts to identify the type of non-linearity and, there-
fore, the form of the non-linear model, prior to the esti-
mation of the parameters in the ®nally identi®ed model.
This helps to ensure that the ®nal non-linear model is
e ciently parameterized (parsimonious) and it should
avoid the over-parameterizatio n that normally accom-
panies neural network and, to a lesser extent, the `black-
box’ NARMAX models. Indeed, the SDP approach has
been developed as a primary tool in Data-Based
Mechanistic (DBM) modelling (e.g. Young, 1998 and
the prior references therein){, where its ability to obtain
parametrically e cient and physically meaningful mod-
els is essential. A practical example of this is given in the
®rst author’s comments on the paper by Hu et al. (2001),
{ Here, the model structure is inferred directly in an
inductive manner from data in relation to a generic class of
black-box models (e.g. SDARX) and then the model is
interpreted in a physically meaningful manner. This can be
contrasted with the hypothetico-deductive `Grey-Box’ mod-
elling, where the simpli®ed model structure is based on prior,
physically-based and possibly subjective assumptions, with
the parameters that characterize this simpli®ed structure
then estimated from data on the basis of this assumed model
form.
 1856 P. C. Young et al.
which appears elsewhere in this Special Issue (Young
2001 b).
SDP estimation also provides a non-parametric
model that can be useful in its own right. As we have
seen, the SDP model can be simulated easily in pro-
grams such as Simulink, thus removing the need for
the ®nal parametric estimation in some applications,
such as simulation, forecasting and automatic control.
Finally, it should be noted that all of the results
presented in this paper were generated using tools
from the CAPTAIN Matlab Toolbox developed at
Lancaster. This Toolbox is currently in the ®nal stages
of ­ -testing and will be generally available towards the
end of 2001. Further information on the Toolbox can
be obtained from the ®rst author and via http://www.
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
es.lancs.ac.uk/cres/captain/.
Acknowledgements
Some of the material in this paper has appeared in
two books (Young 2000, 2001 a) and the author is grate-
ful to the publishers (Cambridge University Press,
Cambridge and Birkhauser, Boston), for permission to
use this material. Part of this research was supported by
the EPSRC under grant GR/K77884; the BBSRC under
grant EO6813; and the EPSRC, together with the
Newton Institute for Mathematical Sciences in
Cambridge, for support during the ®rst author’s six
month stay at the Newton Institute, where most of
this paper was written. The ®rst author is also grateful
for the many stimulating discussions he had with
Professor Alistair Mees, University of Western
Australia, who also provided the squid data (with
the permission of Kazu Aihara and Gen Matsumoto).
The third author received some support from the
Nu eld Foundation. Finally, thanks are also due to
the anonymous referees for their helpful and construc-
tive comments.
References
Brown, R. L., Durbin, J., and Evans, J. M., 1975,
Techniques for testing the constancy of regression relation-
ships over time, Journal Royal Statistical Society, Series B,
37, 141±192.
Bryson, A. E., and Ho, Y. C., 1969, Applied Optimal Control
(Blaisdel).
Co, T., 1996, Parameter estimation of non-linear systems using
modulating functions methods. In M. I. Friswell and J. E.
Mottershead (Eds), Identi®cation in Engineering Systems,
University of Wales, Swansea, UK, pp. 87±96.
Daubechies, I., 1988, Orthonormal bases of compactly
supported wavelets, Communications on Pure Applied
Mathematics, 41, 906±966.
Dempster, A. P., Laird, N. M., and Rubin, D. B., 1977,
Maximum likelihood from incomplete data via the EM
algorithm, Journal Royal Statistical Society, Series B, 39,
1±38.
Durbin, J., and Koopmans, S. J., 1999, Time series analysis
of non-Gaussian observations based on state space models
from both classical and Bayesian perspectives. Journal of the
Royal Statistical Society, Series B, 62, (in press).
Gamerman, D., 1997, Markov Chain Monte Carlo (Chapman
and Hall).
Gauss, K. F., 1821, 1823, 1826, Theoria combinationis obser-
vationum erroribus minimis obnoxiae, Parts 1, 2 and sup-
plement, Werke, 4, 1±108.
Harvey, A. C., 1981, Time Series Models (Oxford: Phillip
Allen, Oxford).
Harvey, A. C., 1984, A uni®ed view of statistical forecasting
procedures (with comments). Journal of Forecasting, 3, 245±
283.
Harvey, A. C., 1989, Forecasting Structural Time Series
Models and the Kalman Filter (Cambridge University Press).
Harvey, A. C., and Peters, S., 1990, Estimation procedures
for structural time series models, Journal of Forecasting, 9,
173±204.
Hastie, T. J., and Tibshirani, R. J., 1996, Generalized
Additive Models (Chapman and Hall).
Hoberock, L. L., and Kohr, R. H., 1966, An experimental
determination of di€ erential equations to describe simple
non-linear systems. Preprints, Joint Automatic Control
Conference, Seattle, Washington, USA.
Holst, U., HoÈ ssjer, O., BjoÈ rklund, C., Ragnarsson, P.,
and Edner, H., 1996, Locally weighted least squares kernel
regression and statistical evaluation of LIDAR measure-
ments, Environmetrics, 7, 410±416.
Hu, J., Kumamaru, K., and Hirasawa, K., 2001, A quasi-
ARMAX approach to modelling non-linear systems.
International Journal of Control, this special issue.
Jakeman, A. J., and Young, P. C., 1979, Recursive ®ltering
and the inversion of ill-posed causal problems, CRES
Report No. AS/R28/1979, Centre for Resource and
Environmental Studies, Australian National University.
Jakeman, A. J., and Young, P. C., 1984, Recursive ®ltering
and the inversion of ill-posed causal problems. Utilitas
Mathematica, 35, 351±376.
Kalman, R. E., 1960, A new approach to linear ®ltering and
prediction problems. ASME Transactions Journal Basic
Engineering, 82D, 35±45.
Kitagawa, G., 1981, A non-stationary time series model and
its ®tting by a recursive ®lter. Journal of Time Series
Analysis, 2, 103±116.
Lee, R. C. K., 1964, Optimal Identi®cation, Estimation and
Control. (MIT Press).
McCabe, A., Chotai, A., and Young, P. C., 2001, State
dependent parameter PIP control of non-linear systems.
Report No. TR/175, Centre for Research on Environ-
mental Systems and Statistics, Lancaster University.
Mees, A. I., 1991, Dynamical systems and tesselations: detect-
ing determinism in data. International Journal of Bifurcation
and Chaos, 1, 777±794.
Mees, A. I., 1993, Parsimonious dynamical reconstruction,
International Journal of Bifurcation and Chaos, 3, 669±675.
Mees, A. I., Aihara, K., Adachi, M., Judd, K., Ikeguchi,
T., and Massumoto, G., 1992, Deterministic prediction
and chaos in squid axon response. Physics Letters A, 169,
41±45.
Mendel, J. M., 1969, A priori and a posteriori identi®cation of
time varying parameters. Second Hawaii International
Conference on System Sciences.
 Identi®cation of non-linear stochastic systems
Norton, J. P., 1976, Identi®cation by optimal smoothing
using integrated random walks. Proceedings of the Institute
Electrical Engineers, 123, 451±452.
Norton, J. P., 1986, An Introduction to Identi®cation
(Academic Press).
Plackett, R. L., 1950, Some theorems in least squares.
Biometrika, 37, 149±157.
Priestley, M. B., 1988, Nonlinear and Nonstationary Time
Series Analysis (Academic Press).
Ruanaidh, J. J., and Fitzgerald, W. J., 1996, Numerical
Bayesian Methods Applied to Signal Processing (Springer).
Schweppe, F., 1965, Evaluation of likelihood function for
Gaussian signals. IEEE Transactions on Information
Theory, 11, 61±70.
Taylor, C. J., Young, P. C., and Chotai, A., 2000, State
space control system design based on non-minimal state
variable feedback: further generalisations and uni®cation
results, International Journal of Control, 73, 1329±1345.
Downloaded by [Case Western Reserve University] at 14:43 13 October 2014
Young, P. C., 1969, Applying parameter estimation to
dynamic systems: Part l, Theory. Control Engineering, 16,
119±125; Part lI Applications, 16, 118±124.
Young, P. C., 1970, An instrumental variable method for real-
time identi®cation of a noisy process. Automatica, 6, 271±
287.
Young, P. C., 1978, A general theory of modeling for badly
de®ned dynamic systems. In G. C. Vansteenkiste (Ed.),
Modeling, Identi®cation and Control in Environmental
Systems (North Holland), 103±135.
Young, P. C., 1984, Recursive Estimation and Time-Series
Analysis (Springer).
Young, P. C., 1988, Recursive extrapolation, interpolation
and smoothing of non-stationary time-series. In H. F.
Chen (Ed.) Identi®cation and System Parameter Estimation
1988 (Pergamon Press), pp. 33±44.
Young, P. C., 1989, Recursive estimation and modelling
of non-stationary and non-linear time-series. In IFAC
Symposium on Adaptive Systems in Control and Signal
Processing, Vol. 1 (Pergamon Press), pp. 49±64.
Young, P. C., 1993, Time variable and state dependent mod-
elling of non-stationary and non-linear time series In T.
Subba Rao (Ed.), Developments in Time Series Analysis
(Chapman and Hall), pp. 374±413.
1857
Young, P. C., 1994, Time-variable parameter and trend esti-
mation in non-stationary economic time series. Journal of
Forecasting, 13, 179±210.
Young, P. C., 1996, A general approach to identi®cation,
estimation and control for a class of non-linear dynamic
systems. In M. I. Friswell and J. E. Mottershead (Eds),
Identi®cation in Engineering Systems (Swansea: University
of Wales), pp. 436±445.
Young, P. C., 1998, Data-based mechanistic modelling of
environmental, ecological, economic and engineering
systems. Environmental Modelling and Software, 13, 105±
122.
Young, P. C., 1999, Nonstationary time series analysis and
forecasting. Progress in Environmental Science, 1, 3±48.
Young, P. C., 2000, Stochastic, dynamic modelling and signal
processing: time variable and state dependent parameter
estimation. In W. J. Fitzgerald, A. Walden, R. Smith
and P. C. Young (Eds) Nonlinear and Nonstationary Signal
Processing (Cambridge: Cambridge University Press),
pp. 74±114.
Young, P. C., 2001 a, The identi®cation and estimation of
non-linear stochastic systems. In A. I. Mees (Ed.)
Nonlinear Dynamics and Statistics (Boston: Birkhauser),
pp. 127±166.
Young, P. C., 2001 b, Comment on `A quasi-ARMAX
approach to the modelling of non-linear systems’ by J. Hu
et al., International Journal of Control, this special issue.
Young, P. C., and Beven, K. J., 1994, Data-based mechanistic
modelling and the rainfall-¯ow non-linearity. Environ-
metrics, 5, 335±363.
Young, P. C., and Pedregal, D. J., 1998, Recursive and en-
bloc approaches to signal extraction. Journal of Applied
Statistics, 26, 103±128.
Young, P. C., Pedregal, D. J., and Tych, W., 1999,
Dynamic harmonic regression. Journal of Forecasting, 18,
369±394.
Zhan-Qian, J. L., and Smith, R. L., 1998, Estimating local
Lyapunov exponents by local polynomial regression, Uni-
versity of North Carolina, Department of Statistics, http://
www.stat.unc.edu./postcript/rs/chaos.