Professional Documents
Culture Documents
Time-Varying Systems
Simo Srkk
Department of Biomedical Engineering and Computational Science
Aalto University
Simo Srkk
Contents
1
Optimal Filtering
Applications
Simo Srkk
Contents of Course
Modeling with stochastic state space models.
Bayesian theory of optimal filtering.
Gaussian approximations: Derivation of Kalman, extended
Kalman and unscented Kalman filters, Gauss-Hermite and
cubature Kalman filters from the general theory.
Monte Carlo methods: Particle filtering, Rao-Blackwellized
filtering.
Optimal smoothing: Bayesian theory of optimal smoothing
and related Kalman (=Gaussian) and particle type
methods.
Various illustrative applications to backup the theory.
Various exercises to practice modeling and estimation.
Simo Srkk
Some History
In 40s, Wieners work on stochastic analysis and optimal
filtering (and cybernetics)
In late 50s, state space models, Bellmans dynamic
programming, Swerlings filter.
In early 60s, Kalman filter and Kalman-Bucy filter, stability
analysis of linear state space models (mostly by Kalman).
In mid 60s, Rauch-Tung-Striebel smoother, extended
Kalman filters (EKF).
In late 60s, Bayesian approach to optimal filtering, first
practical applications (e.g. Apollo program).
In 70s and 80s, first particle filters, square root Kalman
filters, new algorithms and applications.
In 90s, rebirth of particle filters, sigma-point and
unscented Kalman filters (UKF), new applications.
In 00s, new algorithm variations and applications.
Simo Srkk
Simo Srkk
f(t)
Mittalaite
0.25
0.2
0.15
0.15
0.1
0.1
0.05
0.05
0.05
0.05
0.1
0.1
0.15
0.15
0.2
0.2
0.25
0.25
0
8
Time
10
12
14
Signal
Signal derivative
Measurement
0.25
0.2
16
Simo Srkk
8
Time
10
12
14
16
Optimal Filter
y1 y2 y3 y4 ...
Optimal
filter
x1 x2 x3 x4 ...
k = 1, . . . , T .
Prediction distributions:
p(xk +n | y1 , . . . , yk ),
k = 1, . . . , T ,
n = 1, 2, . . . ,
Smoothing distributions:
p(xk | y1 , . . . , yT ),
k = 1, . . . , T .
Simo Srkk
Simo Srkk
Simo Srkk
Simo Srkk
Spread of Disease
Spreading of a disease in population
can be modeled by differential
equations.
1000
Sovitettu kyr
Ennustettu data
Mitattu data
Kuolemia
800
600
400
200
10
15
Viikko
20
25
30
Other Applications
angle 3
target 1
angle 6
Sensor 3
angle 1
angle 2
angle 4
angle 5
Sensor 1
target 2
Sensor 2
160
140
120
100
80
60
40
20
0
950
1000
1050
qk 1 N(0, Q)
yk = H xk + rk ,
rk N(0, R)
x0 N(m0 , P0 ).
In probabilistic notation the model is:
p(yk | xk ) = N(yk | H xk , R)
p(xk | xk 1 ) = N(xk | A xk 1 , Q).
Surprisingly general class of models linearity is from
measurements to estimates, not from time to outputs.
Simo Srkk
Simo Srkk
Simo Srkk
k = 1, . . . , N.
k = 1, . . . , N.
k = 1, . . . , N.
k = 1, . . . , N.
qk 1,i N(0, 2 ),
i = 1, . . . , d .
600
400
Signal
200
200
400
600
500
1000
1500
2000
2500
Time
3000
3500
4000
4500
5000
Simo Srkk
rkx N(0, x2 ).
60
40
450
20
0
400
20
40
350
60
80
300
100
250
1950
1960
1970
1980
1990
2000
2010
2020
2030
2040
2050
120
4900
4910
4920
4930
4940
4950
4960
4970
4980
4990
5000
Simo Srkk
0 0
x1
0 0 1 0
x1
d
x2 = 0 0 0 1 x2 + 0 0 w1 .
1 0 w2
x3
0 0 0 0
dt x3
0 1
0 0 0 0
x4
x4
| {z }
|
{z
}
F
dx
= F x + L w.
dt
Simo Srkk
Simo Srkk
Mittalaite
Simo Srkk
Simo Srkk
300
250
Gravitation law:
200
150
100
F = m a(t) =
50
G m M r(t)
.
|r(t)|3
50
6200
6250
6300
6350
6400
6450
6500
6550
6600
6650
G M r(t)
D(r(t)) |v(t)| v(t) + w(t).
|r(t)|3
Simo Srkk
If we define x = (x1 x2
dx1 dx2
dt dt
dx
= f(x) + L w(t).
dt
where f() is non-linear.
The radar measurement:
q
r = (x1 xr )2 + (x2 yr )2 + er
1 x2 yr
+ e ,
= tan
x1 xr
where er N(0, r2 ) and e N(0, 2 ).
Simo Srkk
Simo Srkk
Simo Srkk
Summary
The purpose of is to estimate the state of a time-varying
system from noisy measurements obtained from it.
The linear theory dates back to 50s, non-linear Bayesian
theory was founded in 60s.
The efficient computational solutions can be divided into
prediction, filtering and smoothing.
Applications: tracking, navigation, telecommunications,
audio processing, control systems, etc.
The formal Bayesian estimation equations can be
approximated by e.g. Gaussian approximations, Monte
Carlo or Gaussian mixtures.
Formulating physical systems as state space models is a
challencing engineering topic as such.
Simo Srkk
http://www.lce.hut.fi/research/mm/ekfukf/
Simo Srkk