# STARTING WITH THE NAME OF ALLAH MOST

BENEFICIENT AND THE MOST MERCIFUL
KALMAN FILTER

RANDOM PROCESS
EE-6413

SHAFAQ EJAZ
11F-MS-EE-01
M. ATIF
11F-MS-EE-04

BRIEF HISTORY

In 1960, R.E. Kalman published his famous paper
describing a recursive solution to the discrete data
linear filtering problem. Since that time the Kalman
filter has been the subject of extensive research and
application.
DEFINITION

The Kalman filter is a set of mathematical equations
that provides an efficient computational (recursive)
means to estimate the state of process, in a way that
minimizes the mean of the squared error.

EXAMPLE

Suppose we have a hydrologic model that predicts river water
level every hour (using the usual inputs). We know that our
model is not perfect and we don‟t trust it 100%. So we want to
send someone to check the river level in person. However, the
river level can only be checked once a day around noon and not
every hour. Furthermore, the person who measures the river
level can not be trusted 100% either. So how do we combine
both outputs of river level (from model and from measurement)
so that we get a „fused‟ and better estimate? KALMAN
FILTERING

APPLICATIONS

 Tracking missiles
 Aircrafts
 Spacecrafts
 GPS technology
 Visual reality
 Progressive method
 Non- stationary
 No large matrices has to be inverted
 Proper dealing with system noise
 Track finding and track fitting
 Detection of outliers
 Merging track from different segments

ASSUMPTIONS OF KALMAN FILTER
 Linear system
 System equations are linear function of parameters
 Measurements are linear function of parameters
 White Gaussian noise
 White: uncorrelated in time
 Gaussian: noise amplitude

DESCRIPTION OF KALMAN FILTER I
System:
our
knowledge
of the system
System
state:
unknown
system
parameters
model
measurement
+
|
noise
noise
? KF
DESCRIPTION OF KALMAN FILTER II

System:
our
knowledge
of the system
System
state:
unknown
system
parameters
model
measurement
KF
Parameters

x

x
k
= A x
k- 1
z
k
= H x
k
+
|
noise
noise
+ w
k
+ v
k
using vectors and matrices
estimation
of
parameters

v
^
STATE-SPACE MODEL
The System Model

process states
output
process noise
measurement noise
system matrix
input matrix
observation matrix
input
k
u
1 k
x
+ k
x
k
H
¿
k
v
k
z 1
z
÷
I
k
A
¿
k
B
Kalman Filter
k
w
1 k k k k k k
k k k k
x A x B u w
z H x v
+
= + +
= +
l
k
n m
k
l n
k
m n
k
m
k
n
k
m
k
n
k
U
H
B
A
v
w
z
x
9 e
9 e
9 e
9 e
9 e
9 e
9 e
9 e
×
×
×
Noise Statistics
Process Noise
Covariance
Measurement Noise
Covariance
Gaussian
Process & measurement noises are independent,
white, zero mean and normal distributed
0
0
k T
n k
k T
n k
Q n k
E w w
n k
R n k
E v v
n k
=
¦
(
=
´
¸ ¸
=
¹
=
¦
(
=
´
¸ ¸
=
¹
0 and
T
n k
E v w n k
(
= ¬
¸ ¸
) , 0 ( ~ ) (
) , 0 ( ~ ) (
k
k
R N v P
Q N w P
STATE ESTIMATION
a-priori State Estimate
a-priori (predicted)
state estimate
a-priori estimation
error
1 1 1 1
ˆ ˆ
k k k k k
x A x B u
÷
÷ ÷ ÷ ÷
= + Update equation
÷
k
x
ˆ
n
9 e
÷ ÷
÷ =
k k k
x x e
ˆ
a-posteriori State Estimate
a-posteriori state
estimate
a-posteriori estimation
error
( )
ˆ ˆ ˆ
k k k k k k
x x K z H x
÷ ÷
= + ÷
Update equation
Innovation
(residual)
Kalman Gain
k k k
x x e
ˆ
÷ =
n
9 e k
x
ˆ
Covariance Matrices
T
k k k
P E e e
÷ ÷ ÷
(
=
¸ ¸
a-priori estimation
error covariance
a-posteriori estimation
error covariance
T
k k k
P E e e
(
=
¸ ¸
KALMAN GAIN COMPUTATION
Optimization Criterion
 Choose Kalman gain that minimizes a-posteriori error covariance
Kalman Filter
T
k k k
P E e e
(
=
¸ ¸
Steps to Kalman Gain Computation
Substitute into

Substitute into

Perform expectations in

Choose so that terms containing are

Solve for to get
Kalman Filter
( )
ˆ ˆ ˆ
k k k k k k
x x K z H x
÷ ÷
= + ÷
ˆ
k k k
e x x = ÷
( )
ˆ ˆ
k k k k k k k
e x x K z H x
÷ ÷
= ÷ ÷ ÷
T
k k k
P E e e
(
=
¸ ¸
k
K
k
K
T
k k k
P E e e
(
=
¸ ¸
( )
1
T T
k k k k k k k
K P H H P H R
÷
÷ ÷
= +
0
k
K
Kalman Gain
( )( )
( ) { } ( ) { }
( ) { }
( ) { }
ˆ ˆ
ˆ ˆ ˆ ˆ
ˆ ˆ
ˆ ˆ
÷ ÷ ÷ ÷
÷ ÷
÷ ÷
(
=
¸ ¸
(
= ÷ ÷
¸ ¸
(
= ÷ ÷ ÷ ÷ ÷ ÷
(
¸ ¸

= ÷ ÷ + ÷ ×

¸
(
÷ ÷ + ÷
(
¸
T
k k k
T
k k k k
T
k k k k k k k k k k k k
k k k k k k k k
T
k k k k k k k k
P E e e
E x x x x
E x x K z H x x x K z H x
E x x K H x v H x
x x K H x v H x
a-posteriori estimate
error covariance
Kalman Gain (‘continued)
( ) { }
( ) { }
( )( ) ( )( )
( )( )
( )( )
ˆ ˆ
ˆ ˆ
ˆ ˆ ˆ ˆ
ˆ ˆ
ˆ ˆ
÷ ÷
÷ ÷
÷ ÷ ÷ ÷
÷ ÷
÷ ÷

= ÷ ÷ + ÷ ×

¸
(
÷ ÷ + ÷
(
¸

= ÷ ÷ ÷ ÷ + ÷

¸
÷ + ÷ ÷
(
+ + ÷ + ÷
(
¸
k k k k k k k k k
T
k k k k k k k k
T T
T
k k k k k k k k k k k k
T
k k k k k k k k
T
T
k k k k k k k k k k k k
P E x x K H x v H x
x x K H x v H x
E x x x x x x H x v H x K
K H x v H x x x
K H x v H x H x v H x K
Kalman Gain (‘continued)
( ) ( ) ( ) ( )
( ) ( )
( ) ( )
( ) ( )
( ) ( )
( )
ˆ ˆ ˆ ˆ
ˆ ˆ
ˆ ˆ
ˆ ˆ
ˆ ˆ
ˆ
T T
T
k k k k k k k k k k k k k
T
k k k k k k k k
T
T
k k k k k k k k k k k k
T
T
k k k k k k k k k k
T
k k k k k k k k
k k k k k k
P E x x x x x x H x v H x K
K H x v H x x x
K H x v H x H x v H x K
P P E x x H x v H x K
E K H x v H x x x
E K H x v H x
÷ ÷ ÷ ÷
÷ ÷
÷ ÷
÷ ÷ ÷
÷ ÷
÷

= ÷ ÷ ÷ ÷ + ÷

¸
÷ + ÷ ÷
(
+ + ÷ + ÷
(
¸
(
= ÷ ÷ + ÷
(
¸ ¸
(
÷ + ÷ ÷
(
¸ ¸
+ + ÷
( )
ˆ
T
T
k k k k k k
H x v H x K
÷ (
+ ÷
(
¸ ¸
Kalman Filter
Kalman Gain (‘continued)
( ) ( )
( ) ( )
( ) ( )
ˆ ˆ
ˆ ˆ
ˆ ˆ
T
T
k k k k k k k k k k
T
k k k k k k k k
T
T
k k k k k k k k k k k k
P P E x x H x v H x K
E K H x v H x x x
E K H x v H x H x v H x K
÷ ÷ ÷
÷ ÷
÷ ÷
(
= ÷ ÷ + ÷
(
¸ ¸
(
÷ + ÷ ÷
(
¸ ¸
(
+ + ÷ + ÷
(
¸ ¸
T
k k k
E v v R
(
=
¸ ¸
( )
T T T T
k k k k k k k k k k k k k k
P P K H P P H K K H P H R K
÷ ÷ ÷ ÷
= ÷ ÷ + +
Linear in Kalman gain
Kalman Gain (‘continued)
Kalman Filter
T T T T
k k k k k k k k k k k k
P P K H P P H K K S S K
÷ ÷ ÷
= ÷ ÷ +
Put
( )
T T T T
k k k k k k k k k k k k k k
P P K H P P H K K H P H R K
÷ ÷ ÷ ÷
= ÷ ÷ + +
into
T T T T T T
k k k k k k k k k k k k
P P K H P P H K K S S K
÷ ÷ ÷
= ÷ ÷ + + ++ ÷++
Used for completing squares
k
T
k k k
T
k k
R H P H S S + =
÷
A
Kalman Gain (‘continued)
Kalman Filter
T T T T T T
k k k k k k k k k k k k
P P K H P P H K K S S K
÷ ÷ ÷
= ÷ ÷ + + ++ ÷++
( )( )
T
T
k k k k k k
T T T T T T T
k k k k k k k k k
P P K S K S
P K S S K K S S K
÷
÷
= + ÷ + ÷ + ÷ ++
= + ÷ + ÷ + + ++ ÷ ++
T T
k k k
P H S
÷ ÷
+ =
T T T T T
k k k k k k k k k k
K S S K K H P P H K
÷ ÷
+ + + = +
Compare the two equations
Holds if
Kalman Gain (‘continued)
Kalman Filter
( )( )
T
T
k k k k k k
P P K S K S
÷
= + ÷ + ÷ + ÷ ++
T T
k k k
P H S
÷ ÷
+ =
Can be minimized if
1
k k
k k
K S
K S
÷
= +
= +
( )
( )
1
1
1
T T
k k k k k
T T
k k k k
T T
k k k k k k
K P H S S
P H S S
P H H P H R
÷ ÷ ÷
÷
÷
÷
÷ ÷
=
=
= +
k
T
k k k
T
k k
R H P H S S + =
÷
A
Interpretation of Kalman Gain
( )
1
T T
k k k k k k k
K P H H P H R
÷
÷ ÷
= +
Kalman Gain weights residual
more heavily
1
0
lim
k
k k
R
K H
÷
÷
=
0
lim 0
k
k
P
K
÷
÷
=
Kalman Gain weights residual less
heavily
( )
ˆ ˆ ˆ
k k k k k k
x x K z H x
÷ ÷
= + ÷
residual
Kalman Gain
Measurement error
covariance
a-priori estimation
error covariance
A- POSTERIORI ESTIMATE ERROR
COVARIANCE
k
P
Computation of (‘continued)
k
P
T T T T
k k k k k k k k k k k
P P K H P P H K P H K
÷ ÷ ÷ ÷
= ÷ ÷ +
( )
k k k k
P I K H P
÷
= ÷
Computation of
( )
T T T T
k k k k k k k k k k k k k k
P P K H P P H K K H P H R K
÷ ÷ ÷ ÷
= ÷ ÷ + +
Substitute
k
P
( )
1
T T
k k k k k k k
K P H H P H R
÷
÷ ÷
= +
into
( ) ( )
1
T T
k k k k k k k k
T T T T
k k k k k k k k k k k
P P K H P P H K
P H H P H R H P H R K
÷ ÷ ÷
÷
÷ ÷ ÷
= ÷ ÷
+ + +
I

A-PRIORI ESTIMATE ERROR
COVARIANCE

k
P
÷
Computation of
k
P
÷
1 1
ˆ ˆ
k k k
x A x
÷
÷ ÷
=
a-priori state
estimate
( )
1 1 1 1 1
1 1 1 1
1 1 1
ˆ
ˆ
ˆ
k k k
k k k k k
k k k k
k k k
e x x
A x w A x
A x x w
A e w
÷ ÷
÷ ÷ ÷ ÷ ÷
÷ ÷ ÷ ÷
÷ ÷ ÷
= ÷
= + ÷
= ÷ +
= +
a-priori state
estimate error
Computation of (‘continued)
Kalman Filter
k
P
÷
( ) ( )
1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1
T
k k k
T
k k k k k k
T T T
k k k k k k k
T T T
k k k k k
P E e e
E A e w A e w
A E e e A A E e E w
E w E e A E w w
÷ ÷ ÷
÷ ÷ ÷ ÷ ÷ ÷
÷ ÷ ÷ ÷ ÷ ÷ ÷
÷ ÷ ÷ ÷ ÷
(
=
¸ ¸
(
= + +
(
¸ ¸
( ( (
= +
¸ ¸ ¸ ¸ ¸ ¸
( ( (
+ +
¸ ¸ ¸ ¸ ¸ ¸
a-priori state
estimate error
covariance
1 k
Q
÷
1 1 1 1
T
k k k k k
P A P A Q
÷
÷ ÷ ÷ ÷
= +
1 1 1
T
k k k
P E e e
÷ ÷ ÷
(
=
¸ ¸
| | | | 0 0
1 1
= . =
÷ ÷
T
k k
w E w E
Discrete Kalman Filter
1 1 1 1
1 1 1
ˆ ˆ
k k k k k
T
k k k k k
x A x B u
P A P A Q
÷
÷ ÷ ÷ ÷
÷
÷ ÷ ÷
= +
= +
Kalman Filter
( )
( )
( )
1
ˆ ˆ ˆ
T T
k k k k k k k
k k k k k k
k k k k
K P H H P H R
x x K z H x
P I K H P
÷
÷ ÷
÷ ÷
÷
= +
= + ÷
= ÷
Time Update
(“predict”)
Measurement Update
(“Correct”)
The time update projects
the current state estimate
The measurement update
adjusts the projected estimate by an
actual measurement at that time
Kalman Filter - Complete Picture
Kalman Filter
1 1 1
T
k k k k k
P A P A Q
÷
÷ ÷ ÷
= +
1 1 1 1
ˆ ˆ
k k k k k
x A x B u
÷
÷ ÷ ÷ ÷
= +
( )
1
T T
k k k k k k k
K P H H P H R
÷
÷ ÷
= +
( )
ˆ ˆ ˆ
k k k k k k
x x K z H x
÷ ÷
= + ÷
( )
k k k k
P I K H P
÷
= ÷
1
ˆ
k
x
÷
÷ 1 k
P
÷

MATLAB SIMULATION
SCALAR KALMAN FILTER
Let's examine the use of a Kalman filter to estimate the
value of a system with a gain a=1. The output gain (h) is
set equal to 3; h=3. The measurement noise (v) covariance
(R) is set to 1; R=1. The initial guess of the state is 1.5;
x0=1.5, and the initial estimate of the a posteriori
covariance is 1; p0=1.
ANALYSING DIFFERENT CASES
Case 1 - No Process Noise
Case 2 - Add Some Process Noise
Case 3 - Increase Process Noise
Case 4 - Increase the Measurement Noise

CASE 1:NO PROCESS NOISE
a=1, h=3, Q=0, R=1; x0=1.5, p0=1

Case 2 - Add Some Process Noise
a=1, h=3, Q=0.01, R=1; x0=1.5, p0=1
Case 3 - Increase Process Noise
a=1, h=3, Q=0.1, R=1; x0=1.5, p0=1
Case 4 - Increase the Measurement Noise
a=1, h=3, Q=0.01, R=2; x0=1.5, p0=1
REFERENCE
 Greg Welch and Gary Bishop “ An Introduction to the Kalman Filter ” ,
TR 95-041 , Department of Computer Science , University of North
Carolina at Chapel Hill , Chapel Hill, NC 27599-3175 , Updated: Monday,
July 24, 2006
 Kalman, R. E. 1960. “A New Approach to Linear Filtering and Prediction
Problems”, Transaction of the ASME--Journal of Basic Engineering, pp.
35-45 (March 1960).
 Saša Fratina and Samo Korpar “ Using Kalman Filter to Track
Particles” , 2004-01-13