An Introduction To Applied Optimal Estimation

BY
Larry N. Lillard, SMRT, Inc

Optimal Estimation Question

~
Give the following linear system where X$ 0 and Y are random variables:

~
Y = A X + v; lq
E v = 0, n s
E vv T = R (1-3)

$ = X + e;
X 0 lq
E e = 0, n s
E ee T = P0 (4-6)

Assuming both random variables have Gaussian distributions, some references ((e.g.
Sorenson) give the minimum variance estimate of X as

X 0c
$ = P −1 + A T R − 1 A h dP−1 −1
0 X 0
~
$ + A T R −1 Y i (7)

{d
$ −X X
E X id
$ −X i } = cP
T
−1
0 + A T R −1 A h −1
(8)

Others (e.g. Gelb), give the Optimal estimate as

c
$ = P −1 + A T R − 1 A
X 0 h −1 ~
A T R −1 Y (9)

{d
$ −X X
E X id
$ −X i } = cP
T
−1
0 + A T R −1 A h −1
(10)

Why the difference?

 A Priori Form -- Linear Case
(see Gelb p145-146 and Kamen & Su p154)

~
Y = A X + v; lq
E v = 0, n s
E vv T = R (1-3)

$ + e;
X=X 0 lq
E e = 0, n s
E ee T = P0 , n s
E Xv T = 0 (4-7)

~
d
J = Y − AX
~
$ T R −1 Y i
$ + X
− AX $ −X
0 0d
$ T P −1 X
$ −X
$
0 i d i d i (8)

∂J
$
∂X
~
= −2 Y − A X d
$ T R −1 A + 2 X
$ −X
0 i
$ T P −1 = 0 T
0 d i (9)

~
d
$ − P −1 X
A T R −1 Y − A X 0
$ −X
$ =0
0 i d i (10)

~ $ − P −1 X
$ + P −1 X
$ =0
A T R −1 Y − A T R −1A X 0 0 0 (11)

X 0 c
$ = P −1 + A T R −1A h dP
−1 −1
0 X 0
~
$ + A T R −1 Y i (12)

$ −X X
P≡E X {d
$ −X id i } = cP
T
−1
0 + A T R −1 A h −1
(13)

Another way (If you want a sequential estimator)

$ =X
Let ∆X $ −X
$ (14)
0

$ + ∆X
Then (12) becomes X 0
$ = P −1 + A T R − 1 A
0 c h dP
−1 −1
0 X 0
~
$ + A T R −1 Y i (15)

cP−1
0 + A T R −1A ∆X 0h
$ + P −1 X
$ + A T R −1 A X
0
$ = P −1 X
0 0 0
~
$ + A T R −1 Y (16)

c
∆X = P0−1 + A T R −1A h −1
d $
A T R −1 Y − A X 0 i (17)
 c
X i +1 = X i + Pi−1 + A T R i−+11A h −1
d $
A T R i−+11 Y i +1 − A X i i (18)

c
Pi +1 = Pi−1 + A T R i−+11A h −1
(19)
 A Priori Form -- Linear Case
(Slightly different approach -- same result)

~
Y = A X + v; lq
E v = 0; n s
E vv T = R (1-3)

$ = X + e;
X 0 lq
E e = 0; n s
E ee T = P0 ; n s
E Xv T = 0 (4-7)

d
~
J = Y − AX i
~
$ T R −1 Y $ + X
− AX $ −X
0 d
$ T P −1 X
0
$ −X
0
$ i d i d i (8)

∂J
$
∂X
~
d
$ T R −1 A − 2 X
= −2 Y − A X i
$ −X
0
$ T P −1 = 0 T
0 d i (9)

~
d $ − P −1 X
A T R −1 Y − A X 0 i
$ −X
$ =0
0 d i (10)

~ $ − P −1 X
$ + P −1 X
$ =0
A T R −1 Y − A T R − 1 A X 0 0 0 (11)

X c
$ = P −1 + A T R − 1 A
0 h dP
−1 −1
0 X 0
~
$ + A T R −1 Y i (12)

{d
$ −X X
P≡E X $ −X id i } = cP
T
−1
0 + A T R −1 A h −1
(13)

For a sequential estimator, solve (13) for P0−1 and substitute in (12) to obtain

P0−1 = P −1 − A T R −1A (14)

$ = P P −1 X
X d
$ − A T R −1 A X
0 0
~
$ + A T R −1 Y ~
$ + PA T R −1 Y
=X 0
$
− AX 0 i d i
$ =X
X c
$ + P −1 + A T R − 1 A
0 0 h −1 ~
d $
A T R −1 Y − A X 0 i (15)
 c
X i +1 = X i + Pi−1 + A T R i−+11A h −1
d $
A T R i−+11 Y i +1 − A X i i (16)

c
Pi +1 = Pi−1 + A T R i−+11A h −1
(17)
 A Priori Form -- Non-Linear Case
(See the linear case for reference)
~
bg
Y = h X + v; lq
E v = 0; n s
E vv T = R (1-3)

$ = X + e;
X 0 lq
E e = 0; n s
E ee T = P0 ; m r
E ev T = 0 (4-7)

$ (X
Linearize (1) about some estimate X $ could be X
$ ) of X to obtain
k k 0

d i d i dX − X$ i + h. o. t. + v
$
∂h X
~ $ + k
Y=h X k k (8)
$
∂X

~ $ , A≡
Defining y ≡ Y − h X
$
∂h X
dik d i d
$ , and ignoring the higher order
, and x ≡ X − X k i
∂X
terms, we can rewrite (8) as

y = Ax + v (9)

d T
i d
$ −X
J = y − A x$ R −1 y − A x$ + X 0
$ T P −1 X
0 i d
$ −X
0
$ i d i (10a)
or
d i T
J = y − A x$ R −1 y − A x$ + X d
$ −X
$ P −1
0 0 i d i d T
$ −X
X $
0 i (10b)

$ −X
Letting x$ 0 = X $ , the cost function to be minimized becomes
0 k

d i T
d i b
J = y − A x$ R −1 y − A x$ + x$ − x$ 0 P0−1 x$ − x$ 0 g b
T
g (11)

∂J
∂ x$
d T
i T
b
= −2 y − A x$ R −1A + 2 x$ − x$ 0 P0−1 = 0
T
g (12)

d i b
A T R −1 y − A x$ − P0−1 x$ − x$ 0 = 0 g (13)

A T R −1 y − A T R −1A x$ − P0−1 x$ + P0−1 x$ 0 = 0 (14)

 c
x$ = P0−1 + A T R −1A h dP−1 −1
0 x$ 0 + A T R −1 y i (15)

$ $
X k +1 = X k + x (16)
$

{d
$ −X X
P≡E X $ −X id i } = cP
T
−1
0 + A T R −1 A h −1
(17)

$ rather than X
If we linearize (1) about X $ , then the cost function (11) becomes
0 k

d i T
d
J = y − A x$ R −1 y − A x$ + x$ P0−1 x$
T
i (18)

And the value of x$ which minimizes (18) is

c
x$ = P0−1 + A T R −1A h −1
A T R −1 y (19)

However if we are iterating, and we should be for the non-linear problem,

then in general X$ k will not be X$ 0 . So in the general we should be using
Equation 15.
 Again the question:
~
Give the following linear system where X$ 0 and Y are random variables:

~
Y = A X + v; lq
E v = 0, n s
E vv T = R (1-3)

$ = X + e;
X 0 lq
E e = 0, n s
E ee T = P0 (4-6)

Assuming both random variables have Gaussian distributions, some references ((e.g.
Sorenson) give the minimum variance estimate of X as

X 0c
$ = P −1 + A T R − 1 A h dP−1 −1
0 X 0
~
$ + A T R −1 Y i (7)

{d
$ −X X
E X id
$ −X i } = cP
T
−1
0 + A T R −1 A h −1
(8)

Others (e.g. Gelb), give the Optimal estimate as

c
$ = P −1 + A T R − 1 A
X 0 h −1 ~
A T R −1 Y (9)

{d
$ −X X
E X id
$ −X i } = cP
T
−1
0 + A T R −1 A h −1
(10)

Why the difference? Email me at lnlillard@aol.com if you have an answer or questions.