Normal equations

Random processes

6.011, Spring 2018

Lec 15 1
 Zillow (founded 2006)

2
© Zillow. All rights reserved. This content is excluded from our Creative Commons license.
For more information, see https://ocw.mit.edu/help/faq-fair-use/
 Zestimates

3
© Zillow. All rights reserved. This content is excluded from our Creative Commons license.
For more information, see https://ocw.mit.edu/help/faq-fair-use/
 LMMSE for multivariate case
min E[(Y { a0 + ⌃ L a X
j=1 j j }) 2
]
a0 ,...,aL | {z }
b`
Y

First min ) a0 = µY – ⌃j =1 aj µXj

L
a0

This ensures unbiasedness of the estimator.

Now min E[(Ye

4
⌃L e
j=1 j j ]
a X ) 2
a1 ,...,aL
Geometric picture

5
 Applying orthogonality gives the
“normal equations”
h⇣ ⌘ i
E Ye - ⌃L e e
j=1 aj Xj Xi = 0

2 32 3 2 3
C X1 X1 CX1 X2 ··· CX 1 X L a1 C X1 Y
6 C X2 X1 CX2 X2 ··· CX 2 X L 76 a2 7 6 CX2 Y 7
6 76 7 6 7
6 .. .. .. .. 76 .. 7=6 .. 7
4 . . . . 54 . 5 4 . 5
C XL X1 CXL X 2 ··· CX L X L aL CXL Y

(CXX ) a = cXY

MMSE: CY2 - cY X (CXX )-1 cXY = CY2 - cY X .a

6
 Estimating mean vector and
covariance matrix from data

Given N independent measurements: Xi , i = 1, · · · , N

N
X
1
Estimate of mean: bX
µ = Xi
N 1

N
X
b XX = 1
Estimate of covariance: C (Xi � µ bX )T
bX )(Xi � µ
N �1 1

7
 Random variable

° Real line

X(c)

8
 Random process

° Amplitude

X(t; c)

c
t1 t

9
Signal ensemble for outcomes a,b,c,d;
& determination of RXX(t1,t2)

X(t) = Xa(t)
t

X(t) = Xb(t)
t

X(t) = Xc(t)
t

X(t) = Xd(t)
t
t1 10
t2
MIT OpenCourseWare
https://ocw.mit.edu

6.011 Signals, Systems and Inference

Spring 2018

For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.

11