38 views

Uploaded by L1architect

MMSE estimation

- HW_CH4(S)
- Gaussian Process Emulation of Dynamic Computer Codes (Conti, Gosling Et Al)
- Modeling and Forecasting Realized Volatility.pdf
- Informal Economy
- CAPM
- Classical linear regression model
- Simo, Styan, Jarkko - Formulas for Linear Regression Analysis.pdf
- Restoring Undersampled Image
- SSPre-PFIEV-Chapter4-2012
- CDVC_LVM_BJMSP.pdf
- HUL215-lecture2.pdf
- dxfgvbdx
- Finding the Efficient Set
- 647221
- tmp722C.tmp
- 03 Systems of Non Linear Equations
- 1.Deep Learning Assignment1 Solutions 1
- cp cpk
- Pobreza e Inteligenciaa
- Isotherm

You are on page 1of 12

S. Lall, Stanford

2011.02.02.01

S. Lall, Stanford

2011.02.02.01

10 - MMSE Estimation

Estimation given a pdf

Minimizing the mean square error

The minimum mean square error (MMSE) estimator

The MMSE and the mean-variance decomposition

Example: uniform pdf on the triangle

Example: uniform pdf on an L-shaped region

Example: Gaussian

Posterior covariance

Bias

Estimating a linear function of the unknown

MMSE and MAP estimation

10 - 2 MMSE Estimation

Suppose x :

Given cost function c : Rn Rn ! R

pick estimate x to minimize E c(x, x)

c(x, x) = kx

xk2

E kx

xk

kx

xk2px(x) dx

10 - 3 MMSE Estimation

S. Lall, Stanford

2011.02.02.01

Lets find the minimum mean-square error (MMSE) estimate of x; we need to solve

min E kx

x

xk2

We have

E kx

xk2 = E (x

x)T (x

x)

= E xT x

2

xT x + xT x

= Ekxk2

2

xT E x + xT x

xmmse = E x

10 - 4 MMSE Estimation

S. Lall, Stanford

The minimum mean-square error estimate of x is

xmmse = E x

E kx

since E kx

xmmsek2 = E kx

E xk2

2011.02.02.01

10 - 5 MMSE Estimation

S. Lall, Stanford

2011.02.02.01

We can interpret this via the MVD. For any random variable z, we have

E kzk2 = E kz

Applying this to z = x

E zk2 + kE zk2

x gives

E kx

xk2 = E kx

x

E xk2 + k

E xk2

The second term is the bias of x; the best we can do is make this zero.

10 - 6 MMSE Estimation

S. Lall, Stanford

2011.02.02.01

We measure y = ymeas.

2

1

0

1

2

: Rm ! R n .

3

2

J = E k (y)

xk2

10 - 7 MMSE Estimation

S. Lall, Stanford

2011.02.02.01

S. Lall, Stanford

2011.02.02.01

Notation

Well use the following notation.

py is the marginal or induced pdf of y

p (y) =

y

p(x, y) dx

p|y (x, y) =

p(x, y)

py (y)

10 - 8 MMSE Estimation

The mean-square-error conditioned on y is econd(y), given by

econd(y) =

k (y)

J = E econd(y)

because

J=

=

Z Z

Z

k (y)

xk2 p(x, y) dx dy

py (y) econd(y) dy

10 - 9 MMSE Estimation

S. Lall, Stanford

2011.02.02.01

S. Lall, Stanford

2011.02.02.01

We can write the MSE conditioned on y as

econd(ymeas) = E k (y)

xk2 | y = ymeas

So we have an MMSE prediction problem for each ymeas

10 - 10

MMSE Estimation

Apply the MVD to z = (y)

x conditioned on y = w to give

xk2 | y = w

econd(w) = E k (y)

= E kx

h(w)k2 | y = w + k (w)

h(w) = E(x | y = w)

To minimize econd(w) we therefore pick

(w) = h(w)

h(w)k2

10 - 11

MMSE Estimation

S. Lall, Stanford

2011.02.02.01

S. Lall, Stanford

2011.02.02.01

With this choice of estimator

econd(w) = E kx

h(w)k2 | y = w

= trace cov(x | y = w)

10 - 12

MMSE Estimation

The MMSE estimator is

mmse (ymeas )

= E(x | y = ymeas)

We often write

xmmse =

mmse (ymeas )

econd(ymeas) = E kx

xmmsek2 y = ymeas

The means and covariance are those of the conditional pdf

The above formulae give the MMSE estimate for any pdf on x and y

10 - 13

MMSE Estimation

S. Lall, Stanford

2011.02.02.01

(x, y) are uniformly distributed on the triangle A.

i.e., the pdf is

(

2 if (x, y) 2 A

p(x, y) =

0 otherwise

0.5

0

0

0.5

the MMSE estimator of x given y = ymeas is

xmmse = E(x | y = ymeas) =

1 + ymeas

2

E kx

10 - 14

xmmsek2 y = ymeas =

1

(ymeas

12

1)2

MMSE Estimation

S. Lall, Stanford

2011.02.02.01

1

0.5

0

E k

mmse (y)

xk2 = E econd(y)

x=0

x=0

1

24

p(x, y) econd(y) dy dx

y=0

1

(y

y=0 6

x

1)2 dy dx

0.5

10 - 15

MMSE Estimation

S. Lall, Stanford

2011.02.02.01

(x, y) are uniformly distributed on the L-shaped

region A, i.e., the pdf is

(

4

if (x, y) 2 A

p(x, y) = 3

0 otherwise

0.5

0

0

xmmse = E(x | y = ymeas) =

(1

E kx

10 - 16

xmmsek

y = ymeas =

if 0 ymeas

2

3

4

if

(1

12

1

48

1

2

0.5

1

2

< ymeas 1

if 0 ymeas

if

1

2

1

2

< ymeas 1

MMSE Estimation

S. Lall, Stanford

2011.02.02.01

1

0.5

0

E k

mmse (y)

xk2 = E econd(y)

0.5

p(x, y) econd(y) dy dx

x=0

1

16

1

2

1

p(x, y) dy dx +

y=0 12

x= 21

1

2

y= 12

1

p(x, y) dy dx

48

10 - 17

MMSE Estimation

S. Lall, Stanford

2011.02.02.01

x

N (, ) where

Suppose

y

x

=

= Tx

y

xy

0.35

0.3

0.25

0.2

0.15

xy

0.1

0.05

0

x | y = ymeas is N (1, 1) where

1 = x +

1

(ymeas

xy

xy

1 T

y

xy

3

2

1

y )

0

1

2

3

10 - 18

MMSE Estimation

S. Lall, Stanford

2011.02.02.01

The MMSE estimator when x, y are jointly Gaussian is

mmse (ymeas )

= x +

econd(ymeas) = trace

xy

(ymeas

xy

y )

1 T

xy

Hence the optimum MSE achieved is

E k

mmse (y)

xk2 = trace

xy

1 T

xy

10 - 19

MMSE Estimation

S. Lall, Stanford

2011.02.02.01

S. Lall, Stanford

2011.02.02.01

Posterior covariance

Lets look at the error z = (y)

x. We have

cov(x | y = ymeas is called the posterior covariance

10 - 20

MMSE Estimation

x

Here

N (0, ) with

y

2.8 2.4

=

2.4 3

3

2

We measure

ymeas = 2

0

1

mmse (ymeas )

Let 2 = QF

= 12 221ymeas

= 0.8ymeas

= 1.6

3

3

xy

yx

= 0.88

10 - 21

MMSE Estimation

S. Lall, Stanford

2011.02.02.01

S. Lall, Stanford

2011.02.02.01

Bias

The MMSE estimator is unbiased; that is the mean error is zero.

E

10 - 22

mmse (y)

x =0

MMSE Estimation

Suppose

p(x, y) is a pdf on x, y

q = Cx is a random variable

we measure y and would like to estimate q

The optimal estimator is

qmmse = C E x | y = ymeas

Because E q | y = ymeas = C E x | y = ymeas

The optimal estimate of Cx is C multiplied by the optimal estimate of x

Only works for linear functions of x

10 - 23

MMSE Estimation

S. Lall, Stanford

2011.02.02.01

Suppose x and y are random variables, with joint pdf f (x, y)

The maximum a posterior (MAP) estimate is the x that maximizes

h(x, ymeas) = conditional pdf of x | (y = ymeas)

xmap = arg max f (x, ymeas)

x

3

2

the conditional pdf is the conditional mean.

1

0

1

to the MAP estimate.

2

3

- HW_CH4(S)Uploaded byAbd Tash
- Gaussian Process Emulation of Dynamic Computer Codes (Conti, Gosling Et Al)Uploaded byMichael Wood
- Modeling and Forecasting Realized Volatility.pdfUploaded byTricia Thaíse
- Informal EconomyUploaded bywaleedthapet
- CAPMUploaded bySeth Okine Bavon
- Classical linear regression modelUploaded byJiya Sharma
- Simo, Styan, Jarkko - Formulas for Linear Regression Analysis.pdfUploaded bydannyleo142
- Restoring Undersampled ImageUploaded bynp_nikhil
- SSPre-PFIEV-Chapter4-2012Uploaded byMinh Hoang
- CDVC_LVM_BJMSP.pdfUploaded byMostafa Salah Elmokadem
- HUL215-lecture2.pdfUploaded byadityalakshay26
- dxfgvbdxUploaded byisaauad
- Finding the Efficient SetUploaded byRochana Ramanayaka
- 647221Uploaded byShiva
- tmp722C.tmpUploaded byFrontiers
- 03 Systems of Non Linear EquationsUploaded byİsmail Eren
- 1.Deep Learning Assignment1 Solutions 1Uploaded byvidhya ds
- cp cpkUploaded byanmol6237
- Pobreza e InteligenciaaUploaded byJuan Ospina
- IsothermUploaded byMaharghya Biswas
- UntitledUploaded byapi-126876773
- Probabilistic analysis of retaining walls.pdfUploaded bySeif15
- AsadSarwar_873_BESE-3A_Ass#1Uploaded byAsad Sarwar
- week8Uploaded byAnonymous 2ZSSleW
- CPU SchedulingUploaded byRsgrsg Rsg
- NonLinFiltNeuroMorphNetworks.pdfUploaded byAnonymous FGY7go
- Bai912.docxUploaded byCao Vũ
- lim2012.pdfUploaded byDan Matthews
- Identification of Frequency Domain Using Quantum Based Optimization Neural NetworksUploaded byInternational Journal of Research in Engineering and Technology
- Combination of Two Exponential Ratio Type Estimators for Estimating Population Mean Using Auxiliary Variable with Double Sampling in Presence of Non-ResponseUploaded byIJSRP ORG

- Motoroal - 802.11ac White PaperUploaded byL1architect
- Base Station Cross Pol AntennasUploaded byL1architect
- Dual Polarized AntennasUploaded byL1architect
- Intro 80211 AcUploaded byHarsha
- sockets-101218053457-phpapp02Uploaded byL1architect
- referat-160404195357.odpUploaded byFakee Zimbo
- sockets-101218053457-phpapp02.pdfUploaded byL1architect
- Channel Models for Massive MimoUploaded byL1architect
- Feb2010 Ims Voip 3g Lte Strategic SwpUploaded byL1architect
- Aruba - 802.11ac - WhitepaperUploaded byL1architect
- Mmse EstimationUploaded byL1architect
- Cisco - 802.11ac - WhitepaperUploaded byL1architect
- Massive MIMOUploaded byL1architect
- 3G NodeB CapacityUploaded byfrangiogt

- WEATHER DISTURBANCE.pptUploaded byKhris Usita
- Robert J. A., Pauline E. K., Linda M. S. - Neural Networks.pdfUploaded byvladislav
- Steel Book 2016Uploaded byThuy Le
- Flood Protection & Future Chances of Flood by Nature & Man Made HAARP FactorUploaded byscparco
- Chapter1.pptxUploaded bykarthiks
- 2slsexampleUploaded byAdy Menon
- Gene_Tierney_A_Biography.pdfUploaded bybluecien
- Potential Risk of Paraffin Wax – RelatedUploaded byJohn Hasting
- 2015 Estimating MDD and OMCUploaded byNg Kok Shien
- Petrobras N-2409-A2Uploaded bytraveller_16812847
- heymes-2006-2.pdfUploaded byAle Eco
- Experiment 4 Lab ReportUploaded bywide fleurantus
- Lecture 1Uploaded byJulian DiVito
- EVIC_05 Notes - Forecasting With Neural Networks Tutorial SFCroneUploaded byasuka123456
- PPC Course newUploaded bymariialuphjc
- 2014May 098 099 HVAC Applications Int HoutUploaded bySantiago J. ramos jr
- Calculating the Amount of Aluminium Roofing Sheet for a BuildingUploaded byDayo Idowu
- Sport Obermeyer Final PDFUploaded bypiyush
- AWSW 2016 Programme WebUploaded byAviation Events International
- Lecture 10 NotesUploaded byLazuardi Hakam
- Zodiac Personalities by Guru Maa Vidyavati JiUploaded byGuruMaaVidyavati
- Transmission Planning ReportUploaded bySurendranath
- 424Uploaded byAykut Başdan
- cogni_ans_1Uploaded bysaif127
- Model Specification.pdfUploaded byRukon Zaman
- 018-025_lawrenceUploaded byibnrafeeq
- Dasa SanthipuUploaded byrajarathnamas
- Use of Meteorological Instruments- 2Uploaded byMinatiBindhani
- [Group1]Samsung Electronics - SCMUploaded byRobert Dang
- Wind Engineering in Sri Lanka – Past, Present and FutureUploaded byNalaka Santhajeewa