Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
1Activity
0 of .
Results for:
No results containing your search query
P. 1
Chapter 5

Chapter 5

Ratings: (0)|Views: 1|Likes:
Published by Jitendra K Jha

More info:

Published by: Jitendra K Jha on Dec 28, 2012
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

04/06/2014

pdf

text

original

 
K.K. GanL5: Maximum Likelihood Method1
Lecture 5Maximum Likelihood Method
 
 x
=
1
 N  x
ii
=
1
 N 
Â
l
Suppose we are trying to measure the true value of some quantity (
 x
).
u
We make repeated measurements of this quantity {
 x
1
,
 x
2
, …
 x
n
}.
u
The standard way to estimate
 x
from our measurements is to calculate the mean value:
 
+
set
 x
=
 x
.
+
DOES THIS PROCEDURE MAKE SENSE???
+
MLM: a general method for estimating parameters of interest from data.
l
Statement of the Maximum Likelihood Method
u
Assume we have made
 N 
measurements of 
 x
{
 x
1
,
 x
2
, …
 x
n
}.
u
Assume we know the probability distribution function that describes
 x
:
 f 
(
 x
,
).
u
Assume we want to determine the parameter
.
+
MLM: pick 
to maximize the probability of getting the measurements (the x
i
's) that we did!
l
How do we use the MLM?
u
The probability of measuring
 x
1
is
 f 
(
 x
1
,
)
dx
u
The probability of measuring
 x
2
is
 f 
(
 x
2
,
)
dx
u
The probability of measuring
 x
n
is
 f 
(
 x
n
,
)
dx
u
If the measurements are independent, the probability of getting the measurements we did is:
u
We can drop the
dx
n
term as it is only a proportionality constant
+
 
 L
=
(
 x
1
,
)
dx
(
 x
2
,
)
dx
(
 x
n
,
)
dx
=
(
 x
1
,
)
(
 x
2
,
)
(
 x
n
,
)
dx
n
 
 L
=
(
 x
ii
=
1
 N 
,
)
Likelihood Function
 
K.K. GanL5: Maximum Likelihood Method2
u
We want to pick the
that maximizes
 L
:
u
Both
 L
and ln
 L
have maximum at the same location.
+
maximize ln
 L
rather than
 L
itself because ln
 L
converts the product into a summation.
+
new maximization condition:
n
could be an array of parameters (e.g. slope and intercept) or just a single variable.
n
equations to determine
range from simple linear equations to coupled non-linear equations.
l
Example:
u
Let
 f 
(
 x
,
) be given by a Gaussian distribution.
u
Let
=
be the mean of the Gaussian.
u
We want the best estimate of 
from our set of 
n
measurements {
 x
1
,
 x
2
, …
 x
n
}.
u
Let’s assume that
is the same for each measurement.
u
The likelihood function for this problem is:
 
∂ 
 L
∂ 
=
*
=
0
 
ln
 L
=
ln
(
 x
i
,
)
i
=
1
 N 
Â
 
∂ 
ln
 L
∂ 
=
*
=
∂ ∂ 
i
=
1
 N 
Â
ln
(
 x
i
,
)
=
*
=
0
 
 f 
(
 x
i
,
)
=
1
s  
2
e
-
(
 x
i
-
)
2
2
s  
2
 
 L
=
(
 x
i
,
)
i
=
1
n
=
1
s  
2
e
-
(
 x
i
-
)
2
2
s  
2
i
=
1
n
=
1
s  
2
ÈÎ͢˚˙
n
e
-
(
 x
1
-
)
2
2
s  
2
e
-
(
 x
2
-
)
2
2
s  
2
L
e
-
(
 x
n
-
)
2
2
s  
2
=
1
s  
2
ÈÎ͢˚˙
n
e
-
(
 x
i
-
)
2
2
s  
2
i
=
1
n
Â
 
K.K. GanL5: Maximum Likelihood Method3
u
 
Find
that maximizes the log likelihood function:
u
If 
are different for each data point
+
is just the weighted average:
 
∂ 
ln
 L
∂ 
=
∂ ∂ 
n
ln1
s  
2
ÊËÁˆ¯˜-
(
 x
i
-
)
2
2
s  
2
i
=
1
n
ÂÈÎ͢˚˙=
0
∂ ∂ 
(
 x
i
-
)
2
i
=
1
n
Â=
02(
 x
ii
=
1
n
Â-
)(
-
1)
=
0
 x
ii
=
1
n
Â=
n
=
1
n x
ii
=
1
n
Â
 
=
 x
i
s  
i
2
i
=
1
n
Â
1
s  
i
2
i
=
1
n
Â
AverageWeighted average

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->