Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Standard view
Full view
of .
0 of .
Results for:
P. 1
Chapter 5

# Chapter 5

Ratings: (0)|Views: 1|Likes:

### Availability:

See more
See less

04/06/2014

pdf

text

original

K.K. GanL5: Maximum Likelihood Method1
Lecture 5Maximum Likelihood Method

x
=
1
N  x
ii
=
1
N
Â
l
Suppose we are trying to measure the true value of some quantity (
x
).
u
We make repeated measurements of this quantity {
x
1
,
x
2
, …
x
n
}.
u
The standard way to estimate
x
from our measurements is to calculate the mean value:

+
set
x
=
x
.
+
DOES THIS PROCEDURE MAKE SENSE???
+
MLM: a general method for estimating parameters of interest from data.
l
Statement of the Maximum Likelihood Method
u
N
measurements of
x
{
x
1
,
x
2
, …
x
n
}.
u
Assume we know the probability distribution function that describes
x
:
f
(
x
,
).
u
Assume we want to determine the parameter
.
+
MLM: pick
to maximize the probability of getting the measurements (the x
i
's) that we did!
l
How do we use the MLM?
u
The probability of measuring
x
1
is
f
(
x
1
,
)
dx
u
The probability of measuring
x
2
is
f
(
x
2
,
)
dx
u
The probability of measuring
x
n
is
f
(
x
n
,
)
dx
u
If the measurements are independent, the probability of getting the measurements we did is:
u
We can drop the
dx
n
term as it is only a proportionality constant
+

L
=
(
x
1
,
)
dx
(
x
2
,
)
dx
(
x
n
,
)
dx
=
(
x
1
,
)
(
x
2
,
)
(
x
n
,
)
dx
n

L
=
(
x
ii
=
1
N
,
)
Likelihood Function

K.K. GanL5: Maximum Likelihood Method2
u
We want to pick the
that maximizes
L
:
u
Both
L
and ln
L
have maximum at the same location.
+
maximize ln
L
rather than
L
itself because ln
L
converts the product into a summation.
+
new maximization condition:
n
could be an array of parameters (e.g. slope and intercept) or just a single variable.
n
equations to determine
range from simple linear equations to coupled non-linear equations.
l
Example:
u
Let
f
(
x
,
) be given by a Gaussian distribution.
u
Let
=
be the mean of the Gaussian.
u
We want the best estimate of
from our set of
n
measurements {
x
1
,
x
2
, …
x
n
}.
u
Let’s assume that
is the same for each measurement.
u
The likelihood function for this problem is:

∂
L
∂
=
*
=
0

ln
L
=
ln
(
x
i
,
)
i
=
1
N
Â

∂
ln
L
∂
=
*
=
∂ ∂
i
=
1
N
Â
ln
(
x
i
,
)
=
*
=
0

f
(
x
i
,
)
=
1
s
2
e
-
(
x
i
-
)
2
2
s
2

L
=
(
x
i
,
)
i
=
1
n
=
1
s
2
e
-
(
x
i
-
)
2
2
s
2
i
=
1
n
=
1
s
2
ÈÎÍ˘˚˙
n
e
-
(
x
1
-
)
2
2
s
2
e
-
(
x
2
-
)
2
2
s
2
L
e
-
(
x
n
-
)
2
2
s
2
=
1
s
2
ÈÎÍ˘˚˙
n
e
-
(
x
i
-
)
2
2
s
2
i
=
1
n
Â

K.K. GanL5: Maximum Likelihood Method3
u

Find
that maximizes the log likelihood function:
u
If
are different for each data point
+
is just the weighted average:

∂
ln
L
∂
=
∂ ∂
n
ln1
s
2
ÊËÁˆ¯˜-
(
x
i
-
)
2
2
s
2
i
=
1
n
ÂÈÎÍ˘˚˙=
0
∂ ∂
(
x
i
-
)
2
i
=
1
n
Â=
02(
x
ii
=
1
n
Â-
)(
-
1)
=
0
x
ii
=
1
n
Â=
n
=
1
n x
ii
=
1
n
Â

=
x
i
s
i
2
i
=
1
n
Â
1
s
i
2
i
=
1
n
Â
AverageWeighted average