You are on page 1of 43

# •PROPERTIES OF ESTIMATORS

1. Unbiased: E (W) = θ (centered around true
value)
2. Consistent: W ≈ θ for large number of
observations
3. Efficient: W1 is better than W2 if
4. Minimum Variance: efficient AND
achieve CRLB
5. Sufficient statistic for θ
2 2
) 2 ( ) 1 ( u u ÷ < ÷ W E W E
ˆ
u
 Let be an estimator for θ, we say
that is unbiased for θ if E( )= θ.
If E( )= θ+b(θ) with b(θ)≠0, then
is a biased estimator for θ with
bias b(θ).
ˆ
u
ˆ
u
ˆ
u
ˆ
u
ˆ
u
If one writes of estimation, then an
accurate estimator would be the one resulting in
small estimation errors, so that estimated values will
be near the true value.
error + =u u
ˆ
 If an estimator t
n
estimates u then difference
between them (t
n
-u ) is called the estimation
error. Bias of the estimator is defined as the
expectation value of this difference:
B
u
=E(t
n
-)=E(t
n
)- u
 If the bias is equal to zero then the estimation
is called unbiased. For example sample mean
is an unbiased estimator:
0 ) (
1
) (
1
)
1
( ) (
1 1 1
= ÷ = ÷ = ÷ = ÷
¿ ¿ ¿
= = =
µ µ µ µ
n
i
n
i
i
n
i
i
x E
n
x E
n
x
n
E x E
 Here we used the fact that expectation and
summation can change order (Remember that
expectation is integration for continuous random
variables and summation for discrete random
variables.) and the expectation of each sample
point is equal to the population mean.
 Knowledge of population distribution was not
necessary for derivation of unbiasedness of the
sample mean. This fact is true for the samples
taken from population with any distribution for
which the first moment exists..
 A biased estimator can be an unbiased
estimator when n is large.
 If , then the estimator is
asymptotically unbiased.
u
ˆ
u
ˆ
lim ( ) 0
n
b u
÷·
=
 Suppose that X
1
, X
2
,…, X
n
constitute a random
sample from a distribution with mean μ
1
and
standard deviation σ
1
, and that Y
1
, Y
2
,…, Y
n
(independent of X
i
’s) constitute a random sample
from a distribution with mean μ
2
and standard
deviation σ
2
.
 Use the rules of expected value to show that X – Y
is an unbiased estimator of μ
1
- μ
2
.
Solution:
2 1
] [ ] [ ] [
µ µ ÷ =
÷ = ÷ Y E X E Y X E
 We have shown that when sampling from
, one finds that the maximum likelihood estimators of and
are
Recalling that the distribution of
Thus, we know that the distribution of
Hence,
2
1 2
( , ) N u µ u o = =
µ
2
o
2
2
1 2
( 1)
ˆ ˆ
ˆ ˆ and
n S
X
n
u µ u o
÷
= = = =
2
is ( , ), we see that ( ) ; X N E X
n
o
µ µ =
is unbiased estimator of . X µ
2 2 2 2
1
2
1
( 1) 1
is where ( )
1
n
n i
i
n
S S X X
n
_
o
÷
=
÷
= ÷
÷
¿
2 2 2
2 2
2
( 1)
( ) ( 1)
1 1
n S
E S E n
n n
o o
o
o
( ÷
= = ÷ =
(
÷ ÷
¸ ¸
 Therefore, S
2
is an unbiased estimator of o
2
 Consequently, since
 Therefore,
2 2 2
2
( 1) ( 1) ( 1)
ˆ
( ) [ ] [ ]
n n n
E E S E S
n n n
u o
÷ ÷ ÷
= = =
2
2 2
ˆ
is a biased estimator of u u o =
 Expectation value of the square of the differences
between estimator and the expectation of the
estimator is called its variance:
Exercise: What is the variance of the sample mean.
 As we noted if estimator for u is t
n
then difference
between them is error of the estimation. Expectation
value of this error is bias. Expectation value of square
of this error is called mean square error (m.s.e.):
2
) ( u
u
÷ =
n
t E M
2
)) ( (
n n
t E t E V ÷ =
u
 It can be expressed by the bias and the
variance of the estimator:
 M.S.E is equal to square of the estimator’s bias plus
variance of the estimator. If the bias is 0 then MSE is
equal to the variance.
 In estimation it is usually trade of between
unbiasedness and minimum variance. In ideal world
we would like to have minimum variance unbiased
estimator. It is not always possible.
) ( ) (
) ) ( ( )) ( ( ) ) ( ) ( ( ) ( ) (
2
2 2 2 2
n n
n n n n n n n n
t B t V
t E t E t E t E t E t E t E t M
u u
u
u u u
+
= ÷ + ÷ = ÷ + ÷ = ÷ =
UNBIASED!!!!
 Let S
2
be a sample variance for a random sample
 from and . Is this
estimator biased or unbiased estimator? Note: mean
of chi-square is it df.
Solution:
since E(S
2
)≠o
2
; therefore S
2
is biased estimator and the
biased value is - o
2
/n.
1 2
, ,...,
n
X X X
2
( , ) N µ o
2
2
( )
i
x x
s
n
÷
=
¿
2
2
2 2
1 1
2
2
2 2
2
2
2
2
2
2
2
( )
( 1) ~ ; ~
( )
( ) [ ]
( )
[ ]
( 1)
i
n n
i
i
x x
s
n
n
x x
E s E
n
x x
E
n
n
n
n
_ _
o
o
o
o
o
o
o
o
÷ ÷
÷
÷
÷
=
÷
=
= ÷
= ÷
¿
¿
¿
 If an estimate is such that it’s biased
converged to zero as n·, then we say that it
is asymptotically unbiased.
 S
2
is asymptotically unbiased when n·
2
2 2
2 2
( )
lim ( ) .
n
E s
n
E s
o
o
o
÷·
= ÷
=
 Solution:
2
2
( )
If , does this estimator is unbiased
1
estimator of ?
i
x x
w
n
o
÷
=
÷
¿
2
2
2
2
2
2
2
2
( )
( ) [ ]
( 1)
( )
[ ]
1
( 1)
1
i
i
x x
E w E
n
x x
E
n
n
n
o
o
o
o
o
o
÷
=
÷
÷
=
÷
= ÷
÷
=
¿
¿
w is unbiased estimator for
o
2
 THEOREM
 Let be an unbiased estimator for u
based on a random sample of size n,
from function , assume we can differentiate
inside the integral whenever needed, then CRLB for
the Var(y) is given by
1 2
( , ,..., )
n
y U x x x =
1 2
, ,...,
n
x x x
( ; ) f x u
2
2
2
2
2
1
ln ( , )
1
ln ( , )
y
f x
nE
f x
n E
o
u
u
u
u
>
¦ ¹
c ¦ ¦ (
´ `
(
c
¸ ¸
¦ ¦
¹ )
=
¦ ¹
( ÷c ¦ ¦
´ `
(
c
¸ ¸
¦ ¦
¹ )
Cramer Rao
Lower bound
 Corrollary
 Suppose y is nor unbiased; say E(y)=u+b(u),
then CRLB for y is:
| |
2
2
2
2
2
1 '( )
ln ( , )
y
b
f x
n E
u
o
u
u
+
>
¦ ¹
(
÷c ¦ ¦
´ `
(
c
¸ ¸
¦ ¦
¹ )
 Get the CRLB and find the variance for this
estimator
1
, 0
( ; ) ; ~ ( 1, )
0 ,
x
e x
f x y gamma
elsewhere
u
o
u o | u
u
÷
¦
< < ¦
= = =
´
¦
¹
 Solution:
2
2
2 2 3
2
2 2 3
2 2
2
ln ( ; ) ln
ln ( ; ) 1
ln ( ; ) 1 2
ln ( ; ) 1 2 ( )
[ ]
1 2
1
x
f x
f x x
f x x
f x E x
E E
u u
u
u
u u u
u
u u u
u
u u u
u u
u
= ÷ ÷
c
= ÷ +
c
c
= ÷ +
c
( c
= ÷ +
(
c
¸ ¸
= ÷ +
=
2
2
2
2
1
1
CRLB:
y
y
n
n
o
u
u
o
>
(
(
¸ ¸
>
2 2 2
1
( ) 1.
( ) 1.
; ( )
ˆ
E X
V X
x
M x E X
n
x
o| u u
o| u u
u
u
= = =
= = =
= = =
=
¿
 Therefore, variance for equal to CRLB.
 If an unbiased estimator achive the CRLB;
we say that is efficient for estimating u and
is called uniformly minimum- variance (or
best) unbiased estimator
2
2
2
2 2
2 2
ˆ
( ) ( ) ( )
1
( )
1
( )
1 1
i
i
x
V V x V
n
V x
n
V x
n
n
n n n
u
u
u u
= =
=
=
= = =
¿
¿
¿
¿
ˆ
u
ˆ
u
ˆ
u
ˆ
u
 If we consider all possible unbiased estimator of
some parameter u , the one with the smallest
variance is called the most efficient estimator of u.
1
ˆ
u
u
2
ˆ
u
3
ˆ
u
Sampling distribution of different estimators of u
Unbiased
estimators
is the best
estimator of u
1
ˆ
u
 Let be an estimator that is unbiased for u.
The relative efficiency for is defined to be:

ˆ
u
ˆ
u
ˆ
( ) 1
ˆ
( )
CRLB
eff
V
u
u
= s
 If and are biased estimators of parameter θ of
a given population, the measure of efficiency of
relative to is denoted by the ratio
where
1
ˆ
u
2
ˆ
u
2 2
ˆ ˆ ˆ ˆ
( ) [( ) ] ( ) ( ) MSE E Var b u u u u u = ÷ = +
1
ˆ
u
2
ˆ
u
2
1 2
1
ˆ
( )
ˆ ˆ
( , )
ˆ
( )
MSE
eff
MSE
u
u u
u
=
Mean square error
CAREFUL HERE
What it means Conclusion
2
1 2
1
ˆ
( )
ˆ ˆ
( , )
ˆ
( )
MSE
eff
MSE
u
u u
u
=
1 2
ˆ ˆ
( , ) 1 eff u u =
1 2
ˆ ˆ
( , ) 1 eff u u >
1 2
ˆ ˆ
( , ) 1 eff u u <
1 2
ˆ ˆ
( ) ( ) MSE MSE u u =
1 2
ˆ ˆ
( ) ( ) MSE MSE u u <
1 2
ˆ ˆ
( ) ( ) MSE MSE u u >
1
ˆ
u
2
ˆ
u
1
ˆ
u
2
ˆ
u
1
ˆ
u
2
ˆ
u
as efficient as
more efficient
than
less efficient
than
 If and are unbiased estimators of parameter θ
of a given population, and , then we say
that is relatively more efficient than .
 The measure of efficiency of relative to is
denoted by the ratio
1
ˆ
u
2
ˆ
u
1 2
ˆ ˆ
( ) ( ) Var Var u u <
1
ˆ
u
2
ˆ
u
1
ˆ
u
2
ˆ
u
2
1 2
1
ˆ
( )
ˆ ˆ
( , )
ˆ
( )
Var
eff
Var
u
u u
u
=
CAREFUL HERE
What it means Conclusion
2
1 2
1
ˆ
( )
ˆ ˆ
( , )
ˆ
( )
Var
eff
Var
u
u u
u
=
1 2
ˆ ˆ
( , ) 1 eff u u =
1 2
ˆ ˆ
( , ) 1 eff u u >
1 2
ˆ ˆ
( , ) 1 eff u u <
1 2
ˆ ˆ
( ) ( ) Var Var u u =
1 2
ˆ ˆ
( ) ( ) Var Var u u <
1 2
ˆ ˆ
( ) ( ) Var Var u u >
1
ˆ
u
2
ˆ
u
1
ˆ
u
2
ˆ
u
1
ˆ
u
2
ˆ
u
as efficient as
more efficient
than
less efficient
than
 Suppose that X
1
and X
2
are random samples
from a normal distribution with mean μ and
standard deviation σ. Two estimators for
mean μ are as below:
 Check whether the two estimators and
are unbiased.
 Determine the measure of efficiency of
relative to .
1 2
1
2
3
X X
µ
+
=
1 2
2
2
X X
µ
+
=
1
µ
2
µ
1
µ
2
µ
 Check whether the two estimators and are
unbiased.
1 2
1
1 2
2
( )
3
1 2
( ) ( )
3 3
1 2
3 3
X X
E E
E X E X
µ
µ µ
µ
+
| |
=
|
\ .
= +
= +
=
1 2
2
1 2
( )
2
1 1
( ) ( )
2 2
1 1
2 2
X X
E E
E X E X
µ
µ µ
µ
+
| |
=
|
\ .
= +
= +
=
1
µ
2
µ
 Since both and ,then the two
estimators and are unbiased.
1
( ) E µ µ =
2
( ) E µ µ =
1
µ
2
µ
 Determine the measure of efficiency of relative to
1 2
1
1 2
2 2
2
2
( )
3
1 4
( ) ( )
9 9
1 4
9 9
5
9
X X
Var Var
Var X Var X
µ
o o
o
+
| |
=
|
\ .
= +
= +
=
1 2
2
1 2
2 2
2
( )
2
1 1
( ) ( )
4 4
1 1
4 4
1
2
X X
Var Var
Var X Var X
µ
o o
o
+
| |
=
|
\ .
= +
= +
=
1
µ
2
µ
 Since both and ,then the two
estimators and are unbiased.
1
( ) E µ µ =
2
( ) E µ µ =
1
µ
2
µ
 So, the measure of efficiency of relative to
( )
( )
2
1 2
1
2
2
( , )
2
5
9
9
0.9 1
10
Var
eff
Var
µ
µ µ
µ
o
o
=
=
= = <
1
µ
2
µ
 Since that is ,then the
estimator is 0.9 times more efficient than .
1
µ
2
µ
1 2
ˆ ˆ
( , ) 1 eff u u <
1 2
ˆ ˆ
( ) ( ) Var Var u u >
 Suppose that X
1
, X
2
and X
3
are
random samples from a normal
distribution with mean μ and standard
deviation σ. Determine the efficiency
measure of relative to
1 2 3
2
4
X X X + +
1 2 3
3
X X X + +
 An estimator is said to be consistent for
ˆ
|
ˆ
if .
P
o u u ÷÷÷
 An unbiased estimator is said to be
consistent if the difference between the
estimator and the parameter grows smaller
as the sample size grows larger.
 E.g. is a consistent estimator of
because: V(X) is
 That is, as n grows larger, the variance of
grows smaller.
X
X
 Let be a random sample from
function f(x;u), ueO. Let y
1
=U( ) be a
statistic with pdf g(y
1
,u). We say that y
1
is a
sufficient statistic for the family u
if and only if
Where for every fix value of the
func H( ) does not depend on u, thus
the conditional pdf of given
is free of u
1 2
, ,...,
n
x x x
1 2
, ,...,
n
x x x
{ }
( , ),
i
F f x
u
u u = eO
( )
1 2
1 2
1 1
1 2
, ( , )... ( , )
joint pdf for , ,...,
( , ) ( , )
( , ,..., )
n
n
n
f x f x f x
x x x
g y g y
H x x x
u u u
u u
=
=
y
1
=U( )
1 2
, ,...,
n
x x x
1 2
, ,...,
n
x x x
1 2
, ,...,
n
x x x
1 1
Y y =
 Let say is a r.s from
1 2
, ,...,
n
x x x
1
1
, 0,1,...
( ; )
!
0 ,
is is a sufficient for ?
x
i
e
x
f x
x
elsewhere
y x
y
u
u
u
u
÷
¦
=
¦
=
´
¦
¹
=
¿
 i. find joint pdf for
 ii.
 Conc: the given func. Is a sufficient statistic.
1 2
, ,...,
n
x x x
1 2
1 1
( , ) ( , )... ( , ) ( , )
!
i
x
n n
n i
i i
e
f x f x f x f x
x
u
u
u u u u
÷
= =
= =
[ [
1
! ( )!
!
!
( )! ( )!
i
i
i i
x
n
x
n
i i
i
x x
n n
i
i i
e
e
x x
x
n x
n e n e
x x
u
u
u u
u
u
u u
÷
÷
=
÷ ÷
¿
= =
¿ ¿
[
[ ¿
[
¿ ¿
Free of u
 Note:
◦ There is another way to find sufficient statistic by
Neyman’s Factorization Theorem.
 Theorem:
◦ Let be a random sample from function
f(x;u), ueO. Let y
1
=U( ) be a statistic of
g(y
1
,u) that y
1
is a sufficient statistic for family u
if and only if there is exist a
function k
1
(.) if k
2
(.)
where for each fix value of y
1
=U( ) the
function k
2
( ) does not depend on u in
form or in domain.
1 2
, ,...,
n
x x x
1 2
, ,...,
n
x x x
{ }
( , ),
i
F f x
u
u u = eO
1 1 1 2 2 1 2
1
( ; ) ( ( , ,..., )) ( , ,..., )
n
n n
i
f x k U x x x k x x x u
=
÷ =
[
1 2
, ,...,
n
x x x
1 2
, ,...,
n
x x x
Is just a statistic
 Let is a r.s from
 Solution:
1 2
, ,...,
n
x x x
1
2
(2 )
[ (1 )] , 0 1
[ ( )] ( ; )
0 ,
Find sufficient statistic for
x x x
f x
elsewhere
u
u
u u
u
÷
I
¦
÷ < <
¦
I =
´
¦
¹

1. 2. 3.

Unbiased: E (W) = θ (centered around true value) Consistent: W ≈ θ for large number of observations Efficient: W1 is better than W2 if

4. 5.

E (W1   ) 2  E (W 2   ) 2 ˆ Minimum Variance:  efficient AND
achieve CRLB Sufficient statistic for θ

ˆ  Let 

be an estimator for θ, we say ˆ ˆ that  is unbiased for θ if E( )= θ. ˆ ˆ If E( )= θ+b(θ) with b(θ)≠0, then  is a biased estimator for θ with bias b(θ).

ˆ If one writes     error of estimation, then an accurate estimator would be the one resulting in small estimation errors, so that estimated values will be near the true value.

If an estimator tn estimates  then difference between them (tn- ) is called the estimation error. Bias of the estimator is defined as the expectation value of this difference: If the bias is equal to zero then the estimation is called unbiased. For example sample mean is an unbiased estimator:
1 n 1 n 1 n E ( x   )  E (  xi   )   E ( xi )     E ( x )    0 n i 1 n i 1 n i 1

B=E(tn-)=E(tn)- 

. Here we used the fact that expectation and summation can change order (Remember that expectation is integration for continuous random variables and summation for discrete random variables. This fact is true for the samples taken from population with any distribution for which the first moment exists.  .) and the expectation of each sample point is equal to the population mean. Knowledge of population distribution was not necessary for derivation of unbiasedness of the sample mean.

  A biased estimator ˆ can be an unbiased estimator when n is large. . then the estimator ˆ is n  asymptotically unbiased. If lim b( )  0 .

X2. Xn constitute a random sample from a distribution with mean μ1 and standard deviation σ1.….  Use the rules of expected value to show that X – Y is an unbiased estimator of μ1 . Solution:  E[ X  Y ]  E[ X ]  E[Y ]  1   2 . Yn (independent of Xi’s) constitute a random sample from a distribution with mean μ2 and standard deviation σ2. Y2.μ2.Suppose that X1.…. and that Y1.

n Thus. we see that E ( X )  . X is unbiased estimator of .2   2 ) . one finds that the maximum likelihood estimators of  and  2 are (n  1) S 2 ˆ ˆ ˆ ˆ 1    X and  2    n 2  2 ).  2 S is  2 2 n 1 1 n where S  ( X i  X )2  n  1 i 1 2   2 (n  1) S 2   2 E (S )  E   (n  1)   2 n 1  2  n 1   2 . We have shown that when sampling from N (1   . Recalling that the distribution of X is N (  . we know that the distribution of (n  1) Hence.

since ˆ )  E[ (n  1) S 2 ]  (n  1) E[ S 2 ]  (n  1)  2 E (2 n n n  Therefore. ˆ is a biased estimator of    2 2 2 . S2 is an unbiased estimator of 2 Consequently.  Therefore.

As we noted if estimator for  is tn then difference between them is error of the estimation. Expectation value of this error is bias.s.): M  E (tn   ) 2 . Expectation value of the square of the differences between estimator and the expectation of the estimator is called its variance: V  E (tn  E (tn ))  2 Exercise: What is the variance of the sample mean. Expectation value of square of this error is called mean square error (m.e.

S. It is not always possible. If the bias is 0 then MSE is equal to the variance. It can be expressed by the bias and the variance of the estimator: V (tn )  B2 (tn ) M  (tn )  E (tn   )2  E (tn  E (tn )  E (tn )   )2  E (tn  E (tn ))2  ( E (tn )   )2    M. In ideal world we would like to have minimum variance unbiased estimator.E is equal to square of the estimator’s bias plus variance of the estimator. In estimation it is usually trade of between unbiasedness and minimum variance. .

.

UNBIASED!!!! .

X n from N (  .  2 ) and s 2   i  .Let S2 be a sample variance for a random 2sample (x  x ) X1 . Solution: s2  ( xi  x )2 2 2  (n  1)  2 ~  n 1 . E ( s 2 )  E[ 2    (x  x ) i 2 n 2 ~  n 1 2 n E[ n  ( xi  x )2 ] ]  2 2 n 2 (n  1)   2 n since E(S2)≠2.. . therefore S2 is biased estimator and the biased value is . Is this n estimator biased or unbiased estimator? Note: mean of chi-square is it df.... X 2 .2/n.

n  . If an estimate is such that it’s biased converged to zero as n. then we say that it is asymptotically unbiased. S2 is asymptotically unbiased when n E (s 2 )   2   2 n lim E ( s 2 )   2 .

does this estimator is unbiased ( xi  x ) 2  (n  1) 2 E ( w)  E[ 2   ] ]  2 n 1 E[ ( xi  x ) 2   2 2 n 1 (n  1) 2 2 w is unbiased estimator for . n 1 estimator of  2 ? Solution: If w  ( xi  x )2  .

x2 . x2 .. assume we can differentiate inside the integral whenever needed... xn from function f ( x..  )  2    nE            1     2 ln f ( x. xn ) be an unbiased estimator for  based on a random sample of size n. then CRLB for the Var(y) is given by   2 y 1 Cramer Rao Lower bound    ln f ( x.. ) .  )  2    n E     2       ..  THEOREM Let y  U ( x1 ... x1 .

say E(y)=+b(). then CRLB for y is:   2 y 1  b '( ) 2    2 ln f ( x.  )  2    n E    2        .  Corrollary Suppose y is nor unbiased.

 )    0  . elsewhere . y ~ gamma(  1.    )  Get the CRLB and find the variance for this estimator .x  1   e f ( x.0  x   .

 )   ln    ln f ( x. )  1 2 E ( x) E  E[  2 ]    2  3    1   1 n 2    2 y  V ( X )   2  1.   ln f ( x. n E( X )   1 2  2 2  1 2 CRLB:   2 y 2 n . 2   2 M1  ˆ x  x  x. ) 1 x   2     2 ln f ( x. Solution: x E ( X )    1. ) 1 2x  2  3  2     2 ln f ( x.

ˆ)  V ( x )  V (  xi ) V ( n 1  2 V ( xi ) n 1  2  V ( x) n 1 1 2 2 2  2    2 n  n n n   ˆ Therefore. ˆ we say that  is efficient for estimating  and ˆ  is called uniformly minimum.variance (or best) unbiased estimator . variance for  equal to CRLB. ˆ If an unbiased estimator  achive the CRLB.

the one with the smallest variance is called the most efficient estimator of . If we consider all possible unbiased estimator of some parameter  . ˆ  Unbiased estimators 1 ˆ 3 ˆ 1 is the best estimator of  ˆ 2  Sampling distribution of different estimators of  .

ˆ The relative efficiency for  is defined to be:  ˆ)  CRLB  1 eff ( ˆ V ( ) . ˆ Let  be an estimator that is unbiased for .

 2 )  ˆ MSE (1 ) CAREFUL HERE ˆ ˆ ˆ ˆ where MSE ( )  E[(   )2 ]  Var ( )  b( ) 2 Mean square error . If ˆ1 and ˆ2 are biased estimators of parameter θ of a given population. the measure of efficiency of ˆ1 relative to ˆ2 is denoted by the ratio ˆ MSE ( 2 ) ˆ ˆ eff (1 .

MSE (ˆ2 ) eff (ˆ1 .ˆ2 )  1 ˆ ˆ MSE (1 )  MSE (2 ) ˆ ˆ MSE (1 )  MSE (2 ) ˆ ˆ MSE (1 )  MSE (2 ) ˆ1 less efficient than ˆ2 .ˆ2 )  1 eff (ˆ1 .ˆ2 )  1 eff (ˆ1 .ˆ2 )  MSE (ˆ1 ) What it means Conclusion ˆ1 as efficient as ˆ2 ˆ1 more efficient ˆ2 than eff (ˆ1 .

and Var (1 )  Var (2 ) . 2 )  ˆ Var (1 ) CAREFUL HERE . then we say that ˆ1 is relatively more efficient than ˆ2 .  The measure of efficiency of ˆ1 relative to ˆ2 denoted by the ratio is ˆ Var ( 2 ) ˆ ˆ eff (1 . If ˆ1 and ˆ2 are unbiased estimators of parameter θ ˆ ˆ of a given population.

ˆ2 )  1 ˆ ˆ Var (1 )  Var (2 ) ˆ ˆ Var (1 )  Var (2 ) ˆ ˆ Var (1 )  Var (2 ) ˆ1 less efficient than ˆ2 .ˆ2 )  Var (ˆ1 ) What it means Conclusion ˆ1 as efficient as ˆ2 ˆ1 more efficient ˆ2 than eff (ˆ1 .Var (ˆ2 ) eff (ˆ1 .ˆ2 )  1 eff (ˆ1 .ˆ2 )  1 eff (ˆ1 .

.

X1  2 X 2 1  3 X1  X 2 2  2   . Determine the measure of efficiency of  1 relative to  2 . Two estimators for mean μ are as below: Check whether the two estimators  1 and  2 are unbiased. Suppose that X1 and X2 are random samples from a normal distribution with mean μ and standard deviation σ.

 Check whether the two estimators  1 and  2 are unbiased.then the two . .  X  2X2  E (1 )  E  1  3   1 2  E( X1 )  E ( X 2 ) 3 3 1 2    3 3   X  X2  E ( 2 )  E  1  2   1 1  E( X1 )  E ( X 2 ) 2 2 1 1    2 2   Since both E ( 1 )   and E (  2 )   estimators  1 and  2 are unbiased.

.Determine the measure of efficiency of  1relative to  2  X  X2   X1  2 X 2  Var (  2 )  Var  1  Var (  1 )  Var   2   3   1 1 1 4  Var ( X 1 )  Var ( X 2 )  Var ( X 1 )  Var ( X 2 ) 4 4 9 9 1 2 1 2 1 2 4 2         4 4 9 9 1 2 5 2     9 2   Since both E ( 1 )   and E (  2 )   estimators  1 and  2 are unbiased.then the two .

9  1 10 ˆ ˆ ˆ ˆ  Since eff (1 . . the measure of efficiency of  1 relative to  2   eff (  . So.then the estimator  2 is 0.9 times more efficient than  1 .2 )  1 that is Var (1 )  Var (2 ).  )  Var    Var  2 1 1 2   2 5 2 2 9 9  0.

 Suppose that X1 . Determine the efficiency measure of X 1  2 X 2  X 3 relative to X1  X 2  X 3 3 4 . X2 and X3 are random samples from a normal distribution with mean μ and standard deviation σ.

.

 ˆ An estimator  is said to be consistent for P ˆ   .  if   .

the variance of X grows smaller. X is a consistent estimator of because: V(X) is That is. E. . as n grows larger.   An unbiased estimator is said to be consistent if the difference between the estimator and the parameter grows smaller as the sample size grows larger.g.

.

x2 . ) g ( y1 . xn given Y1  y1 is free of   . ) joint pdf for x1 . xn ) the func H( x1 .. x2 . x2 .. xn ) Where for every fix value of y1=U( x1 ..... f ( xn . thus the conditional pdf of x1 ......  if and only if f  x1 .  f ( x2 . x2 ...  )  H ( x1 . ).... x2 . xn ) does not depend on ..).. . x2 ... xn ) be a statistic with pdf g(y1.. We say that y1 is a sufficient statistic for the family  F   f ( xi . Let y1=U( x1 . ). xn  g ( y1 ... x2 ...Let x1 .).. xn be a random sample from function f(x.....

.1. xn is a r.... elsewhere is y1 is a sufficient for  ? . x2 . x  0...s from  x e   f ( x. .. )   x !  0  y1   xi . Let say x1 .

 i. Is a sufficient statistic. find joint pdf for x1 . ) f ( x2 . x2 ... xn f ( x1 . )   f ( xi .. )   i 1 i 1 n n  x e i x!  ii..  i 1 n  e xi xi   xi e  n   n  e  n ( xi )! xi n  e  n ( xi )!  x! x ! i  ( xi )! n xi ! Free of  Conc: the given func. ). f ( xn .. ..

 Theorem: ◦ Let x1 ... x2 .. Is just a statistic ... xn ) be a statistic of g(y1.. x2 . x2 .......) that y1 is a sufficient statistic for family  F   f ( xi ....  if and only if there is exist a n function k1(. x2 .) if k2(. x2 . xn ) i 1 where for each fix value of y1=U( x1 . . xn ) the function k2( x1 .. xn ))k2 ( x1. ). Note: ◦ There is another way to find sufficient statistic by Neyman’s Factorization Theorem.. x2 .. Let y1=U( x1 .)   f ( x....). xn be a random sample from function f(x... )  k1 (U1 ( x1. xn ) does not depend on  in form or in domain..

s from  (2 ) [ x(1  x)] 1 ... )  [( )]2  0 .. 0  x  1  f ( x.. xn is a r. Let x1 . x2 . elsewhere  Find sufficient statistic for   Solution: .