STAT 210B HWK #5 SOLUTIONS

GARVESH RASKUTTI

(1) Let X1, ..., Xn be i.i.d according to the Pareto distribution with density
fθ(x) = θcθx−(θ+1),
where 0 < θ and 0 < c < x. Determine the Wald, Rao and likelihood ratio
tests of H0 : θ = θ0 against θ = θ0.
6
The log-likelihood is
Y
l(θ; X1, ..., Xn) = log θcθx−(θ+1)
i
X X
= n log(θ) + θ(n log(c) − log(Xi)) − log(Xi)
i i
The MLE for θ is
ˆθ = P n .
i log(Xi) − n log(c)
The likelihood ratio
test:
Λ = 2l(θˆ; X1, ..., Xn) − 2l(θ0; X1, ..., Xn)
X
ˆ
= 2n log(θ ) − 2θ ˆ log(Xi ) − n log(c)
X
− 2n log(θ0) + 2θ0 log(Xi) − n log(c)

= 2n log θ0
−1− !
θˆ
θ
θ0 ˆ

In order to calculate the Wald and Rao statistics, we first need to calculate the
Fisher information:
X
l(θ, X) = log(θ) − θ log − log(X)
c
∂l 1 X
⇒ (θ, X) = − log
∂θ θ c
∂2l 1
⇒ 2 (θ, X) = − 2
∂θ 1 θ
⇒ I(θ) = 2
θ
Wald test
statistic:
W = n(θˆ − θ0 )2 In (θ0 )
θ !2
ˆ
=n 1−
θ 0

Date: May 2, 2011.

1
2 GARVESH RASKUTTI
Rao test statistic: let ∆n,θ = √1 ∂l
(θ, Xi)
P
n i ∂θ
2
n,θ0 I (θ0)
−1
R=
∆
1n X !2
= − X
i
log θ02
n θ i c
2
1 1 2
=n θ
θ0 − θˆ 0
=n 1− θ 2
0

θˆ

(2) Calculate the asymptotic relative efficiency of the empirical p quantile and
the estimator Φ−1 (p)Sn + X¯ n for estimating the p th quantile of the dis-
tribution of a sample from the univariate normal N (µ, σ 2) distribution
− (where
2 ¯
S = 1/(n − 1) (Xi − X n ) ).2
−
P
(a) Empirical quantile: The asymptotic variance of the empirical p quantile can be
n i
−
obtained using functional delta method as in Example 20.5 VDV
p(1 − p)
σ2 =f (F −1(p))2
1

p(1 p)σ2
= 2 −−1 ,
φ (Φ (p))
where F is the CDF of N (µ, σ2) and f is the corresponding density function, while Φ
and φ are the CDF and PDF of standard normal, respectively. So we have
√ p(1 p)σ2
−1
−1
n(Fn (p) − (p)) → d
0, 2 −1
φ (Φ −(p))
F N
(b) Φ−1 (p)Sn + X¯ n . First use delta method on S 2 as in Example 3.2 VDV, we have
n
√ 4
d N (0, 2σ ),
n(S 2 −
n σ ) →
2

use delta method again we have

√ 1
d N (0 , 2
n(S n − σ) ).
→
d
2
Then note that S and
are independent for Gaussian data, and that −µ) →
X¯n n
√
N (0, σ2), we obtain n(X¯ n
√ √
n(Φ (p)Sn + X n F (p)) = n Φ−1 (p)Sn + X¯ n (Φ−1 (p)σ + µ)
−1 ¯ −1
√ √
−
= Φ (p) n(Sn − σ) + n(X¯ n − µ)
−1

!
−
N 0, 1 + 2
(Φ−1(p))2 σ2!
 2
0@ (Φ−1(p)) A
1
1+
2

= a result, the ARE between the two estimators is: ARE

As p(1−p) .
φ2(Φ−1(p))
 STAT 210B HWK #5 SOLUTIONS 3

(3) Consider the corrupted mixture of Gaussian F = (1 − )N (µ, 1) + N (µ, σ 2)

where and σ are fixed with 0 ≤ 1 and σ > 0.
(a) Find the asymptotic relative efficiency (ARE) of the median relative to
the mean as estimators for µ in terms of and σ.
(b) Prove that the ARE is an increasing function of σ in the range σ ∈ (1, ∞).
(a) Note that the mean of2the distribution F = (1 − )N (µ, 1) + N (µ, σ2) is µ and
the variance is (1 − ) + σ by the law of total variation. Therefore
√ 2
n(X¯ − µ) →d N (0, 1 − + σ ).
For any distribution, note that the asymptotic distribution of the median X2b( n )c
satisfies:
√
n(X n − µ) d N (0, 1 ),
→
b( 2 )c
4f (F −1(1/2))
where f is the pdf and F is the cdf. For the mixture F = (1 )N (µ, 1) + N (µ,
σ2), f (F −1(1/2)) = (1 − )(2π)−1/2 +
(2πσ2)−1/2. −
Therefore the ARE of the median compared to the mean is:
2
ARE = (1 − + σ2)(1 − + /σ)2.
π
(b) To show that the ARE is an increasing function when σ > 1, simply show that
the derivative of log ARE(σ) with respect to σ is an increasing function.
d log ARE 2σ 2
= − .
dσ 1 − + σ2 σ2 − σ2 + σ
Therefore
d log ARE 1 1
d = 2 (1 − )(σ − σ−2) .
1− +σ σ − σ +σ
2 2 2
σ
Clearly the derivative of log ARE(σ) is positive in the range σ > 1 which shows that
the ARE is an increasing function when σ > 1.

(4) Consider the gaussian sequence model Yi = µi + wi, for i = 1, 2, ..., n, and
n
(wi)i= are i.i.d with wi ∼ N (0,n1 ) for each i. Further assume that (µi)∞ i= is an
1
infinite sequence that satisfies the following ellipsoid constraint: 1

∞
X 2 2α
µi i ≤
i=1
1,
where α > 1
. Consider two estimators for the sequence µn = (µi)n : (i)
2 i=1
n
The maximum likelihood estimator µˆ where µˆi = Yi ; and (ii) the
truncated n
estimator µ˜ where µ˜i = Yi 1(i ≤ n1/(2α+1) ).
Pn 2
(a) Prove that MSE(µˆn ) := i= E[(µˆ i − iµ )]2 = 1 and MSE(µ˜ n ) ≤ 2nα−
2α+1
for all
n 1 n
n. Note that MSE(µ˜ )/MSE(µˆ → ) 0 as n → .
(b) Explain in words why your result from (a) does not contradict Theorem
∞ ` = `2 where Tn = µ˜n .
8.11 (page 118) in VDV for the case
(a) First, we compute MSE(µˆn ):
n n
1
MSE(µˆn ) = X E(wi 2 ) = X =1
i= i=
n
1 1
4 GARVESH RASKUTTI

To compute MSE(µ˜n ), we first de-compose the sum as follows:

X X
MSE(µ˜n ) = E(w
2 )+
i µ2i .
i≤bn1/(2α+1) c
i>bn1/
(2α+1) c

First we bound the first sum:

1
E(wi2) = c n−2α/(2α+1).
i≤bnX1/ ≤n
1/(2α+1)
bn
(2α+1) c

Next we bound the second sum:

X 2 X
µi = µ2i2αi−2α
i≥dn1/(2α+1) e i≥dn1/ i
(2α+1) e

X
≤n
−2α/(2α+1)
µi2i2α
i≥dn1/(2α+1) e

−2α/(2α+1)
≤n .
Hence the proof for part (a) is complete.

(b) There are two reasons why the result from part (a) does not violate Theorem
8.11 in VDV. The first reason is that Theorem 8.11 assumes the parameter space is
fixed while for the Guassian sequence model defined here, the parameter space
grows as n grows. Hence Theorem 8.11 does not apply.
The second reason is that Theorem 8.11 in VDV involves taking a supremum
over h belonging to some interval I. Part (a) proves that in the specific case h = 0,
the MSE is asymptotically smaller for the shrinkage estimator µ˜n . Note that for
any finite h the ellipsoid constraint would be violated so the bounds on MSE of
µ˜n do not hold over the shrinking neighborhood. This again highlights the
importance of the shrinking neighborhoods in Le Cam’s theory.