Stat 513

Homework Key 6
February 2016

REGULAR PROBLEMS

1. Suppose Y1 , . . . , Yn are i.i.d. Bernoulli random variables with Yi ∼ B(1, p), p ∈ (0, 1). Denote the target
of inference by θ = V ar(Yi ).

(a) Derive a uniform minimum variance unbiased estimator(UMVUE) θ̃ of θ.

(b) Find the variance of θ̃. Does it meet the Cramer-Rao lower bound for an unbiased estimator of θ?
(c) Derive an exact probability distribution for θ̃.
(d) Derive an asymptotic distribution for θ̃.

Ans:

(a) We know that Ȳ is complete sufficient. An unbiased estimator of θ is the unbiased sample variance,
n
n 1X 2 n
S2 = (Y − Ȳ )2 = Ȳ (1 − Ȳ ).
n − 1 n i=1 i n−1

As S 2 is a function of the complete sufficient statistic Ȳ only, we have that θ̃ = S 2 is the UMVUE
of θ.
(b) The variance of θ̃ can be based on the result of STAT 512 Homework 6, Problem 5. We found that
1 2
V ar(S 2 ) = (γ − 3σ 4 ) + σ4
n n−1
1 2
= (θ(1 − 3θ) − 3θ2 ) + θ2
n n−1

where γ = p(1 − p)(1 − 3p(1 − p)) = θ(1 − 3θ) is the fourth central moment of the Bernoulli(p)
distribution. We know that the CRLB is attainable for g(p) = p, so a BLUE only exists for linear
functions of p. Since θ = p(1 − p), our UMVUE θ̃ does not attain the CRLB.
Pn
(c) The exact distribution of θ̃ is based on the sum of the Yi . Since T = i=1 Yi ∼ B(n, p), we can write
its pmf as
   
n 1 1
P r(θ̃ = s) = P r T 1− T =s
n−1n n
= P r(nT − T 2 = (n − 1)s)
= P r(T 2 − nT + (n − 1)s = 0)
( p )!
n ± n2 − 4(n − 1)s
= Pr T ∈
2
p !
n + n2 − 4(n − 1)s
= 2 × Pr T =
2
n o
t(n−t)
for s ∈ n(n−1) : t = 0, 1, . . . , n/2 .
(d) By the CLT, we have that √
n(Ȳ − p) →d N (0, θ).
By the Delta Method with g(x) = x − x2 , we obtain that
√
n(θ̃ − θ) →d N (0, V (θ)))

with V (θ) = (1 − 2p)2 θ = (1 − θ2 )θ.

1
2. Suppose Y1 , . . . , Yn are i.i.d. normal random variables with Yi ∼ N (µ, σ 2 ) with µ ∈ (−∞, ∞) and
σ 2 ∈ (0, ∞). Denote the target of inference by θ = σ 2 .
(a) Derive a uniform minimum variance unbiased estimator(UMVUE) θ̃ of θ.
(b) Find the variance of θ̃. Does it meet the Cramer-Rao lower bound for an unbiased estimator of θ?
(c) Derive an exact probability distribution for θ̃.
(d) Derive an asymptotic distribution for θ̃.
Ans:

Pn
(a) The complete sufficient statistic in this model is (Ȳ , i=1 Yi2 ). An unbiased estimator of θ is again
the unbiased sample variance,
n n
!
2 n 1X 2 n 1X 2
S = (Yi − Ȳ )2 = Y − Ȳ 2 .
n − 1 n i=1 n−1 n i=1 i

As S 2 is a function of the complete sufficient statistic Ȳ only, we have that θ̃ = S 2 is the UMVUE
of θ.
(b) The variance of θ̃ follows because
σ2 2
θ̃ ∼ χ ,
n − 1 n−1
so that
σ4 2σ 4
V ar(θ̃) = 2(n − 1) = .
(n − 1)2 n−1
The score for θ = σ 2 is just
1
U(σ 2 ) = ((Y − µ)2 − σ 2 ),
2σ 4
so we see that a BLUE can exist for θ in this case. The information for an observation is
1
J(θ) = E[U 2 (θ)] = .
2σ 4
Hence the CRLB is
1 4
J−1 (θ) = 2σ < V ar(θ̃).
n
(c) The distribution of θ̃ is a scaled χ21 distribution,

σ2 2
θ̃ ∼ χ ,
n−1 1
as seen from the course notes as well as Casella-Berger Theorem 5.3.1.
(d) An asymptotic distribution for θ̃ follows from similar arguments from the Homework 1 key, where
we derived an asymptotic joint distribution for the sample mean and variance. Applying that result,
and letting γ4 be the kurtosis of the N (µ, σ 2 ),
√
n(θ̃ − θ) →d N (0, V (θ)).

Recall from STAT 512 Homework 4, Question 2 that γ4 = 3σ 4 so that

V (θ) = γ4 − σ 4 = 3σ 4 − σ 4 = 2θ2 .

3. Suppose Y1 , . . . , Yn are i.i.d. normal random variables with Yi ∼ N (µ, σ 2 ) with µ ∈ (−∞, ∞) and
σ 2 ∈ (0, ∞). Denote the target of inference by θ = µ2 .
(a) Derive a uniform minimum variance unbiased estimator(UMVUE) θ̃ of θ.
(b) Find the variance of θ̃. Does it meet the Cramer-Rao lower bound for an unbiased estimator of θ?
(c) Derive an exact probability distribution for θ̃.
(d) Derive an asymptotic distribution for θ̃.
Ans:

2
(a) Recall that the unbiased sample variance we appealed to in problems 1 and 2 satisfies

E[S 2 ] = V ar(Y1 ) = E[Y12 ] − µ2 .

Hence, a natural choice for an unbiased estimator of θ = µ2 is

n
1X 2
θ̃ = Y − S2,
n i=1 i

since " #
n
1X 2
E Y − S = E[Y12 ] − (E[Y12 ] − µ2 ) = µ2 .
2
n i=1 i

However, as seen in the homework (as well as Casella-Berger Theorem 5.3.1), Ȳ is independent of S 2
for a random sample from a normal distribution. We can express θ̃ in a way that allows us to make
use of this independence:
n
1X 2
θ̃ = Y − S2
n i=1 i
n
1X 2
= Y − Ȳ 2 + Ȳ 2 − S 2
n i=1 i
n−1 2
= S + Ȳ 2 − S 2
n
1
= Ȳ 2 − S 2 .
n

(b) The variance of θ̃ is easily calculated since Ȳ is independent of S 2 ,

1
V ar(θ̃) = V ar(Ȳ 2 ) + V ar(S 2 ).
n2
Now, Ȳ ∼ N (µ, σ 2 /n), so Ȳ 2 ∼ σ 2 /nχ21 (δ) where δ = µ2 /2. Additionally, S 2 ∼ σ 2 /(n − 1)χ2n−1 .
Hence, we can simplify the variance to

σ4 1 σ4
V ar(θ̃) = V ar(Ȳ 2 ) + V ar(S 2 ) = 2(1 + µ2 ) + 2 .
n2 n2 (n − 1)
Since the score for µ is just
U(µ) = σ −2 (Yi − µ),
only linear functions of µ admit a BLUE. Hence, θ̃ will not obtain the CRLB.
(c) The exact distribution of θ̃ is a convolution of the two χ2 distributions detailed above,

σ2 2 2 σ2 2
θ̃ ∼ χ1 (µ /2) − χ .
n n − 1 n−1

(d) Since n−1 S 2 →p 0, we know that if

Ȳ 2 →d X,
then, by Slutsky’s theorem
√ √ 1
n(θ̃ − θ) = n(Ȳ 2 − θ) S 2 →d X.
n1/2
Additionally, we have from the CLT that
√
n(Ȳ − µ) →d N (0, σ 2 ).

We apply the Delta Method with g(x) = x2 , finding

√
n(Ȳ 2 − µ2 ) →d N (0, 4µ2 σ 2 ).

Hence, by our Slutsky’s theorem argument,

√
n(θ̃ − θ) →d N (0, 4θσ 2 ).

3
4. Suppose Y1 , . . . , Yn are i.i.d. exponential random variables with Yi ∼ E(µ) with mean µ ∈ (0, ∞). Denote
the target of inference by θ = V ar(Yi ) = µ2 .
(a) Derive a uniform minimum variance unbiased estimator(UMVUE) θ̃ of θ.
(b) Find the variance of θ̃. Does it meet the Cramer-Rao lower bound for an unbiased estimator of θ?
(c) Derive an exact probability distribution for θ̃.
(d) Derive an asymptotic distribution for θ̃.
Ans:

(a) We know that Ȳ is complete sufficient. Also,

n+1 2
E Ȳ 2 = µ .
n
nȲ 2
Hence by Lehman Scheffe theorem, the UMVUE is θ̃ = .
n+1
Pn
(b) S = i=1 Yi ∼ Gamma(n, µ).
∞
sk+n−1 e−s/µ
Z
ES k = ds
0 µn Γ(n)
Z ∞ k+n−1 −s/µ
k (n + k − 1)! s e
=µ n+k Γ(n + k)
ds
(n − 1)! 0 µ
(n + k − 1)!
= µk .
(n − 1)!

Using the above relation for k = 2, 4 we get,

V ar(S 2 ) = ES 4 − (ES 2 )2 = n(n + 1)(4n + 6)µ4 .

S2 (4n + 6)µ4
V ar(θ̃) = V ar( )= .
n(n + 1) n(n + 1)
Since Pn
i=1 yi
−
e µ
f (y|µ) = ,
µn
we have
−n
U(µ) = (Ȳ − µ). (1)
µ2
Therefore we have a BRUE for µ. However, θ = µ2 is not a linear function of µ. Therefore there is
no BRUE, i.e. CRLB can not be attained by the UMVUE θ̃.
(c) Let

s2
g(s) = .
n(n + 1)
S2
⇒ θ̃ = g(S) =
n(n + 1)
−1
p
g (y) = n(n + 1)y
2s
g 0 (s) =
n(n + 1)
fS (g −1 (y))
fθ̃ (y) = fg(S) (y) = 0 −1
g (g (y))
√
− n(n+1)y/µ
n(n + 1)e (n(n + 1)y)n/2−1
= 1[y > 0].
2(n − 1)!µn

4
 (d) By CLT,
√ D
n(Ȳ − µ) −→ N (0, µ2 ).
By Delta method,
√ D
n(Ȳ 2 − µ2 ) −→ N (0, 4µ4 ).
√ 1
n(Ȳ 2 − θ̃) = Ȳ 2 OP ( √ ) = OP (1)oP (1) = oP (1). Hence, by Slutskey’s theorem,
n
√ D
n(θ̃ − µ2 ) −→ N (0, 4µ4 ).

5. Suppose Y1 , . . . , Yn are i.i.d. exponential random variables with Yi ∼ U(0, µ) with mean µ ∈ (0, ∞).
Denote the target of inference by θ = V ar(Yi ).
(a) Derive a uniform minimum variance unbiased estimator(UMVUE) θ̃ of θ.
(b) Find the variance of θ̃. Does it meet the Cramer-Rao lower bound for an unbiased estimator of θ?
(c) Derive an exact probability distribution for θ̃.
(d) Derive an asymptotic distribution for θ̃.
Ans:

(a) Y(n) is complete sufficient with PDF nµ−n y n−1 I[0,µ] (y). Therefore
Z µ
k n
EY(n) = n xn+k−1 dx (2)
µ 0
nµk
= (3)
n+k

2 nµ2 µ2 n+2 2
Using the above we get that EY(n) = and we know θ = V ar(Yi ) = . Therefore θ̃ = Y
n+2 12 12n (n)
is the UMVUE.
(b) Using (2) we get that

(n + 2)2 µ4 µ4
 
1 n
V ar(θ̃) = − = .
144n n + 4 (n + 2)2 36n(n + 4)

n 4µ4 µ4
We know that the information for µ is 2
. Therefore the CRLB is . For large n, V ar(θ̃) ≈ <
µ n 36n2
4µ4
. This happens because uniform is not a regular family.
n
12n
(c) Let c = . Let us define
n+1
s2
g(s) = .
c
Y(n) 2
⇒ θ̃ = g(Y(n) ) =
c
√
⇒ g −1 (y) = cy
2s
⇒ g 0 (s) =
c
fY(n) (g −1 (y))
⇒ fθ̃ (y) = fg(Y(n) ) (y) =
g 0 (g −1 (y))
ncµ−n √ n−2 √
= ( nc) I[0,µ] ( cy)
2
6n2 µ−n √ n−2 √
= ( nc) I[0,µ] ( cy)
(n + 1)

12n
where c = .
n+1

5
 (d)

P (n(µ − Y(n) ) ≥ y)
  −nµ . −x
x x µ
= 1− → e−x/µ .
µn
Therefore
D
n(µ − Y(n) ) −→ E(µ).
µ2
Let g(µ) = . Hence by Generalised Delta method,
12
2 
Y(n)
 2
µ D µ
n − −→ g 0 (µ)E(µ) ≡ E(µ) ≡ E(µ2 /6).
12 12 6
Now
2 2
Y(n) Y(n) P
|n(θ̃ − )| = | |−
→ 0.
12 6n
Therefore by Slutskey’s theorem,
µ2
 
D
n − θ̃ −→ E(µ2 /6).
12
6. Suppose Y1 , . . . , Yn are i.i.d. exponential random variables with Yi ∼ E(λ) with hazard λ ∈ (0, ∞). Denote
the target of inference by θ = P (Yi > 1).
(a) Derive a uniform minimum variance unbiased estimator(UMVUE) θ̃ of θ.
(b) Find the variance of θ̃. Does it meet the Cramer-Rao lower bound for an unbiased estimator of θ?
(c) Derive an exact probability distribution for θ̃.
(d) Derive an asymptotic distribution for θ̃.
Ans:

Pn
(a) U = I[Y1 > 1] is an unbiased estimator of θ. Recall the complete sufficient statistic is S = i=1 yi .
Hence,

θ̃ = E(U |S = s)
= P (Y1 > 1|S = s)
= P (Y1 /S > 1/s|S = s)

We will use the following result now:

Let X ∼ Gamma(α, 1), Y ∼ Gamma(β, 1) where α, β are the shape parameters. If X and Y are
independent, then
X
∼ Beta(α, β).
X +Y
Following the above result we get that
Y1
∼ Beta(1, n − 1)
S
Y1
which is independent of λ, hence ancillary. Following Basu’s theorem, and S are independent.
S
Therefore,

θ̃ = P (Y1 /S > 1/s)

Z 1
Γ(n)
= 1[s > 1] (1 − x)n−2 dx
Γ(n − 1) 1/s
 n−1
1
= 1− 1[s > 1].
s
 n−1
1 Pn
Therefore the UMVUE is 1 − 1[S > 1] where S = i=1 Yi .
S

6
 e−λs sn−1 λn
(b) S ∼ Gamma(n, λ) with f (s) = .
Γ(n)
∞ 2n−2
e−λs sn−1 λn
Z 
2 1
E θ̃ = 1− ds
1 s Γ(n)
∞
(s − 1)2n−2 e−λs λn
Z
= ds.
1 sn−1 Γ(n)
∞
(s − 1)2n−2 e−λs λn
Z
V ar(θ̃) = ds − θ2 .
1 sn−1 Γ(n)
1 1
From (1) we see that a BRUE for = µ exists. Since θ = e−λ is not a linear function of , no
λ λ
BRUE exists. So the CRLB will not match with the variance of θ̃.
(c) S follows Gamma(n, λ). Let us define
 n−1
1
g(s) = 1− 1[s > 1].
s
 n−1
1
⇒ θ̃ = g(S) = 1− 1[s > 1].
s
1
⇒ g −1 (y) = 1[y > 0]
1− y 1/(n−1)
(n − 1)(s − 1)n−2
⇒ g 0 (s) = 1[s > 1].
sn
fS (g −1 (y))
⇒ fθ̃ (y) = fg(S) (y) = 0 −1
g (g (y))
λ n−2
λn − −
1 − y 1/(n−1)
= e y n − 1 (1 − y 1/(n−1) )−(n+1) 1[y > 0].
(n − 1)Γ(n)

(d) By CLT,
√
   
1 D 1
n Ȳ − −→ N 0, 2 .
λ λ
Therefore using delta method we get
√
 
D
n(e−1/Ȳ − e−λ ) −→ N 0, λ2 e−2λ .

√ 1 √
n(θ̃ − e−1/Ȳ ) = √ + oP (1/( nȲ )).
nȲ
1
Since √ is oP (1), the above term is oP (1). Therefore by Slutskey’s theorem,
nȲ

√
 
−λ D 2 −2λ
n(θ̃ − e ) −→ N 0, λ e .

7
 MORE INVOLVED PROBLEMS

7. Suppose Y1 , . . . Yn are i.i.d. Bernoulli random variables with Yi ∼ B(p), p ∈ (0, ∞). Denote the target of
inference by p. Suppose further that the data are sample sequentially with the following (fairly stupid)
group sequential stopping rule with potential analyses after n1 = 1 and n2 = 2 observations have been
observed:
• Y1 is sampled.
• If Y1 = 1, we stop sampling at n1 = 1 and define (M, s) = (1, Y1 ).
P11
• If Y1 = 0, we sample Y2 , . . . , Y1 1 and then stop sampling at n2 = 11. We define (M, s) = (2, i=2 Yi ).
Note that under this scheme, M is our (random) stopping time, and S. In group sequential terminology,
the “continuation set” at the first analysis is C1 = {0}.
(a) Derive the exact distribution for statistic (M, S).
(b) Show that the exact distribution is a curved exponential family distribution.
(c) Find a minimal sufficient statistic for p.
(d) Show that minimal sufficient statistic for p is a complete sufficient statistic.
(e) Find a uniform minimum variance unbiased estimator (UMVUE) p̃ of p.
(f) Find the maximum likelihood estimator (MLE) p̂ of p, and show that it is a biased estimator.
(g) Compare the MSE of the MLE p̂to the MSE of the UMVUE p̃.
(h) Derive a “bias adjusted estimator” p̌ such that
 
S
E |p = p̌ = p̂
nM

Ans:
P11
(a) We observe that when M = 1, S = 1 and when M = 2, S = i=2 Yi ∼ Bin(10, p).
   m−1
10 s
f (s, m) = p2−m (1 − p) p (1 − p)10−s .
s

(b)
 m−1
10
f (s, m) = e(sm−s+2−m) log p+(m−1)(11−s) log(1−p) (4)
s
Therefore we see that (M, S) has an exponential family distribution. Now we have only one parameter
p but clearly SM − S + 2 − M and (M − 1)(11 − S) are linearly independent. Therefore it is a curved
exponential family. Let us denote T1 = SM − S + 2 − M and T2 = (M − 1)(11 − s).
(c) Clearly T1 and T2 are minimal sufficient. Hence (M, S) is also minimal sufficient since T1 and T2 are
1 − 1 functions of M and S.
(d) We have to show that (M, S) is complete. Now M ∈ {1, 2} and S ∈ {1, . . . , 10}. Suppose, if possible
∃ a no-zero function h : {1, 2} × {1, . . . , 10} → R such that for all p ∈ (0, 1)

Eh(M, S) = 0
E[E[h(M, S)|M ]] = 0 pE[h(1, S)|M = 1] + (1 − p)E[h(2, S)|M = 2] = 0
ph(1, 1) + (1 − p)Eh(2, X) = 0

where X ∼ Bin(10, p). Letting p → 1 we get that

h(1, 1) = 0.

Therefore
(1 − p)Eh(2, X) = 0
for all p. Since X is complete for the family X ∼ Bin(10, p) the above holds only if h(2, s) = 0 for
s ∈ {1, . . . , 10}. Hence h(m, s) = 0 for all (m, s) such that f (s, m) > 0. Therefore (M, S) is complete.
(e) 2 − M ∼ Bin(1, p). Therefore E(2 − M ) = p. Since it is a function of the complete sufficient statistic
it is also the UMVUE.

8
(f) From (4) we get that

l(p) = (sm − s + 2 − m) log p + (11m − 11 − sm + s) log(1 − p) + c

where c is a constant term independent of p. Hence,

sm − s + 2 − m 11m − 11 − sm + s
U(p) = − .
p 1−p
Solving for U(p) = 0, we get that
SM − S + 2 − M
p̂ = .
10M − 9
We also observe that it is equivalent as writing
S
p̂ = (5)
nM

10p
E p̂ = EE[p̂|M ] = p.1 + (1 − p) .
11
P11
since when M = 1, S = 1 and when M = 2, S = i=2 Yi ∼ Bin(10, p). Hence

21p − 10p2
E p̂ = 6= p. (6)
11
(g) M SE(p̂) = V ar(p̂) + (E p̂ − p)2 ,
10p(1 − p) 2
(E p̂ − p)2 = ( ) .
11

V ar(p̃) = E p̂2 − (E p̂)2

E p̂2 = E[E p̂2 |M ]
S 10p(1 − p) + 100p2
= E[E( )|M ]] = p.1 + (1 − p)
nM 121

Therefore,
p(−100p3 + 330p2 − 361p + 131)
V ar(p̂) =
121
130p3 − 261p2 + 131p
M SE(p̂) =
121
M SE(p̃) = V ar(p̃) = p(1 − p)
10p(1 − p)(1 − 13p)
M SE(p̂) − V ar(p̃) =
121
1 1 1
When p < , M SE(p̂) > M SE(p̃). When p > , M SE(p̂) < M SE(p̃). When p = , M SE(p̂) =
13 13 13
M SE(p̃).
S
(h) From (5) we get that is the MLE. Hence
nM
 
S
E |p = p̌ = E[p̂|p = p̌]
nM
Therefore following (6) we get that
21p̌ − 10p̌2
E[p̂|p = p̌] = .
11
Let
21p̌ − 10p̌2
p̂ =
11
⇒ 10p̌2 − 21p̌ + 11p̂ = 0

9
therefore, √
21 ± 441 − 440p̂
p̌ = .
20
Clearly √
21 + 441 − 440p̂
> 1.
20
and √ √ √
21 − 441 − 440.0 21 − 441 − 440p̂ 21 − 441 − 440.1
0= < < = 1.
20 20 20
Therefore the only feasible solution is
√
21 − 441 − 440p̂
p̌ = .
20

10