LSTAT2050 - Analyse statistique II - Practice

session 2

Exercise 1 - UMVUE
iid
Let X1 , . . . , Xn ∼ pθ (x) so that pθ (x) = θ exp[−θx]1(0,∞) (x). Find the UMVUE for g(θ) =
exp(−kθ).

Solution
We are going to use Lehmann-Scheffé theorem (theorem 3.2). Hence, we need to follow three
steps. First, we need to find an unbiased estimator T (X) for g(θ). Second, we’ll find a sufficient
statistic S(X) for g(θ). Finally, well apply Lehmann-Scheffé and compute E[T (X)|S(X)].

Finding an unbiased estimator - T

It’s easy to see that g(θ) = exp[−kθ] = P (X1 > k). Given that, we propose as estimator
T (X) = 1[X1 > k]. Let us check that it is indeed unbiased.
E[T (X)] = E[1[X1 > k]] = P (X1 > k) = g(θ)

Finding a sufficient and complete statistic - S

Pn
It’s trivial to see that pθ ∈ EF (1). From this, we determine that S(X) = i=1 Xi is sufficient
and complete.

Compute E[T|S]
Note that S(X) ∼ Ga(n, θ) and denote its density by pn . Thus, we have (X1 , S(X)) ∼ (pθ , pn−1 ).
With this in mind, we are able to compute E[T (X)|S(X)].
" #
X
E[T (X)|S(X) = s] = E 1[X1 > k]| Xi = s
i
!
X
=P X1 > k| Xi = s
i
s
pθ (x)pn−1 (s − x)
Z
= dx
k pn (s)
Z s
θ exp[−θx]θn−1 [s − x]n−2 exp[−θ(s − x)]Γ(n)
= dx
k Γ(n − 1)θn sn−1 exp[−θs]
Z s
(s − x)n−2
= n−1 dx
k sn−1
" n−1 #k  n−1
s−x s−k
= =
s s
s

LSTAT2050 - Analyse statistique II - Practice session 2 1

Hence, by Lehmann-Scheffé theorem, the UMVUE is
P n−1
∗ i Xi − k
T (X) = E[T (X)|S(X)] = P
i Xi

Exercise 2 - UMVUE
iid iid
Let X1 , . . . , Xm ∼ U [0, θ] and Y1 , . . . , Yn ∼ U [0, θ0 ] be two samples independent from one
another. Assuming n > 1, determine the UMVUE of θ/θ0

Solution
We are going to use Lehmann-Scheffé theoreom (theorem 3.2). Similarly to the previous exercise,
we develop our reasoning in three steps.

Finding an unbiased estimator - T

(m) X
A first good guess would be to use U (X, Y ) = Y(n) . We then need to derive its bias and create
an unbiased statistic upon it. The density of X(m) (and similarly for Y(n) ) writes as follows

pX(m) (x) = mpX (x)[PX (x)]m−1

m  x m−1
=
θ θ
m m−1
= mx
θ
where px and Px are the density and CDF of X1 . We can now compute the expected value of
U (X, Y ).
Z θ0 Z θ
x mn m−1 n−1
E[U (X, Y )] = m 0n
x y dxdy
0 0 yθ θ
Z θ0 Z θ
mn
= m 0n xm y n−1 dxdy
θ θ 0 0
m θm+1 n θ0n−1
=
m + 1 θm n − 1 θ0n
m n θ
=
m + 1 n − 1 θ0
From this we can derive an unbiased statistic, which we call T (X, Y ). More precisely
m+1n−1
T (X, Y ) = U (X, Y )
m n

Finding a sufficient and complete statistic - S

We consider S(X, Y ) = (X(m) , Y(n) ). Let us first consider sufficiency. It is easy to derive that
S1 (X) = X(m) is sufficient for θ and S2 (Y ) = Y(m) is sufficient for θ0 . Since the samples are
independent, S(X, Y ) is sufficient for (θ, θ0 ).

LSTAT2050 - Analyse statistique II - Practice session 2 2

 Let us now turn to completeness. Consider an integrable function g(., .). The expected value
of g(X(m) , Y(n) ) is given by
Z θ0 Z θ
mn m−1 n−1
E[g(X(m) , Y(n) )] = g(x, y) x y dxdy
0 0 θm θ0n
mn
= m 0n G(θ, θ0 )
θ θ
Besides, the cross derivative of G(θ, θ0 ) is given by

∂ 2 G(θ, θ0 )
0
= g(θ, θ0 )θm−1 θ0n−1
∂θ∂θ
As a consequence,

E[g(X(m) , Y(n) )] = 0 ∀θ, θ0 ⇒ G(θ, θ0 ) = 0 a.s. ∀θ, θ0

⇒ g(θ, θ0 ) = 0 a.s. ∀θ, θ0

In conclusion, S(X, Y ) is indeed complete.

Compute E[T|S]
X
Let us recall that T (X) = m+1
m
n−1 (m)
n Y(n)
is unbiased for θθ0 and S(X) = (X(m) , Y(n) ) is complete for
(θ, θ0 ). Using Lehmann-Scheffé theorem, we have that T (X) = E[T (X)|S(X)] is the UMVUE.

Exercise 3 - Using Lehmann-Scheffé theorem to compute

conditional expectations
The Lehmann-Scheffé theorem (theorem 3.2) can be used to compute conditional expectations.
iid
This exercise illustrates this point. Let X1 , . . . , Xn ∼ U (0, θ). Besides, let T (X) = X12 + X2 and
S(X) = X(n) . Use Lehmann-Scheffé theorem to compute E[T (X)|S(X)].

Solution
The general procedure works as follows. Assume T (X) is unbiased for some g(θ) and S(X) is
complete and sufficient for θ, we follow the subsequent steps.
1. Compute g(θ) = E[T (X)].
2. Find h(S(X)) such that E[h(S(X))] = g(θ).
By Lehmann-Scheffé theorem, the only function of S(X) which is unbiased is the UMVUE.
Besides, the UMVUE is obtained as E[T (X)|S(X)]. In conclusion, E[T (X)|S(X)] = h(S(X))

θ θ2
Let us first compute E[T (X)], recalling that E[X] = 2
and V (X) = 12
. Therefore,

θ2 θ2 θ
E[X12 + X2 ] = + +
12 4 2
θ2 θ
= + = g(θ)
3 2

LSTAT2050 - Analyse statistique II - Practice session 2 3

 We then need to find h(S(X)) such that E[h(S(X))] = g(θ). We can compute that E[X(n) ] =
n 2 n
n+1
θ
and E[X(n) ] = n+2 θ2 . Hence, we can take
1n+1 1n+2 2
h(S(X)) = X(n) + X(n)
2 n 3 n
Using Lehmann-Scheffé theorem, we conclude that
1n+1 1n+2 2
E[T (X)|S(X)] = X(n) + X(n)
2 n 3 n

Exercise 4 - simple test - problem 0

Let Θ = {0, 1} and X1 , . . . , Xn be i.i.d copies of a random variable X with the following
probability mass function
X 1 2 3 4 5 6 7 8
f0 0.02 0.02 0.02 0.02 0.02 0.02 0.02 0.86
f1 0.14 0.12 0.10 0.08 0.06 0.04 0.02 0.44

Using the Neyman-Pearson lemma, find a UMP test with level α = 0.04 and α = 0.06 for
H0 : θ = 0 vs. H1 : θ = 1.

Solution
In order to use Neyman-Pearson lemma, we construct a new table showing the likelihood ratios
and their respective probability of occurring under H0 .
X 1 2 3 4 5 6 7 8
f1 22
f0
7 6 5 4 3 2 1 43
P0 0.02 0.02 0.02 0.02 0.02 0.02 0.02 0.86

By Neyman-Pearson lemma, the UMP test has the form of a Neyman test. As a consequence,
the probability of rejection given sample x is given by the following function
f1 (x)

1 if f0 (x) > k

φ(x) = 0 if ff01 (x)
(x)
<k
f1 (x)

γ if f0 (x) = k


We can now exploit the fact that our test is asked to fulfil a level of α. Recalling that the Xi ’s
are i.i.d copies of X, we have
α = E0 [φ(x)]
   
f1 (x) f1 (x)
= 1P0 > k + γP0 =k
f0 (x) f0 (x)
n
! n
!
Y f1 (xi ) Y f1 (xi )
= P0 > k + γP0 =k
f 0 (x i ) f 0 (x i )
 i=1   i=1 
f1 (x) 1/n f1 (x) 1/n
= P0 >k + γP0 =k
f0 (x) f0 (x)
We can now refer to the table above in order to pick the suitable value for k 1/n . We observe
that

LSTAT2050 - Analyse statistique II - Practice session 2 4

  
1/n f1 (x)
1. If we pick k = 4, we get P0 f0 (x)
> 4 = 0.06.
 
f1 (x)
2. If we pick k 1/n = 5, we get P0 f0 (x)
> 5 = 0.04.
In conclusion,
1. For α = 0.06, take k = 4n and γ = 0. The rejection region is thus given by W0.06 =
{f1 (x) > 4n f0 (x)}.
2. For α = 0.04, take k = 5n and γ = 0. The rejection region is thus given by W0.04 =
{f1 (x) > 5n f0 (x)}.

Exercise 5 - simple test - problem 0

Let X1 , . . . , Xn be i.i.d copies of a random variable X with support (0, 1).
— Under H0 , X is uniformly distributed in the interval (0, 1).
— Under H1 , X has the following density function : p1 (x) = 3x2 1[0,1] (x).
Find a UMP test of level α = 0.02.

Solution
By Neyman-Pearson lemma
n andousing the fact that our variable is continuous, the rejection
f1 (x)
region writes as W = f0 (x) > k . Let us now work out this inequality.

n
!2
f1 (x) Y
> k ⇔ 3n Xi >k
f0 (x) i=1
n
Y
⇔ Xi > k ∗
i=1
n
X
⇔ log(Xi ) > k ∗∗
i=1
n
X
⇔ T (X) = − log(Xi ) < k0
i=1

This steps are undertaken in order to express our condition in terms of a statistic T (X) for
which is is easy to find the distribution. First, − log(X) follows an Exp(1) distribution (you
can find this easily by looking at its CDF). Second, since T (X) is a sum of n Exp(1) variables
H
we have that T (X) ∼0 Ga(n, 1). Hence, we need to find k0 such that P0 (T (X) < k0 ) = α. In
conclusion, k0 is the α-quantile of a Ga(n, 1) distribution. In conclusion, we can rewrite our
rejection region as W = {T (X) < k0 }, where k0 is the α-quantile of a Ga(n, 1) distribution.

LSTAT2050 - Analyse statistique II - Practice session 2 5