Statistics for Data Science - 2

Week 9 Graded Assignment Solution

1. Let X1 , X2 , . . . , Xn be i.i.d.
 samples from a distribution
 X with mean µ and standard
X1 + X2 + . . . + Xn
deviation σ. Let µ̂ = 6 be an estimator of µ.
n
i) Is the estimator unbiased?

a) Yes
b) No

Solution:
  
X1 + X2 + . . . + Xn
E[µ̂] = E 6
n
6
= (nµ)
n
= 6µ

And
Bias(µ̂, µ) = E[µ̂] − µ = 6µ − µ = 5µ
Since, Bias(µ̂, µ) ̸= 0, therefore the estimator is not unbiased.
ii) Find the risk of µ̂.
36σ 2
(a) + 25µ2
n
36σ 2
(b) + 5µ
n
6σ 2
(c) + 25µ2
n
6σ 2
(d) + 5µ
n
Solution:
  
X1 + X2 + . . . + Xn
Var(µ̂) = Var 6
n
36
= 2 (nσ 2 )
n
36σ 2
=
n

1
 Risk(µ̂) = Bias(µ̂, µ)2 + Var(µ̂)
36σ 2
= (5µ)2 +
n
2
36σ
= 25µ2 +
n
2. Consider a sample of iid random variables X1 , X2 , . . . , Xn , where n > 20, E[Xi ] =
1 n
µ, Var(Xi ) = σ 2 and the estimator of µ, µ̂n =
P
Xi . Find the MSE of µ̂n .
n − 20 i=21
σ
a) n−20
σ2
b) n−20
σ2
c) n−21
σ
d) n

Solution:
" n
#
1 X
E[µ̂n ] = E Xi
n − 20 i=21
(n − 20)µ
=
n − 20
=µ
This implies that
Bias(µ̂n , µ) = E[µ̂n ] − µ = µ − µ = 0

" n
#
1 X
Var(µ̂n ) = Var Xi
n − 20 i=21
n
1 X
= Var(Xi )
(n − 20)2 i=21
1
= [(n − 20)σ 2 ]
(n − 20)2
σ2
=
(n − 20)

Risk(µ̂) = Bias(µ̂, µ)2 + Var(µ̂)

σ2
=0+
(n − 20)
σ2
=
(n − 20)

2
3. Let X1 , X2 , . . . , Xn ∼ iid X, where X is a random variable with density function
(
θ
θ+1 , x > 1,
fX (x) = x
0, otherwise.
θ
The mean of the random variable X is θ−1
. Find an estimator of θ using method of
moments.
X1 + X2 + . . . + Xn
(a)
X1 + X2 + . . . + Xn − 1
X1 + X2 + . . . + Xn
(b)
1 − X1 + X2 + . . . + Xn
X1 + X 2 + . . . + Xn
(c)
X1 + X2 + . . . + Xn − n
X1 + X 2 + . . . + Xn
(d)
n − X1 + X2 + . . . + Xn
θ
Solution: The mean of the random variable X is θ−1
.
So,
θ
M1 =
θ−1
⇒ M1 θ − M1 = θ
M1
⇒θ=
M1 − 1
X1 +X2 +...+Xn
n
⇒θ= X1 +X2 +...+Xn
n
1 −
X 1 + X2 + . . . + X n
⇒θ=
X1 + X2 + . . . + Xn − n
X1 + X 2 + . . . + Xn
Therefore the estimator of θ is .
X1 + X2 + . . . + Xn − n
4. Let X1 , X2 , X3 ∼ iid Binomial(4, θ). Given a random sample (1, 4, 2), find the maximum
likelihood estimate of θ.
2
a) 3
7
b) 12
1
c) 3
5
d) 12

Solution: Xi ∼ Binomial(4, θ)
⇒ fXi (x) = 4 Cx θx (1 − θ)4−x

3
 Likelihood function is given by
3
Q
L(x1 , x2 , x3 ) = fXi (xi )
i=1
⇒ L(x1 , x2 , x3 ) = 4 Cx1 θx1 (1 − θ)4−x1 × 4 Cx2 θx2 (1 − θ)4−x2 × 4 Cx3 θx3 (1 − θ)4−x3

L(1, 4, 2) = 4 C1 4 C4 4 C2 θ(1+4+2) (1 − θ)12−(1+4+2)

= 24θ7 (1 − θ)5
⇒ log(L(1, 4, 2)) = log(24) + 7 log(θ) + 5 log(1 − θ)

Therefore, ML estimator for θ is given by

θ̂ = arg maxθ [log(24) + 7 log(θ) + 5 log(1 − θ)]

Let Y = log(24) + 7 log(θ) + 5 log(1 − θ)

dY 7 5
⇒ = −
dθ θ 1−θ
Now we will equate this value to zero and find the value of θ

7 5 7
− =0⇒θ=
θ 1−θ 12
7
⇒ θ̂M L =
12

5. Let X1 , X2 , . . . , Xn ∼ iid X, where X is a random variable with density function

(
e−(x−θ) , x > θ,
fX (x) =
0, otherwise.

i) The mean of the distribution is θ + 1. Find the estimator of θ using method of

moments. [1 mark]
X1 + X2 + . . . + Xn
(a)
n
X1 + X2 + . . . + Xn − n
(b)
n
n
(c)
X1 + X2 + . . . + Xn − n
1
(d)
n − X1 + X2 + . . . + Xn

4
 Solution: The mean of the random variable X is θ + 1.
So,

M1 = θ + 1
⇒ θ = M1 − 1
X1 + X2 + . . . + Xn
⇒θ= −1
n
X1 + X2 + . . . + Xn − n
⇒θ=
n
X 1 + X2 + . . . + Xn − n
Therefore the estimator of θ is .
n
ii) Is the method of moments estimator unbiased?

a) Yes
b) No

Solution:
Estimator of θ is
X1 + X 2 + . . . + Xn − n
θ̂ =
n
X1 + X 2 + . . . + Xn − n
E[θ̂] = E[ ]
n
1
= (E[X1 ] + E[X2 ] + . . . + E[Xn ] − n)
n
1
= (nθ + n − n)
n
=θ

And
Bias(θ̂, θ) = E[θ̂] − θ = θ − θ = 0
Since, Bias(θ̂, θ) = 0, therefore the estimator is unbiased.

6. Suppose it is known that a sample consisting of the values 10, 12, 15, 16.5, 18, 19, 20
and 21.5 comes from a population with the density function
( −x
1 θ
e , x > 0,
f (x) = θ
0, otherwise.

Find the maximum likelihood estimate of θ. Enter your answer correct to one decimal.

5
 Solution:
n
Y
L(x1 , x2 , . . . , xn ) = fX (xi )
i=1
n
1 −xi
Y
= e θ
i=1
θ
1  −x1 −x2 −xn

= n e θ e θ ...e θ
θ
1  −(x1 +x2 +...+xn ) 
= n e θ
θ
(x1 + x2 + . . . + xn )
⇒ log(L(x1 , x2 , . . . , xn )) = −n log(θ) −
θ
Therefore, ML estimator for θ is given by
(x1 + x2 + . . . + xn )
θ̂ = arg maxθ [−n log(θ) − ]
θ
(x1 + x2 + . . . + xn )
Let Y = −n log(θ) −
θ
dY n (x1 + x2 + . . . + xn )
⇒ =− +
dθ θ θ2
Now we will equate this value to zero and find the value of θ.
n (x1 + x2 + . . . + xn )
⇒− + =0
θ θ2
x1 + x2 + . . . + xn
⇒θ=
n
x1 + x2 + . . . + xn
⇒ θ̂ =
n
Therefore, maximum likelihood estimate of θ for the given sample will be

10 + 12 + 15 + 16.5 + 18 + 19 + 20 + 21.5
θ̂ =
8
132
=
8
= 16.5

7. Let X be a discrete random variable with the following probability mass function

x 1 2 3 4
1−p p 1−p p
fX (x)
2 2 2 2
Table 8.1.G: PMF of X

6
 Suppose a sample consisting of the values 2, 2, 4, 3, 1, 3, 1 and 2 is taken from the random
variable X. Find the estimate of p using method of moments. Enter your answer correct
to two decimals accuracy.
Solution:
1−p p 1−p p
E[X] = 1 × +2× +3× +4×
2 2 2 2
(1 − p) + 2p + 3(1 − p) + 4p
=
2
=p+2
Now
M1 = E[X] = p + 2
⇒ p = M1 − 2
Therefore, estimate of p will be
X1 + X2 + . . . + Xn
− 2.
n
So, the estimate of p for the given sample will be

2+2+4+3+1+3+1+2
p̂ = −2
8
18
= −2
8
= 0.25

Use the following values of CDF of standard normal distribution to answer the questions:

FZ (1.64) = 0.90, FZ (1.96) = 0.95

8. The heights (in cm) of people in a certain area are normally distributed with mean µ
and standard deviation 40. The heights from a random sample of people are as follows:
160,164,150,160,155,165,170,162,174,150.
Suppose µ̂ is the maximum likelihood estimate for µ. Let X ∼ Normal(µ̂, 1), then find
the value of fX (160).

Solution:
Let the random variable Y represent the heights of people in a certain area.
It is given that Y ∼ Normal(µ, 402 ).

For the given sample 160, 164, 150, 160, 155, 165, 170, 162, 174, 150 of Y , the maximum
likelihood estimate of µ is given by
160 + 164 + 150 + 160 + 155 + 165 + 170 + 162 + 174 + 150
µ̂ = = 161
10
7
Now X ∼ Normal(µ̂, 1) and µ̂ = 161.
1 160 − 161 2
 

− 
1

Therefore, fX (x) = √ e 2 1 = 0.24
2π