Unit IV: The Central Limit Theorem and Point Estimation

 To draw the inferences about a population from the analysis of a sample drawn
from the population.
 Sample is representative of Population.
 Statistical inference includes Estimation and Testing of Hypothesis.
 Any function of the random sample 𝑥1 , 𝑥2 , … , 𝑥𝑛 that is being observed, say
𝑇𝑛 (𝑥1 , 𝑥2 , … , 𝑥𝑛 ) is called a statistic. If it is used to estimate an unknown
parameter 𝜃 of the distribution, it is called an estimator. A particular value of the
estimator is called an estimate of 𝜃.
 There will be always infinite number of functions of sample values, called
statistics, which may be proposed. But the one which falls nearest to the true
value of the parameter to be estimated would be the best estimate.
 Point estimate of some population parameter 𝜃 is a single value 𝜃̂ of a statistic 𝑇.
 The set of all possible values of 𝜃 is called the parameter space Θ.

CHARACTERISTICS OF ESTIMATORS
1. Unbiasedness
2. Consistency

Unbiased Estimator
An estimator 𝑇𝑛 = 𝑇(𝑥1 , 𝑥2 , … , 𝑥𝑛 ) is said to be an unbiased estimator of 𝑔(𝜃 ), if

𝐸 (𝑇𝑛 ) = 𝑔(𝜃 ), for all 𝜃 ∈ Θ.

 Positively Biased

 Negative Biased
Q1. If 𝑥1 , 𝑥2 , … , 𝑥𝑛 is a random sample from normal population 𝑁(𝜇, 1).
1
Show that 𝑇 = ∑ 𝑥𝑖2 is an unbiased estimator of 𝜇2 + 1.
𝑛

2𝑋1 +3𝑋2 +𝛼𝑋3

Q2. For what value of 𝛼, 𝑇 = is an unbiased estimator of mean
3
𝜇, where 𝑋1 , 𝑋2 , 𝑋3 is a random sample drawn from normal population.

(A) -1 (B) -2 (C) 0 (D) 2

Q3. Let 𝑋 be distributed in the Poisson form with parameter 𝜃. Show that
𝑇(𝑋) = (−𝑘) 𝑋 is an unbiased estimetor of 𝑒 −(𝑘+1)𝜃 , 𝑘 > 0.

Q4. If 𝑇 is an unbiased estimator of 𝜃. Which of the following is true?

(A) 2T + 1 is an unbiased estimator of 2θ + 1.

(B) T 2 is unbiased estimator of θ2 .

Q5. Let 𝑥1 , 𝑥2 , … , 𝑥𝑛 be a random sample of size 𝑛 from a large population

with mean 𝜇 and variance 𝜎 2 . Show that sample mean 𝑇1 = 𝑋̅ is a unbiased
estimator of population mean 𝜇, but sample variance is not unbiased
estimator of population variance. Hence find unbiased estimate for
variance.
CONSISTENT ESTIMATOR

𝑃(𝐸 (𝑇𝑛 ) → 𝑔(𝜃 )) → 1 as 𝑛 → ∞

i.e., if for every 𝜖 > 0, 𝜂 > 0, there exists a positive integer 𝑛 ≥ 𝑚 (𝜖, 𝜂) such that

𝑃[|𝑇𝑛 − 𝑔(𝜃)| < 𝜖 ] → 1 as 𝑛 → ∞

𝑃[|𝑇𝑛 − 𝑔(𝜃)| < 𝜖 ] > 1 − 𝜂 ∀𝑛 ≥𝑚

Sufficient condition of consistency

If {𝑇𝑛 } be sequence of estimators for all 𝜃 ∈ Θ

 𝐸𝜃 (𝑇𝑛 ) → 𝑔(𝜃 ) as 𝑛 → ∞

 𝑉𝜃 (𝑇𝑛 ) → 0 as 𝑛 → ∞

Q6. Let 𝑥1 , 𝑥2 , … , 𝑥𝑛 be a random sample of size 𝑛 from a large population

with mean 𝜇 and variance 𝜎 2 . Show that sample mean 𝑇1 = 𝑋̅ is a
consistent estimator of Population mean 𝜇.

Q7. Let 𝑥1 , 𝑥2 , … , 𝑥𝑛 be a random sample from population with p.d.f. given

1, 𝜃 <𝑥 <𝜃+1
as 𝑓 (𝑥 ) = { .
0, 𝑒𝑙𝑠𝑒𝑤h𝑒𝑟𝑒
1
Show that 𝑋̅ is consistent estimator of 𝜃 + 2.

Q8. If 𝑋1 , 𝑋2 , … , 𝑋𝑛 are random observations on a Bernoulli variate 𝑋

taking the value with 1 with probability 𝑝 and the value 0 with
probability 1 − 𝑝,
∑𝑛
𝑖=1 𝑥𝑖
(A) Show that is consistent estimator of 𝑝.
𝑛
INVARIANCE PROPERTY OF CONSISTENT ESTIMATORS

Theorem: If 𝑻𝒏 is a consistent estimator of 𝒈(𝜽) and 𝝍{𝒈(𝜽)} is a

continuous function of 𝒈(𝜽), then 𝝍(𝑻𝒏 ) is a consistent estimator of
𝝍{𝒈(𝜽)}.

∑𝑛
𝑖=1 𝑥𝑖 ∑𝑛
𝑖=1 𝑥𝑖
Q8(B) Show that (1 − ) is consistent estimator of 𝑝(1 − 𝑝)
𝑛 𝑛

2 ∑𝑛
𝑖=1 𝑖 𝑥𝑖
Q9. Check whether 𝑇2 = is unbiased and consistent estimator of
𝑛(𝑛+1)
population mean 𝜇 or not.

Efficient Estimator

Efficiency
If 𝑇1 is the most efficient estimator with variance 𝑉1 and 𝑇2 is any other
estimator with variance 𝑉2 , then the efficiency 𝐸 of 𝑇2 is defined as
𝑉
𝐸 = 1 . 𝐸 cannot exceed unity.
𝑉2

Q10. Of the estimators given in Q6 and Q9, which is more efficient

estimator?
METHOD OF ESTIMATION

Method of Maximum Likelihood Estimation (M.L.E.)

Let 𝑥1 , 𝑥2,…, 𝑥𝑛 be a random sample of size 𝑛 from a population with density

function 𝑓(𝑥, 𝜃). Then the likelihood function of the sample values
𝑥1 , 𝑥2 , … , 𝑥𝑛 denoted by 𝐿(𝜃) is their joint density function given by
𝑛

𝐿 = 𝑓 (𝑥1 , 𝜃)𝑓(𝑥2 , 𝜃) … 𝑓 (𝑥𝑛 , 𝜃) = ∏ 𝑓(𝑥𝑖 , 𝜃)

𝑖=1

 Finding an estimator for the unknown parameter 𝜃 = (𝜃1 , 𝜃2 , … , 𝜃𝑘 )

which maximises the likelihood function.
 M.L.E. is denoted as 𝜃̂.
 Conditions for M.L.E. is
𝜕𝐿 𝜕2𝐿
= 0, <0
𝜕𝜃 𝜕𝜃 2

Q11. Let 𝑥1 , 𝑥2 , … , 𝑥𝑛 be a random sample from a normal population

𝑁(𝜇, 𝜎 2 ). Find the maximum likelihood estimators for
(i) 𝜇 when 𝜎 2 is known.

(ii) 𝜎 2 when 𝜇 is known.

(iii) simultaneous estimation of 𝜇 and 𝜎 2 .

Q12. Suppose that 𝑋 is a discrete random variable with the following

probability mass function, where 0 ≤ 𝜃 ≤ 1 is a parameter.
X 0 1 2 3
P(X) 2𝜃/3 𝜃/3 2(1 − 𝜃)/3 (1 − 𝜃)/3
The following 10 independent observations were taken from such a
distribution: (3, 0, 2, 1, 3, 2, 1, 0, 2, 1). What is maximum likelihood
estimate of 𝜃.
 Q13. Let 𝑥1 , 𝑥2 , … , 𝑥𝑛 be random sample from the uniform distribution in
[0, 𝜃]. Find MLE of 𝜃.

Properties of M.L.E.
Assumptions or Regularity Conditions:
𝜕 log 𝐿 ∂2 log 𝐿
(i) The first and second derivatives 𝜕𝜃 and 𝜕θ2 exist and are
continuous functions of 𝜃 in range 𝑅 (including the true value θ0 of
𝜕
parameter) for almost all 𝑥. For every 𝜃 in 𝑅, |𝜕𝜃 log 𝐿| < 𝐹1 (𝑥) and
𝜕2
|𝜕θ2 log 𝐿| < 𝐹2 (𝑥) where 𝐹1 (𝑥 ) and 𝐹2 (𝑥) are integrable functions over
(−∞, ∞).

∂3 𝜕3
(ii) The third order derivative 𝜕θ3 log 𝐿 exists such that |𝜕θ3 log 𝐿| <
𝑀(𝑥), where 𝐸 [𝑀(𝑥)] < 𝐾, a positive quantity.

(iii) For every 𝜃 in 𝑅,

𝜕2 ∞ 𝜕2
𝐸 (− 𝜕θ2 log 𝐿) = ∫−∞ (− 𝜕θ2 log 𝐿) 𝐿 𝑑𝑥 = 𝐼 (𝜃), is finite and
non-zero.

(iv) The range of integration is independent of 𝜃. But if the range of

integration depends upon 𝜃, then 𝑓(𝑥, 𝜃) vanishes at the extreme
depending upon 𝜃.
This assumption is to make the differentiation under the integral sign
valid.

Important Theorems

 CRAMER-RAO Theorem: With probability approaching unity as 𝑛 → ∞,

𝜕
the likelihood equation log 𝐿 = 0, has a solution which converges in
𝜕𝜃
probability to the true value 𝜃0 .
  HAZOOR BAZAR's Theorem: Any consistent solution of the likelihood
equation provides a maximum of the likelihood with probability
tending to unity as the sample size tends to infinity.

 ASYMPTOTIC NORMALITY OF MLE's: A consistent solution of the

likelihood equation is asymptotically normally distributed about the
1
true value 𝜃0 . 𝜃̂ is asymptotically 𝑁 (𝜃0 , ), as 𝑛 → ∞.
𝐼(𝜃0 )

 If M.L.E. exists, it is the most efficient estimator in the class of such

estimator.

 If a sufficient estimator exists, it is a function of the M.L.E.

 INVARIANCE PROPERTY OF MLE

Theorem: If 𝑇 is the MLE of 𝜃 and 𝜓(𝜃) is a one-one function of 𝜃, then
𝜓(𝑇) is the MLE of 𝜓(𝜃).
Q14. Find the maximum likelihood estimate for the parameter 𝜆𝑡 of Poisson
distribution. Also find its variance.

Central Limit Theorem (CLT)

 Liapounoff’s Form

If 𝑋1 , 𝑋2 , … , 𝑋𝑛 , … be a sequence of independent random variables with

𝐸 (𝑋𝑖 ) = 𝜇𝑖 and 𝑉𝑎𝑟(𝑋𝑖 ) = 𝜎𝑖2 , 𝑖 = 1, 2, … and if 𝑆𝑛 = 𝑋1 + 𝑋2 + ⋯ + 𝑋𝑛 ,
then under certain general conditions 𝑆𝑛 follows a normal conditions with
mean 𝜇 = ∑𝑛𝑖=1 𝜇𝑖 and variance 𝜎 2 = ∑𝑛𝑖=1 𝜎𝑖2 as n tends to infinity.

 Lindeberg-Levy’s Form

If 𝑋1 , 𝑋2 , … , 𝑋𝑛 , … be a sequence of independent identically distributed

random variables with 𝐸 (𝑋𝑖 ) = 𝜇 and 𝑉𝑎𝑟(𝑋𝑖 ) = 𝜎 2 , 𝑖 = 1, 2, … and if
𝑆𝑛 = 𝑋1 + 𝑋2 + ⋯ + 𝑋𝑛 , then under certain general conditions 𝑆𝑛 follows a
normal conditions with mean 𝑛𝜇 and variance 𝑛𝜎 2 as n tends to infinity.

Corollary
𝑋1 +𝑋2 +⋯+𝑋𝑛 𝜎2
̅
If 𝑋 = , then 𝐸 (𝑋) = 𝜇 and 𝑉𝑎𝑟(𝑋) = and 𝑋̅ follows
̅ ̅
𝑛 𝑛
𝜎
𝑁 (𝜇, ) as 𝑛 → ∞.
√𝑛

Q15. The lifetime of certain brand of an electric bulb may be considered a

RV with mean 1200 h and standard deviation 250 h. Find the probability
using central limit theorem, that the average lifetime of 60 bulbs exceeds
1250 h.

Q16. A coin is tossed 200 times. Find the approximate probability that the
number of heads obtained is between 80 and 120.

Q17. The guaranteed average life of a certain type of electric light bulb is
1000 h with a standard deviation of 125 h. It is decided to sample the
output so as to ensure that 90% of the bulbs do not fall short of guaranteed
average by more than 2.5%. Use CLT to find the minimum sample size.

Q18. If 𝑉1 , 𝑉2 , … , 𝑉𝑛 are independent noise voltages received in an adder and

𝑉 is the sum of the voltages received, find the probability that the total
incoming voltage exceeds 105, using CLT. Assume that each of the random
variables 𝑉𝑖 is uniformly distributed over (0, 10).