UNIT 5 INTRODUCTION TO ESTIMATION

Structure
5.1 Introduction
Objectives
5.2 Basic Terminology
5.3 Characteristics of Estimators
5.4 Unbiasedness
5.5 Consistency
5.6 Efficiency
Most Efficient Estimator
Minimum Variance Unbiased Estimator
5.7 Sufficiency
5.8 Summary
5.9 Solutions /Answers

5.1 INTRODUCTION
In many real-life problems, the population parameter(s) is (are) unknown and
someone is interested to obtain the value(s) of parameter(s). But, if the whole
population is too large to study or the units of the population are destructive in
nature or there is a limited resources and manpower available then it is not
practically convenient to examine each and every unit of the population to find
the value(s) of parameter(s). In such situations, one can draw sample from the
population under study and utilize sample observations to estimate the
parameter(s).
Every one of us makes estimate(s) in our day to day life. For example, a house
wife estimates the monthly expenditure on the basis of particular needs, a sweet
shopkeeper estimates the sale of sweets on a day, etc. So the technique of
finding an estimator to produce an estimate of the unknown parameter on the
basis of a sample is called estimation.
There are two methods of estimation:
1. Point Estimation
2. Interval Estimation

In point estimation, we determine a appropriate single statistic whose value is

used to estimate the unknown parameter whereas in interval estimation, we
determine an interval that contains true value of the unknown parameter with
certain confidence. The point estimation and interval estimation are briefly
described in Unit 6 and Unit 7 respectively of this block.
Estimation admits two problems; the first is to select some criteria or properties
such that if an estimator possesses these properties it is said to be the best
estimator among possible estimators and the second is to derive some methods
or techniques through which we obtain an estimator which possesses such
properties. This unit is devoted to explain the criteria of good estimator. This
unit is divided into nine sections. Section 5.1 is introductory in nature. The
basic terms used in estimation are defined in Section 5.2. Section 5.3 is devoted
to criteria of good estimator which are explained one by one in subsequent
5
Estimation sections. Section 5.4 is explored the concept of unbiasedness with examples.
Unbiasedness is based on fixed sample size whereas the concept based on
varying sample size, that is, consistency is described in Section 5.5. There
exists more than one consistent estimator of a parameter, therefore in Section
5.6 is explained the next property efficiency. Section 5.7 is devoted to describe
sufficiency. Unit ends by providing summary of what we have discussed in this
unit in Section 5.8 and solution of exercises in Section 5.9.
Objectives
After studying this unit, you should be able to:
 define the parameter space and joint probability density (mass) function;
 describe the characteristics of an estimator;
 explain the unbiasedness of an estimator;
 explain the consistency of an estimator;
 explain the efficiency of an estimator;
 explain the most efficient estimator;
 explain the sufficiency of an estimator; and
 describe the minimum variance unbiased estimator.

5.2 BASIC TERMINOLOGY

Before discussing the properties of a good estimator, we discuss basic
definitions of some important terms. These terms are very useful in
understanding the fundamentals of theory of estimation discussed in this block.
Discrete and Continuous Distributions
In Units 12 and 13 of MST-003, we have discussed standard discrete and
continuous distributions as binomial, Poisson, normal, exponential, etc. We
know that the populations can be described with the help of distributions,
therefore, standard discrete and continuous distributions are used in statistical
inference. Here, we discuss some standard discrete and continuous distributions
in brief as in tabular form:
S. Distribution Parameter(s) Mean Variance
No.

1 Bernoulli (discrete)
1 x p p pq
P  X  x   px 1  p  ; x  0,1

2 Binomial (discrete)
n&P np npq
P  X  x   n Cx pxq n x ; x  0,1,..., n
Poisson (discrete)
3
e  x λ λ λ
P X  x  ; x  0,1,...&   0
x!
Uniform (discrete)
4
1
n 1 n2  1
n
P  X  x   ; x  0,1,..., n 2 12
n
Hypergeometric (discrete)
5 nM NM  N  M  N  n 
M
Cx N  M Cn  x N, M & n
P X  x  N
; x  0,1,..., min M, n N N 2  N  1
Cn

6
 Introduction to Estimation
6 Geometric (discrete) p p
p
P  X  x   pq x ; x  0,1, 2,... q q2

7 Negative Binomial (discrete) rp rp

r&p
P  X  x   x r1 Cr 1prq x ; x  0,1,2, ... q q2

Normal(continuous)
2
1  x  
8 1   
f x  e 2  
;   x   µ & σ2 µ σ2
 2
&   0,      

Standard Normal(continuous)
9
1  12 x 2 -- 0 1
f x  e ; x
2
Uniform (continuous) 2

1 a&b
ab b  a
10 f x  ; a  x  b, b  a 2 12
ba
Exponential (continuous) 1 1
11 θ
f  x   ex ; x  0 &   0  2
Gamma (continuous)
b b
12 ab a&b
f  x   eax x b 1; x  0 &a  0 a a2
b
Beta First Kind (continuous)
1 b1
ab
13 f x   x a 1 1  x  ; 0  x  1 a 2
B  a,b  a&b  a  b  a  b  1
ab
& a  0, b  0

Beta Second Kind (continuous)

14 1 x a 1 a a  a  b  1
f x  ; x0 a&b 2
B  a, b  1  x a  b b 1  b  1  b  2 
& a  0,b  0

Parameter Space
The set of all possible values that the parameter  or parameters 1, 2, …, k
can assume is called the parameter space. It is denoted by Θ and is read as
“big theta”. For example, if parameter  represents the average life of electric
bulbs manufactured by a company then parameter space of  is    :   0,
that is, the parameter average life  can take all possible values greater than or
equal to 0, Similarly, in normal distribution (, σ2), the parameter space of
parameters  and σ2 is   (, 2 ) :     ; 0    .

Joint Probability Density (Mass) Function

If X1 ,X 2 , ..., X n is a random sample of size n taken from a population whose
probability density (mass) function is f(x,θ) where,  is the population
parameter then the joint probability density (mass) function of sample values is
denoted by f  x1 , x1 ,..., x n ,  and defined as

For discrete case,

f  x1 , x1,..., x n ,   P  X1  x 1 , X 2  x 2 ,..., X n  x n 
7
Estimation since X1 ,X 2 , ..., X n are independent, therefore,

f  x1 , x1 ,..., x n ,   P  X1  x1  P  X 2  x 2 ...P  X n  x n 

In this case, the function f  x1 , x1 ,..., x n ,  represents the probability that the
particular sample x1 , x 2 , ..., x n has been drawn for a fixed (given) value of
parameter θ.
For continuous case,
f  x1 , x1,..., x n ,   f  x1,  .f  x 2 ,  ... f  x n , 

In this case, the function f  x1 , x1 ,..., x n ,  represents the probability density

function of the random sample X1 ,X 2 , ..., X n .

The process of finding the joint probability density (mass) function is described
by taking some examples as:
If a random sample X1 ,X 2 , ..., X n of size n is taken from Poisson distribution
whose pdf is given by
e x
P X  x  ; x  0, 1, 2, ... &   0
x!
then joint probability mass function of X1 ,X 2 , ..., X n can be obtained as

f  x1 , x1,..., x n ,    P  X1  x1  P  X 2  x 2 ...P  X n  x n 

e  x1 e x2 e  xn

 . ...
x1! x 2! x n!
 ... 
 
e  x1  x2  ...  xn
n  times


x1 !x 2 ! ... x n !
n

 nλ
 xi
e λi 1
 n
Π x i!
i1

Similarly, if X1 ,X 2 , ..., X n is a random sample of size n taken from

exponential population whose pdf is given by
f  x,   e x ; x  0,   0
then the joint probability density function of sample values can be obtained as
f  x1 , x1 ,..., x n ,    f  x1 ,  .f  x 2 ,  ... f  x n ,  

 ex1 .ex 2 ... ex n


1  ...
1 1
 n  times
e  x1  x 2 ... x n 
n

n
  xi
f  x1 , x1 ,..., x n ,    e i 1

8
Let us check your understanding of above by answering the following Introduction to Estimation
exercises.
E1) What is the pmf of Poisson distribution with parameter λ = 5. Also find
the mean and variance of this distribution.
E2) If  represents the average marks of IGNOU’s students in a paper of 50
marks. Find the parameter space of .
E3) A random sample X1 , X 2 , ..., X n of size n is taken from Poisson
distribution whose pdf is given by
e  x
P X  x  ; x  0, 1, 2, ... &   0
x!
Obtain joint probability mass function of X1 ,X 2 , ..., X n .

5.3 CHARACTERISTICS OF ESTIMATORS

It is to be noted that a large number of estimators can be proposed for an
unknown parameter. For example, if we want to estimate the average income
of the persons living in a city then the sample mean, sample median, sample
mode, etc. can be used to estimate the average income. Now, the question
arises, “Are some of possible estimators better, in some sense, than the others?”
Generally, an estimator can be called good for two different situations:
(i) When the true value of parameter is being estimated is known− An
estimator might be called good if its value close to the true value of the
parameter to be estimated. In other words, the estimator whose sampling
distribution concentrates as closely as possible near the true value of the
parameter may be regarded as the good estimator.
(ii) When the true value of the parameter is unknown− An estimator may
be called good if the data give good reason to believe that the estimate will
be closed to the true value.
In the whole estimation, we estimate the parameter when the true value of the
parameter is unknown. Hence, we must choose estimates not because they are
certainly close to the true value, but because there is a good reason to believe
that the estimated value will be close to the true value of parameter. In this unit,
we shall describe certain properties, which help us in deciding whether an
estimator is better than others. Prof. Ronald. A. Fisher,
a Great English
Prof. Ronald A. Fisher was the man who pushed ahead the theory of estimation Mathematical
and introduced these concepts and gave some properties of good estimator as Statistician.
follows: (1890-1962)

1. Unbiasedness
2. Consistency
3. Efficiency
4. Sufficiency
We shall discuss these properties one by one in the subsequent sections.
Now, give the answer of the following exercise.
E4) Write the four properties of good estimator.
 Estimation
5.4 UNBIASEDNESS
Generally, population parameter(s) is (are) unknown and if the whole
population is too large to study to find the value of unknown parameter(s) then
one can estimate the population parameter(s) with the help of estimator(s)
which is(are) always a function of sample values.
An estimator is said to be unbiased for the population parameter such as
population mean, population variance, population proportion, etc. if and only if
An estimator is said to the average or mean of the sampling distribution of the estimator is equal to the
be unbiased if the true value of the parameter.
expected value of the
estimator is equal to the Mathematically,
true value of the If X1 ,X 2 , ..., X n is a random sample of size n taken from a population whose
parameter being
estimated. probability density (mass) function is f(x,θ) where,  is the population
parameter then an estimator T = t( X1 ,X 2 , ..., X n ) is said to be unbiased
estimator of the parameter  if and only if
E(T)  θ ; for all θ  Θ
This property of estimator is called unbiasedness.
Normally, it is preferable that the expected value of the estimator should be
exactly equal to the true value of the parameter being estimated. But if the
expected value of the estimator does not equal to the true value of parameter,
then the estimator is said to be “biased estimator”, that is, if
E (T )  θ
then estimator T is called biased estimator of .
The amount of biasness is given by
b(θ)  E (T)  θ
If b(θ) > 0 or E(T) > θ, then the estimator T is said to be positively biased for
parameter .
If b(θ) < 0 or E(T) < θ, then the estimator T is said to be negatively biased for
parameter .
If E (T)  θ as n   i.e. if an estimator T is unbiased for a large sample only
then estimator T is said to be asymptotically unbiased for .
Now, we explain the procedure how to show that a statistic is unbiased or not
for a parameter with the help of some examples:
Example 1: Show that sample mean (X) is an unbiased estimator of the
population mean ( ) if it exists.

Solution: Let X1 ,X 2 , ..., X n be a random sample of size n taken from any

population with mean . Then for unbiasedness we have to show that
E X  

Consider,
 X  X 2  ...  X n 
E  X  E  1   By defination of sample mean 
 n
10
 1 Introduction to Estimation
  E  X1   E  X 2   ...  E  X n    E  aX  bY   aE  X   bE  Y 
n
Since X1, X2,…,Xn are randomly drawn from same population so they also If X and Y are two random
follow the same distribution as the population. Therefore, variables and a & b are two
E(X1 )  E(X 2 )  ...  E(X n )  E(X)   constants then by the
addition theorem of
Thus, expectation, we have
E  aX  bY   aE  X   bE  Y 
1 
E(X)       ...   
n  

n  times


1
  n   
n
E  X  

Hence, sample mean  X  is an unbiased estimator of the population mean .

Also if x1 , x 2 , ..., x n are the observed values of the random sample
1 n
X1 ,X 2 , ..., X n then x   x i is unbiased estimate of population mean.
n i1
Example 2: A random sample of 10 cadets of a centre is selected and
measures their weights (in kg) which are given below:
48, 50, 62, 75, 80, 60, 70, 56, 52, 78
Determine an unbiased estimate of the average weight of cadets of the centre.
Solution: We know that sample mean (X) is an unbiased estimator of the
population mean and its particular value is the unbiased estimate of population
mean, therefore,
1 n X  X2  ...  X n
X 
n i1
Xi  1
n

48  50  62  75  80  60  70  56  52  78
  63.10
10
Hence, an unbiased estimate of the average weight of cadets of the centre is
63.10 kg.
Example 3: A random sample X1 ,X 2 , ..., X n of size n taken from a population
whose pdf is given by
1
f ( x, )  e x /  ; x  0 ,  0

Show that sample mean  X  is an unbiased estimator of parameter .

Solution: For unbiasedness, we have to show that

E X  

Here, we are given that

11
Estimation 1
f ( x, )  e x /  ; x  0 ,  0

Since we do not know the mean of this distribution therefore first of all we find
the mean of this distribution. So we consider,

E  X    x f  x,  dx
0

 
1 1
  x ex / dx   x ex / dx
0
 0

1 21  x /  1 2   n 1  ax n
  x e dx     x e dx  n 
0  1/ 2  0 a 

Since X1, X2,…,Xn are randomly drawn from same population having mean θ,
therefore,

E  X1   E  X 2   ...  E  X n   E  X   

Consider,
If X and Y are two random
 X  X 2  ...  X n 
variables and a & b are two E  X  E  1   By defination of sample mean 
constants then by the  n
addition theorem of 1  E  aX  bY  
  E  X1   E  X 2   ...  E  X n   
expectation, we have n   aE  
X  bE   
Y
E  aX  bY   aE  X   bE  Y 
1
 (

 ... )

n n  times

1
  n   
n
Thus, X is an unbiased estimator of .

Note 1: If X1 ,X 2 , ..., X n is a random sample taken from a population with

mean  and variance σ2, then

1 n 2
S2  
n i1
 Xi  X is a biased estimator of σ2.

n 2 1 n 2
whereas, S2 
n 1
S i.e. S2  
n  1 i1
 Xi  X is an unbiased estimator of σ2.

The proof of the above result is beyond the scope of this course. But for your
convenience, we will show this result with the help of the following example.
Example 4: Consider a population comprising three televisions of certain
company. If lives of televisions are 8, 6 and 10 years then construct the
sampling distribution of average life of Televisions by taking samples of size 2
and show that sample mean is an unbiased estimator of population mean life.
Also show that S2 is not an unbiased estimator of population variance whereas
S2 is an unbiased estimator of population variance where,

12
 1 n 2 1 n 2 Introduction to Estimation
S2  
n i1
 Xi  X and S2  
n 1 i1
 Xi  X

Solution: Here, population consists three televisions whose lives are 8, 6 and
10 years so we can find the population mean and variance as
8  6  10
 8
3
1 2 2 2 8
2   8  8    6  8   10  8     2.67
3   3
Here, we are given that
Population size = N = 3 and sample size = n = 2
Therefore, possible numbers of samples (with replacement) that can be drawn
from this population are Nn = 32 = 9. For each of these 9 samples, we will
calculate the values of X,S2 and S2 by the formulae given below:

1 n 1 n 2 1 n 2
X 
n i1
X i , S2
 
n i1
 X i  X  and S2
 
n  1 i1
 Xi  X

and necessary calculations for these results are shown in Table 5.1 given
below:
Table 5.1: Calculation for X, S2 and S2

Sample Sample X 2
2 S2 S2
Observation  Xi  X 
i 1

1 8, 8 8 0 0 0
2 8, 6 7 2 1 2
3 8,10 9 2 1 2
4 6, 8 7 2 1 2
5 6, 6 6 0 0 0
6 6, 10 8 8 4 8
7 10, 8 9 2 1 2
8 10, 6 8 8 4 8
9 10, 10 10 0 0 0
Total 72 12 24

We calculate X,S2 and S2 as

1 1 1
X1  8  8  8, X 2  8  6   7,..., X 9  10  10  10
2 2 2
1 2 2 1 2 2
S12   8  8  8  8   0, S22  8  7    6  7    1,...,
2   2  
1 2 2
S9  10  10   10  10    0
2

2  

13
Estimation 1  2 2 1  2 2
S12 
 8  8   8  8    0, S 22   8  7    6  7    2,...,
2 1 2 1
1  2 2
S92 
 10  10   10  10    0
2 1
Form the Table 5.1, we have
1 k 1
E X   X i   72  8  
k i 1 9
Hence, sample mean is unbiased estimator of population mean.
Also
1 k 2 1
E S2    Si   12  1.33   2
k i 1 9

Therefore, S2 is not an unbiased estimator of σ2 whereas,

1 k 2 1
E S2    Si   24  2.67   2
k i 1 9
S2 is unbiased estimator of σ2.
Remark 1:
1. Unbiased estimators may not be unique. For example, sample mean and
sample median are unbiased estimators of population mean of normal
population.
2. Unbiased estimators do not always exist for all the parameters. For
example, for a Bernoulli distribution (), there is no unbiased estimator for
2. Similarly, for a Poisson distribution (λ), there exists no unbiased
estimator for 1/ λ.
3. If an estimator is unbiased for all types of distribution, then it is called an
absolutely unbiased estimator. For example, sample mean is an absolutely
unbiased estimator of population mean, if population mean exists.
4. If Tn and Tn* are two unbiased estimator of parameter  then aTn+ (1−a) Tn*
is also unbiased estimator of  where, ‘a’ (0  a  1) is any constant.
For the better understanding of the unbiasedness try some exercises.

E5) If X1 ,X 2 , ..., X n is a random sample taken from Poisson distribution

whose probability mass function is given by
e   x
P X  x  ; x  0, 1, 2, ... &   0
x!
then show that the sample mean X is unbiased estimator of λ.
E6) If X1 ,X 2 , ..., X n is a random sample of size n taken from a population
whose pdf is
f x,   e x   ;  x ; 
then show that the sample mean is an unbiased estimator of (1 + ).

14
One weakness of unbiasedness is that it requires only the average value of the Introduction to Estimation
estimator equals to the true value of population parameter. It does not require
those values of the estimator to be reasonably close to the population
parameter. For this reason, we require some other properties of good estimator
as consistency, efficiency and sufficiency which are described in subsequent
sections.

5.5 CONSISTENCY
In previous section, we have learnt about the unbiasedness. An estimator T is
said to be unbiased estimator of parameter, say,  if the mean of sampling
distribution of estimator T is equal to the true value of the parameter . This
concept was defined for a fixed sample size. In this section, we will learn about
consistency which is defined for increasing sample size.
If X1 ,X 2 , ..., X n is a random sample of size n taken from a population whose
probability density (mass) function is f(x,θ) where,  is the population
parameter then consider a sequence of estimators, say, T1 = t1(X1),
T2 = t2(X1, X2), T3 = t3(X1, X2, X3),…, Tn = tn(X1, X2, ..., Xn) . A sequence of
estimators is said to be consistent for parameter  if the deviation of the values
of estimator from the parameter tends to zero as the sample size increases. That
means values of estimators tend to get closer to the parameter  as sample size
increases.
In other words, a sequence {Tn} of estimators is said to be consistent sequence
of estimators of  if Tn converges to  in probability, that is
p
Tn    as n   for every   … (3)

or for every   0

lim P  Tn       1 ... (4)

n 

or for every   0 and   0 there exist n  m such that

P  Tn       1   ; nm … (5)

where, m is some very large value of n. Expressions (3), (4) and (5) are to
mean the same thing.
Generally, to show that an estimator is consistent with the help of above
definition is slightly difficult, therefore, we use sufficient conditions for
consistency which are given below:
Sufficient conditions for consistency
If {Tn}is a sequence of estimators such that for all   

(i) E  Tn    as n  , that is, estimator Tn is either unbiased or

asymptotically unbiased estimator of  and

(ii) Var  Tn   0 as n  , that is, variance of estimator Tn converges to 0 as

n .

15
Estimation Then estimator Tn is a consistent estimator of .
Now, we explain the procedure based on both the criteria (definition and
sufficient condition) to show that a statistic is consistent or not for a parameter
with the help of some examples:
Example 5: Prove that sample mean is always a consistent estimator of the
population mean provided that the population has a finite variance.

Solution: Let X1 ,X 2 , ..., X n be a random sample taken from a population

having mean  and finite variance σ2. By the definition of consistency, we have
lim P  Tn       lim P  X       Here, Tn  X 
n  n 

 X   n 
 lim P   
n 
 / n  

By central limit theorem (described in Unit 1 of this course), we know that the
X 
variate Z  is a standard normal variate for large sample size n.
/ n
Therefore,
  n
lim P  Tn  θ  ε   lim P  Z  
n  n
  

  n  n
 lim P  Z   X  a  a  X  a 
n
    
 n
 b
 
 lim
n   f zdz  P  a  U  b   f  u  du 
 n  a 

 n

1 z2 / 2  1 z2 / 2 
 lim
n   2
e dz  f  z   2  e 
 n  



1  z2 / 2
  e dz
 2

1 z 2 / 2
Since e is the pdf of a standard normal variate Z therefore, the
2π
integration of this in whole range   to  is unity.
Thus,
lim P  Tn       lim P  X       1 as n  
n  n 

Hence, sample mean is a consistent estimator of population mean.

Note 2: This example can also be proved with the help of sufficient conditions
for consistency as shown in next example.

16
Example 6: If X1 ,X 2 , ..., X n is a random sample taken from Poisson Introduction to Estimation

distribution(λ), then show that sample mean  X  is consistent estimator of λ.

Solution: We know that the mean and variance of Poisson distribution (λ) are
E  X    and Var  X   

Since X1 ,X 2 , ..., X n are independent and come from same Poisson distribution,
therefore,
E  X i   E  X    and Var  X i   Var  X    for all i  1, 2, ..., n

Now consider,

1 
E  X   E   X1  X2  ...  Xn    Bydefination of sample mean
n 

1  E  aX  bY  
  E  X1   E  X 2   ...  E  X n  
n   aE  X   bE  Y  
 

1  1
  
    =  n   
 ... 
n n  times  n

Thus, sample mean  X  is unbiased estimator of .

Now consider,

1 
Var  X   Var   X1  X2  ...  Xn  
n 
 If X and Y are two 
1 independent 
 2  Var  X1   Var  X 2   ...  Var  X n    random variable then 
n  Var(aX + bY) 
= a 2 Var(X) + b2 Var(Y) 
 
1  
  
2  
 
 ... 

n  n  times 
1
 nλ 
n2

Var  X    0 as n  
n
Hence, by sufficient condition of consistency, it follows that sample mean (X)
is consistent estimator of λ.

Remark 2:
1. Consistent estimators may not be unique. For example, sample mean and
sample median are consistent estimators of population mean of normal
population.
2. An unbiased estimator may or may not be consistent.

3. A consistent estimator may or may not be unbiased.

17
Estimation 4. If Tn is a consistent estimator of  and f is a continuous function of  then
f  Tn  is consistent estimator of f (). This property is known as invariance
Continuous function is property. For example, if X is a consistent estimator of population mean θ
described in Sections
5.7 and 5.8 of Unit 5 of then eX also consistent estimator of e because e is a continuous function of
MST- 001. θ.
It is now time for you to try the following exercises to make sure that you have
understood consistency.

E7) If X1 ,X 2 , ..., X n is a random sample of size n taken from pdf

1 ;   x    1
f  x,   
0 ; elsewere
then show that the sample mean is an unbiased as well as consistent
 1
estimator of  θ   .
 2

E8) If X1 ,X 2 , ..., X n are n observations taken from geometric distribution

with parameter , then show that X is consistent estimator of 1/. Also
find consistent estimator of e1/.

5.6 EFFICIENCY
In some situations, we see that there are more than one estimators of an
parameter which are unbiased as well as consistent. For example, sample mean
and sample median both are unbiased and consistent for the parameter  when
sampling is done from normal population with mean  and known variance σ2.
In such situations, there arises a necessity of some other criterion which will
help us to choose ‘best estimator’ among them. A criterion which is based on
the concept of variance of the sampling distribution of the estimator is termed
as efficiency.
If T1 and T2 are two estimators of an parameter . Then T1 is said to be more
efficient than T2 for all sample sizes if
Var  T1   Var  T2  for all n

Let us do some examples about efficiency:

Example 7: Show that sample mean is more efficient estimator than sample
median for estimating mean of normal population.
Solution: Let X1 ,X 2 , ..., X n be a random sample taken from normal
population with mean  and variance σ2. Also let X and X  be the sample
mean and sample median respectively. We have seen in Unit 2 that the
sampling distribution of the mean from a normal population follows normal
distribution with means  and variance σ2/n. Similarly, it can be shown that the
sampling distribution of the median from a normal population also follows
 2
normal distribution with mean  and variance . Therefore,
2 n

18
 σ2 Introduction to Estimation
Var  X  
n
2
   πσ
Var  X
2n

 2 2   2    . Thus, we
But   and  1 therefore, Var  X   Var  X
2n n  2 n 
conclude that sample mean is more efficient estimator than sample median.
Example 8: If X1, X2, X3, X4 and X5 is a random sample of size 5 taken from a
population with mean  and variance σ2. The following two estimators are
suggested to estimate 
X1  X 2  X 3  X 4  X 5 X  2X 2  3X 3  4X 4  5X 5
T1  , T2  1
5 15
Are both estimators unbiased? Which one is more efficient?
Solution: Since X1, X2,…,X5 are independent and taken from same population
with mean  and variance σ2 therefore,
E  X i    and Var  X i   2 for all i  1,2,...,5
Consider,
 X  X2  X3  X4  X5 
E  T1   E  1 
 5
1
  E  X1   E  X 2   E  X 3   E  X 4   E  X 5  
5
1
 µ  µ  µ  µ  µ 
5
E T1   µ
Similarly,
 X  2X 2  3X3  4X4  5X5 
E  T2   E  1 
 15
1
  E  X1   2E  X 2   3E  X 3   4E  X 4   5E  X 5  
15 
1
 µ  2µ  3µ  4µ  5µ
15
1
 15 
15
E  T2   

Hence, both the estimators T1 and T2 are unbiased estimators of .

Now for efficiency, we consider
 X  X 2  X 3  X 4  X5 
Var  T1   Var  1 
 5

19
Estimation 1
  Var  X1   Var  X 2   Var  X 3   Var  X 4   Var  X 5  
25 
1 2

25

σ  σ2  σ2  σ2  σ2 
1

25
 52 
1
Var  T1    2
5
Similarly,
 X  2X2  3X3  4X 4  5X5 
Var  T2   Var  1 
 15

1  Var  X1   4Var  X 2   9Var  X 3  

  
225   16Var  X 4   25Var  X5  


1 2 55σ 2

225

σ  4σ 2  9σ 2  16σ 2  25σ 2 
225

11σ 2
VarT2  
45
Since, Var  T1   Var  T2  , therefore, we conclude that estimator T1 is more
efficient than T2.
5.6.1 Most Efficient Estimator
In a class of estimators of a parameter, if there exists one estimator whose
variance is minimum (least) among the class, then it is called most efficient
estimator of that parameter. For example, suppose T1, T2 and T3 are three
estimators of parameter θ having variance 1/n, 1/(n+1) and 5/n respectively.
Since variance of estimator T2 is minimum, therefore, estimator T2 is most
efficient estimator in that class.
The efficiency of an estimator measured with respect to the most efficient
estimator is called “Absolute Efficiency”. If T* is the most efficient estimator
having variance Var(T*) and T is any other estimator having variance Var(T),
then efficiency of T is defined as

Var  T* 
e
Var  T 

Since variance of most efficient estimator is minimum, therefore,

Var  T* 
e 1
Var  T 

5.6.2 Minimum Variance Unbiased Estimator

An estimator T of parameter  is said to be minimum variance unbiased
estimator (MVUE) of  if and only if
20
(i) E (T) =  for all    , that is, estimator T is unbiased estimator of  and Introduction to Estimation

(ii) Var  T   Var  T for all    , that is, variance of estimator T is less
than or equal to variance of any other unbiased estimator T.
The minimum variance unbiased estimator (MVUE) is the most efficient
unbiased estimator of parameter  in the sense that it has minimum variance in
class of unbiased estimators. Some authors used uniformly minimum variance
unbiased estimator (UMVUE) in place of minimum variance unbiased
estimator (MVUE).
Now, you can try the following exercises.

E9) If X1 ,X 2 , ..., X n is a random sample taken from a population having

1 n
mean  and variance σ2, then show that the statistic T   Xi is
n  1 i1
biased but more efficient than sample mean for estimating the
population mean.
E10) Suppose X1 ,X 2 , ..., X n is a random sample taken from normal
population with mean  and variance σ2. The following two estimators
are suggested to estimate  as
X1  X 2  X 3 X1  X 2
T1  and T2   X3
3 2
Are both estimators unbiased? Which one of them is more efficient?
E11) Define most efficient estimator and minimum variance unbiased
estimator.

5.7 SUFFICIENCY
In statistical inference, the aim of the investigator or statistician may be to
make a decision about the value of the unknown parameter (). The
information that guides the investigator in making a decision is supplied by the
random sample X1 ,X 2 , ..., X n . However, in most of the cases the observations
would be to numerous and too complicated. Directly use of these observations
is complicated or cumbersome, therefore, a simplification or condensation
would be desirable. The technique of condensing or reducing the random
sample X1 ,X 2 , ..., X n into a statistic such that it contains all the information
about parameter  that is contained in the sample is known as sufficiency. So
prior to continuing our search of finding best estimator, we introduce the
concept of sufficiency.
A sufficient statistic is a particular kind of statistic that condenses random
sample X1 , X 2 , ..., X n in a statistic T  t(X1 , X 2 , ..., X n ) in such a way that no
information about parameter  is lost. That means, it contains all the
information about  that is contained in the sample and if we know the value of
sufficient statistic, then the sample values themselves are not needed and can
nothing tell you more about . In other words,

21
Estimation A statistic T is said to be sufficient statistic for estimating a parameter  if it
contains all the information about  which are available in the sample. This
property of an estimator is called sufficiency. In other words,
An estimator T is sufficient for parameter  if and only if the conditional
distribution of X1 , X 2 , ..., X n given T = t is independent of .

Mathematically,
f  x1 , x 2 , ..., x n / T  t   g  x1 , x 2 , ..., x n 

where, the function g  x1 , x 2 ,..., x n  does not depend on the parameter .

Note 3: Generally, the above definition is used to show that a particular

statistic is not a sufficient statistic because it may be very tedious task to obtain
the conditional distribution. Hence, we use factorization theorem which
facilitates us to find sufficient statistic without any difficulty which is given
below.
Theorem (Factorization Theorem): Let X1 , X 2 , ..., X n be a random sample
of size n taken from the probability density (mass) function f(x, ). A statistic
or estimator T is said to be a sufficient for parameter  if and only if the joint
density (mass) function of X1 , X 2 , ..., X n can be factored as

f  x1, x 2 , ..., x n ,    g  t(x),  . h  x 1 , x 2 , ..., x n 

where, the function g  t(x),  is a non-negative function of parameter  and

observed sample values (x1, x 2 ,..., x n ) only through the function t(x) and the
function h  x1 , x 2 ,..., x n  is non-negative function of (x1, x 2 ,..., x n ) and does not
involve the parameter .
For applying factorization theorem, we try to factor the joint density (mass)
function as the product of two functions, one of which is function of
parameter(s) and another is independent of parameter(s).
The proof of this theorem is beyond the scope of this course.
Note 4: The factorization theorem should not be used to show that a given
statistic or estimator T is not sufficient.
Now, do some examples to show that a statistic is sufficient estimator for a
parameter by using the factorization theorem:
Example 9: Show that the sample mean is sufficient for the parameter  of the
Poisson distribution.
Solution: We know that the probability mass function of Poisson distribution
with parameter  is
e x
P X  x  ; x  0, 1, 2, ... &   0
x!
Let X1 , X 2 , ..., X n be a random sample taken from Poisson distribution with
parameter . Then joint mass function of X1 , X 2 , ..., X n can be obtained as

f  x1 , x 2 ,..., x n ,    P  X1  x1  . P  X 2  x 2  ... P  X n  x n 

22
 e  x1 e x2 e  xn Introduction to Estimation
 . ...
x1! x 2! x n!
 ... 
 
e n  times
 x1  x 2  ...  x n

x1 !x 2 ! ... x n !
n
 xi
e  n
 i 1 n 
 n i 1
x i ! repersents the 
 xi !  product of x i ! 
i 1

The joint mass function can be factored as

 
 1   1 n 
f  x1 , x 2 ,..., x n ,     e  n
  nnx
  x  n  xi 
  xi !  i 1 
 i1 
 g  t(x),  . h  x1 , x 2 ,..., x n 

where, g  t(x),    e  n  n x is a function of parameter  and the observed

1
sample values x1 , x 2 ,..., x n only through t(x)  x and h  x 1 , x 2 ,..., x n   n is
 xi !
i 1

a function of sample values x 1 , x 2 ,..., x n and is independent of parameter .

Hence, by factorization theorem of sufficiency, X is sufficient statistic for .

Note 4: Since throughout the course we are using capital letter for statistic or
estimator therefore in the last line of above example we use X in place of x.
Thus, in all the examples and exercises relating to sufficient statistic, we are
using similar approach.
Example 10: A random sample X1 ,X 2 , ..., X n of size n is taken from gamma
distribution whose pdf is given by
a b ax b 1
f  x,a, b   e x
b
Obtain sufficient statistic for
(i) ‘b’ when ‘a’ is known
(ii) ‘a’ when ‘b’ is known
(iii) ‘a’ and ‘b’ both are unknown.
Solution: The joint density function can be obtained as
f  x1, x 2 ,..., x n , a, b   f  x1, a, b  f  x 2 , a, b  ...f  x n , a, b 

a b  ax1 b1 a b ax2 b1 a b axn b1

 e x1 . e x 2 ... e x n
b b b
n
b 1
a nb  xi n
a
 n
e i 1   x i 
 b  i 1 
23
Estimation Case I: When ‘a’ is known then we find sufficient statistic for ‘b’
The joint density function can be factored as
 nb n b 1   a x
i
n
Since ‘a
a  
f  x1 , x 2 ,..., x n ,b    x  .e i 1 ‘a’ is tre
 b n  i1 i  
   
constan

 g  t(x), b .h  x1 , x 2 ,...x n 

b 1
a nb  n 
where, g  t(x), b   n   xi 
is a function of parameter ‘b’ and sample
 b 
i 1 

n
n a  xi
values x1 , x 2 ,..., x n only through t(x)   x i and h  x1 , x 2 ,..., x n   e i 1
is a
i 1

function of sample values x1 , x 2 ,..., x n and is independent of parameter ‘b’.

n
Hence, by factorization theorem of sufficiency,  Xi is sufficient statistic for
i 1
parameter ‘b’
Case II: When ‘b’ is known then we find sufficient statistic for ‘a’
The joint density function can be factored as
n
 a
 xi  b 1
n
 nb
f  x1 , x 2 ,..., x n ,a   a e
i 1 . 1   x 
  n  i
Since ‘b’ is known so ‘b’
is treated as a constant.   b
 i1   
 g  t(x), a .h  x1 , x 2 ,...x n 
n
a
 xi
i 1
where, g  t(x),a   a nbe is a function of parameter ‘a’ and sample values
n b 1
1  n 
x1 , x 2 ,..., x n only through t(x)   x i and h  x 1 , x 2 ,..., x n   n   xi 
is a
i1
 
b 
i 1 
function of sample values x1 , x 2 ,..., x n and is independent of parameter ‘a’.
n
Hence by factorization theorem of sufficiency,  Xi is sufficient statistic for
i1
‘a’.
Case III: When ‘a’ and ‘b’ are unknown then we find jointly sufficient
statistics for ‘a’ and ‘b’
The joint density function can be factored as
n
 a
 xi b 1 
1  n 
f  x 1 , x 2 ,..., x n , a, b    a nbe  .1
i 1
x
n  i 1 i 
 
   b  


 gt1 ( x ), t 2 ( x ), a, b. h x1 , x 2 , ..., x n 

24
 n
Introduction to Estimation
a
 xi 1  n 
b 1
i 1
where, g  t1 (x), t 2 (x), a, b   a nbe n   xi 
is a function of
  b 
i 1 
n
parameters ‘a’& ‘b’ and sample values x1 , x 2 ,..., x n only through t1 (x)   x i
i 1
n
and t 2 (x)   x i whereas, h  x1 , x 2 ,...x n   1 and independent of parameters ‘a’
i 1
and ‘b’.
n n
Hence, by factorization theorem,  X and  X are jointly sufficient for
i1
i
i 1
i

parameters ‘a’& ‘b’.

Example 11: If X1 , X 2 , ..., X n is a random sample taken from uniform
distribution U(, ), find the sufficient statistics for  and .
Solution: The probability density function of U(, ) is given by
1
f x, ,   ;  x 

The joint density function can be obtained as
f  x1 , x 2 ,..., x n , ,    f  x1 , ,   .f  x 2 , ,   ... f  x n , ,  
1 1 1
 . ...
β α βα β α
1

β  αn
ics of Since the range of variable depends upon the parameters so we consider
e ordered statistics X 1 , X 2  , ..., X  n  . Therefore, the joint density function can be
re the factored as
aced
1
er of f  x1 , x 2 ,..., x n , ,    n
;   x 1  x 2   ...  x  n   
e are    
 1 
Xn  I x 1 ,  I2 x n  , .1
n 1
     
where, x 1 and x  n  are the minimum and maximum sample observations
respectively and
1; if x 1  
 
I1 x 1 ,   
0; otherwise
 1; if x  n   
 
I2 x n ,   
 0; otherwise
Therefore,
f  x1 , x 2 ,..., x n , ,    g  t1 (x), t 2 (x), ,   .h  x1 , x 2 ,..., x n 

25
Estimation  
1
where, g  t1 (x), t 2 (x), ,     n
I
1 x 1 , 
 I 2 x n ,   is a function of
     
parameters (α, β) and sample values x1 , x 2 ,..., x n only through t1 (x)  x 1 and
t 2 (x)  x  n  whereas, h  x1 , x 2 ,...x n   1 and independent of parameters ‘α’ and
‘β’.
Hence, by factorization theorem of sufficiency, X 1 and X  n  are jointly
sufficient for  and .
Remark 3:
1. A sufficient estimator is always a consistent estimator.
2. A sufficient estimator may be unbiased.
3. A sufficient estimator is the most efficient estimator if an efficient
estimator exists.
4. The random sample X1 , X 2 ,..., X n and order statistics X 1 , X  2  ,..., X  n  are
always sufficient estimators because both are contain all the information
about the parameter(s) of the population.
5. If T is a sufficient statistic for the parameter  and (T) is a one to one
function of T then (T) is also sufficient for θ. For example, if T   X i is
1 T
sufficient statistic for parameter θ then X 
n
 X i  n is also sufficient
T
for θ because X  is a one to one function of T.
n
Now, you will understand more clearly about the sufficiency, when you try the
following exercises.
E12) If X1 , X 2 , ..., X n is a random sample taken from exp () then find
sufficient statistic for .
E13) If X1 , X 2 , ..., X n is a random sample taken from normal population
N(, σ2), then obtain sufficient statistic for  and σ2or both according as
other parameter is known or unknown.
E14) If X1 , X 2 , ..., X n is a random sample from uniform population over the
interval [0, ]. Find sufficient estimator of .
We now end this unit by giving a summary of what we have covered in it.

5.8 SUMMARY
In this unit, we have covered the following points:
1. The parameter space and joint probability density (mass) function.
2. The basic characteristics of an estimator.
3. Unbiasedness of an estimator.
4. Consistency of an estimator.
5. Efficiency of an estimator.
26
6. The most efficient estimator. Introduction to Estimation

7. Minimum variance unbiased estimator.

8. The sufficiency of an estimator.

5.9 SOLUTIONS / ANSWERS

E1) We know that the pmf of Poisson distribution is
e x
PX  x  ; x  0,1, 2,...&   0
x!
and mean and variance of this distribution are
Mean = Variance = λ
In our case, λ = 5, therefore, pmf of Poisson distribution is
e 5 5x
P X  x  ; x  0,1, 2,...
x!
Also mean and variance of this distribution are
Mean = Variance = λ = 5
E2) Since parameter  represents the average marks of IGNOU’s students
in a paper of 50 marks, therefore, a student can take minimum 0 marks
and maximum 50 marks. Thus, the parameter space of  is
   : 0    50 .

E3) The probability mass function of Poisson distribution is

e x
P X  x  ; x  0, 1, 2, ... &   0
x!
The joint probability mass function of X1 , X 2 , ..., X n can be obtained as

f  x1, x 2 ,..., x n ,    P  X1  x1  . P  X 2  x 2  ... P  X n  x n 

e  x1 e  x2 e  xn

 . ...
x1! x 2! x n!
 ... 
 
e n  times
 x1  x2  ...  x n

x1 !x 2 ! ... x n !
n

 nλ
 xi
e λi 1
 n
Π xi!
i 1

E4) Refer Section 5.3.

E5) We know that the mean of Poisson distribution with parameter λ is λ
i.e.
E(X)  

27
Estimation Since X1 , X 2 , ..., X n are independent and come from same Poisson
distribution, therefore,
E(Xi )  E(X)   for all i  1, 2, ..., n
Now consider,
1   Bydefination of 
E(X)  E  (X1  X2  ...  Xn ) 
n  sample mean 

E  aX  bY 
 E(X1 )  E(X 2 )  ...  E(X n ) 
1 
 
n   aE  X   bE  Y  
1  1
  
      n   
 ... 
n n  times  n
Hence, sample mean (X) is unbiased estimator of parameter .
E6) Here, we have to show that
E X  1  

We are given that

f x,   e x  ;   x  ,      
Since we do not know the mean of this distribution therefore first of all
we find the mean of this distribution. So we consider,

E  X    x f  x,  dx



  xe x   dx


Putting x    y  dx  dy. Also when x    y  0 & when

x    y  . Therefore,

E  X     y   e y dy
0

 
  ye dy   e ydy
y

0 0

 
  y2 1 e y dy   y11e  y dy
0 0


 n 1 
 x e  xdx  n 
 2   1  1 
 0 
 and 2  1  1 

Since X1 , X 2 , ..., X n are independent and come from same population,

so
E(X i )  E(X)  1   for all i  1, 2, ..., n

28
 Now consider, Introduction to Estimation

 X  X2  ...  Xn   Bydefination of 
E  X  E  1 
 n sample mean 

1  E  aX  bY  
  E(X1 )  E(X 2 )  ...  E(X n )    aE  X   bE  Y  
n  

1 
  (1   )  (1   )  ...  (1  )

n   
n  times 

1
  n 1    
n
 1 
Thus, sample mean is an unbiased estimator of (1+).
E7) We have
f x, θ   1 ; θ  x  θ 1

This is the pdf of uniform distribution U[, +1] and we know that for
U[a, b]
2

E X 
ab
and Var  X  
b  a
2 12
In our case, a = θ and b = θ + 1, therefore
2

EX 
   1 1
   and Var  X  
   1    1
2 2 12 12
Since X1 ,X 2 , ..., X n are independent and come from same population,
therefore,
1 1
E  Xi   E  X     and Var  X i   Var  X    i  1, 2, ..., n
2 12

 1
To show that X is unbiased estimator of     , we consider
 2
1   Bydefination of 
E(X)  E  (X1  X2  ...  Xn ) 
n  sample mean 

1  E  aX  bY  
  E(X1 )  E(X 2 )  ...  E(X n )
  aE  X   bE  Y  
n  
 
1  1  1  1 
           ...      
n 
2  2  2
 
 n  times 

1   1 
 n    
n  2 

29
Estimation 1
θ
2

 1
Therefore, X is unbiased estimator of     .
 2

For consistency, we have to show that

1
E X    and Var  X   0 as n  
2
If X and Y are two
independent random Now consider,
variables and a & b are
two constants then 1 
Var  X   Var  (X1  X2  ...  Xn ) 
Var(aX + bY) n 
= a 2 Var(X) + b 2 Var(Y)
1
  Var(X1 )  Var(X 2 )  ...  Var(X n )
n2

 
1  1 1 1 
 2  12  12  ...  12 
n   
 n  times 

1n
  
n 2  12 

1
Now, Var  X    0 as n  
12n
1
Thus, E  X     and Var(X)  0 as n  
2

 1
Hence, sample mean X is also consistent estimator of     .
 2

E8) We know that the mean and variance of geometric distribution(θ) are
given by
1 1 
E(X)  and Var  X   2
 

Since X1 , X 2 , ..., X n are independent and come from same geometric

distribution, therefore,

E(X i )  E(X) and Var  X i   Var  X  for all i  1, 2,..., n

First, we show sample mean X is consistent estimator of 1/.

Therefore, we consider,

1   By defination of 
E  X   E  (X1  X2  ...  X n ) 
n  sample mean 

30
 1  E  aX  bY   Introduction to Estimation
  E(X1 )  E(X 2 )  ...  E(X n )   aE  X   bE  Y  
n  
 
11 1 1
    ...  
 
n    

 n  times 
1n 1
  
n  
Now consider,
1 
Var  X   Var  (X1  X2  ...  Xn ) 
n 
1
  Var(X1 )  Var(X 2 )  ...  Var(X n )
n2
 
1  1     1     1   
 2  2    2   ...   2 
n
      
 n  times 
1   1    1  1   
  n  2    n  2 
n2     
1 1  
Var  X    2   0 as n  
n  
1
Since E  X   and Var(X)  0 as n  

Hence, sample mean X is consistent estimator of 1/.
Since e1/  is continuous function of 1/ therefore, by invariance
property of consistency eX is consistent estimator of e1/  .
E9) Since X1 , X 2 , ..., X n is a random sample taken from a population
having mean  and variance σ2.
Therefore,
E  X i    and Var  X i   2 for all i  1, 2,..., n

Consider,

 1 n 
E  T   E   Xi 
 n  1 i 1 
1
 E  X1  X 2  ...  X n 
n 1

 E(X1 )  E(X 2 )  ...  E(Xn )   aE  X   bE Y 

1  E aX  bY

n 1  

31
Estimation
1  
     ...   
n  1  
n  times


1
  n   
n 1
Therefore, T is biased estimator of population mean .
For efficiency, we find variances of estimator T  and sample mean X as

 1 
Var(T)  Var  (X1  X2  ...  Xn )
 n 1 
If X and Y are two 1
independent random  Var(X1 )  Var(X2 )  ...  Var(Xn )
(n  1)2
variables and a & b are
two constants then
1  2 
Var(aX + bY)    2  ...  2
2   
= a 2 Var(X) + b 2 Var(Y) (n  1)  n  times 

1 n2

(n  1) 2
 n 2
 
(n  1)2
Now consider,
1 
Var(X)  Var  (X1  X2  ...  X n ) 
 n 
1
  Var(X1 )  Var(X 2 )  ...  Var(X n )
n2
1  2 
    2  ...  2
2   
n  n  times 
1 2

n2
  n
n 2


Since, Var(T  ) < Var(X) , therefore, T is more efficient estimator than

sample mean.
E10) Since X1 , X 2 , ..., X n are independent taken from normal population
with mean  and variance σ2 therefore,
E  Xi    and Var  X i   2 for all i  1, 2, 3
To check estimators T1 and T2 are unbiased, we find expectations of T1
and T2 as
If X and Y are two  X  X 2  X3 
E  T1   E  1 
independent random  3 
variables and a & b are
two constants then 1
  E  X1   E  X 2   E  X 3  
E(aX + bY) 3
= aE(X) + bE(Y) and
Var(aX + bY) 1
= a 2 Var(X) + b 2 Var(Y)
E  T1        
3
32
 1 Introduction to Estimation
  3   

Now consider,
 X  X2 X3 
E  T2   E  1  
 4 2 
1 1
  E  X1   E  X 2    E  X 3 
4 2
1   
       
4 2 2 2
ET2   µ
Hence, T 1 and T2 both are unbiased estimators of .
For efficiency, we find the variances of T1 and T2 as
 X  X 2  X3 
Var  T1   Var  1 
 3 
1
  Var  X1   Var  X 2   Var  X 3  
9
1 2 3σ 2

9

σ  σ2  σ2 
9

σ2
VarT1  
3
Now consider,
 X  X 2 X3 
Var  T2   Var  1  
 4 2 
1 1
  Var  X1   Var  X 2    Var  X 3 
16 4
1 2 1
     2    2
16 4
σ 2 σ 2 σ 2  2σ 2
  
8 4 8
3σ 2

8
Since Var  T1   Var  T2  therefore, T1 is more efficient estimator of 
than T2.
E11) Refer Sub-sections 5.6.1 and 5.6.2.
E12) Here, we take random sample from exp () whose probability density
function is given by
f  x,    e x ; x 0 &  0

33
Estimation The joint density function of X1 , X 2 , ..., X n can be obtained and can be
factored as
f  x1 , x 2 ,...x n ,   f  x1 ,  .f  x 2 ,  ...  x n , 

 e  x1 .  e  x 2 ...  e  x n
n

n
  xi
 e i 1

n
   x i 
  n e i1 .1
 
 
L    g  t  x  ,  .h  x1, x 2 ,..., x n 
n
  xi
where, g  t  x  ,    n e i 1
is a function of parameter  and sample
n
values x1 , x 2 ,..., x n only through t  x    x i and h  x1 , x 2 ,..., x n   1 ,is
i 1
n
independent of . Hence by factorization theorem of sufficiency,  Xi
i 1

is sufficient estimator of .
E13) Here, we take random sample from N(, σ2) whose probability density
function is
1
1   x   2
f  x, ,  2   e 2 2
;    x  ,      ,   0
2 2
The joint density function of X1 , X 2 , ..., X n can be obtained as

f  x1 ,x2 ,...x n , , 2   f  x1,, 2  .f  x 2 , , 2  ... f  x n , , 2 

1 1 1
1   x1  2 1   x2  2 1   x n  2
2 2 2 2 2 2
 e . e ... e
2  2 2  2 2  2
n
n 1
 1   2σ2    xi  x    x  µ   2 n
n 1
  e i 1
 1   2 2  xi  
 2πσ 2    e i 1
2
 2 
… (1)
n
n 1
1  2   xi  x  x  µ  2
  2σ i 1
  e 2
 add and subtract x 
 2πσ 

n
 x  x  n 1 2 2
  x    2  x i  x  x  
 1   2 2   i 
  e i 1
2
 2  
n n n
n 1  2 2 
 1   2 2   x i  x     x    2 x     xi  x 

 e i 1 i 1 i 1

2 
 2  

34
 n 1 
n
 x i  x 2  n  x  2  0 
 Introduction to Estimation
 1   2 2   n 
   e
i 1 
  x i  x   0, by 
2  i 1
 2    the 
 property of mean 
n
n 1 2 n 2
 1   2 2   x i x   2  x  
f  x1 , x 2 ,...x n , , 2    e i 1 2
… (2)
2 
 2 
Case I: Sufficient statistic for  when σ2 is known
The joint density function given in equation (2) can be factored as
n
n 1
  n
 x  2  1    2 2
 x i  x 
2

22
f  x 1 , x 2 , ...x n ,     e .  .e i1

  2  2  

 gt x , µ .hx1, x 2 ,..., x n 

n n
 1 
  x  2
2 2
where, g  t  x  ,    e . is a function of parameter 
2 
 2  
and sample values x1 , x 2 ,..., x n only through t  x   x , whereas
n
1

2 2
  x i  x 2
h  x1 , x 2 ,..., x n   e i 1
is independent of . Hence, X is
sufficient estimator for  when σ2 is known.
Case II: Sufficient statistic for σ2 when  is known
The joint density function given in equation (1) can be factored as
n
n 1
 1   22
  x i   2
f  x 1 , x 2 , ...x n ,  2     e
i 1

 2  2 
n
n 1 2
 1   2 2
  x i  
  e
i 1
.1
 2  2 
 gtx , .hx1 , x 2 ,..., x n 
n
n 1 2
 1   2 2   x i  
where, g  t  x  ,      e i 1
is a function of parameter
2
 2  
n
2
σ2 and sample values x1 , x 2 ,..., x n only through t  x     x i    ,
i 1
2
whereas h  x1 , x 2 ,..., x n   1 is independent of σ . Hence by
n
2
factorization theorem of sufficiency,   X i    is sufficient estimator
i 1
2
for σ when  is known.
Case III: When both  and σ2 are unknown
The joint density function given in equation (2) can be factored as
1 
 1   n 1s2  n  x  2  
f  x1 , x 2 ,...x n , ,    
2
2
e 2 2 
 .1
 2 
35
Estimation 1 n 2
where, s2    xi   
n  1 i1

f  x1 , x 2 ,...x n , , 2   g  t1  x  , t 2  x  , , 2  .h  x1,x 2 ,..., x n 

where, g  t1  x  , t 2  x  , , 2  is a function of (,σ2) and sample values

x1 , x 2 ,..., x n only through t1  x   x, t 2  x   s 2 , whereas
h  x1 , x 2 , ..., x n  is independent of (, σ2). Hence by factorization
theorem, X and S2 are jointly sufficient for  and σ2.

Note 5: Here, it is remembered that X is not sufficient statistic for  if

σ2 is unknown and S2 is not sufficient for σ2 if  is unknown.
E14) The sample is taken from U [0,] whose probability density function is
1
f x , θ   ;0xθ θ0
θ
The joint density function of X1 , X 2 , ..., X n can be obtained as

f  x1 , x 2 ,...x n ,   f  x1 ,   .f  x 2 ,  ... f  x n ,  

1 1 1 1
 . ...  n
θ θ θ θ
Since range of variable depends upon the parameter , so we consider
ordered statistics X 1  X  2   ...  X  n  .

Therefore, the joint density function can be factored as

1
f  x1 , x 2 ,...x n ,    ; 0  x 1  x 2   ...  x  n   
n
1 
  n Ix n  , .1
 
where,
 1 ; if x n   θ
Ix n  , θ   
 0 ; otherwise
Therefore,
f  x1 , x 2 ,...x n ,   g  t  x  ,  .h  x1 , x 2 ,..., x n 

where, g  t  x  ,  is a function of  and sample values only through

t(x)  x  n  , whereas h  x1, x 2 ,..., x n  is independent of .

Hence, by factorization theorem X  n  is sufficient statistic for .

36