Introduction To Econometrics, 5 Edition: Review: Random Variables, Sampling, Estimation, and Inference

Type author name/s here
Dougherty
Introduction to Econometrics,
5th edition
Chapter heading
Review: Random Variables,
Sampling, Estimation, and
Inference
© Christopher Dougherty, 2016. All rights reserved.

UNBIASEDNESS AND EFFICIENCY
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
Much of the analysis in this course will be concerned with three properties of estimators:
unbiasedness, efficiency, and consistency. The first two, treated here, relate to finite
sample analysis: analysis where the sample has a finite number of observations.
1
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
Consistency, a property that relates to analysis when the sample size tends to infinity, is
treated in a later slideshow.
2
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
Suppose that you wish to estimate the population mean mX of a random variable X given a
sample of observations. We will demonstrate that the sample mean is an unbiased
estimator, but not the only one.
3
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
We will start with the proof in the previous sequence. We use the second expected value
rule to take the 1/n factor out of the expectation expression.
4
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
Next we use the first expected value rule to break up the expression into the sum of the
expectations of the observations.
5
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
Thinking about the sample values {X1, …, Xn} at the planning stage, each expectation is
equal to mX, and hence the expected value of the sample mean, before we actually generate
the sample, is mX.
6
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
Generalized estimator Z  1 X 1   2 X 2
However, the sample mean is not the only unbiased estimator of the population mean. We
will demonstrate this supposing that we have a sample of two observations (to keep it
simple).
7
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
We will define a generalized estimator Z which is the weighted sum of the two observations,
l1 and l2 being the weights.
8
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
E  Z   E  1 X 1  2 X 2   E  1 X 1   E   2 X 2 
 1 E  X 1   2 E  X 2   1  X   2  X
 X
We will analyze the expected value of Z and determine the condition that must be satisfied
by the weights for Z to be an unbiased estimator.
9
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
E  Z   E  1 X 1  2 X 2   E  1 X 1   E   2 X 2 
 1 E  X 1   2 E  X 2   1  X   2  X
 X
We begin by decomposing the expectation using the first expected value rule.
10
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
E  Z   E  1 X 1  2 X 2   E  1 X 1   E   2 X 2 
 1 E  X 1   2 E  X 2   1  X   2  X
 X
Now we use the second expected value rule to bring l1 and l2 out of the expected value
expressions.
11
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
E  Z   E  1 X 1  2 X 2   E  1 X 1   E   2 X 2 
 1 E  X 1   2 E  X 2   1  X   2  X
 X
The expected value of X in each observation, before we generate the sample, is mX.
12
Unbiasedness of X
1  1
E  X   E   X 1  ...  X n    E  X 1  ...  X n 
n  n
1 1
  E  X 1   ...  E  X n    n X   X
n n
E  Z   E  1 X 1  2 X 2   E  1 X 1   E   2 X 2 
 1 E  X 1   2 E  X 2   1  X   2  X
 X if 1   2  1
Thus Z is an unbiased estimator of mX if the sum of the weights is equal to one. An infinite
number of combinations of l1 and l2 satisfy this condition, not just the sample mean.
13
Efficiency
estimator B
probability density
estimator A
0 mX
How do we choose among them? The answer is to use the most efficient estimator, the one
with the smallest population variance, because it will tend to be the most accurate.
14
Efficiency
estimator B
probability density
estimator A
0 mX
In the diagram, A and B are both unbiased estimators but B is superior because it is more
efficient.
15
Efficiency of X
 Z2  var  1 X 1  2 X 2 
 var  1 X 1   var   2 X 2   2 cov 1 X 1 ,  2 X 2 
 12 X2 1  22 X2 2
 12 X2  22  X2

 12   1  1   X2
2

  212  21  1  X2
We will analyze the variance of the generalized estimator and find out what condition the
weights must satisfy in order to minimize it.
16
Efficiency of X
 Z2  var  1 X 1  2 X 2 
 var  1 X 1   var   2 X 2   2 cov 1 X 1 ,  2 X 2 
 12 X2 1  22 X2 2
 12 X2  22  X2

 12   1  1   X2
2

  212  21  1  X2
The first variance rule is used to decompose the variance.
17
Efficiency of X
 Z2  var  1 X 1  2 X 2 
 var  1 X 1   var   2 X 2   2 cov 1 X 1 ,  2 X 2 
 12 X2 1  22 X2 2
 12 X2  22  X2

 12   1  1   X2
2

  212  21  1  X2
Note that we are assuming that X1 and X2 are independent observations and so their
covariance is zero. The second variance rule is used to bring l1 and l2 out of the variance
expressions.
18
Efficiency of X
 Z2  var  1 X 1  2 X 2 
 var  1 X 1   var   2 X 2   2 cov 1 X 1 ,  2 X 2 
 12 X2 1  22 X2 2
 12 X2  22  X2

 12   1  1   X2
2

  212  21  1  X2
The variance of X1, at the planning stage, is sX2. The same goes for the variance of X2.
19
Efficiency of X
 Z2  var  1 X 1  2 X 2 
 var  1 X 1   var   2 X 2   2 cov 1 X 1 ,  2 X 2 
 12 X2 1  22 X2 2
 12 X2  22  X2

 12   1  1   X2
2
 if 1  2  1
  212  21  1  X2
Now we take account of the condition for unbiasedness and re-write the variance of Z,
substituting for l2.
20
Efficiency of X
 Z2  var 1 X 1  2 X 2 
 var 1 X 1   var  2 X 2   2 cov 1 X 1 ,  2 X 2 
 12 X2 1  22 X2 2
 12 X2  22  X2

 12   1  1   X2
2
 if 1  2  1
  212  21  1  X2
The quadratic is expanded.
21
Efficiency of X
 Z2   212  21  1  X2
To minimize the variance of Z, we must choose l1 so as to minimize the final expression.
22
Efficiency of X
 Z2   212  21  1  X2
d Z2
 0  41  2  0  1   2  0.5
d1
We differentiate with respect to l1 to obtain the first-order condition.
23
Efficiency of X
 Z2   212  21  1  X2
d Z2
 0  41  2  0  1   2  0.5
d1
d 2 Z2
40
d12
The expression is minimized for l1 = 0.5. It follows that l2 = 0.5 as well. So we have
demonstrated that the sample mean is the most efficient unbiased estimator, at least in this
example. (Note that the second differential is positive, confirming that we have a
minimum.) 24
Efficiency of X
f  1.21 
1
0.8
0.6
0.4
0.2
0
0 0.2 0.4 0.6 0.8 1 1
Alternatively, we could find the minimum graphically. Here is a graph of the expression as a
function of l1.
25
Efficiency of X
f  1.21 
1
0.8
0.6
0.4
0.2
0
0 0.2 0.4 0.6 0.8 1 1
Again we see that the variance is minimized for l1 = 0.5 and so the sample mean is the most
efficient unbiased estimator.
26
Copyright Christopher Dougherty 2016.
These slideshows may be downloaded by anyone, anywhere for personal use.

Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.
The content of this slideshow comes from Section R.6 of C. Dougherty,

Introduction to Econometrics, fifth edition 2016, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oxfordtextbooks.co.uk/orc/dougherty5e/
Individuals studying econometrics on their own who feel that they might benefit
from participation in a formal course should consider the London School of
Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics
www.londoninternational.ac.uk/lse.
2015.12.17

Introduction To Econometrics, 5 Edition: Review: Random Variables, Sampling, Estimation, and Inference

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To Econometrics, 5 Edition: Review: Random Variables, Sampling, Estimation, and Inference

Uploaded by

Copyright:

Available Formats

Type author name/s here

© Christopher Dougherty, 2016. All rights reserved.

The first variance rule is used to decompose the variance.

The quadratic is expanded.

To minimize the variance of Z, we must choose l1 so as to minimize the final expression.

We differentiate with respect to l1 to obtain the first-order condition.

These slideshows may be downloaded by anyone, anywhere for personal use.

The content of this slideshow comes from Section R.6 of C. Dougherty,

You might also like