Type author name/s here

Dougherty

Introduction to Econometrics,
5th edition
Chapter heading
Review: Random Variables,
Sampling, Estimation, and
Inference

© Christopher Dougherty, 2016. All rights reserved.

 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

probability density

estimator B

estimator A

0 q

Suppose that you have alternative estimators of a population characteristic q, one

unbiased, the other biased but with a smaller variance. How do you choose between them?

1
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

loss

-3 error (negative) 0 error (positive)3

One way is to define a loss function which reflects the cost to you of making errors,
positive or negative, of different sizes.

2
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE


MSE Z   E  Z   
2
   Z2    Z   
2
probability density

estimator B

bias
0
4
q mZ

A widely-used loss function is the mean square error of the estimator, defined as the
expected value of the square of the deviation of the estimator about the true value of the
population characteristic.
3
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE


MSE Z   E  Z   
2
   Z2    Z   
2
probability density

estimator B

bias
0
4
q mZ

The mean square error involves a trade-off between the variance of the estimator and its
bias. Suppose you have a biased estimator like estimator B above, with expected value mZ.

4
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE


MSE Z   E  Z   
2
   Z2    Z   
2
probability density

estimator B

bias
0
4
q mZ

The mean square error can be shown to be equal to the sum of the variance of the
estimator and the square of the bias.

5
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared


MSE Z   E  Z     2

 E  Z        
2
Z Z

 E   Z           2 Z        
2 2
Z Z Z Z

 E   Z      E         E  2 Z      
2 2
Z Z Z Z

  Z2    Z     2  Z    E  Z   Z 
2

  Z2    Z     2  Z     Z   Z 
2

  Z2    Z   
2

To demonstrate this, we start by subtracting and adding mZ .

6
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared


MSE Z   E  Z     2

 E  Z        
2
Z Z

 E   Z           2 Z        
2 2
Z Z Z Z

 E   Z      E         E  2 Z      
2 2
Z Z Z Z

  Z2    Z     2  Z    E  Z   Z 
2

  Z2    Z     2  Z     Z   Z 
2

  Z2    Z   
2

We expand the quadratic using the rule (a + b)2 = a2 + b2 + 2ab, where a = Z – mZ and b = mZ – q.

7
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared


MSE Z   E  Z     2

 E  Z        
2
Z Z

 E   Z           2 Z        
2 2
Z Z Z Z

 E   Z      E         E  2 Z      
2 2
Z Z Z Z

  Z2    Z     2  Z    E  Z   Z 
2

  Z2    Z     2  Z     Z   Z 
2

  Z2    Z   
2

We use the first expected value rule to break up the expectation into its three components.

8
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared


MSE Z   E  Z     2

 E  Z        
2
Z Z

 E   Z           2 Z        
2 2
Z Z Z Z

 E   Z      E         E  2 Z      
2 2
Z Z Z Z

  Z2    Z     2  Z    E  Z   Z 
2

  Z2    Z     2  Z     Z   Z 
2

  Z2    Z   
2

The first term in the expression is by definition the variance of Z.

9
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared


MSE Z   E  Z     2

 E  Z        
2
Z Z

 E   Z           2 Z        
2 2
Z Z Z Z

 E   Z      E         E  2 Z      
2 2
Z Z Z Z

  Z2    Z     2  Z    E  Z   Z 
2

  Z2    Z     2  Z     Z   Z 
2

  Z2    Z   
2

(mZ – q) is a constant, so the second term is a constant.

10
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared


MSE Z   E  Z     2

 E  Z        
2
Z Z

 E   Z           2 Z        
2 2
Z Z Z Z

 E   Z      E         E  2 Z      
2 2
Z Z Z Z

  Z2    Z     2  Z    E  Z   Z 
2

  Z2    Z     2  Z     Z   Z 
2

  Z2    Z   
2

In the third term, (mZ – q) may be brought out of the expectation, again because it is a
constant, using the second expected value rule.

11
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared


MSE Z   E  Z     2

 E  Z        
2
Z Z

 E   Z           2 Z        
2 2
Z Z Z Z

 E   Z      E         E  2 Z      
2 2
Z Z Z Z

  Z2    Z     2  Z    E  Z   Z 
2

  Z2    Z     2  Z     Z   Z 
2

  Z2    Z   
2

Now E(Z) is mZ, and E(–mZ) is –mZ.

12
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

Mean square error = variance + bias squared


MSE Z   E  Z    2

 E  Z        
2
Z Z

 E   Z           2 Z        
2 2
Z Z Z Z

 E   Z      E         E  2 Z      
2 2
Z Z Z Z

  Z2    Z     2  Z    E  Z   Z 
2

  Z2    Z     2  Z     Z   Z 
2

  Z2    Z   
2

Hence the third term is zero and the mean square error of Z is shown be the sum of the
variance of Z and the bias squared.

13
 CONFLICTS BETWEEN UNBIASEDNESS AND MINIMUM VARIANCE

probability density

estimator B

estimator A

0 q

In the case of the estimators shown, estimator B is probably a little better than estimator A
according to the MSE criterion.

14
Copyright Christopher Dougherty 2016.

These slideshows may be downloaded by anyone, anywhere for personal use.

Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.

The content of this slideshow comes from Section R.6 of C. Dougherty,

Introduction to Econometrics, fifth edition 2016, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oxfordtextbooks.co.uk/orc/dougherty5e/

Individuals studying econometrics on their own who feel that they might benefit
from participation in a formal course should consider the London School of
Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics
www.londoninternational.ac.uk/lse.

2015.12.17