Introduction To Econometrics, 5 Edition: Review: Random Variables, Sampling, Estimation, and Inference

Type author name/s here
Dougherty
Introduction to Econometrics,
5th edition
Chapter heading
Review: Random Variables,
Sampling, Estimation, and
Inference
© Christopher Dougherty, 2016. All rights reserved.

ASYMPTOTIC PROPERTIES OF ESTIMATORS: PLIMS AND CONSISTENCY
Asymptotic properties of estimators:
Properties as n  
• probability limits
• consistency
• central limit theorem
The asymptotic properties of estimators are their properties as the number of observations
in a sample becomes very large and tends to infinity. We shall be concerned with the
concepts of probability limits and consistency, and the central limit theorem.
1
• consistency
These topics are usually given little attention in standard statistics texts, generally without
an explanation of why they are relevant and useful. However, asymptotic properties lie at
the heart of much econometric analysis. For students of econometrics they are important.
2
• consistency
This sequence is addressed to probability limits and consistency. A subsequent one will
treat the central limit theorem.
3
Probability limits
lim P  Z n       0
n
We will start with an abstract definition of a probability limit and then illustrate it with a simple
example.
4
Probability limits
lim P  Z n       0
n
A sequence of random variables Zn is said to converge in probability to a constant a if,

given any positive e, however small, the probability of Zn deviating from a by an amount
greater than e tends to zero as n tends to infinity.
5
Probability limits
lim P  Z n       0
n
plim Z n  
The constant a is described as the probability limit of the sequence, usually abbreviated as
plim.
6
0.10
n X
0.08 1 50
probability density function
0.06
0.04
0.02
n=1
0.00
0 50 100 150 X
We will take as our example the mean of a sample of observations, X, generated from a
random variable X with population mean mX and variance s2X. We will investigate how X
behaves as the sample size n becomes large.
7
0.10
n X
0.08 1 50
0.06
0.04
0.02
n=1
0.00
0 50 100 150 X
For convenience we shall assume that X has a normal distribution, but this does not affect
the analysis. If X has a normal distribution with mean mX and variance s2X, X will have a
normal distribution with mean mX and variance s2X / n.
8
0.10
n X
0.08 1 50
0.06
0.04
0.02
n=1
0.00
0 50 100 150 X
For the purposes of this example, we will suppose that X has population mean 100 and
standard deviation 50, as in the diagram.
9
0.10
n X
0.08 1 50
0.06
0.04
0.02
n=1
0.00
0 50 100 150 X
The sample mean will have the same population mean as X, but its standard deviation will
be 50/ n , where n is the number of observations in the sample.
10
0.10
n X
0.08 1 50
0.06
0.04
0.02
n=1
0.00
0 50 100 150 X
The larger is the sample, the smaller will be the standard deviation of the sample mean.
11
0.10
n X
0.08 1 50
0.06
0.04
0.02
n=1
0.00
0 50 100 150 X
If n is equal to 1, the sample consists of a single observation. X is the same as X and its
standard deviation is 50.
12
0.10
n X
0.08 1 50
4 25
0.06
0.04
0.02 n=4
n=1
0.00
0 50 100 150 X
We will see how the shape of the distribution changes as the sample size is increased. We
have added the distribution of X when n = 4.
13
0.10
n X
0.08 1 50
4 25
25 10
0.06
0.04
n = 25
0.02 n=4
n=1
0.00
0 50 100 150 X
We add the distribution for n = 25. The distribution becomes more concentrated about the
population mean.
14
0.10
n X
0.08 1 50
n = 100 4 25
25 10
0.06 100 5
0.04
n = 25
0.02 n=4
n=1
0.00
0 50 100 150 X
We add the distribution for n = 100. To see what happens for n greater than 100, we will
have to change the vertical scale.
15
0.3 n X
1 50
4 25
25 10
0.2 100 5
0.1
n = 100
0.0
0 50 100 150 X
We have increased the vertical scale.
16
0.3 n X
1 50
4 25
25 10
0.2 100 5
400 2.5
n = 400
0.1
n = 100
0.0
0 50 100 150 X
We increase the sample size to 400. The distribution continues to contract about the
population mean.
17
0.3 n X
1 50
n = 1600 4 25
25 10
0.2 100 5
400 2.5
1600 1.3
n = 400
0.1
n = 100
0.0
0 50 100 150 X
We increase the sample size again. In the limit, the variance of the distribution tends to
zero. The distribution collapses to a spike at the true value. The plim of the sample mean is
therefore the population mean.
18
Probability limits
lim P  Z n       0
n
plim Z n  
Sample mean as estimator of population mean
lim P  X   X     0
n
Formally, the probability of X differing from mX by any finite amount, however small, tends to
zero as n becomes large.
19
Probability limits
lim P  Z n       0
n
plim Z n  
Sample mean as estimator of population mean
lim P  X   X     0
n
plim X   X
Hence we can say plim X = mX.
20
Consistency
Two conditions:
(1) The estimator possesses a probability limit.
An estimator of a population characteristic is said to be consistent if it satisfies two

conditions. The first is that the estimator possesses a probability limit, and so its
distribution collapses to a spike as the sample size becomes large.
21
Consistency
Two conditions:
(2) The limit is the true value of the population characteristic.
The second is that the spike is located at the true value of the population
characteristic.
22
0.3
n = 1600
0.2
0.1
0.0
0 50 100 150 X
The sample mean in our example satisfies both conditions and so it is a consistent
estimator of mX.
23
0.3
n = 1600
0.2
0.1
0.0
0 50 100 150 X
Most standard estimators in simple applications satisfy the first condition because their
variances tend to zero as the sample size becomes large. The only issue then is whether
the distribution collapses to a spike at the true value of the population characteristic.
24
0.3
n = 1600
0.2
0.1
0.0
0 50 100 150 X
A sufficient condition for consistency is that the estimator should be unbiased and that its
variance should tend to zero as n becomes large.
25
0.3
n = 1600
0.2
0.1
0.0
0 50 100 150 X
It is easy to see why this is a sufficient condition. If the estimator is unbiased for a finite
sample, it must stay unbiased as the sample size becomes large.
26
0.3
n = 1600
0.2
0.1
0.0
0 50 100 150 X
Meanwhile, if the variance of its distribution is decreasing, its distribution must collapse to
a spike. Since the estimator remains unbiased, this spike must be located at the true value.
The sample mean is an example of an estimator that satisfies this sufficient condition.
27
0.3
0.2
0.1
n = 25
0.0
80 100 120 140
However the condition is only sufficient, not necessary. It is possible for a biased estimator
to be consistent, if the bias vanishes as the sample size becomes large. In this example,
the true value is 100, and the estimator is biased for sample size 25.
28
0.3
0.2
0.1
n = 100
n = 25
0.0
80 100 120 140
Here the sample size is greatr and the bias is smaller.
29
0.3
0.2
n = 400
0.1
n = 100
n = 25
0.0
80 100 120 140
A further reduction in the bias with a further increase in the sample size.
30
0.3
n = 1600
0.2
n = 400
0.1
n = 100
n = 25
0.0
80 100 120 140
This is an example where the bias disappears altogether as the sample size tends to
infinity. Such an estimator is biased for finite samples but nevertheless consistent because
its distribution collapses to a spike at the true value.
31
0.3
n = 1600
0.2
n = 400
0.1
n = 100
n = 25
0.0
80 100 120 140
We will encounter estimators of this type when we come to Model B, and they will be
important to us.
32
Consistency
Two conditions:
Example
1 n
Z 
n  1 i 1
Xi
The foregoing example was just a general graphical illustration of what might happen as the
sample size increases. Here is a simple mathematical example.
33
Consistency
Two conditions:
Example
1 n
Z 
n  1 i 1
Xi
We are supposing that X is a random variable with unknown population mean mX and that
we wish to estimate mX.
34
Consistency
Two conditions:
Example
1 n
Z 
n  1 i 1
Xi
n
E Z    X   X as n  
n1
As defined, the estimator Z is biased for finite samples because its expected value is
nmX/(n + 1). But as n tends to infinity, n /(n + 1) tends to 1 and the bias disappears.
35
Consistency
Two conditions:
Example
1 n
Z 
n  1 i 1
Xi
n
E Z    X   X as n  
n1
n
var( Z )   X  0 as n  
2
 n  1 2
The variance of the estimator is given by the expression shown. This tends to zero as n
tends to infinity. Thus Z is consistent because its distribution collapses to a spike at the
true value.
36
Consistency
Two conditions:
Why should we be interested in consistency?
In practice we deal with finite samples, not infinite ones. So why should we be interested in
whether an estimator is consistent?
37
Consistency
Two conditions:
• If no unbiased estimator exists, a consistent estimator may be

preferred to an inconsistent one.
One reason is that often, in practice, it is impossible to find an estimator that is unbiased for small
samples. If you can find one that is at least consistent, that may be better than having no estimate
at all.
38
Consistency
Two conditions:

• It is often not possible to determine the expectation of an
estimator, but still possible to evaluate probability limits.
Reason: plim rules are stronger than the rules for handling
expectations.
A second reason is that often we are unable to say anything at all about the expectation of an
estimator. The expected value rules are weak analytical instruments that can be applied in
relatively simple contexts.
39
Consistency
Two conditions:

• It is often not possible to determine the expectation of an
estimator, but still possible to evaluate probability limits.
Reason: plim rules are stronger than the rules for handling
expectations.
In particular, the multiplicative rule E{g(X)h(Y)} = E{g(X)} E{h(Y)} applies only when X and Y are
independent, and in most situations of interest this will not be the case. By contrast, we have a
much more powerful set of rules for plims.
40
plim rules
1. plim  X  Y  Z   plim X  plim Y  plim Z
2. plim bX  b plim X
3. plim b  b
4. plim XY  plim X plim Y
X plim X
5. plim  provided plim Y  0
Y plim Y
6. plim f  X   f  plim X 
Here are six rules for decomposing plims. Note that, in each case, the validity of the rule depends
on plims existing for each of the components on the right side of the equation.
41
plim rules
3. plim b  b
The first three rules are straightforward counterparts for corresponding rules for
decomposing expectations. The first rule, the additive rule, depends on X, Y, and Z each
having their individual plims.
42
plim rules
3. plim b  b
The second rule, a multiplicative rule, b being a constant, depends on X having a plim. The
third rule states the obvious fact that a constant is its own limit.
43
plim rules
3. plim b  b
The fourth rule, a multiplicative rule for two (or more) variables, depends on each variable
having a plim. It does not require X and Y to be independent.
44
plim rules
3. plim b  b
By contrast, the corresponding rule for expectations E{g(X)h(Y)} = E{g(X)} E{h(Y)} applies
only when X and Y are independent. This is often not the case.
45
plim rules
3. plim b  b
The quotient rule for plims, which we shall use very frequently in Model B, requires that X
and Y both have plims and that plim Y is not zero.
46
plim rules
3. plim b  b
X plim X
Y plim Y
There is no counterpart for expectations, even if X and Y are independent. If X and Y are
independent, E(X/Y) = E(X) E(1/Y), provided that both expectations exist, and that is as far
as one can go.
47
plim rules
3. plim b  b
X plim X
Y plim Y
6. plim f  X   f  plim X 
The plim of a function of a variable is equal to the function of the plim of the variable,
provided that the variable possesses a plim and provided that the function is continuous at
that point.
48
Example use of asymptotic analysis
Model: Y  Z
To illustrate how the plim rules can lead us to conclusions when the expected value rules
do not, consider this example. Suppose that you know that a variable Y is a constant
multiple of another variable Z
49
Model: Y  Z
Z is generated randomly from a fixed distribution with population mean mZ and variance s2Z.
a is unknown and we wish to estimate it. We have a sample of n observations.
50
Model: Y  Z
X  Zw
Y is measured accurately but Z is measured with random error w with population mean zero
and constant variance s2w. Thus in the sample we have observations on X, where X = Z + w,
rather than Z.
51
Model: Y  Z Y
Estimator of a :
X  Zw X
One estimator of a (not necessarily the best) is Y/X.
52
Model: Y  Z Y
Estimator of a :
X  Zw X
Y Z w
  
X Zw Zw
We can decompose the estimator into the true value and an error term, as shown.
53
Model: Y  Z Y
Estimator of a :
X  Zw X
Y Z w
  
X Zw Zw
We can decompose the estimator into the true value and an error term, as shown.
54
Model: Y  Z Y
Estimator of a :
X  Zw X
Y Z w
  
X Zw Zw
To investigate whether the estimator is biased or unbiased, we need to take the expectation
of the error term. But we cannot do this.
55
Model: Y  Z Y
Estimator of a :
X  Zw X
Y Z w
  
X Zw Zw
The random variable w appears in both the numerator and the denominator and the
expected value rules are too weak to allow us to determine the expectation of a ratio when
both the numerator and the denominator are functions of the same random variable.
56
Model: Y  Z Y
Estimator of a :
X  Zw X
Y Z w
  
X Zw Zw
plim Z   Z  plim Z   Z
plim w  0  plim  Z  w    Z
However, we can show that the error term tends to zero as the sample becomes large. We
know that a sample mean tends to a population mean as the sample size tends to infinity,
and so plim w = 0 and plim Z = mZ.
57
Model: Y  Z Y
Estimator of a :
X  Zw X
Y Z w
  
X Zw Zw
plim w  0  plim  Z  w    Z
Hence plims exist for both the numerator and denominator of the estimator.
58
Model: Y  Z Y
Estimator of a :
X  Zw X
Y Z w
  
X Zw Zw
plim w  0  plim  Z  w    Z
Y  Z
plim  
X Z
Since the plims of the numerator and the denominator of the error term both exist, we are
able to take the plim of the estimator.
59
Model: Y  Z Y
Estimator of a :
X  Zw X
Y Z w
  
X Zw Zw
plim w  0  plim  Z  w    Z
Y  Z
plim    assumption:  Z  0
X Z
Thus, provided that mZ ≠ 0, we are able to show that the estimator is consistent, despite the
fact that we cannot say anything analytically about its finite sample properties.
60
Copyright Christopher Dougherty 2016.
These slideshows may be downloaded by anyone, anywhere for personal use.

Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.
The content of this slideshow comes from Section R.14 of C. Dougherty,

Introduction to Econometrics, fifth edition 2016, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oxfordtextbooks.co.uk/orc/dougherty5e/
Individuals studying econometrics on their own who feel that they might benefit
from participation in a formal course should consider the London School of
Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics
www.londoninternational.ac.uk/lse.
2015.12.17

Introduction To Econometrics, 5 Edition: Review: Random Variables, Sampling, Estimation, and Inference

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To Econometrics, 5 Edition: Review: Random Variables, Sampling, Estimation, and Inference

Uploaded by

Copyright:

Available Formats

Type author name/s here

© Christopher Dougherty, 2016. All rights reserved.

Asymptotic properties of estimators:

Asymptotic properties of estimators:

Asymptotic properties of estimators:

A sequence of random variables Zn is said to converge in probability to a constant a if,

We have increased the vertical scale.

Sample mean as estimator of population mean

Sample mean as estimator of population mean

Hence we can say plim X = mX.

An estimator of a population characteristic is said to be consistent if it satisfies two

Here the sample size is greatr and the bias is smaller.

Why should we be interested in consistency?

Why should we be interested in consistency?

• If no unbiased estimator exists, a consistent estimator may be

Why should we be interested in consistency?

• If no unbiased estimator exists, a consistent estimator may be

Why should we be interested in consistency?

• If no unbiased estimator exists, a consistent estimator may be

1. plim  X  Y  Z   plim X  plim Y  plim Z

4. plim XY  plim X plim Y

1. plim  X  Y  Z   plim X  plim Y  plim Z

1. plim  X  Y  Z   plim X  plim Y  plim Z

1. plim  X  Y  Z   plim X  plim Y  plim Z

1. plim  X  Y  Z   plim X  plim Y  plim Z

4. plim XY  plim X plim Y

1. plim  X  Y  Z   plim X  plim Y  plim Z

4. plim XY  plim X plim Y

1. plim  X  Y  Z   plim X  plim Y  plim Z

4. plim XY  plim X plim Y

1. plim  X  Y  Z   plim X  plim Y  plim Z

4. plim XY  plim X plim Y

Example use of asymptotic analysis

Example use of asymptotic analysis

Example use of asymptotic analysis

Example use of asymptotic analysis

One estimator of a (not necessarily the best) is Y/X.

Example use of asymptotic analysis

Example use of asymptotic analysis

Example use of asymptotic analysis

Example use of asymptotic analysis

Example use of asymptotic analysis

Example use of asymptotic analysis

Example use of asymptotic analysis

Example use of asymptotic analysis

These slideshows may be downloaded by anyone, anywhere for personal use.

The content of this slideshow comes from Section R.14 of C. Dougherty,

You might also like