Welcome to Scribd. Sign in or start your free trial to enjoy unlimited e-books, audiobooks & documents.Find out more
Standard view
Full view
of .
0 of .
Results for:
P. 1
Asymptotic Theory for OLS

# Asymptotic Theory for OLS

Ratings:
(0)
|Views: 6|Likes:
Econometrics - Asymptotic Theory for OLS
Econometrics - Asymptotic Theory for OLS

### Availability:

See more
See less

09/14/2013

pdf

text

original

Colin Cameron: Asymptotic Theory for OLS
1. OLS Estimator Properties and Sampling Schemes
Consider the OLS model with just one regressor
y
i
=
βx
i
+
u
i
.
The OLS estimator
b
β
=
³P
i
=1
x
2
i
´
1
P
i
=1
x
i
y
i
can be written as
b
β
=
β
+
1
P
i
=1
x
i
u
i
1
P
i
=1
x
2
i
.
Then under assumptions given below (including E
[
u
i
|
x
i
] = 0
)
b
β
p
β
+plim
1
P
i
=1
x
i
u
i
plim
1
P
i
=1
x
2
i
=
β
+0plim
1
P
i
=1
x
2
i
=
β.
It follows that
b
β
is
consistent
for
β
.And under assumptions given below (including E
[
u
i
|
x
i
] = 0
and V
[
u
i
|
x
i
] =
σ
2
)
√
(
b
β
β
) =
1
√
P
i
=1
x
i
u
i
1
P
i
=1
x
2
id
N
h
0
,σ
2
³
plim
1
P
i
=1
x
2
i
´i
plim
1
P
i
=1
x
2
id
N
"
0
,σ
2
µ
plim1
X
i
=1
x
2
i
1
#
.
It follows that
b
β
is
asymptotically normal
distributed with
b
β
a
N
0
,σ
2
³P
i
=1
x
2
i
´
1
¸
.
1.2. Sampling Schemes and Error Assumptions
The key for consistency is obtaining the probability of the two averages (of
x
i
u
i
and of
x
2
i
), by use of laws of large numbers (LLN). And for asymptotic normality the key isthe limit distribution of the average of
x
i
u
i
, obtained by a central limit theorem (CLT).Di
ﬀ
erent assumptions about the stochastic properties of
x
i
and
u
i
ﬀ
erentproperties of
x
2
i
and
x
i
u
i
and hence di
ﬀ
erent LLN and CLT.For the data di
ﬀ
erent
sampling schemes
assumptions include:1. Simple Random Sampling (SRS).SRS is when we randomly draw
(
y
i
,x
i
)
from the population. Then
x
i
are iid. So
x
2
i
are iid, and
x
i
u
i
are iid if the errors
u
i
are iid.

2. Fixed regressors.This occurs in an experiment where we
x the
x
i
and observe the resulting random
y
i
. Given
x
i
xed and
u
i
iid it follows that
x
i
u
i
are inid (even if
u
i
are iid), while
x
2
i
are nonstochastic.3. Exogenous Strati
ed SamplingThis occurs when oversample some values of
x
and undersample others. Then
x
i
are inid, so
x
i
u
i
are inid (even if
u
i
are iid) and
x
2
i
are inid.The simplest results assume
u
i
are iid. In practice for cross-section data the errorsmay be inid due to conditional heteroskedasticity, with V
[
u
i
|
x
i
] =
σ
2
i
varying with
i
.
2. Asymptotic Theory for Consistency
Consider the limit behavior of a
sequence of random variables
b
as
. Thisis a stochastic extension of a sequence of real numbers, such as
a
= 2 + (3
/N
)
.Examples include: (1)
b
is an estimator, say
b
θ
; (2)
b
is a component of anestimator, such as
1
P
i
x
i
u
i
; (3)
b
is a test statistic.
2.1. Convergence in Probability, Consistency, Transformations
Due to sampling randomness we can never be certain that a random sequence
b
, suchas an estimator
b
θ
, will be within a given small distance of its limit, even if the sample isin
nitely large. But we can be almost certain. Di
ﬀ
erent ways of expressing this almostcertainty correspond to di
ﬀ
erent types of convergence of a sequence of random variablesto a limit. The one most used in econometrics is convergence in probability.Recall that a sequence of nonstochastic real numbers
{
a
}
converges to
a
if for any
ε >
0
, there exists
=
(
ε
)
such that for all
N > N
,
|
a
a
|
< ε.
e.g. if
a
= 2 + 3
/N,
then the limit
a
= 2
since
|
a
a
|
=
|
2 + 3
/
2
|
=
|
3
/
|
< ε
for all
N > N
= 3
/ε.
For a sequence of r.v.’s we cannot be certain that
|
b
b
|
< ε
, even for large
,due to the randomness. Instead, we require that the probability of being within
ε
isarbitrarily close to one.Thus
{
b
}
converges in probability
to
b
if
lim
→∞
Pr[
|
b
b
|
< ε
] = 1
,
for any
ε >
0
. A formal de
nition is the following.2

De
nition A1:
(
Convergence in Probability
) A sequence of random variables
{
b
}
converges in probability
to
b
if for any
ε >
0
and
δ >
0
, there exists
=
(
ε,δ
)
such that for all
N > N
,Pr
[
|
b
b
|
< ε
]
>
1
δ.
We write
plim
b
=
b
, where
plim
is short-hand for
probability limit
, or
b
p
b
.The limit
b
may be a constant or a random variable. The usual de
nition of convergencefor a sequence of real variables is a special case of A1.For
vector random variables
we can apply the theory for each element of
b
.[Alternatively replace
|
b
b
|
by the scalar
(
b
b
)
0
(
b
b
) = (
b
1
b
1
)
2
+
···
+(
b
KN
b
)
2
or its square root
||
b
b
||
.]Now consider
{
b
}
to be a sequence of parameter estimates
b
θ
.
De
nition A2
: (
Consistency
) An estimator
b
θ
is
consistent
for
θ
0
if
plim
b
θ
=
θ
0
.
Unbiasedness
;
consistency. Unbiasedness states E
[
b
θ
] =
θ
0
. Unbiasedness permitsvariability around
θ
0
that need not disappear as the sample size goes to in
nity.Consistency
;
1
/N
to an unbiased and consistent estimator- now biased but still consistent.A useful property of plim is that it can apply to
transformations of randomvariables
.
Theorem A3
: (
Probability Limit Continuity
). Let
b
be a
nite-dimensional vector of random variables, and
g
(
·
)
be a real-valued function continuous at a
constant vectorpoint
b
. Then
b
p
b
g
(
b
)
p
g
(
b
)
.
This theorem is often referred to as
Slutsky’s Theorem
. We instead call TheoremA12 Slutsky’s theorem.Theorem A3 is one of the major reasons for the prevalence of asymptotic resultsversus
nite sample results in econometrics. It states a very convenient property thatdoes not hold for expectations.For example,
plim(
a
,b
) = (
a,b
)
implies
plim(
a
b
) =
ab
, whereas E
[
a
b
]
generally di
ﬀ
ers from E
[
a
]
E
[
b
]
.Similarly
plim[
a
/b
] =
a/b
provided
b
6
= 0
.3