Endogeneities

IV under Exact Identification

IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Instrumental Variables
Walter Sosa-Escudero
Econ 507. Econometric Analysis. Spring 2009

March 8, 2009

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Motivation

Consistency of OLS crucially depends on E(ui xi ) = 0.

In general, when this assumption does not hold xi will be
endogenous

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Endogeneities: Examples
Simultaneous Equations
The simplest supply and
s
qi
qd
is
qi

demand system:
= xsi 1s + 2s pi + si
= xdi 1d + 2d pi + di
= qid

In equilibrium:
pi = (2s 2d )1 (xdi 1d xsi 1s + di si )
In both, supply and demand, pi as an explanatory variable depends
on the error term. Simple OLS is not consistent
Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Omitted Variables

Y = 1 X1 + 2 X2 + u
if we omit X2
Y = 1 X1 +
1 X2 . Then the predeterminedness assumption will be
violated unless X1 and X2 are orthogonal or 2 = 0.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Explanatory variable measured with error

Ci = 1 + 2 Xi + ui
and assume all assumptions for consistency hold. Now suppose we
observe a noisy version of Xi :
Xi = Xi + i
i is a measurement error. We will assume i is iid, with
E(i ) = 0, V (i ) = 2 and uncorrelated with Xi and ui .
Replacing Xi = Xi i , the regression model can be written as
Ci = 1 + 2 Xi + i
with = 2 i + ui
Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Now
Cov(Xi , i ) = E(Xi , i ) = E [(Xi + i ) (2 i + ui )]
= 2 2 6= 0
Then the OLS estimator that regresses Yi on Xi is inconsistent.
We will derive more details in the next homework.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

IV under Exact Identification

Model as before. zi is a vector of K instrumental variables.

1

Linearity: yi = x0i 0 + ui

Random sample: {xi , zi , ui } is a jointly i.i.d. process.

i = 1, . . . , n.

IV validity 1): rank. E(zi x0i ) = zx is a finite, invertible

matrix

IV validity 2): orthogonality. E(zik ui ) = 0 for all i and

k = 1, . . . , K.

V (zi ui ) = S finite positive definite.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

The Method-of-Moments and the IV estimator.

The orthogonality condition


E(zi ui ) = E zi (yi x0i 0 ) = 0
is a set of K moment conditions for the K unknown parameters.
Method-of-moments: choose as estimator the values of the
parameters that force the sample moments to hold. IV satisfies:
n

1X
zi (yi x0i IV ) = 0
n
i=1

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

This is a system of K linear equations with K unknowns with

solution
!1 n
!
n
X
X
IV =
zi x 0
zi yi = (Z 0 X)1 Z 0 Y
i

i=1

i=1

Existence: asympotically guaranteed by the rank condition.

Exact identification: same number of variables and
instruments (K).
If xi is exogenous, it is a valid vector of instruments for itself.
Then IV = OLS .

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Large sample properties

p
Consistency: IV 0
Proof (sketch): IV = (Z 0 X)1 Z 0 Y . Replacing:

IV

= 0 +
= 0 +

(Z 0 X)1
Z 0u
 0 1  0 
Z X
n

zx <

Z u
n
p

using the rank and ortogonality conditions.

Asymptotic normality:

p
d
01
(IV 0 ) N (0, 01
zx Szx ).

Finish both proofs as a useful homework

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Sources of valid instruments

Instrument Validity
IV validity 1): rank. E(zi x0i ) = zx , finite and invertible.
Intuitively this requires the instruments to be correlated with
the variables to be instrumented. This might be checked
empirically. More later.
IV validity 2): orthogonality. E(zik ui ) = 0. Instruments must
be uncorrelated with whatever is not observed that is a
determinant of yi . This depends on things we do not observe
and on how we setup the model.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Where do valid instruments came from?

Complex question. It is an econometric and a modeling issue.
Instrument for prices in the demand function: price
determinants related to the supply side (costs).
Angrist and Krueger (1991): wages as a function of
education. Education endogenous. Instrument: month of
birth!: individuals born in the first months of the year may
abandon school earlier, then they should have less education
than the rest. Validity?
Growth regression: What is truly exogenous for
growth?(Durlauf, 2001).

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

IV: the Overidentified Case

Suppose now that there are p > K instruments. Then the MM
logic implies that
n

1X
zi (yi x0i IV ) = 0
n
i=1

is a system of p linear equations with K unknowns.

If we only care about consistency there is something obvious
we can do.
Less obvious?
We will explore a consistent and hopefully more efficient
strategy.
Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Model as before. zi is a vector of p > K instruments.

1

Linearity: yi = x0i 0 + ui

Random sample: {xi , zi , ui } is a jointly i.i.d. process.

i = 1, . . . , n.

IV validity 1): rank. E(zi x0i ) = zx is a p K matrix, that

exists, is finite, an has full column rank.

IV validity 2): orthogonality. E(zi ui ) = 0 for all i.

V (zi ui ) = S finite positive definite.

Weighting matrix: There exists a sequence of possibly random

p
p p symmetric pd matrices Wn , such that Wn W , W is
also symmetric pd.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Estimation strategy: GMM

As mentioned before, we cannot force the sample moments
n

1X
=0
zi (yi x0i )
n
i=1

to hold. Let us adopt the following notation:

gi () = zi ui
gn (b) = 1/n

Pn

i=1 zi (yi

x0i b)

So the sample moment condition is gn = 0.

GMM approach: choose b so gn (b) is as small as possible.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Formally, let J(b) n gn (b)0 Wn gn (b). The GMM estimator is

defined as follows:
g argmin J(b)
b

g minimizes a weighted sum of the elements of the sample

moment condition.
Ill invite you to check that:
g = X 0 ZWn Z 0 X

Walter Sosa-Escudero

1

X 0 ZWn Z 0 Y

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Comments:


The existence of T requires X 0 ZWn Z 0 X invertible. Check
that our assumptions guarantee this asymptotically.(Careful
what you do with n!).
When p = K this reduces to g = IV = (Z 0 X)1 Z 0 Y . WT
plays no role.
What is the value of J(g ) in this latter case?. This will be
crucial later on.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

The plan
1

For any general choice of Wn we will show consistency, AN

and derive a generic estimate for its AVAR.

How to choose Wn optimally.

Many things simplify if we can further assume conditional

homoskedasticity: discuss particular simplifications under this
case.

Revise the classical approaches (2SLS, etc.) from the GMM

perspective. Discuss small sample performance.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Asymptotic Properties of g
Prelude: Identification
Before showing consistency we need to guarantee that the
estimation problem is well defined
In our case, it means that the GMM problem has a unique
solution in the population
Consider the moment equations


E(zi ui ) = E zi (yi x0i b) = 0
which are satistied with b = 0 .
Identification: 0 is identified if it is the only solution.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Write the moment conditions as:

E(zi yi ) E(zi xi )0 = 0
xz 0 = xy
Result (Searle (1982), Simon and Blume (1994, pp. 148): if there
exists a solution to Ax = b, a necessary and sufficient solution that
it is the only solution is that A is of full column rank
Consequently, the rank condition guarantees identification.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

p
Consistency: g 0

Start with:
g = X 0 ZWn Z 0 X

1

X 0 ZWn Z 0 Y

and replace Y = X0 + u to get:

g = 0 + X 0 Z Wn Z 0 X

1

X 0 Z Wn Z 0 u

divide and multiply by n to get:

g = 0 +

X 0Z
Z 0X
Wn
n
n

Walter Sosa-Escudero

1

X 0Z
Z 0u
Wn
n
n

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

There are 6 terms. We know that Wn W . Note that by the

LLN element by element:
n

1 0
1X
p
ZX=
zi x0i E[zi xi ] = zx
n
n
i=1

1 0
1X
p
Zu=
zi ui E[zi ui ]
n
n
i=1

Note that E[xi zi0 ]W E[zi x0i ] is an invertible K K matrix.

1
Call H E[xi zi0 ]W E[zi x0i ] E[xi zi0 ] W . Then,
p
g 0 + H E[zi ui ] = 0

by the moment condition and continuity.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Asymptotic normality: Substract 0 and multiply by

n (g 0 ) =

X 0Z
Z 0X
Wn
n
n
|
{z


1

n to get:

X 0Z
Z 0u
Wn
n
n
}

Since zi ui is an iid process with zero mean (by the population

condition) and finite variance, by the CLT:
n

Z 0u 1 X
d
= n
zi ui N (0, S)
n
n
i=1

Then, by Slutzkys theorem:

d
n ( 0 ) N (0, HSH 0 ).

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Variance estimator: The asymptotic variance of g is


 
0
AV(g ) = HSH 0 = (0xz W xz )1 0xz W S (0xz W xz )1 0xz W
To establish consistency of variance estimation we need to bound
the fourth moments:
Assumption: E[(xik zjl )2 ] exist and are finite, for all k = 1, . . . , K,
l = 1, . . . , p.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS


 
0
AV(g ) = HSH 0 = (0xz W xz )1 0xz W S (0xz W xz )1 0xz W

A consistent estimator can be obtained if we replace

xz 1 Pn zi x0 ,
xz by
i
i=1
n
W by Wn , and
P

S = E(u2i zi zi0 ) by S = n1 ni=1 e2i zi zi0 , with ei yi x0i ,

where is any consistent estimator of 0 .
Then our consistent estimator will be:
h
i h
i0
d (g ) = (
0 Wn
xz )1
0 Wn S (
0 Wn
xz )1
0 Wn
AV
xz
xz
xz
xz

See the close relationship with the White estimator we have seen before.
Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Efficient GMM
Our GMM estimator is consistent and AN for any choice of
Wn and its limit W .
Recall that the asymptotic variance of the GMM estimator is
AV(g ) = HSH 0
The optimal weighting matrix, W 0 is the value that minimizes
AV(g ).
Result: W 0 S 1 .
In such case, replacing, W = S 1 in AV(g ) leads to:
AVo (g ) = 0xz S 1 xz

1

which provides a lower bound for the AVAR of the generic GMM
estimator.
Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Our generic GMM estimator was

g = X 0 ZWn Z 0 X

1

X 0 ZWn Z 0 Y

Then the optimal GMM estimator can be constructed by replacing

Wn by any
consistent estimate of S 1 , up to scale, say, the inverse
P
of S = n1 ni=1 e2i zi zi0 . Then:
g0 = X 0 Z S1 Z 0 X

1

X 0 Z S1 Z 0 Y

This suggests a two step procedure:

1

Choose any matrix Wn in order to produce a consistent

P
estimate of 0 . With this we can obtain S = n1 ni=1 e2i zi zi .
Compute the efficient GMM estimator replacing Wn by S1
obtained before.
Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Proof of efficient GMM: We need to show AV(g ) AVo (g ) is a

positive semidefinite matrix. Remember that it is equivalent to
show
AVo1 (g ) AV1 (g )
is positive semidefinite. Now

 
0
AV(g ) = (0xz W xz )1 0xz W S (0xz W xz )1 0xz W

1
AVo (g ) = 0 S 1 xz
xz

So we need to show
0xz S 1 xz 0xz W xz 0xz W SW xz

1

is psd.

Walter Sosa-Escudero

Instrumental Variables

0xz W xz

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

0xz S 1 xz 0xz W xz 0xz W SW xz

1

0xz W xz

i
h

1 0
xz W xz
0xz S 1 W xz 0xz W SW xz

i
h

1 0
0xz P 0 P W xz 0xz W (P 0 P )1 W xz
xz W xz
where P 0 P = S 1 , P invertible.
i
h

1 0
xz W P 1 P xz
0xz P 0 I P 01 W xz 0xz W P 1 P 01 W xz


0xz P 0 I B(B 0 B)1 B 0 P xz
for B P 01 W xz .
Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

We can write the previous result as:

0xz P 0 MB P xz
with MB I B(B 0 B)1 B 0 . Now, for any vector c
c0 0xz P 0 MB P xz c = c0 0xz P 0 MB MB0 P xz c
= q0q
X
=
qj2 0

(q MB0 P xz c)

since q is a vector. 

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Simplifications under Conditional Homoskedasticity

Suppose we can further assume conditional homoskedasticity:
Assumption: E(ui |zi ) = 2
First, it is easy to verify the following
Result: Let ei = yi x0i where is any consistent estimator of
P
p
0 . Then
n1 ni=1 e2i 2 .
= ui x0 (
0 ). Then take squares, sum, distribute and use the
Proof: Start with ei = yi x0i
i
assumptions.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Now note that S = V (zi ui ) = E(u2i zi zi0 ) simplifies (using LIE) to:
S = E(u2i zi zi0 ) = 2 E(zi zi0 ) = 2 z
which can be consistently estimated by
1
S =
2
n

n
X
i=1

1
zi zi0 =
2 Z 0Z
n

Now replace in the formula for our optimal GMM estimator

1 0 1 0
g0 = X 0 Z S1 Z 0 X
X Z S Z Y

1 0
= X 0 Z(Z 0 Z)1 Z 0 X
X Z(Z 0 Z)1 Z 0 Y T SLS

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

T SLS = X 0 Z(Z 0 Z)1 Z 0 X

least squares estimator.

1

X 0 Z(Z 0 Z)1 Z 0 Y is the Two-stage

Its asymptotic variance is

AV (T SLS ) = 2 0xz 1
zz xz

1

that can be estimated consistently by:

d (T SLS ) =
0
1
AV
2
xz zz xz

1

where we can use T SLS as the consistent estimate for 0 needed

to produce
2.

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

Remarks on TSLS
The TSLS is a very particular case of a GMM estimator for a
particular choice of W 0 that arises under conditional
homoskedasticity.
You have to be very careful with the efficiency assessments
that you want to make about TSLS. From the GMM
perspective it is not more efficient that other choices of W 0
that satisfies the optimality criterion. Why?, Can you quickly
produce another one?.
Careful with the steps. From the GMM perspective there is
only one step since we do not need to produce an estimate of
2 to implement the estimator (it cancelled out!!!).

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

TSLS as an IV estimator
In the standard literature it is called a two-stage estimator
because Pz Z(Z 0 Z)1 Z 0 is idempotent (Pz0 Pz = Pz ) and
symmetric, so:

1 0
T SLS = X 0 Z(Z 0 Z)1 Z 0 X
X Z(Z 0 Z)1 Z 0 Y
= (X 0 Pz X)1 X 0 Pz Y
= (X 0 Pz0 Pz X)1 X 0 Pz0 Pz Y
0

= (X X )1 X Y
with X Pz X and Y Pz Y .

Walter Sosa-Escudero

Instrumental Variables

Endogeneities
IV under Exact Identification
IV: the Overidentified Case
Efficient GMM
Conditional Homoskedasticity and 2SLS

TSLS is actually and OLS estimator where in a first step the

variables X and Y are constructed, and in a second step,
the OLS estimator is produced with these transformed
variables.
The first stage removes the endogeneity problem by replacing
X by its linear projection on the space spanned by the
instruments Z, which are by construction orthogonal to the
error term.

Walter Sosa-Escudero

Instrumental Variables