Redundant and omitted variables. Trade o¤s.

Assume that the variable y is related to X1 and X2 by a linear relation:

DGP : y = X1 1 + X2 2 + ; (1)

where 1 = 0:5, 2 = 0:9; and the assumptions of the classical regression model
hold.

1. Economist A estimates the following model (model A):

A : y = X1 1 + X2 2 +u

Show that the OLS estimators ^ 1 and ^ 2 are unbiased estimators of 1

and 2 .
2. Economist B believes that 2 = 0, and therefore he estimates the following
restricted model (model B):

Estimated mod el : y = X1 1 +v

Estimated mod el : y = X2 2 +v
recall that the true model is:

DGP : y = X1 1 + X2 2 + ; (2)

Show that the OLS estimator ^1 is a biased estimator of 1, unless X1

and X2 are orthogonal. Brie‡y explain.
3. Now assume that 2 = 0, which means we can re-write equation (1) as
follows:
DGP : y = X1 1 + X2 ( 2 = 0) + (3)
Show that under the data generating process in equation (7) the estima-
tors used by Economist A (^ 1 ) and Economist B (^1 ) are both unbiased
estimators of 1 .

4. Still assuming that 2 =0

DGP : y = X1 1 + (4)

show that the estimator used by Economist B (^1 ) is more e¢ cient than
that used by Economist A (^ 1 ). Brie‡y explain why this is the case.

1
a) Economist A estimates the following model (model A):

A : y = X1 1 + X2 2 +u

Show that the OLS estimators ^ 1 and ^ 2 are unbiased estimators of 1

and 2 .

X = [X1 X2 ]
1 1
= ; =
2 2

DGP : y = X + = X1 1 + X2 2 + (5)
Estimated mod el A : y = X + u
the OLS estimator:

^ = (X 0 X) 1
X 0Y
E[^ ] = E[(X 0 X) 1
X 0Y ]
= E[(X 0 X) 1
X 0 (X + )]
0 1
= E[(X X) X 0 X + (X 0 X) 1
X0 ]
= E[ + (X 0 X) 1
X0 ]
= + (X 0 X) 1
X 0 E[ ]
= + (X 0 X) 1 X 0 0 =
^1 1
E[^ ] = E =
^2 2
E[^ 1 ] = 1
E[^ 2 ] = 2

b) Economist B believes that 2 = 0, and therefore he estimates the following

restricted model (model B):

Estimated mod el : y = X1 1 +v

Estimated mod el : y = X2 2 +v
recall that the true model is:

DGP : y = X1 1 + X2 2 + ; (6)

Show that the OLS estimator ^1 is a biased estimator of 1, unless X1

and X2 are orthogonal. Brie‡y explain.

2
 ^1 = (X10 X1 ) 1
X10 y
E[^1 ] = E[(X10 X1 ) 1 0
X1 y]
= E[(X1 X1 ) 1 X10 fX1 1 + X2 2 + g]
0

= E[(X10 X1 ) 1 X10 X1 1 + (X10 X1 ) 1 X10 X2 2 + (X10 X1 ) 1

X10 ]
= E[ 1 + (X10 X1 ) 1 X10 X2 2 + (X10 X1 ) 1 X10 ]
0 1 0
= 1 + (X1 X1 ) X1 X2 2 + (X10 X1 ) 1 X10 E[ ]
0 1 0
= 1 + (X1 X1 ) X1 X2 2 :
X10 X2 = 0
E[^1 ] = 1

E[^2 ] = 0
2 + (X2 X2 )
1
X20 X1 1:
X10 X2 =0

c) Now assume that 2 = 0, which means we can re-write equation (1) as

follows:
DGP : y = X1 1 + X2 ( 2 = 0) + (7)
Show that under the data generating process in equation (7) the estima-
tors used by Economist A (^ 1 ) and Economist B (^1 ) are both unbiased
estimators of 1 .
Start with Economist B:
B : y = X1 1 +v

^1 = (X10 X1 ) 1
X10 y
E[^1 ] = E[(X10 X1 ) 1
X10 y]
= E[(X10 X1 ) 1
X10 (X1 1 + )]
= E[ 1 + (X10 X1 ) 1 X10 ]
= 1

What happens with economist A?

A : y = X1 1 + X2 2 +u

^ = (X 0 X) 1
X 0Y
1
^1 X10 X10
= X1 X2 y
^2 X20 X20
1
^1 X10 X1 X10 X2 X10 y
=
^2 X20 X1 X20 X2 X20 y
X10 X1 X10 X2 ^1 X10 y
=
X20 X1 X20 X2 ^2 X20 y

3
 X 0X ^ = X 0y
X10 X1 X10 X2 ^1 X10 y
=
X20 X1 X20 X2 ^2 X20 y

X10 X1 ^ 1 + X10 X2 ^ 2 = X10 y =) X10 X1 ^ 1 = X10 y X10 X2 ^ 2

X20 X1 ^ 1 + X20 X2 ^ 2 = X20 y =) X20 X2 ^ 2 = X20 y X20 X1 ^ 1

Solve for ^ 1 and ^ 2 .

^1 = (X10 X1 ) 1
(X10 y X10 X2 ^ 2 )
^2 = (X20 X2 ) 1
(X20 y X20 X1 ^ 1 )

consider

X2 ^ 2 = X2 (X20 X2 ) 1
(X20 y X20 X1 ^ 1 )
| {z }
= X2 (X20 X2 ) 1
X20 (y X1 ^ 1 )
= P2 (y X1 ^ 1 )

^1 = (X10 X1 ) 1
(X10 y X10 X2 ^ 2 )
(X10 X1 )^ 1 = (X10 y X10 X2 ^ 2 )
| {z }
(X10 X1 )^ 1 = (X10 y X10 P2 (y
X1 ^ 1 ))
= 0 0
X1 y X1 P2 y + X10 P2 X1 ^ 1
X10 X1 ^ 1 X10 P2 X1 ^ 1 = X10 y X10 P2 y
(X10 X1 X10 P2 X1 )^ 1 = (X10 X10 P2 )y
X10 (I P2 )X1 ^ 1 = X10 (I P2 )y
X10 M2 X1 ^ 1 = X10 M2 y
^1 = (X10 M2 X1 ) 1 X10 M2 y

hence:
^ 1 = (X10 M2 X1 ) 1
X10 M2 y
and

^2 = (X20 M1 X2 ) 1
X20 M1 y
= (^0 ^) 1 0
^y

4
 X2 = X1 +

^ = (X 0 X1 ) 1
X1 X2
1
^ 2 = X1 ^
X
= X1 (X10 X1 ) 1
X1 X2
= P1 X2
^ = X2 ^2
X
= X2 P1 X2
= (I P1 )X2
= M1 X2

1. hence:
^1 (X10 M2 X1 ) 1
X10 M2 y
^= = = (X 0 X) 1
X 0y
^2 (X20 M1 X2 ) 1
X20 M1 y

E[^ 1 ] = E[(X10 M2 X1 ) 1
X10 M2 y]
= (X10 M2 X1 ) 1
X10 M2 E[y]
= (X10 M2 X1 ) 1
X10 M2 E[X1 1 + ]
= (X10 M2 X1 ) 1 X10 M2 X1 1 + (X10 M2 X1 ) 1
X10 M2 E[ ]
= (X10 M2 X1 ) 1 X10 M2 X1 1
= 1

In case the DGP was instead X1 1 + X2 2 + we would have

E[^ 1 ] = E[(X10 M2 X1 ) 1
X10 M2 y]
= (X10 M2 X1 ) 1 0
X1 M2 E[X1 1 + X2 2 + ]
= 1+ (X1 M2 X1 ) 1 X10 M2 X2 2
0

= 1

E[^ 2 ] = E[(X20 M1 X2 ) 1
X20 M1 y]
= (X20 M1 X2 ) 1
X20 M1 E[y]
= (X20 M1 X2 )1 0
X2 M1 E[X1 1 + ]
= (X2 M1 X2 ) 1 X20 M1 X1 1 + 0
0
| {z }
= 0= 2

5
 if we were in part (a) of the excercise:
E[^ 2 ] = E[(X20 M1 X2 ) 1 0
X2 M1 y]
0 1 0
= (X2 M1 X2 ) X2 M1 E[y]
= (X20 M1 X2 ) 1 X20 M1 E[X1 1 + X2 2 + ]
= (X20 M1 X2 ) 1 X20 M1 X1 1 + (X20 M1 X2 ) 1 X20 M1 X2 2 +0
| {z }
= 2

Hence, in this case both economists have unbiased estimators.

d) Still assuming that 2 =0
DGP : y = X1 1 + (8)

show that the estimator used by Economist B (^1 ) is more e¢ cient than
that used by Economist A (^ 1 ). Brie‡y explain why this is the case.

V ar(^ 1 ) = V ar((X10 M2 X1 ) 1 0
X1 M2 y)
= (X1 M2 X1 ) X1 M2 V ar(y)M2 X1 (X10 M2 X1 ) 1
0 1 0

= (X10 M2 X1 ) 1 X10 M2 V ar(X1 1 + )M2 X1 (X10 M2 X1 ) 1

= (X10 M2 X1 ) 1 X10 M2 V ar( )M2 X1 (X10 M2 X1 ) 1
2
= (X10 M2 X1 ) 1 X10 M2 M2 X1 (X10 M2 X1 ) 1
2
= (X10 M2 X1 ) 1 X10 M2 X1 (X10 M2 X1 ) 1
2
= (X10 M2 X1 ) 1

V ar(^1 ) = V ar((X10 X1 ) 1
X10 y)
= (X10 X1 ) 1
X10 V ar(y)X1 (X10 X1 ) 1

= (X10 X1 ) 1 0 2
X1 IX1 (X10 X1 ) 1
2
= 0
(X1 X1 ) 1

V ar(^ 1 ) V ar(^1 ) = 2
(X10 M2 X1 ) 1 2
(X10 X1 ) 1

V ar(^ 1 ) V ar(^1 )
2
= (X10 M2 X1 ) 1
(X10 X1 ) 1

it is possible to show that (X10 M2 X1 ) 1 (X10 X1 ) 1 is positive de…nite.

This means that V ar(^ 1 ) > V ar(^1 ), that is the estimator of Economist
B is more e¢ cient (we know that it is the BLUE estimator).

1 1
a > b () <
a b
1 1
a b > 0 () <0
a b

6
 (X10 M2 X1 ) 1
(X10 X1 ) 1
is pd. if
X10 M2 X1 X10 X1 is nd
X10 X1 X10 M2 X1 is pd
X10 X1 X10 M2 X1 = X10 (I M2 )X1

and clearly X10 (I M2 )X1 is positive de…nite.

z 0 X10 (I M2 )X1 z > 0

for all z.
DGP A: DGP B:
Y = X1 1 + X2 2 + " Y = X1 1 + "
ECO. A: Y = X1 1 + X2 2 +u Correct: BLUE misspeci…cation
(redundant variable)
(unrestricted model) Unbiased, but ine¢ cient.
ECO. B: Y = X1 1 + v mispeci…cation Correct: BLUE
(omitted variable)
(restricted model 2 = 0) biased