Econometrics I: TA Session 3

Giovanna Úbida

INSPER

1 / 20
PS01-Q7

Finite sample properties

Under the assumptions A1 to A3 presented in class, show that:
a) E [βols |X] = β
b) Var (βols |X) = σ 2 (X0 X)−1
c) if E[βols |X] = β and Var(βols |X) = σ 2 (X0 X)−1 , then
Var(β̂ | X) ≥ Var(βols | X), where β̂ is any linear unbiased
estimator. That is, show the Gauss-Markov Theorem.
d) Cov (βols , e|X) = 0

2 / 20
PS01-Q7

d)

cov (β̂, e|X ) = cov ((X 0 X )−1 X 0 y , My |X )

= (X 0 X )−1 X 0 ycov (y , y |X )M 0
= (X 0 X )−1 X 0 var (y |X )M 0
= (X 0 X )−1 X 0 σ 2 In M 0
= σ 2 (X 0 X )−1 X 0 M 0
= σ 2 (X 0 X )−1 (MX )0
=0

Since MX = 0

3 / 20
PS01-Q10
Projection and annihilator matrices
Take e = y − X βols , ε as the true error term, the projection matrix
P and the annihilator M defined as:

P ≡ X(X0 X)−1 X0
M ≡ In − P

Show that:
a) P and M are symmetric and idempotent
b) PX = X
c) MX = 0 , MP = 0 = PM
d) ŷ = Py
e) P ŷ = ŷ , Pe = 0
f) M ŷ = 0, Me = e
g) X 0 ŷ = X 0 y
4 / 20
PS01-Q10

h) y 0 ŷ = y 0 X βols = βols
0 X 0X β 0
ols = ŷ ŷ
i) e = My = Mε
j) SSR = e’e = ε0 Mε
k) e’e = y’My = y’y - ŷ 0 ŷ
0
2 = y Py
l) Ruc
y 0y
m) Every idempotent matrix is non-negative definite. (Does it
imply that an idempotent matrix is full rank? Why?)
n) Every symmetric and idempotent matrix has zeros or ones as
its eigenvalues. (What is its trace then? And what is the
connection with its rank?)
o) Show that 0 ≤ mii ≤ 1 and 0 ≤ pii ≤ 1, where mii and pii
denote the ith diagonal elements of M and P.

5 / 20
PS01-Q10

a) P and M are symmetric and idempotent

I Symmetric

P 0 = [X (X 0 X )−1 X 0 ]0 = X [X (X 0 X )−1 ]0 = X (X 0 X )−1 X 0 = P

Since (X 0 X )−1 is symmetric

Moreover
M 0 = (I − P)0 = I 0 − P 0 = I − P = M
Since P and I are symmetric (I is the identity matrix)

6 / 20
PS01-Q10
I Idempotent

P 2 = P 0 P = PP
0 −1 0
X (X 0 X )−1 X 0

= X
(XX) X


= X (X 0 X )−1 X 0
=P

Since (X 0 X )−1 X 0 X = I

M 2 = (I − P)2 = (I − P)0 (I − P)
= (I 0 − P 0 )(I − P) = (I − P)(I − P)
= I 2 − IP − PI + P 2
= I − 2P + P = I − P
=M

Since P 2 = P and I 2 = I
7 / 20
PS01-Q10
b)
0 −1 0

PX = X 
(XX) XX =X


8 / 20
PS01-Q10
c)
b
MX = (I − P)X = IX − PX = X − X = 0
a
MP = (I − P)P = IP − P 2 = P − P = 0
a
PM = P(I − P) = IP − P 2 = P − P = 0

9 / 20
PS01-Q10
d)
ŷ = Xb = X (X 0 X )−1 X 0 y = Py

10 / 20
PS01-Q10
e)
P ŷ = X (X 0 X )−1 X 0 Xb = Xb = ŷ
Pe = X (X 0 X )−1 X 0 e = X (X 0 X )−1 X 0 (y − Xb)
= X (X 0 X )−1 X 0 y − X (X 0 X )−1 X 0 Xb
= Xb − Xb
=0

11 / 20
PS01-Q10
f)
b
M ŷ = (I − P)Xb = IXb − PXb = Xb − Xb = 0
e
Me = (I − P)e = Ie − Pe = e

12 / 20
PS01-Q10

g)
X 0 ŷ = X 0 Xb = X 0 X (X 0 X )−1 X 0 y = X 0 y
h)
g
y 0 ŷ = y 0 Xb = ŷ 0 Xb = (Xb)0 (Xb) = b 0 X 0 Xb = ŷ 0 ŷ

Where βols = b

i)
d
e = y − Xb = y − ŷ = y − Py = (I − P)y = My
c
= M(X β + ) = MX β + M = M

13 / 20
PS01-Q10

j)
i
X X
SSR = (yi − yˆi )2 = ˆi 2 = e 0 e = (M)0 (M)
a a
= 0 M 0 M = 0 MM = 0 M

k)
i
e 0 e = (My )0 (My ) = y 0 M 0 My = y 0 My
h
= y 0 e = y 0 (y − ŷ ) = y 0 y − y 0 ŷ = y 0 y − ŷ 0 ŷ

14 / 20
PS01-Q10

l) According to Hayashi CH1, one measure Pof 2the variability of the

dependent variable is the sum of squares yi = y 0 y and by it we
can find the uncentered R 2
y 0 y = (ŷ − )0 (ŷ − )
= ŷ 0 ŷ − 2ŷ 0  + 0 
= ŷ 0 ŷ − 2b 0 X 0  + 0 
y 0y ŷ 0 ŷ 2b 0 X 0  0 
= − +
y 0y y 0y y 0y y 0y
ŷ 0 ŷ y 0y 0 
0
= 0 − 0
yy yy yy
2 = ŷ 0 ŷ
by definition, Ruc y 0y

15 / 20
PS01-Q10

Thus,

2 y 0y 0  y 0 y − 0 
Ruc = 0
− 0 =
yy yy y 0y
ŷ 0 ŷ (Py )0 (Py )
= 0 =
yy y 0y
0
y Py
= 0
yy

16 / 20
PS01-Q10

m) Let A be an idempotent and symmetric matrix.

Now, define B = Ac
Then c 0 Ac = c 0 AAc = c 0 A0 Ac = B 0 B =
P 2
bi ≥ 0
i.e., c 0 Ac ≥ 0 and A is positive semi-definite (or non-negative
definite)

Since it is non-negative, all of its eigenvalues (λi ) are positive.

Note that this will imply in full rank iff the eigenvalues are different
from zero.

Qis full rank we have that the det(A) 6= 0, moreover

Recall: if A
det(A) = λi

17 / 20
PS01-Q10

n) Let A be idempotent, λ an eigenvalue and v a corresponding

eigenvector.
Now, let’s calculate the eigenvalues λv = Av (*). Since A be
idempotent
(∗) λ scalar (∗)
λv = AAv = Aλv = λAv = λλv = λ2 v

Thus,

λ(1 − λ)v = 0

Finally, since by definition v 6= 0, we have that λ = 0 or λ = 1

18 / 20
PS01-Q10

Also, for the Spectral Decomposition, we have that if A is

symmetric, it can be can be written as A = PDP 0 , where D is a
diagonal matrix with the eigenvalues of A and P is a orthogonal
matrix whose columns are the standardized eigenvectors of A.

Thus,
P0 P=I (@)
tr (A) = tr (AA) = tr (PDP 0 PDP 0 ) = tr (PD 2 P 0 ) = tr (PDP 0 )
= tr (DP 0 P) = tr (D) = sum of eignvalues of A

Finally, since the eignvalues of A are either 0 or 1, tr (D) = k iff

λi 6== 0 for all i. Thus, tr (D) = k ⇐⇒ rank(D) = k

(@) Since D is the diagonal matrix with only 0 and 1 in its diagonal, we have
that D 2 = D

19 / 20
PS01-Q10

o) Since M is idempotent, by m, it is also positive semi-definite,

thus by definition1 mii ≥ 0. And the same holds for P, thus pii ≥ 0

Recall that P = I − M and let’s pre-multiply and post-multiply it

by the canonical vector in order to get mii and pii . Thus

P = I − M ⇐⇒ ei0 Pei = ei0 Iei − ei0 Mei ⇐⇒ pii = 1 − mii

Since mii and pii , this last equality holds iff 0 ≤ mii ≤ 1 and
0 ≤ pii ≤ 1

1 0
a Ma ≥ 0, let a = ei , thus ei0 Mei = mii
20 / 20