You are on page 1of 103

Violation of OLS assumption - Heteroscedasticity

What, why, so what and what to do?

Lars Forsberg

Uppsala Uppsala University, Department of Statistics

October 22, 2014

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 1 / 103
Econometrics - Objectives and exam

Violations of assumptions - Heteroscedasticity:

What - Explain what is meant by heteroscedasticity


Causes - Account for possible causes of heteroscedastic errors
Consequences - Know what heteroscedasticity does to the OLS
estimators (expectation and variance)
Detection - Account for an informal way to detect heteroscedasticity
(graphing)
Detection - Account for two tests for heteroscedasticity (Explain the
rationale behind them how they are done, and being able to perform
them when given all the relevant numbers

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 2 / 103
Econometrics - Objectives and exam

Violations of assumptions - Heteroscedasticity:


Remedy - Know what to do in the presence of heteroscedasticity
Explain the idea behind the Weighted Least Squares
Perform a WLS estimation using scalar algebra (transformation of the
regression equation and then OLS on the transformed equation)
Interpret the parameters of the original equation after doing WLS

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 3 / 103
Heteroscedasticity - Questions

Outline
How to spell it?
What - is Heteroscedasticity?
Causes - How does Heteroscedasticity come about?
Consequences - Is it a problem? In what way? When/in what
situations?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 4 / 103
Violation of OLS assumption - Heteroscedasticity -
Questions

Outline (cont.)
Detection - How do we know if there is a Heteroscedasticity
problem? (Informal methods, graphs)
Detection - How to test for Heteroscedasticity?
Remedial measures - What can we do about it?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 5 / 103
OLS - Assumptions - Violations - Heteroscedasticity -
How to spell it

Hetero-
sce-
dasti-
city

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 6 / 103
Heteroscedasticity - The word?

The assumption here is Homo-Scedasticity

Constant- Variance

Equal- Variation

Same- Spread

Homo- Scedasticity
That is, constant variance of the Error Term (for the di¤erent values of
the regressor(s))

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 7 / 103
Heteroscedasticity - Animal?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 8 / 103
Heteroscedasticity - Animal?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 9 / 103
Heteroscedasticity - What?

To keep it simple, we have the single linear regression, for a arbitrary


observation
Yi = β1 + β2 X2,i + ui
The assumption (for OLS) is Homoscedasticity

Var (ui ) = σ2

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 10 / 103
Heteroscedasticity - What?

Recall de…ntion of variance

Var (ui ) = E [ui E (ui )]2

recall assumption
E (ui ) = 0

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 11 / 103
Heteroscedasticity - What?

We can write

Var (ui ) = E [ui 0]2

Var (ui ) = E ui2

So the assumption can be written

E ui2 = σ2

Note no i on the sigma-two.

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 12 / 103
Heteroscedasticity - What?

If not homoscedasticity, then we have heteroscedasticity

E ui2 = σ2i

i.e. it depends on i where the i refers to the di¤erent Xi

Remember the income - consumption example.

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 13 / 103
Heteroscedasticity - Why?

Why do we have heteroscedasticity?


When there are natural constraints on the variables,
Consumption - Income
Learning, e.g. number of errors vs time practicing
Improved data quality
Over time
Among the X’s

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 14 / 103
Heteroscedasticity - Why?

Why do we have heteroscedasticity? (cont)


Outliers
Misspeci…cation of model
Skewness of variables

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 15 / 103
Heteroscedasticity - OLS Consequences

Should we worry?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 16 / 103
Heteroscedasticity - OLS Consequences

Should we worry?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 17 / 103
Heteroscedasticity - OLS Consequences - Expectation

OLS: In the presence of Heteroscedastiticy, the OLS-estimator is


still UNBIASED
That is, it is on average correct
The sampling distribution of the estimator b
βj is centered around
the true value βj

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 18 / 103
Heteroscedasticity - Consequences - Variance

Even though the OLS-estimator is unbiased


it no longer have minimum variance
It is NOT BLUE
Not Best, in the sense it has minimun variance among, Linear
Unbiased Estimators
(It is only LUE - Note - Not standard notation...)

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 19 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

To see what happens to the variance in the presence of


heterosceasticity, let us study the simplest case possible, let

Yi = β + ui

where
ui N 0, σ2
That is, a regression on only a constant.

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 20 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

We know that the OLS estimator of β, that is b


β, will be Ȳ , and that it is
unbiased, i.e.
E (Ȳ ) = β

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 21 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

Let’s study the variance


2
V (Ȳ ) = E [Ȳ E (Ȳ )]

2
V (Ȳ ) = E (Ȳ β)

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 22 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

Use de…nition of the sample mean


" #2
1 n
n i∑
V (Ȳ ) = E Yi β
=1

because
Yi = β + ui
we have " #2
1 n
n i∑
V (Ȳ ) = E ( β + ui ) β
=1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 23 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

(Again)
" ! #2
n
1
V (Ȳ ) = E
n ∑ ( β + ui ) β
i =1

Split up sum and since β is a constant ∑ni=1 β = nβ, we have


" ! #2
n
1
V (Ȳ ) = E nβ + ∑ ui β
n i =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 24 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

1
Multiply in n to get
2
nβ ∑ni=1 ui
V (Ȳ ) = E + β
n n
2
∑ni=1 ui
V (Ȳ ) = E β + β
n

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 25 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

(again)
2
∑ni=1 ui
V (Ȳ ) = E β + β
n
2
∑ni=1 ui
V (Ȳ ) = E
n

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 26 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

1
Take the constant n outside the square, and thus, square it, we have
" #2
2 n
1
V (Ȳ ) =
n
E ∑ ui
i =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 27 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

Studying
" #2
n
∑ ui
i =1

again, we have the situation

(a + b + c )2 = a2 + b2 + c 2 + 2ab + 2ac + 2bc

but now with, not three, but n terms...


" #2
n
∑ ui = [u1 + u2 + ... + un ]2
i =1

= u12 + u22 + ...un2 + 2u1 u2 + 2u1 u3 ...

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 28 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

Again
" #2
2 n
1
V (Ȳ ) =
n
E ∑ ui
i =1
" #
n n i 1
1
V (Ȳ ) =
n2
E ∑ ui2 + 2 ∑ ∑ ui uj
i =1 i =2 j =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 29 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

It can be instructive to study


n i 1
2∑ ∑ ui uj = 2u2 u1 + 2u3 u1 + 2u3 u2 + 2u4 u1 + 2u4 u2 + 2u4 u3 + 2u5 u1 + ...
i =2 j =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 30 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

Split up the expectation, (again)


" #
n n i 1
1
V (Ȳ ) = E ∑ ui2 + 2 ∑ ∑ ui uj
n2 i =1 i =2 j =1

" #
n n i 1
1
V (Ȳ ) =
n2 ∑E ui2 +2∑ ∑ E (ui uj )
i =1 i =2 j =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 31 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

Again study
n i 1
∑ ∑ E (ui uj )
i =2 j =1

We assume that
E (ui uj ) = 0, i 6= j
that is, we have...?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 32 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

So (again)
" #
n n i 1
1
V (Ȳ ) = 2
n ∑E ui2 + 2 ∑ ∑ E (ui uj )
i =1 i =2 j =1

" #
n n i 1
1
V (Ȳ ) = 2
n ∑E ui2 + 2 ∑ ∑0
i =1 i =2 j =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 33 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

That is
" #
n
1
V (Ȳ ) = 2
n ∑E ui2 + 0
i =1

" #
n
1
V (Ȳ ) = 2
n ∑E ui2
i =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 34 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

Since now, (under heteroscedasticity)

E ui2 = σ2i

Again
" #
n
1
V (Ȳ ) =
n2 ∑E ui2
i =1
n
1
V (Ȳ ) =
n2 ∑ σ2i
i =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 35 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

So in the model
Yi = β + ui
the variance of the sample mean (the OLS-estimator) is given by

∑n σ 2
V (Ȳ ) = V b
β = i =12 i
n

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 36 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

Note that if σ2i would be the same for all i that is, we would have
(homoscedasticity)
E ui2 = σ2
Then
" #
n
1
V (Ȳ ) =
n2 ∑E ui2
i =1

" #
n
1
V (Ȳ ) =
n2 ∑ σ2
i =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 37 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

and we know that, for all constants


n
∑c =n c
i =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 38 / 103
Heteroscedasticity - Consequences - Variance - Y-bar

So, we have
" #
n
1
V (Ȳ ) =
n2 ∑ σ2
i =1

1
= n σ2
n2

nσ2
=
n2

σ2
V (Ȳ ) =
n
which is the "usual" variance of the sample mean.

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 39 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Now, back to the (simple linear) regression model

Yi = β1 + β2 Xi + ui

For the OLS estimator of β2 :


2
V β̂2 = E β̂2 E β̂2

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 40 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Recall
∑nj=1 (Xi X̄ ) ui
β̂2 = β2 +
2
∑ (X X̄ )

and
E β̂2 = β2

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 41 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

So
h 2
i
V β̂2 = E β̂2 E β̂2

20 12 3
∑nj=1 (Xi X̄ ) ui
V β̂2 = E 4 @ β2 + 2
β2 A 5
∑ (X X̄ )
20 12 3
n
4 @ ∑ j =1 i
(X X̄ ) ui
V β̂2 = E A 5
2
∑ (X X̄ )

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 42 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

2 3
2
6 ∑nj=1 (Xi X̄ ) ui 7
V β̂2 = E 4 5
2 2
∑ (X X̄ )

h i
2
E (∑ni=1 (Xi X̄ ) ui )
V β̂2 =
2 2
∑ (X X̄ )

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 43 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

2 !2 3
n
Numerator = E 4 ∑ (Xi X̄ ) ui 5
i =1
" #
n n
Numerator = E ∑ ∑ (Xi X̄ ) (Xj X̄ ) ui uj
i =1 j =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 44 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Study numerator (Num.)


2 !2 3
n
Num. = E 4 ∑ (Xi X̄ ) ui 5
i =1
" #
n n
Num. = E ∑ ∑ (Xi X̄ ) (Xj X̄ ) ui uj
i =1 j =1
" ! !#
n n i 1
∑ (Xi ∑ ∑ 2 (Xi
2
Num. = E X̄ ) ui2 + X̄ ) (Xj X̄ ) ui uj
i =1 i =2 j =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 45 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Split up expecation
! !
n n i 1
∑ (Xi 2∑ ∑ (Xi
2
Num. = E X̄ ) ui2 +E X̄ ) (Xj X̄ ) ui uj
i =1 i =2 j =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 46 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

The Xi0 s are constants, so we can move them out of the expectation,
(again)
! !
n n i 1
∑ (Xi 2∑ ∑ (Xi
2
Num. = E X̄ ) ui2 +E X̄ ) (Xj X̄ ) ui uj
i =1 i =2 j =1

n n i 1
∑ (Xi X̄ ) E ui2 + 2 ∑ ∑ (Xi
2
Num. = X̄ ) (Xj X̄ ) E (ui uj )
i =1 i =2 j =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 47 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Recall that
E (ui uj ) = 0, i 6= j
assuming ?, so
n n i 1
∑ (Xi X̄ ) E ui2 + 2 ∑ ∑ (Xi
2
Num. = X̄ ) (Xj X̄ ) E (ui uj )
i =1 i =2 j =1

n n i 1
∑ (Xi X̄ ) E ui2 + 2 ∑ ∑ (Xi
2
Num. = X̄ ) (Xj X̄ ) 0
i =1 i =2 j =1

n
∑ (Xi
2
Num. = X̄ ) E ui2 + 0
i =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 48 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

and that now, (under heteroscedasticity)

E ui2 = σ2i

so
n
∑ (Xi
2
Num. = X̄ ) E ui2
i =1
n
∑ (Xi
2
Num. = X̄ ) σ2i
i =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 49 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

That was the numerator of the variance expcession. Recall

E ∑ni=1 ∑nj=1 (Xi X̄ ) (Xj X̄ ) ui uj


V β̂2 =
2 2
∑ (X X̄ )

we just showed that


" #
n n n
∑ ∑ (Xi ∑ (Xi
2
E X̄ ) (Xj X̄ ) ui uj = X̄ ) σ2i
i =1 j =1 i =1

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 50 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Putting it together we get


2
∑ni=1 (Xi X̄ ) σ2i
V β̂2 =
2 2
∑ (X X̄ )

(Why cannot we just move the σ2i outside the summation?)

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 51 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

So the (correct) variance of the OLS-estimator for β2 in the presence of


heteroscedastiticy is given by
2
∑ni=1 (Xi X̄ ) σ2i
V β̂2 =
2 2
∑ (X X̄ )

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 52 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Note that if
σ2i = σ2 for all i
that is, if we would have homoscedasticity, we have
2
∑ (Xi X̄ ) σ2i
V β̂2 =
2 2
∑ (X X̄ )

2
∑ (Xi X̄ ) σ2
V β̂2 =
2 2
∑ (X X̄ )

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 53 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Recall
a 1
=
a2 a
here
∑ (Xi
2
a= X̄ )

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 54 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

so
2
σ2 ∑ (Xi X̄ )
V β̂2 =
2 2
∑ (X X̄ )

σ2
V β̂2 =
2
∑ (X X̄ )

σ2
V β̂2 = 2
∑ (X X̄ )

i.e. the "usual" variance of the estimator.

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 55 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Compare the two variance expresstions

Homo- Hetero-
Scedasticity Scedasticity

(A) (B)
2
σ2 ∑ni=1 (X i X̄ ) σ2i
V β̂2 = 2 V β̂2 = 2 2
∑ (X X̄ ) ( ∑ (X X̄ ) )

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 56 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Now, unlike in the case of multicollinearity, where multicollinearity always


in‡ates the variance, here there is no general result.
So if we use
σ2
V β̂2 = 2
∑ (X X̄ )
when we should use
2
∑ni=1 (Xi X̄ ) σ2i
V β̂2 =
2 2
∑ (X X̄ )

Do we under or over-estimate true variance?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 57 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

That is, is there a way of knowing


2
∑ni=1 (Xi X̄ ) σ2i σ2
>
2 2 ∑ (X X̄ )
2
∑ (X X̄ )

or

2
∑ni=1 (Xi X̄ ) σ2i σ2
<
2 2 ∑ (X X̄ )
2
∑ (X X̄ )

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 58 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

If we mistakenly ignore heteroscedasticity. What happens? For the


following reasoning, introduce the (temporary) notation
2
∑ni=1 (Xi X̄ ) σ2i
VTRUE β̂2 =
2 2
∑ (X X̄ )

σ2
VFALSE β̂2 = 2
∑ (X X̄ )

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 59 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

So, in testing the signi…cance of the individual parameters

H0 : β j = 0

H1 : β j 6 = 0

Using
b
βj 0
zobs =
σ bβ
j

where s
σ2
σ bβ = 2
j
∑ (X X̄ )

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 60 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Note that here we are dealing with population quantities (pretending to


know σ2 , thus we use
b
βj 0
zobs =
σ bβ
j

since we assume that the error term is normal, and the estimator is a linear
estimator, thus also normal.

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 61 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

Of course in practice, we do not know σ2 and need to estimate it by

Σb
ui2
b2 =
σ
n k
where, of course
bi = Yi
u b
β1 + b
β2 Xi ,2

and k is the number of estimated coe¢ cients in the regressrion equation,


in this case 2. Then we use
b
βj 0
tobs = tn k
b bβ
σ
j

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 62 / 103
Heteroscedasticity - Consequences - Variance - Slope est.
If we UNDER-estimate the true variance, that is

VUsed but FALSE β̂2 < VTRUE β̂2

σ bβ to SMALL
j
+
b
βj
σ bβ to BIG
j
+
Reject H0 : βj = 0 to Often
+
Signi…cance to Often

Think model is "better"


that it actually is

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 63 / 103
Heteroscedasticity - Consequences - Variance - Slope est.
On the other hand, If we OVER-estimate the true variance, that is

VUsed but FALSE β̂2 > VTRUE β̂2

σ bβ to BIG
j
+
b
βj
σ bβ to SMALL
j
+
Never Reject H0 : βj = 0
+
Never Signi…cance

Think model is "worse"


that it actually is

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 64 / 103
Heteroscedasticity - Consequences - Variance - Slope est.

In practice, unfortuently, there is no way of knowing if we under or over


estimate the variance when ignoring heteroscedasticity.

One would need to be careful in the analysis, and/or use Whites


heteroscedastic robust standard errors.

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 65 / 103
Heteroscedasticity - Detection - Graphs
Quick and dirty preliminary analysis: Just plot Y vs X

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 66 / 103
Heteroscedasticity - Detection - Tests

Some tests of Heteroscedasticity, outline:


Parks test
Gleijser test
Goldfeldt-Quandt test (GQ-test)
Breusch-Pagan-Godfrey test (BPG-test)
White’s test

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 67 / 103
Heteroscedasticity - Detection - Tests - Parks test

Parks test:

σ2i = σ2 X2,i e v
β

bi2 = β1 + β2 ln (X2,i ) + vi
ln u

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 68 / 103
Heteroscedasticity - Detection - Tests - Gleiser test

Gleiser test:
jubi j = β1 + β2 X2,i + vi
or other functions of X2

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 69 / 103
Heteroscedasticity - Detection - Tests - BPG test

Breusch-Pagan-Godfrey (BPG) test:


BPG - Test: Idea

Regress
bi2
u
= β1 + β2 X2,i + β3 X3,i + ei
σ̂2
Test statistic
ExplSS
χ2
2

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 70 / 103
Heteroscedasticity - Detection - Tests - White’s tests

Run regression of f (u ) on g (X )
White’s test:

Using square regresors

bi2 = β1 + β2 X2 + β3 X22 + β4 X3 + β5 X32 + e


u

What problem could we encounter running this regression?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 71 / 103
Heteroscedasticity - Detection - Tests - White’s tests

To see if the residual variance increases with the increasing values of the
regressors, we do a
F test of

H0 : βj = 0, j > 1
H1 : At least one βj 6= 0 j = 2, ..., k

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 72 / 103
Heteroscedasticity - Detection - Tests - White’s tests

We can include cross-terms in the regression

bi2 = β1 + β2 X2 + β3 X22 + β4 X3 + β5 X32 + β6 X2 X3 + e


u

Again perform a F test of

H0 : βj = 0, j > 1
H1 : At least one βj 6= 0 j = 2, ..., k

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 73 / 103
Heteroscedasticity - Detection - Tests - White’s tests

Note: With many regressors


What regressors to include in the auxiliry regression
What functional form?
What regressors to square?
What, if any, crossproducts to include?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 74 / 103
Heteroscedasticity - Detection - Tests - Goldfeld-Quandt

Golfeld-Quandt (GQ) test:

σ2i = σ2 Xi2

Yi = β1 + β2 Xi + ui

1 Divide sample into two parts with respect to X


2 Estimate the model on each part
3 Test using
RSS1 /df1
RSS2 /df2

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 75 / 103
Heteroscedasticity - Detection - Tests - Properties - Size

Sample size 10 and 50:


White, White + cross terms, BPG, F and χ2 :

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 76 / 103
Heteroscedasticity - Detection - Tests - Properties - Power
Power comparison

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 77 / 103
Heteroscedasticity - Remedy

OK, we have heterosceasticity, what can we do?


White’s consistent estimator
WLS (Weighted Least Squares)

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 78 / 103
Heteroscedasticity - Remedy - Whites HAC estimator

bi2 as a proxy for σ2i


Idea: Use u
2
∑n (Xi X̄ ) u bi2
V\β̂2 = i =1
2 2
∑ (X X̄ )
use this estimate of the variance instead of the usual.

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 79 / 103
Heteroscedasticity - Remedy - WLS

Instead of OLS, we should use the Weighted Least Squares (WLS))


Without matrix algebra notation, hard to show exactly what is going on.

How can we "do" WLS without matrix algebra?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 80 / 103
Heteroscedasticity - Remedy - WLS

Assume a functional form for

E ui2 = σ2i

We usually assume that the increasing variance of ui is proportional to


some function of (some) X .

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 81 / 103
Heteroscedasticity - Remedy - WLS

Commonly used (assumed, mathematically convenient) functional forms:

σ2i = σ2 Xi2

σ2i = σ2 jXi j

p
σ2i = σ2 Xi

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 82 / 103
Heteroscedasticity - Remedy - WLS

In general, if we have the variable ui that has the variance

V (ui ) = cσ2

What transform can we do on ui , that is, how can we standardize ui to


get the standardized variable, say ui , to have variance σ2 ?

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 83 / 103
Heteroscedasticity - Remedy - WLS

Hint recall
V (aY ) = a2 V (Y )

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 84 / 103
Heteroscedasticity - Remedy - WLS

We try
ui
ui =p
c
then
u
V (ui ) = V pi
c
1
= p 2
V (ui )
c
1
= V (ui )
c
V (ui )
V (ui ) =
c

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 85 / 103
Heteroscedasticity - Remedy - WLS

Recall
V (ui ) = cσ2
so (again)

V (ui )
V (ui ) =
c

cσ2
=
c

V (ui ) = σ2

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 86 / 103
Heteroscedasticity - Remedy - WLS

Back to the regression.

How to implement, we start with

Yi = β1 + β2 Xi + ui

Assuming
σ2i = σ2 jXi j

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 87 / 103
Heteroscedasticity - Remedy - WLS

p
Do the transformation, that is divide by jXi j to get
Y 1 Xi ui
p i = β1 p + β2 p +p
jXi j jXi j jXi j jXi j

Yi = β1 Xi + β2 Xi + ui

Note: No constant in the "new", transformed regression line.

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 88 / 103
Heteroscedasticity - Remedy - WLS

(again)
Yi = β1 Xi + β2 Xi + ui
study the "transformed" error term
ui
ui = p
jXi j

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 89 / 103
Heteroscedasticity - Remedy - WLS

Expectation
!
u
E (ui ) = E p i
jXi j
1
= p E (ui )
jXi j
1
= p 0
jXi j

= 0

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 90 / 103
Heteroscedasticity - Remedy - WLS

Study the variance of ui

Var (ui ) = E (ui E (ui ))2

= E (ui 0)2

V (ui ) = E (ui )2

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 91 / 103
Heteroscedasticity - Remedy - WLS

Focus on E (ui )2

Replace
ui
ui = p
jXi j
and we get
!2
u
E (ui )2 = E p i
jXi j

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 92 / 103
Heteroscedasticity - Remedy - WLS

Recall for any constant a


h i
E (a ui )2 = E a2 (ui )2
= a2 E (ui )2

here
1
a=
jXi j

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 93 / 103
Heteroscedasticity - Remedy - WLS

(again)
!2
u
E (ui )2 = E p i
jXi j

2 !2 3
1
E (ui )2 = E 4 p (ui )2 5
jXi j

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 94 / 103
Heteroscedasticity - Remedy - WLS

1
E (ui )2 = p 2
E ui2
jXi j

1
= E ui2
jXi j

E ui2
E (ui )2 =
jXi j

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 95 / 103
Heteroscedasticity - Remedy - WLS

We have heteroscedasticity, so

E ui2 = σ2i

That is (again)

E ui2
E (ui )2 =
jXi j
σ2i
E (ui )2 =
jXi j

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 96 / 103
Heteroscedasticity - Remedy - WLS

Recall that we assumed that σ2i had the functional form

σ2i = σ2 jXi j

We have, (again)

σ2i
E (ui )2 =
jXi j
σ2 jXi j
E (ui )2 =
jXi j

E (ui )2 = σ2

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 97 / 103
Heteroscedasticity - Remedy - WLS

In the transformed regression,

Yi = β1 Xi + β2 Xi + ui

we have constant variance, i.e. homoscedasticity

V (ui ) = σ2

So, here, WLS is equivalent to using OLS on transformed data!

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 98 / 103
Heteroscedasticity - Remedy - WLS

We started with
Yi = β1 + β2 Xi + ui
but we estimate
Yi = β1 Xi + β2 Xi + ui
How do we interpret β1 and β2 ?
Note that, in e¤ect, when tranforming the PRF, we transform the data -
not the parameters, so in terms of the original regression, we keep the
interpretation of the parameters.

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 99 / 103
Heteroscedasticity - Remedy - WLS

What if we choose (assume) another functional form of the


heterscedasticity, again, for the single linear regression

Yi = β1 + β2 Xi + ui

Assuming
σ2i = σ2 Xi2
given all Xi > 0

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 100 / 103
Heteroscedasticity - Remedy - WLS

Do
q the transformation, we divide the terms of the PFR by that is divide by
Xi2 = Xi to get

Yi 1 Xi ui
= β1 + β2 +
Xi Xi Xi Xi

Yi = β1 Xi + β2 + ui

Yi = β2 + β1 Xi + ui

Note: The coe¢ cients has changed "roles".


It is straightforward to see that now Var (ui ) = σ2 , i.e. constant. We
have transformed away the heteroscedasticity

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 101 / 103
Heteroscedasticity - Remedy - WLS

The transformed regression again

Yi = β2 + β1 Xi + ui

Once estimated, we interpret b


β2 as the slope in the orginal regression

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 102 / 103
Heteroscedasticity - Remedy - WLS

For the WLS - some words of caution:


What if we assume the wrong functional form?
If we assume the wrong functional form, we might induce even
"more" heteroscedasticity than we had to begin with, thus, causing
more damage than good.
So, if uncertain of what kind of functional form we have on the

Lars Forsberg (Uppsala University) 1410 - Hetero-sce-dasti-city October 22, 2014 103 / 103