You are on page 1of 17

RELAXING THE ASSUMPTIONS OF CLRM

RELAXING THE ASSUMPTIONS OF CLRM Dr. Obid A.Khakimov Senior lecturer, Westminster International University in Tashkent

Dr. Obid A.Khakimov Senior lecturer,
Westminster International

University in Tashkent

RELAXING THE ASSUMPTIONS OF CLRM Dr. Obid A.Khakimov Senior lecturer, Westminster International University in Tashkent
RELAXING THE ASSUMPTIONS OF CLRM Dr. Obid A.Khakimov Senior lecturer, Westminster International University in Tashkent

ASSUMPTIONS OF CLASSICAL LINEAR REGRESSION MODEL

The regression model is linear in parameters The values of independent variables are fixed in repeated sampling Conditional mean of residuals is equal to zero

For given X’s there is no autocorrelation in the residuals

Independent variables , X’s , and residuals of the regression are independent.

ASSUMPTIONS OF CLASSICAL LINEAR REGRESSION MODEL  The regression model is linear in parameters  The

ASSUMPTIONS OF CLASSICAL LINEAR REGRESSION MODEL

The number of observations must be greater than number of parameters.

There must be sufficient variability in the values of variables.

The regression model should be correctly specified.

There is no linear relationship among

2

independent variables.

e

i

e

~ (0,

N

)

Residuals of the regression normally distributed

ASSUMPTIONS OF CLASSICAL LINEAR REGRESSION MODEL  The number of observations must be greater than number

MULTICOLLINEARITY:

Agenda:

The nature of multicollinearity. Practical consequences. Detection. Remedial measures to alleviate the problem.

MULTICOLLINEARITY: Agenda:  The nature of multicollinearity.  Practical consequences.  Detection.  Remedial measures to

REASONS:

Data collection process

Constraints on model or in the population being sampled.

Model specification An over-determined models

REASONS:  Data collection process  Constraints on model or in the population being sampled. 

PERFECT V.S LESS THAN PERFECT

Perfect multicollinearity is the case when two ore more independe variables Can create perfect linear relationship.

X + X + X 1 1 2 2 3 3 2 3 X = X
X
+
X
+
X
1
1
2
2
3
3
2
3
X
=
X
+
X
1
2
3
1
1
..... X k k k ..... X k 1
.....
X
k
k
k
.....
X
k
1

=

0

Perfect multicollinearity is the case when two ore more independe variables Can create less than perfect linear relationship.

X

1

 
  • 2

X

  • 2 +

  • 1

  • 3 X

k

1

=

3

.....

X

k

+

  • 1 1

1

e

i

PERFECT V.S LESS THAN PERFECT Perfect multicollinearity is the case when two ore more independe variables
MULTIPLE REGRESSION MODEL Y = X + X + u i 2 2 3 3 i
MULTIPLE REGRESSION MODEL
Y = X + X + u
i
2
2
3
3
i
2
2
ˆ
ˆ
ˆ
min u ˆ
=
(
Y
X
X )
i
i
1
2
2
i
3
3
i
ˆ
ˆ
ˆ
= Y
X
X
1
2
2
3
3
(
)
2
y x
(
x
)
(
y x
)(
x x
)
ˆ
i
2
i
3
i
i
3
i
2
i
3
i
=
2
2
2
2
(
x
)(
x
)
(
x x
)
2 i
3
i
2
i
3
i
(
2
y x
)
(
x
)
(
y x
)(
x x
)
ˆ
i
3
i
2
i
i
2
i
2
i
3
i
=
3
2
2
2
(
x
)(
x
)
(
x x
)
2 i
3
i
2
i
3
i

MULTIPLE REGRESSION MODEL

ˆ

2

X = X 2 i 3 i ( ) 2 y x ( x ) (
X
=
X
2
i
3
i
(
)
2
y x
(
x
)
(
y x
)(
x x
)
i
2
i
3
i
i
3
i
2
i
3
i
=
2
2
2
(
x
)(
x
)
(
x x
)
2 i
3
i
2
i
3
i
(
)
2
y x
(
x
)
(
y x
)(
x x
)
ˆ
i
3
i
3
i
i
3
i
3
i
3
i
=
2
2
2
2
(
(
x
) )(
x
)
(
x x
)
3
i
3
i
3
i
3
i
(
)
2
2
y x
(
x
)
(
y x
)(
x
)
3
i
ˆ
i
3
i
3
i
i
3 i
=
=
2
2
2
2
2
(
(
x
)(
x
)
(
x
)
3
i
3
i
3 i
if
_ =
a
(
)
y x a
(
y x a
)
0
ˆ
i
3
i
i
3
i
=
=
2
2
aa
( a )
0

OLS ESTIMATION

ˆ

var( ) =

1

var( ˆ ) =

2

2 2 2 2 1 X x + X x 2 X X x x 2
2
2
2
2
1
X
x
+
X
x
2
X X
x x
2
3 i
3
2
i
2
3
2
i
3
i
+
2
2
2
n
x
x
(
x x
)
2 i
3
i
2
i
3
i
2
2
var( ˆ ) =
2
2
3
2
2
x
(1
r
)
x
(1
r
)
2
i
2,3
3
i
2,3

2

cov( ˆ , ˆ ) =

2

3

2 r 2,3 2 2 2 (1 r ) x x 2,3 2 i 3 i
2
r
2,3
2
2
2
(1
r
)
x
x
2,3
2 i
3 i

As degree of collinearity approaches to one, the variances of coefficients approaches to infinity.

OLS ESTIMATION ˆ var( ) = 1 var( ˆ ) = 2 2 2 2 2

Thus, the presence of high collinearity will

PRACTICAL CONSEQUENCES

The OLS is BLUE but large variances and covariances making process estimation difficult.

Large variances cause large confidence intervals and accepting or rejecting hypothesis are biased.

T statistics are biased

Although t-stats are low, R-square might be very high.

The sensitivity of estimators and variances are very high to small changes in dataset

PRACTICAL CONSEQUENCES  The OLS is BLUE but large variances and covariances making process estimation difficult.

VIF

VARIANCE INFLATION FACTOR

var( ˆ ) =

2

x

2

2

i

2

r

2,3

)

(1

2

1

=

x

2

2

i

(1

2

r

2,3

)

2

=

x

2

2

i

VIF

0 120 100 80 60 40 20 0.8 Correlation 0.6 0.4 0.2 1.2 0 1
0
120
100
80
60
40
20
0.8
Correlation
0.6
0.4
0.2
1.2
0
1
VIF VARIANCE INFLATION FACTOR var( ˆ ) = 2 x 2 2 i 2 r 2,3

IMPLICATION FOR K VARIABLE MODELS

Y

i

ˆ

=

X

i

=

ˆ

1

X

1

ˆ

2

X

2

ˆ

3

X

3

0

X

0

uˆ

+

+

+

+

k

X

k

+

.....

2

x

2

j

(1

R

2

j

)

2

1

2

x

2

j

var(

ˆ

j

) =

=

=

VIF

x

2

j

(1

R

2

j

)

+

ˆ

3

X

3

ˆ

0

X

0

ˆ

1

X

1

ˆ

2

X

ˆ

k

+

+

+

X

k

.....

i

R

2

=

R

2

j

IMPLICATION FOR K VARIABLE MODELS Y i ˆ = X i = ˆ 1 X 1

CONFIDENCE INTERVALS AND T- STATISTICS

±1.96 se ( ) VIF k k k k 0 t = se ( ) VIF
±1.96
se
(
)
VIF
k
k
k
k 0
t =
se
(
)
VIF
Due to low t-stats we can not reject our
Null Hypothesis
k
H
:
=
=
...
=
=
0
0
2
3
k

H a : Not all slope coefficients are simultaneously zero

2 2 n k ESS n k R R /( k 1) F = = =
2
2
n k ESS
n
k
R
R /( k 1)
F =
=
=
2
2
k
1
RSS k
11 R
(1
R
)/(
n
k
)
ue to high R square the F-value will be very high and rejection of Ho will be easy

DETECTION

Multicollinearity is a question of degree. It is a feature of sample but not population.

How to detect :

High R square but low t-stats. High correlation coefficients among the independent variables. Auxiliary regression High VIF

Eigenvalue and condition index.***

DETECTION  Multicollinearity is a question of degree.  It is a feature of sample but

AUXILIARY REGRESSION

Ho: The Xi variable is not collinear

X

i

=

ˆ

0

X

0

+

ˆ

1

X 1
X
1

+

ˆ

2

X

2

+

ˆ

3

X

3

.....

+

ˆ

k

X

k

R

2

=

R

2

j

Run regression where one X is dependent and other X’s are independent and Obtain R square

F =

i

R

2 /( k 2) x x , , x ... x i 2 3 k 2
2
/( k 2)
x x
,
,
x
...
x
i
2
3
k
2
) /( n k
x x
,
,
x
...
x
i
2
3
k

(1 R

+ 1)

Df num = k-2 Df denom = n-k+1

k- is the number of explanatory variables including intercept. n- is sample size.

If F stat is higher than F critical then Xi variable is collinear

Rule of thumb: if R square of auxiliary regression is higher than over R square then it might be troublesome.

AUXILIARY REGRESSION Ho: The Xi variable is not collinear X i = ˆ 0 X 0

WHAT TO DO ?

Do nothing. Combining cross section and time series Transformation of variables (differencing, ratio transformation) Additional data observations.

WHAT TO DO ?  Do nothing.  Combining cross section and time series  Transformation

READING

Gujarati D., (2003), Basic Econometrics, Ch. 10

READING  Gujarati D., (2003), Basic Econometrics, Ch. 10