1

Topic:

Multicollinearity
 Outline 2

 What is multicollinearity
 Types of multicollinearity
 Cause of multicollinearity
 Consequences of multicollinearity
 Detection methods of multicollinearity
 Remedies of multicollinearity
 Example
 What is Multicollinearity? 3

 In statistics, multicollinearity is a phenomenon in which one

predictor variable in a multiple regression model can be linearly
predicted from the others with a substantial degree of accuracy
 Examples: If some one want to predict the age of a person by
using weight and height
 4

Types of Multicollinearity

There are two types of multicollinearity

 Perfect multicollinearity
 High or non-perfect multicollinearity
 Perfect Multicollinearity 5
 Perfect multicollinearity refers to a situation in which two or more
explanatory variables in a multiple regression model are perfectly
correlated
 Consider the model:

If

Thus only would be estimable. We cannot get the estimates of and

separately. In this case, there is perfect multicollinearity because and
perfectly correlated
 6

High or Non-Perfect Multicollinearity

 Multicollinearity always exists.  It occurs when there are

high correlations among predictor variables, leading to
unreliable and unstable estimates of regression coefficients
 How Multicollinearity Occurs? 7

 It is caused by inaccurate use of dummy variables

 It is caused by inclusion of a variable which is computed
from other variables in the data set
 Multicollinearity can also results from the replication of
same kind of variable
 Multicollinearity generally occurs when the variables are
highly correlated with each other
 8

Consequences of Multicollinearity
 Even in the presence of multicollinearity, OLS is blue and
consistent
 Stand error of estimate tends to be large
 The probability of committing type-II error will be large
 Estimates of standard errors and parameters tends to be
sensitive to changes in the data and specification of the
model
 9

Detection Method of Multicollinearity

 Correlation Matrix
 Tolerance Measure
 Variance Inflation Factor (VIF)
 Condition Index (CI)
 10

What is Correlation
Matrix?
A correlation matrix is a table showing correlation
coefficient between sets of variables
 11
What is Tolerance Measure?

 The percentage of the variance in the predictor variable

that is not accounted by the other predictor variables.
 Most common tolerance values of 0.10 or less are cited as
problematical
 Although 0.20 has also been suggested.

2
Tolerance  1  R
 What is Variance Inflation Factor 12
(VIF)?
 In statistics,
the variance inflation factor (VIF)quantifies the
severity of multicollinearity in an ordinary least
square regression analysis. It provides an index that measures
how much the variance of an estimated regression coefficient
is increased because of collinearity

 VIF of 5 or 10 and above indicates a multicollinearity problem

 13
Condition Index (CI)
 The condition index is supposed to measure the sensitivity of the
regression estimates to small changes in the data. It is defined as
the square root of the ratio of the largest to the smallest
eigenvalue of the matrix of the explanatory variables

 CI lies between 10 to 30 then there is moderate multicollinearity

 CI exceeds 30 then there will be severe multicollinearity
 Remedies of Multicollinearity 14

The remedies for multicollinearity range from modification of the

regression variates to the use of specialized estimation procedures
 By weighted least square
 By using the Ridge Regression
 Omit one or more highly correlated independent variables
 Use the model with the highly correlated independent variables
 Example 15

Klein and Goldberger want to attempt the following U.S.

economy from 1936-1949.
 Where Y = consumption, X1 = wage income, X2 = nonwage,
nonfarm income, and X3 = farm income. But since X1, X2,
and X3 are expected to be highly collinear.
 Detect the multicollinearity and reduce it.
 Raw data give for consumption, wage income, nonwage and
farm income.
 16
Data
Analyze > Regression > Linear 17
Statistics… 18
Output 19
 20
Remedy by Omitting variable
Analyze > Regression > Linear 21
Statistics… 22
 23
Output
 24
1 2
Remedy Through Weighted Least Square 25
By using weighted average 26
Output 27
 28

1 2