Multiple regression analysis: Estimation

Chapter 3_Review

Course tutor: Ms. Le Thi Ngoc Mai 1

Definition of multiple linear regression model

“Explains variable in terms of variables ”

Intercept Slope parameters

Dependent variable,
Error term,
explained variable, Independent variables, disturbance,
response variable,… explanatory variables, unobservables,…
regressors,…

2
Interpretation of multiple regression model
Example: Determinants of college GPA

Grade point average at college High school grade point average Achievement test score

Interpretation:
• The intercept 1.29 is the predicted college GPA if hsGPA and ACT are 0: not
meaningful in this case.
• Holding ACT fixed, another point on hsGPA is associated with another 0.453 points
on college GPA.
• Or: If we compare two students with the same ACT, but the hsGPA of student A is
one point higher, we predict student A to have a colGPA that is 0.453 higher than
that of student B.
• Holding hsGPA fixed, another 10 points of ACT are associated with another 0.094
points on college GPA.

3
Five Gauss-Markov assumptions
• Assumption MLR.1 (Linear in parameters)
In the population, the
relationship between y and the
explanatory variables is linear

• Assumption MLR.2 (Random sampling)

The data is a random sample
drawn from the population

Each data point therefore follows the population equation

4
Five Gauss-Markov assumptions
• Assumption MLR.3 (No perfect collinearity)
“In the sample (and therefore in the population), none of the independent
variables is constant, and there are no exact linear relationships among the
independent variables.”

• Assumption MLR.4 (Zero conditional mean)

The value of the explanatory variables
must contain no information about the
mean of the unobserved factors

If MLR.4 holds, we have exogenous explanatory variable.

If is correlated with u, then is an endogenous explanatory variable.
5
Five Gauss-Markov assumptions
• Assumption MLR.5 (Homoskedasticity)
The value of the explanatory
variables must contain no
information about the variance of the
unobserved factors

6
Gauss-Markov Theorem
Theorem 3.1: Under assumptions MLR.1 through MLR.4:
OLS estimators are unbiased estimators of the population parameters.
Theorem 3.4: Under assumptions MLR.1 through MLR.5:
OLS estimators for are the best linear unbiased estimators (BLUE).

Note: Heteroskedasticity (failure of assumption MLR.5) does not

cause OLS to be biased, but OLS no longer has the smallest variance
among linear unbiased estimators (no longer BLUE)

7
Model misspecification
• Overspecifying the model: including an irrelevant variable in the model.
This does not affect the unbiasedness of the OLS estimators, but it can
have undesirable effects on the variances of OLS estimators.

• Underspecifying the model: excluding a relevant variable in the model.

This problem causes OLS estimators to be biased: Omitted variable bias

8
Multicollinearity
• High correlation between two or more independent variables is called
multicollinearity.
• Multicollinearity may be detected through variance inflation factors
(VIF)

is obtained from a regression involving only independent variables

of the original model and plays the role of a dependent variable.
As a rule of thumb, the variance inflation factor should not be larger
than 10.