1

Chapter 1: The Nature of Econometrics and Economic Data

Chapter 2: The Simple Regression Model
PA R T

Chapter 3: Multiple Regression Analysis: Estimation

Chapter 4: Multiple Regression Analysis: Inference

Chapter 5: Multiple Regression Analysis: OLS Asymptotics

2

Chapter 6: Multiple Regression Analysis: Further Issues

P A R T

Chapter 7: Multiple Regression Analysis with Qualitative

Information: Binary (or Dummy) Variables
Chapter 8: Heteroskedasticity

1
 Chapter
3
Multiple Regression Analysis:
Estimation

Part 1

Ruslan 2
 Outlin
1. e Regression
Motivation for Multiple
2. Mechanics and Interpretation of Ordinary Least Squares
3. The Expected Value of the OLS Estimators
4. The Variance of the OLS Estimators
5. Efficiency of OLS: The Gauss-Markov Theorem
6. Some Comments on the Language of Multiple Regression
Analysis

3
 1. Motivation for Multiple
Regression

1. Motivation for Multiple Regression

• Definition of the multiple linear regression

model
“Explains variable in terms of variables ”
Intercept Slope parameters

Dependent
variable, explained Error term,
Independent variables, disturbance,
variable, response
explanatory variables, unobservables,…
variable,… regressors,…

4
 1. Motivation for Multiple
Regression

Motivation for multiple

regression
– Incorporate more explanatory factors into the model
– Explicitly hold fixed other factors that otherwise
would be in u
– Allow for more flexible functional forms

5
 1. Motivation for Multiple
Regression

• Example: Wage
equation

Now measures effect of education explicitly holding experience

fixed

All other factors…

Hourly wage Years of education Years of labor market experience

6
 1. Motivation for Multiple
Regression

• Example: Average test scores and per

student spending

Other factors

Average standardized Per student Average family income

test score of school spending at this of students at this
school school

• Per student spending is likely to be correlated with average

family income at a given high school because of school financing
• Omitting average family income in regression would lead to
biased estimate of the effect of spending on average test scores
• In a simple regression model, effect of per student spending
would partly include the effect of family income on test scores

7
 1. Motivation for Multiple
Regression

• Example: Family income and family

consumption
Other factors

Family consumption Family income Family income

squared

– Model has two explanatory variables: inome and

income squared
– Consumption is explained as a quadratic function of
income
By how much does Depends on how
consumption increase if much income is
income is increased by one already there
unit?
– One has to be very careful when interpreting the
coefficients:

8
 1. Motivation for Multiple
Regression

• Example: CEO salary, sales, and CEO

tenure

Log of CEO salary Log sales Quadratic function of CEO tenure with the firm

– Model assumes a constant elasticity relationship between

CEO salary and the sales of his or her firm
– Model assumes a quadratic relationship between CEO
salary and his or her tenure with the firm

• Meaning of “linear” regression

– The model has to be linear in the parameters (not in the
variables)

Ruslan 9
 2. Mechanics and Interpretation of Ordinary Least
Squares

2. Mechanics and Interpretation of

Ordinary Least Squares
• OLS Estimation of the multiple regression model
• Random sample

• Regression residuals

• Minimize sum of squared residuals

Minimization will be carried out by

computer

Ruslan 10
 2. Mechanics and Interpretation of Ordinary Least
Squares

• Interpretation of the multiple regression model

By how much does the dependent variable change if the j-th
independent variable is increased by one unit, holding all
other independent variables and the error term constant

– The multiple linear regression model manages to hold

the values of other explanatory variables fixed even
if, in reality, they are correlated with the explanatory
variable under consideration
– “Ceteris paribus”-interpretation
– It has still to be assumed that unobserved factors do
not change if the explanatory variables are changed

11
 2. Mechanics and Interpretation of Ordinary Least
Squares

• Example: Determinants of college

GPA

College grade point average High school grade point average Achievement test score

• Interpretation
– Holding ACT fixed, another point on high school GPA is
associated with another .453 points college GPA
– Or: If we compare two students with the same ACT, but the
hsGPA of student A is one point higher, we predict student A
to have a colGPA that is .453 higher than that of student B
– Holding high school GPA fixed, another 10 points on ACT are
associated with less than one point on college GPA

12
 2. Mechanics and Interpretation of Ordinary Least
Squares

Properties of OLS on any sample of

data
• Fitted values and
residuals

Fitted or predicted Residual

values s

• Algebraic properties of OLS

regression

Deviations from Covariance between Sample averages of y and of

regression line sum up to deviations and regressors are the regressors lie on
zero zero regression line

13
 2. Mechanics and Interpretation of Ordinary Least
Squares

"Partialling out” interpretation of multiple

regression
• The estimated coefficient of an explanatory variable in a
multiple regression can be obtained in two steps:
i. Regress the explanatory variable on all other explanatory
variables
ii. Regress y on the residuals from this regression

• Why does this procedure work?

– The residuals from the first regression is the part of the
explanatory variable that is uncorrelated with the other
explanatory variables
– The slope coefficient of the second regression therefore represents
the isolated effect of the explanatory variable on the dependent
variable

14
 2. Mechanics and Interpretation of Ordinary Least
Squares

Simple vs Multiple
Regression
• Simple regression: =�
�𝑦 𝑦 𝛽� 𝑥𝑥
𝛽 𝛽 +𝛽
0 1 1

• Multiple regression: = 𝛽̂ 𝛽+𝛽

�𝑦 𝑦 𝑥̂ +𝛽
𝑥
𝛽 𝑥̂
𝑥
𝛽
0 1 1 2 2

= 𝛽̂ 𝛽 + 𝛽̂ 𝛽
�
𝛽 𝛽1 δ�
1
2
̃ 𝛿 𝛿1 - the slope coefficient from
1 the simple regression of
x2 on x1

15
 2. Mechanics and Interpretation of Ordinary Least
Squares

• Example

regress colGPA hsGPA

colGPA Coef. Std. Err. t P>|t| [95% Conf. Interval]

hsGPA .4824346 .0898258 5.37 0.000 .304833 .6600362

_cons 1.415434 .3069376 4.61 0.000 .8085635 2.022304

regress colGPA hsGPA ACT

colGPA Coef. Std. Err. t P>|t| [95% Conf. Interval]

hsGPA .4534559 .0958129 4.73 0.000 .2640047 .6429071

ACT .009426 .0107772 0.87 0.383 -.0118838 .0307358
_cons 1.286328 .3408221 3.77 0.000 .612419 1.960237

16
 2. Mechanics and Interpretation of Ordinary Least
Squares

Goodness-of-
Fit
• Decomposition of total variation

Notice that R-squared can

• R-squared only increase if another
explanatory variable is
added to the regression

• Alternative expression for R-squared

R-squared is equal to the

squared correlation coefficient
between the actual and the
predicted value of the
dependent variable

17
 2. Mechanics and Interpretation of Ordinary Least
Squares

• Example: Explaining arrest

records
Number of Proportion prior Months in prison Quarters employed
times arrested arrests that led to 1986 1986
1986 conviction

• Interpretation:
– If the proportion prior arrests increases by 0.5, the
predicted fall in arrests is 7.5 arrests per 100 men
– If the months in prison increase from 0 to 12, the predicted
fall in arrests is 0.408 arrests for a particular man
– If the quarters employed increase by 1, the predicted fall in
arrests is 10.4 arrests per 100 men

18
 2. Mechanics and Interpretation of Ordinary Least
Squares

• Example: Explaining arrest records (cont.)

– An additional explanatory variable is added:

Average sentence in prior

convictions

R-squared increases only slightly

• Interpretation:
– Average prior sentence increases number of arrests (?)
– Limited additional explanatory power as R-squared
increases by little
• General remark on R-squared
– Even if R-squared is small (as in the given example),
regression may still provide good estimates of ceteris
paribus effects

19