Chapter 06

Ch.6 Multiple Regression: Further Issues

Multiple Regression Analysis
1. Effects of Data Scaling on OLS
Statistics
y = β0 + β1x1 + β2x2 + . . . βkxk + u 2. More on Functional Form
3. More on Goodness-of-Fit & Selection
4. Further Issues of Regressors
4. Prediction & Residual Analysis*

Econometrics 1 Econometrics 2

6.1 Effects of Data Scaling Cont. Redefining Variables

If xj is multiplied by c, its coefficient is
Redefining Variables divided by c.
Changing the scale of the variable will If y is multiplied by c, all OLS coefficients is
lead to a corresponding change in the multiplied by c.
scale of the coefficients and standard Neither t nor F statistics are affected by
errors, but no change in the changing the units of measurement of any
variables.
significance or interpretation.
If the variables appears in logarithmic form,
→ see table 6.1. changing unit of measurement does not affect
the slope coefficient.
Econometrics 3 Econometrics 4

Effects of Data Scaling Beta Coefficients

Dependent
y cy y Idea is to replace y and each x variable with a
Independent
standardized version – subtract mean and
x1 β1 (se1) cβ1 (c*se1) ―― divide by standard deviation.
dx1 ―― ―― β1/d (se1/d) Coefficient reflects standard deviation of y
x2 β2 (se2) cβ2 (c*se2) β2 (se2) for a one standard deviation change in x.
„ We can compare the magnitudes of the resulting
Intercept β0 (se0) cβ0 (c*se0) β0 (se0)
beta coefficients and conclude that “which
R-squared R2 R2 R2 variable is most important,” etc.
SSR SSR c2*SSR SSR
„ Whether we use standardized or unstandardized
variables does not affect statistical significance.
Standard errors in parentheses
Econometrics 5 Econometrics 6

Multiple Regression 4: Further Issues 1

Chapter 06

6.2 More on Functional Form Interpretation of Log Models

ln(y) = β0 + β1ln(x) + u
OLS can be used for relationships that „ β1 is the elasticity of y with respect to x
are not strictly linear in x and y by
using nonlinear functions of x and y – ln(y) = β0 + β1x + u
will still be linear in the parameters. „ β1 is approximately the percentage change in y
given a 1 unit change in x
„ the natural log of x, y or both
„ quadratic forms of x
y = β0 + β1ln(x) + u
„ β1 is approximately the change in y for a 100
„ interactions of x variables percent change in x

Econometrics 7 Econometrics 8

Functional forms with Logarithms Why use log models?

Model
Dependent Independent Interpretation Log models are invariant to the scale of the
of β 1
Variable Variable
variables since measuring percent changes.
level-level y x ∂y/∂x They give a direct estimate of elasticity.
level-log y log(x) ∂y/(∂x/x) For models with y > 0, the conditional
distribution is often heteroskedastic or
log-level log(y) x (∂y/y)/∂x skewed, while ln(y) is much less so.
log-log log(y) log(x) (∂y/y)/(∂x/x) The distribution of ln(y) is more narrow,
limiting the effect of outliers.

Econometrics 9 Econometrics 10

Some Rules of Thumb for “log” Quadratic Models

What types of variables are often used in
For a model of the form
log form?
„ Dollar amounts that must be positive y = β0 + β1x + β2x2 + u,
„ Very large variables, such as population we can’t interpret β1 alone as measuring the
What types of variables are often used in change in y with respect to x, we need to take
level form? into account β2 as well, since
„ Variables measured in years
∂yˆ ˆ
„ Variables that are a proportion or percent ≈ β1 + 2 βˆ2 x
∂x
Econometrics 11 Econometrics 12

Multiple Regression 4: Further Issues 2

Chapter 06

More on Quadratic Models Interaction Terms

For the case of the coefficient on x > 0 and the For a model of the form
coefficient on x2 <0, y is increasing in x at first, y = β0 + β1x1 + β2x2 + β3x1x2 + u
but will eventually turn around and be decreasing
in x (see fig.6.1). we can’t interpret β1 alone as measuring the
For the case of the coefficient on x < 0 and the change in y with respect to x1, we need to
coefficient on x2 > 0, y is decreasing in x at first, take into account β3 as well, since
but will eventually turn around and be increasing ∂y
in x (see fig.6.2). = β1 + β 3 x 2
For both case, the turning point will be at ∂x1
( )
x* = βˆ1 2 βˆ2 .
„ To summarize the effect of x1 on y, we typically
evaluate the above at average of x2.
Econometrics 13 Econometrics 14

6.3 More of Goodness-of-Fit Cont. Adjusted R-Squared

Adjusted R-Squared
Recall that the R2 will always increase as
more variables are added to the model.
The adjusted R2 takes into account the
number of variables in a model, and may
decrease.
R 2 ≡ 1−
[SSR (n − k − 1)] = 1 − (1 − R 2 ) n − 1
[SST (n − 1)]
Most PC packages will give you both R2
and adj-R2.
You can compare the fit of 2 models (with
the same y) by comparing the adj-R2 .
However, you cannot use the adj-R2 to
compare models with different y’s.
(6.22) „ e.g. y vs. ln(y).
n − k −1

Econometrics 15 Econometrics 16

Goodness of Fit
It is important not to fixate too much on adj-
R2 and lose sight of theory and common sense.
„ If economic theory clearly predicts a variable
belongs, generally leave it in the model.
„ Don’t want to include a variable that prohibits a
sensible interpretation of the variable of interest –
remember ceteris paribus interpretation of
multiple regression.
e.g. housing price = f ( no. of rooms, square footage)

Econometrics 17

Multiple Regression 4: Further Issues 3