Multiple Regression

Introduction

• In this chapter, we extend the simple linear

regression model. Any number of independent
variables is now allowed.

• We wish to build a model that fits the data better

than the simple linear regression model.
• Computer printout is used to help us:
– Assess/Validate the model
• How well does it fit the data?
• Is it useful?
• Are any of the required conditions violated?
– Apply the model
• Interpreting the coefficients
• Estimating the expected value of the dependent variable
Model and Required Conditions

• We allow for k independent variables to

potentially be related to the dependent variable

Coefficients Random error variable

Y = 0 + 1X1+ 2X2 + …+ kXk + 

Dependent variable Independent variables

 Multiple Regression for k = 2,
Graphical Demonstration
Y The simple linear regression model
allows for one independent variable, “X”
Y =  0 +  1X + 
X X
Note how the straight line Y = 0 + 1 = 0 +  1
 X
Y + 2 2
becomes a plane  +  X1
1
Y = 0

X +  2X 2 +  X2
=  + 1 1
+  X
1 1
2
Y Y = 0 X1
0

The multiple linear regression model

allows for more than one independent variable.
Y = 0 + 1X1 + 2X2 + 
X2
 Required Conditions for the Error Variable

• The error is normally distributed.

• The mean is equal to zero and the standard
deviation is constant ( for all possible values
of the Xis.
• All errors are independent.
Estimating the Coefficients and
Assessing the Model
• The procedure used to perform regression analysis:
– Obtain the model coefficients and statistics using Excel.

– Diagnose violations of required conditions. Try to remedy

problems when identified.

– Assess the model fit using statistics obtained from the

sample.

– If the model assessment indicates good fit to the data, use it

to interpret the coefficients and generate predictions.
• Example 18.1 Where to locate a new motor inn?
– La Quinta Motor Inns is planning an expansion.
– Management wishes to predict which sites are likely to be
profitable, defined as having 50% or higher operating margin (net
profit expressed as a percentage of total revenue).
– Several potential predictors of profitability are:
• Competition (room supply)
• Market awareness (competing motel)
• Demand generators (office and college)
• Demographics (household income)
• Physical quality/location (distance to downtown)
 Operating Margin
Profitability

Competition/ Market Demand/

Community Physical
Supply Awareness Customers

Rooms Nearest Office College Income Disttwn

Space Enrollment
Number of Distance to Median Distance to
hotels/motels the nearest household downtown.
rooms within motel. income.
3 miles from
the site.
 Model and Data
• Data were collected from 100 randomly-selected inns that belong
to La Quinta, and ran for the following suggested model:

Margin = Rooms NearestOffice

College + 5 Income + 6 Disttwn + 
Xm18-01
Margin Number Nearest Office Space Enrollment Income Distance
55.5 3203 4.2 549 8 37 2.7
33.8 2810 2.8 496 17.5 35 14.4
49 2890 2.4 254 20 35 2.6
31.9 3422 3.3 434 15.5 38 12.1
57.4 2687 0.9 678 15.5 42 6.9
49 3759 2.9 635 19 33 10.8
Excel Output
SUMMARY OUTPUT
This is the sample regression equation
Regression Statistics
Multiple R (sometimes called the prediction equation)
0.7246
R Square
Adjusted R Square
Margin = 38.14 - 0.0076 Rooms +1.65 Nearest
0.5251
0.4944
Standard Error + 0.020 Office + 0.21 College
5.51
Observations 100
+ 0.41 Income - 0.23 Disttwn
ANOVA
df SS MS F Significance F
Regression 6 3123.8 520.6 17.14 0.0000
Residual 93 2825.6 30.4
Total 99 5949.5

Coefficients Standard Error t Stat P-value

Intercept 38.14 6.99 5.45 0.0000
Number -0.0076 0.0013 -6.07 0.0000
Nearest 1.65 0.63 2.60 0.0108
Office Space 0.020 0.0034 5.80 0.0000
Enrollment 0.21 0.13 1.59 0.1159
Income 0.41 0.14 2.96 0.0039
Distance -0.23 0.18 -1.26 0.2107
Model Assessment

• The model is assessed using three measures:

– The standard error of estimate
– The coefficient of determination
– The F-test of the analysis of variance
• The standard error of estimates is used in the
calculations for the other measures.
Standard Error of Estimate

• The standard deviation of the error is estimated

by the Standard Error of Estimate:
SSE
sε =
n − k −1

(k+1 coefficients were estimated)

• The magnitude of s is judged by comparing it to:
€
Y.
• From the printout, s = 5.51
• The mean value of Y can be determined as:

Y = 45.739
• It seems that s is not particularly small (relative
to the mean of Y).
• Question:
€ Can we conclude the model does not fit the data well?
Not necessarily.
Coefficient of Determination
• The definition is:

2 SSE
R = 1−
∑ i
(Y − Y ) 2

• From the printout, R2 = 0.5251

• 52.51% of the variation in operating margin is explained by the six
€ variables. 47.49% are unexplained.
independent
• When adjusted for the impact of k relative to n (intended to flag potential
problems with small sample size), we have:
Adjusted R2 = 1-[SSE/(n-k-1)] / [SS(Total)/(n-1)] =
= 49.44%
Testing the Validity of the Model
• Consider the question:
Is there at least one independent variable linearly related to
the dependent variable?
• To answer this question, we test the hypothesis:

H0: 1 = 2 = … = k = 0
H1: At least one i is not equal to zero.

• If at least one i is not equal to zero, the model has some

validity.
• The test is similar to an Analysis of Variance ...
 • The hypotheses can be tested by an ANOVA
procedure. The Excel output is:
MSR/MSE

ANOVA
df SS MS F Significance F
Regression k = 6 3123.8 520.6 17.14 0.0000
Residual n–k–1 = 93 2825.6 30.4
Total n-1 = 99 5949.5

SSR MSR=SSR/k

SSE MSE=SSE/(n-k-1)

SSR: Sum of Squares for Regression

SSE: Sum of Squares for Error
• As in analysis of variance, we have:

[Total Variation in Y] = SSR + SSE.

Large F indicates a large SSR; that is, much of the
variation in Y is explained by the regression model.
Therefore, if F is large, the model is considered valid
and hence the null hypothesis should be rejected.

The Rejection Region:

SSR F>F,k,n-k-1
F= k
SSE
n − k −1
ANOVA
df SS MS F Significance F
Regression 6 3123.8 520.6 17.14 0.0000
Residual 93 2825.6 30.4
Total 99 5949.5

F,k,n-k-1 = F0.05,6,100-6-1=2.17
F = 17.14 > 2.17

Also, the p-value (Significance F) = 0.0000

Reject the null hypothesis.

Conclusion: There is sufficient evidence to reject

the null hypothesis in favor of the alternative hypothesis:
at least one of the i is not equal to zero. Thus, at least
one independent variable is linearly related to Y.
This linear regression model is valid
Interpreting the Coefficients
• b0 = 38.14. This is the intercept, the value of Y when all the
variables take the value zero. Since the data range of all the
independent variables do not cover the value zero, do not interpret
the intercept.
• b1 = – 0.0076. In this model, for each additional room within 3 mile
of the La Quinta inn, the operating margin decreases on average
by .0076% (assuming the other variables are held constant).
• b2 = 1.65. In this model, for each additional mile that the nearest
competitor is to a La Quinta inn, the operating margin increases
on average by 1.65%, when the other variables are held constant.
• b3 = 0.020. For each additional 1000 sq-ft of office space, the
operating margin will increase on average by .02%, when the
other variables are held constant.
• b4 = 0.21. For each additional thousand students, the operating
margin increases on average by .21%, when the other variables
are held constant.
• b5 = 0.41. For each increment of $1000 in median
household income, the operating margin would increase
on average by .41%, when the other variables remain
constant.
• b6 = -0.23. For each additional mile to the downtown
center, the operating margin decreases on average
by .23%, when the other variables are held constant.
Testing Individual Coefficients
• The hypothesis for each i is:

H0: i  0 Test statistic

H1: i  0 b −βi
t= i d.f. = n - k -1
sbi
• Excel output:

Ignore Insufficient Evidence

Coefficients Standard Error t Stat P-value

Intercept 38.14 6.99 5.45 0.0000
Number -0.0076 0.0013 -6.07 0.0000
Nearest 1.65 0.63 2.60 0.0108
Office Space 0.020 0.0034 5.80 0.0000
Enrollment 0.21 0.13 1.59 0.1159
Income 0.41 0.14 2.96 0.0039
Distance -0.23 0.18 -1.26 0.2107

Insufficient Evidence
La Quinta Inns, Point Estimate
Xm18-01
• Predict the average operating margin of an inn at a site
with the following characteristics:
– 3815 rooms within 3 miles,
– Closet competitor .9 miles away,
– 476,000 sq-ft of office space,
– 24,500 college students,
– $35,000 median household income,
– 11.2 miles distance to downtown center.
MARGIN = 38.14 - 0.0076 (3815) +1.65 (.9) + 0.020 (476)
+0.21 (24.5) + 0.41 (35) - 0.23 (11.2) = 37.1%
Regression Diagnostics
• The conditions required for the model
assessment to apply must be checked.
– Is the error variable normally
distributed? Draw a histogram of the residuals
– Is the error variance constant? Plot the residuals versus the
predicted values of Y
– Are the errors independent? Plot the residuals versus the
time periods
– Can we identify outlier?
– Is multicolinearity (correlation between the Xi’s) a problem?