6

Regression Analysis

Business Analytics, 1e
By Sanjiv Jaggia, Alison Kelly, Kevin Lertwachara, and Leida Chen

Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior
7/29/2020 written consent of McGraw-Hill Education.
6-1
Chapter 6 Learning Objectives (LOs)

LO 6.1 Estimate and interpret a linear regression

model.
LO 6.2 Interpret goodness-of-fit measures.
LO 6.3 Conduct tests of significance.
LO 6.4 Address common violations of the OLS
assumptions.

BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-2

Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-2
 6.2: Model Selection (1/7)
• Example: Recall the introductory case and consider three
models. Which should we choose?

• Several “goodness-of-fit” measures summarize how well the

sample regression equation fits the data.
– The standard error of the estimate, 𝑠𝑒
– The coefficient of determination, 𝑅2
– The adjusted coefficient of determination, adjusted 𝑅2

BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-3

Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-3
 6.2: Model Selection (2/7)
• Recall that a residual is the difference between the observed and predicted value
of the response, 𝑒𝑖 = 𝑦𝑖 − 𝑦ො𝑖 .
• The sample regression equation provides a good fit when the dispersion of the
residuals is relatively small.
• The sample variance is the average squared deviation between the observed and
predicted values.
• The standard deviation of the residuals (standard error of the estimate) has the
same units of measurement as the response.
• For a given sample size, increasing the number of predictors reduces the
numerator and denominator.
– The net effect allows us to determine if the added predictor variables improve the fit.
– When comparing models with the same response, the model with the smaller 𝒔𝒆 is preferred.

BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-4

Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-4
 6.2: Model Selection (3/7)
• The coefficient of determination, 𝑅2 , quantifies the sample variation in the
response that is explained by the sample regression equation.
• It is the ratio of the explained variation of the response variable to its total
variation.
– The proportion of the sample variation in the response explained by the sample
regression equation.
– Falls between 0 and 1
– The closer to 1, the better the fit

BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-5

Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-5
 6.2: Model Selection (5/7)
• We cannot use 𝑅2 for model comparison when the competing
models do not include the same number of predictor
variables (but have the same response).
– 𝑅2 never decreases as we add more variables.
– May include variables with no economic or intuitive foundation.
• Adjusted 𝑅2 explicitly accounts for the sample size 𝑛 and the
number of predictor variables 𝑘.
– Imposes a penalty for any additional predictors.
– The higher the adjusted 𝑅2 , the better the model.
• When comparing models with the same response, the model
with the higher adjusted 𝑅2 is preferred.

BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-6

Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-6
 6.2: Model Selection (6/7)
• Example: Recall the introductory case and consider three models.

a. Which of the three models is the preferred model?

b. Interpret the coefficient of determination for the preferred model.
c. What percentage of the sample variation in annual post-college earnings
is unexplained by the preferred model?

BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-7

Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-7
 6.2: Model Selection (7/7)

a. Model 3 has the lowest standard error of the estimate and

the highest adjusted 𝑅2 .
b. Model 3 explains 42.92% of the sample variation in the
earnings.
c. Model 3 does not explain 57.08% of the sample variation in
earnings.

BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-8

Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-8
 Exercises

BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-9

Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-9
 BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-10
Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-10
 BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-11
Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-11
 BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-12
Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-12
 Quiz

BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-13

Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-13
 Problem Set

BUSINESS ANALYTICS, 1e | Jaggia, Kelly, 6-14

Copyright © 2021 McGraw-Hill Education. AllLertwachara,
rights reserved.Chen
No reproduction or distribution without the prior
written consent of McGraw-Hill Education.
Copyright © 2021 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education. 6-14