A Decision-Making Approach

Linear Regression and Correlation

Analysis

Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-1

 Introduction to
Regression Analysis
 Regression analysis is used to:
 Predict the value of a dependent variable (y), based on the
value of at least one independent variable (x)
 Dependent variable: the variable we wish to explain.
 Independent variable: the variable used to explain the
dependent variable
 Explain the impact of changes on an independent variable
on the dependent variable

Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-2

 Simple Linear Regression Model
 Only one independent variable, x
 Relationship between x (iv) and y (dv) is
described by a linear function
 independent: iv, dependent: dv
 Changes in y (dv) are assumed to be
caused by changes in x (iv)

Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-3

 Types of Regression Models
Positive Linear Relationship Relationship NOT Linear

Negative Linear Relationship No Relationship

Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-4

 Population Linear Regression

The population regression model:

Population Random
Population Independent Error
Slope
y intercept Variable term, or
Coefficient
Dependent residual

y  β0  β1x  ε
Variable

Linear component Random Error

component

Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-5

 Linear Regression Assumptions

 Error values (ε) are statistically independent.

 The difference between the expected value and actual

value is called error term.

 i.e. stock prices, no. of sales etc.

Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-6

 Population Linear Regression
(continued)

y y  β0  β1x  ε
Observed Value
of y for xi

εi Slope = β1
Predicted Value Random Error
of y for xi
for this x value

Intercept = β0

xi x
Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-7
 Interpretation of the
Slope and the Intercept

 b0 is the estimated average value of y

when the value of x is zero.

 b1 is the estimated change in the

average value of y as a result of a
one-unit change in x

Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-8

 Least Squares Criterion

 b0 and b1 are obtained by finding the values

of b0 and b1 that minimize the sum of the
squared residuals (error)

e 2
  (y ŷ) 2

  (y  (b 0  b1x)) 2

Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-9

 Finding the
Least Squares Equation

 The coefficients b0 and b1 will usually be

found using computer software, such as
Excel or Minitab

 Other regression measures will also be

computed as part of computer-based
regression analysis

Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-10

 Simple Regression Steps

1. Specify the independent (x) and the

dependent (y) variables
2. Develop a scatter plot
3. Calculate the correlation coefficient
4. Determine the linear regression equation

Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-11

 Explained and Unexplained
Variation
 Total variation is made up of two parts:

SST  SSE  SSR

Total sum of Sum of Squares Sum of Squares
Squares Error Regression

SST   ( y  y)2 SSE   ( y  ŷ)2 SSR  ( ŷ  y)2

where:
y = Average value of the dependent variable
y = Observed values of the dependent variable
ŷ = Estimated value of y for the given x value
Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-12
 Explained and Unexplained
Variation
(continued)

 SST = total sum of squares

 Measures the variation of the yi values around their
mean y
 SSE = error sum of squares
 Variation attributable to factors other than the
relationship between x and y
 SSR = regression sum of squares
 Explained variation attributable to the relationship
between x and y

Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-13

 Explained and Unexplained
Variation
(continued)
y
yi 
 2
SSE = (yi - yi ) y
_
SST = (yi - y)2

y  _2
_ SSR = (yi - y) _
y y

Xi x
Copyright ©2011 Pearson Education, Inc. publishing as Prentice Hall 14-14