Regression Analysis

Prof. Prema Muthuswamy

KCT, Coimbatore
 Purpose of Regression
• To predict the value of a
dependent variable from the
values of at least one explanatory
or independent variable
(predictors)
• To explore the nature of the
relationship among these
variables
 Regression Analysis
• Majority of regression analyses
use a linear regression model

• Dependent variable expressed in

terms of linear combinations of
independent variables
 Relationship

To PREDICT, there should be a

statistical relationship between
variables ( variables that are NOT
perfectly related)
 Statistical Relationship
Examples
• Height & Weight
• Driving speed & fuel mileage
• Running speed and oxygen intake

To check if relationship exists between variables,

run Correlation test
Types of Linear Regression

• Simple Linear Regression

• Multiple Linear regression
 Simple Linear Regression
• One predictor variable
(Independent Variable)
• Predictor can be continuous,
dichotomous or categorical
• Only one response (dependent
variable), and it is continuous.
Simple Linear Regression -
Equation

This is of the form: Y = C + mx

(C is a constant, m is the slope of the line)
 Simple Linear Regression

Slope dy/dx
(b1)

Intercept
(Constant
bo)
Linear Regression –
Examples
Simple Linear Regression
– Example 1 (Data)
Height in Weight in
Age in months
inches pounds
191 62.5 112.5
160 62 94.5
157 64.5 123.5
191 65.3 107
141 61.8 85
185 63.3 101
149 64.3 110.5
169 62.3 99.5
173 62.8 102.5
….. ….. …..
 Simple Linear Regression
Example 1 – Model Significance
62% of the variation
in Weight is
explained by Height

Regression Model is Significant & Prediction is possible

Simple Linear Regression
Example 1- Parameter Estimates

Predicted equation:
Weight = - 127.91 + (3.729 X Height)
 Simple Linear Regression
Example 1 – Regression Plot
 Simple Linear Regression Example 1 –
Regression Plot BEST FIT LINE

Actual Value
Regression
Line

Error/ Predicted Value

Deviation
Simple Linear Regression Example 1 –
Regression Plot BEST FIT LINE
Age in Height in Weight in Predicted
Error (Error)2
months inches pounds Weight
191 62.5 112.5 105.2 7.3 54
160 62 94.5 103.3 -8.8 77
157 64.5 123.5 112.6 10.9 119
191 65.3 107 115.6 -8.6 74
185 63.3 101 108.1 -7.1 51
149 64.3 110.5 111.9 -1.4 2
169 62.3 99.5 104.4 -4.9 24
173 62.8 102.5 106.3 -3.8 14
150 61.3 94 100.7 -6.7 45
144 59.5 93.5 94.0 -0.5 0
….. …… ……. ……. ….. …….

Sum of Square of the Error term = 4554

 Simple Linear Regression Example 1 –
Regression Plot BEST FIT LINE

• Sum of Square of the Error term = 4554

• Best FIT line is the one that has this

sum as small as possible
(Least Squares Criterion)

• No other line will have the this

SUM = or < 4554
 Simple Linear Regression
Example 1 – Actual by Predicted Plot
Multiple Linear Regression
– Example 2 (Same Data)
Height in Weight in
Age in months
inches pounds
191 62.5 112.5
160 62 94.5
157 64.5 123.5
191 65.3 107
141 61.8 85
185 63.3 101
149 64.3 110.5
169 62.3 99.5
173 62.8 102.5
….. ….. …..
 Multiple Linear Regression
Example 2 – Model Significance
65% of the variation
in Weight is
explained by Height
& Age

Regression Model is Significant & Prediction is possible

Multiple Linear Regression
Example 2- Parameter Estimates

Predicted equation:
Weight = - 121.26 + (3.06 X Height) + (0.2067 X Age)
Multiple Linear Regression
Example 2 – Actual by Predicted Plot