Regression Analysis

1
Supervised Learning
◦ kNN (k Nearest Neighbors)
◦ Linear Regression
◦ Naïve Bayes
◦ Logistic Regression
◦ Support Vector Machines
◦ Random Forests

2
What is Regression Analysis
•Regression analysis is a statistical method to model the relationship
between a dependent (target) and independent (predictor)
variables.

•It is used in the real world where we need some future predictions
such as weather condition, sales prediction, marketing trends, etc.,

•It helps to predict real/continuous values.

3
Types of Regression
There are various types of regressions which are used in machine
learning. Some of the types of regression are as follows.
•Linear Regression
•Logistic Regression
•Polynomial Regression
•Support Vector Regression

4
Linear Regression
•Linear regression shows the linear relationship between the
independent variable (X-axis) and the dependent variable (Y-axis),
hence called linear regression.
•f there is only one input variable (x), then such linear regression is
called simple linear regression.
•if there is more than one input variable, then such linear regression
is called multiple linear regression.

5
Linear Regression Cont…
For instance, predicting the salary of an employee on the basis
of the year of experience.

6
Linear Regression Cont…
The population regression model:

Population Random
Population Independent Error
Slope
y intercept Variable term, or
Coefficient
Dependent residual

y = β0 + β1x + ε
Variable

Linear component Random Error

component

7
Linear Regression (continued)

y y = β0 + β1x + ε
Observed Value
of y for xi

εi Slope = β1
Predicted Value Random Error
of y for xi
for this x value

Intercept = β0

xi x
8
Least Square Error

9
Linear Regression

10
Explained and Unexplained Variation
y
yi 
 2
SSE = (yi - yi ) y
_
SST = (yi - y)2

y  _2
_ SSR = (yi - y) _
y y

Xi x
Logistic Regression
Why Use Logistic Regression?
▪There are many important topics for which the dependent
variable is "limited."
▪For example: Gender Classification
▪Dependent variable is discrete variable: coded 0 (did not
vote) or 1(did vote)

12
Logistic Regression
Equation

Where, e is euler’s constant and its value is 2.718

13
Logistic Regression

14