Lecture 4

Logistic Regression
Recap

Linear Regression
Cost Function
Gradient Descent
Topics to be Covered

Logistic Regression

Sigmoid Function

Examples

Advantages of LR

Disadvantages of LR
What is Logistic
Regression?
• Logistic regression is also used to estimate the relationship
between a dependent variable and one or more independent
variables.
• It is used to make a prediction about a categorical variable
versus a continuous one.
• A categorical variable can be true or false, yes or no, 1 or 0,
et cetera.
Why Logistic
Regression?
• Logistic regression
analysis is valuable for
predicting the likelihood
of an event.
• It helps determine the
probabilities between any
two classes.
Why Logistic not Linear?
Steps of Logistic Regression
Step 3: Finally, the
Step 1: To calculate output value of the
the binary separation, sigmoid function gets
first, we determine converted into 0 or
In logistic regression, the best-fitted line by 1(discreet values)
we decide a following the Linear based on the
probability threshold. Regression steps. threshold value.

If the probability of a Step 2: Thus, the

particular element is predicted value gets
higher than the converted into
probability threshold probability by feeding
then we classify that it to the sigmoid
element in one group function.
or vice versa.
Sigmoid Function
• Sigmoid is a mathematical function that takes any real
number and maps it to a probability between 1 and 0.
Logit function to Sigmoid Function
• where p(x)/(1-p(x)) is termed odds,
• and the left-hand side is called the logit or log-odds
function.
• The odds are the ratio of the chances of success to the
chances of failure.
• As a result, in Logistic Regression, a linear combination
of inputs is translated to log(odds), with an output of 1.
Inverse of the function
• This is the Sigmoid function, which produces an S-shaped curve.
• It always returns a probability value between 0 and 1.
• The Sigmoid function is used to convert expected values to probabilities.
• The function converts any real number into a number between 0 and 1.
• We utilize sigmoid to translate predictions to probabilities in machine learning.
Types of Logistic Regression

Binary Logistic Multinomial Logistic Ordinal Logistic

Regression Regression Regression
• two or binary • three or more • three or more like
outcomes like yes outcomes like first, multinomial logistic
or no second, and third regression but here
class or no class with the order like
degree customer rating in
the supermarket
from 1 to 5
Binary logistic regression

Whether or not to lend to a bank customer (outcomes are yes or

no).

Assessing cancer risk (outcomes are high or low).

Will a team win tomorrow’s game (outcomes are yes or no).

Multinomial logistic regression

Classifying texts into what language they come from.

Predicting whether a student will go to college, trade

school or into the workforce.

Does your cat prefer wet food, dry food or human food?
Ordinal Regression

Ranking restaurants on a scale of

0 to 5 stars.

Predicting the podium results of

an Olympic event.

Assessing a choice of candidates,

specifically in places that institute
ranked-choice voting.
 An email is a spam

It’ll rain today

A tumor is fatal

An individual will purchase a car

Example An online transaction is fraudulent

Scenarios A contestant will win an election

A group of users will buy a product

An insurance policyholder will expire before the policy term

expires
A promotional email receiver is a responder or non-responder
Example
• Let us consider obese or
non-obese
• The sigmoid function returns the probability for each output value from the regression line. Now
based on a predefined threshold value, we can easily classify the output into two classes Obese or
Not-Obese.
 Overfitting occurs when a model
is excessively complex, such as
having too many parameters
relative to the number of
observations. A model that has
been overfit has poor predictive
performance, as it overreacts to
minor fluctuations in the training
data.