Week #7

Logistic Regression

Artificial Intelligence &

Intelligent Systems
Slides By Dr.Rami Ibrahim
Objective
s
 Logistic Regression as a method for classification.
 Some classification examples:
🞑 Spam vs. No Spam
🞑 Bank Loan (Approve vs. Reject)

🞑 Tumor diagnosis (Cancer/No cancer)

 What is common is previous examples?
Logistic Regression
 Logistic Regression allow us to solve classification
problems with discrete categories.
 The convention for binary classification is to have two
classes 0 and 1.
Logistic vs. Linear Regression
 If training data has two values 0 and 1, fitting a line can cause
prediction of negative values (doesn’t make any sense).
 Using a sigmoid function with values between 0 and 1 can
predict the output properly.
Logistic Regression
 We can set a cutoff point at 0.5, anything below it results in 0,
anything above is class 1.
 Transformation
Probability of
disease

x
Transformation

a = log odds of disease

in unexposed

b = log odds ratio associated

with being exposed

e b = odds ratio
{

logit of P(y|x)
When use Logistic regression
 Dependent variable is discrete.
 At least one of the independent variables is continuous.
Logistic Regression Evaluation
 After training logistic regression model on training data, we will
evaluate the performance using test data.
 We can use confusion matrix to evaluate logistic regression
since it is a classification model.
Logistic Regression Advantages
 Easy to implement and interpret.
 Make no assumption about distribution of classes.
Logistic Regression Disadvantages
 Overfit for small datasets (number of observations lesser than
number of features).
 Non-linear problems can't be solved with logistic regression
because of its linear decision surface.
 Logistic regression assumes multicollinearity between
independent variables.
 References
Hands-On Machine Learning with Scikit-Learn, Keras, and
TensorFlow: Concepts, Tools, and Techniques to Build Intelligent
Systems’ 2nd Edition