5 views

Uploaded by Navid Anjum Aadit

save

- The Identification of Insurance Fraud - An Empirical Analysis
- GLM Models
- Corrosion Prediction of Carbon Steel in.pdf
- Cash Management
- 178-hw3
- Great Parameter Estimation Monod
- QA03-Display Inspection Lot.doc
- An Overview of Gradient Descent Optimization Algorithms
- Conditional Logistic Regression Models
- 2005 10 Iccv Levenberg
- Accountatnt as Source 4 Small Business
- Honeywell Equipment Condition Monitor Performance Monitoring Offshore Gas Compressor Tin659
- Machine Learning Terms
- HW_17_ISB_2015
- Subsidence prediction using regression mode
- High Low Method
- Children`s conceptual knowledge of lying.pdf
- 2896-7291-1-PB
- An Appraisal of the Factors that Influence Maintenance of Residential Buildings’ Standards
- Setting Up Release Procedure for PR Without Cla..
- Hotel Industry in Thailand
- 03409_12.pptx
- Chapter2.1.pdf
- nmf_nature.pdf
- SSau Exercises
- Chapter 14 Association Rules Collaborative Filtering
- Risk Analizi
- Chap08Slides.pdf
- Chapter 02
- Iet 603 Assignment 6
- History, Complexity and Health Systems Research
- İNTEGRAL-FORMÜLLERİ.pdf
- logistic_regression.pdf
- Optimization
- Statistics Exercises2
- 6720 Labs Chapter 9
- 6720 Labs Chapter 7
- YTUJENS-2017-8-2.1835
- 10.1.1.35.5072.pdf
- Exercise 7
- Constraint Prog
- 6720 Labs Chapter 2

You are on page 1of 29

Logistic Regression

Ingmar Schuster

Patrick Jähnichen

using slides by Andrew Ng

Institut für Informatik

**This lecture covers
**

●

●

●

●

●

●

Logistic regression

hypothesis

Decision Boundary

Cost function

(why we need a new one)

Simplified Cost function &

Gradient Descent

Advanced Optimization

Algorithms

Multiclass classification

Logistic regression

2

Logistic regression Hypothesis Representation Logistic regression 3 .

Classification Problems ● ● Classification ● malignant or benign cancer ● Spam or Ham ● Human face or no human face ● Positive Sentiment? Binary Decision Task (in most simple case) ● ● ● Want Data point belongs to class if close to 1 Doesn't belong to class if close to 0 Logistic regression 4 .

1] ● 0 asymptote for ● 1 asymptote for Sigmoid Function (S-shape) Logistic Function Logistic regression 5 .Logistic Function (Sigmoid Function) ● maps into interval [0.

. Logistic regression 6 . . define ● If interpret as 70% chance data point belongs to class ● If classify as positive sentiment.● Hypothesis ● Interpretation ● Because probabilites should sum to 1.. malignant tumor.

Logistic regression Decision boundary Logistic regression 7 .

● ● If or equivalently predict y = 1 If or equivalently predict y = 0 Logistic regression 8 .

Example ● If and ● Prediction y = 1 whenever Logistic regression 9 .

Example ● If and ● Prediction y = 1 whenever Logistic regression 10 .

Logistic regression Cost Function Logistic regression 11 .

Training and cost function ● Training data wih m datapoints. n features where ● Average cost Logistic regression 12 .

Reusing Linear Regression cost ● Cost from linear regression with logistic regression hypothesis leads to non-convex average cost ● Convex J easier to optimize (no local optima) All All function function values values below below intersection intersection with with any any line line 13 .

learning algorithm will be penalized by large cost Logistic regression 14 .Logistic Regression Cost function ● If y = 1 and h(x) = 1. Cost = 0 ● But for ● Corresponds to intuition: if prediction is h(x) = 0 but actual value was y = 1.

Logistic Regression Cost function ● If y = 0 and h(x) = 0. Cost = 0 ● But for Logistic regression 15 .

Logistic regression Simplified Cost Function & Gradient Descent Logistic regression 16 .

use the simplified cost function to calculate Logistic regression 17 .Simplified Cost Function (1) ● ● ● Original cost of single training example Because we always have y = 0 or y = 1 we can simplify the cost function definition to To convince yourself.

Simplified Cost Function (2) ● Cost function for training set ● Find parameter argument ● To make predictions given new x output that minimizes J: Logistic regression 18 .

Gradient Descent for logistic regression ● Gradient Descent to minimize logistic regression cost function with identical algorithm as for linear regression Logistic regression 19 .

Advanced Optimization Logistic regression 20 .Beyond Gradient Descent .

Advanced Optimization Algorithms ● Given functions to compute ● ● an optimization algorithm will compute Optimization Algorithms Advantages ● (Gradient Descent) ● Often faster convergence ● Conjugate Gradient ● No learning rate to choose ● BFGS & L-BFGS Disadvantages ● Complex Logistic regression 21 .

Preimplemented Alorithms ● Advanced optimization algorithms exist already in Machine Learning packages for important languages ● Octave/Matlab ● R ● Java ● Rapidminer – under the hood Logistic regression 22 .

Multiclass Classification (by cheap trickery) Logistic regression 23 .

2. Lung Cancer ● Weather: Sunny. . Friends. 3. Job Offers ● Medical diagnosis: Not ill. Cloudy. Invoices. Rain.Multiclass classification problems ● Classes of Emails: Work.. Logistic regression 24 .. Snow ● Number classes as 1. Asthma.

Binary vs. Multiclass Classification Logistic regression 25 .

One versus all Logistic regression 26 .

● ● Train logistic regression classifier for each class i to predict probability of y = i On new x predict class i which satisfies Logistic regression 27 .

This lecture covered ● ● ● ● ● ● Logistic regression hypothesis Decision Boundary Cost function(why we need a new one) Simplified Cost function & Gradient Descent Advanced Optimization Algorithms Multiclass classification Logistic regression 28 .

org. License CC SA NC Lightbulb picture from openclipart. public domain Machine Learning Introduction 29 .Pictures ● ● Tumor picture by flickr-user bc the path.

- The Identification of Insurance Fraud - An Empirical AnalysisUploaded byKee Chun Siong
- GLM ModelsUploaded byManoj Paliwal
- Corrosion Prediction of Carbon Steel in.pdfUploaded bySalem Karrab
- Cash ManagementUploaded bynarayan
- 178-hw3Uploaded byjagaenator
- Great Parameter Estimation MonodUploaded byReza Mahoor
- QA03-Display Inspection Lot.docUploaded byBhupendra Singh
- An Overview of Gradient Descent Optimization AlgorithmsUploaded byAhmad Karlam
- Conditional Logistic Regression ModelsUploaded byds532
- 2005 10 Iccv LevenbergUploaded bySlamet Widodo
- Accountatnt as Source 4 Small BusinessUploaded bysiscani
- Honeywell Equipment Condition Monitor Performance Monitoring Offshore Gas Compressor Tin659Uploaded bytsiporn
- Machine Learning TermsUploaded byRam Prasad
- HW_17_ISB_2015Uploaded bycvikasgupta
- Subsidence prediction using regression modeUploaded byAudrey Robinson
- High Low MethodUploaded byCharles Kioko Mulonzi
- Children`s conceptual knowledge of lying.pdfUploaded byjelena
- 2896-7291-1-PBUploaded byPrasenjit Saha
- An Appraisal of the Factors that Influence Maintenance of Residential Buildings’ StandardsUploaded byAlexander Decker
- Setting Up Release Procedure for PR Without Cla..Uploaded bykanchan2chaudhari
- Hotel Industry in ThailandUploaded byshah_kinjal1990

- 03409_12.pptxUploaded bysweetie05
- Chapter2.1.pdfUploaded bysweetie05
- nmf_nature.pdfUploaded bysweetie05
- SSau ExercisesUploaded bysweetie05
- Chapter 14 Association Rules Collaborative FilteringUploaded bysweetie05
- Risk AnaliziUploaded bysweetie05
- Chap08Slides.pdfUploaded bysweetie05
- Chapter 02Uploaded bysweetie05
- Iet 603 Assignment 6Uploaded bysweetie05
- History, Complexity and Health Systems ResearchUploaded bysweetie05
- İNTEGRAL-FORMÜLLERİ.pdfUploaded bysweetie05
- logistic_regression.pdfUploaded bysweetie05
- OptimizationUploaded bysweetie05
- Statistics Exercises2Uploaded bysweetie05
- 6720 Labs Chapter 9Uploaded bysweetie05
- 6720 Labs Chapter 7Uploaded bysweetie05
- YTUJENS-2017-8-2.1835Uploaded bysweetie05
- 10.1.1.35.5072.pdfUploaded bysweetie05
- Exercise 7Uploaded bysweetie05
- Constraint ProgUploaded bysweetie05
- 6720 Labs Chapter 2Uploaded bysweetie05