ME-438

AI AND INTERNET OF THINGS

ELECTIVE COURSE

NED University of Engineering & Technology

1
THIS WEEK

 Introduction to Classification
 Decision boundary
 Confusion Matrix
 Cost Function
 Logistic Regression Example - Python

DR. HAIDER ALI AI and Internet of Things 2

 INTRODUCTION TO CLASSIFICATION

DR. HAIDER ALI AI and Internet of Things 3

 CLASSIFICATION

Question Answer “y”

Is this email spam? No Yes
Is the transaction fraudulent?No Yes
Is the tumor malignant? No Yes
false true
y can only be one of two values 0 1
“binary classification”

DR. HAIDER ALI AI and Internet of Things 4

 EXAMPLE Decision Decision
boundary boundary

Yes “1”

Threshold
“0.5”

No “0”
Tumor size x
If fw,b(x) < 0.5 → y = 0 “Logistic Regression”
If fw,b(x) ≥ 0.5 → y = 1
DR. HAIDER ALI AI and Internet of Things 5
 EXAMPLE Decision Want outputs between 0 and 1
boundary

Yes “1”

Threshold
“0.7”

No “0” z
Tumor size x
Sigmoid function
1 Logistic function
𝑔(𝑧) = −𝑧
0<𝑔(𝑧)<1 Output between 0 and 1
1+𝑒
DR. HAIDER ALI AI and Internet of Things 6
 Want outputs between 0 and 1

𝑓𝑤,𝑏
ഥ (𝑥)
ҧ

𝑧 = 𝑤.
ഥ 𝑥ҧ + 𝑏

1
𝑔(𝑧) =
1 + 𝑒 −𝑧

1
𝑓𝑤,𝑏
ഥ 𝑥ҧ = 𝑔(𝑤.
ഥ 𝑥ҧ + 𝑏) = ҧ
ഥ 𝑥+𝑏)
z
1 + 𝑒 −(𝑤.

1 Logistic Regression
Sigmoid function
𝑔(𝑧) = −𝑧 0<𝑔(𝑧)<1 Logistic function
1+𝑒
DR. HAIDER ALI AI and Internet of Things 7
INTERPRETATION OF LOGISTIC REGRESSION OUTPUT
1
𝑓𝑤,𝑏
ഥ 𝑥ҧ = −( ഥ
𝑤. ҧ
𝑥+𝑏) 𝑓𝑤,𝑏
ഥ 𝑥ҧ = 𝑃(𝑦 = 1|𝑥;ҧ 𝑤,
ഥ 𝑏)
1+𝑒
“probability” that class is 1 Probability that y is 1,
given input 𝑥,ҧ parameters 𝑤,
ഥ 𝑏
Example
x is tumor size
y is 0 (tumor is not malignant) 𝑃 𝑦 =0 +𝑃 𝑦 =1 =1
y is 1 (tumor is malignant)

𝑓𝑤,𝑏
ഥ 𝑥ҧ = 0.7
70% chance that y is 1
DR. HAIDER ALI AI and Internet of Things 8
PLOT OF LOGISTIC FUNCTION - PYTHON

DR. HAIDER ALI AI and Internet of Things 9

DECISION-BOUNDARY
1
𝑓𝑤,𝑏
ഥ 𝑥ҧ = 𝑔(𝑤.
ഥ 𝑥ҧ + 𝑏) = ҧ
ഥ 𝑥+𝑏)
1 + 𝑒 −(𝑤.

𝑓𝑤,𝑏
ഥ 𝑥ҧ = 𝑃(𝑦 = 1|𝑥;ҧ 𝑤,
ഥ 𝑏)

Is 𝑓𝑤,𝑏
ഥ 𝑥ҧ > 0.5? (threshold)
Yes: 𝑦 = 1 No: 𝑦 = 0

When is 𝑓𝑤,𝑏
ഥ 𝑥ҧ > 0.5?
𝑔 𝑧 ≥ 0.5
𝑧≥0
𝑤.
ഥ 𝑥ҧ + 𝑏 ≥ 0 𝑤.
ഥ 𝑥ҧ + 𝑏 < 0
𝑦=1

DR. HAIDER ALI AI and Internet of Things 10

 𝑓𝑤,𝑏
ഥ 𝑥ҧ = 𝑔(𝑧) = 𝑔(𝑤1 𝑥1 + 𝑤2 𝑥2 + 𝑏)
DECISION-BOUNDARY 𝑧 = 𝑥1 + 𝑥2 − 3 = 0
𝑥1 + 𝑥2 = 3
x2
y=1
3
y=0
2

1 2 3 x1
DR. HAIDER ALI AI and Internet of Things 11
NON-LINEAR DECISION-BOUNDARY
x2
𝑓𝑤,𝑏
ഥ 𝑥ҧ = 𝑔(𝑧) = 𝑔(𝑤1 𝑥12 + 𝑤2 𝑥22 + 𝑏)

1
𝑧 = 𝑥12 + 𝑥22 − 1

x1
-1 1 𝑥12 + 𝑥22 = 1

-1

DR. HAIDER ALI AI and Internet of Things 12

Confusion Matrix

Actual

Positive Negative
Negative Positive

Ture False
Predicated

Positive Positive

False Ture
Negative Negative

DR. HAIDER ALI AI and Internet of Things 13

 Actual
Positive Negative

Predicated
Positive Ture Positive (TP) False Positive(FP)

False Negative
Negative Ture Negative (TN)
(FN)

PRECISION
ACCURACY

𝑇𝑃 + 𝑇𝑁 𝑇𝑃
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = Precision =
𝑇𝑃 + 𝑇𝑁 + 𝐹𝑁 + 𝐹𝑃 𝑇𝑃 + 𝐹𝑃
DR. HAIDER ALI AI and Internet of Things 14
 COST FUNCTION

DR. HAIDER ALI AI and Internet of Things 15

COST FUNCTION

i = 1,…,m Training example

j = 1,…,n Features

X1 Xn y Target y is 0 or 1
i=1
1
𝑓𝑤,𝑏
ഥ 𝑥ҧ = ҧ
ഥ 𝑥+𝑏)
1 + 𝑒 −(𝑤.

i=m

How to choose 𝑤
ഥ = [𝑤1 𝑤2 … 𝑤𝑛 ] and b?

DR. HAIDER ALI AI and Internet of Things 16

COST FUNCTION

Loss

DR. HAIDER ALI AI and Internet of Things 17

COST FUNCTION

log(f)

-log(f)

Loss → 0
0 then Loss → ∞
DR. HAIDER ALI AI and Internet of Things 18
COST FUNCTION

DR. HAIDER ALI

1 then Loss → ∞ AI and Internet of Things 19
SIMPLIFIED COST FUNCTION

DR. HAIDER ALI AI and Internet of Things 20

LOGISTIC REGRESSION - PYTHON

 Please see the Python Code

DR. HAIDER ALI AI and Internet of Things 21

 THANK YOU

DR. HAIDER ALI AI and Internet of Things 22