A SIMPLE ANALOGY FOR NEURAL

NETWORKS
ANALOGY Shopping

+100%

+100%

-100%
Gaming

-100%
ANALOGY Shopping

+100%

+100%

-100%
Gaming

-100%
ANALOGY Shopping

+100%

+100%

-100%
Gaming

-100%
ANALOGY Shopping
Gaming %

+100%
? Girl or Boy?
Shopping %

+100%

-100%
Gaming

-100%
ANALOGY Shopping
Gaming %

+100%
? Girl or Boy?
Shopping %

+100%

-100%
Gaming

-100%
ANALOGY Shopping

+100%

+100%

-100%
Gaming

-100%
ANALOGY
𝑦 − 𝑦0 = 𝑚 𝑥 − 𝑥0

Shopping

+100%

+100%

Gaming
ANALOGY 𝑦 − 𝑦0 = 𝑚 𝑥 − 𝑥0
𝑦−0 =1 𝑥−0

Shopping

+100%

+100%

Gaming
ANALOGY 𝑦 − 𝑦0 = 𝑚 𝑥 −
𝑦 − 0 𝑥0
=1 𝑥−
0
𝑦=𝑥
Shopping

+100%

+100%

Gaming
ANALOGY 𝑦 − 𝑦0 = 𝑚 𝑥 −
𝑦 − 0 𝑥0
=1 𝑥−
𝑥 −0𝑦 = 0
𝑦=𝑥
Shopping

+100%

+100%

Gaming
ANALOGY 𝑦 − 𝑦0 = 𝑚 𝑥 − x%
𝑦 − 0 𝑥0
=1 𝑥−
x-y Gender?
y%
𝑥 −0𝑦 = 0
𝑦=𝑥 𝑥−𝑦> 0
1(𝑏𝑜𝑥)
Shopping 𝑔𝑒𝑛𝑑𝑒𝑟 = ቊ 0(𝑔𝑖𝑟𝑙) 𝑥 − 𝑦 < 0

+100%

+100%

Gaming
ANALOGY 1(𝑏𝑜𝑦) 𝑥 − 𝑦 > 0
𝑔𝑒𝑛𝑑𝑒𝑟 =
0(𝑔𝑖𝑟𝑙) 𝑥 − 𝑦 < 0
ቊ
𝑥 = −100, 𝑦 = 100
−100 − 100 → −200 < 0
Shopping
𝐺𝑒𝑛𝑒𝑟 = 𝑔𝑖𝑟𝑙

+100%

+100%

-100%
Gaming

-100%
ANALOGY 1(𝑏𝑜𝑦) 𝑥 − 𝑦 > 0
𝑔𝑒𝑛𝑑𝑒𝑟 =
0(𝑔𝑖𝑟𝑙) 𝑥 − 𝑦 < 0

𝑥 = 10, 𝑦 = −5
10 − −5 = 15 → 15 >
Shopping
0
𝐺𝑒𝑛𝑒𝑟 = 𝑏𝑜𝑦
+100%

+100%

-100%
Gaming

-100%
ANALOGY 1(𝑏𝑜𝑦) 𝑥 − 𝑦 > 0
𝑔𝑒𝑛𝑑𝑒𝑟 =
0(𝑔𝑖𝑟𝑙) 𝑥 − 𝑦 < 0
ቊ
𝑥 = 10, 𝑦 = 60
10 − 60 = −50 → −50 < 0
Shopping
𝐺𝑒𝑛𝑒𝑟 = 𝑔𝑖𝑟𝑙

+100%

+100%

-100%
Gaming

-100%
ANALOGY 1(𝑏𝑜𝑦) 𝑥 − 𝑦 > 0
𝑔𝑒𝑛𝑑𝑒𝑟 =
0(𝑔𝑖𝑟𝑙) 𝑥 − 𝑦 < 0

𝑥 = 150, 𝑦 = 0
150 − 0 = 150 → 150 > 0
Shopping
𝐺𝑒𝑛𝑒𝑟 = 𝑏𝑜𝑦

+100%

+100%

-100%
Gaming

-100%
 LET’S BE MORE
MATHEMATICA
L
 Shopping

+100%

+100%

-100%
Gaming

-100%
LET’S BE MORE Shopping

+100%
𝑝1

f 𝑜

MATHEMATICA
+
1
0
0

𝑝2
%

-100%
Gaming

-100%

L
𝑦−𝑥>0
𝑜 = 𝑓 𝑥, 𝑦 =
0 𝑦−𝑥<
1
ቊ 0
LET’S BE MORE Shopping

+100%
𝑝1

f 𝑜

MATHEMATICA
+
1
0
0

𝑝2
%

-100%
Gaming

-100%

L
𝑦−𝑥>0
𝑜 = 𝑓 𝑥, 𝑦 = 0 𝑦 − 𝑥 <
1 Let's use weights
0

𝑝1𝑤1 + 𝑝2𝑤2 > 0

𝑜 = 𝑓 𝑝1 , 𝑝2 =
1 0 𝑝1𝑤1 + 𝑝2𝑤2 < 0
ቊ

Let's use 𝒑𝟏 , 𝒑𝟐
LET’S BE MORE Shopping

+100%
𝑝1

f 𝑜

MATHEMATICA
+
1
0
0

𝑝2
%

-100%
Gaming

-100%

L 𝑝1𝑤1 + 𝑝2𝑤2 > 0

𝑜 = 𝑓 𝑝1 , 𝑝2 =
1 0 𝑝1𝑤1 + 𝑝2𝑤2 < 0
ቊ
LET’S BE MORE Shopping

+100%
𝑝1

f 𝑜

MATHEMATICA
+
1
0
0

𝑝2
%

-100%
Gaming

-100%

L 𝑝1𝑤1 + 𝑝2𝑤2 > 0

Instead of 0, let’s use 𝒃
𝑜 = 𝑓 𝑥1 , 𝑥2 =
1 0 𝑝1𝑤1 + 𝑝2𝑤2 < 0
ቊ

𝑝1𝑤1 + 𝑝2𝑤2
𝑓 𝑝1 , 𝑝2 =
1 > 𝑏 0 𝑝1𝑤1 + 𝑝2𝑤2
ቊ
<𝑏
LET’S BE MORE Shopping

+100%
𝑝1

f 𝑜

MATHEMATICA
+
1
0
0

𝑝2
%

-100%
Gaming

-100%

L 𝑝1𝑤1 + 𝑝2𝑤2 > 0

𝑜 = 𝑓 𝑝1 , 𝑝2 =
1 0 𝑝1𝑤1 + 𝑝2𝑤2 < 0
ቊ

𝑝1𝑤1 + 𝑝2𝑤2
𝑓 𝑝1 , 𝑝2 =
1 > 𝑏 0 𝑝1𝑤1 + 𝑝2𝑤2
ቊ
<𝑏

1 𝑝1𝑤1 + 𝑝2𝑤2 − 𝑏 > 0

𝑓 𝑝1 , 𝑝2 =
ቊ 0 𝑝1𝑤1 + 𝑝2𝑤2 − 𝑏 < 0
LET’S BE MORE Shopping

+100%
𝑝1

f 𝑜

MATHEMATICA
+
1
0
0

𝑝2
%

-100%
Gaming

-100%

L 𝑝1𝑤1 + 𝑝2𝑤2 > 0

𝑜 = 𝑓 𝑝1 , 𝑝2 =
0 𝑝1𝑤1 + 𝑝2𝑤2 < 0
ቊ
1 Let’s use additions
only since 𝒃 can
1 𝑝1 𝑤1 + 𝑝2 𝑤2 > 𝑏
be negative too
𝑓 𝑝1, 𝑝2 =
0 𝑝1𝑤1 + 𝑝2𝑤2 < 𝑏
ቊ

1 𝑝1𝑤1 + 𝑝2𝑤2 + 𝑏 > 0

𝑓 𝑝1 , 𝑝2 =
ቊ 0 𝑝1𝑤1 + 𝑝2𝑤2 + 𝑏 < 0
LET’S BE MORE Shopping

+100%
𝑝1

f 𝑜

MATHEMATICA
+
1
0
0

𝑝2
%

-100%
Gaming

-100%

1 𝑝1𝑤1 + 𝑝2𝑤2 + 𝑏 > 0

𝑜 = 𝑓 𝑝1 , 𝑝2 =
0 𝑝1𝑤1 + 𝑝2𝑤2 + 𝑏 < 0
ቊ
LETS BUILD A MODEL WITH THIS EQUATION
Shopping

+100%

+
1
0
0
%

-100%
Gaming

-100%

𝑝1𝑤1 + 𝑝2𝑤2 +𝑏> 0

𝑜 = 𝑓 𝑝1 , 𝑝2 =
0 𝑝1𝑤1 + 𝑝2𝑤2 +𝑏< 0
1
ቊ
LETS BUILD A MODEL WITH THIS EQUATION
Shopping

+100%

+
1
0
0
%

-100%
Gaming

-100%

𝟏 𝒑𝟏 𝒘𝟏 + 𝒑𝟐 𝒘𝟐 + 𝒃 > 𝟎
𝒐 = 𝒇 𝑝1 , 𝑝2 =
ቊ 𝟎 𝒑𝟏 𝒘𝟏 + 𝒑𝟐 𝒘𝟐 + 𝒃 < 𝟎

𝑝1 𝑤1

∑ 𝑓 𝑜

𝑤2
𝑝2 𝑏
NUMERICAL EXAMPLE
Shopping

+100%

+
1
0
0
%

-100%
Gaming

-100%

𝑝1 ∗ 1 + 𝑝2 ∗ −1 +0> 0
𝑓 𝑝1 , 𝑝2 =
0 𝑝1 ∗ 1 + 𝑝2 ∗ −1 +0< 0
1
ቊ

𝑝1 1

∑ 𝑓 𝑜

−1
𝑝2 0
NUMERICAL EXAMPLE
Shopping

+100%

+
1
0
0
%

-100%
Gaming

-100%

10,2 girl or
boy?

𝑝1 ∗ 1 + 𝑝2 ∗ −1 +0> 0
𝑓 𝑝1 , 𝑝2 =
0 𝑝1 ∗ 1 + 𝑝2 ∗ −1 +0< 0
1
ቊ

𝑝1 1

∑ 𝑓 𝑜

−1
𝑝2 0
NUMERICAL EXAMPLE
Shopping

+100%

+
1
0
0
%

-100%
Gaming

-100%

10,2 girl or
boy?

𝑝1 ∗ 1 + 𝑝2 ∗ −1 +0> 0
𝑓 𝑝1 , 𝑝2 =
0 𝑝1 ∗ 1 + 𝑝2 ∗ −1 +0< 0
1
ቊ

10 𝑝1 1

∑ 𝑓 𝑜 1

−1
2 𝑝2 0
NUMERICAL EXAMPLE
Shopping

+100%

+
1
0
0
%

-100%
Gaming

-100%

2,10 girl or
boy?

𝑝1 ∗ 1 + 𝑝2 ∗ −1 +0> 0
𝑓 𝑝1 , 𝑝2 =
0 𝑝1 ∗ 1 + 𝑝2 ∗ −1 +0< 0
1
ቊ

𝑝1 1

∑ 𝑓 𝑜

−1
𝑝2 0
NUMERICAL EXAMPLE
Shopping

+100%

+
1
0
0
%

-100%
Gaming

-100%

2,10 girl or
boy?

𝑝1 ∗ 1 + 𝑝2 ∗ −1 +0> 0
𝑓 𝑝1 , 𝑝2 =
0 𝑝1 ∗ 1 + 𝑝2 ∗ −1 +0< 0
1
ቊ

2 𝑝1 1

∑ 𝑓 𝑜 0

−1
10 𝑝2 0
WHAT YOU SAW JUST NOW WAS
THE BUILDING BLOCK FOR ALL
NEURAL NETWORKS
𝑝1 𝑤1

∑ 𝑓 𝑜

𝑤2
𝑝2 𝑏
IMPORTANT TERMINOLOGIES
 Shopping

𝑝1
𝑤1
+100%

∑ 𝑓 𝑜
+1
00

-100%
%
𝑝2 𝑤2 𝑏

-100%

1- Features 1- Features (inputs)

𝑝1 𝑝2 𝑝3 …

Example:

• Number of sales: 1000

• Age: 23
• Height: 1.5
• Price: 12
• Pixel colour: 255
• Angle: 360
• Weight: 70
• Hair length: 0.1
• Temperature: 100
• …
 Shopping

𝑝1
𝑤1
+100%

∑ 𝑓 𝑜
+1
00

-100%
%
𝑝2 𝑤2 𝑏

-100%

2- Class 2- Class

0 or 1
 Shopping

𝑝1
𝑤1
+100%

∑ 𝑓 𝑜
+1
00

-100%
%
𝑝2 𝑤2 𝑏

-100%

3- Training (learning)
? 3- Training (learning)

𝑤1 = ?
𝑤2
=?
𝑏=?
✓
 Shopping

𝑝1
𝑤1
+100%

∑ 𝑓 𝑜
+1
00

-100%
%
𝑝2 𝑤2 𝑏

-100%

4- Training samples 4- Training samples

10,
−10
Training set
−50,
−2, 19 Training set

200
…
 Shopping

𝑝1
𝑤1
+100%

∑ 𝑓 𝑜
+1
00

-100%
%
𝑝2 𝑤2 𝑏

-100%

5- Test samples 5- Test samples

30,
−20,
−1
Test set
88 Test set

…
 Shopping

𝑝1
𝑤1
+100%

∑ 𝑓 𝑜
+1
00

-100%
%
𝑝2 𝑤2 𝑏

-100%

6- Data set 6- Data set

Training set
Training set

Test set
Test set
𝑝1
𝑤1
7- Weights or connection weights
∑ 𝑓 𝑜
𝑤1, 𝑤2
𝑝2 𝑤2
𝑏
𝑝1
𝑤1
8- bias
∑ 𝑓 𝑜
𝑏
𝑝2 𝑤2
𝑏
𝑝1
𝑤1
9- Summation unit
∑ 𝑓 𝑜

𝑝2 𝑤2
𝑏
∑

𝑝1𝑤1 + 𝑝2𝑤2
 𝑝1
𝑤1
10- Activation (transfer) function
∑ 𝑓 𝑜

𝑝2 𝑤2
𝑏
𝑓

1 𝑝1𝑤1 + 𝑝2𝑤2 + 𝑏 > 0

𝑓 𝑝1 , 𝑝2 =
ቊ 0 𝑝1𝑤1 + 𝑝2𝑤2 + 𝑏 <
0