Future Institute of Technology ,Kolkata

DEEP LEARNING, 2024

PCCAIML 602
Report On

CALCULATIONS OVER PERCEPTRON

Author: Shrutanik Chatterjee

Course Instructor: Dr. Pradipta Kr.Banerjee

Department of CSE (AI & ML)

University Roll No: 34230822046

University Registration No: 223420120181of 2022-23

6th Semester
Contents
1 Schematic Representation of Problem: 3

2 Pre-Requisites of Problem: 3
∂𝐿
3 Find the Sum of Elements for : ∂𝑤 4
3

3.1 Computation of 𝑧1 and 𝑎1 : . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3.2 Computation of 𝑧2 , 𝑎2 and 𝑧3 : . . . . . . . . . . . . . . . . . 5

∂𝐿
3.3 Computation of :...............................6
∂𝑤3

∂𝐿
3.4 Sum of all the Elements of ∂𝑤 : . . . . . . . . . . . . . . . . . . . . . . . . . 6
3

∂𝐿
4 Find the Sum of Elements for : ∂𝑤 7
2

∂𝐿
4.1 Computation of ∂𝑤 : . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2

4.2 Mathematical Computation: . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

∂𝐿
4.3 Sum of all the Elements of :.........................8
∂𝑤2

4.4 Hadamard product (matrices): . . . . . . . . . . . . . . . . . . . . . . . . . . 8

∂𝐿
5 Find the Sum of Elements for : 𝟗
∂𝑤1

∂𝐿
5.1 Computation of ∂𝑤 : . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1

5.2 Mathematical Computation: . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

∂𝐿
5.3 Sum of all the Elements of : . . . . . . . . . . . . . . . . . . . . . . . . . 10
∂𝑤1
1 Schematic Representation of
Problem:

2 Pre-Requisites of Problem:
From the above diagram, we can say that,

𝑧 (1) = 𝜔1 𝑥
𝑎(1) = 𝜎(𝑧 (1) )
𝑧 (2) = 𝜔2 𝑎(1)
𝑎(2) = 𝜎(𝑧 (2) )
𝑧 (3) = 𝜔3 𝑎(2) = 𝑦ˆ
𝑦ˆ = 𝑎(3) = 𝑧 (3) = Linear activation.
1 1 2
1 1 2
𝑤1 = [3 1 1] 𝜔2 = [ ] 𝜔 = [2 5]
3 1 1 3
1 2 3
1
𝑥 = [1]
1

3 Find the Sum of Elements for : ∂𝐿

∂𝑤3
3.1 Computation of 𝑧1 and 𝑎1 :
 1 1 21 1
𝑧1 = 𝜔1 𝑥 = [3 1 1] [ 1] ⋅
1 2 3 3×3 1 3×1
|𝑥| + |𝑥| + 2𝑥 ∣ 1+1+2 4
=[ 3𝑥| + |𝑥| + |𝑥 ∣ ] = [3 + 1 + 1] = [5]
|𝑥| + 2𝑥| + 3𝑥| 3𝑥1 1 + 2 + 3 3𝑥∣ 6 3×1

0.982
= [0.993]
0.997
(1)
Here, 𝑎1 = 0.982

(1)
𝑎2 = 0.993
(1)
𝑎3 = 0.997

3.2 Computation of 𝑧2 , 𝑎2 and 𝑧3 :

0.982
1 1 2
𝑧2 = 𝑤2 𝑎(1) =[ ] + [0.993]
3 1 1 2×3
0.997 3×1
1 × 0.982 + 1 × 0.993 + 2 × 0.997
=[ ]
3 × 0.982 + 1 × 0.993 + 1 × 0.997 2×1
0.982 + 0.993 + 1.994 3.969
=[ ] [ ] .
2.946 + 0.993 + 0.997 2×1 4.936 2×1
3.969 1 1 1
𝑎2 = 𝜎(𝑧 (2) ) =𝜎 [ ] > 𝜎(3.969) ≈ −3.969
≈ ≈ ≈ 0.981
4.936 1+𝑒 1 + 0.018 1.018
0.981
=[ ]
0.993
(2)
Here, 𝑎1 = 0.981

(2)
𝑎2 = 0.993
2 × 0.981
0.981
𝑧3 = 𝑤˙3 𝑎(2) = [2 5] [ ] =[ + ]
0.993 1×2
5 × 0.993
= [6.927] = 𝑌ˆ
∂𝐿
3.3 Computation of ∂𝑤 :
3
 𝐿 = (𝑌ˆ − 𝑌)2
∂𝐿 ∂𝐿 ∂𝑌ˆ
= ×
∂𝜔3 ∂𝑦ˆ Δ𝜔 3
∂ ∂
= (𝑌ˆ − 𝑌)2 × 3 (𝑊3 𝑎(2) )
∂𝑌ˆ 𝜔
= 2(𝑌ˆ − 𝑌) × 𝑎(2)
∂𝐿
3.4 Sum of all the Elements of ∂𝑤 :
3

∂𝐿
Sum of all the elements of ∂𝑤 = 2[𝑌ˆ − 𝑌] × 𝑎(2)
3

0.981
= 2[6.927 −10] × [ ]
0.993
0.981
= 2[−3.073] × [ ]
0.993
0.981
= −6.146 × [ ]
0.993
−6.029
=[ ]
−6.102
∂𝐿
∵ = −6.029 + (−6.102)
∂𝜔3
= −12.131

4 Find the Sum of Elements for : ∂𝐿

∂𝑤2
∂𝐿
4.1 Computation of ∂𝑤 :
2

∂𝐿 ∂𝐿 ∂𝑦ˆ (2) ∂𝑎(2) ∂𝑧 (2)

= × × ×
∂𝜔2 ∂𝑦ˆ ∂𝑎(2) ∂𝑧 (2) ∂𝜔2
∂ ∂ ∂ ∂
= (𝑦ˆ − 𝑌)2 × (2) (𝜔3 𝑎(2) ) × (2) (𝜎(𝑧 (2) )) × (𝜔 𝑎(1) )
∂𝑦 ∂𝑎 ∂𝑧 ∂𝜔2 2
(2)
=⏟
2(𝑦ˆ − 𝑦) × 𝜔
⏟3 × {𝜎(𝑧
⏟ ) × [1 − 𝜎(𝑧 (2) ))} × 𝑎⏟
(1)

1×1 1×2 3×1

2×1

Loss is a scalar quantity → size is 1

4.2 Mathematical Computation:
0.981
𝜎(𝑧 (2) ) = 𝑎(2) = [ ]
0.993
0.019
1 − 𝜎 (𝑧 (2)) = [ ]
0.007

∵ Both the matrix are of equal dimension → Hadamard Product

𝜎(𝑧 (2) ) ⊙ [1 − 𝜎(𝑧 (2) )]

0.981 0.019
=[ ][ ]
0.993 0.007
0.0186
=[ ]
0.007
2 2 0.0186
𝜔3⊤ = [ ] ⊙ {𝜎(𝑧 (2) ): (1 − 𝜎(𝑧 (2) ))} = [ ] [ ]
5 5 0.007
[Hadamard Product Applied]
0.037
=[ ]
0.035
𝑎1 (𝑇) = [0.982 0.993 0.997]:
Now 𝜔3⊤ ⊙ {𝜎(𝑧 (2) ) ⋅ (1 − 𝜎(𝑧 (2) ))} × 2(𝑌ˆ − 𝑌)
0.037 −0.227
=[ ] [−6.146] = [ ]
0.035 −0.215
∂𝐿
4.3 Sum of all the Elements of :
∂𝑤2

⊤
𝑤3⊤ ⊙ {𝜎(𝑧 (2) ) ⋅ (1 − 𝜎(𝑧 (2) ))} × 2(𝑌ˆ − 𝑌) × 𝑎(1)
−0.227 [
=[ ] 0.982 0.993 0.997]
−0.215
−0.223 −0.225 −0.226 ∂𝐿
=[ ]=
−0.211 −0.213 −0.214 ∂𝜔2
∂𝐿
∴ Sum of all elements for ∂𝜔
2

= {(−0.223) + (−0.225) + (−0.226) + (−0.211) + (−0.213) + (−0.214)}

= −1.312

4.4 Hadamard product (matrices):

In mathematics, the Hadamard product (also known as the element-wise product,
entrywise product or Schur product) is a binary operation that takes in two matrices of
the same dimensions and returns a matrix of the multiplied corresponding elements.
This operation can be thought as a "naive matrix multiplication" and is different from the
matrix product. It is attributed to, and named after, either French mathematician
Jacques Hadamard or German mathematician Issai Schur.
∂𝐿
5 Find the Sum of Elements for : ∂𝑤
1
 ∂𝐿
5.1 Computation of :
∂𝑤1

∂𝐿 ∂𝐿 ∂𝑦ˆ ∂𝑎(2) ∂𝑧 (2) ∂𝑎(1) ∂𝑧 (1)

= × (2) × (2) × (1) × (1) ×
∂𝜔𝐿 ∂𝛾ˆ ∂𝑎 ∂𝑧 ∂𝑎 ∂𝑧 ∂𝜔1
=2(𝛾ˆ − 𝑦) × 𝜔3 × {𝜎(𝑧 (2) ) ⋅ (1 − 𝜎(𝑧 (2) ))} × 𝜔2
2×1
× {𝜎(𝑧 (1) ) ⋅ (1 − 𝜎(𝑧 (1) ))} × 𝑥3×1

Let 2(𝑦ˆ − 𝑦) × 𝜔3 × {𝜎(𝑧 (2) ) ⋅ (1 − 𝜎(𝑧 (2) ))} = 𝑝. which is 2 × 1 and is having.
Hadamard product properties

1 1 2
Let 𝑤2 = 𝑞 which is 2 × 3. [ ] Let {𝜎(𝑧 (1) ) ⋅ (1 − 𝜎(𝑧 (1) )} = 𝑟 which is 3 × 1 having
3 1 1
Hadamard Product properties

1
Let 𝑥 = which is 3 × 1 [1] and is having Hadamard Product
1

∴ 𝑟 × 𝑠 = (3 × 1) ⊙ (3 × 1) = (3 × 1) → Hadamard Product

𝑞 × 𝑟𝑠 = (2 × 3) × (3 × 1) = (2 × 1)
𝑝 = (2 × 1) 𝑞 × 𝑟𝑠 = (2 × 1)

∴ 𝑃 and 𝑞 × res is also having Hadamard Product Properties

∴ 𝑝 ⊙ 𝑞𝑟𝑠 → 𝑝 ⊙ 𝑞 × 𝑟 ⊙ 𝑠 = (2 × 1)

5.2 Mathematical Computation:

0.982 0.018 0.0176
→ {𝜎(𝑧 (1) ) ⊙ (1 − 𝜎(𝑧 (1) ))} = [0.993] [0.007] = [0.0069]
0.997 0.003 0.0029
0.0176 1 0.0176
⟶ {𝜎(𝑧 (1) )𝒪 (1 − 𝜎(𝑧 (1) ))} ⊙ 𝑥 = [0.0069] [1] = [0.0069]
0.0029 1 0.0029
0.0176
1 1 2 0.0303
⟶ 𝜔2 × {𝜎(𝑧 (1) ) ⊙ (1 − 𝜎(𝑧 (1) ))} ⊙ 𝑥 = [ ] × [0.0069] =[ ]
3 1 1 2×3 0.0626 2×1
0.0029 3×1
1 × 0.0176 + 1 × 0.0069 + 2 × 0.0029
=[ ]
3 × 0.0776 + 1 × 0.0069 + 1 × 0.0029 2×1
∂𝐿
5.3 Sum of all the Elements of ∂𝑤 :
1

∂𝑙
sum of all the elements of ∂𝜔
1
 ∂𝐿 −0.227 0.0303
∴ =[ ][ ] ← Applying Hadamard product
∂𝜔1 −0.215 0.0626
−0.0068
=[ ]
−0.0134
∂𝐿
∴ sum of all the elements of ∂𝜔 = (−0.0068) + (−0.0134)
1

= −0.02