TECHIN513 – Managing

Signal and Data Processing

Week 8
Today’s Agenda
• CNN
• YOLO
• ICTE
• FPDAWT
Today’s Agenda
• Convolutional Neural Network
• You Only Look Once
• In Class Team Exercise
• Final Project Discussion And Work Time
Announcement
• Purchasing supplies for final project
• Budget of $40 per team
• Requests must be made by Monday, February 26 at 9:59am

Link to Request Form:

TECHIN513 Final Project Supply Request Form - Google Sheets
What is a convolutional neural network?
• A network architecture for deep learning
• CNNs can have tens or hundreds of hidden layers
• Includes a typical artificial neural network architecture
• Useful for finding patterns in images to recognize objects
Stages of a CNN
• Input image
• Convolution
• Activation
• Pooling
• Flattening
• Fully Connected ANN
• Activation
image source

• Output

Convolutional Operations | Medium

 pixel values range
Greyscale Image Data from 0 to 255

24x16 matrix

How Do Machines Read and Store Images? | Analytics Vidhya

Color Image Data

one image has

three matrices or
pixel values range “channels”
from 0 to 255

How Do Machines Read and Store Images? | Analytics Vidhya

CNN Overview
Feature Extraction

Feature Extraction with CNNs | Towards Data Science

Typical Artificial Neural Network
• Each neuron in the input layer
is connected to a neuron in the
hidden layer
• Each connection has a weight
value
• Each neuron has a bias value
• The model learns these values
during the training process
• Values are updated with each
new training example

Introduction to Deep Learning - MATLAB

Typical Artificial Neural Network
• Each neuron in the input layer
is connected to a neuron in the
hidden layer
• Each connection has a weight
value
• Each neuron has a bias value
• The model learns these values
during the training process
• Values are updated with each
new training example

Introduction to Deep Learning - MATLAB

Convolutional Neural Network
• The weights and bias values are
the same for all neurons in a
hidden layer
• All hidden layers are detecting
the same feature (e.g. edge) in
different regions of an image
• The network is better equipped
to detect the feature regardless
of its location in an image

Introduction to Deep Learning - MATLAB

Convolutional Neural Network
• The weights and bias values are
the same for all neurons in a
hidden layer
• All hidden layers are detecting
the same feature (e.g. edge) in
different regions of an image
• The network is better equipped
to detect the feature regardless
of its location in an image

Introduction to Deep Learning - MATLAB

Convolutional Operation

An operation on two functions

which produces a third
combined function

Convolution Integral | Statistics How To

Convolutional Operation
kernel types

• A convolutional kernal is a
small 2D matrix
• The kernal maps on to the
input image by matrix
multiplication and addition
• The output is a matrix of
lower dimensions
Sliding window protocol
where stride =1

Lower dimension matrix

(feature map) Convolutional Operations | Medium
Convoluting to Create Feature Maps

CNNs | simplilearn
 45*0
+ 12*(-1)
+ 5*0
+ 22*(-1)
+ 10*5
+ 35*(-1)
+ 88*0
+ 26*(-1)
+ 51*0
= - 45
Activation Step Rectified
Linear
Unit
• Activation function takes the
output of a neuron and maps it
to the highest positive value
• If output is negative, the
function maps it to zero
• ReLU is a commonly used
activation function in deep
learning

Introduction to Deep Learning - MATLAB

ReLu activation retains only positive values

CNNs | simplilearn
CNN Overview
Pooling Step New
Feature
Map
• Pooling reduces dimensionality
of features map by using
different filters
• Condenses regions of neurons
into a single output
• Simplifies model by reducing
the number of parameters the
model needs to learn
• Pooling retains the most
important information but
lowers resolution

Introduction to Deep Learning - MATLAB

Pooling Applies Various Filters

CNNs | simplilearn
Pooling Enhances Edges Three iterations of
max pooling using a
(2, 2) kernel

Features (edges) are

enhanced, but
resolution is reduced

Pooling In Convolutional Neural Networks | paperspace

CNN Overview
Flattening
• The flatten layer lies
between the CNN and the
Softmax
ANN
• Converts the feature map
from the pooling layer into
an input that the ANN can
understand
• The ANN requires a one-
dimensional array as input
Artificial Neural Network

Feature Maps | educative.io , Dense layers | Pysource

Softmax Activation Step
Mathematical
representation
Last fully
• Often used as the last connected layer
activation function to
normalize the output of a
network to a probability
distribution over predicted
output classes
• The output of a Softmax is a
vector with probabilities of
each possible outcome.

Softmax Activation Function | Towards Data Science

CNN Output Layer
The final layer of the CNN architecture provides the final
classification output
A vector of length K
equal to the
number of classes

Introduction to Deep Learning - MATLAB

Classification, Detection, & Segmentation

or object localization

Object Segmentation vs. Object Detection | LinkedIn

You Only Look Once
• "You Only Look Once" (YOLO)
• YOLOv1 paper published May 2016
• Uses CNN as its backbone
network architecture
• YOLO predicts bounding boxes
and class probabilities for these
boxes simultaneously
• Improvement on previous model:
R-CNN

https://arxiv.org/abs/1506.02640
YOLO

https://pjreddie.com/darknet/yolo/

https://arxiv.org/abs/1506.02640
Previous Model for Image Detection: R-CNN
• Regions with CNN features
• Published Oct 2014
• link to article
• Splits an image into 2000
regions in boundary boxes
then classify each region
• Drawbacks:
• Long time to train – classify
2000 regions per image
• Detection not in real-time: 47
sec for test image
• Boundary box inaccuracies

R-CNN | Towards Data Science

How does YOLO work?
• Resizes the input image into YOLO Architecture
448x448
• A 1x1 convolution is first applied
to reduce the number of
channels
• 24 convolutional layers
• 4 max pooling layers
• The activation function is ReLU
• Two fully connected layers

https://arxiv.org/abs/1506.02640
What is Object
Detection?
First let’s talk about
object localization

36
What is object localization?
width (bw)
Object localization is
finding what and where a
(single) object exists in a
single image

height
(bh)

(bx, by)
 How is object localization described
numerically in YOLO?
• The coordinates of a bounding x_train

box are described as a vector

y_train

Pc 1
Probability Bx 0.5
of class By 0.6
Bw 0.4
Bh 0.3
C1 1
C2 0
C1 = car class
C2 = motorcycle class
 How is object localization described
numerically in YOLO? (0.5,0.6)
• The coordinates of a bounding (0,0) x_train

box are described as a vector

y_train

Pc 1 (bx,by)
Probability Bx 0.5 bh
of class By 0.6
0.3
Bw 0.4
Bh 0.3 bw
C1 1
C2 0 (1,1)
C1 = car class 0.4
C2 = motorcycle class
 How is object localization described
numerically in YOLO? (0.5,0.6)
• The coordinates of a bounding (0,0)
box are described as a vector

Output of
Neural Network

Pc 1 (bx,by)
Probability Bx 0.5 bh
of class By 0.6
0.3
Bw 0.4
Bh 0.3 bw
C1 0.97
C2 0.03 (1,1)
C1 = car class 0.4
C2 = motorcycle class
 How is object localization described
numerically in YOLO?
• The coordinates of a bounding x_train

box are described as a vector

y_train

Pc 0
Probability Bx -
of class By -
Bw -
Bh -
C1 -
C2 -
C1 = car class
C2 = motorcycle class
What about multiple objects?

YOLO algorithm | YouTube

What about multiple objects?

Pc 0
Bx -
By -
Bw -
Bh -
C1 -
C2 -

C1 = dog class
C2 = person class

YOLO algorithm | YouTube

What about multiple objects?
Person’s
object
belongs to
this cell

Pc 1
Bx 0.05
By 0.3
Bw 2
Bh 1.3
C1 1
C2 0

C1 = dog class
C2 = person class

YOLO algorithm | YouTube

What about multiple objects?

Pc 1
Bx 0.32
By 0.02
Bw 2.2
Bh 1.7
C1 0
C2 1

C1 = dog class
C2 = person class

YOLO algorithm | YouTube

What about multiple objects?

All other cells 4x4x7 matrix

Pc 0
Bx -
By -
Bw -
Bh -
C1 -
C2 -

C1 = dog class
C2 = person class

YOLO algorithm | YouTube

Training the YOLO Model

YOLO algorithm | YouTube

YOLO Prediction

YOLO algorithm | YouTube

Evaluating Image Detection Models
• Common Objects in Context
(COCO) dataset
• Published by Microsoft
• Used to evaluate algorithms’
performance of real-time
object detection
• 330,000 images
• 200,000 are labeled Pc 1

• 1.5 million object instances y_train

Bx
By
0.5
0.6
Bw 0.4
• 5 captions per image Bh
C1
0.3
1
C2 0

COCO Dataset | viso.ai

Evaluating Image Detection Models
Error Matrix

• Mean Average Precision (mAP)

• Benchmark metric used to
evaluate the robustness of
object detection models
• Incorporates mathematics image source

from:
• Error matrix
• Intersection over union (IoU)
ratio for bounding box

image source

Understanding Confusion Matrix | Towards Data Science

Best Object Detection Models

Object Detection | viso.ai

YOLOv8

YOLOv8 Tutorial - Colaboratory (google.com)

YOLOv8

Ultralytics YOLOv8 | GitHub

ICTE