Adv.

Computer Vision

Term Project Presentation:

One-shot Image Classification

Naufal Suryanto
(201999137)
 Table of
Background
01 One common problem in Contents
Computer Vision Task

One-Shot Learning
02 Learning from a single
training example

Siamese Network
03 Framework for learning the Face emotion recognition
similarity 2nd Presentation

Siamese Losses
04 Loss function for Siamese
network
 Background

Deep Convolutional Neural Network have become the

state-of-the-art methods for image classification tasks.
However, one of the biggest limitations is they require a
lots of labelled data.

CIFAR-10 IMAGENET
MNIST

10 Classes
60.000/10.000 train/test
10 Classes >20.000 Classes
50.000/10.000 train/test > 14.000.000 images
 Background
How if we just have a single training example for each
label ?
We will need to collect a lot of different images for each person

Training Sample Output Layer

Elon Musk
Mark Zuckerberg
Deep Convolution Neural Bill Gates
Network
Stave Jobs

Does it works?
 Background
How if we want to add new label to the model ?

Training Sample Output Layer

Elon Musk
Mark Zuckerberg
Deep Convolution Neural Bill Gates
Network
Stave Jobs

Jeff Bozos (new

label)
Retraining the
model ?
One Shot
Learning
 One-shot Learning
Classification task where one, or a few, examples are
used for each class to classify many new examples in
future
Test Sample Training Sample

Simila Not
r Similar
 One-shot Learning
Learning a “similarity” function
 
d(img1, img2) = degree of difference between images

If d(img1,img2) threshold : “same”

> threshold: “different”
Test Sample Training Sample

d(x1,y1) =
0.9
1 ) =
, y
d(x2 1
0. d(x1,
y2) =
0 .0 5
d(x2,y2) =
0.9
d(x1,
y3) =
d(x2,y3) = 1. 0
1.2

Simila Not
r Similar
 One-shot Learning
If we can develop NN as a good distance function, we
will be able to solve this one-shot learning problem

Deep Convolution Neural Distance Score:

Network as Distance Function 0.9 (Different)

Deep Convolution Neural Distance Score:

Network as Distance Function 0.05 (Similar)
Siamese
Neural
Network
 Siamese Neural Network

A Neural Network that uses the same weights while

working in tandem on two different input vectors to
compute comparable output vectors.

Embedding vector

Deep Convolution Neural

Network
Distance
Shared Score
Calculation
weight
Deep Convolution Neural
Network
 Siamese Neural Network

Embedding vector = Features vector

A good model must output extracted features which can
differentiate between input

Embedding vector

Examples:
Deep Convolution Neural - Roundness of face
Network - Distance between eyes
- Nose width
- Check bone structure
- Jaw line length
- etc
 Siamese Neural Network
Pros:

• More Robust to class Imbalance: With

the aid of One-shot learning, given a
few images per class is sufficient for
Siamese NetworksCons: to recognize those
images
quadratic pairs to in
learn the
from (tofuture
• Needs more training time than normal networks: Since Siamese Networks involves
see all information available) it is slower than normal
classification type of learning (pointwise learning)
•• Doesn’t
Can be
output combine
probabilities: Since with
training the
involves best
pairwise learning, it won’t output
the probabilities of the prediction, but the distance from each class
classifier: Given that its learning
mechanism is somewhat different from
Siamese Loses
 Siamese Loses
1. Contrastive Loss

it will take two inputs data to

compare at each time step. These
two-input data could either be similar
Embedding vector

Deep Convolution Neural

or dissimilar.NetworkThis is modelled by a
Distance
binary class weightvariable Y whose
Shared
valuesLoss(d,y)
Calculation
Deep Convolution Neural
are: Network

- 0 if dissimilar
Siamese Loses
1. Contrastive Loss
 Siamese Loses
2. Triplet Loss

It will take three inputs data to

compare at each time step. Oppositely
Embedding vector
Deep Convolution Neural
Pos
to the Contrastive
Network Loss, Distance
Shared

Ancho
the inputs are intentionally
weight
Deep Convolution
Network
Neural
Calculation
Loss
L(d1,d2)
r
sampled regarding Shared
weight
their class:
Distance
Calculation
Deep Convolution Neural
Neg
• We sample an anchor object, used as
Network
 Siamese Loses
2. Triplet Loss

d2 d2
d1 d1
 Siamese Loses
Training strategies
Offline Mining Online Mining

• Find the training pairs at • Find the training pairs on

the beginning of process the fly (some pairs maybe
invalid)

CNN

…
 NEXT:
Face emotion recognition