Classifictaion Approaches - Algorithms II

ML Classification II
Agenda
Linear Classification
Overview
Linear classifier
References
Overview
Linear classifier
References
Contact Information
Dept. of Computer Science

19 Helwan University
Farmer Example: Nearest neighbor
2 Overview Linear
classifier
References
) The classifier must remember all of the training data and

store it for future comparisons with the test data. This is
space inefficient.
) Classifying a test image is expensive since it requires a
comparison to all training images. 19
Helwan University
Farmer Example: Linear Classification
3 Overview Linear
classifier
References

4 Overview Linear
classifier
References

5 Overview Linear
classifier
References

Linear classifier: parametric approach
Overview
We will start with the simplest possible function, a 6 Linear classifier
References
linear mapping:
f (xi , W, b) = Wxi + b
) Image xi has all of its pixels flattened out to a single
column vector of shape [Dx1].
) The matrix W (of size [KxD]), and the vector b (of size
[Kx 1]) are the parameters of thefunction.
) The parameters in W are often called the weights, and b
is called the bias vector because it influences the output
scores, but without interacting with the actual data xi .

Linear classifier: Example
Overview
7 Linear classifier
References
for 10 classes

Linear classifier: Example
Overview
Linear classifier
References
8
) Input Data is fixed, we have control over the setting of the

parameters W, b.
) Our goal will be to set these in such way that the
computed scores match the ground truth labels across the
whole training set. Dept. of Computer Science
Interpreting a linear classifier: Example
Overview
9 Linear classifier
References
) Each classifier is a row of W.

Overview
9 Linear classifier
References

) A linear classifier computes the score of a class as a
weighted sum of all of its pixel values across all 3 of its
color channels.

Overview
9 Linear classifier
References

) A linear classifier computes the score of a class as a
weighted sum of all of its pixel values across all 3 of its
color channels.
) The function has the capacity to like + or dislike −
view
r classifier
Over
10 Linea
References
) We stretch the image pixels into a column and perform

matrix multiplication to get the scores for each class.

view
r classifier
Over
10 Linea
References
) We stretch the image pixels into a column and perform

matrix multiplication to get the scores for each class.
) Note that this particular set of weights W is not good atall.

Overview
11 Linear classifier
References
) Every row of W is a classifier for one of theclasses.

) As we change one of the rows of W , thecorresponding
line in the pixel space will rotate in different directions. 19
Helwan University
Overview
References
) The biases b, on the other hand, allow our classifiers to

translate the lines. In particular, note that without the bias
terms, plugging in xi = 0 would always give score of zero Dept. of Computer Science
Helwan University
regardless of the weights. 19
Interpretation of linear classifiers as tem-
plate matching
Overview
) Another interpretation for the weights W is that each row 13 Linear classifier
of W corresponds to a template for one of the classes. References
) The score of each class for an image is then obtained by

comparing each template with the image using a dot
product one by one to find the one that “fits” best.
) The linear classifier is doing template matching, where the
templates are learned.

Interpretation of linear classifiers as tem-
plate matching
Overview
) Another interpretation for the weights W is that each row
References
of W corresponds to a template for one of the classes.
) The score of each class for an image is then obtained by
comparing each template with the image using a dot
product one by one to find the one that “fits” best.
) The linear classifier is doing template matching, where the
templates are learned.

Linear classifier
Overview
) + the training data is used to learn the parameters W, b, 15 Linear classifier
References
but once the learning is complete, we can discard the
entire training set and only keep the learned parameters.

Linear classifier
Overview
but once the learning is complete, we can discard the References

) That is because a new test image can be simply forwarded
through the function and classified based on the
computed scores.

Linear classifier
Overview
References
computed scores.
) + To classifying the test image involves a single matrix
multiplication and addition, which is significantly faster
than comparing a test image to all training images.

Linear classifier
PatternRecognition
Ghada Ahmed
Overview
References
computed scores.
) + To classifying the test image involves a single matrix
multiplication and addition, which is significantly faster
than comparing a test image to all training images.
) Convolutional Neural Networks will map image pixels to
scores exactly as shown above, but the mapping (f ) will be
more complex and will contain more parameters.

Is it good enough?
Pattern Recognition
Ghada Ahmed
Overview
References

Next..Decide what W should be?
Pattern Recognition
Ghada Ahmed
Overview
References
) Quantifying what it means to have a "good" W ,measure

our unhappiness with outcomes [Loss function]
) Start with random W and find a W that Minimize the loss
function [Optimization]
) Tweak the functional form of f [ConvNets]

References
PatternRecognition
Ghada Ahmed
Overview
Linear classifier
) Online: 18 References
http://www.computervisionblog.com/2015/03/deep-
learning-vs-machine-learning-vs.html
) Online: http://nkonst.com/machine-learning-explained-
simple-words/
linear-classifier Demo
http://vision.stanford.edu/teaching/cs231n-demos/linear-
classify/
) Book: François Chollet, Deep learning with Python
Cross-Validation
https://scikit-
learn.org/stable/modules/generated/sklearn.model_selectio
n.KFold.html#sklearn.model_selection.KFold

Classifictaion Approaches - Algorithms II

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Classifictaion Approaches - Algorithms II

Uploaded by

Copyright:

Available Formats

ML Classification II

Dept. of Computer Science

) The classifier must remember all of the training data and

Dept. of Computer Science

Dept. of Computer Science

Dept. of Computer Science

We will start with the simplest possible function, a 6 Linear classifier

Dept. of Computer Science

Dept. of Computer Science

) Input Data is fixed, we have control over the setting of the

) Each classifier is a row of W.

Dept. of Computer Science

) Each classifier is a row of W.

Dept. of Computer Science

) Each classifier is a row of W.

) We stretch the image pixels into a column and perform

Dept. of Computer Science

) We stretch the image pixels into a column and perform

Dept. of Computer Science

) Every row of W is a classifier for one of theclasses.

) The biases b, on the other hand, allow our classifiers to

of W corresponds to a template for one of the classes. References

) The score of each class for an image is then obtained by

Dept. of Computer Science

Dept. of Computer Science

) + the training data is used to learn the parameters W, b, 15 Linear classifier

Dept. of Computer Science

) + the training data is used to learn the parameters W, b, 15 Linear classifier

but once the learning is complete, we can discard the References

entire training set and only keep the learned parameters.

Dept. of Computer Science

) + the training data is used to learn the parameters W, b, 15 Linear classifier

Dept. of Computer Science

) + the training data is used to learn the parameters W, b, 15 Linear classifier

Dept. of Computer Science

Dept. of Computer Science

) Quantifying what it means to have a "good" W ,measure

Dept. of Computer Science

You might also like