SIF910: Deep Learning

Lecture 3: Machine Learning Basics (2)

Chandra Kusuma Dewa

2021/2022
Outlines for Today

● Introduction to MNIST Dataset

● Learning without labels: K-Means
● Learning different representations: PCA
● Learning language: The bag of word representations
MNIST Dataset

● Secondary dataset which consist of

60.000 training images and 10.000
testing images
● All images have dimension of 28 X
28 pixels in one channel grayscale
ranges between 0 (white) and 255
(black)
Classifying MNIST Dataset With Logistic Regression

● Since we have 10 digits to classify, this

becomes multiclass classification
problem
● Therefore, softmax activation function
must be used inside the logistic
regression model → multinomial logistic
regression
● We will use all pixels of an image as the
features → 784 features
Load the MNIST dataset
Check the Dataset Properties
Convert the Dataset to Tensor
Use All Pixels as Features
Create the Model Class
Prepare the Training
Start the Training
Evaluate The Training
What if we have huge amounts of training data?

● We cannot load all 500GB

training data at once to the
500GB 24GB 24GB GPU
Training Data GPU
● Therefore, we need to split the
data into batch
Create the MNIST Dataset class
Create the Data Loader objects
Create the model class (same as the previous one)
Prepare the training
Start the training
Calculate the accuracy
 Learning without labels: K-Means

● A simple unsupervised learning

algorithm
● Instead of performing
classification, we perform
clustering
● Even though we do not need
labels, we still have to define the
number of centroids

Image source: https://miro.medium.com/max/1400/0*irrlUXS1tmYanvT0.png

Two phases in the K-Means algorithm

● Assign = each datapoint is assigned

to the nearest centroid in term of
Euclidean distance
● Minimize = centroids are moved in a
direction that minimize the sum of
the distance of all data points
assigned to it
 Principal Component Analysis

● PCA is used to preprocess

the data. This means that
it has to transform the
data before the data is fed
to the classifier, to make it
digestible
● PCA can also be used for
dimensionality reduction.

Image source: https://miro.medium.com/max/1400/1*mgncZaKaVx9U6OCQu_m8Bg.jpeg

Learning Language: The Bag of Words

● Can we use machine

learning in the field of
Natural Language
Processing (NLP)?
● One of the ways is to use
the bags of words
representations
Can we classify the comments?
The bag of words representations (BOW)
Latihan - Kelompok

● Modifikasi kode program mnist logistic regression tadi sehingga bobot

pelatihan dapat disimpan (file .pth)!
● Load bobot hasil pelatihan (file .pth) tadi ke model, kemudian coba lakukan
prediksi untuk beberapa data image angka!
● Kerjakan secara kelompok
● Presentasikan hasilnya minggu depan (21 Oktober 2021)!
● Tugas ini akan dinilai sebagai Quiz pertama!