Course Name : Pattern Recognition

Student ID: 2019226752

Name: Shahzad Iqbal

Introduction

The MNIST database (Modified National Institute of Standards and Technology database) is a
large database of handwritten digits that is commonly used for training various image processing
systems. The database is also widely used for training and testing in the field of machine learning.
The MNIST database contains 60,000 training images and 10,000 testing images. We have used
k-means, confusion matrix, and neural network to play with the data. In this project we applied
different machine learning (ML) techniques to MNIST data to build a classifier or detect certain
patterns.

Method
We have used the following models:

K-means clustering:

K-means clustering is one of the simplest and popular unsupervised machine learning algorithms.
Typically, unsupervised algorithms make inferences from datasets using only input vectors
without referring to known, or labelled, outcomes. You’ll define a target number k, which refers to
the number of centroids you need in the dataset. A centroid is the imaginary or real location
representing the centre of the cluster. Every data point is allocated to each of the clusters
through reducing the in-cluster sum of squares.

In other words, the K-means algorithm identifies k number of centroids, and then allocates every
data point to the nearest cluster, while keeping the centroids as small as possible. The ‘means’ in
the K-means refers to averaging of the data; that is, finding the centroid. To process the learning
data, the K-means algorithm in data mining starts with a first group of randomly selected
centroids, which are used as the beginning points for every cluster, and then performs iterative
(repetitive) calculations to optimize the positions of the centroids

It halts creating and optimizing clusters when either:

  The centroids have stabilized — there is no change in their values because the clustering
has been successful.

 The defined number of iterations has been achieved.

Confusion Matrix:

In the field of machine learning and specifically the problem of statistical classification, a
confusion matrix, also known as an error matrix. A confusion matrix is a table that is often used to
describe the performance of a classification model (or “classifier”) on a set of test data for which
the true values are known. It allows the visualization of the performance of an algorithm. It allows
easy identification of confusion between classes e.g. one class is commonly mislabelled as the
other. Most performance measures are computed from the confusion matrix.

A confusion matrix is a summary of prediction results on a classification problem. The number of

correct and incorrect predictions are summarized with count values and broken down by each
class. This is the key to the confusion matrix. The confusion matrix shows the ways in which your
classification model is confused when it makes predictions. It gives us insight not only into the
errors being made by a classifier but more importantly the types of errors that are being made.

Neural Networks:

A neural network is a network or circuit of neurons, or in a modern sense, an artificial neural

network, composed of artificial neurons or nodes. Thus a neural network is either a biological
neural network, made up of real biological neurons, or an artificial neural network, for solving
artificial intelligence (AI) problems. The connections of the biological neuron are modelled as
weights. A positive weight reflects an excitatory connection, while negative values mean
inhibitory connections. All inputs are modified by a weight and summed. This activity is referred
as a linear combination. Finally, an activation function controls the amplitude of the output. For
example, an acceptable range of output is usually between 0 and 1, or it could be −1 and 1.

Results and Discussions

We have used MNIST data set. Each data entry is text data that represents a handwritten digit
as shown below. Size of each image is 28x28 pixels and are in greyscale, 0 representing white
and 255 black. The data is split into two sets, one with labels (training data) and another one
without labels (testing data). Each image has 784 pixels (28x28), where the pixels are rolled out
from left to right and then continuing at the next line (top-down).
Case 1: Using k-means clustering

Figure shows the results and accuracy of k-means. In my opinion the results are good and
accurate.
Case 2: Using confusion matrix

As the figure shows the result are good and accurate.

Case 3: using Neural networks

The accuracy of using Neural Networks is 0.94 which is very good.

Conclusion
We have used k-means, transfusion matrix and Neural networks on our MNIST data set and
found the results quite good and accurate. But across all these Neural networks results are very
good and accurate. Among all Neural Networks results have the highest accuracy.