Handwritten Digit Recognition Using

Convolutional Neural Networks (CNN)

ABSTRACT

Handwritten Digit Recognition is one of the challenging task in computer vision due to its
variability in shape, size and orientation. Our project aims to develop a model that identifies
handwritten digits from the image with better accuracy. We proposed a model for this by
using Convolutional Neural Network and MNIST dataset which consists of 70,000 images
ranging the values from 0-9. In this model every neuron in one layer is coupled to every other
neuron in the layer above. At first, Input layer having a filter map by taking 28x28 pixel
image as input. Next there will be a convolutional layer which is used to extract features from
the images. Then they are downsampled by the maxpooling layers. The performance of CNN-
based Handwritten Digit Recognition systems is typically evaluated using accuracy. Our
proposed method achieved an accuracy of 97%.
Key words : Handwritten Digit Recognition, Convolutional Neural Network, MNIST
dataset, Input layer, Convolutional layer, Maxpooling layer.

1. INTRODUCTION

Handwritten recognition employs machine learning, particularly convolutional neural networks,

to convert handwritten text into digital format. It involves preprocessing steps, feature
extraction, and a training phase where the system learns patterns for accurate character
recognition. Challenges include handling diverse handwriting styles and continuous
refinement for improved accuracy.
In the year 2023 the research paper explains about different computer programs to recognize
handwritten numbers, by using the machine learning algorithms on two datasets named
MNIST and HDC. The results show that neural networks performed the best, even though
they weren't perfect. This suggests that while neural networks are really good, other computer
methods can still do a decent job recognizing handwritten digits [1]. This will tells about
using special computer programs called Convolutional Neural Networks (CNNs) to recognize
handwritten numbers better. The aim is to make these programs super accurate (99.87%)
without needing complicated and expensive setups. The study managed to set a new best
score for recognizing MNIST digits at 99.89%, using a simpler CNN approach that
outperformed fancier models from other researchers. Future studies could try mixing different
types of CNNs and using special algorithms to make things even better [2].
By using the clever program, HDR, that changes handwritten numbers into digital ones,
saving money. Our mission is to improve accuracy using the MNIST dataset. This uses a
smart tool called Convolutional Neural Network (CNN) with Keras, Matplotlib, CV2, and
Tensorflow to keep things simple and reached a high accuracy of 99.63%. Comparing it with
other methods, CNN proved to be the best. In the future, we plan to make our program even
better by including
different handwriting styles [3,4].
By comparing the four smart computer programs to predict zeros in handwritten numbers
using the MNIST dataset. These programs, named Support Vector Machine (SVM),
Stochastic Gradient Descent (SGD), AdaBoost, and Random Forest, were tested to see which
one works the best. SVM turned out to be the most accurate at 99.77% and used less
computer power. It also got the highest score at 98.84%. While Random Forest was accurate
too, it needed more computer resources, and SGD was the quickest with an accuracy of
98.82% and very little memory use [5].
This research will works with the seven smart computer programs—KNN, SVM, Logistic
Regression, Neural network, Random Forest, Naïve Bayes, and Decision Trees—on two sets
of handwritten numbers: MNIST and HDC. The best one turned out to be Neural networks,
even though it didn't get everything perfect. To see which computer programs are best at
recognizing handwritten numbers, and the results showed that Neural networks are really
good. But, other simpler computer programs are still useful for this job [6].
To recognize handwritten numbers using different smart computer methods in WEKA. To
find good and reliable ways to recognize these numbers, using things like Multilayer
Perceptron, Support Vector Machine, Naïve Bayes, Bayes Net, Random Forest, J48, and
Random Tree. This mainly focused on extracting important information from the numbers
and using it to recognize them, and the best method they found was Multilayer Perceptron,
which got 90.37% accuracy. This is a first step in making a program that recognizes
handwritten numbers without using regular methods, and it showed good results in how
accurate it is and how quickly it works [7].
Introduces a Q-ADBN, a clever strategy merging deep learning and reinforcement learning
for improved accuracy and speed in recognizing handwritten digits. Drawing inspiration from
deep reinforcement learning, the model, based on ADAE, excels at extracting features from
digit images, utilizing them as states for decision-making through Q-learning [8].
Computer methods to improve a strategy, testing Support Vector Machine (SVM) with
99.36% accuracy and Convolutional Neural Network (CNN) with an impressive 99.89%.
Although Double Q learning had high accuracy, it was limited to MATLAB data. The layered
design of CNNs enhances computer vision, connecting neurons for detailed processing [9].
The method tackles numeral recognition challenges with EfficientDet-D4, a deep learning
technique. Using annotated images to emphasize important regions, EfficientDet-D4, built on
EfficientNet-B4, enhances accuracy in identifying and categorizing numbers from zero to
nine. The study introduces the robust EfficientDet-D4 system, overcoming hurdles in
accurate numeral recognition, and achieves an impressive 99.83% average accuracy on the
MNIST dataset, showcasing effective detection and categorization of numbers 0 to 9 [10].
The main goal of this article is to talk about the Convolutional Neural Networks (CNN). By
using the python library like Tensorflow which is used for the machine learning algorithms.
We used some of the libraries like keras, matplotlib, numpy and cv2 which are useful for the
recognition of the digits.
2. LITERATURE SURVEY
Paper Paper Title Abstract
Journal of ML Handwritten Rec- The study assessed seven machine
Computing ognition : Handwritten learning algorithms i.e, kNN, SVM,
and Digit Recognition using Logistic Regression, Neural network,
Communicati Machine Learning Random Forest, Naïve Bayes and
Algorithms Decision Trees on two datasets:
on[1]
MNIST and Handwritten Digit
Classification dataset (HDC). Neural
networks emerged as the top
performer, although not yielding
perfect results, indicating their
suitability for this task.
TECHRXIV Influence of Comparison of four competing
[2] various ML- Machine Learning Classifiers that
Based Binary suit the MNIST dataset to
Classifiers on predict 0-Image by using a
the boosting approach. Accuracy of
Performance Support Vector Machine (SVM),
on handwritten Stochastic
digit Gradient Descent (SGD),
recognition AdaBoost and Random Forest
models are assessed through 3-
fold and 10-fold Cross
Validation.
The paper introduces an off-line
GJCST[3] Handwritten Digit handwritten digit recognition
Recognition using approach employing diverse
Machine Learning machine learning techniques in
Algorithms
WEKA. The primary objective is
to establish effective and reliable
methods for recognizing
handwritten digits using various
algorithms, including Multilayer
Perceptron, Support Vector
Machine, Naïve Bayes, Bayes
Net, Random Forest, J48, and
Random Tree.
IJIRCST[4] Handwritten Digit This paper tested variety of
Recognition Using Machine Learning algorithms to
Various Machine upgrade the productivity of
Learning Algorithms strategy and decrease the
and Models
complexity. Machine Learning
algorithms include Support Vector
Analysis
This paper aimed to evaluate
machine learning models for
handwritten digit recognition,
using two medium-sized
datasets, including MNIST.
The findings demonstrate
neural networks' superiority
over classical ML algorithms,
but the gap in performance
suggests that classical ML
algorithms remain viable
options for this task.
Research found that SVM
Classifier demonstrated the
highest accuracy of 99.77%,
while consuming lower CPU
resources during training. It
also yielded the highest F1
score of 98.84% on the test
set, based on PR curve
analysis.
Random Forest Classifier
achieved 99.44% accuracy but
required the most
computational resources, while
SGD Classifier provided the
fastest training and recognition
with 98.82% accuracy and
minimal memory usage of
0.22 MB.
The study highlights feature
extraction and classification in
recognition, achieving 90.37%
accuracy with Multilayer
Perceptron. As an initial effort,
it aims to enable handwritten
numeral recognition without
standard classification methods,
demonstrating promising
results in accuracy and time
complexity.
It is demonstrated that SVM
gives an accuracy of 99.36%
and CNN accuracy rises to
99.89%. Double Q learning
algorithm also gives higher
accuracy but it is limited to
 Machine, Convolutional Neural
Network, Quantum Computing, K-
Nearest Neighbor algorithm.
Elsevier[5] An adaptive deep Q- This project presents an
learning strategy for adaptive Q-learning strategy to
handwritten digit enhance accuracy and reduce
recognition runtime in handwritten digit
recognition. It merges deep
learning and reinforcement
learning creating a Q-ADBN
model.
The method addresses
A Novel Technique for limitations with EfficientDet-D4,
Hindawi[6]
Handwritten Digit a deep learning approach, for
Recognition Using Deep numeral categorization.
Learning
Annotated input images highlight
ROIs, and the EfficientDet-D4,
based on EfficientNet-B4,
enhances accuracy and
performance in detecting and
classifying numerals from zero to
nine.
The study aims to improve
Combining Multiple handwritten digit classification by
TUBITAK[7]
Representations for combining dynamic movement
Pen-based Handwritten and image-based representations
Digit Recognition
using Multi-Layer Perceptron
(MLP) classifiers. These
representations show
complementary error patterns, and
techniques like cascading enhance
accuracy without added
complexity or recognition time.
IJCRT[8] HANDWRITTEN
DIGIT HDR turns handwritten numbers into
RECOGNITION digital ones, saving money. We're
USING OPENCV making a smart program to recognize
AND CNN scanned handwritten numbers, testing
different methods to improve accuracy
using the MNIST dataset.
MATLAB dataset only. CNNs
excel due to their layered
architecture, enhancing
computer vision through a
hierarchical model that
connects neurons in fully
connected layers, facilitating
comprehensive output
processing.
ADBN draws inspiration from
deep reinforcement learning,
excelling in feature extraction
and decision-making. It
employs an ADAE for feature
extraction from handwritten
digit images and integrates Q-
learning, where these features
serve as current states for
decision-making.
The study addresses
challenges in accurate numeral
recognition from images,
introducing a robust DL-based
HDR system, EfficientDet-D4.
It uses image annotations to
train the model, achieving an
impressive average accuracy
of 99.83% on the MNIST
dataset, demonstrating
effective
numeral detection and
categorization from 0 to
9.
Combining diverse MLP
classifiers improves accuracy
through error complementarity,
with classifier choice being
more critical than the
combining method. Correlated
errors diminish the benefits of
combination. Multi-
representation, especially
cascading, reduces costly
preprocessing and enhances
efficiency, particularly for tasks
like static image generation.
Trained a Convolutional Neural
Network (CNN) with libraries
like Keras, Matplotlib, CV2,
and Tensorflow to simplify the
model and achieve high
accuracy (99.63%). Then
compared it to other machine
learning algorithms and found
CNN to be the most accurate.
 ICCIDS[9] Hybrid CNN-SVM The purpose is to create a cross
Classifier for breed demonstrate of a effective
Handwritten Digit Convolutional Neural Network
Recognition (CNN) and Support Vector
Machine (SVM) to recognize
handwritten digits from MNIST
dataset. The suggested model
integrates the salient
characteristics of both classifiers.
Improved Handwritten This study explores the potential of
Sensors[10]
Digit Recognition Using Convolutional Neural Networks
Convolutional Neural (CNNs) for handwritten digit
Networks (CNN) recognition, optimizing design options
and evaluating SGD optimization
algorithms. The goal is to achieve
exceptional accuracy (99.87%)
without the computational
complexity and cost associated with
ensemble architectures, setting a new
record for MNIST digit classification.
IEEE[11] Better digit recognition They introduce a novel approach to
with a committee of training one-hidden-layer neural net
committee members. Rather than
simple Neural Nets
training multiple nets on subsets of the its best. They demonstrated how
training data, they preprocess each
model's training data so that the
associated errors are decorrelated.
Using a committee of 25 one-hidden- aspect ratio and slant of
layer neural nets, they achieve a
recognition error rate of 0.39% on the the motivation for the
MNIST digit recognition benchmark preprocessing that is applied.
set.
Elsevier[12] Improved method of They have developed a novel neural
handwritten digit classifier dubbed LImited Receptive
recognition tested Area (LIRA) for photo identification.
on MNIST database The output, associative, and sensor
layers are the three neural layers that
comprise the classifier LIRA. There
are a lot of MNIST database
discoveries in the literature. In the
optimal cases, the error rates are 0.7,
0.63, and 0.42%. The classifier LIRA
indicates that the mean value of three
trials is the error rate of 0.61%.
IEEE[13] Handwriting This paper gives an overview of the
recognition using multi-class categorization of the
photos using these models is
Deep Learning in
provided in this study, along with a
Keras performance evaluation of the models
using several metrics.
A hybrid CNN-SVM model for
handwritten digit recognition
combines CNN for feature
generation and SVM for output
prediction, benefiting from
both classifiers. It achieves an
impressive 99.28%
classification accuracy on the
MNIST dataset.
The goal was to improve
handwritten digit recognition
using a straightforward CNN
approach, achieving a
remarkable 99.89% accuracy on
MNIST. The simpler CNN
surpassed more complex models
used by other researchers. Future
studies could explore hybrid
CNN models and evolutionary
algorithms for further
improvements.
The errors of the individual
experts should not be associated
for the committee to function at
doing a little preprocessing on
the data before training can do
this. The observed variances in
handwritten numbers serve as
The LIRA neural classifier,
developed for image recognition,
demonstrated excellent
performance in handwritten
recognition and pin-hole relative
position estimation, with a
99.41% recognition rate on the
MNIST database.
For the categorization of
handwritten digits,
convolutional neural
networks are shown to
perform more accurately
than feedforward neural
networks.
IEEE[14] A Robust Model for In this study, they evaluate the
Handwritten Digit effectiveness of CNN with SVM and
KNN and propose CNN as a deep
Recognition using
learning approach using Keras for
Machine and Deep MNIST handwritten digit recognition.
Learning Technique The suggested CNN, which is based
on the Keras model, uses the
RMSprop optimizer to optimize the
model and is used to categorize
photos of handwritten numbers.
IEEE[15] Performance Evaluation In this research, they examined the
of Machine Learning effectiveness of a few machine
Algorithms in learning methods that are used to
Handwritten Digits classify complicated and multiclass
Recognition issues. In order to predict handwritten
digits, they used four machine
learning methods in this work: K-
Nearest Neighbors, Deep Neural
Network, Decision Tree, and Support
Vector Machine.
3. PROPOSED SOLUTION
Our proposed work mainly separated into stages, as shown in Figure 1, Data Collection,
Preprocessing the data, Model construction (CNN), Train the Model, Evaluate the Model,
Predictions.
Figure 1 : Stages of Proposed Solution
By increasing the
convolutional layer with
pooling and dropout and fine-
tuning the model with filter,
kernel size, and neuron count,
this research effort primarily
contributes to the field. With
epoch 10, the suggested CNN
model generates training
accuracy of 99.06% and
testing accuracy of 98.80%.
According to experiment
findings, the suggested CNN
works better than
alternative methods.
The classification produced
by deep neural networks is
the most accurate when
compared to the other
classifiers, according to the
findings obtained using
several similarity criteria,
including accuracy,
sensitivity, and specificity.
3.1 Data Collection
The Dataset we have used is MNIST (Modified National Institute of Standards and
Technology database) Dataset. It is a very popular dataset in the field of image recognition. It
has total images of 70,000 in which it consists of training set of 60,000 examples, and a test
set of 10,000 examples, ranging of values from 0-9. Some of the sample images from the
MNIST database are shown in Figure 2. Every image is of 28x28 resolution with pixel values
in the range of 0- 255 (Gray scale). 0 Gray value represents the background of digit (black
color), while digit is appeared as 255 Gray value (white color).

Figure 2 : Sample MNIST data

3.2 Preprocessing the Data

The crucial step in building a model is Preprocessing. Preprocessing the dataset means
making the dataset that builds and suits perfectly in making our desired model. Dataset has to
be clean, rectified in order to improve the model accuracy and quality. The model will learn
best from the dataset that does not contain null values, has all numerical data. We have to
split the dataset into training and testing. To view the digits present in the dataset we can use
y_train[index] to display the digits. If we use the x_train we will get a two-dimensional
matrix. So we have to scale the dataset as shown in Figure 3. To scale the dataset we have to
import numpy library and we have to use the reshape function to convert the 2-D matrix to 1-
D array. Now the dataset will look good and perfectly fits our model.

Figure 3 : Scaling the data

3.3 Model Construction
After preprocessing the data, the images and labels are ready to be fitted into our model. To
recognize the handwritten digits, we have used an Advanced Machine Learning Technique
i.e. Convolutional Neural Networks (CNN). The basic process of recognizing digit using
CNN is shown in Figure 4.

Figure 4 : Basic Process involved in the Digit Recognition

CNN Architecture
CNN is one of the type of the feed-forward neural network that learns feature engineering by
itself using filters or kernal optimization. CNN is also called as Shift Invariant or Space
Invariant Artificial Neural Network. Feed-forward neural networks are completely linked
networks, meaning that every neuron in one layer is coupled to every other neuron in the
layer above.
An input layer, hidden layers, and an output layer make up a convolutional neural network.

Figure 5: Brief architecture involved in CNN

The CNN architecture was composed of two primary components: the feature classifier and
the feature extractor as shown in Figure 6. Every network layer in the feature extraction unit
gathers the output from the layer immediately before it as input and sends the current output
to the layer immediately after it as inputs; in contrast the classification unit produces the
anticipated outputs.
1. Convolutional Layer :
The CNN's first layer, or CNL, learns the properties of an input image by scanning its
whole region with vertical and horizontal sliding filters. After adding a bias to each zone,
the scalar product of the filter values and the image regions is assessed. Repaired linear
units are used to apply element-wise activation functions, such as max(0, x), sigmoid, and
tanh, to this layer's output in order to perform thresholding.
2. Pooling Layer :
The pooling layer, also known as the max pooling layer or subsampling, appears next.
The pooling layer (PL) reduces the amount of data in order to facilitate quicker and easier
network processing. The two primary tools for implementing pooling are max pooling
and average pooling. By applying vertical and horizontal sliding filters through the input
image, this layer minimizes the amount of data and determines the maximum value or
average value for each region of the input data.
3. Fully Connected layer or Dense layer :
In a conventional neural network, the fully connected layer comes after the convolution
and pooling layers. It consists of n neurons, where n is the expected class number, with a
single neuron for each pixel. For instance, in the digit character classification issue, there
are 10 neurons for each of the ten classes (0–9). However, for the English character
classification problem, there should be 26 neurons for each of the 26 classes (a–z).
4. Output layer :
The next layer of Dense layer is Output layer. It represents the final output classifications
of the network.

Figure 6 : Different Layers of Neural Network

3.4 Train the Model
Once the model is constructed, it can be trained with training data for the required number of
iterations, but increasing the number of iterations may lead to overfitting. We train the model
with 5 Epochs as shown in Figure 7.

3.5 Evaluate the Model

Test data was used to evaluate the trained model as shown in Figure 8.

4. RESULTS AND PREDICTIONS

We have created the model Sequential and added convolutional layers and filters (kernals)
to recognize the digit. Some of the pictures while training the model :

Figure 7 : Training the Model

Figure 8 : Evaluating the model on test data

We have given the inputs of our hand written digits using paint tool as ‘.png’ format to
predict the digits. Some of the pictures and their predictions were shown below :
1) Input : Prediction:

Figure 9 : Input and Prediction for digit 0

2) Input : Prediction :

Figure 10 : Input and Prediction for digit 3

3) Input : Prediction :

Figure 11 : Input and Prediction for digit 1

4) Input : Prediction :

Figure 12 : Input and Prediction for digit 4

5) Input : Prediction :

Figure 13 : Input and Prediction for digit 7

6) Input : Prediction :

Figure 14 : Input and Prediction for digit 8

7) Input : Prediction :

Figure 15 : Input and Prediction for another style of digit 4

8) Input : Prediction :

Figure 16 : Input and Prediction for digit 9

9) Input : Prediction :

Figure 17 : Input and Prediction

10) Input : Prediction :

Figure 18 : Input and Prediction for another style of digit 5

11) Input : Prediction :

Figure 19 : Input and Prediction for another style of digit 1

12) Input : Prediction :

Figure 20 : Input and Prediction for digit 2

13) Input : Prediction :

Figure 21 : Input and Prediction for another style of digit 2

The Training and Validation Accuracy of digits 0-9 are shown in below Figure 22. We can observe
that the Accuracy for all the digits is same.

Figure 22 : Training Accuracy and Validation Accuracy for digits 0-9

5. CONCLUSION
The performance of CNN model was significant in recognizing the hand written digits. The
model obtained 97% accuracy and can recognize photographs from the real world. The
evaluation and training loss percentage is also less. The effect of increasing the number of
convolutional layers in CNN architecture on the performance of handwritten digit recognition
is clearly presented through the experiments. We can increase the epochs in order to increase
our accuracy. But more epochs can leads to Overfitting so we have to specify the epochs very
accurately. The parameters like pooling layers, activation functions also plays a major role in
recognizing the pictures clearly.
6. REFERENCES
[1] Essam, Farah, Hanein Samy, and Judy Wagdy. "Mlhandwrittenrecognition: Handwritten digit
recognition using machine learning algorithms." Journal of Computing and Communication 2.1 (2023):
9-19.
[2] Shahid, Muhammad Farhan. "Influence of various ML-Based Binary Classifiers on the Performance
on handwritten digit recognition." Authorea Preprints (2023).
[3] Shamim, S. M., et al. "Handwritten digit recognition using machine learning algorithms." Global Journal
Of Computer Science And Technology 18.1 (2018): 17-23.
[4] Patil, Pranit. "Handwritten digit recognition using various machine learning algorithms and
models." International Journal of Innovative Research in Computer Science & Technology (IJIRCST)
ISSN (2020): 2347-5552.
[5] Qiao, Junfei, et al. "An adaptive deep Q-learning strategy for handwritten digit recognition." Neural
Networks 107 (2018): 61-71.
[6] Ahmed, Syed Sohail, et al. "A novel technique for handwritten digit recognition using deep
learning." Journal of Sensors 2023 (2023).
[7] Alimoğlu, Fevzi, and Ethem Alpaydin. "Combining multiple representations for pen-based handwritten
digit recognition." Turkish Journal of Electrical Engineering and Computer Sciences 9.1 (2001): 1-12.
[8] Swetha, Hithaishi, Tejaswini, Parthasaradhi, Venkateswara Rao, “HANDWRITTEN DIGIT
RECOGNITION USING OPENCV AND CNN”, International Journal of Creative Research Thoughts
(IJCRT), Volume 9, Issue 6, June-2021 | ISSN: 2320-2882.
[9] Ahlawat, Savita, and Amit Choudhary. "Hybrid CNN-SVM classifier for handwritten digit
recognition." Procedia Computer Science 167 (2020): 2554-2560.
[10] Ahlawat, Savita, et al. "Improved handwritten digit recognition using convolutional
neural networks (CNN)." Sensors 20.12 (2020): 3344.
[11] Meier, Ueli, et al. "Better digit recognition with a committee of simple neural nets."
2011 international conference on document analysis and recognition. IEEE, 2011.
[12] Ernst Kussul, Tatiana Baidyk, “Improved method of handwritten digit recognition tested on
MNIST database”, Image and Vision Computing, Volume 22, Issue 12, 2004, Pages 971-
981,
ISSN 0262-8856.
[13] S. Arora and M. P. S. Bhatia, "Handwriting recognition using Deep Learning in Keras," 2018
International Conference on Advances in Computing, Communication Control and
Networking (ICACCCN), Greater Noida, India, 2018, pp. 142-145, doi:
10.1109/ICACCCN.2018.8748540.
[14] A. K. Agrawal, A. K. Shrivas and V. k. Awasthi, "A Robust Model for Handwritten Digit
Recognition using Machine and Deep Learning Technique," 2021 2nd International
Conference for Emerging Technology (INCET), Belagavi, India, 2021, pp. 1-4, doi:
10.1109/INCET51464.2021.9456118.
[15] S. HAMIDA, B. CHERRADI, A. RAIHANI and H. OUAJJI, "Performance Evaluation of
Machine Learning Algorithms in Handwritten Digits Recognition," 2019 1st International
Conference on Smart Systems and Data Science (ICSSD), Rabat, Morocco, 2019, pp. 1-6,
doi: 10.1109/ICSSD47982.2019.9003052.