Handwriting Recognition Using Machine Learning

VISVESVARAYA TECHNOLOGICAL UNIVERSITY
Jnana Sangama, Belagavi-590018, Karnataka
ON
“HANDWRITING RECOGNITION USING MACHINE LEARNING”
Submitted in partial fulfillment of the requirements for the award of degree of
BACHELOR OF ENGINEERING
in
ELECTRONICS AND COMMUNICATION ENGINEERING
By
GTaniyasri 1BG16EC037
MeghanaBN 1BG17EC407
NehaP 1BG17EC411
ShreeRakshaB 1BG17EC417
Under the guidance of

Mrs. Anuradha JP
Assistant Prof, Dept. of ECE, BNMIT
An Autonomous Institution under VTU, Approved by AICTE, Accredited as Grade A Institution by NAAC.
All Eligible UG branches – CSE, ECE, EEE, ISE &Mech.Engg. Accredited by NBA for academic years 2018-19 to 2021-22 & valid upto 30.06.2022
Post box no. 7087, 27th cross, 12th Main, Banashankari 2nd Stage, Bengaluru- 560070, INDIA
Ph: 91-80- 26711780/81/82 Email: principal@bnmit.in, www.bnmit.org
Department of Electronics & Communication Engineering

An Autonomous Institution under VTU, Approved by AICTE, Accredited as Grade A Institution by NAAC.
All Eligible UG branches – CSE, ECE, EEE, ISE &Mech.Engg. Accredited by NBA for academic years 2018-19 to 2021-22 & valid upto 30.06.2022
Post box no. 7087, 27th cross, 12th Main, Banashankari 2nd Stage, Bengaluru- 560070, INDIA
Ph: 91-80- 26711780/81/82 Email: principal@bnmit.in, www.bnmit.org
Department of Electronics & Communication Engineering
CERTIFICATE
Certified that the project work entitled “HANDWRITING RECOGNITION USING MACHINE
LEARNING” carried out by G Taniyasri (1BG16EC037), Meghana B N (1BG17EC407), Neha P
(1BG17EC411) and Shree RakshaB (1BG17EC417) bona fide students of VIII semester in partial
fulfillment for the award of Bachelor of Engineering degree in Electronics and Communication
Engineering of the Visvesvaraya Technological University, Belagaviduring the year 2020-2021. It is
certified that all corrections/suggestions indicated for internal assessment have been incorporated in there
port deposited in the department library. The project work report has been approved as it satisfies the
academic requirements in respect of Project work prescribed for the said degree.
Mrs. Anuradha J P Dr. P.A. Vijaya Dr. Krishnamurthy. G.N

Assistant Professor Professor and Head Principal
Dept. of ECE Dept. of ECE BNMIT
BNMIT, Bangalore BNMIT, Bangalore Bangalore
External Viva:
SL Name of the Examiners Signature and date

N0
2
ACKNOWLEDGEMENT
We would like to place on record our sincere thanks and gratitude to the concerned people, whose
suggestions and words of encouragement has been valuable. We express our heartfelt gratitude to
BNM Institute of Technology, for giving us the opportunity to pursue Degree of Electronics and
Communication Engineering and helping us to shape our career. We take this opportunity to thank
Prof. T. J. Rama Murthy, Director Dr. S. Y. Kulkarni, Additional Director, Prof. Eishwar N
Maanay, Dean and Dr. Krishnamurthy G.N., Principal for their support and encouragement to
pursue this project. We would like to thank Dr. P.A. Vijaya, Professor and Head, Dept. of
Electronics and Communication Engineering, for her support and encouragement.
We would like to thank our guide Mrs. Anuradha J P, Assistant Professor, Dept. of Electronics and
Communication Engineering, who has been the source of inspiration throughout our project work
and has provided us with useful information at every stage of our project, who is kind enough to
extend her help for our project whenever the needarose.
Finally, we are thankful to all the teaching and non-teaching staff of Department of Electronics and
Communication Engineering for their help in the successful completion of our project. Last but not
the least we would like to extend our sincere gratitude to our parents and all our friends who were a
constant source of inspiration.
G Taniya Sri
Meghana B N
Neha P
Shree Raksha B
i
ABSTRACT
Handwriting Recognition is a technique or ability of a Computer to receive and interpret

intelligible handwritten input from source such as paper documents, touch screen, photographs
etc. Handwritten Text recognition is one of area pattern recognition. The purpose of pattern
recognition is to categorizing or classification data or object of one of the classes or categories.
Handwriting recognition is defined as the task of transforming a language represented in its
spatial form of graphical marks into its symbolic representation. Each script
hasasetoficons,whichareknownascharactersorletters,whichhavecertainbasicshapes.
The goal of handwriting is to identify input characters or image correctly then analyzed too
many automated process systems. This system will be applied to detect thewritings of different
format. The development of handwriting is more sophisticated, which isfound various kinds of
handwritten character such as digit, numeral, cursive script, symbols, and scripts including
English and other languages. The automatic recognition of handwritten text can be extremely
useful in many applications where it is necessary to process large volumes of handwritten data,
such as recognition of addressesandpostcodes on envelopes, interpretation of amounts on bank
checks, document analysis, and verification of signatures. Therefore, computer is needed to be
able to read document or data for ease of document processing. Key Terms-E.G –For Example,
NLP -Natural Language Processing, CNN – Convolution NeuralNetwork
ii
CONTENTS
Sl No. Topic Page No

1 Acknowledgement i
2 Abstract ii
3 Contents iii
4 List of figures iv
5 Chapter 1 - Introduction
1.1 Why is handwriting recognition needed? 1
1.2 Motivation 2
1.3 ProblemStatement 4
1.4 Objective 5
1.5 Organization of theReport 6
6 Chapter 2 – Literature Survey
2.1 Paper 1 7
2.2 Paper 2 8
2.3 Paper 3 9
2.4 Paper 4 11
2.5 Paper 5 14
7 Chapter 3 – Project Decription
3.1 Steps Involved 17
3.2How do convolution neural network work 20
3.2.1 Convolution of an image 21
3.3 Market Potential 30
8 Chapter 4 – Software requirement
4.1 Python 3 32
4.2 Tensor Flow 33
4.3 NumPY 33
4.4Open CV 33
4.5 Application 34
4.6 Functional Requirements 35
4.7 Non Function requirements 35
iii
9 Chapter 5 – Software Description
5.1 UML Diagram 37
5.2 System Testing 38
5.3 Software Code 39
5.4 Future Enhancement 42
10 Chapter 6 - Conclusion 43
11 References 44
iv
LIST OF FIGURES
Figure No Topic Page No
Figure 1.2 Steps for recognizing the character. 4

Figure 2.2 Recognition of online handwritten characters. 9
Figure 2.3 Convolution of images. 10
Figure 2.4 Decision tree. 12
Figure 2.5 Character recognition using neural network. 14
Figure 3.1 Block diagram of Neural Network. 17
Figure 3.2 Gray value image simplifies the task for NN 17
Figure 3.3 Output of CNN Layer. 18
Figure 3.4 Visualization of the RNN output. 19
Figure 3.5 Examples of Cnn 20
Figure 3.6 Basic binary classification 21
Figure 3.7 Convolution 22
Figure 3.8 Convolution with 1 filter 22
Figure 3.9 Result of first filter 23
nd
Figure 3.10 Convolution 2 filter 23
nd
Figure 3.11 Result of 2 filter 24
Figure 3.12 Final result after convoluting with 1 filter 24
Figure 3.13 Taking up positive values 25
Figure 3.14 Replacing negative values with zero 25
Figure 3.15 Result 26
Figure 3.16 Result of all filters 26
Figure 3.17 Take out the highest value 27
Figure 3.18 Result of one filter 27
Figure 3.19 Result of all filter 28
Figure 3.20 Arrange into List 29
Figure 3.21 Final trained data 30
Figure 3.22 Global Handwriting Recognition 31

CHAPTER – 1
INTRODUCTION
HAND WRITING RECOGNITION USING MACHINE LEARNING
Chapter 1: INTRODUCTION
Handwriting character recognition is a field of researching artificial intelligence,

computer vision, and pattern recognition. A computer performing handwriting
recognition is said to be able to acquire and detect characters in paper documents,
pictures, touch-screen devices and other sources and convert them into machine-
encoded form. Its application is found in optical character recognition and more
advanced intelligent character recognition systems. Most of these systems
nowadays implement machine learning mechanisms such as neural networks.
Machine learning is a branch of artificial intelligence inspired by psychology and

biology that deals with learning from a set of data and can be applied to solve
wide spectrum of problems. A supervised machine learning model is given
instances of data specific to a problem domain and an answer that solves the
problem for each instance. When learning is complete, the model is able not only
to provide answers to the data it has learned on, but also toyetunseen data with
high precision.
Neural networks are learning models used in machine learning. Their aim is to
simulate the learning process that occurs in an animal or human neural system.
Being one of the most powerful learning models, they are useful in automation
of tasks where the decision of a human being takes too long, or is imprecise. A
neural network can be very fast at delivering results and may detect connections
between seen in stances of data that human cannot see.
1.1 AIM
The main aim of our project is to develop an hand written characters
reorganization, This project seeks to classify an individual handwritten word so
that handwritten text can be translated to a digital form.
BE. Dept of ECE, Page 1 2020-21

BNMIT
1.2 WHY IS HANDWRITING RECOGNITION

NEEDED?
Manually transcribing large amounts of handwritten data is an arduous process
that's bound to be fraught with errors. Automated handwriting recognition can
drastically cut down on the time required to transcribe large volumes of text, and
also serve as a framework for developing future applications of machine
learning.
1.3 MOTIVATION
Several motivations led to the design of the proposed model. Leveraging research
experience from other fields: a lot of research and good results were reported with
convolution neural networks on images (e.g. for object recognition) and LSTMs
on language (e.g. speech recognition, machine translation). Since the inputs of
our system are images, and the outputs are sentences, it makes sense to use the
presented architecture.
A human can easily solve and recognize any problem, but this is not the same in
the case of a machine. Many techniques or methods should be implemented to
work as a human.
Online handwriting recognition is more accurate when compared to offline

handwriting recognition because of the lack of information. Therefore, there
can be research done in this area to improve offline handwriting recognition. The
main task in offline handwriting recognition is to recognize the Therefore, there
can be research done in this area to improve offline handwriting recognition.
The main task in offline handwriting recognition is to recognize the character

of words. Initially, a dataset is given as input. This is followed by preprocessing,
where an image is subjected to various operations like noise reduction, document
skew correction, slant correction, normalization,

BNMIT
Smoothing, and Skeletonization. The result of this preprocessing can be given as

an input to feature generation.
There are a large number of classifiers, which reduces the performance of

classification at each step. There are also classifiers for recognition including the
statistical, the structural, and the stochastic classifiers and finishing on a
combination of classifiers. At each step, selecting the appropriate parameters
could affect the final classification performance, which results in the complete
recognition of an image. There are different approaches for recognizing the
characters of a word in offline handwriting. The following are the processing
steps:
Figure 1.2: Steps for recognizing the character.

BNMIT
1.4 PROBLEMSTATEMENT
• Despite the abundance of technological writing tools, many people still choose to
take their notes traditionally: with pen and paper. However, there are drawbacks
to handwriting text. It’s difficult to store and access physical documents in an
efficient manner, search through them efficiently and to share them with others.
• Thus, a lot of important knowledge gets lost or does not get reviewed
becauseofthefactthatdocumentsnevergettransferredtodigitalformat.
.
• We have thus decided to tackle this problem in our project because we
believe the significantly greater ease of management of digital text
compared to written text will help people more effectively access, search,
share, and analyze their records, while still allowing them to use their
preferred writing method.
• The purpose of the project is to take the hand written characters as an input
process the character, train the neural network effectively by using the
algorithm to recognize the pattern. To classify, we need to use the best
template to compare with segmented image and to determine how the
template will be used to compare with the image.
• To create segmented image of individual Characteristics, we need to find

the characteristics to be used in boundary, to segment the image in
OFFLINE mode.
• Given a handwritten character, the system needs to predict the type of the
character. In other words if we can write the character“A” the system
predict the character that it is truly “A” or the input character is nearer to
“A” or something else. The purpose of this project is to take

BNMIT
The hand written characters as an input process the character, train the
neural network effectively by using the algorithm to recognize the pattern.
1.5 OBJECTIVE
Manually transcribing large amounts of hand written data is an arduous process

that is bound to be fraught with errors. Hence we propose to bring improvements
to develop an automated handwriting recognition using machine learning for
future applications that:
1. Recognize handwritten digits in real works for autonomous machine

processing.
2. Measures are accuracy and recognition speed.
3. Deformable templates: where an image deformation is used to match an
unknown image against a database of known images.
4. Investigate the deformation of character image outline as a source of
information for recognition.
5. We can go further in recognizing characters in a more challenging
scenario.
6. Machine learning has been applied to number of applications.
1.6 ORGANIZATION OF THEREPORT
This report gives detailed information about the project, “Handwriting

Recognition using Machine Learning”. It explains the algorithms used, the results
observed and the literature survey which assisted us in coming up with the
prototype which is described in this report. The report is organized into 5
chapters.
Chapter-1: Introduction- This chapter gives a brief introduction about the project

BNMIT
Undertaken, the motivation which was the cause for taking up this project, the Problem
statement and the objective of the project.
Chapter-2: Literature Survey-This chapter summarizes the five chosen papers
which is Relevant for project.
Chapter-3: Project Description -This chapter shows the description of our
Project.
Chapter-4: Software Requirements-This chapter describes the software used in
our Project
Chapter-5: Software Description-.This chapter testing of the system, coding and future
enhancement.
Chapter-6: Conclusion- This chapter talks about the information been covered
so far.A referenceisal so mentioned towards the end of the report.

BNMIT
CHAPTER - 2
LITERATURE SURVEY
HAND WRITNG RECOGNITION USING MACHINE LEARNING
Chapter 2: LITERATURE SURVEY

2.1 PAPER1:
Ayush Purohi, Shardul Singh Chauhan:
” A Literature Survey on Handwritten Character Recognition”, International Journal of
Computer Science and Information Technologies, Vol. 7 (1)2016,
1-5.
Handwriting recognition has gained a lot of attention in the field of pattern
recognition and machine learning due to its application in various fields. Optical
Character Recognition (OCR) and Handwritten Character Recognition (HCR) has
specific domain to apply. Various techniques have been proposed to for character
recognition in handwriting recognition system. Even though, sufficient studies
and papers describe the techniques for converting textual content from a paper
document into machine readable form. In coming days, character recognition
system might serve as a key factor to create a paperless environment by digitizing
and processing Existing paper documents. This paper presents a detailed review
in the field of Handwritten Character Recognition.
Character recognition is a fundamental, but most challenging in the field of

pattern recognition with large number of useful applications. It has been an
intense field of research since the early days of computer science due to it being a
natural way of interactions between computers and humans. More precisely
Character recognition is the process of detecting and recognizing characters from
the input image and converts it into ASCII or other equivalent machine editable
form. The technique by which a computer system can recognize characters and
other symbols written by hand in natural handwriting is called handwriting
recognition system . If handwriting is scanned and then understood by the
computer, it is called offline handwriting recognition. In case, the handwriting is
recognized while writing through touch pad using stylus pen, it is called online
handwriting recognition.

BNMIT
2.2 PAPER2:
Megha Agarwal Shalika, Vinam Tomar, Priyanka Gupta:
”Handwritten Character Recognition using Neural Network and Tensor Flow”,
International Journal of Innovative Technology and Exploring Engineering (IJITEE)
ISSN: 2278-3075, Volume-8, Issue- 6S4, April 2019.
The paper will describe the best approach to get more than 90% accuracy in the
field of Handwritten Character Recognition (HCR). There have been plenty of
research done in the field of HCR but still it is an open problem as we are still
lacking in getting the best accuracy. In this paper, the offline handwritten
character recognition will be done using Convolution neural network and Tensor
flow.
A method called Soft Max Regression is used for assigning the probabilities to
handwritten characters being one of the several characters as it gives the values
between 0 and1 summing up to 1. The purpose is to develop the software with a
very high accuracy rate and with minimal time and space complexity and also
optimal. Keywords— Handwritten Character Recognition, Convolution Neural
Network, Tensor Flow, Soft Max Regression. Understanding the handwritten
characters or typed documents is simple to the human beings as we have the
ability to learn. The same ability can be induced to the Machines also by the use of
Machine Learning and Artificial Intelligence.
The field which deals with this problem is called as the OCR or also known as
Optical Character Recognition. It’s the area of study among various fields such as
recognizing of pattern, also Image vision and also AI. This is the system for
changing electronic and image text into the digital character to be read by the
machines. The time used in entering the data and also the storage space required
by the documents can be highly reduced by the use of OCR or in other words it can
be retrieved fast.
By using the OCR in banking field, legal scenarios, etc. many important and sensitive
documents can be processed faster without human intervention. OCR

BNMIT
In advance can be inferred in two ways based on type of the text and document
acquisition (Figure 1).Handwritten text and PCR (Printed Character Recognition).
Figure 2.2: Recognition of online handwritten characters.

BNMIT
2.3 PAPER3:
Karishma Verma, Dr. Manjeet Singh:
”A Literature Survey on Handwritten Character Recognition using Convolution Neural
Network” International Journal of Management, Technology And Engineering ISSN NO:
2249-7455 Volume 8, Issue VII, JULY/2018.
Automatic handwritten character recognition has many academic and commercial

interests. Nowadays, deep learning techniques already excel in learning to
recognize handwritten characters. In computer vision handwriting recognition is
the core problem and the main challenge in this field is to deal with enormous
variety of handwriting style by different writers in different language. The
similarities of the neighboring characters make further complicate the character
recognition problem. The large variety of writing style of different writers and the
complex features of the handwritten characters are very challenging for
accurately classifying the handwritten characters. This paper presents a detailed
review in the field of handwriting recognition using convolution neural network.
The convolution step: Convolution neural network derive their name from the
operator “convolution”. The primary purpose of this operator in case of CNNs is
to extract features from the input image. This is the first layer in CNN; the input
to this layer is a 3D array (32*32*3) of pixel value. Convolution is a mathematical
operation to merge two set of information in other our case, first is input image
and the second set is the filter.
Figure 2.3: Convolution of images.

BNMIT
On the left side we have input image, convolution filter is located in the center
which is also called kernel, detector or feature whereas on the right side of figure
2.3the output of the convolution process called activation map also called
feature map, convolved feature. We perform convolution operation by sliding this
filter over the input image. At every location we do an element wise matrix
multiplication and sum the result, this resultant matrix is called Convolved
feature. CNN learn the value of these filters on its own during training process.

BNMIT
2.4 PAPER4:
Tsehay Admassu Assegie, Pramod Sekharan Nair:
”Handwritten digits recognition with decision tree Classification: a machine learning

approach”, International Journal of Electrical and Computer Engineering (IJECE) Vol.
9, No. 5, October 2019, pp. 4446~4451 ISSN: 2088 -8708.
Handwritten digits recognition is the process of identifying digits written by

human using a machine learning approach [1, 2, 3, 4, 5, 6, 7, 8, 9]. Machine that
performs the recognition task has to be trained so that the machine can perform
the task based on the training given with the past data sets with the known class
labels. Machine learning models such as, artificial neural network, convolution
neural network, decision tree classifier and improved chain cod e histogram
feature [1, 2, 3, 4] can also be applied to handwritten digits recognition problem,
however, the recognition of handwritten digits is still a research issue, numerous
researchers have been carried out and none of them has proposed a perfect
approach to recognize handwritten digits.
This implies that the problem of recognizing a handwritten digit is an important

research issue and there are different approaches and algorithms proposed by
researchers for recognitions of handwritten digits.
The main problems of handwritten digits recognition using machine learning

approaches are listed as follows:
1) Handwriting styles vary based on the writer- it is usually very difficult even
for human to recognize handwritten digits because of the significant difference of
handwriting styles.
2) Similarity between handwritten digits, for example six and four may look the
same digit based on the handwriting style of the writer similarly, one and seven
may look the same digit.
3) There is no perfect machine learning model that is suitable and effective in
recognizing handwritten digits. Different researchers use different models but
nomodelisperfectlysuitedforrecognitionofhandwrittendigits.Thisissueis
BNMIT
Focused in this paper. Efficiency and accuracy of proposed decision tree

classification model is evaluated.
4) The handwritten digits cannot be recognized by using fixed image recognition
patterns, because they are different for different writers.
A decision tree classifier is a machine learning model widely used to solve
classification problems, like handwritten digits recognition. As the name decision
tree suggests, the model breaking down our data by making decisions based on
asking a series of questions. Let's consider the following example where we use a
decision tree to decide upon recognition of a 0 and a 1 digit. The classifier tests all
of the possible sets with digits 0 and 1 until it finds a matching digit as shown
inFigure2.4.1
Figure 2.4: Decision tree.

BNMIT
2.5 PAPER5:
KarishmaTyagi ,VedantRastogi: ”Survey on Character Recognition using OCR
Techniques”, International Journal of Engineering Research &Technology
(IJERT) Vol. 3 Issue 2, February – 2017 ISSN:2278-0181.
Optical Character Recognition has number of applications in day-to-day life.

OCR has been widely used in banking, legal, health care, finance etc. Handwriting
recognition has been one of the most interesting and challenging research areas in
field of image processing and pattern recognition in the recent years. This paper
describes the techniques for converting textual content from a paper document
into machine readable form. The computer actually recognizes the characters in
the document through a transforming technique called Optical
CharacterRecognition.
In this paper, Several techniques like OCR using correlation method and OCR using
neural networks has been discussed. OCR has three processing steps, Document scanning
process, Recognition process and Verifying process. In the document scanning step, a
scanner is used to scan the hand written or printed documents. So, a scanner with high
speed and color quality is desirable.
The recognizing process includes several complex algorithms and previously

loaded templates and dictionary which are crosschecked with the characters in
thedocument and the corresponding machine editable ASCII characters. The
verifying is done either randomly or chronologically by human Intervention.
Difference in font and sizes makes recognition task difficult if preprocessing,
feature extraction and recognition are not robust. There maybenoise pixels
thatareintroducedduetoscanningoftheimage.
Character Recognition in the license plate recognition has important role in

optical recognition system which is relateddirectly with success or failure of the
system. We used Back Propagation Neural Network to opticallyrecognize
theimage.

BNMIT
The basic idea of BP algorithm is the learning process is divided intotwo phases:
PHASE I: Forward Propagation.
PHASE II: Back Propagation.
Figure 2.5: Character recognition using neural network.

BNMIT
CHAPTER – 3
PROJECT DESCRIPTION
HAND LEARNING RECOGNITION USING MACHINE LEARNINIG
Chapter 3: PROJECT DESCRIPTION
Neural networks are composed of simple elements operating in parallel. These

elements are inspired by biological nervous systems. As in nature, the network
Function is determined largely by the connections between elements. We can
traina neural network to perform a particular function by adjusting the values of
the connections (weights) between elements. Commonly neural networks are
adjusted, or trained, so that a particular input leads to a specific target output.
There, the Network is adjusted, based on a comparison of the output and the
target,untilthenetworkoutputmatchesthetarget.
Typically many such input/target pairs are used, in this supervised learning, to
train a network. An artificial neural Network as the backend is used for
performing classification and recognition tasks.Analgorithm that performs
handwriting recognition can acquire and detect characteristics from pictures,
touch-screen devices and convert them to a machine-readable form. There are
two basic types of handwriting recognition systems – online and offline .Both
types can be implemented in applications to progressively learn based on the
user’s feedback while performing offline learning on data in parallel.
Several approaches have been used for online and offline handwriting recognition
fields, such as statistical methods, structural methods, neural networks and
syntactic methods. There are two basic types of handwriting recognition systems
– online and offline.
The use of NN for our task, It consists of convolution NN (CNN) layers, recurrent
NN (RNN) layers and a final Connectionist Temporal Classification (CTC)

layer.

BNMIT
Figure 3.1: Block diagram of Neural Network.
3.1 STEPS INVOLVED:
STEP 1: It is a gray-value image of size 128×32. Then, we copy the image into
a (white) target image of size 128×32. Finally, we normalize the gray-values of
theimagewhichsimplifiesthetaskfortheneuralnetwork.
Figure 3.2: Gray value image simplifies the task for NN.

BNMIT
STEP 2: CNN (convolution neural network): The input image is fed into the
CNN layers. These layers are trained to extract relevant features fromthe image.
Figure 3.3: Output of CNN Layer.
CNN output: Fig. shows the output of the CNN layers which is a sequence of length 32.
Each entry contains 256 features. Of course, these features are further processed by the
RNN layers, however, some features already show a high correlation with certain high-
level properties of the input image: there are features which have a high correlation with
characters (e.g. “e”), or with duplicate characters (e.g. “t”), or with character-properties
such as loops (as contained in handwritten “l” s or “e” s).
STEP 3: RNN (recurrent neural network) : The feature sequence contains 256 features
per time-step, the RNN propagates relevant information through this sequence.

BNMIT
Figure 3.4: Visualization of the RNN output.
Figure 4.2.3 shows a visualization of the RNN output matrix for an image
containing the text “little”. The matrix shown in the top-most graph contains the
scores for the characters including the CTC blank label as its last (80th) entry.
The other matrix-entries, from top to bottom, correspond to the following
characters:“!”#&’()*+,/0123456789:ABCDEFGHIJKLMNOPQRSTUVWXYZ
“Abcdefghijklmnopqrstuvwxyz”.
It can be seen that most of the time, the characters are predicted exactly at the
position they appear in the image (e.g. compare the position of the “i” in the
image and in the graph). Only the last character “e” is not aligned. But this is OK,
as the CTC operation is segmentation-free and does not care about absolute
positions. From the bottom-most graph showing the scores for the

BNMIT
characters “l”, “i”, “t”, “e” and the CTC blank label, the text can easily be decoded:
STEP 4: CTC (connectionist temporal classification): while training the NN,

the CTC is given the RNN output matrix and the ground truth text and it
computes the loss value. While inferring, the CTC is only given the matrix
and it decodes it into the final text.
3.2 How Do Convolutional Neural Networks Work?

There are four layered concepts we should understand in Convolutional Neural
Networks:
1. Convolution,
2. ReLu,
3. Pooling and
4. Full Connectedness (Fully Connected Layer).
Example of CNN:
Consider the image below:
Figure 3.5: Examples of CNN.

BNMIT
The computer understands every pixel. In this case, the white pixels are said to be
-1 while the black OneCare 1. This is just the way we’ve implemented to
differentiate the pixels in a basic binary classification.
Figure 3.6: Basic Binary Classification
3.2.1 Convolution Of An Image
• We have 4 steps for convolution:

• Line up the feature and the image
• Multiply each image pixel by corresponding feature pixel
• Add the values and find the sum
• Divide the sum by the total number of pixels in the feature

BNMIT
Figure 3.7: Convolution
Figure 3.8: convolution with one filter

BNMIT
Figure 3.9: Result of first filter
Figure 3.10: Convolution for 2 nd filter

BNMIT
Figure 3.11: Result of 2 nd filter
Figure 3.12: Final result after convoluting with one filter

Here we considered just one filter. Similarly, we will perform the same
convolution with every other filter to get the convolution of that filter.
ReLU Layer: ReLU is an activation function. But, what is
an activation function? Rectified Linear Unit (ReLU) transform function

only activates a node if the input is above a certain quantity, while the input is
below zero, the output is zero, but when the input rises above a certain
threshold, it has a linear relationship with the dependent variable.

BNMIT
Figure 3.13: Take out positive values as it is
Figure 3.14: Replace negative values with 0

BNMIT
Figure 3.15: Result
Figure 3.16: Result of all filters

BNMIT
Pooling Layer: In this layer we shrink the image stack into a smaller size. Pooling
is done after passing through the activation layer.
We do this by implementing the following 4 steps:
• Pick a window size (usually 2 or3)

• Pick a stride (usually2)
• Walk your window across your filtered images
• From each window, take the maximum value
Figure 3.17: Take out the highest value
Figure 3.18: Result of one filter

BNMIT
Figure 3.19: Result for all filters
Stacking Up the Layers

So to get the time-frame in one picture we’re here with a 4×4 matrix from a
7×7 matrix after passing the input through 3 layers – Convolution, ReLU
and Pooling as shown below:

BNMIT
Figure 3.20: Arrange into list

BNMIT
Figure 3.21 : Final Trained Data
3.3 MARKETPOTENTIAL:
Handwriting recognition technology plays a crucial role in corporate and

government enterprises as it facilitates document preservation and their digital
storage. The handwriting recognition software based on algorithm designed to
translate different languages and symbols into machine-readable electronic text.
The most prominent factor fueling the market growth is the rising demand from corporate
and government enterprises for effective document management.
Substantial amount of the government and corporate enterprises still rely upon physical
documents and files.
Additionally these organizations face significant challenge in preserving old documents

and converting them to electronic media. Handwriting recognition is proven to be viable
solution for effective document management.

BNMIT
Figure 3.22: Global handwriting recognition (HWR) market segmentation.

BNMIT
CHAPTER – 4
SOFTWAREREQUIREMENT
Chapter 4: SOFTWAREREQUIREMENT
The software tools used in this project are:
1. Python3
2. Tensor Flow1.3
3. Numpy
4. OpenCV
4.1 PYTHON3:
Python is an open source programming language that was made to be easy to read.
A Dutch programmer named Guido van Rossum made Python in 1991. He named
it after the television show Monty Python's Flying Circus. Many Python examples
and tutorials include jokes from the show . Python is an interpreted language.
Interpreted languages do not need to be compiled to run. A program called an inter
preterruns Python code on almost any kind of computer.
This means that a programmer can change the code and quickly see the results.
This also means Python is slower than a compiled language like C, because it is
not running machine code directly. Python is a good programming language for
beginners. It is a high-level language, which means a programmer can focus on
what to do instead of how to do it. Writing programs in Python takes less time
than in some other languages. Python drew inspiration from other programming
languages like C,C++,Java Per land Lisp.
Python's developers try to avoid changing the language to make it better until they
have a lot of things to change. Also, they try not to make small repairs, called
patches, to unimportant parts of the C Python reference implementation that
would make it faster.
Python 3.0 was released in 2008. and is interpreted language i.e, it’s not compiled and the
interpreter will check the code line by line. This article can used to learn very basics of
Python programming language. Python makes the development and debugging fast

BNMIT
because there is no compilation step included in Python development, and edit-test-debug

cycle is very fast.
4.2 TENSORFLOW:
Tensor Flow is an open-source software library. Tensor Flow was originally developed
by researchers and engineers working on the Google Brain Team within Google’s
Machine Intelligence research organization for the purposes of conducting machine
learning and deep neural networks research, but the system is general enough to be
applicable in a wide variety of other domains as well.
Tensor Flow is a free and open-source software library for machine learning. It can be
used across a range of tasks but has a particular focus on training and inference of deep
neural networks. It is used for both research and production at Google. Among the
applications for which Tensor Flow is the foundation, are automated image-captioning
software, such as Deep Dream.
4.3 NUMPY:
Numpy is a library for the Python programming language, adding support for large,
multi-dimensional arrays matrices, along with a large collection of high- level
mathematical functions to operate on these arrays. The ancestor of Numpy,
Numeric, was originally created by Jim Hugunin with contributions from several
other developers.
Numpy is a general-purpose array-processing package. It provides a high-performance

multidimensional array object, and tools for working with these arrays. It is the fundamental
package for scientific computing with Python. Besides its obvious scientific uses, Numpy
can also be used as an efficient multi-dimensional container of generic data.
4.4 OPENCV:
OpenCV is a cross-platform library using which we can develop real- time

computer vision applications. It mainly focuses on image processing,

BNMIT
video capture and analysis including features like face detection and object detection.
Originally developed by Intel, it was later supported by Willow Garage then

Itseez(which was later acquired by Intel ] ). The library is cross-platform and free
for use under the open-source Apache 2 License. Starting with 2011, Open CV
features GPU acceleration for real-timeoperations.
Officially launched in 1999 the OpenCV project was initially an Intel Research
initiative to advance CPU intensive applications, part of a series of projects
including real-time ray tracing and 3D display walls. The main contributorsto the
project included a number of optimization experts in Intel Russia, as well as
Intel's Performance Library Team. In theearly days of Open CV, the goals ofthe
project weredescribed.
The first alpha version of Open CV was released to the public at the IEEE Conference on
Computer Vision and Pattern Recognition in 2000, and five betas were released between
2001 and 2005. The first 1.0 version was released in 2006. A version 1.1 "pre-release"
was released in October 2008. The second major release of the Open CV was in October
2009. Open CV 2 includes major changes to the C++ interface, aiming at easier, more
type-safe patterns, new functions, and better implementations for existing ones in terms
of performance. Official releases now occur every six months and development is now
done by an independent Russian team supported by commercial corporations. In July
2020, Open CV announced and began a Kick starter campaign for the Open CV AI Kit a
series of hardware modules and additions to Open CV supporting SpatialAi.
4.5 APPLICATIONS:
1. 2D and 3D featuretoolkits.
2. Ego motionestimation.
3. Facial recognitionsystem.
4. Gesturerecognition.

BNMIT
5. Human–computer interaction(HCI).
6. Mobilerobotics.
7. Motionunderstanding.
8. Objectdetection.
9. National ID number recognitionsystem.
10.Postal office automation with code number recognition onEnvelope.
11.Automatic license plate recognitionand
12. Bankautomation.
4.6 FUNCTIONALREQUIREMENTS
• The system should supportthe three stages of the writing process, these
are planning, translation (writing), andreview.
• It should include some spelling support and should incorporate file-
handling facilities. The recognition component should be able to workeven
when children write slowly, it should be able to deal with ‘wobbly’
writing, and should be able to recognize common miss constructions of
characters.
• Wehaveclassifiedthesefunctionalrequirementsasfollow:
1. Taking/ choosing the desired textimage.
2. Recognition of thetext.
3. Copying the text for differentuses.
4.7 NON FUNCTIONALREQUIREMENT
1. UsabilityRequirements
The application shall be used friendly and doesn't require any guidance to be
used. In other words, the application has to be as simpleas possible, so itsusers
shall use it easily. Actually, the interface is quite simple and straight forward so
that anyone can understandit .

BNMIT
2. ReliabilityRequirements
The application should not have any unexpected failure. In order to avoid any
failure's occurrence, the specifications have been respected and followed
correctly. The only problem that may occur in some cases
isthattheapplicationdoesnotget100%ofthecharactersinthepicture.
3. EfficiencyRequirement
Performance: The application response time shall be adequate and sufficient

enough, that's why the time required for this application toresponse to its user's
actions has to been managed and controlled. But in order to maintain the
performance of the application, the user has to follow the required steps to get
the desiredresult.
BE. Dept ofECE,BNMIT Page 36 2020-21

CHAPTER - 5
SOFTWARE DESCRIPTION
LearnilLearning learning
CHAPTER 5: SOFTWARE DESCRIPTION
5.1 UMLDIAGRAM
In the Unified Modeling Language (UML), a use case diagram can summarize the details
of your system's users (also known as actors) and their interactions with the system. To
build one, you'll use a set of specialized symbols and connectors. An effective use case
diagram can help your team discuss and represent:
• Scenarios in which your system or application interacts with people,

organizations, or externalsystems
• Goals that your system or application helps those entities (known as actors)
achieve
• The scope of your system.

BNMIT
5.2 SYSTEM TESTING
TEST DESCRIPTION EXPECTED ACTUAL RESULT

CASE OUTPUT OUTPUT
NUMBER
TC01 Choose and load dataset to Dataset loaded Dataset Pass
model and analyzed loaded and
analyzed
TC02 Feed dataset to model and Model should Model Pass
train. train and trained and
create check created
point check point
TC02 Validate model through Validate data Validate data Pass
validation data.
TC03 Feed test data to the model Pre-process Pre-process Pass
and recognize and
text recognize
text
TC04 Feed raw data to Segment Sentence Pass
segmentation model sentence with segmented
summary with
summary
TC05 Feed segmented data to Recognize text Text Pass
CNN model and display recognized
with score and
displayed
withscore

BNMIT
5.3 Software Code:

BNMIT

BNMIT

BNMIT
5.4 FUTURE ENHANCEMENT
Firstly, to have more compelling and robust training, we could apply additional
pre-processing techniques such as jittering. We could also divide each pixel by its
corresponding standard deviation to normalize the data. Next, given time and
budget constraints, we were limited to 20 training examples for each given word
in order to efficiently evaluate and revise our model. Another method of
improving our character segmentation model would be to move beyond a greedy
search for the most likely solution. We would approach this by considering more
exhaustive but still efficient decoding algorithm such as beam search. We can use
a character/word-based language-based model to add a penalty/beneﬁt score to
each of the possible ﬁnal beam search candidate paths, along with their
combined individual soft max probabilities, representing the probability of the
sequence of characters/words. If the language model indicates perhaps the most
likely candidate word according to the soft max layer and beam search is very
unlikely given the context so far as opposed to some other likely candidate words,
then our model can correct it-self accordingly.

BNMIT
Chapter 6
CONCLUSION
Chapter 6: CONCLUSION
As the purpose of the project is to take the handwritten character as an input,

process the character, and train the neural network effectively by using algorithm
to recognize the pattern. The algorithms used in character recognition can be
divided into three categories: Image Pre-processing, Feature Extraction, and
Classification.
The recognition system first accepts a scanned image as an input. The image can
be in JPG OR BMT format. Digital capture and conversion of an image often
introduces noise, which makes it hard to identify what is actually a part of the
object of interest. Considering the problem of character recognition, we want to
reduce as much noise as possible, while preserving the strokes of the characters,
since they are important for correct classification.
After reviewing the papers, it was observed that some techniques like direction
feature extraction and diagonal feature extraction techniques were proved to be
better ingenerating higher accuracy results compared to the traditional horizontal
and vertical methods. Also neural networks provide a plus feature of having
higher tolerance to noise.
It was also observed that bigger the training data set helps in achieving a higher
accuracy rate when features are extracted from similar looking characters. This is
highly beneficial; as handwritten characters appear similar, so good feature
extraction techniques need to be used to avoid such anomalies.
To conclude, the steps which have been mentioned previously consists of 5

CNNand2RNNlayersgivesthebriefdescriptionofhowourprojectworks.

BNMIT
REFERENCES:
[1] AyushPurohit, Shardul Singh Chauhan: “A Literature Survey on Handwritten

Character Recognition”, International Journal of Computer Science and
Information Technologies, Vol. 7 (1) ,2016,1-5.
[2] MeghaAgarwal, Shalika, VinamTomar, Priyanka Gupta: ”Handwritten
Character Recognition using Neural Network and Tensor Flow “,International
Journal of Innovative Technology and Exploring Engineering (IJITEE) ISSN:
2278-3075, Volume-8, Issue- 6S4, April2019.
[3] KarishmaVerma, Dr. Manjeet Singh: ”A Literature Survey on Handwritten
Character Recognition using Convolution Neural Network” International Journal
of Management, Technology and Engineering ISSN NO: 2249-7455 Volume
8, Issue VII,JULY/2018.
[4] TsehayAdmassuAssegie,PramodSekharanNair:”Handwritten digits recognition
with decision tree classification: a machine learning approach”, International
Journal of Electrical and Computer Engineering (IJECE) Vol. 9, No. 5, October
2019, pp. 4446~4451 ISSN:2088-8708.
[5] KarishmaTyagi,VedantRastogi: ”Survey on Character Recognition using
OCR Techniques”, International Journal of Engineering Research
&Technology (IJERT) Vol. 3 Issue 2, February – 2017 ISSN: 2278-0181.
[6]AnishaSharma, SoumilKhare, SachinChavan: “A Review on Handwritten
Character Recognition” ,MukeshPatel School of Tech. Management and Engg.,
IJCST Vol. 8, Issue 1, Jan - March2017.
[7] SandeepKaur, Rekha Bhatia: “A Systematic Review of Optical Character
Recognition”, International Journal of Advanced Research in Computer Science
and SoftwareEngineering,
Volume 6, Issue 9, September 2018 ISSN: 2277 128
[8] HandwrittendigitrecognitionusingdeeplearningbyAkshiKumar.
[9] Website: https://www.mygreatlearning.com/blog/how-to-recognise-
handwriting-with-machine-learning/

BNMIT

Handwriting Recognition Using Machine Learning

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Handwriting Recognition Using Machine Learning

Uploaded by

Copyright:

Available Formats

VISVESVARAYA TECHNOLOGICAL UNIVERSITY

Jnana Sangama, Belagavi-590018, Karnataka

Under the guidance of

Department of Electronics & Communication Engineering

Department of Electronics & Communication Engineering

Mrs. Anuradha J P Dr. P.A. Vijaya Dr. Krishnamurthy. G.N

SL Name of the Examiners Signature and date

Handwriting Recognition is a technique or ability of a Computer to receive and interpret

Sl No. Topic Page No

Figure No Topic Page No

Figure 1.2 Steps for recognizing the character. 4

Figure 3.20 Arrange into List 29

Figure 3.21 Final trained data 30

Figure 3.22 Global Handwriting Recognition 31

Handwriting character recognition is a field of researching artificial intelligence,

Machine learning is a branch of artificial intelligence inspired by psychology and

BE. Dept of ECE, Page 1 2020-21

1.2 WHY IS HANDWRITING RECOGNITION

Online handwriting recognition is more accurate when compared to offline

The main task in offline handwriting recognition is to recognize the character

BE. Dept of ECE, Page 2 2020-21

Smoothing, and Skeletonization. The result of this preprocessing can be given as

There are a large number of classifiers, which reduces the performance of

Figure 1.2: Steps for recognizing the character.

BE. Dept of ECE, Page 3 2020-21

• To create segmented image of individual Characteristics, we need to find

BE. Dept of ECE, Page 4 2020-21

Manually transcribing large amounts of hand written data is an arduous process

1. Recognize handwritten digits in real works for autonomous machine

1.6 ORGANIZATION OF THEREPORT

This report gives detailed information about the project, “Handwriting

BE. Dept of ECE, Page 5 2020-21

BE. Dept of ECE, Page 6 2020-21

Chapter 2: LITERATURE SURVEY

Character recognition is a fundamental, but most challenging in the field of

BE. Dept of ECE, Page 7 2020-21

BE. Dept of ECE, Page 8 2020-21

Figure 2.2: Recognition of online handwritten characters.

BE. Dept of ECE, Page 9 2020-21

Automatic handwritten character recognition has many academic and commercial

Figure 2.3: Convolution of images.

BE. Dept of ECE, Page 10 2020-21

BE. Dept of ECE, Page 11 2020-21

”Handwritten digits recognition with decision tree Classification: a machine learning

Handwritten digits recognition is the process of identifying digits written by

This implies that the problem of recognizing a handwritten digit is an important

The main problems of handwritten digits recognition using machine learning

Focused in this paper. Efficiency and accuracy of proposed decision tree

Figure 2.4: Decision tree.

BE. Dept of ECE, Page 13 2020-21

Optical Character Recognition has number of applications in day-to-day life.

The recognizing process includes several complex algorithms and previously

Character Recognition in the license plate recognition has important role in

BE. Dept of ECE, Page 14 2020-21

Figure 2.5: Character recognition using neural network.

BE. Dept of ECE, Page 15 2020-21

Chapter 3: PROJECT DESCRIPTION

Neural networks are composed of simple elements operating in parallel. These

NN (RNN) layers and a final Connectionist Temporal Classification (CTC)

BE. Dept of ECE, Page 16 2020-21

Figure 3.1: Block diagram of Neural Network.

3.1 STEPS INVOLVED:

BE. Dept of ECE, Page 17 2020-21