Book Recommendation System

A Thesis Submitted in Partial Fulfillment of the Requirements for the

Degree of
Bachelor of Science in Computer Science and Engineering

By
Abdullah Basar Maruf
CSE 063 07404
&
Md Nazim Uddin
CSE 063 07429

Supervised by: Samiul Islam

Senior Lecturer

Department of Computer Science and Engineering

STAMFORD UNIVERSITY BANGLADESH

July 2021
Abstract
Handwritten character recognition is a complex task, complexity varies among different
languages because of their distinct shapes, strokes and number of characters. There exist a
number of works in English handwritten character recognition with respect to other major
languages such as Bangla. Distinct feature extraction techniques and various classification
tools are used in those existing methods for their recognition schemes. Nowadays, English
handwritten character recognition found appreciable result by using Neural Network (NN). In
this report, a NN based Bangla handwritten character recognition is researched. The proposed
method normalizes the written character images and then employ NN to classify individual
characters. It does not employ any feature extraction method like other related works. 6875
handwritten characters with different shapes and variations are used in this study. The
proposed method is shown satisfactory recognition accuracy. Therefore, this report proposes
that using neural network architecture and recently released BanglaLekha-Isolated[8] dataset,
we find a competitive result which is quite good.
 Approval

The thesis report “Handwritten Bangla Characters(Vowel) Classification Using Neural

Networks” submitted by SUMIYA ARAFIN ID: CSE 05406737, ELORA AFRIN ID: CSE
05406774, to the Department of Computer Science & Engineering, has been accepted as
satisfactory for the partial fulfillment of the requirements for the degree of Bachelor of
Science (B.Sc.) in Computer Science & Engineering and as to its style and contents.

Board of Examiner’s Name, Signature and Date:

……………………………... …………………………….. ……………………………..

(Board Member 1) (Board Member 2) (Board Member 3)

Date: 01-11-2018 Date: 01-11-2018 Date: 01-11-2018

Supervisor’s Signature and Date:

……………………………...
Asma-ull-Hosna
Date: 01-11-2018
 Declaration

We, hereby, declare that the work presented in this Thesis is the outcome of the investigation
performed by us under the supervision of Asma-ull-Hosna, Senior Lecturer, Department of
Computer Science & Engineering, Stamford University Bangladesh. We also declare that no
part of this Thesis and thereof has been or is being submitted elsewhere for the award of any
degree or Diploma.

Signature and Date:

……………………………...
Student Name: Sumiya Arafin
Date: 01-11-2018

……………………………...
Student Name: Elora Afrin
Date: 01-11-2018
 Dedicated to…
Asma-ull-Hosna
 Acknowledgements

We would like to sincerely thank our honorable supervisor Asma-ull-Hosna, Senior Lecturer,
Department of Computer Science and Engineering, Stamford University Bangladesh, for this
worthy guidance, suggestions and encouragement to complete our work properly. Finally, we
thank our friends and family members, who have been our source for inspiration and
encouragement during the last four months of thesis research, without whom this work would
not have been possible.
Table of Contents
List of Figures...........................................................................................................................7

List of Tables............................................................................................................................3

1: Introduction......................................................................................................................4

1.1 Overview.....................................................................................................................4
1.2 Problem Definition......................................................................................................4
1.3 Motivation...................................................................................................................5
1.4 Objective......................................................................................................................5
1.4 Chapter Summary........................................................................................................5

2: Literature Review.............................................................................................................6

2.1 Background Study.......................................................................................................6

2.2 Neural Network (NN)............................................................................................................. 7
2.2.1 Historical Background…...................................................................................8
2.3 Chapter Summary........................................................................................................9

3: Architecture....................................................................................................................10

3.1 Dataset Preparation....................................................................................................10

3.2 Proposal Strategy......................................................................................................12
3.3 Preprocessing of raw character..................................................................................12
3.4 Classification Using Neural Network….....................................................................13
3.4.1 Sigmoid Neurons..............................................................................................14
3.4.2 Gradient Descent...............................................................................................15
3.4.3 Three Layers of the Proposed Neural Network…............................................15
3.5 Chapter Summary.......................................................................................................18

4: Implementation..............................................................................................................19

4.1 Implementation..........................................................................................................19
4.1.1 Description of Configuration Code..................................................................19
4.1.2 Results...............................................................................................................28
 4.2 Chapter Summary......................................................................................................38

5: Conclusion.......................................................................................................................39

5.1 Limitations.................................................................................................................39
5.2 Future Work...............................................................................................................39

References...............................................................................................................................40

Appendix.................................................................................................................................42
List of Figures

2.1: A Simple Neural Network...........................................................................................7

3.1: Classification of Dataset............................................................................................10

3.2: Some Original Image (Without resize)......................................................................12

3.3: resized image (28×28 dimension)..............................................................................13

3.4: Input-Output Structure of Sigmoid Neurons.............................................................14

3.5: Three layers architecture of proposed network.........................................................16

3.6: Raw input images......................................................................................................17

4.1: Corresponding resized images...................................................................................17

4.1: Decimal Valued Image..............................................................................................20

4.2: Double Valued Image................................................................................................20

4.3: Image under Labelling...............................................................................................21

4.4: Image of Doubled Value under Labelling.................................................................21

4.5: Images after Shuffling...............................................................................................22

4.6: All data in 1 matrix (5500*784)................................................................................23

4.7: Images label in 1 matrix (1*5500).............................................................................24

4.8: Labelled and Image mat file......................................................................................24

4.9: The Structure of Labelled and Image mat file...........................................................25

4.10: The main matfile (bangla_vowel.mat).....................................................................25

4.11: The Structure of Labelled and Image mat file.........................................................26

1
4.12: Result(99% accuracy) for Same Training and Testing data....................................28

4.13: Result(72% accuracy) for Different Training and Testing data.............................29

4.14: Some Training Images for This Experiment...........................................................30

4.15: Some Testing Images for This Experiment.............................................................30

4.16: Values of the Parameter and Result(69% accuracy) of the Experiment.................31

4.17: Values of the Parameter and Result(51% accuracy) of the Experiment.................32

4.18: Some Training Images for This Experiment...........................................................33

4.19: Some Testing Images for This Experiment.............................................................33

4.20: Values of the Parameter and Result(84% accuracy) of the Experiment.................34

4.21: Values of the Parameter and Result(89% accuracy) of the Experiment.................35

4.22: Values of the Parameter and Result(75% accuracy) of the Experiment.................36

List of Tables

3.1:The Number of Training and Testing Data Used for 80:20 Ratio..............................11

3.2:The Number of Training and Testing Data Used for 5:1 Ratio..................................11

3.3: Bangla 11 vowels.......................................................................................................13

4.1: Result for Different Dataset.......................................................................................38

1 Introduction
1.1 Overview

In recent years, the necessity of automatic character recognition is getting popularized.

Handwritten character recognition is more perplexing. It is more difficult comparing to
printed forms of characters. Handwritten characters of different persons are not identical but
there are lots of variation in their size and shape. This countless variations in writing styles of
individual character make the recognition task more challenging. The main task of
handwriting character recognition is to deal with the abundant variety of handwriting styles
by different writers in different languages. Sometimes, characters are diverse from each other
(e.g., Thai, Laos and Japanese), In some other cases, characters are wreath and sometimes the
characters are connected with each other (e.g., English, Bangladeshi and Arabic). Moreover,
accurately classifying the hand written characters are very challenging because of the large
diversity of writing styles, writers, and the complicated features of characters.
Bangla is one of the most spoken language counted as the seventh most spoken native
language in the world. It is glorious with its rich heritage. It is the second most popular
language in the Indian subcontinent and also it is the language of 220 million people of this
region and 300 million from all over the world. As opposed to, it is the first language of
Bangladeshi people. 21st February is declared as the International Mother Language Day by
UNESCO to respect the language martyrs for the language in Bangladesh in 1952. Bangla is
an Indo-Aryan language. Its primary root is Sanskrit and kept enlarging by the exploitation of
foreign words over thousands of year since its origination. The modern version of the
language consists of a total of 50 basic alphabets including 11 vowels and 49 consonants and
complementing them are 24 compound characters, along with 10 distinct digits. Here, we are
working only for the Bangla Handwritten Vowel Alphabet classification.

1.2 Problem Definition

Nowadays, Information distribution has shifted from handwritten hard-copy documents to

digital file formats. This process of converting information from handwritten hard-copy to
digital file format is occurring over the past several decades, because of reliability and make
the information more durable. As opposed to, it creates a new problem. A massive portion of
the older documents are still stored as handwritten forms. Convert this massive portion of
data to digital format with traditional method is difficult because this method completely
relies on manual typing to copy an existing storage. However, our government is trying to
convert their stored data which are written in Bangla, into digital formats. But, this slow and
unimaginative traditional process will waste a lot of time to explore the documents, on the
other side, a considerable manpower will require to make accurate copies of each and every
document. Everyone has a unique approach to Bangla handwriting. That’s why trying to
understand the writing style of the handwritten documents is more difficult. Moreover,
Bangla characters are more complex with a complicated arrangement of curvatures.
1.3 Motivation

We have inspired by the problem. This problem has motivated us to create an approach which
can identify Bangla handwritten characters, where we have used a model that is trained with
machine learning algorithms, and it can classify handwritten Bangla vowel alphabets from
images of documents. Slow and unimaginative traditional process will waste time to explore
the documents. However, our proposed system which is trained with machine learning
algorithms offers an alternative solution to the old traditional method of Replication
handwritten Bangla documents and also reduces the waste of valuable manpower, associated
costs, and the time required for the whole process. Apart from this, large numbers of potential
applications of Bangla Handwritten Character Recognition exist. Such as- Bangla traffic
number plate recognition, automatic ID card reading, extracting data from hardcopy forms,
automatic postal code identification, automatic reading of bank checks and digitalization of
documents etc. We have tried to make a system which will allow governments and
organization alike to increase their efficiency of document handling, saving storage space,
and ensuring the security of documents.

1.4 Objective

Our goal is to classify handwritten Bangla vowel characters from sample images of
handwritten isolated words, to do this work, we have decided to classify all the 11 vowel
characters. It’s a tenacious task to cover the entire domain of Bangla language family.
Because, the task requires proper utilization of complex and efficient machine learning
algorithms, and top-tier hardware to train a model within the least amount of time. However,
it is essential for us that the model achieves the minimum expected value accuracy on Bangla
handwritten vowel characters.

In this report, we have described our work which is about the Neural Network (NN) based
Bangla handwritten character classification. The proposed method first normalizes the written
character images and then employ NN to classify individual characters. It does not employ
any feature extraction method like other related works. Handwritten characters of different
persons having different shapes and variations are used in this study. Experimental studies
reveal that the proposed NN based method shows satisfactory classification accuracy.

1.4 Chapter Summary

In this chapter, we gave a small description about handwritten character classification task
and our work Bangla Handwritten Character Classification using Neural Network (BHCC-
NN). We also clearly described why we do this task, our inspirations of the work, our goal.
Moreover, here we tried to introduce about the task, reasons for doing this task, motivation
and its goal.
2 Literature Review

There are some appreciable works exist for Bangla handwritten characters. Which works are
applied in the field of classification and identification using machine learning algorithms,
various scholars worked with the different state of the art implementations on Bangla
handwritten character and digit classification, within all those early systems most of them
based on standard shallow learning methods like feature extraction and Multilayer Perceptron
(MLP) techniques and A. Roy, Bhattacharya, B. Chaudhuri, and U. Pals have done some of
the most prominent work, they are the pioneers in the field of Bangla handwritten character
and digit classification, and also create a high standard for future scholarly works and
implementation.

2.1 Background Study

Numerous methods based on the artificial neural network are investigated for handwritten
English character recognition. Those methods follow distinct feature extraction techniques.
Morphological /Rank/Linear Neural Network (MRL-NN) [1], in which the combination of
inputs in every node is formed by hybrid linear and nonlinear (of the morphological/rank
type) operations, is investigated for handwritten digit. A hybrid Multilayer Perceptron-
Support Vector Machine (MLP-SVM) based method was used for English digit [2] and
Chinese character [3] recognition. Support Vector Machine (SVM) with Radial Basis
Function (RBF) network is used in Ref. [4] for both small and capital handwritten English
character set.
There exist some noteworthy works for Bangla handwritten character recognition. A fusion
classifier using Multilayer Perceptron (MLP), RBF network and SVM considered by
Bhowmik et al. [5]. The Wavelet transform is used for their feature extraction from character
images. They proposed some similar characters which are the single pattern and train the
classifier for 45 classes in classification. As opposed to, a hierarchical approach to segment
characters from words and MLP is introduced by Basu et al. [6] which is used for
classification. Three different feature extraction techniques are used in their segmentation
stage. However, they reduced character patterns. Character pattern reduced into 36 classes
because of the merging similar characters in a single class.
Battacharya et al. [7] introduced a new method recently. Here a two-stage recognition scheme
for 50 basic character classes is used. Overlaying a rectangular grid consisting of regularly
spaced horizontal and vertical lines over the character bounding box computed the feature
vector for the first classifier. Then, the first classifier response, which is to identify its
confusion between a pair of similarly shaped characters. To resolve the confusion, they used
the second stage of classification. Here, overlaying another rectangular grid but consisting of
irregularly spaced horizontal and vertical lines over the character bounding box computed the
feature vector. Respectively, Modified Quadratic Discriminant Function (MQDF) classifier
and MLP is used as the classifiers in the first and second stages.

Handwritten character classification is more complex task than classification from the printed
form. It is a high-dimensional complex task. To work with a grayscale image, it required
numerous computations. At first, features are extracted by some traditional methods [5-7].
After feature extraction, they use MLP based method for the classification task.
Over and above, numerous successful works exist for automatic character classification of
English handwritten characters [8], on the other side, handwriting classification of Bangla
content is falling behind. Deep learning technique is very useful for handling classification of
handwritten characters. Usually, a massive number of labeled data is required for these types
of technique. BanglaLekha-Isolated [8], create an opportunity to decrease the deficiency.

2.2 Neural Network

Neural network is a beautiful biologically-inspired programming paradigm, which is a

generic term in Deep Learning that works on the basis of the structure and functions of a
human brain. It enables a computer to learn from observational data. The Human brain has
interconnected neurons that constantly transmit signals. Likewise, a neural network also has
interconnected artificial neurons and it can transmit data among each other, those are called as
nodes.

Among different types of neural networks, the setup of a neural net can vary tremendously.
However, in most cases, nodes of the neural networks can be split up into three main layers.
These layers are showed below in Figure 2.1.

Figure 2.1: A simple Neural Network.

 Input Layer – It is the top layer. Here each node represents an input and it can accept
inputs in different forms.
 Hidden Layer - The real work is done in this middle layer, but it is a hidden layer that’s
why we specifically don't know what's going on. Generally, hidden layers transform the
inputs and do several calculations and feature extractions. Usually, this layer consists of
multiple layers which are feeding into each other and more layer provides a better result
 (once the system is fully trained). However, the result also tends to increase the training
time exponentially.
 Output Layer – It is the bottom layer. It produces the desired output. Here each node
represents an output.

Every node in the network has specific random weights and every layer has a bias, which is
attached to it that influences the output of every node. To decide which nodes to fire,
particular activation functions are applied to each layer.

2.2.1 Historical Background

Human brain is the mysterious organ of the human body. Study of the human brain is not a
new topic. It is started from almost thousands of years ago. This thinking process developed
with the advent of modern electronics. In 1943 Warren McCulloch, a neurophysiologist, and
a young mathematician, Walter Pitts, wrote a paper which was about how neurons work and
they also modeled a simple neural network using electrical circuits.
In 1949 Organization of Behavior was written by Donald Hebb. This book was about the
concept of neurons and how they work. The strength of neural pathways is described in this
book.

A simple neural network was simulated in 1950. Nathanial Rochester led the first effort to
simulate this. He was a researcher from IBM research laboratories. However, the first attempt
was not successful and the later attempts were effective. During this time the strength in
computing left the neural research in the background.

Dartmouth Summer Research Project on Artificial Intelligence helps to both artificial

intelligence and neural networks in 1956.

John von Neumann suggested Dartmouth Project for following simple neuron functions by
using telegraph relays or vacuum tubes in the following year. Frank Rosenblatt start work on
Perceptron. He was a neuro-biologist of Cornell. He did his research with the operation of the
eye of a fly. Perceptron was the result of this research and it was built in hardware. Still,
Perceptron is the oldest neural network.

Bernard Widrow and Marcian Hoff developed ADALINE and MADALINE model in 1959.
Multiple Adaptive Linear Elements were used in these model. In the real world problem,
MADALINE was the first neural network which is an adaptive filter. Still, this neural
network is using commercially.

the American Institute of Physics began meeting - Neural Networks for Computing in 1985
what has become an annual meeting. the Institute of Electrical and Electronic Engineers
(IEEE) arranges first International Conference on Neural Networks in 1987.

The Deep neural network is a commonly discussed topic in the recent world. They give very
brilliant commitments. That proof this kind of works will give a satisfactory result. However,
the whole technology and its future depend on hardware development. Recently, the
fundamental works are absolutely providing by the most neural network. Study of the deep
Neural network is developing day by day. But, it takes time to learn because of it processing
abridgment. Nowadays, three types of neuro chips are using in different Companies. Such as -
digital, analog, and optical. These three types of neuro chips are used to take these prototypes
out of the lab. "Silicon compilers” are also used in some companies to produce a neural
network Application Specific Integrated Circuit (ASIC).

2.3 Chapter Summary

In this chapter, we have described some previous works, which works are applied are applied
for Bangla Handwritten Characters classification and identification using machine learning
algorithms. We also gave a description about Neural Network(NN) which we used in our
method and provide a historical background of this network. Moreover, in this chapter, we
have described our total background study.
3 Architecture

Our goal of the experiment is to identify scanned images of Bangla Handwritten Characters
(অ, আ, ই, ঈ, উ, ঊ, ঋ, এ, ঐ, ও, ঔ) assemble to the input image. To get the best possible
solution we have used a famous architecture and also customized this. The architecture
of the proposed method will be described in this section.

3.1 Dataset Preparation

Recently, Classification of Bangla handwritten character is being very popular and becoming
a very important issue. It’s a very important task indeed, especially for Bangla speaking
population. BanglaLekha-Isolated[8] is a comprehensive Bangla handwritten character
dataset [8]. Bangla handwritten numerals, basic characters and compound characters are
included in this dataset. It is a huge dataset consist of 84 classes, which is constructed with 50
class of basic letters, 10 class of numeral and 24 classes are for often used compound letters
(Figure 3.1). However, we have worked here only for 11 Bangla vowels.

Figure: 3.1 Classification of Dataset

This dataset is extremely different from other dataset. Data are collects from various region
and different age’s people. This age range is 4 to 27.
Table: 3.1 The Number of Training and Testing Data Used for 80:20 Ratio

Dataset Training Testing Validation

অ 500 125 125
আ 500 125 125
ই 500 125 125
ঈ 500 125 125
উ 500 125 125
ঊ 500 125 125
ঋ 500 125 125
এ 500 125 125
ঐ 500 125 125
ও 500 125 125
ঔ 500 125 125
Total 5500 1375 1375

Table: 3.2 The Number of Testing and Testing Data Used for 5:1 Ratio

Dataset Training Testing Validation

অ 500 100 100
আ 500 100 100
ই 500 100 100
ঈ 500 100 100
উ 500 100 100
ঊ 500 100 100
ঋ 500 100 100
এ 500 100 100
ঐ 500 100 100
ও 500 100 100
ঔ 500 100 100
Total 5500 1100 1100
3.2 Proposed Strategy

A Neural Network (NN) comprised in multiple layers between input and output layers.
Generally, it is a feedforward network where without loopback data flows from input layers
to output layers. Pixel values are the input of an NN.
NN is a classic machine learning benchmark. In our experiment, we used 5500 Bangla
handwritten grayscale vowel, and the task is to identify them. The vowel run from 0 to 10 for
11 vowels, this is a multiclass classification problem. There are 11 possible classes, one for
each character. For testing we also used 1375 Bangla handwritten grayscale vowel, we have
done the same task for the validation part. We had to choose an efficient model to do the
work properly. On the other side, we had to preprocess the dataset to achieve our desired
result.

3.3 Preprocessing of the raw character images

The forms of the BanglaLekha-isolated[8] dataset are scanned. We manually extract the
dataset. Each content extracted automatically. The images were in grayscale format. In this
format, each pixel values represent with a single integer number (from 0 to 255). The
brightness of the pixel represents by these integer number. Generally, 255 represent the white
pixels and 0 represent the black pixels.
The size of different image files of the dataset was not the same and original image size was
too large for our experiment. Large size image required a large memory also. To sustain
definite and equal inputs for all the characters we resized the dataset into the 28×28

dimension.
Figure: 3.2 Some Original Images (Without resize)
 Figure: 3.3 resized image (28×28 dimension)

The original images wrote on a black background using white color. For this reason, the
grayscale image contains too much black pixel than white. To increase the accuracy, we have
converted the background from black to white and foreground characters from white to black.
However, it decreases the accuracy. That’s why we have used the images which are written
on the black background with white foreground characters.

3.4 Classification Using Neural Network

Table. 3.3: Bangla 11 vowels

0 1 2 3 4 5

অ আ ই ঈ উ ঊ

6 7 8 9 10
ঋ এ ঐ ও ঔ
3.4.1 Sigmoid Neurons

Sometimes, multilayer networks are known as Multilayer Perceptron or MLP [10]. However,
Perceptron is not made up with sigmoid neurons and it has some uncertainty to get the best
output. In MLP, a little change in weights and biases creates a change in the whole network.
As opposed, in Sigmoid Neurons a little change in weights and biases makes a small change
in output. On the other hand, Sigmoid neuron can produce output as a real number between 0
to
1. However, perceptron can only produce 0 or 1. That’s why we have used Sigmoid Neurons
instead of Multilayer Perceptron terminology in our experiment. Sigmoid Neurons
terminology is a modified form of MLP.

X1

X2 Output

X3

Figure 3.4: Input-Output Structure of Sigmoid Neurons

Generally, Sigmoid Neuron has input like X1, X2, X3…….and so on. These inputs can take
values between 0 to 1. For example, 0.89 is a valid input for a sigmoid neuron. For each input
sigmoid neuron has weights like W1, W2, W2……... and a bias for all the inputs and weights.
Without σ (ω. X + b) the output will not be 0 or 1. Here, σ is called sigmoid function [10]
which is defined by:

σ(z) = 1/(1 + 𝑒𝑧)

When z = ω. X + b produces a very large number then the sigmoid neuron will be 1 and when
this function produces a very negative value then sigmoid neuron will be 0. Sometimes, σ is
also called logistic function and the new class of neurons called logistic neurons.
3.4.2 Gradient Descent

To learn dataset from training data is the first task of the network. In our experiment notation
x denoted training data. Each input takes 28×28=784 dimensional vector and every single
pixel represents gray value. We denote corresponding desired output by y=y(x). Here, y is an
11
dimensional vector. Suppose there is a training input which is an image of “ই” then the
desired output of the network is y(x) = (0,0,1,0,0,0,0,0,0,0,0) T. Here, T is a transpose
operation which turning a row vector into a column vector.

For this task. we required an algorithm which can find weights and biases so that for all
training inputs, the output of the network approximate y(x). To achieve this goal, we need a
cost function which can find a set of weights and biases which make the cost as small as
possible. We have done this task by using an algorithm which is known as gradient descent.
The cost function of the network defined by:
𝐶(𝑤, 𝑏) ≡ 12𝑛 ∑ 𝑥||𝑦(𝑥) − 𝑎||2 [10]
In this function C is the quadratic cost function, the collection of all weights and biases
respectively denoted by w and b, n represents the total number of training inputs when x is
input a is the vector of outputs from the network. output a always depends on x, w, and b. C is
also known as Mean Square Error or MSE. When the value of C(w,b) will be very large then
it can’t make a good result. However, when C(w,b)=0 then our training algorithm will do a
good job. That’s why, the goal of training algorithm is to minimize the cost C(w,b) which is a
function of weights and biases.

3.4.3 Three Layers of the Proposed Neural Network

A neural network has three layers. Previously we have mentioned these three layers. Here, we
have drawn the three layers’ architecture of our networks:
 Hidden Layer (40 Nodes)
Output
Layer
Input Layer (11 outputs)
(11 Inputs)

Figure: Three Layers Architecture of Bangla Handwritten Characters Classification

using Neural Network
we have mentioned Previously that there are three layers in a neural network. The leftmost
layer is called the input layer and the input neurons of the input layer are called input
neurons. The rightmost layer is called the output layer and neurons within this layer are called
output neurons. The middle layer is known as the hidden layer. Neurons of this layer are not
input nor outputs. The term hidden layer sounds like it’s a mysterious part. Generally, it
means "not an input or an output". We have shown the structure of the neural network of our
experiment in figure 3.4 which has a single hidden layer with 40 nodes.
Values of the input pixels are encoded by the neurons. This neuron contained by the input
layer of the network. As mentioned earlier, for this network we used training data which are
Bangla handwritten scanned characters(Vowel) and images are consist of 28 by 28 pixels.
That means, 28×28=784 neurons are contained by this input layer. We used the grayscale
image for input layer. In greyscale image white pixels are representing with a value of 0.0 and
black pixels are representing with 1.0. Other dark shades are representing with the value
between 0.0-1.0.

Figure 3.4: Raw Input Images

Figure 3.5: Corresponding resized (28×28) images

The second layer that means the middle layer is known as the hidden layer. The number of
neurons of the hidden layer is denoted by n. In this experiment, we have used many values of
n for gain the best result. i.e. we got 54% and 54% accuracy for n=30 and n=20 respectively.
Finally, we used n=40 because this value gives us 63% accuracy which is the best output in
our experiment.

The third layer which represents the output of the experiment contains 11 neurons. When the
first neuron fire then the output will be 1 and the network will think that the character will be
“অ”. Similarly, When the second neuron fires, the network will think that the character is “আ”
and It will occur for others character also. All the output neurons precisely numbered from 0
to 10 and the task is to find out the neuron with large activation value. If a neuron represents
the value 4 then the network will conceive that the input character was “উ” and it will be
similar for other output neurons.

3.5 Chapter Summary

In this chapter, we have described the structure of neural network which we used for Bangla
handwritten character classification. Four major steps such as: Dataset preparation, proposed
strategy, preprocessing of the raw character image, Classification using NN are explained in
this section which has described the architecture of BHCC-NN in details. Dataset preparation,
proposed strategy, preprocessing of the raw character image are explained at the beginning
for better understanding.
4 Implementation
Classification of Bangla Handwritten characters is not much easy like some other language.
It’s a complex and different task than some other language. We used deep neural network to
solve this task. Our proposed method can classify Bangla Handwritten Characters(Vowel).
We have experimented with a famous and global architecture. In this section we will describe
the implementation of the method which we used to Classify Bangla Handwritten Characters.

4.1 Implementation

Our dataset has a total of 11 classes. In this experiment, we have applied the different ratio of
training and testing dataset in the method. 80:20 ratio gives us the best output. We used core
i5(gen) with 4 gigabyte RAM on Windows 10(64 bit) and we used MATLAB software for
implementation.

4.1.1 Description of Configuration Code

The raw dataset contains different size of images and the dimension of the dataset was too
large, where 180×180 dimension was the minimum size of an image. Which required a large
storage. Due to storage insufficiency, we need such images which contain the number of
pixel as small as possible and we also need an equal size of inputs for our proposed method.
However, the appearance of the image has to be clear. That’s why at first we have resized all
the raw image into different size. From those size, 28×28 dimension gives us the better
output, which images also required a small storage of the memory and then rename all the
image to get a better output.
All the codes are given in the Appendix section. We will try to describe all the code here.

Description of Code for Resized and Rename Dataset:

At first we have declared the location of the images into a variable. The variable name is
“myfolder”. “A” is another variable and it contains the name of the single image. If the folder
is empty or there are no “.png” file exist or this folder does not existed, then the code will
show the error message “Error: The following folder does not exist”.
If there is no error occurred, then it will read all the image one by one and resize them. It also
renames the images. Imresize (imageArray,[28 28]) resized the image into 28×28 dimension
and imwrite (I2,sprintf( '%04d.png', k )) rename the resized image and save them.
After resize and rename the all training and testing image we have labeled all the image and
create two “. mat” files.
Read Image from folder, Convert them Floating Value and labeling them:
In this section, we labelled all the data and after labelling, we randomly shuffled it for the
bangla_vowel.matfile which is same as their matfile. At first, we take all the data from the
device folder and read all the images individually. On the matrices, it shows the pixel value
between 0 to 255 in 28 by 28 columns.
 Figure 4.1: decimal valued image
We used Sigmoid Neurons in our work. Sigmoid Neurons required real number of value from
0 to 1. That’s why we need to show the float value of all the images. So we convert our
dataset from decimal to double. We create a cell ‘doub1’ and insert all double value into this
cell.

Figure 4.2: double valued image

Then we make a structure S1 which has two columns- Image and Label (S1 = struct(‘Image’,
[], ‘Label’, ‘’)). The Image column of S1 structure is taken all the double Images in one row
with 784 column that’s why we write the code S1(i1).Image = doub1(1:784).

Besides, the Label column of S1 structure, we give the one label value of all same character
Such as for ‘অ’ the label is 0, for ‘আ’ the label value is 1 etc.
Similarly, for the other vowel, they are made in the same way.
 Figure 4.3: Image under labelling

Figure 4.4: images of doubled value under labelling

Description of Code for Shuffle and label the Training Data:

we put all the structure S1, S2, S3…, S11 in a variable called training_A. Then we take a
2dimensional matrix and put the training_A into this matrix according the row-column sizes.
Then we call a function ‘repmat’ which creates a large matrix randomly of X by Y tilling and
its accomplish the same result as the training_A. The first six lines are mainly used for
shuffling the images and label.
 Figure 4.5: Images after shuffling

And the rests are for dividing the Image column and label column in different matrices.
Figure 4.6: All data in 1 matrix (5500*784)
 Figure 4.7: Image’s label in 1 matrix(1*5500)

Code for Creating matfile for Shuffled images and Labelled matfile is made for load the
shuffled images and the labelled images in different matrix. We make two matfile where we
load this values.

Figure 4.8: Labelled and image matfile

 Figure 4.9: the structure of labelled and image matfile

This is our main matfile ‘bangla_vowel.mat’ code where we make 3*2 cell and every cell
contains the total images of training, testing and validation images and their labelling.

Figure 4.10: The main matfile (bangla_vowel.mat)

 Figure 4.11: The structure of ‘bangla_vowel.mat’ matfile

Sigmoid:
As mentioned previously, we have used Sigmoid Neuron terminology. SIGMOID use
sigmoid function to calculate the value of neurons. y= sigmoid(X) is returning the sigmoid
function of the elements of the vector and y = 1./(1+exp(-X)) is described in 3.4.1 section.
Feedforward:
Activation value of every node are returned by Feedforward. Values returns after forwarding
propagation.
[y,y_dot] = feedforward(x,W,b,l) returns y , Activation value of each network nodes and the
corresponding derivative, computed sigmoid function. They are using and the derivative of
the sigmoid function are respectively represented by y_dot. Here, W contains the value of
weights and b is the biases and l represents the number of layers in the network.

ValidateNetwork:
The number of test inputs returned by ValidateNetwork. It returns those test inputs for which
the neural network can produce the correct results.
The output of a network compared by correct = VALIDATENETWORK(X,Y,W,b,l) with a 1
by n vector. Y is representing the desired output and also return the number of correct values.
W represents weights and b is bias, the number of layers are represented by l and x is a m by
n matrix which represents the network’s input where m is the number of third layer’s nodes
and n is the number of test data.
Correct=0 is initializing the output of the function.a{1} = X(:,i) fill in the input layers and
[~,idx] = max(a{l})is used for finding the maximum index value of the output layer. [a,~] =
feedforward(a,W,b,l) is written for fill the hidden layer nodes up to the output layer.
updateWeightBias:
The task of UpdateWeightBias is to update the value of weights and biases of the network
using gradient descent and backpropagation. Here,
[W,b]=updateWeightBias(X,Y,eta,W,b,n,l) computes the new values of weights w and biases
b of the network where network use l as layers, matrix X as a set of network inputs, matrix Y
as a set of output which is desired by the network.
size(X,1) is the number of nodes of the input and size(Y,2) is the number of nodes of the
output when size(X,2) and size(Y,2) are the numbers of training data respectively. The
number of nodes in each network’s layer is stored in n and eta is representing the learning
rate.
SigmoidPrime:
The derivative of the sigmoid function is computed by sigmoidPrime. Where,
y=sigmoidPrime(x) returns the sigmoid function derivative. They are elements of X.
backProp:
[dnabla_W,dnabla_b] = backprop(X,Y,W,b,l) returns dnabla_W and dnable_b which
respectively represents the gradients of weights W and biases b of the network. Where l is the
layers of the network, vector X is the network input and vector Y is the desired output. The
task of backProp is to return the gradient of the weights and biases.
a{1} = X fill in the input nodes and [a,a_dot] = feedforward(a,W,b,l) fill in the hidden layer
up to the output layer.
characterClassification (Initialize Parameter and load mat file):
Here, nh= [20] represents one hidden layer which has 40 nodes. The variable epoch and mini
batch size represent the expected number of the epoch to train and mini batch size use when
sampling. After this part, bangla_vowel.mat file will be loaded by using
load(‘bangla_vowel.mat’)
Now, creating a network is the most important part of this implementation. The code is given
below which we used to initialize network object:
characterClassification (Create Network):
The network contains l = 1+length(nh)+1layers. Where, nh is the number of nodes in the
hidden layer. Each node in the input layers are corresponding with the pixel of each
image and in the output layer each node corresponds with each character (অ, আ, ই, ঈ, উ,
ঊ, ঋ, এ, ঐ, ও, ঔ). The nodes are initialized by n = zeros(1,1).n(1) =
size(bangla_vowel{1,1},2) is the
number of nodes on input layer and n(1)=11 is the number of nodes on output layer.
characterClassification (Initialize the weights and biases):
A random number generator is used to generate weights and biases randomly. Normally, a
random number is with mean 0 and variance 1. rng(0,'twister’) initializes the random number
generator. W=cell([1,l-1]) and b= cell([1,l-1]) initialize the weights and bias respectively.

4.1.2 Results

To optimize result we have used several types of handwritten dataset. We have

experiminented with different values of parameter also and diferent values of parameter gave
us different accuracy. We have changed in the number of nodes in hidden layer, batch size,
learning rate. On the other side, The different ratio of training and testing data make a
mutation in the test result. At first, we have experimented with the same testing and training
dataset. Which gave us 99% accuracy. Apart from this, in every case, we used a different set
of training and testing data respectively. As opposed, we have also tried to experiment with
the different size of image. Unfortunately, additional cost and training time required for the
increasing image size. We will demonstrate the results of all the experiments in this section
which we have done with different values of parameter and different ratio of testing and
training data.
To check the performance of our system, at first we have experimented with the same training
and testing dataset. Result of this experiment is giving below:
Figure 4.12: Result(99% accuracy) for Same Training and Testing Dataset
After that, we used totally different data in training and testing dataset respectively. Results
and parameters of the experiment using real data are showing below:

Figure 4.13: Result(74% accuracy) for Different Training and Testing Dataset

In this experiment, we used total 5,500 data for training and 1,375 data for testing. We have
used totally different types of data where training data and testing data were collected from
two different sources. We used training dataset from BanglaLekha-Isolated[8] and testing
dataset from BasicFinalDatabase[11-16]. Where the shape of training and testing data are
totally different and after resizing the testing dataset the were being obscure. We have used
the 80:20 ratio of training and testing data here.
Figure 4.14: Some Training Images for this Experiment

Figure 4.15: Some Testing Images for this Experiment

To enhance the accuracy, we have increased the training and testing dataset. Here, we also
used taring data from BanglaLekha-Isolated[8] and testing data from BasicFinalDatabase[11-
16]. We used total 10,560 training data where 980 training data exist for each alphabet and
2,640 testing data, where 240 testing data used for each alphabet.

Figure 4.16: Values of the Parameter and Result(69% accuracy) of the Experiment

This experiment gives us 69% accuracy. Where, we used 30 nodes of hidden layer, 15
epochs, 10 mini batch size, 3 learning rate. We used black and white images with a black
background and white foreground. Here, we also used 80:20 ratios. But, this experiment
didn’t give us any improved accuracy. It has given us 69% accuracy which is lower than the
previous one.
Then, we use complementary images, where we used the white background and black
foreground. The result of the experiment given below:

Figure 4.17: Values of the Parameter and Result(51% accuracy) of the Experiment

Here, we increase the number of epochs. We used 20 epochs here. However, this experiment
gave us 51.4% accuracy which is inferior to the previous one.
After these three experiments, we used training and testing data from BanglaLekha-
Isolated[8]. Where training and testing data are different but after resize this dataset it
produces almost clear image than the previous testing dataset. We used 5,500 training data
and 1375 testing data here.
Figure 4.18: Some Training Images for this Experiment

Figure 4.19: Some Testing Images for this Experiment

The results and parameter of the experiment are showing below:

Figure 4.20: Values of the Parameter and Result(84% accuracy) of the Experiment

Here we used 80:20 ratio of training and testing data and it give us a better accuracy which is
a. Here, we have used 30 nodes of the hidden layer, 15 epochs, 10 mini batch size and 3
learning rate which is represented by eta.
After this experiment, we used mixed data for testing dataset where we used 50 data from
training dataset and 75 different data. We also increase the number of nodes in the hidden
layer. Where, we have increased the number of epochs also.
The result of the experiment given below:

Figure 4.21: Values of the Parameter and Result(89% accuracy) of the Experiment

This experiment give us 89% accuracy which is better than other experiment where we usd
totally different training and testing dataset. 80:20 ratio of training and testing dataset is used
here.
We also experimented with different ratio of training and testing dataset. However, 80:20
ratio of dataset giving us the best accuracy. From another ratio 5:1 gives us the better
accuracy.
We have given the accuracy of 5:1 ratio’s training and testing data below:

Figure 4.22: Values of the Parameter and Result(75% accuracy) of the Experiment

Here, we used total 5,500 training data and 1,100 testing data. 30 nodes of the hidden layer,
15 epochs, 10 mini batch size, 3 learning rates have used as parameter. This experiment gives
us 75% accuracy which is lower than the accuracy which was achieved by using 80:20 ratio
of training and testing data.
 Table 4.1: Results for Different Dataset

Training Data Testing Data Data Types Ratio Result(Accuracy)

5500 1375 Same Training and 80:20 99%
Testing Data from
BanglaLekha-
Isolated[8] Training
Dataset
5500 1375 Different Training Data 80:20 74%
from BanglaLekha-
Isolated[8] Training
Dataset and Testing
Data from
BasicFinalDatabase[11
-
16] Training Dataset
10560 2640 Different Training Data 80:20 69%
from BanglaLekha-
Isolated[8] Training
Dataset and Testing
Data from
BasicFinalDatabase[11
- 16] Training Dataset
10560 2640 Different Training Data 80:20 51%
from BanglaLekha-
Isolated[8] Training
Dataset and Testing
Data from
BasicFinalDatabase[11
- 16] Training Dataset,
using Complementary
Image
5500 1375 Different Training Data 80:20 84%
from BanglaLekha-
Isolated[8] Training
Dataset and Testing
Data from
BanglaLekha-
Isolated[8] Testing
Dataset
 5500 1375 Mixed Training Data 80:20 89%
from BanglaLekha-
Isolated[8] Training
Dataset and Testing
Data from
BanglaLekha-
Isolated[8] Testing
Dataset
5500 1100 Different Training Data 50:10 75%
from BanglaLekha-
Isolated[8] Training
Dataset and Testing
Data from
BanglaLekha-
Isolated[8] Testing
Datase

4.2 Chapter Summary

In this chapter, we explained about the implementation and result of our work. In
implementation section we tried to describe all the codes which we have used in Bangla
Handwritten Character Classification using Neural Network (BHCC-NN) and we explained
the result of different experiment in result section. We tried to exhibit a comparative result in
this section.
5 Conclusion
We have tried to show that, a better performance can be achieved in Classification of Bangla
Handwritten Characters into digitally readable formats by using Neural Network (NN). To
achieved a better performance, it required effective utilization of Neural Network. It is an
efficient model than some other shallow learning method like MLP and SVM. Due to
inadequate hardware support, we have faced the deficiency of memory storage. Moreover, the
accuracy would have been better, if we had more hardware support. With more resources and
bigger Neural Network (NN), we can achieve a better result in future and can enhance the
status of Bangla Handwritten Character Classification.

5.1 Limitations

The prosperity of implementation obstructs by several factors. Sometimes, these obstacles

eroded our time, sometimes created some restrictions. For example, we have faced with
insufficiency of RAM and this insufficiency was limiting the accuracy. Since we used an
ordinary GPU and CPU, increasing input makes various complexity. For example, a large
Image size created insufficiency of memory. That’s why we have to resize all the image into
28*28 dimension whereas, images larger than 28*28required a large memory storage which
makes the whole system too slow. On the other side, we used only 5500 train data. Because,
when we tried to use a large dataset the system was obstructed due to insufficient memory.
Bangla Handwritten Dataset is not so available also. A small number of the dataset are
available online where included data are not good enough. However, in deep Neural Network,
a large training dataset is a big factor of better accuracy. Increasing number of nodes in the
hidden layer also makes the system slower. Increasing number of hidden layer, Mini batch
size would greatly increase the training time needed for the model also. These problems
forced us to compromise and work on a simple network instead of a complex network.
However, we have tried to fix all the restrictions. Moreover, we can say that better
opportunities can improve the efficiency of the model.

5.2 Future Work

Our future works for this system will be to increase the obtained accuracy by using highly
configured hardware which will allow us to use a more complex network. For insufficiency
of memory, we can’t use larger data than 28*28 pixel. On the other side, the dataset of
Bangla character consists of 84 classes, which is constructed with 50 class of basic letters, 10
class of numeral and 24 classes are for often used compound letters. However, we worked
with only 11 characters(vowels) or classes. In future we will try to increase the number of
classes to include all the characters of Bangla handwritten dataset and also this can be
improved as handwritten sentence classifier.
References

[1] L. Pessoa and P. Maragos, "Neural networks with hybrid morphological/rank/linear

nodes: a unifying framework with applications to handwritten character recognition",
Pattern Recognition, vol. 33, no. 6, pp. 945-960, 2000.
[2] Gilloux, M., Gallinari, P. and Bellili, A. (2003). An MLP-SVM combination architecture
for offline handwritten digit recognition. International Journal on Document Analysis and
Recognition, 5(4), pp.244-252.
[3] Dong, J., Krzyżak, A. and Suen, C. (2005). An improved handwritten Chinese character
recognition system using support vector machine. Pattern Recognition Letters, 26(12),
pp.1849-1856..
[4] Vamvakas, G., Gatos, B. and Perantonis, S. (2010). Handwritten character recognition
through two-stage foreground sub-sampling. Pattern Recognition, 43(8), pp.2807-2816..
[5] Bhowmik, T., Ghanty, P., Roy, A. and Parui, S. (2009). SVM-based hierarchical
architectures for handwritten Bangla character recognition. International Journal on
Document Analysis and Recognition (IJDAR), 12(2), pp.97-108.
[6] Basu, S., Das, N., Sarkar, R., Kundu, M., Nasipuri, M. and Basu, D. (2009). A
hierarchical approach to recognition of handwritten Bangla characters. Pattern
Recognition, 42(7), pp.1467-1484.
[7] Bhattacharya, U., Shridhar, M., Parui, S., Sen, P. and Chaudhuri, B. (2012). Offline
recognition of handwritten Bangla characters: an efficient two-stage approach. Pattern
Analysis and Applications, 15(4), pp.445-458.
[8] Biswas, M., Islam, R., Shom, G., Shopon, M., Mohammed, N., Momen, S. and Abedin,
A. (2018). BanglaLekha-Isolated: A multi-purpose comprehensive dataset of Handwritten
Bangla Isolated characters.
[9] D. Wan, M. Zeiler, S. Zhang, S., Y. LeCun., & R. Fergus. Regularization of neural
networks using drop connect. in: Proceedings of the 30th International Conference on

Machine Learning (ICML-13), pp. 1058–1066.

[10] M. Nielsen, "Neural Networks and Deep
Learning", Neuralnetworksanddeeplearning.com, 2018. [Online]. Available:
http://neuralnetworksanddeeplearning.com/chap1.html. [Accessed: 23- Oct- 2018].
[11] N. Das, R. Sarkar, S. Basu, M. Kundu, M. Nasipuri and D. Basu, "A genetic
algorithm based region sampling for selection of local features in handwritten digit
recognition application", Applied Soft Computing, vol. 12, no. 5, pp. 1592-1606, 2012.
[12] B. Duerr, W. Haettich, H. Tropf and G. Winkler, "A combination of statistical and
syntactical pattern recognition applied to classification of unconstrained handwritten
numerals", Pattern Recognition, vol. 12, no. 3, pp. 189-199, 1980.
[13] S. Basu, N. Das, R. Sarkar, M. Kundu, M. Nasipuri and D. Basu, "A hierarchical
approach to recognition of handwritten Bangla characters", Pattern Recognition, vol. 42,
no. 7, pp. 1467-1484, 2009.
[14] N. Das, S. Basu, R. Sarkar, M. Kundu, M. Nasipuri, and D. K. Basu, "Handwritten
Bangla Compound character recognition: Potential challenges and probable solution," in
4th Indian International Conference on Artificial Intelligence, Bangalore, 2009, pp. 1901-
1913.
[15] N. Das, S. Basu, R. Sarkar, M. Kundu, M. Nasipuri, and D. K. Basu, "An Improved
Feature Descriptor for Recognition of Handwritten Bangla Alphabet," in International
conference on Signal and Image Processing, Mysore, India, 2009, pp. 451-454.
[16] N. Das, K. Acharya, R. Sarkar, S. Basu, M. Kundu, and M. Nasipuri, "A Benchmark
Data Base of Isolated Bangla Handwritten Compound Characters," IJDAR( Revised
version communicated)
Appendix

Configuration Code

Code for Resized and Rename Dataset

myFolder = 'C:\Users\User\Desktop\before resize';

A =dir( fullfile(myFolder, '*.png') );

fileNames = { A.name };

if ~isfolder(myFolder)
errorMessage = sprintf('Error: The following folder does not
exist:\n%s', myFolder);
uiwait(warndlg(errorMessage));
return;
end

filePattern = fullfile(myFolder, '*.png');

jpegFiles = dir(filePattern);

for k = 1:length(jpegFiles)
baseFileName = jpegFiles(k).name;
fullFileName = fullfile(myFolder, baseFileName);
fprintf(1, 'Now reading %s\n', fullFileName);
imageArray = imread(fullFileName);
drawnow;

I2 = imresize(imageArray,[28 28]);

imwrite(I2,sprintf( '%04d.png', k ));

end
Code for Read Image from folder, Convert them Floating Value and Labelling them:
folderPath1 = 'C:\Users\User\Desktop\train\1';

A1 =dir( fullfile(folderPath1, '*.png') );

fileNames1 = { A1.name };
filePattern1 = fullfile(folderPath1, '*.png');
pngFiles1 = dir(filePattern1);

totalFiles1 = numel(pngFiles1);
S1 = struct('Image', [], 'Label', '')
for i1 =1:totalFiles1

Fileaddress1{i1,1}=strcat(folderPath1,'\',pngFiles1(i1).name);
file1{i1} = imread(Fileaddress1{i1,1});
doub1{i1} = im2double(file1{i1}) ;

S1(i1).Image = doub1{i1}(1:784);

S1(i1).Label = 0;

end

Code for Shuffle and label the Training Data:

training_A = [S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11];

[X,Y] = size(training_A);

rowIndex_training = repmat((1:X)',[1 Y]);

[~,randomizedColIndex_training] = sort(rand(X,Y),2);

newLinearIndex_training=sub2ind([X,Y],rowIndex_training,random
izedColIndex_training);

B_training = training_A(newLinearIndex_training);

vals_training = [B_training(1:5500).Label]

B_new_training = int64(vals_training);
vals_train2=zeros(0,784);

fori=1:5500
vals_train1 = [B_training(i:i).Image];
vals_train2 = [vals_train2;vals_train1];
end
vals_training_single = single(vals_train2);
Code for Creating matfile for Shuffled images and Labelled matfile:

save('labelled_matfile.mat', 'B_new_training',
'B_new_testing', 'B_new_validation');
load_labelled_matfile = load('labelled_matfile.mat');

save('img_matfile.mat', 'vals_training_single',
'vals_testing_single', 'vals_validation_single');
load_img_matfile = load('img_matfile.mat');

Code for Creating Main Matfile “bangla_vowels”:

aa = load('img_matfile.mat');
bb = load('labelled_matfile.mat');

bangla_vowel = {aa.vals_training_single ,bb.B_new_training}

bangla_vowel(2,:) ={aa.vals_testing_single,bb.B_new_testing}
bangla_vowel(3,:) = {aa.vals_validation_single,
bb.B_new_validation}

save('bangla_vowel.mat', 'bangla_vowel');
bangla_vowel_load = load('bangla_vowel.mat');

Sigmoid:
function y = sigmoid(X)

y = 1./(1+exp(-X));
end

Feedforward:
function [y,y_dot] = feedforward(x,W,b,l)

y = x;
y_dot = cell(1,l-1);

fori = 1:l-1
y{i+1} = sigmoid(W{i}*y{i}+b{i});
y_dot{i} = sigmoidPrime(W{i}*y{i}+b{i});
end
end
ValidateNetwork:

function correct = validateNetwork(X,Y,W,b,l)

correct = 0;

fori = 1:size(Y,2)
a{1} = X(:,i);

[a,~] = feedforward(a,W,b,l);

[~,idx] = max(a{l});

if idx-1 == Y(i)
correct = correct + 1;
end
end
end

updateWeightBias:

function [W,b] = updateWeightBias(X,Y,eta,W,b,n,l)

% initialize the gradient of the weighting matrices and biases

nabla_W = cell([1,l-1]);
nabla_b = cell([1,l-1]);
fori = 1:l-1
nabla_W{i} = zeros(n(i+1),n(i));
nabla_b{i} = zeros(n(i+1),1);
end

% compute the gradient of the weights and biases using

backprogpagation
fori = 1:size(X,2)
[dnabla_W, dnabla_b] = backprop(X(:,i),Y(:,i),W,b,l);
for j = 1:l-1
nabla_W{j} = nabla_W{j}+dnabla_W{j};
nabla_b{j} = nabla_b{j}+dnabla_b{j};
end
end

% update the value of weight and bias

fori = 1:l-1
W{i} = W{i} - (eta/size(X,2)).*nabla_W{i};
 b{i} = b{i} - (eta/size(X,2)).*nabla_b{i};
end
end

SigmoidPrime:

function y = sigmoidPrime(X)

y = sigmoid(X).*(1-sigmoid(X));
end
backProp:

function [dnabla_W,dnabla_b] = backprop(X,Y,W,b,l)

a{1} = X;

[a,a_dot] = feedforward(a,W,b,l);

% gradient descent
delta = (a{end}-Y).*a_dot{end};

dnabla_W{l-1} = delta*a{l-1}';
dnabla_b{l-1} = delta;

fori = l-2:-1:1
delta = W{i+1}'*delta.*a_dot{i};
dnabla_W{i} = delta*a{i}';
dnabla_b{i} = delta;
end
end
characterClassification (Initialize Parameter and load mat file):
nh = [40];
epoch = 10;
mini_batch_size = 4;
eta = 9.0;
load('Bangla_Vowels.mat');

characterClassification (Create Network):

l = 1+length(nh)+1;

n = zeros(l,1);
n(1) = size(bangla_vowel{1,1},2);
for i = 1:length(nh)
n(1+i) = nh(i);
end

n(l) = 11;

characterClassification (Initialize the weights and biases):

rng(0,'twister');

W = cell([1,l-1]);
b = cell([1,l-1]);
for i = 1:l-1
W{i} = randn(n(i+1),n(i));
b{i} = randn(n(i+1),1);
end