2019 Global Conference for Advancement in Technology (GCAT)
Bangalore, India. Oct 18-20, 2019
Offline Character recognition on Segmented
Handwritten Kannada Characters
Kevin George Joe
Computer Science and Engineering Computer Science and Engineering
National Institute of Technology National Institute of Technology Karnataka
Karnataka
Surathkal, India meghna.savit99@gmail.com
kevingeorgejoe@gmail.com
Abstract—Optical character recognition (OCR) is the
conversion of pictures of typed or handwritten characters into
machine encoded characters. We chose to work on a subfield of
OCR, namely offline learning of handwritten characters.
Kannada script is agglutinative, where simple shapes are
concatenated horizontally to form words. This paper presents a
comparative study between different machine learning and
deep learning models on Kannada characters.A Convolutional
Neural Network (CNN) was chosen to show that handcrafted
features are not required for recognizing classes to which
characters belong to. The CNN beats the accuracy score of
previous models by 5%.
Keywords—Optical Character Recognition, Handwritten
Kannada Characters, Convolutional Neural Network
I. INTRODUCTION
Handwritten text is the most widely used method for
recording information in everyday life. Handwriting
Recognition is the ability of a computer to detect and
process handwritten input from images, photographs,
reports etc. Offline recognition is centered around
converting text from an image to codes which can be
processed by a computer application such as text editors.
The offline handwritten recognition system accepts images
of handwritten documents as input and the recognized
characters from document image are outputted. Handwritten
character recognition has a wide range of applications, from
recognizing number plates of automobiles to processing
bank cheques and form data entry. The challenge of
recognising and classifying handwritten characters lies in
the difference between the handwriting styles of different
writers. Each person has a unique handwriting. Some have
handwriting that is difficult to decipher the . Some
documents may be damaged therefore it is difficult to
recognize the characters in those documents. There might be
variations in font styles and thickness. In addition to this,
data is often presented in the form of a scanned image or
photograph which requires accounting for sources of
variability like camera positions that cause distortion,
illumination and background and foreground colour.
Handwriting recognition has become a popular research
area in the past couple of decades. HWR of english
characters is a relatively easy and an already conquered
problem space. However, offline HWR systems have been
developed for only a few languages and most Indic
languages like Kannada are still beyond the pale of current
HWR techniques. Indic scripts have rich morphological
structure where in a character can lead to many other
978-1-7281-3694-3/19/$31.00 ©2019 IEEE
Meghna Savit K. Chandrasekaran
Computer Science and Engineering
National Institute of Technology
Surathkal, India Karnataka
Surathkal, India
kchnitk@ieee.org
characters simply by the addition of a single dot or stroke.
This results in a large number of classes (657). Modern
Kannada script has 34 consonants, 13 vowels as well as two
other letters . All consonants combine with all the vowels to
form consonant-vowel combinations. All possible
consonants and consonant-vowel combinations which are
possible, have been considered, Figure 1.
The dataset was chosen from [1] and worked exclusively
on the offline handwritten Kannada characters. Three
different models were implemented on the dataset including
a Random Forest Classifier(RFC)[6], a MultiNomial Naive
Bayes Classifier (MNNBC)[7] and a CNN[8]. Optical
Character Recognition (OCR) using machine learning stated
above fared in the same accuracy percentage as that of [1].
The CNN model was chosen due to the promising results on
[2] and [3] to test if using a deep learning model with no
feature extraction would improve accuracy compared to
methods deployed in [1] and [9]. We show that deep
learning techniques like CNN, even without feature
extraction would show a higher accuracy as compared to the
machine learning models.
II. BACKGROUND
The following are attempts made by various authors on
the topic of optical character recognition of Indic languages.
This section briefly discusses the methods used for feature
extraction, data pre-processing and classifying that were
used for similar problems in the past.
In [21] authors attempt offline cursive handwritten tamil
character recognition using HMM and feature extraction
using a combination of Time domain and frequency domain
features. Their work focuses on Tamil words rather than
isolated characters. In [22] researched attempt both online
and offline character recognition for Bangla numerals using
a database created from Bangla numerals which were
extracted from postal addresses using state of the art flatbed
scanner. In [23] authors outline various methods that were
and are being used for character recognition outlining
feature extraction methods like Fourier, Cosine, Slant or
Wavelet transform. They outline other methods like Fuzzy
rules, Mahalanobis and Hausdorff distance, Evolutionary
algorithms and explain why they aren’t used on Indic
scripts. The paper also goes into detail about the various
advances made in character recognition for Indic scripts. In
[24] authors propose handwritten Kannada character
recognition based on Fisher linear discriminant analysis
(FLD)[25]. Their system extracts performs feature
1
extraction using FLD, Two dimensional FLD (2D-FLD) and
diagonal FLD. And uses distance measure techniques for the
purpose of classification. In [1] authors performed
classification using algorithms like Nearest Neighbors[13]
and Support Vector Machines[14] using feature extraction
methods like Shape Contexts[15], Geometric Blur[16],
Scale Invariant Feature Transform[17], Patch
Descriptor[18]. In [26] the authors subject the scanned
Kannada image by a flatbed scanner to pre-processing
which eliminates noise followed by line, word and character
segmentation is performed to segment the characters.
Moment features are extracted using Hu and Zernike
moments passed to a neural classifier.
III. PROBLEM DESCRIPTION
Character recognition is a field that has made a lot
of progress in recent years and shows great field for
improve-ment. Unlike Romance Languages like Spanish,
Portuguese,French, Italian, English etc. where there has
been a great deal of research done in handwritten
character recognition, Indic languages remain relatively
uncharted territory. The goal is to be able to increase the
accuracy of character recognition of handwritten
Kannada characters than previous attempts like[1] and
[9]. These characters are segmented individually andare
collected by [1] where the authors go into full detail as
to how they collected and pre-processed the dataset.
This problem is a supervised offline learning problem
and refers to situations where the program is not
operating and takingin new information in real time.
Instead, it has a static set of input data.
IV. PROPOSED SOLUTION
This section explains the methods that were followed to
solve the problem of recognizing characters from Indic
scripts, particularly Kannada. It explains how the dataset
was acquired and preprocessed as well as the machine
learning models that were implemented.
A. The Dataset
The Char74k Dataset consists of 657 Kannada characters
collected using a tablet PC. Each character has 25 samples.
The samples have different stroke thickness and style, just
as in the case of most handwritten characters. There are
several modifiers which are added to the base class
characters to obtain new characters which belong to the
modified class. The base classes and the modified classes
cover all the consonant vowel combinations. The 657
characters are either vowels, base classes, modified classes
or numeric digits.
The dataset consists of images of street scenes taken in
Ban- galore. In addition to these images there are also
handwritten characters and other characters generated by
computer fonts. From the images, the individual characters
were obtained by manual segmentation. The handwritten
characters were captured using a tablet PC.
B. Data Collection and Preprocessing
Most of data pre-processing was done converting the
2
grayscale 900x1200 .png files of the Char74k dataset [1] to
an intermediary file of 30x40. png and reading individual
pixel values of the intermediary image to a csv which was
uploaded to a cloud based system. A cloud graphics
processing unit (GPU) was used to train all other models. It
was observed that the RFC and MNNBC models chosen
showed no improvement on using the GPU. Whereas, the
CNN trained approximately three times faster.
Each image from the handwritten Kannada characters
was re- sized to make it smaller by a factor of 30, for easy
computation during machine training. Each pixel value was
written to .csv file where the first column stood for the label
of the Kannada character and the subsequent columns
stored the value of each pixel. The value of each pixel was
normalized by dividing by 255. This was done in order to
speed up the time it takes for a model to converge to
the global optima.
The data was split into a train and validation set where
the validation set is 10% of the original dataset. Then we
trained our various different models on this new training
set.
C. Random Forest Classifier
RFC is a machine learning model that creates a collection
of decision trees from a subset of the training set. This
model can be implemented for both classification as well as
regression problems.It the aggregates the votes to
decide the class of the object. The RFC was imple-
mented setting the following parameters as
n_estimators=100, min_samples_split=10,
Fig. 1. Pairs of similar kannada characters
Fig. 2. Kannada handwritten characters in the Char74k dataset collected
using a tablet PC[9]
min_samples_leaf=5.
It works on the principle of bootstrap
aggregating(bagging). It is a meta-algorithm, which takes M
sub-samples (with replacement) from the initial dataset and
trains the predic- tive model on those subsamples. Random
Forest introduces randomness to the model, while growing
the decision trees. The final model is obtained by averaging
the ”bootstrapped” models and usually yields better results.
Bagging can be used to provide ongoing estimation of
overfitting error as well as the estimation of strength and
correlation.
The drawback of a RFC is that they take a considerable
amount of time for classification due to the decision tree
type structure. The decisions and classes predicted must be
 aggregated.
D. Multinomial Naive Bayes Classifier
MNNBC[7] is a generative model. The principle behind
MNNBC is probabilities that each pixel values is
independent from each other. They are probabilistic
classifiers, therefore will calculate the probability of each
category using Bayes theorem. The MNNBC starts by
calculating the previous prob- ability of each label. This
was implemented by checking the frequency of all the
labels in the training set. The contribution of each character
is then combined with this prior probability to obtain
estimates for each of the labels in the dataset. The label
with the highest probability will be output.
A subtle issue with the MNNBC is that if there are no
occurrences of a class label and a certain attribute value
together then the frequency-based probability estimate will
be zero. Given the conditional independence assumption,
when the probabilities are multiplied you will get zero. This
model also assumes that the features are independent of
each other, which is not always true.
E. Convolutional Neural Network
Convolutional Neural Networks (CNNs)[8] are
comprised of neurons that self-optimise through learning.
Each neuron receives an input and perform a specific
operation. From the input raw image vectors to the final
output of the class score, the entire of the network will still
express a single perceptive score function (the weight). The
last layer contains loss functions associated with the
classes. CNNs are primarily used in the field of pattern
recognition within images. CNN was implemented using
Keras as an API. The CNN architecture implemented
consists of the following layers. The first is the
convolutional (Conv2D) layer. It is a set of learnable filters
that creates a convolution kernel that is convolved with the
layer input over a single spatial dimension. The first layer
uses 32 filters and the next uses 64. Each filter transforms a
part of the image using the kernel filter. The kernel filter
matrix is applied on the whole image. The parameters for
the first Conv2D layer were set as kernel size= (3, 3),
activation = ’relu’ and input_shap = input_shape. The
parameters for the second Conv2D layer were set as
kernel_size = (3, 3) and activation = ’relu’. The activation
function used is Rectified Linear Unit(ReLu). The ReLu
activation function accelerates convergence and prevents all
the neurons from firing in an analogous manner, ensuring
that a few neurons in the network do not activate. This
makes the activations more sparse and efficient. The
rectifier activation function is used to add non linearity to
the network. The next layer in the CNN is the pooling
(MaxPool2D) layer. It is a Max pooling operation for
spatial data. It examines the 2 neighboring pixels and picks
the maximal value. This operation is performed to reduce
computational cost, and to some extent also reduce
overfitting. Higher the pooling dimension, greater is the
downsampling. The parameters were set as
pool_size=(2,2)|. The remaining parameters were assigned
the default sklearn values. These parameters will halve the
image in each dimension. Combining convolutional and
3
pooling layers, CNNs are able to combine local features and
learn more global features of the image. The Dropout layer
randomly drops a portion of the network and forces it to
learn features in a distributed way. Dropout is a
regularization method, where a proportion of nodes in the
layer are randomly ignored for each training sample.
Dropout is done to improve generalization and reduce
overfitting. The probability of dropping is set to 0.25. The
Flatten layer is used to convert the final feature maps into a
single 1D vector. This flattening step is required to make
use of fully connected layers after some
convolutional/maxpool layers. It combines all the found
local features of the previous convolutional layers. A Dense
layer is applied which is an artificial neural network(ANN)
classifier. The parameter values are set as activation=’relu’.
An additional Dropout layer is applied in which the
probability of dropping is set to 0.5. Ultimately the features
are used in two fully-connected (Dense) layers. The last
layer (Dense(657, activation="softmax")) the net outputs
distribution of probability of each class. The softmax
activation function returns the probabilities of classes which
sum to one. The model is compiled and an F1 score is use
to evaluate the model. F1 is an overall measure of a model’s
accuracy that combines precision and recall. A good F1
score indicates infrequent false positives and false
negatives. The parameters are set as loss =
keras.losses.categorical_crossentropy, optimizer =
keras.optimizers.Adadelta(), metrics= [f1]. The CNN is
trained for 128 epochs(iterations on a dataset). The
arguments of the fit function are passed as x_train, y_train,
batch_size= batch_size, epochs = 128, verbose = 1,
validation_data= (x_val, y_val). The value of batch_size
was set to 128.
V. RESULTS
Comparing our accuracies to the ones show in papers [1]
and [9] we can see that our CNN architecture outperforms 4
Fig. 3. Time taken by different Classifiers. all other models
and shows an improvement in accuracy by 5 % compared to
[9] even without manual feature extraction as shown in
Figure 2. But our machine learning models suffer the same
low accuracy as that in [1]. Discussed in Section VI are the
possible reasons for the differences in accuracies. These
results were calculated on a cloud GPU. The CNN predicts
the handwritten Kannada characters with an accuracy of 57
percent whereas the other models show low accuracy.
Random Forest Classifier shows an accuracy of 5 percent
whereas Multinomial Naive Bayes Classifier shows an
accuracy of 4 percent. We see a higher accuracy due to the
deep learning nature of the CNN algorithm. The time taken
by the CNN to train is significantly higher than that of
either the machine learning models at 139980 seconds. The
random forest takes 90000 seconds followed by the
Multinomial Naive Bayes Classifier at 947 seconds. One of
the reasons the CNN took almost 1.5 to 2 times longer to
train than the machine learning algorithm is due to having
to train over multiple epochs, 128 in our experiment.

Fig. 3. Time taken by different Classifiers.
Fig. 4. Accuracy of different ClassifiersFig. 4.
Accuracy of different Classifiers
Fig. 4. Accuracy of different Classifiers
VI. DISCUSSION
In this paper we chose to tackle the problem of
handwritten character recognition. [1] showed an accuracy
of Fig. 4. Accuracy of different Classifiers 2.77% and 3.4%
using machine learning models like Nearest Neighbors[13]
and Support Vector Machines[14] using feature extraction
methods like Shape Contexts[15], Geometric Blue[16],
Scale Invariant Feature Transform[17], Patch
Descriptor[18] on handwritten Kannada characters. One
reason for the poor accuracy is the fact that Kannada is
agglutinative and the fact that handwriting varies from
person to person. Then, [9] showed an accuracy of using
Hidden Markov Models[19] using implicit segmentation.
Our machine learning models RFC and MNNBC perform
poorly in terms very similarly to [1] showing that we have
achieved a baseline for our experiment. The reason the
CNN implementation does so well with an accuracy of 57%
even without the feature extraction is due to convolutions,
multiple filters, dense and dropout layers highlighted in [20]
that the CNN leverages in order to find spatially invariant
features.
VII. CONCLUSION AND FUTURE WORK
In this paper we have shown that deep learning models ,
here the CNN trains and classifies handwritten characters
4
far more accurately than probabilistic classifiers like the
Multinomial Naive Bayes Classifier and Bootstrap
Aggregating methods such as Random Forest Classifier.
Future work will involve implementing a capsule
network[11], as it doesn’t have the same issue that a CNN
has with spatial features, that are vast in Indic languages. A
capsule neural network is a deep learning model that model
hierarchical relationships better. Capsule networks add
structures called capsules to a CNN, reuse output from
those capsules to perform classifications. In capsule
networks the extra layers are added inside a single layer.
They use minimal preprocessing as compared to other
classification models. The dynamic routing is performed
between capsules instead of neurons as in CNNs. The layers
are arranged in functional pods which enable designers to
distinguish between the various elements.The model can be
evaluated and compared to see if it outperforms the CNN
model. Manual feature extraction methods can be
implemented on CNN to see if they help improve the
accuracy of models. Feature extraction maximizes the
recognition rate with the least amount of elements. Feature
extraction starts from an initial set of measured data and
derives values (features) that are informative facilitating the
subsequent learning and classification steps, which leads to
more accurate interpretations. Methods like Zoning,
Histogram of Gradients[12] and Gradient Based
features[13] or other more recent feature extraction methods
may be implemented. We may improve on cross validation
scheme and use more Indic scripts that are agglutinated in
nature along the horizontal direction like Telugu,
Devanagari, Tamil etc. Cross-validation is a statistical
method used to estimate the skill of machine learning
models. Cross validation scheme is a resampling technique
used to evaluate the model when data is limited.
TABLE I RESULTS
Classifier Time taken to train Accuracy on test
and predict set
Random Forest 1 min 30 secs 5.234
MNNBC 947 ms 4.199
CNN 2 min 19 secs 57.002
REFERENCES
[1] M. V. T. E. de Campos, “Character recognition in natural images,”
VISAPP, 2009.
[2] [2] Marti and H. Bunke, “The IAM-database: an English sentence
database for offline handwriting recognition,” International Journal on
Document Analysis and Recognition, vol. 5.1
[3] Patil and S. Shimpi, “Handwritten English character recognition
using neural network,” Elixir Comput Sci Eng, vol. 41, pp. 5587–
5591, 2011.
[4] A. C. A. Y. N. T. Wang, “End-to-end text recognition with
convolutional neural networks,” International Conference on
Document Analysis and Recognition, 2011.
[5] B. B. K. Wang and S. Belongie, “End-to-End Scene Text
Recognition.”
[6] Breiman and Leo, “Random forests,” Machine learning, vol. 45.1, pp.
5–32, 2001.
[7] Rennie and J. D, “Tackling the poor assumptions of naive bayes text
classifiers,” Proceedings of the 20th international conference on
machine learning, 2003.
[8] I. S. Krizhevsky and G. E. Hinton, “Imagenet classification with deep
convolutional neural networks,” Advances in neural information
processing systems, 2012.
[9] V. M. Venkatesh and D. Vijayasenan, “Implicit segmentation of
Kannada characters in offline handwriting recognition using hidden
Markov models.”
[10] Smith and Ray, “An overview of the Tesseract OCR engine,” Ninth
International Conference on Document Analysis and Recognition,
vol. 2, 2007.
[11] N. F. Sabour and G. E. Hinton, “Dynamic routing between capsules,”
2017.
[12] Dalal and B. Triggs, “Histograms of oriented gradients for human
detection,” International Conference on computer vision & Pattern
Recognition, vol. 1, 2005.
[13] Cover and P. E. Hart, “Nearest neighbor pattern classification,” IEEE
transactions on information theory, 21-27, 1967.
[14] Suykens and J. Vandewalle, “Least squares support vector machine
classifiers,” Neural processing letters, vol. 9.3, 1999.
[15] M. J. Belongie and J. Puzicha, “Shape matching and object
recognition using shape contexts,” IEEE Transactions on Pattern
Analysis and Machine Intelligence, 2002.
[16] B. T. L. Berg and J. Malik, “Shape matching and object recognition
using low distortion correspondence,” Computer Vision and Pattern
Recognition, 2005.
[17] D. G. Lowe, “Object recognition from local scale-invariante
features,” Computer Vision, Corfu, Greece, 1999.
[18] M. Varma and A. Zisserman, “Texture classification: Are filter banks
necessary?” IEEE Conf on Computer Vision and Pattern Recognition,
Madison WI, June 18-20, vol. 2, 1999.
[19] Y. S. Fine and N. Tishby, “The hierarchical hidden Markov model:
Analysis and applications,” Machine learning 32.1, 41-62, 1998.
[20] Zeiler and R. Fergus, “Visualizing and understanding convolutional
networks,” European conference on computer vision. springer,
Cham,, 2014.
[21] Patil and S. Shimpi, “Handwritten English character recognition using
neural network,” Elixir Comput Sci Eng, vol. 41, pp. 5587–5591,
2011.
[22] Y. S. Fine and N. Tishby, “The hierarchical hidden Markov model:
Analysis and applications,” Machine learning 32.1, 41-62, 1998.
[23] R. J. Kannan, R. Prabhakar, and R. M. Suresh, Off-line cursive
handwritten Tamil character recognition. Technology. IEEE, 2008.
[24] B. B. Chaudhuri, A complete handwritten numeral database of
Bangla–a major Indic script., 2006.
[25] U. Pal and B. B. Chaudhuri, “"Indian script character recognition: a
survey." pattern,” Recognition, vol. 37, no. 9, pp. 1887–1899, 2004.
[26] S. K. Niranjan, V. Kumar, and H. Kumar, FLD based unconstrained
handwritten Kannada character recognition., 2008, vol. 3.
[27] S. Mika, “"Fisher discriminant analysis with kernels." Neural
networks for signal processing,” vol. 98. Ieee, 1999.
[28] R. S. Kunte and R. S. Samuel, “A simple and efficient optical
character recognition system for basic symbols in printed Kannada
text.” Sadhana, vol. 32, no. 5, p. 521, 2007