2018 Conference on Information and Communication Technology (CICT’18)

Handwritten Text Recognition in Bank Cheques

Rajib Ghosh Chinmaya Panda

Department of Computer Science and Engineering Department of Computer Science and Engineering
National Institute of Technology Patna National Institute of Technology Patna
Patna, India Patna, India
rajib.ghosh@nitp.ac.in chinmayapanda9732@gmail.com

Prabhat Kumar
Department of Computer Science and Engineering
National Institute of Technology Patna
Patna, India
prabhat@nitp.ac.in

Abstract—In spite of rapid evolution of digital technologies,
a huge number of applications still rely on the use of paper
based medium. This is especially true for processing of bank
cheques. The pre-printed account number and cheque number
might be easily readable and processed automatically. However,
the handwritten texts in cheque are usually processed manually
involving important time and cost. An attempt has been made in
this paper to develop a bank cheque handwritten text recognition
(BCHWTR) system for cheques of Indian banks by recognizing
the handwritten characters present in the ’payee name’, ’courtesy
amount (both in words and figures)’ and ’date’ fields by using
image processing techniques on handwritten cheque images.
Images of bank cheques are fed as input to the proposed
system. There are four stages in the proposed system: cropping
the image at a specific location; segmentation of handwritten
lines, words and characters; feature extraction from individual
characters and digits using Histogram of Oriented Gradients
(HOG) method and Grey Level Co-occurrence Matrix (GLCM)
texture features; recognition of isolated characters and digits
using the Support Vector Machine (SVM) based classification
process that ensures correct recognition. The performance of
present BCHWTR system is evaluated on a self-generated dataset
of bank cheques and it has provided a promising result.
Index Terms—Handwritten text recognition, Bank cheques,
Line segmentation, HOG feature, GLCM texture features, SVM
based classification.
I. I NTRODUCTION
Automatic bank cheque processing is a field of interest in
banking industry, as a large part of cheques is still processed
manually that involves the manual reading of the cheques and
keying their respective values into the computer. Bank cheques
are still widely used all over the world for financial transac-
tions where the cheques are usually processed manually in
almost all countries. In such manual verification, handwritten
text portions such as payee name, courtesy amount (both in
words and figures), date and signature of each cheque are to
be verified through observation by the bank employees. This
makes the process not only time consuming but also error-
prone and costly. As many countries use cheque truncation
systems (CTS) nowadays, much time, effort and money can
be saved if this entire process of recognition, verification and
data entry is done automatically using images of cheques.
The account number and the cheque number are printed
on the cheques in magnetic ink (MICR). Since the MICR
character set is a special type font, these fields can be au-
tomatically read using magnetic machines or optical character
recognition (OCR) systems. Recognition of handwritten char-
acters by computing approach is a serious problem because of
variations of handwritten style and shapes written by different
individuals. Moreover, lack of a specific pattern for characters
make it a daunting task. Differences in size, style of cheques,
low image contrast and complex background often makes the
process of automatic text extraction an extremely challenging
task. The basic goal of the present research work is to develop
a fast and accurate bank cheque handwritten text recognition
(BCHWTR) system able to read the handwritten text (payee
name, courtesy amount and date) in a cheque automatically
with minimal errors. This will largely automate the system
by reducing workload, time and cost per transaction. The
present system has been proposed for Indian bank cheques.
The character recognition process has been carried out using
Support Vector Machine (SVM) based classification technique.
The overall framework of the proposed BCHWTR system is
shown in Fig. 1.
The rest of the paper is organized as follows. Section II
details the related works. The process of data collection and
pre-processing steps have been discussed in Section III. The
proposed methods of line, word and character segmentation in
bank cheques are discussed in Section IV. Section V details
the proposed feature extraction approach. Performance of the
proposed system is detailed in Section VI. Section VII presents
the conclusion of the paper.

978-1-5386-8215-9/18/$31.00
2018
c IEEE
 2018 Conference on Information and Communication Technology (CICT’18)
Fig. 1. Overall framework of the proposed BCHWTR system
II. R ELATED WORK
The foremost technique in offline character recognition is to
extract different characters appearing in the image and there
by recognition of individual characters. Several research ex-
plorations are available in the literature on offline handwritten
text recognition, but works on handwritten text recognition in
bank cheques are very little. Some of those related works are
discussed below.
Roy et al. [3] dealt with the segmentation of a word image
containing native Indian language Bangla script into three
zones- upper, middle and lower. The components in middle
zone are modeled using Hidden Markov Model (HMM).
The authors have considered local gradient histogram (LGH)
features for their character dataset and have achieved an
promising result. A fuzzy logic based sliding window approach
is described in [4] to extract signatures from cheque and
other legal document images. In another study [5], Roy et
al. have proposed an HMM-based Indic script handwritten
word recognition using zone segmentation, considering LGH,
GABOR and pyramid histogram of oriented gradient (PHOG)
features but could not achieve a promising result. Recognition
of multi-oriented text in Devanagari and Bangla scripts has
also been reported [6], where convex hull technique and water
reservoir principle were applied. After character segmentation
from the document using the background information of the
text, individual characters were recognized using rotation
invariant features (e.g. Relative angular information of the
pixel with respect to its neighboring pixels) obtained from the
foreground portion of the characters. Roy et al. [7] proposed
a method to develop an automated system for detection of
seal (stamp) from documents with cluttered background by
considering angular radial transform features and SVM based
classification technique. A word recognition and verification
scheme based on HMMs is presented by Morita et al. [8].
The authors have considered frequent occurring words in
Brazilian bank cheques and have used concavity measurements
features viz., concavities, feature vector, auxiliary directions,
and 4-Freeman directions. Cursive unconstrained handwritten
word recognition based on input and class (lexicon) feature
vector was proposed by Guillevic et al. [9]. This work has
used K-Nearest Neighbour (KNN) classifier for classification
purpose and has shown a promising result. A method to
extract the items of a cheque by applying three sub processes
was proposed in [11]. First, elimination of the horizontal
lines and background pixels by image subtraction method was
performed, which is followed by the second step, in which
each connected object on the image is described by its minimal
rectangular bounding box, which is represented by a label.
In the last step, reconstruction of truncated written lines was
performed on basis of geometrical parameters. Suen et al. [12]
presented an approach by combining image processing and
pattern recognition techniques for handwritten text recogni-
tion in bank cheques. Initially, authors used different image
processing techniques to isolate and detect handwritten text
portions on a cheque. Various recognizers were then used
to recognize the handwritten legal amounts as well as the
date on the cheque. Freitas et al. [13] proposed a system to
recognize handwritten legal amount in Brazilian bank cheques.
Yu et al. [14] presented a system for recognizing handwritten
legal amount on bank cheques written in Chinese script. A
method for legal amount recognition written on bank cheques
is proposed in [15] based on word segmentation hypothesis.
Tang et al. proposed [16] a spiral recognition method to rec-
ognize handwritten text in Chinese bank cheques. Jayadevan
et al. [17] proposed a lexicon driven segmentation-recognition
scheme of words from legal amounts of Indian bank cheques.
The authors have considered a lexicon of 36 different words
occurring frequently in the courtesy amount in Indian bank
cheques. Water reservoir concept was used to pre-segment the
words into primitive components.
III. DATA COLLECTION AND PRE - PROCESSING
As the input to the proposed system is a scanned image
of a bank cheque, hence to generate the test dataset we had
to collect images of bank cheque with different handwriting
styles from various persons for introducing variations in the
collected data. To generate this dataset, 100 different people
provided their handwritten text in 100 separate bank cheques.
Fig. 2 shows the image of a sample bank cheque.
It is found that, usually, in Indian bank cheques, the hand-
written part to be filled manually, is located in the upper part
of the cheque, which if suitably and automatically cropped out,
will ease the process of detection. Hence we have cropped the
original cheque image at specified coordinates. The cropped
part (at a coordinate of [166.5 62.5 4135 1074] of the original
scanned image) contains the name of the payee, date, courtesy
amount both in words and digits. Whereas, the other part
contains primarily the information related to branch, IFSC
code and the logo of the bank, which are irrelevant portions in
 2018 Conference on Information and Communication Technology (CICT’18)
Fig. 2. Sample image of a bank cheque
light with the objective of the proposed system. Fig. 3 shows
the image of a cropped cheque.
Fig. 3. Cropped part of the bank cheque at a specific location
To accomplish the training phase, we have collected 100
handwritten samples of each character and digit in Latin script
as in the proposed system character based classes have been
considered during training. These samples were collected from
100 different persons of various educational backgrounds.
Each person provided one sample of each character and digit.
IV. P ROPOSED APPROACHES OF LINE , WORD AND
CHARACTER SEGMENTATION
As in the proposed system character based classes have
been considered during training the system using SVM, so,
to accomplish the recognition of handwritten text in bank
cheques, we need to segment each line of the cheque portion
where handwritten text is written, each word in a line and each
character in a word.
A. Line segmentation
Though in printed text line segmentation is quite easy
but in handwritten text it is quite difficult due to existence
of several issues like overlapping of characters, touching of
characters and variability in writing styles of different writers.
In the present system, a connected component based line
segmentation algorithm has been proposed on the basis of
threshold from input image.
The step-wise algorithm for line segmentation is given
below.
Step 1: Convert scanned cheque image to grayscale. This
process minimizes the intensity level information.
Step 2: Find vertical profile of the image.
Step 3: Convert grayscale image to binary image on the basis
of certain threshold.
Step 3: Find white spaces from vertical profile on basis of
the same threshold.
Step 4: Calculate centroids for connected components in the
white spaces.
Step 5: Concatenate structure array containing centroids into
a single matrix.
Step 6: Extract only Y- co-ordinates of the centroid and this
will constitute a line.
Step 7: Repeat the process until all lines are collected.
Fig. 4 shows the extracted lines after segmentation of
various lines from the handwritten portion of a cheque.
Fig. 4. Extracted lines after segmentation of various lines from the handwrit-
ten portion of a cheque- (a) Line containing date, (b) Line containing payee
name, (c) Line containing amount in words and (d) Line containing amount
in digits
B. Word segmentation
After line segmentation, each word in a line need to be
segmented so that each word can further be passed to the
individual character segmentation process.
The step-wise algorithm for word segmentation is given
below.
Step 1: Take a previously segmented line as input image.
Step 2: Convert it into a binary image.
Step 3: Remove small objects (noise) from the image.
Step 4: Calculate areas and the bounding boxes (the smallest
rectangle containing the region) for connected components in
the binary image.
Step 5: Concatenate structure array containing areas into a
single matrix and store the length of the matrix in a new
variable k.
Step 6: Crop the bounding box region and display the region.
Repeat the process k number of times. This will extract all
words in that line.
Fig. 5 shows the extraction of each word from the line
containing payee name.
C. Character segmentation
After extracting each word from a line, we need to segment
each character from a word which will allow the recognition
of each character using SVM.
The step-wise algorithm for word segmentation is given
below.
Step 1: Input the binary image of the word.
 2018 Conference on Information and Communication Technology (CICT’18)
Fig. 5. Extracted words from the line containing payee name
Step 2: Find vertical projection and from there find the
location of letters.
Step 3: Find rising and falling edges of starting and ending
columns from the difference in letter locations.
Step 4: Continuous connected pixels from starting edge to
ending column will form a letter.
Fig. 6 shows the extraction of each character of the payee
first name.
Fig. 6. Extracted characters of the payee first name
V. P ROPOSED APPROACH OF FEATURE EXTRACTION
Features are distinctive properties of input patterns that help
in differentiating between the classes of input patterns. As in
the proposed system the patterns are basic characters and digits
of Latin script, so we have extracted Histogram of Oriented
Gradients (HOG) [18] features along with Grey Level Co-
occurrence Matrix (GLCM) texture features [19] from each
basic character and digit. Fig. 7 demonstrates the overview of
the proposed feature extraction approach.
Fig. 7. Block diagram of the proposed feature extraction approach
Initially, luminance gradient is calculated at each pixel of the
binary image of segmented character and a gradient orientation
histogram is created for each cell. Here, the cell area is an area
that consists of 8 x 8 pixels. Next, features are normalized
for each descriptor block. Luminance gradient represents the
change in luminance in terms of magnitude m and orientation
θ. So, luminance magnitude m of (x, y) coordinates of any
image is defined using (1) (Here L denotes the luminance value
of pixel):
q
m(x, y) = (L(x + 1, y) − L(x − 1, y))2 + (L(x, y + 1) − L(x, y − 1))2 (1)
Using the magnitude and orientation of the gradient, the
’luminance gradient histogram’ for each cell area is calculated.
To normalize this histogram, the orientation bins are evenly
divided in 9 bins over 0°– 180°. We have tried with other
bin divisions also, but the one using 9 bins provided best
performance. So, the feature vector of one cell is having 9
dimension.
We have also extracted the GLCM texture features along
with the HOG features and combined the feature values
obtained from these two techniques into a single feature vector.
In GLCM texture features, following three parameters are
evaluated using (2 - 4 ):
N
X −1
Energy = Pij2 (2)
i,j=0
N −1
X (i − µ)(j − µ)
Correlation = Pij (3)
i,j=0
σ2
N −1
X Pij
Homogeneity = (4)
i,j=0
1 + |i − j|
Where i and j denote row and column of a pixel respectively,
N is the number of grey levels in the image as specified by
number of levels in quantization, σ 2 is the variance of the
intensities of all reference pixels and µ is the GLCM mean.
VI. E XPERIMENTAL RESULTS AND ANALYSIS
In the proposed system, SVM classifier has been used
for training and testing purpose to recognize the handwritten
text of bank cheques. Different kernels of SVM like linear,
polynomial and (Gaussian) radial basis function (RBF) have
been tested. As in the proposed system character based classes
have been considered during training the system using SVM,
so in testing phase, after segmenting each character from a
word, HOG and GLCM texture features (mentioned in section
V) are extracted from each character and digit. Next, feature
values obtained from HOG and GLCM texture features are
combined into a single feature vector. Finally, this feature
vector is fed to SVM classifier to know the label of the
character or digit.
A. Results of character segmentation
The accuracy of the line segmentation in handwritten text
highly depends upon the type of the text. If sufficient gap
exists between text lines and the document scanned properly
then the performance of line segmentation method will be
better. In the present system, results of the proposed line
segmentation algorithm from the cropped cheque image are
satisfactory. Similarly, accuracy of word segmentation from
a line depends upon the nature of the text obtained from
 2018 Conference on Information and Communication Technology (CICT’18)

a line. If the space between words is high, then words are present on the bank cheques as well as handwritten text
easily extracted. In the present system, except for some noise, recognition of cheques of various foreign banks. Application
the proposed word segmentation algorithm performed satis- of this proposed system for the real world scenario will be our
factorily. The accuracy has been increased by using the noise real challenge.
removal technique before word extraction. Finally, despite of
encountering some faulty segmentation, the proposed character R EFERENCES
segmentation algorithm has provided an accuracy of 91.83%.
[1] Arica, N., Yarman-Vural, F. T., ”An overview of character recognition
B. Text recognition performance using SVM focused on off-line handwriting”, IEEE Transactions on Systems, Man,
and Cybernetics, Volume 31, Issue 2, 2001, pp. 216-233.
Different kernels of SVM like linear, polynomial and (Gaus- [2] Marinai, S., Marino, E., Soda, G., ”Font adaptive word indexing of
sian) RBF have been tested to evaluate the performance of modern printed documents”, IEEE Transactions on Pattern Analysis and
Machine Intelligence, Volume 28, Issue 8, 2006, pp. 1187-1199.
text recognition of the proposed system. The text recognition [3] Roy, P. P., Dey, P., Roy, S., Pal, U., Kimura, F., ”A novel approach of
accuracies of the proposed system using various kernels of Bangla handwritten text recognition using HMM”, Proceedings of the
SVM are shown in Table I. The optimal set of values of various 14th International Conference on Frontiers in Handwriting Recognition,
2014, Island of Crete, Greece, pp. 661-666.
hyper parameters in SVM is shown in Table II. This optimal [4] Madasu, V. K., Mohd. Hafizuddin Mohd. Yusof, Hanmandlu, M.,
set has been found using Bayesian optimization technique. Kubik, K., ”Automatic Extraction of Signatures from Bank Cheques and
Other Documents”, Proceedings of the 7th International Conference on
Digital Image Computing: Techniques and Applications, 2003, Sydney,
TABLE I Australia, pp. 591-600.
T EXT RECOGNITION RESULTS OF THE PROPOSED SYSTEM USING VARIOUS [5] Roy, P. P., Bhunia, A. K., Das, A., Dey, P., Pal, U., ”HMM-based
KERNELS OF SVM Indic handwritten word recognition using zone segmentation”, Pattern
Recognition, Volume 60, 2016, pp. 1057-1075.
(Gaussian) RBF Kernel Linear Kernel Polynomial Kernel [6] Pal, U., Roy, P. P., Tripathy, N., Lladós, J., ”Multi-oriented Bangla and
85.42% 86.36% 89.73% Devanagari text recognition”, Pattern Recognition, Volume 43, Issue 12,
2010, pp. 4124-4136.
[7] Roy, P. P., Pal, U., Lladós, J. ”Document seal detection using GHT and
character proximity graphs”, Pattern Recognition, Volume 44, Issue 6,
TABLE II 2011, pp. 1282-1295.
O PTIMAL VALUES OF VARIOUS H YPER PARAMETERS OF SVM
[8] Morita, M., Sabourin, R., Bortolozzi, F., Suen, C. Y., ”A recognition and
verification strategy for handwritten word recognition”, Proceedings of
Soft-margin Width of (Gaussian) Degree of Polynomial 7th International Conference on Document Analysis and Recognition,
constant (C) RBF Kernel (γ) Kernel 2003, Edinburgh, U.K, pp. 482-486.
8 2.2 2 [9] Guillevic, D., Suen, C. Y., ”Cursive script recognition applied to the
processing of bank cheques”, Proceedings of the 3rd International
Conference on Document Analysis and Recognition, 1995, Quebec,
C. Comparative performance analysis Canada, pp. 11-14.
[10] Pal, U., Datta, S., ”Segmentation of Bangla unconstrained handwritten
The present study is not comparable directly with any other text”, Proceedings of the 7th International Conference on Document
existing studies because the efficiency of the proposed system Analysis and Recognition, 2003, Edinburgh, UK, pp. 35-40.
has not been evaluated on any publicly available dataset, it has [11] Djeziri, S., Nouboud, F., Plamondon, R., ”Extraction of items from
checks”, Proceedings of the 4th International Conference on Document
been evaluated on a self-generated dataset. But, we present Analysis and Recognition, 1997, Ulm, Germany, pp. 749-752.
here the comparative performance analysis in the recognition [12] Suen, C. Y., Lam, L., Guillevic, D., Strathy, N. W., Cheriet, M., Said, J.
performances of the present work with few existing studies N., Fan, R., ”Bank check processing system”, International Journal of
Imaging Systems and Technology, Volume 7, Issue 4, 1998, pp. 392-403.
to get the idea of comparative study. The comparative perfor- [13] Freitas, C. O. A., Yacoubi, A. E., Bortolozzi, F., Sabourin, R., ”Brazilian
mance analysis is shown in Table III. Bank Check Handwritten Legal Amount Recognition”, Proceedings
of the 13th Brazilian Symposium on Computer Graphics and Image
Processing, 2000, Gramado, Brazil, pp. 97-104.
TABLE III [14] Yu, M. L., Kwok, P. C. K., Leung, C. H., Tse, K. W., ”Segmentation and
C OMPARATIVE PERFORMANCE ANALYSIS WITH FEW EXISTING STUDIES recognition of Chinese bank check amounts”, International Journal on
Document Analysis and Recognition, Volume 3, 2001, Gramado, Brazil,
Reference Accuracy Rejection Error pp. 207–217.
Tang et al. [16] 60% 39% 1% [15] Kim, K. K., Kim, J. H., Chung, Y. K., Ching, Y. S., ”Legal Amount
Kim et al. [15] 72% 0% 28% Recognition Based on the Segmentation Hypotheses for Bank Check
Yu et al. [14] 74% 15.6% 10.4% Processing”, Proceedings of International Conference on Document
Proposed method 89.73% 2.13% 8.14% Analysis and Recognition, 2001, Seattle, USA, pp. 964-967.
[16] Tang, H., Emmanuel, A., Ching, Y. S., Baret, O., Cheriet, M., ”Spiral
Recognition Methodology and Its Application for Recognition of Chinese
Bank Checks”, Proceedings of 9th International Workshop on Frontiers
VII. C ONCLUSION in Handwriting Recognition, 2004, Tokyo, Japan, pp. 263-268.
Automatic handwritten text recognition in bank cheques is [17] Jayadevan, R., Pal, U., Kimura, F., ”Recognition of words from legal
amounts of Indian bank cheques”, Proceedings of the International
an interesting field of research from both scientific and com- Conference on Frontiers in Handwriting Recognition, 2010, Kolkata,
mercial points of view. In this paper, SVM based BCHWTR India, pp. 166-171.
system has been proposed for Indian bank cheques. The [18] Dalal, N., Triggs, B., ”Histograms of oriented gradients for human
detection”, Proceedings of IEEE Computer Society Conference on
proposed system has provided promising result. This research Computer Vision and Pattern Recognition, 2005, San Diego, USA, pp.
work can further be extended for verification of signatures 886-893.
 2018 Conference on Information and Communication Technology (CICT’18)

[19] Haralick, R. M., Shanmugam, K., ”Textural features for image classifi-
cation”, IEEE Transactions on systems, man, and cybernetics, Volume
6, 1973, pp. 610-621.
[20] Meyer, D., Leisch, F., Hornik, K., ”The support vector machine under
test”, Neurocomputing, Volume 55, Issue 1, 2003, pp. 169-186.