Recognition of
handwritten characters
a review
R H Davis and J Lyall
In the quest to study h~an perception, the field of
character recognition remains prominent. Research has
progressed from early, cumbersome optical character
readers used for the recognition ?j’ a limited set of
individual characters to the present-day automatic recog-
nition C$ handwritten cursive script. This paper attempts
to clarify the,fundamentals of character recognition, high-
lighting the processes involved in using a standard database
for ‘leurning’ character sets and also the .~tandards and
~on~~trai~tsimposed by researcller.~ concerning the consti-
tution of a valid ~hararter. A number of various feature
extraction techniques which enable individuul characters
to he recognized are discussed and compared.
Keywords: pattern recognition, handwritten characters,
feuture extraction
While much work in character recognition (CR) has been
concentrated on multi-font and fixed-font reading, the
most interesting field, scienti~caily, is that of recognition
of handwritten characters. Machines which read fixed
fonts, such as those on cheques and credit cards, are
very reliable due to the constraints of using magnetic
ink and the stylization of the character set. These readers
perform a one-dimensional scan combined with wave-
form correlation, and recognition is achieved by correlat-
ing the resulting signal with stored waveforms.
Operating on a less idealized set of characters, a multi-
font reader has to cope with noise factors such as paper
quality and reflectivity, ribbon wear and type element
alignment. The use of template matching is the most
widely used approach to multi-font reading and, because
ideally the number of acceptable fonts is finite, a proto-
type of each character can be stored for the purpose
of matching. In the recognition process the character
is optically scanned and converted into digital patterns
Department of Computer Science, Heriot-Watt University,
Grassmarket, Edinburgh EHI IHJ, UK
0262-8856/86/04208-l 1 $03.~
208
before comparison with several prototypes and eventual
recognition or rejection.
Much more difficult, and hence more interesting to
researchers, is the ability to recognize handwritten
characters automatically. The complexity is greatly
increased by the noise problem, and the almost infinite
variability of handwriting. Noise arises from the writing
instrument used, the quality of the paper or the pressure
applied when writing, for example. Variability, on the
other hand, is virtually certain as even one person’s
handwriting will change depending upon su~oundings,
the mood of the person and the nature of the writing.
The recognition of handwritten characters can be sub-
divided into that of cursive script (joined writing) and
printed script. Even humans, with the best optical
readers available, have difficulty in recognizing some
cursive script, and in this area of CR the greatest
problem is that of segmentation of letters within the
word and detection of features.
The operation of character recognition is a process
which can be carried out online or offline. In the former
case, characters are recognized as they are written. This
approach necessitates the use of a graphics tablet, or
a light pen, for the drawing of characters. Information,
usually in the form of coordinates, is collected at regular
time intervals as the tablet is moved. The tasks of prepro-
cessing, feature extraction and classification follow,
enabling either successful recognition, misrecognition or
rejection of the character.
In an offline system, characters are recognized on
completion of drawing. An optical scanner is required
to obtain a digitized representation of the input
character. The stages involved in optical character recog-
nition (OCR) are shown in Figure 1.
In all recognition methods the fundamental task is
that of extracting useful information from the image
and then performing classification, usually by com-
parison with known results. Although the character may
79 be represented in matrix form, or by line segments, the
basic approach is the same - that of processing the
@ 1986 ~utterworth & Co. (Publishers) Ltd
image and vision computing
 To
preprocessmg
Input Digitization Character
n-&lx
Smoothing, noise Identifi~atian
elimination and character
size narmalizat~on
Figure 1. Stages in recognition of handprinted characters
character(s) and extracting features to compile what is
known as a feature vector.
The feature vector is the means by which the character
is represented in the system, and is the basis of the
recognition stage, Before any classification takes place
it is usual to build up some prior knowledge of the
character set to be tested, by supplying a learning set
of characters, and establishing feature vectors which can
be compared with those of the characters of the testing
set.
The stages in recognition by an OCR are
Input characters are read and digitized by an optical
scanner.
Each character is located and segmented from the
data sheet under software control.
The resulting matrix is fed into a preprocessor for
2 smoothing i.e. filling holes and breaks in line
segments,
fi noise elimination (isolated dots),
0 size normalization.
Distinctive features are then extracted, which are seen
to be ‘descriptive’ of the character.
Identification is performed on the basis of comparison
between extracted features and statistics of features
from a learning set of examples.
Early character recognition was hindered by three
aspects’,
The adequate representation of a handprinted
character requires at least 500 bits of memory (to
enable 20 x 25 matrix size).
The cost in terms of computing power and time was
high.
The lack of scanning equipment and a combination
of the above forced many researchers to work on
limited sets of characters; as a result their results were
very inconclusive.
With the limited set of characters proving to be a great
drawback some researchers established ‘standard’ data-
bases, containing many variations of different
characters. which could be used as both learning and
testing sets. The two most commonly used are those
created by Munson’ and HighIeyman~.
Munson’s data was represented in 24 x 24 matrix
form, and consisted of 46 characters of the FORTRAN
IIlanguage. He obtained 12 760 samples, 6 762 of these
from 49 authors, with each printing three sets of
FORTRAN characters on standard coding sheets. The
remaining characters were collected from individual
authors, or from fragments of coding sheets. Munson
instructed authors to slash the characters ‘0’ and ‘Z’,
~014 no 4 november 1986
print ‘1’ without serifs and place horizontal bars on
the letter ‘I’.
Highleyman’s data consisted of the 36 alphanumeric
characters, represented by a 12 x 12 matrix. He
obtained 1 800 letters and 500 numerals from 50 authors
who were instructed to print neatly on l/4 in quadruled
paper, approximately tilling these boxes.
of
Since Highleymans’ data is of low resolution, poor
recognition rates were achieved using his database.
Although Munsons’ data varies widely in quality, it is
still considered to be the most difficult set of almost
unconstrained handprinted characters for machine
recognition.
While researchers have been developing and designing
recognition techniques, OCR committees in some
countries have developed handprint standards in order
to reduce the number of shape variations, resulting in
less confusion over identification of characters and
higher recognition rates.
A large proportion of optical readers in OCR indus-
tries can only recognise handprinted characters within
the scope of numerals and a few letters, entered in boxes
with or without guidelines. The writers are usually
instructed to keep to the standards and models provided
as closely as possible, avoiding gaps and flourishes in
the characters.
Another common constraint is that characters should
be printed as large as possible, within specified bound-
aries, to retain good resolution, and it is usual to have
characters written on forms with OCR specifications
in weight, dirt count, colour and reflectance.
As well as specifying exact areas in which to enter
characters, many researchers place rules on what consti-
tutes a valid character: commonly these are ‘0’ and ‘Z’
with strokes, ‘I’ with bars and ‘4’ with vertical and
horizontal lines only.
Comparison of recognition techniques
Feature extraction is the central issue in handprint recog-
nition, and is followed by classification. Recognition,
or classification, is fairly straightforward and totally
dependent on the merit of the extraction process. Table
1 shows a comparison of various techniques used in
feature extraction based on:
sensitivity to the deformation of the image of a
character, caused by
CD]) noise - disconnected line segments, bumps,
gaps in lines, filled loops etc.;
032) distortion - local variations, rounding of
corners, improper protrusions, dilation and
shrinkage;
(D3) style variation - use of different shapes to
represent the same character, serifs, slants etc.;
(D4) transl~ti~~n- movement of the whole character
or its components;
(D5) roration - change in orientation;
practical implementation of the recognition technique,
iudged by
iIl)- aut&atic musk making - whether or not it is
easy to build masks automatically corresponding
to each class of characters to match the different
features used in such a technique:
w ,speed 0f rt~~.~)g~~iti~)n~
209
Table 1. Comparison of recognition techniques

Tolerance to image Ease of

deformation implementation

Dl D2 D3 D4 D5 11 I2 I3 I4

Global Template matching + 0 - - - + - 0 -

features Transformation - + + + + 0 - - 0

Zoning _ 0 - - 0 - + + -

Moments 0 0 - + + 0 - 0 -

Distribution of points n-tuple 0 - 0 - 0 - + + 0

Characteristic loci - + + + or0 + + -

Crossings and distances _ + + + 0 - + + -

Geometrical and topological _ + + + 0 - + 0 +

+ high or easy 0 medium - low or difficult

(13) comple.uity qf implementation; and the densities of points in these regions are used
(14) independence ~ whether supplementary as feature vectors.
techniques are needed. Moments. The moments of black points about a
chosen centre, such as the centre of gravity, are used
The above criteria are applied to compare recognition as features.
techniques which can be classified into three categories, n-tuples. The occurrence of black or white elements,
according to the way in which features are extracted or joint occurrences of these elements, are used as features.
Characteristic loci. For each white (i.e. empty) point
l global features in the background of the character, vertical and horizon-
l distribution of points tal vectors are generated, and the number of times the
0 topological features line segments are intersected by these vectors are used
as features.
Crossings and distances. For crossings, features are
Global features measured by the number of times line segments are
crossed by vectors in specified directions. In the case
This technique extracts features from every point inside of distances, the distances of elements or line segments
a frame surrounding the character, and features show from a given boundary, such as the frame, are used
no reflection of any local, geometrical or topological as features.
properties.
Template matching and correlations. Features are
Geometrical and topological features
determined by the state (white or black) of all points
within the frame. A measure of the similarity between
This approach is based on extracting the geometrical
the input character and store references is obtained by
and topological features which describe the physical
the matching and correlation of points or groups of
makeup of the character. They may represent both global
points in the frame.
and local properties. This is by far the most popular
Transformations and series expansions. Using the
method used by researchers and common features to
points in the frame as features, as in template matching,
extract include:
results in a very large feature vector. (The dimension
will be high because all points in the frame will be held
l strokes and curves in various directions e.g.
in the vector.) It is possible to reduce the dimensionality
and extract features invariant to some global defor-
mation, such as a translation or rotation. Rotational
I I \
transformations include those of Fourier, Walsh, Haar l end points, intersections of line segments and loops,
and Hadamard whilst Karhunen-Loeve use series l stroke relations, angular properties, sharp protrusions
expansions. and intrusions in connection with contour analysis.

An approach to geometrical and topological features,

Distribution of points and zoning, is explained later.

The dimensionality of the feature vector can also be

reduced by considering as features the statistical distri- USING TOPOLOGICAL FEATURES FOR
bution of points, rather than every individual point of RECOGNITION
the character.
Zoning. This involves splitting the character frame Printed script recognition has been researched by many
into several overlapping, or non-overlapping, regions people. Particularly useful is the manipulation of the

210 image and vision computing

 Stage I - Feature -
The pattern is centred in an octagonal grid (see Figure
3a), and features are extracted according to a set of
rules. By using a symmetrical grid the detection software
can operate independently of the character size and
avoid the necessity of size normalization.
The features are represented by:
I

Stage a&no part of a line in an octant;

feature
al-a stroke crossing octants 1,2,5, or 6 is approximated
extractor
by a horizontal stroke;
a2-a stroke crossing octants 3,4,7 or 8 is approximated
Relectlon DeClSlOrl Decision by a vertical stroke;
Figure 2. Recognition system using two-level decision a3-a stroke terminating in an octant in the clockwise
making direction;
a&a stroke terminating in an octant in the anti-
geometrical and topological features of the character4. closewise direction;
This approach to recognition is concerned with the a5-a loop in an octant, with opening to the right, con-
physical makeup of a letter or digit, and the features sidered only in octants I ,6,7.8;
of interest may involve a&a loop in an octant, with opening to the left, con-
sidered only in octants 2,3,4,5;
0 strokes and bays (curves); a7-a loop in an octant, facing upwards, considered
0 end points of characters, intersections of line segments only in octants 4,5,6,7;
and loops; a8-a loop in an octant, facing downward, considered
l contour analysis: stroke relations, angular properties, only in octants 1,2,3,8;
sharp protrusions and intrusions. a9-a stroke contained entirely in an octant.

The popularity of such an approach can be attributed In the feature list, a loop is signified by a continuous
to its high tolerance to distortion and style variations stroke, starting in one octant, proceeding to the adjacent
when compared with other recognition techniques, as octant, and returning to the original octant. In Figure
well as its ability to tolerate a certain amount of rotation 3a the octagonal grid is shown, with the octant numbers
and translation. used. Figures 3b to 3h show examples of characters to
It is possible to involve decision making at various illustrate the selected features.
stages during the recognition process, and thus avoid There are ten possible symbols, aO,al a9, which
the waste of time and resources if a character can be are assigned to each octant depending on the nature
rejected at an early stage in processing4. of the line segment within that octant. Each octant may
Such a recognition model would be split into two have up to four features, indicating four different stroke
phases, as shown in Figure 2. It starts with global detec- segments, and for any octant having less then four fea-
tion of features, which is then followed by local analysis tures (i.e. parts of strokes) the entries are zero rather
when the first stage provides a non-unique solution, i.e. than one of the symbols a0.. a9.
the character cannot be uniquely identified. The feature vector for a character consists of these
Within these stages four symbols for each octant, or zero, and a boolean
value to indicate whether or not two strokes cross in
The conversion takes place of the physical image to an octant; m = 2 if two strokes do cross, and m = I
a digital pattern through the use of an input device if none cross. The vector can be represented thus
such as a high speed optical scanner.
The preprocessor centres and thins the pattern, and X = (.ul 1,x12,.~13,.~14,m;.u21,.~22,.u23,.~24,m; .;
locates the start and end of each stroke. .uil ,.ui2,xi3,.ui4,m; x8 1..u82..~83..~84,m)
The first stage feature extractor determines the basic
features which are collected as a feature vector. where
The first stage classifier operates on feature vectors
and assigns characters to a primary category or rejects i = 1,2,3...8
.x,, > (a,!,
the character, involving no detailed analysis. j = 1,2,3,4
The first level decision maker uses previous experi- .k = 0,1.2.. . 9
ence, from the learning set, to see if the character
can be uniquely identified without involving more
or
detailed local analysis.
If a non-unique classification has been reached, the
X,, = 0
second stage classifier is used to detect local features
to remove any of the ambiguities realised at the first
stage. Although these features a0.. a9 help categorize a
character with a minimum of feature extraction. it is
not possible to identify uniquely all characters as some
First stage analysis will have identical vectors. In such cases the second stage
classifier will have to remove any ambiguities.
The first stage of the recognition process involves group- E.ytraction of ,fiatures. Once the start, intermediate
ing the input character into one of several subgroups. and end points of each stroke have been found (by the

vol4 no 4 novemher 1986 211

 a b

9 h

Figure 3. The octugonaf grid (a) and examples offeature types (h)-(h)

preprocessor), yielding individual strokes, feature o determining when a stroke crosses a boundary,
extraction is performed. Each stroke is tracked and l calculating features a0. . , a9 as each octant is crossed;
features obtained by: l ordering of features into feature vectors.

212 image and vision computing

Reduction I$ the ,j&ure set. This is used to determine ledge. They can be described as general purpose for
which features can be classified as identical, while mak- the detection of sharp inflections or special purpose for
ing sure the loss of information does not lose the class resolving specific ambiguities between characters.
uniqueness for any pattern in the learning set. Before
reduction, a regrouping process is necessary as several
General purpose tests
classes contain identical feature vectors. At this stage
it is impossible to remove any of the eight measurements
One approach for detecting sharp inflections is by way
from the vector, but the feature set can be simplified
of a curve follower, which establishes three points on
by:
a curve and follows the curve, keeping the distances
between points similar. The smallest angle between two
l ignoring the orientation of strokes;
line segments connecting three points is stored, and the
l not assigning different codes to the same stroke
maximum inflection along a line can be determined
occurring in different octants;
(enabling discrimination between ‘U’ and ‘V’ among
l ignoring the relative position of loops. In fact, the
others).
only significance is whether loops are facing left, right,
A discriminant-function detector is useful for dis-
up, or down, hence the basic topological features
tinguishing a straight stroke from an inflection. This
shown.
involves locating an elliptical boundary that has its
major (horizontal) axis containing the two end points
Recognition ut thr ,first stuge. The main advantage of
of a stroke. For a straight line most of the stroke will
using the topological features as shown is the reduction
lie within the ellipse, while an inflection will exist mainly
of variability of characters among the same class, thus
outwith the boundary.
allowing the use of a lookup table for recognition at
the first stage. A lookup table can be easily updated,
and allows for rejection. Updating the table merely Special purpose’tests
entails adding entries. while rejection is necessary in
many application of CR. such as in mail sorting. These are concerned with resolving specific ambiguities
between characters. As the first stage in recognition has
greatly narrowed the range of possible outcomes, the
Classification extraction is much easier than had recognition been per-
formed in one step. The specific conflicts encountered
First, classify the feature vector by searching for an entry are listed below, where * denotes the use of a curve-
in the table. If no entry is found the character is rejected, follower inflection and ** denotes use of discriminant-
i.e. it is not recognised by the absence of a corresponding function detection.
vector from the learning set.
If an entry is found, then determine if the feature DO/UVW** - examine degree of opening in upper
vector represents a unique pattern using the first level region to yield (D,O) or (U,V,W);
decision maker. which has an auxiliary storage for the D/O* ~ check lower section for sharp inflection (0);
previous conflicts between patterns of different classes. U/V/W* -W has multiple inflections in lower half;U
The auxiliary storage is identical in structure to the 1’sV: single inflection;
lookup table, and its size is determined by how well X/Y/4* - 4: stroke starting in upper left has sharp
features at the first stage are able to distinguish between inflection; X vs. Y: consideration of relationship
classes. If the feature vector is not present in the storage between two strokes;
then the character is recognized, but if it has an entry C/L** - check straightness of upper 2;3 of pattern;
then the character cannot be uniquely recognized. This D/P - check height of lower horizontal stroke in
necessitates second stage classification which performs comparison with height of character:
more local analysis. depending on which subgroup the F/I - check height of lower horizontal for being
character belongs to (one of a possible i characters). more than 15% up the height of the vertical;
J,;T,!Y** - Y has sharp inflection in top stroke; T
r.s J: check for tail in bottom of character;
Second stage analysis J:‘5 ~ check relationship between top stroke and the
rest of the character;
The second stage is used when the pattern has been 5&9.G,6* - 5 has sharp inflection in upper left;
classified into a subgroup but:annot be uniquely identi- S.9/G,6 - examines region below and left of lower
fied. In this event specialized features on a local level endpoint: G and 6 have a section of character present,
are analysed, according to which subgroup the character S and 9 do not:
has been restricted. G!‘6* - examine lower right for inflection;
At the first stage, for example, differentiation between S:‘9* - check for lower section being curved or
a ‘U’ and a ‘V’ would not be possible because of their straight;
identical feature vectors, this fact being held in the l Z/2* ~- examine upper section of character for sharp
auxiliary storage. On entering the second stage, the inflection.
feature extractor would look for a means of classifying
the character. In the case of ‘U’ and ‘V’, the presence
or absence of a sharp inflection at the bottom of the DISTRIBUTION OF POINTS AS FEATURES
character would enable discrimination.
The procedures required for the extraction of features Another of the features of a character to be used for
at this stage are governed by experimentation and know- recognition is that of the distribution of points, i.e. tilled

rol4 no 4 novernher 1986 213

points within a matrix, to assess the probability of an
input pattern being a specific character.
One such technique is zonings. This method works
by splitting the matrix up into several non-overlapping
regions, and building a feature vector which contains
as elements the count of black points in each of these
regions. Probabilities are required for the value of region
k (i.e. the count of black points) on condition of class
Cj, which could, for example, be one of the 26 upper
case letters of the English alphabet. Also required is
the probability of an input pattern being Cj. This may
be such that there is an equal probability of each
character occurring, so P(Cj = l/26) for all Cj, or it
may be related to the known probabilistic occurrences
of each letter.
On obtaining these values, a formula is applied for
each Cj and the Cj yielding the highest result is deemed
the recognition answer. In any recognition process the
task is to extract features from the input pattern, and
use stored data from the learning (or training) set in
conjunction with these features to enable classification.
As there is no optimum way of extracting features it
is usual to select some well known ones rather than
make a simple random selection. Obviously, the result
of having too large a feature set is to involve a vast
amount of storage and data processing, as well as
necessitating many training samples to build the initial
learning set. It is for this reason that such importance
is placed on preprocessing of the pattern in character
recognition.
Preprocessing the pattern before feature extraction
allows the complexity of the feature extraction process
to be reduced, and is vitally important in such an
approach as zoning.
A specific form of preprocessing, size normalization,
is achieved by Hussain et ~l.~, as performed using the
3 822 upper case alphabetic letters from Munson’s data-
base (49 different persons each supplied three sets of
the alphabet). The need for size normalization when
a zoning technique is involved is apparent because of
the use of point densities within a region for features,
so a scaling up or scaling down is required to obtain
a normalized pattern involving most of the matrix area.
The use of zoning for character recognition is a statis-
tically based approach once the feature vectors have
been compiled.
For each of the 26 upper case letters, the character
set in this case
R = f ln~P(x~/C~)] + ln[P(Cj)] (1)
i=l
is calculated where:
Cj is letter ‘j’
P(Cj) is the a priori probability of the character
being Cj
P(xi/CJ:, is the conditional probability of the value
of xi given Cj
The Cj yielding the maximum value of R is that which
is resultantly recognized. The probability of a recog-
nition error is reduced if the feature vectors’ components
are statistically independent and when probabilities in
(I) are obtained using Bayes Estimation.
To obtain some quantified value of the merit of a
recognition system, Touissant and Donaldson suggest
214
splitting the alphabets into M disjoint sets’. Bayes’ esti-
mates are used for probabilities for M - 1 of these
sets (used as training sets for the system) and formula
(1) for the other set (testing set). The procedure is
repeated, with each set as the test set, and the average
probability of error for each set is calculated.
ONLINE RECOGNITION OF CHARACTERS
In online recognition the processing occurs as the
characters are drawn rather than on completion’.
Commonly, a graphics tablet is used which either
samples points at a constant rate or in spatial increments.
The creation of noise is a problem in such systems and
is usually due to the distance between the tip of the
light pen and the point of whose coordinates are taken.
Characteristics of online recognition
As patterns are represented by line drawings, the need
for costly processes such as skeletonization or contour
extraction is removed, and it is more usual to gather
information on the input stroke sequence in preference
to the actual shape of the character.
The time to recognize may be longer than in an optical
character reader, as it is unnecessary to recognize
characters faster than they can be drawn.
There are less constraints on characters, allowing size
and orientation variation.
Problems of online recognition compared with an
OCR
If the online recognizer is being used for computer aided
design, the set of symbols encountered may be very large,
usually more than 85 as opposed to, say, the 36 alpha-
numeric characters.
It may be necessary to enlarge the vocabulary as the
system expands, thus a learning capability is required.
Sloppy writing or ineffectual writing devices will
reduce the recognition rate.
Noise is a problem, particularly if the character is
small, due to the precision of the graphics tablet. The
limit for size is usually 0.1 to 0.5 mm.
Preprocessing
As with all recognition systems, a preprocessor is used
to extract the relevant information that will be used
for classification. In an online system it is generally the
points that make up the character that are selected.
Methods of obtaining these points include
smoothing i.e. averaging a point with its neighbours
to reduce the effects of noise;
filtering by forcing a minimum distance between
points; this eliminates perturbations while the pen
is not moving and reduces data redundancy;
angular segmentation by sampling a point as soon
as the tangent to the drawing changes by a certain
angle;
normalization of the size and position of the
character.
image and vision computing
Extraction of features and ciassifica tion
Most online systems use as features the sequence of
subareas traversed by the pen or the sequence of
directions of the drawing. Typically, feature vectors are
created by means of angles, structure (i.e. clockwise or
anti-clockwise curves etc.), zoning or Fourier descrip-
tors. Classi~cation is achieved by a table lookup scheme
i.e. check if that vector has been previously encountered
and thus known. In an attempt to reduce the amount
of processing of data, a hierarchical scheme can be
adopted. operating on only part of the feature vector.
This would be particularly advantageous when the
vector has many dimensions.
Capabilities and performances
Separation of characters
In an OCR system. separation is implied by a space
between characters or by considering a pattern within
a restricted area, but in the online case it is more dif~cult
to distinguish the completion of one character from the
initiation of another. It is possible to find this separation
by having the user signal he has finished one character
by interaction, operating a time-out of about 0.5 s
between characters or using forced physical separation.
~earnj~g
While nearly all offline recognition systems (i.e. process-
ing performed on completion not during the drawing
of the character) operate on trainable algorithms which
involve the use of a learning set, online systems lack
the standard databases. e.g. those of Munson and
Highleyman. because taking data one item at a time
would prove to be ineffective in terms of building the
lookup table, or dictionary.
Although this appears to be a major drawback, learn-
ing is feasible through user interaction and, feedback.
In the case of correct recognition there would be no
feedback and different features or parameters could be
weighted accordingly to indicate success. On the other
hand. if the character is rejected then interaction would
be used to indicate to the system the character the user
had drawn, and an entry in the table with the name
and description (i.e. feature vector) would be established.
A substitution, i.e. misrecognition of the intended
character, would result in a new test or the addition
of a feature to remove the ambiguity as seen by the
recognition system.
Size of recognizable characters
Due to the cost of increasing the precision of the graphics
tablet, it is usually difficult to recognize small characters
(3-5 mm), and as a result most researchers systems are
restricted to characters SlOmm high which is unsatis-
factory for typical applications such as billing machines.
~014 no 4 novcmtwr 1986
Recognizable character set
Online systems have been designed to recognize a vary-
ing set of alphabets from numerals and alphanumerics
to Japanese Katakana characters, as well as for specific
purposes such as input of computer programs of com-
puter aided design requiring FORTRAN symbols and
special symbols for CAD applications.
Recognition rates
In terms of recognition rates it is not normal for an
online system to exceed 95% accuracy, but this figure
may be seen as more than acceptable because:
They are sensitive to experimental conditions as there
are no standard databases.
Recognition varies considerably as the user acclima-
tizes to the system and training of the user is an
important part of online recognition.
The possibility exists to correct errors in drawing
almost as soon as they occur.
Time requirements and memory
All online systems operate recognition in a time neg-
ligible in comparison with the time taken to draw a
character, and a figure of under 50 ms is usually possible.
With the task of recognition being so involved in data
processing, the time can be optimized and memory
requirements reduced by adopting techniques such as
hash coding or binary search of a dictionary containing
condensed codes.
Most programs run on large computers would require
drastic alterations if they were to be run on mini or
microcomputers, although in the case of CAD this is
not possible as these systems tend to operate on large
and powerful computers.
Use of Fourier coefficients
In an online recognition system a useful device for the
construction of feature vectors is that of Fourier
expansion coefficients. The stage of feature description
is concerned with analysing the preprocessed informa-
tion, i.e. smoothed, normalized and sampled, and pin-
pointing the features of importance in a character that
will be used for classification.
Classification is performed at two levels, involving
a preliminary grouping according to the number of
strokes in the character. In the experimentation by
Arakawa’, using the 26 lower case letters and digits,
the characters i. j. t. x, 4, 5 and 7 are found to comprise
two strokes, with the remainder being drawn in one
complete stroke. These two subgroups are further
categorized according to the shape of the strokes as
represented by Fourier coefficients.
The character feature vector is then compared with
reference pattern vectors, which consist of average values
and a covariance matrix of Fourier coefficients for learn-
ings samples, with the classification being based on
Bayes’ decision rule.
215
CURSIVE SCRIPT RECOGNITION
In the recognition methods described so far, the task
has been to identify an individual character, but it would
undoubtedly be of more interest to humans if it were
possible to recognize entire words that consisted of
cursively joined characters.
Work in the area of cursive script recognitions (CSR)
has evolved over some 25 years, and began with
Frishkopf and Harmon’s work at Bell Telephone
Laboratories. Frishkopf aimed to recognize the word
as a whole, i.e. as a single complex symbol, while
Harmon decided to segment the word and perform
recognition on the individual characters.
Much work was carried out in CSR until about 1965,
when many of the researchers turned to speech recog-
nition instead, and it was a further 15 years before the
research intensified again. Although CSR is a major
area of character recognition, very little of the work
turns up in literature, as a great deal of the work is
being promoted by proprietary concerns. It is the capa-
bility to alleviate problems associated with man-machine
communication more efficiently, with the development
of relatively cheap processing power, that has made the
area all the more attractive.
As even humans find it difficult to recognize some
cursive script. it is clear that in automatic recognition
the task or preprocessing is even more important than
in single character script. With such a variety of styles
possible it is necessary to eliminate much of this varia-
bility which proves to be problematical in recognition.
Brown and Ganapathyy implemented a CSR using
an Amdahl 470 system, linked to a Tektronix graphics
tablet and a Tektronix 4010 graphics terminal. As the
word is written the computer is supplied with X and
Y coordinates from the tablet at the rate of 96 coordi-
nates/s. and with a realistic drawing rate of characters/s,
this yields about 24 sets of coordinate points per
character.
The script data is stored in the Amdahl 470 in the
form of X and Y integer vectors, with a typical four
to six letter word being represented by 200&300 coordi-
nates -this information is known as the pattern vector.
In an unnormalized form this vector represents the com-
plete input script, including the variability of the script
caused by a particular writer, and it is this variability
that is required to be removed by preprocessing. This
will result in a normalized pattern vector which is author
independent and is followed by feature extraction and
classification.
Shridhar and Badreldin’“,” use a structural classifi-
cation scheme in which the recognition algorithm is
derived as a tree classifier.
The first phase describes topological features to
uniquely characterize isolated numerals. Features are
derived from the left to right profiles of threshold images
of the numerals. Using a structural scheme features are
defined and combined to yield a logical description. The
second phase deals with the segmentation of connecting
strings using a hierarchical approach.
Huang and Chuang” similarly pursue a means to
recognize handwritten numerals. Here a heuristic
approach is adopted whereby structural differences
between numerals are extracted and a learning table
216
built to classify numerals of large variability with high
accuracy.
As with all recognition techniques, the main features
of the pattern are extracted before an attempt is made
to classify. Brown and Ganapathy’s model consists of
individual routines for each of 183 features, 15 of these
being globally extracted from all parts of the word, and
the others are extracted from isolated parts of the word.
Before these methods of extraction can work, the
height of the script, script location and orientation must
be known. It is these tasks which preprocessing performs
through normalization.
Preprocessing techniques
Typically a preprocessor for cursive script is concerned
with coordinate translation, rotation and scaling, curve
smoothing (and noise elimination) and character
deskewing. An open loop preprocessor would handle
these stages one by one and involve a single pass of
the pattern, so there is no verification that the data
has been satisfactorily modified by the preprocessor. In
contrast to this, a closed loop preprocessor can reduce
error by applying verification and feedback.
Open loop preprocessing
Brown and Ganapathy’s implementation uses ten bits
per coordinate, with an average space of 100 units
between successive characters within the script.
The pattern vector which represents the original input
pattern is stored as a 3 x N array, with N being the
number of sample points taken and the three dimensions
representing X-coordinate, Y-coordinate and visibility.
This information enables the script to be redrawn in
a sequential process from the first to last coordinates,
with the visibility factor distinguishing between a move
and a draw. This indicates, for example, a move of the
tablet to place a dot on an ‘i’ rather than a continuously
drawn stroke.
Preprocessing is achieved by performing two-
dimensional linear transformations using homogeneous
coordinate transformation matrices. A fourth dimension
is augmented to the 3 x N matrix, containing as
elements the value 1, to enable post-multiplication of
3 x 3 matrices as the visibility information is not used
in calculations.
By using vector representation, several transfor-
mations may be accumulated into one 3 x 3 transfor-
mation matrix. A measure of the current orientation,
size and shape is obtained by the extraction of various
features from the pattern vector, and is a necessary
requirement before performing the linear processes such
as translation and rotation.
Before it is possible to scale the script, the height
of the central body is found, ignoring upper and lower
loops, dots and crossbars on ‘t’s. The relative height
of the script can be determined by looking at the prob-
ability density of Y coordinates, as in most cases the
majority of Y coordinates will lie in the central body
of the script. Ideally, the density function will have low,
flat tails representing coordinates of upper and lower
loops, with a plateau in the centre representing the
central body.
image and vision computing
 To determine this density function, horizontal
thresholds are placed over the script at pr~dete~ined
levels, and a count of the number of times the script
crosses each is kept in a histogram.
Deskewing is the last of the preprocessing tasks to
be performed, and is concerned with removing slant 4.----- Verification --9
variation that is typical of cursive script. The slant of
the script can be measured by placing two horizontal
thresholds through the centre of the script and finding
reciprocal gradients at the points at which the script
crosses over these lines. Determining the slant of the
complete script is then merely a task of averaging all Pattern Transformation
the slopes taken. transformotion -

In most cases (more than 80%) Brown and

Ganapathy found that open loop preprocessing is able
to properly translate, rotate and scale the script, but
problems do arise when critical baseline points that are
selected do not all lie on the true baseline of the script,
particularly if the number of critical points is low. While
most classifiers that have been proposed” assume
handwritten characters to be continuous, isolated and Figure 4. Closed loop preprocessing
completely described by their boundaries, Shridhar and
Badreldin attend not only to the more commonly Table 2. Computer selected words for recognition
encountered broken form of handwritten numeral but
also to connected adjacent numerals. Their recognition Recognition error
algorithm uses a tree classifier to determine individual
numerals with a reported accuracy of 99% and Open loop Closed loop
connected numerals with an accuracy of 93O/.
rxxi 10.0 0
xxxi 9.1 9.1
Closed loop preprocessing rxxi 33.3 0
exxi 20.0 10.0
In terms of operations performed, closed loop and open xrxi 30.0 40.0
loop preprocessing are very similar, but the main advan- xxxr 0 0
tage of the former is that the error rate can be reduced rxxe 22.2 30.0
by applying verification and feedback before repeating rxxx 0 0
the process, while an open loop system operates on only cxxi 20.0 20.0
one pass. ixxi 20.0 0
Verification of the location and orientation of the rxri 22.2 0
word are usually easy so the measurement algorithms exxr 40.0 50.0
can be simplified to give more consistent results. This xxri 40.0 10.0
veri~cation gives an approximate measurement of the rixx 20.0 20.0
positional or rotational error, and although this value xrxx 50.0 30.0
is not exact the measurement error will diminish as the rexx 60.0 10.0
word becomes properly normalized. By placing the rxxs 0 0
verification stage before the first application of a trans- rxre 0 0
formation, to obtain a first transformation estimate, it exxe 20.0 20.0
is possible to reduce the amount of processing if the exri 30.0 10.0
word is already partially normalized in terms of xxxe 20.0 0
orientation and size,
The diagram of the closed loop preprocessor (Figure Average error 22.1% 12.4%
4) shows the veri~cation and transformation processes
in expanded form. Change in recognition error - 43.9%
A set of thresholds is placed over the script, as was
done in open loop scaling, and the histogram computed
for the non-orientated pattern. In this closed loop
method a trial rotation is performed and the new histo- before the first transfo~ation, thus avoiding these
gram is obtained. processes (scaling and deskewing) if they are not necess-
These processes are repeated until there is no further ary.
improvement, in which case the angular change is halved The usefulness of a closed loop preprocessor can be
before repeating the whole process again. seen by the reduction of recognition errors shown in
Closed loop scaling and deskewing are similar to their Table 2.
respective open loop counterparts, except that in the Processing times for the various algorithms were
closed loop case the histogram is calculated after each found by Brown and Ganapathy on the Amdahl 470
transformation, rather than just once, to determine the using operating systems software, and they established
result of that transformation. Verification is performed that preprocessing could be performed at a rate exceed-

vol4 no 4 novemher 1986 217

ing 300 words/min, which is typical of human reading
rates. With the use of special purpose hardware much
of the preprocessing could be speeded up and pipelined
with the recognition stage to maximize throughout.
A recognition rate of about 97% was established by
Huang and Chuang using a digital TV camera input
with 128 x 128 resolution. The closed loop effectively
was achieved through use of a dictionary set up during
input of the first 200 of 400 randomly written numerals.
CONCLUSIONS
Character recognition has been of great interest to
scientists for over 25 years. It is the quest to study human
perception, in combination with the advantages of com-
puter processing, that has made this field one of such
significance. Research has progressed from early
cumbersome optical character readers, involving massive
hardware facilities, for recognition of a limited set of
individual characters, to the automatic recognition of
handwritten cursive script.
In these systems, the problem of storing the vast
amounts of information involved in an unprocessed
character proved to be particularly acute. The advent
of relatively cheap processing power, along with the
advance in technology, made CR more within the
reaches of automation. This resulted in much research
work from the mid to late 1960s onwards, initially
dealing with recognition of individual characters before
progressing to the more complex feat of cursive script
recognition.
As the automation of CR can be achieved offline or
online, attention has been given to each of these to
understand the similarities and differences between such
systems. It was found that although online systems tend
to use graphics facilities for the writing of characters,
with offline adopting an optical scanning technique, the
processes necessary were basically the same. These are,
namely, segmentation of the characters (if needed),
preprocessing, feature extraction and cfassification.
For the recognition of individual characters three
varying techniques were studied. These are
l recognition of characters as they are drawn using
a graphics tablet,
l recognition using the geometrical features of a
character
l recognition of matrix represented characters by con-
218
sidering the distribution of points as descriptive of
the character.
Cursive script recognition will prove most advantageous
to industry in terms of the efficiency gained, and it is
already possible for recognition of words at a speed
exceeding that of human reading.
REFERENCES
1 Suen, C Y, Berthod, M and Mori, S ‘Automatic recog-
nition of handprinted characters - the state of the
art’ Proc. IEEE Vol68 No 4 (April 1980) pp 472487
2 Munson, J H ‘Experiments in the recognition of
handprinted text: part I - character recognition’
Proc. Fall Jt. Comp. Conf. Vol 33 (Dec. 1968) pp
1125-1138
3 Highleyman, W H ‘An analog method for character
recognition’, IRE Trans. Electron. Comput. Vol 10
(Sept. 1961) pp 502-512
4 Tou, J T and Gonzalez, R C ‘Recognition of hand-
written characters by topological feature extraction
and multilevel categorization’ IEEE Trans. Comput.
Vol21 (Feb. 1972) pp 321-331
5 Hussain, A B S, Toussaint, G T and Donaldson, R
W ‘Results obtained using a simple character recog-
nition procedure on Munson’s handprinted data’
IEEE Trans. Comput. Voi 21 (Feb. 1972) pp 201-205
6 Toussaint, G T and Donaldson, R W ‘Algorithms for
recognizing contour-traced handprinted characters’
IEEE Trans. Comput. Vol 19 (June 1970) pp 541-546
7 Arakawa, H ‘On-line recognition of handw~tten
characters ~ alphanumeric, Hiragana, Katakana,
Kanji’ Pattern Recognition Vol 16 No 1 (1983) pp
25-34
8 Harmon, L D ‘Automatic recognition of print and
script’ Proc. IEEE Vol60 (Oct. 1972) pp 1165-I 176
9 Brown, M K and Ganapathy, S ‘Preprocessing
techniques for cursive script word recognition’
Pattern Recognition Vol 16 NO 1 (1983) pp 35-42
10 Shridhar, M and Badreldin, A ‘Recognition of
isolated and simply connected handwritten numerals’
Pattern Recognition Vol 19 No 1 (1986) pp l-12
11 Shridhar M and Badreldin, A ‘Handwritten numeral
recognition by tree classification methods’ Image
Vision Comput. Vo12 No 3 (August 1984) pp 143-149
12 Huang, J S and Chuang, K ‘Heuristic approach to
handwritten numeral recognition’ Pattern Recog-
nition Vol 19 No 1 (1986) pp 15-20
image and vision computing