SIViP (2011) 5:165183
DOI 10.1007/s11760-010-0152-1
ORIGINAL PAPER
Statistical modeling for the detection, localization and extraction
of text from heterogeneous textual images using combined
feature scheme
Chitrakala Gopalan D. Manjula
Received: 29 April 2009 / Revised: 1 January 2010 / Accepted: 1 January 2010 / Published online: 29 January 2010
Springer-Verlag London Limited 2010
Abstract Discriminating between the text and non text
regions of an image is a complex and challenging task. In con-
trast to Caption text, Scene text can have any orientation and
may be distorted by the perspective projection. Moreover, it
is often affected by variations in scene and camera parameters
such as illumination, focus, etc. These variations make the
design of unified text extraction from various kinds of images
extremely difficult. This paper proposes a statistical unified
approach for the extraction of text from hybrid textual images
(both Scene text and Caption text in an image) and Document
images with variations in text by using carefully selected fea-
tures with the help of multi level feature priority (MLFP)
algorithm. The selected features are combinedly found to be
the good choice of feature vectors and have the efficacy to
discriminate between text and non text regions for Scene text,
Caption text and Document images and the proposed system
is robust to illumination, transformation/perspective projec-
tion, font size and radially changing/angular text. MLFP fea-
ture selection algorithm is evaluated with three common ML
algorithms: a decision tree inducer (C4.5), a naive Bayes
classifier, and an instance based K-nearest neighbour learner
and effectiveness of MLFP is shown by comparing with three
Chitrakala Gopalan (B)
Department of Computer Science and Engineering,
Easwari Engineering College, Anna University,
Ramapuram, Chennai, Tamil Nadu, India
e-mail: ckgops@gmail.com
Chitrakala Gopalan
4/3, Kaliappa Naicker St., Plot no: 80, Nehru Nagar,
Ramapuram, Chennai 600089, Tamil Nadu, India
D. Manjula
Department of Computer Science and Engineering,
College of Engineering, Anna University, Guindy,
Chennai, Tamil Nadu, India
e-mail: manju@annauniv.edu
feature selection methods with benchmark dataset. The pro-
posed text extraction system is compared with the Edge based
method, Connected component method and Texture based
method and shown encouraging result and finds its major
application in preprocessing for optical character recognition
technique and multimedia processing, mobile robot naviga-
tion, vehicle license detection and recognition, page segmen-
tation and text-based image indexing, etc.
Keywords Text extraction Non sub sampled Contourlet
Transform Gray level run length matrix Caption text
Scene text Document image
Abbreviations
NSCT Non sub sampled Contourlet Transform
NSP Non sub sampled pyramid
NSDFB Non sub sampled directional filter bank
CC Connected component
NLV Normalized local variance
GLRLM Gray level run length matrix
GLCM Gray level co-occurrence matrix
MLFP Multi level feature priority
VM Closing Vertical morphological closing
HM Closing Horizontal morphological closing
1 Introduction
The growing popularity of Internet and the World Wide Web
has resulted in the tremendous growth of multimedia data
containing still images and video, in addition to the textual
information. Text in images/video usually carries important
messages about the content. Indexing, querying and retriev-
ing multimedia information uses the textual information
embedded in the multimedia data.
123
166
This has created an overriding need to provide efficient
means of text extraction methodologies from textual images
for effective data storage, retrieval, search, querying and
interaction capabilities and preprocessing for OCR. Extract-
ing embedded/inserted text in images often gives an indica-
tion of a scenes semantic content. Automatic extraction of
text is a challenging job due to variation in font style, size,
orientation, alignment and complexity of background.
Text in images/videos is classified into Caption text and
Scene text [1]. Caption text image is the one which is the
inserted text and is otherwise called as superimposed/artifi-
cial text. Natural textual images/embedded texts are called as
Scene texts or graphics text images. Electronic documents,
images of paper documents, images acquired by scanning
book covers, CD covers or other multi-colored documents
are called Document images.
Literature studies so far addressed three different app-
roaches to extract the text from images namely Bottom-up,
Top-down and Hybrid approaches. Bottom-up approach
starts with the identification of sub-structures, such as con-
nected components (CCs) or edges and then merging these
sub-structures to mark bounding boxes for text. Top-down
approach looks for global information in the page and splits
the page from column level to word level. Edge based and
Connected component (CC) based methods are categorized
under Bottom-up methods and Texture based methods under
Top-down approach. The proposed system employs Non sub-
sampled Contourlet transform and texture analysis to extract
text from Caption text image, Scene text image and Doc-
ument image with variations in illumination, font size, per-
spective projection and orientation by using MLFP algorithm
with Neural network classifier.
2 Related work
The Literature has been surveyed to find existing methods for
text extraction from different kinds of images which are [2]
based on the combination of connected component and tex-
ture feature analysis of unknown text region contours. Each
candidate text region is verified with texture features derived
from wavelet domain followed by expectation maximization
algorithm to binarise each text region. Jiang et al. [3] pro-
posed a learning-based method for text detection and text
segmentation in natural scene images. Here, the input image
is decomposed into multiple CCs by Niblack clustering algo-
rithm. Then all the CCs including text CCs and non text CCs
are verified on their text features by a two-stage classification
module.
Karatzas and Antonacopoulos [4] follows a split-and-
merge strategy based on the Hue-Lightness-Saturation (HLS)
representation of color as a first approximation of an anthro-
pocentric expression of the differences in chromaticity and
123
SIViP (2011) 5:165183
lightness. Character-like components are then extracted as
forming text lines in a number of orientations and along
curves. Kumar et al. [5] proposed globally matched wavelet
filters with Fisher classifiers for text extraction from Docu-
ment images and Scene text images. Liu and Samarabandu
[6] proposed Edge based method with edge strength, density
and the orientation variance as distinguishing characteristics
of text embedded in images which can handle printed doc-
ument and Scene text images. This method used multi scale
edge detector for text detection and dilation operator for text
localization stages.
Gllavata et al. [7] proposed connected component based
method which uses color reduction technique followed by
horizontal projection profile analysis which can extract text
from Caption text images. Li et al. [8] presented algorithms
for detecting and tracking text in digital video and implement
a scale-space feature extractor that feeds an artificial neural
processor to detect text blocks. Lin and Tan [9] proposed a
method to apply a neural network on canny edges with both
spatial and relative features like sizes, color attributes and
relative alignment features. By making use of the alignment
information, we can identify the text area from the charac-
ter level rather than the conventional window block level.
In [10], the region-of-interests (ROI) probably containing
the overlay texts are decomposed into several hypothetical
binary images using color space partitioning. A grouping
algorithm is then conducted to group the identified character
blocks into text lines in each binary image.
Jeong et al. [11] classifies text pixels and non text pixels
using a network that operates as a set of texture discrim-
ination filter to find and locate text regions from Caption
text images using histogram analysis after removing errors
in the classification results. Pan et al. [12] proposes a hybrid
method to localize texts in natural scene images with a Con-
ditional Random Field (CRF) model, considering the unary
component property as well as binary neighboring compo-
nent relationship. Finally text components are grouped into
text lines with an energy minimization approach.
Phan [13] proposes a text detection method from Video
based on the Laplacian operator. K-means is then used to
classify all the pixels into two clusters: text and non text.
This method undergoes projection profile analysis to deter-
mine the boundary of the text blocks and employ empirical
rules to eliminate false positives based on geometrical prop-
erties. Experimental results show that the proposed method
is able to detect text of different fonts, contrast and back-
grounds.
Shi et al. [14] presents an algorithm using adaptive local
connectivity map for retrieving text lines from the complex
handwritten documents such as handwritten historical manu-
scripts. The algorithm is designed for solving the problems
like fluctuating text lines, touching or crossing text lines and
low quality image seen in handwritten documents.
SIViP (2011) 5:165183
The above mentioned approaches focused on extracting
text from Scene text, Caption text and Document images sep-
arately or on some combinations only. In contrast to Caption
text, Scene text can have any orientation and may be distorted
by the perspective projection. Moreover, it is often affected
by variations in scene and camera parameters such as illumi-
nation, focus, etc. These variations make the design of uni-
fied text extraction from various kinds of images extremely
difficult.
Recently we [15] proposed an Image analysis based
approach called Sub Band Texture Analysis based Text
Detection/Text Localization (SBTA-TD/TL) technique for
text extraction from heterogeneous images which is robust to
limited orientation (horizontal, vertical) and limited font size
of images. Comparable performance for Scene text images
was not produced as well. It was also observed that SBTA-
TD/TL technique suffers and not showing encouraging
results for the following variations
Variation in illumination
Variation in wide range of font size
Variation in skewness
Variation in angularity of text
Perspective projection
It is required to produce equal performance for Scene
text images also with robustness to the variations in textual
images. Consequently, all these observations, drawbacks and
advantages were analyzed and a more complex Image analy-
sis plus Machine learning based approach called multi level
feature priority (MLFP) technique is proposed for the Text
detection and localization from heterogeneous images which
takes care of variation in illumination, transformation/per-
spective projection, font size, angular and radially chang-
ing text so as to make the system suitable for heterogeneous
images.
3 Approach
The goal is to build a unified extraction of text from hetero-
geneous range of textual images and to focus on transformed,
illuminated and angular form. The major contributions of the
proposed text extraction system are as follows:
Extracts text from Scene text images, Caption text images,
Document images and hybrid text (which has both scene
and caption text) images with comparable performance
for al kinds of images.
Supports multi lingual text images.
Takes care of variation in font size, skewness, angularity
of text and perspective projection
167
Text extraction has been carried out in two phases, namely,
Offline processing (training phase) and Online processing
(testing phase). Offline processing is carried out to extract
and generate feature vectors for training images from the
image corpus. Experimental analysis on various images
indicated that text regions typically have different texture
properties than the non text areas. This is analyzed by decom-
posing the input image with the variation of Contourlet
transform such as Non sub sampled Contourlet Transform
(NSCT) which decomposes the image into a set of direc-
tional 2n sub bands with texture details or edges captured in
different orientations at various scales for n level specified.
Each and every decomposed sub band has high intensity tex-
ture details in various directions with high prominent values
shown for NSCT coefficients which are merged by adding
eight sub bands to form an image with edges detected in
various directions. The transformed NSCT coefficients in an
edge image are used for the calculation of feature vectors for
text and non text regions of images.
Five different sets of features are extracted and analyzed
from the above edge image with MLFP algorithm and best
features are selected. With this MLFP algorithm, textural
distribution of the spatial frequency components within the
decomposed regions (Normalized Local VarianceNLV)
and the feature based on the run which is a series of pix-
els having the same gray level on a definite direction in the
image (Gray level run length matrixGLRLM-based fea-
tures) are collectively found to be the good choice of feature
vector and have the efficacy to discriminate between text and
non text regions for three kinds of images with variations in
lighting, orientation and font size.
During Online processing, when the user supplies an input
image, Selective features are extracted from the transformed
and merged contourlet coefficients by capturing the textural
distribution of the pixels and produce feature vector.
Neural network classifier is used to classify the regions as
candidate text and non text regions from the textual image
by using the extracted feature vector. Candidate text regions
will undergo selected verification rules to eliminate the
unwanted non text regions. Then binarization algorithm is
applied to extract text from text regions by eliminating the
background.
System architecture of the proposed system is shown in
Fig. 1. Proposed approach includes:
(a) Candidate text region detection
Image decomposition using NSCT
Feature extraction and selection using MLFP
algorithm
(b) Text region classification
Neural network classifier
123
168 SIViP (2011) 5:165183

Input Training Images

Textual image

Text NSCT

NSCT
Binarization
Feature extraction

Feature extraction Verification of text regions

Feature selection
Candidate Non
Features Representation of
Text text
Features
images

Neural network
Neural network classifier
Training
(Testing )

ONLINE PROCESSING OFFLINE PROCESSING

Fig. 1 System architecture of the proposed system

(c) Text region Localization
Verification rules
(d) Binarization
4 Candidate text region detection
Always Text extraction from images starts with the candi-
date text region detection which involves the detection of the
presence of the text in the image. In this paper, this detection
is done by
(1) Capturing multi oriented texture components with high
intensity texture details representing primitively and
roughly identified text regions from the NSCT decom-
posed merged sub bands.
(2) Extracting best features from the merged sub bands so
as to distinguish text and non text regions better with
the help of the classifier stage.
These detected candidate text regions are later verified
with the text localization stage.
4.1 Image decomposition using Non sub sampled
Contourlet Transform
The contourlet transform is an extension of the wavelet trans-
form which uses multi scale and directional filter banks.
Here images are oriented at various directions in multiple
scales, with flexible aspect ratios. The contourlet transform
effectively captures smooth contour images that are the dom-
inant feature in natural images. The contourlet transform
[16] is a multi directional and multi scale transform that is
constructed by combining the Laplacian pyramid with the
Directional Filter Bank (DFB). Due to down samplers and
up samplers present in both the Laplacian pyramid and the
DFB, the contourlet transform is not shift-invariant. An over
complete transform, the Non sub sampled Contourlet Trans-
form (NSCT) has been proposed in [17] and applied in the
proposed system. The NSCT is a fully shift-invariant, multi
scale and multi direction expansion that have a fast imple-
mentation. Here filters are designed with better frequency
selectivity thereby achieving better sub band decomposition.
This NSCT transform [17] can thus be divided into two
shift-invariant parts: (1) a non sub sampled pyramid struc-
ture that ensures the multi scale property of the NSCT and is
obtained from a shift-invariant filtering structure that achieves
sub band decomposition similar to that of the Laplacian pyr-
amid and (2) A non sub sampled DFB structure that gives
directionality. A shift-invariant directional expansion is also
obtained with NSDFB. The NSDFB is constructed by elimi-
nating the down samplers and up samplers in the DFB. This is
done by switching off the down samplers/up samplers in each
two-channel filter bank in the DFB tree structure and up sam-
pling the filters accordingly. This results in a tree composed
of two-channel NSDFBs. Refer [15] for detailed description
of NSCT.
The NSCT applied on the input image produces 2n sub
bands for n level specified. Here, eight sub bands have been

123
SIViP (2011) 5:165183
Fig. 2 NSCT coefficients
produced after three level decomposition and they have Multi
oriented texture components with high intensity texture
details representing candidate text regions as shown in Fig. 2,
for Scene text images. The NSCT is a fully shift-invariant,
multi scale and multi direction expansion that have a fast
implementation. A non sub sampled pyramid structure in
NSCT ensures the multi scale property. Large variations in
scale are captured in candidate text region detection stage
itself with the help of NSP in NSCT. Each and every decom-
posed sub band has texture details in various directions with
high prominent values shown for NSCT coefficients. These
prominent values of NSCT coefficients from decomposed
eight sub band images are merged by adding eight sub bands
to form an image with edge detected in various directions so
as to facilitate the extraction of features in next stage.
4.2 Feature extraction
Once the image is converted into gray image, its feature vec-
tor represents them. Several feature extraction strategies are
proposed in the literature and classification using various
image features is analyzed [18]. Texture is the term used
to characterize the surface of a given object or region and it
is one of the main features utilized in image processing and
pattern recognition.
Existing methods for text detection, localization and
extraction can broadly be classified as gradient features
based, color segmentation based and texture features based
[19]. The proposed system employs texture feature based
method. Text is viewed as a unique texture that exhibits a cer-
tain regularity that is distinguishable from the background.
Humans can identify text of foreign languages even when
they do not understand them, largely due to its distinct tex-
ture. Various researchers have exploited this fact to detect
text in images. The texture methods are largely used for text
detection. Texture features can be extracted directly from the
pixels spatial relationships or from frequency data. However,
these methods are often computationally expensive, but lead
to good results. Herewith five sets of texture features are con-
sidered for investigation to analyze the following properties
of the image:
169
(1) Spatial distribution of the gray levels in the texture
image with GLCM-based features
Gray level co-occurrence matrix based features
(2) Textures at different scales with Fractal dimension
(3) Textural distribution of the spatial frequency compo-
nents with NLV
Normalized local variance
(4) Density distribution in the image statistically with
GLRLM-based features in various directions (positive
and negative 0, 45, 90 and 135 directions)
Gray level run length matrix (GLRLM)
(5) Intensity of pixels in the image with Intensity Histogram
based features
Now feature vectors are calculated for each 8 8 block
from the decomposed merged sub band edge image rather
than on original input image because these merged sub bands
have primitively identified candidate text regions. The
extracted feature vectors will be passed through MLFP algo-
rithm and then submitted to a neural network classifier. The
methodology employed in the approach to represent images
is the combination of finally selected NLV and GLRLM
based features which are used to extract the primitives of
an image. The above mentioned 5 sets of features compris-
ing 55 features are extracted from the decomposed merged
sub bands in each block of an image and the list is shown
in Table 1. Features considered for investigation are detailed
below:
4.2.1 Normalized local variance (NLV)
Information about textural distribution of the spatial fre-
quency components within regions in frequency domain is
analyzed and can be used as one of the features to dis-
criminate text and non text areas. This is derived by cal-
culating local energy coefficients of the NSCT of an image
as in Eq. (3). The derived local energy is normalized using
global standard deviation of respective energy coefficient as
in Eq. (2) to obtain a feature set containing additional infor-
mation of textures. It is observed from experimentation that
this NLV feature is high for text and low for non text.
Global mean (GM):
w h
1 
GMi = Ci (u, v) (1)
wh
u=0 v=0
Global standard deviation (GSD) for each coefficient:

w h
v=0 (Ci (u, v) GMi )
2
u=0
GSDi = (2)
wh
123
170 SIViP (2011) 5:165183

Table 1 Set of features

S. No Feature group
analyzed
1 Local energy deviation
of NSCT coefficients
2 Histogram based features
3 Gray level co-occurrence matrix / GLCM
4 Fractal dimension
5 Gray Level Run Length Matrix / GLRLM
based features checked in various
directions. (positive and negative
0, 45, 90 and 135 directions)
Feature name
Normalized Local Variance (NLV)
Mean, Variance, Skew ness, Kurtosis,
Energy, Entropy
Contrast, Correlation, Energy,
Entropy Homogeneity
Short run emphasis (SRE)
Long Run Emphasis (LRE)
Gray level non uniformity (GLN)
Run length non uniformity (RLN)
Low gray level run emphasis (LGRE)
High gray level run emphasis (HGRE)
Short run low gray level emphasis (SRLGE)
Short run high gray level emphasis (SRHGE)
Long run low gray level emphasis (LRLGE)
Long run high gray level emphasis (LRHGE)

Local mean in the window of size N N is: From the original run length matrix p(i, j), many numer-
ical texture measures can be computed [20] such as Short
1 
N N
Mi = Ci (u, v) (3) Run Emphasis (SRE), Long Run Emphasis (LRE), Gray
N2 Level Non uniformity (GLN), Run Length Non uniformity
u=0 v=0
(RLN), Low Gray Level Run Emphasis (LGRE), High Gray
Local coefficient variance V Ci , Level Run Emphasis (HGRE), Short Run Low Gray Level
N N Emphasis (SRLGE), Short Run High Gray Level Emphasis
(Ci (u, v) Mi )2
V Ci (u, v) = u=0 v=0 2 (4) (SRHGE), Long Run Low Gray Level Emphasis (LRLGE)
N
and Long Run High Gray Level Emphasis (LRHGE).
where, w and h are the number of pixels in horizontal and ver- For a given image, a run-length matrix p(i, j) is defined
tical directions respectively, in the image, i = 0 to 9, N = as the number of runs with pixels of gray level i and run
2n (=16), Ci is ith coefficient and the normalized variance length j. Let M be the number of gray levels and N be the
NV Ci (u, v) for each pixel is calculated as follows: maximum run length. Pg vector represents the sum distribu-
V Ci (u, v) tion of the number of runs with gray level i.nr is the total
NVCi (u, v) = (5) number of runs. The subsets of GLRLM features which have
GSDi
been selected with MLFP algorithm as in Table 2 are only
The relative deviation in local coefficient in the region N N described as follows:
window is computed for each pixel of the image.
(1) Gray level non uniformity (GLN):
4.2.2 GLRLM based features
2
1   1 
M N M
Next set of features were extracted from decomposed merged
sub bands using texture analysis based on the GLRLM. Tex- GLN = p(i, j) = Pg (i)2 (6)
nr nr
i=1 j=1 i=1
ture analysis is the method to analyze the density distri-
bution in the image statistically. Texture analysis based on
the GLRLM is one such analysis and is a method based (2) High gray level run emphasis (HGRE):
on the run which is a series of pixels having the same gray
1  1 
level on a definite direction in the image [20]. Since M N M
density, length and direction in the run are included, the HGRE = p(i, j) i 2 = pg (i) i 2
nr nr
aggregation of the run from the image represents features i=1 j=1 i=1
of the texture. (7)

123
SIViP (2011) 5:165183 171

Table 2 Selected features with multi level feature priority algorithm

S. No Feature group Feature name Priority Non text (NT) Text (T)

1 Local energy deviation Normalized local variance (NLV) (1) Low High
of NSCT coefficients
2 GLRLM based features High gray level run emphasis HGRE-0 High Low
3 HGRE-90 High Low
4 Short run high gray level emphasis SRHGE-0 High Low
5 SRHGE-90 High Low
6 Long run high gray level emphasis LRHGE-90 High Low
7 Gray level non uniformity GLN-135 Low High
8 Short run high gray level emphasis SRHGE-45 (2) High Low
9 Long run high gray level emphasis LRHGE-45 High Low
10 High gray level run emphasis HGRE-45 High Low
11 Gray level non uniformity GLN-0 Low High
12 GLN-45 Low High
13 Long run high gray level emphasis LRHGE-135 (3) High Low

(3) Short run high gray level emphasis (SRHGE):
1   p(i, j) i 2
M N
SRHGE = (8)
nr j2
i=1 j=1
(4) Long Run High Gray Level Emphasis (LRHGE):
1 
M N
LRHGE = p(i, j) i 2 j 2 (9)
nr
i=1 j=1
These features are used to capture apparent properties of
run length distribution. It is observed from experimentation
that the above selected features have shown distinguishable
difference between text and non text as shown in Fig. 4ae.
4.2.3 Gray level co-occurrence matrix based features
(GLCM)
Gray level co-occurrence matrix (GLCM) can reveal certain
properties about the spatial distribution of the gray levels in
the texture image as proposed by Haralick et al. [21] and
is also known as the Gray Level spatial dependence matrix.
Hence, GLCM-based features were considered for feature
extraction. GLCM is a tabulation of how often different com-
binations of pixel values (grey levels) occur in an image.
When divided by the total number of neighboring pixels in
the image, this matrix becomes the estimate of the joint prob-
ability of two pixels, a distance d apart in a direction e having
particular (co-occurring) grey values i and j. The dimension
of GLCM is Gx G where G is the number of grey levels
used to construct the matrix. Various statistics can be derived
from the created GLCM and can provide information about
the texture of an image. Five out of the fourteen Haralick
features [21] have been used in the approach.
These features are: contrast, homogeneity, energy, entropy
and correlation. To calculate different features, the joint prob-
ability density of grey level co occurrence computed by
GLCM is weighted differently. The first two features (con-
trast, homogeneity) can be grouped and named as Contrast
Group. They compute quantity of contrast in a window. The
second group called Orderliness Group contains features
(energy and entropy) which indicate how regular (orderly)
the pixel values are within the window.
(a) Contrast measures the local variations in the gray level
co-occurrence matrix. It is calculated as

L1
Contrast = Pi j (i j)2 (10)
i, j=0
Here Pi j = Element i, j of the normalized symmetrical
GLCM.
N = Number of gray levels in the image.
(b) Homogeneity measures the closeness of the distribution
of elements in the GLCM to the GLCM diagonal. It is
calculated as

L1
Homogeneity = Pi j /1 + (i j)2 (11)
i, j=0

123
172 SIViP (2011) 5:165183

(c) Energy provides the sum of squared elements in the where m 3 is the third moment
GLCM. Also known as uniformity or the angular sec-

L1
ond moment. It is calculated as Kurtosis = 1/4 (x m 4 )4 Pu(x) (19)
x=0

L1
Energy = (Pi j )2 (12) 4.2.5 Fractal dimension
i, j=0

Fractal dimension is a parameter frequently used to analyze

(d) Entropy measures the average, global information con- textures at different scales. Fractal dimension values indicate
tent of an image in terms of average bits per pixel. the complexity of a pattern or the quantity of information
embodied in a pattern in terms of morphological, entropy,

N 1 spectral and variance [22]. FD serves as an index of the mor-
Entropy = Pi j log2 Pi j (13) pho metric complexity and variability of the object being
i, j=0 studied. The fractal dimension indicates the complexity of
images. Because the picture components are more compli-
(e) Correlation measures the linear dependency of grey lev- cated than the text components, the picture components have
els on those of neighboring pixels. It is calculated as higher fractal dimensions than the text components. To cal-
culate fractal dimension using the box dimension method, a
square grid of size s is superimposed over the image. N (s),

L1
Correlation = Pi j (i )( j )/ 2 (14) the number of grids that contain part of the image is counted.
i, j=0 Box dimension D can be calculated using the following
formula:
4.2.4 Intensity histogram based features D = log N (s)/ log 1/s (20)

The intensity of the pixels in an image helps to differentiate

4.3 Feature selection algorithm
the text pixels from that of the background pixels. Hence, the
first order intensity histogram statistical features were con-
Finding an optimal set of features from a large set of can-
sidered for feature extraction. The intensity histogram was
didate features is a problem which occurs in many contexts.
created by quantizing the gray-scale intensity values into the
Several feature selection strategies are proposed in the liter-
range 0255 and then making a 256-bin histogram for these
ature and analyzed in [23].
values. The histogram was then normalized by dividing all
Heuristic algorithms for finding near-optimal solutions
values by the total sum. The mean, second and third moments
[24] require both the definition of a strategy for selecting
are derived from the intensity histogram. The histogram fea-
feature subspaces and the definition of a function, i.e. how
tures are calculated as follows:
well classes result separated in the selected feature subspace.
Let u be a random variable representing a gray level in a
In this paper, proposed MLFP algorithm follows sequential
given region of the image. Define:
backward selection approach, which removes one feature at
Pu (x) = Prob(u = x) = no. of pixels with gray level x/ a time with low discrimination ability until no improvement
in the criterion function is obtained. The criterion function
total no. of pixels in the region is problem specific and can vary from simple performance
(15) measures to complex multistage evaluation procedures.
The evaluation methods for evaluating the goodness of
Now the mean, variance, skewness and kurtosis are defined each selection can be divided into two wide classes: (1) Fil-
as follows: ter method evaluate a feature subset independent of the clas-

L1 sifier and are usually based on some statistical measures of
Mean = E(u) = x Pu(x) (16) distance between the samples belonging to different classes
x=0 and (2) Wrapper method is based on the classification results

L1 achieved by a given classifier. Filter methods are usually
Variance = (x m 1 )2 Pu(x) (17) faster than wrapper ones, but selecting comparatively more
x=0 number of features. The approach here follows filter-based

L1 method for evaluation and aims at producing small number
Skewness = (x m 3 )3 Pu(x) (18) of features at less running time with Difference between clas-
x=0 ses (DBC) based consistency. This algorithm also categorizes

123
SIViP (2011) 5:165183
the selected features, meeting the criteria into various priority
levels at the time of the selection of the features itself.
4.3.1 Multi level feature priority (MLFP) algorithm
Multi level feature priority algorithm aims for the selection
of the most appropriate subset of features that adequately
describes a given classification task. MLFP algorithm is based
on sequential backward selection method, which considers
all features and removes a feature with low discrimination
ability to distinguish text and non text regions of an image.
This algorithm also categorizes the selected features, meet-
ing the criteria into various priority levels at the time of the
selection of the features itself.
This algorithm selects and categorizes the features as first
level (top) features if difference between feature values
of text and non text regions exceeds the threshold and the
same is maintained for all n images/consistent for all
n images.
The algorithm selects and categorizes the features as sec-
ond level features if the difference between feature val-
ues of text and non text regions exceeds the threshold but
consistent only for large subset of images n1, where
n1 n.
The algorithm selects and categorizes the features as
third level features if the difference between feature val-
ues of text and non text regions exceeds the threshold and
consistent not for all images but only for small subset of
images n2, where n2 n.
Various steps involved in MLFP algorithm during offline
processing are as follows:
1. For each image of m blocks, compute the following for
each feature:
Fi T = f i T 1 + + f i T n /N
Fi N T = f i N T 1 + + f i N T m /M
DFi = Fi T Fi N T
where f i T 1 , . . . , f i T n and f i N T 1 , . . . , f i N T n are fea-
ture values in various text and non text regions of the
image respectively and N and M are the total number
of text and non text regions in the image. Fi T , Fi N T are
the mean feature values for text and non text regions,
respectively, and DFi is the difference between those
mean values.
2. flag = flag + 1; {if DFi > threshold}
3. Repeat the above two steps for all N images.
173
4. Maximum relevant features are selected and categorized
into various priority sets such as P1 , P2 , P3 (with P1 as
the highest priority) based on Difference between clas-
ses (DBC) based consistency i.e., based on the differ-
ence between the flag and number of images N . If the
difference between text and non text of a particular fea-
ture is being maintained a value above the threshold for
all images, choose these kinds of features as a candidate
for first highest priority level P1 .
P1 = { f i / |flag| N = 0} (23)
P2 = { f i / |flag| N < t1} (24)
P3 = { f i / |flag| N < t2 where t2  t1} (25)
where  is a partial ordering such that t1  t2 is
defined as t1 t2 0.
5. Selected priority set of features must be maximum
relevant and minimum redundant in order to qualify as
significant features. Maximum relevancy is rechecked
by consistency and Minimum redundancy is checked by
correlation among the features.
Consistency: Optimal set of features to distinguish text
and non text regions of an image should be consistent or
homogeneous. It indicates that difference between text and
non text of a particular feature should be consistent for all
images irrespective of the types of images such as Caption
text, Scene text and Document images. These kinds of fea-
tures will become candidature for optimal set of features.
Consistency is measured by coefficient of variation (C.V)
and is calculated as follows:
Coefficients of dispersion (C.D) are calculated [25] to
compare the variability of the two series which differ widely
in their averages or which are measured in different units.
Coefficient of dispersion (C.D) based upon standard devi-
ation is given as:
C.D = S.D/Mean = / X (26)
(21) Coefficient of Variation: 100 times the coefficient of disper-
sion based upon standard deviation is called coefficient of
variation (C.V) and it is calculated as
C.V = 100 (/ X ) (27)
According to Professor Karl Pearson who suggested this
measure, C.V is the percentage variation in the mean,
standard deviation being considered as the total variation
in the mean. The series having greater C.V are said to be
more variable than the other and the series having lesser C.V
are said to be more consistent (or homogeneous) than the
(22) other.
Correlation: Optimal subset of features should not be
redundant. Redundancy is checked by the correlation
123
174 SIViP (2011) 5:165183

between the features. Correlation criteria can detect linear Procedure Multi level Feature Priority ( )

dependencies between variable and target. It is a measure Step 1: Start

flag = 0
of the strength of linear association between two variables. Step 2: For i=1 to n features
Correlation coefficient r is calculated as follows: Correlation Step 3: For each j= 1 to x images
Step 4: For each k = 1 to m of Id
Coefficient (r ) = Covariance (x, y)/x y Compute Mean (i) for text region
 

Compute Mean (i) for non text region
r=N XY X Y / Step 5: Next
Compute dtn betwn text and non text values from Mean


2    
2 
fix_ threshold()
 If dtn > = Th
N X2 X N Y2 Y flag = flag +1
Step 6: Next
If flag = = x
(28) Include i into Priority SET 1
Else if | flag - x | < t1
where N is the number of values. Include i into Priority SET 2
Else if | flag - x | < t2
The value of r lies between 1 and 1, inclusive. If X and Include i into Priority SET 3
Y are completely correlated, r takes the value of 1 or 1. If X Step 7: Next
Step 8: Priority Set Evaluation ()
and Y are independent, r is zero. It is a symmetrical measure {Min_ redundancy check ();
for two variables. If X and Y have a strong positive linear Max_ relevancy check () ;}
{/*Returns Significant
correlation, r is close to +1. A feature with r value of exactly Features*/}
+1 indicates a perfect positive fit which are truly redundant in Step 9: End

the sense that no additional information is gained by adding

Procedure fix_threshold ( )
them. If X and Y have a strong negative linear correlation, Step 1: Start
r is close to 1. If there is no linear correlation or a weak Step 2: For i=1 to n features
Step 3: For j=1 to x images
linear correlation, r is close to 0. A correlation greater than Draw histogram for text and non text regions
0.8 (close to 1) are generally described as strong, whereas a Th = Min (Valley) of histogram
Step 4: Next
correlation less than 0.5 (close to 0) are generally described Step 5: Next
Step 6: End
as weak.
6. Then qualified top level significant features are used to
define a primitive to distinguish text and non text regions Table 3 Selected features with coefficient of variation (CV)
of the image. S. No Feature Name CV
The Procedure for MLFP is summarized as follows:
1 SRHGE-0 10.47
Declare 2 HGRE-0 14.92
n: no. of features 3 LRHGE-90 15.46
4 SRHGE-90 17.30
k: no. of levels
5 (NLV) 19.86
m: no. of blocks in segmented
6 GLN-135 20.48
decomposed image 7 HGRE-90 21.99
x: no. of images 8 GLN-0 25.15
N : Sub bands count 9 GLN-45 25.46

Th: Threshold for difference value 10 SRHGE-45 27.68

11 LRHGE-45 33.13
dm : Difference between the text and non text values
12 HGRE-45 36.08
t1, t2, t3: Thresholds to decide priority, 13 LRHGE-135 40.39
such that t1  t2  t3

4.3.2 Analysis of Features with MLFP algorithm
Above 5 sets of features comprising 55 features are analyzed
and 13 features are selected and categorized into 3 priorities
using MLFP algorithm as in Table 2. C.V is calculated for
the series with difference of value between text and non text
regions for all 55 features. It is observed that top seven fea-
tures comprising P1 have scored lesser C.V than others and
show that they are consistent. There is a marginal rise in C.V
for Priority sets P2 , P3 than P1 as in Table 3.
Correlation coefficient matrix values for all 55 features
are calculated and observed that r value for P1 set features is
close to 0 ranging between 0.5 to 0 and 0 to 0.5 and shows
that features are weakly correlated and are not redundant.
r Value for seven features are listed in Table 4 with self corre-
lation shown as 1. Distributions of some features from Fig. 4

123
SIViP (2011) 5:165183 175

Table 4 Selected Priority 1

level features with correlation r NLV HGRE-0 HGRE-90 SRHGE-0 SRHGE-90 LRHGE-90 GLN-135
coefficient (r )
NLV 1.00 0.07 0.43 0.05 0.30 0.21 0.18
HGRE-0 0.07 1.00 0.15 0.22 0.50 0.24 0.22
HGRE-90 0.43 0.15 1.00 0.41 0.38 0.04 0.36
SRHGE-0 0.05 0.22 0.41 1.00 0.50 0.03 0.43
SRHGE-90 0.30 0.50 0.38 0.50 1.00 0.27 0.33
LRHGE-90 0.21 0.24 0.04 0.03 0.27 1.00 0.38
GLN-135 0.18 0.22 0.36 0.43 0.33 0.38 1.00

LRHGELong Run High Gray Level Emphasis

H, V, DHorizontal, Vertical & Diagonal direction

It is observed from experimentation that above selected

features have shown distinguishable difference between text
and non text and two features are shown in Figs. 3 and 4a and
all seven features showing remarkable difference between
text and non text for set of images as shown in Fig. 4b.

Fig. 3 NLV feature differentiating text and non text 4.3.3 Results of feature selection algorithm
and comparison with other methods

may seem to be same but their variance differs which make
them weakly correlated and not redundant.
Subsequently, out of 13 features, only 7 top level prior-
ity features are selected based on the filter-based evaluation
method and are used to define a primitive P to distinguish
text and non text regions and is described as a seven-attribute
tuple where
P = (NLV, HGRE-H, HGRE-V, SRHGE-H,
SRHGE-V, LRHGE-V, GLN-D-135)
NLVNormalized Local Variance
GLNGray Level Non uniformity
HGREHigh Gray Level Run Emphasis
SRHGEShort Run High Gray Level Emphasis
Our data set for feature selection algorithm is created by
pooling features from 5 different texture models compris-
ing 55 features. In addition to this, some more data sets are
selected from UCI Machine Learning Repository. WEKAs
collection (27) is used to implement all the algorithms cho-
sen for comparison. In this section, we evaluate the efficiency
and effectiveness of our method by comparing MLFP with
representative feature selection algorithms such as Genetic
search algorithm, ReliefF and Greedy stepwise search
algorithm [27]. For each data set, we first run all the feature
selection algorithms in comparison, and obtain the selected
features for each algorithm as in Table 5. It is observed that
all these algorithms achieve significant reduction of dimen-
sionality by selecting only a small portion of the original
features. MLFP on average selects the smallest number of
features.

Fig. 4 Various features distinguishing text (T) and non text (NT). a One of GLRLM feature- GLN 135, b selected features differentiating
T and NT

123
176 SIViP (2011) 5:165183

Table 5 Number of features

selected by each feature Data set (No. of features) MLFP Genetic search Greedy stepwise ReliefF
selection (FS) algorithm
Our textual dataset 55 7 10 8 8
Splice 60 9 12 11 10
Chemical 150 8 14 12 11
CoIL2000 85 10 18 17 16
Average feature count 350 34 54 48 45

Table 6 Accuracy of various

classifiers on selected features Classifier FS
for each FS algorithm MLFP-FS Genetic search Greedy stepwise ReliefF

F1 RMSE F1 RMSE F1 RMSE F1 RMSE

IBk 0.89 0.3282 0.75 0.5 0.5 0.679 0.69 0.5

NBC 0.85 0.4097 0.67 0.5 0.75 0.5993 0.33 0.7071
C4.5 0.90 0.2236 0.5 0.71 0.5 0.7071 0.73 0.5

F1-measure for FS algorithms 5 Text region classification

IBk NBC C4.5
F1-measure

1
Here neural network classifier is used to classify objects into
0.5 one or two or more groups based on a set of features that
0
describe the objects. The above mentioned seven features
MLFP-FS Genetic search Greedy ReliefF are used as the basis for classification. After extracting the
Method features, a neural network is trained to identify text-like win-
Fig. 5 F1-measure for feature selection algorithm dows. The network used is the general feed-forward network
trained with back propagation. To train the network,

In addition, three bench mark learning algorithms, Naive
bayes Classifier [28], lazy instance-based algorithm (k-NN) -
IBk nearest neighbour [29] with k = 5 and eager model
based C4.5 Decision tree algorithm [30] are used to evalu-
ate the predictive accuracy on the selected subset of features.
F1 measure and Root mean square error (RMSE) calculated
for all datasets and average of those measures are listed in
Table 6. These measures are estimated with the use of tenfold
stratified cross-validation tests.
It is observed from Table 6 and Fig. 5 that for each of the
four data sets, the highest accuracy is achieved by applying
IBk and C4.5 on the feature subset selected by MLFP.
MLFP achieves higher accuracy, lower RMSE, smallest
number of selected features compared with other FS algo-
rithms. The reason is that importance of the features is
checked at two levels in MLFP. Relevant features are selected
based on DBC-based consistency and features are catego-
rized into priority sets with high relevance. Second time of
consistency check is incorporated during priority set eval-
uation. So MLFP produces comparatively small number of
features with high relevance and is faster as it follows filter
method.
1. Choose the mask for typical text and non text regions.
2. Extract features for these regions separately
3. Calculate mean of appropriate feature for all text and non
text regions separately
4. Pass that mean for training
Each small window is classified using the trained neural
net on the feature vector associated with that window. If a
window is classified as text, all the pixels in this window
will be labeled as text. Those pixels which are not covered
by any text window are labeled as non text. As the result of
classification, a label map of the original image is generated
as in Fig. 6b.
6 Text region localization
6.1 Verification of text regions
The generated label map of the original image after text
region classification consists of detected text regions, but
may include non text as text regions also. These detected
regions have to be verified as text regions with the removal

123
SIViP (2011) 5:165183 177

Fig. 6 Various steps in the MLFP method. a Input image, b O/P of tal projection, j vertical projection, k logical AND of horiz verti image,
NN classifier, c VM closing, d HM closing, e morphological opening, l binary image, m O/P text region, n ground truth image
f bounding box region, g superimpose with i/p, h edge image, i horizon-

of unwanted non text regions. This verification stage under where z = {X, Y j }, j = 0, 1, . . . n, X = C1 for VM
text localization stage groups and localizes the text regions closing
identified by the detection stage into text instances. The ver-
ification rules are used for text localization to
(A B) = (A B) B = {z |(B)z A = } (30)
(a) Identify and remove small unwanted non text regions.
(b) Group the unfilled pixels by closing all the gaps.

The following verification rules are shown in Fig. 6. After the
mask region is obtained from neural network,
1. Perform vertical morphological (VM) closing and hori-
zontal morphological (HM) closing operation as in
Eqs. (29) and (30) to smooth the contour, fuses narrow
breaks and eliminates small holes. Closing is performed
with dilation followed by erosion where C1, C2 are
constants.
(A B) = (A B) B = {z |(B)z A = } (29)
where z = {X i , Y }, i = 0, 1, . . . n, Y = C2 for HM
closing
2. Binary area open operation is performed to remove all
connected components (objects) that have fewer than P
pixels from a binary image.
3. Place the bounding box for every region identified and
corresponding region is taken. Superimpose this mask
region with input image to get candidate text region.
4. Perform edge detection with sobel operator.
5. Calculate horizontal projection (HP) for every row from
an edge image.

123
178 SIViP (2011) 5:165183

(a) Initialize new binary image with 0 same as size as 7. Perform logical AND of the output images from HP and
input image VP to get the text region
(b) Assign 1 for only Rows with (HP) > Mean (HP) T = i j (35)
x


1| Hi > H
8. This binary image is superimposed with input image to
i=0
i = (31) get text from image.


x
0| Hi > H
i=0
(c) Perform open operation vertically
7 Binarization
(A B) = (A B) B = {(B)z|(B) Z A}
(32) Before the extracted text is parsed and recognized by the
Common OCR, it has to be binarised to extract the text
where z = {X, Y j }, j = 0, 1, . . . n, X = C1 from the identified text region. Well known global thres-
6. Calculate Vertical projection (VP) for every column from holding method described by Otsu [26] is tried for binari-
edge image zation. All intensity values below a selective threshold are
converted to one intensity level and intensities higher than
(a) Initialize new binary image with 0 same as size as this threshold are converted to the other chosen intensity.
input image It segments an image into foreground and background. The
(b) Assign 1 for only columns with VP > mean (VP) foreground contains interested characters and this process

y
generates an output image with white text against a black

1| V j > V
background.
j=0
j = y (33)

<


0| V j V

j=0 8 Results and performance analysis
(c) Perform open operation horizontally
8.1 Experimentation setup and Performance measures
(A B) = (A B) B = {(B)z|(B) Z A}
(34) In the experimental part, data sets for various textual images
with variations in illumination, perspective, font size and
where z = {X i , Y }, i = 0, 1, . . . n, Y = C2 orientation are considered as in Table 7 and tested using

Table 7 Data set for text

extraction experimentation Image group Image type No. of images

Caption text images News telecast & video titles 25

Scene text images Name plate images
Number plate images
Book cover images
General embedded text images 90
Variations:
Skewed images 15
Perspective projection 15
Angular text 20
Radially changing text 20
Illuminated images (day, evening and night light ) 15
Language tested (English, Tamil and Hindi) 15
Document images Scanned document image 50
Web document image 20
Hybrid Scene and Caption text in same image 15
Total images 300

123
SIViP (2011) 5:165183
1.79 GHz system having 1GB RAM. These data set images
have been gathered from sites of several research groups such
as Laboratory for Language and Media Processing (LAMP),
Automatic Movie Content Analysis (MoCA) Project, Com-
puter Vision Lab., Pennsylvania State University. The sig-
nificance of testing the algorithms on the above variations is
to ensure the efficiency of this methodology to function as a
unified technique.
The performance of the proposed technique has been eval-
uated based on its precision and recall rates and F score
obtained. In order to have a common method to evaluate and
compare the results from each algorithm, in this project each
text pixel in ground truth and output image is considered
in the calculation of precision and recall rate. Precision and
recall rates are calculated as follows where CDP (Correctly
Detected Pixels) is the number of pixels matching between
output mask image (O) and Ground Truth image (GT).
CDP = O GT (36)
Precision rate (PR) = CDP/(CDP + FP) (37)
Recall rate (RR) = CDP/(CDP + FN) (38)
Precision rate takes into consideration the false positives,
which are the non text pixels in output image and have been
detected by the algorithm as text pixels. They are not text
pixels in ground truth image but detected as text pixels in
output mask image
FP = O GT (39)
Recall rate takes into consideration the false negatives,
which are text pixels in output image, and have not been
detected by the algorithm. They are text pixels in ground
truth image but not detected as text pixels in output mask
image
FN = O GT (40)
Thus, precision and recall rates are useful as measures to
determine the accuracy of each algorithm in locating correct
text regions and eliminating non text regions. F-score is the
harmonic mean of recall and precision rates as represented
in Eq. (29)
F-score = (2 PR RR)/(PR + RR) (41)
Table 8 Average F-score for
the proposed and other methods Image type Method
Edge-based
method (%)
Caption text images 66
Scene text images 63
Document images 71.6
179
8.2 Results and comparison with other text extraction
methods
The proposed text extraction system has been tested with the
dataset mentioned in Table 7. The proposed system produces
good performance for Scene text, Caption text and Docu-
ment images by producing better recall rate missing relatively
weak texts and shown promising precision rate as non text
objects are correctly classified as in Table 8, Figs. 7 and 8. The
results have also been verified against other existing algo-
rithms such as Edge-based method [6], Connected compo-
nent (CC) method [7] and our previous Texture-based method
SBTA-TD/TL [15]. Even though, Edge based method is
designed for Scene text and Document images and
CC method is designed for Caption text images, proposed
method is compared with the above two methods in order to
show the ability of the proposed method to extract text from
all three kinds of images in much better way. We imple-
mented the above mentioned three methods and estimated
with our data sets and compared with the proposed sys-
tem. Performance measures for comparison are shown in
Tables 8 and 9.
The results of the proposed system shows clear improve-
ment over [6,7] methods and when compared to texture based
SBTA method [15], proposed system is showing better per-
formance with F-score of 90 and 81.5% for Caption text and
Scene text images, respectively, as far as the basic three kinds
of images are concerned without any mentioned variations
in text as in Fig. 8 and Table 8.
Subsequently, the proposed system is showing apprecia-
ble improvement in performance over [6,7,15] for hetero-
geneous images with variations in text such as variation in
orientation, font size, perspective projection and lighting con-
ditions, as the proposed system is intended to handle varia-
tions in text so as to qualify as a better unified framework
for text extraction from heterogeneous images than texture
based SBTA method [15].
It is observed from experimentation that MLFP method
produces good and comparable results with F-score of 80,
82.7 and 78.3% for Orientation variation, font size varia-
tion, lighting variation, respectively, as shown in Table 9 and
Fig. 9. The reason for this improvement in performance is
twofold. (1) NSCT is invariant to scale and orientation and
CC Texture Proposed
method (%) method (%) MLFP method (%)
75 86 90
34 78 81.5
75 91.7 91
123
180 SIViP (2011) 5:165183

Fig. 7 Heterogeneous images with variations. a Caption text with small font, b scene text with large font, c scene text with mixed font size,
d document image, e hybrid image with oriented text, f scene image with radially changing text

Table 9 Comparative results

for variations in textual images Methods Variations

Orientation variation Font size variation Lighting variation

P R F-score P R F-score P R F-score

Edge based method 35 80 48.7 46 70.8 55.8 41 87 55.7

CC method 55 82 65.8 50 83 62.4 36 75 48.6
Texture method 70 83 76 67 85 75 60 87 71
MLFP method 75 85 80 78 88 82.7 70 89 78.3

Document images Scene text images Caption text images 76 75 71 80 82.7 78.3
65.8 62.4
55.7 55.8
48.7 48.6
MLFP

Texture
Methods
CC Edge based CC Texture MLFP

Edge based Orientation variance Font size variance Lighting variance

0% 20% 40% 60% 80% 100%

F-score Fig. 9 Comparison of F-score for various variations in images

Fig. 8 Comparison of F-score for heterogeneous images

and results are more robust to the variation in orientation, font

GLRLM features are also extracted in various directions.
(2) Optimal set of features suitable for better discrimina-
tion between text and non text with variations are chosen
by MLFP algorithm. It is observed that the proposed system
is showing clear improvement over other compared methods
size, perspective projection and lighting conditions. One such
result for Scene text with perspective projection and differ-
ent lighting conditions is shown for all above methods and
MLFP method in Figs. 10 and 11. Results of various cases
have been shown in Figs. 7, 10 and 11 as follows:

123
SIViP (2011) 5:165183 181

Fig. 10 Results of images for various methods at three different lighting conditions. a Images at three different lighting conditions, b results from
Edge based algorithm, c CC based algorithm, d texture based algorithm, e proposed MLFP algorithm

Fig. 11 Scene text with perspective projection. a Input image, b CC method, c edge method, d texture, e MLFP method

123
182
1. Caption text with small font in Fig. 7a.
2. Scene text with large font in Fig. 7b.
3. Scene text with mixed font size in Fig. 7c.
4. Document image in Fig. 7d.
5. Hybrid image (with Scene and Caption text) with ori-
ented text in Fig. 7e.
6. Scene text with radially changing text in Fig. 7f.
7. Scene text with different lighting conditions for Edge
based, CC based, texture based and Proposed method in
Fig. 10.
8. Scene text with perspective projection for Edge based,
CC based, Texture based and Proposed method in Fig. 11.
9 Conclusion
In this paper, the objective of extracting text from hetero-
geneous images is achieved by focusing with the variations
involved in orientation, font size, perspective projection and
different lighting conditions with MLFP algorithm applied
on the transformed Contourlet coefficients. Effectiveness of
MLFP feature selection algorithm is shown by producing
optimal set of features with the compared three feature selec-
tion methods. Experimental results also show that the
proposed text extraction method using MLFP algorithm out-
performs the Edge based method and Connected component
method and performs marginally well with Texture based
method for heterogeneous images with appreciable improve-
ment shown for Scene text images. It is showing clear
improvement in the performance over all three methods when
the images are considered with variations. The results indi-
cate that our methodology using NSCT and NLV and
GLRLM feature based MLFP algorithm have the efficacy
to discriminate between text and non text for three kinds of
images with variations in text. As an extension of this system,
it is planned to tune binarization stage to work equally well
for the three kinds of text images also.
References
1. Jung, K., Kim, K.I., Jain, A.K.: Text information extraction in
images and video: a survey. J. Pattern Recogn. Soc. 37(5), 977
997 (2004)
2. Liu, Y., Goto, S., Ikenaga, T.: A contour-based robust algorithm
for text detection in color images. IEICE Trans. Inf. Syst. E89
D(3), 12211230 (2006)
3. Jiang, R., Qi, F., Xu, L., Wu, G., Zhu, K.: A learning-based method
to detect and segment text from scene images. J. Zhejiang Univ.
Sci. A 8(4), 568574 (2007)
4. Karatzas, D., Antonacopoulos, A.: Text extraction from web
images based on a split-and-merge segmentation method using col-
our perception. In: Proceedings of 17th international conference on
pattern recognition (ICPR 2004), August 2004. IEEE Computer
Society Press, pp. 634637
123
SIViP (2011) 5:165183
5. Kumar, S., Gupta, R., Khanna, N., Chaudhury, S., Joshi, S.D.: Text
extraction and document image segmentation using matched wave-
lets and MRF model. IEEE Trans. Image Process. 16(8), 2117
2128 (2007)
6. Liu, X., Samarabandu, J.: Multiscale edge-based text extrac-
tion from complex images. IEEE Int. Conf. Multimedia Expo.,
pp. 17211724 (2006)
7. Gllavata, J., Ewerth, R., Freisleben, B.: A Robust algorithm for
text detection in images. In: Proceedings of 3rd international
symposium on image and signal processing and analysis, vol. 2,
pp. 611616 (2003)
8. Li, H., Doermann, D., Kia, O.: Automatic text detection and
tracking in digital video. IEEE Trans. Image Process. 9(1), 147
156 (2000)
9. Lin, L., Tan, C.L.: Text extraction from name cards using neural
network. In: IJCNN 05. Proceedings of IEEE international joint
conference neural networks, vol. 3, pp. 18181823 (2005)
10. Zhang, D., Chang, S.-F.: Accurate Overlay Text Extraction for
Digital Video Analysis. Int. Conf. on Information Technology:
Research and Education, ITRE2003, pp. 233237
11. Jeong, K.-Y., Jung, K., Kim, E.Y., Kim, H.J.: Neural network-based
text location for news video indexing. IEEE Proc ICIP 3, 319
323 (1999)
12. Pan, Y.-F., Hou, X., Liu, C.-L.: Text localization in natural scene
images based on conditional random field (ICDAR 09), pp. 610
(2009)
13. Phan, T.Q., Shivakumara, P., Tan, C.L.: A Laplacian method for
video text detection (ICDAR 09), pp. 6670 (2009)
14. Shi, Z., Setlur, S., Govindaraju, V.: Text extraction from gray
scale historical document images using adaptive local connectivity
map. (ICDAR 05) 2, 794798 (2005)
15. Gopalan, C., Manjula, D.: Contourlet based approach for text iden-
tification and extraction from heterogeneous textual images. Int. J.
Comput. Sci. Eng. 2(4), 202211 (2008)
16. Do, M.N., Vetterli, M.: The contourlet transform: an efficient direc-
tional multiresolution image representation. IEEE Trans. Image
Proc. 14(12), 20912106 (2005)
17. da Cunha, A.L., Zhou, J., Do, M.N.: The Non sub sampled Con-
tourlet Transform: theory, design and applications. IEEE Trans.
Image Process. 15(10), 30893101 (2006)
18. Chen, N., Blostein, D.: A survey of document image classification:
problem statement, classifier architecture and performance evalu-
ation. Int. J. Document Anal. Recogn. 10, 116 (2007)
19. Liang, J., Doermann, D., Huiping, L.: Camera-based analysis of
text and documents: a survey. Int. J. Document Anal. Recogn.
7, 84104 (2005)
20. Galloway, M.M.: Texture analysis using gray level run
lengths. Comput. Graph. Image Process 4, 172179 (1975)
21. Haralick, R., Shanmugam, K., Dinstein, I.: Textual features for
image classification. IEEE Trans. Syst. Man Cybern. SMC-3(6),
610621 (1973)
22. Gnitecki, J., Moussavi, Z.: Classification of lung sounds during
bronchial provocation using waveform fractal dimension. In: Pro-
ceeding 26th conference IEEE engineering in medicine and biology
society (EMBS), pp. 38443847 (2001)
23. Molina, L.C., Belanche, L., Nebot, A.: Feature selection algo-
rithms: a survey and experimental evaluation data mining. In:
ICDM 2002. Proceedings of IEEE international conference,
pp. 306313 (2002)
24. Guyon, I., Elisseeff, A.: An introduction to variable and feature
selection. J. Mach. Learn. Res. 3, 11571182 (2003)
25. Gupta, S.C., Kapoor, V.K.: Fundamentals of mathematical statis-
tics, chap. 2. Sultan Chand and Sons, New Delhi, pp. 2.432.45
(1970)
26. Otsu, N.: A threshold selection method from grey-level histo-
grams. IEEE Trans. Syst. Man Cybern. SMC-1, 6266 (1979)
SIViP (2011) 5:165183
27. Witten, I.H., Frank, E.: Data miningpractical machine learn-
ing tools and techniques with JAVA implementations, Morgan
Kaufmann, Menlo Park (2000)
28. John, G.H., Langley, P.: Estimating continuous distributions in
Bayesian classifiers. In: Eleventh conference on uncertainty in
artificial intelligence, pp. 338345 (1995)
183
29. Aha, D.W., Kibler, D., Albert, M.K.: Instance-based learning
algorithms. Mach. Learn. 6(1), 3766 (1991)
30. Quinlan, R.: C4.5: Programs for machine learning. Morgan
Kaufmann, Menlo Park (1993)
123