CHAPTER-3

Image Retrieval using Color, Texture and Shape

Features
3.1 Introduction
While designing image retrieval systems, one of the key steps is the choice of feature
representations. Color and texture exploration has been attracting a great exchange of
interest due to its potential value in pattern recognition and computer vision. The
exercise of combing various combinations of local features to build feature vector has
given successful results in image retrieval. Texture feature is based on the local
intensity of an image. Therefore, neighborhood and statistical features are exposed to
texture pattern. Local patterns are generally used to find the relative information for
each pixel in the image from its neighborhood.

The color feature represents the division of intensity at various color channels.
Therefore, the techniques color correlogram, color histogram, and color coherence
vector are used to intend as color feature descriptors. Shape features can be extracted
by Generic Fourier Descriptor (GFD) and Moments.

In this chapter, all low level features are combined for efficient image retrieval has
been discussed. The existing texture feature descriptor such as local maximum edge
binary pattern (LMEBP), HSV color model and LZM for shape features are integrated
to build a feature vector for efficient image retrieval.

3.1.1. Related Work

Ojala et al.[37] proposed a rotation invariant texture feature descriptor called local
binary pattern (LBP) in which each pixel is encoded with a binary value to extract the
texture features of an image. The neighbor pixels Pi of a center pixel Pc are assigned
with a binary value depending on the intensity of the center pixel as shown in Fig 3.1.
The neighbor pixels are angular, uniformly distributed over a radius of a circle to a
center pixel. The compatation procedure of LBP to a center pixel is given in
Eqns.(3.1) & (3.2).
 Figure 3.1: Local Binary Pattern computational structure

∑ (3.1)

{ (3.2)

Where Pc represents the center pixel and Pi represents the ith neighboring pixel,
i=1,2…8. In our operation R=1, n=8 and ‘d’ is the difference between the neighbor
pixel and center pixel. In the same way, all pixels in the image are considered as
center pixel once and a binary pattern generated for every pixel. Finally, a histogram
of pattern map has been constructed to build a feature vector. The rotational structure
of LBP made it dominant in rotational invariant applications like object tracking,
recognition of facial expressions, face recognition and texture classification.

Heikkilä et al.[42] proposed a modified version of a local binary pattern called

center-symmetry local binary pattern (CS_LBP). It is also considering each pixel in
an image as center pixel but the local differences are calculated only for center
symmetric pixels which are independent of the center pixel. A binary label is assigned
for each local difference based on its sign i.e., ‘0' for negative and ‘1' for the positive
difference. Therefore a four-bit binary pattern generated for every center pixel, later
weights have been assigned to this pattern to summed up to get a decimal value.
CS_LBP explained mathematically in the following Eqns.(3.3) & (3.4).

⁄
∑ ⁄ (3.3)

{ (3.4)
Where Pi and Pi+(N/2) are respective gray values of center-symmetric pairs of pixels of
N equally distributed pixels on a circle of radius R. The value of threshold T is 1% of
the pixel intensity range of an image. Later, build histogram to construct the feature
vector.

3.2. Proposed Optimized Image Retrieval Method.

To improve the retrieval rate and accuracy with state of art techniques, all the local
features (color, texture, and shape) are optimized to construct the feature vector due to
their richness in the content information. The proposed optimized IR system
framework is shown in Fig.3.2. All the color images including the query image are
converted into HSV color space. The Hue and Value components are quantized to
appropriate levels to extract the maximum color information from the image. Here,
the Hue and Value are quantized to 18/36 and 10/20 levels respectively and
corresponding histograms are constructed. The quantization level 18 makes the Hue
divided into 20 parts where each bin in the H- histogram represents 200 ranges of
colors and quantization level 36 makes Hue divided into 10 parts where each bin in
the H-histogram represents 100 ranges of colors. More no.of bins in the histogram
provides annoying discrimination in between the relative intensity and brightness of
pixels. In this thesis maximum no.of bins used for H-histogram is 18 and S-histogram
is 10.

To extract the texture features, LMEBP is applied on the Value component of HSV
color space to extract the texture features and build the histogram for it individually.
The shape features extracted from the grayscale image, which is a converted version
of the original RGB image. The LZM (Lower order Zernike Moment) technique is
used to extract the shape features for the order n=5 and constructed a vector for each
image. All the individual histograms on H,S,V histograms and LZM vector are
combined to construct the feature vector and similarity metric is applied to retrieve
the similar images.
 Figure 3.2: Proposed system flowchart
3.2.1. Color Space
Generally, images are three types in nature: Binary, Grayscale and Color. The binary
image, which consists only two pixel intensities of white and black. The gray scale
image, which contains a particular range of intensities in only one band. The last color
image, which has three bands: Red, Green and Blue and each band has a particular
range of intensities. The natural color images are known as RGB images, which
contain three color bands i.e., red, green and blue. These three color bands can give
only color information, not of color intensity information. Therefore, RGB color
space not suitable for computer vision applications. To overcome the above problem
HSV (Hue, Saturation and Value) color space is used which gives the color,
brightness and intensity information respectively. Hue gives color information in
terms of an angle from 00 to 3600 , and each degree represents a unique color.
Saturation ranges 0 to 1, and these values change from low to high when the intensity
of color increases. Value also ranges 0 to 1. The distributions of pixel intensities for
RGB and HSV color model shown in Fig.3.3. HSV color space provides the wide
information about color ranges and their intensities.

In the proposed method appropriate quantization levels (i.e.,18&10) has been taken
on Hue and Saturation components for better results. More quantization levels make
undesirable discriminations in between pixel intensities and brightness.
 Figure 3.3: Distribution of pixel intensities in RGB & HSV color spaces

3.2.2 Local Maximum Edge Binary Pattern (LMEBP)

LBP finds the relation between the center pixel and surrounding eight neighborhood
pixels by comparing intensity values [37]. In the continuation of LBP,
M.Subramanyam et al.[61] designed a new approach for texture feature extraction
entitled local maximum edge binary pattern (LMEBP). It captures edge information
for each pixel with the help of eight surrounding pixels.

For a center pixel and corresponding eight neighbor pixels , LMEBP

calculation procedure explained in the following steps:

Step1: Calculate the local difference di between neighbor pixel Pi and center pixel
Pc for an image Im..

(3.5)

Step2: Sort the magnitudes of differences irrespective of the sign.

| || | | | (3.6)

Where computes the maximum value in the array I. Sort is the function where
it sorts the array in descending order irrespective of the magnitude of I.

Step3: Assign the binary label to each local difference according to its sign.

(3.7)

Where f(x) is defined in the edge is positive assign 1 for this center pixel otherwise 0.

{
Step4: Repeat steps 1to 3 for all the pixels in 3x3 pattern and computes the edges.

LMEBP defined as

( ) { } (3.8)

Step5: Construct the histogram to build feature vector.

After the calculations of LMEBP, the entire image is represented by constructing a

histogram based on

∑ ∑ (3.9)

Here, the image size is mxn.

Steps involved in LMEBP Algorithm:

1. Assume a 5x5 matrix of a given image as shown in Fig 3.4(a).
2. Take a 3x3 window from the above-said matrix, which should have 8
neighbors for a center pixel.
3. Compute the local differences like LBP (local binary pattern) using Eqn. (3.1)
& shown in Fig. 3.4(b).
4. Sort the magnitudes of differences in descending order.
5. Assign '0' &'1' to the differences according to sign as shown in Fig.3.4(c).
6. Shift the center pixel, such that each pixel in the pattern becomes a center
pixel at once.
7. Repeat the steps 3 to 6 until 9 patterns gets generated.
8. Calculate the eight edges using all binary patterns as shown in Fig 3.4(d).
 Figure 3.4: Execution procedure of LMEBP algorithm.

The simulation results of LMEBP on a face image have been shown in Fig.3.5, and it
has been observed that the edge information about the object from a first edge to the
eighth edge has been decreased.
 Figure 3.5: Simulation results of LMEBP on a face image.

3.2.3. Similarity metrics for Query matching.

After calculating the feature vector for all database images, including the query
image, similarity measurement plays an important role in retrieving similar images
from the database.
There are various metrics used to find the distance between two vectors, three
significant metrics given from Eqns. (3.10)-(3.12).

∑ | | (3.10)
 ∑ | | (3.11)

∑ | | (3.12)

Where q is the query image, b is the database image. The d1 distance is widely used,
based on its performance [62,63,66] and it is confirmed form the Fig.3.9.

3.3. Simulation Results and Comparisons.

Image retrieval techniques require different image databases to do the experiments

and to validate the results. There exist enormous databases based on the requirement,
in that some are Corel database, MIT-Vistex, Brodtz, S-Tex etc. Corel database is
well-known and often used to verify the retrieval results and it is available in three
different sizes i.e., 1K,5K and 10K. MIT-VisTex database consists color, texture
images and Brodtz database consists of texture images. To compare the proposed
method results to the existing methods, two renowned metrics called precision and
recall are used. In all experiments, each and every image from the database has taken
as a query image for retrieval performance analysis. The computational equations for
precision, recall, average retrieval rate (ARR) and average retrieval precision (ARP)
has given in Chapter1 and Table 3.1 gives ARP and ARR results for Corel-10K and
MIT-Vistex for various algorithms. To measure the competence of the proposed
method, experiments have been done on two databases, where the first one is Corel-
10K and second one MIT-VisTex Color database.

Table 3.1. Results for Corel-10K and MIT-Vistex in precision (for n=10) and recall
(for n=100)

Method Corel-10K MIT-Vistex

ARP ARR ARP ARR
Wavelet+Color [32] 41.5 17.4 87 93
CS-LBP+Color [42] 45 19.1 91 94
LEP+Color [52] 46.4 20.4 92.3 94.6
LEPINV+Color[86] 45.5 19.4 91.1 93.9
LEPSEG+Color[86] 48.2 21.4 90.2 94.89
PM 50.1 22.8 95.89 97.8
3.3.1 Experimental results on Corel-10K.

The Corel-10k database [111] comprises of 10,000 images of 100 types; each type has
100 images as it is larger and adaptable than Corel-1K and 5K. The images of wild
animals, e.g, lion, polar bear, tiger, deer, etc., army, nature sceneries, ships, food,
buses, etc., army, ocean, cats, airplanes etc. A few sample images from Corel-10K are
shown in Fig.3.6. The retrieval performance of the proposed work on Corel-10K is
calculated in terms of precision, recall, ARP, ARR, and F-measure.

The performance of the proposed method compared with modern techniques,

Such as Wavelet, CS_LBP, LEP, LEPINV, and LEPSEG methods along with color
histograms for each method. The top 20 & 30 retrieved results for two different query
images are shown in Fig.3.7 (a)-(d) and Fig 3.8(a)-(b) represents the category wise
retrieval performance in terms of precision and recall curves for each query image
from each category. Fig.3.8 (c)&(d) characterize average rate precision (ARP) and
average rate retrieval (ARR). Owing to the shape features in the feature vector, the
precision of shape oriented image categories has shown a significant improvement
example categories 17,18,37,42,54 etc. It is clearly showing a considerable
improvement in the average precision around 5.74% as compared to Wavelet+Color,
2.94% as compared with CS_LBP+Color, 1.67% as compared with LEP+Color,
4.71% as compared with LEPINV+Color and 5.46% as compared with the LEPSEG+
color histogram methods. Fig.3.6(e) is showing the relation between top matched
images to F-measure where the computational Eqn.(1.6) has been given in Chapter 1
and comparative results in Table.3.1. The results demonstrated that the proposed
method outperforms than other existing methods.

Figure 3.6: Sample images from Corel-10K

Query Image

(a)
Query Image

(b)
 Query Image

(c)

Query Image

(d)
Figure 3.7: The top 20 & 30 retrieved results of the proposed method for the given
query image in (a) & (b) on the Corel-10K database.
 Wavelet+Color CS-LBP+Color LEP+Color
LEPINV+Color LEPSEG+Color PM
100

80

60
Precision

40

20

0
0 10 20 30 40 50 60 70 80 90 100
No.of Images Category

(a)

Wavelet+Color CS-LBP+Color LEP+Color

LEPINV+Color LEPSEG+Color PM
100

80

60
Recall

40

20

0
0 10 20 30 40 50 60 70 80 90 100
No.of Images Category

(b)
 Wavelet+Color CS-LBP+Color LEP+Color
LEPINV+Color LEPSEG+Color PM
60

50

40
ARP

30

20

10

0
10 20 30 40 50 60 70 80 90 100
No.of Images Retrieved

(c)

Wavelet+Color CS-LBP+Color LEP+Color

LEPINV+Color LEPSEG+Color PM
25

20

15
ARR

10

0
10 20 30 40 50 60 70 80 90 100
No.of images Retrieved

(d)
 Wavelet+Color CS-LBP+Color LEP+Color
LEPINV+Color LEPSEG+Color PM

0.25

0.2

0.15
F-MEASURE

0.1

0.05

0
10 20 30 40 50 60 70 80 90 100
NO.OF TOP MATCHES

(e)
Figure 3.8: Corel-10k Database:(a) Precision and image category number (b)Recall
and image category (c) ARR and images retrieved (d) ARP and images retrieved (e)F-
measure vs top matched images.

d1 Canberra Euclidian Manhattan

55
50
45
40
35
30
ARP

25
20
15
10
5
0
10 20 30 40 50 60 70 80 90 100
No.of Top Matches

Figure 3.9: Graphic representation of the performance of proposed method with

different distance measures in terms of ARP on Corel-10K database
3.3.2 Experimental results on MIT-VisTex.

MIT- Vistex database consists of a lot of color, texture images [112]. This database
consists of 40 different color, texture images and each of size 512x512. For image
retrieval, these images are divided into 16 blocks, where each block size is 128x128.
Therefore 640 (40x16) images database has been created. Some sample images from
this database shown in Fig.3.10(a) and Fig.3.10(b) shows the 16 blocks of an image.
The top 20 retrieved results for a query image shown in Fig.3.11 and it is observed
that all the 20 images are related in content wise to the query image.

From Fig.3.12(a)&(b) the retrieval performance of the proposed method compared

with the state-of-the-art approaches. It has been demonstrated that the proposed
descriptor has shown weighty improvement in terms of precision around 7.8% as
compared with Wavelet+Color, 6.2% as compared with CS_LBP+Color, 4.3% as
compared with LEPINV+Color, 3.3% as compared with LEPSEG+Color, and 2.7%
as compared with LEP + Color. The Fig 3.12(c) is representing the top matched
images vs respective precision and recall relations, which is called F-measure.

(a)
 (b)
Figure 3.10: (a) Samples from MIT-Vistex Database (b) 16 divided blocks of an
image

Query Image

Figure 3.11: Top 20 retrieved results for a query image as shown on MIT-Vistex
database.
 Wavelet+Color CS-LBP+Color LEP+Color
LEPINV+Color LEPSEG+Color PM
100
90
80
70
60
ARP

50
40
30
20
10
0
10 20 30 40 50 60 70 80 90 100
No.of images Retrieved

(a)

Wavelet+Color CS-LBP+Color LEP+Color

LEPINV+Color LEPSEG+Color PM
100
95
90
85
80
ARR

75
70
65
60
55
50
0 10 20 30 40 50 60 70 80 90 100
No.of Images Retireved

(b)
 Wavelet+Color CS-LBP+Color LEP+Color
LEPINV+Color LEPSEG+Color PM

0.9

0.8

0.7
F-Measure

0.6

0.5

0.4

0.3

0.2

0.1

0
0 2 4 6 8 10 12 14 16
No.of Top matches

(c)
Figure 3.12: MIT-Vistex Database: (a) ARP and images retrieved (b) ARR and
images retrieved (c)F-measure vs Top matched images.

Table 3.2. Feature vector length for a given query image and respective feature
extraction and retrieval times using several methods.

Feature Image
Method Feature vector length Extraction Retrieval Time
Time(s) (s)
Wavelet+Color [32] 24+192=216 0.081 0.211
CS-LBP+Color [42] 16+24=40 0.012 0.038
LEP+Color [52] 16x8x8x8=8192 0.169 2.519
LEPINV+Color [86] 72+24=96 0.070 0.042
LEPSEG+Color [86] 512+24=536 0.243 0.173
LMEBP +Color [61] 8X511=4088 0.638 0.199
PM(H18-S10-V) 18+10+8x511+21=4137 0.641 0.213
PM(H18-S20-V) 18+20+8x511+21=4147 0.645 0.214
PM(H36-S10-V) 36+10+8x511+21=4155 0.651 0.220
PM(H36-S20-V) 36+20+8x511+21=4165 0.652 0.222
 3.4 Conclusion.
An optimized image retrieval technique has been proposed for better precision and
recall than the existing algorithms. All the local features are extracted to construct the
feature vector. The color information has taken from Hue and Saturation components
of the HSV color model. The performance with different quantization levels has been
tested on different databases, e.g., for Corel-10K database, Hue 18 and Saturation 10
has been observed with good results. The Value space is utilized to extract texture
information using LMEBP algorithm and shape features extracted from proposed
lower order ZM. All these features are concatenated to construct a feature vector and
the results are extracted. The texture feature descriptor LMEBP is limited to find the
relative information of a center pixel & its eight neighbors in every pattern. The
performance of LMEBP can be improved further by adding the more directional
pixels to the algorithm in calculating the edges.

A novel hybrid local quantized pattern (LQP) based Local quantized edge binary
pattern (LQEBP) algorithm for texture feature extraction has been proposed. LQEBP
computes the total 16 edges for each and every pixel, which clears the maximum
extraction of relative information from its neighbors. The proposed algorithm applied
to Value space of HSV color model and color information also added from Hue and
Saturation components for efficient retrieval. It has been demonstrated that the
proposed algorithm shows an impressive Precision, Recall, ARP & ARR over
Wavelet+Color, CS-LBP+Color, LEPINV+Color, LEPSEG+Color, and
LECoP+Color.