Professional Documents
Culture Documents
Abstract—Images have become a standard for information into multi-dimensional feature vectors for each image. These
consumption and storage, far replacing text in various domains feature vectors for all the images in the database is referred
such as museums, news stations, medicine and remote sensing. to as a feature database. In order to retrieve relevant images,
Such images constitute of the majority of data being consumed
on the Internet today and the volume is constantly increasing day user would provide with an example image(s) or sketched
by day. Most of these images are unlabeled and devoid of any figure(s). The CBIR system would transform these inputs
keywords. The swift and continuous increase in the use of images into its representation of feature vectors. The distances or
and their unlabeled characteristics have demanded the need for similarities between the feature vectors of the example query
efficient and accurate content-based image retrieval systems. A and those of the images in the database are then computed
considerable number of such systems have been designed for
the task that derive features from a query image and show the and based on it the retrieval is performed. The results of the
most similar images. One such efficient and accurate system is search can then be sorted based on their distance or similarity
attempted in this paper which makes use of color and texture measure to the query example. In literature, many measures
information of the images and retrieves the best possible results of similarity models (or image distance) have been developed.
based on this information. The proposed method makes use of The paper mainly utilizes two most commonly used features
Color Coherence Vector (CCV) for color feature extraction and
Gabor Filters for texture features. The results were found to be for CBIR system, texture and color [2], [3]. Since image size
significantly higher and easily exceeded a few popular studies as or its orientation has no impact on the color composition of the
well. image, color is amongst the most widely employed features
Index Terms—Color coherence vector, Image retrieval, Gabor for image classification. Swain et. al. in [4] and Viet [5]
Filter, Legendre moment proposed use of color histogram method for representing color
feature. However, color histogram methods do not include
I. I NTRODUCTION spatial information and hence are not avert to significant spatial
Content Based Image Retrieval (CBIR), also referred to changes [6]. Pass et al. [7] classified each image pixel as either
as Content-Based Visual Information retrieval (CBVIR) and coherent or not, based on the similarity in color between pixel
Query By Image Content (QBIC), applies the techniques of and its neighbors. He represented this classification of color by
computer vision to the problem of searching relevant digital a split histogram called Color Coherence Vector (CCV). It has
images in large datasets, also known as the image-retrieval proved to be most effective for color feature extraction. While
problem. ‘Content-Based’ implies that the search instead of color extraction is simple and fast in implementation, it alone
analyzing metadata (e.g. tags, keywords, or any other descrip- cannot capture the complete information about the image.
tion associated with the image), analyzes the contents of the Different methods of texture representations have been studied
image. The term content suggests any kind of information that in the fields of pattern recognition and computer vision.
can be directly derived from the image, such as colors, shapes, Gabor and wavelet transform amongst other multi-resolution
textures etc. Even though it is possible to annotate images filtering techniques, characterizes texture by the statistical
manually by providing metadata for each image in a database, distribution of the image intensity. Manjunath and Ma [8]
such a process will be time consuming (specially for large demonstrated that multi-filtering approach using a Gabor filter
datasets) and it is also possible that the metadata does not bank yields most accurate results. Gabor wavelets inherently
represent the image properly. Because of the limitations of provide multi-resolution of both spatial frequency and orien-
such systems as mentioned above, as well as the large range tation and they can be conveniently designed for efficiently
of possible uses for efficient image retrieval, the interest in extracting regions of varying textures from an image. It has
CBIR has grown in the recent times. also been proven that image analysis with Gabor filters is in
CBIR method [1] is centered around extraction of features on synchronization with the human visual system which provides
basis of which different images can be compared in the dataset superior analytical resolutions in comparison to other methods
and relevant images can be retrieved. A typical CBIR system of texture extraction.
extracts the visual contents of the images in the database In this paper, authors propose a CBIR system based on an
978-1-5386-6678-4/18/$31.00 2018
c IEEE 290
efficient combination of CCV for color feature extraction and by most techniques. In the next stage of algorithm, Color Co-
Gabor Filters for texture feature extraction. In Section II the herence Vectors are used for color feature extraction of image
proposed technique is discussed in brief. Section III presents dataset. CCV technique provides a more accurate analysis of
the detailed description of feature extraction process. Different the color space of an image as opposed to the techniques that
subsections include texture feature vector extraction in III-A, were used previously in the study. The idea of using pixels
Image segmentation in III-B, color moments extraction using as part of connected components not only segments pixels on
Legendre polynomial in III-C and color coherence vectors basis of similar color in a discrete color space but also does
in III-D. Simulation results are given in section IV. Lastly this in a computationally efficient manner.
section V includes concluding remarks based on the results After computing the entire feature database, the distances
obtained from simulations. of the color and texture features of the query image are
calculated from the database using the ‘Manhattan’ similarity
II. P ROPOSED T ECHNIQUES measure. The distances obtained are combined on application
Huang [9] proposed a CBIR technique involving combined of suitable weights. Finally, the top-k images are retrieved
features of color and texture. Moments (mean, variance and after sorting. However, Color and texture features may not be
skewness) of Hue, Saturation and Value (HSV) components of equally important for the database under consideration. When
image are used as its color features. Gabor texture descriptors distances are computed of the database from the query image,
are utilized as texture features. In the work proposed by [10], different weights must be assigned to the respective color and
image retrieval system is designed using a combined feature texture distances. These weights may vary from 0.1 to 0.9 and
set including color auto-correlogram feature, Gabor texture sum of the weights should be 1. For achieving the optimal
feature for color and texture respectively. Wavelet transform is results, a grid search technique is used where a linear scan on
added in feature set for representing shape to improve results. the weights is performed, keeping the sum to be 1. The best
Extraction of texture features using Gabor filters includes weights chosen are thereby used for further computation.
appropriate design of a filter bank tuned to several orientations
III. F EATURE E XTRACTION
and spatial-frequencies to comprise the spatial-frequency space
[11]. Each wavelet of the filter bank can be tuned to a In case of CBIR, as the name suggests, the features of the
specific frequency and orientation. This multi-channel filtering image are to be retrieved from the content itself. In this paper,
approach decomposes the image into a number of filtered only color and texture features of the images are considered.
images and extraction of features is done from the filtered A. Gabor Filters for Texture Feature space
images. In this paper, the mean (μ) and standard deviation (σ)
of filtered images are computed which have been subsequently Gabor filters are linear filters used for texture analysis. They
used for obtaining the texture feature vectors. can analyse presence of specific frequencies in an image in
A lot of different algorithms were tried to analyze color space specified directions for a localized region around the region of
in this paper. In initial stage of proposed technique, higher interest. Fundamentally, Gabor filters are a group of wavelets.
order Legendre moments have been computed to generate the Each wavelet captures energy at a specific frequency and at
color feature vector for the image. In image processing, com- a specific direction [13] . Expanding a signal using this basis
puter vision and related fields, image moment [12] are widely provides a localized frequency description. The energy distri-
chosen to have some attractive property or interpretation of butions obtained can be used to extract the texture features.
images pixel intensities. Further, color information is gathered Furthermore, Gabor filters provide optimal resolution in both
from the image (HSV color space) and stored in form of color time and frequency domains as compared to Fourier transform.
feature vector, which constitute the feature database. Same In the spatial domain, mathematically, a 2D Gabor filter is
technique was repeated after separating the foreground objects a Gaussian kernel function modulated by a sinusoidal plane
from the background of the images. To achieve this, color wave.
based segmentation technique using the K-means clustering f2 f2 f2
ψ(x, y) = exp(−( 2 x2 + 2 y 2 )) exp(j2πx f )
algorithm and L ∗ a ∗ b color space was employed. Once the πγη γ η
segmentation of an image is complete, higher order Legendre
x = x cos θ + y sin θ , y = −x sin θ + y cos θ
moments are extracted from the region of interest to represent
color features. f and θ indicate the scale and orientation of the Gabor wavelet
An extensive study of already existing algorithms revealed that respectively, γ is the sharpness along the major axis, η is
positional attribute of colors were ignored while analyzing the sharpness along the minor axis. The multi-resolution√and
u
the color space. Sparse and aggregated color distributions multi-orientation analysis can be defined as: fu = fmax / 2
produced similar feature vectors because pixels having same and θv = vπ/V Where fmax is the maximum central
colors were simply aggregated without taking their relative frequency. Different central frequencies for different wavelets
positions into consideration. CCV is superior than other tech- should be near the characteristic texture frequency of the
niques such as color histograms in a way that they prevent co- corresponding region. Total six orientations (M ) and four
relation between coherent pixels in one image from incoherent scale (N ) have been chosen to generate a total of 24 filters
pixels in another, thus allowing fine distinctions not provided in the filter bank. The Gabor filters are applied on the image
B. Image Segmentation
For image segment based classification, the image is seg- image, the first order and second order moments are computed
mented into many homogeneous areas. Image features are then to generate the color feature vector.
extracted based on the specific requirements. To achieve sep- N N
1 1
aration of the foreground objects from background, the color Er,i = Iij , σr,i = (Iij − Er,i )2 (3)
based segmentation technique using the K-means clustering N j=1 N j=1
algorithm and L ∗ a ∗ b color space was employed. The K-
means clustering algorithm in [14] is an unsupervised machine The number of features for each image will be 6 (mean,
learning technique which is generally used for unlabeled data. variance for each of the 3 channels). When these color features
The aim of this algorithm is to find some structure or pattern and gabor texture features were combined to form the feature
in the data by dividing the data into K distinct groups such database for image retrieval, precision was quite low. This
that the observations within a cluster are similar. Each cluster led to the conclusion that computing only mean and variance
can be represented by its own centroid. The L ∗ a ∗ b color for color features may not be enough. We need to capture
space describes all perceivable colors in three dimensions: L higher order color information from the image also. The
for lightness and a and b for the color components green-red computation of color moment by mean and variance can be
and blue-yellow. All of the color information is stored in a and extended by calculating Legendre moments that may contain
b. Following steps are carried out in separating the foreground higher order information. There are many advantages of using
homogeneous region from the background: legendre moments over basic moments (mean, variance, etc.)
as [15] : values of these features are invariant to geometric
1) Read the image and convert to L ∗ a ∗ b color space.
transformations, these are also helpful to identify objects with
This helps in better quantifying the visual differences.
unique shapes, etc. There are defined Legendre polynomials
The color information is contained in a ∗ b color space.
which are used to determine Legendre Moments. The fol-
2) K-means algorithm is used to segment the objects
lowing recurrence relation can be used to represent Legendre
into 2 clusters: foreground and background, using the
polynomials of order k :
Euclidean distance measure. For every object, it returns
an index corresponding to the cluster. Each pixel was (2k − 1)mLPk−1 (m) − (k − 1)Pk−2 (m)
LPk (m) = (4)
labeled by this index. k
3) The indices obtained were reshaped to form an image Here LP0 (m) = 1, LP1 (m) = m and k > 1. Similarly,
consisting of only black and white pixels, corresponding polynomial LPl (n) can also be defined. Using the Legendre
to the original image polynomials LPk (m) , LPl (n) and image intensity function
4) Pixel corresponding to the index value at the top left I(m, n), the two-dimensional Legendre moments of order (k+
corner of the image was considered as background and l), are defined in [16]. Normalizing the equation in discrete
its value set to 0. This separated background from the form results in Legendre moments as:
foreground.
N
−1 N
−1
Once the segmentation of an image is complete, Legendre
LMkl = λkl LPk (mi )LPl (ni )I(i, j) (5)
moments are used on the segmented image (region of interest)
i=0 j=0
to extract the higher order color features.
Here λkl is defined as the normalizing constant and is given
C. Moments for Color Feature Space by: λkl = (2k + 1)(2l + 1)/N 2 . In equation (5), mi and nj
Color moments are most commonly used to extract and are in the range of [−1, 1], and known as the normalized pixel
represent the color feature of an image. For each channel of the coordinates: mi = N2i−1 − 1, nj = N2j −1 − 1
CP = ((α1 , β1 ), (α2 , β2 ), . . . , (αn , βn )) and compared with all the vectors in the feature database and
are checked for similarity using a distance factor. Images
CP = ((α1 , β1 ), (α2 , β2 ), . . . , (αn , βn )) (6) are then sorted in increasing order of distance and top ’k’
Since the total number of coherent and incoherent pixels images are retrieved. In this paper, Manhattan distance
for a bucket j can be same for images P and P , that is, metric [19] is used for computing similarity distance.
αj +βj = αj +βj . But these pixels may entirely be coherent in This section also presents the simulation results to analyze the
P and entirely incoherent in P . So applying a naive absolute performance of the proposed work. Gabor Filter is used for
TABLE I
P RECISION OF PROPOSED CBIR SYSTEM USING CCV AND G ABOR FOR
Nr (T OTAL RETRIEVED IMAGES )=10 AND 20.
3 MONUMENTS 8 11
4 BUSES 10 20
5 DINOSAURS 10 20
6 ELEPHANTS 10 18
7 FLOWERS 10 17
8 HORSES 9 15
9 MOUNTAINS 8 12
10 FOOD 8 15
MEAN = 8.9 MEAN = 15.4