JOURNAL OF COMPUTING, VOLUME 2, ISSUE 12, DECEMBER 2010, ISSN 2151-9617

HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 51

Image Retrieval from Web

Mrs. Tripti Bhatiyani

Abstract—In past recent years there has been much growth in the field of digital imaging technologies and hence
popularity of digital images is increasing very fast. But the task of obtaining images from the Internet which meet user’s
requirement is not a trivial task. Images were initially retrieved on the basis of textual information supplied through various
sources e.g. URL, filename, surrounding text, alt text etc. But all the images on the web are not annotated properly and still
results obtained on this basis were not always satisfactory. Imprecise annotation, perception subjectivity and requirement of
much time and labor in annotation paved the way for content based image retrieval.

Index Terms—Image retrieval, shape, texture, color, edge, low level features

——————————  ——————————

analyzing the actual contents of image rather than the

1 INTRODUCTION
World Wide Web is the medium wherefrom information in
large scale can be obtained very easily and at very low cost
too. In past few years the growth of World Wide Web has
been phenomenon. And because of this growth the
inclination towards retrieving images from Web has also
increased. By using digital imaging devices various kinds of
digital images from the real world can be obtained easily.
With the growing popularity of digital images on the Web,
it has become important to focus on research on
technologies for image retrieval on Web. Images can be
retrieved either on the basis of text or content. In text based
search URL’s, page titles, alt text, hyperlinks and
surrounding text etc are used to search images. While in
content based search, images are retrieved by matching low
level features e.g. color, texture, edge, shape, shadow and
temporal details.
Image Retrieval is the mechanism of browsing, searching
and retrieving images from image databases. The most
common methods of image retrieval are based on adding
captions, keywords and descriptions to images. The image
retrieval is based mainly on the annotation words in these
methods. The task of annotating the images manually
requires lot of time and efforts. Hence a lot of research has
been done in the field of automatic image annotation. Due
to the problem with traditional methods of image indexing
there is keen interest in techniques for retrieving images on
the basis of automatically-derived features such as color,
texture i.e. Content-Based Image Retrieval (CBIR). MIT was
the first microcomputer based image database retrieval
system, developed during 1980’s by Bani Reddy Prasad,
Amar Gupta, Toong and Madnick.
2 CONTENT-BASED IMAGE RETRIEVAL
In content based image retrieval search is performed by
————————————————
 Mrs. Tripti Bhatiyani is with the Institute of Applied Sciences & Computer
Applications, ITM Universe, Gwalior.
metadata e.g. keywords, tags, descriptions attached with the
image. The content of images are low level features e.g.
colors, shapes, textures and all the other information
derived from the image itself. The origin and need of
content based image retrieval lies in the difficulties faces in
the area of text-based image retrieval. In text based search
first images are needed to be annotated by text and then a
text based database management system is used to perform
image retrieval. The annotation of each and every image is
not possible if the collection is very large as it requires vast
amount of labor. Images are also rich in content and still
there is more difficult problem i.e. different perception of
same image by different people. Imprecise annotations and
perception subjectivity paved way for content based image
retrieval.
Content based image retrieval is based mainly on three
fundamental tasks, i.e. Feature Extraction, Indexing and
Design of Retrieval System.
3 FEATURE EXTRACTION
A feature is function of one or more measurement which is
the value of quantifiable property which is computed to find
some significant features of object. Feature extraction is the
primary task of CBIR. It is the preprocessing step to obtain
image features like color histogram, shapes and textures.
Features e.g. corner points or interest points are also being
used in image retrieval. Scale and affine invariant interest
points for affine transformations and illumination changes
are important features for image retrieval. Besides that
wavelet-based salient points can also be used. To improve
generalization and efficiency in classification and indexing
of large number of image features, feature subset can be
derived. After making the decision about visual features,
next step is retrieval of image. Color, texture and shape are
application independent features. These features can be
further divided into:
1. Pixel level feature which includes color and its
location.
JOURNAL OF COMPUTING, VOLUME 2, ISSUE 12, DECEMBER 2010, ISSN 2151-9617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG
2. Local features e.g. segmentation and edge detection.
3. Global features which are calculated over entire image
or regular sub area of image.
Histogram based descriptors, dominant color descriptors,
spatial color descriptors and texture descriptors are used for
browsing and image retrieval. Representation of shape
attribute in segmented image regions plays an important
role in image retrieval. Shape context is the shape descriptor
for matching shape, is very compact and robust to geometric
transformations [9]. For shape estimation in images reliable
segmentation is very important. The problem of
segmentation from the point of view of human perception is
difficult to solve but still some solutions are obtained in this
direction. In normalized cuts criteria cues of contours and
texture differences are used for segmentation [10]. Corner
points or intersect points are also being used extensively for
image retrieval. Scale and affine invariant interest points,
which can handle affine transformation and illumination
changes, are also important features for image retrieval.
Another classification is:
1. Low level feature
2. High level feature
Low level features are those features extracted directly from
images e.g. edges, texture, color, shape, corner. While high
level features extraction is based on low level features.
3.1 COLOR
This is the most widely used visual feature in Image
Retrieval. Images can be retrieved on the basis of color
similarity by computing color histogram for the image.
Color Histogram is the most common color feature
representation. The color histogram identifies the
proportion of pixels in an image holding certain values.
Color histograms denote the joint probability of three colors.
Color histogram may define number of pixels of each red
channel value in range of 0-255. Histograms lose
information related to spatial distribution of colors hence
two different images can have similar histograms. There is
need to access such spatial information. Correlograms and
anglograms are the two well known approaches to access
spatial information. Correlgrams finds the distribution of
colors of pixels in certain areas around pixel of certain color,
while anglograms finds signature of spatial arrangement of
areas having common properties like same colors.
Similarity is measured for the color by Histogram
Intersection, a L1 metric [1]. Similarities between similar but
non-identical colors can be measured by L2-related metric in
comparing the histograms [2].
In addition to Color Histograms Color Moments and Color
Sets are also used to retrieve images. Any color distribution
can be described by its moments. First moment (mean),
second moment (variance) and third moment (skewness) are
found out as most of the information is based on low order
moments [3]. Color sets can be used to search quickly from
large-scale image collections. (R, G, B) color space can be
transformed into uniform space and quantized into M bins.
A color set is a selection of colors from the quantized color
space [4].
52
3.2 TEXTURE
Texture is look of visual pattern in images and their spatial
arrangement. It is the natural property of all surfaces which
tells about the structural composition of surface. Visual
patterns are innate properties of all surfaces which possess
information about surfaces and their relationship to
environment [5]. A co-occurrence matrix is constructed
which is based on the orientation and distance between
image pixels. Textures can be divided into various sub
categories e.g. surface texture, image texture, deterministic
texture and statistical textures.
Texture representation can also by given as computational
approximations of the visual texture properties e.g. contrast,
regularity, roughness, coarseness, directionality, line
likeness etc [6]. Besides that texture representations were
given by mean and variance extracted from the Wavelet sub
bands. This approach achieved 90% accuracy on 112 Brodatz
texture images. Tree structured wavelet transform was used
to explore middle band features in order to improve the
classification accuracy.
3.3 SHAPE
Shapes of images and of specific region within them are
identified by applying segmentation or edge detection to an
image. When edges are detected and meaningful
information are extracted from images, outer shape can be
described using some statistical expression. Shape
description can be either geometric or topological.
Geometric descriptors are area, length, perimeter,
eccentricity, principle axis of inertia, compactness, moments
of inertia etc. Topological descriptors are connectivity i.e.
neighboring feature adjoining the region and Euler number.
Shapes are retrieved on the basis of features e.g. lines,
boundaries, aspect ratio and circularity etc. areas of change
or stability are identified by region growing and edge
detection. Boundary-based shape representation uses outer
boundary of the shape while region-based representation
use entire shape region. Fourier Descriptor and Moment
Invariants are examples of such boundary-based and
region-based shape representation respectively.
4 IMAGE INDEXING
Image indexing is the process where feature vectors in s are
sorted so that to process a query there is no need to
sequentially scan database. Images in the database go
through feature extraction process. Each image is
represented as k-vector, dimensional feature where k is the
no of features of the image. If s is the set of feature vectors
and Ek is k dimensional feature space then S Ek .
Images are 2-D array of pixels, where each pixel is
represented with fixed precision e.g. 8 bits or 16 bits per
pixel. High level abstractions of images are formed from
raw images in many ways, one of which is making
histograms on the basis of color distributions and object
contours. This is known as feature extraction. Images are
presented as K-dimensional feature vector, where k is
JOURNAL OF COMPUTING, VOLUME 2, ISSUE 12, DECEMBER 2010, ISSN 2151-9617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG
number of features representing an image. If S be the set of
feature vectors representing the images in database and Ek
are K-dimensional feature spaces then S Ek [7]. Image
indexing aims to sort the feature vectors in S to avoid
sequential scan through entire database while executing the
query. The choice of Ek and discrete matrix affects the
performance of search mechanism significantly. Data image
closest to query image according to discrete matrix many
not always be the most similar data image according to
visual perception or other objective criteria.
Searching for nearest neighbor for a given point can be done
by sorting points in the database first. Partitioning methods
are used to divide the multidimensional vector space into
partitions. Partitioning methods are based on absolute
coordinate values of vector space. These partitioning
structures are useful for queries based on absolute
coordinates but not for nearest-neighbor search as search
structure does not maintain distance information between
points within a partition and its boundaries.
Image indexing is the process where feature vectors in s are
sorted so that to process a query there is no need to
sequentially scan the dbase. The choice of feature space and
distance metric affects the performance of search
mechanism to great extent. It may happen that data image
closest to query image in feature space according to distance
metric may not be most similar visually. What criteria
should be decided to match image similarity is still a big
research issue.
5 RELEVANCE FEEDBACK TO CONTENT BASED
IMAGE RETRIEVAL
In CBIR initially feature representation scheme is to be
chosen for each feature then during the retrieval of image
visual feature is selected for the process of retrieval. If there
are multiple features, then weights are specified for the
representations. On the basis of selected feature and
specified weights images are searched, matching to users
query. This approach is totally dependent on the computer
and doesn’t involve human interaction. It lacks in
performance as there is gap between high level concept and
low level features. Secondly in this approach there is
subjectivity of human perception. Different persons may
have different viewpoint over similar things and /or the
same person under different circumstances may perceive
same visual content differently.
Due to these difficulties posed by computer centric
approach, relevance feedback approach is applied. In this
approach human and computer interact to refine high level
queries to representation based on low level features.
Queries are adjusted automatically using the relevance
feedback given by the user. Such adjusted query is better
approximation to users information need. User abstains
from the responsibility of concept mapping and
specification of weights. Weights in the query object are
dynamically updated to model high level concepts and
perception subjectivity [8].
53
Relevance Feedback attempts to capture user’s precise needs
by iterative feedback and query refinement. As there is no
framework to characterize high level semantics of image
and human subjectivity of perception, relevance feedback
gives case specific query semantics. It is an active learning
process where feedbacks given by user are selected in
several rounds. Man algorithm are based on just 2
classifications of image i.e. relevant or irrelevant, while
some consider multiple relevant and irrelevant groups of
images using proper user interfaces.
6 REQUIREMENTS
Content Based Image Retrieval is applied in various fields
e.g. Medical Science, Astronomy, Space Research,
Mineralogy, Remote Sensing. While applying CBIR in any of
these fields there are several requirements which must be
fulfilled by the System. Most critical feature is the
performance of system i.e. quality of retrieval and its
relevancy. Efforts are directed to improve performance of
precision and recall. The web is large storehouse of images
which is increasing enormously by each day. System must
be able to handle large volume of images, indexing and
retrieving them. Images taken from different sources differ
in quality, resolution and color depths, hence there may be
variations in extracted color and texture. A robust system is
needed to tackle such variations. System should be such
which does not exhaust host server resources, as in online
image retrieval system there would be multiple concurrent
users. Time is also very important factor in online retrieval,
for better performance response time must be lesser.
7 IMAGE RETRIEVAL SYSTEMS
There are many image retrieval systems, which are available
today to be used for commercial purpose and research.
These systems support many tasks e.g. browsing images,
searching image either by text, sketch or example and
permit navigation with customized image categories. QBIC
(Query By Image Content) was the first content based image
retrieval system. In this system queries are based on
example image, drawings and sketches etc. (R,G,B) color
feature and k-element color histograms are used in it. Virage
is another content based image retrieval system, developed
by Virage Incorporation. Queries are based on color, texture
and shape. RetrievalWare system was developed by
Excalibur Technologies and searches images on the basis of
color, shape, texture, brightness, color layout and aspect
ratio of image. It allows users to adjust weights of visual
features. Photobook, developed by MIT Medial Lab
consisted of three subparts to extract shape, texture and
surface features respectively. Netra developed in UCSB
Alexandria Digital Library, retrieved images on the basis of
color, texture, shape and spatial location information. Visual
Seek and WebSeek retrieval system were developed in
Columbia University and searches image on the basis of
visual features. But WebSeek also accepts text based queries.
It is made up of three modules i.e. Image collection,
classification and indexing, search and retrieval module.
MARS is Multimedia Analysis and Retrieval System,
JOURNAL OF COMPUTING, VOLUME 2, ISSUE 12, DECEMBER 2010, ISSN 2151-9617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 54
developed at university of Illinois, includes relevance
feedback in image retrieval.

8 CONCLUSIONS
Image Retrieval is not an independent task, but is an
integration of multiple research community’s efforts. In past
recent years there has been much research work in the field
of Image Retrieval on the basis of text and content. Earlier
approach in the field of content based retrieval was
computer centric, where images were retrieved on the basis
of similarities of individual feature representation with fixed
weights. But this approach was not perfect as it is very
difficult to represent high level concept using low level
features and also perception difference is also another big
issue. As an after effect to this, human intervention was
involved in retrieval process as relevance feedback. In this
approach, user after giving his query can refine it later on
using relevance feedback.

Still a lot is to be done in the field of image retrieval. Even

after much research and efficient system of image retrieval,
still there is an urge for robust and reliable image
understanding technology.

REFERENCES
[1] Michael J. Swain and Dana Ballard. Color Indexing. International
Journal of Computer Vision, 1991.

[2] W. Niblack, R. Barber. The QBIC project: Querying images by content

using color, texture and shape, 1994.
[3] Markus Stricker and Orengo. Similarity of color images. 1995
[4] John Smith and Shih-Fu Chang. Tools and techniques for color image
retrieval. 1995.
[5] Robert M. Harlick, K. Shanmugam, Itshack Dinstein. Texture features for
image classification. 1973.
[6] Tamura, Shunji Mori and Takashi Yamawaki. Texture features
corresponding to visual perception. 1978.
[7] Tzi-cker Chiueh. Content based Image Indexing.
[8] Yong Rui, Thomas S. Huang, Sharad Mehrotra. Relevance feedback
technique in interactive CBIR.
[9] S. Belongie, J. Malik, J. Puzicha. Shape matching and object recognition
using shape context.
[10] J. Shi, J. Malik. Normalized Cuts and Image Segmentation, 2000.