2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare (WCFTR’16)
Object Recognition for Blind people Using
Portable Camera
Vikky Mohane
Dept. of Electronics & Tel., YCCE,
Nagpur, India.
vikkymohane@gamil.com
Abstract- For blind persons it is very essential to recognize
a product of their daily use so we implied a method to identify
product in their everyday routine by use of camera. To
separate a object from un-necessary background, movement
based technique is used to spot object of concern from the
video by instructing person to shake the object that gives
moving object area by K-means clustering background
separation technique. After detection of moving area feature
extraction SIFT technique is used to take out key points of that
product and that key-point is matched with the data base
images if it matches efficiently then it identify a product and
produce name of that product to blind persons in speech.
Instead of OCR we used SIFT because in OCR orientation of
camera must be towards label of product and for blind person
it is very tough to locate label efficiently whereas in SIFT
product orientation is not a big deal as key-points can be
matched from any position.
Index terms-Blind, Feature extraction, OCR, SIFT.
I. INTRODUCTION
According to the World Health Organization, 314
million people in the world are visually impaired and out of
them 15% blind. 13% of visually impaired people reside in
developed countries. Recently, several electronic assisting
devices [2], [3], [4], [5], [6], [7], [8] & [9] developed for
these persons but they cannot handle product recognition
efficiently. For example, moveable bar code readers used to
assists visually impaired people to recognize different
products but to find the position of the bar code and to point
the bar code reader at the bar code efficiently is not easy for
blind people.
The majority of existing systems are for mobility and
obstacle detection but still blind people face big problem to
identify of their environment and its components [10].
Identifying and locating objects is the need for mobility of
blind people by travel assistance and aid to navigation. For
this reason, we were interested in the evaluation of an
efficient algorithm for fast and Robust computer vision to
identify and locate objects in a video scene .We aim to
describe our method [10] to identify the nearby object for
blind people by proposing an approach for analyzing video
scenes based on feature extraction.
II. OBJECTS RECOGNITION USING FEATURES
EXTRACTION
978-1-4673-9214-3/16/$31.00 © 2016 IEEE
Prof. Chetan Gode
Dept. of Electronics & Tel., YCCE,
Nagpur, India.
chetan_gode@rediffmail.com
The task of finding and identifying objects is still a
challenge for computer vision systems. It is noted that
approach used for object recognition is feature extraction.
A. Features extraction
When we used features instead of raw pixel data
classification becomes easier [10]. Visual substitution
systems generally exploit a single camera to capture image
data. Further processing for Recognition is then processed
based on different features extracted from that data.
Features extraction is the process by which interesting parts
of an image are detected and represented for further
processing. It is important step in most computer vision and
image processing solutions because it marks the transition
from pictorial to non pictorial (alphanumerical, usually
quantitative) data representation [11]. Types of features
extracted from image depend on the type of image (e.g.
binary, gray-level, or color), the level of granularity
(complete image or individual regions) desired, and the
context of the application. The features which are extracted
are usually represented in an alphanumerical way for further
processing. The actual representation based on the technique
used.
B. Features extraction based SIFT
Scale Invariant Features Transform approach [12] was
designed to match pictures or objects of a different scene.
The features are partially invariant to illumination and 3D
camera viewpoint and invariant to scale and rotation. There
are two important steps to extract SIFT features: localization
key-points by extreme scale space detection and generation
of descriptors of points of interest. SIFT algorithm is applied
to object recognition. First, key-points are extracted from a
reference image and stored in the database. Then objects in
the target image are compared with the stored references
based on Euclidean distance between features vectors. The
object is identified by the matching between key-points.
In principle, SIFT works like a multi-scale corner
detector with sub-pixel positioning accuracy and a rotation-
invariant features descriptor attached to each candidate point
[1].
SIFT features carry with the location of key-points,
scale, and orientation. Lowe uses a four-dimensional
similarity transformation as the original Hough banning
structure, thus each bin denotes a particular location for the
object center, scale, and in-plane rotation. Each matching
 2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare (WCFTR’16)
feature votes for the nearest 24 bins and peaks in the
transform are then selected for a more careful affine motion
fit [13].
III. ALGORITHM OF PROPOSED SYSTEM:
The figure above shows the algorithm of our proposed
system our proposed system use a lone camera to capture
images of the view in front of the user. Different features
are then extracted from those images and objects in the
scene are recognized by comparing these features with
those of known database objects.
Before applying SIFT algorithm to the image,
preprocessing of image takes place. It includes various steps
are as shown in Figure 1.
In image acquisition, the camera captures scenes which
contains region of interest in the form of images. RGB
information is extracted from an image to apply proper
median filtering on an image. To eliminate noise median
filtering is used. As a pre-processing step median filtering
is used for better results of advance processing. Median
filtering conserves edges while eliminate noise and hence
it has significant application in digital image processing.
The dilation is performed over medium filtered image.
Fig. 1. Our proposed approach for object recognition for blind people
The basic effect of the operator on a binary image is to
gradually broaden the boundaries of regions of foreground
pixels. Thus areas of foreground pixels widen in size while
holes within those regions will reduce in size. After dilation,
to separate object from un-necessary background, movement
based technique is used to identify object of concern from
the video by instructing person to shake the object that gives
moving object area by K-means clustering background
separation technique. In this technique n observations are
separated into k clusters in which each observation belongs
to the cluster with the nearest mean, allocated as element of
the cluster. After foreground and background separation of
an image, in segmented filled image filling holes in binary
image takes place. The shapes of input image are expanded
by the dilation operation using structuring element and
after dilation noise is removed by removing small objects
that is noise then with the processing of segmented noise
removed image, processed Image is generated. Features are
extracted from the processed image which is used for
matching with data base images. If key points are matched
then identify the name of product by auditory system
otherwise detected as unknown object by our system.
IV. EXPERIMENTAL RESULTS
In our system we first check the algorithm over images
to check efficiency of the algorithm and then we apply it on
video scene to recognize objects from video scene. To
extract features SIFT method is used. In Figure 2 image
acquisition takes place from the camera.
Fig. 2. Input image from camera
2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare (WCFTR’16)
Fig. 3 . Database images
Fig. 4. Processed image over which sift performed
Fig. 5. SIFT operation
Data base images shown in Figure 3, after
preprocessing and foreground and background separation
SIFT algorithm is performed and key-point of input image is
drawn and that is matched with the features of database
image after successful matching image recognition takes
place. Distinct features can be find by SIFT even in the
small objects for efficient recognition of object we have to
improve the rate of object recognition. For efficient
matching we will use SIFT in object recognition.
In the above operation 564 key-points found, object
detected as bourn vita and processing time is 5.35 ms
similarly various objects are recognized using same
algorithm efficiently.
V. CONCLUSION
In recent years, some solutions have been devised to
help blind people or visual impaired in recognizing objects
in their environment But it is not that efficient to recognize
surrounding objects our purpose is to provide a robust and
easy system for blind people to recognize their surrounding
objects. Our proposed system uses a single camera to
capture images of the view in front of the user. Different
features are then extracted from those images and objects in
the scene are recognized by comparing these features with
those of known database objects. We have evaluated SIFT
algorithm for feature extraction. Our future work consists
on to increase the object recognition rate, to use our system
for different application like obstacle detection and to find
distance information between camera and object.
REFERENCES
[1] HanenJabnoun, FaouziBenzarti,and Hamid Amiri
“Object recognition for blind people based on features
extraction”In INTERNATIONAL IMAGE PROCESSING
APPLICATIONS AND SYSTEMS CONFERENCE 2014
[2] B. Durette, N. Louveton, D. Alleysson, and J. H´erault.
”Visuo-auditorysensory substitution for mobility
assistance: testing The VIBE”. InWorkshop on Computer
Vision Applications for the Visually Impaired,Marseille,
France, 2008.
[3] A. F. R. Hern´andez et al. ”Computer Solutions on
Sensory Substitution for Sensory Disabled People”. In
Proceedings of the 8th WSEAS International Conference
on Computational Intelligence, Man-machine Systems
and Cybernetics, pages 134–138, 2009.
[4] H. Tang and D. J. Beebe. ”An oral tactile interface for
blind navigation”. IEEE Trans Neural SystRehabilEng,
14(1):116–123, March 2006.
[5] M. Auvray, S. Hanneton, and J. K. O’Regan. ”Learning
to perceive with a visuo - auditory substitution system:
Localisation and object recognition with ’The vOICe’ ”.
Perception, 36:416–430, 2007.
[6] S. Kammoun et al. ”Navigation and space perception
assistance for the visually impaired: The NAVIG
project”. IRBM, Numrospcial ANRTECSAN, 33(2):182–
189, April 2012.
[7] F.G. Brian Katz et al. ”NAVIG: Guidance system for the
visuallyimpaired using virtual augmented reality”.
Technology and Disability, 24(2):163–178, June 2012.
2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare (WCFTR’16)
[8] S. Abboud, S. Hanassy, S. Levy-Tzedek, S.
Maidenbaum, and A. Amedi. ”EyeMusic: Introducing a
”visual” colorful experience for the blind using auditory
sensory substitution”. Restorative neurology and
neuroscience, January 7 2014.
[9] B.D.C. Martnez, O.O. Vergara-Villegas, V.G.C. Snchez,
H. de JessOchoaDomnguez, and L.O. Maynez. ”Visual
Perception Substitutionby the Auditory Sense”. In
BeniaminoMurgante, Osvaldo Gervasi, Andrs Iglesias,
David Taniar, and Bernady O. Apduhan, editors, ICCSA
(2), volume 6783 of Lecture Notes in Computer Science,
pages 522–533. Springer, 2011.
[10] H. Jabnoun, F. Benzarti, and H. Amiri. ”Visual
substitution system for blind people based on SIFT
description”. In International Conferenceof Soft
Computing and Pattern Recognition. Tunsia, pages 300–
305,August 2014.
[11] L. Juan and O. Gwon. ”A Comparison of SIFT, PCA-
SIFT and SURF”.International Journal of Image
Processing (IJIP), 3(4):143–152, 2009
[12] G.D. Lowe. ”Distinctive Image Features from Scale-
Invariant Keypoints”.Int. J. Comput. Vision, 60(2):91–
110, November 2004.
[13] R. Szeliski. ”Computer Vision: Algorithms and
Applications”. Springer-Verlag New York, Inc., New
York, NY, USA, 2010.