Professional Documents
Culture Documents
Marina Bay Sands, Singapore, 10-12th December 2014 (ICARCV 2014) Fr47.5
Abstract—One of the major driving forces behind technology II. U SED T ECHNOLOGIES
development is the establishment of equality among everyone.
It is intended that the strategic use of technology complements Radio Frequency Identification, RFID, is a method of
competencies or diminished capacities, improving self confidence
and autonomy. Blind and partially sighted people often deal with
automatic identification by radio signal. A tag identifies and
this inequality in the performance of tasks or daily activities. object and is able to respond to radio signals from radio
The presented work reviews some of the proposed technologies waves transceivers. Tags can be active or passive: active tags
to assist visually impaired people in common activities, specifically have external power supply and therefore a greater range.
navigation and going shopping. The Electronic Product Code, EPC, is a universal identifier
designed to be stored on an RFID tag.
I. I NTRODUCTION UPC, Universal Product Code, is the foundation of EPC.
UPC is a barcode symbology that stores information within
Visually impaired people are defined as those with little black bars and white spaces. There are several UPC types that
or no visual capacity, also named blind and partially sighted can be used for many different applications. Quick Response,
people. These conditions often limit peoples ability to perform QR, are 2D barcodes codes that can easily be deciphered by a
common tasks and affects their quality of life. It is estimated camera. Those codes usually store text information, numbers,
that there are over 285 million[23] people with this disability geographic coordinates or other information.
worldwide. Bluetooth is a wireless transfer technology of a small range
Among the activities affected by visual impairment, navi- widely used on mobile phones, computers and accessories.
gation plays a fundamental role, since it enables the person to ZigBee is another wireless communication technology used
independently move in safety. The heterogeneous environment, between devices, known by its low powered operation and long
easily perceived by visually enabled people, is hardly known distance range. Unlike Bluetooth and ZigBee, that are personal
by partially sighted people. The environment features are area networks, WiFi is a local area network with wider range
perceived through hearing and touch, usually blind people use and enables devices to exchange data or connect to the Internet
dogs and white canes for that. using radio waves.
Another challenging task for these people is independent The famous GPS standing for Global Positioning System,
shopping. In most supermarkets information about products is a satellite based positioning system, enabling to specify
is only presented for visually enabled people, making blind the position of any point on the earth’s surface with an
people dependent on others. unobstructed line of sight to four or more GPS satellites.
The work here presented provides an overview over some The Global System for Mobile Communications, GSM, is the
technologies intended to support visually impaired people in standard used by mobile phones. GSM operates in the 900MHz
some daily tasks, such as navigation and going shopping. or 1800MHz for the second generation networks, 2G.
Systems and applications here presented were divided into Infrared radiation is part of the electromagnetic radiation
three categories: Assistive Technologies for Navigation, As- with longer wavelengths than visible light, 700 nanometers.
sistive Technologies for Shopping and Assistive Technologies Infrared radiation is widely used on remote controls of televi-
for Navigation and Shopping. Within shopping activity it is sions and other devices. Ultrasounds are sound waves with a
included knowing and finding product categories or specific frequency greater than the human hearing range, approximately
products. 20kHz. Ultrasounds are often used in medical imaging and on
The paper is organized as follows: Section II introduces the boats to calculate distances.
major used technologies in the systems, Section III presents Voice recognition software is used to convert speech to
systems for assisted navigation, Section IV presents systems words. Several software can be used for this purpose, the
for assisted shopping, Section V systems for both navigation choice of the best to use depends on the application features.
and shopping assistance and Section VI concludes the paper. Speech synthesizers execute the opposite operation, converting
from words and sentences to speech.
B. RFID System 2
S. Chumkamon proposes a system with three subsystems:
the track infrastructure, the navigation device and the navi-
gation server[3]. The first one is related to the RFID tags
network, it intends to place tags on a tactile orientation system
on the footpath, which is used by visually impaired people for
orientation. The navigation device is composed of an RFID
reader, a microprocessor, a communication module, a user Fig. 1: BlindAid[13].
interface and a memory module.
When the user wants to start the navigation, the commu-
nication module of the navigation device asks the server, via E. Smart-Robot
GPRS network, for a route and stores the information received. Smart-Robot is a robot proposed for indoor and outdoor
This way, communication with the server is only required on navigation[24]. Indoor navigation is performed using RFID
the route start or when the user wants to start a new one, or technology and outdoor by GPS. The robot is equipped with an
gets lost. The system enables user guidance at least in the analog compass for orientation, ultrasonic and infrared sensors
decision points, but obstacle detection relies on user’s abilities for obstacle avoidance, a speaker and a vibrating glove for
and sensory skills. instructions delivery, and a keypad for user inputs.
Destination is recorded by the keypad and the robot per-
C. RG forms navigation checking for obstacles on the way. Indoors
Robotic Guide, RG, is a robot that escorts the user through the Smart-Robot travels close to the wall where it finds best
a route, it is intended to assist users in navigating unknown RFID signal intensity. Although it is known that the GPS
environments[9]. Indoors navigation is performed by potential system has an error of some meters, this system has the strong
fields and by finding empty spaces around the robot. Potential advantage of performing indoor and outdoor navigation.
fields is an algorithm that calculates robot motion by attracting
it to the desired end and repulsing it from obstacles. This F. GPS System 1
technique enables the robot to follow corridors without ori- This system integrates a mobile phone with camera and GPS
entation sensors. RFID tags are used for navigation and for technology, and motion and magnetic induction equipment[15].
robot location. The aim is to provide information to travel a specific route
This system has the advantage of not requiring a global calculated for the system regarding user desired destination,
map of the environment to navigate, the last position is always this will comprise traffic and obstacle information.
stored on the connectivity graph and precise locations are Localization and route calculations are based on GPS,
known when the robot receives an RFID tag. and orientation and motion are received by the motion and
1930
to deliver right alarms: fast and slow moving obstacles through
sound and vibrations for different levels of warning.
The presented system only helps the user on his arbitrary
path, not enabling the blind user to travel through a new path
with confidence, in order to not get lost. The user is enabled
to use the system on any environment, receiving information
about obstacles everywhere.
1931
IV. A SSISTIVE T ECHNOLOGIES FOR S HOPPING As the previous systems, the presented here is designed
A. Voice Shopping Guide only for product identification, so the user will not receive
instructions to reach it. The system relies on user’s smartphone,
This system places voice emitters near the products on the
but considering the products are not yet equipped with QR
shelves that displays information when the user approaches
code and Braille seals, there are several modifications needed
them[6]. Infrared sensors are used to detect the proximity of
to me done on the environment. A great advantage of this
blind users, that proximity distance ranges since 10cm to 50cm.
system is that the user is enabled to receive information about
Information displayed can introduce the section the user just
the unique product he is holding.
entered, or identify product types in the user’s vicinity, and can
be easily changed anytime to adapt the environment changes. E. RFID and EPC Based System
he required changes on the environment will depend on the An EPC-RFID combined system was proposed by Sreekar
level of specification intended to achieve, the user does not Krishna[7]. A RFID reader collects the EPC code from the
need to carry any device to use the proposed system. Although products on the shelves as user approaches them, and the
the system can provide much information, the user will rely product’s information is accessed by user’s PDA or smartphone
on his skills to have sure he has the specific wanted product. and delivered to the user via synthesized speech. The users
B. GroZi device communicates with the central server in the supermarket
through Internet. Central server is essentially a relational
A different proposal is the presented by GroZi[14][18] database with information of all the products which performs
project. This proposal is composed of a web site for wanted the task of associating EPC sent by the portable computing de-
product’s list creation, a computer vision software for products vice to a unique product’s information. The interface between
recognition in stores and a portable device that can execute the user’s device and the database is implemented as a middle-
computer vision algorithms and give the user haptic and verbal ware layer. RFID reader is mounted on a bracelet and ensures
feedback. The user compiles a shopping list of products and Bluetooth communication with the user’s device.
uploads it on the portable device. In the store, the user wears Since the product is identified by an unique EPC, the user
a hand glove with a small camera and vibrating motors to is enabled to receive information about one specific product.
point aisles, the application will return information about With information about the surrounding products the user can
products there and instructions to find the desired product. be lead to the wanted product, but all products need to be
When a product on the shopping list is detected in the video equipped with EPC or RFID identification.
stream, the application will give the user either haptic or
verbal instructions to reach it. GroZi uses in vitro and in situ
images: the first are taken under ideal lighting and perspective
conditions, the second are obtained from video streams in the
store.
Since it relies on images, minimal environment adjustments
are required, however the system does not indicate where the
products are, only tells what is the product the user is pointing
at.
C. Barcode System
The system is based on barcode recognition: barcodes are
detected in vision range of the user. With code information
and its known location, the user is informed about commodity Fig. 4: ShopTalk[16].
regions, products arrangement, and name, trademark and price
of a single product[17]. Barcode identification comprises a set
of image processing steps, such as filtering to noise and texture F. Trinetra
detection. Trinetra is a solution based on RFID and UPC
Assuming the system can be implemented in a simple identification[11]. Trinetra is composed of an Internet and
device, or even in the smartphone, this system as the advantage Bluetooth enabled cell phone, text-to-speech software and a
of enabling the user to user his own portable device. portable barcode reader. The shopper’s phone receives informa-
tion from the barcode reader, RFID reader or keypad; the first
D. QR Code System two are received via Bluetooth. When an ID is received, phone
This proposal is implemented on a smartphone equipped application matches it with a most recently used ID’s looking
with QR code reader software[1]. The user uses the mobile for a match. When no match is found, phone communicates
phone to scan QR codes on products, the device will connect with the server over the Internet, the server will match the ID
to Internet and a verbal description of the product will be with all product ID’s available in the store. Once a product
delivered. To enable blind people to find the QR code, its is identified, its description will be returned to the user.
location is marked with a Braille seal. Information is delivered to the user via synthesized speech.
1932
Although this solution identifies well each product, it does
not allows the user to navigate autonomously within the store
and easily find wanted products. When the environment does
not is provided with RFID tags, the UPC identification prevails
and it is considered that all products have a unique UPC.
V. A SSISTIVE T ECHNOLOGIES FOR NAVIGATION AND
S HOPPING
A. Virtual Walking Stick
This system intends to allow detection of obstacles, identifi-
cation of colors, recognition of products or objects by barcode
or RFID reading, orientation, reproduction of audio content,
signaling the user on dark environment and recognition of
letters, words or sentences through a digital camera. This
device is aimed to provide as much information as the user
may need, since date and hours information to points of
interest in public spaces, replacing the traditional white cane
by a stick with technology that collects information about
the environment without having to physically touch objects Fig. 5: RoboCart[8].
and obstacles[21]. Ultrasonic sensors are used for obstacles
and depressions detection (R4) and alarms or warnings are
delivered by vibrations and sound alerts. A digital compass the user a product selection interface, when confirmed the
and a GPS system are installed for outdoor navigation and selection guides him to the vicinity of the product. Once there,
positioning information. Product recognition is enabled by RoboCart uses the user’s egocentric frame of reference to
digital camera or RFID reader and a wireless connection find the product and a barcode reader ergonomically modified
to an existing database on the establishment, which delivers to help the alignment with the shelves, where the products
information about the product or object. barcodes are placed. The floor of the store is converted into an
RFID-enabled surface, where each RFID tag had its own 2D
B. BlindShopping coordinates. This RFID tags are used as recalibration areas,
BlindShopping is an RFID and QR-code based mobile when the robot reaches a recalibration area its localization is
solution[12], that demands inexpensive off-the-shelf technol- well known.
ogy. The products are grouped into different categories, and Two drawbacks of RoboCart are that the user has to scan
these are divided into product types which are separated into all barcodes until reach the target one, and he can completely
concrete brand products. The supermarket surface is mapped miss the barcode he is looking for and never find the product.
with cells containing shelves and passageway cells, and RFID On the other hand, the RFID identification enables constant
tags are distributed throughout the floor creating an RFID and precise positioning.
map. BlindShopping maps the IDs of the RFID tags within
a cell to navigation and product localization, instructions are D. ShopTalk
given to the user through voice messages. Once in the target ShopTalk[16] is a system composed of a processor, a
aisle, this solution offers support for product recognition by numeric keypad and a barcode scanner. ShopTalk guides the
shelf section identification, by QR code, product own ID, user to the vicinity of the target product with vocal instructions
or barcode scanning. The user will use personal smartphone with route directions and descriptions of the store layout.
and the supermarket must have a wireless network and server The topological map of the store, used for route calculations,
installed. RFID readers, attachable to the white cane, are lent is a directed graph whose nodes are decision points like
to visually impaired clients by supermarket. aisle entrances or cashier lane entrances. Route directions are
Once the floor is mapped with RFID tags, user’s position is constructed from a database of parametrized route directions,
always known, but obstacle detection relies on user abilities. and the keypad is used to set instructions as done, the system is
As the system implies the deployment of an RFID web to unaware of the shopper’s actual location and orientation. The
user’s location and QR codes on the shelves for products barcode scanner was modified to align more easily with the
identification, there are several environment required changes. shelves where the product barcodes are. The key data structure
The user is enabled to specify a wanted product and be guided that associates barcodes with aisles, aisle sides, shelf sections,
to it, but he can not specify a list of products in one time. specific shelves and relative positions on the shelves. Users
location can be known once he/she scans a barcode on an
C. RoboCart aisle, allowing the user to use the system even if he/she gets
Another solution uses a robot to help the visually impaired lost.
costumer to find wanted products. RoboCart[4] presents to The user is guided to the vicinity of the wanted product,
1933
but relies on own sensory abilities for obstacles detection. [5] Guidance, navigation and information system especially adapted
for blind or partially sighted people. https://www.google.com/patents/
The system comprises a whole new user device, however uses WO2010002284A2?cl=en Google Patents, 7 January 2010.
barcodes on the shelves that may already exist, so minimal [6] Induction voice system capable of performing shopping guide for blind
environment modifications are required. people. https://www.google.com/patents/CN202775358U?cl=en Google
Patents, 13 March 2013.
1) ShopMobile: ShopMobile is an updated version of [7] KRISHNA Sreekar et al., A Wearable Wireless RFID System for Accessi-
ShopTalk that runs on a mobile phone. A vision-based barcode ble Shopping Environments. BodyNets ’08 Proceedings of the ICST 3rd
scanning method[10] is proposed. Once again are applied international conference on Body area networks, Article No. 29.
[8] KULYUKIN Vladimir and KUTIYANAWALA Aliasgar, Accessible Shop-
stabilizers to help the visually impaired user align the camera ping Systems for Blind and Visually Impaired Individuals: Design Require-
with shelves, where product barcodes are placed. The system ments and the State of the Art. The Open Rehabilitation Journal, 2010, 3,
finds barcodes in images: when part of a barcode is detected, 158-168.
[9] KULYUKIN Vladimir et al., RFID in Robot-Assisted Indoor Navigation
instructions are given to slide the phone along the shelf in one for the Visually Impaired. Proceedings of 2004 IEEE/RSJ International
specific way. Once the target barcode is reached, the user takes Conference on Intelligent Robots and Systems. September 28 - October
a product from the shelf. 2, 2004, Sendai, Japan.
[10] KULYUKIN Vladimir and KUTIYANAWALA Aliasgar, From ShopTalk
This system has the strong advantage of enabling the user to ShopMobile: Vision-Based Barcode Scanning with Mobile Phones for
to use his own mobile phone, does not requiring effort to learn Independent Blind Grocery Shopping. RESNA Annual Conference, 26-30
how to use a new one. June 2010, Las Vegas, Nevada.
[11] LANIGAN Patrick et al., Trinetra: Assistive Technologies for Grocery
Shopping for the Blind. 10th IEEE International Symposium on Wearable
VI. C ONCLUSION Computers, 11-14 October 2006.
The technologies proposed cannot really be generally com- [12] LÓPEZ-DE-IPIÑA Diego, LORIDO Tania and LÓPEZ Unai, Indoor
Navigation and Product Recognition for Blind People Assisted Shopping.
pared, as some are only intended to assist navigation and other Third International Workshop, IWAAL 2011, Held at IWANN 2011,
to find products. Among the systems for assisting navigation, Torremolinos-Mlaga, Spain, June 8-10, 2011.
GPS is widely used as it is a common technology. However it [13] MAU Sandra et al., BlindAid: An Electronic Travel Aid for the Blind. The
Robotics Institute - Carnegie Mellon University, Pittsburgh, Pennsylvania.
is already known that GPS have an error of a few meters, for May 2008.
pedestrian navigation that is a wide error. And so, for indoor [14] MERLER Michele, GALLEGUILLOS Carolina and BELONGIE Serge,
navigation, which means more accuracy, RFID technology is Recognizing Groceries in situ using in vitro Training Data. SLAM,
Minneapolis, MN 2007.
more used. For system intended to find specific products, a [15] Navigation mobile phone suitable for blind people. https://www.google.
relevant technology used is barcode and QR code identifica- com/patents/CN101980515A?cl=en Google Patents, 23 February 2011.
tion, as it is already used for products unique identification it [16] NICHOLSON John, KULYUKIN Vladimir and COSTER Daniel,
ShopTalk: Independent Blind Shopping Through Verbal Route Directions
is highly advantageous to support systems on it. and Barcode Scans. The Open Rehabilitation Journal, 2009, 2, 11-23.
This study presents some proposal systems and applications [17] Shopping guide method and apparatus for blind in supermarket based on
designed for support daily tasks of blind and visually impaired bar code identification. https://www.google.com/patents/CN101474020B?
cl=en Google Patents, 8 January 2010.
people, with relevance to navigation and going to the supermar- [18] TRAN Robert et al., GroZi: Grocery Shopping Assistant for the Blind.
ket. In order to integrate blind people on society, more public Continuity Report, ENG 100L.
spaces should be equipped with such systems or alternative [19] Ultrassonic technology based electronic terminal for blind peo-
ple and navigation method thereof. https://www.google.com/patents/
methods with the same intention. From the proposed systems CN102743271A?cl=en Google Patents, 24 October 2012.
here presented it is clear that RFID is a promising technology [20] VARPE Kanchan and WANKHADE M. P., Survey of Visually Impaired
in this field, due to its clear accuracy and low acquisition cost Assistive System. International Journal of Engineering and Innovative
Technology (IJEIT), Volume 2, Issue 11, May 2013.
and deployment. [21] Virtual walking stick for assisting blind people. https://www.google.com/
patents/WO2011104589A1?cl=en Google Patents, 1 September 2011.
ACKNOWLEDGMENT [22] VTEK Stanislav et al., New Possibilities for Blind People Navigation.
International Conference on Applied Electronics (AE), Czech Republic,
This work was partially financed by iCIS Intelli- 7-8 September 2011.
gent Computing in the Internet Services (CENTRO-07- [23] Visual impairment and blindness. http://www.who.int/mediacentre/
factsheets/fs282/en/ World Health Organization. March 29, 2014.
ST24FEDER002003), Portugal. [24] YELAMARTHI Kumar et al., RFID and GPS Integrated Navigation
System for the Visually Impaired. 53rd International Midwest Symposium
R EFERENCES on Circuits and Systems (MWSCAS). 1-4 August 2010.
[1] AL-KHALIFA Hend, Utilizing QR Code and Mobile Phones for Blinds
and Visually Impaired People. Computers Helping People with Special
Needs. Lecture Notes in Computer Science Volume 5105, 2008, pp 1065-
1069
[2] Blind people communication navigation device. https://www.google.com/
patents/CN201476794U?cl=en Google Patents, 19 May 2011.
[3] CHUMKAMON Sakmongkon, TUVAPHANTHAPHIPHAT Peranitti and
KEERATIWINTAKORN Phongsak, A Blind Naviagtion System Using
RFID for Indoor Environments. 5th International Conference on Electrical
Engineering/Electronics, Computer, Telecommunications and Information
Technology. ECTI-CON 2008. 14-17 May 2008.
[4] GHARPURE Chaitanya and KULYUKIN Vladimir, Robot-assited shop-
ping for the blind: issues in spatial cognition and product selection. Intel
Serv Robotics, 2008, 1:237-251.
1934