2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE)
Improving reality perception for the visually
impaired
2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE) 978-1-7281-4142-8/20/$31.00 ©2020 IEEE 10.1109/ic-ETITE47903.2020.093
Shreyansh Banthia
School of Information Technology
Engineering
Vellore Institute of Technology
Vellore, India
shreyanshbanthia@live.com
Abstract— In this paper, three different assistive products
are discussed, one is an attempt at augmenting the walking cane
used by the visually impaired persons, another product attempts
at creating a solution that can convert text into braille in real
time and the last product is for providing image recognition and
classification capabilities to those who have visual impairment.
Keywords—Blind, Visually impaired, Internet of Things,
Machine Learning, obstacle detection, image recognition, image
classification, braille
I. INTRODUCTION
The World Health Organization has made the observation
that people with disabilities are one of the fastest growing
minority groups in the world and south-east Asian countries
like India and China are home to one of the biggest
populations of persons with visual impairment. [1] While the
benefits from advancements in technology have been
relatively slower to reach such groups. An important reason
for this happens to be that commercial innovation in this space
is not very lucrative for companies since the majority of the
potential customers don’t have deep pockets to buy technical
products that aren’t cost effective. Though, the number of
companies working in this space have certainly increased over
the years, but the market of products is still very niche and
isn’t accessible to all.
For assistive technology to take off, it needs to be cost
effective while working with the products that their target
audience is already very comfortable with. Rate of adoption
of a new solution is likely to be much higher if its something
that it’s audience is even remotely familiar with and can adapt
to easily.
Internet of Things or IoT can be defined as a system of
computing devices and sensors which can be uniquely
identified in the given network and have to capability to
communicate with each other without human intervention.
The term was coined by Kevin Ashton of Procter & Gamble
in 1999 [5] and has seen rapid growth since then where
Ericsson has estimated that there will approximately eighteen
billion interconnected IoT devices by 2022. [17]
Some of the best examples where IoT based solutions have
been able to make a huge difference to people with disabilities
include Microsoft’s Seeing AI application, special shoe
insoles developed by Ducere Technologies, assistive wearable
technology by VTT Technical Research Centre of Finland,
applications like Crosswalk and many more.
978-1-7281-4141-1/$31.00 ©2020 IEEE
978-1-7281-4142-8/$31.00 ©2020 IEEE
V Priya
School of Information Technology
Engineering
Vellore Institute of Technology
Vellore, India
vpriyacse@vit.ac.in
II. LITERATURE REVIEW
In the paper titled ‘An overview of the Internet of Things
for people with disabilities’, the author discusses the various
scenarios in which IoT based solutions can be implemented to
assist people with different types of disabilities. Apart from
discussing broader ideas, the author also goes ahead and
presents some general architecture models detailing how these
solutions might work in tandem. The paper ends with a
commentary on possible challenges and their possible
solutions. [2]
While in the paper titled ‘Mobile applications for assisting
mobility for the visually impaired using IoT infrastructure’,
the authors first discuss why the innovations in the assistive
technologies for the visually impaired have not been readily
accepted by the target audience, main reasons being
requirement of dedicated hardware which is usually expensive
and cumbersome. They then go ahead and discuss how
location-based services, that come bundled with the
smartphones of today, be leveraged along with IoT
infrastructure to assist a visually impaired person to travel
independently. [4]
In another paper titled ‘IoT based smart stick for blind
people’, its authors have explored the possibility to augment
the walking cane of a visually impaired person with IoT
sensors and here the walking cane was fitted with proximity
and ultrasonic sensors that identify different types of obstacles
with limited accuracy and then information on these obstacles
is alerted to the user via vibration and sound outputs. [3]
The paper titled ‘Application of Deep Learning for Object
Detection’ goes in-depth comparing the deep learning
frameworks, datasets available and the deep learning methods
available for object detection. These are compared across
metrics like specific advantages offered, scenarios in which
each combination will find use etc. This paper served as a
basis for choosing TensorFlow for object recognition in the
smart glasses, which is addressed later in this paper. [15]
In the paper titled ‘Low-cost real-time braille keypad for
communication’, the authors detail out a braille-based keypad
solution that converts inputs in braille format into text in real
time. For achieving 9 buttons are connected to a
microcontroller which processes the inputs from these buttons
into English characters. The six buttons form the braille script
while the remaining three are mapped to delete, space and line
break. The solution in itself is very intuitive but can’t deal with
special characters in it’s current form. [16]
1
 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE)
III. SMART NAVIGATOR
The smart navigator is basically an IoT augmented
walking cane that is supposed prove, in concept, that
traditional walking canes used by the visually impaired to
navigate can be improve and made much more helpful with
the help of IoT sensors.
Fig. 1. Smart Navigator Cane
A. Proposed system
The following figures demonstrate the system architecture
and the entity relationship diagram for the smart navigator –
Fig. 2. System diagram of Smart Navigator
Fig. 3. ER diagram for Smart Navigator
B. Working methodology
The smart navigator consists of two logically separate entities
that work individually to achieve two separate goals, while
sharing some common hardware components. These are
listed and explained in the following paragraphs –
• The obstacle detection circuit.
• The panic button circuit.
The obstacle detection circuit consists of an ultrasonic sensor,
a piezo speaker, an Arduino board and supporting breadboard
and connecting wires. The logic related to this circuit in the
Arduino board works in the following way – The ultrasonic
sensor, operating at 40000 Hz, sends out a sound wave every
10s. This wave after bouncing back from an obstacle in the
path is picked up by the ultrasonic sensor which then
calculates the approximate distance of the obstacle from the
walking cane on which this setup is mounted. This calculated
value is then passed on to the Arduino which checks if the
value is in the range of 4-3 meters or in 3-2 meters or in 2-0
meters and based on that it adjusts the volume of the piezo
speaker and trigger it. This way, the visually impaired user of
the stick can get a rough idea about how far the different
obstacles are.
Talking about the panic button circuit, it consists of a
capacitive touch sensitive button, a GSM/GPRS module, an
Arduino and supporting breadboard with connecting wires.
The logic related to this circuit works in the following way –
The Arduino upon receiving three continuous tap inputs from
the capacitive button triggers a hardcoded distress message to
be sent to the visually impaired user’s guardian on his/her
phone via the GSM/GPRS module. The guardian can then
take necessary steps to reach out to the user and make sure
that he/she is safe. This panic button functionality is
standalone and doesn’t depend on the user’s phone to operate
in any way.
C. Components required
1) ATmega328/p (Arduino)
ATmega328p is a CMOS 8-bit low power microcontroller
based on the modified Harvard architecture of the RISC
processor core. Some of it’s features that make it a very
popular option for use in IoT projects are a 32 Kbyte ISP flash
memory, it’s various power saving modes, it’s advanced
RISC architecture that allows faster program execution,
ability to be used at extreme temperatures etc. The popular
Arduino development platform is also based on this
microcontroller.
Fig. 4. ATMega328P Microcontroller Pin diagram (Source: Adapted
from [6])
2
 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE)
2) SIM900A GSM/GPRS Modem
The technology underlying the SIM900A is the Dual Band
GSM/GPRS which is based on the SIM900A modem by
SIMCOM. It supports two frequencies – 900 MHz and 1800
MHz, which are tuned into automatically. The processor
powering it is a single-chip AMR926EJ-S core and that helps
keep the size of this modem relatively compact and cost-
effective.
Fig. 5. SIM900A GSM/GPRS Modem (Source: Adapted from [7])
3) Piezo speaker
A piezo speaker, like the one shown above, consists of piezo
crystals between two semiconductors and whenever voltage
difference is applied across these semiconductors the
resulting pressure variation causes the crystals to produce
sound. Piezo speakers usually operate between 3 – 250 V and
have a current consumption of less than 30 mA.
Fig. 6. Piezo speaker (Source: Adapted from [8])
4) Capacitive Touch Sensor (TTP223-IC)
TTP223 is a single touch detector pad and can detect small
capacitive element variations. It can operate on relatively low
power and its operating voltage lies between 2 – 5.5 V, which
makes it suitable for use in low powered circuits. Its fast
mode response time is around 60 mS while in the low
powered mode it is around 220 mS. The sensitivity for this
sensor can be adjusted with an external capacitor (0 – 50pF).
Fig. 7. Capacitive Touch Sensor (TTP223) (Source: Adapted from [9])
5) Ultrasonic distance sensor
This ultrasonic distance sensor uses SONAR for non-contact
range detection of objects in the near distance with relatively
high accuracy. It is not affected by other sources of light, but
its accuracy takes a hit while detecting soft objects. It requires
a current of 15 mA to operate and has a range of 1 inch - 13
ft. Other details include an effectual angle of less than 15
degrees, a resolution of 0.3 cm and measuring angle of 30
degrees.
Fig. 8. Ultrasonic distance sensor (Source: Adapted from [10])
D. Results
The smart navigator cane prototype does suffer from
accuracy issues owing to the limited accuracy that the
ultrasonic sensor used provides and at the same time the
limited processing power that an Arduino microcontroller
comes with. On the other hand, in theory, it validates that a
product of this kind can be developed owing to advancements
in technologies like LIDAR, machine learning on edge
devices etc.
E. Advantages
• Portable
• Works as a standalone product without critical
dependencies.
• Augments a tool that a visually impaired person is
already likely to be comfortable with
F. Scope for further improvement
• The ultrasonic sensor and the Arduino can be replaced
with a much more powerful LIDAR sensor and a chip
powerful enough to process it’s data in the real time.
This will provide a much more accurate idea of the
environment to the user of the smart navigator.
3
 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE)
• The GSM/GPRS module can be augmented with a
GPS module using which the GPS coordinates of the
location of the user can be included in the distress
message to the guardian.
IV. BRAILLE NEWS READER
The braille news reader is a hardware-based solution that
makes converts news articles fetched from the internet via an
API into braille in real time. Since the braille script consists of
six dots which when embossed in different combinations form
the characters, the same can be replicated through six
solenoids which can be selectively pushed up and down to
form each character. While building this Proof of Concept
product, due to some operational issues, the solenoids were
replaced with LEDs, but the concept still stands.
Fig. 9. Braille News Reader
A. Proposed system
The following figures show the system architecture and the
entity relationship diagram for the braille news reader –
Fig. 10. Braille News Reader system diagram
Fig. 11. ER diagram for Braille News Reader
B. Working methodology
The Raspberry Pi 3 powering the braille news reader has the
logic for this application written in python. Inside the code an
API call is made to the NEWS API and an article is fetched.
The fetched response contains the article headline, the author
name, the publishing date and time, the article body and the
site where the article was published on. The article headline
and the article body are then split across individual
characters, which are then one-by-one passed on to the logic
that converts the characters into mechanical outputs of the
solenoid.
Each character of the alphabet, along with the decimal system
and the special characters including spaces have been linked
to a corresponding representation in braille in the form of
zeroes and ones. For example, ‘a’ is represented as ‘100000’
while ‘b’ is represented as ‘101000’ and a white space is
represented as ‘111111’. As one might notice, the braille
representation consists of six spaces which can be filled with
zeroes or ones based on the character which is to be formed.
Each of these six spaces represent the six solenoid output pins
and a one means the up state for a solenoid while a zero means
a down state for a solenoid. Based on the braille
representation received, the solenoids move up and down to
form the characters in braille. The speed at which the braille
news reader operates is new character being formed every
500 milliseconds. In this fashion a news article can be
converted into braille in real time allowing a visually
impaired person to consume such content easily.
C. Components required
1) Raspberry Pi 3 Model B
Raspberry Pi 3 Model B is an ARM based small computer
that has been developed by the Raspberry Pi Foundation. It is
a third-generation upgrade to the ever-popular Raspberry Pi
and comes with significant improvements over the previous
generations. It supports a host of operating systems and
provides the crucial capability to interact with I/O
components like devices, sensors etc. via it’s 40 I/O pins.
Fig. 12. Raspberry Pi 3 Model B Pin diagram (Source: Adapted from [11])
2) 1N4004 diode
A diode only allows unidirectional current flow, usually from
the anode to the cathode. The 1N4004 diode can carry current
up to 1A and withstand up to 30A, making them suitable for
4
 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE)

circuits designed to operate at less than 1A. The reverse • A user interface of a suitable form can be created to
current rating for this diode is a mere 5uA while the peak allow a user to choose the kind of articles he/she
reverse voltage rating is up to 400V. wants to read and the source they want to read them
from.
V. SMART GLASSES
The smart glasses make use of image recognition and
computer vision capabilities that a deep learning library like
TensorFlow provides, to help a visually impaired person
understand the object that is in front of them.
Fig. 13. 1N4004 diode diagram (Source: Adapted from [12])

3) 5V push pull solenoid

The solenoid used has a 5V operating voltage, is 27mm in
length and gives a 10mm long throw. It has a captive armature
with a return spring. At 5V operating voltage, it draws in 800
mA current and thus requires a strong driver to operate.

Fig. 15. Smart Glasses prototype

A. Proposed system
The following figures explain the system architecture and the
entity relationship diagram of the smart glasses –

Fig. 14. 5V push pull solenoid (Source: Adapted from [13])

D. Results
This prototype, even though crude, can reliably convert all
English language characters into braille characters as long
they are mapped in braille. Compared to a product like
HumanWare Brailliant, which retails for over $2500 in the
market, allowing its users to type and consume digital text in
braille, this prototype costed approx. $70 to be built. It goes
without saying that a matured miniaturized product built on Fig. 16. Smart Glasses system diagram
this prototype will cost considerably more than the current
cost but still would be much cheaper than most products on
the market out there.
E. Advantages
• Exponentially cheaper compared other commercial
alternatives in the market.
• Can be used to convert any form of digital text into
braille in real time.
• The speed at which the text is converted into braille
is adjustable.
F. Scope for further improvement
Fig. 17. Smart Glasses ER diagram,
• Currently this works only with text fetched from an
API, but it can be made to work with e-books and the B. Working methodology
various formats that they come in. The Raspberry Pi 3 powering the smart glasses runs a version
• The size of the product can be reduced to an extent of Android Things which basically allows a TensorFlow
where most of it’s user would find it very based computer vision program to run as sort of an Android
comfortable. application. The machine learning model being used here

5
 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE)
works completely offline but suffers from limited categories
in which it can classify images of objects being fed into it.
The circuit here consists of a push button, a Raspberry Pi 3, a
LED and a camera module for Raspberry Pi 3. The user
brings the object to be detected in front of glasses and press
the push button on the side, causing the LED to glow
signaling that the camera module has been triggered. The
camera module then takes a picture of the object, sends it to
the Raspberry Pi 3, which is then fed into image classification
algorithm that is provided by the TensorFlow library. The
output of the algorithm is some categories/object names with
probability percentages in front of them, this is provided in a
JSON format. The response is then parsed and the object with
the highest probability percentage is read out to the user via
the headphones attached to the Raspberry Pi 3.
C. Components required
1) Raspberry Pi 3 Model B
Raspberry Pi 3 Model B is an ARM based small computer
that has been developed by the Raspberry Pi Foundation. It is
a third-generation upgrade to the ever-popular Raspberry Pi
and comes with significant improvements over the previous
generations. It supports a host of operating systems and
provides the crucial capability to interact with I/O
components like devices, sensors etc. via it’s 40 I/O pins.
Fig. 18. Raspberry Pi 3 Model B Pin diagram (Source: Adapted from [11])
2) Raspberry Pi 5 MP camera board
This camera module interfaces with a Raspberry Pi 3 Model
B via the Camera Serial Interface and provides a resolution
of 5 MP. It uses an Omnivision 5647 sensor in a fixed focus
module while connecting with the Raspberry Pi Model B via
the dedicated 15-pin MIPI ribbon cable. The CSI bus supports
high speed data rates and exclusively carries pixel data to the
BCM2835 processor.
Fig. 19. Raspberry Pi 5 MP camera board (Source: Adapted from [14])
3) LED
4) Push Button
D. Results
The smart glasses prototype performs with limited accuracy
owing to the offline object detection model it works with
along with the limited 80 object classes that the version of
TensorFlow used here provides. The accuracy is also
considerably affected by the lighting of the surroundings,
distance of the object from the camera, stability of the object
at the time of taking the picture and other factors like these.
In comparison to products like Intel’s Vaunt glasses and
Google’s Google Glass which come with an OS built
specifically for them and high-grade components, this
prototype falls short in a lot of areas. But competing with
those products isn’t what this prototype was built to do. The
aim was to prove that offline object detection capabilities can
be provided to the visually impaired at a low-cost point and
to that extent, this prototype delivers.
E. Advantages
• Portable
• Can be fixed onto the glasses used by visually
impaired persons
• Works offline, thus no dependency on internet
connectivity
• Cost effective
F. Scope for further improvement
• Capabilities of a cloud-based offering like Google
Cloud Vision can be made use of in exponentially
improving the image classification capabilities of the
glasses.
• The size of the attachment can be brought down, and
the weight of the attachment can be reduced as well.
• Better camera sensors can be used to allow higher
resolution images to be processed.
VI. CONCLUSION
With advancements in technologies like Machine Learning
and IoT becoming more accessible to the masses, there is a
strong case to be made for the benefits of these to reach the
vulnerable minority groups. The authors of this paper believe
that the tools and products used by the members of one such
minority groups, the visually impaired, can be augmented
with technologies like IoT and machine learning to vastly
6
 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE)

improve their experience of interacting with the environment [9] www.cytron.io. TTP223 Capacitive Touch Sensor Module [Image].
Retrieved from
around them. The three products discussed in this paper were https://www.cytron.io/image/cache/catalog/products/SN-TTP223-
mere proof of concepts that such products can be developed MOD/SN-TTP223-MOD%20(5)-0-1-1-100x100.jpg
and can make a huge difference in the lives of their users. [10] www.piborg.org. Ultrasonic Distance Sensor (HC-SR04) [Image].
Retrieved from
VII. REFERENCES https://www.piborg.org/image/cache/catalog/freeburn/BURN-
0019/DSC_0245-747x569.jpg
[1] World Health Organisation, 2010, Global Data on Visual Impairments
2010 [11] www.jameco.com. Raspberry Pi Pinout Diagram | Circuit Notes
[2] Mari Carmen Domingo, March 2012, An overview of the Internet of [Image]. Retrieved from
Things for people with disabilities, Journal of Network and Computer https://www.jameco.com/Jameco/workshop/circuitnotes/raspberry_pi
Applications 35(2):584-596 _circuit_note_fig2.jpg
[3] Arvind R. Tamboli, Avinash N. Suryawanshi, Aman D. Bohra, Dipak [12] components101.com. 1N4004 Rectifier Diode [Image]. Retrieved from
D. Pal, Jagdish Y. Kapadnis, July 2017, IoT based smart stick for blind https://components101.com/sites/default/files/inline-images/1N4004-
people, IJARIIE-ISSN(O)-2395-4396 DO41-Dimensions.png
[4] Jee-Eun Kim, Masahiro Bessho, Noboru Koshizuka, Ken [13] Adafruit. Medium Push-Pull Solenoid - 5V or 6V [Image]. Retrieved
from https://cdn-shop.adafruit.com/145x109/3992-01.jpg
Sakamura,July 2017, Mobile applications for assisting mobility for the
visually impaired using IoT infrastructure, TRON Symposium [14] Element14. RPI CAMERA BOARD - RASPBERRY PI CAMERA
(TRONSHOW) BOARD, 5MP [Image]. Retrieved from
[5] Internet of Things. (n.d.). In Wikipedia. Retrieved on October 1, 2019, https://in.element14.com/productimages/standard/en_GB/2302279-
from https://en.wikipedia.org/wiki/Internet_of_things 40.jpg
[15] Ajeet Ram Pathaka, Manjusha Pandey, Siddharth Rautaray, June 2018,
[6] components101.com. (2018). ATMega328P Microcontroller [Image].
Application of Deep Learning for Object Detection, International
Retrieved from
Conference on Computational Intelligence and Data Science, Procedia
https://components101.com/sites/default/files/component_pin/ATMeg
Computer Science 132 (2018) 1706–1717
a328P-Pinout.png
[16] Prabha, Lakshmi & Ej, Sanjanamenon & Samuel, Christina & Ramesh,
[7] Astvatronics. SIM900A GSM MODULE [Image]. Retrieved from
Ganga. (2018). Low cost real time braille keypad for communication.
http://astvatronics.in/image/cache/catalog/WIRELESS/SIM900A/1-
700x850.jpg International Journal of Engineering & Technology. 7. 77.
10.14419/ijet.v7i2.8.10334.
[8] create.arduino.cc. USE a BUZZER MODULE (PIEZO SPEAKER)
USING ARDUINO UNO [Image]. Retrieved from [17] Ericsson, Ericsson Mobility Report June 2019, Retrieved from
https://hackster.imgix.net/uploads/attachments/495698/FII0D11IAP6 https://www.ericsson.com/49d1d9/assets/local/mobility-
UG5A.png?auto=compress%2Cformat&w=900&h=675&fit=min report/documents/2019/ericsson-mobility-report-june-2019.pdf