International Journal of Advanced Science and Technology

Vol. 29, No. 9s, (2020), pp. 3115-3123

A Novel Based Intelligent Spectacles For Visually Impaired

Sandhya. B1, Suraj. R2, Tamilselvan. A3, Sathishkumar. P4

1
Assistant Professor , Department of Information Technology, Easwari Engineering College,
Chennai, India
2,3,4
student , Department of Information Technology, Easwari Engineering College, Chennai,
India
1
Email: sandhiya5705@gmail.com

Abstract
Visually challenged people face a lot of difficulties in their routine life; they depend on others for all
assisting them in managing their basic tasks. There are a lot of technical solutions working in the same
technology to help the visually challenged people. This project aims at creating an intelligent spectacle
for blind people that help them roam independently in society with higher confidence. An intelligent
spectacle will be built using Artificial Intelligence technology. This device will help the user identify the
approaching object along with position and will notify them. A deep analysis of the object detection
algorithms with respect to software speed, accuracy and efficiency will be provided and the best
algorithm is implemented in this project.

Keywords: Bounding box, Feature extraction, MobiNet, Single Shot Detector, Feature detection and
Recognition

1 INTRODUCTION

Object Detection is a new methodology getting advanced in the field of Deep learning. It is the method
of detecting instances of objects in the given video or image frame. The frame can be processed from
both real time videos or local images or videos. Object detection has a lot of real time applications when
combined with other image processing techniques. The variance in applications includes security,
logistics, healthcare, marketing, human resources, anomaly detection etc.
Visually impaired people around many countries shows interest among various kit like instruments in
both Engineering technology and Retailing purpose. The Mechatronic tool can solve many of the
problems faced by the sightlessness to few extents by providing many information of the neighboring of
the visually impaired people. Thus the tool can be proposed by some deep learning algorithm, Raspberry
pi and computer vision glass.

1.1 RELATED WORK

Various processes are carried out with many features implanted into it in the field of wearable assistive
technology. The [1] author discusses the design of two smart devices namely a smart glass and a smart
show and the integration of both. Smart glasses consist of an object detection module which works on a
neural network framework namely tensor flow. The labels in the detected bounding boxes will be
converted to Braille script. Labels will be read out using Google's tesseract API. A smart shoe integrated
with this system, aims at protecting the users from accidents. The shoe consists of a sensor built on an
arduino board along with an IR sensor to detect minimal distance from an approaching object and
intimate the user about the same.
The main idea of research implementation in paper [2] is focused on object detection for detecting live
surroundings of the blind user. This object detection model is implemented using a neural network
algorithm called MobileNet Single Shot Detector with a reasonable frame processing speed. The COCO
model is used to load the initial dataset. A microprocessor named Raspberry pi will take care of all these

ISSN: 2005-4238 IJAST 3115

Copyright ⓒ 2020 SERSC
 International Journal of Advanced Science and Technology
Vol. 29, No. 9s, (2020), pp. 3115-3123

processing, along with a pi camera for taking video feed input. An ultrasonic sensor for calculating
distance along with a GSM model for live location tracking is used. This proposed system does not
concentrate much on informing the user about the environment.
Another research paper [3] aims at developing a smart bio chip and placing it in the retina of the eye to
overcome the problem of blindness. The biochip consists of two modules namely recognition and
auditory system and a face recognition system. The recognition and auditory system will detect the live
environment and will inform the user about the same. The face recognition system will recognize known
faces and will inform the user. This method ensures better confidence to the user than the other system,
but may find difficulty in getting modeled as a smart processing bio chip.
A convolution neural network based paper[4] ,concentrates more on implementing a face detection and
recognition system. The model will be trained to detect faces by learning the common features such as
eyes, nose, eyebrows, lips using tensor flow. A face recognition system to recognize the known and pre
trained faces will make the model more customizable when used. The output of this paper has been
proven to give more accuracy and robustness in a complex environment.
In paper[5], an affordable solution for the blind people is provided. This is similar to our idea as the
project uses Raspberry pi for image processing. The image processing module precisely, detects objects
and provides the output to the users in the form of vibrating motors. Here the training stage has been
accomplished off-line while the on-line stage classifies images in 1.12 seconds on average. A precision
scale of 79% was achieved in this project.

1.2 WORKING PRINCIPLE

Here the smart vision glass for visually impaired aid model comprises of the apparatuses like Raspberry
pi 3B, bone conduction headset, pinhole HD camera, ultrasonic sensor and android phone. The features
like, GSM, GPS, Wi-Fi are accessed from android phone. All the components of the prototype are
connected using Bluetooth / Wi-Fi feature to provide a feedback without any postponement. Here the
Raspberry pi 3B acts as a primary part of our project.

Fig.1 Working principle of Smart glass prototype.

In our project the smart vision spectacles with pinhole camera will communicate Raspberry pi over Wi-Fi
and provides the live video feedstuff, using that video the Raspberry pi can train each and every structure.
Ultrasonic sensor senses the distance feature information of the obstacles and that will be sent to
raspberry pi. With the help of bone conduction headset, microcontroller sends the warning and response
to the visually impaired person, so that they can act consequently (Figure.1). The overall work flow is
mentioned in the below figure 2.

ISSN: 2005-4238 IJAST 3116

Copyright ⓒ 2020 SERSC
 International Journal of Advanced Science and Technology
Vol. 29, No. 9s, (2020), pp. 3115-3123

Fig.2 Block diagram of our system.

2. PROPOSED WORK

Our aid mainly consists of three important components namely: Ultrasonic sensor, Vision glass embedded
with Raspberry pi 3 B and HD Camera, Bone conduction headset.

2.1 ULTRASONIC SENSOR

The main problem faced by the visually impaired people is object distance detection near him. Our work
rectify this with many number of ultrasonic distance sensor devices, with that it can provide real time
feedback to him also give alert in case of any obstacle is nearby.
Ultrasonic distance sensors works on the standard of reflection of sound waves and distance. To calculate
the distance of the sound travel parameter will be used.

Distance (d) = (Time(t) x SpeedOfSound)/2.

Here ‘2’ is used because the sound will travel front and back.

2.2 VISION GLASS EMBEDDED WITH RASPBERRY PI 3B

The vision glass comprises mainly of two hardware apparatuses, and the components are Rasperry pi 3 B
and pinhole Camera. Here the pinhole camera acts as a input sensor which gets the live video feed of the
neighboring environments and sends the feedback to the Raspberry pi, using deep algorithms all the
surrounding objects are detected and recognized. For detection and recognition of the items deep learning
with MobileNets and SSD (Single Shot Detector) is used.

ISSN: 2005-4238 IJAST 3117

Copyright ⓒ 2020 SERSC
 International Journal of Advanced Science and Technology
Vol. 29, No. 9s, (2020), pp. 3115-3123

2.3 BONE CONDUCTION HEADSET

Bone conduction headset is integrated with the vision glass embedded with Raspberry pi 3B.The signals
or alerts received by the Raspberry pi is communicated to the visually impaired persons for their
assistance like navigation and obstacle detection.

Fig.3 Object detection and recognition algorithm graph.

ISSN: 2005-4238 IJAST 3118

Copyright ⓒ 2020 SERSC
 International Journal of Advanced Science and Technology
Vol. 29, No. 9s, (2020), pp. 3115-3123

3. HARDWARE AND SOFTWARE SPECIFICATION

 Raspberry Pi 3B+
 A headset
 Graphical Processing Unit
 5V Battery
 A sample spectacle
A raspberry pi camera will be attached to the board. This camera will capture real time video and will
give it to the software modules for further processing.

Fig 4. Raspberry pi 3B with attached pi camera

Specification Raspberry Pi 3 B

CPU ARM Cortex-A53 TYPE 1.2GHz SPEED

RAM 1GB SRAM – SIZE

Wi-Fi 2.4GIGAHERTZ

Ethernet 10/100 Mbps – SPEED

Bluetooth 4.2 VERSION

Table 1. Specification of the board.

3.1 SOFTWARE SPECIFICATION

 Rasbian Buster Operating System
 Python 3.7
Here the libraries such as Numpy, Imutils, Dlib, ComputerVision are used. The software model is
embedded in a microprocessor named Raspberry pi, which is cost effective. Our model provides credit-
card dimensioned system that is inserted to a computer monitor, and utilizes a regular keyboard and mouse.
It is a potential small tool that assists people of all ages to explore computing, and to gain knowledge how
to code in programming languages like Scratch and Python.

ISSN: 2005-4238 IJAST 3119

Copyright ⓒ 2020 SERSC
 International Journal of Advanced Science and Technology
Vol. 29, No. 9s, (2020), pp. 3115-3123

3.2 DATASET DETAILS

The common objects in context are commonly called as the COCO dataset. It is an open source
dataset which trains the large scale object detection, segmentation and capturing dataset. The frame
features of the captured dataset are as follows; Object Segmentation, Object Recognition, Super pixel
stuff, 500K images, 1 million object instances, 90 object frame categories, 101 stuff categories, 8
captions per picture, 300,000 people with points , 90,000 training frames, 40,000 validation picture.
Frames are formatted in JSON. It is a collection of “info”, “licenses”, “images”, “annotations”,
“categories”, and “segment info”.

Fig. 5 Sample image of coco dataset

4 EXPERIMENTATION RESULTS AND SCREENSHOTS

Here the Table 2 represents various test cases with scenario, expected output, status, and secondary
considerations. TC ID is the test case ID. Scenario is the environment in which the considered test case
will act and produce the output. Secondary considerations include the parameter which also needs to be
considered for the given test case. Status marks the success or the failure of the scenario.

TC ID SCENARIO SECONDARY EXPECTED OUTPUT STATUS

CONSIDERATIONS

Instantiation of software
TC01 Camera switching on Not Applicable Pass
module

Speaking the name of the

TC02 Obstacle in front Camera turned on Pass
obstacle

ISSN: 2005-4238 IJAST 3120

Copyright ⓒ 2020 SERSC
 International Journal of Advanced Science and Technology
Vol. 29, No. 9s, (2020), pp. 3115-3123

Speaking the name and

TC03 Obstacle in front Position considerations Pass
position of the obstacle

For a range of
TC04 Obstacle in front Range of obstacle detection Pass
5-10 meters

Does not properly detect the The software module enters

TC05 Obstacle in the side Fail
object a deadlock

Connection between glass and

Headset working and Speaking of the output with
TC06 headset coordinated with the Pass
positioning proper clarity
raspberry pi

TC07 Camera tilting Adjustment of glasses Proper image processing Pass

User is able to hear

Headset that causes very less
TC08 Speaker module surrounding sound along Pass
damage to ear
with the output spoken

Speaker and glass Very comfortable for the

TC09 End user level comfort Pass
integrated users

Support high end image

TC10 Raspberry pi GPU level hardware support Pass
processing

Clarity of the camera for feeding

Decent and manageable level
TC11 Raspberry pi camera input to get into high end Pass
quality
processing

TABLE.2 Test Cases of our Work.

Our proposed work software module screen shots and our proposed working model are stated below:

ISSN: 2005-4238 IJAST 3121

Copyright ⓒ 2020 SERSC
 International Journal of Advanced Science and Technology
Vol. 29, No. 9s, (2020), pp. 3115-3123

Fig.6 Software module output.

Fig.7 Our developed model.

5. ADVANTAGES OF PROPOSED SYSTEM

From the survey every 150 000 children in India have severe visual impairment and approximately 20,000
are in blind school. There are few glasses available in market for blind but the algorithms used for object
detection are not scalable and efficient in real time. This is much efficient and affordable. The accurate
efficiency of our software integrated with a voice assistance makes this project more productive.

6. CONCLUSION AND FUTURE WORK

Our paper tries to bring smart vision glass can be preferred better than the devices available in the
business market for the purpose of cost effectiveness and working. From the produced result our work is

ISSN: 2005-4238 IJAST 3122

Copyright ⓒ 2020 SERSC
 International Journal of Advanced Science and Technology
Vol. 29, No. 9s, (2020), pp. 3115-3123

shown that it can be converted to product with some minor modifications in the design. The device will
be useful for the blind persons for the need of object detection and navigation.
Our designed product was surveyed on different basis and at different levels. The future improvements of
the designed product are briefly discussed below:
Real time data storage and processing in cloud environment. Since there will be enormous amount of data
involved, a cloud environment can be integrated for effective processing of the data and software;
Installation of GPS module for more location related information. Location based accuracy will improve
the efficiency of the product and the user; Notification based system, integrated with IOT for enhanced
safety. Any safety or emergency calls or notifications can be done for more user friendliness of the
product. A GPU- software based integration for more efficient and fast real time image processing will
provide more accurate results.
REFERENCES:

1.Smart Assistive Navigation Devices for Visually Impaired People, 2019 IEEE 4th International
Conference on Computer and Communication Systems.
2.Intelligent Smart Glass for Visually Impaired Using Deep Learning Machine Vision Techniques
and Robot Operating System (ROS) Springer International Publishing AG, part of Springer
Nature 2019.
3. J.-H. Kim et al. (eds.), Robot Intelligence Technology and Applications 5, Advances in Intelligent
Systems and Computing 751, https://doi.org/10.1007/978-3-319-78452-6_10, Deep Learning
Assisted Smart Glasses as Educational Aid for Visually Challenged Students, 978-1-7281-2882-
5/19/$31.00 ©2019 IEEE.
4. Smart Guiding Glasses for Visually Impaired People in Indoor Environment. IEEE Transactions
on Consumer Electronics, Vol. 63, No. 3, August 2017.
5. Dalal Khalid Aliasem, Michael Heeney, Armando PesentiGritti, Franco Raimondi, On-the-fly
Image Classification to Help Blind People, IEEE Interntional Conference on Intelligent
Environments, 2016.
6. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing
Ren, Kaiming He, Ross Girshick, and Jian Sun.
7. Smart Guiding Glasses for Visually Impaired People in Indoor Environment. IEEE Transactions
on Consumer Electronics, Vol. 63, No. 3, August 2017
8. A Wearable Mobility Aid for the Visually Impaired based on embedded 3D Vision and Deep
Learning. IEEE Workshop on ICT solutions for eHealth 2016 978-1-5090-0679-3/16/$31.00
©2016 IEEE
9. A Convolutional Neural Network based on TensorFlow for Face Recognition. 978-1-4673- 8979-
2/17/$31.00 ©2017 IEEE
10. A Wearable Portable Electronic Travel Aid for Blind. International Conference on Electrical and
Electronics and Optimization Techniques (ICEEOT) @ 2016 IEEE.
11. Visual impairment and blindness Fact Sheet N°282". August 2014. Retrieved 23 May 2015
12. Syed Tehzeeb Alam, Sonal Shrivastava - Smart Device for Blind people, International Journal of
Engineering Research and Technology (IJERT), ISSN: 2278-0181, Vol 4, Issue 03, March 2015.
13. SSD: Single Shot MultiBox Detector Wei Liu1 , Dragomir Anguelov2 , Dumitru Erhan3 ,
Christian Szegedy3 , Scott Reed4 , Cheng-Yang Fu1 , Alexander C. Berg1 1UNC Chapel Hill
2Zoox Inc. 3Google Inc. 4University of Michigan, Ann-Arb

ISSN: 2005-4238 IJAST 3123

Copyright ⓒ 2020 SERSC