Proceedings of the Fourth International Conference on Smart Systems and Inventive Technology (ICSSIT-2022)

IEEE Xplore Part Number: CFP22P17-ART; ISBN: 978-1-6654-0118-0

Virtual Smart Glass for Blind using Object Detection

V.P.Gowtham Raj1 , E.Esakki Vigneswaran2 , M.Deshnaa 3 ,K. RajPrasanth4
Assistant Professor 2,, Student 1,3,4
Department of Electronics and Communication Engineering
Sri Ramakrishna Engineering College,
Coimbatore, India
esakkivignesh@srec.ac.in
2022 4th International Conference on Smart Systems and Inventive Technology (ICSSIT) | 978-1-6654-0118-0/22/$31.00 ©2022 IEEE | DOI: 10.1109/ICSSIT53264.2022.9716412

Abstract— One among the prevailing
problems is that the vision disabilities are rapidly
increasing. The disability of the person with vision
difficulty is the inability to see and recognize people or
objects. Therefore, the visually impaired person has to
walk depending on a stick or dog or another person. The
main motive of this experiment is to help the blind by
modelling a device that make them walk like normal
people and to overcome their disability. This device, the
virtual smart glass assists them in their ways without the
need of human help and help them walk independently.
The device is installed on the glass frame and these
glasses help to figure out the surroundings. The object
detection technology is given as input to the
microcontroller, to find out if there is any obstacle in
front of it. If there is an obstacle, the device will produce
an audio form for alerting the user.
Keywords- microcontroller, GPU processor, numPy, OpenCV,
matlab, yolo, API, XML, TTS, Co-create.
I. INTRODUCTION
In the recent days, many people suffer from different
diseases or are disabled. NCBI tells that 1.5% of the
population in Saudi are blind and 7.8% have vision
difficulties problem. They require some help to make their
life easier and comfortable. A new technology needed to be
introduced to help them. Thus, the motive Smart Glass is to
aid them walk independently and freely. This system
consists of wearable smart glass with object detection
using neural network methodology to identify objects like
person, rupees, QR, car , bus etc. Individual image
processing description labels are used to detect the objects
and are converted into voice command. Therefore, smart
glasses are a device which alters the wearer’s vision, where
one is physically located or where he/she looks.
There are several ways for differentiating the visual
information which a wearer perceives. The human brain
make vision seem very much easy. The brain need not work
work hard to identify, apart from human identification,
reading a sign, or recognizing a human's face. These are
complicated to solve using a computer. They only seem
easier just because our brains are good at understanding and
recognizing the image. Image recognition and object
detection is a software which helps in identification of
objects, places, people, writing and actions in images.
Computers uses computer vision technologies in
combination with a camera and artificial intelligence
software to achieve image recognition. Software for image
recognition requires deep machine learning. In
convolutional neural networks , processors performance is
best but requires massive amount of power.
II. RELATED WORK
In recent times, the number of vision difficulties in the
world has been increased, which led to the innovation in
such area to serve the blind community.
Mayuresh Banne (2020)[5], proposed a system
which consists of a box like structure which has a portable
camera and a processing system which will process the
input image. The pictures are captured with a camera
mounted device with real-time image recognition on
existing object detection methodology. V.Balaji (2020) [9],
explained the idea and methodology and architecture of the
system. He implemented the system and obtained immediate
results and analyzed the performance of the tools . On
comparing these papers , the virtual smart glass can be
implemented in any device that is easily portable for the
visually impaired people to detect objects from a particular
distance.
Bhumika Gupta (2017) [14], briefly explained that, in
several purposes, image processing and object detection can
be used including video surveillance and also its retrieval.
In the study, various basic concepts including image
processing were used in object detection while making use
of the library file called OpenCV library of python 2.7.
which improves the efficiency and accuracy of object
detection and can be brought into picture. Lijun Yu (2018)
[10], proposed that the experimental result of the device
shows that the object recognition system has major
significant effects on the image detection and recognition of
single-target motion objects and has good accurate output
values. This concept has good engineering application
prospect.
The above provide a brief explanation on the object
recognition steps by various tools, the uniqueness of the

978-1-6654-0118-0/22/$31.00 ©2022 IEEE 1419

Authorized licensed use limited to: Higher College of Technology. Downloaded on July 16,2022 at 06:14:47 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Fourth International Conference on Smart Systems and Inventive Technology (ICSSIT-2022)
IEEE Xplore Part Number: CFP22P17-ART; ISBN: 978-1-6654-0118-0

audio conversion and 3D printed glass frame that have been
implemented in order to provide an appropriate solution.
A. Interpreting images and determining each object in the
images through tensorflow
The camera tracks any object using Image processing
algorith m. This leads the device to receive informat ion on
the user’s surroundings. The smart glasses ’ unique
computing device detects more intensive objects.
B. Conversion from Voice to Audio
After recognizing each object, they are labelled with text,
then using Speech Synthesis the Text to voice conversion
technique is performed and can be heard using the
headphone.
C. Prerequisisites
The prerequisites are processor devices with mini-GPU and
nano cameras in g lasses frames. In this s mart glass, software
solution plays a wide role using image processing.
The mach ine learn ing algorith m set is trained to make the
glass able to recognize objects of the input images. The
training set in recognition phase contains the data about
objects known to the computer through the training
algorith m (examp le: YOLO). The classification phase uses
the training set to recognize the objects read fro m the
camera’s images.
III. PROPOSED WORK
3. OpenCV
4. NumPy 1.14
The pre-trained model of python OpenCV, recognizes
objects using camera. It works based on real-time object
detection with access of minicamera in an efficient manner
for each frame. The minicamera is pre-installed in the
system. The live stream can access with minicam output.
The video frame detects the objects depending on the speed
of computer’s CPU or GPU resources.
Anaconda command prompt can be used, but make sure that
the anaconda commands prompt without changing the
directories of the path.
D. Object Recognition
The Object Recognition Workflow of object detected model
is created to avoid complexities in system. The TensorFlow
and open cv are models with dataset. The deep learning
frameworks use library for real time computer neural
network in many real time applications like medical
recognition, face detection, object detection.
E. Speech Synthesis
The process of synthesis ing text to voice comment is
calibrated using Google’s Text to Speech. API commonly
used is the GTTS API which computes system speech
synthesizer and is implemented in python library files. Thus
the system converts the speech format into traditional
speech text.

A. Image Input
The system detection is the task to identify image along with
label bo xes. System visions’ deep learning algorithm
possesses approx. 90 FPS, accurate SSD and Faster-
datasets. Database support for blobs is not universal. Hence
there is no need to use server because the data is fixed,
because the things which the smart glas s captures and
recognizes have particular feature and shape, for examp le, at
home (tables, chairs, doors and so on ) all these things have
same feature in all p laces (in streets, cars, sidewalk, vehicle
and so on). Therefore, the database is fixed on pattern
recognition, that it can be stored in device which run the
application process the eyeglasses.
B. Audio Input Fig. 1. Block Diagram of the Proposed Work

The result of the detected objects and the distance it is in,

are converted to text. Then using the prepared audio library
for conversion of text to voices , the voice output is heard
through the headphones.
C. Tools Utlized
Stereo views pre-trained model along with the necessary
packages has about 30 trained datasets useful system to
detect the object. Framework model used to create deep
learning networks solves the object detection problem.
Some necessary Tools Utilized are:
1. Python 3.5
2. Caffe model framework
From the figure 1, Object detection images and videos
inputted to the high GPU (raspberry pi 3) is a quadcore
processor and has roughly 50-60 percent better performance
in 32-bit mode and 10x faster than the original single-core
Raspberry Pi. Also, the camera placed in front of the camera
slot, detects the images using the detection algorithm and
glasses can capture images using the camera to recognize
images and determine each object present in front by
computer vision technique & pattern recognition system.
The system can calculate the distance between the person
and each specific object and convert the information to
voice commands which is to be understood by the blind user

978-1-6654-0118-0/22/$31.00 ©2022 IEEE 1420

Authorized licensed use limited to: Higher College of Technology. Downloaded on July 16,2022 at 06:14:47 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Fourth International Conference on Smart Systems and Inventive Technology (ICSSIT-2022)
IEEE Xplore Part Number: CFP22P17-ART; ISBN: 978-1-6654-0118-0
via earphones and intimates them if they are very near to the
object.
The proposed system is well trained and has a unique
novelty in providing good accurate detection results in a
simplified manner. The input of the virtual glass is taken
from the output provided by the strip camera and provides
subsequent feedback to the person through the earphones
and thus it can help the blind while walking alone in an
outdoor environment. This is a form of computing device,
used and worn in head.
When the person is wearing the glass moves from his or her
position, the camera adjusts such that it gets clear vision
according to the position and orientation. Thus, the glasses
are smart and unique compared to other computing devices
of such kind. They also possess some unique features such
as enabling new applications which is not easily adaptable
compared to others.
IV. METHODOLOGY
Initially the image is processed by several
methodologies like detection, extraction, classification and
finally the objects are being recognized. Binary images are
processed in the form of pixels, then they are extracted and
applied for real time binary image processing.
Fig.2.Object Recognition M ethodology
The above fig. explains the methodology of object
identification and recognition techniques incorporated in the
work.
A. Feature Detection
One of the methodologies employed in the system
is called as feature detection, where the computing
abstractions of the specific set of image information occurs
and detection by making decisions and recognition at every
978-1-6654-0118-0/22/$31.00 ©2022 IEEE
Authorized licensed use limited to: Higher College of Technology. Downloaded on July 16,2022 at 06:14:47 UTC from IEEE Xplore. Restrictions apply.
point of the image whether there is feature of the image set
or not. The resulting features are considered as the subsets
of the image domain. Feature detection are mostly applied
by four ways such as edges, corner (interest points), blobs
(regions of interest point) and ridges .
B. Edges
The boundary between two images is considered to
be the edge. Generally, they should include junctions and
are mostly arbitrary in shape. During the application, the
edges are usually considered as set of points within the
particular image having a robust gradient magnitude. The
Gradient points in the image are collectively chained
together inorder to make the image a more complete
description of a foothold.
C. Interest points
The corner or interest point is a methodology
where the terms corners are referred to point-like features in
a picture (fig.3). They have a two dimensional (2D)
structure. The term "Corner" was first performed by the
edge detection and then the edges are individually analyzed
to find the rapid changes in direction or corners.
However. it was noticed that the corners were also
detected on individual parts of the image which weren't
corners in the sense.
Fig.3 Corner Interest points
D. Feature Extraction
The feature extraction is the method of extracting
the useful and crucial information from the images. In this
step, the image is transformed from input file to a group of
specific features. Features are nothing but distinctive
properties of exclusive input patterns that help in
differentiating between the categories of input by deriving
new features from the existing initial features inorder to
scale back the cost and work of the feature measurement and
to extend the efficiency of the image classifier with a higher
order of classification accuracy. While extraction, if the
features extracted are carefully chosen, the classification
phase are reportedly proved to produce better results.
Several data analysis software packages exist in
order to supply and support the feature extraction and also
the dimension reduction. Therefore, for this process the
numerical programming environments that are commonly
1421
 Proceedings of the Fourth International Conference on Smart Systems and Inventive Technology (ICSSIT-2022)
IEEE Xplore Part Number: CFP22P17-ART; ISBN: 978-1-6654-0118-0
used such as MATLAB and C language, supports a number
of simpler feature extraction techniques. The fig.4 explains
the feature extraction techniques.
EXTRACTED
DATA
Fig.4. Feature Extraction
E. Classification
Image classification is the methodology of
recognizing the images using the feature vector which is the
output of the feature extraction phase supported to the
training set given to this program. Spectral signatures are
obtained from the training samples using the supervised
classification method employed in classification of the
picture (fig.5). Training samples can be easily created to
represent the classes that have to be extracted, using the
image classification toolbar.
Fig.5. Detection Result
This project uses the Speech Application
Programming Interface or SAPI that allows the use of
speech recognition and speech synthesis within the
respective applications. There are several applications that
use SAPI, such as MS Office, MS Agent and Microsoft
Speech Server. The Speech API generally could be a free
redistributable component. They can be transferred with
Windows application that uses speech technology. Many
other versions of the speech recognition and synthesis
engines are also available to freely redistribute.
F. Text to Speech Conversion (TTS)
The text to speech conversion is built up though
nonfunctional COM framework. Various new steps are
available for every new function demonstrating the use
XML tags to modify speech in sample. This step is to create
978-1-6654-0118-0/22/$31.00 ©2022 IEEE
Authorized licensed use limited to: Higher College of Technology. Downloaded on July 16,2022 at 06:14:47 UTC from IEEE Xplore. Restrictions apply.
the voice of assistance by simply declaring the instance and
using CoCreate Instance where SAPI uses intelligent
defaults. This generally requires a minimal amount of
initialization where the voice can be used immediately.
In this, the defaults are located on top of things
Panel in the speech properties which includes a variety of
voices and several languages including the native ones such
as English, Japanese, etc. Speech properties can be
programmatically set to default values. The voices of the
speech could also be modified using some sort of methods
such as using XML commands on to the stream. In this
situation, a relative rating will lower the pitch of the voice.
V. RESULT ANALYSIS
Fig.6. Developed Hardware Setup
Fig.7. Hardware Model
Fig.6 represents the hardware setup of the project, where the
outlook skeleton of the frame of the glass has been designed
using the 3D printed frame in which the camera has been set
up in order to recognize the objects detected in front of the
model. This is inbuilt with the earphones on the side which
gives the speech output of the detected objects through TTS
from the earphones.
The object recognition is done by extracting features
from each object and comparing them with the objects in the
training set. After segmentation in (2D) and triangulation
(3D), each object is connected to its Z dimension values to
calculate the space between the blind man and every object
within the image. Results of distance measurement and
recognition are converted to text then to voice that helps the
blind man to beat obstacles.
The virtual smart glass (fig.7) is a device used for
guidance and just guides the blind man the way to traverse.
1422
 Proceedings of the Fourth International Conference on Smart Systems and Inventive Technology (ICSSIT-2022)
IEEE Xplore Part Number: CFP22P17-ART; ISBN: 978-1-6654-0118-0
This device guarantees that if a person with vision difficulty
uses this, he/she can walk almost like a normal person.
Since the system requires some amount of time to process
the detected values and guide them, they need to be walking
at a relatively less speed.
TABLE I. COMPARISON METRICS
Metrics YOLO YOLO V3
Precision % 83.12 93.27
Recall % 81.04 92.11
Inference Time 5.0 2.6
The table-1 shows the comparison metrics of the YOLO and
YOLO V3. From this table, it is evident that, YOLO V3
produces very high accuracy. YOLO V3 is employed in the
proposed project and is fed into the system for producing
the sound signal of the corresponding word.
Fig:8. Detected objects
It is seen that the device consisting of a camera on the
frame placed in a camera slot, is an individual computer
processing unit which is accurate enough to urge slot in the
pocket and transparent and thus the displays on the
eyepieces are accessible to the software that gives the
images of the objects. This device is suitable for completely
blind people and also it is recommended just for people with
low vision, nyctalopia or night blindness.
VI CONCLUSION
The main objective of this paper is explained
briefly which is the need of voice assistant system for the
blind people all over the globe. The " Virtual Smart Glass for
Blind" is a practically feasible device which can be easily
worn and used by any person. The above figure (fig:8)
shows the detected images of the objects in front of the
camera and these detected images are then converted to
audio and fed as audio. Thus , the smart glass will be a major
help for the blind society in guiding them.
REFERENCES
[1] Sungheetha, Akey, and Rajesh Sharma. “3D Image
Processing using Machine Learning based Input Processing
for Man-Machine Interaction.” Journal of Innovative Image
Processing (JIIP) 3, no. 01 (2021): 1-6.
978-1-6654-0118-0/22/$31.00 ©2022 IEEE
Authorized licensed use limited to: Higher College of Technology. Downloaded on July 16,2022 at 06:14:47 UTC from IEEE Xplore. Restrictions apply.
[2] Dhaya, R. “Analysis of Adaptive Image Retrieval by
TransitionKalman Filter approach based on Intensity
Parameter.” Journal of Innovative Image Processing (JIIP)3,
no. 01 (2021): 7-20.
[3] Manoharan, J. Samuel. “Capsule Network Algorithm for
Performance Optimization of Text Classification.” Journal
of Soft Computing Paradigm (JSCP) 3, no. 01 (2021): 1-9.
[4] Tesfamikael, HadishHabte, Adam Fray, Israel
Mengsteab, AdonaySemere, and ZebibAmanuel.
“Simulation of Eye Tracking Control based Electric
Wheelchair Construction by Image Segmentation
Algorithm.” Journal of Innovative Image Processing (JIIP)
3, no. 01 (2021): 21-35.
[5] MayureshBanne, Rahul Vhatkar, RuchitaTatkare(2020),
Object Detection and Translation for Blind People Using
Deep Learning, International Research Journal of
Engineering and Technology (IRJET), Volume: 07 Issue:
03.
[6] M. Murali, Shreya Sharma, Neel Nagansure(2020),
Reader and Object Detector for Blind, International
Conference on Communication and Signal Processing, July
28 - 30, 2020, India.
[7] Qi-Chao Mao,Hong-Mei Sun,Yan-Bo Liu,Rui-Sheng
Jia. (2019) “Mini-YOLOv3:Real-Time Object Detector for
Embedded Applications” IEEE Access 7 .
[8]Mohammad Marufur Rahman Milon Islam
Shishir Ahmmed , Saeed Anwar Khan(2020), Obstacle
and Fall Detection to Guide the Visually Impaired People
with Real Time Monitoring, SN Computer Science (2020)
1:219
[9] V.Balaji, S. KanagaSuba Raja , C.J. Raman ,
S.Priyadarshini , S.Priyanka5 , S.P.Salaikamalathai,(2020),
real time object detector for visual impaired using open cv,
European Journal of Molecular & Clinical Medicine ISSN
2515-8260 Volume 7, Issue 4, 2020.
[10]Lijun Yu, Weijie Sun, HuiWang,Qiang Wang and
Chaoda Liu(2018), The Design of Single Moving Object
Detection and Recognition System Based on OpenCV,
Proceedings of 2018 IEEE International Conference on
Mechatronics and Automation August 5 - 8, Changchun,
China.
[11] HemalathaVadlamudi,(2020), Evaluation of Object
Tracking System using Open-CV In Python, International
Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181, Vol. 9 Issue 09.
[12] M. Ramalingeswara Rao, Sai Sri Harsha,
AlekhyaSaiSree, Srikanth, SaiChaitanya(2020),raspberry pi
based accessory for blind people, 2020 JETIR April 2020,
Volume 7, Issue 4.
[13] Chen LB, Su JP, Chen MC, Chang WJ, Yang CH, Sie
CY (2019). An Implementation of an Intelligent Assistance
System for Visually Impaired/Blind People.
[14] Bhumika Gupta, Ashish Chaube, Ashish Negi,
UmangGoel, UmangGoel,(2017) Study on Object Detection
using Open CV - Python, International Journal of Computer
Applications (0975 – 8887) Volume 162 – No 8.
1423
 Proceedings of the Fourth International Conference on Smart Systems and Inventive Technology (ICSSIT-2022)
IEEE Xplore Part Number: CFP22P17-ART; ISBN: 978-1-6654-0118-0

[15] Wei Fang,LinWang,PeimingRen. (2019) “Tinier-

YOLO:A Real-Time Object Detection Method for
Constrained Environments” IEEE Access 8 .

978-1-6654-0118-0/22/$31.00 ©2022 IEEE 1424

Authorized licensed use limited to: Higher College of Technology. Downloaded on July 16,2022 at 06:14:47 UTC from IEEE Xplore. Restrictions apply.