See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/330959114

Voice Recognition based Intelligent Wheelchair and GPS Tracking System

Conference Paper · February 2019

DOI: 10.1109/ECACE.2019.8679163

CITATIONS READS

24 7,353

3 authors:

Nasrin Aktar Israt Jahan

6 PUBLICATIONS 37 CITATIONS
Linnaeus University
9 PUBLICATIONS 40 CITATIONS
SEE PROFILE
SEE PROFILE

Bijoya Lala
Independent University, Bangladesh
5 PUBLICATIONS 33 CITATIONS

SEE PROFILE

All content following this page was uploaded by Bijoya Lala on 08 February 2019.

The user has requested enhancement of the downloaded file.

2019 International Conference on Electrical, Computer and Communication Engineering (ECCE), 7-9 February,
2019

Voice Recognition based Intelligent Wheelchair and

GPS Tracking System
Nasrin Aktar1, Israt Jahan2, Bijoya Lala3
Department of Electrical and Electronic Engineering
University of Chittagong
Chittagong-4331, Bangladesh
1
aktarnasrin33@gmail.com, 2israt.ononna.002@gmail.com, 3bijoyalala@gmail.com

Abstract— Development of a voice recognition based
intelligent wheelchair system for physically handicapped people
who are unable to drive the wheelchair by hand is represented in
this paper where the patient can operate the wheelchair using
voice commands and the location of patient can be traced using
GPS module in the wheelchair that tracks and sends the
information to smartphone application (app) via Firebase. Voice
module V3 is used to record patient’s voice and recognize that
voice to follow the instructions of the patient. This kit converts
the voice commands to hexadecimal numbers and then the data is
fed to the Wi-Fi module to control the wheelchair. Wi-Fi module
directs the motor driver IC to move the wheels in desired
direction. Motor speed can also be controlled in three stages-low,
medium, high. This system also offers obstacle detection
automatically using IR sensor and a smartphone app has been
developed for the family members of patient to know about the
location of the patient. Use of firebase makes the system fast and
android app offers low cost and user friendly environment than
the conventional GSM based navigation systems. As this system
simultaneously offers voice operated wheelchair, motor speed
control, obstacle detection and GPS tracking of patient using
android app, hopefully it will be a fruitful system for the
handicapped people worldwide.
Keywords— voice recognition; GPS module; smartphone
application; Firebase; Wi-Fi module; speed control; obstacle
detection;
I. INTRODUCTION
Physical disability is increasing rapidly due to ageing,
accidents and diseases like quadriplegics, paralysis and so on.
Report from World Health Organization (WHO) says that,
around 15% of the world population are handicapped [1]. For
this, the use of wheelchair is increasing. But, some people still
can’t operate the wheelchair because of having injury in hand
or some other issue in their hand.
The first automated electric wheelchair was introduced by
George Klein in 1953 for the World War II veterans [2]. Later,
many researchers have proposed and developed many designs
for wheelchair based on image processing, eye movement
signal (EOG) control, tongue control, EMG control and EEG
control and so on. However, many limitations including
complex signal processing, high cost and additional hardware
requirement made these systems not much suitable [3-8].
Other than these, joystick controlled electric wheelchair has
also been developed which is of no use to the users with
paralyzed hand or who have problems in hand. After that,
voice recognition based wheelchair has been proposed as
voice is an easy and popular way of communication. The first
voice recognition based wheelchair was designed and used in
1999 by Siamo University of Alcala, Spain [9]. Some of the
proposed systems later on this basis are explained here:-
M.Senthil Sivakumar [10] proposed a Speech Controlled
Automatic Wheelchair that is an intelligent home navigation
system. It offers the elderly or physically disabled persons to
move inside home using voice command. The wheelchair also
incorporates obstacle avoidance system using ultrasonic
sensor.
Megha Muralidharan [11] proposed a Voice Recognition
Based Intelligent Wheelchair where voice recognition module
is used to recognize patient’s voice in order to drive a
wheelchair. The user can go front, back, right, left using
his/her voice. Two ultrasonic sensors are used for obstacle
depth detection as well as the system incorporates joystick
control using hand movement.
Android Based Closed Loop Speed Control of Dc Motor
Using Voice Recognition via Bluetooth is proposed by B.
Gokul [12]. This system offers voice recognition via mobile
app. This app receives voice from patient which then
transferred to the arduino via Bluetooth. Arduino then directs
the motor driver IC to work accordingly. This system also
offers the advantage of knowing the speed of motor using a
speed sensor. Speed information is displayed in the app.
There is another proposed system, not yet implemented by
Romil Chauhan [13] named Study of Implementation of Voice
Controlled Wheelchair that proposed to use Raspberry Pi as
the processing unit rather than using arduino/microcontroller.
Obstacle detection sensor can be connected to Raspberry Pi. It
will reduce the number of hardware needed like motor driver.
However, it’s a proposed idea only.

978-1-5386-9111-3/19/$31.00 ©2019 IEEE

 Design of an Arduino Based Voice-Controlled Automated independent systems can be operated for any person only the
Wheelchair is proposed by Zannatul Raiyan [14] where a words are needed to be trained in the system [15].
voice module is used to drive the wheelchair through voice
In our system, we have used voice recognition module v3
command. It offers obstacle detection and also offers GSM
which is speaker-dependent device. This module can record
based navigation system to track current location of user and
maximum 80 voice commands, with each 1500 ms [16].
send the information to the people concerned.
Before using this module, the user needs to train his/her voices
These existing systems include voice operated wheelchair so that the device can recognize the voice of the user. We have
and GSM based navigation system, that is not cost effective trained our commands by using Arduino IDE software. Seven
and not user friendly. Moreover, these systems provide commands such as Forward, Reverse, Right, Left, Stop, Low,
extremely slow transportation. Our developed system offers and High have been recorded in our system to achieve our
voice recognition based wheelchair as well as GPS tracking of target. The training can be done in any of the language that is
patient using android app. Voice module is trained with comfortable for the user. The screenshot of our training is
patient’s voice first. The patient only need to speak to the shown in Fig. 2.
microphone of the module and then the converted
hexadecimal numbers are feed to the Wi-Fi module that
controls the wheel direction via motor driver. Motor speed can
also be controlled. Automatic obstacle detection is provided
using IR sensor. Family members of patient can be aware of
patient’s location using android app. Use of firebase makes the
navigation system speedy. Moreover, speed control and
android app make the system user friendly. The system is also
cost effective than the GSM based navigation systems.

II. IMPLEMENTATION DETAILS

The methodology of the prototype is described with proper
diagram.

Fig. 2. Training the commands

As shown in Fig. 1, a microphone is connected with the

Fig. 1. Block diagram of the system
A. Voice Recognition Module
There are two types of voice recognition system one is
speaker-dependent and another one is speaker-independent.
Speaker-dependent systems are operated only by the person
whose voice is trained in the system. Whereas, speaker-
voice recognition module. Microphone is used for collecting
the audio signal of the user and converting that audio signal to
the electrical signal. The electrical signal is then passed to the
voice module. The voice module compares the received
command to the trained command. If the command doesn’t
match with the trained command, the wheelchair won’t
activate.
B. IR Sensor
An Infrared sensor is an electronic device which can
radiate infrared ray. It can be used to sense some aspects of its
surroundings [17]. In our system, we have used this module to ways to proceed this login session and they are via Google,
detect the objects around it. If there is any object in front of Facebook, Twitter or GitHub account. Firebase has several
the wheelchair, the wheelchair will automatically stop. Even if kinds of features such as Authentication, Real-time database,
the user commands to go forward, it won’t work. App indexing, Storage, Messaging, Notification, AdMob,
Crash reporting etc. [20].
C. GPS Technology
GPS is the abbreviation of global positioning system. It
consists of 24 satellites circling around the earth. There are
several kinds of GPS having the same basic function and that
is to track the accurate location of the device. The tracking is
done by the emission of radio signals from the satellites. The
GPS receiver receives the radio signals from at least three
satellites so that it can provide accurate data. It is used in
different fields such as agriculture, astronomy, automated
vehicle, cellular telephony, emergency services, navigation,
telematics and so on [18].
In our system, the GPS is used to track the location of the
handicapped people who use wheelchair for their regular
movement. We have used a GPS module to receive the co-
ordinates from the satellites.

D. Wifi Module
In our system, we have used Wi-Fi module NodeMcu as a
processing unit. As shown in Fig. 1, this module is interfaced
with different input and output devices. Voice recognition
module, IR sensor, GPS module work as input devices while
motor driver module is connected as output devices. We have
programmed using arduino IDE software to achieve our goal.
It is designed to control the direction of the motor as well as to
control the speed of the wheelchair by the person’s voice. We
have also designed to send the location information of the user
which is tracked by the GPS module to the firebase. The flow
chart of the motor direction is shown in Fig. 3. The system is
designed to first check the obstacle. If there is no obstacle, the
command will be accepted. And the wheelchair will run
according to the command.
For the speed control of the motor, pulse width modulation
technique is used. In our prototype, we have only trained two
commands for controlling the speed and they are low and
high. When the user commands, the stage of the pulse width
modulation is set according to the predefined program and is
applied to the motors. The speed is then changed and
controlled depending upon the signal that is applied to it. The
initial speed of the motor is predefined in the code and if there
is no command regarding speed of the vehicle, the motor
rotates with its initial speed. The working procedure of speed
control of the motor is shown in Fig. 4. Fig. 3. Flow chart of motor direction of the wheelchair

Using firebase authentication, developers can define who can

E. Firebase
Firebase, an online server, is a mobile and web app
development platform provided by Google [19]. It saves data
using NoSQL database and as a JSON format [20]. It receives
and transmits data very quickly [21]. It provides service to
integrate with android, iOS or web app [19]. Login process
has to be followed to integrate with them. There are different
access the data and who can’t. Firebase can link more than one
app under one database [21]. In our system, we have designed
firebase to serve as a communication medium between Wi-Fi
module and location tracker app. Wi-Fi module updates the
location information to the firebase continuously. Firebase
then updates the app by sending the location information. We
have also designed firebase in a way that it will store new data
to a specified database replacing the existing data of that
reference.

Fig. 5. Hardware implementation of the system

B. Firebase
The storage of location information in firebase is shown in
Fig. 4. Flow chart of the speed control of the wheelchair Fig. 6.
F. Motor driver Module
To control the motor, motor driver module, L298N has
been used. NodeMcu can’t be directly connected to motors
because it can’t provide sufficient current to drive them.
Motor driver is acted as a current enhancing device. So, it is
inserted between NodeMcu and motors [22].
G. Location Tracker Application
We have developed an android application using android
studio. It is designed so that the family members can track the
location of the user of the wheelchair. It is integrated with
firebase under the same database of the NodeMcu. For the
reception of the co-ordinates information in the app, the user
needs to activate the internet connection. Two buttons are
displayed in the 1st page of the app. One is to show location in
text format and another is to show location in map. By using
the geocoder class in android studio, the location co-ordinates
are converted to the address in text format and it is displayed
in the 2nd page when the user clicks the 1st button. By clicking
another button, map is displayed in the app’s interface. From
that, the family members can also obtain information about the
Fig. 6. Storage of location information in firebase
patient’s whereabouts.
C. Testing of Location Tracker app
III. RESULTS AND DISCUSSONS
To demonstrate our system, the app was installed in a
A. Hardware Implementation smartphone. The screenshots of our working app are shown in
Fig. 7.
Hardware implementation of our prototype is shown in Fig.
5.
 controlled by eye-tracking system” Optica Applicata. 2006, Vol. 36
Issue 2/3, p401412.
[5] Tsui, Chun Sing & Jia, Pei & Gan, John & Hu, Huosheng & Yuan, Kui.
(2008), “EMG-based Hands-Free Wheelchair Control with EOG
Attention Shift Detection”, Proceedings of IEEE International
Conference on Robotics and Biomimetics, 1266 – 1271,
10.1109/ROBIO.2007.4522346.
[6] Guin and B. B. Baishya, “Brain Controlled Wheelchair using
LabVIEW”, Jan. 2013.
[7] M. Mahmud, D. Hawellek, A. Bertoldo, “EEG Based Brain Machine
Interface for Navigation of Robotic Device, Proc. 3rd IEEE/RAS-EMBS
Intl. Conf. Biomed. Robotics & Biomechatronics (BioRob’10), Tokyo,
Japan, 26-29 Sept. 2010, pp. 168-172, doi:
10.1109/BIOROB.2010.5627015.
[8] M.S. Kaiser, Z. Chowdhury, S. Al Mamun, A. Hussain, M. Mahmud, “A
Neuro-Fuzzy Control System Based on Feature Extraction of Surface
(a) Electromyogram Signal for Solar-Powered Wheelchair”, Cogn.
Comput., 8(5): 946-954. doi: 10.1007/s12559-016-9398-4.
[9] R. C. Simpson, “Smart wheelchairs: A literature review,” Journal of
rehabilitation research and development, vol. 42, no. 4, p. 423, 2005.
[10] M.Senthil Sivakumar, Jaykishan Murji, Lightness D Jacob, Frank
Nyange, M.Banupriya, “SPEECH CONTROLLED AUTOMATIC
WHEELCHAIR”, Pan African International Conference on Information
Science, Computing and Telecommunications(2013).
[11] Megha Muralidharan, P.T.Jabir, Vinod Pottakulath, “Voice Recognition
Based Intelligent Wheelchair”, International Journal of Innovative
Research in Science, Engineering and Technology, Volume 5, Special
Issue 5, April 2016.
[12] B.Gokul, K.Karthi, A.Thiyagaseelan, V.Santhosh Kumar, “Android
Based Closed Loop Speed Control of Dc Motor Using Voice
Recognition via Bluetooth”, International Journal of Advanced Research
in Electrical, Electronics and Instrumentation Engineering, Vol. 5, Issue
3, March 2016.
[13] Romil Chauhan, Yash Jain, Harsh Agarwal, Abhijit Patil, “Study of
(b) Implementation of Voice Controlled Wheelchair”, 3rd International
Fig. 7. Interfaces of location tracker app-(a) 1st activity offers two ways to get Conference on Advanced Computing and Communication Systems
the location, (b) Next two activities offer respected results (ICACCS -2016), Jan. 22 – 23, 2016, Coimbatore, INDIA.
[14] Zannatul Raiyan, Md. Sakib Nawaz, A.K.M Asif Adnan, Mohammad
IV. CONCLUSION Hasan Imam, “Design of an Arduino Based Voice-Controlled
Automated Wheelchair”, IEEE Region 10 Humanitarian Technology
To make the world a better place, it is essential to reduce Conference (R10-HTC), 21 - 23 Dec 2017, Dhaka, Bangladesh.
the sufferings of living human being. Scientists and [15] ‘Types of Speech Recognition’, 2018. [Online]. Available:
researchers have been trying for many years in the field of https://www.lumenvox.com/resources/tips/types-of-speech-
technology to bring something new for a better surveillance. recognition.aspx
In this paper, we have represented an assistive system for [16] W. Shen,
https://www.elechouse.com/elechouse/images/product/VR3/VR3_manu
physically disabled persons who can’t move their wheelchair al.pdf. 2018.
without assistance. The system offers movement of wheelchair [17] ‘Infrared IR Sensor Circuit Diagram and Working Principle’, 2018.
using voice command. Also, patient can control the speed of [Online]. Available: https://www.elprocus.com/infrared-ir-sensor-
wheelchair by voice command. A tracking system has also circuit-and-working/. [Accessed: 29-Sep-18].
been developed to track current location of patient using [18] ‘Global Positioning System-Wikipedia’, 2018. [Online]. Available:
android app that makes our system user friendly, speedy and https://en.wikipedia.org/wiki/Global_Positioning_System. [Accessed:
29-Sep-18].
cost effective. Proper positioning of the hardware inside the
[19] ‘Firebase-Wikipedia’, 2018. [Online]. Available:
wheelchair is a major concern. https://en.wikipedia.org/wiki/Firebase. [Accessed: 29-Sep-18-18].
[20] S. Khedkar, S. Thube, W. I. Estate, N. M. W, and C. Naka, “Real Time
Databases for Applications,” Int. Res. J. Eng. Technol., vol. 4, no. 6,
References 2017.
[1] ‘WHO | World report on disability’, 2019. [Online]. Available: [21] ‘Documentation | Firebase’, 2018. [Online]. Available:
https://www.who.int/disabilities/world_report/2011/report/en/. https://firebase.google.com/docs/. [Accessed: 11-May-2018].
[Accessed: 13-Jan-19]. [22] ‘SP ROBOTIC WORKS’, 2018. [Online]. Available:
[2] A. Sasou and H. Kojima, “Noise robust speech recognition applied to https://sproboticworks.com/blog/choosing-the-right-motor-driver.
voice-driven wheelchair”, EURASIP Journal on Advances in Signal
Processing, vol. 2009, p. 41, 2009.
[3] R. C. Simpson, J. Rehabil, Res. Dev, “Smart wheelchairs: A literature
review”, vol. 42, no. 4, p. 423, 2005.
[4] Lin, Chern-Sheng & Chien-Wa, Ho & Wen-Chen, Chen &
ChuangChien, Chiu & Mau-Shiun, Yeh. “Powered wheelchair

View publication stats