K.

Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN
2278-5841, Vol 1, Issue 6, November 2012.

Voice and Gesture Based Electric-Powered

Wheelchair Using ARM
K. SUDHEER1, T.V.JANARDHANA RAO2, CH. SRIDEVI3, M.S.MADHAN MOHAN4
1

M.Tech Student(Embedded Systems), 2Professor , 3,4Associate professor,

Department of Electronics and Communication Engineering, B.V.C. Engineering College, Odalarevu, AP, INDIA
4
E- mail : 1sudheer.kanuri@gmail.com, 2dean.admissions@bvcgroup.in, 3chavakula.sridevi@gmail.com, msmm141@gmail.com
ABSTRACT
A voice and gesture based system has been developed to
control a wheelchair using voice commands and moment of hand
i.e. Mems sensor is connected to hand. The system is divided into
two main components: speech recognition module with Mems
sensor and wheelchair control unit. In the speech recognition
module, hidden Markov models are used as the main method in
recognizing voice commands. The Mems sensor senses the angle
of the hand, i.e. according to the tilt of hand it gives voltages to
microcontroller. The wheelchair control unit is developed using
ARM controller.

Keywords- Voice Recognition, Gesture recognition , ARM

controller.
I .INTRODUCTION
Todays world comprises of a large variety of
people. Some of them depend on others for their living. But
in todays fast world, everyone is busy and there are less
people to care for the increasing number of elderly and the
physically challenged people. Also these people find it tough
to even navigate inside the home without external aids. The
elderly people find automated wheelchairs as an easy way
for locomotion. Having known about these facts, our aim
was to bring an automated navigation system which can be
used by both the elderly and the physically challenged
people in a user friendly manner using voices and hand
gestures for operation.
In biomedical sector, a wheelchair is an important
device because of the recent shift in the industrial
populations. The demand of the physically handicapped and
the aged are ever rising. The present wheelchairs do not have
integration of technologies for their working. It either
requires constant monitoring by the helper or hence lot of
effort.

for developing an advanced wheelchair. Traditional

wheelchairs used by the elderly and handicapped severe
disabilities have some limited functions and flexibility
several teams are currently at work to develop robotic
wheelchairs to overcome all these limitations. In this project
we are trying to include some sensors to develop an
automated wheelchair which can help the patient to control
the direction of the wheelchair based o voice and gesture[3].
In existing systems pc will be used for geture
recognition and processing of voice commands[1] . hence
along with wheelchair a pc has to be equiped which increases
complexity in system. That complexity will be reduced by
using voice recognition IC and MEMS accelerometer which
is a very small IC placed on the finger tips of the patient.
Block Diagram:
Implementation of this proposed problem mainly
involves three steps. They are voice recognition, gesture
recognition and controlling direction of wheelchair using
microcontroller based on the received voice or gesture
commands by making use of motors. These motors will be
driven by controller. The block diagram of the system is
shown in Figure 1.
DC motors are fixed to the wheels of wheelchair
hence based on rotation of motor direction of wheelchair will
be controlled. Motors will be interfaced to microcontroller
through motor drivers.

The recent developments in the robotics artificial

intelligence or sensor technology promises enormous scope

www.ijrcct.org

chiefeditor@ijrcct.org

Page 278

K. Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN
2278-5841, Vol 1, Issue 6, November 2012.
1.
2.
3.
4.

5.

Figure 1: Block diagram

Voice recognition module recieves the voice of the
speaker and sends commands corresponding to the recieved
voice command to controller. Gesture recognition module
sends command to the microcontrolller based on the geture of
the user. Control unit controls the direction of rotation of
motors.
II. SPEECH RECOGNITION MODULE
Related Work
Various system applications using automatic speech
recognition (ASR) technology have been in the market for just
over 10 years [7]. In 1994, the Dragon Systems released its
DragonDictate for Windows 1.0, using discrete ASR
technology, which is a word based ASR. Discrete ASR
technology requires the user to pause after each word making
the dictation unnatural for the user. In addition, the
performance of discrete ASR is slow and usually
unacceptable. In 1996, the first continuous speech recognition
system, called MedSpeak/Radiology, was released by IBM.
Continuous speech technology allows users to speak in a
natural manner conversationally, as they are speaking to
another person. At the time of their releases, these systems
were often priced to be over 10,000 dollars and required very
expensive computer systems to support them to function as
desired [7].
Now instead of using cost effective computer
systems directly speech recognition modules can be used
which reduces complexity as well as size of the total system
There are two essential characteristics for any
effective ASR system: accuracy and speed. In addition, to
meeting these demands, ASR systems face a number of
additional challenges including the large variance that exists
among individual human speech patterns (e.g. pitch, rate,
inflection). A successful ASR system requires extraordinary
flexibility to accommodate these variances. The process of
ASR typically follows these steps:

www.ijrcct.org

Acoustic speech is captured by a microphone and

undergoes analog-to-digital conversion.
Short time frames of speech are converted into signal
features that can be used to discern speech
components.
Series of frames are mapped to phonemes, the
smallest building blocks of speech, and series of
phonemes are mapped to words.
Contextual information is simultaneously processed
to increase the accuracy of the ASR system. For
example, contextual information would be able to
assist in the selection of a series of homonyms (e.g.
merry, marry, and Mary).
The series of words are determined based on the
probabilistic measure and provided in a format
suitable for the next stage in the application (e.g. text
format).

Hidden Markov Model

Hidden Markov models (HMMs) are widely used in
the area of ASR [8]. HMMs are comprised of two main
components: a Markov chain of states, which is the hidden
portion of the model, and statistical descriptors for each state
in the Markov chain that are associated with the probabilistic
nature of the observed data. This structure enables the HMM
to simultaneously characterize local signal properties (i.e.
regularities in the signal that occur in a short time span) and
global signal properties by modeling local signal properties as
a sequence of events. As speech can be considered a
quasistatic signal, the structure of HMMs make it ideal for
ASR.
Proposed model
From the above procedures, to implement speech
recognition system it requires a pc but it is very difficult for a
patient on wheelchair to carry a PC along with him hence for
speech recognition HM2007 IC based SR07 circuit can be
used[3] which reduces circuit complexity.
This circuit recieves user voice commands through
microphone and processes and then sends 8 bit data output
corresponding to the recieved command which can be
interfaced with any microcontroller
Features:
1 It has a stand alone speech recognition ciruit.
2 User programmable
3 It can recognize up to 20 word vocabulary of duration
two second each
4 Multi-lingual

chiefeditor@ijrcct.org

Page 279

K. Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN
2278-5841, Vol 1, Issue 6, November 2012.
5

Non-volatile memory back up with 3V battery

onboard. Will keep the speech recognition data in
memory even after power off.
6 Easily interfaced to control external circuits &
appliances
The heart of the speech recognition circuit is the
HM2007 speech recognition IC.

HM2007 can recognize 20 words each of length 1.92

second or equivalently it can recognize up to 40 words each of
length 0.98 seconds[4]. In this ciruit along with HM2007 there
are decoders keypads and digital display. The keypad and
digital display are used to communicate and program the
HM2007 chip. The keypad have 12 normally open contact
switches. When the circuit is turned on, 00 is on the digital
display, the red LED (READY) is lit and the circuit waits for a
command.

Speech recognition is divided into two broad

processing categories; speaker dependent and speaker
independent. Speaker dependent systems are trained by the
individual who will be using the system. These systems are
capable of achieving a high command count and better than
95% accuracy for word recognition. The drawback to this
approach is that the system only responds accurately only to
the individual who trained the system. Speaker independent is
a system trained to respond to a word regardless of who
speaks. Therefore the system must respond to a large variety
of speech patterns, inflections and enunciation's of the target
word. The command word count is usually lower than the
speaker dependent however high accuracy can still be
maintain within processing limits. Speech recognition circuit
is shown in Figure 3.

Figure 2: SR07 Speech Recognition System

Press 1 (display will show 01 and the LED will
turn off) on the keypad, then press the TRAIN key ( the LED
will turn on) to place circuit in training mode, for word one.
Say the target word into the onboard microphone (near LED)
clearly. The circuit signals acceptance of the voice input by
blinking the LED off then on.
If any errors occur the following codes will be
displayed based on the type of error
Error Codes:
The chip provides the following error codes.
55 = word to long
66 = word to short
77 = no match
Once stored commands or data can be tested.
Data present in IC can be cleared by pressing 99 and
then CLR. Then all the numbers will quickly scroll on the
digital display and data will be erased

www.ijrcct.org

Figure 3: Speech Recognition IC

III. GESTURE RECOGNITION
Gesture means A movement of part of the body, esp. a
hand or the head, to express an idea or meaning. Here the
project is implemented based on hand gesture. Based on the
hand movement wheelchair will be controlled. To recognise
this hand gesture MEMS accelerometer[2] is ussed.
Micro Electro Mechanical Systems (MEMS) is the
integration of mechanical elements, sensors, actuators, and
electronics on a common silicon substrate through micro
fabrication technology. MEMS is an enabling technology
allowing the development of smart products, augmenting the
computational ability of microelectronics In most cases, the
physics behind the behavior of MEMS devices can be
expressed by mathematical expressions. MEMSolver works by
creating a mathematical model of the system and generates
analytical solutions to explain the behavior of the MEMS
device. The user just has to enter the input parameters like
length and width of the beam for example in a user friendly
GUI, and the software will immediately calculate the relevant

chiefeditor@ijrcct.org

Page 280

K. Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN
2278-5841, Vol 1, Issue 6, November 2012.
results and plot graphs that fully explain the MEMS device or
part of it.

MEMS accelerometer is a single chip with small size

and low cost. Because of their small size and weight.
Accelerometers are attached to the fingertips and back of the
hand. Arrows on the hand show the location of accelerometers
and their sensitive directions. That the sensitive direction of
the accelerometer is in the plane of the hand.
In this praposed model MMA7361L accelerometer is
used. An accelerometer is an electromechanical device that
measures acceleration forces. The device can be used for
sensor data changes, product orientation, and gesture
detection. This is 3-axes accelerometer which produces three
voltages in three directions based on the tilt[4] of the
accelerometer.
Orientation can be detected based on the capacitance
produced between parallel plates. The figure 4 shows the
detection of orientation. In this there are two fixed and one
movable plates based on the orientation, position of movable
plate changes which results in change in capacitance
Figure 5: Voltages corresponding to orientation
Finally this accelerometer produces three output
votages in three directions corresponding to each orientation
which are in analog form hence they will be interfaced to
controller through analog to digital conversion pins as shown
in figure 6.

Figure 4: Orientation detection

Based on the tilt of the accelerometer different
voltages will be generated in three directions i.e., X,Y and Z.
Voltages that will be produced corresponding to tilt are
shown in figure 5.

Figure 6: Accelerometer interfacing to controller

VI. WORKING
Speech recognition module receives voice commands
of the user and sends binary code corresponding to the
recieved command to the microcontroller.
Similarly gesture recognition module sends analog
voltages in three directions corresponding to each orientation

www.ijrcct.org

chiefeditor@ijrcct.org

Page 281

K. Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN
2278-5841, Vol 1, Issue 6, November 2012.
to the microcontroller. Output of the speech recognition
module and gesture recognition module are interfaced to
controller. This controller controls the wheelchair based on
recieved gesture or voice command. To control wheelchair DC
motors will be drived through the microcontroller where DC
motors are connected to the wheels of wheelchair, but DC
motors will not be driven directly by microcontroller as thier
operating voltages are not compatible. Hence to drive motor
L293D motor driver is used.
V. RESULTS AND DISCUSSION

Motors will be fixed to front wheels of wheelchair

and hence based on the rotation of motors direction of
wheelchair will be controlled.

In this system for the operation of hardware software

is needed. Controller has to be programmed to perform the
required task. the code was written on assembly language
using Keil Vision-3 software. The assembly code now
converted into Hex file. Now the code was burn onto the
controller.
Table 1: Tesa results based on voice commands

By using above procedure hardware setup is done

figure 7 shows input part that is interfacing of speech
recognition module and accelerometer to ARM controller.

No.of
words
spoken

No.of times
system
correctly
responded

Speaker-I

20

19

95%

Speaker-II

45

43

95.5%

Speaker

Accuracy

The recognition rate of the voice commands is 72.5%

using the Speech recognized electronic device controller,
where as it is improved by using HM2007 IC based SR07
voice recognition module and it produces 95% accuracy .in
voice recognition as shown in Table 1.
Gesture recognition also produces more than 90% of
accuracy from test results.
VI. CONCLUSION
Figure 7: Hardware setup input part
Controller controls wheelchair by properly controlling
the direction of rotation of motors hence motors will be
interface to controller through the motor driver as shown in
figure 8.

By using this sytem physically challenged people

finds easy way to navigate within the home using wheelchair
without any external aid. This provides ease of operation as
the system is user friendly and size is small. Voice
commands will be received through microphone and for
gesture recognition small IC will be placed at finger tips.
This system canbe extended by including GSM which
sends an SMS during emergency by assigning pirticular
voice or gesture command. By including GPS position of the
wheelchair can also be known.
VII. REFERENCES
1.

2.

Rajesh Kannan Megalingam, Ramesh Nammily Nair, Sai

Manoj Prakhya, Automated Voice based Home
Navigation System for the Elderly and the Physically
Challengedproc.ICACT2011P .603-08,Feb.2011.
O.Mirabella, M.Brischetto, G.Mastroeni Mems based
gesture recognition, proc.HSI P.599 604, May

2010
Figure 8: hardware setup output part

www.ijrcct.org

chiefeditor@ijrcct.org

Page 282

K. Sudheer ,et al, International Journal of Research in Computer and Communication technology, IJRCCT, ISSN
2278-5841, Vol 1, Issue 6, November 2012.
3.
4.
5.
6.

7.
8.

R.Posada Gomez, L.H.Sanchez Medel, A Hand Gesture

System
Of
Control
For
An
Intelligent
Wheelchair,Proc.ICEEE,P.68-71,5-7,sept.2007.
http://www.analog.com/en/content/over_five_motion_sen
ses/fca.html,1995-2012.
J.Z. Yi , Y.K. Tan Z.R. Ang, Microcontroller Based
Voice-Activated Powered Wheelchair Control, ACM
publishers, 2007.
C.Chandramouli
and
Vivek Agarwal, Speech
Recognition based Computer Keyboard Replacement for
the Quadriplegics, Paraplegics, Paralytics and Amputees,
ACM publishers,
Munro, Jay. "Watch What You Say." PC Magazine
Online. March 10, 1998. http://www.zdnet.com/pcmag/
features/speech/intro1.html (23 October 1998).
Rabiner L. R., A Tutorial on Hidden Markov Models and
Selected Applications in Speech Recognition, Proc

www.ijrcct.org

chiefeditor@ijrcct.org

Page 283