A WHEELCHAIR

CONTROLLED BY HAND
GESTURES

1
 UNDERGRADUATE FINAL YEAR PROJECT THESIS
Department of Mechanical Engineering
University of Engineering & Technology Peshawar

A Wheelchair Controlled by Hand

Gestures

Group# Batch 2018-22

Executive Summary

The wheelchair is an important assistive device for handicapped and old-aged

people. These days it is a demand for automated wheelchairs to drive people
without having any assistance to help them. It is the age of robotics, AI, and
automation, so this is why this wheelchair will help out in case of any disability.
This wheelchair is completely wireless and based on a sensing mechanism.
Traditional wheelchairs exist in the market, but they are comparatively hard to
drive. Also, we want to lead towards automation in the field of mechatronics

2
and electromechanical systems. If you look at the population of the world so 2-
4% of them are disabled to move from one place to another they depend upon
others. , so this wheelchair is just like a self-driven system for them to move
safely. This wheelchair is based on a hand gesturing mechanism, which only
needs a gesturing sensor or accelerometer sensor mounted on the glove. It
controls the movement and direction according to the user. We have two main
circuits in the whole project, called transmitter and receiver. Both of them are
based on Arduino UNO with an ATMEGA328 microcontroller. This project is
really valuable because of having a low design cost, a smaller number of
components used and a sensing range which is about 30 meters. The movement
is controllable in 5 axes, forward. Backward, right, left, and stop when the
gesture sensor is in a straight position the basic purpose of this project is to just
transfer signals from the gesturing sensor and transfer it to the motors driver
decoder to drive the motors. After having this chair any disabled person will be
able to do his daily basis work in a safe and better way.
The maximum load that can be bear by the chair is about 60kg, for which this
wheelchair is designed. The biggest advantages of this project are having low
cost, solar-based, sensitive sensing mechanism, smooth motion, and self-
dependency. The inclination and declination angle is about 10-15 degrees. This
hand gesture-controlled wheelchair is capable of carrying the disabled person to
their destination safely and easily.

3
Declaration:

This work contains no material which has been accepted for the award of any
other degree or diploma in any university or any tertiary institution, and the best
of our knowledge and belief contains no materials previously published or
written by another person, except where due reference has been made in the text
and where theory about something has been explained.
We give consent to this copy of our thesis when deposited in the university
library, being made available for loan and photocopying.

……………………………………………………………………………………

Engr. Adnan Rasheed

(Project supervisor)

.......................................................................................

Prof. Dr. Rizwan M. Gul

(Chairman mechanical Engg. Dept.)

4
Acknowledgment:

We would like to acknowledge the efforts of all the people who have
contributed to the work in this thesis. We are all deeply indebted to our
supervisor, Engr. Adnan Rasheed sb, without whom this project and thesis
would never have been completed. His kind is to share his knowledge and
insight has always made us motivated for doing more work.
We are thankful to many other academics in our university who have
helped us in our project. We who have helped and motivated us. We are
sincerely indebted to all of them who have given us continuous assistance
throughout the project.
In the end, we dedicated this thesis to our parents and are also grateful to
them for giving us deep love, encouragement, and support during our education.

5
 Table of contents

Executive Summary……………………………………………………. (3)

Declaration……………………………………………………………… (4)
Acknowledgment………………………………………………………... (5)
Table of contents………………………………………………………… (6)
Chapter 1 Introduction………………………………………………….. (7)

1.1 Background Information……………………………………... (7)

1.2 Significance & Motivations………………………………….. (7)
1.3 Aims & Objectives…………………………………………… (8)

Chapter 2 Literature Review…………………………………………….. (9)

2.1 Previous proposed system……………………………………. (9)

2.2 Based on MEMS sensor……………………………………... (9)
2.3 Based on touch sensor……………………………………….. (9)
2.4 Raspberry pi approach………………………………………. (10)
2.5 Wired transmission control…………………………………... (10)
2.6 Human machine interfaces…………………………………… (11)

Chapter 3 Methodology (Research Design & Methods) ………………… (12)

3.1 Components of Hand Gestures Controlled Wheelchair……… (12)

3.2 Conversion of hand movements into numeric data…………... (14)
3.3 Control commands of servo motors………………………….. (16)

Chapter 4 Transmitter and Receiver Codes………………………………. (18)

4.1 Transmitter Codes……………………………………………. (18)

4.2 Receiver Codes………………………………………………. (21)

Chapter 5 Summary, Applications, Discussion, Conclusion……………… (25)

5.1 Summary………………………………………………………..(25)
5.2 Applications…………………………………………………… (25)
5.3 Discussion………………………………………………………(26)
5.4 Conclusion………………………………………………………(27)

6
Chapter 1
Introduction

1.1 Background information

In recent times there is a need for automated wheelchairs because of the high
ratio of disabled people throughout the world. They need an assistant for their
help to move from one place to another place. Almost these days 2% of the
people are facing disability because of different reasons, like the weather they
would be older too much or any physical disease. On the other side if we look at
the population of the world, so it is increasing day to day, which again leads to a
high rate of disabled people being born? So, to overcome these problems we
need to understand that having the availability of such kind of wheelchair that
we are going to describe in this paper can ease the work of quadriplegic people.
Even if somebody cannot move, he can only gesture his hand so he can reach
out himself to their destination by using this wheelchair. The traditional chairs
have certain limitations like flexibility, bulkiness, and many more, but this chair
can be used friendly in any way. To operate this chair only needs fewer efforts
to move it. Here we go to make the smart type of wheelchair whose motion can
be synchronized according to the user. This wheelchair is free of jerks and has
full convenience so that the user can use it very easily and have no problem
while driving. Most of us already know about the wheelchair based on the
joystick which is electrically operated. But the problem with this kind of
wheelchair needs a high force to operate it. But in the case of gesture control,
you need no additional force but you have to shake your hand according to the
direction in which the user wants to go if the user wants to behave in a stop
position make sure your hand must be in a straight position. This project is
preferred over another one because of having low cost and market demand.
Also, this kind of wheelchair has higher sensitivity instead of a lag system. In
this project, we trying out to make a low-cost hand gesture accelerometer-based
controlled wheelchair. This chair is controlled wirelessly using sensors through
IR rays. As the world leads toward automation and convenience so we also try
out to make such projects based on advanced and automated approaches.

1.2 Significance and motivations

The hand gesture-controlled wheelchair is to make disabled people independent.

This wheelchair is for those who are even not able to apply much force on the
joystick to move. The hand gesture-controlled wheelchair is an advanced level
of approach in the field of mechatronics as well. This wheelchair is not much
complicated in structure. It is wireless, which is another big achievement for the
user to not disturb while using. The hand gesture-controlled wheelchair needs
less power and less cost to design it. Also, it has a big sensitivity. To drive this
wheelchair, need no assistance, but is a self-driven system. It is based on dc 12
7
voltage, so it needs a low input voltage source as well. It can climb on the 10-
degree inclination as well. The other significance of the hand gesture is
controlled wheelchair good balance. It can also Bes used to carry the luggage
and goods as well. It has less cost to design. If you have this kind of wheelchair,
disabled people can do their work and can take the economy of any country to
altitude.
The motivation for this project is to lead toward automation. Already we have
the traditional systems, but in the case of this system, you will be more
comfortable and safer. The other thing for the selection of this project is having
a low level of manufacturing upon this project. The market demand is much
higher as compared to the other traditional ones. The design cost of the
wheelchair is less while the selling coat is much better. I like the automation
whether that is industrial automation or any automated system. Keeping the
perception of the disabled people in the heart made the low-cost hand gesture-
controlled wheelchair.

1.3 Aims and objectives

A wheelchair is an important assistive device for handicapped and old-aged

people. These days it is a demand for automated wheelchairs to drive people
without having any assistance to help them. It is the age of robotics, AI, and
automation, so this is why this wheelchair will help out in case of any disability.
This wheelchair is completely wireless and based on a sensing mechanism.
Traditional wheelchairs exist in the market, but they are comparatively hard to
drive. Also, we want to lead towards automation in the field of mechatronics
and electromechanical systems. It has high scope in the market as well as
throughout the country where there is a high ratio of disabled people. If you
look at the population of the world so 2-4% of them are disabled to move from
one place to another they depend upon others., so this wheelchair is just like a
self-driven system for them to move safely. Following are some of the main
objectives of this project.
1. To make a wireless hand gestured controlled wheelchair
2. To make a hand gesture-controlled wheelchair which is self-driven and
independent.
3. To make a hand-gestured controlled wheelchair that is fully comfortable
and balanced.
4. To make a fully developed wheelchair to be driven by the simple flip
movement of a disabled person’s hand
5. To minimize jerk in motion and to make it possible to have a uniform
velocity of the hand-gestured controlled wheelchair.

8
Chapter 2
Literature review

2.1 Previous proposed systems

Over the years, several approaches, such as a joystick, gesture, and chain-based
control mechanisms, have been proposed to help physically disabled people
reduce their level of dependence on others for movement. An image processing
technique has been proposed to control the wheelchair. It was easy to control. It
was based on image processing to recognize the gesture. However, it required a
camera with wires to perform the operations. Which was a difficult thing?

2.2 Based on MEMS sensor

The motion of the wheelchair was controlled by using a MEMS sensor. Which
was fixed on the hand of the user? The sensor was connected with the 3-axis
accelerometer sensor with digital output. Which detects and converts into a 6-bit
digital value.
The hand gesture-controlled wheelchair was proposed later, but it was based on
the remote control. Then we proposed another smart system by the name of
hand gesture-controlled wheelchair having no wire systems. Which is under
consideration in this paper. It works on the bases of sensors. It was a remote
control, so it was difficult for the user to use it. It has certain complications and
many more drawbacks.

2.3 Based on touch sensor

A hand-gesture-based approach using a touch sensor for users to control a

wheelchair effectively has been proposed. This approach however uses a wired
hand gesture hardware component, which makes it difficult for the elderly and
physically disabled people to use. the authors proposed a hand-gesture-based
approach to controlling a wheelchair. This approach relies on the use of the
global positioning system (GPS) and GSM to identify locations, which makes it
unreliable to use in noisy environments.

9
2.4 Raspberry pi approach

The hand-gesture-based approach that uses Raspberry pi and image processing

techniques to control a wheelchair has been proposed this approach relies on an
image processing technique and a USB web camera to recognize gestures. The
main drawback of this approach is that it is difficult to handle and operate by
users, and it requires a wired web camera to perform the operations. An
accelerometer-based-gesture approach has been proposed to control a
wheelchair using GPS and GSM navigation. In this approach, a wheelchair
component should be transmitted to a control room and the location of the user
is determined by a navigation application. The authors presented a framework to
help people who 2950 Sensors and Materials, Vol. 31, No. 9 (2019) cannot walk
owing to physiological or physical illness. The proposed framework involves
the use of computer-controlled wheelchairs, which will be unfamiliar to the
elderly and physically disabled people.

2.5 Wired transmission control

A model for a hand-gesture-wired-controlled user interface has been. This

model uses an accelerometer sensor to control the direction of a wheelchair
using hand movements. The main drawback of the proposed model is that the
transmission control is wired, making it difficult for users to handle and operate.
The authors developed hand-gesture-based methods to control wheelchair
movement using MEMS and acceleration technologies. The main drawbacks of
these methods are their non-cost-effectiveness and difficulty to use and operate
because they are unfamiliar to the elderly and severely physically disabled
people. A gesture-recognition-based approach has been proposed to control a
wheelchair using an android application. A smartphone and a connection should
be used to control a wheelchair. Using the touch screen of the smartphone, the
user has to choose the direction specified within the four quadrants on the
screen, which is difficult for users to use the authors present a hand-gesture-
reorganization approach using a real-time tracking method and a hidden Markov
model that introduces the hand gesture reorganization system to recognize
continuous gestures before a stationary background. In this approach, the
motion of the object gives important and useful information for object
localization and extraction. However, to recognize the gesture, the complexity is
higher and the accuracy is lower, so it is not beneficial and not compatible with
the users.

10
2.6 Human machine interfaces

A visual-based human-machine interface (HMI) solution has been proposed to

control a wheelchair by head gestures, which were recognized by detecting the
position of the nose on a user’s face. The proposed approach is uncomfortable
and not applicable to all users, which requires constant shoulder or neck
movements. The head of the user should always be within the range of the
sensor; otherwise, the user cannot control the movement of the cursor the
authors presented an approach using closeness matching to detect hand motion,
the authors presented a method that utilizes a strategy for signal coordination
with the utilization of the ARM 11 Raspberry pi and Sigsbee module. Another
approach that uses wearable hand gloves to capture hand movement to control a
wheelchair has been proposed the authors described their work on gesture
reorganization applied to wheelchair control. In this case, gestures are
recognized through a three-axis accelerometer sensor

11
Chapter 3
Methodology (Research Design & Methods)

The design and procedures utilized in a Hand Gestures Controlled

Wheelchair will be discussed in this chapter. To assist the elderly and totally or
partly quadriplegic persons in doing everyday life tasks without the assistance
of others by overcoming physical impairment problems. For our system, we
assume the following.
 An accumulator is put on the user's hand to detect hand movement and
convert it to analog signals, which are then routed to an encoder for
transmission through an nRF transmitter.
 To operate the wheelchair, the user's hand gestures will be converted into
movement commands by an Arduino microcontroller unit that is directly
attached to it.
 Transmission and receiving circuits will turn the user's hand gestures into
numeric data.
 The forward, backward, turn right, turn left, and stop instructions are the
five commands that provide adequate and proper control.

3.1 Components of Hand Gestures Controlled Wheelchair

In the HGCWC system, an accumulator with an nRF transmitter, an nRF

receiver, and microcontrollers (AT89C51 and ATMEGA328P-PU) is used as its
core components. These components are integrated with a wheelchair and
programmed to fully control the movement and directions of the wheelchair.
Figure 1 shows the core components and work mechanism of the HGCWC
system.

12
3.1.1 Hand Gestures

For partially paralyzed people who can only move their hands only
straightly, we provide a solution that depends on hand gestures to represent the
directions of a wheelchair, as shown in Fig. 2.

3.1.2 Accelerometer

The accelerometer in this HGCWC is used to sense the movements of the

hand and change them into analog signals, which will then be directed to an
encoder sent by an nRF transmitter.

3.1.3 AT89C51 Microcontroller

After the signals are received by the nRF receiver to be directed to the
encoder (to be
encoded), we use another microcontroller to analyze the encoded signals and
then send them to the ATMEGA328P microcontroller to convert them into
movement
commands.

3.1.4 Servo Motors

The motors that are used in HGCWC are permanent magnet 24-volt DC motors
of 17 watts. When there is no load then each motor consumes 0.73-ampere
current while on full load each motor consumes 9.61 amperes. To increase the
torque, these motors have inbuilt gears, the speed of these motors is 60rpm.

13
3.2 Conversion of hand movements into numeric data

There are two circuits used in this system i.e., the transmission and
reception of data. The working mechanism of each will be explained separately
as follows.

3.2.1 Transmission circuit

The transmission circuit will be placed on a user's hand to detect forward,

backward, rightward, and leftward hand motions. Following the identification of
the readable hand movements, we allocated each movement to one of the five
instructions as follows:

14