2019 IEEE Region 10 Humanitarian Technology Conference
Depok, Indonesia | November 12-14, 2019
Eye Blink Controlled Low Cost Smart Wheel Chair
Aiding disabled People
Md Ashiqur Rahman Apu, Imran Fahad,S. A. Fattah, Celia Shahnaz*
Department of Electrical and Electronic Engineering
Bangladesh University of Engineering and Technology, Dhaka-1205
Email: celia@eee.buet.ac.bd
Abstract—In this paper, a smart wheelchair is proposed to help
out physically disabled people by using their eye blink activities.
A prototype is implemented by processing eye ball images of the
subject to detect intentional eye blinks. An algorithm is proposed
to produce wheelchair movement command from consecutive
intentional eye blinks. To identify intentional eye blinks from eye
ball images, advanced image processing techniques are applied
using raspberry-pi. The commands are wirelessly transferred to
the motor driver of the wheel chair.The HC-05 Buletooth receiver
is connected to the wheelchair which receives transmitted data
from raspberry-pi and motor driver implements the command
according to predefined decision table. It can also sense obstacle
using APDS 9960 proximity sensor to avoid collision with
surroundings. The proposed smart wheelchair has shown False
Command Rate (FCR) of 2.1% to 13.2%. The experiment is
performed on four subjects and each of them has attended three
sessions. Overall, the eye blink controlled aiding wheelchair is
going to create a humanitarian impact on physically disabled
people.
Index Terms—Eye Blink, Haar Cascade, Google Glass, Wheel
Chair, Muscular Dystrophy, Amylotropic Lateral Sclerosis (ALS).
I. I NTRODUCTION
A major task of designing a smart wheelchair control is to
produce sufficient commands rapidly and accurately including
left and right turns, forward and backward motions, stopping,
acceleration and deceleration. The facilities provided by a
typical wheelchair require people with disabilities having
intact manipulation ability like to use a joystick and operate
the vehicle. However, many people with disabilities can not
manipulate mechanical device and thus unable to use typical
wheelchair [1]. People with limited muscular function usually
suffer from Amylotropic Lateral Sclerosis (ALS), paralysis
and other Motor disabilities which causes loss of body move-
ment. These diseases introduce difficulties in their daily life
interaction with their surrounding environment [2] [3]. They
have quite low mobility and may lose communication abilities
locked in their own body which causes frustration,depression
and anxiety. They become dependent on others for daily
activities like moving one place to another place, eating food
etc [4] [5]. In this paper, body movement problem is targeted to
find reasonable solution with a smart aiding wheelchair. The
system can improve the personal autonomy of people with
disabilities through development of machines which can aid
movement and communication [3] [6] [7].
978-1-7281-0834-6/19/$31.00 ©2019 IEEE
Authorized licensed use limited to: University of Exeter. Downloaded on May 08,2020 at 12:42:38 UTC from IEEE Xplore. Restrictions apply.
A. Disabilities due to paralysis
Paralysis is often caused by damage in the nervous system,
stroke, poliomyelitis, cerebral palsy, peripheral neuropathy,
Parkinson’s disease etc. Pseudo paralysis is voluntary restric-
tion of motion because of pain, lack of coordination and is not
due to actual muscular paralysis. It can be caused by extreme
mental stresses such as panic anxiety disorder. Paralysis can
occur in localized or generalized form [5]. Paralysis can occur
in newborns due to a congenital defect. It causes vertebrae
failing to form vertebral arches within which the spinal cord
protrudes from the rest of the spine. The cessation of spinal
cord function can result in paralysis of lower extremities [8].
B. Amyotrophic lateral sclerosis (ALS)
Amyotrophic lateral sclerosis is also known as motor neuron
disease (MND). It is a specific disease which causes the death
of neurons controlling voluntary muscles. ALS is characterized
by muscle twitching, stiff muscles and gradually worsening
weakness due to muscles decreasing in size. This results in
speaking, swallowing difficulty but the cause is not known
in 90% to 95% of cases. The remaining 510% of cases are
inherited from parents. About half of these genetic cases are
due to one of two specific genes [3].
C. Spinal Cord Injuries
Spinal cord injuries can result in paralysis of the limbs.
Paralysis of the legs is called paraplegia and paralysis of both
legs and arms is called quadriplegia. Patients with paraplegia
generally have no difficulty accessing the Internet. Individuals
with quadriplegia, however, may have significant difficulties,
depending on the type injury. Individuals with quadriplegia
have some use of their hands but not enough to manipulate a
mouse or type on a keyboard [9].
D. Muscular Dystrophy
”Muscular Dystrophy” indicates a group of genetic dis-
eases which are characterized by progressive weakness and
degeneration of the person’s voluntary muscles used to control
movement. ”Duchene” is the form of Muscular Dystrophy
which affects children mostly. Myotonic Muscular Dystrophy
is the most common form of the disease affecting adult
populations. Muscular Dystrophy can affect persons of any
age group. Physical therapy and corrective orthopedic surgery
may improve a person’s quality of life.
99
Fig. 1. An MRI with increased signal in the posterior part of the internal
capsule which can be tracked to the motor cortex, consistent with the diagnosis
of ALS.
Fig. 2. Block diagram representation of the proposed wheel chair system.

A prototype is implemented based on the proposed system

to get an overview of the performance. A HD camera mounted
on a glass is used to acquire eyeball images. The captured
images are processed sequentially for detecting eye blinks.
An eye blink detection algorithm is designed to count the
number of consecutive eye blinks. The number of consecutive
eye blinks is used to generate different commands to control
a wheelchair. After generating command, it is transmitted by
using wireless communication modules.

II. P ROPOSED S MART W HEEL C HAIR S YSTEM

In the proposed wheelchair system, bluetooth module HC-
05 is used incorporating with Arduino board. Arduino works
as a controller here. It reads bluetooth input and control
the wheel chair. In this system, obstacle avoidance proximity
sensor APDS 9960 module is used. It sends feedback to the
Arduino based controller and depending on the feedback of the
Fig. 3. Raspberry-Pi Image Processing Unit.
proximity sensor movement of the wheelchair is performed. A
printed circuit board is designed to accommodate the circuit
elements in a congested place. The prototype is powered by B. Image Input and Image Processing Unit
using a lithium ion battery. Here is the complete block diagram
Eye ball images are captured using webcam (Model:
of the proposed system sequentially placed and the functions
A4Tech PK-333E). The images are given as input to the
are described later.
proposed system. Input image is processed using Open-CV
library of Python. Different functions have been used for
A. Hardware Design Elements
manipulating the captured image. Eye pupil is the subject
The elements used to implement the proposed system is of interest for the system. Eye pupil is detected using ”Haar
as following: 1.Webcam 2.Raspberry Pi 3.Bluetooth Module Cascade” classifier. The classifier marks the eye pupils in the
HC-05 4.APDS 9960 Proximity sensor 5.Arduino uno 6.Motor captured images by the webcam.
Driver L293D 7.Motor 8.Wheel 9.800mAh 7.4V Lipo Battery This is the full block diagram of the proposed image pro-

100

Authorized licensed use limited to: University of Exeter. Downloaded on May 08,2020 at 12:42:38 UTC from IEEE Xplore. Restrictions apply.
cessing unit. The operations are performed by using raspberry
pi.
Fig. 4. Image Processing Unit Block diagram.
This algorithm is developed to implement the image pro-
cessing operations on the captured images.
Algorithm 1 Image Processing in Microprocessor Algorithm
1: Input: Eye Blink Signal, e[n]
2: Output: Command, C[n]
3: Eye Ball Detection: d[n] = Pretrained Eye Ball detector,
(e[n])
4: Compare: IF Blink Gap ¡ Threshold
5: Compute: Consecutive Blink
6: Decision Making: IF Blink NO. == Command
7: Wireless Transfer: Detected Command, C[n]
C. Eye Blink Based Decision Making Criteria
Eye blink has been used for decision making purpose.
Eye blink is defined as consecutive eye opening followed by
closure of eye pupil. The duration of each blink is 20ms on an
average and inter blink is very large compared to blink period
for natural blink. First of all, eye blink is detected and the
system starts to count no. of blinks and their corresponding
times.. Then, inter blink gap of two consecutive blinks is
compared with a threshold time of five seconds for differen-
tiating intentional blinks from natural blinks. The actual inter
blink time period is relatively lower than the threshold time
considered. As there is a system delay, the threshold has been
set up by trial and error process [4] [1] [6].
101
Authorized licensed use limited to: University of Exeter. Downloaded on May 08,2020 at 12:42:38 UTC from IEEE Xplore. Restrictions apply.
Fig. 5. Time period of consecutive eye blink compared to natural eye blink.
Here is the visual scenario of eye blink moments. Humans
blink for a very short period of time and it is defined as
blink period. The time gap between two blinks is defined as
inter blink gap. These two time periods are used to distinguish
between natural blinks and intentional command blinks.
D. Wireless Command Transfer and Controller
Raspberry Pi built in Bluetooth module works as command
data transmitter and HC-05 bluetooth module works as com-
mand data receiver. Movement command is generated from
Raspberry Pi and is transmitted via Bluetooth to Arduino.
Algorithm 2 Wireless Command Transfer Algorithm
1: Input: Command Signal, Tx
2: Output: Wireless Communication, Rx
3: Scan: Specific IP Address
4: Connect: Data transfer
Arduino is the motor driver controller of the proposed
system. HC-05 Bluetooth module receives command from the
microprocessor and feeds to Arduino. Arduino sends command
to motor driver. Then, the motor driver drives the motor
according to given command.
Algorithm 3 Wheel Chair Controlling Algorithm
Input: Transferred Command, C[n]
2: Output: Movement, move[n]
Check: IF Command matches any case
4: Call respective function
Motor Operation
6: Feedback: Proximity Sensor feedback
E. Feedback to the Controller B. Cost Analysis
A proximity sensor has been placed in front of the wheel The proposed system is developed with a view to aiding
chair prototype. It uses IR-ray for obstacle detection purpose. mobility of the disabled people with. Remarkable efforts have
When an obstacle comes in front the prototype, the sensor been put on focusing high performance while keep the cost at
sends a feedback to the controller and it is immediately minimum level. It is possible to use recycled parts and still
stopped. have satisfactory performance. The smart wheelchair prototype
is constructed within $90. The price list is given in Table III.
III. R ESULTS AND A NALYSIS
TABLE III
C OMPREHENSIVE C OST A NALYSIS
A prototype is implemented to observe the wheelchair
performance in real world environment. Motor driver gets Component Unit
Cost/Unit Total Cost
command from the microcontroller connected to it and thus the (USD) (USD)
Webcam 1 10 10
plant is operated. This plant can be operated in four possible Raspberry Pi 3 1 41.45 41.45
modes: idle, forward, rotate and stop. Bluetooth Module HC-05 2 2.37 4.74
.APDS 9960 Proximity sensor 1 9.47 9.47
Arduino Uno 1 5.33 5.33
TABLE I Motor Driver L293D 2 3.55 7.1
COMMAND TABLE OF THE SYSTEM. Motor 2 0.95 1.9
Wheel 2 0.83 1.66
No. of Consecutive Blinks Command
800mAh 7.4Volt Lipo Battery 1 9.47 9.47
0,1,2 Idle Total 91.12
3 Stop
4 Forward
5 Rotate
IV. H UMANITARIAN I MPACT O N S OCIETY
This smart wheelchair has significant impacts on disabled
people of both underdeveloped and developing countries who
A. Performance Evaluation of the Prototype
have limited economic resources but great potential. It will
The experiment is performed by four human subjects for cost little compared to other smart wheelchairs. Moreover, it
three sessions. Eye blink generated commands are used to has easily adaptable control system compared to other avail-
control the designed the prototype. The metrics used for the able state-of-the-art wheelchair like EEG or EOG signal based
evaluation of the proposed system performance are provided wheelchairs. This propose scheme will alleviate the physical
as following [5] [2]: labor necessary to operate other conventional wheelchairs.
In a country like Bangladesh, there are an estimated 16
Average time = Sum of all Session times million people having physical disabilities which is 10% of
N o. of Sessions
the country’s population. In 2004, the prevalence of disabil-
ity was about 6% among those below the age of 18 and
N o. of f alse commands
F alse command ratio(F CR) = T otal N o. of commands about 14% among those above that age. Disabled people
in Bangladesh are granted government support but programs
The subjects volunteered for three sessions and various are limited due to sufficient financial support and bounded
movement of the wheel chair prototype is performed for to town areas. In addition to governmental programs, NGOs
multiple times. The performance of the proposed system varies like ”Centre for Disability in Development (CDD)” is playing
from session to session for individual subject [10]. significant role to provide assistance to people with disabilities.
In collaboration with Government institutions and NGOs, the
TABLE II proposed smart wheelchair can make significant impact on
RESULTS OF THE EXPERIMENT ON SUBJECTS. rural areas upgrading lifestyle of the disabled people. So, it is
notable that commercializing such smart wheelchair will have
Subject Session Commands No. Time (s) Avg. (s) FCR (%)
1 11 58.3 2.1 a overwhelming humanitarian impact on the socio-economic
S1 2 9 58.0 58.9 4.2 structure of the country.
3 12 60.5 13.2
1 9 52.0 5.1 V. C ONCLUSION
S2 2 11 59.4 56.5 3.8
3 10 58.2 7.9 A major challenge in the design of the smart wheelchair
1 14 78.0 7.4 is to rapidly, accurately, and sufficiently produce control
S3 2 10 55.5 71.2 6.5
3 13 80.2 6.8
commands. In this paper, no. of eye blinks based on different
1 9 52.3 3.6 types of eye movement (blink) is presented for wheelchair
S4 2 9 52.5 56 4.1 control. The algorithm discriminates the intended blinks from
3 14 63.2 3.9 the unintended blinks mainly based on the time duration
between the eye blinks. According to the experimental results,

102

Authorized licensed use limited to: University of Exeter. Downloaded on May 08,2020 at 12:42:38 UTC from IEEE Xplore. Restrictions apply.
maximum False Command Rate is found about 13.2%. In
the future work, experiments will be conducted to increase
the no. of commands associated with direction control with
different degrees, forward and backward motions, accelera-
tion,deceleration, and stopping. One of the major focuses will
be to reduce the cost of the system and expand the application
range.
R EFERENCES
[1] Q. Huang, S. He, Q. Wang, Z. Gu, N. Peng, K. Li, and Y. Zhang,
“An EOG-Based Human-Machine Interface for Wheelchair Control,”
vol. 9294, no. c, pp. 1–11, 2017.
[2] J. Xiong, W. Xu, W. Liao, Q. Wang, J. Liu, and Q. Liang, “Eye Control
System Base on Ameliorated Hough Transform Algorithm,” vol. 13,
no. 9, pp. 3421–3429, 2013.
[3] W. Zhang, B. Cheng, and Y. Lin, “Driver Drowsiness Recognition Based
on Computer Vision Technology *,” vol. 17, no. 3, pp. 354–362, 2012.
[4] R. Zhang, S. He, X. Yang, X. Wang, K. Li, Q. Huang, Z. Gu, and Z. Yu,
“An EOG-based Human Machine Interface to Control a Smart Home
Environment for Patients with Severe Spinal Cord Injuries,” vol. 9294,
no. c, pp. 1–12, 2018.
[5] J. Ma, Y. Zhang, A. Cichocki, and F. Matsuno, “A Novel EOG / EEG
Hybrid Human-Machine Interface Adopting Eye Movements and ERPs
: Application to Robot Control,” vol. 9294, no. c, pp. 1–14, 2014.
[6] Y. Wang, G. Zhai, S. Zhou, S. Chen, X. Min, and Z. Gao, “Eye Fatigue
Assessment Using Unobtrusive Eye Tracker,” IEEE Access, vol. PP,
no. c, p. 1, 2018.
[7] A. Maksud, R. I. Chowdhury, T. T. Chowdhury, S. A. Fattah, C. Sha-
hanaz, and S. S. Chowdhury, “Low-cost EEG based electric wheelchair
with advanced control features,” vol. 2017-December, pp. 2648–2653,
dec 2017.
[8] S. K. Sivanath, S. A. Muralikrishnan, and P. Thothadri, “Eyeball and
Blink Controlled Firing System for Military Tank Using LabVIEW,”
pp. 4–7, 2012.
[9] T. F. Bastos-filho, F. A. Cheein, S. Mara, T. Müller, W. C. Celeste, C. D.
Cruz, D. C. Cavalieri, M. Sarcinelli-filho, P. Faria, S. Amaral, E. Perez,
C. M. Soria, R. Carelli, and S. Member, “Towards a New Modality-
Independent Interface for a Robotic Wheelchair,” vol. 22, no. 3, pp.
567–584, 2014.
[10] R. Chakravarthy, “Implementation of Home Automation using Eye Blink
Sensor,” pp. 242–244, 2014.

103

Authorized licensed use limited to: University of Exeter. Downloaded on May 08,2020 at 12:42:38 UTC from IEEE Xplore. Restrictions apply.