VISVESVARAYA TECHNOLOGICAL UNIVERSITY

Belagavi-590018

Technical Seminar Report on

“Wheel chair controller using eye ball movement”

Submitted in partial fulfillment as per VTU curriculum for VIII semester for the
award of degree of

Bachelor of Engineering
In
Electronics and CommunicationEngineering

by

TASMIYA S
1EP17EC062

Under the guidance of

Dr V NAVYA
Associate Professor
Department of Electronics and Communication Engineering.

Department of Electronics & Communication Engineering

Jnana Prabha Campus, Virgo Nagar Post, Bidarahalli.
Bengaluru – 560049
2019-2020
 Department of Electronics and Communication Engineering

CERTIFICATE

This is to certify that the Technical Seminar report entitled “Wheel chair
controller using eye ball movement” is a bonafide work carried out by TASMIYA S
bearing USN: 1EP17EC062 in partial fulfillment of the requirements for the eighth
semester for the award of degree of Bachelor of Engineering in Electronics and
Communication Engineering of Visvesvaraya Technological University, Belagavi,
during the academic year 2020-2021. The Technical seminar report has been approved as it
satisfies the academic requirements prescribed by the university.

Signature of the Guide Signature of the HOD

Dr V Navya, Dr Yogesh G S,
Associate Professor, Head of theDepartment,
Dept of ECE, Dept of ECE,
EPCET. EPCET.
 DECLARATION

I, TASMIYA S[1EP17EC062] student of 8th semester BE in Electronics and

Communication Engineering, East Point college of engineering and Technology,
Bengaluru, hereby declare that the “Technical Seminar” has been done during the
academic year 2020-2021, is a record of an original work done by me under the
guidance of Dr. V. Navya, Department of Electronics and Communication
Engineering, East Point college of engineering and Technology, Bengaluru. This
Technical seminar is done in partial fulfillment of the requirements for the
completion of 8th semester Bachelor of Engineering in Electronics &
Communication Engineering of the Visvesvaraya Technological University.

Date: TASMIYA S
Place: Bengaluru USN:1EP17EC062
 ACKNOWLEDGEMENT

Behind every achievement there lies a profound sea of gratitude to those who actuated it,
without whom it would never have come to existence. To them my praise of gratitude imprinted
not just on the paper but deep in our heart.

I express my profound gratitude towards Late Dr S M VENKATAPATHI, Founder

Chairman, East Point College of Engineering and Technology, for providing necessary
infrastructure and also honorable Dr PRAKASH S, Principal, EPCET, for creating good
environment for carrying out my internship.

I extend my earnest gratitude to Dr YOGESH G S, Head of the Department, Electronics

and Communication Engineering. I would also like to sincerely thank my guide Dr V Navya,
Associate Professor, Department of ECE, for her guidance and encouragement throughout
the course of my Technical Seminar. Her timely assistance and valuable suggestion were
motivation factor for completion of my Technical Seminar.

Dr Anita R, Associate Professor and Dr V Navya , Associate Professor, Dept of ECE,

Technical Seminar Coordinator, who has helped me in several ways to learn and explore things
by guiding me and providing me with all the required support. I also thank all teaching and non-
teaching staff of ECE Department, for their valuable guidance.

Last but not the least, I would like to utilize this opportunity to express love and sense
of gratitude to my beloved family and to my dearest friends for their support, strength.
 ABSTRACT

The disabled and elderly people who are in need of a wheelchair are estimated to be
over 100 million worldwide. A lot of these disabled people cannot use the traditional wheelchairs
as users may have restricted limb movements. An alternative is a powered wheel chair, a
mobility-aided device for persons with moderate/severe physical disabilities or chronic diseases
as well as the elderly. In order to take care for different disabilities, various kinds of interfaces
have been developed for powered wheelchair control; such as joystick control, head control, and
sip-puff control. Many people with disabilities do not have the ability to control powered wheel
chair using the above mentioned interfaces. The planned model is a possible alternative. It is an
attempt to make lives of the disabled people simpler, i.e., self-reliant, which will thereby reinstate
their confidence and their happiness. The idea is to create an Eye Monitored System which allows
movement of the patient’s wheelchair depending on the eye movements. A patient looks directly
at the camera mounted on a head gear and is able to move in a direction just by looking in that
direction. The camera signals are monitored by an Open CV application, which then guides the
motors wired to the Micro-controller over the Serial Interface to move in a particular direction.
The system is cost effective and thus can be used by patients spread over a large economy range.
 TABLE OF CONTENTS

CHAPTER 1 INTODUCTION…………………………………………………1
CHAPTER 2 DIFFERENT EYE TRACKING MECHANISM………………..3
2.1 ELECTRO OCLOGRAM (EOG) METHOD………………………………3
2.2 LENS TRACKING SYSTEMS…………………………………………….3
2.3 HEAD MOUNTED CAMERA SYSTEM………………………………….4
CHAPTER 3 BLOCK DIAGRAM……………………………………………..5
CHAPTER 4 PROPOSED SYSTEM…………………………………………...7
4.1 INDUINO X………………………………………………………………...8
4.2 RELAY BOARD……………………………………………………………8
4.3 ARDUINO…………………………………………………………………..9
CHAPTER 5 HARDWARE AND SOFTWARE REQUIREMENTS……….…11
CHAPTER 6 EYE BALL SENSOR WORKING………………………………16
6.1 PUPIL DETECTION………………………………………………………..17
CHAPTER 7 WORKING PROCESS…………………………………………..18
7.1 LEFT MOTION……………………………………………………………..19
7.2 RIGHT MOTION…………………………………………………………...19
7.3 STRAIGHT MOTION………………………………………………………19
7.4 BRAKING SYSTEM………………………………………………………..19
CHAPTER 8 METHODOLOGY……………………………………………….21
CHAPTER 9 ADVANTAGES AND APPLICATIONS………………………..27
CHAPTER 10 EXPERIMENTAL RESULT……………………………………28
10.1 TESTING STRATOGY……………………………………………………28
10.2 RESULTS………………………………………………………………….29

REFERENCE
 LIST OF FIGURES

Fig No Title Page No

3.1 Block diagram of eye movement based electric wheel chair 5

4.1 Flow of the proposed system 7
5.1 IBall CHD20.0 web camera 11
5.2 AT 89S52 development board 12
6.1 Eye ball sensor 17
6.2 Eye anatomy 17
7.1 Flow of controls from sensor 18
7.2 Flow of controls from sensor for braking the system 20
8.1 Face detector 24
8.2 Artificial example with a dark circle on a light background 25
8.3 The process flow of eye movement detection 26
10.1 Division of eye’s rectangular box 28
10.2 person looking in the centre 29
10.3 person looking right 30
10.4 person looking left 30
10.5 person looking up 30
Wheel chair controller using eye ball 2020-21

CHAPTER 1
INTRODUCTION

The ability to move freely is highly valued by all people. However, it is

sometimes difficult for a person with a physical disability. Conventional
wheelchair use tends to focus exclusively on manual use which assumes users
are still able to use their hands which excludes those unable to do so. Diseases
or accidents injuring the nervous system also frequently make people lose their
ability to move their voluntary muscle. Since voluntary muscle is the main
actuator enabling people to move their body, paralysis may cause a person not
move their loco motor organs such as arms and legs.

Transportation is of vital importance in this advanced world. The world is

changing towards automation. The aim of this seminar topic is to introduce
automation in transportation for the paralytic people. This seminar proposes
advanced model of existing system .This existing system can operate using
joysticks in hand or using head movement detection sensor. But this proposed
system operates in eyeball movement detection.

Nowadays, an electric wheelchair is commercially available for disabled

people. It generally requires considerable skill to operate. Moreover, some
disabled people cannot drive an electric wheelchair manually, even with a
joystick, because they lack the physical ability to control the movement.

Therefore it is required to enable a disabled person to drive a wheelchair safely

and easily so that they can enjoy a higher quality of life. The aim of the project
is to design a system for electric wheelchair control using the movement of eyes.
A commercially available web camera mounted on a head gear which the user
wears is used to capture moving pictures of the users face.

Department of ECE, EPCET Page 1

Wheel chair controller using eye ball 2020-21

A computer mounted on the electric chair processes the captured image data,
detecting and tracking movements of the user’s eyes, estimating the line-of-
sight vector, and actuating the electric wheelchair in the desired direction
indicated by the user’s eyes. One of the key essentials of the planned system is
detecting and tracking the eye movements.

Inside house paralytic people reaching a desired destination is little bit difficult.
So, paralytic people can’t move freely inside their house they depend on others
for their motion. Here with the help of eyeball movement detection system, the
new idea of providing a cost effective, less hardware complex embedded system
that helps the paralytic people to move freely inside their house.

Department of ECE, EPCET Page 2

Wheel chair controller using eye ball 2020-21

CHAPTER 2
DIFFERENT EYE TRACKING MECHANISMS

There are three eye tracking mechanisms to compute the position of pupil.

2.1 Electro-Oculogram (EOG) Method:

The Electro-Oculogram method obtains the gaze direction by sensing the

electro-oculographic potential. This is done by measuring the potential using
electrodes placed on face where human eye is an electric dipole with a negative
pole at the fundus and positive pole at the cornea. One electrode is placed to the
side of the left eye and another to the side of the right eye. This pair shall detect
horizontal eye movements. One electrode is placed above the left eye and
another below the left eye. This pair shall detect vertical eye movements. A fifth
electrode is attached by the ear to provide reference voltage. These electrodes
send the electrical signals to two EOG circuits of similar design to detect the
horizontal and vertical movement of the pupil. This information is sent for
computation. The big advantage of this method is the ability to detect eye
movements even when they are closed.

2.2 Lens Tracking Systems:

In this method a non slipping contact lens fits over corneal bulge. The tracking
of the pupil is recorded by affixing a magnetic coil or mirror to the lens. The
integrated mirror in the contact lens allows measuring reflected light;
alternatively, the integrated coil in the contact lens allows detecting the coil’s
orientation in magnetic field. The big advantage of this method is high accuracy
and nearly unlimited resolution in time. Both methods explained so far are
Department of ECE, EPCET Page 3
Wheel chair controller using eye ball 2020-21

obtrusive and are not suited well for interaction by gaze. The third and preferred
method for eye-gaze interaction is using video camera.

2.3 Head Mounted Camera System:

The most common mechanical setup involves use of desktop computer with
integrated eye tracker camera. The desktop computer is loaded with software
package for analyzing the eye gaze data. This setup require head fixation and
hence restricts the head movements. An alternative method uses a head mounted
eye tracking

Department of ECE, EPCET Page 4

Wheel chair controller using eye ball 2020-21

CHAPTER 3
BLOCK DIAGRAM

The block diagram of Eye Movement Based Electric Wheel Chair is shown in
Figure 3.1. The web camera is used to detect the eyes movement which is
further processed to drive the motors. Serial communication is used to
communicate between the web camera and the micro-controller. The micro-
controller placed on the wheel chair which is connected to the motors, driving
the wheel chair in the direction the person desires to move in.

Figure 3.1: Block Diagram of Eye Movement Based Electric Wheel Chair.

A. Camera

The first block is the video capturing module that captures sequence of eyes
images from the subject using a specially designed camera.

B. Open-CV Application

The camera is wired to the patient’s computer that runs an Open-CV

application designed to monitor and react to eye movements. Based on a series
of snapshots, image processing is performed and thereafter motion of the user’s
eye is detected.

Department of ECE, EPCET Page 5

Wheel chair controller using eye ball 2020-21

C. Serial Interface

The next block establishes the serial connection between the computer and the
micro controller. The decision to move the motor in a particular direction is
communicated to the micro-controller using the serial interface.

D. Micro-controller

The micro-controller on reception of signal from the serial interface forces the
port pin high on which the motors have been connected for desired motion of
the Wheel Chair.

E. Wheel Chair Prototype

The final block is the assembly consisting of motors, wheels and the chassis
responsible for the disabled person’s movement. The motors receive the
electrical signal from the Micro-controller and drive the wheel chair in the
desired direction.

Department of ECE, EPCET Page 6

Wheel chair controller using eye ball 2020-21

CHAPTER 4
PROPOSED SYSTEM

In this seminar topic proposals importance is to serve all types of paralytic

people to move freely to smaller distances inside their residence. This system
uses only eyeball detection using eyeball IR sensor. The sensor consists of two
parts infrared transmitter and receiver.

Infrared transmitters and receivers are housed into a single unit and fixed in
front of the goggles. Necessary wiring's are takes through the goggles to the
sensor processing. The person should wear the goggles for the sensor to work.
Since the sensor is shielded it can be isolated from the external light, thus
external light illumination will not affect the sensor output values.

Figure 4.1: Flow of the proposed system.

Department of ECE, EPCET Page 7

Wheel chair controller using eye ball 2020-21

The flow of our proposed system starts with the detection of the eyeball
position. In order to detect the position of the eyeball we use the eyeball IR
sensor. The values generated by the sensor depending on the position of the
eyeball are routed to the Arduino kit. Next the values in the Arduino kit are
processed and sent to the relay as the enable signals to switch on the relays and
as the final step when the relays are made to switch on the motors fitted to the
wheels of the wheelchair will start to rotate and thus the wheelchair will be
locomoted.

4.1 InduinoX

The Induino board consists of an 8-bit Atmel AVR micro-controller with

complementary components to facilitate programming and incorporation into
other circuits. An important aspect is the standard way that connectors are
exposed, allowing the CPU board to be connected to any external devices and a
variety of interchangeable add-on modules (known as shields). Also, the on-
board components on Induino-X can be connected/disconnected to/from the i/o
pins by using jumpers. The Induino can be connected to a computer and
programmed using a standard USB. A FTDI FT232 is used for USB to serial
conversion. In-spite of an internal oscillator in the microcontroller, the boot-
loader is configured such that it works with a 16MHz external clock oscillator.

Shields

Shields are the add-on circuits which are mounted on the Induino board, which
gives extended functionality from the available functionality of the board.

Department of ECE, EPCET Page 8

Wheel chair controller using eye ball 2020-21

4.2 Relay Board

A relay is an electrically operated switch. Many relays use an electromagnet to

mechanically operate a switch, but other operating principles are also used, such
as solid-state relays. Relays are used where it is necessary to control a circuit by
a low-power signal (with complete electrical isolation between control and
controlled circuits), or where several circuits must be controlled by one signal.
The first relays were used in long distance telegraph circuits as amplifiers: they
repeated the signal coming in from one circuit and re-transmitted it on another
circuit. Relays were used extensively in telephone exchanges and early
computers to perform logical operations.

A type of relay that can handle the high power required to directly control an
electric motor or other loads is called a contactor.Solid-state relays control
power circuits with no moving parts, instead using a semiconductor device to
perform switching. Relays with calibrated operating characteristics and
sometimes multiple operating coils are used to protect electrical circuits from
overload or faults; in modern electric power systems these functions are

performed by digital instruments still called "protective relays".

4.3 Arduino

The Arduino is what is known as a Physical or Embedded Computing platform,

which means that it is an interactive system, that through the use of hardware
and software can interact with it's environment. For example, a simple use of
the Arduino would be to turn a light on for a set period of time, let's say 30
seconds, after a button has been pressed. In this example, the Arduino would

have a lamp connected to it as well as a button. The Arduino would sit patiently
waiting for the button to be pressed. When you press the button it would then

Department of ECE, EPCET Page 9

Wheel chair controller using eye ball 2020-21

turn the lamp on and start counting. Once it had counted 30 seconds it would
then turn the lamp off and then carry on sitting there waiting for another button
press. You could use this set-up to control a lamp in an under-stairs cupboard
for example. You could extend this example to sense when the cupboard

door was opened and automatically turn the light on, turning it off after a set
period of time.

Department of ECE, EPCET Page 10

Wheel chair controller using eye ball 2020-21

CHAPTER 5
HARDWARE AND SOFTWARE REQUIREMENTS

5.1 HARDWARE REQUIREMENTS

The hardware components used in the system design are explained in detail in
the following sections.

A. Camera

A 720 pixels high definition USB web camera of 5500 x 3640 pixels image
resolution and 1920 x 1080 pixel video resolution has been used for better
clarity of image. It has a frame rate of 30 frames per second for fast image
processing. It uses 6 LED's for better image processing during night time and
has a cable length of 1.5 meters. The Web camera is shown in Figure 5.1.

Figure 5.1: iBall CHD20.0 Web camera.

Department of ECE, EPCET Page 11

Wheel chair controller using eye ball 2020-21

B. AT 89S52 Development Board

With the development board one can develop and prototype any of 8051 40
pin Micro-controllers. The RS232 driver on board allows easy connection with
PC or other embedded hardware. The board has User buttons and status LED's.
The bridge rectifier allows this board to be powered with both AC and DC
power supply adapters. The Development board is shown in Figure 5.2. It has
following features:

1. RS232 TX, Rx interface with MAX232 IC on socket

2. Quartz crystal 11.0592Mhz

3. Reset button

4. Power plug-in jack

5. Extension slot on every uC pin

6. Gnd bus

7. Vcc bus

8. Four mounting holes 3.3 mm (0.13")

Figure 5.2 : AT 89S52 Development Board.

Department of ECE, EPCET Page 12

Wheel chair controller using eye ball 2020-21

C. AT89S52 Micro-controller

The AT89S52 is a low-power, high-performance CMOS 8-bit micro-controller

with 8Kbytes of insystem programmable Flash memory. The device is
manufactured using Atmels highdensity nonvolatile memory technology and is
compatible with the industry standard 80C51 instruction set and pinout. The
AT89S52 provides the following standard features: 8K bytes of Flash, 256 bytes
of RAM, 32 I/O lines, Watchdog timer, two data pointers, three 16-bit
timer/counters, a six-vector two-level interrupt architecture, a full duplex serial
port, on-chip oscillator, and clock circuitry. In addition, the AT89S52 is
designed with static logic for operation down to zero frequency and supports
two software selectable power saving modes. The Idle Mode stops the CPU
while allowing the RAM, timer/counters, serial port, and interrupt system to
continue functioning. The Powerdown mode saves the RAM contents but
freezes the oscillator, disabling all other chip functions until the next interrupt
or hardware reset.

D. RS 232 Standard

Communication defined in the RS-232 standard is an asynchronous serial

communication method. The word serial means that the information is sent one
bit at a time. Asynchronous tells that the information is not sent in predefined
time slots. Data transfer can start at any given time and it is the task of the

receiver to detect when a message starts and ends. The RS 232 standard defines
the voltage levels that correspond to logical one and logical zero levels. Valid
signals are plus or minus 3 to 25 volts. The range near zero volts is not a valid

Department of ECE, EPCET Page 13

Wheel chair controller using eye ball 2020-21

RS 232 level; logic one is defined as a negative voltage, the signal condition is
called marking and has the functional significance of OFF. Logic zero is
positive; the signal condition is spacing and has the function ON. So a logic
zero represented as +3V to +25V and logic one is represented as -3V to -25V.
Since RS232 voltage levels is not compatible to TTL voltage level, voltage or
level converter is needed which can convert TTL to RS232 and RS232 to TTL
voltage levels. MAX 232 IC includes charge pump which can generate RS232
voltage levels (-10V and +10V) from 5V power supply. It also includes two
receiver and two transmitters and is capable of full-duplex Universal
Asynchronous Receiver Transmitter/Universal Synchronous Asynchronous
Receiver Transmitter (UART/USART) communication.

E. Motor Driver Circuit

L293D is a typical Motor driver or Motor Driver IC which allows DC motor to

drive on either direction. L293D is a 16pin IC which can control a set of two
DC motors simultaneously in any direction. It means that you can control two
DC motor with a single L293D IC. It works on the concept of H-bridge. H-
bridge is a circuit which allows the voltage to be own in either direction.

As we know voltage need to change its direction for being able to rotate the
motor in clockwise or anticlockwise direction, hence H-bridge IC are ideal for
driving a DC motor. In a single l293d chip there two h-Bridge circuit inside the
IC which can rotate two dc motor independently. Due its size it is very much
used in robotic application for controlling DC motors.

Department of ECE, EPCET Page 14

Wheel chair controller using eye ball 2020-21

5.2 SOFTWARE REQUIREMENTS

The software's used for creating and programming of the various modules to
obtain the desired results are as follows:

A. Open-CV 3.0.0

Open-CV (Open Source Computer Vision Library) is an open source BSD-

licensed library that includes several hundreds of computer vision algorithms.
Open-CV has a modular structure, which means that the package includes

several shared or static libraries.

B. Eclipse 3.8

In computer programming, Eclipse is an integrated development environment

(IDE). It contains a base workspace and an extensible plug-in system for
customizing the environment. Eclipse is written mostly in Java and its primary
use is for developing Java applications, but it may also be used to develop
applications in other programming languages through the use of plugins,
including: Ada, ABAP, C, C++, COBOL,Open CV, Fortran, Haskell, JavaScript,
Julia, Lasso, Lua, NATURAL, Perl, PHP, Prolog, Python, R, Ruby (including
Ruby on Rails framework), Rust, Scala, Clojure, Groovy, Scheme, and Erlang.

Department of ECE, EPCET Page 15

Wheel chair controller using eye ball 2020-21

CHAPTER 6
EYE BALL SENSOR WORKING

The eyeball sensor is based on the concept that “white color region of the eye
will scatter the light and black color region of the eye will absorb the light”. The
sensor consists of IR transmitter and receiver circuitry. The IR transmitter will
transmit the light. The iris of the eye which is in black color will absorb all the
light and it won’t reflect where as the white part will reflect the light.

At the receiver side there will be LDR or Light Dependent Resistor these
LDRs are used at the dark/light sensing area. Thus when the eyeball of the
patient is illuminated by the light in the transmitter side, partial of the
illuminated light will be absorbed by the black region of the eye and part of the
light will be reflected by the white region of the eye as mentioned above.

Thus by receiving the analog light pulses the Light Dependent Resistors at the
receiver side which has ohmic resistance about 100000 will absorb the light, on
absorbing the light pulses the ohmic values of the sensor will decrease or
increase depending on the intensity of the light which in turn sends digital
values as the output to the Arduino kit.

Department of ECE, EPCET Page 16

Wheel chair controller using eye ball 2020-21

Fig 6.1: Eyeball sensor.

6.1 Pupil detection

The pupil is a hole located in the center of the iris of the eye that allows light
to enter the retina. It appears black because light rays entering the pupil are
either absorbed by the tissues inside the eye directly, or absorbed after diffuse
reflections within the eye that mostly miss exiting the narrow pupil. Pupil - the
opening in the center of the iris- it changes size as the amount of light changes
(the more light, the smaller the hole).

Figure 6.2 : Eye anatomy

Department of ECE, EPCET Page 17
Wheel chair controller using eye ball 2020-21

CHAPTER 7
WORKING PROCESS
The eyeball sensor which is focused to the eyeball of the patient generates
three different ranges of values depending upon the position of the eyeball.

Figure 7.1: Flow of controls from sensor.

As shown in the above flowchart, values generated by the sensor is fed into the
Arduino kit. Now the Arduino kit matches it with the values that is written in
the coding

Department of ECE, EPCET Page 18

Wheel chair controller using eye ball 2020-21

7.1 Left Motion

If the values generated by the sensor matches with the values of left in the
coding, then the arduino will automate the relay connected to the right motor.
Thus the right motor will start to rotate in the forward direction. As a result the
wheelchair will turn left. During this time the left wheel is stopped.

7.2 Right Motion

If the values generated by the sensor matches with the values of right in the
coding, then the arduino will automate the relay connected to the left motor.
Thus the left motor will start to rotate in the forward direction. As a result the
wheelchair will turn right. During this time the right wheel is stopped.

7.3 Straight Motion

If the values generated by the sensor matches with the values of straight in the
coding, then the arduino will automate the relay connected to the right motor
and also the left motor. Thus the right and left motors will start to rotate in the
forward direction. As a result the wheelchair will move straight.

7.4 Braking System

The system doesn’t have any manual braking. Eyeball sensor will also produce
two binary values ‘1’ and ‘0’. The value ‘1’ is produced when the patients eye is
opened and one of the above three motions will be selected and the wheel chair
will be moved accordingly. The value ‘0’ is produced when the patient eye is
closed and this value is passed into the Arduino kit, which in turn will switch
off the relay(s) which are enabled at that time.

Department of ECE, EPCET Page 19

Wheel chair controller using eye ball 2020-21

Figure 7.2: Flow of controls from sensor for braking the system.

Department of ECE, EPCET Page 20

Wheel chair controller using eye ball 2020-21

CHAPTER 8

METHODOLOGY

There are two major components from the system design standpoint:

1. Eye-Detection and motion tracking.

2. Micro-controller controlled Wheel Chair Assembly.

8.1 Eye-Detection and Motion Tracking

A web camera is mounted on a cap, continuously staring at the users eyes. The
web camera wired to the users laptop runs an Open CV application designed to
monitor and react to eye movements. Based on a series of snapshots taken and
thereafter processed, the motion of the users eyes are detected and decision to
move the Wheel Chair in a particular direction is taken. An image processing
toolbox which uses Viola Jones Algorithm [3] is used that is capable of
detecting eye-shaped objects based on their shape and size. Further the pupil of
eye is detected using gradient method [4] and its centre is located. Based on the
position of pupil, a movement is detected and this is communicated to the
wheelchair assembly via the serial port.

8.2 Micro-controller controlled Wheelchair Assembly

A decision based on the processing done by the Open CV application is

communicate and received by the micro controller. The controller on reception
forces the port pin high on which the motors have been connected for desired
motion of the Wheel Chair.

Department of ECE, EPCET Page 21

Wheel chair controller using eye ball 2020-21

8.3 Process Flow

1. Initialization

Initially the serial communication is set up is to be used later for the interface
between Open CV and the controller, the video capture and the program
variables.

2. Image and Video Processing

Then continuous video frames is taken and the input is sampled and saved as
screenshots. Each frame is then converted into black and white frames. For
accurate results, contrast stretching is performed on each frames to make the
dark regions much darker and bright regions much brighter. This ensures better
detection of the eyes.

3. Estimation

Now, after working on each frame, the eyes are to be detected. This is done by
estimation of the position of left as well as the right eye. Thus, a threshold is set
and the position of the eyes are detected which can be used for the further
processing.

4. Detection

Now, the eye movements are to be detected. The idea is to compare the current
position of the eye with the previous position. The difference in the coordinates
helps to predict the motion in the particular eye. But sometimes, it might be

Department of ECE, EPCET Page 22

Wheel chair controller using eye ball 2020-21

possible that only one of the either eye is detected. In that case, the eye that is
detected correctly is preferred.

5. Error Handling

To avoid errors in detection, an error handling mechanism is incorporated,

which specifies a threshold for the height and width of a valid eye, through
calibration of system for a particular user. If the detection results give a height
and width value lesser or greater than the threshold, the value is avoided and it
is not considered for the decision making.

6. Motion

Now after detection of the eye movements, a decision algorithm needs to be

considered that helps the controller drive the motors accordingly:

a) Valid Left: The decision to turn left is considered as valid if the eye turns left
and stays there for a cycle. This action is detected as a left turn request. After
that, the patient turns right to again look forward.

b) Valid Right: Similarly, the decision to turn right is considered as valid if the
eye turns right and stays there for a cycle. This action is detected as a right turn
request.

c) Valid Straight: The signal to go straight is when a person looks upwards. This
is detected as a forward movement request.

d) Halt: The decision to bring the wheel chair to a halt is considered when the
pupil is in center or the patient is looking directly into the camera.

Department of ECE, EPCET Page 23

Wheel chair controller using eye ball 2020-21

8.4 Eye Centre Localisation by means of Gradient

A multi-stage scheme is followed that is usually performed for feature-based

eye centre localisation (see Figure 8.1), and following observations are made:

1. A novel approach for eye centre localisation, which defines the centre of a

circular pattern as the location where most of the image gradients intersect.

Therefore, a mathematical function is shown that reaches its maximum at the

centre of the circular pattern. By using this mathematical formulation a fast

iterative scheme can be derived.

2. We incorporate prior knowledge about the eye appearance and increase the
robustness.

3. We apply simple post processing techniques to reduce problems that arise in

the presence of glasses, reflections inside glasses, or prominent eyebrows.
Furthermore, we evaluate the accuracy and the robustness to changes in lighting,
contrast, and background by using the very challenging Bio ID database.

Figure 8.1: A face detector is applied first; based on the face location rough eye
regions are

extracted (left), which are then used for a precise estimation of each eye centre
(middle and right). The relationship is described mathematically between a
possible centre and the orientations of all image gradients. Let c be a possible

Department of ECE, EPCET Page 24

Wheel chair controller using eye ball 2020-21

centre and gi the gradient vector at position xi. Then, the normalized
displacement vector di should have the same orientation (except for the sign) as
the gradient gi (see Figure 8.2). If we use the vector field of (image) gradients,
we can exploit this vector field by computing the dot products between the
normalized displacement vectors (related to a fixed centre) and the gradient

vectors gi. The optimal centre c* of a circular object in an image with pixel
positions xi,. i ϵ {1,...,N }, is then given by

Figure 8.2: Artificial example with a dark circle on a light background, similar
to the iris and the sclera. On the left the displacement vector di and the gradient
vector gi do not have the same orientation, whereas on the right both
orientations are equal.

Department of ECE, EPCET Page 25

Wheel chair controller using eye ball 2020-21

8.5 Flowchart

The process flow is shown in Figure 8.3.

Figure 8.3: The Process Flow of Eye Movement Detection

Department of ECE, EPCET Page 26

Wheel chair controller using eye ball 2020-21

CHAPTER 9

ADVANTAGES AND APPLICATIONS

9.1 ADVANTAGES:

o The existing eye tracking methods for loco-mating the powered wheel chair

are based on image processing techniques thus it is tedious to work with images.
But our system uses only eyeball sensor which tracks the position of the eye by
using a simple components light dependent resistor, comparator, IR led.

o Calculating the threshold values of position of the image processing

techniques are complex but using the eyeball sensor we can easily calculate the
threshold values.

o Using the eyeball sensors instead of the CCD camera will also reduce the cost
of the total system dramatically

9.2 APPLICATIONS:

 Application to mobile and communication aid for paralytic people.

 Medical Field.

Department of ECE, EPCET Page 27

Wheel chair controller using eye ball 2020-21

CHAPTER 10

EXPERIMENTAL RESULT

10.1 Testing Strategy

After detection of the eye, a rectangular box is created around the eye. The
rectangular box is divided vertically into four parts and horizontally into three
parts as shown in Figure 10.1. If the pupil is found in the 1st horizontal division

(shown in Purple), the wheelchair moves forward. Similarly if it is found in the

centre of horizontal as well as vertical division or in the 3rd horizontal division
(shown in Brown), the wheelchair halts. If the pupil is found in the 4th vertical
division (shown in Blue), the wheelchair turns left. If the pupil is found in the
1st vertical division (shown in Pink), the wheelchair turns right.

Figure 10.1: Division of eye's rectangular box

Department of ECE, EPCET Page 28

Wheel chair controller using eye ball 2020-21

10.2 Results

The various eye movements detected by the system corresponds to the

movement of wheel chair in the indicated direction. Figure 10.2 shows a person
who looks straight into the camera, the pupil is detected in the centre of the
rectangular box which is processed by the system as centre and therefore makes
the wheel chair halt. Similarly, Figure 10.3 shows a person with movement of
eye in right direction, the pupil is detected in the 1st vertical division that leads
to right turn of wheel chair. Figure 10.4 depicts a person with left eye
movement, the pupil is detected in the 4th vertical division that corresponds to
the left turn of wheel chair. Figure 10.5 shows a person who looks upward, the
pupil touches the 1st horizontal division, unlike Figure 10.2 where the pupil lies
in the centre, the system guides the wheel chair forward. The system is tested on
four users under different lighting conditions (day, night, dim light, no light) for
100 to 200 attempts. The numbers of successful attempts are noted against the
total number of attempts and accuracy is computed for each user which is
tabulated in Table 10.1. The accuracy results after testing is found to be more
than 90%.

Figure 10.2: Person looking in the centre, wheel chair doesn't move (or stops).

Department of ECE, EPCET Page 29

Wheel chair controller using eye ball 2020-21

Figure 10.3: Person looking right, wheel chair turns right.

Figure 10.4: Person looking left, wheel chair turns left.

Figure 10.5: Person looking up, wheel chair moves forward.

Department of ECE, EPCET Page 30

Wheel chair controller using eye ball 2020-21

Table 5.1: Accuracy Results

Department of ECE, EPCET Page 31

Wheel chair controller using eye ball 2020-21

REFERENCE

[1] Kohei Arai and Ronny Mardiyanto. Eyes based electric wheel chair control
system. International Journal of Advanced Computer Science and Applications,
2(12):98-105, November 2011.

[2] Gunda Gautam, Gunda Sumanth, Karthikeyan K C, Shyam Sundar, and

D.Venkataraman. Eye movement based electronic wheel chair for physically
challenged persons. International Journal of Scientific and Technology
Research, 3(2):206-212, February 2014.

[3]Yi-Qing Wang. An analysis of the violajones face detection algorithm. Image

Processing On Line, 4:128-148, June 2014.

[4] Fabian Timm and Erhardt Barth. Accurate eye centre localisation by means
of gradients. Proceedings of the Int. Conference on Computer Theory and
Applications (VISAPP), 1:125-130, 2011.

Department of ECE, EPCET Page 32