Fin Irjmets1657369160

e-ISSN: 2582-5208
International Research Journal of Modernization in Engineering Technology and Science

( Peer-Reviewed, Open Access, Fully Refereed International Journal )
Volume:04/Issue:07/July-2022 Impact Factor- 6.752 www.irjmets.com
EYE GAZE COMMUNICATION SYSTEM FOR
DIFFERENTLY ABLED PEOPLE
Adarsh R*1, Biswa Ranjan Rao*2, Febi Sarah Binu*3,
Pooja R*4, Prof. Srinivas Halvi*5
*1,2,3,4B.E, Student, Department Of Medical Electronics, Dayananda Sagar College Of
Engineering, Bangalore, India.
*5Assistant Professor, Department Of Medical Electronics, Dayananda Sagar College Of
Engineering Bangalore, India.
ABSTRACT
The number of paralyzed people keep increasing year by year perhaps because of two common factors. Some
people have unfortunate mishaps, while others are born with it.. Since communication is an important aspect in
our life, these paralyzed people may not be able to communicate. They cannot use their hands or mouth to
conversate and communicate but they might have a sound vision and can control their eye movements. Hence
by using the movements of the eye, we propose a low-cost eye gazing communication system. We use a camera
module which is used to video capture the face of the subject and by using various image processing algorithms
such as the Voila-Jones algorithm, we detect the face of the individual. We use the facial landmarks to slice out
the eyes of the subject, i.e., the left eye and the right eye. To precisely detect the motion of the eye, we try to
track the iris movement which is the most prominent feature in the eye. Using the Hough’s circle algorithm
which detects the most circular shape object in the given image i.e., the iris, we successfully track the movement
of the iris. Once we track the movement of the iris, we design a Graphical User Interface (G.U.I) using Python
which is displays various communication features such as Doctor/Nurse, Washroom Assist, Family,
Food/Water etc. on the screen in front of the subject where the subject can navigate through and select the
desired option using the position of iris and eye blink i.e., if the patient gazes right, the screen shifts to a new
communication option, if the patient gazes left, the screen shifts to the previous communication option and if
the patient blinks the current communication option is selected. When the patient blinks on a particular option,
message gets sent to the concerned person using GSM module where the concerned person gets notified that
the patient needs assistance. All the modules and monitor are interfaced with a small minicomputer called
Raspberry Pi which controls the entire system.
Keywords: Face Detection, Iris Detection, Graphical User Interface, Raspberry Pi.
I. INTRODUCTION
Communication is an important aspect in the life of a living being. Without communication life becomes harder.
People suffering from paralysis and motor neuron disease may not be able to communicate like us. As paralyzed
people not have sensation to make any motion using their limbs. Hence they lose their ability to communicate
which prevents them to express their basic needs and necessity which results them of being dependent on others
for all their needs. These complications of paralysis can dramatically affect their both physical and mental health
and also quality of life[2]. Hence providing a chance to communicate at least for their needs gives them hope.
The only voluntarily motion controlled by disabled is movement of their eye. It is estimated 150,000 severely
disabled persons able to control only the muscles of their eyes without any problem. Under this circumstance, an
eye tracking system may provide an alternative option for people with severe disabilities who only retain the
ability to move their eyes[3]. We can develop system that detects the motion of the eye gaze of the disabled
person for communication purpose.
Hence by using the movement of eye, we propose a low cost-based eye gaze communication system to aid in
communication[1]. The movement of the eye ball is tracked by locating the position of iris using various image
processing techniques. Once the eye tracking is obtained, various eye gestures can be implemented in a graphical
user interface helpful in communication.
www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

[887]
e-ISSN: 2582-5208
Infrared sensors are used to detect the presence of the patient within a certain proximity, when the patient is
detected the camera module switches through which face is detected using the Viola-Jones algorithm[9],
followed by extracting the eye coordinates by using facial landmarks and using Hough’s circle to detect the
presence of iris. Once the iris is detected, using the position of iris we interface this feature to a Graphical User
Interface (GUI) displayed on a monitor, which contains various options for communication that aids the patient.
The communication messages will be sent to the concerned person via SMS using the GSM module. Raspberry Pi
3B acts as a mini computer onto which all components are interfaced and operated.
II. RELATED WORK
A technique of dominant mouse pointer on a monitor victimizes the electrical potentials developed by the eye
movements called Electrooculography (EOG) signals [4] which are employed to detect blink patterns and motion
of the eye.The technology is advantageous as it yields high accuracy but incorporation of the above idea may
cause discomfort to the subject when used over a longer period of time. This paper was taken to be the base
ideology for this project by eradicating all the strains faced by the patient.
Poonam Potraje el. and Prof. P.S. Kulkarni[5] proposed a system to control the cursor of the computer with the
help of user’s eye and allows the user to use the computer system with the help of user’s eyesight and move the
cursor as per the direction that are given by the user. Using web camera, the subjects face is detected using
feature-based approach where the author uses Haar Cascade classifiers to obtain integral values of facial
landmarks and by using the Viola Jones algorithm feature such as eye is extracted by applying the Leonardo da
Vinci rule and store it in a template. The cursor movement is obtained by serving the central coordinate as an
initial point and by calculating the distance between the initial point and the distance moved by the eye, the
cursor would move accordingly. We incorporate the above methodology of eye detection using the Haar
classifier and Voila Jones algorithm and modify the methodology a continuous iris detection.
Arulmozhi. K, Dharshini. K, and Kaviyasree. P. [6] designed and implemented the cursor control system based on
the movement of the eye ball. The cursor is then controlled by tracking the eyeball's movement accordingly and
gestures like blinking enables enter and blinking twice enables right click and left click. Raspberry pi is the key
element in processing module which keeps on monitors eye movement by interfacing with various sensor .Once
when the sensor detects the movement of eye pupil the camera starts to capture the images and send it into the
Raspbian board through an USB cable. Then the transferred image is processed and monitored out. We
incorporate the idea of using hardware components such as PIR sensor, Raspberry pi along with the camera
module which is interfaced with the microcomputer which can be advantageous in terms of portability.
The system is a computer interface that provides the functionality of an input device like mouse based on eye
actions such as eye blink, eye-gaze and gaze control [7]. For initialization, the subject just holds his/her head still
for a few seconds: using involuntary blinks that would naturally occur during this time, the system both locates
the user’s eye pair as well as forms online templates of the open and shut eyes of the specific user, valid for the
rest of that session. We incorporate the idea of blink detection which when implemented can be used in our GUI.
Eye tracking technology has been used for human–computer interaction, virtual reality, eye disease diagnosis,
human behavior studies, etc. An efficient eye tracking system having a feature of eye blink detection for
controlling an interface that provides an alternate way of communication is proposed by Sidra Naveed et al [8]
for the disabled by using their pupil portion for tracking the movement of eyes. We use a similar modified
algorithm in our project and implement the above application on a Graphical User Interface.
III. METHODOLOGY
3.1. Hardware
The proposed system has a camera module, PIR Sensor, GSM module, Monitor and SD card interfaced onto
Raspberry Pi as shown in block diagram in Fig 1

[888]
e-ISSN: 2582-5208
Fig 1: Hardware Block diagram

IR Sensor: It uses Infrared light to detect the presence of a person in its field of view which in turn activates the
camera module for image acquisition.
Camera Module: The camera module is activated using the PIR sensor which helps in video capturing of face of
the subject and by using image processing techniques the subjects face and iris are accordingly detected.
Monitor: The monitor is interfaced with HDMI port of Raspberry Pi. It is used to display the Graphical User
Interface designed. The monitor is setup in front of the subject which have soft keys where they can navigate
through using iris movement which will aid assistance to the subject.
GSM Module: It provides a data link to remote network. The GSM module is interfaced to the GPIO pins of
Raspberry Pi. It is used to send a message to the doctors and family members in case of emergencies or in
requirement of any aid.
Level Shifter circuit: The Raspberry Pi is a 3.3V single-board computer, but the SIM900A modem is a 5V
device. The SIM900 modem's TTL serial port cannot be directly connected to the RPi's serial TTL port. Because
the 5V modem can withstand 3.3V UART voltages, if we connect the serial ports of the modem and RPi directly,
it will communicate the serial data to the modem. However, the 5V UART signals from the modem can harm or
even destroy the serial reception pin on the Raspberry Pi when we attempt to accept data from it. Therefore,
we use a 5V to 3.3V TTL Logic Shifter to be safe. The 1N4148 diode can be used to create a straightforward 5V-
to-3V3 logic shifter, as shown in the circuit schematic in Fig 2:
Fig 2: Level Shifter Circuit

SD Card: It is used to store and load the operating system and program into the Raspberry pi module through
which a Raspberry Pi can be used as a mini computer.
3.2. Software
A detailed block diagram of software implementation is shown in Fig 3.
Video Capture: The Raspberry Pi camera module captures the real time video of the subject.
[889]
e-ISSN: 2582-5208
Face Detection: After the camera captures the video of the individual the face is detected using the Viola Jones
algorithm used to detect face of the user very rapidly and correctly in the image.
Fig 3: Block diagram for Iris Detection

3.2.1. VOILA JONES ALGORITHM
Typically, we use a deep learning algorithm for face detection, but deep learning will be computationally
inefficient due to large amount of data set required hence we use Voila Jones algorithm which uses very simple
features for face detection[9]. It majorly comprises of 4 steps:
Haar Feature: Haar features are used to detect the presence of features in the given image. These Haar features
are masks which scans through the image and the difference between the sum of the black region and the white
region gives us a single value which is the unique feature. It uses a 24x24 window as base window and starts
evaluating the features in the given image. The total parameters extracted in the image is approximately more
than 160000 features.
Fig 4: Haar Feature

Integral image: Summing up the intensity values of an image does not look computationally efficient hence we
use the idea of integral images by performing simple computation by using the corner values of the region.
Adaboost: When an image is passed through, the algorithm detects more than 160000 features which is
computationally inefficient, hence Adaboost eliminates all the redundant features and preserves the relevant
features. Adaboost is a machine learning algorithm which helps in finding the best features among the 160000
features. It provides weights to all these features and a linear combination of all features is used to decide
whether the image is a face or not. Generally, 2500 features are used to form a strong classifier.
Fig 5: Adaboost
[890]
e-ISSN: 2582-5208
Cascading: For example, an image of resolution 640x640, we move 24x24 window all across the image and in
each window it evaluates through those 2500 features which is computationally inefficient. Hence we use the
process of cascading. Cascading is the process where the features are subdivided into various cascades in the
form of a binary tree pattern where the image slides through each cascade where the features are matched and if
matched are continued on rest of the cascades.
Fig 6: Cascading
Eye Detection: Once the face is detected we locate the position of the eye using facial landmarks which is a
group of 64 points which marks all the features on the face as shown in the diagram. Using these landmarks, we
can slice the positions of the eye and use it for iris detection.
Fig 7: Facial Landmarks

Iris Detection: Once the eye is detected our next step is to locate the iris where we look for the most circular
object in eye region which can be facilitated by using Hough’s Circle.
Hough’s Circle
In order to find circles in defective images, the circle Hough Transform (CHT) is a fundamental feature extraction
technique used in digital image processing. The circle candidates are created by choosing local maxima in an
accumulator matrix after "voting" in the Hough parameter space.. A Hough circle transform is an image
transformation that enables the extraction of circular objects from images, regardless of how full the circle may
be. Additionally, the transform is only selective for circles and typically ignores extended ellipses. High radial
symmetry objects are effectively sought after by the transform, which gives each degree of symmetry one "vote"
in the search space.
The Hough’s algorithm is applied on the sliced image of the eye where it finds the most circular object present in
the image (eye) i.e., the iris. After finding the edge map of input eye image by canny edge detector, next step is to
find radius and center co-ordinates of the outer and inner circular boundary of iris region. Here the radius and
center co-ordinates of both the circles are to be found. The Hough transform is used to determine parameters of
simple geometric objects, such as lines, ellipses and circles, present in an image. The circular Hough transform is
employed to find out the radius and center coordinates of the circular boundary of pupil and iris outer boundary.
3.3. Graphical User Interface
Once the iris is detected and the location of iris is tracked, we interface this mechanism on a graphical user
interface which is shown in Fig 8.

[891]
e-ISSN: 2582-5208
Fig 8: Block diagram for integration of Iris Detection with Graphical User Interface
Instead of text-based UI’s, typed command labels, or text navigation, the graphical user interface enables people
to interact with electronic devices through graphical icons and auditory indicators like primary notation. In the
field of human-computer interaction, designing a GUI's visual layout and temporal behavior is a crucial
component of software application programming. Initially the face of the patient is detected and then the eye is
captured followed by which the iris is detected and used for eye gaze movement. When the patient is at a
certain accessible distance, the screen which was initially in an idle mode, switches on with the detection of a
nearby patient using the IR sensor. The information of the patient is shown on the very 1st screen which
includes various information about the patient.
For the patient to move to the next application of screen, the patient has to gaze towards the right side of the
screen which contains a button to move to the next screen where the succeeding applications can be accessed.
If the patient blinks the any current screen, the main application contained in that screen with get executed. For
e.g., if the patient blinks on the screen which consists of the function of doctor /nurse, a message will go to the
concerned professionals. If the patient has to go back to any previous screen, the patient can gaze towards the
left which has the option for switching to the previous page and therefore the screen shifts back to the previous
page.
Tkinter
The Tk GUI toolkit has a Python binding called Tkinter. The Tk GUI toolkit's standard Python interface is what it
is called. Using the widgets in the Tk toolkit, this framework gives Python programmers a quick way to make
GUI elements. Button, menu, data field, and other widgets can be built with Tk widgets in a Python program.
These graphic elements, once developed, can be connected to or interact with features, functionality, processes,
data, or even other widgets. A button widget, for instance, can both accept mouse clicks and be programmed to
carry out an action, like closing the application. Various GUI communication screens aiding the patients are
shown in Fig 9.
Fig 9a: Patient information screen and index Fig 9b: Doctor/Nurse screen

[892]
e-ISSN: 2582-5208
Fig 9c: Washroom assist screen Fig 9d: Family assist screen
Fig 9e: Food/Water Assist Screen Fig 9f: Confirmation Screen
Fig 9g: Shutdown Screen

Fig 9: Various Screens Of Graphical User Interface Which Aids The
Patient In Communication
IV. RESULTS AND DISCUSSION
The facial features are detected using the Viola Jones algorithm and facial landmarks. Using Hough’s circle, we
detect the presence of iris in the eyes which can be used for communication via GUI. The position of iris was
successfully tracked highlighting the position such as right, left, center as shown in the below figure which can
help the patient to navigate through the various communication screen shown on monitor. The blink was
successfully detected by using the concept of Euclidean distance which can help the patient to select the
particular communication aid he needs and a message will be sent to the concerned person for his need. This is
designed primarily for paralyzed patients where their eye movements could be a key factor in communication.
The procedure of this system is explained below in the Figure 8. When the patient accesses the GUI, an index
screen(Fig:7a) that shows the options provided in subsequent screens. If the patient needs family assistance
screen to have their presence, he gazes onto the next page option until he gets to that screen passing a set of
confirmation messages. In the family screen, the patient blinks on family assist option and soon a confirmation
screen pops up and he blinks on ‘YES’. Then the request is initiated and a message is sent to the respective

[893]
e-ISSN: 2582-5208
contact number via GSM module. When the patient gazes on previous page option, the screen shifts back.
Likewise, the patient can access any communication provided in GUI to get assistance from caretaker, doctor or
family.
This system is reliable and cost efficient and it holds equally good for subjects with glasses. This system provides
a feasible and practical way of communication for differently abled subjects. This technique can be implemented
in wheelchairs so that they can locomote and communicate with other people. A major disadvantage of this
system is – requires ambient lighting surrounding for efficient detection, inefficient on people suffering from
squint eyes, patient might strain his eyes due to long exposure on screen.
Fig 10a: Patient accessing GUI Fig 10b: Washroom Assist screen Fig 10c: Detection of right gaze
Fig 10d: Confirmation screen Fig 10e: Family assistance screen Fig 10f: Blink Detection
Fig 10g: Request initiated Fig 10h: Message sent to family members Fig 10i: Moving to previous page
V. CONCLUSION
In consideration with the means of communication, we are trying to create an effective environment for
differently abled patients to interact with doctors’ family members or nurses in hospitals or rehabilitation center.
A computer interface that provides the functionality of an input device like mouse based on eye actions such as
eye blink, eye-gaze and gaze control. The implementation is such that, Initially, by detecting the facial features of
the patient and then capturing the iris which is the integral part of the eye gaze movement Secondly, by using
various image processing techniques and other algorithms which enhances the clarity and accuracy of the image
thereby giving us the desired output. In addition to this, the project also considers the use of a Graphical User
Interface for making the application more interactive which will create a user- friendly experience and also by
providing a hassle free accessing of the eye gaze system. To conclude, the main motive of the application is to
minimize the physical strains a patient has to go through for communicating, thereby creating a convenient and a
futuristic approach to communication for differently abled patients.

[894]
e-ISSN: 2582-5208
VI. REFERENCES
[1] Eye Tracking: https://en.m.wikipedia.org/wiki/Eye_tracking
[2] Tobii Tech: https://tech.tobii.com/technology/what-is-eye-tracking/
[3] Spinal Cord Injury Zone: https://spinalcordinjuryzone.com/info/19933/blink-to-speak-system-for-
people-suffering-from-paralysis
[4] Kristie Huda, Mohiuddin Ahmad and Shazzad Hossain,“Command the Computer with your eye- an
Electro-ocular-based approach”. In International Conference on Software Knowledge Information,
Industrial Management and Applications (SKIMA) 2015.
[5] Poonam Potraje and Prof.P.S. Kulkarni ,“Cursor Controlling with the Help of Eye” International Journal
of Emerging Engineering Research and Technology(IJEERT), July 2014.
[6] Arulmozhi. K, Dharshini. K, and Kaviyasree. P,“Controlling Digital Device Using Smart Eye-Tracking
System”,International Journal of Scientific Research in Science and Technology (IJSRST) ,2019.
[7] Sidra Naveed, Bushra Sikander, and Malik Sikander Hayat Khiyal, “Eye Tracking System with Blink
Detection”, Journal of Computing, Volume 4, Issue 3, March 2012.
[8] Veena N. Hegde, Ramya S. Ullagaddimath, Kumuda. S,“Low cost eye based human interface computing
system”, IEEE India Conference, February 2017.
[9] Paul Viola Michael Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features”
Conference on Computer Vision and Pattern Recognition, 2001.

[895]

Fin Irjmets1657369160

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Fin Irjmets1657369160

Uploaded by

Copyright:

Available Formats

e-ISSN: 2582-5208

International Research Journal of Modernization in Engineering Technology and Science

www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

Fig 1: Hardware Block diagram

Fig 2: Level Shifter Circuit

Fig 3: Block diagram for Iris Detection

Fig 4: Haar Feature

Fig 7: Facial Landmarks

www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

Fig 9e: Food/Water Assist Screen Fig 9f: Confirmation Screen

Fig 9g: Shutdown Screen

www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

You might also like