BLUE EYES TECHNOLOGY

SEMINAR REPORT

Submitted by

GLADWIN P JOJO
Reg No: 21021088337

In partial fulfillment of the requirements

for
the award of the degree of

BACHELOR OF COMPUTER APPLICATIONS

DEPARTMENT OF COMPUTER APPLICATIONS

Yeldo Mar Baselios College,

Kothamangalam Puthuppady,
Kerala – 686691
2021-2024

1
 YELDO MAR BASELIOS COLLEGE
(Affiliated to Mahatma Gandhi University)

KOTHAMANGALAM

DEPARTMENT OF COMPUTER APPLICATIONS

CERTIFICATE

This is to certify that seminar entitled “Blue Eyes Technology” submitted in

partial fulfilment for the award of degree of BACHELOR OF COMPUTER
APPLICATIONS is a report of seminar done by GLADWIN P JOJO
(Reg No:210021088337) during the year 2021-2024

Date:
Head of the department

Examiner: Asst. Prof. NEETHU M MATHEW

2
 DECLARATION

We hereby declare that the seminar work entitled “BLUE EYES

TECHNOLOGY” submitted to Mahatma Gandhi University in partial fulfilment

of requirement of the award of degree of Bachelor of Computer Applications from

Yeldo Mar Baselios College, Kothamangalam is a record of bonafide work done

under the guidance of Dr SHAJU VARGHESE, seminar guide Yeldo Mar

Baselios College, Puthuppady. This seminar has not been submitted in partial or

fulfilment of any other degree/diploma/fellowship or similar of this university or

any other university.

Place: Puthuppady GLADWIN P JOJO 210021088337

Date:

3
 ACKNOWLEDGMENT

I express my gratitude to Prof K M George Principal of Yeldo Mar Baselios

College for providing me with adequate facilities, ways and means by which I am

able to complete the Seminar. With immense pleasure I take this opportunity to

record out sincere thanks to Asst. Prof. Neethu M Mathew, head of the

department, Department of Computer Applications, Yeldo Mar Baselios College

Kothamangalam.

I especially thank guide Dr SHAJU VARGHESE , Department of Computer

Applications in examining the daft of this seminar and suggestions, modifications

for having shown keen interest in the development of the seminar.

I also express my gratitude to all other members of the faculties and well wishers

who assisted me in various occasions during the seminar. Last but not least, I

would like to thank my parents and friends who encouraged me and gave the

motivation to complete the seminar. Above all my utmost gratitude. The Almighty

God for being with us always.

SEMINAR GLADWIN P

JOJO

4
5
Serial Contents Page
No No
ABSTRACT
1. 6

AIM
2. 7

INTRODUCTION
3. 8

NEED OF BLUE EYE

4. TECHNOLOGY 10

PERFORMANCE
5. REQUIREMENT 12

DESIGN METHODOLOGY
6. 13

TYPES OF EMOTION SENSOR

7.  HAND
 EMOTIONAL 14
MOUSE
 MAGIC
 EYE TRACKER
 VOICE
 ARTIFICIAL
INTELLIGENT
SPEECH TRACKER
 SUITOR

APPLICATIONS OF BLUE EYE

8. TECHNOLOGY 20
 APPLICATIONS
 POD
 PONG
ADVANTAGES AND
9. DISADVANTAGES OF BLUE 23
EYES TECHNOLOGY
CONCLUTION
10.  FUTURE WORK 25

BIBLIOGRAPHY
11. 26

6
 ABSTRACT

Human error is still one of the most frequent causes of catastrophes and ecological
disasters. The main reason is that the monitoring systems concern only the state of
the processes whereas human contribution to the overall performance of the system
is left unsupervised. Since the control instruments are automated to a large extent, a
human operator becomes a passive observer of the supervised system, which results
in weariness and vigilance drop. This, he may not notice important changes of
indications causing financial or ecological consequences and a threat to human life.

It therefore is crucial to assure that the operator's conscious brain is involved in an

active system supervising over the whole work time period. It is possible to measure
indirectly the level of the operator's conscious brain involvement using eye motility
analysis. Although there are capable sensors available on the market, a complex
solution enabling transformation, analysis and reasoning based on measured signals
still does not exist. In large control rooms, wiring the operator to the central system is
a serious limitation of his mobility and disables his operation. Utilization of wireless
technology becomes essential.

Blue Eyes is intended to be the complex solution for monitoring and recording the
operator's conscious brain involvement as well as his Physiological condition. This
required designing a Personal Area Network linking all the Operators and the
supervising system. As the operator using his sight and hearing senses the state of the
controlled system, the supervising system will look after his physiological condition.

7
 AIM

1) To design smarter devices

2) To create devices with emotional intelligence
3) To create computational devices with perceptual abilities

The idea of giving computers personality or, more accurately, emotional

intelligence" may seem creepy, but technologists say such machines would offer
important advantages.

De-spite their lightning speed and awesome powers of computation, today's PCs
are essentially deaf, dumb, and blind. They can't see you, they can't hear feel.
Every computer user knows the software, and abrupt system crashes. 1 :14 hut, of
course, you, and they certainly don't car frustration of nonsensical error 1 the s in
front of their Rosalind Picard, a computer science professor Media Lab in
Cambridge.

A computer endowed with emotional intelligence, on the other hand, could

recognize when its operator is feeling angry or frustrated and try to respond in an
appropriate fashion. Such a computer might slow down or replay a tutorial
program for a confused student, or recognize when a designer is burned out and
suggest he take a break. It could even play a recording of Beethoven's "Moonlight
Sonata" if it sensed anxiety or serve up a rousing Springsteen anthem if it detected
lethargy. The possible applications of "emotion technology" extend far beyond the
desktop. A car equipped with an affective computing system could recognize
when a driver is feeling drowsy and ad-vise her to pull over, or it might sense
when a stressed-out motorist is about to explode and warn him to slow down and
cool off.

These machines have got their own personality and this personality
depends upon the moods of the user.

8
 INTRODUCTION

Blue Eyes system provides technical means for monitoring and recording the
operator's basic physiological parameters. The most important parameter is
saccadic activity', which enables the system to monitor the status of the operator's
visual attention along with head acceleration, which accompanies large
displacement of the visual axis (saccades larger than 15 degrees). Complex
industrial environment can create a danger of exposing the operator to toxic
substances, which can affect his cardiac, circulatory and pulmonary systems.
Thus, on the grounds of plethysmography signal taken from the forehead skin
surface, the system computes heart beat rate and blood oxygenation.

[Saccades are rapid eye jumps to new locations within a visual environment
assigned predominantly by the conscious attention process.]

The Blue Eyes system checks above parameters against abnormal (e.g., a low
level of blood oxygenation or a high pulse rate) or undesirable (e.g., a longer
period of lowered visual attention) values and triggers user-defined alarms when
necessary. Quite often in an emergency situation operator speak to themselves
expressing their surprise or stating verbally the problem. Therefore, the operator's
voice, physiological parameters and an overall view of the operating room are
recorded. This helps to reconstruct the course of operators' work and provides data
for long-term analysis.

Blue Eyes consists of a mobile measuring device and a central analytical system.
The mobile device is integrated with Bluetooth module providing wireless
interface between sensors worn by the operator and the central unit. ID cards
assigned to each of the operators and adequate user profiles on the central unit
side provide necessary data personalization so different people can use a single
mobile device (called hereafter DAU- Data Acquisition Unit). The overall system

9
diagram is shown in Figure 1. The tasks of the mobile Data Acquisition Unit are
to maintain Bluetooth connections, to get information from the sensor and sending
it over the wireless connection, to deliver the alarm messages sent from the
Central System Unit to the operator and handle personalized ID cards. Central
System Unit maintains the other side of the Bluetooth connection, buffers
incoming sensor data, performs on-line data analysis, records the conclusions for
further exploration and provides visualization interface.

Figure 1. Overall system diagram

The task of the mobile Data Acquisition Unit is to maintain Bluetooth connection,
to get information from the sensor and sending it over the wireless connection, to
deliver the alarm messages sent from the Central System Unit to the operator and
handle personalized ID cards. Central System Unit maintains the other side of the
Bluetooth connection, buffers incoming sensor data, performs on-line data
analysis, records the conclusion for further exploration and provides visualization
interface.

1
0
 NEED OF BLUE EYE TECHNOLOGY

Artificial intelligence aims in creating intelligence to the Machines'. Intelligence

alone cannot make a machine perfect; even then it can only perform the work it is
designed for. But they never understood the perceptions or emotions of Human
beings. Blue Eye Technology now at incorporating these perceptions and sensory
abilities into the computational machines. Machines will be more powerful,
effective and helpful if they even have a little of these perceptions in them. In
future, we might see in reality of a aims computer playing your favourite movie,
songs or photos to cheer you up. You might just check your mails by asking your
computer, or call your friends by telling the computer to make the call for you.

Those which were considered to be a science fiction will be reality soon enough
with the help of this technology. Cars with emotional intelligence can reduce the
number of accidents, by judging the emotional state of the driver and help him
driving safely. The technology aims to enable these perceptions in machines by
using natural inputs like facial expressions. This technology has a power to
change the future of the world.

A computer on every desk was the vision of Bill gates when he started Microsoft.
The vision has now become the almost reality of the present world. As
technologies evolve, old ones are replaced by the new ones.

Like Floppies were got ridden by CDs and Pen drives, Walkman which almost
dominated the market for a long time is dead now because of mp3 players and
iPods. Similarly prospective future of Computers is Blue Eye Technology. Since
nineties, the world has seen a rapid change because of computers. It has changed
the way the world was working before and has brought us to a scenario where no

1
1
business can be called big and successful without computers in them as apart of it.
Animal survival depends on highly developed sensory abilities. Likewise human

cognition depends on highly developed abilities to perceive, integrate and

interpret visual, auditory and touch information. If such perceptions are added to
computers even in a small fraction, definitely computers would become even
more powerful than they are today. Also, they can help human beings in much
better manner. Blue Eye aims at creating the perceptual and sensual abilities to
computers. IBM scientists, at Almaden Research Centre (ARC) in San Jose, the
main IBM research laboratory, are trying to enable the computers to listen to our
speech, sense our gaze and read our body language since 1997.

1
2
 PERFORMANCE REQUIREMENTS

The portable nature of the mobile unit results in a number of performance

requirements. As the device is intended to run on batteries, low power
consumption is the most important constraint. Moreover, it is necessary to assure
proper timing while receiving and transmitting sensor signals. To make the
operation comfortable the device should be lightweight and electrically safe.

Finally, the use of standard and inexpensive IC's will keep the price of the device
at relatively low level.

The priority of the central unit is to provide real-time buffering and incoming
sensor signals and semi-real-time processing of the data, which requires speed-
optimizes filtering and reasoning algorithms. Moreover, the design should assure
the possibility of distributing the processing among two or more central unit nodes
(e.g., to offload the database system related tasks to a dedicated server).

1
3
 DESIGN METHODOLOGIES

In creating the Blue Eyes system, a waterfall software development model was
used since it is suitable for unrepeatable and explorative projects. During the
course of the development UML standard notations were used. They facilitate
communication between team members, all the ideas are clearly expressed by
means of various diagrams, which is a sound base for further development.
The results of the functional design phase were documented on use case diagrams.
During the low-level design stage, the whole systems was divided into five main
modules. Each of them has an independent, well-defined functional interface
providing precise description of the services offered to the other modules. All the
interfaces are documented on UML class, interaction and state diagrams. At this
point each of the modules can be assigned to a team member, implemented and
tested in parallel. The last stage of the project is the integrated system testing.

1
4
 TYPES OF EMOTION SENSORS

Blue Eyes technology uses most modern cameras, microphones and advanced
non-obtrusive sensing techniques to interact with humans and understand the
emotions of human beings. The machine has the ability to grasp the eye
movement of the user, the needs of the user and also can understand the emotional
and physical states of a user in front of the machine. The steps involved for
designing such type of computers are given below.

1. Process of giving sensing capacity: Blue Eyes utilizes many sensor

mechanisms, which is equivalent for the ears, eyes and other sensory
organs that human beings used to express emotions and recognize each
other. Blue Eyes uses voice recognition software, cameras and biometric
sensors to understand and respond to the emotional levels of humans.

2. Human Emotion detection or Affect Detection: The Blue Eyes

enables the machines to identify these minor emotional variations of
human beings even by a single touch on the mouse or key board and the
machines started to react with the users according to these emotional
levels. This is done with the guidance of intelligent devices like Emotion
Mouse". Along with this Emotion Mouse, Simple User Interest Tracker
(SUITOR) and Artificial Intelligent Speech Recognition are equipped
with the Blue Eyes technology to understand the speech and identify the
interest of the peoples at that instance of time.

For implementing the Affective Computing, we need Emotion Sensors. The

different types of Emotion Sensors used in Blue Eyes Technology are as follows:
For Hand: Emotion Mouse

1
5
For Eyes: Magic Pointing, Eye Tracking
For Voice: Artificial Intelligence Speech Recognition

HAND
 EMOTIONAL MOUSE

One proposed, non-invasive method for gaining user information through touch is
via a computer input device, the mouse. This then allows the user to relate the
cardiac rhythm, the body temperature, electrical conductivity of the skin and other
physiological attributes with the mood. This has led to the creation of the
"Emotion Mouse”. The device can measure heart rate, temperature, galvanic skin
response and minute bodily movements and matches them with six emotional
states: happiness, surprise, anger, fear, sadness and disgust. The mouse includes a
set of sensors, including infrared detectors and temperature-sensitive the mouse
includes a set of sensors, including infrared detectors and temperature-sensitive
chips. These components, user researchers' stress, will also be crafted into other
commonly used items such as the office chair, the steering wheel, the keyboard
and the phone handle. Integrating the system into the steering wheel, for instance,
could allow an alert to be sounded when a driver becomes drowsy.

INFORMATION OBTAINED FROM EMOTIONAL MOUSE

 BEHAVIOR
 Mouse movement

1
6
  Button click frequency
 Finger pressure when a user presses his/her button

 PHYSICIOLOGICAL INFORMATION
 Heart rate (Electro cardio gram (ECG/EKG), &
Photoplethysmogram (PPG))
 Skin temperature (Thermistor)
 Skin electricity (Galvanic skin response, GSR)
 Electromyographic activity (Electromyogram, MG)

 MAGIC (MANUAL AND GAZE INPUT CASCADED POINTING)

MAGIC stands for Manual Acquisition with Gaze Tracking Technology. a

computer with this technology could move the cursor by following the direction
of the user's eyes. This type of technology will enable the computer to
automatically transmit information related to the screen that the user is gazing at.
Also, it will enable the computer to determine, from the user's expression, if he or
she understood the information on the screen, before automatically deciding to
proceed to the next program. The user pointing is still done by the hand, but the
cursor always appears at the right position as if by MAGIC. By varying input
technology and eye tracking, we get MAGIC pointing.

1
7
EYES TRACKING

Since the goal of this work is to explore MAGIC pointing as a user interface
technique, the IBM Amaden Eye Tracker was used experimentally.
When the light source s placed on-axis with the camera optical axis, the camera is
able to detect the light reflected from the interior of the eye, and the image of the
pupil appears bright. This effect is often seen as the red-eye in flash photographs
when the flash is close to the camera lens. Bright (left) and dark (right) pupil
images result from on- and off axis illumination. The glints, or corneal reflections,
from the on- and off-axis light sources can be easily identified as the bright points
in the iris. The Almaden system uses two near infrared (IR) time multiplexed light
sources, composed of two sets of IR LED's, which were synchronized with the
camera frame rate. One light source is placed very close to the camera's optical
axis and is synchronized with the even frames. Odd frames are synchronized with
the second light source, positioned off axis. The two light sources are calibrated to
provide a proximately equivalent whale-scene illumination.

VOICE

 ARTIFICIAL INTELLIGENT SPEECH RECOGNITION

It is important to consider the environment in which the speech recognition

system has to work. The grammar used by the speaker and accepted by the
system, noise level, noise type, position of the microphone, and speed and manner
of the user's speech are some factors that may affect the quality of speech
recognition. The user speaks to the computer through a microphone, which, in
used; a simple system may contain a minimum of three filters. The more the
number of filters used, the higher the probability of accurate recognition. Preso

1
8
switched capacitor digital filters are used because these can be custom-built in
integrated circuit form These are smaller and cheaper than active filters using

operational amplifiers. The filter output is then fed to the ADC to translate the
analogue signal into digital word. The ADC samples the fitter outputs many times
a second. Each sample represents different amplitude of the signal. Each value is
then converted to a binary number proportional to the amplitude of the sample. A
central processor unit (CPU) controls the input circuits that are fed by the ADCS.
A huge RAM (random access memory) stores all the digital values in a buffer
area. This digital information, representing the spoken word, is now accessed by
the CPU to process it further. The normal speech has a frequency range of 200 Hz
to 7 kHz Recognizing a telephone call is more difficult as it has bandwidth
limitation of 300 Hz to3.3 kHz.

 THE SIMPLE USER INTREST TRACKER

Computers would have been much more powerful, had they gained perceptual and
sensory abilities of the living beings on the earth. What needs to be developed is
an intimate relationship between the computer and the humans. And the Simple
User

Interest Tracker (SUITOR) is a revolutionary approach in this direction. By

observing the webpage someone is browsing, the SUITOR can help by fetching
more information at his desktop. By simply noticing where the user's eyes focus
on the computer screen, the SUITOR can be more precise in determining his topic
of interest. It can even deliver relevant information to a handheld device.
IBM's Blue Eyes research project began with a simple question, according to
Myron Flicker, a manager in Almaden's USER group: Can we exploit nonverbal
cues to create more effective user interfaces? One such cue is gaze the direction in
which a person is looking. Flicker and his colleagues have created some new
techniques for tracking a person's eyes and have incorporated this gaze-tracking
technology into two prototypes. One, called SUITOR (Simple User Interest

1
9
Tracker), fills a scrolling ticker on a computer screen with information related to
the user's current task. SUITOR knows where one is looking, what applications

one is running, and what Web pages one may be browsing. "If I'm reading a Web
page about IBM, for instance," says Paul Maglio, the Almaden cognitive scientist
who invented SUITOR, "the system presents the latest stock price or business
news stories that could affect IBM."

2
0
 APPLICATIONS OF BLUE EYES TECHNOLOGY

One of the main benefits of speech recognition system is that it lets user do other
works simultaneously. The user can concentrate on observation and manual
operations, and still control the machinery by voice input commands. Another
major application of speech processing is in military operations. Voice control of
weapons is an example. With reliable speech recognition equipment, pilots can
give commands and information to the computers by imply speaking into their
microphones they don't have to use their hands for this purpose. Another good
example is a radiologist scanning hundreds of X-rays, ultra-sonograms, CT scans
and simultaneously dictating conclusions to a speech recognition system
connected to word processors. The radiologist can focus his attention on the
images rather than writing the text. Voice recognition could also be used on
computers for making airline and hotel reservations. A user requires simply
stating his needs, to make reservation, cancel a reservation, or making enquiries
about schedule.

 APPLICATIONS

1. To control weapons by voice commands

2. Pilot give commands to computers by speaking into microphones
3. For making airline and hotel reservations
4. For making reservations, cancelling reservations or making enquiries
5. Can be connected to word processors and instead of writing, simply dictate to
them.

2
1
Some of the blue Eyes enabled devices are discussed below:
 POD:

The first blue Eye enabled mass production device was POD, the car
manufactured by TOYOTA. It could keep the driver alert and active. It could tell
the driver to go slow if he is driving too fast and it could pull over the driver when
he feels drowsy. Also, it could hear the driver some sort of interesting music when
he is getting bored.

 PONG

2
2
TBM released a robot designed for demonstrating the new technology. The Blue
Eyes robot is equipped with a computer capable of analysing a person's glances

and other forms of expressions of feelings, before automatically determining the

next type of action. IBM has released a robot called PONG, which is equipped
with the Blue Eyes technology. PONG is capable of perceiving the person
standing in front of it, smiles when the person calls his name, and expresses
loneliness when it loses sight of the person.

2
3
 ADVANTAGES AND DISADVANTAGES

OF BLUE EYES TECHNOLOGY

 ADVANTAGES

Intuitive interaction: Blue Eyes Technology enables natural and intuitive

interaction between humans and machines, making it easier for users to
communicate with technology.

 Improved user experience: By recognizing and responding to human

emotions, Blue Eyes Technology can provide personalized and more
engaging user experiences.

 Increased efficiency: Blue Eyes Technology can increase efficiency in

various industries by improving customer service and identifying areas of
improvement.

 Enhanced safety and security: Blue Eyes Technology can be used to

improve safety and security in various settings by monitoring individuals'
emotional states and alerting authorities if necessary.

 Versatility: Blue Eyes Technology can be applied in a wide range of

industries, including healthcare, retail, and security, making it a versatile
solution for various business needs.

2
4
  Potential for automation: As Blue Eyes Technology continues to evolve,
it has the potential to automate various tasks, leading to increased
efficiency and cost savings for businesses.

 DISADVANGES

While Blue Eyes Technology offers many potential benefits, there are also several
disadvantages to consider:

 Privacy concerns: Blue Eyes Technology relies on collecting and

analysing sensitive data about individuals, raising concerns around privacy
and data security.

 Limited accuracy: The technology is still in the early stages of

development, and its accuracy in recognizing and interpreting human
emotions may be limited.

 High cost: Implementing Blue Eyes Technology can be expensive,

particularly for smaller businesses with limited resources.

 Technical limitations: The technology is still limited by technical

constraints such as the quality of sensors and the processing power
required to interpret data.

 Ethical concerns: The technology could be used for unethical purposes,

such as manipulating or exploiting individuals' emotions for commercial
or political gain.

 Potential for misinterpretation: The technology may misinterpret certain

emotions or physical signals, leading to incorrect responses or actions.

2
5
 CONCLUTION

The nineties witnessed quantum leaps interface designing for improved man
machine interactions. The BLUE EYES technology ensures a convenient way of
simplifying the life by providing more delicate and user-friendly facilities in
computing devices. Now that we have proven the method, the next step is to
improve the hardware. Instead of using cumbersome modules to gather
information about the user, it will be better to use smaller and less intrusive units.
The day is not far when this technology will push its way into your house hold,
making you lazier. It may even reach your hand-held mobile device. Any way this
is only a technological forecast.
The report presents two results of emotional sensory world. First, observation
reveals the fact that different eye colours and their results in change in emotions.
It changes without giving any information on shape and actual detected emotion.
It is used to favourably recognize four different emotions of eyes. This developed
methodology can be widespread to other activities. Second result were achieved
for converging in good emotions using a mixture of features, shapes, colours
based on eye points. After this favourable capturing of eye spots, it will help to
talk about the state of a person and also helps to cheer up by playing songs or
other sources. The motive of this research proves to be a source of economic
growth over all.

FUTURE WORK

This study work can be extended to home appliances where it can perform various
tasks within home premises through blue eye technology. Further as a world is
digitizing and we are moving towards robotic world, several human activities can
be shrunk with emotion sensory world tool. The tool or system is fitted in robot

2
6
with eye emotions which detects what is the demand and the action can be taken
by robot accordingly.

BIBLIOGRAPHY

 Levin J.L, An eye-controlled computers.

 Silbert L and R Jacob, The advantage of eye gazing interactions.
 Richard A Bolt, Eyes at interface.
 Colin ware, Harutune H Mikaelian, An evalution of eye tracker.
 OPEN AI - https://chat.openai.com/
 https://www.electroniclinic.com/blue-eye-technology-and-application-of-
the-blue-eye-technology/

2
7