Professional Documents
Culture Documents
( blue eyes –
Human operator
monitoring system
)
BY
Agenda
n Motivation
n What is BlueEye technology ?
n What is BlueEyes ?
n System designing
n System overview
n DAU
n CSU
n Data security
n IBM research
n Conclusion
Motivation
n Human error - a frequent reason of
catastrophes and ecological disasters
– temporal indisposition
– weariness
– oversight
• Monitoring of conscious
brain involvement
How can we make
computers “see” and
“feel” ?
n Blue eyes uses sensing technology to
identify a user’s actions and to extract
key information.
n Information is then analyzed to
determine the user’s physical
,emotional ,or informational state.
What is Blue Eyes ?
n The complex solution for human-
operator monitoring:
– Visual attention monitoring
– Physiological condition
– Operator’s position detection
– Wireless data acquisition using bluetooth
What is BlueEyes not?
n Doesn’t predict nor interfere with
operator’s thoughts
n Cannot force directly the operator to
work
IBM research
n BLUE EYE – EMOTIONAL MOUSE
sensors in the mouse ,sense the
physiological attributes which are
correlated to emotions using
correlation model
-by simply touching the mouse ,the
computer will be able to determine a
person’s emotional state.
n BLUE EYE enabled TELEVISION – could
become active when the user makes
an eye contact.
Physiological
foundations
Human brain
Jazz Multisensor
Jazz
Design objectives
n A mobile data acquisition device
• A reliable real-time buffering, processing
and recording
• A clear visualization interface
• Mechanisms for creating custom data
analyzers
• To ensure a possibility of distributing the
processing among a number of computers
Implementation and
engineering
considerations
-THREE groups of users
n OPERATOR -a person whose physiological parameters are
supervised
- authorization in the system
- receiving alerts
n SUPERVISOR – a person responsible for analyzing operator’s
condition and performance.
n SYSTEM ADMINISTRATORS- maintains the system
deliver tools for adding
- new operator’s to the database
- defining alarm conditions
- configuring logging tools
- creating new analyzer modules
Designing
Connection Visualizati
Atmel 89C52 Bluetoot Bluetoo Manager on
microcontroll h th
er Module
n Lightweight
Atmel 89C52 Bluetoot
microcontroll
er
h n Runs on batteries -
Jazz low power
Multisensor
consumption
n Easy to use - does not
disturb the operator
working
n ID cards for operator
authorization
DAU - components
microphone MC 145483
earphone PCM codec
Jazz Bluetoot
Multisensor h
modul
Beeper Atmel 89C52
UART
microcontroller
LCD display MAX2
UART
32
LED indicators
BlueEyes
ID card Simple Data Acquisition Unit
interface keyboar
d
ID card
Microcontroller & blue
tooth module
EEPROMS & the PCB
Implementation - DAU
n
• Made from scratch by the team
Batteries and
voltageBluetooth module
level monitor
CSU - features
Central System Unit
n Connection
management Connection Visualizati
Bluetoo Manager on
n Data th Module
processing Data
Analysis
Data
Logger
n Visualization
n Data recording
n Access
verification
n System maintenance
CSU – components
n CONNECTION MANAGER – main task to
perform low-level blue tooth communication
n DATA ANALYSIS MODULE – performs the
analysis of the raw sensor data in order to
obtain information about operator’s
physiological condition
n DATA LOGGER MODULE – provides support
for storing the monitored data.
n VISULAIZATION MODULE – provides user
interface for the supervisors
CSU - Visualization
Module
n Raw and processed data visualization
using:
– VU-meters
– pie-charts
– time series
Velocity
50
0
-50
-100
-150
-200
-250
Differentiation 0 8000 16000 24000 32000
Time [ms]
60
Attention level (Lva)
50
40 Conscious
30 brain
20
10
involvement
0 level
0 8000 16000 24000 32000
Time [ms]
CSU - Connection
Manager
Operator • Initializes Bluetooth
Data •Data demultiplexer
Manager modules
•Data producer queues
• Inquires for new devices
in range
Bluetoo • Establishes Raw connections
Separated
• Operator Data
th Supports
Manager
operator data streams
Buffer
roamingAlert Alerts
• Feeds Operator
buffer Manager
... with...physiological data
Bluetooth
... Connection ...
Manager
Raw
Operator Data
Manager Buffer
Bluetoo
Alert Alerts
th buffer
CSU - Data Analysis
• Custom data analyzers - decision tree based classifiers
Assigned
Data samples category
“Yes”
No
... ... answer Decision tree
“Yes” coefficients
“No”
No
answer
CSU - Data Analysis
n Custom data analyzers - decision tree
based classifiers
Data
consumer
creates
D
Data
Data Data Data
Processing
consumer &
ata
q
consumer
producer queue producer ueu
e
Data
D
consumer
Data ata
Data
Processing
consumer & q
consumer
producer ueu
e
CSU - intermodule
communication
Connection
Connection
Manager
Manager
Operator
Operatormanager
manager ...
... OO OO
(OM)
(OM) MM MM
Visualization
Visualization Data
Data
DataAnalysis
Analysis
Module
Module Logger
Processed data
n Custom-built
Bluetooth
communication
routines
n Intel USB camera
n ID card programmer
n Developed using Visual C++ 6.0
n Deployment environment:
Windows 2000,
MS SQL Server
Future improvements
DAU
small CMOS camera to monitor
the operator’s point of gaze
single PCB (SMD technology)
low voltage ICs - LiIO batteries
power
CSU
data mining algorithms
advanced database encryption
using e.g. AES algorithm
Data security
n Only registered mobile devices
can connect to the system
n Bluetooth connection authentication
n Bluetooth connection encryption
n Access rights restrictions
n Personal and physiological data
encryption
App licat ions of Blue Eyes
n Generic control rooms
(System can be applied in every working
environment
requiring permanent operator’s attention)
– Power station
– Captain bridge
– Flight control centers
– Operating theatres – anesthesiologists
Blue E yes - benefits
n Minimization of
– ecological consequences
– financial loss
– a threat to a human life
n The reconstruction
of the course of
operator’s work
References
n www.seminar4u.com
n www.howstuffswork.com
n www.cs.put.poznan.com
n www.whitepapers.com
n www.ibmresearchcenter.com
n www.almaden.ibm.com
Conclusion