You are on page 1of 10

ABSTRACT: Is it possible to create a computer which can interactw i t h u s a s w e i n t e r a c t e a c h o t h e r ?

F o r e x a m p l e i m a g i n e i n a f i n e m o r n i n g y o u w a l k o n t o y o u r computer room and switch on your computer, and t h e n i t t e l l s yo u “ H e y f r i e n d , g o o d m o r n i n g y o u s e e m t o b e a b a d m o o d t o d a y. A n d t h e n i t o p e n s your mailbox and shows you some of the mails andtries to cheer you. It seems to be a fiction, but it willbe the life lead by “BLUE EYES” in the very nearfuture. The basic idea behind this technology is togive the computer the human power. We all havesome perceptual abilities. That is we can understandeach others feelings. For example we can understando n e s e m o t i o n a l s t a t e b y a n a l y z i n g h i s f a c i a l expression. If we add these perceptual abilities ofh u m a n t o c o m p u t e r s w o u l d e n a b l e c o m p u t e r s t o w o r k t o g e t h e r w i t h h u m a n b e i n g s a s i n t i m a t e partners. The “BLUE EYES” technology aims atc r e a t i n g c o m p u t a t i o n a l m a c h i n e s t h a t h a v e p e r c e p t u a l a n d s e n s o r y a b i l i t y l i k e t h o s e o f humanbeingsH o w c a n w e m a k e c o m p u t e r s " s e e " a n d " f e e l " ? B l u e E ye s u s e s s e n s i n g t e c h n o l o g y t o i d e n t i f y a user's actions and to extract key information. Thisinformation is then analyzed to determine the user'sphysical, emotional, or informational state, which int u r n c a n b e u s e d t o h e l p m a k e t h e u s e r m o r e productive by per forming expected actions or byproviding expected information. For example, infu ture a Blue Eyes-enabled television could becomeactive when the user makes eye contact,at which point the user could then tellthe television to "turn on". This paper isabout the hardware, software, benefitsand interconnection of various partsinvolved in the “blue eye”technology.

For example we can understand onesemotional state by analyzing his facial expression. or evenscream aloud.The BLUEEYES technology aims at creating computationalmachines that have percep tual and sensory abilitylike those of human beings. recognize human presents. It realizes theu r g e n c y o f t h e s i t u a t i o n t h r o u g h t h e mouse. T h u s . or even guess their feelings.T h e m a c h i n e c a n u n d e r s t a n d w h a t a u s e r w a n t s . That is we can understand eachothers feelings. speech recog nition. Human cognition depends primarily ont h e a b i l i t y t o p e r c e i v e . Youa r e s i t t i n g i n f r o n t o f y o u r p e r s o n a l computer that can listen. ande s t a b l i s h e s a c o n n e c t i o n . where he is looking at. which can affect his cardiac. System overview B l u e E ye s s ys t e m p r o v i d e s t e c h n i c a l m e a n s f o r m o n i t o r i n g a n d r e c o r d i ng the operator’s basicphysiological parameters. Itv e r i f i e s y o u r i d e n t i t y . It uses nonobtrusives e n s i n g m e t h o d .INTRODUCTION I m a g i n e y o u r s e l f i n a w o r l d w h e r e humans interact with computers. talk. W e a l l h a v e s o m e perceptual abilities. The mosti m p o r t a n t p a r a m e t e r i s s a c c a d i c activity 1 .dials your friend at his office. w h i c h e n a b l e s t h e s ys t e m t o m o n i t o r t h e s t a t u s o f t h e o p e r a t o r ’ s v i s u a l a t t e n t i o n a l o n g w i t h h e a d acceleration. which accompanies l argedisplacement of the visual axis (saccadesl a r g e r t h a n 1 5 d e g r e e s ) .talk. a n d integrate audiovisuals and censoringi n f o r m a t i o n . e m p l o yi n g m o s t m o d e r n v i d e o c a m e r a s a n d m i c r o p h o n e s t o i d e n t i f y t h e u s e r s ’ actions through the use of imparted sensory abilities. listen. Researchers are attempting to addmore capabilities to computers that will allow themto interact like humans. f e e l s y o u r p r e s e n t s .e t c . The basic idea behind this technology is to give thec o m p u t e r t h e h u m a n p o w e r . a n d s t a r t s i n t e r a c t i n g w i t h you . A d d i n g e x t r a o r d i n a r y p e r c e p t u a l a b i l i t i e s t o c o m p u t e r s w o u l d e n a b l e computers to work together with human beings asintimate partners. The "BLUEEYES" technology aims at creating computationalmachines that have perceptual and sensory abilitylike those of human beings. It has the ability to gatherinformation about you and interact withy o u t h r o u g h s p e c i a l t e c h n i q u e s l i k e facial recognition. C o m p l e x i n d u s t r i a l e n v i r o n m e n t c a n c r e a t e a danger of exposing the operator to toxicsubstances.c i r c u l a t o r y a n d p u l m o n a r y s y s t e m s .You can ask the computer to dial toyour friend at his office. I t c a n e v e n u n d e r s t a n d y o u r emotions at the touch of the mouse. i n t e r p r e t . and even realize his physicalo r e m o t i o n a l s t a t e s . Ifw e a d d t h e s e p e r c e p t u a l a b i l i t i e s o f h u m a n t o computers would enable computers to work togetherwith human beings as intimate partners. o n .

I t i s n o t . C e n t r a l S y s t e m U n i t maintains the other side of the Bluetooth connection. I D c a r d s a s s i g n e d t o e a c h o f t h e operators and adequate user profiles on the centralunit side provide necessary data personalization sod i f f e r e n t p e o p l e c a n u s e a s i n g l e m o b i l e d e v i c e (called hereafter DAU – Data Acquisition Unit). The authorization processshall be repeated after reinserting the IDc a r d .The operator wears the DAU. Implementation and engineeringconsiderations Functional design During the functional design phase we used UMLs t a n d a r d u s e c a s e n o t a t i o n . r e c o r d s t h e c o n c l u s i o n s f o r f u r t h e r exploration and provides visualization interface. Oncethe connection has been established andauthorization process has succeeded (thePIN code is correct) central system startsmonitoring the operator’s physiologicalparameters. w h i c h s h o w s t h e functions the system offers to particularusers. h o w e v e r . Theoverall system diagram is shown in Figure 1. T h e r e f o r e .buffers incoming sensor data. The onlyf u n c t i o n s o f f e r e d t o t h a t u s e r a r e a u t h o r i z a t i o n i n t h e s y s t e m a n d r e c e i v i n g a l a r m a l e r t s . a lowl e v e l o f b l o o d o x y g e n a t i o n o r a h i g h pulse rate) or undesirable (e. to get informationf r o m t h e s e n s o r a n d s e n d i n g i t o v e r t h e w i r e l e s s connection. s u p e r v i s o r s a n d s y s t e m administrators. This helpsto reconstruct the course of operators’work and provides data for long-termanalysis. Afterinserting his personal ID card into themobile device and entering proper PINcode the device will start listening forincoming Bluetooth connections. The mobile device isintegrated with Bluetooth module providing wirelessinterface between sensors worn by the operator andt h e c e n t r a l u n i t . Operator i s a p e r s o n w h o s e physiological parameters are supervised.Quite often in an emergency situationo p e r a t o r s s p e a k t o t h e m s e l v e s e x p r e s s i n g t h e i r s u r p r i s e o r s t a t i n g verbally the prob l e m . 2).t h e g r o u n d s o f plethysmographic signal taken from thef o r e h e a d s k i n s u r f a c e . to deliver the alarm messages sent fromthe Central System Unit to the operator and handlep e r s o n a l i z e d I D c a r d s . p h y s i o l o g i c a l parame ters and an overall view of theoperating room are recorded. t h e o p e r a t o r ’ s v o i c e . We defined three groups of users:o p e r a t o r s . t h e s y s t e m c o m p u t e s h e a r t b e a t r a t e a n d b l o o d oxygenation. S u c h l i m i t e d functionality assures the device does notdisturb the work of theoperator (Fig.T h e B l u e E y e s s y s t e m c h e c k s a b o v e parameters against abnormal (e.The tasks of the mobile Data Acquisition Unit are tomaintain Bluetooth connections. Receiving alerts – the function supplies the operatorwith the information about the most important alertsr e g a r d i n g h i s o r h i s c o - .g. performs on-line dataa n a l y s i s . Authorization – t h e f u n c t i o n i s u s e d when the operator’s duty starts. r e q u i r e d o n reestablishing Bluetooth connection. a longerp e r i o d o f l o w e r e d v i s u a l a t t e n t i o n ) values and triggers user-defined alarmswhen necessary. Our system consists of a mobile measuring deviceand a central analytical system.g.

g .g. forworking night shifts).w o r k e r s . batteryl o w ) . The results received from the newmodules can be used in alarm conditions.g.w o r k e r s ’ c o n d i t i o n a n d mobile device state (e. Supervisor i s a p e r s o n r e s p o n s i b l e f o r a n a l yz i n g o p e r a t o r s ’ c o n d i t i o n a n d p e r f o r m a n c e .g. A c o m p r e h e n s i v e d a t a a n a l ys i s c a n b e p e r f o r m e d e n a b l i n g t h e s u p e r v i s o r t o draw conclusions on operator’s overallperformance and competency (e. connection lost. Defining alarm conditions – the functionenables setting up user-defined alarmconditions by writing conditionactionrules (e. a u d i o a n d v i d e o d a t a . A l a r m s a r e s i g n a l e d b y u s i n g a b e e p e r . t h e p r e s e n c e o f t h e h u m a n s u p e r v i s o r i s n o t n e c e s s a r y s i n c e t h e s y s t e m i s e q u i p p e d w i t h r e a s o n i n g algorithms and can trigger userdefinedactions (e. System administrator i s a u s e r t h a t maintains the system.During the on-line browsing it is possible to watch alist of currently working operators and the status oftheir mobile devices. Designing new analyzer modules – based on earlierr e c o r d e d d a t a t h e a d m i n i s t r a t o r c a n c r e a t e n e w analyzer modules that can recognize other behaviorsthan those which are built-in the system. Selecting one of the operatorse n a b l e s t h e s u p e r v i s o r t o c h e c k t h e o p e r a t o r ’ s c u r r e n t p h ys i o l o g i c a l c o n d i t i o n ( e . earphone providing central system sound feedbacka n d a s m a l l a l p h a n u m e r i c L C D d i s p l a y . T h e s u p e r v i s o r r e c e i v e s t o o l s f o r i n s p e c t i n g p r e s e n t values of the parameters ( On-line browsing ) as wellas browsing the results of long-term analysis ( Off-line browsing ). low saccadic activity during al o n g e r p e r i o d o f t i m e → informo p e r a t o r ’ s c o .H o w e v e r . configuring logging toolsand creating new analyzer modules.While registering new operators thea d m i n i s t r a t o r e n t e r s a p p r o p r i a t e d a t a (and a photo if available) to the systemdatabase and programs his personal IDcard. w h i c h shows more detailed information. T h e a d m i n i s t r a t o r n a m e s t h e n e w behavior to be recognized and points t h e d a t a associated with it. a p i e c h a r t showing active brain involvement) and a history ofalarms regarding the operator. defining alarmconditions. The administratori s d e l i v e r e d t o o l s f o r a d d i n g n e w operators to the database. w a k e h i m u p using the beeper or playing appropriate sound and log the event in the database).During off-line browsing it is possible toreconstruct the course of the operator’sd u t y w i t h a l l t h e p h y s i o l o g i c a l p a r a m e t e r s . The newmodules are created using decision tree inductiona l g o r i t h m . to inform the operator’s co-workers). Monitoring setup . All new incomingalerts are displayed immediately so thatthe supervisoris able to react fast.g.

The “uninteresting” data is suggested bythe built-in reasoning system. the level of blood oxygenation. T h e s e n s o r s e n d s a p p r o x i m a t e l y 5 . 2 k B o f data per second . P h y s i o l o g i c a l d a t a sensor To provide the Data Acquisition Unitw i t h n e c e s s a r y p h ys i o l o g i c a l d a t a w e d e c i d e d t o p u r c h a s e a n o f f s h e l f e ye movement sensor – Jazz Multisensor. a d e l a y b e t w e e n s t o r i n g consec utive frames can be set (e. t h e D A U h a r d w a r e c o m p o n e n t s a n d m i c r o c o n t r o l l e r software.Eye movement is measured using directinfrared oculographic transducers.The eye movement is sampled at 1kHz. one picture everytwo seconds). Loggersetup providestools forselecting the parameters to be recorded.t h e o t h e r p a r a m e t e r s a t 2 5 0 H z . Database maintenance – here the administrator canr e m o v e o l d o r “ u n i n t e r e s t i n g ” d a t a f r o m t h e database. For audiodata sampling frequency can be chosen. As regardst h e v i d e o s i g n a l . Data Acquisition Unit (DAU) In this section we describe the hardware part of theBlueEyes system with regard to the physiologicald a t a s e n s o r .e n a b l e s t h e a d m i n i s t r a t o r t o choose the parameters to monitor as wel l a s t h e a l g o r i t h m s o f t h e d e s i r e d a c c u r a c y t o c o m p u t e parameter values.g. Itsupplies raw digital data regarding eyeposition.a c c e l e r a t i o n a l o n g h o r i z o n t a l a n d vertical axes and ambient light intensity.

f i l t e r s . Additionally. Bluetooth Connection Manager isr e s p o n s i b l e f o r e s t a b l i s h i n g a n d m a i n t a i n i n g c o n n e c t i o n s u s i n g a l l a v a i l a b l e B l u e t o o t h d e v i c e s . Any number of consumer modules canregister to receive the data supplied by a producer. Data Logger and Visualization. Theauth e n t i c a t i o n p r o c e s s i s p e r f o r m e d using system PIN code fetched from thedatabase. Once the connection has beena u t h e n t i c a t e d t h e m o b i l e u n i t s e n d s a d a t a f r a m e c o n t a i n i n g t h e o p e r ator’si d e n t i f i e r . A f t e r e s t a b l i s h i n g a c o n n e c t i o n a n authentication procedure occurs. Atthe startup it communicates with the Operator DataManager in order to get more detailed personal data. Connection Manager authori z e s operators. manages their sessions. every consumer may be a producerfor other consumers (for system core class diagram – see Appendix C).Only with those devices the ConnectionM a n a g e r w i l l c o m m u n i c a t e . demultiplexes andb u f f e r s r a w p h ys i o l o g i c a l d a t a . Naturally. 1): Connection Manager. I n t h i s s e c t i o n w e d e s c r i b e t h e f o u r m a i n CSU modules (see Fig. I t periodically inquires new devices in anoperating range and checks whether theyare registered in the system database. F i n a l l y .Every single consumer can register at any number ofproducers. receiving therefore different types ofdata. F i g u r e 1 1 s h o w s Connection Manager architecture. This approach enables high systems c a l a b i l i t y – n e w d a t a p r o c e s s i n g m o d u l e s ( i . which will managethe new operator’s session (for detailedBluetooth communication flow chartsa n d p r o t o c o l s – see Appendix E A d d i t i o n a l l y. t h e C o n n e c t i o n M a n a g e r a d d s a S C O l i n k ( v o i c e connection) and runs a new dedicatedOperator Manager.Data Analysis. d a t a a n a l yz e r s a n d l o g g e r s ) c a n b e e a s i l y added by simply registering as a consumer. i n c a s e o f l a r g e r r e s o u r c e d e m a n d s t h e processing can be distributed among a number ofn o d e s . It is designed tocooperate with all available Bluetooth devices ino r d e r t o s u p p o r t r o a m ing (for CSU detailedhardware speci fication – s e e A p p e n d i x D ) .The most important Operator Manager’s task is tob u f f e r t h e . Transport Layer Manager hides the details interface (which can be either RS232 orUART or USB standard) and providesuniform HCI command interface.so that when the operators roam around the covereda r e a a c o n n e c t i o n w i t h a n a p p r o p r i a t e B l u e t o o t h d e v i c e i s e s t a b l i s h e d a n d t h e d a t a s t r e a m i s redirected accordingly. Themodules exchange data using specially designedsingleproducer-multi-consumer buffered thread-safe queues.Central System Unit (CSU) CSU software is located on the delivered Toshibal a p t o p . Connection Manager Connection Manager’s main task is t perform lowl e v e l B l u e t o o t h c o m m u n i c a t i o n u s i n g H o s t Controller Interf ace commands.The data of each supervised operator is bufferedseparately in the dedicated Operator Manager. t h e C o n n e c t i o n M a n a g e r m a p s t h e operator’s identifiers into the Bluetooth connections. e .

5 decision tree induction algorithm.i n a n a l y z e r m o d u l e s w e implemented a saccade detector. T h e m o d u l e c o n s i s t s o f a number of smaller analy zers extracting differenttypes of information. g .g.t h e c u s t o m m o d u l e s a r e c r e a t e d b y applying a supervised machine learninga l g o r i t h m t o a s e t o f e a r l i e r r e c o r d e d examples containing the charact eristicf e a t u r e s t o b e r e c o g n i z e d . Ananalyzer can be either a simple signalfilter (e. Operator Data Manager provides an interface to theoperator database enabling the other modules to reado r w r i t e p e r s o n a l d a t a a n d s y s t e m a c c e s s information. a c t i n g a s a producer. Data Analysis Module The module performs the analysis of the raw sensord a t a i n o r d e r t o o b t a i n i n f o r m a t i o n a b o u t t h e operator’s p hysiological condition. walkingand lying) or whether his eyes are closedor opened. Thec o m p u t e d f e a t u r e s c a n b e e . visualattention level. Each of the analyzers registersa t t h e a p p r o p r i a t e O p e r a t o r M a n a g e r o r a n o t h e r a n a l y z e r a s a d a t a c o n s u m e r a n d . blood oxygenation andpulse rate analyzers.T h e a l g o r i t h m w e u s e d p e r f o r m s t w o main steps:A. saccade detector) or acustom detector module. After buffering approx. t h e O p e r a t o r M a n a g e r provides an interface for sending alert messages tothe related operator. 5 sof the signal differentiate it using threepoint central difference algorithm. Finite Input Response (FIR)filter) or a generic data extractor (e. As we are notable to predict all the supervisors’ needs. Sort the velocities by absolute value and calculate upper15% of the border velocity along both X – v 0x and Y – v 0y axes . t h e operator’s position (standing. provides the results of the analysis. F u r t h e r m o r e . the split data streams are available for theother system modules through producerconsumerq u e u e s . whichwill give eye velocity time series.A s b u i l t .The saccade detector registers as an eyem o v e m e n t a n d a c c e l e r o m e t e r s i g n a l v a r i a n c e d a t a c o n s u m e r a n d u s e s t h e data to signal saccade occurrence.signal variance. As a result v 0x and v 0y are cut-offvelocities. I n t h e p r o t o t y p e w e u s e d a n i m p r o v e d C 4 . User adjustment step .g. On-line analyzer flow .i n c o m i n g r a w d a t a a n d t o s p l i t i t i n t o separate data streams related to each of the measuredp a r a m e t e r s .B. Sincesaccades are the fastest eye movementsthe algorithm calculates eye movementv e l o c i t y a n d c h e c k s p h y s i o l o g i c a l constraints. T h e r a w d a t a i s s e n t t o a L o g g e r Module. T h e p h a s e takes up 5 s. The separatelyrunning Data Analysis Module supervises each oft h e w o r k i n g o p e r a t o r s .

there is a dedicated videod a t a l o g g e r . Data Logger Module T h e m o d u l e p r o v i d e s s u p p o r t f o r s t o r i n g t h e monitored data in order to enable the supervisor toreconstruct and analyze the course of the operator’sduty. Moreover. The Data Logger is ableto use any ODBC-compliant databases y s t e m . I f t h e v e l o c i t y e x c e s s precalculated v 0 ( b o t h a x e s a r e c o n s i d e r e d separately) there is a possibil i t y o f s a c c a d e occurrence. Check the following conditions (if anyof them is satisfied do not detect a saccade):the last saccade detection was less than 130 ms ago(physiological constraint – the saccades will noto c c u r m o r e f r e q u e n t l y ) t h e m o v e m e n t i s nonlinear (physiological constraint) compare thesignal with accelerometer (rapid head movementmay force eye activity of comparable speed) ifthe accelerometer signal is enormously unevenc o n s i d e r i g n o r i n g t h e s i g n a l d u e t o p o s s i b l e sensor device movements. Applications: . video and audiodata the supervisor is able to reconstruct the courseo f t h e s e l e c t e d o p e r a t o r ’ s d u t y ( f o r d e t a i l e d Visualization Module structure – see Appendix I). Continuously calculate eyem o v e m e n t v e l o c i t y u s i n g t h r e e p o i n t c e n t r a l d i f f e r e n c e a l g o r i t h m ..Apart from the raw or processed physiological data. The Logger’stask is to add appropriate time stamps toe n a b l e t h e s y s t e m t o r e c o n s t r u c t t h e voice. Each working operator’s data isrecorded by a separate instance of the Data Logger. It enables them to watch e a c h o f t h e w o r k i n g o p e r a t o r ’ s p h y s i o l o g i c a l condition along with a preview of selected videos o u r c e a n d h i s r e l a t e d s o u n d s t r e a m . The voiced a t a i s d e l i v e r e d b y a V o i c e D a t a A c q u i s i t i o n m o d u l e . I n t h e p r o t o t yp e w e u s e d M S S Q L S e r v e r .e. The module is designedt o h a n d l e o n e o r m o r e c a m e r a s u s i n g V i d e o F o r W i n d o w s s t a n d a r d . A l l t h e in coming alarm messages are instantly signaled tothe supervisor. the visualization modulecan be set in the off-line mode. alarms.The raw data is supplied by the relatedO p e r a t o r M a n a g e r m o d u l e ( F i g . w h i c h i s a p a r t o f t h e Project Kit. w h i c h r e c o r d s t h e d a t a supplied by the Video Data Acquisitionmodule (in the prototype we use JPEGcompression). where all the data isfetched from the database. Visualization Module The module provides user interface forthe supervisors.alerts and operator’s voice are stored. T h e came ras can be either directly connectedt o t h e s ys t e m o r a c c e s s i b l e t h r o u g h a UDP network connection (for d etailedc a m e r a p r o t o c o l s p e c i f i c a t i o n – s e e Appendix H). w h e r e a s t h e D a t a A n a l y s i s m o d u l e delivers the processed data. 1 1 ) . The module registers as a consumer of the datato be stored in the database (for database schema – see Appendix G). T h e m o d u l e registers as an operator’s vo i c e d a t a consumer and optionally processes thesound to be stored (i.Additionally. reduces noise orr e m o v e s t h e f r a g m e n t s w h e n t h e operator does not speak). Watching all the recordedphysiological parameters.

and the Computer Sound Morpher arec o m m e r c i a l l y a v a i l a b l e s m a r t t o y p r o d u c t s d e v e l o p e d b y t h e I n t e l S m a r t T o y L a b i n . BlueEyes is developingways for computers to anticipate users' wants bygathering video data on eye movement and faciale x p r e s s i o n . but have no sense of whethert h a t i n f o r m a t i o n i s e v e r v i e w e d o r understood. t h i s i s w a y i t c a p t u r e s t h e person's income and buying preferences. andt h e s o f t w a r e r e g i s t e r s t h e c u s t o m e r ' s a t t e n t i o n . i t c o u l d b e u s e f u l t o h e l p w i t h c r i t i c a l d e c i s i o n s like:" I k n o w yo u w a n t t o g e t i n t o t h e f a s t lane. including. In contrast. thepupil appears bright to the sensor. n ew real-timecomputer. O n e theme that is common across these PCconnectedt o y s i s t h a t u s e r s i n t e r a c t w i t h t h e m u s i n g a combinat ion of visual. When monitoring pupils. But the first practical use for t h e r e s e a r c h t u r n s o u t t o b e s n o o p i n g o n s h o p p e r s . for example.Typically targeting commercial business. TheIntel Play QX3 Computer Microscope. thesystem uses a camera and two infrared light sourcesplaced inside the product display. eyebrowand mouth movement. t h e other is slightly off axis. The presentation will provide anoverview of the interaction design of these productsand pose some unique challenges faced by designersand engineers of such experiences targeted .H o w m a n y s h o p p e r s i g n o r e d a p r o m o t i o n ? H o w many stopped? How long did they stay? Did theirf a c e s r e g i s t e r b o r e d o m o r d e l i g h t ? H o w m a n y r e a c h e d f o r t h e i t e m a n d p u t i t i n t h e i r s h o p p i n g carts? BlueEyes works by tracking pupil. Y o u r g a z e m i g h t r e s t o n a W e b s i t e heading. and that would prompt yourcomputer to find similar links and to call them up ina n e w window. 4. but I'm afraid I can't do that. report that a number ofl a r g e r e t a i l e r s h a v e i m p l e m e n t e d s u r v e i l l a n c e s y s t e m s that record and i nterp ret custom ermovements. i t c o u l d g i v e i n d i v i d u a l challenges to customer s playing video games. what activity is occuring. CA. Another application would be in theautomobile industry. When the eyelooks into the camera-aligned light. 2. f o r c a r s .Yourtoo upset right now" and therefore assisti n d r i v i n g s a f e l y .The integration of children's toys. technologies andcomputers is enabling new play experiences thatwere not commercially feasible until recently. 3.BlueEyes is actively been incorporatedi n s o m e o f t h e l e a d i n g r e t a i l o u t l e t s . BlueEyes software makes sense of what the camerassee to answer key questions for retailers.Engineers at IBM's ffice:smarttags" ResearchCenter in San Jose. the computer system is designedt o b e a b l e t o d e t e r m i n e a p e r s o n ' s e m o t i o n a l s t a t e . the Me2Camwith Fun Fair.1. audible and tactile input &output modalities. We could see its use in video gamesw h e r e . Current interfaces between computersa n d h u m a n s c a n p r e s e n t i n f o r m a t i o n vividly. using s o f t w a r e f r o m A l m a d e n ' s BlueEyes research project. One light source isa l i g n e d w i t h t h e c a m e r a ' s f o c u s . By simply touchinga c o m p u t e r i n p u t d e v i c e s u c h a s a mouse. andt h e r e f o r e w h a t d i s p l a y o r m e s s a g i n g modalities are most appropriate to use inthe current situation. The results of ourresearch will allow the interface betweencomputers and human users to becomem o r e n a t u r a l a n d i n t u i t i v e .

s a f e t y a n d a s s u r e s proper Quality of system performance.These new possibilities can cover areassuch as industry. sensors willnotice when we enter a space.pump iron. It avoids potential threats resulting fromh u m a n e r r o r s . 5 . t h e y s h o w t h e c h a n g e i n their value though patina. T h e f a m i l i a r a n d u s e f u l c o m e f r o m t h i n g s w e recognize. where we are. and actions to give thecomputer the experience it needs to work with us. sit down. lie down. Many of our favorite things' appearancec o m m u n i c a t e t h e i r u s e . Pervasive infrastructure is recordingit. o r e v e n g u e s s t h e i r f e e l i n g s . militarycommand centers or operation theaters.tiredness. n a m e l y y o u n g c h i l d r e n . feelings.Researchers are attempting to add morecapabilities to computers that will allowthem to interact like humans. .We use our looks. recognizeh u m a n p r e s e n t s . oversight. etc. t a l k . transportation.at novicec o m p u t e r u s e r s . This talk will cover projects from the ContextAware Computing Conclusion: The wireless link between the sensorsworn by the operator and the supervisings ys t e m m a k e s i t p o s s i b l e t o i m p r o v e o v e r a l l r e l i a b i l i t y. s u c h a s w e a r i n e s s . l i s t e n .Keyboards and mice will not continue to dominatecomputer user interfaces. Se nsors are gaining fidelity andubiquity to record presence and actions. Keyboard input will ber e p l a c e d i n l a r g e m e a s u r e b y s ys t e m s t h a t k n o w w h a t w e w a n t a n d r e q u i r e l e s s e x p l i c i t communication. As technologists we arenow poised to imagine a world where computingobjects communicate with us in-situ.