PRESENTED BY

SIDDHARTH SOAM
SUMIT TIWARI
VIVEK CHAWRA
SHREYANSH SINGH

INTRODUCTION

Sixth Sense is a wearable gestural interface

that augments the physical world around us with
digital information and lets us use natural hand
gestures to interact with that information.

Steve Mann is considered as the father of Sixth

Sense Technology who made wearable computer
in 1990. He implemented the Sixth Sense
Technology as the neck worn projector with a
camera system (which Mann originally referred to
as Synthetic Synesthesia of the Sixth Sense).

THEN HIS WORK WAS CARRIED FORWARD BY PRANAV MISTRY (PH.D

STUDENT IN THE FLUID INTERFACES GROUP AT THE MIT MEDIA LAB).

THE DEVICES WHICH ARE USED

IN SIXTH SENSE TECHNOLOGY
ARE:
CAMERA
COLORED MARKER.
MOBILE COMPONENT
MIRROR
PROJECTOR

Camera

Captures an object in view and tracks the users hand gestures

It sends the data to smart phone
It acts as a digital eye, connecting you to the world of digital
information

Color Markers

It is at the tip of the users fingers .

Marking the users fingers with red, yellow, green, and blue tape helps
the webcam recognize gestures

The movements and arrangements of these makers are interpreted into

gestures that act as interaction instructions for the projected application
interfaces.

Mirror
MIRROR

The usage of the mirror is significant as the projector dangles

pointing downwards from the neck.

Smart Phone

A Web-enabled smart phone in the users pocket processes the video

data

Other software searches the Web and interprets the hand gestures

Projector

The projector projects visual information enabling surfaces and

physical objects to be used as interfaces

The project itself contains a battery inside, with 3 hours of battery

life.

A tiny LED projector displays data sent from the smart phone on
any surface in viewobject, wall, or person.

BLOCK DIAGRAM

APPLICATIONS

Viewing Map

Taking Pictures

Making Calls

Interacting with physical

objects

Drawing Application

Getting Information
Product information
Book Information
Flight Updates

Make a call

You can use the Sixth Sense

to project a keypad onto your
hand, then use that virtual
keypad to make a call.

Call up a map

With the map application we

can call up the map of our
choice and then use thumbs
and index fingers to navigate
the map

Check the time

Draw a circle on your wrist to

get a virtual watch that gives
you the correct time

Create multimedia
reading experiences

Sixth Sense can be

programmed to project related
videos onto newspaper
articles you are reading

Get flight updates

The system will recognize your

boarding pass and let you know
whether your flight is on time
and if the gate has changed.

Feed information on
people

The system will project relevant

information about a person such
as what they do, where they
work, and so on.

Take pictures

If you fashion your index fingers

and thumbs into a square
("framing" gesture), the system
will snap a photo.

After taking the desired number

of photos, we can project them
onto a surface, and use gestures to
sort through the photos, and
organize and resize them.

Drawing application

The drawing application lets

the user draw on any surface
by tracking the fingertip
movements of the users index
finger.

Zooming features

The user can zoom in or zoom

out using intuitive hand
movements

Get product information

Sixth Sense uses image

recognition or marker
technology to recognize
products we pick up, then feeds
us information on those
products.

Get book information

The system can project Amazon

ratings on that book, as well as
reviews and other relevant
information

HOW DOES IT WORK???

ADVANTAGES
Portable
Support Multi touch and Multi user interaction
Cost Effective(300$)
Data access directly from the machines in real time
Mind map the idea anywhere
Open Source Software

DISADVANTAGES

Hardware limitations of the devices, that we

currently carry around with us.

For example many phones will not allow the

external camera feed to be manipulated in real
time.

Post processing can occur however.

CONCLUSION

Sixth Sense recognizes the objects around us, displaying

information automatically and letting us to access it in any
way we need.

The Sixth Sense prototype implements several applications

that demonstrate the usefulness, viability and flexibility of
the system.

The potential of becoming the ultimate "transparent" user

interface for accessing information about everything
around us.

Allowing us to interact with this information via natural

hand gestures

FUTURE ENHANCEMENTS
To get rid of color markers.
To incorporate camera and projector inside
mobile computing device.
To have 3D gesture tracking.
To make Sixth Sense work as fifth Sense for
disabled person.

REFERENCE
1. http://www.ted.com/talks/pattie_maes_demos_the_sixth_sense.html
2. http://www.pranavmistry.com
3. P. Mistry, P. Maes. Sixth Sense A Wearable Gestural Interface. To be
appeared in SIGGRAPH Asia 2009, Emerging Technologies. Yokohama,
Japan. 2009
4. P. Mistry. The thrilling potential of Sixth Sense technology TED India 2009.
Mysore, India

5. http://ieeexplore.ieee.org Rao, S.S.;

Electron. & Commun. Eng., Anna Univ. of Technol., Coimbatore, India This
paper appears in: Communication and Computational Intelligence
(INCOCCI), 2010 International Conference on
Issue Date: 27-29 Dec. 2010
On page(s): 336 - 339
Location: Erode
E-ISBN: 978-81-8371-369-6
References Cited: 11
INSPEC Accession Number: 11887446
Date of Current Version: 24 March 2011

REFERENCES
Sources:

http://www.pranavmistry.com/projects/sixthsense

http://boingboing.net/2009/11/12/sixth-sense-technolo.html

http://news.bbc.co.uk/2/hi/technology/7997961.stm

http://news.softpedia.com/news/Next-Gen-039-Sixth-Sense039Device-Created-at-MIT-103879.shtml

Publications:

Pranav Mistrys: The Thrilling Potential of SixthSense

Computing-Future of Interface. Boxesandarrows. Apr 1 2010.

P. Mistry, P. Maes. SixthSense A Wearable Gestural Interface.

To be appeared in SIGGRAPH Asia 2009, Sketch. Yokohama,
Japan. 2009