Professional Documents
Culture Documents
1. Introduction
1.1 Introduction:
Microsoft Surface is an interactive table top that can do everything a network computer can do plus more without using a keyboard or a mouse. There are four key features: direct interaction, multi-touch ability, multi-user ability and object recognition. irect interaction allows you to touch or grab digital information with your hands and use natural gestures to open, grasp, and command virtual objects, pages and images. The multi-touch feature enables the Surface to recognize many points of contact simultaneously so you can enlarge an image by touching the opposite corners and dragging those outwards. !long with the multi-touch feature, the shape and design of the Surface allows for multi-users at once, therefore, the user sitting across from you can be doing something completely different or independent of you. The last key feature, object recognition, enables the system to identify physical objects just by setting them on the Surface and to respond by displaying the appropriate software related to that item. "urrently, Microsoft Surface is being marketed and sold directly to large scale leisure, entertainment and retail companies, such as !T#T in various cities, $io in %as &egas, and Sheraton 'otels in various cities.
' out of ()
Seminar report
Having studied about this cocept in newspaper made me eager and enthusiastic to "now about this topic and collect information about it
1.5 Applications:
9nteractive 8lassrooms: The multi-touch surface computers will encourage the students to interact with content and each other promoting group wor" and team building s"ills Students would have custom built hardware where they can create their assignments and teachers may be able to see it instantly and help the students Students sitting around the table may open a file# push it across# drag it# modify it# let another student add or delete information and then save the document 9n a photography class# the students could share their images instantly 9n an art class# one student could be painting with a paint brush while another is drawing with her finger %oth the paint brush and the finger would be recogni:ed 9n %usiness classes# specifically accounting# having access to a computer right at your finger tips will help the students learn faster and comprehend on a higher level 9 believe 9t$s a lot easier to follow along on an ;xcel spreadsheet when you can highlight the cell and see for yourself what the formula is or where that amount came from Allowing students the ability to actively participate while teaching them about constructing a balance sheet will ma"e it easier for the students to not only comprehend the material but also retain the material in my opinion 9n a geography class each student could find a specific location and the maps could be displayed instantly Teachers would not have to worry about finding space in a computer lab in order for the students to create pro&ects or conduct research %ho& reddy engineering college for women + out of ()
Seminar report
Students could share podcasts or other information related to a certain pro&ect that they have saved to their flash drive &ust by laying the device on the surface
Seminar report
dinner being eaten based on the type of wine set on the Surface# or in con&unction with a password# offering user authentication A commercial Microsoft Surface unit is H'+#3// <unit only=# whereas a developer Microsoft Surface unit costs H'3#/// and includes a developer unit# five seats and support ,artner companies use the Surface in their hotels# restaurants# and retail stores The Surface is used to choose meals at restaurants# plan vacations and spots to visit from the hotel room Starwood Hotels plan to allow users to drop a credit card on the table to pay for music# boo"s# and other amenities offered at the resort 9n AT>T stores# use of the Surface include interactive presentations of plans# coverage# and phone features# in addition to dropping two different phones on the table and having the customer be able to view and compare prices# features# and plans MS?%8Cs coverage of the +//0 !S presidential election used Surface to share with viewers information and analysis of the race leading up to the election The anchor analy:es polling and election results# views trends and demographic information and explores county maps to determine voting patterns and predict outcomes# all with the flic" of his finger 9n some hotels and casinos# users can do a range of things# such as watch videos# view maps# order drin"s# play games# and chat and flirt with
people between Surface tables.
2.2 (istory:
The product idea for Surface was initially conceptuali:ed in +//' by Steven %athiche of Microsoft Hardware and Andy Iilson of Microsoft -esearch 9n 5ctober +//'# 7D Jurlander# Michael Jim# Doel 7ehlin# %athiche and Iilson formed a virtual team to bring the idea to the next stage of development 9n +//(# the team presented the idea to the Microsoft 8hairman %ill Eates# in a group review Fater# the virtual team was expanded and a prototype nic"named T' was produced within a month The prototype was based on an 9J;A table with a hole cut in the top and a sheet of architect vellum used as a diffuser The team also developed some applications# including pinball# a photo browser and a video pu::le 5ver the next year# Microsoft built more than 03 early prototypes for Surface The final hardware design was completed in +//3 A similar concept was used in the +//+ science fiction movie Minority -eport As noted in the 7K7 commentary# the director Steven Spielberg stated the concept of the device came from consultation with Microsoft during the ma"ing of the movie 5ne of the filmCs technology consultantCs associates from M9T later &oined Microsoft to wor" on the Surface pro&ect Surface was unveiled by Microsoft 8;5 Steve %allmer on May (/# +//2 at The Iall Street DournalCs C7: All Things 7igitalC conference in8arlsbad# 8alifornia Surface 8omputing is part of %ho& reddy engineering college for women ) out of ()
Seminar report
MicrosoftCs ,roductivity and ;xtended 8onsumer ;xperiences Eroup# which is within the ;ntertainment > 7evices division The first few companies to deploy Surface will include HarrahCs ;ntertainment# Starwood Hotels > -esorts Iorldwide# T-Mobile and a distributor# 9nternational Eame Technology April '2# +//0# AT>T became the first retail location to launch Surface 9n Dune +//0 Harrah$s ;ntertainment launched Microsoft Surface at -io i%ar and 7isneyland launched it in Tomorrowland# 9nnoventions 7ream Home 5n August '(# +//0 Sheraton Hotels introduced Surface in their hotel lobbies at 3 locations 5n September 0th# +//0 MS?%8 began using the Surface to wor" with election maps for the +//0 !S ,residential ;lection on air MS?%8Cs political director# 8huc" Todd# was placed at the helm.
3 out of ()
Seminar report
4 out of ()
Seminar report
cameras with a net resolution of '/+) x 240# allowing it to sense# and react to items touching the tabletop Surface will ship with basic applications# including photos# music# virtual concierge# and games# that can be customi:ed for the customers A uniMue feature that comes preinstalled with Surface is the pond effect GAttractG application Simply# it is a GpictureG of water with leaves and roc"s within it <a lot li"e Microsoft Surface Fagoon# included in the Surface Touch ,ac"= %y touching the screen# users can create ripples in the water# much li"e a real stream Additionally# the
6ig + + ob&ect recognition
pressure of touch alters the si:e of the ripple created# and ob&ects placed into the water create a barrier that ripples bounce off# &ust as they would in real life
9ntel 8ore + Nuad Aeon GIoodcrestG O + 44 EH: with a custom motherboard form factor about the si:e of two ATA motherboards )E% 77-+-'/44 -AM 'T% 2+//-,M Hard 7rive
The discontinued <as of 4 Danuary +/''= commercially available version had the following specificationsP'2Q:
2 out of ()
Seminar report
9ntel 8ore + 7uo O + '( EH: +E% 77-+ -AM +3/E% SATA Hard 7rive
0 out of ()
Seminar report
3. Multi+touch technolo*y
3.1 'hat is multi+touch:
(n computing, multi-touch refers to a touch sensing surface)s *trackpad or touchscreen+ ability to recogni,e the presence of two or more points of contact with the surface. This plural-point awareness is often used to implement advanced functionality such as pinch to ,oom or activating predefined programs. (n an effort of disambiguation or marketing classification some companies further breakdown the various definitions of multi-touch. !n e-ample of this is .M defining multi-touch as a touch-screen)s ability to register three or more distinct positions.
3.2 (istory:
Multi-touch technology is used to develop a new type of human machine interface <HM9= for the control room of the Super ,roton Synchrotron particle accelerator The use of touchscreen technology to control electronic devices pre-dates multi-touch technology and the personal computer ;arly synthesi:er and electronic instrument builders li"e Hugh Fe 8aine and %ob Moog experimented with using touch-sensitive capacitance sensors to control the sounds made by their instruments 9%M began building the first touch screens in the late '14/s# and# in '12+#8ontrol 7ata released the ,FAT5 9K computer# a terminal used for educational purposes that employed single-touch points in a '4x'4 array as its user interface
6ig ( ' The prototypes of the x-y mutual capacitance multi-touch screens <left= developed at 8;-?
5ne of the early implementations of mutual capacitance touchscreen technology was developed at 8;-? in '122 based on their capacitance touch screens developed in '12+ by 7anish electronics engineer %ent Stumpe This technology was
1 out of ()
Seminar report
9n a handwritten note dated '' March '12+# Stumpe presented his proposed solution L a capacitative touch screen with a fixed number of programmable buttons presented on a display The screen was to consist of a set of capacitors etched into a film of copper on a sheet of glass# each capacitor being constructed so that a nearby flat conductor# such as the surface of a finger# would increase the capacity by a significant amount The capacitors were to consist of fine lines etched in copper on a sheet of glass L fine enough <0/ Rm= and sufficiently far apart <0/ Rm= to be invisible <8;-? 8ourier April '12) p''2= 9n the final device# a simple lacMuer coating prevented the fingers from actually touching the capacitors Multi-touch technology began in '10+# when the !niversity of TorontoCs 9nput -esearch Eroup developed the first human-input multi-touch system The system used a frosted-glass panel with a camera placed behind the glass Ihen a finger or several fingers pressed on the glass# the camera would detect the action as one or more blac" spots on an otherwise white bac"ground# allowing it to be registered as an input Since the si:e of a dot was dependent on pressure <how hard the person was pressing on the glass=# the system was somewhat pressure-sensitive as well 9n '10(# %ell Fabs at Murray Hill published a comprehensive discussion of touch-screen based interfaces P4Q 9n '10)# %ell Fabs engineered a touch screen that could change images with more than one hand 9n '103# the !niversity of Toronto group including %ill %uxtondeveloped a multi-touch tablet that used capacitance rather than bul"y camera-based optical sensing systems A brea"through occurred in '11'# when ,ierre Iellner published a paper on his multi-touch S7igital 7es"T# which supported multi-finger and pinching motions Karious companies expanded upon these inventions in the beginning of the twenty-first century The company 6ingerwor"s developed various multi-touch technologies between '111 and +//3# including Touchstream "eyboards and the iEesture ,ad Several studies of this technology were published in the early +///s by Alan Hedge# professor of human factors and ergonomics at 8ornell !niversity Apple acMuired 6ingerwor"s and its multi-touch technology in +//3 Mainstream exposure to multi-touch technology occurred in +//2 when the i,hone gained popularity# with Apple stating they Cinvented multi touchC as part of the i,hone announcement# however both the function and the term predate the announcement or patent reMuests# except for such area of application as capacitive mobile screens# which did not exist before 6ingerwor"s*AppleCs technology <Apple filed patents for in +//3-+//2 and was awarded with in +//1-+/'/= ,ublication and demonstration using the term Multi-touch by Defferson U Han in +//3 predates these# but Apple did give multi-touch wider exposure through its association with their new product and were
'/ out of ()
Seminar report
the first to introduce multi-touch on a mobile device MicrosoftCs table-top touch platform Microsoft Surface# which started development in +//'# interacts with both the users touch and their electronic devices Similarly# in +//'# Mitsubishi ;lectric -esearch Faboratories <M;-F= began development of a multi-touch# multi-user system called 7iamondTouch# also based on capacitance but able to differentiate between multiple simultaneous users <or rather# the chairs in which each user is seated or the floorpad the user is standing on=@ the 7iamondtouch became a commercial product in +//0 Small-scale touch devices are rapidly becoming commonplace# with the number of touch screen telephones expected to increase from +//#/// shipped in +//4 to +' million in +/'+
Apple has retailed and distributed numerous products using multi-touch technology@ most prominently including its i,hone smartphone and i,ad tablet Additionally# Apple also holds several patents related to the implementation of multi-touch in user interfaces Apple additionally attempted to register GMulti-touchG as a trademar" in the !nited States however its reMuest was denied by the !nited States ,atent and Trademar" 5ffice because it considered the term generic Multi-touch sensing and processing occurs via an AS98 sensor that is attached to the touch surface !sually# separate companies ma"e the AS98 and screen that combine into a touch screen@ conversely# a trac"padCs surface and AS98 are usually manufactured by the same company There have been large companies in recent years that have expanded into the growing multi-touch industry# with systems designed for everything from the casual user to multinational organi:ations 9t is now common for laptop manufacturers include multi-touch trac"pads on their laptops# and tablet computers respond to touch input rather than traditional stylus input and it is supported by many recent operating systems %ho& reddy engineering college for women '' out of ()
Seminar report
A few companies are focusing on large-scale surface computing rather than personal electronics# either large multi-touch tables or wall surfaces These systems are generally used by government organi:ations# museums# and companies as a means of information or exhibit display
3.4 Implementations:
Multi-touch has been implemented in several different ways# depending on the si:e and type of interface The most popular form are mobile devices# tablets# touchtables and walls %oth touchtables and touch walls pro&ect an image through acrylic or glass# and then bac"-light the image with F;7s Types
Multitouch 8apacitive Technology Surface 8apacitive Technology ,ro&ected 8apacitive Touch <,ST= 9n-cell: 8apacitive Touch -esistive Technology Analog -esistive 7igital -esistive or 9n-8ell: -esistive Multitouch 5ptical technologies 5ptical 9maging or 9nfrared technology -ear 7iffused 9llumination <79= 9nfrared Erid Technology <opto-matrix= or 7igital Iaveguide Touch <7IT=V or 9nfrared 5ptical Iaveguide
6rustrated Total 9nternal -eflection <6T9-= or 7iffused Surface 9llumination <7S9= 7ispersive Signal Touch <7ST= Jinect 9n-8ell: 5ptical Touch Iave Technologies Surface Acoustic Iave <SAI= %ending Iave Touch <%IT= 6orce-%ased Sensing or ?ear 6ield 9maging <?69=
The optical touch technology functions when a finger or an ob&ect touches the surface# causing the light to scatter# the reflection is caught with sensors or cameras that send the data to software which
'+ out of ()
Seminar report
dictates response to the touch# depending on the type of reflection measured Touch surfaces can also be made pressure-sensitive by the addition of a pressure-sensitive coating that flexes differently depending on how firmly it is pressed# altering the reflection Handheld technologies use a panel that carries an electrical charge Ihen a finger touches the screen# the touch disrupts the panelCs electrical field The disruption is registered and sent to the software# which then initiates a response to the gesture 9n the past few years# several companies have released products that use multi-touch 9n an attempt to ma"e the expensive technology more accessible# hobbyists have also published methods of constructing 79U touchscreens
Ma-e
Model
#orm #actor
!peratin* "ystem
Acer
Aspire AS32(0,E
Faptop
Iindows 2
'3 4 inch
'(44 W 240
H211 11
++*'/*+//1
Acer
Aspire X34'/!1/2+
All-in5ne
Iindows 2
+( inch
'1+/ W '/0/
H011 11
'+*+//1
Acer
Aspire '0+/,T
!ltra-thin Tablet
Acer
T+(/H
Monitor
?*A
+( inch
'1+/ W '/0/
H'01 // H(33 //
/0*+/''
Acer
Aspire
Faptop
Iindows 2
'3 4 inch
'(44 W 240
H211 11
++*'/*+//1
'( out of ()
Seminar report
Ma-e
Model
#orm #actor
!peratin* "ystem
AS32(0,E
Acer
9conia
Faptop
Iindows 2
'/
'(44 W 240
H'#'11 11
'*)*+/''
Acer
Aspire X34'/!1/2+
All-in5ne
Iindows 2
+( inch
'1+/ W '/0/
H011 11
'+*+//1
Apple
i,ad
All-in5ne
i5S
''
1 2 inch <diagonally=
April +/'/
%lac"berry ,layboo"
F87 7isplay
%lac"berry 5S
+/
2 inch
H33' 2) to H'3)1 //
September +/'/
8yberdyne 9nc
Tacto
All-in5ne
Finux
!nlimited
)4 inch
'1+/ W '/0/
'/*+/'/
H,
H, TouchSmart 4//
All-in5ne
Iindows 2
<+=Y
+( inch
'/0/p
H'#/)1 11
++*'/*+//1
H,
H, TouchSmart tx+
Tablet
Iindows 2
'+0/ W 0//
H211 11
++*'/*+//1
H,
H, TouchSmart 1'//
All-in5ne
Iindows 2
<+=Y
+( inch
'1+/ W '/0/
H'#+11 11
++*'/*+//1
H,
H, F7)+//tm
F87 7isplay
<+=Y
)+ inch
'1+/ W '/0/
H+#211 11
'+*+//1
H,
H, F+'/3tm
F87
+' 3 inch
'1+/ W '/0/
H+11 //
'/*+//1
') out of ()
Seminar report
Ma-e
Model
#orm #actor
!peratin* "ystem
7isplay
6u&itsu
Tablet
Iindows 2
'( ( inch
H'#231 //
'+*+//1
6u&itsu
Tablet
Iindows 2
H'#')1 //
'+*+//1
6u&itsu
Fife%oo" !H1//
Handheld ,8
Iindows 2
3 4 inch
'+*+//1
All-in5ne
Iindows 2
+( inch
'1+/ W '/0/
H021 11
''*+//1
IAEA H7 Eigabyte T'///, ?etboo"* Tablet Iindows 2 '/ inch '(44x240 F;7 bac"light H411 /(*+/'/
Multitouch Elobus Multitouch Eroup Elobe * EF5%!S = multitouch device All in Iindows 2 unlimited 'm Z <7iameter= '/3/ W '/3/ ('*'/*+/'/
Solution Spherical<domed
8ustomi:ed Microsoft Microsoft Surface All-in- Iindows Kista 5ne Iith Surface Shell 3+ (/ inch '/+) W 240
Iindows
!nlimited
)4 inch
'1+/ x '/0/
Approx
1*+/'/
'3 out of ()
Seminar report
Ma-e
Model
#orm #actor
!peratin* "ystem
)42 Advanced
A,*2# Finux# 5S A
F87 7isplay
<+=Y
IAEA
H+#+11 11
'+*+//1
(+# )/# )+# ?eprash Technology ??-Touch Touch MultiTouch 7evice All-in5ne Iindows 2 (+ )4# 3+# 33# 32# 2/# 0+# '// and +// inches as well as custom screen si:es '1+/ [ '/0/ Danuary +/'/
H+#(11 All-in5ne Iindows A,*Kista*2#Ma c (+ )+ inch '1+/ [ '/0/ <MultiTouch E+# (+ inches# screen only= H'/#///L Dune +/'/
,N Fabs
iTable
'4 out of ()
Seminar report
Ma-e
Model
#orm #actor
!peratin* "ystem
Samsung
All 9n 5ne
Android
2 inch <diagonally=
Karies by -egion
Sony
F Series
All 9n 5ne
Iindows 2
<+=Y
+) inch
'1+/ W '/0/
H'#+11 11
'+*+//1
Toshiba
Faptop
Iindows 2
'( ( inch
H13/ //
++*'/*+//1
4. 2esture reco*nition
/ig 2.3 ! child being sensed by a simple gesture recognition algorithm detecting hand location and movement
Eesture recognition is a topic in computer science and language technology with the goal of interpreting human gestures via mathematical algorithms Eestures can originate from any bodily motion or state but commonly originate from the face or hand 8urrent focuses in the field include emotion recognition from the face and hand gesture recognition Many approaches have been made using cameras and computer vision algorithms to interpret sign language However# the
'2 out of ()
Seminar report
identification and recognition of posture# gait# proxemics# and human behaviors is also the sub&ect of gesture recognition techniMues Eesture recognition can be seen as a way for computers to begin to understand human body language# this building a richer bridge between machines and humans than primitive text user interfaces or even E!9s <graphical user interfaces=# which still limit the ma&ority of input to "eyboard and mouse Eesture recognition enables humans to interface with the machine <HM9= and interact naturally without any mechanical devices !sing the concept of gesture recognition# it is possible to point a finger at the computer screen so that the cursor will move accordingly This could potentially ma"e conventional input devices such as mouse# "eyboards and even touch-screens redundant Eesture recognition can be conducted with techniMues from computer vision and image processing The literature includes ongoing wor" in the computer vision field on capturing gestures or more general human pose and movements by cameras connected to a computer
5ffline gestures: Those gestures that are processed after the user interaction with the ob&ect An example is the gesture to activate a menu 5nline gestures: 7irect manipulation gestures They are used to scale or rotate a tangible ob&ect.
4.3 %ses:
'0 out of ()
Seminar report
Eesture recognition is useful for processing information from humans which is not conveyed through speech or type As well# there are various types of gestures which can be identified by computers
4.3.1 "i*n lan*ua*e reco*nition: Dust as speech recognition can transcribe speech to text# certain types of gesture recognition software can transcribe the symbols represented through sign language into text 4.3.2 #or socially assistive ro/otics: %y using proper sensors <accelerometers and gyros= worn on the body of a patient and by reading the values from those sensors# robots can assist in patient rehabilitation The best example can be stro"e rehabilitation 4.3.3 3irectional indication throu*h pointin*: ,ointing has a very specific purpose in our society# to reference an ob&ect or location based on its position relative to ourselves The use of gesture recognition to determine where a person is pointing is useful for identifying the context of statements or instructions This application is of particular interest in the field of robotics 4.3.4 Control throu*h facial *estures: 8ontrolling a computer through facial gestures is a useful application of gesture recognition for users who may not physically be able to use a mouse or "eyboard ;ye trac"ing in particular may be of use for controlling cursor motion or focusing on elements of a display 4.3.5 Alternative computer interfaces: 6oregoing the traditional "eyboard and mouse setup to interact with a computer# strong gesture recognition could allow users to accomplish freMuent or common tas"s using hand or face gestures to a camera 4.3.4 Immersive *ame technolo*y: Eestures can be used to control interactions within video games to try and ma"e the game playerCs experience more interactive or immersive 4.3.5 6irtual controllers: 6or systems where the act of finding or acMuiring a physical controller could reMuire too much time# gestures can be used as an alternative control mechanism 8ontrolling secondary devices in a car# or controlling a television set are examples of such usage 4.3.7 Affective computin*: 9n affective computing# gesture recognition is used in the process of identifying emotional expression through computer systems 4.3.8 emote control: Through the use of gesture recognition# Gremote control with the wave of a handG of various devices is possible The signal must not only indicate the desired response# but also which device to be controlled
'1 out of ()
Seminar report
4.4.1 'ired *loves: These can provide input to the computer about the position and rotation of the hands using magnetic or inertial trac"ing devices 6urthermore# some gloves can detect finger bending with a high degree of accuracy <3-'/ degrees=# or even provide haptic feedbac" to the user# which is a simulation of the sense of touch The first commercially available hand-trac"ing glovetype device was the 7ataElove# a glove-type device which could detect hand position# movement and finger bending This uses fiber optic cables running down the bac" of the hand Fight pulses are created and when the fingers are bent# light lea"s through small crac"s and the loss is registered# giving an approximation of the hand pose 4.4.2 3epth+a9are cameras: !sing speciali:ed cameras such as time-of-flight cameras# one can generate a depth map of what is being seen through the camera at a short range# and use this data to approximate a (d representation of what is being seen These can be effective for detection of hand gestures due to their short range capabilities 4.4.3 "tereo cameras: !sing two cameras whose relations to one another are "nown# a (d representation can be approximated by the output of the cameras To get the camerasC relations# one can use a positioning reference such as a lexian-stripe or infraredemitters 9n combination with direct motion measurement <47-Kision= gestures can directly be detected 4.4.4 Controller+/ased *estures: These controllers act as an extension of the body so that when gestures are performed# some of their motion can be conveniently captured by software Mouse gestures are one such example# where the motion of the mouse is correlated to a symbol being drawn by a personCs hand# as is the Iii -emote# which can study changes in acceleration over time to represent gestures P++QP+(QP+)Q 7evices such as the FE ;lectronics Magic Iand# the Foop and the Scoop use Hillcrest FabsC 6reespace technology# which uses M;MS accelerometers# gyroscopes and other sensors to translate gestures into cursor movement The software also compensates for human tremor and inadvertent movement
+/ out of ()
Seminar report
4.4.5 "in*le camera: A normal camera can be used for gesture recognition where the resources*environment would not be convenient for other forms of image-based recognition Although not necessarily as effective as stereo or depth aware cameras# using a single camera allows a greater possibility of accessibility to a wider audience
4.5 Al*orithms:
6ig ) 3 ' 7ifferent ways of trac"ing and analy:ing gestures exist# and some basic layout is given is in the diagram above 6or example# volumetric models convey the necessary information reMuired for an elaborate analysis# however they prove to be very intensive in terms of computational power and reMuire further technological developments in order to be implemented for real-time analysis 5n the other hand# appearance-based models are easier to process but usually lac" the generality reMuired for Human-8omputer 9nteraction .
7epending on the type of the input data# the approach for interpreting a gesture could be done in different ways However# most of the techniMues rely on "ey pointers represented in a (7 coordinate system %ased on the relative motion of these# the gesture can be detected with a high accuracy# depending of the Muality of the input and the algorithm$s approach 9n order to interpret movements of the body# one has to classify them according to common properties and the message the movements may express 6or example# in sign language each gesture represents a word or phrase The taxonomy that seems very appropriate for Human8omputer 9nteraction has been proposed by Nue" in SToward a Kision-%ased Hand Eesture 9nterfaceT He presents several interactive gesture systems in order to capture the whole space of the gestures: ' Manipulative@ + Semaphoric@ ( 8onversational Some literature differentiates + different approaches in gesture recognition: a (7 model based and an appearance-based The foremost method ma"es use of (7 information of "ey elements of the body parts in order to obtain several important parameters# li"e palm position or &oint angles 5n the other hand# Appearance-based systems use images or videos for direct interpretation.
+' out of ()
Seminar report
6ig ) 3 + A read hand <left= is interpreted as a collection of vertices and lines in the (7 mesh version <right=# and the software uses their relative position and interaction in order to infer the gesture
The (7 model approach can use volumetric or s"eletal models# or even a combination of the two Kolumetric approaches have been heavily used in computer animation industry and for computer vision purposes The models are generally created of complicated (7 surfaces# li"e ?!-%S or polygon meshes The drawbac" of this method is that is very computational intensive# and systems for live analysis are still to be developed 6or the moment# a more interesting approach would be to map simple primitive ob&ects to the person$s most important body parts < for example cylinders for the arms and nec"# sphere for the head= and analyse the way these interact with each other 6urthermore# some abstract structures li"e super-Muadrics and generalised cylinders may be even more suitable for approximating the body parts Kery exciting about this approach is that the parameters for these ob&ects are Muite simple 9n order to better model the relation between these# we ma"e use of constraints and hierarchies between our ob&ects 4.5.2 "-eletal+/ased al*orithms:
6ig ) 3 ( The s"eletal version <right= is effectively modelling the hand <left= This has less parameters than the volumetric version and itCs easier to compute# ma"ing it suitable for real-time gesture analysis systems
9nstead of using intensive processing of the (7 models and dealing with a lot of parameters# one can &ust use a simplified version of &oint angle parameters along with segment lengths This is "nown as a s"eletal representation of the body# where a virtual s"eleton of the person is computed and parts of the body are mapped to certain segments The analysis here is done using the position
++ out of ()
Seminar report
and orientation of these segments and the relation between each one of them< for example the angle between the &oints and the relative position or orientation= Advantages of using s"eletal models:
Algorithms are faster because only "ey parameters are analy:ed ,attern matching against a template database is possible !sing "ey points allows the detection program to focus on the significant parts of the body
6ig ) 3 ) These binary silhouette<left= or contour<right= images represent typical input for appearance-based algorithms They are compared with different hand templates and if they match# the correspondent gesture is inferred
These models don$t use a spatial representation of the body anymore# because they derive the parameters directly from the images or videos using a template database Some are based on the deformable +7 templates of the human parts of the body# particularly hands 7eformable templates are sets of points on the outline of an ob&ect# used as interpolation nodes for the ob&ect$s outline approximation 5ne of the simplest interpolation function is linear# which performs an average shape from point sets # point variability parameters and external deformators These template-based models are mostly used for hand-trac"ing # but could also be of use for simple gesture classification A second approach in gesture detecting using appearance-based models uses image seMuences as gesture templates ,arameters for this method are either the images themselves# or certain features derived from these Most of the time# only one < monoscopic= or two < stereoscopic = views are used
4.4 Challen*es:
+( out of ()
Seminar report
There are many challenges associated with the accuracy and usefulness of gesture recognition software 6or image-based gesture recognition there are limitations on the eMuipment used and image noise 9mages or video may not be under consistent lighting# or in the same location 9tems in the bac"ground or distinct features of the users may ma"e recognition more difficult The variety of implementations for image-based gesture recognition may also cause issue for viability of the technology to general usage 6or example# an algorithm calibrated for one camera may not wor" for a different camera The amount of bac"ground noise also causes trac"ing and recognition difficulties# especially when occlusions <partial and full= occur 6urthermore# the distance from the camera# and the cameraCs resolution and Muality# also cause variations in recognition accuracy 9n order to capture human gestures by visual sensors# robust computer vision methods are also reMuired# for example for hand trac"ing and hand posture recognition or for capturing movements of the head# facial expressions or ga:e direction
5. (uman+computer interaction
HumanLcomputer 9nteraction <H89= involves the study# planning# and design of the interaction between people <users= and computers 9t is often regarded as the intersection of computer science# behavioral sciences# design and several other fields of study The term was coined by 8ard# Moran# and ?ewell in their germinal boo"# GThe ,sychology of Human-8omputer 9nteraction G The term connotes that# unli"e other tools with only limited uses <such as a hammer# useful for driving nails# but not much else=# a computer has many affordances for use and this ta"es place in a sort of open-ended dialog between the user and the computer 9nteraction between users and computers occurs at the user interface <or simplyinterface=# which includes both software and hardware@ for example# characters or ob&ects displayed by software on a personal computerCs monitor# input received from users via hardware peripherals such as "eyboards and mouses# and other user interactions with large-scale computeri:ed systems such
+) out of ()
Seminar report
as aircraft and power plants The Association for 8omputing Machinery defines human-computer interaction as Ga discipline concerned with the design# evaluation and implementation of interactive computing systems for human use and with the study of ma&or phenomena surrounding them G An often-sought facet of H89 is the securing of user satisfaction # although user satisfaction is not the same thing as user performance by most meaningful metrics %ecause human-computer interaction studies a human and a machine in con&unction# it draws from supporting "nowledge on both the machine and the human side 5n the machine side# techniMues in computer graphics# operating systems# programming languages# and development environments are relevant 5n the human side# communication theory# graphic and industrial design disciplines# linguistics#social sciences# cognitive psychology# and human factors such as computer user satisfaction are relevant ;ngineering and design methods are also relevant 7ue to the multidisciplinary nature of H89# people with different bac"grounds contribute to its success H89 is also sometimes referred to as manLmachine interaction <MM9= or computerLhuman interaction <8H9= Attention to human-machine interaction is important# because poorly designed human-machine interfaces can lead to many unexpected problems A classic example of this is the Three Mile 9sland accident where investigations concluded that the design of the human-machine interface was at least partially responsible for the disaster Similarly# accidents in aviation have resulted from manufacturersC decisions to use non-standard flight instrument and*or throttle Muadrant layouts: even though the new designs were proposed to be superior in regards to basic human-machine interaction# pilots had already ingrained the GstandardG layout and thus the conceptually good idea actually had undesirable results
+3 out of ()
Seminar report
Historically# pen computing <defined as a computer system employing a user-interface using a pointing device plus handwriting recognition as the primary means for interactive user input= predates the use of a mouse and graphical display by at least two decades# starting with the Stylator and -A?7 tablet systems of the '13/s and early '14/s
4. Multi+touch devices
Multi-touch gestures are employed by some touchscreen devices to perform various actions A gesture refers to a motion used to interact with multipoint touch screen interfaces
+4 out of ()
Seminar report
The latest Mac%oo" and Mac%oo" ,ros <late +//0 models= both have a new SbuttonlessT trac"pad which is bigger and made of a touch-friendly# and wear-resistant glass The entire trac"pad has been completely redesigned and it$s also one large button so it$s clic"able everywhere on the surface ?o separate button means there$s more room for additional multi-touch gestures and your fingers can move with ease on the smooth and sil"y glass surface
+2 out of ()
Seminar report
Ie got to play around with a SMA-T Table in a classroom full of luc"y "ids at Haines ;lementary School in 8hicago this morning# and we came away impressed with how much they loved it The multitouch table is built on the same basic idea and hardware as Microsoft Surface -- Kista ,8# AEA pro&ector# infrared camera -- but itCs a custom patented SMA-T design# not Surface lite or anything li"e that That said# the multitouch system isnCt Muite as responsive as Surface# and the "id%ho& reddy engineering college for women +0 out of ()
Seminar report
proof plastic screen felt a little weird# but it certainly wor"s well enough -- the Table recogni:es up to )/ touches and we saw some interesting demos# ranging from the standard rotate * :oom photo app to painting and pu::le games Teachers get admin access with a special !S% "ey that enables them to manage apps# and thereCs an S7J in the wor"s# so hopefully thereCll be Muite a few to manage SMA-T says the Table should start shipping next spring for somewhere between H2#/// to H0#/// each -- obviously the company will be targeting school systems with its extensive SMA-T %oard sales networ"# but well-off parents will be able to score one for their darling children as well 8hec" a few vids of the table in action after the brea"
5. Applications
The following is an example of a possible application using Microsoft Surface: '= 5n the left you have your device which has stored your information += 5n the right you have your friend$s device which has stored his*her information (= 9n the center it$s showing how you can pull the information needed from each device and compile it to complete the final pro&ect
+1 out of ()
Seminar report
(/ out of ()
Seminar report
construction of pro&ects easier Also# students will be able to wor" on class assignments together or help each other and sometimes students are able to learn and understand better when the information is delivered or reiterated from their peers in a more creative fashion
(' out of ()
Seminar report
8. Conclusion
There wouldn$t be a surprise if each student$s des" top be replaced by a multi-touch technology similar to the Microsoft Surface ;ach classroom and teacher would have their Surface applications customi:ed to fit their specific curriculum These devices offer various ways of visuali:ing the information in order to improve understanding which enables our students to excel 9 feel we need to find ways to "eep up with the rapidly growing world of technology and integrate it into our classrooms or our students are going to surpass us and figure out ways to do things better and faster at home on their own personal computer Iith Microsoft Surface the opportunities are endless with the ability to create custom applications for specific businesses or educational purposes or building pac"aged applications for use across a range of industries or schools
(+ out of ()
Seminar report
(( out of ()