Praxis Business School Augmenting Our World

A report submitted to, Prof. Prithwis Mukherjee Business Information Systems

By, Rewati Rane Roll No: B10042 Batch of 2010-2012

Imagine a world where you walk down the road with your phone at arm¶s length guiding you to the nearest eatery or museum, or getting information about a building you walk by. This is what is coming soon to the world of technology. Augmented reality is the latest buzz on the streets. It is the ability to overlay information on a live video feed of the world. In much easier terms, augmented reality is the incorporation of virtual and artificial elements into the physical world, in real time. That is, unlike virtual reality which supplements the physical world, augmented reality blends together virtual and actual components.

Augmenting Our World
For the longest time now, video games have been entertaining us and have made us believe in a world apart from what¶s real. Since the introduction of these video games in the 1970¶s computer graphics have become much more advanced and these games are pushing the barriers of photorealism. Researchers are now taking graphics from the television and computers and integrating them into the real world. This is what augmented reality is. It makes the line between what¶s real and what¶s virtual hazy by enhancing what we see, hear, feel and smell. Augmented reality is closer to the real world in the sense that it adds graphics, sounds, feedback, and smell, to the world as it exists. The basic idea of augmented reality is to superimpose graphics and sensory enhancements in real time. However, as simple as it may sound, it really isn¶t. It sounds like the thing that television has been doing for the longest time with graphics but it is so different from anything we ever imagined. Augmented reality takes graphics and virtual reality to a completely different level. Where, TV creates graphics for only one point of view, augmented reality systems will create graphics for different perspectives. Augmented reality research has been going on for many years now around the world. Two researchers in 2009 presented their augmented reality system at the TED conference in 2009. They called this model SixthSense. The model uses a camera, smart phone, projector and mirror as its components. All these components are strung together and worn around a person¶s neck. The user of this model also wears coloured caps on his fingers which are used to manage the images the projector emits. The model works with the projector turning any surface into an interactive screen. Basically the model uses the camera and the mirror to gather information from the surroundings. This information then gets fed into the phone which processes it and then gets projected by the projector on any surface be it the empty space in front of your eyes, your palm or even the windshield of your car. This model is so interesting that because the user wears the camera around his neck even if he is looking at a poster of a movie, the camera will capture the picture, feed it into the phone which will process information about the movie and project it in front of the person on any surface.

Typical Augmented Reality System
A standard virtual reality system seeks to completely immerse the user in a computer generated environment. This environment Is maintained by the system in a frame of reference registered with the computer graphics system that creates the rendering of the virtual wworld, for this immersonf to be effective, the ego-centered frame of reference maintained by the user¶s body and the brain must be registered with the virtual world reference. This requires that motions or changes made by the user will result in the appropriate changes in the perceived virtual world. Because the user is looking at a virtual world there is no natural connection between these two reference frames and a connection mst be created. An augmented reality system could be considered the ultimate immersive system. The user can not become more immersed in the real world. The task now is to register the virtual frame of reference with what the user is seeing. This registration is more critical in an augmented reality system.

The object or scene is viewed by an imaging device which can be a camera or video camera. The imaging devise then processes the image of the 3D world onto a 2D image plane. The intrinsic and extrinsic parameters of the device determine wxactly whats is projected onto the image plane. The generation of the virtual image is done with a standard computer graphics system. The virtual objects are modeled in an object reference frame. The graphics system requires the information about the imaging of the real scene so that it can correctly render these objects. This data will control the synthetic camera that is used to generate the image of the virtual objects. This image is then merged with the image of a real scene to form the augmented reality image. This video imaging and graphic rendering is pretty straight forward. The research activities in augmented reality center around two aspects of the problem. One is to develop methods to register the two distinct sets of images and keep them registered. Some new work in this area has started to make use of compute vision techniques. The second is in the display technology for merging the two images.

The Technology used
The main technology that is used for augmented reality to work are input devices, cameras, display and tracking devices. Also needed is that technology that is present on smart phones as well as a powerful CPU, GPS systems and a compass. In terms of display there are three basic platforms, head-mounted dislplay, handheld displays and spatial displays. Head- mounted displays are those that place images of both the physical world and the graphics that have been processed by the smart device and place it in front of the user. The head mounted displays are either optical see through or video see through in nature. An optical see through display uses half-silver mirror technology to let views of the physical world pass through the lens and be reflected in the users eyes. The main advantage of the head mounted displays are that it allows the user to get an in depth experience of augmented reality.

Handheld displays are those that employ a computing device with a display which fits into the users hands. Handheld augmented displays employ video see through techniques to overlay the graphical information to the physical world. The two main advantages of handheld devices are, they are portable and hence convenient to use and also they are omnipresent in nature


Spatial displays make use of digital projectors graphical displays onto physical objects. In Spatial augmented reality, the displays are separated from the users. Spatial augmented reality or SAR as it is usually called has a good number of advantages in comparison with head mounted displays and handheld augmented systems. The SAR does not need to be carried around by the user unlike the head mounted display or the handheld device. Apart from that, it can be used by many people at one time, it can provide good collaborative work and users can see each others faces, it does not require a specific area to project displays like the head mounted and handheld devices. The SAR can display images on any given surface. Because of the use of any number of devices, the projection area can be expanded according to the users will.

Apart from these three main types of augmented reality, there are others which include projection, recognition, location and outline. Projection is the most common type of augmented reality which augments virtual images. Which is basically, what you see live on tv. An example of this is when football matches are telecasted live on TV, usually when the line of scrimmage and yardage need to be indicated. These lines are not actually there on the field but can only be seen by TV viewers. This is augmented reality by means of projection. The next type is recognition which uses the recognition of faces and other real world elements to to provide one with real time information. An example of this would be the face recognition device that is present on some laptops in order for the user to access their work. Location, which uses technology such as GPS to provide you information about the direction in which your travelling and finally there¶s outline which is a type of augmented reality which outlines your body or a part of your body with virtual materials and allows you to pick up objects that do not exist in reality.

What is already available?
As of now, augmented reality is available only in nooks and corners of this world. In most places where it is a feasibility only handheld devices are available as these are the smart phones. However, augmented reality has made headway into different fields and is being used for many different purposes as of now. Some of these are; Advertising: where marketers use AR to promote their products. For example, at the 2008 LA Auto show, Nissan unveiled the concept vehicle cube and presented the visitors with a brochure which when held against a webcam would show several versions of the vehicle. Just recently in 2010, Walt Disney used augmented reality to connect a movie experience to outdoor advertising. Navigation devices: AR can be used as a navigation device for a number of applications. For example, building navigation can be upgraded for maintaining outdoor plants. Outdoor navigation can be used for military and disaster management purposes. Other uses are in airplanes and by vehicles in order to navigate their directions. Industrial applications: AR can be used to compare the data of digital mock-ups with physical mockupsfor efficiently finding discrepancies between two sources. It can further be used to safeguard digital data in combination with real existing prototypes. Music: Some pop bands such a Duran Duran have incorporated augmented rea in their concerts lity to make them more appealing and fascinating. These are only some of the uses of AR, there are still a number of uses of AR that are in practice. Among these are, military and emergency services, Art, architecture, sightseeing, collaboration, entertainment and education, among others.

What can be expected in the future?
Augmented reality is slowly gaining recognition today. In an ever changing world like ours is it is but obvious that the uses of augmented reality will keep increasing if not change. Even though at present it may not be very widespread there is not a shadow f a doubt that in the years to come, AR will take on a very large part of the world as the most widely used technology by every single person. The future of augmented reality is very bright. For one you may learn things about countries that you have lived in since ages by just pointing your AR enabled phone to a particular place. Expanding a PC screen into the real environment in real time, this means that program windows and icons will appear right before your eyes on any surface just by gazing or pointing. Other future applications of augmented reality include, virtual plants, wallpapers, panoramic views, artwork decorations, etc for enhancing everyday life. For example, a virtual window could be placed on a real wall showing a live feed of a camera placed on the exterior of the building. Apart from this, augmented reality can be put to use when costs need to be cut down during the construction of a building. This happens because yu can save on materials by using virtual markers to designate where a beam should go or which structural support to inspect. Paleontologists working in shifts to assemble a dinosaur could leave µnotes¶ to team members on the bones themselves, artists could produce

virtual graffiti and doctors could overlay a digital image of a patients X-rays onto a mannequin for added realism. With augmented reality systems getting into the market we may soon start seeing posters, graffiti and even festival decorations on any given surface just by pointing a finger or gaze at one particular place. The future of augmented reality is clearly very bright even though as of now it has already found its way to our cell phones which are now smart phones. We are talking about the future of augmented reality and how it will impact our lives in the years to come. In the near future just one of many possible applications of the concept of augmented data will be the use of it in airplanes. This will help make air travel more convenient. Like robotics, there is a visceral and physical representation of the underlying artificial intelligence involved. From expediting everyday business travel to fuelling potential military research, facilitating heightened responses in emergency scenarios and powering the world¶s most immersive video games, augmented reality will change the way we think about data and how we process information. According to Sam Bergen, an associate art director for digital innovation at Ogilvy and Mather, ³Augmented reality will become a part of everyday life.´One thing is for certain, Augmented Reality is definitely making the world and technology easily understandable and useful for the generations to come. In many ways, AR is an attempt to integrate digital content with physical objects. One of the foremost examples is the US Postal Service¶s µpriority mail shipping stimulator.¶ It begins with the individual first seeing a digital representation of the shipping container. Next, the object that needs to be shipped is selected and then help up to a webcam that assesses whether the object, say a toy, will fit or a larger or smaller container needs to be used, depending on the size of the object to be shipped. Like this augmented reality is used in many different ways and for mnay different purposes. Some say that AR could become so widespread that we would use it just as frequently as we sign on to the internet. It could become a part of our routine and how we work and manage our lives. Augmented Reality should be looked at something that can change the very basis of our lives. There may eve come a n time when we wonder about how we ever lived without AR in our lives and we may even have people write books about the way it was before AR came into being. However, far out that may sound. So where will all this innovation lead? Some researchers say that in the next 10-20 years, augmented reality could become a commonplace. But they also warn of the harm that this can cause because of the careless use of augmented reality as it will be ubiquitous. So to conclude this, experts agree on one thing that even though AR will definitely be a major step towards integrating virtual reality with the physical world, enhancing our perception and providing clues about the underlying data that we would not normally understand. It is the responsibility of the future innovators to make sense of this technology and compute it for the lay man. Only then can this whole idea of augmented reality really become something worth living for. While on the topic of the uses of augmented reality and its future let us make a few comparisons between augmented reality and other platforms.

Augmented Reality VS Virtual Reality
Virtual reality is a technology that encompasses a broad spectrum of ideas. It is defined as a computer generated, interactive, three-dimensional environment in which a person is immersed. This implies that virtual reality is computer generated and the interaction is only between the computer and the person and lastly, the person is immersed in the environment. This goes to show that virtual reality leaves no scope for the user to get distracted from the activity he may be doing. This is done with the help if a head mounted display which is worn by the user, After the user wears it, the outside world gets blocked entirely and the user is focused only on what is being shown on the screen in front of him. The computer generated graphics are so powerful that they keep the use glued to what is happening as it provides an adequate amount of realism and this increases the level of interaction. A very distinct difference between the two is that virtual reality is a totally immersive system. This also means that once a person is a part of virtual reality he loses all sense of what¶s happening around him. He is completely under the control of the system. Whereas in augmented reality this is not the

case as the user controls what he wants and doesn¶t wasn¶t. Here the system is under the control of the user and not the other way around. Milgram describes how augmented reality and virtual reality work are related. He does this with the help of the diagram shown below.

In the figure above, the real environment and the virtual environment are at two extreme ends of this virtual continuum, as he called it. The middle region was called the mixed reality. Augmented reality lies next to the real environment end of the continuum meaning that augmented reality is much closer to the real world than the virtual environment. According to Milgram, augmented virtuality is a term used to identify systems which are mostly synthetic with some real world imagery added. Such as texture mapping videos on to virtual objects. He suggested three axes for categorizing these systems; Reproduction Fidelity, Extent of Presence Metaphor and the extent of World Knowledge. Reproduction Fidelity refers to the quality of the computer generated imagery ranging from simple wireframe approximations to complete photorealistic renderings. The Extent of Presence Metaphor refers to the level of immersion of the user within the the displayed scene. Finally, Extent of World Knowledge relates to the fact that one requires knowledge apart from that of computer graphics.

Augmented Reality VS Application Domains
Researchers working with augmented reality systems have proposed them as solutions in many domains. The areas that have been discussed range from enter tainment to military training. Many of the domains such as medical are also proposed for the augmented reality systems.

Pre-operative imgaging studies such as CT or MRI scan have to be performed to provide the surgeon with the necessary view of the are to be operated on. This helps them in carrying out the operation in a specific manner. After this, it is up to the surgical team to pre-empt its course of action properly. If augmented reality is used then it will help perform the surgery with ease. This is because with AR the CT or MRI data will be registered on the patient correctly and the surgeons can look at this while the procedure is progressing.

Augmented reality is also used in the entertainment and news business. In the weather report we usually see the reporter standing in front of weather maps and satellite images of a specific area. In actuality, the reporter is standing in front of a blue or green screen. The real image is augmented with computer generated maps using a technology called chroma-keying. The same technology is applied in movies too especially when stunts are being performed.

Military training
The military makes use of augmented reality in the cockpit of their aircrafts that presents the pilot wit information on the windshield of the aircraft or the visor of the flight helmet. By equipping military

personnel with helmet mounted visor displays or a special purpose rangefinder, the activities of other units participating in the exercise can be imaged.

Robotics and telerobotics
In the domain of robotics and telerobotics, an augmented display system can assist the user of the system. This is used by the operator the robot or telerobot to guide it in performing all the required actions.

Consumer design
Consumer design is already, at present making use of virtual reality. However, with the development of augmented reality it will be very easy to show interior decorators or even furniture stores. With Ar consumers can carry video tapes shot from various viewpoints of the house. Augmented reality can also be used in the fashion and beauty industry. If a store does not a have a particular style in someone¶s size, then the correct size can be used to augment the image of the person. In this way, when one looks in a three sided mirror, they would see the image of the new dress on their body. Then change in the hem length, shoulder style and any other changes can be made before placing the order. In the same way when going into a beauty salon, the image of a hairstyle can be augmented onto a person¶s head and they can see how it looks on them before getting their hair cut.

Limitations of Augmented Reality
The limitations of augmented are plentiful however interesting its uses may be. For example, the GPS system used in vehicles can deliver information accurately only within 30 feet. Also, GPS does not work indoors. People may not want to depend on their mobile phones which have small screens on which information can be seen. Another drawback of augmented reality will be the reduction of interaction between people. People will no longer require tour guides when going to a new city, nor will they require decorators for their homes. All this will be done through AR systems and this can really hamper trade and business in the world. Hair stylists will become redundant as they will only be required to cut hair and not think up styles and thus their creativity and expertise will not be put to full use. Other concerns are related to privacy. Image recognition software coupled with AR will quite soon allow us to point our phones at people, even strangers, and instantly see information about them from the various social networking sites. With most of these services, people willingly put information about themselves online, but it may be an unwelcome shock to meet someone, only to have him instantly know so much about your life and background.

The current users of augmented reality
While it may take a while before a gadget like SixthSense gets recognition the world over and is used by people in their day to day live, the primitive version of AR are slowly becoming available to people in some parts of the world. There are a few versions of augmented reality that are available with certain applications on the Iphone and other android phones. In the Netherlands, cell phone owners can download an application called µLayar¶ that uses the phone¶s camera and GPS system to gather information about the surrounding area. Layar also shows information about nearby restaurants and visiting locations in that are. All this can be done just by downloading the app and then pointing your phone in the direction from where you want the information. Layar will then overla this information on your phone¶s screen. By pointing your phone at a building, Layar will also be able to tell you what companies that have offices in that particular building are hiring. However, Layar isn¶t the only application available of this kind. There is an application available on Apple¶s Iphone called Yelp. Basically, this application works with augmented reality to give you information about areas in your surroundings. It is called Yelp Monocle. All one has to do is shake their Iphone 3GS thrice and the application will start up. Once it starts, it will automatically give you information about surrounding restaurants and the like. There are also a few other applications similar to the two above which work on an augmented reality system. Some of these are Urbanspoon, Wikitude which finds information from Wikipedia and delivers it to your phone. Another field that¶s using augmented reality is gaming. Games like Pac-man and many others are using augmented reality to make them more interactive. All these games are to be played using a pair of data glasses and the computer overlays digital information onto what the viewer physically sees. A company called Total Immersion makes software that applies augmented reality to baseball cards. All one has to do is download the software from the internet and then hold the baseball up to the camera which will process the image and then display the related video on the computer. Now what a person has to do is move the card in their hands, making sure it is in front of the camera and watch the 3D figure on the screen perform different actions. there is a Canadian company called Arcane Technologies which has sold augmented reality systems to the U.S military. These AR enabled head-mounted devices could overlay blueprints or a a view from a satellite or overhead drone directly on to a soldiers field of vision. However, all these efforts that have been made to incorporate augmented reality in different ways s just the tip of the iceberg. In the next couple of years we will definitely see more advanced uses of the realm of augmented reality. We may see the world cup which is happening in some country in our very own park or scavenger hunts that use virtual objects. Anything is possible!

The Hurdles Present in Augmented Reality
Even though we are very kicked about eh idea of augmented reality as it probably one of the more innovative ones, with strong visuals and the general feeling of been in a video game making it a bit more exciting than the structured data web, another hotly tipped trend. However, augmented reality is still a little set back in some respects. In terms of the AR technology used, there are still some hurdles that need to be overcome. Some of the hurdles that need tobe overcome are as follows:

The margin of error is large. It¶s 10 metres on today¶s GPS system and when you¶re trying to superimpose an image on another at close range, that is not accurate enough. Techniques like skyhook and image recognition can be used to mitigate this. In 2014 a new GPS satellite network will be launched giving a much higher degree of accuracy.

There are no open standards amongst the current crop of AR browsers which means you have to make a judgement call on the platform of choice. As in other walled garden scenarios, that means data can¶t be shared between platforms.

Availability of AR capable devices:
AR is currently limited to smartphones as the complexity of displaying AR conrent requires numerous bits of smartphone hardware such as GPS, camera, gyroscope or accelerometer+ compass, 3D rendering capability etc. this is found only in the latest handsets as of now but will soon filter down throughout the mobile landscape.

From everything that we have read so far it clear to see that augmented reality is one day going to become a reality in totality for all of us living on this planet. Augmented reality has not yet touched all spheres and become something that even the common man can use. It is only the very wealthy and the techies that actually have access to this technology. This means that in order for augmented reality to become a part of all our lives, costs of using these systems will need to be cut. It is not feasible for every single person to purchase an iphone or an android or even a very high tech head mounted display and so researchers and experts will have to come up with newer ways of augmenting our world. Soon we will find that our entire lives are being controlled by augmented reality and that augmented reality will have be what¶s real.

y y y y y y y -reality-hud.jpg

Sign up to vote on this title
UsefulNot useful