Introduction to Emerging Technology

Chapter Five
Augmented Reality (AR)

1
Presentation Outline
 Introduction
 History of AR
 Types of AR
 Technologies of AR /AR Devices
 Architecture of AR Systems
 AR System Client/Server Communications
 AR System Clients
 AR System Database Server
 Applications of AR
2
Introduction
 What is AR?
 Collection of technologies used to blend computer-
generated information with the viewer’s natural senses
 Italso refers to a high-end user interface that involves
real-time simulation and interactions through multiple
sensorial channels.
AR-Virtual
Chairs
and
Lamps

3
Introduction (Cont’d)
 Why AR?
 AR is able to immerse you in a computer-generated world of your own
making:
 a room, a city, the interior of human body.
 With AR, you can explore any uncharted territory of the human
imagination.
 AR uses appearance and behavior to the realism of virtual objects to
make the user feel more immersed in the AR world
 Appearance: Photon mapping for realistic appearance
 Behavior: using a physical law, the motion of a virtual object can be
simulated in a physically correct manner
 Enhances our perception of the surroundings by combining sensing
(Human Vision in Particular), computing and display technologies

4
Introduction (Cont’d)
 Characteristics of AR
 Combines Virtual and Real
 3D- Display of Virtual Object to register and align with Real World objects
 Interactive in Real-time
 Mixed Reality
 The middle of AR and Augmented Virtuality (AV)
 AR-adds virtual contents to the Real Environment
 AV-adds real content to a predominantly virtual environment
 AR techniques exploit the spatial relationships between the user, the digital
information, and the real environment, to enable intuitive and interactive
presentations of data.

5
Introduction (Cont’d)
 3D ultrasound scan is registered with a pregnant woman’s body- Looking into
the Inner part of the body

6
Types of VR
 Windows on World(WoW)
 Also called Desktop VR.
 Using a conventional computer monitor to display the 3D virtual world.
 Immersive VR
 Completely immerse the user's personal viewpoint inside the virtual 3D world.
 The user has no visual contact with the physical word.
 Often equipped with a Head Mounted Display (HMD).
 Vendors of VR (Facebook, google, Samsung, etc…), Please read page #100 of your module for
devices produced by these manufacturers
 Three types of VR devices
 tethered,
 Stand-alone, and
 smartphone VR
7
 Types of VR (Cont’d)
 Augmented Reality (AR)
 overlays computer-generated content on top of the real world.
 can superficially interact with the environment in real-time
 primarily experienced via a wearable glass device or through
smartphone applications.
 It is contents do not recognize or interact with real world objects
but enhance user experiences
 Can be delivered on existing hardware such as smartphone. Hence,
demanded more than VR and Mixed Reality.
 exploits users’ perceptual-motor skills in the real world, creating a
special type of human-machine interaction

8
Types of VR (Cont’d)
 Telepresence
 A variation of visualizing complete computer generated worlds.
 It aims to achieve the illusion of presence at a remote location.
 Links remote sensors in the real world with the senses of a human
operator. The remote sensors might be located on a robot. Useful
for performing operations in dangerous environments.

9
Types of VR (Cont’d)
 Mixed Reality(Augmented Reality)
 The seamless merging of real space and virtual space.
 Integrate the ccomputer-generated virtual objects into the physical world
which become in a sense an equal part of our natural environment.
 lenses or headsets present an overlay of digital content that interacts
with objects in the real world in real-time

Fig. Reality-Virtuality Continuum

10
Types of VR (Cont’d)
 Distributed VR
 A simulated world runs on several computers which are connected over
network and the people are able to interact in real time, sharing the
same virtual world.

11
 Architecture of VR System: AR Components
• device or software • The objects in the real
Tracking System
Scene Generator
Display
responsible for rendering and virtual worlds must
the scene be properly aligned with
respect to each other
• is not currently one of the • Otherwise, the illusion
major problems in AR, that the two worlds
because a few virtual coexist will be
objects need to be drawn compromised.
• Many applications demand
accurate registration,
especially on medical
systems.
• Allows the user to see a
combined view of virtual
imagery and real objects.
• It is under development.
• e.g. HMD (Head Mounted
Display)
• There are still no see-
through displays that have
sufficient brightness,
resolution, field of view,
and contrast (these are
determining factors for
Optical and Video
Technologies) to
seamlessly blend a wide
range of real and virtual
imagery
12
AR System Architecture

13
AR System Architecture (Cont’d)
Client Tier
• Present Information to users Takes information from users

Mid Tier
1. Contains components and add-in services that run on a web
makes the user interface available in browsers
server, enabling users to view applications on the web

Server
implements the workflow functions, access control, and flow of Tier
contains server-side applications (such as Approval Server, Email
data into and out of the database and other data sources in the
Engine, and the Flashboards server)
data tier

Data Tier
Contains database servers and other data sources that can The database server acts as the data storage
14 and retrieval
be accessed by the AR System server engine.
AR System Architecture (Cont’d)
Client Tier

Mid Tier

Server Tier: Multiple Servers (Server

Group) accessing the same database-
Scalability, Reliability, Load Sharing
and backup to each others

Data Tier
15
AR System Architecture: High-level
Architecture

16
AR System Architecture: Distributed
Environment 
Use Distributed Server Option
(DSO) to build large-scale,
distributed environments that
behave like a single virtual
system.
 DSO enables you to share
common information among
servers and to keep that
information consistent
 Layers are independent of one
another combine operating
system platforms to fulfill
different functions
 E.g. An AR System server on
Windows can interact with a
database on UNIX.
17
AR System Architecture: Functional
Components
 Presentation Layer
 Responsible for presenting services
and displaying data to clients through
various interfaces:
 Browsers, Cellphones, PCs, PDA, BMC
Developer Studio, and APIs.
 Business Processing
 manage the data entered through
clients
 perform services based on a business
processes, and
 enforce business rules
 Mid Tier, AR System Server, DSO,
Atrium Integrator, and Web services
 Data Storage
 supports Oracle and Microsoft SQL
18
Server databases for storing data.
AR System Architecture: Client/server
communication
• using remote procedure calls (RPCs) on top of a TCP/IP transport stack
• The type of RPC is the Oracle ONC RPC-(RPC uses minimal Network bandwidth)
Clients • The AR System web server communicates with the browsers using HTTP or HTTPS
and AR
System
Server

• The AR System server is a database client

• AR System server uses the JDBC to connect to the database
1. AR • AR System server supports SQL Server and Oracle databases to communicate with the database servers using type 4 JDBC drivers
System • AR System server uses the JDBC connection pooling to allow multiple database connections in parallel.
Servers • The AR server process comprises many threads to support user requests and share the connection pool.
and • This improves throughput and system scalability
Databa
se
Servers

• AR System server can theoretically support any number of AR System client

connections and Vice-versa
Many-
to- • limited by network bandwidth and server host and database performance
Many • Servers can be any mix of server hosts and underlying database engines
Connec
tions

19
AR System Architecture: AR System
clients
user client and developer clients
Client is composed of a set of native applications (tools) that use the standard user
interface conventions for that environment. Individual users can run these tools as
necessary
User Client Developer Client
Through Mid tier, users submit and The developer clients create, modify,
modify new requests, search for and extend AR System applications
information about requests, and
generate reports

For detail tasks of both user and developer clients, please refer to the module on
pages #110-111

20
AR System Architecture: AR System
Database Server
 AR System uses standard relational database engines for the actual storage
and retrieval of data.
 separate from the AR System server processes
 It can be running on the same server host as the AR System server
 or on a different host
 Can be Any platform that database engine supports
DB1
DB2
Search DB3
Facilities
AR System
DB Server

External 21
Database
AR System Architecture: AR System
Database Server (Cont’d)
 Support for External Database
 can use AR System database connectivity (ARDBC) to work with data not stored in
databases
 Support for multiple database types
 An application created on an AR System server running one type of database
engine can easily be moved to a server running a different database engine
through a simple export/import process
 Support for database searches
 Clients can use the following types of searches
 Query-by-example (QBE)
 Advanced search
 Predefined
 Recent
22
AR Devices
 Four major classes of AR can be distinguished by their display type:
 Optical See Through
 Virtual Retinal Systems
 Video See-Through
 Monitor Based AR and
 Projector Based AR

23
AR Devices: Optical See-through HMD
 uses a transparent Head Mounted Display to show the virtual
environment directly over the real world
 It works by placing optical combiners in front of the user’s eyes
 combiner is to provide an optically direct view of the environment, with a
simultaneous presentation of computer-generated imagery

24
AR Devices: Video See-through HMD
 uses an opaque HMD to display a merged video of the VE and view from
cameras on the HMD
 is based on a camera that acquires the view of the environment,
 a computer that adds virtual content, and
 an ordinary video display that presents the combined view to the user
 two cameras should be used to acquire a stereo view, with one perspective
for each eye, but monoscopic single-camera systems are common and easier
to design and implement

25
AR Devices: Video See-through HMD
(Cont’d)
 Some video see-through displays use a camera to capture the scene, but
present the combined view on a regular, typically handheld, computer
display.
 A window-like effect often referred to as a “magic lens,” is achieved if the camera
is attached on the back of the display, creating the illusion of see-through
 The figure below (NaviCam Project) exhibits annotation overlaid onto the
video captured from the environment.
 Mobile is popularly used in Video See-through implementation

26
 AR Devices: Project Based AR
 uses real-world objects as the projection surface for the virtual environment
 Applied in industrial assembly, product visualization, and the like
 The alignment of projectors and the projection surfaces is critical for
successful applications

examples of how the real world can

be modified through controlled
light that alters its appearance

A child uses a tracked brush to

apply virtual paint, which is
projected onto physical objects

27
Application of AR System

 Medical
 Education
 Entertainment
 Engineering Design
 Manufacturing, Maintenance, and Repair
 Collaborative AR

28
Application of AR System: Medicine
 Practice Performing Surgery.
 Perform surgery on a remote patient.
 Teach new skills in a safe, and controlled environment.
 Ultrasound, CT (Computed Tomography) and/or MRI (Magnetic Resonance
Imaging) scans provide the surgeon with the necessary view of the internal
anatomy

29
Application of AR System: Medicine
(Cont)
A 3D Virtual Reality Model of Unborn
Baby

30
Application of AR System: Education

 Driving Simulators.
 Ship Simulators.
 Flight Simulators.
 Tank Simulators.

31
Application of AR System: Entertainment
a. Games using a virtual
table and synthetic
objects
b. Advertisement in a
Football game
• More vivid
• Move exciting
• More attractive

32
Application of AR System: Engineering
Design
 3D prototyping of Engineering Design
 Design augmented with a
computer-generated image of the current
design built from the CAD files describing it

33
 Application of AR System: Manufacturing,
Maintenance, and Repair

 The image of the equipment would be augmented with annotations and

information pertinent to the repair.
 Makes much easier than consulting a lot of repair manuals
 Boing made an experimental system, where the technicians are guided by
the augmented display that shows the routing of the cables on a generic
frame used for all harnesses

34
Application of AR System: Collaborative AR
 AR addresses two major issues with
collaboration:
 seamless integration with existing tools and
practices, and
 enhancing practice by supporting remote and
co-located activities that would otherwise be
impossible
 E.g. Collaborative AR systems using see-
through displays include both those that use
see-through handheld displays and see-
through head-worn displays.

35
End of AR

36