GPU, VR & AR

CSE101: Introduction to Computer Science

01/13
 Contents
• What is GPU? • What is VR? • What is AR?
• Structure: CPU vs. GPU • How do we perceive • Augmented reality vs.
reality?
• A Glimpse at a GPU virtual reality
• Reality vs. virtual reality
• Evolution of GPU • Use of technologies to • How does AR work?
simulate senses • Architecture of AR
• GPU Graphics Trends
• The history of VR system
• GPU Pipeline • Types of VR • Applications of AR
• Applications of GPU • Architecture of VR
system
• Applications of VR
01/13
2
What is GPU?
What is Graphic Processing Unit (GPU)?
• It is a processor optimized for 2D/3D graphics, video, visual
computing, and display.
• It is highly parallel, highly multithreaded multiprocessor
optimized for visual computing.
• It provides real-time visual interaction with computed
objects via graphics images, and video.
• It serves as both a programmable graphics processor and a
scalable parallel computing platform.
• Heterogeneous Systems: combine a GPU with a CPU
What is CUDA?
• The CUDA stands for Compute Unified Device
Architecture. It is a C (language) extension for
programming for NVIDIA GPUs
• Straightforward to learn
• Challenge is in getting performance 01/13
3
Structure: CPU vs. GPU

01/13
4
A glimpse at a GPGPU: GetForce 8800

01/13
5
Evolution of GPU
Evolution of GPU:
1980’s – No GPU. PC used VGA controller. VGA - video graphics array
1990’s – Add more function into VGA controller
1997 – 3D acceleration functions: Hardware for triangle setup and rasterization.
1) Texture mapping & 2) Shading
2000 – A single chip graphics processor ( beginning of GPU term)
2005 – Massively parallel programmable processors
2007 – CUDA (Compute Unified Device Architecture)
01/13
6
GPU Graphic Trends
o OpenGL – an open standard for 3D programming
o DirectX – a series of Microsoft multimedia programming interfaces
o New GPU are being developed every 12 to 18 months
o New idea of visual computing: combines graphics processing and parallel
computing
o Heterogeneous System – CPU + GPU
o GPU evolves into scalable parallel processor
o GPU Computing: GPGPU and CUDA
o GPU unifies graphics and computing
o GPU visual computing application: OpenGL, and DirectX
01/13
7
GPU Applications
• Bioinformatics: Sequencing and protein docking are very calculating intensive tasks that see a
larger performance benefit by using CUDA enabled GPU. There is a relatively bit of ongoing
work on GPUs for a range of bioinformatics and life sciences codes.
• Computational Finance: NVIDIA TESLA GPU acceleration offers financial service firms a
competitive advantage by enabling applications. Calculating pricing and risk for compound
options and OTC derivatives within seconds versus hours allows you to run more simulations
thereby increasing the quality of results and more confidence in data.
• Computational Fluid Dynamics: Several ongoing projects on Navier-Stokes models and Lattice
methods have shown very large speedups using CUDA-enabled GPUs.
• Data Science, Analytics, and Databases: A increasing number of customers are using GPUs for
big data analytics to make better, real –time business decisions.
• Defense and Intelligence: The defense and intelligence community heavily rely on accurate
and timely information in its strategic and day to day operations. Intelligence gathering and
assessment are essential parts of these activities that comprise data coming from the number
01/13and radar.
of move away sources such as satellites, surveillance cameras, UAVs,
8
What is VR? VR Experience: https://www.youtube.com/watch?v=qYfNzhLXYGc
VR Tool: https://www.youtube.com/watch?v=jnqFdSa5p7w
• Virtual Reality (VR) means feeling the
imaginary(virtual) world, rather than
the real one.
• The imaginary world is a simulation
running in a computer. Virtual reality is
the term used for computer generated
3D environments that allow the user to
enter and interact with alternate
realities.
• The definition of ‘virtual’ is near and
‘reality’ is what we experience as
human beings.
• Virtual Reality refers to
a high-end user
interface that involves real-time
simulation and interactions through
multiple sensorial channels. 01/13
9
How do we perceive reality?
We understand the world through
our senses:
• Sight, Hearing, Touch, Taste, Smell
(and others..)

Two basic processes:

• Sensation – Gathering information ✔ Sight/visualising
• Perception – Interpreting information ✔ Hearing
✔ Taste
✔ Smell
01/13 ✔ Sense/Touch
10
Reality vs. virtual reality

✔ In a VR system, there are input and output devices between

human perception and action 01/13
11
Use of technologies to simulate senses
Creating an Immersive
Experience:
✔ Head Mounted Display
✔ Immerse the eyes
✔ Projection/Large Screen
✔ Immerse the head/body

Future Technologies:
✔ Neural implants
✔ Contact lens displays, etc

01/13
12
 HMD Display Principles
VR Stereo images
Principle of Stereo image generation and
illusion on brain
VR Stereo images

Left-eye image Right-eye image

Fig. Principle of display with optics to create illusion of virtual screen 01/13
13
History of virtual reality
✔ In mid 1950s, a visionary cinematographer, Morton
Heilig developed the Sensorama (patented 1962)
which was an arcade-style theatre cabinet that would
stimulate all the senses, not just sight and sound.
✔ It featured stereo speakers, a stereoscopic 3D display,
fans, smell generators and a vibrating chair. The
Sensorama was intended to fully immerse the
individual in the film.
✔ He also created six short films for his invention all of
which he shot, produced and edited himself.
✔ The Sensorama films were titled, Motorcycle, Belly
Dancer, Dune Buggy, helicopter, A date with Sabina
and I’m a coca cola bottle!.
01/13
14
Types of virtual reality
Immersive Virtual Reality
Non-immersive Virtual Reality
Window on world Virtual Reality
Immersive Virtual Reality:
Immersion into virtual reality is a
perception of being physically
present in a non-physical world.
Elements of virtual environments
that increase the immersiveness
of the experience:
✔ Continuity of surroundings
✔ Conformance to human vision
✔ Freedom of movement
✔ Physical interaction
✔ Physical feedback
01/13
15
Types of virtual reality (cont.)
Non-Immersive Virtual Reality:
Large display, but doesn’t
surround the user.

01/13
16
Types of virtual reality (cont.)
Window on world Virtual Reality:
Desktop- based Virtual Reality involves displaying
a 3-dimensional virtual on regular desktop display
without use of any specialized movement tracking
environment.

01/13
17
Architecture of a VR System

Components of a VR System:
o Input Processor
o Simulation Processor
o Rendering Processor
o World Database
Fig. Architecture of a VR System
01/13
18
Components of VR System
Input Processor: Input Devices

o Control the devices used to input

information to the computer.
o The object is to get the
coordinate data to the rest of the Prio VR Razer Hydra
system with minimal lag time.
o Keyboard, mouse, 3D position
trackers, a voice recognition
system, etc. 01/13
Stem Control VR
19
Components of VR System (cont.)
Output devices
Simulation Processor :
3D Audio
o Core of a VR system.
o Takes the user inputs along with
any tasks programmed into the VR headset
world and determine the
Helsinki
actions that will take place in
the virtual world.

01/13
Sensing Gloves
20
Components of VR System (cont.)
Rendering Processor :
o Create the sensations that are output to the user.
o Separate rendering processes are used for visual, auditory,
haptic and other sensory systems.
o Each renderer takes a description of the world starts from the
simulation process or derive it directly from the World
Database for each time step.

01/13
21
Components of VR System (cont.)
World Database (World Description Files):
o Store the objects that inhabit the world, scripts that describe
actions of those objects.

01/13
22
Applications of VR
VR in Military VR in Healthcare

01/13
23
Applications of VR (cont.)
VR in Education VR in Scientific Data Visualization

01/13
24
Applications of VR (cont.)
VR in Environment

01/13
25
Advantages and Disadvantages of VR
Advantages Disadvantages
o Virtual reality creates a realistic world. o The equipment used in virtual
reality are very expensive.
o It enables user to explore places.
o It consists of complex technology.
o Through Virtual Reality user can experiment
o In virtual reality environment, we
with an artificial environment.
can’t move by our own like in the
o Virtual Reality makes the education more real world.
easy and comfortable.

01/13
26
What is Augmented Reality(AR)?
https://www.youtube.com/watch?v=T5O3vijPGUw https://www.youtube.com/watch?v=d3YT8j0yYl0

❑ Augmented Reality:
o A combination of a real scene
viewed by a user and a virtual
scene generated by a computer
that augments the scene with
additional information.
o An AR system adds virtual
computer-generated objects,
audio and other sense
enhancements to a real-world
environment in real time.
01/13
27
Augmented Reality vs. Virtual reality?
Comparison of AR & VR:
• Augmented reality system:
✔ Augments the real world scene with computer generated objects
✔ User maintains a sense of presence in real world
✔ Needs a mechanism to combine virtual and real worlds

• Virtual reality system:

✔ Totally immersive environment
✔ Visual senses are under control of system

01/13
28
 How does AR work?

✔ Pick a real world scene.

✔ Add your virtual objects in it.

✔ Delete real world objects if

needed.

✔ Not virtual reality since

environment real.

01/13
29
Displays used in AR
❑ Head-mounted Display(HMD)

✔ Device paired to a headset such as a

harness or helmet

❑ Eye Glasses

✔ Eye wear that employs cameras to

intercept the real world view and
re-display it's augmented view through
the eye pieces
01/13
30
Architecture of an AR System

❑ Video see-through HMD based AR

System
✔ Capturing of real world scene
✔ Head tracking
✔ Generation of graphical/artificial
objects
✔ Combining the real world scene and
computer generated objects/ scene
✔ Displaying the combined video through
a head mounted display

01/13
31
Applications of AR
✔ Medical
✔ Entertainment
✔ Military Training Engineering
✔ Design
✔ Robotics and Telerobotics
✔ Manufacturing, Maintenance, and Repair
✔ Consumer Design Hazard Detection Audio

01/13
32
Medical Applications of AR

01/13
33
AR in Entertainment

01/13
34
AR in Defence

01/13
35
AR in Educations

01/13
36
Hardware and Specifications

01/13
Choosing Computer Components
Components Options

Processor Intel 9th Gen Core i5 AMD Ryzen 5 3600

Motherboard Gigabyte H110M MSI B450M

Memory/RAM DDR4 16GB 3200MHz 4GB DDR3 1333MHz

Storage Samsung SSD Western Digital HDD

01/13
38
Choosing Mobile Components
Components Options

Operating
iOS Android
System

Memory/RAM 2GB 8GB

Storage 4GB 64GB

Camera 12 MP Telephoto Lens 48 MP Wide Angle Lens

01/13
39
HDD & SSD
HDD:
An electro-mechanical data storage device that
uses one or more rapidly rotating platters
coated with magnetic material to store and
retrieve digital data.

SSD:
A storage device that uses integrated circuit
assemblies or instantly-accessible memory
chips to store data.

01/13
40
SSD vs HDD
SSD HDD

Full-Form Solid State Drive Hard Disk Drive

Structure Uses integrated circuit assemblies
to store and retrieve data.
Operation Much Faster than HDD
Speed
Durability SSDs don’t have moving parts like
spinning disks and so they tend to
be more durable than HDDs.
Noise SSDs make very little noise while
performing operations.
Cost More expensive compared to HDD. Less expensive compared to SSD.
Uses rapidly rotating platters to store
and retrieve data.
Slower than SSD
HDDs contain moving parts and are
less durable compared to SSDs.
The spinning disks in HDDs can make
a lot of noise while performing
operations.
01/13
41
 Smartwatch
A smartwatch is a wearable computer in
the form of a watch.

Features might include:

✔ Accelerometers ✔ Thermometers
✔ Pedometers ✔ Digital Cameras
✔ Heart rate monitors ✔ Speakers
✔ Compasses ✔ MicroSD card slot
✔ GPS receivers And so on…

01/13
42
Laptop-Tablet Convertible
A portable computer that can function like both a laptop and a tablet.

01/13
43
Thanks

01/13
44