Simulations in IIoT: Augmented
Reality and Virtual Reality
Contents
• Industry 4.0: Technologies and benefits
• Augmented Reality and Virtual Reality : IIoT
• Augmented Reality (AR)
• AR Types
• Virtual Reality (VR)
• VR Types
• Differences between AR and VR
Industry 4.0: Technologies
Industry 4.0 comprises a set of latest technologies
which are deployed to deliver the complete
transformation of modern factories into smart
manufacturing factories.
1. The Cloud
2. Sensors And Connected Devices
3. Immersive Technologies (AR/VR/MR)
4. Artificial Intelligence
5. Big Data
6. Digital Twin
7. Cybersecurity
8. Additive Manufacturing and Digital Scanning
Image source : Batechnology.it
Augmented Reality and Virtual Reality: IIoT
• Overview
• Popularly known as AR and VR
• Important technologies
• General features
• Types
• Relationship and difference
Industry 4.0: Benefits
• Predictive Maintenance
• Demand Prediction
• Inventory Optimisation
• Productivity
• Expedited And Improved Training
• Improved Robotics
• Costs
Augmented Reality and Virtual Reality in IIoT
• From the technological perspective,
Augmented Reality (AR) and Virtual Reality
(VR) are used in several contexts and sectors in
Industry 4.0.
• AR and VR play an important role in the primary
stages of manufacturing, where optimisation and
productivity are important in the manufacturing
industry (quantity, quality, speed, flexibility)
• To empower the workforce and train operators to
be more efficient in complex production process
• The efficiency of warehouses is improved using
various AR applications
• AR and VR also play an important role in safety
training, thereby the potential safety hazards can
be easily located
Source: gorodenkoff/Getty Images
Use cases where AR/VR augmented with Industrial
IoT is transforming the manufacturing space are:
Real-Time Training Preventing Physical
Designing and
and Employee Damages and Manage Inventory
Prototyping
Instruction Disruptions
• Product appearance • Providing • Potential threats • Organisation of
without physical instructions to alert through inventory involves
prototype service engineers simulation of manual labour
• Ford Immersive during repair and production • Guidance to worker
vehicle environment maintenance environment. through AR
: Virtual car • Employee • Reduction of wearables to exact
environments instruction and downtime and location of
education maintenance particular item
• Real time expenses. • Time saving
instruction • Enhancement of
projected into AR employee security
glasses.
Benefits of AR/VR in Industry 4.0:
No
Better time
downtime/
to market
disruptions
Increased Better
security Quality
Reduced
expenses
Augmented Reality
• An enhanced version of real physical world
• It involves overlaying visual, auditory, or other
sensory information onto the world in order to
enhance one's experience.
• Direct/indirect views of physical world
environments are "augmented" with computer-
generated superimposed images
• Primary goals
• To highlight specific features of the physical
world
• To increase understanding of those features
• Examples: AR Furniture’s in shopping apps. ,
Google glass , Gaming app (Pokemon)
Source: PTC.com
Augmented Reality in Industry 4.0
• To support technicians in their
real working environment.
• Step-by-step procedures of the
task to be performed
• Visual instructions in real time
from experts with remote
assistance systems.
Source: Innovae.eu
Augmented Reality (AR) in manufacturing
• Augmented reality combines real and virtual worlds generated by
the computer
• It starts with capturing a real image that gets infused or
augmented with different layers of digital information
• The technology has proven useful in the manufacturing process in
terms of measuring changes, visualizing the end product, and
seeking out dangerous working conditions
• Assisted Reality Wearable: Like Google Glass, this displays an
image of computer screen typically to one eye, providing on-the-
spot data.
• Phone/Tablet: When you hold up your device to view a piece of
equipment, a digital overlay can provide additional data regarding
that equipment, KPI’s, Graphical Data, Schematics, and Digital
twin data etc.
• Example:
• Worker’s current environment in manufacturing industry.
• A workpiece real-time temperature details through a smart
wearable device, thus avoiding the mishaps.
• Broken machine details, Restricted area of factory
Source: Plugxr
Key Features of AR
The key features of AR are:
• It lies in the middle of the mixed reality spectrum.
• It provides multiple sensor modalities - visual, auditory, and haptic.
• It utilizes the existing environment and overlays new information on
top of it.
Image Source: Lovreglio, R., 2018, A Review of Augmented Reality Applications for Building Evacuation, 17th International
Conference on Computing in Civil and Building Engineering, ICCCBE 2018, Tampere, Finland
AR: First head mounted display (1968)
Source: Cable Vision
Development timeline of AR 1968 – 2020. Image: AppVenturez
AR Applications
Tourism
Entertain
Education
-ment
AR
Military Retail
Applications
Flight Industrial
training design
Medical
Types of Augmented Reality
• Marker based AR
• Markerless AR
• Projection based AR
• Superimposition
based AR
Source: INEXTRIX
Types of Augmented Reality (contd.)
• Marker-based AR
• It use target images (markers) to position objects in a given space.
• It gives an outcome when the reader is sensed by the camera and visual marker.
• It is tied to a specific physical image pattern marker in a real-world environment to
superimpose the virtual 3D object, text, or animation on top of it.
• Camera: differentiates between a marker and a real object.
• Marker: recognizes simple, distinct patterns and can be easily processed.
• Markerless AR
• It is commonly utilised for mapping directions. The location is provided based on
the GPS, digital compass, or accelerometer, which is attached to the device.
• It refers to the software application that does not require any camera to put a mark
for image pattern recognition.
• it works with the digital data obtained by the sensors capable of recording a
physical space in real time.
• It merges the digital data with input from real-time data that are registered to a
physical space.
Source: Argentics Source: Program-ace
Marker based AR
Source: Program-ace
A Silicon Valley startup is enabling sales through markerless AR. Their application lets customers
preview a wide variety of products inside their homes, particularly on common surfaces like floors and
desks. They make the most of advanced phone sensors to display the products in lifelike scale and
position.
Types of Augmented Reality (contd.)
• Projection-based augmented reality
• It gives an outcome by projecting light onto real world surfaces.
• Projected AR uses a combination of projectors and vision sensors to display
step by step interactive graphics onto any work surface.
• It is a method of delivering digital data within a stationary context as
projection-based AR focuses on rendering virtual 3D objects within or on a
user’s physical space.
• Projection-based AR can be applied for proper guidance to simplify the
complex manual tasks in companies or industries.
• It eliminates the need for computers and screens as the instructions can be
placed on a particular task space.
• It allows human interaction by sending light.
Types of Augmented Reality (contd.)
Superimposition-based AR
• It is used either for partial or full replacement of the original view of
an object with an updated augmented view of that object for the
human eye.
• Superimposition AR provides multiple views of a target object with
the option of showing extra relevant information on that object.
• It partially or fully substitutes the original view of the object with the
augmented view
• Object recognition plays an important role
• The application cannot replace the original view with the augmented one.
Superposition based AR in History field Trip Projection based AR in a Geography class
Virtual Reality (VR)
• Virtual Reality is
• a mixture of interactive hardware and software
based artificial environment
• a realistic three-dimensional image is created
• presented to the user in such a way so that they
interact with the real or physical world through
specially designed equipment's.
• Virtual reality is the term used to describe a three-
dimensional, computer generated environment
which can be explored and interacted with by a
person.
• That person becomes part of this virtual world or is
immersed within this environment and whilst
there, is able to manipulate objects or perform a
series of actions.
Key Features of VR
The key features of VR are:
• Virtual world
• It creates and enhances an imaginary reality .
• Interactivity
• It gives the perception of being physically present in a non-
physical world .
• Sensory feedback
• It incorporates auditory and feedback.
• Feedback on senses like the taste, smell, and forces to provide
an immersive experience.
• Immersion
• It allows users to get naturally absorbed into the virtual
environment.
• Making the experience closer to reality using immersive
graphical content and sound.
Source: Plugxr
Types of VR
• Non immersive simulations
• Semi immersive simulations
• Fully immersive simulations
Source: Popsci.com
Types of VR
Non-immersive simulations
• It utilise only a subset of the
user's senses.
• The user enters into the virtual
environment through a portal or
window
• Users allows a peripheral
awareness of the reality outside
the virtual reality simulations.
• Example: Video game console
with input devices like
keyboard/joystick
Source: Pixabay
Types of VR
Semi immersive
• It provides an experience wherein a user can
be a part of the virtual environment and
partially connected with the physical world.
• The simulations are:
• Powered by high performance graphical computing
system, and
• Coupled with a large screen projector
• Example of semi-immersive virtual reality is a
flight simulator.
• It often consists of a large, concave screen,
projection system and monitor and is similar to the
large screen experiences seen at IMAX cinemas.
• They also involve high end computer graphics.
• The viewer becomes partly but not fully immersed
in this environment.
Types of VR
Fully immersive simulations
• It provide most realistic experience to the users.
The simulations
• Deliver a wide field view and
• Use head mounted displays and motion detecting
devices to simulate users experiences.
• Immersive videos & sounds
• To experience and interact with fully immersive VR,
the user needs VR equipment like VR glasses or
head mount displays (HMD).
• The technology is highly used in the gaming &
entertainment industry.
Source: Cyprus Mail
Virtual Reality in Industry 4.0
• The application of virtual reality in
industry 4.0 has a different
orientation due to the level of
immersion it provides to the user
• Exact simulations of products,
processes or production plants can
be built in order to see their
operation in first person and in an
immersive way
• Design phase of products /
processes and validation of
prototypes : visual and interactive
way
• Errors in this phase can be reduced
and productivity increased
Augmented and Virtual Reality: the difference
Augmented Reality Virtual Reality
• It overlays and positions digital • Immersive application, which
elements/virtual things to the affects the experience of user.
actual environment. • It offers a digital recreation of a
• It delivers virtual elements as real life setting.
encrust of the real world. • Stimulates the senses to provide
• Microsoft HoloLens : It overlays the feeling of presence in virtual
apps, images, and information on experience.
the walls or surroundings of your
home, in the same way in which
apps are laid out on our
smartphones.
The underlying aim of the two technologies is the same - altering a user’s
perspective for an enhanced experience.
Source: Nanotechsys
Source: Intel
References
• Introduction to Industry 4.0 and Industrial Internet of Things, IIT Kharagpur
[NPTEL Course by Prof. Sudip Mishra]
• https://www.mouser.in/blog/industry-40-supported-by-machine-learning
• https://www.pratititech.com/bridging-the-digital-and-physical-worlds-using-ar-
vr-and-industrial-iot/
• https://www.fierceelectronics.com/components/iiot-and-industry-4-0-basics-
you-need-to-know
• https://www.innovae.eu/en/industry-40-augmented-and-virtual-reality/
• Lovreglio, R., 2018, A Review of Augmented Reality Applications for Building
Evacuation, 17th Internationa lConference on Computing in Civil and Building
Engineering, ICCCBE 2018, Tampere, Finland
Thank You
