Project Report

DS3093 AI in Design

Creating a digital twin of a classroom and deploying it on a

cloud based AVR platform along with literature study.

Department of Design

IIT Hyderabad

Narumugai T BD19BDES11004
Shreyas KB BD19BDES11005
Mandira VR BD19BDES11006
Ritik BD19BDES11007
Swapnish S BD19BDES11008
 1

Contents:

1. Acknowledgement

2. Abstract

3. Introduction

Team contribution
Technical difficulties
Project management
Reflection

4. Literature Study

Research Methodology
Literature Search
Literature Analysis
Content Findings
Use cases

5. Project

3D modelling
Exporting assets to Unreal engine
Adding textures and preparing environment
Create interactions
Deploying model into cloud based AVR platform ( EON XR)

6. Conclusion

7. Bibliography

1) Acknowledgements
 2

Foremost we would like to express our sincere gratitude to our professor Srikar A V
R for supervising and guiding us through this course. We also thank our HOD
Deepak John Mathew and the rest of the Department of Design for providing us this
opportunity to take up this project.

Our sincere thanks also goes to Ambresh Kulkarni sir, John Dcoutho sir and Nikunj
Khetani sir for taking out their time to collaborate with us and teach us so many
topics in such depth in a very short time period. We’re grateful for the
encouragement, insightful comments and knowledge shared with us.

We also wanna thank our classmate Aryaman for helping and accompanying us
through the whole process.

2) Abstract
 3

The purpose of design is to devise a specification for how the world is to be

changed. Artificial Intelligence (AI) is the science concerned with understanding
intelligent behaviour and how it can be artificially created. The adoption of artificial
intelligence (AI) has received enormous attention and emphasis across virtually
every industrial setting, from healthcare delivery to automobile manufacturing.

Advances in digital technologies such as digital twin(DT) have seen enormous

utilisation in digitally advanced industries including the manufacturing and
automotive industries. It presents an opportunity for the integration of the
physical world to the digital world and helps add digital information to the world
that you can interact with in the same manner that you interact with the physical
world.

In this project we will be modeling our classroom and it’s components using 3D
software and prepare an interactive AVR model (software and hardware workflow)

We will also be producing a report based on our findings and gaps in research
related to virtual concept development, digital twins and related subjects.

Keywords: AI, AVR, Digital twin, Concept development Technology,

3) Introduction
 4
Through this project it is aimed to give an overview of AI applications in design by
covering the skills and knowledge required to holistically optimise the information
flows using Industry 4.0 tools. We will be learning to research and use various
technologies which include modelling software, augmented reality/virtual reality
software.

We also extensively read and analysed articles/papers pertaining to AVR concept

development and digital twin technologies and how it changes the world.

3.1) Team contributions

Literature review:
Narumugai helped outline the literature review and search, areas of study and
format it.
Ritik read and analysed virtual concept development through AVR and current
industry practices.
Shreyas and Mandira delved deeper into the digital twins and its use cases, key
findings and realising gaps in them and further possible research areas related to the
legal and cultural aspects of creating digital twins of buildings and objects that are
of cultural value.
Swapnish also contributed by further adding to our literature review.

3Ds Max:
Shreyas and Ritik initially created a replica of the basic structure of the design
rooms- Ablock 618. These include roofs, walls, beams, windows and doors.
Later as a team we hunted for free assets similar to the real ones and built a few not
so accurate to the original ones on 3Ds Max.
The .3ds Max file was shared and assets were modified, scaled and placed in the
right places by the members and monitored by Ritik.

Unreal engine:
 5
Narumugai helped export the model and assets through datasmith and worked with
layer/asset migration into VR settings.
The team members sat together and rewatched the classroom recordings and tried
to replicate the actions. We also scouted for videos explaining sky settings, motion
control pawn, physics, etc in order to obtain better understanding.
Ritik added textures, lighting and the environment for the room giving it life.
Team members also played around and tuned the interactions, led by Ritik.

3.2) Technical difficulties

The vastness of research on this area was overwhelming and tiring to go through.
It took the team a week to do the research in order to finally zero down on any gap in
the field of research. The Zotero plugin had a few issues as well.
3Ds Max offered the least complications as it was easy to adapt to. Optimisation and
asset management were some difficulties faced.
Unreal engine was a completely new realm for us, it took a considerable amount of
time to get used to its interface and navigation. A lot of doubts and errors arose
starting with installation of datasmith in the right drive and folder.
We had initially imported our model into a blank game file and found it very
confusing to migrate it into a VR environment. Implementing physics was very time
consuming and was not resolved. We later used the file that was sent to us and
replaced our model in the place of the pre-existing model. This resolved most of the
issues.
Some textures made the shaders skyrocket causing the laptop to crash sometimes.
The major set back was when we realised the scaling of the room was multiple
times bigger than its actual size leading to a very low perspective in VR mode and
every asset had to be scaled down and moved individually in order for the VR to
function properly.

3.3) Project management

All team members were involved throughout the process and stayed connected
 6
everyday.
In terms of the literature report and the project report preparation online modes
were undertaken since the division of labour was very smooth. We worked on
google docs simultaneously while sitting on our discord server.
All of us took up responsibility for parts of this project to ensure a smooth process
and to collect contributions from all members:
Narumugai - Project report
Mandira - Literature study
Shreyas - 3D modeling
Ritik - Unreal Engine and VR implementation

The modeling and setting up on the Unreal engine part was taken one person at a
time due to limitations in the number of powerful systems. While some worked on
these systems others assisted with the troubleshooting, etc with the help of tutorials.

3.4) Reflection
This was an interesting project to take up in terms of learnings, challenges faced and
project process. Division of labour and responsibility really turned out to work in our
favour. Time management is one thing we wish could have been handled and
distributed better. Some of us even plan to take this further by using the softwares
and topics learnt in this course for the coming semester’s project.

4) Literature Study

4.1) Research Methodology

The overarching aim of this study was to analyse the current state of applications in
the AI industry and how it can be merged with other technology like AVR, giving rise
to new experiences and purposes.This study comprehensively reviews and analyses
DT concept, AVR technologies, and their deployment on mobile applications.

After a complete search of several databases and careful selection in line with
 7
the proposed criteria, 10 papers on the integration of these technologies were
identified and analysed.

The research analysed in detail the status, evolution of the concept, key
technologies, AI in design flows, AVR for concept development and potential
industry impact, building information modeling. This research shows that there is
a high potential for DT, AVR to enable solutions to numerous challenges.

4.2) Literature search

Title/abstract/keyword” field of Scopus with the search string consisting of two

parts. The first part consisted of keywords like “product design”, “technology” “day to
day life” “IOT” and the 2nd part consisted of keywords like “ AI”, “AVR”, “MR”, “Digital
twin”.
A systematic review, as an essential feature of research, was implemented in
this study. This presents a summary of previous studies in this field of study
and also identifies knowledge gaps in the published papers.
Scopus was used for the initial search for literature since it has a broader
coverage of scientific publications. Furthermore, the Scopus search engine is
recognised to perform better than other search engines like Web of Science,
PubMed Google Scholar Meho and Rogers (2008) stated that the Scopus
database includes conference papers, has a faster process of indexing and
availability of more recent publications compared to other databases.

Table 1: some papers/articles read:
Title
Literature Review on
Artificial Intelligence
AI-aided design of novel
targeted covalent inhibitors
against SARS-CoV-2 |
bioRxiv
AI-based methodology of
integrating affective design,
engineering, and marketing
An AI-Based Methodology
for Factory Design
Applying model-base
situational awareness and
augmented reality to
next-generation physical
security systems
AR-based product design in
automobile industry
8
Abstract URL/citation
Research on artificial | Elsevier Enhanced Reader
intelligence in the last two (no date).
decades has greatly
improved perfor-mance of doi:10.1016/j.engappai.2015.
both manufacturing and 04.001.
service systems....
The focused drug repurposing of https://www.biorxiv.org/con
known approved drugs (such as tent/10.1101/2020.
lopinavir/ritonavir) has been
reported failed for curing
SARS-CoV-2...
In the early stage of product https://reader.elsevier.com/r
design, particularly for eader/sd/pii/S09521976150
consumer products, 00822?token=92BBB61F3A1
affective design, 0745A29ACCD93DA5443E4D
engineering,and marketing AC29A6234598DCD3CF67B
issues must be taken into 3D7BDE345640BA912B7292
consideration...
This article provides a https://ojs.aaai.org/index.ph
discussion of factory design p/aimagazine/article/view/
and an artificial intelligence 560
(AI) approach to this
problem. Major issues
covered include knowledge
acquisition and
representation, design
methodology, system
architecture, and
communication..
Mixed, augmented, and Scopus
virtual reality holds promise
for many security related (Raybourn and Trechter,
applications including 2018)
physical security systems.
When combined with
models of a site, an
augmented reality (AR)
approach can be designed
to enhance knowledge....
This paper describes Scopus
different scenarios for die
use of Augmented reality in (Fruend, Matysczok and
the early design phase for Radkowski, 2002)

Digital twin application in the
construction industry: A
literature review
Enhancing Immersiveness in
AR-Based Product Design
4.3) Literature Review
Table 2 : list of Papers/articles in the area of interest read with their key findings.
Authors
Alonzo C. Addison ,
University of California at
Berkeley, Berkeley, CA,
United States
Teller, M
University Côte d'Azur,
Nice, France
9
new cars. The prototype
uses different tracking
systems like AR-ToolKit,
Polhcmus Fastrack...
The construction industry is Scopus
faced with numerous
challenges including low (Opoku et al., 2021)
productivity, lack of research
and development, and poor
technology advancements...
Recently, various AR-based Scopus
product design
methodologies have been (Ha et al., 2006)
introduced. In this paper, we
propose technologies for
enhancing the immersive...
Description Key Findings
Virtual Heritage - Use of technology to
Technology in the Service preserve and protect
of Culture world’s cultural legacy
Legal aspects related to Digital replicas of
digital twin people in the medical
field raises issues
related to privacy.
Looks at applying legal
aspects to digital twins
of individuals.

Adriana Marra 1 , Combining integrated
Salvatore Gerbino 2 , informative system and
Alessandro Greco 2 and historical digital twin for
Giovanni Fabbrocino 1 maintenance and
preservation of artistic
assets
10
Advantages from
integrating traditional
and novel procedures
in the conservation of
artistic assets.

JEREMY HUGGETT, Virtually Real or Really Argues that there is a

University of Glasgow, UK Virtual: Towards a Heritage
Metaverse?
disjunction between the
present reality of virtual
heritage and virtual
reality and discusses
the twin challenges of
presence and realism
within virtual reality.

4.4) Content Findings

Virtually Real or Really Virtual: Towards a Heritage Metaverse?

Six major aims for virtual heritage:

“First, it should aim to carefully capture objects and processes of scientific, social or
spiritual value. Second, it should present this information as accurately, authentically,
and engagingly as possible. Third, it should attempt to distribute the project in a
sensitive, safe and durable manner to as wide and long-term an audience as
possible. Fourth, it should aim to provide an effective and inspirational learning
environment that best communicates the intended pedagogical aims. Fifth, it should
allow the possibility of participating in its construction. Finally, it should attempt to
carefully evaluate its effectiveness with regard to the above five aims in order to
improve both the project and virtual heritage in general.” [Pujol and Champion 2012:
86]
 11

Virtual Realism
The value and utility of a virtual model is commonly linked to its realism and
accuracy/authenticity – indeed, this realism is seen as a key factor in successful
immersion in a virtual world.
Realism in conjunction with authority/authenticity is seen as necessary for a
successful model, yet simultaneously the same realism may actually undermine its
authority.

Virtual heritage reconstructions instead often seem to be primarily an alternative

form of archaeological illustration and presentation rather than something that is or
should be capable of driving forward archaeological inquiry which the existence of a
metaverse or new class of information system might imply.

In a typically perceptive paper on VR presented in 1997, Gillings asked what are

models for?
“… we often find that far from embracing the Virtual-models as a new means of
exploring ideas, we have exhausted ourselves in addressing and maintaining this
cycle of suspicion. As a result the models are left as little more than ingenious
images to be viewed and consumed ... They become a closed, end-product,
something passive to be gazed at, whether from without, within, a fixed viewing
point or a flexible one. In effect, they become the passive images rather than objects
of analysis.” [Gillings 1999: 253].

4.5) Use Cases

Analyse AVR use cases for concept development and potential industry impact
Medical- telemedicine, Real wear during hands off meetings, etc.
Education - learning applications using AVR, etc.
Training - TeamViewer pilot, connecting industrial headsets, etc.
Security - Librestream Onsight software, thermal imaging, etc.
Energy - AR glasses for frontline workers, etc.
 12
Motor vehicle industry- Hyundai Captivating showroom, Audi E-tron VR
experience, etc.

Analyse AI use cases and impact on design workflows

Repetitive processes

High-volume processes

Rule-based processes

Personal assistants, Market research, Sales, Digital marketing, Customer services,

Accounting, Law, Travel industry, Transportation, Health, Education, Entertainment.

AI workflow:

Data Preparation: Data cleansing, Human Insight, Data from simulation

AI Modeling: Model design,tuning, Hardware acc. Training, Interoperability

Simulation and Test Integration: with complex systems, System simulation, System
verification

Deployment Embedded devices, Enterprise systems, Edge, cloud, desktop

Potential of AI in Design

AI stimulates creativity by taking care of some usual work that saves time and efforts
for the designer to spend on idea building.

AI identifies user preferences by using algorithm to identify designs and platforms

that users like spending time on

AI creates multiple variants after recognizing a pattern. The algorithm extracts

colors and patterns of a design and then creates thousands of variants, within the
range of identified colors and patterns

AI creates a humanized experience, identifies and interprets the way people engage
with designs and digital products. It helps create a personality for products.
 13
AI brings a personalized user experience which is the key to customer satisfaction.
Platforms that make users feel loved and cared for attract and retain users.

AI helps non-professionals with AI-driven designing platforms, non-professionals

can easily design, without any hassle crafting solutions for users, driven from the
data.

5) Project

Objective:

Digital twin of a room and deploying it using cloud-based AVR platforms.

Time:

4 weeks

Deliverables:

Digital twin of a room arranged in 3ds max

Rendering and VR development in Unreal Engine 4.0.

Deployed into EON XR

We started with scouting for a place to replicate and finally zeroed down to one of
the design classrooms (A block 618) due to its compactness and ability to makeshift
according to the occasion or the assignment in progress.
 14

We measured every edge of the room with help of the Apple Measure app in order
to come up with the floorplan and layout.

5.1) 3D Modelling
We constructed the basic layout before creating/importing other assets.
 15

5.2 Exporting assets to Unreal engine

Datasmith plugin was installed and connected to 3Ds Max and the model was exported
in Unreal engine, where it was scaled and set up.

5.3 Adding textures and preparing environment

Twinmotion was installed from epic games for texture assets. Textures were then
applied to tables, chairs, walls, floors, etc
 16

5.4 Create interactions

In play mode you can pick, place and customise assets by enabling collision mesh,
physics and gravity. Used grab component and chose drop settings.

5.6 Deploying model into cloud based AVR platform ( EON XR)

Sign into EON XR and import the .fbx file.

 17
8) Conclusion:

In the assignment related to recreating our design lab, we gained skills on using
tools related to 3D modeling and AR/VR. While undertaking this we also learnt the
importance of digital twins and ways in which it can be implemented in numerous
fields like medical training, designing smart cities, optimization of manufacturing
lines and more. Reading up on these different applications of digital twins helped us
realise the extent to which this technology can be utilized.

As for the part of producing a report based on our findings and gaps in research
from existing literature, this assignment introduced us on how to do research. We
had to go through a lot of material which helped us learn more about the different
ways in which digital twins have been applied in various sectors and industries. We
also came up with gaps in research we went through online and came up with
possible research areas related to the legal and cultural aspects of creating digital
twins of buildings and objects that are of cultural value.
 18
9) Bibliography:

M. LI, Y. MA, Z. YIN and C. WANG, "Structural Design of Digital Twin Laboratory Model
Based on Instruments Sharing Platform," 2020 Chinese Control And Decision
Conference (CCDC), 2020, pp. 797-802, doi: 10.1109/CCDC49329.2020.9164813.

Opoku, D.-G., Perera, S., Robert, O.-K., Rashidi, M., 2021. Digital twin application in the
construction industry: A literature review. Journal of Building Engineering 40, 102726.
https://doi.org/10.1016/j.jobe.2021.102726

M. LI, Y. MA, Z. YIN and C. WANG, "Structural Design of Digital Twin Laboratory Model
Based on Instruments Sharing Platform," 2020 Chinese Control And Decision
Conference (CCDC), 2020, pp. 797-802, doi: 10.1109/CCDC49329.2020.9164813.

Developing Real-Time Digital Twins for Cloud Deployment [WWW Document], 2020.
. ScaleOut Software. URL
https://www.scaleoutsoftware.com/featured/developing-real-time-digital-twins-for
-cloud-deployment/ (accessed 9.2.21).

Bolton, R.N., McColl-Kennedy, J.R., Cheung, L., Gallan, A., Orsingher, C., Witell, L., Zaki,
M., 2018. Customer experience challenges: bringing together digital, physical and
social realms. Journal of Service Management 29, 776–808.
https://doi.org/10.1108/JOSM-04-2018-0113

Digital twins [WWW Document], n.d. . Deloitte Insights. URL

https://www2.deloitte.com/us/en/insights/focus/tech-trends/2020/digital-twin-a
pplications-bridging-the-physical-and-digital.html (accessed 9.2.21).

Qi, Q., Tao, F., Hu, T., Anwer, N., Liu, A., Wei, Y., Wang, L., Nee, A.Y.C., 2021. Enabling
technologies and tools for digital twin. Journal of Manufacturing Systems, Digital
Twin towards Smart Manufacturing and Industry 4.0 58, 3–21.
https://doi.org/10.1016/j.jmsy.2019.10.001
Liu, M., Shuiliang, F., Dong, H., Xu, C., 2020. Review of digital twin about
concepts,technologies, and industrial applications. Journal of Manufacturing
Systems 58. https://doi.org/10.1016/j.jmsy.2020.06.017

J. Huggett, “Virtually real or really virtual: Towards a heritage metaverse?,” Studies in

Digital Heritage, vol. 4, no. 1, pp. 1–15, 2020, doi: 10.14434/sdh.v4i1.26218.

M. Teller, “Legal aspects related to digital twin,” Philosophical Transactions of the

 19
Royal Society A: Mathematical, Physical and Engineering Sciences , vol. 379, no. 2207,
2021, doi: 10.1098/rsta.2021.0023.

A. Marra, S. Gerbino, A. Greco, and G. Fabbrocino, “Combining Integrated Informative

System and Historical Digital Twin for Maintenance and Preservation of Artistic
Assets,” Sensors, vol. 21, no. 17, Art. no. 17, Jan. 2021, doi: 10.3390/s21175956.

A. Addison, “Virtual heritage: technology in the service of culture.,” Jan. 2001, pp.
343–354. doi: 10.1145/584993.585055.