Proceedings of the 29th International Conference

2018 Cybernetics & Informatics (K&I)

Jan. 31 - Feb. 3, 2018, Lazy pod Makytou, Slovakia

Interactive and Virtual / Mixed Reality Applications

for Mechatronics Education Developed in Unity
Engine
Erik Kučera, Oto Haffner and Roman Leskovský
Faculty of Electrical Engineering and Information Technology
Slovak University of Technology in Bratislava
Bratislava, Slovakia
Email: erik.kucera@stuba.sk

Abstract—The main objective of the proposed project was

to improve quality of teaching for graduates and specialists in
technical study programs (Applied Mechatronics and Automotive
Mechatronics) and related areas of automation using latest ICT
technologies, virtual reality and interactive learning in modelling,
simulation and control. Visualization of the modelling and control
processes of complex mechatronic systems, components and de-
vices (e.g cars, electric vehicles, mechatronic systems in consumer
electronics, health care, etc.) in virtual and mixed realities will
give students more insight and better understand the studied
stuff compared with conventional learning methods.
Index Terms—Unity engine, virtual reality, mixed reality, e-
learning, Microsoft HoloLens, mechatronics

I. I NTRODUCTION
Modern forms of visualisation are now realized on the basis
of the develop-ment of new ICT technologies (e.g. interac-
tive applications made in 3D engine [1], virtual reality or
augmented/mixed reality). Visualisation of process modelling,
identification and control of complex mechatronic systems,
elements and drives using virtual and augmented/mixed reality
allows students to get a much quicker and better understanding
of the studied subject compared to conventional education
methods.
A. 3D applications for education and presentation
Nowadays, there is a trend of using 3D interactive applica-
tions and virtual/mixed reality in virtual tours for houses, cars,
apartments and other products. Also, many modern interactive
3D applications for education are being developed [2] [3].
Automotive company Toyota offers modern 3D virtual
showroom for their customers [4]. This 3D showroom was
developed in Unreal Engine.
There are also more interactive applications from Animech
Technologies. Animech Technologies offers a lot of education
modules like Virtual Car, Virtual Truck or Virtual Gearbox [5].
Using these interactive applications students can understand
the functionality of mentioned devices and they can look into
their interior and detach their individual components in detail.
Very interesting project is a virtual clinic [6]. This virtual
clinic is supported by the University of Miami or Charles R.
Drew University of Medicine and Science in Los Angeles,
Fig. 1. Microsoft HoloLens - mixed reality application (Volvo) [8]
United States. This interactive application offers an insight
into the functioning of a larger clinic, and students can also
try to diagnose patients. They are thus trained through a real
experience with the health system, but this complex system is
modelled and simulated in virtual reality.
An absolute novelty is Microsoft HoloLens [7], the arrival of
which has led to the emergence of a completely new segment
of mixed reality (formerly Microsoft Holographic Platform).
Mixed reality has unquestionable advantages over virtual real-
ity, as the user perceives a real world and also a virtual world in
the same time. The use of this feature is in practice undisputed
and it is as-sumed that mixed reality is going to become a
new standard in many areas such as marketing, modeling of
complex mechatronic systems, education, etc.
For Microsoft HoloLens there are many education and
virtual tour applications.
Application HoloTour [9] provides 360-degree spatial video
of historical places like Rome or Peru. The application com-
plements 3D models of important landmarks that have not
been retained or supplementary holographic information about
elements in the scene.
B. Differences between virtual, augmented and mixed reality
Virtual reality (VR) replicates an environment that simulates
a physical presence in places in the real world or an imagined

978-1-5386-4421-8/18/$31.00 ©2018 European Union

 Fig. 2. Example of augmented reality
Fig. 3. Example of augmented reality
world, allowing the user to interact in that world. Devices for
virtual reality are Google Cardboard, HTC Vive, Oculus Rift,
etc.
Augmented reality (AR) is a live, direct or indirect view
of a physical, real-world environment whose elements are
augmented (or supplemented) by computer-generated sensory
input such as sound, video, graphics or GPS data. Augmented
reality is an overlay of content on the real world, but that
content is not anchored to or part of it. The real-world content
and the CG content are not able to respond to each other (Fig.
2 and Fig. 3).
Mixed reality (MR) is the merging of real and virtual
worlds to produce new environments and visualisations where
physical and digital objects co-exist and interact in real time.
MR is an overlay of synthetic content on the real world
that is anchored to and interacts with the real world. The
key characteristic of MR is that the synthetic content and
the real-world content are able to react to each other in real
time. Technologies for mixed reality are Microsoft HoloLens
(Windows Mixed Reality platform), Android ARCore and
Apple ARKit.
II. P ROJECT D ESCRIPTION
The main objective of the proposed project was to improve
quality of teaching for graduates and specialists in techni-
Fig. 4. Example of mixed reality - Microsoft HoloLens
Fig. 5. Example of mixed reality - Android ARCore and Apple ARKit
cal study programs (Applied Mechatronics and Automotive
Mechatronics) and related areas of automation using latest
ICT technologies, virtual reality and interactive learning in
modelling, simulation and control. Visualization of the mod-
elling and control processes of complex mechatronic systems,
components and devices (e.g cars, electric vehicles, mecha-
tronic systems in consumer electronics, health care, etc.) in
virtual and mixed realities will give students more insight and
better understand the studied stuff compared with conventional
learning methods.
The project focused on launching of modern learning forms
in the bachelor and master study programs in the broad-
spectrum field of mechatronics (Bc.- Automotive Mechatron-
ics, MSc. - Applied Mechatronics and Electromobility) using
multimedia-aided education systems, with emphasis on latest
knowledge from applied informatics, communication systems
and technologies, automation and other related disciplines.
One of the key objectives of the project was to modernize
and complete the multimedia laboratory for teaching bachelor
and master program courses in mechatronics using innovative
learning forms and developed (in this project) interactive ap-
plications and teaching materials in virtual and mixed realities.
The prepared learning materials and education methods will be
applicable in both presence and distance forms of high-school
and university education.
This project is a follow-up to the successful KEGA project
Multimedial Education in Mechatronics. Within it, quality
education and promotional videos from the subject field
were created; they are available on the modern e-learning
website http://elearning.mechatronika.cool, which was one of
the outcomes of the mentioned project. Since its launch, the
website gained an extraordinary popularity in students. The
created videos were awarded at various film competitions, e.g.
CINEAMA.
The basic idea of the proposed project was to build on the
hitherto results and use the acquired methodological experi-
ence to create learning applications for completely innovative
learning forms in virtual and mixed realities. The developed
applications were placed on the above-mentioned e-learning Fig. 6. Meeting with students from the high school at the Institute automotive
website. The project outcomes will further be used in distance mechatronics FEI STU
learning for practitioners within lifelong learning as well as
in promoting mechatronics and automation among the general
public and secondary schools pupils (dual forms of mechatron-
ics education) - potential students of technical universities.

III. P ROJECT R ESULTS

The main goal of the project was the creation of mod-

ern interactive educational 3D applications and applications
in virtual and mixed reality for mechatronics. The project
outputs will be further usable for the purposes of training of
practitioners in lifelong learning, and also for popularization
of mechatronics and automation for the general public and
high schools pupils (dual forms of mechatronics education) -
potential students of technical universities, as these materials
will be publicly available. The benefit of the project was also
the acquisition of methodical experience for the creation of
interesting and attractive educational materials for mechatron-
ics.
Both objectives and results of the project can be divided
into these points:
1) Workshops and popularization meetings / lectures
2) Interactive 3D applications and applications in virtual /
mixed reality from selected topics of mechatronics
3) Teaching improvement
4) Visualization of the new laboratory
Results are usually described on the modern e-learning
website https://elearning.mechatronika.cool developed during
the project solution.
A. Workshops and popularization meetings / lectures
1) Meeting with students from the high school in Trencin
at the Institute of Automotive Mechatronics FEI STU: The
meeting was held at the Institute of Automotive Mechatronics
of the Faculty of Electrical Engineering and Information
Technology of the Slovak Technical University in Bratislava
(UAMT FEI STU). Within the area of virtual and mixed
reality, the Microsoft HoloLens Mixed Reality Headset was
presented to high school pupils. This meeting can be seen in
Fig. 6.
Fig. 7. Open Day FEI STU
2) Open Day FEI STU: On October 2017, the Open
Day was held at the faculty. The researchers presented their
promotional and educational videos to visitors, and especially
the Microsoft HoloLens Mixed Reality application "FEI STU
Electric Cart", and also an interactive application for a com-
puter with a showroom with electric vehicles created at the
institute. This meeting can be seen in Fig. 7.
B. Interactive 3D applications and applications in virtual /
mixed reality from selected topics of mechatronics
1) Solar System - interactive application for PC: Interac-
tive educational application for web browser where you can
see planets of our solar system.
Link: https://elearning.mechatronika.cool/blog/slnecna-
sustava/
2) Electric kart showroom + game - application for Sam-
sung Gear VR and PC: Interactive application for PC and
virtual reality application for Samsung Gear VR headset.
This application presents the first electric kart developed in
Slovakia. Applications also contains a simple game where the
player can race with this kart - Fig. 8.
Link: https://elearning.mechatronika.cool/blog/elektromotokara-
hra-a-showroom-pc-gear-vr/
3) Hangar with electric vehicles of the Institute of Auto-
motive Mechatronics FEI STU - interactive application for
PC: Application contains showroom (hangar) with electric
 Fig. 8. Electric kart showroom + game Fig. 10. Electric vehicles showroom - electric kart

Fig. 11. Electric vehicles showroom - room with videos

Fig. 9. Hangar with electric vehicles - interactive application for PC

C. Teaching improvement
kart, 6x6 rescue vehicle and also a promotional video with e-
As part of the project solution, members of the team have
mobility projects of Institute of Automotive Mechatronics FEI
acquired invaluable know-how in the field of virtual and mixed
STU - Fig. 9. User uses a mouse for control of this application
reality. This know-how was then transformed into the prepara-
like in 3D editors.
tion of a new lecture for courses "Smart Mechatronic Systems"
Link: https://elearning.mechatronika.cool/blog/hangar-s- and "Multimedia and Telematics for Mobile Platforms".
elektrickymi-vozidlami-uamt-fei-stu-pc/
4) Electric vehicles showroom - application for PC and D. Visualization of the new laboratory
Oculus Rift: Interactive application for PC and virtual reality For teaching improvement of the described area, the project
application for Oculus Rift headset. Application contains a members created visualization of the prepared laboratory for
showroom with electric kart (Fig. 10), 6x6 rescue vehicle.
Application also contains a room with promotional and ed-
ucational videos (Fig. 11).
Link: https://elearning.mechatronika.cool/blog/showroom-
s-elektrickymi-vozidlami-uamt-fei-stu-z-pohladu-prvej-osoby-
pc-oculus-rift/
5) FEI STU Electric Cart - mixed reality application for Mi-
crosoft HoloLens: Microsoft HoloLens application controlled
by voice instructions. The user can see the electric kart model
(Fig. 12 and Fig. 13) with a description of its parts. You can
also see the GUI with information about other vehicles from
institute and promotional video e-mobility. The application is
available on the Microsoft Store.
Link: https://www.microsoft.com/sk-
sk/store/p/elektromotok%C3%A1ra-fei-stu/9p1nsd4prnsv Fig. 12. Microsoft HoloLens mixed reality application
 No. 2017vs022 (Collaborative education applications in mixed
reality for mechatronics).
R EFERENCES
[1] Triseum. (2017) Variant: Limits. [Online]. Available: https://triseum.
com/calculus/variant/
[2] J. Majernik, M. Madar, and J. Mojzisova, “Integration of virtual patients
in education of veterinary medicine,” in 2017 Federated Conference on
Computer Science and Information Systems (FedCSIS). IEEE, 2017. doi:
10.15439/2017F134
[3] K. Zhang, J. Suo, J. Chen, X. Liu, and L. Gao, “Design and implemen-
tation of fire safety education system on campus based on virtual reality
technology,” in 2017 Federated Conference on Computer Science and
Information Systems (FedCSIS). IEEE, 2017. doi: 10.15439/2017F376
[4] K. Sloan. (2016) Rotor brings toyota showroom 360 to life with unreal
engine. [Online]. Available: https://www.unrealengine.com/showcase/
Fig. 13. Microsoft HoloLens mixed reality application rotor-brings-toyota-showroom-360-to-life-with-unreal-engine
[5] A. Technologies. (2014) Virtual gearbox. [Online]. Available: http:
//www.animechtechnologies.com/showcase/virtual-gearbox/
[6] D. Parvati, W. L. Heinrichs, and Y. Patricia, “Clinispace: a multiperson
3d online immersive training environment accessible through a browser,”
Medicine Meets Virtual Reality 18: NextMed, vol. 163, p. 173, 2011.
[7] P. A. Rauschnabel, A. Brem, and Y. Ro, “Augmented reality smart glasses:
definition, conceptual insights, and managerial importance,” Working
paper, The University of Michigan-Dearborn, Tech. Rep., 2015.
[8] E. Uhlemann, “Connected-vehicles applications are emerging [connected
vehicles],” IEEE Vehicular Technology Magazine, vol. 11, no. 1, pp. 25–
96, 2016.
[9] M. Corporation. (2017) Holotour. [Online]. Available: https://www.
microsoft.com/en-us/hololens/apps/holotour

Fig. 14. Visualization of the new laboratory

the Internet of Things, virtual and mixed reality. The visuali-

sation is in Fig. 14.

IV. C ONCLUSION
Nowadays, there is a trend of using interactive 3D applica-
tions and virtual / mixed reality applications in virtual tours for
houses, cars, furniture and other products. This paper describes
the results of the project that dealed with development of
interactive 3D applications, virtual / mixed reality applications
for mechatronics education. The project focused on launching
of modern learning forms in the bachelor and master study
programs in the broad-spectrum field of mechatronics (Bc.-
Automotive Mechatronics, MSc. - Applied Mechatronics and
Electromobility) using multimedia-aided education systems,
with emphasis on latest knowledge from applied informatics,
communication systems and technologies, automation and
other related disciplines.

ACKNOWLEDGMENT
This work has been supported by the Cultural and Educa-
tional Grant Agency of the Ministry of Education, Science,
Research and Sport of the Slovak Republic, KEGA 030STU-
4/2017 and KEGA 038STU-4/2018, by the Scientific Grant
Agency of the Ministry of Education, Science, Research and
Sport of the Slovak Republic under the grant VEGA 1/0733/16
and VEGA 1/0819/17, and by the Tatra banka Foundation
within the grant programme Quality of Education, project