Development of IoT Security Exercise Contents for
Cyber Security Exercise System
1st Sanggyu Shin
Dept. of Applited Computer Engineering
Tokai University
Kanagawa, Japan
shin@tsc.u-tokai.ac.jp
Abstract—In this paper, we discuss the development of the
IoT security exercise content and the implementation of it to the
CyExec. While the Internet of Things (IoT) devices are becoming
more popular, vulnerability countermeasures are insufficient, and
many incidents have occurred. It is because there is insufficient
protection against vulnerabilities specific to IoT equipment. Also,
the developers and users have low awareness of IoT devices
against vulnerabilities from the past. Therefore, the importance
of security education on IoT devices is increasing. However, the
enormous burden of introduction and operation costs limited
the use of commercial cybersecurity exercise systems. CyExec
(Cyber Security Exercise System), consisting of a virtual envi-
ronment using VirtualBox and Docker, is a low-cost and flexible
cybersecurity exercise system, which we have proposed for the
dissemination of security education. And the content of the
exercises for CyExec is composed of the Basic exercises and
Applied exercises.
Index Terms—cybersecurity, security education, IoT, CyExec
I. I NTRODUCTION
As IoT devices become popular, cyberattacks targeting it
are increasing. And these attacks make business and service
failures, information leakage, and financial damage occur both
at home and abroad, threatening economic development and
safety and security of daily life.
In September 2016, there was a DDoS attack on a blog site
by botnet infected with IoT equipment in the United States [1].
In April 2018, the system was tampered with by unauthorized
access to a floodgate water monitoring camera of a river in
Japan [1].
The cyberattacks on IoT devices cause economic and social
losses, and then, the social awareness and need for the security
of IoT devices will increase more. The Ministry of Internal
Affairs and Communications in Japan has listed “the human
resources development” as one of the tasks in its “Compre-
hensive measures for IoT security,” which has drawn up the
issue of IoT security. [2].
As an effort of talent development, training by the Cyber
Range, which experiences cyber-attack and defense using a
dedicated application, is carrying out to obtain knowledge and
technology of cybersecurity in some universities and public
institutions [3].
Using the commercial Cyber Range exercise, a learner can
learn how to respond systematically to learners’ role to protect
themselves from cyber-attacks based on training scenarios on
978-1-7281-7392-4/20/$31.00 ©2020 IEEE
horized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 05,2022 at 07:40:25 UTC from IEEE Xplore. Restrictions ap
2nd Yoichi Seto
Dept. of Computer Science
Tokyo Metropolitan University
Tokyo, Japan
yoichi-s@tmu.ac.jp
a network built in a virtual environment [4] [5]. However,
the higher education institutions such as universities, there
is a high cost of introducing the Cyber Range system and a
shortage of personnel to maintain and manage the exercise en-
vironment. For these reasons, the educational environment for
developing security personnel has not progressed. Therefore,
higher education institutions must develop an inexpensive ex-
ercise system that facilitates joint development and utilization
of exercise programs.
To promote practical security education, we have developed
a cybersecurity exercise system CyExec (Cyber Security Exer-
cise System) with a low-cost and flexible consisting of a virtual
environment using VirtualBox and Docker [4] [5]. Exercises
content of CyExec is composed of the basic exercises and
applied exercises.
We explain the outline of the CyExec in Chapter II, brief
an overview of IoT security exercises in Chapter III, explains
basic exercise content in Chapter IV, and provides information
on developing and implementing IoT security-related applied
exercises in Chapter V.
II. OVERVIEW OF C Y E XEC
CyExec is, assuming the introduction of higher education
institutions and small and medium-sized enterprises, is the
exercise system to learn the basic techniques of cyber attack
and defense. Figure 1 shows an overview of the CyExec
exercise system [4] [5].
The below lists shown the characteristics of the CyExec.
(1) highly portable exercise environment at a low cost
(2) exercise environment of proper joint development
and utilization
CyExec enhances the implementation of exercise programs
by joint development and use of not only a single organization
but also a related organization. In order to realize joint devel-
opment and use by multiple higher education institutions, it
is necessary to develop and utilize exercise programs between
different institutions efficiently. We achieved this task using a
container technology like Docker.
Our system placed a container on Docker. It has imple-
mented the Docker on the virtual environment configured with
VirtualBox. Facilitate the creation of a purpose-specific exer-
cise environment by implementing and running a variety of
281
 Fig. 1. Overview of CyExec exercises system.

Fig. 2. The architecture of IoT system.

vulnerability diagnoses, attack, and defense exercise programs
on containers. Also, by creating an image file of the container
which runs the developed exercise program, and it can be used
jointly by publishing it in the related organization.
III. I OT S ECURITY AND E XERCISE C ONTENT
A. Type and Security of IoT Systems
Figure 2 shows the architecture of IoT systems. IoT systems
consist of IoT devices at the lowest layer, edge computing, and
cloud computing at the top layer [6].
The architecture of the IoT system can classify into three
patterns depending on the lower layer configuration.
(1) Pattern 1: IoT devices cannot process the data. The
acquired data without processing, to the cloud com-
puter via IoT gateway. As an example, digital water
temperature systems and weather sensors are there.
(2) Pattern 2: Install IoT devices within an IoT area net-
work consisting of near-range communications, such
as wireless LAN and Bluetooth. IoT devices cannot
process data. Edge server process acquired data and
transmit the processed data to a cloud computer. For
example, a product management system using IC
tags or smart appliances are there.
(3) Pattern 3: An IoT device has a function as an edge
server and processes acquired data. It is sent the
processed data to cloud computing. As an exam-
ple, multi-function printers and network cameras are
there.
B. Structure of training contents
Figure 3 shows the configuration of the CyExec exercise
content. CyExec exercise content consists of law and ethics,
basic and applied exercises [4].
We provide the law and ethics practice before entering the
main exercises to prevent the participants from exploiting to
a crime the attack techniques they have learned. The lecturer
of the exercise obliges the lecture of law and ethics education
before the practice.
In the part of the basic exercise, uses OSS (Open Source
Software) exercise content to learn the basics of vulnerability
knowledge and detection methods.

282

horized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 05,2022 at 07:40:25 UTC from IEEE Xplore. Restrictions ap

Fig. 3. Configuration of CyExec exercises.
In the applied exercise part, learners learn the practical skills
of attack and defense using IoT devices reproduced in virtual
environments.
IV. BASIC EXERCISES CONTENT
A. The positioning of the basic exercises
The purpose of the learning and acquisition skills of the
basic exercise part is as below.
(1) Purpose of the exercise: The basic exercise part
aims to learn essential vulnerability detection and
countermeasures using WebGoat [7]. In this part,
learners build the necessary skills to learn more
practical applied exercises.
(2) Learning technology: In the basic exercise, the
learner will learn the basics of attack and defense
techniques listed below.
• Necessary knowledge of the Vulnerability As-
sessment
• The essential skill about threats and vulnerabil-
ities on the system and application security
B. Implementing Basic Exercises
Figure 4 shows an implementation of the basic exercises in
CyExec. The basic exercise consists of two parts. One is the
OWASP (Open Web Application Security Project) WebGoat.
We implemented it in the container image on CyExec. And
others are Metasploitable2 and Kali Linux. We implemented
there in the VM image on VirtualBox.
We will discuss the IoT security exercises using WebGoat
in detail in Chapter IV-E.
C. Exercises content used OWASP
OWASP is a non-profit organization that aims to solve
security problems, mainly in Web applications [8]. They
have regularly announced the vulnerability information and
exercises content for Web applications and IoT systems.
Concerning the vulnerability information for the IoT system,
there was the following announcement (Table I) in December
2018. However, the content of the corresponding exercise for
IoTGoat is under development [9].
283
horized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 05,2022 at 07:40:25 UTC from IEEE Xplore. Restrictions ap
Fig. 4. Implementing Basics Exercise in CyExec.
Table I shows the OWASP IoT Top10 [10]. OWASP has a
plan to release the IoTGoat in 2020 (has not released any
available content as of March 2020). For this reason, we
used WebGoat for web application exercise content at this
development version.
TABLE I
OWASP I OT T OP 10
Rank Threat
I1 Weak, Guessable, or Hardcoded Passwords
I2 Insecure Network Services
I3 Insecure Ecosystem Interfaces
I4 Lack of Secure Update Mechanism
I5 Use of Insecure or Outdated Components
I6 Insufficient Privacy Protection
I7 Insecure Data Transfer and Storage
I8 Lack of Device Management
I9 Insecure Default Settings
I10 Lack of Physical Hardening
D. Exercises content by Metasploitable2
Metasploitable is an intentionally vulnerable Linux virtual
machine. This VM can be used to conduct security train-
ing, test security tools, and practice conventional penetration
testing techniques [11] [12]. Kali Linux is a Debian-based
Linux distribution aimed at advanced Penetration Testing and
Security Auditing [13].
Implement Metasploitable2 and Kali Linux VM images on
VirtualBox and attacks Metasploitable2 using a vulnerability
inspection tool on Kali Linux [14].
In this study, to learn the attack method considering IoT se-
curity exercises, we developed a scenario in which a backdoor
is installed using the vulnerability of the FTP server based on
the pre-survey by port scanning and the information obtained
in the survey. Also, CyExec users can independently develop
the basic exercises they need by combining Metasploitable2
with Kali Linux.
E. Configuring Basic Exercises with WebGoat
To compose the basic exercise theme, we considered the
high-priority vulnerabilities identified in the OWASP IoT Top
10. As the first step, among these high-priority vulnerabilities,
we selected what can import from WebGoat to our system
 for learning “Pattern 3” security described in Chapter III-A
OWASP has scheduled 12 exercises for IoTGoat exercise.
Among these exercises, five of which were can available in
the WebGoat exercise.
Table II shows the corresponding exercises theme and
available WebGoat exercises for the subject of IoTGoat.
TABLE II
C ORRESPONDENCE OF I OTG OAT AND W EB G OAT EXERCISES THEME
IoTGoat WebGoat
Weak, Guessable, or Hardcoded Passwords Authentication Flaws
General
Injection Flaws
Authentication Flaws
Cross-Site Scripting
Insecure Ecosystem Interfaces Access Control Flaws
Insecure Communication
Insecure Deserialization
Request Forgeries
Client side
Use of Insecure or Outdated Components Vulnerable Components
Insufficient Privacy Protection Access Control Flaws
Insecure Data Transfer and Storage Access Control Flaws
Insecure Communication
As an example of the exercise, we will explain “Insecure
Data Transfer and Storage.” This exercise is an exercise to
eavesdrop unencrypted communication over HTTP or Telnet
using the packet monitoring tool OWASP ZAP. This exercise
corresponds to exercise “Access Control Flaws” and exercise
“Insecure Communication” of WebGoat.
An exercise “Access Control Flaws” of WebGoat learns
about settings for which access control does not function
correctly, and falsifies communication by using an OWASP
ZAP. An exercise “Insecure Communication” learns about the
HTTP encryption and intercepts authentication information
included in communication by using the OWASP ZAP.
Figure 5 shows an exercise screen that the learner, in the
CyExec exercise system, uses OWASP ZAP to steal authenti-
cation information through network communication.
Fig. 5. Exercise screen to uses OWASP ZAP for steal authentication
information.
V. D EVELOPMENT OF APPLIED EXERCISE CONTENTS
A. Exercise theme
IoT devices have vulnerabilities in common with Web
applications. As a result, there has been an increase in cyber-
attacks, etc. using IoT equipment as a springboard. However,
284
horized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 05,2022 at 07:40:25 UTC from IEEE Xplore. Restrictions ap
current, the developers and users of IoT devices have not been
understanding enough of these issues. So, most of all, it is of
importance to raise awareness. Therefore, we developed two
application exercises. One is about an IoT camera, and the
other is an exercise on digital signage. The following describes
the exercises on digital signage.
We assumed the exercise about a CSRF (Cross-Site Re-
quest Forgeries) attack on digital signage web applications
connected to the network. In order to make it easier for trainees
to imagine, we used the digital signage with a display as an
IoT device, and CSRF, a typical Web application vulnerability,
was taken up as a vulnerability.
Among digital signage installed on the street and display-
ing images and characters, we assumed that the system is
connected to the network, and the administrator can log in
remotely and changes the display contents by HTTP commu-
nication. The learner will experience cyberattacks and defense
through the exercise of unauthorized changes in passwords for
digital signage devices, faking images held by digital signage
devices, and displaying them contrary to the intentions of the
user (administrator).
B. Point of exercise
The point of this exercise is to understand how to attack the
vulnerability and how to control it. That means, in this exer-
cise, learners must understand that when CSRF vulnerability in
the interface of IoT devices, results can control by an attacker
to contrary to the intention of the user (administrator). Also,
through this training, we want the learners to recognize the
necessity of countermeasures. Paradoxically, attack technology
learns to understand vulnerabilities deeply.
C. Configuration of an exercise system
Figure 6 shows the system configuration of the applied
exercise. We implemented Docker on a virtual environment
configured with VirtualBox and installed a container on
Docker. By implementing various exercise programs related
to attack and defense, such as virtual digital signage and trap
server used by attackers, and running them on containers, it is
possible to construct an exercise environment for each purpose
easily.
We built a physical environment that imitated digital signage
using Raspberry Pi. By connecting via CyExec, it is possible
to perform safe exercises in an isolated environment from the
outside while using the actual machine.
By preparing IoT devices as a rial physical environment,
the learner could imagine the actual situation of the attack.
Therefore, we expect to enable easy-to-understand exercises
with high educational effects. If a lecturer cannot prepare the
physical environment, exercises in virtual environments only
are also possible.
D. Exercise scenario
The attacker installs the attack file on the trap server, sends
a hyperlink linked the attack file to the user (administrator)
who logged in to digital signage. If the user clicked the
 Fig. 6. System configuration of the applied exercise.

link, the attacker (learner) performs exercises of exploit CSRF (administrator) clicks the link. The user (admin-
vulnerabilities to change passwords and falsification of the istrator) changes the password unintentionally
display images. and uploads the image. The displays change in
digital signage is executed.
E. Exercise procedure
To using the CyExec, a learner needs to implement CyExec
on PCs and then implement OWASP-created attack exercise
environment containers within CyExec and run on Docker.
WebWolf plays the role of a trap server that installs the attack
file created by the attacker and a link to the attack file that
is sent to the user (administrator). Also, a learner needs to
implement OWASP ZAP on theCyExec.
Connect the Raspberry Pi with this exercise program to a
PC and operate it through CyExec. Connect a display to the
HDMI terminal of the Raspberry Pi. The Raspberry Pi and the
display imitated digital signage.
The attacker uses the packet monitoring tool OWASP ZAP,
text editor, and WebWolf in CyExec. The user (administrator)
Fig. 7. An overview of the attack exercises.
operates the digital signage implemented on the Raspberry Pi
through the browser in CyExec. Figure 8 shows the result screen when succeeded in the
(1) Confirmation of normal operation: change password.
• The user (administrator) inputs the ID and pass-
word into the digital signage web application and
logs in.
• Perform the password change task and confirm
that the password is changed as intended. Per-
form the image upload and confirm that the
aimed image is displayed on the digital signage.
(2) CSRF attack exercise:
Figure 7 shows the flow of the attack exercises.
• The attacker activates OWASP ZAP and in-
tercepts the request sent from the browser to
digital signage when changing the password and Fig. 8. Result screen after changed password by an attacker (learner).
uploading the image in (1) above. Figure 9 shows the original image and the image that was
• Create an attack file based on the intercepted
changed by the successful attack exercise using the CyExec.
content and write the code to realize a CSRF
attack in JavaScript. VI. C ONCLUSION
• Upload the attack file to the trap server using With IoT devices become popular, cyberattacks targeting
WebWolf. The attack is triggered when the user it are increasing, it is urgent to develop human resources that

285

horized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 05,2022 at 07:40:25 UTC from IEEE Xplore. Restrictions ap
 [6] S. Norio et al., “Latest Developments of IoT Architecture,” The journal
of the Institute of Electronics, Information and Communication Engi-
neers, Vol. 100, NO. 3, March 2017, pp. 214-221.
[7] The OWASP WebGoat Homepage.
https://owasp.org/www-project-webgoat/
[8] The OWASP Homepage.
https://owasp.org/
[9] OWASP Internet of Things Project-IoT Goat Homepage.
https://www.owasp.org/index.php/OWASP Internet of Things
Project#tab=IoTGoat
[10] OWASP Internet of Things Project-IoT Top10 Homepage.
https://www.owasp.org/index.php/OWASP Internet of Things Project
[11] T. Yaki et al., “Practical Cyber Security Monitoring,” Tokyo, Japan:
CORONA Publishing Co., 2016.
[12] Metasploitable 2 Homepage.
https://metasploit.help.rapid7.com/docs/metasploitable-2
[13] Kali Linux Homepage.
https://www.kali.org/
[14] Kali Linux Tools Listing Homepage.
https://tools.kali.org/tools-listing

Fig. 9. The original image and changed image.

can cope with the security of IoT equipment. However, human

resources development is not progressing because of due to the
introduction cost of the exercise system and the shortage of
personnel who maintain and manage the exercise environment.
Therefore, we developed a cybersecurity exercise system,
CyExec. CyExec is based on an ecosystem consisting of a
virtual computer environment using VirtualBox and Docker.
Additionally, we developed and implemented IoT security
exercise content.
The contents of basic exercises are referenced on OWASP
IoT Top10 and based on WebGoat exercises. In this part, the
learner can experient the outline of vulnerabilities related to
IoT devices and how to detect and countermeasures. For the
applied practice contents, we developed attacks and defenses
exercise against IoT devices. This exercise imitated digital
signage considering high-priority vulnerabilities and imple-
mented them in containers.
ACKNOWLEDGEMENT
This research carried out in the PBL (Project Based Learn-
ing) in the AIIT (Advanced Institute of Industrial Technol-
ogy). In advancing the PBL, we got the cooperation of Ryo
Watanabe, Katsumi Komano, Shigeo Hatatani, Nobuaki Maki,
Chen Sheng, and Daisuke Ishikawa. We would like to express
our appreciation here. This work was supported by JSPS
KAKENHI Grant Number JP 19K03006.
R EFERENCES
[1] Information and Security White Paper 2019 (in Japanese), white paper,
Information-technology Promotion Agency, Japan (IPA), August 2019.
[2] Ministry of Internal Affairs and Communications, Comprehensive mea-
sures for IoT security, MIC, Japan, October 2017.
[3] Institute of information security, Short-term Reeducation Program -
Security Course for Working People - Homepage.
https://www.iisec.ac.jp/banner/20160316st refrsh prog.html
[4] S. Toyoda et al., “Proposal of Cyber attack and defense Exercise system
CyExec composed of ecosystem,” CSS2018, October 2018.
[5] N. Maki et al.,“An Effective Cybersecurity Exercises Platform CyExec
and its Training Contents,” International Journal of Information and
Education Technology, vol. 10, no 3, 2020, pp. 215–221.

286

horized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on August 05,2022 at 07:40:25 UTC from IEEE Xplore. Restrictions ap