This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
1

Design and Operation of a Lightweight Educational

Testbed for Internet of Things Applications
Mahmoud AbdelHafeez, (Student Member, IEEE), Ali H.
Ahmed, and Mohamed AbdelRaheem, (Member, IEEE)

Abstract—In this paper, we present the design and operation

process of AssIuT-IoT testbed; an educational and remotely
accessible testbed for internet of things applications. The testbed
adopts the Experiment as a Service (EaaS) model in which users
are able to access the testbeds and reserve/use the available
resources remotely over the internet. Also, the testbed is
federated with other ones distributed among different locations
to allow sharing of resources, user accounts, and policies.
The federated testbeds form an educational consortium that
facilitates students’ access to hardware resources available at
different locations. AssIuT-IoT testbed consists of a control
server and a set of IoT nodes (hardware resources) supported
with different wireless communication capabilities. We present Fig. 1: IoT system layered architecture [1].
the design, operation, and performance details of the testbed in
terms of the hardware and software components. We describe
the steps needed to complete experiments using our testbed, and
we provide some examples. Moreover, we evaluate the testbed used in this paper. It consists of five layers organized as
and the IoT networks performance. Finally, we present the follows2 :
internet of things students’ competition as one of the activities
where the testbed has been used.
• Layer 0 (Things layer): This layer includes the physical
Index Terms—Internet of Things, Testbed, EaaS, WiFi, Zig- devices or “things” which are needed to be connected
Bee to the internet. From its definition, this layer includes
a very wide spectrum of different members including
humans, appliances, vehicles, animals, etc.
I. I NTRODUCTION • Layer 1 (Sensors Layer): This layer includes the
1 sensors, transducers, and actuators attached to layer 0
The internet of Things (IoT) [2] [3] is an emerging members. It is responsible for acquiring the necessary
technology that is expected to change our life in many information from layer 0 members and their surrounding
aspects. According to the International Telecommunication environments. This layer has a high heterogeneity level
Union (ITU), the number of IoT nodes is expected to exceed due to the wide range of available sensors; however, it
28.1 billion by 2020 [4] with a market value of about 7.1 has less members than layer 0 as one type of sensors
trillion US dollars. In IoT paradigm, the devices “things” can be used for many “things”.
are equipped with appropriate internet connectivity modules • Layer 2 (Nodes Layer): This layer includes the pro-
to allow these devices to communicate with each other cessing power of the IoT nodes. It is responsible for
or with central cloud services which creates a large-scale receiving the sensors’ data and performing the necessary
heterogeneous network. pre-processing functions. Examples of the members of
As IoT paradigm targets almost everything surrounding this layer are microcontrollers (MCUs), Raspberry Pis
us, it has unlimited applications in different sectors like (RPis), and smartphones.
healthcare, farming, construction, transportation, etc. Due • Layer 3 (Communication Layer): This layer includes
to the high diversity level of IoT applications and their the different communication modules responsible for
heterogeneity, it is better to apply the layered model on IoT exchanging the data and commands between the IoT
systems to achieve a clear understanding of their different nodes and their peers or between the IoT nodes and the
components. Fig. 1 shows the IoT system layered architecture cloud-based services. The communication technologies
used in this layer include Ethernet, Zigbee, WIreless
M. AbdelHafeez and M. Abdelraheem are with the Department of FIdelity (WiFi), and Cellular communications among
Electrical Engineering, Assiut University, EGYPT e-mail: (mahmoud-
hafeez@aun.edu.eg and m.abdelraheem@aun.edu.eg).
Ali H. Ahmed is with the Faculty of Computers and Information, Assiut
University, Assiut, EGYPT e-mail: (ali.hussein@aun.edu.eg)
1 An earlier version of this paper appeared in the proceedings of the 2018 2 In some cases, one device may represent more than one layer like the
IEEE Global Conference on Internet of Things [1]. smartphone which may contain the upper four layers

Copyright (c) 20xx IEEE. Personal use of this material is permitted. However, permission to use this material for any
must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
2

Fig. 2: AssIuT-IoT testbed access scenario.

others3 . IoT systems and to build prototype projects before final

• Layer 4 (Cloud Layer): This layer includes the cloud-
based platforms responsible for receiving, storing, pro-
cessing, and visualizing the IoT nodes’ data. Examples
of the cloud platforms include Amazon Web Service
(AWS), Google cloud platform, Microsoft Azure, etc.
Also, it may be a customized platform as in our testbed.
As can be noticed from the described architecture, the IoT
systems are characterized by their high level of heterogeneity
in terms of the utilized technologies such that an IoT system
designer has to gain a variety of knowledge in more than
one field to be able to design and build a complete IoT
system. For students and junior researchers, gaining sufficient
experience in these fields is a challenge that prevents them
from exploring the IoT benefits.
To overcome this challenge, we designed and implemented
a remotely accessible IoT testbed located at Assiut Uni-
versity, Egypt and hence named AssIuT-IoT4 . The testbed
design allows the users to access the IoT nodes in an easy
and low-cost way over the internet. While other IoT testbeds
already exist [5]–[9], they are more suitable for experienced
users as they include up to thousands of advanced IoT nodes
(mostly, using Real-Time Operating System (RTOS)) which
may not be of interest for students and junior researchers.
Also, they may not be able to replicate the hardware design
if they want to implement their own projects. The purpose
of AssIuT-IoT testbed is to provide an educational tool for
students and junior researches to understand the basics of
3 Higher internet protocols like Internet Protocol (IP), Transmission Con-
trol Protocol (TCP), and User Datagram Protocol (UDP) are implemented
in layer 2 or 3 according to the capabilities of the hardware used in these
layers.
4 This work is a part of the Collaborative Research and Teaching Testbed
for Wireless Communications and Networks-Phase II (CRC II) project which
aims to build a number of federated testbeds in different technologies. In
this paper, we briefly describe the CRC II project and the federation process.
implementations. Also, a set of educational materials and
video tutorials are provided to facilitate the testbed usage.
Our design follows the layered architecture for IoT systems
depicted in Fig. 1 and the following points summarize our
testbed design philosophy:
• The separation, if practicable, between the different lay-
ers of IoT system so that students can better recognize
the various elements of the IoT structure.
• The testbed usage is free of charge and can be accessed
remotely.
• The testbed replicates real-life design scenarios, such
that the user does not need any additional skills.
• The testbed utilizes over the counter hardware compo-
nents available globally as well as in the local markets.
Fig. 2 depicts a simple scenario to illustrate the general
operation mechanism of AssIuT-IoT testbed. In this scenario,
the testbed user has only a computer connected to the internet
to access the testbed and to build an IoT project. The user,
offline, writes and compiles the codes required for the IoT
nodes needed for the experiment. Then the user logs into
the control portal using a secure authentication procedure,
and reserves some of the available IoT nodes for a certain
time which should be enough to complete the required
experiments5 . Then, using the testbed portal, the user uploads
the code and programs the reserved IoT nodes. After proper
configuration of the IoT nodes, they start to work according
to the user setup. After the expiry of the reservation time,
the IoT nodes are released for other users.
In this paper, we explain the testbed design including its
two main components; the IoT nodes (hardware resources)
5 The design of AssIut-IoT testbed allows more than one to access it at
the same time as long as they are not reserving the same resources.

2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
Fig. 3: AssIuT-IoT testbed
and the control portal server. The current setup of the testbed
is shown in Fig. 3.
Paper Organization –The rest of the paper is organized
as follows. A brief review of IoT testbeds is provided in
Section II. An overview of the CRCII parent project and
the federated testbeds is given in Section III. The hardware
design of the IoT nodes is described in Section IV. The
control server architecture and its roles are described in
Section V. Section VI presents the basic steps required to use
the testbed and run a complete experiment. In Section VII,
we present some basic experiments using the testbed. In
Section VIII, we evaluate the testbed and the configured IoT
network performance. Section IX presents the IoT student
competition. Finally, the paper is concluded in Section X.
II. I OT T ESTBEDS
The maturity and rapid expansion of IoT systems deploy-
ment has motivated the research and industrial communities
to develop a large number of IoT testbeds for research and
development purposes [10]. In this section, we provide a brief
survey of the recent IoT testbeds.
FIT-IoT [5] testbed contains over 1500 IoT nodes dis-
tributed around six different locations including both static
and mobile nodes. This testbed utilizes different IoT nodes
based on different processors (MSP430, STM32, and Cortex-
A8) and different wireless communication modules (Zigbee
and WiFi). Every IoT node needs two other intermediate
processing devices; gateway which is a Linux computer and
a control node to control its access and configuration. Also,
all nodes run using 5 different RTOSs.
FIESTA-IoT [6] is a large IoT consortium that consists of a
number of testbeds located across the European Union (EU)
and covers different IoT applications like smart cities, mar-
itime, water quality, smart grid, smart building, and crowd-
sensing. The total number of available IoT devices exceeds
5,000 units. They produce millions of sensors’ observations
per day providing a rich resource of IoT data for researchers.
In [11], the authors designed a higher layer platform that
enables performing experiments over multiple IoT testbeds
through a federation framework. The platform was validated
by applying it on ten testbeds from the FIESTA-IoT project.
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
3
IoTBed [8] is a generic architecture for testbeds as a
service model that allows creating testbeds from the IoT
nodes already connected to the internet. This feature allows
the developers to choose the appropriate testbed for their
purpose and to rent the available resources for the required
time. The architecture was validated using a testbed that has
IoT nodes powered by Contiki operating system.
Smart Santander [12], [13] is a testbed that focuses on
smart cities IoT applications. In this testbed, the IoT nodes
are distributed around a city covering an area of 1 km2 . The
IoT nodes are equipped with a number of sensors including
noise level, carbon monoxide, temperature, and light inten-
sity. The same authors provided a complete business model
for IoT testbeds in [14].
JOSE [9] is a large scale IoT testbed located in Japan that
aims to integrate three types of IoT resources, namely IoT
nodes (sensors boards), IoT storage resources, and computa-
tional resources. The testbed elements are distributed over 5
locations in Japan and utilize network visualization to ensure
sensors’ data security and privacy.
In [15], the authors proposed a laboratory IoT testbed that
focuses on Zigbee performance over different layers like the
Physical (PHY) and Media Access Protocol (MAC) layers.
The testbed is used to evaluate the network recovery scenar-
ios and the interaction between the Zigbee based network and
the internet. This testbed is not remotely accessible. Also,
IoT testbeds which focus on studying specific aspects of
IoT systems like security and power consumption have been
proposed. In [16], the authors introduced a testbed that is
dedicated to studying the security and privacy aspects of IoT
devices by analyzing the WiFi or Bluetooth traffic transmitted
from these devices. The testbed is capable of conducting a
number of experiments to test vulnerability scans, encryp-
tion solutions used, identifying insecure protocol versions,
monitor idle IoT node traffic, investigating firmware up-
dates, authentication issues, and privacy violations. Authors
of [17] designed a security testbed that employs standard
and advanced techniques like artificial intelligence to test the
security robustness of IoT services.
The authors of [18] proposed a testbed that measures the
timing and energy consumption characteristics of a single
IoT node. The testbed utilizes MSP430 family MCU and
Zigbee module. The testbed is interfaced with the user via
a control server which, on its turn, communicates with each
IoT node using a small Linux-based node that is responsible
for programming the MCU.
As can be noticed from the above survey, current IoT
testbeds are dedicated to advanced research aspects of IoT
systems. Some of these testbeds [6], [9], [12], [13] focus
on the IoT applications which benefit from the data received
through the deployment of a large number of IoT nodes.
Other testbeds allow the researchers to build their own
customized IoT networks for certain applications like in [5],
[15], [18]. These testbeds utilize advanced IoT nodes which
mainly run over RTOS and require a dedicated control unit
for each IoT node to allow its access and programming. In
addition, they focus on other aspects like nodes coverage
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
4

TABLE I: A features comparison between AssIuT IoT and other testbeds.

AssIuT IoT FIT IoT [5] Smart JOSE [9] St. Petersburg
Santander [12] SUT [15]
Scale Small Medium Large Large Small
Environment Lab Lab Outdoor Outdoor Lab
Mobility No Yes Yes No No
Purpose Education Research Research Education Education
and Research and Research
Communication Yes Yes Yes Yes No
heterogeneity
Level of Beginner Professional Professional Professional Beginner
Experience
Federation Yes Yes Yes No No
Reproducibility Easy Difficult Difficult Difficult Difficult

and mobility effect, and are not intended to be used by
students and junior researchers. While these testbeds are very
useful for developing and enhancing IoT systems, they are far
too advanced for students and junior researchers. AssIuT-IoT
tries to fill this gap by providing an easy-to-use educational
testbed for IoT applications using hardware resources which
are available as over the counter items. Also, it allows the
users to replicate their projects after validating their design
on the testbed. Table I shows a features comparison between
AssIuT-IoT and other testbeds.
III. CRCII
Collaborative Research and teaching Testbed for wireless
Communications and networks phase II (CRC II) is a multi-
universities project which gives students and researchers
remote access to the hardware resources available at different
Egyptian universities. The project aims to reduce the cost of
purchasing duplicated hardware available in other universities
and benefit from the accumulated technical experience of
testbeds’ administrators through tutorials and teaching ma-
terials prepared for every testbed. The testbeds are federated
together in a manner that allows them to be connected and
synchronized with each other while providing a common
interface to the user using minimum resources and costs. The
federation service provides the following features:
• Synchronizing resources and users databases among the
testbeds.
• Providing a single entity for registration and access
to different testbeds through a unified portal (besides
individual testbeds’ portals).
• Enforcing sharing policies among testbeds.
• Collecting utilization statistics from different testbeds.
The federation mechanism is based on the Slice-based Fed-
eration (SFA) framework as will be described in Section V.
The CRC II federation is formed by :
• The universal Software Radio Peripheral (USRP) testbed
hosted at Alexandria University (the host of CRC I
project [19]). It also contains the federation controller
which is responsible for adding new sites to the federa-
tion and the unified portal which acts as a central access
point to all testbeds [20].
•The Internet of Things (IoT) testbed (AssIuT-IoT) [1]
hosted at Assiut university.
• The Field Programmable Gate Array (FPGA)
testbed [21] hosted at Cairo University.
• A lightweight version of USRP testbed controlled by
RPis [22] is hosted at Alexandria University.
In the rest of the paper, we will focus only on AssIuT-IoT
testbed.
IV. I OT N ODES D ESIGN
In this section, the hardware design of the testbed IoT
nodes is described.
A. IoT node components
Here, we will describe the different components of the
designed IoT nodes according to the above layered model.
Layer 1: The sensor layer is occupied by the DHT-
11 [23], a digital temperature and humidity sensor. The
DHT utilizes a single-wire communication protocol to
send the data to the IoT node MCU. This sensor was
chosen to be used in layer 1 in the testbed because of
its worldwide availability at affordable cost and its ease
of use. Also, a set of Light Emitting Diodes (LEDs) are
connected to the MCU which are controlled through the
cloud platform to represent real actuators in IoT applications.
Layer 2: The onboard processor for the IoT node is the
Arduino Mega 2560 Rev3 [24]. The Arduino Mega board
is built around the ATmega2560 which is an 8-bit AVR
RISC-based MCU [25]. This platform is selected because
it represents an easy-to-use introductory programming
environment that suits basic educational purposes.
Layer 3: The AssIuT-IoT testbed nodes are equipped
with two wireless communication modules, namely WiFi
and Zigbee.6 The WiFi connectivity is provided by the
6 It is worth mentioning that our testbed does not test the WiFi or
the Zigbee communication properties like the transmission/retransmission
mechanism, data rate, etc. The parameters of both protocols depend on the
hardware modules used.

2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
Fig. 4: The IoT node and the shield Printed Circuit Board
(PCB) layout.
ESPressif Systems ESP8266-12E module [26], which is a
low-cost WiFi module with a complete TCP/IP stack. The
WiFi wireless topology was chosen to provide the IoT nodes
with a direct internet access or to form a local IoT network.
The Zigbee networking is provided through the Digi’s Zig-
Bee (Xbee) module [27]. This module adopts IEEE 802.15.4
protocol to provide peer-to-peer or point-to-multipoint net-
working. In AssIuT-IoT testbed, the ZigBee network topol-
ogy was adopted due to its special features which do not exist
in WiFi networks. The Zigbee network has a greater wireless
range which extends to hundreds of meters compared to
few dozens of meters supported by WiFi networking. The
topology also provides mesh networking, compared to the
WiFi star topology, which allows simple extension of the
network’s physical range by routing data through mesh routes
between different IoT nodes. Digi’s Zigbee module has
been chosen because its firmware enables the module to
be configured with ATtention (AT) commands via Universal
Asynchronous Receiver-Transmitter (UART) terminals. This
introduces another degree of customizability by enabling the
remote user to configure the ZigBee network using the code
uploaded to the MCU of the IoT node.
B. Hardware implementation of the IoT nodes
As illustrated in Fig. 4, the hardware of the IoT node
was designed and fabricated as an Arduino shield on a
Printed Circuit Board (PCB). This consolidated design was
adopted to achieve a small size of the IoT node. The Arduino
shield form also provides a reliable connection and wiring to
the Arduino board with the shortest possible electric paths.
Electronic components like level shifters and regulators are
used to interface the Arduino board with the communication
modules, as both of the modules require a supply voltage of
3.3V while the Arduino board operates on a 5V supply.
V. A SS I U T-I OT T ESTBED C ONTROL P ORTAL
In AssIuT-IoT testbed, the control portal has two main
roles. First, the portal acts as the testbed management con-
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
5
troller. It is responsible for managing the users’ Authentica-
tion, Authorization, and Accounting (AAA) processes. Also,
it synchronizes the information about users and resources
with other testbeds to enable the federation. The second role
is to act as the resources and experiments controller. Through
the portal, users reserve and configure the available resources,
execute experiments, debug code, and visualize the collected
data. In this section, the main components of the control
portal are presented and their functions are explained. Also,
the utilization of the RPi, which acts as a relay between the
control server and the resources, is presented. At the end of
this section, we describe the debugging utility used in our
testbed. Fig. 5 shows the control portal architecture with its
different components and depicts its relation to the users, the
resources, and the other testbeds.
A. Control Portal Architecture
The control portal is built using Django web frame-
work [28] on Linux Debian operating system [29]. The server
is HP Prodesk with an Intel Core i7 processor and 8 GB of
RAM. To understand the different components of the control
portal, we divide its architecture into three layers according to
their roles regarding the users, resources, and the federation
service. These layers are:
Users Layer: This layer contains the software services which
deal with the testbed users directly. These services are:
• The portal page templates: This service includes the
different control portal web pages which interface the
user to the testbed. The portal pages can be classified
into unified and customized ones. The unified pages are
common to all testbeds like those used for users regis-
tration/access and resource reservations. The customized
pages are tailored to the type of resources found in
different testbeds. For example, in AssIuT-IoT testbed,
there are customized pages like the “Upload Binary
File” page which is accessed by the user to upload
the MCU binary file to the portal which, on its turn,
programs the IoT node. Also, the “Variables Creation”
page which enables the user to define the variables to
be received from the IoT nodes.
• IoT data visualizing: The testbed uses a javascript-
based library to visualize the captured sensors data. In
the testbed portal “Variables Creation” page, the user
adds the desired variables like temperature and humidity
which must match the same quantities collected by
the IoT node7 . After executing the experiment, the
control portal receives the declared variables updates
and visualizes them.
Resources Layer: This layer contains the set of software
services that deal with the testbed resources. The portal
communicates directly with the resources (IoT nodes) or via
the intermediate RPi which acts as a relay. These services
are:
7 This service may not be used if the user decided to use one of the
cloud-based services like Ubidots or AWS.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
6

Fig. 5: AssIuT-IoT testbed control server architecture and its relation to experiment elements (user, CRC II testbeds,
resources).

• Node Programming: This service utilizes AVRDude transmitting IoT node ID and type. Then, the informa-
software tool [30] to communicate with the IoT node tion is stored in the system database and visualized. It
MCU. AVRDude enables the users, via the Command- depends on establishing a TCP/IP connection between
Line Interface (CLI), to download, upload, and manip- the IoT node and the portal using a unified packet format
ulate the program memory and EEPROM contents of to extract the IoT node ID and the sensor values. The
the AVR MCU using the In-System Programming (ISP) format of the packet is shown in Fig. 6.
technique. The user uploads the binary file that contains • Serial Monitoring: This service provides a debugging
the code to program a specific IoT node MCU through tool for the testbed users. Debugging messages sent from
the “Upload Binary File” page. The server automatically each IoT node over the USB interface is buffered and
generates the AVRDude commands with the targeted relayed to the corresponding user over the internet. This
Universal Serial Bus (USB) port and binary file location service utilizes Pyserial Library [31] to receive the IoT
to program the IoT node MCU. nodes MCU messages sent over the serial (USB) port.
• Node Identifying: This service is used to link the IoT The implementation of this service depends on the way
node unique identifier (ID) stored in its MCU EEPROM the IoT nodes are connected to the testbed (directly
and the server USB port connected to it as the latter is to the server or via an RPi) and will be discussed in
used by the AVRDude tool in the programming step. In Section V-C.
the server or the RPi, the communication with each IoT
The previous services form the necessary components to
node is done using the Abstract Control Model (ACM)8 ,
communicate directly between the portal server and the IoT
which emulates the serial (USB) port number. For that,
nodes. For the IoT nodes connected via an intermediate RPi,
the node identifying service invokes the AVRDude tool
some additional services are needed as the user has no direct
to scan the available ACMs and discover the IoT nodes
access to the RPi and the communication has to be done
connected to each of them by reading their IDs from
through the control portal. These additional services are:
the MCUs’ EEPROMs. The mapping between the IoT
node ID and its ACM number is stored in the portal • RPi Programming: As the testbed user has no direct
database to be used later by the AVRDude in the access to the RPi, this service relays the binary file,
node programming service. This service runs after the uploaded by the user on the portal targeting an RPi-
server is powered up and then periodically to update the connected IoT node, to the RPi over the Local Area
database for any hardware changes. Network (LAN) or the Wide Area Network (WAN)
• IoT Data Receiving: This service receives the sensor according to the location of the RPi. On its turn, the
values transmitted by the IoT nodes to the control portal RPi uses AVRDude tool to program the code on the
over the internet. It classifies the data according to the IoT node MCU.
• RPi Messages Archiving: This service is used to receive
8 The
ACM is equivalent to the COMmunication (COM) port in the the debugging messages from the RPi-connected IoT
Windows environment. nodes by using the Pyserial library. Then, the RPi

2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Fig. 6: Frame format for the update message payload trans-
mitted from an IoT node to the control server.
transmits these messages to the server where they are
saved and relayed to the IoT node user.
Federation Layer: It is responsible for facilitating the
federation between AssIuT-IoT and the other testbeds. This
layer consists of two main components:
• Slice-based Federation (SFA): The federation is carried
out using the SFA mechanism which is responsible
for the discovery, reservation, and provisioning of the
shared resources. It defines the minimal set of interfaces
and data types that enable a federation of slice-based
network components to interoperate and synchronize.
In SFA, slices are the primary abstraction used for
accounting and accountability. A slice is defined by the
set of resources spanning a set of network components,
plus any associated set of users that is allowed to
access the resources for the purpose of running an
experiment. In the CRC II federation consortium, the
SFA is used to synchronize the database of the differ-
ent testbed portals. The SFA also facilitates database
synchronization between sites to ensure that all the
resources are available. As the federated testbed contains
different types of resources, the federation between the
testbeds is divided into multiple levels according to each
testbed resource nature and capability. Table II shows
the different levels of federation allowed for the CRC
II testbeds. For AssIuT-IoT testbed, the federation is
limited to L1 where the testbed shares its resources and
sharing policy. On the other hand, higher-levels like L2
and L3 require additional capabilities to enable the user
to run an experiment that spans resources from different
testbeds simultaneously using a single controller like
the cOntrol and Management Framework (OMF) [32].
As there are no experiments required to be done using
combined resources from AssIuT IoT and the other CRC
testbeds, we have limited the federation level of AssIuT-
IoT to L1.
TABLE II: Different CRC II federation levels.
Federation function L0 L1
Remote-Login x x
On-site reservation x x
Resource sharing policy x x
Remote site reservation x x
Multiple reservations
from multiple site
Accessing multiple
testbeds independently
Accessing multiple testbeds
using a single controller
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
Things Journal
7
• Testbed Database: The server database is used to store
the data related to the users and resources as well as the
data received from the IoT nodes’ sensors9 . Through the
mySQL database management system, query requests
from the Django framework models to the database
tables are handled. The database tables can be classified
into federation tables and local tables (customized ta-
bles). The federation table entities define the minimum
set of information that has to be shared between the
federated testbeds. The federation tables are:
1) Portal User: Contains the primary user information
like username, password, affiliation, and type (student
or instructor).
2) Portal Reservation: Holds the reservations informa-
tion such as user ID and node ID.
3) Portal Reservation Details: Contains the start and the
end times of the reserved node (slice time).
4) Portal Phy Node: Contains nodes information such
as node ID, name, Interface, or IP address if applica-
ble.
The customized tables are those tailored to utilize spe-
cific resources. In AssIuT-IoT testbed, these tables are:
1) Node Phy Details: Contains the attributes which de-
scribe how to access and program the IoT nodes such
as the node ID, ACM number, programming baud
rate, and the RPi IP address in case of connecting to
remote nodes attached to the RPi.
2) Node Vars: Contains the user-modified list of sensing
quantities which will be received from each IoT
node. These table fields are modified by the user
to add/remove a new/existing sensing parameter for
individual IoT nodes.
3) Node Vars Output: Stores time-stamped values for
specific output variables for all of the IoT nodes.
B. Raspberry Pi as an Intermediate Relay
In AssIut-IoT testbed design, the RPi acts as a relay
between some of the IoT nodes and the control server. The
RPi is responsible for programming its directly connected
resources and relaying their debugging messages to the server
but it is not involved in other tasks like resources and users
management. Here, we utilize the Raspberry Pi 3 Model
B [33] running Raspbian [34] which is a customized version
of Debian Linux optimized for RPi. In this design, the RPi
L2 L3 is directly connected to the IoT nodes via the USB cables
x x and communicates with the control server over any IP-based
x x network using its Ethernet or WiFi interfaces. The RPi may
x belong to the same network of the testbed control server or
x to another network as long as both networks are reachable
to each other. As the RPi communicates with the control
x
server using any IP based network type, the testbed resources
(IoT nodes) will not be limited to those which are directly
x x
connected to the server via its USB ports. This feature
x
9 The database also belongs to the resources layer as it is used to store
the variables’ update.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
Fig. 7: The RPi as an intermediate relay between the server
and the IoT nodes.
enables the testbed to serve a large number of IoT nodes
which may be distributed among different locations with only
one server. The RPi setup in AssIuT-IoT testbed is depicted
in Fig. 7.
When the RPi starts up, it notifies the control portal
such that the server marks the RPi-connected IoT nodes as
available. When a user requests an IoT node connected using
the RPi, the same procedure of reserving and accessing a
server-connected IoT node takes place. The portal receives
the binary file from the user and forwards it to the RPi
which uses the AVRDude tool to program the MCU of
the targeted IoT node. After that, the IoT node collects
and transmits the sensor data to the control server as usual
without any interaction from the RPi. At the end of using
the reserved slice, the server instructs the RPi to flush the
program memory of the IoT node MCU.
C. Serial Monitoring as a Debugging Tool
The Arduino Integrated Development Environment (IDE)
does not support hardware debugging and, as an alternative
solution, it communicates messages between the onboard
MCU and the host computer USB interface over the UART
serial terminal. In the application code, the user can check the
values of different variables by using the Serial.print
Arduino function which prints messages including the desired
variable values on the UART interface. The user receives
these messages using any terminal software on the user’s
personal computer. This method enables the user to check
the correctness of the execution of the user’s code. However,
the serial communication requires that the Arduino board to
be directly connected to the host computer via the USB cable
which is not feasible in our remotely-accessible testbed. To
overcome this problem, a message logging service is added
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
8
Fig. 8: Testbed serial monitoring service mechanism.
to the control server. This service receives different messages
from the Arduino board and relays them to the corresponding
user who reserves the IoT node at this time. The design of this
service depends on whether the IoT node is connected to the
server directly or through the RPi. Fig. 8 shows the design of
the serial monitor service. The design of this serial monitor
service is based on PySerial, a Python-based library that is
responsible for accessing and communicating using the serial
port of the host (virtual COM port in Windows and ACM in
Linux environments). For the IoT nodes connected directly to
the server, the PySerial receives the messages directly from
the IoT node onboard MCU. For the IoT nodes connected
via the RPi, the process is carried out over two steps. In
the first one, the RPi receives the serial messages from the
IoT node using the PySerial library, then it forwards the
received messages to the control server where the messages
are saved in an intermediate buffer before being relayed to
the corresponding user.
VI. T ESTBED E XPERIMENTS S CENARIOS
In this section, the procedure that should be followed to
properly use the testbed is described. Fig. 9 shows a flow
chart that depicts the required steps to implement a typical
experiment on the testbed from the user’s point of view.
These steps are as follows:
1) Design Stage:
a) Network topology design: The user, offline, designs
the IoT network topology needed to be tested accord-
ing to the needed application.
b) Coding and compiling: The testbed user should write
and compile the software code, usually in C program-
ming language for Atmega MCUs, which implements
the required design and application for each IoT node.
This step is usually performed offline before accessing
the testbed in order to limit the resources’ reservation
time.
2) Authentication & reservation stage: This stage con-
tains the steps required from the user to successfully
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
9

Fig. 9: Experiment steps flowchart.

access the testbed and reserve the needed resources.

It can be further divided into the following sequential
steps:
a) Authentication: The user must be registered at one of
the federated testbeds to be able to access the AssIuT-
IoT portal.
b) Nodes reservation: Based on the targeted IoT network
design, the user reserves the needed IoT nodes from
the available resources.
3) Variables creation stage: The configured IoT nodes
may send their collected sensor data to the control portal
to be visualized. To do that properly, the user has to
create a new entry on the “Variables Creation” page
for every variable that matches the name used on the
corresponding MCU code.
4) Programming the IoT nodes stage: In this step, the
user uploads the compiled application binary files to
each of the reserved nodes. After completion of the
upload to the control server, it programs the targeted
IoT node MCU by invoking the AVRDude tool.
5) Monitoring results & debugging stage: In this stage,
the user observes the data collected on the “Variables Fig. 10: (a) WiFi based IoT network (b) Mixed IoT network.
Monitoring” page or received through the serial monitor
debugging service. The user may eventually need to
change the code if the IoT network did not behave VII. E XPERIMENT E XAMPLES
according to the targeted design.
In this section, we describe small experiments that can be
carried out at the students level using AssIuT-IoT testbed.

At the end of the reservation time, the control portal auto- A. WiFi-based topology
matically uploads an empty binary file on the IoT node MCU As an overview of the WiFi based topology, the reserved
to flush the expired code. IoT nodes form a star connection using their WiFi modules

2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
(Zigbee modules are deactivated in all nodes) with only the
central (parent) node connected to the internet as shown in
Fig. 10(a). The central node acts as a hub for the leaf nodes
as it collects and re-formats their data before sending it to
the control server. After collecting the data, the central node
initiates a remote connection with the control server through
the internet to send all the collected data. The detailed
description of the tasks carried out by the central node are
as follows:
1) Configuring the ESP module to be in the dual-mode
(station + access point).
2) Networking with the internet. The ESP station mode
allows it to connect to an available WiFi internet con-
nection to communicate with the control server for data
sending.
3) Constructing an access point. The access point mode
allows it to establish another isolated local WiFi network
for the leaf nodes.
4) Establishing TCP connections with the leaf nodes to
receive their data.
5) Waiting a predefined interval of time for the data to be
sent from the leaf nodes over TCP.
6) Closing the TCP connections with the leaf nodes.
7) Collecting its own sensors’ data.
8) Formatting the collected data in a payload which is
compatible with the control server format shown in
Fig. 6.
9) Initiating a TCP connection with the control server.
10) Sending the data to the control server.
11) Closing the TCP connection with the control server.
12) Repeating the steps beginning from step 4 again.
The leaf node performs the following tasks:
1) Connecting to the local WiFi network created by the
central node.
2) Gathering the required data from its sensors.
3) Sending the gathered data through the TCP connection
to the central node.
4) Repeating the steps beginning from step 2 again.
The main drawback of this configuration is that all the
workload is centralized in the parent node, as it is responsible
for creating the local network, gathering data, formatting, etc.
which may reduce the speed performance of the network if
compared with other topologies that balance the workload
among all network nodes. As an example for the limitations,
the firmware installed on the ESP modules allows only 5
simultaneous TCP connections, which limits the number of
leaf nodes connected to one central node. In addition, the
synchronization of the local transmissions between the nodes
and the remote transmission to the control portal (or cloud
service), heavily depends on the central node as it actually
does all the work. This results in a complex firmware of
the central node which reflects on its performance (energy
consumption) as will be detailed in the next section.
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
10
B. WiFi & Zigbee-based topology
In this topology, the local communication between the IoT
nodes is achieved through a Zigbee mesh network, while
the remote communication to the control server is achieved
through a WiFi connection at the central node only. The
Zigbee topology as shown in Fig. 10(b) has three main
superior advantages if compared with the WiFi star-based one
shown in Fig. 10(a). First, the number of nodes allowed in the
network is much larger. Second, the physical reach of the net-
work is much longer. Third, the mesh topology automatically
configures the routing between the nodes to achieve the best
configuration without the need for any interference from the
user. Zigbee modules also allow transparent data transmission
between the modules through their UART interface, which
makes coding and synchronization between the nodes far
simpler if compared with the TCP server approach used by
the WiFi based topology, as the user needs only to configure
the addresses once at the beginning of the code. The central
node aggregates the data received by its Zigbee module and
transmits them to the control portal using its WiFi module.
At the control portal, the experimenter can check each of the
transmitted variables using a separate graph.
The central node performs the following tasks:
1) Connecting to the internet through the available wireless
connection using the ESP module configured in station
mode.
2) Reconfiguring the Xbee module to specific Zigbee net-
work parameters by utilizing the proper AT commands
if necessary.
3) Sending a pre-agreed data-request message to the first
leaf node ordering it to send specific data from its sensor
and waiting for its response.
4) Repeating step 3 for all leaf nodes.
5) Collecting the data from the its sensors.
6) Formatting all the acquired data into a suitable payload
structure compatible with the control server, as shown
in Fig 6.
7) Establishing a TCP connection with the control server.
8) Sending all the formatted data to the control portal.
9) Repeating the steps beginning from step 3 again.
The role of the leaf node is to:
1) Reconfigure, if necessary, the XBee module to specific
ZigBee network parameters that match those at the
central node.
2) Wait for the central node “data-request” message, that
would be sent over the ZigBee network.
3) Parse the received message from the central node.
4) Respond to the request message according to its content
and to send the required unformatted data to the central
point.
5) Repeat the steps beginning from step 2 again.
VIII. A SS I U T-I OT P ERFORMANCE E VALUATION
In this section, the measurements and analysis of different
performance metrics of the AssIuT IoT testbed are presented
including:
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
30
25
CPU utilization (%)
20
15
10
0
Server (Peak) Server (AVG)
CPU
Fig. 11: CPU utilization for both the Server and the RPi.
•The Central Processing Unit (CPU) utilization of the
control server and the RPi during the experiments.
• The performance of the wireless networks from the per-
spective of transmission time and energy, when utilizing
the ESP8266 by the WiFi network and Xbee module by
the Zigbee network10 .
• Performance of the IoT nodes processing capabilities
from the perspective of computation time and energy.
All the results are shown with a confidence interval of 95%.
A. Performance of the control server and RPi CPUs
In this experiment, we evaluate the ability of the control
server CPU (Intel Core i7 with 8 cores each of 3.6 GHz)
and RPi CPU (Broadcom BCM2837 Quad Core 1.2 GHz)
to handle their tasks. The CPU utilization is monitored when
the maximum number of four nodes that can be connected to
the RPi, together with the current number of nodes connected
directly to the server are programmed. The size of the binary
file in use is 253KB, which is 98.8% of the maximum
possible size for the Atmega2560. Fig. 11 shows the peak and
average (AVG) CPU utilization during this experiment. As
can be inferred from the figure, the handling of simultaneous
experiments does not consume much of the CPU processing
power. Notably, while more than one user may be using the
testbed server simultaneously, it is not expected that all of
them will be programming their reserved nodes at the same
time. Thus, the AssIut IoT is characterized as a lightweight
testbed.
B. Performance of the IoT wireless networks
In this experiment, we compare the performance of two
different small size IoT networks established using WiFi and
Zigbee technologies. For the WiFi network, the implemented
topology is based on an access point initiated by the parent
node and several child nodes. The parent node initiates a
TCP connection with each of the child nodes, one at a
time. The parent node sends a data request message to each
child, and the latter responds with the required data (for
example, the current temperature value). The parent node
10 Here, we measure and compare the IoT networks performance not that
of the wireless modules themselves. The modules parameters, as indicated
in their datasheets, are listed in Table III.
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
Things Journal
11
RPi (Peak) RPi (AVG)
Fig. 12: Experimental setup for time and energy measurement
of wireless transmission.
cycles through the available child nodes sequentially. For
the Zigbee network, the implemented topology is based on a
Zigbee network initiated by the parent node, as a coordinator,
and all child nodes are connected to it. The parent node
sends a data request message to each child, one at a time,
and the latter responds with the required data. The parent
node cycles through the available child nodes sequentially.
The data request/respond frame used in both topologies has
a typical size of 13/18 bytes. For both networks, the parent
node aggregates the data and forwards it to the cloud server
using its WiFi interface. A special experimental setup is used
to measure the total time and energy used by the parent node
during the data request/respond transmission phase among
the local networks. Fig. 12 shows the setup used in the
measurement process. A 0.5Ω resistor is placed before the
Vcc pin of the wireless module (Zigbee or WiFi) to allow
current sensing, by measurement of the voltage across the
resistor and dividing by its value. A trigger signal is raised
by the microcontroller immediately before the transmission
begins to start the data acquisition by the digital scope. The
trigger signal goes to zero immediately after the transmission
end to stop data acquisition.
The energy and time of transmission are measured during
the period at which a full request/respond process is per-
formed with all child nodes in the network. The experiment
is performed with one parent node and two, three, and four
child nodes connected to the network. To measure the WiFi
module transmission time, a TCP connection is initialized
with a child node, followed by the request-respond cycle, and
then terminated before moving on to the next child. Fig. 13
shows the total transmission time needed for one transmission
cycle, which includes all of the child nodes, for both the
WiFi and Zigbee based networks. For the transmission energy
measurement, the energy E is calculated as:
 
X V0.5Ω (n)
E= (3.3 − V0.5Ω (n)) · · 10−5 , (1)
n
0.5
where n indicates the samples taken during transmission
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
12

TABLE III: Xbee and ESP8266 operation characteristics.

Serial Port Protocol Wireless Transmission Range Average current Operating voltage
speed (typical) rate (MAX)
Xbee 9.6 Kb/sec. IEEE 802.15. 4 35 Kbit/sec 120 m 40 mA 3.3 V
ESP8266 115.2 Kb/sec. IEEE 802.11 (b) 11 Mbit/sec 27 m [35] 71 mA 3.3 V

700
works [36], it is vital to implement cryptographic measures
to provide different security services through IoT networks
600 Zigbee
Wi-Fi like confidentiality and authenticity. One of the challenges in
500
IoT security is the limited available resources in IoT devices.
Time (msec)

400 Limitations in storage and processing power put restrictions

300 on the security algorithms that can be implemented in IoT de-
200 vices. The Advanced Encryption Standard (AES) symmetric
100
block cipher is chosen to be the code block under inspection
to quantify the loading effect of applying the cryptographic
0
2 3 4 code blocks and its execution cost on the processing unit at
Number of Child Nodes
run time.
Fig. 13: Data transmission total time in WiFi and Zigbee The measurements were done from two perspectives:
networks. • Required processing time
• Energy consumption
800 The previous parameters were measured for the application
Zigbee
of the AES symmetric encryption algorithm between the
600 Wi-Fi IoT nodes. The tests were done corresponding to the Cipher
Energy (mJ)

Block Chain (CBC) mode of operation. All the experiments

400
200
0 done to both encryption and decryption processes individ-
2 3
Number of Child Nodes
Fig. 14: Data transmission energy in WiFi and Zigbee net-
works.
process, V0.5Ω is the voltage across the 0.5Ω resistance, and
10−5 sec. is the sampling time. The whole equation repre-
sents a power integration with time into energy. Fig. 14 shows
the consumed energy during a complete request/respond
cycle for networks consisting of a parent and two, three, and
four child nodes in WiFi and Zigbee networks.
As can be concluded from the figures, the WiFi network
operates at a higher speed compared to the Zigbee network,
despite the overhead time associated with the TCP connection
establishment and termination. This is because of the higher
data rate of WiFi compared to the Zigbee. On the other hand,
the energy consumption of the parent node in the Zigbee
based network is less than that of the WiFi. Besides, the
Xbee has a longer transmission range compared to that of
the ESP module.
C. Performance of the processing units of the IoT nodes
To test the performance of the IoT nodes processing units,
a block of code is chosen to be executed while monitoring
the time and energy consumption of the node during the
code execution. Due to the importance of security in IoT net-
were performed on a 16 Byte (128 bit) plain/cipher text i.e.
one AES encryption/decryption block operation. The AES
library used is called tiny-AES-c by kokke [37].
1) Processing time measurements: Measurements were
4
ually with the key sizes: 128, 192, and 256 bit. The At-
mega2560 was set to run at the maximum speed of 16 MHz
and the internal built-in Timer unit was used to calculate the
time precisely.
Fig. 15 shows the required time for executing the
encryption and decryption processes of AES in CBC
mode with different key sizes: 128, 192, and 256 bit. If
we compare the time required by the processing node to
perform decryption or encryption of a message with the
transmission time shown in Fig. 13, we can notice that
processing a computational intensive block of code like AES
CBC takes only a small fraction of the total time compared
to the time consumed in the transmission of packets.
2) Energy Measurements: Since the IoT nodes usually
depend on limited power sources (batteries), the energy
consumed by the running code segments has a direct impact
on the nodes lifetime. This section is concerned with the
measurement of energy consumed while executing the en-
cryption and decryption segments of the code. Fig. 16 shows
the amount of energy consumed in the AES CBC mode, for
encryption and decryption processes with different key sizes:
128, 192, and 256 bit.
Again, the power consumed in a computationally intensive
block of code like AES CBC encryption or decryption algo-
rithm represents only a small fraction of the power consumed
in the transmission mechanism as shown in Fig. 14. As

2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
13

2500
and scoring. After the announcement of the competition,
Encryption a duration of two weeks was given for different teams to
Decryption
2000 register in an online form. Tutorial sessions were conducted
Time (Microseconds)

remotely in the form of video-conference online meetings to

1500
help the teams know more about the testbed architecture and
1000 usage. The tutorial sessions included presentations, program
demos, and question sessions from the teams. After that, the
500
competition schedule was announced to the teams.
Since the number of the teams was greater than the avail-
0
128 192
Key Size (bits)
256 able hardware nodes, a time multiplexing plan was conducted
so that different teams have different time windows for
Fig. 15: Measured encryption and decryption time in CBC accessing the available hardware. Competitors were given a
mode with different key sizes duration of two weeks to perform all the required missions.
The competition consisted of five missions as below:
1200 • Mission no.1: Basic WiFi (60 points): This mission

1000 Encryption
required the competitors to write a code utilizing a single
Decryption
IoT node to connect it to the internet through a pre-
Energy (MicroJoules)

800
600
400
200
0 • Mission no.2: WiFi + Zigbee (70 points): This mission
128 192
Key Size (bits)
Fig. 16: Measured encryption and decryption energy con-
sumption in AES CBC mode with different key sizes
a conclusion, the previous experiments show that the IoT
node can perform complex tasks while consuming a smaller
fraction of time and energy compared to those consumed in
the communication process.
IX. A SS I U T-I OT S TUDENT C OMPETITION
A competition was organized between university-level stu-
dent teams to build different IoT networks by using AssIuT-
IoT testbed. The competition design ensured that teams from
different locations and universities have the same privilege
no matter how far they are from the testbed server as
long as they have remote access to the testbed through the
internet. Also, organizing a competition with different teams
accessing all the testbed hardware and software services is
a real benchmark for the testbed performance. The problems
which were occurred during the competition and the feedback
received from the competitors helped to improve the testbed
significantly afterwards11 . In this section, the competition
phases and missions are described and the statistical results
of the competition progress and results are discussed.
announced Service Set IDentifier (SSID) and password.
The user code should read the temperature and humidity
values through the sensor and send these values through
the WiFi to the server to be monitored periodically every
3 seconds.
256
required utilizing two nodes; a parent node and a child
node. The parent node connects to the internet through
a WiFi connection and connects to the child node
through a Zigbee connection. The child has no internet
connection and it sends its temperature and humidity
values to the parent node, which aggregates the child
data and sends it along with its own data to the control
server.
• Mission no.3: Advanced WiFi (70 points): Similar to
mission 2 except that the connection between the parent
and child node is through the WiFi module. The parent
node acts as a hotspot and sets up a local WiFi network
besides connecting to the internet WiFi network. The
child connects to the parent through the local WiFi
network to send its data. This mission was scaled up
to include more than one child node.
• Mission no.4: Execution time measurement (36
points): The mission required the competitors to use the
timer module of the MCU to measure the time required
for executing a pre-given block of code.
• Mission no.5: Security (46 points): The mission re-
quired the competitors to repeat mission 2 or 3 after
applying a 128-bit AES encryption to the messages sent
from the child node to the parent node. The parent node
should decrypt the messages before transmitting them to
the server.

B. Results, Statistics and Feedback

A. Competition design
The competition consisted of multiple phases, namely:
registration, tutorial sessions, missions tackling, evaluation,
11 The serial monitor debugging utility described in Section V-C was
developed in its final version based on the feedback received during the
competition.
Nine teams, distributed among different Egyptian univer-
sities, participated in the competition. Detailed score sheets
were created and filled by the judges to evaluate the function-
ality of the submitted codes, in addition to its reliability and
readability. Fig. 17 shows the average degrees in percentage
relative to the full mark scored by the teams in each mission.

2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
Fig. 17: Average mission’s degree relative to the full mark
X. C ONCLUSIONS
In this paper, we provided the detailed design and opera-
tion of AssIuT-IoT, a remotely accessible, federated testbed
for IoT applications. The purpose of the testbed is to provide
a practical educational tool for IoT systems through facili-
tating user access to remotely located IoT nodes. The IoT
nodes are based on the Arduino Mega board and the testbed
is controlled by a single server. The testbed design allows it
to serve a large number of resources as it uses Raspberry Pis
as intermediate relays between the server and some of the IoT
nodes instead of directly connecting all of them to the server.
Through this paper, we provided a step-by-step guidance for
one complete experiment from the user point of view. Also,
the testbed platform performance was evaluated where both
the server and the RPi were able to perform their tasks with-
out showing significant CPU utilization. Moreover, the IoT
network performance was evaluated when utilizing Zigbee or
WiFi modules to show the advantages and disadvantages of
each type of communication. Finally, we bench-marked the
testbed by holding a student competition which utilized the
testbed in all its steps.
ACKNOWLEDGMENT
This work was supported in part by the Egyptian National
Telecommunication Regulatory Authority (NTRA).
R EFERENCES
[1] M. AbdelHafeez and M. AbdelRaheem, “AssIUT IOT: a remotely
accessible testbed for internet of things,” in 2018 IEEE Global Con-
ference on Internet of Things (GCIoT). IEEE, 2018, pp. 1–6.
[2] A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and
M. Ayyash, “Internet of things: A survey on enabling technologies,
protocols, and applications,” IEEE Communications Surveys Tutorials,
vol. 17, no. 4, pp. 2347–2376, 2015.
[3] J. A. Stankovic, “Research directions for the internet of things,” IEEE
Internet of Things Journal, vol. 1, no. 1, pp. 3–9, Feb 2014.
[4] International Telecomunication Union , “Harnessing the
internet of things for global development,” [Online]. Available:
https://www.itu.int/en/action/broadband/Documents/Harnessing-IoT-
Global-Development.pdf, [Accessed: 8 April 2020].
[5] C. Adjih, E. Baccelli, E. Fleury, G. Harter, N. Mitton, T. Noel,
R. Pissard-Gibollet, F. Saint-Marcel, G. Schreiner, J. Vandaele, and
T. Watteyne, “Fit IoT-LAB: A large scale open experimental IoT
testbed,” in 2015 IEEE 2nd World Forum on Internet of Things (WF-
IoT), Dec 2015, pp. 459–464.
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.
14
[6] E. FIESTA-IoT, “Project, federated interoperable semantic IoT testbeds
and applications, grant agreement no,” CNECT-ICT-643943. Available
online: http://fiesta-iot. eu [Accessed: 8- April - 2020], Tech. Rep.
[7] L. Sanchez, J. A. Galache, V. Gutierrez, J. M. Hernandez, J. Bernat,
A. Gluhak, and T. Garcia, “SmartSantander: The meeting point be-
tween future internet research and experimentation and the smart
cities,” in 2011 Future Network Mobile Summit, pp. 1–8.
[8] M. Hossain, S. Noor, Y. Karim, and R. Hasan, “IoTbed: A generic
architecture for testbed as a service for internet of things-based
systems,” in 2017 IEEE International Congress on Internet of Things
(ICIOT), June 2017, pp. 42–49.
[9] Y. Teranishi, Y. Saito, S. Murono, and N. Nishinaga, “JOSE: An
open testbed for field trials of large-scale IoT services,” Journal of
the National Institute of Information and Communications Technology,
vol. 62, no. 2, 2015.
[10] M. Chernyshev, Z. Baig, O. Bello, and S. Zeadally, “Internet of Things
(IoT): Research, Simulators, and Testbeds,” IEEE Internet of Things
Journal, vol. 5, no. 3, pp. 1637–1647, 2017.
[11] J. Lanza, L. Sanchez, J. R. Santana, R. Agarwal, N. Kefalakis, P. Grace,
T. Elsaleh, M. Zhao, E. Tragos, H. Nguyen et al., “Experimentation as
a service over semantically interoperable Internet of Things testbeds,”
IEEE Access, vol. 6, pp. 51 607–51 625, 2018.
[12] L. Sanchez, L. Muñoz, J. A. Galache, P. Sotres, J. R. Santana,
V. Gutierrez, R. Ramdhany, A. Gluhak, S. Krco, E. Theodoridis
et al., “Smartsantander: IoT experimentation over a smart city testbed,”
Computer Networks, vol. 61, pp. 217–238, 2014.
[13] P. Sotres, J. R. Santana, L. Sánchez, J. Lanza, and L. Munoz,
“Practical lessons from the deployment and management of a smart
city internet-of-things infrastructure: The smartsantander testbed case,”
IEEE Access, vol. 5, pp. 14 309–14 322, 2017.
[14] E. M. Silva and P. Maló, “IoT testbed business model,” Advances in
Internet of Things, vol. 4, no. 04, p. 37, 2014.
[15] R. Kirichek and A. Koucheryavy, “Internet of things laboratory test
bed,” in Wireless Communications, Networking and Applications.
Springer, 2016, pp. 485–494.
[16] A. Tekeoglu and A. Ş. Tosun, “A testbed for security and privacy
analysis of IoT devices,” in 2016 IEEE 13th International Conference
on Mobile Ad Hoc and Sensor Systems (MASS). IEEE, 2016, pp.
343–348.
[17] S. Siboni, V. Sachidananda, Y. Meidan, M. Bohadana, Y. Mathov,
S. Bhairav, A. Shabtai, and Y. Elovici, “Security testbed for internet-
of-things devices,” IEEE Transactions on Reliability, vol. 68, no. 1,
pp. 23–44, 2018.
[18] A. Pötsch, “A scalable testbed infrastructure for embedded industrial
wireless sensor and actuator networks: Ph. d. forum abstract,” in
Proceedings of the 15th International Conference on Information
Processing in Sensor Networks. IEEE Press, 2016, p. 45.
[19] S. S. Hanna, A. Guirguis, M. A. Mahdi, Y. A. El-Nakieb, M. A.
Eldin, and D. M. Saber, “Crc: Collaborative research and teaching
testbed for wireless communications and networks,” in Proceedings of
the Tenth ACM International Workshop on Wireless Network Testbeds,
Experimental Evaluation, and Characterization, ser. WiNTECH ’16.
ACM, 2016, pp. 73–80.
[20] O. M. Mahmoud Mahdi and M. Elnainay, “Research and educational
federation testbed: A new architecture design for the collaborative re-
search cloud,” in International Telecommunications Society Conference
(ITS), February 2019.
[21] A. E.-R. Mohsen, M. Y. GadAlrab, Z. elhaya Mahmoud, G. Alshaer,
M. Asy, and H. Mostafa, “Remote FPGA Lab For ZYNQ and Virtex-
7 Kits,” in 2019 IEEE 62nd International Midwest Symposium on
Circuits and Systems (MWSCAS). IEEE, 2019, pp. 185–188.
[22] A. Attaby, N. Osman, M. Elnainay, and M. Youssef, “WiPi: A low-cost
large-scale remotely-accessible network testbed,” IEEE Access, vol. 7,
pp. 167 795–167 814, 2019.
[23] “DHT11 temperature and humidity sensor,” [Online].
Available: https://www.waveshare.com/wiki/DHT11 Temperature-
Humidity Sensor.
[24] “ARDUINO Mega 2560 -Rev 3,” [Online]. Available:
https://store.arduino.cc/usa/arduino-mega-2560-rev3, [Accessed:
8 April 2020].
[25] “Atmega 2560,” [Online]. Available:
https://www.microchip.com/wwwproducts/en/ATmega2560, [Accessed:
8 April 2020].
[26] “The internet of things with ESP8266,” [Online]. Available:
http://esp8266.net/, [Accessed: 8 April 2020].
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3012022, IEEE Internet of
Things Journal
15

[27] “Zigbee embedded module,” [Online]. Available:

https://www.digi.com/xbee., [Accessed: 8 April 2020].
[28] “Django web framework,” [Online]. Available:
https://www.djangoproject.com/, [Accessed: 8 April 2020].
[29] “Debian linux distribution,” [Online]. Avail-
able:https://www.debian.org/, [Accessed: 8- April 2020].
[30] “AVRDUDE- avr Downloader/UploaDEr,” [Online].
https://www.nongnu.org/avrdude/, [Accessed: 8 April 2020].
[31] “Pyserial platform,” [Online]. Available:
https://pythonhosted.org/pyserial, [Accessed: 8 April 2020].
[32] “OMF Controller,” [Online]. Available:
https://www.fibre.org.br/infrastructure/experimentation-environments/,
[Accessed: 8 April 2020].
[33] “Raspberry pi 3 model b,” [Online].
https://www.raspberrypi.org/products/raspberry-pi-3-model-b/,
[Accessed: 8 April 2020].
[34] “Raspbian operating system,” [Online]. https://www.raspbian.org/,
[Accessed: 8 April 2020].
[35] B. Fernandes, F. Silva, C. Analide, and J. Neves, “Crowd sensing for
urban security in smart cities.” J. UCS, vol. 24, no. 3, pp. 302–321,
2018.
[36] J. Granjal, E. Monteiro, and J. S. Silva, “Security for the internet of
things: a survey of existing protocols and open research issues,” IEEE
Communications Surveys & Tutorials, vol. 17, no. 3, pp. 1294–1312,
2015.
[37] “Github - kokke/tiny-aes-c: Small portable AES128/192/256 in c,”
[Online]. Available: https://github.com/kokke/tiny-AES-c, [Accessed:
8- April - 2020].

2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 04,2020 at 03:58:24 UTC from IEEE Xplore. Restrictions apply.