Edge Computing

Digital Health
Smart
Manufacturing
Careers

JUNE 2023www.computer.org
 IEEE COMPUTER SOCIETY CAREER CENTER

Evolving Career
Opportunities
Explore new options—upload your resume today

Changes in the marketplace shift demands for vital skills

careers.computer.org and talent. The IEEE Computer Society Career Center is a
valuable resource tool to keep job seekers up to date on
the dynamic career opportunities offered by employers.
Take advantage of these special resources for job seekers:

JOB ALERTS TEMPLATES WEBINARS

CAREER RESUMES VIEWED

ADVICE BY TOP EMPLOYERS

No matter what your career level, the IEEE Computer

Society Career Center keeps you connected to
workplace trends and exciting career prospects.
 IEEE COMPUTER SOCIETY computer.org

STAFF
Editor Periodicals Portfolio Senior Managers Director, Periodicals and Special Projects
Cathy Martin Carrie Clark and Kimberly Sperka Robin Baldwin

Production & Design Artist Periodicals Operations Project Specialist Senior Advertising Coordinator
Carmen Flores-Garvey Christine Shaughnessy Debbie Sims

Circulation: ComputingEdge (ISSN 2469-7087) is published monthly by the IEEE Computer Society. IEEE Headquarters, Three Park Avenue, 17th
Floor, New York, NY 10016-5997; IEEE Computer Society Publications Office, 10662 Los Vaqueros Circle, Los Alamitos, CA 90720; voice +1 714 821 8380;
fax +1 714 821 4010; IEEE Computer Society Headquarters, 2001 L Street NW, Suite 700, Washington, DC 20036.
Postmaster: Send address changes to ComputingEdge-IEEE Membership Processing Dept., 445 Hoes Lane, Piscataway, NJ 08855. Periodicals Postage
Paid at New York, New York, and at additional mailing offices. Printed in USA.
Editorial: Unless otherwise stated, bylined articles, as well as product and service descriptions, reflect the author’s or firm’s opinion. Inclusion in
ComputingEdge does not necessarily constitute endorsement by the IEEE or the Computer Society. All submissions are subject to editing for style,
clarity, and space.
Reuse Rights and Reprint Permissions: Educational or personal use of this material is permitted without fee, provided such use: 1) is not made for
profit; 2) includes this notice and a full citation to the original work on the first page of the copy; and 3) does not imply IEEE endorsement of any third-
party products or services. Authors and their companies are permitted to post the accepted version of IEEE-copyrighted material on their own Web
servers without permission, provided that the IEEE copyright notice and a full citation to the original work appear on the first screen of the posted copy.
An accepted manuscript is a version which has been revised by the author to incorporate review suggestions, but not the published version with copy-
editing, proofreading, and formatting added by IEEE. For more information, please go to: http://www.ieee.org/publications_standards/publications
/rights/paperversionpolicy.html. Permission to reprint/republish this material for commercial, advertising, or promotional purposes or for creating new
collective works for resale or redistribution must be obtained from IEEE by writing to the IEEE Intellectual Property Rights Office, 445 Hoes Lane,
Piscataway, NJ 08854-4141 or pubs-permissions@ieee.org. Copyright © 2023 IEEE. All rights reserved.
Abstracting and Library Use: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy for private use of patrons,
provided the per-copy fee indicated in the code at the bottom of the first page is paid through the Copyright Clearance Center, 222 Rosewood Drive,
Danvers, MA 01923.
Unsubscribe: If you no longer wish to receive this ComputingEdge mailing, please email IEEE Computer Society Customer Service at help@
computer.org and type “unsubscribe ComputingEdge” in your subject line.
IEEE prohibits discrimination, harassment, and bullying. For more information, visit www.ieee.org/web/aboutus/whatis/policies/p9-26.html.

IEEE Computer Society Magazine Editors in Chief

Computer IEEE Intelligent Systems IEEE Pervasive Computing

Jeff Voas, NIST San Murugesan, Western Fahim Kawsar, Nokia Bell Labs
Sydney University (Interim EIC) and University of Glasgow
Computing in Science
& Engineering IEEE Internet Computing IEEE Security & Privacy
Lorena A. Barba, George George Pallis, University Sean Peisert, Lawrence
Washington University of Cyprus Berkeley National
Laboratory and University of
IEEE Annals of the History IEEE Micro California, Davis
of Computing Lizy Kurian John, University
Ramesh Subramanian, of Texas at Austin IEEE Software
Quinnipiac University and Ipek Ozkaya, Software
Yale Law School IEEE MultiMedia Engineering Institute
Balakrishnan Prabhakaran,
IEEE Computer Graphics University of Texas at Dallas IT Professional
and Applications Charalampos Z.
André Stork, Fraunhofer IGD Patrikakis, University of
and TU Darmstadt West Attica

2469-7087/23 © 2023 IEEE Published by the IEEE Computer Society June 2023 1
JUNE 2023 • VOLUME 9 • NUMBER 6

8
Utilizing AI
16
DROO: Integrated
26
Medical Device
Planning on Learning and Forensics
the Edge Optimization for
Edge Computing
Offloading
 Edge Computing
8 Utilizing AI Planning on the Edge
ILIR MURTURI, ADAM EGYED, AND SCHAHRAM DUSTDAR

16 DROO: Integrated Learning and Optimization for Edge

Computing Offloading
QIAN ZHANG

Digital Health
20 TELECOVID: Remote Vital Signs Monitoring of
COVID-19 Risk Patients in Home Isolation With an
In-Ear Wearable
MELANIE BALDINGER, AXEL HEINRICH, TIM ADAMS, EIMO MARTENS,
MICHAEL DOMMASCH, ALEXANDER MÜLLER, ADONIA SIEGMANN, AND
GEORG SCHMIDT

26 Medical Device Forensics

VERONICA SCHMITT

Smart Manufacturing
32 Trusting Digital Twins
PHIL LAPLANTE

38 Architectural Concerns of Digital Twins

IPEK OZKAYA

Careers
42 “Well Paid for a Woman”: Gloria Hardman’s 50-Year
Career in Computing
PHILIP NEL

50 Masterminds of the NSFnet: Jennings, Wolff, and

Van Houweling
GEORGE STRAWN

Departments
4 Magazine Roundup
7 Editor’s Note: Optimizing Edge Computing
55 Conference Calendar

Subscribe to ComputingEdge for free at

www.computer.org/computingedge.
 Magazine Roundup

T he IEEE Computer Society’s lineup of 12 peer-reviewed technical magazines covers cutting-edge topics rang-
ing from software design and computer graphics to Internet computing and security, from scientific appli-
cations and machine intelligence to visualization and microchip design. Here are highlights from recent issues.

Revisiting the Spaceborne
Illuminators of Opportunity
for Airborne Object Tracking
This article from the January
2023 issue of Computer reviews
existing research for coopera-
tive passive coherent location
and tracking employing space-
borne opportunistic illuminators.
The article highlights the need
for novel, cooperative, real-time
object detection and tracking
approaches. It also discusses the
lack of multi-static experimental
data for future research.
Responsibly Reckless
Matrix Algorithms for HPC
Scientific Applications
High-performance computing (HPC)
simulations on massively parallel
systems are often driven by matrix
computations, whose rate of exe-
cution depends on their floating-
point precision. Referred to by Jack
Dongarra—the 2021 ACM A.M. Tur-
ing Award Laureate—as “respon-
sibly reckless” matrix algorithms,
the authors of this article from the
July/August 2022 issue of Comput-
ing in Science & Engineering high-
light the implications of mixed-pre-
cision (MP) computations for HPC
applications. Introduced 75 years
ago, long before the advent of HPC
architectures, MP numerical meth-
ods turn out to be paramount for
increasing the throughput of tra-
ditional and artificial intelligence
workloads beyond riding the wave
of the hardware alone.
When the New Magic
was New: The Claritas
Corporation and the
Clustering of America
In 1982, geodemographics was
“the new magic.” The system
promised to pinpoint voters. One
of its key developers, Jonathan
Robbin, boasted, “Tell me some-
one’s ZIP code, and I can predict
what they eat, drink, drive—even
think.” What was the secret? Rob-
bin and his company, the Clari-
tas Corporation, were among the
first to merge newly digitized cen-
sus data and computational social
science into a geodemographic
system called PRIZM. It produced
new categories of populations
known as clusters. These clusters
provided a novel and legible form
of computer demographics for
marketers reliant on direct mail.
Drawing on interviews and histor-
ical research, this article from the
October–December 2022 issue
of IEEE Annals of the History of
Computing chronicles the under-
studied history of the Claritas
Corporation.
Finding Their Data Voice:
Practices and Challenges of
Dashboard Users
Dashboards are the ubiquitous
means of data communication
within organizations. Yet, we
have limited understanding of
how they factor into data prac-
tices in the workplace, particu-
larly for data workers who do not
self-identify as professional ana-
lysts. The authors of this article
from the January/February 2023
issue of IEEE Computer Graphics
and Applications focus on data
workers who use dashboards
as a primary interface to data,

4 June 2023 Published by the IEEE Computer Society  2469-7087/23 © 2023 IEEE
 reporting on an interview study
that characterizes their data prac-
tices and the accompanying barri-
ers to seamless data interaction.
While dashboards are typically
designed for data consumption,
the authors’ findings show that
dashboard users have far more
diverse needs.
A Framework for
Reasoning About Uncertainty
in Ontologies
This IEEE Intelligent Systems
November/December 2022 arti-
cle introduces a logic-based argu-
mentation framework for rea-
soning under uncertainty of
description logic-based ontolo-
gies. The authors found the frame-
work on four new argumentation-
based semantics: the classical
semantics, the IAR semantics fam-
ily, the cardinality-based family,
and the agent preference-based
semantics. Within this frame-
work, multiple new inference rela-
tions can be distinguished to
draw meaningful answers to que-
ries even from imperfect ontolo-
gies. The authors investigate logi-
cal properties and the productivity
of these new entailment mecha-
nisms and their relationships with
other well-known existing logical
entailments.
www.computer.org/computingedge
The Future of HPC in
Nuclear Security
Experience spanning almost 80
years shows that qualitative leaps
in simulation capabilities have
allowed new contributions to
national security. Through detailed
quantitative analysis of represen-
tative simulations, the authors of
this article from IEEE Internet Com-
puting’s January/February 2023
issue show that reliance on tech-
nologies developed for popular
market applications will likely stall
progress. They argue that if the
United States is to continue bene-
fiting from advances in computing,
investments in deeper co-design
of hardware and software—
addressing levels of branching
and sparsity not found in machine
learning or most other major mar-
ket applications—will be needed.
Data Centers on Wheels:
Emissions From
Computing Onboard
Autonomous Vehicles
While much attention has been
paid to datacenters’ greenhouse
gas emissions, less attention has
been paid to autonomous vehi-
cles’ (AVs’) potential emissions. In
this January/February 2023 IEEE
Micro article, the authors intro-
duce a framework to probabilis-
tically model the emissions from
computing on board a global fleet
of AVs and show that the emis-
sions have the potential to make
a nonnegligible impact on global
emissions, comparable to that of
all datacenters today. Anticipating
a future scenario with high adop-
tion of AVs, business-as-usual
decarbonization, and workloads
doubling every three years, hard-
ware efficiency must double every
1.1 years for emissions in 2050 to
equal 2018 datacenter emissions.
The rate of increase in hardware
efficiency needed in many scenar-
ios to contain emissions is faster
than the current rate.
Enhanced Local and
Global Learning for
Rotation-Invariant Point
Cloud Representation
Various studies have been under-
taken to learn point cloud repre-
sentations that are both discrimi-
native and robust. However, most
of them suffer from rotation dis-
turbance and insufficient labeled
data. To solve the problem of rota-
tion disturbance, the authors of
this article from IEEE MultiMedia’s
October–December 2022 issue
5
MAGAZINE ROUNDUP

propose a novel rotation-invariant
network that is equipped with the
following core modules: enhanced
local representation learning mod-
ule and global alignment module.
The enhanced local representa-
tion learning module captures the
geometric relationship among the
neighbors defined in both 3D Car-
tesian space and a latent space to
exploit the local context and long-
distance context. The global align-
ment module is devised to address
the lack of global information and
supplement the absolute locations
of points by adaptively generating
the rotation-invariant coordinates.
An Opt-in Framework for
Privacy Protection in Audio-
Based Applications
not require retraining of recogni-
tion algorithms, making the obfus-
cated data compatible with exist-
ing methods.
Eight Lightweight
Usable Security Principles
for Developers
The authors of this IEEE Security
& Privacy article from the Janu-
ary/February 2023 issue propose
eight usable security principles
that provide software developers
with a lightweight framework to
help them integrate security in a
user-friendly way. These principles
should help developers who must
weigh usability and security trad-
eoffs to facilitate adoption.
combines multiple virtual spaces.
A novel technology like meta-
verse gives rise to various chal-
lenges such as the standing of
virtual avatars in legal systems in
case of injury or harm, demand for
expensive resources in its upscal-
ing, privacy of user data, and
increased risk of cybercrimes.
This November/December 2022
IT Professional article presents
a brief overview of technologies
used in the development of meta-
verse, the challenges, and the
potential aspects of the virtual
world.

Installing audio-based applica- Unscheduled Meetings in

tions exposes users to the risk
of the data processor extract-
ing additional information beyond
the task the user permitted. To
solve these privacy concerns, the
authors of this article from IEEE
Pervasive Computing’s October–
December 2022 issue propose to
integrate an on-edge data obfus-
cation between the audio sen-
sor and the recognition algorithm.
They introduce a novel privacy loss
metric and use adversarial learning
to train an obfuscator. Contrary to
existing work, this technique does
not require users to specify which
sensitive attributes they want to
protect (opt-out), but instead only
provide permission for specific
tasks (opt-in). Moreover, it does
Hybrid Work
By studying four development
teams in the Norwegian compa- Join the IEEE
nies NAV and Entur, the authors of
Computer
this article from the March/April
2023 issue of IEEE Software found
Society
that unscheduled meetings are computer.org/join
maintained by using tools like Dis-
cord and Zoom to create virtual
rooms. The authors provide three
recommendations for hybrid work.
Virtual Dimension—
A Primer to Metaverse
The metaverse is a concept of a
persistent, online, 3D world that

6 ComputingEdge June 2023

 Editor’s Note
Optimizing Edge Computing
E dge computing is a promising
technology that can enable
smart cities and Internet of Things
(IoT) applications to achieve
energy-efficient, low-latency, and
high-performance computing.
New research suggests that we
can extract the maximum bene-
fits of edge computing by apply-
ing artificial intelligence. This
ComputingEdge issue considers
ways to optimize edge comput-
ing through state-of-the-art tech-
niques such as deep reinforce-
ment learning.
“Utilizing AI Planning on the
Edge,” from IEEE Internet Comput-
ing, proposes a conceptual frame-
work for executing AI planners
on distributed devices in edge
networks. As a case study, the
authors simulate a waste manage-
ment problem and find the optimal
route for disposing of waste bins
in a city. Computer’s “DROO: Inte-
grated Learning and Optimization
2469-7087/23 © 2023 IEEE Published by the IEEE Computer Society June 2023
for Edge Computing Offloading”
discusses an approach for achiev-
ing optimal online provisioning of
edge services in a large IoT net-
work under fast-varying wireless
channel conditions.
In the healthcare field, medi-
cal devices can often help improve
patient outcomes. “TELECOVID:
Remote Vital Signs Monitoring of
COVID-19 Risk Patients in Home
Isolation With an In-Ear Wearable,”
from IEEE Pervasive Computing,
shows how wearables played an
important role in disease detec-
tion and monitoring during the
pandemic. The authors of IEEE
Security & Privacy’s “Medical
Device Forensics” argue that—in
light of increased hacking—med-
ical devices should be manufac-
tured with security and incident
response in mind.
In smart manufacturing, dig-
ital twins are used to optimize
production systems. Computer’s
“Trusting Digital Twins” consid-
ers whether emerging digital
twin technology will provide the
desired operational functionality
along with an acceptable level of
quality. The authors of IEEE Soft-
ware’s “Architectural Concerns
of Digital Twins” encourage orga-
nizations beyond those in manu-
facturing to adopt digital twins to
assist with operations and main-
tenance in cyberphysical systems.
This ComputingEdge issue
concludes with two articles that
detail the careers of computing
pioneers. “’Well Paid for a Woman’:
Gloria Hardman’s 50-Year Career
in Computing,” from IEEE Annals
of the History of Computing, pro-
vides a biography of an influential
programmer and computer edu-
cator. IT Professional’s “Master-
minds of the NSFnet: Jennings,
Wolff, and Van Houweling” high-
lights the team that created a pre-
cursor to the Internet.
7
 EDITOR:Schahram
EDITOR: SchahramDustdar,
Dustdar,dustdar@dsg.tuwien.ac.at
dustdar@dsg.tuwien.ac.at
This article originally
appeared in
DEPARTMENT:
DEPARTMENT:INTERNET
INTERNET OF
OF THINGS, PEOPLE, AND PROCESSES
PEOPLE, AND PROCESSES vol. 26, no. 2, 2022

Utilizing AI
Utilizing AI Planning
Planning on
on the
the Edge
Edge
Ilir Murturi , Adam Egyed, and Schahram Dustdar , TU Wien Vienna, Vienna 1040, Austria
Ilir Murturi , Adam Egyed, and Schahram Dustdar , TU Wien Vienna, Vienna 1040, Austria

The convergence between AI planning techniques and the Internet of Things (IoT) can
solve
The various operational
convergence betweenand business techniques
AI planning challenges. However, IoT systems’
and the Internet stringent
of Things (IoT) can
requirements
solve such as latency
various operational andand scalability
business have introduced
challenges. However,several challenges
IoT systems’ to
stringent
execute and scale
requirements suchplanners
as latencyin cloud environments.
and scalability haveEdge computers
introduced placed
several close to the
challenges to execute
and
IoT scale
domainplanners in cloud
(e.g., sensors) can environments. Edge
be leveraged for computersplanners
implementing placed close to the IoT domain
and overcoming
(e.g., sensors)
scalability can We
issues. be leveraged for implementing
propose a conceptual frameworkplanners and overcoming
highlighting executing scalability
issues. We propose
Expressive NumericaHeuristic
conceptual framework
Search Planner onhighlighting
distributedexecuting
devices in Expressive Numeric
edge networks.
Heuristic Search Planner on distributed devices in edge networks. As a proof
As a proof of concept, we develop a simulator to show the applicability and feasibility of concept,
of
we develop a simulator to show the applicability and feasibility of
running planners on the edge. As a case study, we simulate a waste management running planners on
the edge. As a case study, we simulate awaste management problem and find the optimal
problem and find the optimal route for disposing of? waste bins in a city. Throughout
route for disposing of? waste bins in a city. Throughout the experiments, the user can
the experiments, the user can discover insightful information regarding the planner’s
discover insightful information regarding the planner’s applicability on the edge.
applicability on the edge.

T
he Internet of Things (IoT) has widely dissemi-
nated into society, and many services in various
industries are built on top of IoT technologies.
Managing and controlling a high number of heteroge-
neous devices (i.e., mobile devices, sensors, etc.) is
becoming increasingly complex. On the other hand,
the evergrowing number of devices challenges the
cloud-centric environments to process incoming sen-
sory data streams within the critical time-frame.1,2 In
this context, processing IoT data streams over distrib-
uted computation entities and gaining authority over
a smaller segment of the large network is critical to
achieving various system goals. These distributed
computation entities are usually referred to as edge
devices. Essentially, edge devices allow for prepro-
cessing incoming sensory data streams as well as pro-
vides a seamless opportunity to deploy multiple edge
applications (i.e., IoT applications) aiming at providing
low-latency services for end-users. For instance, a
neighborhood may be composed of hundreds of net-
worked devices providing various sensory information
about the environment. An edge device in proximity
This work is licensed under a Creative Commons Attribution
4.0 License. For more information, see https://creativecom-
mons.org/licenses/by/4.0/
Digital Object Identifier 10.1109/MIC.2021.3073434
Date of current version 16 May 2022.
can act as an intermediary computation device to pro-
cess and store the incoming data. Besides that, an
edge device may decide whether the pumped IoT data
must be processed at the edge network or forwarded
to a cloud infrastructure.
In recent years, urban growth became an endemic
problem for large cities in the world. In cities with rapid
population growth and with high number of visitors, daily
public services or public safety services should operate
efficiently. In this sense, IoT resources (i.e., sensors,
actuators, etc.) distributed over the city have been seen
as critical to improve the quality of life. For example,
within a city district, several smart devices provide real-
time information about the environment to the closest
edge device benefiting from high connectivity to it and
its awareness of other resources in its surroundings. An
edge device may act as an authority to coordinate avail-
able resources in its district and becomes responsible
for processing data generated by the smart devices in
the field. At the same time, various edge applications
(e.g., waste management service) can be deployed on
edge, process sensory data, and provide useful informa-
tion to the end-users. However, not all services can be
trivially obtained from IoT resources—for instance, opti-
mizing the energy efficiency of street lighting systems in
a neighborhood when no citizen is walking. In that case,
several combinations of other IoT devices’ resources
must be considered to achieve the desired goal.3 Such a
goal can be achieved through utilizing AI planning tech-
niques, which are well-known approaches developed to

8 28 June 2023
IEEE Internet Computing Published by the IEEE
Published Computer
by the SocietySociety
IEEE Computer  2469-7087/23 © 2023
March/April 2022IEEE
 INTERNET OFOF
INTERNET THINGS, PEOPLE,
LISTING 1. Domain-Predicates.
1 (:predicates
2 (truck ?t - vehicle)
3 (bin ?b - wastebin)
4 (node ?n - location)
5 (street ?s - street)
6 (plant ?p - facility)
7 (is-at ?p ?n - location)
8 (connected ?s - street ?n - location)
9 (truck-at ?t - vehicle ?n - location)
10 (plant-at ?p - facility ?n- location)
11 (emptied ?b - wastebin)
12 (empty ?t - vehicle))
solve planning autonomously and without human inter-
vention. Essentially, the planning is defined as the task of
coming up with a sequence of actions that will achieve a
user goal.
AI planning techniques aim to address NP-hard
problems. The bigger the problem instance is, the more
time and resources are required to find an optimal
solution. In this sense, utilizing the planners in the con-
text of IoT and smart cities becomes an increasingly
complex as well as resource-intensive process.4 Exe-
cuting AI planning techniques on the cloud is naturally
possible. However, in the context of IoT such techni-
ques do not scale very well. In addition, the bandwidth
of the networks that carry IoT sensory data streams to
and from the cloud has not increased appreciably.
Essentially, the network bottleneck causes higher
latencies than expected response time for IoT systems.
Considering the above-mentioned challenges, we argue
that Edge Computing can help to tackle the scalability
problem by splitting the problem, and also overcome
networks bottleneck by processing sensory data on
the edge. Therefore, this calls for novel resource man-
agement approaches, resource coordination, and the
need to investigate the feasibility and the applicability
of AI planning techniques executed in a distributed
manner on the edge. We refer to the applicability
aspect by considering real-life scenarios and their con-
crete requirements to achieve user goals. On the other
hand, we refer to the feasibility aspect to show the per-
formance requirements to execute AI planning models
on edge infrastructure.
In this article, we propose a conceptual framework
that highlights executing Expressive Numeric Heuristic
LISTING 2. Domain-Functions.
1 (:functions
2 (total-cost)
3 (distance ?s - street)
4 (capacity ?t - vehicle)
5 (wastebin-status ?b - wastebin)
6 (wastebin-capacity)
7 (max-capacity ?t - vehicle))
www.computer.org/computingedge
March/April 2022
THINGS, AND
PEOPLE, ANDPROCESSES
PROCESSES
LISTING 3. Problem skeleton.
1 (define (problem smart-waste)
2 (:domain smart-waste)
3 (:objects %1$)
4 (:init %2$)
5 (:goal %3$)
6 (:metric minimize (total-cost))
Search Planner (ENHSP)a on distributed edge devices
in edge networks. As a proof of concept, we develop a
simulator to simulate the waste management problem
to find the optimal route for the disposal of waste bins
in a city. Subsequently, we discuss the motivation and
reasons for choosing to model the waste management
problem as well as explain the syntax required to model
both the domain and the actual problem for the ENHSP
planner. Furthermore, the simulator is configurable and
provides functionalities that enable the user to interact
at runtime. Throughout the experiments, the user can
discover insightful information regarding the planner’s
applicability on the edge, such as performance req-
uirements when executing the planner in different city
areas with different parameter settings, and also opti-
mizing the overall waste disposal process.
BACKGROUND AND RELATED
WORK
In the past few years, researchers from academia and
industry have been focused on utilizing the power of
AI planning techniques in different contexts in edge-
based systems (i.e., IoT systems). Several approaches
discussed in the literature are based on AI plan-
ning techniques such as enabling engineering
resource coordination at runtime,3 energy-efficient
task offloading,5 deploying self-adaptive IoT systems,6
and adaptation of goal-driven IoT systems.7 AI-Plan-
ning consists of several developed techniques to solve
planning and scheduling problems autonomously and
without human intervention. Essentially the planning
is defined as the task of coming up with a sequence of
actions that will lead from an initial state to a specified
goal state. A goal can be represented as a single goal
state (i.e., predicate) or a complex goals states (i.e.,
multiple goal states) that can either be true or false.
To reach a goal state, the planner is provided with
actions that can alter these states. These actions may
define preconditions that need to be met to change
the states of the problem according to the action’s
effect. For each task, the planner requires the domain
a
The ENHSP Planning System, https://sites.google.com/view/
enhsp/
IEEE Internet Computing 29 9
 INTERNETOF
INTERNET OFTHINGS,
THINGS, PEOPLE,
PEOPLE, AND
AND PROCESSES
PROCESSES
FIGURE 1. An overview of the conceptual framework.
and problem models as inputs (discussed in the sec-
tion “Scalability and Performance”).
Engineering edge-based systems is challenging,
partly due to the heterogeneity, dynamicity, and uncer-
tainty of the devices. Tsigkanos et al.3 proposed a goal-
driven approach for engineering resource coordination
at runtime. The approach adopts goal modeling to cap-
ture objectives opportunistically at runtime and with-
out any operational status knowledge. Bounded model
checking is used as the foundational technique to com-
pute coordination plans that satisfy device goals.
Essentially, it considers dependencies among IoT
things to achieve a particular goal. Alkhabbas et al.7
proposed an approach for enabling the automated for-
mation and adaptation of goal-driven IoT systems by
exploiting context-awareness and AI-planning techni-
ques. On the contrary to the mentioned works, while
focusing primarily on AI planning techniques, we delve
more into showing the feasibility and emphasize the
applicability of AI planning techniques executed in a
distributed manner on the edge. As the case study, we
consider an age-old matter and well-known waste
management problem. We developed a configurable
simulator, which is interactable at runtime. The simula-
tor provides functionalities to the user to create
diverse use-cases for the ENHSP planner to solve.
MOTIVATION
Providing an efficient waste disposal service in
crowded and highly frequented areas (i.e., squares,
parks, etc.) by visitors remains a prime concern for
metropolitan cities. This is because the cleaner cities
are, the more tourists they attract. Parks for instance
may become visitor spots because of their cleanliness,
view, and fresh air. On the other side, waste disposal
operators traditionally collect waste from bins on a
fixed schedule (e.g., hourly or daily). However, the
number of visitors and citizens roaming in the various
city areas changes frequently. Such public spaces can
get crowded very fast, meaning that the waste bin fill-
ing rate may increase drastically. In such situations,
1030
ComputingEdge
IEEE Internet Computing
the fixed schedules become ineffective while driving
to an almost empty garbage can cause wasting of
resources and money.
To provide real-time scheduling, we assume that
several smart vehicles (i.e., smart trucks) may operate
continuously around the city areas. Through the inte-
grated IoT sensors, each waste bin provides sensory
information to the closest edge device responsible for
a particular area in the city (i.e., neighborhood, square,
park, or district). On the other side, edge devices are
preconfigured regarding the maximum waste level
allowed in bins as well as the truck capacity. For
instance, when the waste level in bins is above the pre-
defined threshold, a responsible edge device places a
lease on the closest available truck to empty all bins in
the responsible authority radius. However, we will face
an NP-hard problem when considering several factors
(i.e., streets, bins, trucks, etc.) to provide most optimal
routes for the trucks operating in the city. Therefore,
when utilizing AI planning techniques and processing
sensory data on the edge, a crucial aspect is to mea-
sure performance issues in different scenarios.
UTILIZING AI PLANNING ON THE
EDGE
Conceptual Framework
In the Smart City context, we may have multiple com-
plex problems, which can be solved by using planners.
More precisely, this means that for each task, we have
different planning domains and planning require-
ments. Thus, this calls for a new framework that allows
developers or system administrators to deploy models
on available devices at the edge. To overcome the
mentioned challenges, we introduce a conceptual
framework that enables easy deployment and opera-
tion of planners on the edge infrastructure. The con-
ceptual framework is presented in Figure 1.
The proposed framework comprises two main
parts: a cloud-based IoT platform and edge device core
functions. The platform consists of three main modules
i) Runtime Orchestration, ii) Smart City Layouts, and
iii) AI Planning Models. The runtime orchestration is
responsible for providing an environment, where it
allows planners’ execution in heterogeneous environ-
ments. The platform can interact with the edge system
components (i.e., resource manager) to access, govern,
and orchestrate planners at the edge. For instance,
Dockerb containers are a way to wrap up a planner into
its own isolated package. Additionally, the runtime
b
Docker, https://docker.com
June
March/April 20222023
 INTERNET OFOF
INTERNET THINGS, PEOPLE,
THINGS, AND
PEOPLE, ANDPROCESSES
PROCESSES

orchestration manages edge devices, automates con- Thus, the simulator requires the geospatial informa-
tainer provisioning, networking, load-balancing, secu- tion in a format called shapefile.
rity, and scaling across the distributed edge devices.
The cloud layer assumes to contain various plan-
Use Case: Waste Management Domain
ning models that a system administrator can upload on
Modeling
distributed edge devices. An edge device may run mul-
As outlined in section , PDDL is comprised of planning
tiple planners (i.e., ENHSP instances) depending on the
domain model and problem description files. The
device resources. Furthermore, models are written
domain describes the planning world in a general
using the Planning Domain Definition Language
sense, which contains the actions and predicates.
(PDDL), which is comprised of planning domain model
Essentially, it defines which objects are existent, the
and problem description. Planning domain model and
actions that can alter these objects’ state, including
problem description are essential files required to
the preconditions that need to be fulfilled, and the
instantiate a planner (see section “Technology and
effects of these actions on the objects. The domain
Social Constraints”). In addition to the models, the
file does not define the set amount of objects (i.e.,
cloud platform may contain simple city layouts (i.e.,
bins or trucks), neither does it define the planning
maps) or accurate digital architectural models.8
problem’s goal.
The second part of the framework represents edge
In Listing 1, we show the exact syntax for defining
device core functionalities. The Resource Manager is
the domain problem. Most of the predicates are self-
responsible for monitoring the infrastructure-specific
explanatory, such that objects are vehicles, bins,
metrics such as edge devices’ resource capabilities
streets, etc. In order to derive the city structure from
(i.e., hardware). In addition, the module implements an
the shapefile retrieved from the OpenGov data, we
IoT resource discovery mechanism9 and various com-
introduce the notion of whether a node is connected
munication protocols (i.e., Bluetooth, ZigBee, LoRa,
to a street or not. The rest of the predicates express
WiFi, etc.) to connect with various surrounding IoT
positioning on the map alongside the ones telling the
devices. On the other hand, the AI Planning module
planner whether a waste bin has been emptied or a
executes ENHSP instances. The module listens contin-
vehicle is empty after visiting a waste disposal facility
uously for new requests from the system administrator
and regaining full capacity.
to start new solver instances. Essentially, after the
With the :fluents requirement, we are able to create
required files (i.e., domain and problem model) are
numeric predicates to keep track of the capacity of the
uploaded, a new container-based service is started at
vehicle or the current status of the waste bins or assign
the edge device.
a numeric cost called distance for travelling along a
street (see Listing 2). The most important function is
Smart City Layout the (total-cost) one that is getting modified after every
As outlined in the motivation section, we consider the movement of the truck and essentially keeps track of
waste management problem by deploying multiple the plan’s current cost. This is the metric that will ulti-
edge devices and coordinating available resources mately need to be optimized by the planner. Further-
(i.e., trucks) in the entire city. Before starting with the more, vehicle move actions and preconditions
problem modeling in PDDL, first it is required to relay considered in the problem are rather straightforward
and document the process of finding the right underly- (i.e., pick up bin, move truck, etc.). Further modeling
ing data-structure for the city itself. Generally speak- details can be found.10
ing, the waste management problem is consolidated
to Nodes (waste bins), Streets connecting them with a
Problem Skeleton
given distance (representing the cost of the route), and
The problem description file defines the initial and the
one or multiple trucks. However, to visualize the pro-
goal states. Thus, once the format of the domain is
cess and enable user interaction with the simulator,
set, the problem file is initialized with the actual part
we require a static city representation with wide-
of the city that needs to be planned including nodes,
spread usage implementation. To overcome such a
streets, facilities, waste bins, and the trucks. To initiate
challenge, we utilize an open-source java library called
the planner, we require the planning domain and prob-
GeoToolsc to read, parse, and display the datasets.
lem as two static files. However, edge devices receive
real-time status values from smart devices, and these
c
GeoTools The Open Source Java GIS Toolkit, https:// data need to be translated and inputted into the plan-
geotools.org/ ner continuously. Thus, to build our simulator and

www.computer.org/computingedge
March/April 2022 IEEE Internet Computing 31 11
 INTERNETOF
INTERNET OFTHINGS,
THINGS, PEOPLE,
PEOPLE, AND
AND PROCESSES
PROCESSES
FIGURE 2. Overview of the three-step configuration process.
enable the user to interact with it, we generate the
static file on the fly. This calls for creating a skeleton
of the actual problem file, populated by real-time val-
ues and edge configuration parameters (see Listing 3).
As can be noted in Listing 3, the %1$ part is where
the actual initialization occurs. Essentially, it gets built
as a string in the application itself and uses string
replacement to populate the placeholders. Ultimately
this file gets written to the file system and the planner.
Afterward, the planner receives its input for the prob-
lem definition. Furthermore, to cut down generation
times, the static part of the problem is generated as
soon as an edge device is placed on the map. Given
the edge device’s authority radius, it is possible to pre-
generate the city structure that needs to be planned
to speed up the generation process. This means that
only the values that are changing over time (i.e., the
bin status and the truck’s position) need to be proc-
essed before the planning can start.
Another issue that may arise is when the smart
truck is out of the edge device’s authority radius. An
edge device can call the closest available smart truck
in the city. Solving such a challenge within the problem
skeleton becomes unfeasible in performance terms
when the truck is too far away (i.e., the entire city needs
to be considered). To overcome such a challenge, we
create another problem skeleton intending to move
the smart truck within the caller radius (i.e., edge
radius) in the most optimal way possible.
SIMULATOR IMPLEMENTATION
Workflow
To show the applicability and feasibility of executing
planners on edge networks, we develop a simulator to
simulate the waste management problem.10 The cur-
rent version of the prototype is written in Java and
provides basic functionalities to enable resource coor-
dination (i.e., between edge devices and trucks) and
executing multiple planners on the simulated edge
devices. The end-user can interact with the simulator
at runtime, configure it, and simulate various scenar-
ios. An overview of the three-step configuration pro-
cess of the simulator is presented in Figure 2.
1232
ComputingEdge
IEEE Internet Computing
Smart City Layouts
We first choose a city that provides data that can be
used in conjunction with the ENHSP planner. For
instance, the City of Vienna, which is part of the Coop-
eration Open Government Data Austriad provides a
vast amount of static and real-time data spanning
from general street structure to real-time construction
works. The above-mentioned site provides an API for
developers to use such data free of charge for their
purposes. Nevertheless, since the aim is to develop a
standalone version of our prototype, downloading the
geospatial information in a format called shapefile ful-
filled the need for offline usage.
Configure Edge Device
The end-user can place edge devices anywhere on the
map. Prior to its placing on the map, an edge device is
essentially configured with the parameters such as
authority radius, number of bins, bin capacity, etc. The
waste bins are distributed randomly inside the edge
devices authority zone. For each edge device placed on
the map, a new server instance is started on a specific
port to generate sensory data for the simulator (i.e.,
explained in the following section). The waste bins posi-
tion on the map are generated randomly. Moreover,
trucks operate in one of two possible modes: i) available
and ii) not-available. The available mode represents the
state when edge devices can place a lease on the truck.
The not-available mode represents the state when
the truck is leased by another edge device. Besides that,
the end-user configures trucks with the following
parameters such as truck capacity, speed, etc. Each
edge device can query at anytime the state of trucks
placed on the map.
Simulate
The end-user can place trucks at any given place on the
map. After the main components are set, the user can
either set a fixed interval for plan generation or do it on
demand. The plan generation occurs in two steps. First,
the edge device places a lease on the truck, and it cal-
culates the fastest route to the closest point in its
authority radius. The truck then moves there, and in
the second step, the planner generates the most opti-
mal route to empty all the bins in the edge device’s
authority. Once these bins are emptied, the truck is
released, and other edge devices can apply for a lease
on the said vehicle. The entire process (i.e., planning
and executing the plans) can thus be automated and
d
Cooperation Open Government Data Austria, https://www.
data.gv.at/
June
March/April 20222023
 INTERNET OFOF
FIGURE 3. An overview of the simulator.
evaluated over a longer period of time. An overview of
the simulator is presented in Figure 3.
Simulating Sensory Data
The time until a waste bin is full usually varies depend-
ing on population density and some other variables.
For instance, according to the City of Vienna,e the bins
are emptied from once a day to six times a week. To
generate waste bins sensory data, we developed a
Python-based web server that allows the planner to
query actual sensor status and create new instances
when the user adds new edge devices on the map.
With generating pseudorandom numbers between
0 and 1 for each bin, we get the fill rate function
fðxÞ ¼ 0:6 þ x3 � ð4:167 � 0:6Þ. Every random number
generated gives us an average of fð0:5Þ � 1:045% per
day. Moreover, waste bins are full until an average of
3.98 days and make it somewhat closer to the situa-
tion we encounter in real life (see Figure 4).
EXPERIMENTS
Throughout the experiments, the user can discover
insightful information regarding the planner’s app-
licability on the edge, such as i) performance req-
uirements when executing the planner in different
city areas with different parameter settings and
e f
Vienna Waste Disposal Frequency, https://bit.ly/39qpEIN
www.computer.org/computingedge
March/April 2022
INTERNET THINGS, PEOPLE,
THINGS, AND
PEOPLE, ANDPROCESSES
PROCESSES
ii) optimizing the waste disposal process. First, we
show the planner performance aspects when scaling
by the number of trucks considered to generate a sin-
gle plan. The planner will thus try to find the optimal
emptying plan by using one or multiple trucks avail-
able. Besides that, we compare the performance scal-
ing by the number of bins considered for generating a
single plan. Second, we explain other experiments
that the user can realize within the current simulator
version.
To monitor resource utilization during the runtime
process, we use a tool called VisualVM,f which displays
the performance statistics of a specific Java Virtual
Machine instance in real time. Figure 5 presents the
planner performance when scaling with increasing the
number of trucks assigned to a single edge device. We
consider a city area that consists of 129 streets, 8 bins
placed within the 71 nodes on the map. As can be
noted, with increasing the number of trucks, the aver-
age planning time increases as well. The average plan-
ning time and memory consumption remain in the
acceptable range with three trucks; however, the plan-
ner will not scale well when the number of trucks
increases. Nevertheless, edge devices are considered
resource-constrained devices, and creating multiple
edge devices with a single truck responsible for
VisualVM, https://visualvm.github.io/
IEEE Internet Computing 3313
 INTERNETOF
INTERNET OFTHINGS,
THINGS, PEOPLE,
PEOPLE, AND
AND PROCESSES
PROCESSES
FIGURE 4. Fill-rate graph of waste bins.
FIGURE 5. Scaling by vehicle count.
FIGURE 6. Scaling by number of bins.
emptying bins at a time overcomes the scalability
challenges introduced by the planner.
Figure 6 presents the planner performance when
scaling with increasing the number of bins assigned to
a single edge device. We consider a similar city area as
in the previous experiment. On the contrary to the first
experiment, we consider a single truck and the various
number of bins. As can be noted, increasing the num-
ber of bins in the problem instance results in an aver-
age planning time increase. Similarly, as in the first
experiment, the planner will not scale well when the
number of bins considered in the problem instance is
high. Therefore, the user may concretely discover the
planner’s resource requirements to run it in a particular
region of a city.
Since we are subdividing the city into small self-
governing areas, determining optimal truck count and
capacity may determine the overall operating cost. For
1434
ComputingEdge
IEEE Internet Computing
instance, regularly operating multiple smaller capacity
trucks with lower operating costs can be more cost-
efficient than having fewer high-capacity trucks cover-
ing a larger area. With the parameters revolving around
the truck count, we can immediately notice the drop in
bins emptied per km driven. Therefore, we can empiri-
cally put the cost of dispatching a garbage truck in
shorter intervals versus the cost of doing longer routes
but emptying more waste bins along the way.
CONCLUSION
The developed simulator is the initial attempt at
designing a system for executing AI planning tech-
niques in a distributed manner on the edge. The
current version is in the early stage of development
and provides essential functionalities to simulate
the waste management problem. Besides that,
edge devices are simulated, and the planner instan-
ces for each edge device placed on the map are
executed on the same machine. However, when
placing multiple edge devices on the map, the sim-
ulator may face performance issues. In our future
work, we plan to extend the prototype with a mod-
ule through which developers and users can easily
configure edge devices and connect them to their
physical entities.
entities.Furthermore,
Furthermore,we weplan to extend
plan the
to extend
the simulator with other AI models supporting
simulator with other AI models supporting various vari-
ous Smart
Smart City City problems.
problems. Finally,
Finally, we to
we plan plan to per-a
perform
form a more extensive evaluation of
more extensive evaluation of the approach. the approach.
ACKNOWLEDGMENTS
This work was supported in part by the “Smart Commu-
nities and Technologies (Smart CT)” and it has received
funding from the EU’s Horizon 2020 Research and Inno-
vation Programme under grant agreement No. 871525.
EU web site for Fog Protect: https://fogprotect.eu.
REFERENCES
1. W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge
computing: Vision and challenges,” IEEE Internet Things
J., vol. 3, no. 5, pp. 637–646, Oct. 2016.
2. S. Dustdar and I. Murturi, “Towards distributed edge-
based systems,” in Proc. IEEE 2nd Int. Conf. Cogn.
Mach. Intell., 2020, pp. 1–9.
3. C. Tsigkanos, I. Murturi, and S. Dustdar, “Dependable
resource coordination on the edge at runtime,” Proc.
IRE, vol. 107, no. 8, pp. 1520–1536, Aug. 2019.
4. M. Ghallab, D. Nau, and P. Traverso, Automated
Planning: Theory and Practice. Amsterdam, The
Netherlands: Elsevier, 2004.
June
March/April 20222023
 INTERNET OFOF
INTERNET THINGS, PEOPLE,
THINGS, AND
PEOPLE, ANDPROCESSES
PROCESSES

5. J. Zilic, A. Aral, and I. Brandic, “EFPO: Energy efficient and 10. U. A. P.On the Edge (The Waste Management Problem,
failure predictive edge offloading,” in Proc. 12th IEEE/ACM 2021. [Online]. Available: https://hyde.infosys.tuwien.ac.
Int. Conf. Utility Cloud Comput., 2019, pp. 165–175. at/utilizing-ai-planning-on-the-edge.
6. F. Alkhabbas, I. Murturi, R. Spalazzese, P. Davidsson,
and S. Dustdar, “A goal-driven approach for deploying
self-adaptive IoT systems,” in Proc. IEEE Int. Conf. ILIR MURTURI is currently working toward the Ph.D. degree
Softw. Archit., 2020, pp. 146–156. in edge computing under the supervision of Prof. S. Dustdar
7. F. Alkhabbas, R. Spalazzese, and P. Davidsson, “Eco-IoT:
with the Distributed Systems Group, TU Wien, 1040, Vienna,
An architectural approach for realizing emergent
Austria. Contact him at imurturi@dsg.tuwien.ac.at.
configurations in the Internet of Things,” in Proc. Eur.
Conf. Softw. Archit., 2018, pp. 86–102.
8. P. M. Lerones, J. L. Ferna  M. Gil, J. Go
ndez, A.  mez- ADAM EGYED received the B.Sc. degree in software and infor-
García-Bermejo, and E. Z. Casanova, “A practical mation engineering from the TU Wien, Vienna, Austria, in 2020.
approach to making accurate 3D layouts of interesting Contact him at adam.egyed@protonmail.com.
cultural heritage sites through digital models,” J.
Cultural Heritage, vol. 11, no. 1, pp. 1–9, 2010.
9. I. Murturi, C. Avasalcai, C. Tsigkanos, and S. Dustdar, “Edge- SCHAHRAM DUSTDAR is a professor of Computer Science
to-edge resource discovery using metadata replication,” in with the Distributed Systems Group, TU Wien, 1040, Vienna,
Proc. IEEE 3rd Int. Conf. Fog Edge Comput., 2019, pp. 1–6. Austria. Contact him at dustdar@dsg.tuwien.ac.at.

ADVERTISER INFORMATION

Advertising Coordinator Central US, Northwest US, Southeast US, Asia/Pacific:

Eric Kincaid
Debbie Sims Email: e.kincaid@computer.org
Email: dsims@computer.org Phone: +1 214-553-8513 | Fax: +1 888-886-8599
Phone: +1 714-816-2138 | Fax: +1 714-821-4010 Cell: +1 214-673-3742

Midwest US:
Advertising Sales Contacts Dave Jones
Email: djones@computer.org
Mid-Atlantic US: Phone: +1 708-442-5633 Fax: +1 888-886-8599
Dawn Scoda Cell: +1 708-624-9901
Email: dscoda@computer.org
Phone: +1 732-772-0160
Cell: +1 732-685-6068 | Fax: +1 732-772-0164 Jobs Board (West Coast and Asia), Classified Line Ads

Southwest US, California: Heather Bounadies

Mike Hughes Email: hbuonadies@computer.org
Email: mikehughes@computer.org Phone: +1 623-233-6575
Cell: +1 805-208-5882

Northeast, Europe, the Middle East and Africa: Jobs Board (East Coast and Europe), SE Radio Podcast
David Schissler
Email: d.schissler@computer.org Marie Thompson
Phone: +1 508-394-4026 Email: marie.thompson@computer.org
Phone: +1 714-813-5094

www.computer.org/computingedge
March/April 2022 IEEE Internet Computing 3515
 EDITOR: Ron Vetter, University of North Carolina Wilmington, vetterr@uncw.edu
DEPARTMENT: SPOTLIGHT ON TRANSACTIONS
DROO: Integrated Learning
and Optimization for Edge
Computing Offloading
Qian Zhang, Hong Kong University of Science and Technology
This installment of Computer’s series highlighting the work published in IEEE
Computer Society journals comes from IEEE Transactions on Mobile Computing.
E
dge computing is recognized as a key
enabling technology for Internet of Things
(IoT) applications to achieve energy-efficient,
ultralow-latency, and high-performance computing
service beyond 5G/6G. IoT devices of limited comput-
ing power and onboard battery life can significantly
improve their data processing capability by offloading
intensive computation tasks to a nearby edge server
for remote execution. Constrained by the available
computation and communication resources in an
edge access network, efficient provisioning of edge
computing services requires judiciously deciding the
set of IoT devices allowed to offload as well as the
assets allocated to them based on real-time edge
parameters. Achieving optimal online provisioning
of edge services is challenging in a large IoT network
under fast-varying wireless channel conditions. In
particular, it requires repeatedly solving mixed-integer
nonlinear programming (MINLP) problems, each to be
finished within a channel coherence time of at most
several seconds. Conventional model-based integer
optimization methods are too slow in their numerical
iterations to search large solution vector spaces, while
model-free learning-based methods suffer from slow
learning convergence or even divergence in finding an
optimal hybrid integer-continuous online policy.
In “Deep Reinforcement Learning for Online Com-
putation Offloading in Wireless Powered MobileEdge
Digital Object Identifier 10.1109/MC.2021.3055923
Date of current version: 17 November 2021
16 June 2023 Published by the IEEE Computer Society 
This article originally
appeared in
vol. 54, no. 12, 2021
Computing Networks,” Huang et al.1 propose a novel
integrated optimization and learning approach, deep
reinforcement learning-based online offloading
(DROO). That can provision close-to-optimal comput-
ing services that are adaptive to fast-varying wireless
channels. As shown in Figure 1, instead of solving for
the hybrid integer-continuous solution altogether,
DROO decomposes the original optimization problem
into a zeroone binary offloading decision subproblem
TO BETTER APPROXIMATE THE
SUBMANIFOLD, THEIR METHOD
CONSIDERS ONLY A FEW BASIS
MODES IN THE VICINITY OF THE
DESIRED DEFORMATION.
and a continuous resource allocation subproblem and
then tackles them separately through a model-free
learning module and a model-based optimization mod-
ule, respectively. In each time slot, the actor module
uses a fully connected deep neural network (DNN)
to map the input instant system parameters to mul-
tiple binary offloading action vectors, and the critic
module optimizes the resource allocation for each
action vector and selects the best one. DROO uses the
selected binary offloading action and the correspond-
ing resource allocation as the control decision in the
current time slot and then stores the selected binary
action in a replay buffer for updating the policy of the
DNN in the future. With a new input channel condition
2469-7087/23 © 2023 IEEE
 Model-Free Model-Based
Actor Module Critic Module

Relaxed Selected
Action Resource
Action
Instant System Allocation Real-Time
Parameters and Decision
Argmax

Quantized
Deep Neural Network Actions

Train Neural Networks

FIGURE 1. A schematic of DROO.1

in the next time slot, the update of the DNN model
parameters repeats until converging to the optimal
online policy.
Compared with model-based optimization, DROO
takes negligible computation time, as it can directly
map any new channel input to the optimal output
control action without the need for numerical opti-
mization. On the other hand, compared with conven-
tional deep reinforcement learning (DRL) methods
that treat both the integer offloading and continuous
resource allocation decisions as the action, DROO
significantly reduces the action space of the DNN,
which greatly simplifies the learning task to a classi-
cal classification problem. Perhaps more importantly,
the optimization-based critic module provides precise
evaluation of the integer offloading decisions gener-
ated by the actor module. This greatly improves the
convergence of the training process as compared with
conventional DRL, whose convergence is often jeop-
ardized by the inaccurate evaluation of actions before
the critic network is sufficiently trained. Simulation
results show that DROO quickly converges in fewer
than 3,000 iterations in a 30-user network, and the
provisioned computing service achieves 99.9 of the
optimum in less than 0.1 s after convergence.
B esides edge service provisioning, DROO has
important applications in a wide range of areas,
such as wireless communications and industrial con-
trol, where MINLP instances need to be frequently
MODEL-FREE LEARNING-BASED
METHODS SUFFER FROM SLOW
LEARNING CONVERGENCE OR EVEN
DIVERGENCE IN FINDING AN OPTIMAL
HYBRID INTEGER-CONTINUOUS
ONLINE POLICY.
resolved. In addition, its decoupled structure facili-
tates simple modifications to handle more compli-
cated use cases. For instance, by slightly modifying the
critic module with a Lyapunov optimization technique,
DROO can effectively tackle stochastic MINLP with
long-term performance constraints2; by replacing the

www.computer.org/computingedge 17
SPOTLIGHT ON TRANSACTIONS

DNN in the actor module with a convolutional neural QIAN ZHANG is the Tencent Professor of Engineering and
network, recursive neural network, or state-of-the-art chair professor in the Department of Computer Science and
transformer, DROO can improve its convergence Engineering, Hong Kong University of Science and Technol-
performance and handle more sophisticated learn- ogy, Hong Kong, China. Contact him at qianzh@cse.ust.hk.
ing tasks. In this sense, DROO makes it truly viable to
obtain real-time and optimal solutions to a variety of
hard MINLP problems in fast-varying environments.

REFERENCES
1. L. Huang, S. Bi, and Y. J. Zhang, “Deep reinforcement
learning for online computation offloading in wireless
powered mobile-edge computing networks,” IEEE
Trans. Mobile Comput., vol. 19, no. 11, pp. 25812593, Nov.
2020. doi: 10.1109/TMC.2019.2928811
2. S. Bi, L. Huang, H. Wang, and Y. J. Zhang,
“Lyapunov-guided deep reinforcement learning for
stable online computation offloading in mobile-edge
computing networks,” IEEE Trans. Wireless Commun., WWW.COMPUTER.ORG/COMPUTINGEDGE
2021, arXiv:2010.01370.

CG& A IEEE Computer Graphics and Applications bridges the theory

and practice of computer graphics. Subscribe to CG&A and
www.computer.org/cga

• stay current on the latest tools and applications and gain

invaluable practical and research knowledge,
• discover cutting-edge applications and learn more about
the latest techniques, and
• benefit from CG&A’s active and connected editorial board.

18 ComputingEdge June 2023

 EXECUTIVE COMMITTEE
President: Nita Patel
President-Elect: Jyotika Athavale
Past President: William D. Gropp
www.computer.org First VP: Hironori Washizaki
Second VP: Grace A. Lewis
Secretary: Carolyn McGregor
PURPOSE: Engaging professionals from all areas of
Treasurer: Michela Taufer
computing, the IEEE Computer Society sets the standard for
VP, Membership & Geographic Activities: Fernando Bouche
education and engagement that fuels global technological
VP, Professional & Educational Activities: Deborah Silver
advancement. Through conferences, publications, and
programs, IEEE CS empowers, guides, and shapes the future Interim VP, Publications: Greg Byrd
of its members, and the greater industry, enabling new VP, Standards Activities: Annette Reilly
opportunities to better serve our world. VP, Technical & Conference Activities: Grace A. Lewis
OMBUDSMAN: Direct unresolved complaints to 2023–2024 IEEE Division VIII Director: Leila De Floriani
ombudsman@computer.org. 2022–2023 IEEE Division V Director: Cecilia Metra

CHAPTERS: Regular and student chapters worldwide provide 2023 IEEE Division V Director-Elect: Christina M. Schober
the opportunity to interact with colleagues, hear technical
experts, and serve the local professional community. BOARD OF GOVERNORS
Term Expiring 2023:
AVAILABLE INFORMATION: To check membership status,
report an address change, or obtain more information on any Jyotika Athavale, Terry Benzel, Takako Hashimoto, Irene Pazos
of the following, email Customer Service at help@computer. Viana, Annette Reilly, Deborah Silver
org or call +1 714 821 8380 (international) or our toll-free Term Expiring 2024:
number, +1 800 272 6657 (US): Saurabh Bagchi, Charles (Chuck) Hansen, Carlos E. Jimenez-
• Membership applications Gomez, Daniel S. Katz, Shixia Liu, Cyril Onwubiko
• Publications catalog
Term Expiring 2025:
• Draft standards and order forms
İlkay Altintaş, Nils Aschenbruck, Mike Hinchey, Joaquim Jorge,
• Technical committee list
Rick Kazman, Carolyn McGregor
• Technical committee application
• Chapter start-up procedures
• Student scholarship information EXECUTIVE STAFF
• Volunteer leaders/staff directory
Executive Director: Melissa Russell
• IEEE senior member grade application (requires 10 years
practice and significant performance in five of those 10) Director, Governance & Associate Anne Marie Kelly
Executive Director:
Director, Conference Operations: Silvia Ceballos
PUBLICATIONS AND ACTIVITIES Director, Information Technology & Sumit Kacker
Computer: The flagship publication of the IEEE Computer Services:
Society, Computer, publishes peer-reviewed technical Director, Marketing & Sales: Michelle Tubb
content that covers all aspects of computer science, Director, Membership Development: Eric Berkowitz
computer engineering, technology, and applications. Director, Periodicals & Special Projects: Robin Baldwin
Periodicals: The Society publishes 12 magazines, 19 journals.
Conference Proceedings & Books: Conference Publishing IEEE BOARD OF DIRECTORS
Services publishes more than 275 titles every year.
President & CEO: Saifur Rahman
Standards Working Groups: More than 150 groups produce
President-Elect: Thomas M. Coughlin
IEEE standards used throughout the world.
Director & Secretary: Forrest (Don) Wright
Technical Communities: TCs provide professional interaction
Director & Treasurer: Mary Ellen Randall
in more than 30 technical areas and directly influence
Past President: K. J. Ray Liu
computer engineering conferences and publications.
Director & VP, Educational Activities: Rabab Ward
Conferences/Education: The society holds more than 215
Director & VP, Publication Services & Sergio Benedetto
conferences each year and sponsors many educational
Products:
activities, including computing science accreditation.
Director & VP, Member & Geographic Jill Gostin
Certifications: The society offers three software Activities:
developer credentials.
Director & President, Standards Yu Yuan
Association:
COMPUTER SOCIETY OFFICES Director & VP, Technical Activities: John Verboncoeur
Director & President, IEEE-USA: Eduardo Palacio
Washington, D.C.: Los Alamitos:
2001 L St., Ste. 700, 10662 Los Vaqueros Cir.,
Washington, D.C. 20036-4928; Los Alamitos, CA 90720;
Phone: +1 202 371 0101; Phone: +1 714 821 8380;
Fax: +1 202 728 9614; Email: help@computer.org
Email: help@computer.org

MEMBERSHIP & PUBLICATION ORDERS

Phone: +1 800 272 6657; Fax: +1 714 821 4641;
Email: help@computer.org revised 6 February 2023
 EDITORS:Oliver
EDITORS: OliverAmft,
Amft, oliver.amft@fau.de
oliver.amft@fau.de
Kristof Van Kristof Van Laerhoven,
Laerhoven, kvl@eti.uni-siegen.de
kvl@eti.uni-siegen.de This article originally
appeared in

DEPARTMENT:
DEPARTMENT:WEARABLE
WEARABLE COMPUTING
vol. 20, no. 2, 2021

TELECOVID: Remote Vital Signs

TELECOVID: Remote Vital Signs Monitoring
Monitoring
of COVID-19of COVID-19
Risk Risk
Patients Patients
in Home in
Isolation
Home Isolation
With an With an In-Ear Wearable
In-Ear Wearable
Melanie Baldinger , Axel Heinrich, and Tim Adams, Cosinuss GmbH , Munich, 81379 , Germany
Melanie Baldinger , Axel Heinrich, and Tim Adams, Cosinuss GmbH , Munich, 81379 , Germany
Eimo Martens , Michael Dommasch, Alexander Müller , Adonia Siegmann , and Georg Schmidt,
Eimo Martens
Technische , Michael
Universität Dommasch,
Munich, Alexander
Munich, Mu€ ller , Adonia Siegmann , and Georg Schmidt, Technische
80333, Germany
Universitat Munich, Munich, 80333, Germany
€

The TELECOVID study (https://www.telecovid.de) is designed to remotely monitor

The TELECOVID study (https://www.telecovid.de) is designed to remotely monitor the
the physiological health status of COVID-19 positive-risk patients in home isolation.
physiological health status of COVID-19 positive-risk patients in home isolation. Key vital
Key vital parameters are measured day and night using an in-ear biosensor technol-
parameters are measured
ogy. These data day and
are streamed night
to the using
clinic an time
in real in-ear
tobiosensor technology.
enable timely These
interaction in data
are streamed to the clinic
case of deterioration. in real time to enable timely interaction in case of deterioration.

T
he main goal of the study at the Klinikum rechts
der Isar (MRI) of the Technical University of
Munich (TUM) is to detect the transition into
the potentially life-threatening course of the disease
at an early stage and, thereby, improve the survival
chances of the patients. To this end, we continuously
record relevant vital parameters, such as blood oxy-
gen saturation, core body temperature, respiration
rate, and heart rate, using the cosinuss remote vital
signs monitoring system and the in-ear wearable
cosinuss Two. Recent research has shown that wear-
ables can play an important role in disease detection1
and monitoring2 during the COVID-19 pandemic. Espe-
cially digital health platforms for remote patient moni-
toring and managing this pandemic are needed.3 The
aim of this article is to make the pandemic outbreak
more manageable and to keep up a functioning
healthcare system.
In total, 95 patients (n = 95) participated in the
study so far. We instructed participants to use the
wearable as much as possible during day and night for
the time of their home isolation. Inclusion criteria are
as follows.
› COVID-19 patients over 60 years of age with con-
firmed diagnosis, which do not require intensive
care or mechanical ventilation at the time of
inclusion.
› Immunosuppressed COVID-19 patients of all
ages.
All participants signed a declaration of consent
provided by the MRI, TUM. This article was approved
by the local medical ethics committee (Ethikkommis-
sion der Technischen Universita
€t Munich).

REMOTE VITAL SIGNS

ABOUT THIS ARTICLE
Health professionals monitor the patients during
home isolation day and night and, in the event of criti-
cal deterioration, patients are immediately admitted
to the clinic.
1536-1268  2021 IEEE
Digital Object Identifier 10.1109/MPRV.2021.3066825
Date of current version 25 May 2021.
MONITORING SYSTEM
The cosinuss remote vital signs monitoring system
(https://www.cosinuss.com/en/portfolio-items/study-
on-remote-patient-monitoring-for-covid-19) is used for
data collection and remote monitoring of COVID-19
patients in home isolation. It consists of a wearable in-
ear device for monitoring vital signs (cosinuss Two) a
data gateway (cosinuss LabGateway) and a cloud
server database (cosinuss LabServer). The components
are shown in Figure 1. The cosinuss Two is an in-ear
wearable incorporating a photoplethysmographic (PPG)

2058
June 2023
IEEE Pervasive Computing PublishedPublished
by the IEEE
byComputer Society  Society
the IEEE Computer 2469-7087/23 © 2023
April-June 2021IEEE

FIGURE 1. Components of the remote vital signs monitoring system. (a) Wearable in-ear device for monitoring vital signs
remotely: cosinuss Two. (b) Data gateway, cosinuss LabGateway, receives the data from the in-ear wearable and transmits it
to the cloud server, cosinuss LabServer. (c) Example dashboard of the LabServer showing pseudonymized patients with
example vital signs data and deviation scores.
sensor with red and infrared LEDs, a contact tempera-
ture sensor, and a 3D accelerometer.
The in-ear wearable device collects physiological
parameters from the patient and sends it to the LabGa-
teway. The PPG and other vital signs data are transmit-
ted with a proprietary protocol including lossless data
compression using Bluetooth Low Energy. The LabGa-
teway, which runs a Python application on a Debian-
based Linux OS, stores the data—also offline—and
transmits it to the cosinuss LabServer via REST proto-
col. For optimal performance and scalability, MongoDB
is used as a backend cloud database in combination
with Redis, an in-memory data storage application, and
the web framework Flask. Data management is based
on proprietary logic to efficiently store, transmit, and
process the vast amounts of raw data accumulating
during a patients monitoring period.
The web interface of the LabServer provides a
comprehensive dashboard of all monitored patients.
Latest vital parameters can be tracked and data visu-
alization with meaningful plots supports healthcare
professionals with identifying critical patients.
MEASURING AT PATIENTS’ HOMES
Patients that gave their written consent to participate
in the study, receive the in-ear wearable, cosinuss data, the following four vital signs are calculated: core
Two, and a LabGateway from a courier that is part of
the study team at MRI.
The cosinuss remote vital signs monitoring sys-
tem is then set up as convenient and easy-to-use as
possible: The cosinuss Two is placed inside the outer
ear canal. The LabGateway is plugged into a socket. It
is a plug-and-play device, requiring minimum user
interaction. The wearable device automatically starts
measuring PPG data and vital signs and sends the col-
lected data to the LabGateway. The collected data are
www.computer.org/computingedge
April-June 2021
WEARABLE
WEARABLECOMPUTING
COMPUTING
then transmitted via Wi-Fi or the mobile network to
the cloud server, where a group of students that is
supervised by physicians have access to all monitored
vital data and further information, e.g., the measure-
ments and trends over the last days. This information
supports the study team in determining the urgency
of the respective health situation and the necessity of
hospital admission. The health professionals at the
MRI call the patients at home at least once a day as
well as in critical situations to inquire about the gen-
eral condition and the well-being of the patient.
Additionally to the real-time data display, all data
can be downloaded for further analysis using the
cosinuss REST API. The data flow is shown in Figure 2.
All health data on the cloud server are pseudony-
mized. Only authorized personnel have access to the
data. The data transfer is encrypted.
DATA COLLECTION AND
PROCESSING
The cosinuss Two in-ear wearable collects the follow-
ing data: Optical pulse wave (PPG signal) with a sam-
pling rate of 200 Hz, core body temperature with a
sampling rate of 1 Hz and 3D acceleration of the head
with a sampling rate of 100 Hz. Based on these raw
body temperature, heart rate, blood oxygen saturation
(SpO2 ), and respiration rate.
We designed and provide an indicator for the PPG
signal’s quality additionally to the vital data, indicating
the reliability of the calculated heart rate of the current
measurement. If the signal quality indicator (Q) drops
below a set threshold, the calculated values should be
scrutinized by the health professionals. This quality indi-
cator enables them to determine whether the wearable
is correctly adjusted in the ear canal and whether it is
IEEE Pervasive Computing 5921
WEARABLECOMPUTING
WEARABLE COMPUTING

FIGURE 2. Data flow from the wearable device through the data gateway to the cloud server, where data can be displayed at
hospitals or health authorities and data are provided through the API for downloading or integrating the data in already existing
data infrastructures.

able to determine reliable measurements. Additionally, decision makers in case of deterioration.6 This is in line
the perfusion index (PI) is an indicator for the quality of with the recommendations of Liao et al.7
the measurement of SpO2 . PI is the ratio of pulsatile
blood flow to nonpulsatile blood flow measured using INTERMEDIATE RESULTS
pulse oximetry.4 The quality indicator as well as the PI
From the total of 95 patients, 7 dropped out of the
are particularly important for the health professionals
study early due to various reasons ranging from pain
monitoring the patients. The in-ear wearable device
in the ear due to long wearing times of the sensor in
measures the vital signs every 15 min for a period of 3
three cases, daily phone calls from the clinical person-
min. After the 3-min measurement period, all data are
nel being too stressful in two cases and no specified
sent to the cloud server and displayed on the dashboard
reasons in two cases.
in the web interface. For SpO2 , the maximum value over
Patients were monitored between 1 and 21 days,
the 3-min measurement is displayed; for the respiration
with an average duration of 8.2 days (median = 8).
rate, heart rate, core body temperature, and the quality
Patients recorded their vital data using the cosinuss
indicator, the median value over the last 3 min is shown;
Two in-ear wearable on average 12.9 h per 24 h
and for the PI, the minimum value is shown. These
(median = 13.3), recording time ranging between 1.3
measurement cycles can be configured individually. For
and 21.1 h (assuming that 3-min data acquisition corre-
this article, we optimized the measurement intervals for
sponds to 15-min usage time). The actual wearing time
a 24-h runtime from one battery charge; thus, the
of the device might potentially be longer since we did
cosinuss Two wearable only needs to be charged 1 h per
not include the time when patients were outside of
24-h monitoring. Furthermore, the battery status and the
the Bluetooth range of the gateway, which means no
time that has passed since the last measurement are dis-
data can be recorded and transmitted. Therefore, the
played on the dashboard for each patient. Additionally to
time when the patient is outside of Bluetooth range
the periodic recordings, a vital parameter measurement
corresponds to “not wearing the device.” At peak
can be triggered on demand through the web interface.
times, 17 patients were monitored in parallel.
A physiological deviation score based on the four
Based on the continuous monitoring, 9 patients
vital parameters (heart rate, SpO2 , respiration rate, and
have been hospitalized so far. Of these, one had to be
core body temperature) is calculated for each measure-
ventilated (noninvasively) and one put into an artificial
ment. This score indicates an overall deviation from the
coma after hospital admission.
normal physiological health state of a person. The cal-
culation is similar to the National Early Warning Score
(NEWS) system5: A deviation is given to each parameter LESSONS FROM THE FIELD STUDY
as they are measured, with the magnitude reflecting the High data quality especially during the night and a low
deviation from the norm. The deviation of each of the dropout rate of 7.7% strongly indicate the feasibility
four vital parameters is added up to calculate the over- and usability of the solution to use in remote patient
all deviation score, which is indicated on the dashboard monitoring applications.
(low, intermediate, and high deviation). The Royal Col- Talking and chewing affects the measurements.
lege of Physicians suggest using the National Early The setting in this article is different from a station-
Warning Score system when managing patients with ary hospital setup. Patients at home are moving
COVID-19 to ensure timely assessment by clinical around within their flat or house and engage in

22
60
ComputingEdge
IEEE Pervasive Computing June
April-June 20212023
 WEARABLE
WEARABLECOMPUTING
COMPUTING

parameters, the PPG raw data (see Figure 3) helps

them to better understand if the signal is disturbed
or can be relied on when interpreting the vital
parameters. In addition to motion artifacts, PI, and,
therefore, the quality of the SpO2 measurement is
furthermore influenced by various factors including
blood circulation, ambient temperature, body posi-
tion (sitting, standing, or lying), or skin color.
In-ear fit is important. Another important factor
is the size of the wearable itself. A good and tight
fit of the in-ear wearable device inside the outer
ear canal is essential to get high-quality data. Par-
ticipants decide which size (S, M, L) fits using a
specifically designed measuring tool without previ-
ous experience and only little guidance on how to
apply it. An ill-fitting device can considerably impair
the measurement quality, which is reflected in low
quality values.
These promising results suggest the cosinuss
remote vital signs monitoring system to be a reason-
able solution to tackle COVID-19 crisis by keeping
patients safely monitored at home.

We would like to acknowledge and express thanks to
the whole TELECOVID study team.
FIGURE 3. Varying quality of PPG raw data signals derived
from the cosinuss Two depending on the patients’ activity
level. (a) Undisturbed PPG. (b) Mildly disturbed PPG. (c)
Severely disturbed PPG.
activities of daily living. Patients’ activities at home
can affect the measurement quality (see Figure 3).
First research has already been done in the area of
gait and movement classification using the in-ear
wearable and has shown promising results.8 Further
research in the area of movement detection would
be an additional benefit for remote monitoring of
patients. However, measurements of heart rate and
blood oxygen saturation using the in-ear wearable
are strongly influenced by talking or chewing due
to the movement in the jaw. Highest accuracy and
reliability of the measurements were observed
when the patient was in a resting position without
talking or eating.
PI and Q help to determine if data are reliable.
The health professionals at the clinic use the PI
and Q parameters to determine if the recorded
data point is reliable or if the measurement is
affected by motion artifacts. Additionally to these
ACKNOWLEDGMENTS
REFERENCES
1. O. Amft, L. Lopera, P. Lukowicz, S. Bian, and P. Burggraf,
“Wearables to fight COVID-19: From symptom tracking
to contact tracing,” IEEE Ann. Hist. Comput., vol. 19,
no. 4, pp. 53–60, Oct.–Dec. 2020, doi: 10.1109/
MPRV.2020.3021321.
2. A. Natarajan, H.-W. Su, and C. Heneghan, “Assessment of
physiological signs associated with COVID-19 measured
using wearable devices,” NPJ Digit. Med., vol. 3, no. 1,
pp. 1–8, 2020, doi: 10.1038/s41746-020-00363-7.
3. D. R. Seshadri et al., “Wearable sensors for COVID-19: A
call to action to harness our digital infrastructure for
remote patient monitoring and virtual assessments,”
Front. Digit. Health, vol. 2, pp. 1–11, 2020, doi: 10.3389/
fdgth.2020.00008.
4. A. Lima and J. Bakker, “Noninvasive monitoring of
peripheral perfusion,” in Appl. Physiology Intensive Care
Medicine, Springer, Berlin, Heidelberg. 2006, doi: 10.1007/
3-540-37363-2_26.
5. Royal College of Physicians, “National early warning
score (NEWS) 2,” 2017. Accessed: Feb. 1, 2021. [Online].
Available: https://www.rcplondon.ac.uk/projects/
outputs/national-early-warning-score-news-2

www.computer.org/computingedge
April-June 2021 IEEE Pervasive Computing 6123
WEARABLECOMPUTING
WEARABLE COMPUTING

6. Royal College of Physicians, “NEWS2 and deterioration
in COVID-19,” 2020. Accessed: Feb. 1, 2021. [Online].
Available: https://www.rcplondon.ac.uk/news/news2-
and-deterioration-covid-19
7. X. Liao, B. Wang, and Y. Kang, “Novel coronavirus
infection during the 2019-2020 epidemic: Preparing
intensive care units—The experience in Sichuan
Province, China,” Intensive Care Med., vol. 46, no. 2,
pp. 357–360, 2020, doi: 10.1007/s00134-020-05954-2.
8. C. P. Burgos et al., “In-ear accelerometer-based
sensor for gait classification,” IEEE Sensors J., vol. 20,
no. 21, pp. 12895–12902, Nov. 2020, doi: 10.1109/JSEN.
2020.3002589.
MELANIE BALDINGER is currently a Project Manager for
research projects at cosinuss (Cosinuss GmbH), Munich,
Germany. Contact her at melanie.baldinger@cosinuss.com.
AXEL HEINRICH is currently a Data Engineer with cosinuss
(Cosinuss GmbH), Munich, Germany. Contact him at axel.
heinrich@cosinuss.com.
EIMO MARTENS is currently a Senior Physician of the TELE-
COVID study with the Department of Internal Medicine I, Kli-
nikum rechts der Isar, Munich, Germany. Contact him at
eimo.martens@mri.tum.de.
MICHAEL DOMMASCH is currently a Senior Physician of the
TELECOVID study with the Department of Internal Medicine
I, Klinikum rechts der Isar, Munich, Germany. Contact him at
michael.dommasch@mri.tum.de.
€
ALEXANDER MULLER is currently a Software Engineer with
the Biosignal Analysis Working Group, Klinikum rechts der
Isar, Munich, Germany. Contact him at alexander.mueller@-
tum.de.
ADONIA SIEGMANN is currently the Project Manager of the
TELECOVID study with the Department of Internal
Medicine I, Klinikum rechts der Isar, Munich, Germany. Con-
tact her at adonia.siegmann@tum.de.

GEORG SCHMIDT is currently the Clinical Director of the

TIM ADAMS is currently the Head of Software Development TELECOVID study with the Department of Internal Medicine
with cosinuss (Cosinuss GmbH), Munich, Germany. Contact I, Klinikum rechts der Isar, Munich, Germany. Contact him at
him at tim.adams@cosinuss.com. gschmidt@tum.de.

IEEE COMPUTER SOCIETY

Call for Papers
Write for the IEEE Computer
Society’s authoritative
computing publications
and conferences.

GET PUBLISHED
www.computer.org/cfp

2462
ComputingEdge
IEEE Pervasive Computing June
April-June 20212023
IEEE
Computer
Society Has
You Covered!
WORLD-CLASS CONFERENCES — Stay
ahead of the curve by attending one of our
189+ globally recognized conferences.

DIGITAL LIBRARY — Easily access over

893k articles covering world-class peer-
reviewed content in the IEEE Computer
Society Digital Library.

CALLS FOR PAPERS — Discover

opportunities to write and present your
ground-breaking accomplishments.

EDUCATION — Strengthen your resume

with the IEEE Computer Society Course
Catalog and its range of offerings.

ADVANCE YOUR CAREER — Search the

new positions posted in the IEEE Computer
Society Career Center.

NETWORK — Make connections that count

by participating in local Region, Section,
and Chapter activities.

Explore all of the member benefits

at www.computer.org today!
 EDITORS: Pavel Gladyshev, pavel.gladyshev@ucd.ie,
Joshua James, joshua.i.james@pm.me
CYBERCRIME AND FORENSICS
Medical Device Forensics
Veronica Schmitt, Noroff University
Traditionally, medical devices were built with a focus on clinical care, not security. As
health care moves to Industry 4.0, practitioners need to evolve and determine what
digital forensics and incident response entail when dealing with medical devices.
I
n 2020, the world was turned on its head. With the
pandemic not showing any signs of regressing,
the medical industry has had to adapt quickly
to ensure that patients receive the necessary care.
More than ever before, physicians needed to be able
to monitor their patients while at home. This brought
about what is referred to as “Health Care 4.0,”1 derived
from “Industry 4.0.”
This article details how health care has moved to
form part of the digital frontier. This change meant
that the push for interconnected medical devices was
no longer a pipe dream but had to be made a reality
much faster. It became a necessity for a physician to
have real-time access to patients’ data.
Generally, the medical industry has not had the
best track record when it comes to incorporating
fast-paced changes. As we introduce these instru-
ments into our hospital infrastructure and patients’
homes, the age-old question is asked: “Can these
devices be hacked?” Although this is a valid concern,
it would be far better to determine the likelihood that
a medical device will be hacked. The easy answer to
this is probably one that every digital forensics prac-
titioner is familiar with: “It depends.” Many factors are
involved—for example, how the device is connected,
how the network is segmented, and the manufactur-
ing specifications of the device.
It is often proposed that the likelihood of a medi-
cal device being hacked is not very high.9 When one
Digital Object Identifier 10.1109/MSEC.2021.3127490
Date of current version: 25 January 2022
26 June 2023 Published by the IEEE Computer Society 
This article originally
appeared in
vol. 20, no. 1, 2022
examines the threat landscape around health care,
the probability starts looking much greater. In 2020,
there were an alarming number of incidents in which
health-care information was exposed, with 24.1 mil-
lion records being disclosed in one occurence.4 These
events represent an alarming 91.2 of all breached
records for that year.
Health-care data are a significant source of con-
cern. Generally, within health care, it takes anywhere
between 96 and 236 days to detect and recover from
an IT breach.9 Overall, health care has seen an increase
of 25 in the frequency of breaches in the sector.5
Hacking and IT incidents accounted for about 67 of all
data breaches and 92 of all data breaches for 2020.4
These numbers indicate that health care, as a whole,
is under attack. When dealing with interconnected
instruments, medical devices could potentially be a
stepping-stone toward obtaining access to a hospital
network or electronic health-care records.
There have been numerous hcal devices. It should
be no surprise that these devices, like any other sys-
tem, have flaws that could lead to a more significant
compromise. According to the Center for Devices
and Radiological Health,2 security researchers
identified 12 vulnerabilities, which they named
SweynTooth. These particular vulnerabilities were
associated with the wireless communication tech-
nology Bluetooth Low Energy. This communication
technology allows two devices to effectively pair and
perform their intended function without excessively
impacting battery life. The potential attacks that
could stem from this vulnerability would crash the
device, stopping the communication from working
2469-7087/23 © 2023 IEEE

and causing the device to freeze, much like a denial
of service. This would leave the device unable to
respond and allow researchers to bypass security
to access functions on the device that otherwise
would not be available. There are many manufactur-
ers affected by this vulnerability. 2
The question is whether these devices have log-
ging capabilities to offer early detection or even
allow postmortem investigations to occur. From the
research and examinations I have done, this would
seem not to be the case. The focus should be on early
detection and mitigation for vulnerabilities like this.
There have been other suggestions made in the indus-
try relating to managing the manufacturing material
and code libraries used in terms of introducing the
software bill of materials, which certainly helps
manufacturers be in a position to identify whether
a vulnerability impacts their devices. However, this
does not solve the problem of determining whether a
device has been compromised by a vulnerability and
exploited.3
When I have these conversations with patients and
medical personnel, I am often asked whether the risk
outweighs the benefit of these devices. There is no
satisfactory answer here, except it depends on many
variables. One cannot argue that medical devices
stand between life and death for patients. I have had
an implantable device that ensures that my heart
functions as it should and does not stop. This device
has added many years to my life expectancy.
The concern when dealing with medical devices
has always been one of legacy. When a device is
manufactured, it goes through many checks and tests
to ensure that it is safe and built appropriately. It is
essential to understand that security is not a func-
tional requirement of a medical device. Functional
requirements are defined as necessary functions of
the system or its components. These are described
as a specification of the behavior between the inputs
www.computer.org/computingedge
©SHUTTERSTOCK.COM/VEKTORJUNKIE
and outputs. Medical devices are manufactured to
last for an extended period and apply treatment to
patients. On a functional level, they were not built
with security in mind.
As a result, some devices were the most advanced
at one point. However, as time has moved on, they
have become vulnerable due to the progression of
technology. Many of the more traditional medical
devices cannot dynamically evolve in the way that
software might. The medical device software is
built on hardware that might be embedded within a
patients’ body and so cannot be updated. As soft-
ware and firmware grow over time and become more
sophisticated, they surpass the capabilities of the
physical hardware. In 10 years, we will still be deal-
ing with legacy devices, which are the biggest pitfall
when it comes to traditional incident response and
digital forensics.
Forensic readiness is a term that has many defini-
tions, partially because it is relatively new. Unclear
definitions lead to confusion when discussing how
forensic readiness can be achieved in medical devices.
There is a balance to forensic readiness, specifically,
the ability to collect credible digital evidence while
reducing the costs to perform digital forensics on
a medical device. 8 To determine whether a medical
device has forensic readiness maturity means that
the digital forensics practitioner or manufacturer
has to determine what information is stored on the
file system.
The first step is to understand what hardware
architecture is used on a particular device. Hardware
is very diverse across a manufacturer. The functional-
ity and firmware support determine its selection. Next,
we should consider the file system, which is the link
between the hardware and software components. It is
the guiding principle for storing the data and recording
the file creation, deletions, and modification. These
could even apply to devices that do not have a file
27
CYBERCRIME AND FORENSICS
Evidence
Locations
Cloud
Application
Operating System
File System
Physical
Hardware
FIGURE 1. The medical device locations of the investigation.
system but function as “bare-metal devices” These
can be likened to memory-based devices, which pose
many challenges to digital forensics.
Other file systems in medical devices are ones that
most digital forensics practitioners are already famil-
iar with. The file system I have encountered the most
is the File Allocation Table (FAT), which has unique
features that allow easier data recovery. The Exten-
sible FAT (exFAT) is the next most common, a UNIX file
system with its own rules that govern data storage.
The next area of understanding in the layers of
medical devices is the operating systems; these vary
across devices, manufacturers, and many other fac-
tors. One you can expect to encounter is VxWorks
(https://www.windriver.com/products/vxworks), an
embedded real-time operating system. You might
even encounter embedded Microsoft Windows or
Windows CE, which has since been deprecated. How-
ever, as previously discussed, medical devices last for
many years. An investigator might encounter medical
device forensics that includes the Android and iOS
operating systems. These, again, have forensic arti-
facts a digital forensics practitioner is accustomed
to. We could also encounter some operating systems
that are custom versions of Linux or BusyBox (https:
//busybox.net/).
We then proceed to the application layer, which
contains the information generated by the application
itself. This area is unique to each application regard-
ing what is created in terms of artifacts. However, how
data are stored is governed by the operating and file
systems. Knowing the forensics rules surrounding
these components enables the digital forensics prac-
titioner to digest the application level’s information. It
is critical to record the application functionality, the
28 ComputingEdge
settings or calibration of the device, patient data, and
clinical information. The data from these devices in
terms of clinical and health data are often integrated
with different cloud solutions at the health-care facil-
ity and connected with the electronic health-care sys-
tems.4 The question is whether there are logs that can
be ingested from medical devices into cloud solutions
for log aggregation (Figure 1).
When a medical device is compromised, several
things are needed to respond to and remediate the
incident. The first is whether the medical device
manufacturer, hospital, or patient is in a position to
detect the breach. Since 2015, reports indicate that
early detection in all industries has been challeng-
ing. 2 Medical devices are not manufactured with
breach detection in mind. Presently, however, the
landscape is changing, with medical manufacturers
exploring security by design and including this as part
of their operating models.
When we talk about medical device forensics,
there is no clear definition in the literature. The field is
made up of three main disciplines, called the triangle
approach, as seen in Figure 2.13 There is also a wide
variety of medical devices. We explore some in more
detail, as this is important to understand the problem
statement. Infusion pumps make up more than half of
all of the medical Internet of Things devices deployed
within hospitals. They play a critical role in patient
care and ensuring that the hospital controls the infu-
sion of medicine. These pumps have given medical
professionals the ability to care for patients remotely
and automate some health-care delivery. This type
of technology invites malicious threat actors inter-
ested in disrupting these life-sustaining advances in
medicine. 5
In January 2020, a security vulnerability was pub-
lished involving certain GE Healthcare clinical infor-
mation central stations and telemetry servers.4 These
devices are specifically used in health-care facilities
to display information about patients, including phys-
iological parameters, such as temperature, heart-
beat, blood pressure, for monitoring. A statement
made by the U.S. Food and Drug Administration says
that, to date, there have been no reports of incidents
related to these vulnerabilities.4 The question that
should be raised here, the same as before, is whether
these devices can retain information forensically,
June 2023
 CYBERCRIME AND FORENSICS

allowing possible malicious access

Medical Device Engineering
through this vulnerability. When To Debug
explored, concerns around logging Digital Forensics
and forensic data indicate that To Preser ve, Examine, and Analyze
the maturity of forensic readiness Medical Forensics
To Determine Clinical Information
within medical devices is very low.

MEDICAL
DEVICE FORENSICS
Medical device forensics is made FIGURE 2.The medical device forensics triangle.
up of three distinct processes:

› medical device engineering
› digital forensics
› medical forensics.
These three disciplines cover the skill sets needed
when looking at medical devices, which are often
are composed of complex hardware, software, and
data structures. Medical device engineering is the
process of designing, manufacturing, and maintain-
ing a medical device. An understanding of the hard-
ware and operating system is needed. This process is
also used extensively to examine whether a medical
device functions as it should and does fault finding.
It is useful when considering the process of acquiring
a forensic image from a medical device, as this often
requires an understanding of the hardware and con-
straints you might face dealing with these devices.7
Knowledge of device functionality can aid the
digital forensics investigation by determining what
condition the medical device was in during acquisi-
tion. The second part of what needs to be understood
is the sequence of events on the device and whether
there are any data that need to be recovered.
The discipline of digital forensics involves recon-
structing events on a digital device and recovering
data that may have been lost. Data recovery is the
second portion of examining the forensic image
that you have acquired. Recovery and reconstruc-
tion require a low-level understanding of the disk
structures and file systems used to store the data
on a medical device. It is important to note that a file
system should not be confused with an operating
system. Some medical devices, from a digital foren-
sics point of view, look more like what we know in
traditional memory forensics.
Once the sequence of events has been estab-
lished, one more aspect needs to be looked at: the
actual recorded medical data. Recorded medical
data should be examined to reconstruct events and
determine the overall medical sequence of them. This
analysis requires knowledge of the medical nature of
the device and patient. It is essential to understand
that these devices vary in how data can be acquired,
the data they store, and the medical information they
record. Some devices run on bare-metal systems and
are primarily memory based. The analysis of these
can be time-consuming and cumbersome. They are
also often implanted within a patient, making the
data more complicated to acquire. With implanted
devices, one cannot simply plug something in to make
a forensic acquisition, as there is a person attached
to the device.
Informal medical device forensics is not a new dis-
cipline; it has been done by manufacturers for years
when receiving back faulty devices. They have been
fault finding and determining why a device failed as
part of the medical engineering portion. It is possible
to analyze a medical device to provide an overall pic-
ture of its functionality, the sequence of events, and
the recorded medical information.
Important to note is that most medical devices
do not contain robust logging capabilities designed
for incident response. I have a saying: ”When noth-
ing goes right, just go left;“ this way, you build what
you do not have. The landscape should change from
simply hoping that medical devices will not be com-
promised. Instead, manufacturers must build in the
detection and forensic evidence needed to prove
that is the case or adequately deal with an incident.
It is probably not when but whether you would even

www.computer.org/computingedge 29
CYBERCRIME AND FORENSICS
know about such an incident. That being said, medi-
cal devices save lives and extend the life expectancy
of patients. It is time we focus on building forensics
for future breaches.
LOGGING AND MONITORING ON
MEDICAL DEVICES
Logging and monitoring are critical steps in the early
detection of IT incidents. Log monitoring speeds up
the identification of specific exceptions; for example,
when a device is not behaving expectantly, the logs
can alert people to that fact. Additionally, log moni-
toring provides developers and support personnel
with observability and visibility of the behavior of their
applications. Cauchi et al.11 state that their research
examined the logs from Braun Infusomat Space
pumps and found that they are split between two files.
The first that they encountered was the device log,
which contains comprehensive device event informa-
tion. The second is the keystroke log, which records
inputs. These logs are deficient in many ways.
For example, logs are often incomplete and contain
anomalies.12 Ingest logs in real time via cloud-based
solutions can help detect and clean these anomalies.
The key to having these logs ingested is that the logs
remain time synchronized and tamper evident. This
would allow for the real-time monitoring of the con-
dition of patients and their medical devices. Nguyen
et al.12 recommend using Software Guard Extension
(SGX) and the Trusted Platform Module (TPM). They
propose that the logger relies on SGX, TPM, and stan-
dard encryption to facilitate secure communication.
This process is used assuming that the communica-
tion channel is in a hostile environment and operat-
ing system. The article identifies options that have
promise. However, the concern is how older systems
can be refactored to use a cloud solution for real-time
logging, given that the hardware potentially does not
support this.
Building a secure cloud logger is something that
should be considered for future medical device manu-
facturing. Medical devices generally have logs that
developers design to do fault and error finding and
pull statistics to understand the device’s functional-
ity. Instead, devices should contain logs with an equal
balance among the system, performance, and debug-
ging, and security logging should be considered. The
30 ComputingEdge
balance needs to have multiple stakeholders look at
the same logs and understand if the provided data
suits their needs.
A developer looks at logs to determine whether
something within the workflow is working as it
should. A digital forensics practitioner looks at logs to
rebuild events that took place across a given period.
Upcoming work must ensure that legacy devices can
be monitored and forensic readiness applied in some
form, and medical devices should be built with future
breaches in mind.
T he biggest constraint in dealing with medical
devices includes legacy systems, which have
been an ever-growing problem in the medical device
world. This means that a digital forensics practitio-
ner investigating medical devices should be versed
in older systems. These devices are not forensically
ready; they do not have the hardware build, sufficient
logging, or necessary artifacts to conduct incident
response or perform digital forensics adequately.
We have the opportunity to influence how com-
panies build future devices by building forensic readi-
ness within the manufacturing phase. However, it also
means that there are potentially legacy devices that
cannot support a costly upgrade to make significant
changes in terms of forensic readiness. There needs to
be a layered approach to device usage within a hospital
network and its protection. The future of more secure
and better forensics-ready medical devices rests
in the hands of those building them. More research
is needed in unpacking the forensic needs to better
reconstruct information on a medical device.
ACKNOWLEDGMENTS
I would like to thank Prof. Barry Irwin from Noroff Uni-
versity, who has been instrumental in supporting my
writing and research; Emlyn Butterfield from Noroff
University for the support and editing that have gone
into the article and being a sounding board for the dig-
ital forensics component; and the Biohack Village at
DEF CON, especially Janine Medina, for the late-night
conversation about how we can get better manu-
factured medical devices. Then, there are othersyou
know who you arewho have given me the support to
explore my ideas. Last but not least, I would like to
thank William Hagestad for his continuous support
June 2023
 CYBERCRIME AND FORENSICS

and many interesting conversations as well as Kyle Rochefortia Sw. (Ehretiaceae, Boraginales),” Biodiver-
Erickson for his support and knowledge. sity Data J., vol. 4, p. e7720, Jun. 2016, doi: 10.3897/BDJ
.4.e7720.
REFERENCES 11. A. Cauchi, H. Thimbleby, P. Oladimeji, and M. Harrison,
1. P. P. Jayaraman, A. R. M. Forkan, A. Morshed, P. D. “Using medical device logs for improving medical
Haghighi, and Y.-B. Kang, “Healthcare 4.0: A review of device design,” in Proc. 2013 IEEE Int. Conf. Healthcare
frontiers in digital health,” Wiley Interdisciplinary Rev., Inform., pp. 5665, doi: 10.1109/ICHI.2013.14.
Data Mining Knowl. Discovery, vol. 10, no. 2, p. e1350, 12. H. Nguyen et al., “Cloud-based secure logger for medi-
2019, doi: 10.1002/widm.1350. cal devices,” in Proc. 2016 IEEE 1st Int. Conf. Connected
2. “SweynTooth cybersecurity vulnerabilities may affect Health, Appl., Syst. Eng. Technol. (CHASE), pp. 8994, doi:
certain medical devices: FDA safety communication,” 10.1109/CHASE.2016.48.
U.S. Food and Drug Administration. https://www.fda 13. N. Ellouze, S. Rekhis, N. Boudriga, and M. Allouche,
.gov/medical-devices/safety-communications/ “Cardiac implantable medical devices forensics:
sweyntooth-cybersecurity-vulnerabilities-may-affect Postmortem analysis of lethal attacks scenarios,”
-certain-medical-devices-fda-safety-communication Digit. Investigation, vol. 21, pp. 1130, Jun. 2017, doi:
(accessed Jun. 24, 2021). 10.1016/j.diin.2016.12.001.
3. S. Carmody et al., ”Building resilient medical technol-
ogy supply chains with a software bill of materials,” npj
Digit. Med., vol. 4, p. 34, Feb. 23, 2021, doi: 10.1038 VERONICA SCHMITT is an assistant professor at Noroff Uni-
/s41746-021-00403-w. versity, Oslo, 4645, Norway. Her research interests include
4. J. Johnson, “Percentage of U.S. healthcare data security vulnerabilities in medical devices forming part of
breaches caused by hacking from 2014 to 2020,” the Internet of Things and how these could be exploited by
Statista, Feb. 12, 2021. [Online]. Available: https://www malicious attackers as well as what types of forensic arti-
.statista.com/statistics/972228/health-data-breaches facts could be identified from any attacks. Schmitt received
-caused-by-hacking-us/ (accessed Jun. 24, 2021). an M.S. degree in information security with a specialization
5. F. Langston, “Top 6 hackable medical IoT devices,” Criti- in the forensic analysis of malware from Rhodes University.
cal Insight. https://www.criticalinsight.com/resources She is involved in DEF CON and various conferences and has
/news/article/top-6-hackable-medical-iot-devices spoken on this topic.
(accessed Jun. 24, 2021).
6. “Cybersecurity vulnerabilities in certain GE healthcare
clinical information central stations and telemetry
servers: Safety communication,” U.S. Food and Drug
Administration. https://www.fda.gov/medical-devices
/safety-communications/cybersecurity-vulnerabilities
-certain-ge-healthcare-clinical-information-central
-stations-and (accessed Jun. 24, 2021).
7. F. E. Block, “The role of forensic engineering investiga-
tions in medical device reports,” J. Clin. Eng., vol. 42, no.
2, pp. 8588, doi: 10.1097/JCE.0000000000000208.
8. A. Kyaw, B. Cusack, and R. Lutui, “Digital forensic
readiness in wireless medical systems,” in Proc. 2019
29th Int. Telecommun. Netw. Appl. Conf. (ITNAC), pp. 16,
F O LLOW US
doi: 10.1109/ITNAC46935.2019.9078005.
9. “Resources for retail delivery leaders,” Convey. https: @s e curit y p riva c y
//www.getconvey.com/resource/medical-devicex
-statistics/ (accessed Jun. 24, 2021).
10. R. Irimia and M. Gottschling, “Taxonomic revision of

www.computer.org/computingedge 31
 EDITOR: Phil Laplante, The Pennsylvania State University, plaplante@psu.edu

This article originally

appeared in
DEPARTMENT: SOFTWARE ENGINEERING
vol. 55, no. 7, 2022

Trusting Digital Twins

Phil Laplante, The Pennsylvania State University

A digital twin is a virtual representation of real-world entities and processes

synchronized at a specified frequency and fidelity. This new technology has the
same trust concerns as do other systems, and we explore them here.

O
ur current virtual world seems little different
than it was in past decades when we talked
about specifications and software, both of
which are virtual. Specifications and software were
key software engineering topics back then and still are
today. Here, I explore a new technology termed digital
twins, and ask, “What has really changed?” The trust
concerns are essentially the same, so let’s revisit them.
For those of you who are unfamiliar with “digital
twins,” the Digital Twin Consortium, which is currently
developing a digital twin standard, has offered the fol-
lowing description for digital twins:
“A digital twin is a virtual representation of
real-world entities and processes synchro-
nized at a specified frequency and fidelity.
› Digital twin systems transform business by
accelerating holistic understanding, optimal
decision-making, and effective action.
› Digital twins use real-time and historical data
to represent the past and present and simulate
predicted futures.
› Digital twins are motivated by outcomes,
tailored to use cases, powered by integration,
built on data, guided by domain knowledge, and
implemented in IT/OT systems.1”
We won’t focus on that definition here because
there are others. Instead, we will focus on the 14 trust
Digital Object Identifier 10.1109/MC.2022.3149448
Date of current version: 4 July 2022
considerations that were proposed in Voas et al. 2021.2
These 14 should be considered by any proposed defini-
tion for digital twin.
These proposed trust concerns are not directly
focused on risk assessment and mitigation, but rather
on trust. That is, will digital twin technology provide
the desired operational functionality with an accept-
able level of quality? Answering this question begins
with an understanding of trust. Here, trust is the
probability that the intended behavior and the actual
behavior are equivalent given a fixed context, fixed
environment, and fixed point in time. Trust is viewed as
a level of confidence.
Trust should be considered at several levels. 1) Is
the digital twin functionally equivalent to the physical
object? 2) Can a specific digital twin be composed with
another digital twin? 3) Is enough information available
about the environment and context of the physical
object? 4) Can digital twin technology be standardized
to the point where certification of a digital twin is pos-
sible? Now, let’s explore the 14 trust issues.
DIGITAL TWIN
CREATION ORDERING
The point in time at which a digital twin is created will
have an impact on the correctness of the digital twin.
For example, is it created before the physical object is
created, or is it reverse-engineered from the physical
entity (that it is intended to mirror)? Both approaches
are valid. However, the fidelity of the digital twin may
be reduced if it is created after the physical entity
exists because there may be internal unknowns about
the existing physical entity that cannot be discov-
ered. A good analogy here is commercial off-the-shelf

32 June 2023 Published by the IEEE Computer Society  2469-7087/23 © 2023 IEEE
 software. Such products are black boxesthe source
code is usually unavailable to the customer or integra-
tor and, thus, hides internal syntax. For digital twins,
this is a trust consideration.
TEMPORAL
Digital twin technology has an implied temporal com-
ponent to it, particularly since it deals with physical
objects, and physical objects are bound by time. Hard-
ware reliability theory dictates that physical objects
will degrade and fatigue over time after periods of
usage. Even when idle, physical systems suffer from
levels of decay over time. For example, if a car has not
been turned on for years, it is likely that the battery will
be dead, rust and oxidation has occurred, and perhaps
insect or rodent pests have damaged the vehicle. (A
friend had his car “totaled” by the insurance company
because after years of it lying idle in his barn, he dis-
covered that mice had made nests, clogging compo-
nents and chewing through all of the wiring.)
A digital twin will not degrade or fatigue over time.
Therefore, at some point the real-world entity and
digital twin will be in conflict on some level, and syn-
chronization of the two should occur. For example, a
metal part could develop hairline fractures after usage
that are not represented in the digital twin. This situ-
ation might suggest that the digital twin needs to be
reworked or maintained to account for this. For exam-
ple, a physical object at time t + 1 will likely be different
than at time t. However, the digital twin should be the
same at times t and t + 1 unless it updates dynamically
with feeds from the physical object. Having access to
an accurate time stamp for the physical object and
digital twin is a trust consideration.
ENVIRONMENT
Digital twin technology has an implied or explicit envi-
ronmental component that cannot be overlooked. For
physical objects, a description of the environmental
www.computer.org/computingedge
tolerances or expected usage profiles is needed for
many of the “ilities,”3 particularly interoperability. For
example, bricks used to construct buildings are made
from a variety of materials; some bricks will break more
easily under stress than others, and some bricks are
better suited to certain temperatures and climates.4
This additional expected operational usage infor-
mation should be reflected in the digital twin. With-
out this information, it will be difficult to determine if
the physical object is ”fit for purpose” since purpose
implies environment and context. Unknown environ-
mental influences have plagued safety-critical sys-
tems and software.
Consider PowerPoint running during a presenta-
tion. Usually, the presenter does little more than touch
the Page Up or Page Down keys. One could argue
that the operational profile for executing PowerPoint
during a presentation is twofold: 1) the loaded presen-
tation and 2) the button inputs from the presenter.
Whether the presentation goes smoothly (reliably
and in a timely manner) is also a function of all of the
inputs that PowerPoint is receiving from the disk,
memory, and the operating system in real time. If, for
example, the presentation gets stuck going from slide
x to slide x + 1, then something related to “unknown”
(phantom-like) environmental influences is probably
involved (for example, another process running on the
machine at the same time and stealing resources and
computing cycles). Accurately defining as many envi-
ronmental factors as possible is a trust consideration.
MANUFACTURING DEFECTS
Digital twin technology has an interesting relation-
ship to mass production. A digital twin may be used to
guide a manufacturing process. For example, a factory
that produces light bulbs will have a certain defect
rate per thousand bulbs. Not all bulbs produced will
be usable, and for those that are usable, there will still
be small (possibly microscopic) distinctions between
33
SOFTWARE ENGINEERING
individual bulbs. These small distinctions may impact
the lifetime of a specific bulb. The packaging on a
set of light bulbs will offer an approximation for how
long a bulb will operate before burnout. This facet
highlights the idea that a digital twin could not only
describe the underlying components of an average
bulb but also suggest how it should be manufactured
if the representation also details a metric, such as
time to burnout. Ensuring that a manufacturing pro-
cess produces a product with the correct life expec-
tancy based on the information in a digital twin is a
trust consideration.
FUNCTIONAL EQUIVALENCE
Digital twin technology requires a means to determine
functional equivalence between the digital twin and
the physical object. Without this function, trust is sus-
pect. If the digital twin is an executable specification,
DOES THE INSERTION OF VIRTUAL
PROBES IN THE DIGITAL TWIN MIMIC
THE BEHAVIOR OF THE PROBES IN
THE REAL SYSTEM?
then for the inputs that are presented, it should pro-
duce the same outputs that the physical object pro-
duces for the same input data. If this does not occur,
then functional equivalence has not been achieved.
This situation could occur for many reasons, such as
decay and fatigue, manufacturing variances, or other
environmental influences that the physical object
experiences during operation but the digital twin does
not. Without some assessment of the level of func-
tional equivalence, it is difficult to assert trustworthi-
ness. (Verification and validation can be used to pro-
vide evidence of functional equivalence.)
COMPOSABILITY
AND COMPLEXITY
There is a trust consideration regarding the size and
complexity of the digital twin for its physical object.
A digital twin that is too complicated can introduce a
composability problem in terms of predicting the trust-
worthiness of a final composed system from more than
one digital twin. Assume that a system has five physical
34 ComputingEdge
components (real-world entities), and each compo-
nent has a corresponding digital twin definition. Physi-
cally connecting the five components may be straight-
forward, but composing the five digital twins may not
be, particularly if the digital twins contain information
such as tolerances and expected operational usages.
Standards should be useful to prune extraneous infor-
mation contained in a digital twin since standards can
define required interconnects between components
of a domain, enabling the composition to be modeled
and tested. One approach might be separating classes
of information into categories, such as “essential,”
“need to know,” or “extraneous.”
INSTRUMENTATION
AND MONITORING
Instrumentation of a digital twin (the ability to provide
dashboard information during operations) is a bene-
ficial and unique advantage that digital twin technol-
ogy offers. While one might not be able to instrument
the physical object, one may be able to instrument the
digital twin. However, it is not as simple to correctly
inject instrumentation and probes into a digital twin
as might be expected; much can be learned here from
the safety-critical software community. First, a deter-
mination of where to inject the probes is necessary.5
This is not often easy, and it can be more art than sci-
ence. Second, how many probes to inject is also a con-
sideration. As shown in real-time systems, probes can
slow down performance and timing. This may cause
a problem for synchronization between the digital
twin and physical object. That said, there are ways to
reduce this impact by having the probes only collect
raw data and not compute internal test results, such
as built-in self-tests. Collecting the “right” information
from the internal state of an executing digital twin is
an expensive and error-prone effort. Finally, the inser-
tion of probes in real systems can sometimes alter the
system performance in subtle ways that might not be
reflected in the twin.6 But does the insertion of virtual
probes in the digital twin mimic the behavior of the
probes in the real system? I don’t think so.
HETEROGENEITY OF STANDARDS
The heterogeneity of different formats for digital
twins may cause composability problems.7 If ven-
dors misuse standardized formats for the digital twin
June 2023

definitions of their components, composing digital
twin definitions from different component vendors
may not be achievable.8 This is a consideration for
trusting composed digital twins.
NONFUNCTIONAL REQUIREMENTS
A trust consideration for systems composed of many
components deals with quality attributes often
referred to as “ilities.” This also applies to digital twin
technology. Functional requirements state what a
system shall do, negative requirements state what a
system shall not do, and nonfunctional requirements
(the “ilities”) typically state what level of quality the
system shall exhibit for both the functional and nega-
tive requirements. “ilities” apply to both “things” and
the systems they are built into. It is unclear how many
“ilities” there are, though examples include availabil-
ity, composability, compatibility, dependability, dis-
coverability, durability, fault tolerance, flexibility,
interoperability, insurability, liability, maintainability,
observability, privacy, performance, portability, pre-
dictability, probability of failure, readability, reliabil-
ity, resilience, reachability, safety, scalability, cyberse-
curity, sustainability, testability, traceability, usability,
visibility, and vulnerability.3 The issue for digital twin
technology concerns how many of the nonfunctional
requirements can be written for the functional and
negative requirements (thus defining the level of
quality for what the system should and should not
do). The ability to write these nonfunctional require-
ments will affect the ability to claim the trustworthi-
ness of a composite object.
DIGITAL TWIN ACCURACY
If the accuracy of a digital twin is questionable, or
even found to be faulty, then trust is an issue. For soft-
ware, faulty specifications lead to faulty designs that
lead to faulty implementations. In digital twin tech-
nology, the degree to which the digital twin is cor-
rect is a trust consideration. It begs the question as
to whether it might be prudent to have more than one
independently created digital twin for a specific phys-
ical object. In N-version programming,9 more than
one independent software implementation is created
for highly critical systems that the software impacts
because no single implementation can be assumed
to be adequately trustworthy. To address this, each
www.computer.org/computingedge
SOFTWARE ENGINEERING
independent implementation is run in parallel, and the
outputs from each implementation are sent to a voter,
which then decides on the final output that the sys-
tem receives.
TESTING
The testability of a digital twin refers to measuring how
likely an error or defect will be detected during test-
ing. Systems that are less likely to reveal the presence
of defects are deemed less testable. Physical objects
are testable to different degrees using this definition,
though the methods for testing digital twins that are
most likely to demonstrate that the digital representa-
tion is correct are unclear. One option is to ignore this
trust consideration and decide that a digital twin is
untestable and, therefore, stands alone as the “oracle”
or “gold standard.” Moreover, although testing usually
involves expected use cases, consideration should
also be given for cases of misuse.
CERTIFICATION
Certification usually occurs in two different ways.10
One type certifies the process used to develop, while
the other certifies the final artifact that comes from
that process. These two types of certification are dis-
tinct.1114 For digital twin technology, this means that
one could attempt to certify how the digital twin was
created or certify the accuracy of the digital twin itself.
Certification of a twin will be complicated. For exam-
ple, the pharmaceutical industry has illuminated the
problem of information overload. Most prescription
drugs come with warnings concerning who can take
them based on gender, age, underlying conditions,
negative drug interactions, and other factors. Most
drugs also come with disclaimers about negative side
effects and when to discontinue use. This information
is made available to patients, doctors, pharmacists,
and other medical providers.
The problem stems from the vast amount of infor-
mation known about a drug and the vaster amount of
unknown information about a drug at time t that will
not be known until time t + 1. Further, much of the
information is only understandable by medical experts
but is vital to determine a drug’s fitness for purpose.
The trust consideration here for digital twin technol-
ogy is how much of this information can be provided in
a specific digital twin without overloading a user with
35
SOFTWARE ENGINEERING
extraneous information that leads to confusion about
how to use the twin or what the twin even represents.
PROPAGATION
One of the greatest trust concerns with any system
of systems is how errors and corrupt data propagate
(cascade) during execution.15 Digital twin technol-
ogy experiences this trust consideration, particularly
when different twins representing different physical
objects are composed. This may, perhaps, suggest
that a digital twin should be packaged with precon-
ditions and postconditions to determine if the out-
put from one digital twin will be acceptable as input to
another digital twin.
COUNTERFEITING
It is possible that a digital twin could be tampered
with or counterfeited. There are schemes that could
be used to protect against this. Digital twin defini-
tions could be hashed and the hash posted to a pub-
lic webpage; users of a digital twin definition could
hash their copy and compare it against the hash on the
public webpage. That said, webpages and other simi-
lar publicly accessible repositories can be hacked. To
enhance trust, one could use a blockchain and post a
digital twin definition hash publicly in an immutable
data structure (it could never be changed even by mali-
cious attackers). In these ways modifications to digital
twin definition files could be discovered. Alternatively,
identical copies of a digital twin definition (and related
instances) could be stored in separate locations (for
example, in offline backups).
I n summary, these 14 trust concerns apply to digital
twins, but really, to anything virtual. I recommend
you think about them in your domain and environment,
especially as we enter the metaverse.
REFERENCES
1. “The definition of a digital twin,” Digital Twin Consor-
tium, 2020. https://www.digitaltwinconsortium.org
/hot-topics/the-definition-of-a-digital-twin.htm
2. J. Voas, P. Mell, and V. Piroumian, “Considerations
for digital twin technology and emerging standards,”
Nat. Inst. Standards Technol., Gaithersburg, MD, USA,
NISTIR 8356, 2021. [Online]. Available: https://csrc.nist
.gov/publications/detail/nistir/8356/draft
36 ComputingEdge
3. J. Voas, “Software’s secret sauce: The ‘–ilities’
[Software Quality],” IEEE Softw., vol. 21, no. 6, pp. 23,
2004, doi: 10.
1109/MS.2004.54.
4. Standard Specification for Building Brick, ASTM C62
17, 2017. [Online]. Available: https://www.astm.org
/Standards/C62.htm
5. J. M. Voas and K. W. Miller, “Putting assertions in
their place,” in Proc. Int. Symp. Softw. Reliability Eng.,
Monterey, CA, USA, 1994, pp 152157. doi: 10.1109/ISSRE
.1994.341367.
6. P. Laplante, “Heisenberg uncertainty,” ACM SIGSOFT
Softw. Eng. Notes, vol. 15, no. 5, pp. 122, 1990, doi: 10
.1145/101328.101333.
7. J. M. Voas, “Networks of Things’,” Nat. Inst. Standards
Technol., Gaithersburg, MD, USA, NIST Special Publica-
tion (SP) 800-183, 2016.
8. J. M. Voas and P. Laplante, “Standards confusion and
harmonization,” Computer, vol. 40, no. 7, pp. 9496, 2007,
doi: 10.1109/MC.2007.252.
9. L. Chen and A. Avizienis, “N-version programming:
A fault-tolerance approach to reliability of software
operation,” in Proc. 8th Int. Symp. Fault-Tolerant
Comput., 1978, pp. 39.
10. J. Voas, “The software quality certification triangle,”
Crosstalk, vol. 11, no. 11, pp. 1214, 1998.
11. J. M. Voas and G. Hurlburt, “Third party software’s trust
quagmire,” Computer, vol. 48, no. 12, pp. 8087, 2015, doi:
10.1109/MC.2015.372.
12. J. Voas, “Toward a usage-based software certification
process,” Computer, vol. 33, no. 8, pp. 3237, 2000, doi:
10.1109/2.863965.
13. J. Voas and P. Laplante, “The IoT blame game,”
Computer, vol. 50, no. 6, pp. 6973, 2017, doi: 10.1109/
MC.2017.169.
14. J. Voas and P. Laplante, “IoT’s certification quagmire,”
Computer, vol. 51, no. 4, pp. 8689, 2018, doi: 10.1109/MC
.2018.2141036.
15. J. Voas, “Error propagation analysis for COTS systems,”
IEEE Comput. Control Eng. J., vol. 8, no. 6, pp. 269227,
1997, doi: 10.1049/cce:19970607.
PHIL LAPLANTE is a professor of software and systems engi-
neering at The Pennsylvania State University, Malvern, Penn-
sylvania, 19355, USA, a Fellow of IEEE, and an associate editor
in chief of Computer. Contact him at plaplante@psu.edu.
June 2023
IEEE Security & Privacy is a bimonthly magazine
communicating advances in security, privacy,
and dependability in a way that is useful to a
broad section of the professional community.

The magazine provides articles with both a

practical and research bent by the top thinkers in
the field of security and privacy, along with case
studies, surveys, tutorials, columns, and in-depth
interviews. Topics include:
• Internet, software, hardware, and systems security
• Legal and ethical issues and privacy concerns
• Privacy-enhancing technologies
• Data analytics for security and privacy
• Usable security
• Integrated security design methods
• Security of critical infrastructures
• Pedagogical and curricular issues in security education
• Security issues in wireless and mobile networks
• Real-world cryptography
• Emerging technologies, operational resilience,
and edge computing
• Cybercrime and forensics, and much more

www.computer.org/security

E-Currency and Fairness ■ Ransomware Defense ■ A National Cybersecurity Policy

IEEE SECURITY & PRIVACY

Software and Cybersecurity ■ Big Data: Privacy Versus Accessibility ■ Resiliency in Cloud Computing
AI ETHICS

Blockchain Technologies ■ The Fuzzing Revival ■ Cybersecurity for the Public Interest
IEEE SECURITY & PRIVACY

IEEE SECURITY & PRIVACY

DIGITAL FORENSICS, PART 2

PRIVACY AND AUTOMATED AIRPORT SCREENING

VOLUME 17

VOLUME 16
NUMBER 1
JANUARY/FEBRUARY 2019

NUMBER 3

January/February 2019 CYBERSECURITY AND

PRIVACY ISSUES IN BRAZIL
Vol. 17, No. 1
WWW.COMPUTER.ORG/SECURITY

VOLUME 17

November/December 2018
Vol. 16, No. 6
NUMBER 2
MAY/JUNE 2018

MARCH/APRIL 2019

Join the IEEE Computer Society May/June 2018

WWW.COMPUTER.ORG/SECURITY

Vol. 16, No. 3 March/April 2019

Vol. 17, No. 2
WWW.COMPUTER.ORG/SECURITY

for subscription discounts today!
www.computer.org/product/magazines/security-and-privacy
 EDITOR IN CHIEF: Ipek Ozkaya, Carnegie Mellon Software Engineering Institute,
ipek.ozkaya@computer.org
DEPARTMENT: FROM THE EDITOR
Architectural Concerns of
Digital Twins
Ipek Ozkaya
S
ystems where software and hardware ele-
ments need to be designed, deployed, and sus-
tained in orchestration are no longer limited to
manufacturing systems such as smart grids, autono-
mous vehicles, medical monitoring, industrial control,
robotics, or avionics. In these systems, the boundar-
ies of software, hardware, and data become hard to
isolate and control. Digital twins provide a solution
to managing smart manufacturing lifecycles in such
systems as well as their operation and maintenance.
While manufacturing systems pose key software
maintenance and evolution challenges due to their
scale, complexity, and safety criticality, embedded
and cyberphysical systems increasingly dominate
the technologies we use on a daily basis, ranging
from home appliances to personal health and enter-
tainment. Digital twins can assist managing and
monitoring maintenance and evolution lifecycles
with a data-driven approach in these systems as well.
Developing digital twins lies at the intersection of
model-driven software engineering, data manage-
ment, and the ability to scale analysis, including using
artificial intelligence-based techniques. This article
summarizes architectural concerns to consider when
developing digital twin-enabled systems.
A digital twin is a virtual representation that serves
as the real-time counterpart of a physical object, pro-
cess, or component. The concept originates from work
in product lifecycle management in the early 2000s
but found increasing implementations as businesses
recognized the cost savings and timely response to
Digital Object Identifier 10.1109/MS.2021.3130872
Date of current version: 14 February 2022
38 June 2023 Published by the IEEE Computer Society 
This article originally
appeared in
vol. 39, no. 2, 2022
needs that a digital twin enables.1 The concept of digi-
tal twins is more than an effective data collection and
analysis idea for manufacturing maintenance. Digital
twins’ fundamental objective is to disrupt the cost of
operations and maintenance. However, they also have
the potential to influence cost-effective software
development and maintenance strategies as well as
different software-based business models.
Organizations that embrace digital twin-based
approaches often aim to enable both reactive and
proactive data collection strategies. Digital twins not
only assist with operations and maintenance but also
give organizations an opportunity to monetize their
and their partners’ data as well as services developed
to monitor different system components. Taking
advantage of the new data and service economy
expands the scope of digital twins from improving the
maintenance and operation of a single cyberphysical
component, e.g., an aircraft engine, to a broader focus,
e.g., an entire plane and the operation optimization
of fleets. With such an expanded scope, the need to
develop twins of twins is not uncommon. All these
scenarios are driven by the goal of collecting as much
relevant data as possible to drive timely decision
making. The ability to take advantage of digital twins
requires designing them with these several scoping
goals in mind. The data architecture, the design of the
data collecting and servicing infrastructure, and the
digital model of the asset are all part of the scope.
ARCHITECTURALLY SIGNIFICANT
REQUIREMENTS FOR
DIGITAL TWINS
Designing systems that meet quality, business, and
long-term sustainability goals is a stakeholder-focused
2469-7087/23 © 2023 IEEE

activity. Architecturally significant requirements are
elicited in light of stakeholder and business goals. In
addition to typical stakeholders, such as users, main-
tainers, development teams, and customers, digital
twin systems’ stakeholders include data scientists
and business domain analysts. In addition, twin build-
ers or twin implementers may need to be considered
as stakeholders. Data scientists and business ana-
lysts create their own analysis steps by using digital
twin platforms and application programming inter-
faces, accessing data and other analytics capabilities.
Some may also take on the special role of twin builder,
those who create digital twins from existing digital
models and analytics.
Architecturally significant concerns can be
grouped into three areas: data, operation, and evolu-
tion and maintenance. These apply whether the digital
twin is for a physical element, software component,
business process, or service. Eliciting and expressing
these concerns as quality attribute scenarios2 will
assist software engineering teams to assess tech-
nologies they need to take into account and their risks
when designing systems that incorporate digital twins.
Data-driven quality concerns for digital twins
include scalability, consistency across an asset or
process lifecycle, accessibility, the usability of analyt-
ics, and security and privacy. There are other critical
attributes related to data, including quality, ownership
and intellectual property, versioning, and lifecycles
(such as how long to keep information). Not all these
characteristics can be controlled with architectural
strategies. However, being aware of these properties
and concerns informs architects about other potential
constraints to consider when selecting technologies
and design approaches.
Operational quality is related to how the digital
twin functions as well as when and how data analysis
is conducted. Performance, availability, security, and
scalability concerns apply, especially as operational
data are collected and while the system operates. Dif-
ferent quality response measures may apply to the twin
versus the asset in operation. Real-time data collec-
tion and servicing, which may be used to orchestrate
scenarios such as real-time repair and availability, call
for stringent design strategies, as opposed to systems
where the same data are used for business analytics
after the fact. Data consistency is critical and applies
www.computer.org/computingedge
FROM THE EDITOR
to information-related services and operational con-
cerns. In particular, architectural decisions related to
latency, shared resources, storage, versioning, and
logging will significantly impact data consistency.
Evolution- and maintenance-related quality relies
on accounting for the evolution of the asset that
the digital twin represents as well as the evolution
of the technologies that the digital twin architec-
ture relies on. Legacy integrability and reusability,
configurability, and platform alignment- and digital
twin evolution-related considerations drive differ-
ent classes of architectural decisions. In particular,
evolvability is often a higher-priority quality concern in
digital twin-enabled systems due to the rapid pace of
change in underlying technologies as well as data.
QUESTIONS TO DRIVE DIGITAL
TWIN DESIGN DECISIONS
Software design decisions for digital twins operate at
two related but distinct levels. The first is the architec-
ture of the digital twin, in essence, the choices that are
involved in deciding how to model the asset. The sec-
ond level concerns decisions related to incorporating
the digital twin into the system, particularly as it oper-
ates and as maintenance and evolution choices are
made based on collected data.
Model-based engineering techniques drive digital
twin design, whether the twin represents a physical,
digital, or process asset. Domain expertise is essential
to making dependency and modeling decisions related
to different asset elements. Established architectural
tactics, such as localizing change, support ease of
evolution of the asset model. A number of generaliz-
able design concerns can assist architects with mak-
ing correct technical as well as design element deci-
sions. Regardless of the domain and context of the
digital twin, addressing what concrete architecture
choices are made to answer key design questions can
help teams get on the same page quickly. Examples of
these questions include the following:
› How does the digital twin ensure the availability
of all the moving parts at run time as well as the
data when needed?
› How does the digital twin deal with versioning,
including the versioning of data?
› How is provenance of collected data handled?
39
FROM THE EDITOR

› What are the atomic units where
architecture-level scalability applies: to a single
digital twin and its operations, many digital
twins of the same kind and their operations, and
digital twins of twins and their operation?
› How are communications between the digital
twin model, asset, and business analytics
handled? How are concerns, such as message
caching, network routing, parallel operations,
and the decoupling of runtime services,
designed to keep the digital twin in service?
› How will the correctness and consistency of
the digital twin models and collected data be
ensured? What architecture strategies are used
to guarantee reliability and assure that the
analysis results depend on the intended data?
› How are key security and privacy attributes
assured? Are these attributes assumed to be
provided by the cloud platforms and data analyt-
ics software frameworks that digital twins often
rely on? What assumptions are made? What
additional architectural strategies are needed to
enforce security and privacy?
likely rely on digital twins to optimize quality, opera-
tional effectiveness, and cost of system evolution. Vir-
tualization enabled by digital twins empowers orga-
nizations to ask about the cost of maintenance and
operations and change questions in a timelier man-
ner. Such analytics also inform improved design deci-
sions. Architectural concerns span multiple levels,
including the creation of digital twins, understanding
how twins of twins need to be designed and behave,
platforms that host data collected from operations,
and systems that incorporate digital twins. Address-
ing architectural concerns at these multiple levels will
both improve the likely success of digital-twin enabled
systems and also accelerate their successful applica-
tion to different business areas.
REFERENCES
1. M. Grieves, Product Lifecycle Management: Driving
the Next Generation of Lean Thinking. New York,
McGraw-Hill, 2006.
2. L. Bass, P. Clements, and R. Kazman, Software
Architecture in Practice, 4th ed. Reading, MA, USA:
Addison-Wesley, 2021.

Capabilities enabled by more and timely data also

imply amultiple-ownership context. Different entities
and software platforms may be responsible for data,

Cutting Edge
models that contribute to the digital twin architec-
ture, digital twin development, system development,
stay
and maintenance and operations. An underestimated
but critical cross-cutting architectural concern then
on the

of Artificial Intelligence
arises around ownership: the party responsible for
end-to-end quality needs to be clearly defined for
the successful development and operations of digital
twin-enabled systems. J a n ua ry/ f E b r ua ry 2 016

IEEE Intelligent Systems provides peer-

IEEE

Also in this issue:

aI’s 10 to Watch 56
real-Time Taxi Dispatching 68
IEEE
January/FEBruary 2016

from flu Trends to Cybersecurity 84

SOFTWARE ENABLES
P U T T I N G A I I N T O P R A C T I C E

reviewed, cutting-edge articles on the

DIGITAL TWINS theory and applications of systems
Online BehAviOrAl AnAlysis

A number of model-based engineering software sys- that perceive, reason, learn, and
tems, such as Ansys and Microstation, include digi-
act intelligently.
VOLuME 31

tal twin builder functions as well as underlying physi-

nuMBEr 1

www.computer.org/intelligent

IS-31-01-C1 Cover-1 January 11, 2016 6:06 PM

cal models for common asset types, such as circuits,

sensors, and engines, and the ability to develop cus-
tomized analytics on top of them. Such systems ease
the initial entry for conceptualizing digital twins. As
The #1 AI Magazine
www.computer.org/intelligent
IEEE

the boundaries between system software, hardware,

and data elements get fuzzier, more organizations will

40 ComputingEdge June 2023

Get Published in the New IEEE Open
Journal of the Computer Society

Submit a paper to the new

IEEE Open Journal of the
Computer Society covering
computing and informational
technology.

Your research will benefit from

the IEEE marketing launch and 5
million unique monthly users of
the IEEE Xplore® Digital Library.
Plus, this journal is fully open
and compliant with funder
mandates, including Plan S.

Submit your paper today!

Visit www.computer.org/oj to learn more.
 DEPARTMENT
EDITOR: EDITOR:
Alexander Alexander
Magoun, Magoun, a.b.magoun@ieee.org
a.b.magoun@ieee.org
This article originally
appeared in
DEPARTMENT:
ANECDOTE ANECDOTE
vol. 44, no. 3, 2022

“Well Paid for a Woman”:

“Well Paid for a Woman”: Gloria Hardman’s
Gloria Hardman’s 50-Year Career
50-Year Career in Computing
in Computing
Philip Nel , Kansas State University, Manhattan, KS, 66506, USA
Philip Nel , Kansas State University, Manhattan, KS, 66506, USA

In 1964, Gloria Nel (ne

e Webb) arrived in England and began her 50-year career in
computing at IBM in London (1964-1966). She continued as a programmer at the
InLondon
1964, Gloria
Press Nel (née Webb)
Exchange arrived and
(1966–1968), in England
then inand began
the U.S. at her
MIT50-year career in computing
(1968–1969),
at IBM in London (1964-1966). She continued as a programmer at the London Press Exchange
followed by time as a self-employed educator (1979–1982), at Shore Country Day
(1966–1968), and then in the U.S. at MIT (1968–1969), followed by time as a self-employed
School (1982–1984), Choate Rosemary Hall (1984–1989), IBM in Milford Connecticut
educator (1979–1982), at Shore Country Day School (1982–1984), Choate Rosemary Hall
(1989–1993), and Yale University (1994–2014). Her life in computing started in the era
(1984–1989), IBM in Milford Connecticut (1989–1993), and Yale University (1994–2014). Her
of punch cards and ended in the era of smart phones.
life in computing started in the era of punch cards and ended in the era of smart phones.

“W
ho would want to read this?” When
encouraged to write her memoirs, Glo-
ria Hardman often asked that ques-
tion. I told her that lots of people would: she was a
pioneer in the fields of computing and computing edu-
cation. When she brushed off that reason, I said that
her two children—I and my sister Linda—would. So
would her granddaughter Emily. In the 2000s and the
early 2010s, Gloria wrote about her life. This account
derives from those memoirs, and is augmented by her
letters and emails, interviews with and emails from
people who knew her, other items in our family’s
archive, and my own memories.
Gloria’s career in computing began with an inter-
view at IBM’s London office in September of 1964.
Newly arrived to the country and to the field of com-
puters, she was unsure what to expect. A 22-year-old
graduate of Natal University in Durban, the former
Gloria Webb had taught high school math for two
years in South Africa.
Born in Pretoria in 1941, Gloria’s attraction to num-
bers began in childhood. “When I was a small child I
remember my brothers and my father teaching me
card games—solitaire, simplified Bridge (auction), and
rummy. The cards and their symbols attracted me.”
When her cousins observed that four-year-old Gloria
wanted to attend school (as her two older brothers
1058-6180  2022 IEEE
Digital Object Identifier 10.1109/MAHC.2022.3193854
Date of current version 27 August 2022.
did), they persuaded her mother to enroll her in Peter
Pan, a tiny private school in the Durban neighborhood
where they lived. She was so adept that, after a little
over two years of school, Gloria was already two
grades ahead of her age [1], [3].
When her arithmetic teacher taught “rules for dis-
covering which numbers were divisible by, for exam-
ple, 2, 3, 4, 5,” Gloria spent her walk home from school
looking at license plate numbers and working out
what they were divisible by. “I doubt that any of the
other girls in the class found this fascinating. It seems
that an interest in numbers may be genetic and I was
just lucky that numbers appealed to me” [3].
Though South African society did not value girls’
education as highly as boys’, her brother Terry noticed
that his sister was bright and inquisitive. Her parents
would have been happy for Gloria to take “domestic
science,” but Terry insisted that she enroll in serious
academic courses, including Latin and math.
Graduating at the top of her class, Gloria was Dux
(the equivalent of valedictorian in the U.S.) of Durban
Girls High School. As a student, she so loved mathe-
matics that she tutored classmates. “I loved to explain
it to others and would spend time tutoring my friends
and others in the class. Teaching was such fun and I
knew I wanted to be a math teacher.” Gloria “was sure
that if I taught math properly every girl in my class
would find it interesting and be successful” [3].
At the age of 16, Gloria enrolled in the University of
Natal at Durban, pursuing degrees in math and
English. For the first time, she was not in a girls-only
school. “The ratio of girls to boys at was about 1 to 5
which was quite a culture shock to someone who had

4280
June 2023
IEEE Published by Published
Annals of the History of Computing the IEEE Computer Society
by the IEEE 
Computer Society 2469-7087/23 © 2023
July-September 2022IEEE
 ANECDOTE
ANECDOTE

FIGURE 1. Gloria Nel in the spring of 1966 poses in London at what was probably the completion of an IBM PL1 class with her stu-
dents from local IBM offices and IBM's clients. Left to right: Frank Johnston (from one of Great Britain’s largest privately owned
firms), Graham Manson (later finance director for IBM Norway), Nicholas Blacklock (IBM's Computer Services Centre in London),
Mike Allen (Information Services, Beecham Pharmaceuticals), Nick Bark (back; ITT Data Services, whose Data Centre opened in
London that September), Gloria Nel (front), Roger Prosser, Bill Clark (data systems at Clark Johnston Whitehead Ltd.), Guy
Parker (later managing director, Bellard Investments), George Sutton, George Smith. Courtesy of the author.

attended girls-only schools.” Her first-year math class
had about a hundred students, but only “perhaps 5
girls in total and the others arrived early and sat
together in the front row. I was coming from my
English class some distance away and would arrive
just minutes before the class began.” Each day as she
stepped into the large, tiered lecture hall, the boys
greeted Gloria with “some foot stomping, whistles and
calls to ‘sit here – there’s room for you.’” The experi-
ence was “more than a little intimidating and today it
would be classified as harassment but back then
(1957) that was just the way it was” [3].
She persisted. By the final year of her degree, the
increasing challenge shrunk the math class to 12,
with just two girls. When the other girl failed an exam,
19-year-old Gloria became the sole female student
remaining. The work was getting harder, and more
philosophical, “like proving there were more irrational
than rational numbers.” Gloria worried that she would
be the next to fail [3].
Her professor was not much help. His style of
teaching was to write on the board at high speed, say
“it is intuitively obvious that. . .” and then write the next
line of a proof without further explanation. On the rec-
ommendation of a classmate, Gloria went to tutorials
run by a younger professor, Siegfried Gra €sser [3].
Prof. Gra€sser taught Gloria how to study. He told her
she should “get a large pad of scratch paper and prac-
tice writing all the proofs until they were embedded in
my brain.” She must spend six hours a day studying

www.computer.org/computingedge
July-September 2022 IEEE Annals of the History of Computing 8143
ANECDOTE
ANECDOTE
math, he said. She followed his advice. Each day for a
month, Gloria rose at 5 a.m., worked on math for two
hours, took a break by reading English, and then
returned to math. At the end of the term, Gloria aced
the exam, receiving the highest mark in the class [3].
For the next two years, Gloria taught math—one
year at Pietermaritzburg Girls High School, and the next
back at Durban Girls High. Gloria at first “enjoyed the
challenge of trying to get the nonmath girls at least to
feel that they could do it.” However, she soon concluded
“that the damage had been done in elementary school
and that it is really hard to turn around the perception
that ‘I’ll never get it.’” She also missed the coeducational
environment and social life of her university days [3].
In September 1963, Gloria needed a date to the Dur-
ban Girls High dance—which included some younger
teachers. A friend set her up with Pierre Nel. After that
night, they began dating. A few months later, when Pierre
shared his ambition of going to England for his Ph.D. in
chemistry, Gloria said that was fine, but she would not
continue dating him exclusively while awaiting his return.
In response to Gloria’s nonexclusive clause, Pierre pro-
posed in December—just three months after they had
met. They married in July 1964 [4].
In August, she and Pierre arrived in the U.K. so that
he could pursue his Ph.D. at Imperial College London.
They found a small basement flat with a rent of £8 per
week. At 56 Penywern Road, they had a bedroom,
small dining room, tiny kitchen, and a bathroom. But
no hot water, nor any heat. To get those, they put
money into a meter that “swallowed it faster than you
can imagine.” As Gloria recalled, “Landlords in London
made extra money that way—they paid the electric
bill and had the key to your meter so they could set
the rates somewhat higher than they paid” [4].
Pierre had a British Council Scholarship, but Gloria
would need to be the primary breadwinner. Thinking
that computing might pay better than teaching, she
scheduled an interview at IBM at 101 Wigmore Street
in Marylebone [4].
When she arrived, “they put me in a little office
with their aptitude test . . .. It was great fun—just up
my street.” They graded the test—which had arith-
metic, logic, and algebra sections—while she waited.
Gloria aced it. They interviewed her, and set up
another interview at their Education Department on
Lodge Road. There, “they asked whether I was plan-
ning to have a baby and then quit. I managed to
convince them that was not part of the game plan.”
Her starting salary was £1000 per year—far “better
than I would have got as a teacher. And as a
teacher in a tough East End school I would probably
not have survived!” [4].
4482
ComputingEdge
IEEE Annals of the History of Computing
To prepare her to teach the corporation’s salesmen
and customers, IBM enrolled Gloria in a six-month training
program. She was the youngest person and sole woman
in a class of 24, and one of only two students learning to
be instructors. The others would become systems engi-
neers and salesmen. It was a “trial by fire.” They began by
learning about machines that used punch cards: key-
punch machines, verifiers, sorters, and calculators. That
was but a warmup to the very latest in tabulation technol-
ogy: the 421 accounting machine, a calculator larger than
a contemporary washer-dryer. For each new task, one had
to rewire its control panel. The 421 would then read
instructions from a stack of 80-column punch cards [4].
After two months of course work, IBM sent Gloria
out into the field with an experienced salesman.
Together, they “had to figure out how to sell these
behemoths and would have to create sales plans to
solve the accounting or inventory control problems in
a factory or business.” She felt “horribly out of my
depth—I had no business experience and did not
know an invoice from a hole in the wall” [4].
Gloria persevered, learned, and then returned to
IBM’s classrooms for two more months, where she began
learning programming languages: Assembler, Fortran,
Cobol, and RPG. She learned to write code for the punch
cards. This meant that, first, she wrote all the code on
paper. Next, she keypunched the code into the card.
Then, she gave the cards to the operator, who loaded
them, ran the program, and gave her a printout with all
the error messages. Gloria then corrected her code, and
repeated the entire process until the program performed
as intended [4].
After Gloria completed the six-month training pro-
gram, IBM sent her into its classrooms, teaching cus-
tomers who had come down to London for one or two
weeks of training—the cost of which was underwritten
by IBM. “Fortunately my ‘students’ were all very polite,
kind Brits who did not shoot holes in my teaching or ask
really embarrassing questions that I could not answer.”
When asked a difficult question, Gloria used her tea
break to find the answer. She would dash to the instruc-
tors’ lounge and find a knowledgeable man who would
tell her what she needed to know. “It was a steep learn-
ing curve, but I was young, I needed the job and I sur-
vived by studying at night for the next day’s class” [4].
She also began putting what she had learned into
practice. As Gloria watched Pierre process complex cal-
culations for his own research, she realized “we could
write a program to automate this.” Assisted by her IBM
colleagues, she wrote a program that Pierre then put on
punch cards and ran on the Imperial College computer.
It worked! for Gloria, programming was no longer a theo-
retical endeavor. She could actually do it [4].
June
July-September 2023
2022

Soon Gloria was teaching customers how to use
the brand-new IBM System/360 computers, which the
corporation began shipping to customers in the spring
of 1965. With a faster programming language, PL1, that
ran on earlier machines and peripherals compatible
with third-party manufacturers, the 360 series domi-
nated the market by the late 1960s. But in 1965, they
were so new that salespeople had not yet had the
chance to test them. Gloria found herself telling a class
what the 360 would do, only to be interrupted by one
student whose company had already installed one:
“Well, it didn’t work like that on the one we have!” [4].
In May 1966, after about 20 months with IBM, Glo-
ria learned from conversations in the staff room that
her male colleagues were being paid a lot more—even
men who started when she did and were doing exactly
the same job. She thought that “since I am the primary
breadwinner in our little family I should be paid at the
same rate as the male breadwinners.” So, she plucked
up her courage, and went to the office of her manager,
Mr. Cooper. She explained the situation, and asked
politely for a raise. Cooper responded, “You’re being
well paid for a woman” [4].
As Gloria left his office, she thought to herself: with my
IBM training, I could get a raise by getting a better job.
Looking through the newspaper ads, she saw a better-pay-
ing job programming in assembler language on IBM’s Sys-
tem/360 for the London Press Exchange, a market-
research firm. The company hired her, and she quit IBM [5].
Though she never forgot the discrimination she
faced there, the years at IBM also launched her career
as a computer educator. But it would take a decade for
her to return to teaching people how to use computers.
Pierre defended his Ph.D. in August of 1967. He got a
job at Carter’s Ink in the United States. While waiting for
their U.S. visas, Pierre and Gloria returned to South Africa.
In the spring of 1968, they flew to Boston, Massachusetts.
Gloria got a programming job at MIT’s Office of Insti-
tutional Studies. In December 1968, when she was
nearly six months pregnant, Carter’s Ink fired Pierre. He
thought the company insufficiently innovative, and
voiced frustration that his research project was taking
so long to be approved. Though his tendency for blunt
criticism likely cost him his job, by early 1969, Pierre had
a new job in the research division of Polaroid, and joined
the team that designed the battery for Polaroid’s SX-
70—the iconic camera that instantly ejected pictures
which developed as you watched. Meanwhile, Gloria
went from a full-time MIT employee to working part-
time from home. At the end of March she had her first
child—me. Two years later, she had her second—Linda.
Though she was primarily a stay-at-home mom from
mid-1969 until 1976, Gloria’s interests in mathematics,
www.computer.org/computingedge
July-September 2022
ANECDOTE
ANECDOTE
computers, and education were never far away. We
were the only family in our neighborhood that used com-
puter punch cards as scrap paper. When she learned
about Cuisenaire Rods (colored blocks designed to
teach mathematical ideas), she bought a set and used
them with her children. These blocks are the reason I
still think of numbers spatially.
Exercise led her back to the classroom. In 1974,
Gloria read Mildred Cooper’s Aerobics for Women,
which advocated regular exercise that raises the heart
rate to 130. A year later, with both children in school,
she started taking and then teaching Jacki Sorenson’s
Aerobic Dancing. In 1978, Gloria began working with
Math Learning Inc., and developed and taught courses
to help people of all ages overcome math anxiety [7].
In 1979, the family acquired a Radio Shack TRS-80
computer. Gloria began giving workshops at home on
how to use personal computers. Within a year, she was
teaching Lynnfield Massachusetts school teachers
and courses on programming in BASIC (Beginners’ All-
purpose Symbolic Instruction Code) in the town’s adult
education program. In 1982, Pierre took early retire-
ment from Polaroid and set himself up as a computer
salesman and consultant. Since that was not particu-
larly lucrative, the family needed a new breadwinner.
THOUGH SHE NEVER FORGOT THE
DISCRIMINATION SHE FACED THERE,
THE YEARS AT IBM ALSO LAUNCHED
HER CAREER AS A COMPUTER
EDUCATOR.
In meeting this necessity, Gloria transformed her
life and the family’s. For the next 32 years, Gloria
devoted her professional life to teaching people how to
use computers. In 1982, she designed the classroom,
selected the equipment and planned the computer
curriculum at Shore Country Day School in Beverly
Massachusetts, where she would teach for two years.
Under her direction, the school installed about 15 Apple
computers—likely the Apple II Plus—in a classroom on
the second floor of the main upper-school building. In
1983, those were joined by some Franklin Ace 1200
computers—the lower-priced, short-lived clone of the
Apple II. She taught the students BASIC, including the
“IF. . . THEN. . .” routine, simple graphics and animation,
the random number generator, and how to debug one’s
program via TRON (Trace On).
Gloria described the experience as “tremendously
challenging and enjoyable.” It was “a teacher’s
IEEE Annals of the History of Computing 8345
ANECDOTE
ANECDOTE

dream—the students are lined up outside the com-

puter room in the morning, eager to get in, and they
are working productively on programming in BASIC. It
is easily the most popular course in the school and I
have ideal conditions, with one student to a computer
in most of my classes.”1
In addition to Gloria’s expertise, her students
remember her patience, her supportiveness, and her
gently irreverent sense of humor. Steve Sloan (Shore
’84) recalled, “because we all started with different levels
of exposure to computers,” Gloria “supported us 1:1 from
wherever we were starting.”2 Alex Rigopulos (also Shore
’84) became a successful game designer, co-founding
Harmonix Music Systems, the company that developed
both the Guitar Hero and Rock Band series of video
games. He calls Gloria’s classes “the most energizing
and inspirational classes in my 3 years at Shore,” credit-
ing them with “changing the trajectory of my life,” and
adding “that there’s pretty much a direct line from [her]
classes to my career making music games.”3
Though a private school paid a lower salary than a
public school, Gloria was in one respect now well paid
and not just for a woman. Since faculty children could
attend Shore tuition-free, her work gave my sister and
me access to a first-rate education.
When I was to graduate from ninth grade, Gloria
found a new job that would allow both of us to continue FIGURE 2. Gloria helping a student in Choate Rosemary
our private secondary education. For five years (1984- Hall's Academic Computer Center, 1987. Photo from
1989), she directed Computer Education at Choate Course Catalog: Choate Rosemary Hall 1987-1989. Photo-
Rosemary Hall in Wallingford, Connecticut. Rare among graph courtesy of the Choate Rosemary Hall School
private schools at the time, Choate in 1984 had a super-
Archives, Wallingford, CT.
minicomputer—a DEC VAX 11/750, running Unix.
Roughly the size of a refrigerator, it sat behind a glass
window, in a climate-controlled room, and was con- field of computing.” His computer science instructors
nected to a series of terminals in a larger classroom. at his high and middle schools had “all been men.”5
Though few used it, every Choate student had an In taking over computer education at Choate,
account, via which they could send email—via Yale— Gloria inherited an unusual managerial structure:
anywhere in the world, if they specified exactly how they members of the almost entirely male Choate Com-
wanted it routed. As Jan Miksovsky (Choate ’86) recalls, puter Club were the system administrators. On the
to send a message to somebody at Stanford, he would one hand, as Jan observed, “it’s not a universal good”
“send it to Yale, and then to this other hub and then to to put teenage boys “in charge of anything.”6 Gloria
Stanford, and then to their final email address.”4 maintained order by reminding them of their responsi-
Jan is one of Gloria’s students who realized in ret- bility and intervening as needed. On the other hand,
rospect how unusual it was to have a woman teaching as Gloria said at the time, managing a computer
computer science. As he recalled, “she was the first system “provides an opportunity for students to be
woman that I knew who was sort of a leader in the unrestricted without a bureaucratic system hanging
above them. They have independence” [6].
When they devised independent projects, Gloria
1
G. Nel and P. Nel, letter to Phil Penney, 29 December 1982.
supervised and evaluated their work. In one of them,
2
S. Sloan, “Re: Research question about Gloria Nel at Shore,”
email to author, 12 June 2022.
3
A. Rigopulos, “Re: Research question about Gloria Nel at
5
Shore,” email to author, 11 June 2022. Miksovsky, Zoom interview.
4 6
J. Miksovsky, Zoom interview with author, 13 June 2022. Miksovsky, Zoom interview.

4684
ComputingEdge
IEEE Annals of the History of Computing June
July-September 2023
2022
 ANECDOTE
ANECDOTE

FIGURE 3. Gloria Hardman, “Hello, World!” Editor's Comments, inaugural issue of IBM's AIXpert, Winter 1991. Reprint Courtesy of
IBM Corporation © 1991.

www.computer.org/computingedge
July-September 2022 IEEE Annals of the History of Computing 8547
ANECDOTE
ANECDOTE
Jan wrote an interpreter for the programming lan-
guage Karel, allowing it to run on Unix. He shared it on
Usenet and, soon, Jan’s compiler was being used by
people at AT&T, Digital Equipment Corporation, and in
courses at Stanford University. The education at and
internships attained via Choate served as “an impor-
tant foundation for [his] career.” Jan spent 15 years at
Microsoft before leaving to co-found two compa-
nies—Cozi and Component Kitchen.7
In July of 1989, Gloria—now Gloria Hardman—
returned to IBM, but this time at the company’s offices
in Milford, Connecticut. She became a project admin-
istrator in the AIX Programs Department. In the winter
of 1991, she launched and edited the company’s AIX-
pert Journal. As she wrote in its inaugural issue, the
publication’s goal was to provide software developers
and others “with timely, relevant information that will
optimize your AIX development work.” [2]
In 1993, when IBM wanted her to relocate to Texas,
Gloria left the company. The next year, Phil Long hired
her to work in Yale University’s Instructional Technol-
ogy Department, where she remained until she retired
in 2014.
For her first decade at Yale, Gloria supported
Classesv1, helping faculty and graduate students with
their technological needs, including (in the mid-1990s)
introducing many of them to email. She not only
answered emails and phone calls, but went to their offi-
ces and classrooms. When conservative icon William F.
Buckley returned to Yale to teach writing in the late
1990s, he insisted on using WordStar—a mid-1980s
word processing program that had to be adapted to
current computing systems. Gloria ensured that he
could use WordStar, and he “was extremely consider-
ate and courteous to his ‘computer geek assistant’ and
sent me autographed copies of his books for several
years after he taught at Yale.”8
In March 2005, Yale’s Academic Technology Direc-
tor Chuck Powell decided to bring Sakai to Yale’s
course management system. Gloria became part of
the main team implementing Sakai on Classesv2, led
by project manager David Hirsch and programming
team manager Andy Newman, with technical manager
Michael Appleby, and programmers Jen Bourey and
Drew Mazurek. From then until her retirement, Gloria
was its sole full-time support person and “the face of
Classesv2,” as her colleague Matt Regan said.9
7
Miksovsky, Zoom interview.
8
G. Hardman, “Re: Shiny & Busy & Bow-tie’d,” email to author,
3 March 2008.
9
M. Regan, “Re: Query for IEEE article about Gloria Hardman,”
email to author, 12 July 2022.
4886
ComputingEdge
IEEE Annals of the History of Computing
Assistant Dean Risa Sodi recalled that “It was a com-
fort” to know that Gloria “was the real, live person
behind the Classesv2 help button.” Gloria wrote
much of the documentation and balanced the inter-
ests of IT management and the faculty. On the one
hand, she supported Sakai and the people using it. On
the other, to improve the system, she brought their cri-
tiques back to management, and helped people use v2
in ways not anticipated by its designers [8].
The onset of posterior cortical atrophy, a variant of
Alzheimer’s, began impeding Gloria’s ability to read
certain typefaces. She devised workarounds, such as
changing or enlarging the font. But PCA erodes the
brain’s capacity to interpret what the eyes see, and
Gloria soon realized she would need to retire. As she
said then, “I don’t want to stay on past my sell-by
date.” In the fall of 2014, she moved to The Commons
in Lincoln, Massachusetts.
Though Alzheimer’s has mostly silenced Gloria,
when I told her that I was writing about her role in the
history of computing, she opened her eyes and man-
aged to utter a single legible word: “Good.”
ACKNOWLEDGMENTS
For answering my queries, sincere thanks to Jenn
Borggaard, Ann Cahill, Brian Donlon, David Hirsch, Trip
Kirkpatrick, Jan Miksovsky, Linda Nel, Alina Nevins,
Paul Parry, Ba Penney, Chris Penney, Jeff Pettiross,
Matt Plum, Matt Regan, Josh Rose, Stephen Sloan,
Peter Snow, Risa Sodi, Jill Speed, and Terry Wong. For
additional guidance, thanks to Annals editors Alexan-
der Magoun and David Hemmendinger.
BIBLIOGRAPHY
[1] G. Hardman, “Gloria’s memories,” unpublished
manuscript, 2003 and 2013.
[2] G. Hardman, “Hello, world!,” AIXpert, Winter 1991, p. 3.
[3] G. Hardman, “Memoirs — How mathematics gave me
opportunities,” unpublished manuscript, c. 2010.
[4] G. Hardman, “Pierre,” unpublished manuscript, 2004 and
2013.
[5] G. Hardman, Re
sume , c. 1990.
[6] K. Kassover, “A decade of computers at CRH,” Choate
News, vol. 76, no. 10, p. 1, Feb. 1986.
[7] “New faculty,” unspecified Shore Country Day School
publication, p. 2, Fall 1982.
[8] “Wonder woman,” Booklet of reminiscences given to
Gloria Hardman upon her retirement from Yale, 2014.
PHILIP NEL is University Distinguished Professor of English at
Kansas State University, Manhattan, KS, 66506, USA. Find him
online at philnel.com. Contact him at philnel@ksu.edu.
June
July-September 2023
2022
IEEE COMPUTER SOCIETY D&I FUND

Drive Diversity
& Inclusion in
Computing

Supporting projects
and programs that
positively impact
diversity, equity, and
inclusion throughout
the computing DONATE TODAY!
community.
 EDITOR: George Strawn,
EDITOR: Strawn, gostrawn@gmail.com
gostrawn@gmail.com
DEPARTMENT:
DEPARTMENT: MASTERMIND
Masterminds of the NSFnet:
Masterminds of the NSFnet: Jennings, Wolff,
Jennings, Wolff, and
and Van Houweling
Van Houweling
George Strawn , National Academy of Sciences, Engineering, and Medicine, Washington, DC, 20418, USA
George Strawn , National Academy of Sciences, Engineering, and Medicine, Washington, DC, 20418, USA
T
he Arpanet origins of the Internet are well
known. For example, see “Masterminds of the
Arpanet.”1 Not as well known is the NSFnet,
which was the essential step between the Arpanet
and the Internet. This article will highlight the unlikely
origin of the NSFnet, its development over ten years
(1985–1995), and its transition to the Internet of today.
I should note that I was a part of the NSFnet project
team, so this article is part history, part reminiscence.
LAX AND SCIENCENET REPORTS
In the early 1980s, both Europe and Japan were provid-
ing their researchers with more access to supercom-
puters than the United States. A committee of
distinguished computational scientists, chartered by
the National Science Foundation (NSF) and chaired by
Peter Lax of NYU, recommended in December 1982 that
the U.S. government address this problem.2 In 1984, an
IBM Vice President, Eric Bloch,3 became the Director of
NSF and accepted the challenge (along with some new
money) to create supercomputing centers for general
use by scientists at 100 U.S. research universities.
The Lax Report also recommended that a com-
puter network be constructed to interconnect the
research universities with the new supercomputer
centers. As a fortuitous coincidence, in January 1983,
TCP/IP replaced NCP on the Arpanet. Significantly,
the Arpanet was the only network at that time
that was “vendor neutral.” The Sciencenet Report
appeared in December 1984, exactly two years after
the Lax Report.4 Its subtitle was “A report on the evo-
lution of a supercomputer access network.” An inter-
national participant in the Sciencenet deliberations
was Dennis Jennings, the director of Computing Serv-
ices at the University College Dublin. This report
1520-9202  2021 IEEE
Digital Object Identifier 10.1109/MITP.2021.3123419
Date of current version 17 December 2021.
50 June 2023 2021
November/December Published
Published by theby theComputer
IEEE IEEE Computer
This article originally
appeared in
vol. 23, no. 6, 2021
referred to a previous action of an NSF Program Advi-
sory Committee, which had recommended that “the
standard DoD TCP and IP protocols be designated as
the initial national network transport and internet
protocols, respectively.”
Dennis Jennings arrived at the NSF from Ireland in
1985 and was appointed as a Network Program Officer
in the Division for Supercomputing. In concert with this
recommendation, he selected TCP/IP for the incipient
NSFnet. After making the decision for TCP/IP, Jennings
architected a three-layer network: national backbone,
regional nets, and campus nets. Then, he made an
interim backbone network award to interconnect the
six supercomputing centers with a 56 Kb/s network
(which was the same bandwidth as the Arpanet). The
developing regional networks began connecting to this
interim backbone. The three-layer decision was also for-
tuitous, as will be seen below. Then, Jennings returned
to his regular job in Ireland in the summer of 1986.
Stephan Wolff, a Network Specialist from the U.S.
Army Ballistics Research Laboratory (home of the
ENIAC), was selected by NSF to replace Jennings.
Shortly, a question arose as to how funds would be
divided between supercomputers and the NSFnet. In
order to forestall intradivision disputes, a new NSF
division was created for the network, and Steve Wolff
became its director. He proceeded to staff the new
division and to prepare a solicitation for the new
NSFnet backbone.
NSFNET
The three-layer network was developed and deployed
as follows. A national backbone network competition
was won by a partnership led by the State of Michigan
and its higher education network, MERIT, and included
the private sector companies, MCI and IBM. The
leader of the partnership was Douglas Van Houweling,
CIO of the University of Michigan. The regional net-
works formed in different geographical areas of the
country and interconnected the research universities
in their areas with the backbone network. The campus
SocietySociety  67
2469-7087/23 © 2023 IEEE
IT Professional
MASTERMIND
networks were the responsibility of the universities,
who had to assure that TCP/IP traffic could be
accepted and transmitted by campus computers. The
56 Kb/s backbone had become very congested, and
the installation in 1988 of the 1.5 Mb/s backbone was
much appreciated by users.
MCI was relatively a new company that provided
long-distance phone service, after having broken the
AT&T long-distance monopoly. (MCI’s original name
was Microwave Communications Inc. They argued in
court that AT&T’s natural long-distance monopoly
was moot in the presence of microwave communica-
tions. The court agreed. MCI was acquired by World-
com in 1998 and then by Verizon in 2006.) IBM was the
computer colossus of that time, a provider of hard-
ware and software (like Intel and Microsoft combined).
Significantly, it was the IBM research division that was
the NSFnet partner. IBM’s task was to develop Inter-
net routers, and MCI’s task was to provide the fiber
optic communications lines. There were many out-
standing contributors in the NSFnet partnership, but
as stated previously, the most important contributors
were Jennings, Wolff, and Van Houweling. It should be
noted that, in addition to contributing excellent
human resources, MCI, led by Dick Liebheber, and
IBM, led by Al Weis, contributed more than $150 mil-
lion in like-kind effort to the project, making the
NSFnet a truly public–private partnership.
On the one hand, both the backbone network and
the regional networks were funded by NSF as support
awards to the academic community, as opposed to
the government procuring services for itself. This
meant that the awards were outside of the jurisdiction
of the Federal Acquisition Regulations, which can be a
nemesis for agencies (disputed awards, etc.). The
backbone was awarded as a cooperative agreement,
which meant that decisions would be made jointly by
NSF and the partnership. On the other hand, the
regionals were awarded as standard grants, which
meant that they were at an arm’s length from govern-
ment requirements. This turned out to be crucial
because it enabled the regionals to seek additional
members beyond the research universities, as long as
the network traffic was restricted to open research.
Midway through NSFnet’s life (in 1991), half of the
members of the regionals were private companies,
who wanted to “keep an eye” on this novel network. In
my opinion, it was this early interest by the private
sector that convinced entrepreneurs that there was
“money to be made” in the Internet. At the same time
that the private sector was quietly joining the network,
colleges and universities beyond the first 100 were
offered the opportunity to join by means of NSF grants
68
www.computer.org/computingedge
IT Professional
MASTERMIND
that helped fund connecting to the local regional. At
the time, there were about 4000 colleges and universi-
ties in the United States, and by 1995, half of them
were on the NSFnet. And, via an international connec-
tion program, European and Asian research networks
began connecting. But the Internet was not yet public.
PRIVATIZATION AND
COMMERCIALIZATION
In 1992, interested entrepreneurs called for and
obtained a congressional hearing aimed at “getting
the government out of the way so that they could
take over the Internet.” After the hearing, it was clear
to NSF that congress wanted this to happen, so Wolff
began thinking about how to retire the NSFnet in a
helpful way. This turned out to be a complicated task
that took three years to accomplish.
When the NSFnet began operating, DARPA was
still supporting Internet number and domain name
transactions. As the NSFnet grew, it was appropriate
that NSF assume those responsibilities. Consequently,
a solicitation was issued that called for a new man-
ager for these services. It also called for a user serv-
ices organization to help new academic network
users. Finally, it also called for Internet “white and yel-
low page services.” Very soon the World Wide Web
made its dramatic appearance on the network, and
Google (in 1998) and other search engines filled this
role in a revolutionary manner.
Another challenge that had to be met before retir-
ing the NSFnet was related to its increasing popularity.
That is, the 1.5 Mb/s network was becoming con-
gested. The partnership came to NSF with a proposal
to replace the 1.5 Mb/s backbone with a 45 Mb/s net-
work. This was approved by NSF, and (none too soon)
the new network went into parallel operation with the
old one. It was the first network to operate at such a
high bandwidth, and a number of “interesting” prob-
lems were discovered and solved. The fast network
came into full service just at the time when the slow
network would have collapsed from congestion, as
had its 56 Kb/s predecessor in 1987.
With the network stabilized, attention turned to
creating a solicitation for a new, post-NSFnet Internet.
The initial discussion regarding NSF’s future was
whether it should have one new backbone or two (for
private sector development purposes). But the private
sector signaled that it was wary of using a Federally
supported network for business purposes, and in the
end, the solicitation called for no NSF backbone. It did,
of course, call for the creation of network access
points where the new commercial Internet providers
November/December 2021 51
MASTERMIND
could interconnect their services (those commercial
services would be where the higher ed community
could get service to replace the soon-to-be-retired
backbone). The solicitation was first released as a
public draft, and many comments and suggestions
were received, some of which were incorporated in
the final solicitation. In due course, all the required
awards were made, and in the spring of 1995, the
NSFnet was retired and the commercial Internet was
born. For much more detail on the NSFnet, see its
Wikipedia article5 and the Computer History Museum
history of the Internet.6
AFTERMATH
As it happened, the handoff to the private sector
occurred just in time. In that summer, Internet usage
exploded and the new Internet providers struggled to
keep up with demand. They invested tremendous
resources to do so, and if NSF had still been in charge, it
would have either been bankrupted or else the Internet
would have failed. As it was, NSF did not get the Internet
number and domain services privatized in time and
spent the next several years struggling to do so.
IBM had developed the first high-speed Internet
routers for the NSFnet and might have become the
leader of the router business. But their research divi-
sion could not convince the business division to com-
mercialize routers, so Cisco (and then others) stepped
in. His job well done, Steve Wolff left NSF at this time
and joined Cisco.
One other part of the new Internet should be men-
tioned. The supercomputing centers were intercon-
nected with an ultrahigh-bandwidth network (cryptically
named the vBNS) to support research in distributed
high-performance computing. And when an NSF review
of the new network architecture (chaired by Doug Van
Houweling) highlighted the grave concern that profes-
sors could not get their new Internet-related work done
on the newly privatized but highly congested networks,
NSF responded by providing support for the 100 (then
200) research universities to connect to the vBNS, and
this provided the spark for the creation of the university
consortium named Internet2, which Van Houweling ably
led for the next ten years. About the time when Van
Houweling stepped down from his Internet2 responsibil-
ities, Wolff joined it as Interim VP and CTO.
52 ComputingEdge
November/December 2021
MASTERMIND
By the late 1970s, the idea of a vendor-neutral com-
puter network had come to be seen as important. Sev-
eral international standards bodies created the Open
Systems Interconnect (OSI) model with the anticipa-
tion that OSI networks would be the future. But his-
tory provided an Internet future instead. As the
Wikipedia article on OSI says.
“OSI was an industry effort, attempting to get
industry participants to agree on common network
standards to provide multivendor interoperability. It
was common for large networks to support multiple
network protocol suites, with many devices unable to
interoperate with other devices because of a lack of
common protocols. For a period in the late 1980s and
early 1990s, engineers, organizations and nations
became polarized over the issue of which standard,
the OSI model or the Internet protocol suite, would
result in the best and most robust computer networks.
However, while OSI developed its networking stand-
ards in the late 1980s, TCP/IP came into widespread
use on multivendor networks for internetworking.’’7
In other words, the de jure OSI development
became bogged down in politics, and the de facto
TCP/IP took its place in history. Politics had played
almost no role in the development of TCP/IP and its
method of “rough consensus and running code,”
proved to be a better development strategy than cor-
porate politics.
REFERENCES
1. G. Strawn, “Masterminds of the Arpanet,” IT
Professional, vol. 16, no. 3, pp. 66–68, May/Jun. 2014.
2. Lax Report, 1982. [Online]. Available: https://www.pnnl.
gov/scales/docs/lax_report1982.pdf
3. G. Strawn and C. Strawn, “Erich Bloch: Mastermind of
computer technology and science administration,” IT
Professional, vol. 19, no. 2, pp. 70–72, Mar./Apr. 2017.
4. “The Sciencenet report: A report on the evolution of a
supercomputer access network,” 1984, unpublished.
5. Accessed: Mar. 2021. [Online]. Available: https://en.
wikipedia.org/wiki/National_Science_Foundation_
Network
6. Accessed: 2021. [Online]. Available: https://www.
computerhistory.org/internethistory/1980s/
7. Accessed: 2021. [Online]. Available: https://en.wikipedia.
org/wiki/OSI_model
IT Professional 69
June 2023
IEEE Computer Architecture Letters is a forum for fast
publication of new, high-quality ideas in the form of
short, critically refereed technical papers. Submissions
are accepted on a continuing basis and letters will be
published shortly after acceptance in IEEE Xplore and in
the Computer Society Digital Library.

Submissions are welcomed on any topic in computer

architecture, especially:
• Microprocessor and multiprocessor systems
• Microarchitecture and ILP processors
• Workload characterization
• Performance evaluation and simulation techniques
• Interactions with compilers and operating systems
• Interconnection network architectures
• Memory and cache systems
• Power and thermal issues at the architectural level
• I/O architectures and techniques
• Independent validation of previously published results
• Analysis of unsuccessful techniques
• Domain-specific processor architecture
(embedded, graphics, network)
• High-availability architectures
• Reconfigurable computer architectures

www.computer.org/cal

Join the IEEE Computer Society

for subscription discounts today!
www.computer.org/product/journals/cal
 SA
,U
Register Today!
A
W
E,
U
V
LE

IEEE
EL
B

QUANTUM
WEEK

23
20
PT
SE
2
–2
17

qce.quantum.ieee.org
Don’t miss IEEE Quantum Week 2023—the IEEE International
Conference on Quantum Computing and Engineering (QCE)
bridging the gap between the science of quantum computing
and the development of the industry surrounding it. In-person
registration space is limited. Register today!

Council on Superconductivity

®
 Conference Calendar
I EEE Computer Society conferences are valuable forums for learning on broad and dynamically shifting top-
ics from within the computing profession. With over 200 conferences featuring leading experts and thought
leaders, we have an event that is right for you. Questions? Contact conferences@computer.org.
JULY
2 July
• CLOUD (IEEE Int’l Conf. on
Cloud Computing), Chicago,
USA
• EDGE (IEEE Int’l Conf. on Edge
Computing & Communica-
tions), Chicago, USA
• ICDH (IEEE Int’l Conf. on Digi-
tal Health), Chicago, USA
• ICWS (IEEE Int’l Conf. on Web
Services), Chicago, USA
• QSW (IEEE Int’l Conf. on Quan-
tum Soft ware), Chicago, USA
• SERVICES (IEEE World Con-
gress on Services), Chicago,
USA
• SSE (IEEE Int’l Conf. on Soft-
ware Services Eng.), Chicago,
USA
3 July
• EuroS&P (IEEE European Sym-
posium on Security and Pri-
vacy) Delft , Netherlands
10 July
• CSF (IEEE Computer Secu-
rity Foundations Symposium)
Dubrovnik, Croatia
• ICALT (IEEE Int’ l Conf. on
Advanced Learning Technolo-
gies) Orem, USA
• ICME (IEEE Int’l Conf. on Mul-
timedia and Expo) Brisbane,
Australia
55 June 2023
18 July
• ICDCS (IEEE Int’l Conf. on Dis-
tributed Computing Systems)
Hong Kong
• SCC (IEEE Space Computing
Conf.) Pasadena, USA
• SMC-IT (IEEE Int’l Conf. on
Space Mission Challenges for
Information Technology) Pasa-
dena, USA
19 July
• ASAP (IEEE Int’l Conf. on Appli-
cation-specific Systems,
Architectures and Processors)
Porto, Portugal
28 July
• ICCP (IEEE Int’l Conf. on Com-
putational Photography), Mad-
ison, Wisconsin, USA
AUGUST
1 August
• IRI (IEEE Int’l Conf. on Informa-
tion Reuse and Integration for
Data Science), Bellevue, WA,
USA
14 August
• FiCloud (Int’l Conf. on Future
Internet of Things and Cloud),
Marrakesh, Morocco
21 August
• PST (Int’l Conf. on Privacy,
Security & Trust), Copenhagen,
Denmark
Published by the IEEE Computer Society
23 August
• HotI (IEEE Hot Interconnects
Symposium), virtual
25 August
• SmartIoT (IEEE Int’l Conf. on
Smart Internet of Things), Xin-
ing, China
27 August
• HCS (IEEE Hot Chips Sympo-
sium), Palo Alto, California, USA
30 August
• MIPR (IEEE Int’l Conf.on Multi-
media Information Processing
and Retrieval), Singapore
SEPTEMBER
4 September
• ARITH (IEEE Int’l Symposium
on Computer Arithmetic),
Portland, Oregon, USA
• RE (IEEE Int’ l Require -
ments Eng. Conf.), Hannover,
Germany
16 September
• SmartCloud (IEEE Int’l Conf. on
Smart Cloud), Tokyo, Japan
25 September
• ACSOS (IEEE Int’l Conf. on
Autonomic Computing and
Self- Or ganizing S ys tems),
Toronto, Canada
• IC2E (IEEE Int’l Conf. on Cloud
Eng.), Boston, USA
• MASS (IEEE Int’ l Conf. on
2469-7087/23 © 2023 IEEE
 Mobile Ad-Hoc and Smart Sys-
tems), Toronto, Canada
• STC (IEEE Software Technol-
ogy Conf.), virtual
OCTOBER
2 October
• ICCV (IEEE/CVF Int’l Conf. on
Computer Vision), Paris, France
• LCN (IEEE Conf. on Local Com-
puter Net works), Day tona
Beach, USA
9 October
• e-Science (IEEE Int’l Conf. on
e-Science), Limassol, Cyprus
18 October
• SecDev (IEEE Secure Develop-
ment Conf.), Atlanta, USA
22 October
• VIS (IEEE Visualization and
Visual Analytics), Melbourne,
Australia
31 October
• CLUSTER (IEEE Int’l Conf. on
Cluster Computing), Santa Fe,
USA
NOVEMBER
1 November
• CIC (IEEE Int’l Conf. on Collab-
oration and Internet Comput-
ing), Atlanta, USA
• CogMI (IEEE Int’l Conf. on Cog-
nitive Machine Intelligence),
Atlanta, USA
• TPS (IEEE Int’l Conf. on Trust,
Privacy and Security in Intel-
ligent Systems and Applica-
tions), Atlanta, USA
2 November Multimedia), Laguna Hills, USA
• CyberC (Int’l Conf. on Cyber- 14 December
Enabled Distributed Comput- • BCD (IEEE/ACIS Int’l Conf. on
ing and Knowledge Discovery), Big Data, Cloud Computing,
Suzhou, China and Data Science Eng.), Ho Chi
4 November Minh City, Vietnam
• ICEBE (IEEE Int’l Conf. on 15 December
E- Business Eng.), S ydney, • BigData (IEEE Int’l Conf. on Big
Australia Data), Sorrento, Italy
12 November 18 December
• SC (Int’l Conf. for High-Perfor- • HiPC (IEEE Int’l Conf. on High-
mance Computing, Network- Performance Computing, Data,
ing, Storage, and Analysis), and Analytics), Goa, India
Denver, USA • iSES (IEEE Int’l Symposium on
Smart Electronic Systems),
DECEMBER Ahmedabad, India
1 December
• ICDM (IEEE Int’l Conf. on Data
Mining), Shanghai, China
4 December
• CloudCom (IEEE Int’l Conf. on
Cloud Computing Technol-
ogy and Science), Napoli, Italy
CloudCom (IEEE Int’l Conf. on
Cloud Computing Technology
and Science), Napoli, Italy
• CSDE (IEEE Asia-Pacific Conf.
on Computer Science and
Data Eng.), Nadi, Fiji
• ICA (IEEE Int’l Conf. on Agents), Learn more
Kyoto, Japan
about IEEE
5 December
• BIBM (IEEE Int’l Conf. on Bio- Computer
informatics and Biomedicine), Society
Istanbul, Turkey
• RTSS (IEEE Real-Time Systems
conferences
Symposium), Taipei, Taiwan computer.org/conferences
11 December
• ISM (IEEE Int’l Symposium on
Science Led.
Enterprise Driven.

Through the pioneering work of

its scientific and engineering
teams around the world,
Quantinuum has become a global
leader for quantum computing
hardware, software, and the open
source tools used by quantum
developers and engineers.

Today, Quantinuum is
accelerating the adoption of
applications in cybersecurity,
chemistry, optimization, and
true artificial intelligence.

Join us as we accelerate
quantum computing.

Scan the QR code

or visit us online at
quantinuum.com
to learn more . . . . . . . . . . . . .