applied

sciences
Article
Value-Oriented and Ethical Technology Engineering
in Industry 5.0: A Human-Centric Perspective for the
Design of the Factory of the Future
Francesco Longo 1, * , Antonio Padovano 1 and Steven Umbrello 2
1 Department of Mechanical, Energy and Management Engineering, University of Calabria,
87036 Rende, Italy; antonio.padovano@unical.it
2 Department of Philosophy and Educational Sciences, University of Turin, 10124 Turin, Italy;
steven.umbrello@unito.it
* Correspondence: francesco.longo@unical.it




Received: 6 May 2020; Accepted: 16 June 2020; Published: 18 June 2020


Featured Application: This work can be used to drive the design and development of Industry
4.0 technological solutions and the Factories of the Future in a human-centric, value-oriented
and ethical direction.

Abstract: Although manufacturing companies are currently situated at a transition point in

what has been called Industry 4.0, a new revolutionary wave—Industry 5.0—is emerging as
an ‘Age of Augmentation’ when the human and machine reconcile and work in perfect symbiosis with
one another. Recent years have indeed assisted in drawing attention to the human-centric design of
Cyber-Physical Production Systems (CPPS) and to the genesis of the ‘Operator 4.0’, two novel concepts
that raise significant ethical questions regarding the impact of technology on workers and society
at large. This paper argues that a value-oriented and ethical technology engineering in Industry
5.0 is an urgent and sensitive topic as demonstrated by a survey administered to industry leaders
from different companies. The Value Sensitive Design (VSD) approach is proposed as a principled
framework to illustrate how technologies enabling human–machine symbiosis in the Factory of
the Future can be designed to embody elicited human values and to illustrate actionable steps that
engineers and designers can take in their design projects. Use cases based on real solutions and
prototypes discuss how a design-for-values approach aids in the investigation and mitigation of
ethical issues emerging from the implementation of technological solutions and, hence, support the
migration to a symbiotic Factory of the Future.

Keywords: Industry 4.0; Industry 5.0; Human-Machine Interaction; Operator 4.0; Smart Operator;
ethics; technology engineering; Value Sensitive Design; smart factory; human factors

1. Introduction
Recent years have witnessed technology-oriented manufacturing megatrends such as robotics,
computer-integrated manufacturing and cyber-physical production systems (CPPS), molding how
industrial production is seen and understood [1]. As we continue to reap the benefits of the age of
‘smart production’, scholars and practitioners have recently begun emphasizing the critical function
that workers form as a constituent of human-centric CPPS [2], as well as the importance of designing
industrial systems and work processes to integrate and embody human factors [3]. Despite numerous
studies focused on the interplay between the design of work processes or workplaces and human
well-being [4], technologies shaping the Factories of the Future are far more impactful. In particular,
their sociotechnical influence in industrial settings has been typically overlooked, perhaps resulting

Appl. Sci. 2020, 10, 4182; doi:10.3390/app10124182 www.mdpi.com/journal/applsci

Appl. Sci. 2020, 10, 4182 2 of 25

from the technocratic paradigm that characterizes the 4th Industrial Revolution [5]. Although Industry
4.0 is not yet full-grown and it is still the major revolution on manufacturers’ minds, some industry
pioneers and visionary technology leaders have already opened up the floor to the 5th Industrial
Revolution, Industry 5.0. Unlike Industry 4.0, the new revolutionary wave combines the diverging
strengths of CPPS and human agents to create symbiotic factories [6]. To address this phenomenon,
institutions and policy makers are shifting their attention towards ‘human-centered design’ as well as
‘ethical and responsible innovation’ in the Factories of the Future [7]. The European Union, for example,
has been encouraging research into methodologies that centralize technology design for values that also
encircles socio-cultural and ethical considerations [8,9]. Despite the inclination to frame technologies
as neutral objects, they unavoidably manifest values, whether those of their designers or users [10],
consequentially shaping and transforming how workers using them can actualize their potentials,
identities, relationships, and aims [11]. Industry 5.0 is compelling computer scientists, designers,
industrial engineers, as well as philosophers and legal experts to concentrate on the means by which
technologies within 5.0 industrial systems can be designed for human values, rather than relegating
them as an afterthought [12]. This is particularly apparent when converging technologies blur the
lines between human and technological capabilities [13], such as in the case of the emerging Operator
4.0 paradigm. Scholars [13] have recently taken up the invitation to find acceptable and ethically
sustainable solutions together with the employees. Given the impacts that manufacturing systems
have on both workers and society, coupled with the fact that the human-centric CPPS paradigm is still
in its infancy, it warrants investigating the role of human values and ethical design for the converging
technologies enabling the Operator 4.0.

Scope and Aims

This paper argues that a 5th revolutionary wave in the industrial sector, Industry 5.0, is imminent
and develops in tandem with a growing consciousness of the value of a human-machine symbiosis in
industry. The emergence of Industry 5.0 as the Age of Augmentation is discussed in Section 2 where
ethical questions are presented regarding the impact of technology on human workers in the field
of future industrial systems engineering. The second aim of this study is to ascertain whether the
industrial community is truly attentive to the human-centric evolution of the Industry 4.0 paradigm
towards an Industry 5.0 and whether human values and ethical issues represent an actual concern for
the workers at different levels and in different kind of organizations. A survey has been administered
to industry leaders from different companies already leading the 4th industrial revolution to different
extents and the results are discussed in Section 3. What makes this paper comparatively unique in
its approach is that it then proposes a Value Sensitive Design (VSD) methodology as an overarching
framework for a human-centered design of the Factories of the Future, with technologies shaping
and embodying human values. Guidelines and recommendations are provided in Section 4 that are
based on the results of the survey, including a list of values—that may serve as starting point for
future studies—to uphold during techno-social change and human-machine symbiosis towards the
Industry 5.0. Three use cases are finally proposed in Section 5 and discussed. The use cases build upon
technological solutions and prototypes for the Operator 4.0 and describe prospective scenarios that
industrial workers will deal with in the Factories of the Future. Insights describe how a design for
values aids in the investigation and mitigation of ethical issues concealed behind the implementation of
specific technological solutions and, hence, support the migration to a symbiotic Industry 5.0. Finally,
Section 6 draws the conclusions by summarizing the contribution of the study, illustrates the limitations
and suggests directions for future research.
 Appl. Sci. 2020, 10, 4182 3 of 25

2. Industry 5.0

2.1. The Emergence of Industry 5.0 as the Age of Augmentation

Almost a decade after the term Industry 4.0 has been coined and the benefits of an interconnected,
highly digitalized, and automated smart factory are well-documented in the literature, most industrial
firms are still ‘transitioning’ to the smart manufacturing paradigm, rather than ‘revolutionizing’ [14].
They are currently experimenting and piloting standalone solutions to establish a digital foundation
Appl. Sci. 2020, 10, x FOR PEER REVIEW 3 of 25
that may yield in the short or medium term greater efficiency into traditional manufacturing
93routines [15].a digital
establish Despitefoundation
the fact that the 4th
that may yieldindustrial
in the short revolution
or medium seems to set aside
term greater the into
efficiency human
94component,
traditionalthe ubiquitousroutines
manufacturing integration of digital
[15]. Despite andthat
the fact automation technologies
the 4th industrial revolution in seems
the industrial
to set
95socio-technical
aside the human component,
framework hasthe ubiquitous
today integrationtriggered
spontaneously of digital and
the automation
genesis of technologies
a futuristic in the of
vision
96a new
industrial socio-technical framework has today spontaneously triggered
evolutionary stage where the human and machine reconcile and work in perfect symbiosis withthe genesis of a futuristic
97one vision
another of (Figure
a new evolutionary
1). After the stage
lastwhere the human ages’—the
two ‘industrial and machine Agereconcile and work (Industry
of Automation in perfect 3.0)
98andsymbiosis with one another (Figure 1). After the last two ‘industrial ages’—the
the Digital Age (Industry 4.0)—where smarter and smarter connected devices and computer Age of Automation
99technologies
(Industry 3.0) and the Digital Age (Industry 4.0)—where smarter and smarter connected devices and
have automated manufacturing processes and workflows, the next ‘age’—the Age of
100 computer technologies have automated manufacturing processes and workflows, the next ‘age’—the
Augmentation (Industry 5.0)—will be focused on the cooperation between human intelligence and
101 Age of Augmentation (Industry 5.0)—will be focused on the cooperation between human intelligence
102cognitive computing and on treating automation as a further enhancement of the human’s physical,
and cognitive computing and on treating automation as a further enhancement of the human’s
103sensorial, and cognitiveand
physical, sensorial, capabilities
cognitive[16]. By putting
capabilities [16].humans
By putting back into the
humans loop,
back intoIndustry
the loop,5.0Industry
profoundly
104restructures human tasks in the realm of manufacturing in ways that benefit
5.0 profoundly restructures human tasks in the realm of manufacturing in ways that benefit the the workers. They will be
105upskilled
workers.to shift
Theyfrom manual
will be to cognitive
upskilled labor
to shift from [17], to
manual provide value-added
to cognitive tasks in value-added
labor [17], to provide production and
106to work—with peace of and
tasks in production mind—alongside
to work—withan autonomous
peace workforce, an
of mind—alongside i.e.,autonomous
collaborative robots, that
workforce, i.e. will
107be perceptive
collaborative and informed
robots, about
that will human intention
be perceptive and desire
and informed about[18].
human intention and desire [18].
108

109
110 Figure
Figure 1. Moving
1. Moving to to the
the fifthindustrial
fifth industrialrevolution
revolution or
orIndustry
Industry5.0:
5.0:the Age
the of Augmentation.
Age of Augmentation.

111 Although
Although thisthis
discourse
discoursemaymayappear
appear premature
premature and andIndustry
Industry 5.05.0 literature
literature does
does not not
existexist
yet, yet,
112the 5th
the industrial
5th industrial revolution
revolution is is already
already underwayamong
underway amongthe theenterprises
enterprises that
that are
are just now adopting
adopting the
113principles
the principles of Industry
of Industry 4.0. Recent
4.0. Recent years years have indeed
have indeed assisted
assisted in drawing
in drawing attention
attention to the
to the human-
human-centric
114design
centric design of CPPS [19] and to the genesis of the ‘Operator 4.0’, defined
of CPPS [19] and to the genesis of the ‘Operator 4.0’, defined as a hybrid agent established as a hybrid agent as
115the product
established of as the productrelationship
a symbiotic of a symbiotic relationship
between betweenand
the human thethe
humanmachineand the machine
[20]. [20]. The
The impact of the
116technologies
impact of the technologies enabling the Operator 4.0 on human perception, cognition and interaction
enabling the Operator 4.0 on human perception, cognition and interaction capabilities
117 capabilities is explained in Figure 2. Coined as an enabler for computers to perceive the world, the
is explained in Figure 2. Coined as an enabler for computers to perceive the world, the Internet of
118 Internet of Things (IoT) paradigm evolved now full circle to aim at enriching the human perception
119Things (IoT) paradigm evolved now full circle to aim at enriching the human perception of the CPPS
of the CPPS (and of the environment in general) with a multitude of sensing devices and technologies.
120(and of the environment
Nonetheless, in general)
the abundance with
of data a multitude
generated of sensing
by such devices devices
has not and
beentechnologies.
yet fully exploitedNonetheless,
in a
121the abundance of data generated by such devices has not been yet fully
production environment [21]. Augmented cognitive capabilities are needed to process, understand exploited in a production
122environment
and analyze [21]. Augmented
this amount of cognitive
data to get capabilities are needed to
valuable information process,
from understandenhancing
it. Technologies and analyze
123this cognitive
amount of data to get
capabilities dovaluable
not only information
include thosefrom it. Technologies
belonging enhancing
to the Artificial cognitive
Intelligence capabilities
(AI) sphere—
124do not
i.e. only includecomputing,
cognitive those belonging to the Artificial
computer Intelligence representation,
vision, knowledge (AI) sphere—i.e.,machine cognitivelearning,
computing,
125computer
recommender systems andrepresentation,
vision, knowledge planning, schedulingmachineandlearning,
optimization algorithms.systems
recommender They also andinclude
planning,
126scheduling
simulation andfor what-if scenario
optimization analysis,They
algorithms. big data
also analytics, cloud computing
include simulation for what-if(thatscenario
providesanalysis,
on-
127big data
demand resources, especially data storage and computing power) and virtual
analytics, cloud computing (that provides on-demand resources, especially data storage and reality (for training
128computing
and experiential learning purposes). In order to move towards more cognitive and intelligent spaces
power) and virtual reality (for training and experiential learning purposes). In order to
129 [22], human workers are also asked to cooperate with the CPPS and complement the robotic and
move towards more cognitive and intelligent spaces [22], human workers are also asked to cooperate
130 virtual world of the automated production system through novel technologies enabling faster and
131withmore
the CPPS and complement the robotic and virtual world of the automated production system
intuitive workflows [23]. A considerable amount of information and knowledge is indeed
132 generated during the augmented cognition process that should be made available to the operator in
133 a meaningful and intuitive way. Augmented Reality, mobile device and smartwatch apps and
134 advanced human–machine interfaces in industry will be augmented with voice-enabled interaction,
135 thus giving birth to digital intelligent assistants. While Industry 4.0 creates the foundation for the
136 Smart Factory, Industry 5.0 is the era of a ‘Social Smart Factory’, where every single cooperative
 Appl. Sci. 2020, 10, 4182 4 of 25

through novel technologies enabling faster and more intuitive workflows [23]. A considerable amount
of information and knowledge is indeed generated during the augmented cognition process that should
be made available to the operator in a meaningful and intuitive way. Augmented Reality, mobile device
and smartwatch apps and advanced human–machine interfaces in industry will be augmented with
voice-enabled interaction, thus giving birth to digital intelligent assistants. While Industry 4.0 creates the
foundation for the Smart Factory, Industry 5.0 is the era of a ‘Social Smart Factory’, where every
Appl. Sci. 2020, 10, x FOR PEER REVIEW
single
4 of 25
cooperative building block of a CPPS will be able to communicate with the human component through
138enterprise social networks.
social networks. In the
In the ‘Social ‘Social
Smart Smart
Factory’, Factory’,
humans willhumans will work
work alongside alongsiderobots
collaborative collaborative
[24],
139robots
will[24],
use will
exoskeletons to performtostrenuous
use exoskeletons perform tasks [25] and
strenuous will[25]
tasks be able
and to control
will be ableandtoact upon and
control the act
140upon CPPS
the remotely
CPPS remotelyif needed. Given theGiven
if needed. close interplay
the closeand symbiosis
interplay between
and human
symbiosis and machine,
between humanit and
141machine,
becomes compelling
it becomes to pay attention
compelling to paytoattention
the possible consequences
to the of Industry 5.0ofon
possible consequences the welfare
Industry of the
5.0 on
142welfare
humans interacting with these systems [26]. Bringing back human workers to factory
of humans interacting with these systems [26]. Bringing back human workers to factory floors floors is about
143 creating meaningful and sustainable socio-technical systems to support harnessing smart
is about creating meaningful and sustainable socio-technical systems to support harnessing smart
144 technologies in organizations [27].
technologies in organizations [27].
145

146
Figure 2. Towards human–machine symbiosis in the Industry 5.0: technology-augmented human
147 Figure 2. Towards human–machine symbiosis in the Industry 5.0: technology-augmented human
capabilities as part of a cognitive process.
148 capabilities as part of a cognitive process.
2.2. The Role of Ethics in Industry 5.0
149 2.2. The Role of Ethics in Industry 5.0
Since organizations have witnessed an increasing human–machine cooperation level, more and
150more ethical
Since organizations have witnessed an increasing human–machine cooperation level, more and
questions and concerns have been raised regarding the impact of technology on humans
151 more ethical questions and concerns have been raised regarding the impact of technology on humans
in the field of future industrial systems engineering [28]. Just like ethics acts as a self-governing
152 in the field of future industrial systems engineering [28]. Just like ethics acts as a self-governing
153system to keep human self-interest and the good of society in equilibrium, so we argue that ethics is
system to keep human self-interest and the good of society in equilibrium, so we argue that ethics is
154expected to fuel
expected a symbiotic
to fuel relationship
a symbiotic relationshipbetween
between humans
humansand andthe
thecyber-physical
cyber-physical world in Industry
world in Industry 5.0.
155Although literature specifically focusing on ethical issues related to technologies is steadily
5.0. Although literature specifically focusing on ethical issues related to technologies is steadily expanding,
156muchexpanding, much of this research focuses on separate individual values or technologies. A significantbody
of this research focuses on separate individual values or technologies. A significant
157of research focuses,focuses,
body of research for example, on ethical
for example, concerns
on ethical concernsrelated
related to informationand
to information and communication
communication
158technologies
technologies (ICT)
(ICT) and and roboticengineering.
robotic engineering. Human–robot
Human–robot co-working
co-working issues have
issues been,
have for example,
been, for example,
159identified
identified in psychological issues, lack of social interaction, skepticism towards
in psychological issues, lack of social interaction, skepticism towards learning learning robots,
robots,
160shrinking
shrinking human workforce, and human–robot competition, among others [29].
human workforce, and human–robot competition, among others [29]. Although this research Although this
161is very
research is very
valuable to valuable
flesh outtospecific
flesh out specificit values,
values, generallyit generally
does notdoes
takenot take a holistic
a holistic approach.
approach. This can
162be aThis can be a miss particularly when values are competing. Hence, even if the theoretical background
miss particularly when values are competing. Hence, even if the theoretical background exists
163 exists and face-to-face interviews are widely adopted to elicit values influencing the future of
and face-to-face interviews are widely adopted to elicit values influencing the future of industrial
164 industrial workforce and its connection to technology [30], current research in this field remains
165workforce and its connection to technology [30], current research in this field remains scarce and most
scarce and most of the research efforts focus asymmetrically on the technical aspects of the Factories
166of the
of research
the Future. efforts focus asymmetrically on the technical aspects of the Factories of the Future.
167 In Industry 5.0, instead, researchers are compelled to design CPPS using a ‘human-centered’
168 rather than a ‘techno-centered’ approach [26], with a focus on social and ethical considerations [5].
169 The vision of Industry 5.0 comprises of two prospective scenarios, commonly known as the
170 automation and the tooling scenarios, which are characterized by a different degree of self-X abilities
171 of the CPPS [31]. On the one hand, the ‘automation’ scenario proposes an extensive automation of
Appl. Sci. 2020, 10, 4182 5 of 25

In Industry 5.0, instead, researchers are compelled to design CPPS using a ‘human-centered’ rather
than a ‘techno-centered’ approach [26], with a focus on social and ethical considerations [5]. The vision
of Industry 5.0 comprises of two prospective scenarios, commonly known as the automation and the
tooling scenarios, which are characterized by a different degree of self-X abilities of the CPPS [31].
On the one hand, the ‘automation’ scenario proposes an extensive automation of work processes
and tasks, including planning of production process that might eventually lead to a devaluation
of simple working tasks and to maintenance and supervision efforts performed only by highly
qualified personnel [32]. Conversely, in the ‘tooling’ scenario, technology is mostly implemented to
extend human capabilities and support the worker to make the right decision [16]. Which of the
two scenarios will be dominant is not predictable and will depend on the pace of technological
advances, and company-related conditions among other factors. However, in both cases, serious ethical
harms can arise because of either poor design, inappropriate application or misuse of multiple types of
technologies in industrial environments.
Two sub-fields of applied ethics are directly relevant: (1) how to design industrial systems based
on ethical principles and (2) the ethics of the technologies themselves, or machine ethics. Due to
greater intelligence embedded into production systems, machine ethics—i.e., how these systems can
themselves behave ethically—is becoming relevant, but further discussion of this subject is not within
the scope of this article. Instead, the role of industrial systems designers is to examine human values
and ethical aspects a priori so that they are not construed as costs, but instead as design requirements.
Designers have traditionally employed cost-benefit analyses in determining design requirements
where, for example, safety and efficacy are at odds and safety is usually categorized as a cost [33].
Today, this approach is no longer acceptable. Any poorly value-oriented design can result in a lack of
technological transformation due to barriers for the adoption of technologies [34] or serious ethical
issues. Besides the huge investments required for the current industrial transformation, companies are
experiencing significant worker hesitation to adopt Industry 4.0 technologies. This might be intended
as a genuine concern due to the lack of a robust ethical framework that encompasses all the aspects of
the Factory of the Future.

3. Ethics and Technology in Industrial Enterprises: A Survey

In order to ascertain to what extent the industrial community is attentive to the human-centric
evolution of the Industry 4.0 paradigm towards an Industry 5.0, and whether human values and
ethical issues represent a barrier for adoption and an actual concern for the workers, a survey has
been administered to industry leaders from different companies already leading the 4th industrial
revolution at different levels and with different purposes.

3.1. Survey Structure and Administration

A Scalable Information Dimension (SID) and Value-oriented Semi-structured Survey [35] has been
constructed to tap into respondents’ understandings, views and values about the technologies adopted
to augment the capabilities of the workers as well as to tease apart the influence of a technology’s
pervasiveness, proximity, granularity of information, and other scalable dimensions [35]. After the
opening section for the collection of anonymous socio-demographic data, the survey comprises of four
sections with a total of 42 questions. The questions delve into their prior knowledge and opinions about
a large set of technologies enabling today the human–machine symbiosis in industrial environments,
examine the interest of their enterprises in these kinds of technologies, and elicit human values
and ethical questions related to the augmentation of workers’ capabilities in a hypothetical Industry
5.0 scenario. For the sake of simplicity, technologies included in the survey were identified with
the following acronyms: Activity and Motion Trackers (AT); Artificial Intelligence, including all its
subfields (AI); Augmented Reality (AR); Big Data (BD); Cloud Computing (CC); Collaborative Robots
(COB); Digital Voice-enabled Assistants (DA); Exoskeletons (EXO); Health Monitoring Devices (HMD);
Human-Machine Interface (HMI); Internet of Things, including machine sensors and identification
Appl. Sci. 2020, 10, 4182 6 of 25

sensors, e.g., RFID (IOT); Mobile Devices, e.g., smartphone or tablets (MD); Simulation (SIM); Social
Networks, including tools for collaborative working and smart interaction with work equipment (SN);
Teleoperated Systems, including devices for remote supervision and control, e.g., PLCs, actuators
(TOS); Virtual Reality (VR); Wearable Devices, including smart clothes and smart watches (WD);
and Work Environment Sensors, including visual, temperature or other sensors (WES).
In order to aid the elicitation process, this survey applies Schwartz’s 10 values to the workplace,
thus defining a potential map of work values that guide individuals in work-related activities [36].
Items were created in the form of brief statements which reveal diverse aspects of each value type
(three items per dimension, for a total of 30 items) and describe an example concrete work situation
considered important by a hypothetical person (e.g., ‘for him/her it is important . . . to succeed more
than others’). Participants were asked to evaluate the extent to which they view themselves as similar
to the person in the item on a 5-point Likert scale (from 1 = ‘Not at all like me’ to 5 = ‘Very much
like me’). To enhance the reliability of the survey, in the other questions, the same values were proposed
by using different wording and synonyms (e.g., the value ‘Achievement’ was replaced by ‘recognition
of one’s abilities and results’). Furthermore, throughout the value section of the survey, the values from
VSD [37] that are considered important for technological design were also proposed to the respondents
as an input for the elicitation, namely Welfare, Ownership and Property, Privacy, Freedom from Bias,
Universal Usability, Trust, Autonomy, Informed Consent, Accountability, Courtesy, Identity, Calmness
and Environmental Sustainability.
The survey was designed in Google Forms for its ease of access and use, and distributed via
email to industry experts. At the beginning of the survey, the respondents were informed of the
survey’s purposes, terms and conditions. Thereafter, the Operator 4.0 concept and the technologies
enabling human–machine symbiosis in the Factory of the Future were briefly presented to uniform
their interpretation of the context under study. Illustrative images of mock-ups, narratives, and stories
of use of 4.0 technologies as well as videos of workers using prototypes of such technologies in daily
situations were also used to investigate value implications of technologies that are yet to be built or
widely adopted. An example video that shows one of the prototypes of a voice-enabled knowledge
assistant on the factory floor used for the survey is provided in the supplementary materials.

3.2. Participants
About a hundred different-sized enterprises, with at least one Italian headquarter and currently
transitioning (according to the press and their website information) from Industry 3.0 to Industry
4.0, were selected from several industry sectors, including Manufacturing, Industrial Automation,
Electronics, Oil&Gas, Energy, Shipping, Waste Management, Food, Fashion, Construction, and ICT.
ICT enterprises were included in this survey, albeit not strictly related to the industrial and
manufacturing domain, given their role in designing new technologies. Eighteen enterprises
responded to the survey: three microenterprises (fewer than 10 employees), eight small enterprises
(10 to 49 employees), two medium-sized enterprises (50 to 249 employees), and five large enterprises
(250 or more employees). Reports from different online survey platforms demonstrated that surveys
with lower response rates (near 20%) had more accurate measurements compared to surveys with
higher response rates (near 60 or 70%) [38]. The answers’ accuracy was crosschecked in this study
during some follow-up meetings and emails with the respondents. It is also important to mention that
the aim of the survey is not to carry out a statistical analysis over a huge dataset, but to understand
which is the first feeling and perception of industry leaders in different industrial sectors about the
topics of the Operator 4.0 (or Augmented Operator) as well as Value-oriented and Ethical Technology
Engineering. The survey, which begins with a section gathering anonymous socio-demographic data,
was answered by 18 people (83% males), ranging in age from 28 to 64 and distributed as follows:
11.1% between 25–34 y.o., 38.9% between 35–44 y.o., and 22.2% between 45–54 y.o. The majority of
the respondents (≈ 72%) hold a managerial role, while the remaining ones are operatives, specialists
and technicians. Their professional experience is as follows: 5.6% less than 5 years, 22.2% between
 Appl. Sci. 2020, 10, 4182 7 of 25

5–10 years, 27.8% between 11–15 years, 16.7% between 16–20 years, 11.1% between 21–25 years,
5.6% between 26–30 years, and 11.1% more than 30 years. With respect to their level of education, the
61.1% reported to have a master degree, the 22.2% a bachelor’s degree, while the remaining ones have
other post-graduate education (e.g., Ph.D. or Professional Master).

3.3. Survey Results and Discussion

Appl. Sci. 2020, 10, x FOR PEER REVIEW 7 of 25
3.3.1. Section 1: Prior Knowledge
271 3.3.1. Section 1: Prior Knowledge
The human-centric design of the Factory of the Future and the Operator 4.0 rose in popularity
272 over theThe last
human-centric
few years design of the Factory
in the industrial of the Future
community. Onlyand the Operator
11.1% 4.0 rose in popularity
of the respondents never heard
273 over the last few years in the industrial community. Only 11.1% of the respondents never heard about
about this concept, while the remaining respondents heard about this shift towards a human-centered
274 this concept, while the remaining respondents heard about this shift towards a human-centered
industry from meetings, fairs, and conferences, but also newspapers and political talks. The 72.2%
275 industry from meetings, fairs, and conferences, but also newspapers and political talks. The 72.2% of
of the surveyed companies have already initiated a roadmap for the transformation of the worker
276 the surveyed companies have already initiated a roadmap for the transformation of the worker into
into an Operator 4.0, while all the respondents reported that the management of their enterprises
277 an Operator 4.0, while all the respondents reported that the management of their enterprises show
278 show an obvious interest to implement and use such technologies. It is notable that technologies
an obvious interest to implement and use such technologies. It is notable that technologies enabling
279 enabling human–machine
human–machine symbiosis symbiosis in an industrial
in an industrial context
context do do not receive
not receive the same thelevel
sameoflevel of attention.
attention. A
280 Asnapshot
snapshotofofthe thedigital
digital transformation
transformation of of
thethe enterprises
enterprises underunder
studystudy in this
in this workwork is presented
is presented in in
281 Figure
Figure3.3. While
While all the technologies
all the technologiesare arewidely
widelyknown
known bybythethe respondents,
respondents, onlyonly
a fewa few of them
of them are are
282 extensively used in the workplace. Among others, IOT, AI, CC, and MD receive
extensively used in the workplace. Among others, IOT, AI, CC, and MD receive the greatest attention the greatest attention
283 atatthe
theexecutive
executive level,
level, probably fortheir
probably for theirobvious
obviousreturn
returnononinvestment.
investment. Nevertheless,
Nevertheless, executives
executives and and
284 managers
managersstill
stillhave
havelimited
limited confidence
confidence with such
with technologies
such technologies andandare are
skeptical about
skeptical theirtheir
about real benefits
real
285 tobenefits
the business. For this reason,
to the business. For thismost of them
reason, mosthave started
of them have to started
implement them at the
to implement workplace
them at the as
286 workplace
piloting as piloting
standalone standalone
solutions solutionsindeveloped
developed in collaborations
collaborations with research with research institutions.
institutions. Furthermore,
287 itFurthermore,
can be noted it cantechnologies
that be noted that suchtechnologies
as AT, EXO such as AT,are
and HMD EXO
not and
usedHMDat the are not usedWhile
workplace. at theEXO
288 workplace. While EXO are characterized by notable investment costs and
are characterized by notable investment costs and still few beneficial applications were documented, still few beneficial
289 ATapplications
and HMDwere imply documented,
significant AT and HMD
privacy imply
concerns forsignificant privacythus
the workforce, concerns
showing for the
how workforce,
a discussion
290 thus showing how a discussion over the ethical issues connected to the use
over the ethical issues connected to the use of technology and their design for values is an of technology and their
imperative
291 indesign for values
Industry 5.0. is an imperative in Industry 5.0.
292
AI
WES 100% AR
WD 80% AT

60%
VR BD
40%

TOS 20% CC
0%

SN COB

SIM DA

MD EXO

IOT HMD
HMI

Percentage of respondents w ho knew the technology

Percentage of respondents w ho used the technology atthe w orkplace
Percentage of enterprises that have adop ted the technology
Percentage of enterprises w here the technology is considered critical at the executive level
293
Figure 3. Prior knowledge and use of the Operator 4.0 enabling technologies.
294 Figure 3. Prior knowledge and use of the Operator 4.0 enabling technologies.

295 3.3.2. Section 2: Opinion

296 Despite a certain skepticism, the respondents unanimously agree on the prospective beneficial
297 impact of augmenting the worker’s capabilities. What emerges from the survey in the first place is
298 the importance to extend the worker’s cognitive capabilities to fill the gap with technology (they had
 Appl. Sci. 2020, 10, 4182 8 of 25

3.3.2. Section 2: Opinion

Despite a certain skepticism, the respondents unanimously agree on the prospective beneficial
impact of augmenting the worker’s capabilities. What emerges from the survey in the first place is
the importance to extend the worker’s cognitive capabilities to fill the gap with technology (they had
a 4.1 score on a 5-point Likert scale). The capability to interact with other workers and with the
CPPS in an intuitive and smart manner resulted to be almost as important as the ‘cognitive’ one
(the ‘interaction capabilities’ scored 3.8), unlike the ‘sensorial capabilities’, whose augmentation is
considered mildly important by the respondents (3.5 score). A more detailed view on this segmentation
is given in the treemap chart of Figure 4 that is a qualitative illustration to compare proportions
within groups of technologies augmenting human’s capabilities. In this chart, each technology is
represented by a rectangular shape, where bigger rectangles indicate a higher perceived beneficial
impact of the technology by the industry expert. The color allows to distinguish the categories
of human capabilities (i.e., cognitive, interaction and sensorial) that the technology augments and
to understand the significance of each technology set. This result is in line with the literature and
the data from the previous section that pictures the current situation of the enterprises’ digital
transformation. Indeed, studies focusing on the augmentation of the worker’s cognitive capabilities
in industrial environments recently emerged in the perspective of the ‘tooling’ scenario. The survey
respondents pointed out several prospective benefits of implementing such technologies, especially at
the field level. Human–robot collaboration would increase production efficiency, whereas an enhanced
problem solving (e.g., AR tools may help maintenance technicians to minimize machine stop time) is
enabled by a faster and intuitive access to data (e.g., workers’ health data), information, competences,
and knowledge available at the workplace. In their opinion, maintenance, manufacturing and process
supervision tasks will be impacted the most, but they do also mention improvements for the capability to
respond
Appl. Sci. to market
2020, changes,
10, x FOR for planning activities, manual assembling, process control, quality checks,
PEER REVIEW 9 of 25
as well as back-office activities if such technologies are integrated with enterprise systems (e.g., MES).

339
Figure 4. Treemap chart of the perceived beneficial impact of Operator 4.0 enabling technologies.
340 Figure 4. Treemap chart of the perceived beneficial impact of Operator 4.0 enabling technologies.
Another significant aspect emerging from this section of the survey refers to the beneficiaries of the
341 current
3.3.3. Section 3–4:transformation
4.0 digital Human Valuesand and explicitly
Ethics calls for an Industry 5.0. Based on the feedback of the
342 industry leaders, the technologies under study are expected agreed
The same inclination holds for when the respondents to positively
to theimpact (e.g.,
fact that the by simplifying)
technologies the
that
343 work of technicians, deskless operators, warehouse workers, and field specialists, but
the workers use (or will use) in the Factory of the Future should embed or be designed for human also production
344 and maintenance
values: managers/supervisors.
88.9% responded ‘yes, definitely’However, they are also
while the remaining the ones ‘maybe’.
responded that are bearing (or will
The majority of bear)
the
345 the negative impacts of this transformation. In the end, a paradox is established:
surveyed enterprises revealed to be very attentive to human values and formal ethical codes of the workers,
346 whose
conductwork
are may get the
available highest
in 72% benefits,
of the are also the
cases, especially in ones that may
the larger observe a deterioration of their
enterprises.
347 However, when it comes to technology, respondents are even more attentive to human values,
348 as demonstrated in Figure 5. The questionnaire items analysis revealed that, while the perceived level
349 of importance for some values does not change when associated to technology, it significantly rises
350 for some others if compared to their personal value scale, such as in the case of ‘conformity’,
351 ‘tradition’, ‘achievement’, but also ‘universalism’ and ‘security’. This result captures the conflict
 Appl. Sci. 2020, 10, x FOR PEER REVIEW 10 of 25
Appl. Sci. 2020, 10, 4182 9 of 25
374 Autonomy, courtesy, calmness, and universal usability also scored more than four among the
375 VSD values. As emerged from the analysis of the Schwartz’s values, ‘universality’ should also be a
work conditions, or even lose their job if the ‘tooling’ scenario evolves in an uncontrolled manner
376 core value in the human-centered design of the Factories of the Future. The human–machine
towards the ‘automation’ scenario. Among others, the respondents reported significant implications,
377 symbiosis
such as the inworker’s
Industryalienation,
5.0 should be faircontrol
a higher and respectful
on working of times
everyone’s dignity
and duties, withand opinionreduced
consequent equally. It
378 should not create relational gaps, but foster social interaction: the respondents
freedom and autonomy for the workers. The collection of personal data increases privacy issues, frequently revealed
379 their
thusconcern
causing about worker
resistance alienation
to change and by
especially work depersonalization
workers that anorethical
with an old mindset design of the
with work-related
380 human–machine
stress. Despite the fact that the workforce downsizing risk is of course the most predominant design
symbiosis in Industry 5.0 should avoid. In this case, poor technology potentialthat
381 does not respect
negative aspect the roles and
of Industry expertise
4.0, of a person
the majority in the company
of the workers may negatively
in the responding impact
enterprises the are
(≈89%) value
382 ofresolutely
experienced willing to undertake this transformation. It then bears investigating the role of humanthat
people and, ultimately, lead to job displacement. Indeed, some respondents argue
383 technology
values and mayethicaloverpower
design for thetheconverging
workers, technologies
for example,enablingif technology is designed
a human–machine with episodic
symbiosis from
384 memory like humans,
the perspective of a 5th with the aim
industrial to improve
revolution the the
that places workers’
humanproblem solving
worker again skills.
at the centerHowever,
of the
385 Factory of the
technology may Future.
also give some workers (e.g. younger ones) the opportunity to overpower
386 (consciously or not) other people (e.g. older ones). Instead, value-oriented technology should be
3.3.3. Section 3–4: Human Values and Ethics
387 centered on the workforce’s continuous learning and growth thanks to a sharing and collaboration
388 mindset The(for
sameexample, knowledge
inclination holds forand when expertise). It is eventually
the respondents agreed to the opportunity
the fact for the executives
that the technologies that
389 themanagers
and workers use to (or will use)
express in the Factory
a ‘positive of theand
leadership’ Future should embed
to establish or bework
a healthy designed for humanand
environment
390 values: 88.9%
workplace culture. responded ‘yes, definitely’ while the remaining responded ‘maybe’. The majority of
391 theSimilarly
surveyed enterprises
to Figure 4, revealed to be very
the treemap attentive
chart of Figureto human
6 is avalues and formal
qualitative ethical codes
illustration to make of a
392 conduct are available in 72% of the cases, especially in the larger enterprises.
comparison between and within groups of technologies augmenting human’s capabilities where the
393 size ofHowever, when itshape
a rectangular comesis to proportional
technology, respondents are even more
to the respondents’ attentive
concern andto opinion
human values,
that the
394 as demonstrated in Figure 5. The questionnaire items analysis revealed
technology implies bigger ethical issues and challenges. It shows that the respondents expressed that, while the perceived
level of importance for some values does not change when associated to technology, it significantly
395 ethical concerns mainly towards BD and AI, while SN, DA, HMI, and AT were of second-most
rises for some others if compared to their personal value scale, such as in the case of ‘conformity’,
396 criticality. Although the ethical issues connected to these technologies are quite well-known, the low
‘tradition’, ‘achievement’, but also ‘universalism’ and ‘security’. This result captures the conflict
397 awareness towards the potential ethical issues that may derive from the use of co-bots at workplace
between the disruptive effect of technology and the workers’ inclination to preserve the past and
398 is a surprising result.
resist change in the workplace in order to maintain the status quo. The ‘security’ value confirms this
399 It can be concluded that, despite some skepticism by small enterprises, all the respondents
interpretation, as workers—especially older adults—strive to get stability and safety in their workplace
400 reported that the workers are aware of the ethical issues in the workplace and agree on the fact that
(something that the introduction of new technologies cannot guarantee in their opinion). In contrast to
401 a value-oriented
the ‘conservation’ design of a future
mindset, human–machine
workers—in symbiosis
particular younger in industrial
adults—also aimorganizations
at reaching goalsmayand aid in
402 the amelioration of the ethical issues associated with technologies
personal success, demonstrating their competence with technological solutions. and lead the enterprises towards a
403 new industrial age, Industry 5.0.

404
405 Figure 5.
Figure Perceived importance
5. Perceived importanceof
ofthe
theSchwartz’s
Schwartz’sten values.
ten values.
Appl. Sci. 2020, 10, 4182 10 of 25

The analysis of the extent to which values from VSD should be considered when designing
new technologies and solutions for the factory workers confirmed the findings above. All the values
were rated on a range between 3.45 (for ‘Ownership & Property’) and 4.36 (for ‘Trust’). It proves that
respondents are concerned about the fact that technology should be trustful, serve for the common good
and be designed to prevent any possible misuse. Linked to the need of digital trust, the respondents
cited ‘honesty’, ‘explainability’ and ‘transparency’ as well as ‘integrity’ and ‘professionality’ as the
foremost values that any technology used by the Operator 4.0 at the workplace should embed.
In fact, the analysis of a list of seven ethical issues proposed to the respondents—including unethical
conduct (lying, deception, theft, . . . ), toxic workplace culture, discrimination and harassment,
privacy and confidentiality, unethical leadership, unrealistic and conflicting goals, and misuse of
technology—corroborated the value of the ethical conduct. Despite the fact that all the issues scored
a mean value between 3.36 and 4.09 on a 5-point Likert scale (thus demonstrating the importance of
a design for ethics), digital and automation technology is explicitly considered as a tool that should
support accountability and responsibility. This may be seen somehow in contrast to the values of
second-most importance for the respondents, namely informed consent, privacy and confidentiality.
Autonomy, courtesy, calmness, and universal usability also scored more than four among the VSD
values. As emerged from the analysis of the Schwartz’s values, ‘universality’ should also be a core
value in the human-centered design of the Factories of the Future. The human–machine symbiosis in
Industry 5.0 should be fair and respectful of everyone’s dignity and opinion equally. It should not
create relational gaps, but foster social interaction: the respondents frequently revealed their concern
about worker alienation and work depersonalization that an ethical design of the human–machine
symbiosis in Industry 5.0 should avoid. In this case, poor technology design that does not respect
the roles and expertise of a person in the company may negatively impact the value of experienced
people and, ultimately, lead to job displacement. Indeed, some respondents argue that technology may
overpower the workers, for example, if technology is designed with episodic memory like humans,
with the aim to improve the workers’ problem solving skills. However, technology may also give
some workers (e.g., younger ones) the opportunity to overpower (consciously or not) other people
(e.g., older ones). Instead, value-oriented technology should be centered on the workforce’s continuous
learning and growth thanks to a sharing and collaboration mindset (for example, knowledge and
expertise). It is eventually the opportunity for the executives and managers to express a ‘positive
leadership’ and to establish a healthy work environment and workplace culture.
Similarly to Figure 4, the treemap chart of Figure 6 is a qualitative illustration to make a comparison
between and within groups of technologies augmenting human’s capabilities where the size of
a rectangular shape is proportional to the respondents’ concern and opinion that the technology implies
bigger ethical issues and challenges. It shows that the respondents expressed ethical concerns mainly
towards BD and AI, while SN, DA, HMI, and AT were of second-most criticality. Although the ethical
issues connected to these technologies are quite well-known, the low awareness towards the potential
ethical issues that may derive from the use of co-bots at workplace is a surprising result.
It can be concluded that, despite some skepticism by small enterprises, all the respondents
reported that the workers are aware of the ethical issues in the workplace and agree on the fact that
a value-oriented design of a future human–machine symbiosis in industrial organizations may aid in
the amelioration of the ethical issues associated with technologies and lead the enterprises towards
a new industrial age, Industry 5.0.
 Appl.
Appl.Sci. 2020,10,
Sci.2020, 10,4182
x FOR PEER REVIEW 1111ofof2525

406
Figure 6. Treemap chart of the perception of the most ethically problematic technologies in industry.
407 Figure 6. Treemap chart of the perception of the most ethically problematic technologies in industry.
4. Methods
408 4. Methods
4.1. Value Sensitive Design in the Industry 5.0 Era
409 4.1. Value Sensitive Design in the Industry 5.0 Era
Currently used by the European Union’s Framework Programmes, Responsible Research and
410 Innovation
Currently(RRI)used
is founded
by the on an interactionist
European principle that
Union's Framework technology and
Programmes, humans co-construct
Responsible Research and
411 and co-vary one another [7]. Several design approaches to technology engineering
Innovation (RRI) is founded on an interactionist principle that technology and humans co-construct have been proposed
412 that
andaccount
co-varyforone theanother
social embeddedness of technologies
[7]. Several design approaches andto their impacts. engineering
technology Particular emphasishave been is
413 given to thethat
proposed involvement
account for andtheelicitation of stakeholders,ofeither
social embeddedness those directly
technologies or indirectly
and their impacts.implicated
Particular
414 by technology
emphasis design.
is given Approaches
to the involvement such
andaselicitation
universalofdesign [39], inclusive
stakeholders, design
either those [40], sustainable
directly or indirectly
415 design [41], participatory
implicated by technologydesign design.[42], and valuessuch
Approaches sensitive design [43–45],
as universal among
design [39], others design
inclusive have been [40],
416 theorized;
sustainable Value
designSensitive Design (VSD)
[41], participatory becoming
design [42], one
and of the most
values popular
sensitive approaches
design over theothers
[43–45], among last
417 20have
years [46].theorized;
been EmergingValue from Sensitive
the field of human–computer
Design (VSD) becoming interaction,
one of the VSD is precisely
most popular predicated
approaches
418 on thethe
over same lastpremise
20 yearsthat
[46].technology
Emergingisfrom not the
value-neutral, rather it is sensitive
field of human–computer to stakeholder
interaction, values,
VSD is precisely
419 whether they are direct stakeholders such as users and designers
predicated on the same premise that technology is not value-neutral, rather it is sensitive or indirect such as industry CEO’s,to
420 governments,
stakeholder values,and thewhether
biosphere theyat are
large [35,47].
direct stakeholders such as users and designers or indirect such
421 VSD is chosen
as industry CEO’s,asgovernments,
the theoreticaland andthemethodological
biosphere at large framework[35,47].to guide the design of the Factory
422 of the VSD
Future in the perspective
is chosen of Industry
as the theoretical 5.0 for two main
and methodological reasons: (1)
framework to because
guide the of design
its inherent
of the
423 self-reflexivity
Factory of the and emphasis
Future in theon perspective
enrolling both of direct
Industry and 5.0
indirect
for twostakeholders into the
main reasons: (1)design
because process
of its
424 and the philosophical
inherent self-reflexivity investigation
and emphasis of implicated
on enrolling values,
botha direct
coupling andthat is notstakeholders
indirect explicitly called intoforthe
425 asdesign process and
a fundamental the philosophical
process in other designinvestigation
approaches [48]; of implicated
and (2) for values, a coupling
its robustness, that isthe
meaning not
426 scope and applicability
explicitly called for asofa VSD across different
fundamental processdesign
in other spaces.
design VSD aims to account
approaches [48]; and for (2)
multiple
for its
427 robustness,
dynamic meaning
variables the scope
ranging from aand wideapplicability of VSD
array of values, across different
technologies, socialdesign
contexts,spaces. VSD aims to
and stakeholder
428 groups
account among others. In
for multiple doing so,
dynamic VSD promotes
variables rangingworking
from a wide at thearray
meta-level rathertechnologies,
of values, than focusing on
social
429 ancontexts,
exhaustive andset stakeholder
of values suchgroups amongorothers.
as safety usability,In doing so, VSD promotes
or a particular working such
set of stakeholders at theasmeta-
the
430 level rather than This
differently-abled. focusing on an exhaustive
meta-level set of values
approach divorces VSD suchfrom as any safety or usability,
specific or a particular
moral or theoretical vows set
431 of permits
and stakeholdersit to besuch
widelyas applicable
the differently-abled. This meta-level
across sociodemographic approach
contexts divorces
and design spaces.VSD To from any
this end,
432 specific
the moral or theoretical
VSD methodology aims to be vows and permits
fundamentally it to beand
proactive widely applicable
anticipatory, across with
engaging sociodemographic
stakeholders
433 tocontexts and design spaces.
guide technological To thisinend,
development the VSD
Industry 5.0 methodology
from an early aims stage to
andberecursively
fundamentally proactive
self-improve
434 and anticipatory,
throughout the designengaging
processwith stakeholders
as new to guide emerge.
values or tensions technological development
This reflexivity in Industry
towards beneficial 5.0
435 from an early stage and recursively self-improve throughout the design process as new values or
development is on account of the approach’s tripartite structure illustrated in Figure 7. Three types of
436 tensions emerge. This conceptual,
investigations—namely reflexivity towards
empirical beneficial
and technical development is on account
investigations—are of the approach’s
necessarily iterative
437 tripartite
and structure
in a continual illustrated
feedback cycle in
[49].Figure 7. Three types of investigations—namely conceptual,
438 empirical and technical investigations—are necessarily iterative and in a continual feedback cycle
439 [49].
 Appl. Sci. 2020, 10, 4182 12 of 25
Appl. Sci. 2020, 10, x FOR PEER REVIEW 12 of 25

440
441 Figure 7.
Figure The tripartite
7. The tripartite iterative
iterative Value
Value Sensitive
Sensitive Design
Design methodology.
methodology.

Not only this, but VSD encourages designers to consider always building malleable and
442 Not only this, but VSD encourages designers to consider always building malleable and flexible
flexible architectures into technologies to permit post-deployment augmentations to be possible
443 architectures into technologies to permit post-deployment augmentations to be possible as new
as new values and issues emerge [12]. Previous studies have looked at how VSD can be applied
444 values and issues emerge [12]. Previous studies have looked at how VSD can be applied to discrete
to discrete technologies while generally referring to their interactions with other technologies in
445 technologies while generally referring to their interactions with other technologies in only a cursory
only a cursory way. To illustrate, VSD has been applied to energy systems [50,51], mobile phone
446 way. To illustrate, VSD has been applied to energy systems [50,51], mobile phone usage [52],
usage [52], architecture projects [53], and augmented reality systems just to name a few [54]. However,
447 architecture projects [53], and augmented reality systems just to name a few [54]. However, it works
it works speculatively too for future technologies, both near and long term. Examples include its
448 speculatively too for future technologies, both near and long term. Examples include its exploratory
exploratory application to nanopharmaceuticals [55], molecular manufacturing [45], intelligent agent
449 application to nanopharmaceuticals [55], molecular manufacturing [45], intelligent agent systems
systems [56], and, less futuristic, autonomous vehicles [57]. Although these studies provide useful
450 [56], and, less futuristic, autonomous vehicles [57]. Although these studies provide useful
illustrations of the intricacies of how the VSD approach can be accommodated to various technologies
451 illustrations of the intricacies of how the VSD approach can be accommodated to various technologies
in particular, they do not provide discussions of the convergence factor of these technologies in general
452 in particular, they do not provide discussions of the convergence factor of these technologies in
or how they can be designed with this structural embeddedness and interdependence. Furthermore,
453 general or how they can be designed with this structural embeddedness and interdependence.
no applications of VSD to the industrial domain have been found so far in the literature. Guidelines
454 Furthermore, no applications of VSD to the industrial domain have been found so far in the literature.
and recommendations for the three different investigations are proposed in the following subsections,
455 Guidelines and recommendations for the three different investigations are proposed in the following
while use cases are illustrated in the next section.
456 subsections, while use cases are illustrated in the next section.
4.2. Conceptual Investigations
457 4.2. Conceptual Investigations
The VSD approach aims to embed stakeholder values both early on and throughout the design
458 The VSD approach aims to embed stakeholder values both early on and throughout the design
process to steer the design of the technology in such a way as to successfully map the values deemed
459 process to steer the design of the technology in such a way as to successfully map the values deemed
critical [12,45]. Similar to other human experiences, work experiences are ‘objects’ rich in meaning
460 critical [12,45]. Similar to other human experiences, work experiences are ‘objects’ rich in meaning
and defined by both individual and collective values. Indeed, values contribute to ideally delineating
461 and defined by both individual and collective values. Indeed, values contribute to ideally delineating
the job with respect to potential and/or favorable goals/aims to pursue. Hence, a more holistic
462 the job with respect to potential and/or favorable goals/aims to pursue. Hence, a more holistic
approach, taking into account sets of values (rather than individual values) is highly preferable,
463 approach, taking into account sets of values (rather than individual values) is highly preferable, as it
as it allows a more comprehensive view and further balancing of values. The starting point of
464 allows a more comprehensive view and further balancing of values. The starting point of a conceptual
a conceptual investigation is to look for any valuable literature containing list of values, from different
465 investigation is to look for any valuable literature containing list of values, from different ethical
ethical perspectives, regardless of the context for which they are intended or in which they are used.
466 perspectives, regardless of the context for which they are intended or in which they are used. Using
Using well-consolidated value system measures is a good starting point for this analysis. Research has
467 well-consolidated value system measures is a good starting point for this analysis. Research has
proposed and utilized the WVI—Work Values Inventory [58], the WIS—Work Importance Study [59]
468 proposed and utilized the WVI—Work Values Inventory [58], the WIS—Work Importance Study [59]
and the PVQ—Portrait Values Questionnaire [60]. Beyond theoretical discussions, value lists that are
469 and the PVQ—Portrait Values Questionnaire [60]. Beyond theoretical discussions, value lists that are
prominent in particular disciplines or professions (i.e., professional codes of ethics or ethical codes
470 prominent in particular disciplines or professions (i.e. professional codes of ethics or ethical codes of
of conduct) as well as application-based approaches (e.g., Google’s AI principles) can be assessed.
471 conduct) as well as application-based approaches (e.g. Google’s AI principles) can be assessed. Lists
Lists relevant for novel and emerging technologies include the anticipatory emerging technology ethics
472 relevant for novel and emerging technologies include the anticipatory emerging technology ethics
list [61] or the technomoral value list [62]. Technomoral values include the following 10 values as most
473 list [61] or the technomoral value list [62]. Technomoral values include the following 10 values as
worthy of upholding: honesty, self-control, humility, justice, courage, empathy, care, civility, flexibility,
474 most worthy of upholding: honesty, self-control, humility, justice, courage, empathy, care, civility,
perspective, magnanimity, and technomoral wisdom. Instead, the values listed in anticipatory emerging
475 flexibility, perspective, magnanimity, and technomoral wisdom. Instead, the values listed in
476 anticipatory emerging technology ethics are the following: harms and risks, rights and freedoms,
477 autonomy, human dignity, privacy, property, animal rights and animal welfare, (distributive) justice,
Appl. Sci. 2020, 10, 4182 13 of 25

technology ethics are the following: harms and risks, rights and freedoms, autonomy, human dignity,
privacy, property, animal rights and animal welfare, (distributive) justice, and well-being and the
common good. The recent interest towards the implementation of values into emerging technologies,
such as Big Data, find application in [63], that distilled a comprehensive set of 10 ethical values for
new technologies: human welfare, autonomy, non-maleficence, justice, accountability, trustworthiness,
privacy, dignity, solidarity, and environmental welfare. VSD studies also have provided various lists of
human values that may be important to technologies design, they typically include values such as:
Welfare, Ownership and Property, Privacy, Freedom from Bias, Universal Usability, Trust, Autonomy,
Informed Consent, Accountability, Courtesy, Identity, Calmness and Environmental Sustainability
among others. Besides the document review of relevant literature and research projects dealing with the
value-oriented design of the Factory of the Future and related technologies or the analysis of industrial
practices, there are a multitude of strategies from the social sciences that establish ways of eliciting
such values such as the use of stakeholder tokens, value source analysis, and value scenarios among
others [64–66]. After having selected the most appropriate lists and narrowed down the spectrum
of values, integration and contextualization for the specific technological solution being adopted at
the workplace is the next step. It is fundamental to distil an a priori comprehensive list of values
from previous conceptual work in an abstract manner. A list of values to uphold during techno-social
change toward human–machine symbiosis in Industry 5.0 and recommendations for value-oriented
design of technologies are presented in Table 1. The list is of course not exhaustive given that there
exists no framework to provide such a list; certain values are technology-specific and some emerge
over the design process and, in many cases, after the deployment of the technology [67]. Speaking
about robotics, for example, [30] identified the following values: social interaction, movement and
exercise, (human) autonomy, problem solving, task variety, and building with their hands. However,
values may have common points of contact and are not mutually exclusive. They overlap to a certain
extent; therefore, the list can be considered a priori comprehensive.

Table 1. Values to uphold during techno-social change towards human–machine symbiosis in

Industry 5.0.

Human Value Technology Shall Be Designed to . . .

Self-actualization . . . enable the workers to accomplish their own goals and obtain personal success, to
demonstrate their own capabilities and competence in order to advance their career.
Accountability . . . ensure full transparency of how the symbiotic CPPS operates and makes decision.
It includes that the actions of a worker or group of workers is shared transparently to
ensure that any problem can be traced back to the cause.
Trustworthiness . . . be honest, trustworthy, reliable, and avoid systematic bias, thus performing its
required function under given conditions for a stated time interval, as well as to improve
interpersonal trust in a human-machine hybrid system.
Privacy . . . ensure informational privacy, respect for the workers’ private sphere and the right to
determine what information can be communicated to others through informed consent.
Welfare . . . ensure workers’ health (physical well-being and peaceful psychological state) also
thanks to a proper work-life balance, a balanced workload and a comfortable and pleasant
work environment.
Autonomy . . . allow workers to be independent, free and flexible while at work in order to express
themselves, engage in autonomous thinking, make independent choices, utilize their
creativity and fully use their own intelligence.
Altruism . . . enhance teamwork and mutual care among the workers and to favour the wellbeing of
all those one comes into contact with during one’s professional activities, thus
demonstrating oneself as sincere, open to helping and responsible.
Appl. Sci. 2020, 10, 4182 14 of 25

Table 1. Cont.

Human Value Technology Shall Be Designed to . . .

Common Good . . . ensure environmental welfare and sustainability as well as to contribute to the beauty
of the world and make good things.
Security . . . provide a secure return (economic or abstract) to the workers and preserve order,
stability and harmony within the physical environment, work-related relationships and
professional activities.
Stimulation . . . keep stimulating workers with challenging, various and novel tasks, that require
continuous learning and professional growth and may sometimes require even a certain
level of risk.
Sociability . . . stimulate good social relationships and interactions with fellow workers, thus working
in a social environment rather than in isolation.
Identity . . . to ensure people’s equality, universality, respect and inclusion and avoid
discrimination of any kind. It also requires proper actions to guarantee to everyone the
same possibilities of others as well as a certain amount of human control over the events.
Authority . . . allow workers to attain and/or exercise a prestigious or authoritative position and/or
social status that increases one’s influence or ability to control other members of the
organization and resources.
Conformity . . . support the workers to respect rules and expectations, thus demonstrating social
discipline and loyalty. At the same time, it shall restrict one’s actions and/or conditioning
one’s choice, inclinations, impulses and desires.

4.3. Technical Investigations

Technical investigations follow up on the previous analyses to determine how the technical
aspects of the technology itself can support or restrict the values distilled in the previous investigation.
Value Sensitive Design adopts the position that technologies in general provide value that follow from
their properties. That is, a given technology is more suitable for certain activities and more readily
supports certain values while rendering other activities and values more difficult to realize. In one form,
the ultimate aim is to move from the values to the design requirements through the definition of norms.
As mentioned, values can often be abstract philosophical concepts that are difficult to conceptualize in
terms of concrete engineering requirements. Because of this, VSD aims at the embodiment of values
that extend beyond—but not excluding—economic value, which is what technological solutions are
traditionally designed for. Norms, which are situated in the transition point between values and
design requirements, are the contextual designations of those values and can be understood as the
design objectives of any given project (i.e., ‘maximize safety/usability/efficiency’ or ‘minimize cost’).
The norms of any given value are established on the basis of the industrial context-of-use in which
the technology is or will eventually be deployed. To this end, the norms of any particular context
can be used to derive specific design requirements that the technology can embody to support any
given set of norms. Doing this appears prima facie abstract and meta-level, and for this reason we
provide some salient use cases to illustrate how designers can begin to conceptualize the bi-directional
process. Van de Poel is clear in saying that the ‘elements at the lower levels cannot be logically
deduced from the higher level elements’ [68]. The reason for this is because there is no one avenue
from values to design requirements; there are numerous ways that values can be translated and
embodied in design requirements. In many design spaces, the successful construction of a value
hierarchy will almost certainly need to be conducted in both directions. Figure 8 illustrates simply the
value-norm-requirement hierarchy. Each of the norms and design requirements are not inextricably
exclusive to any single value. Any of the below design requirements can just as easily satisfy other
norms which in turn can satisfy other values. What engineering teams need to do is specify as best
as possible how different design requirements, norms and values relate to one another in any given
design project and determine how to most aptly satisfy them.
 Appl. Sci. 2020, 10, 4182 15 of 25
Appl. Sci. 2020, 10, x FOR PEER REVIEW 15 of 25

533
Figure 8. Bi-directional value-norm-requirement hierarchy.
534 Figure 8. Bi-directional value-norm-requirement hierarchy.
4.4. Empirical Investigations
535 4.4. Empirical Investigations
Empirical investigations conclude the iteration of the VSD cycle by enrolling stakeholders to
536 Empirical
determine investigations
their relationship conclude the iteration of
with the technologies in the VSD
Factorycycle
of bythe enrolling
Future and stakeholders
how well the to
537 determine their relationship with the technologies in the Factory of the Future
identified values map on to the current design architecture [35]. As such, the functional directive of and how well the
538 identified
this step isvalues mapstakeholders
to enroll on to the current
directly design architecture
into the [35]. As
design process bysuch, the functional
eliciting their values directive
in orderofto
539 this steptechnologies
design is to enroll stakeholders directly into
that more accurately the design
mirror processMethods
their values. by eliciting their but
include values
are in
notorder to
limited
540 design technologies that more accurately mirror their values. Methods include
to scalable information dimensions, value-oriented semi-structured surveys, analysis of prospective but are not limited to
541 scalable
workinginformation
scenarios ordimensions,
value-oriented value-oriented
mock-ups and semi-structured
prototypes. In surveys,
any case,analysis
empiricalofinvestigations
prospective
542 working scenarios or value-oriented mock-ups and prototypes. In any case,
will eventually identify ethical issues with a bottom-up approach and answer the following questions:empirical investigations
543 will eventually identify ethical issues with a bottom-up approach and answer the following
• Which values should be driving technology design and development to achieve a human–machine
544 questions:
symbiosis in the Factory of the Future?
545  Which values should be driving technology design and development to achieve a human–
• Which values are involved with the technology at today’s point in its development in industries?
546 machine symbiosis in the Factory of the Future?
547 • Which
How do thoseare
values critical values
involved in the
with industrial workplaces
technology at today’salign with
point societal
in its prioritiesinatindustries?
development large?
548 • How do stakeholders prioritize competing values in design trade-offs?
How do those critical values in industrial workplaces align with societal priorities at large?
549 • Howdo
How dostakeholders
they prioritize individual
prioritize values and
competing usability
values considerations?
in design trade-offs?
550 • How Are there differences
do they prioritize between espoused
individual valuespractice (what people
and usability say) compared with actual practice
considerations?
551  (what
Are people
there do)?
differences between espoused practice (what people say) compared with actual
552 • practice
Do organizations
(what people dodo)?appropriate value considerations during the technology design and
553  implementation
Do organizationsprocess?
do appropriate value considerations during the technology design and
554 implementation
This process shouldprocess?
be reiterated every time there are technological advancements, when people
555 at the workplace changebeorreiterated
This process should the socialevery timechanges.
context there areThis
technological advancements,
is done continually through whenthepeople
design
556 atprocess.
the workplace
As new empirical values emerge and are defined, they may come into conflict with thedesign
change or the social context changes. This is done continually through the a priori
557 process. As newdelineated
values initially empirical values emergeinvestigations,
in conceptual and are defined, they may
triggering come into conflict
a revaluation with the
of how those a
initial
558 priori values initially delineated in conceptual investigations, triggering
values and their technical understanding impact design. Similarly, post-deployment of any given a revaluation of how those
559 initial
system values and their
may trigger the technical
emergenceunderstanding
of a previouslyimpact design.value,
unconsidered Similarly, post-deployment
triggering a revaluationof of any
how
560 given system may trigger the emergence of a previously unconsidered
to account for such emergence and the technical capacity for actualizing such a change. value, triggering a revaluation
561 of how to account for such emergence and the technical capacity for actualizing such a change.
 Appl. Sci. 2020, 10, 4182 16 of 25
Appl. Sci. 2020, 10, x FOR PEER REVIEW 16 of 25

562 5. Use
5. Cases: Preliminary
Use Cases: Preliminary Results
Results and
and Discussion
Discussion
563 Three uses
Three usescases
caseswere
weredefined
definedwith
withthe
the purpose
purpose to to explore
explore thethe ethical
ethical issues
issues concealed
concealed behindbehind
the
564 the implementation of specific technological solutions towards a human–machine
implementation of specific technological solutions towards a human–machine symbiosis in Industry 5.0. symbiosis in
565 Industry
The 5.0. The
use cases builduseupon
casessolutions
build upon
andsolutions
prototypesanddeveloped
prototypes bydeveloped
CAL-TEKby CAL-TEK
S.r.l., S.r.l.,
a Spin-off a Spin-
company
566 off company of the University of Calabria, Italy and carried out at the Modeling
of the University of Calabria, Italy and carried out at the Modeling & Simulation Center-Laboratory & Simulation Center-
567 Laboratory
of EnterpriseofSolutions
Enterprise Solutions at(MSC-LES)
(MSC-LES) the Departmentat the of Department
Mechanical, of Energy
Mechanical, Energy and
and Management
568 ManagementofEngineering
Engineering the University of the University of Calabria.
of Calabria.
569 Regarding the empirical investigationsfor
Regarding the empirical investigations forthese
theseuse usecases, experts
cases, expertswere consulted
were consulted within the
within
570 professional networks of the researchers involved in the projects, but also
the professional networks of the researchers involved in the projects, but also at expert forums. at expert forums. In
571 particular,
In during
particular, during a major
a majorIndustry
Industry4.0
4.0conference,
conference,experts
expertswere
wereasked
asked to to consider
consider the the prototypes
prototypes
572 illustrated here in the following and identify the values from Table 1 that are
illustrated here in the following and identify the values from Table 1 that are more relevant or more relevant or identify
identify
573 new ones. In most instances, experts confirmed that the proposed list of values
new ones. In most instances, experts confirmed that the proposed list of values was comprehensive. was comprehensive.
574 Thediscussions
The discussionswith with
themthem
werewere particularly
particularly valuablevaluable in deepening
in deepening the understanding
the understanding of and thinkingof and
on
575 thinking on how to regard particular values when designing technological
how to regard particular values when designing technological solutions for achieving human–machine solutions for achieving
576 human–machine
symbiosis symbiosis
in the context in the context
of Industry 5.0. of Industry 5.0.

577 5.1. Use Case #1: Augmenting

Augmenting the Workers’ Perception Capabilities ->
-> Monitoring
Monitoring Workers’
Workers’ Health
Health and
578 Movements via Wearable
Wearable Sensors
Sensors
579 Wearable technology, including
including smart
smartgarments
garments(e.g.
(e.g., gloves)
gloves) wristbands
wristbands andand smartwatches,
smartwatches, are
580 are designed
designed to collect
to collect thethe data
data of of workers’
workers’ personal
personal healthasaswell
health wellasastototrack
track the
the movement
movement of the
581 operators. Prototypes were developed to track the movements of the worker’s arms and fingers and,
582 at the same
same time, to collect
collect data
data about
about their
their heart
heart rate
rate variability
variability to
to monitor
monitor the task-related
task-related stress
stress
583 during manual assembling tasks in a manufacturing company (Figure (Figure 9).
9). It
It is
is worth mentioning that
584 using these devices requires handling
handling and processing
processing big data,
data, therefore,
therefore, the the developed
developed solution
solution
585 embeds big data analytics methodologies.
analytics methodologies.
586

587
588 9. Example of wearable technology—from left to right, wristbands, gloves, smartwatches—
Figure 9.
589 augmenting
augmenting workers’
workers’ perception
perception capabilities.
capabilities.

590 Figure
Figure 10
10 lays
lays out
out one
one potential
potential way
way toto translate
translate values,
values, through
through norms
norms andand into
into design
design
591 requirements that embody the value of welfare. The value was considered important
requirements that embody the value of welfare. The value was considered important in the in the environments
592 in which the augmented
environments in which operator works. As
the augmented mentioned
operator above,
works. the bi-directional
As mentioned above,hierarchy of values
the bi-directional
593 does not prescribe one direction over the other, but instead allows for various permutations
hierarchy of values does not prescribe one direction over the other, but instead allows for various to be
594 undertaken
permutationsdepending on the context
to be undertaken of design/use
depending on theand the current
context design paradigms
of design/use and the that are specific
current design
595 to the context. Because of this, various design requirements can ultimately satisfy the final value(s).
paradigms that are specific to the context. Because of this, various design requirements can ultimately
596 satisfy the final value(s).
597
 Appl. Sci. 2020, 10, 4182 17 of 25
Appl. Sci. 2020, 10, x FOR PEER REVIEW 17 of 25

598
Figure 10. Bi-direction hierarchy showcasing how the value of welfare can be translated through norms
599 Figure 10. Bi-direction hierarchy showcasing how the value of welfare can be translated through
and into technical design requirements for wearable technology.
600 norms and into technical design requirements for wearable technology.
While values such as welfare have already some background literature (e.g., ergonomics of
601 While
wearable valuesthat
devices) suchmay as help
welfare have already
the designers, somesome
other background
values are moreliterature (e.g.for
intricate, ergonomics
example the of
602 wearable devices) that may help the designers, some other values are more intricate,
value of trustworthiness. Some of the issues that may arise here can include the various ways that for example the
603 value of trustworthiness. Some of the issues that may arise here can include the various ways that
these existent wearable technologies adopted for the Operator 4.0 uses change efficacy depending on
604 these
the existent
user. wearabletrustworthiness
For example, technologies adoptedcan befor the Operator
severely limited4.0 uses change
between efficacy depending
the technology and the user on
605 the user. For example, trustworthiness can be severely limited between the technology and the user
depending on the physical build of the user. Arm band technologies must ensure that they are adaptable
606 depending
to on the
different skin physical
types, build of the
arm diameters anduser. Arm band
hair density. technologies
Likewise, it may must ensure that
be implicitly theythat
required are
607 adaptable to different skin types, arm diameters and hair density. Likewise, it may be implicitly
systems recognize the type of user that is currently employing the technology. How a technological
608 required that
intelligent systems
system recognize
provides the an
the user type of user that
explanation ofisitscurrently
goals and employing the technology.
actions depends on the userHowthata
609 technological intelligent system provides the user an explanation of its goals and actions depends on
prompts it, and such can change at any time. These systems must be designed in such a way as to
610 the user that prompts it, and such can change at any time. These systems must be designed in such a
handle dynamic use. How existent wearable technologies employed for the Operator 4.0 use can be
611 way as to handle dynamic use. How existent wearable technologies employed for the Operator 4.0
designed retrospectively to satisfy these issues remains to be seen.
612 use can be designed retrospectively to satisfy these issues remains to be seen.
5.2. Use Case #2: Augmenting the Workers’ Cognitive Capabilities -> Using Simulation-Based VR Training
613 5.2. Use Case
Solutions #2: Augmenting
to Enhance Learning the Workers’ Cognitive Capabilities -> Using Simulation-based VR Training
614 Solutions to Enhance Learning
Complexity has increased significantly in production environments, thus making traditional
615 Complexity
training methodshas increased
outdated. significantly
This has made in production
necessary environments,
the integration thus making
of Modeling traditional
& Simulation
616 training methods
methods outdated.
with Virtual Reality toThis has made
deliver necessary
a meaningful the integration
environment of Modeling
for personnel & Simulation
training. Prototypes
617 methods
of with VirtualVR
simulation-based Reality to deliver
training tools fora meaningful
augmentingenvironment
workers’ cognitivefor personnel training.
capabilities and Prototypes
enhancing
618 of simulation-based
the learning speed areVR training in
presented tools for augmenting
Figure workers’
11. In such contexts, atcognitive
the beginningcapabilities and enhancing
of the training session,
619 learningare
the players speed are presented
usually given certainin Figure
directives11. and
In such
rulescontexts,
to respectat in
theorder
beginning of theMultiple
to succeed. training
620 session, the
scenarios players
(even are usually
very unlikely ones,given
such ascertain directives
industrial accidents)andmayrulesbeto respect intoorder
experienced to worker’s
test the succeed.
621 Multiple scenarios
capability (even
to minimize very unlikely
impulsive ones,
behavior such
and as industrial
assess the player’s accidents) may be experienced
improvements over time in to test
terms
622 thecompliance
of worker’s capability to minimize
to rules and impulsive
expectations. Figurebehavior
12 laysand out assess the player’s
one potential wayimprovements
to translate designover
623 time in termsthrough
requirements, of compliance to rules and
norms embodying the expectations.
conformity value. Figure 12 lays various
Of course, out oneother
potential way to
instantiations
624 translate
in terms ofdesign
designrequirements,
requirementsthrough
can better norms embodying
suit such the conformity
translations. value. Of
This is, of course, notcourse, various
an exhaustive
625 other instantiations in terms of design requirements can better suit such translations. This is, of
626 course, not an exhaustive illustration, as values are interdependent. While on one side, players are
 Appl. Sci.
Appl. Sci. 2020,
2020, 10,
10, 4182
x FOR PEER REVIEW 18 of
18 of 25
25

627 Appl. Sci. 2020,

required 10, x FOR PEER
to conform REVIEW
to the procedure and expectations, on the other side easygoing and uncritical 18 of 25

628 illustration,
adhesion toasnorms
valueswould
are interdependent.
minimize the While on one
worker’s side, players
capability are required
to make informedto decisions,
conform tothus the
627
629 required
procedure to conform
and to the
expectations, procedure
on the and
other expectations,
side easygoing on the
and other side
uncritical easygoing
adhesion to
potentially leaving the worker in unknown situations that are risky for health and safety. It emerges and
norms uncritical
would
628
630 adhesion
minimize to
therefore thethenorms
need would
worker’s minimize
ofcapability the
to make
a discussion worker’s
informed
over capabilitythus
decisions,
the correlation oftothe
make informed
potentially leaving
conformity decisions,
with thus
the worker
value in
the
629
631 potentially
unknown leaving
situations the
thatworker
are in
risky unknown
for health situations
and that
safety. It are risky
emerges for health
therefore and
the
trustworthiness of VR technology. Technologies that are used or will be used by the workers are safety.
need of a It emerges
discussion
630
632 therefore
over
shapedtheby the need
correlation
human ofof
theabut
values discussion
conformity overwith
value
also transform thethe
correlation
workers’ of theof
trustworthiness
(and people’s) conformity
VR technology.
identities. value with the
Technologies
631
633 trustworthiness of VR technology. Technologies that are used or will be used by
that are used or will be used by the workers are shaped by human values but also transform workers’ the workers are
632 shaped by human values
(and people’s) identities. but also transform workers’ (and people’s) identities.
633

634
635
634 Figure 11. Prototypes of simulation-based VR training tools for augmenting workers’ cognitive
636 capabilities.
635 Figure 11.Prototypes
Figure 11. of simulation-based
Prototypes VR training
of simulation-based tools fortools
VR training augmenting workers’ cognitive
for augmenting workers’capabilities.
cognitive
636 capabilities.

637
638
637 12. Bi-direction hierarchy showcasing how the value of conformity can be
Figure 12. be extrapolated
extrapolated from
639 technical design
design requirements
requirements through
through norms
norms for
for simulation-based
simulation-basedVR
VRtraining
trainingtools.
tools.
638 Figure 12. Bi-direction hierarchy showcasing how the value of conformity can be extrapolated from
639 technical design requirements through norms for simulation-based VR training tools.
 Appl. Sci. 2020, 10, 4182 19 of 25
Appl. Sci. 2020, 10, x FOR PEER REVIEW 19 of 25

640 5.3. Use

5.3. Use Case
Case #3:
#3: Augmenting
Augmenting thetheWorkers’
Workers’Interaction
InteractionCapabilities
Capabilities->->Using
UsingIntelligent
IntelligentVoice-Enabled
Voice-EnabledDigital
641 Digital Assistants
Assistants at the Factory
at the Factory
642 Voice-enabled digital assistants
Voice-enabled assistantsarearedrastically
drasticallychanging
changing the way
the wewe
way interact with
interact the CPPS,
with and
the CPPS,
643 prototypes
and are are
prototypes today available,
today available,forfor
example,
example,totosupport
support technicians
technicians and
and specialists during
during
644 maintenance tasks at
maintenance atthe
thefactory
factoryasasshowed
showed inin
Figure
Figure13.13.
Prospectively,
Prospectively,every single
every autonomous
single autonomousand
645 cooperative
and building
cooperative blockblock
building of a CPPS will bewill
of a CPPS ablebeto able
interact with thewith
to interact human
theworker
humanvia digitally-
worker via
646 generated voice responses,
digitally-generated thus giving
voice responses, thus birth
givingtobirth
the to
Social SmartSmart
the Social Factory. Digital
Factory. assistants
Digital give
assistants
647 workers
give intuitive
workers and quick
intuitive access
and quick to a plethora
access of information
to a plethora and knowledge,
of information processed
and knowledge, and made
processed and
648 available
made by a ubiquitous
available intelligence,
by a ubiquitous but concerns
intelligence, exist exist
but concerns on howon to
howdesign ‘ethical
to design digital
‘ethical agents’
digital that
agents’
649 treattreat
that people withwith
people consideration and and
consideration courtesy.
courtesy.
650

651
652 Figure
Figure 13. Prototypes of voice-enabled digital assistants
13. Prototypes assistants that
that augment
augment the
the worker’s
worker’s interaction
interaction
653 capabilities towards the CPPS.
capabilities towards the CPPS.

654 Figure 14 tackles

Figure 14 tacklesthe
thetranslation
translationof of
thethe value
value of identity
of identity for thefordesign
the design of voice-enabled
of voice-enabled digital
655 digital assistants, showing how several ethical issues (discrimination of physical-impaired
assistants, showing how several ethical issues (discrimination of physical-impaired workers, workers,
or by
656 or by language, culture, or current mood) could potentially emerge if value-oriented design
language, culture, or current mood) could potentially emerge if value-oriented design requirements requirements
657 are
are not
not embedded
embedded intointo ‘assistance
‘assistance technology’.
technology’. One One of
of the
the issues
issues that
that can
can arise
arise is
is work-related
work-related and and
658 emotional over-reliance, thus creating a potentially harmful reliance-dependence
emotional over-reliance, thus creating a potentially harmful reliance-dependence paradigm where paradigm where
659 operators
operators are
are continually
continually encouraged
encouraged to to use
use these
these devices
devices for
for most
mostof oftheir
theirduties.
duties.TheTherisk
riskisisthat
thata
660 akind
kindof of emotional dependence develops, accompanied by overconfidence in the
emotional dependence develops, accompanied by overconfidence in the ability of the ability of the machine
661 to
machine toone’s
support tasks,one’s
support even tasks,
in unexpected
even in situations.
unexpectedPoor design requirements
situations. that encompasses
Poor design requirements that
662 all
encompasses all the human values simultaneously can lead to an exacerbation of potentiallytechnical
the human values simultaneously can lead to an exacerbation of potentially harmful harmful
663 outcomes
technical that then put
outcomes thathuman welfare
then put humanin jeopardy
welfare given the system
in jeopardy givenreliance on suchreliance
the system systems. on such
664 systems.
665
 Appl. Sci. 2020, 10, 4182 20 of 25
Appl. Sci. 2020, 10, x FOR PEER REVIEW 20 of 25

666
667 Figure 14.
Figure Bi-direction hierarchy
14. Bi-direction hierarchy showcasing
showcasing how
how the
the value
value of
of identity
identity can
can be
be extrapolated
extrapolated from
from
technical design requirements through norms for voice-enabled digital assistants.
668 technical design requirements through norms for voice-enabled digital assistants.
6. Conclusions
669 6. Conclusions
6.1. Contribution of the Study
670 6.1. Contribution of the Study
This paper envisages a new revolutionary wave in the industrial sector, Industry 5.0 or Age of
671 This paper where
Augmentation, envisages a newand
humans revolutionary wave in the
machines cooperate industrial
in perfect sector, Industry
symbiosis with one5.0 or Age
another of
and
672 Augmentation, where humans and machines cooperate in perfect symbiosis
human capabilities are augmented by a human-friendly technology. Despite arguments that industry is with one another and
673 human capabilities
being ‘versioned’ aretoo
way augmented
arbitrarily,bythe
a human-friendly
human-centered technology. Despite
transformation arguments
of the smart and that industry
automated
674 is being is
factories ‘versioned’
a completely waynewtooparadigm
arbitrarily, thatthe
hashuman-centered transformation
been recently anticipated of thelike
by concepts smart and
‘socially
675 automated factories is a completely new paradigm that has been recently anticipated
sustainable workplaces’, ‘human-centric factory’, and ‘Operator 4.0’ and is markedly different from by concepts like
676 ‘socially sustainable workplaces’, ‘human-centric factory’, and ‘Operator
the ‘techno-centric’ Industry 4.0. In this regard, the paper reveals the growing concern in several 4.0’ and is markedly
677 different
industrialfrom theregarding
sectors ‘techno-centric’ Industry
the ethical issues4.0.
thatInarise
this with
regard, the paper that
technologies reveals the growing
workers currently concern
use or
678 in several industrial sectors regarding the ethical issues that arise with technologies
are considering using towards the Factory of the Future. Likewise, they all agreed that all or some of that workers
679 currently
the issues use
thatormayarearise
considering
with these using towards the
technologies canFactory of the Future.
be responded Likewise, they
to and ameliorated by all agreed
designing
680 that all or some of the issues that
these technologies explicitly for human values.may arise with these technologies can be responded to and
681 ameliorated by designing these technologies explicitly for human values.
This paper argued that a value-oriented and ethical technology engineering in Industry 5.0 is
682 This paper
an urgent argued that
and sensitive a value-oriented
topic. and ethical
As is true for values technology
in general, which engineering in Industryin5.0
tend to be construed is an
highly
683 urgent and sensitive topic. As is true for values in general, which tend
sociocultural nuanced ways, designers require tools to help translate these abstract values, throughto be construed in highly
684 sociocultural
sociocultural nuanced ways, designers
norms (including require
legal/policy tools to
norms) intohelp translate
concrete theserequirements.
design abstract values, Thethrough
Value
685 sociocultural
Sensitive Design norms
(VSD)(including
approachlegal/policy norms) into
has been proposed concrete design
as a principled frameworkrequirements.
that design The Value
teams of
686 Sensitive Design (VSD) approach has been proposed as a principled framework
the Factories of the Future can adopt to contribute towards systemic robustness. It illustrates how that design teams of
687 the Factories enabling
technologies of the Future can adopt to symbiosis
human–machine contribute in towards systemic
the Factory of therobustness.
Future can It be
illustrates
designed howto
688 technologies
embody elicited human values and shows actionable steps that engineers and designers can take to
enabling human–machine symbiosis in the Factory of the Future can be designed in
689 embody elicited
their design human
projects. Forvalues and shows
this reason, actionable
this paper steps that engineers
is comparatively unique in and designersRather
its approach. can take in
than
690 their design
providing a projects. For this reason,
solely speculative this paper
application of theis comparatively
VSD methodology unique in its
to the approach.technologies
individual Rather than
691 providing a solely speculative application of the VSD methodology to the
that converge to form the Factory of the Future, we evaluate the applicability of VSD as a pragmatic individual technologies
692 that converge to form the Factory of the Future, we evaluate the applicability of VSD as a pragmatic
693 approach to informing the work of engineers at a specific nexus of practice. In this case, this paper
694 looks at how the adoption of VSD can be used to guide the development of Industry 5.0 solutions.
Appl. Sci. 2020, 10, 4182 21 of 25

approach to informing the work of engineers at a specific nexus of practice. In this case, this paper
looks at how the adoption of VSD can be used to guide the development of Industry 5.0 solutions.

6.2. Limitations of the Study

Still, there are areas of limitations that this paper does not explore but warrant future research.
The insights derived from the survey are applicable to a large number of enterprises and sectors but
are not exhaustive. Cross-cultural or cross-national comparisons to identify similarities and differences
in the specificity of value systems may be needed. Researchers have shown how a value system is not
entirely stable but rather dynamically influenced by socio-cultural factors, the changing work world,
and transformations in production activities. Therefore, the results of this survey should be intended
with reference to enterprises with similar characteristics.
Regarding the VSD approach to the human–machine symbiosis in Industry 5.0, the manner
in which such values express themselves in a particular culture at a particular point in time
can be quite different [35,69]. Investigating relevant stakeholders, determining their values and
translating those values into norms and design requirements for any given technological artifact is
demanding. Values, as already mentioned, are typically abstract, and conceptualizing how such abstract
concepts can be turned into concrete design requirements is understandably laborious. Even the most
experienced engineers and design specialists can be stopped in their tracks when trying to determine the
inter-relations of values that are in tension and determining how to address such tensions. What must
be resisted is the urge towards perfection, something that VSD actively resists. This tendency can
hinder learning and meaningful progress [12]. We stipulate this given that the above discussion,
until this point, seems idealistic in its attempt to balance so many potential stakeholders and values.
Although robustness can be reasonably increased in any project if it is allotted more time and resources,
most innovation hubs are constrained by such variables. VSD at its foundation pushes engineers to
consider how to achieve the most good given the present constraints and to resist the urge towards the
burdensome ideal of perfect innovation.

6.3. Further Research

This discussion paves the way for an Industry 5.0 designed for human values. Most of the
research efforts focus asymmetrically on the technical and technological aspects of the Factories of the
Future while it neglects that a human–machine symbiosis requires profound ethical considerations.
The results of our survey showed that 28% of respondent were part of organizations that did not have
any formalized code of ethics. This presents both challenges and opportunities for how technologies
enabling human–machine symbiosis are designed and used currently within organizations. However,
the opportunity to reflect on how to begin codifying the values of the stakeholders both in the
organization as well as those who are impacted by that organizations’ actions becomes apparent.
Future research should look at the salience of existent standards and codes of practice towards O4.0
technologies as well as practices for developing new standards and guidelines alongside the ethical
development and use of those technologies. Co-construction provides the most salient mode of
policy innovation and VSD is similarly apt to providing a means by which both technologies and
their respective governance and policy structures can be co-constructed [68]. Future work will also
be designed to assess whether there are significant differences between organizational members
and leaders, comparing for instance, the values of pairs of supervisors and subordinates as well as
significant differences among enterprises depending on their size. Further elicitation methods will be
used to ask for additional value considerations, organizational motivations, methods of training and
dissemination, reward structures, and economic incentives with reference to ethics at the workplace.
Ethics training for executives, managers and employees may be required in the next future as well as
the need for a 24 h hot line for ethical situational advice that may aid workers or employers in the
case of ambiguous situations. Future works should clarify when automation starts decreasing and
affecting the workers’ quality of life (‘over-automation’) and understand who decides that developing
Appl. Sci. 2020, 10, 4182 22 of 25

a certain kind of new technology is ‘safe’ for the workforce. From our review, we noticed that the main
proposals fostering the adoption of techno ethics considerations are often related to the idea of charters
to be signed (‘Hippocratic oath’). However, the legal system is having trouble in catching up with the
increasing speed of technological advances. In this sense, the iterative VSD approach may dramatically
help in the identification of ethical rules and frameworks that technologies for a human–machine
symbiosis in the Factory of the Future must be designed for.

Supplementary Materials: The following are available online at http://www.mdpi.com/2076-3417/10/12/4182/s1,

Video S1: Voice-enabled Digital Assistant.
Author Contributions: The authors have contributed equally to the paper. All authors have read and agreed to
the published version of the manuscript.
Funding: The research work described in this paper is part of the activities carried out in the context of the
SO4SIMS project (Smart Operators 4.0 based on Simulation for Industry and Manufacturing Systems) funded by
the Italian Ministry of Education, Universities and Research MIUR (Project PRIN – 2017FW8BB4).
Acknowledgments: The use cases and prototypes showed in the survey build upon solutions and projects
developed by CAL-TEK S.r.l., a Spin-off company of the University of Calabria, Italy and carried out at the
Modeling & Simulation Center – Laboratory of Enterprise Solutions (MSC-LES) at the Department of Mechanical,
Energy and Management Engineering of the University of Calabria, such as the W-Artemys project, financed by
INAIL (Italian National Institute for Insurance against Accidents at Work). The authors also acknowledge all the
people who participated to the survey.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Monostori, L.; Kádár, B.; Bauernhansl, T.; Kondoh, S.; Kumara, S.; Reinhart, G.; Sauer, O.; Schuh, G.; Sihn, W.;
Ueda, K. Cyber-physical systems in manufacturing. CIRP Ann. 2016, 65, 621–641. [CrossRef]
2. Krugh, M.; Mears, L. A complementary Cyber-Human Systems framework for Industry 4.0 Cyber-Physical
Systems. Manuf. Lett. 2018, 15, 89–92. [CrossRef]
3. Hancock, P.A.; Jagacinski, R.J.; Parasuraman, R.; Wickens, C.D.; Wilson, G.F.; Kaber, D.B. Human-automation
interaction research: Past, present, and future. Ergon. Des. 2013, 21, 9–14. [CrossRef]
4. Cohen, Y.; Golan, M.; Singer, G.; Faccio, M. Workstation-Operator Interaction in 4.0 Era: WOI 4.0.
IFAC-PapersOnLine 2018, 51, 399–404. [CrossRef]
5. Özdemir, V.; Hekim, N. Birth of Industry 5.0: Making Sense of Big Data with Artificial Intelligence,
“The Internet of Things” and Next-Generation Technology Policy. Omics A J. Integr. Biol. 2018, 22, 65–76.
[CrossRef]
6. European Economic and Social Committee. Industry 5.0. Available online: https://www.eesc.europa.eu/en/
agenda/our-events/events/industry-50 (accessed on 7 April 2020).
7. Philbeck, T.; Davis, N.; Engtoft Larsen, A.M. Values, Ethics and Innovation—Rethinking Technological Development
in the Fourth Industrial Revolution; World Economic Forum: Cologny, Switzerland, 2018.
8. de Saille, S. Innovating innovation policy: The emergence of ‘Responsible Research and Innovation’.
J. Responsible Innov. 2015, 2, 152–168. [CrossRef]
9. United Nations. The Age of Digital Interdependence—Digital cooperation: Report of the UN Secretary-
General’s High-level Panel on Digital Cooperation. Available online: https://digitalcooperation.org/wp-
content/uploads/2019/06/DigitalCooperation-report-for-web.pdf (accessed on 25 January 2020).
10. van de Poel, I. Design for Values in Engineering. In Handbook of Ethics, Values, and Technological Design: Sources,
Theory, Values and Application Domains; van den Hoven, J., Vermaas, P.E., van de Poel, I., Eds.; Springer:
Dordrecht, The Netherlands, 2014; pp. 1–20. ISBN 978-94-007-6994-6.
11. Kiran, A.H.; Oudshoorn, N.; Verbeek, P.P. Beyond checklists: Toward an ethical-constructive technology
assessment. J. Responsible Innov. 2015, 2, 5–19. [CrossRef]
12. Friedman, B.; Hendry, D.G. Value Sensitive Design: Shaping Technology with Moral Imagination; Mit Press:
Cambridge, MA, USA, 2019; ISBN 9780262039536.
13. Kaasinen, E.; Schmalfuß, F.; Özturk, C.; Aromaa, S.; Boubekeur, M.; Heilala, J.; Heikkilä, P.; Kuula, T.;
Liinasuo, M.; Mach, S.; et al. Empowering and engaging industrial workers with Operator 4.0 solutions.
Comput. Ind. Eng. 2020, 139, 105678. [CrossRef]
Appl. Sci. 2020, 10, 4182 23 of 25

14. Calabrese, A.; Levialdi Ghiron, N.; Tiburzi, L. ‘Evolutions’ and ‘revolutions’ in manufacturers’ implementation
of industry 4.0: A literature review, a multiple case study, and a conceptual framework. Prod. Plan. Control
2020, 1–15. [CrossRef]
15. Müller, J.M.; Buliga, O.; Voigt, K.I. Fortune favors the prepared: How SMEs approach business model
innovations in Industry 4.0. Technol. Forecast. Soc. Chang. 2018, 132, 2–17. [CrossRef]
16. Romero, D.; Bernus, P.; Noran, O.; Stahre, J.; Fast-Berglund, Å. The operator 4.0: Human cyber-physical
systems & adaptive automation towards human-automation symbiosis work systems. In Proceedings
of the APMS 2016: Advances in Production Management Systems. Initiatives for a Sustainable World,
Iguassu Falls, Brazil, 3–7 September 2016; Nääs, I., Vendrametto, O., Reis, J.M., Gonçalves, R.F., Silva, M.T.,
von Cieminski, G., Kiritsis, D., Eds.; Springer: Berlin/Heidelberg, Germany, 2017; Volume 488, pp. 677–686.
17. Guerin, C.; Rauffet, P.; Chauvin, C.; Martin, E. Toward production operator 4.0: Modelling Human-Machine
Cooperation in Industry 4.0 with Cognitive Work Analysis. IFAC-PapersOnLine 2019, 52, 73–78. [CrossRef]
18. Salaken, S.M.; Nahavandi, S.; McGinn, C.; Hossny, M.; Kelly, K.; Abobakr, A.; Nahavandi, D.; Iskander, J.
Development of a cloud-based computational framework for an empathetic robot. In Proceedings of the
11th International Conference on Computer and Automation Engineering, Perth, Australia, 23–25 Feburary
2019; pp. 102–108.
19. Zolotová, I.; Papcun, P.; Kajáti, E.; Miškuf, M.; Mocnej, J. Smart and cognitive solutions for Operator 4.0:
Laboratory H-CPPS case studies. Comput. Ind. Eng. 2018, 139, 105471. [CrossRef]
20. Romero, D.; Stahre, J.; Wuest, T.; Noran, O.; Bernus, P.; Fast-Berglund, Å.; Gorecky, D. Towards an Operator 4.0
Typology: A Human-Centric Perspective on the Fourth Industrial Revolution Technologies. In Proceedings
of the CIE 2016: 46th International Conferences on Computers and Industrial Engineering, Tianjin, China,
29–31 October 2016.
21. Romero, D.; Mattsson, S.; Fast-Berglund, Å.; Wuest, T.; Gorecky, D.; Stahre, J. Digitalizing occupational
health, safety and productivity for the operator 4.0. In Proceedings of the IFIP Advances in Information and
Communication Technology, Seoul, Korea, 23–26 August 2018; Springer: New York, NY, USA, 2018; Volume
536, pp. 473–481.
22. Papcun, P.; Kajati, E.; Koziorek, J. Human machine interface in concept of industry 4.0. In Proceedings of
the DISA 2018—IEEE World Symposium on Digital Intelligence for Systems and Machines (DISA), Kosice,
Slovakia, 23–25 August 2018; Institute of Electrical and Electronics Engineers Inc.: Piscataway, NJ, USA,
2018; pp. 289–296.
23. Taylor, M.P.; Boxall, P.; Chen, J.J.J.; Xu, X.; Liew, A.; Adeniji, A. Operator 4.0 or Maker 1.0? Exploring the
implications of Industrie 4.0 for innovation, safety and quality of work in small economies and enterprises.
Comput. Ind. Eng. 2020, 139. [CrossRef]
24. Koch, P.J.; van Amstel, M.K.; D˛ebska, P.; Thormann, M.A.; Tetzlaff, A.J.; Bøgh, S.; Chrysostomou, D.
A Skill-based Robot Co-worker for Industrial Maintenance Tasks. Procedia Manuf. 2017, 11, 83–90. [CrossRef]
25. Rahman, M.H.; Saad, M.; Kenné, J.P.; Archambault, P.S. Control of an exoskeleton robot arm with sliding
mode exponential reaching law. Int. J. Control. Autom. Syst. 2013, 11, 92–104. [CrossRef]
26. Trentesaux, D.; Rault, R. Designing Ethical Cyber-Physical Industrial Systems. IFAC-PapersOnLine 2017, 50,
14934–14939. [CrossRef]
27. Bednar, P.M.; Welch, C. Socio-Technical Perspectives on Smart Working: Creating Meaningful and Sustainable
Systems. Inf. Syst. Front. 2019, 1–18. [CrossRef]
28. Pacaux-Lemoine, M.P.; Trentesaux, D. Ethical Risks of Human-Machine Symbiosis in Industry 4.0: Insights
from the Human-Machine Cooperation Approach. IFAC-PapersOnLine 2019, 52, 19–24. [CrossRef]
29. Demir, K.A.; Döven, G.; Sezen, B. Industry 5.0 and Human-Robot Co-working. Procedia Comput. Sci. 2019,
158, 688–695. [CrossRef]
30. Welfare, K.S.; Hallowell, M.R.; Shah, J.A.; Riek, L.D. Consider the Human Work Experience When Integrating
Robotics in the Workplace. In Proceedings of the ACM/IEEE International Conference on Human-Robot
Interaction, Daegu, Korea, 11–14 March 2019; pp. 75–84.
31. Gräßler, I.; Pöhler, A.; Pottebaum, J. Creation of a Learning Factory for Cyber Physical Production Systems.
Procedia CIRP 2016, 54, 107–112. [CrossRef]
32. Dworschak, B.; Zaiser, H. Competences for cyber-physical systems in manufacturing-First findings and
scenarios. Procedia CIRP 2014, 25, 345–350. [CrossRef]
Appl. Sci. 2020, 10, 4182 24 of 25

33. van de Poel, I. Conflicting Values in Design. In Handbook of Ethics, Values, and Technological Design: Sources,
Theory, Values and Application Domains; van den Hoven, J., Vermaas, P.E., van de Poel, I., Eds.; Springer:
Dordrecht, The Netherlands, 2014; pp. 1–23. ISBN 978-94-007-6994-6.
34. Horváth, D.; Szabó, R.Z. Driving forces and barriers of Industry 4.0: Do multinational and small and
medium-sized companies have equal opportunities? Technol. Forecast. Soc. Chang. 2019, 146, 119–132.
[CrossRef]
35. Friedman, B.; Hendry, D.G.; Borning, A. A Survey of Value Sensitive Design Methods. Found. Trends®Hum.
–Comput. Interact. 2017, 11, 63–125. [CrossRef]
36. Avallone, F.; Farnese, M.L.; Pepe, S.; Vecchione, M. The Work Values Questionnaire (WVQ): Revisiting
Schwartz’s Portrait Values Questionnaire (PVQ) for work contexts. BPA-Appl. Psychol. Bull. 2010, 261, 59–76.
37. Friedman, B.; Kahn, P.H.; Borning, A. Value Sensitive Design and Information Systems. In The Handbook
of Information and Computer Ethics; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2008; pp. 69–101.
ISBN 9780471799597.
38. Lindemann, N. What’s the Average Survey Response Rate? [2019 Benchmark]. Available online:
https://surveyanyplace.com/average-survey-response-rate/#:~{}:text=A%20low%20response%20rate%
20can,the%20participants%20regarding%20the%20outcome.&text=However%2C%20recent%20studies%
20have%20shown,near%2060%20or%2070%25 (accessed on 25 January 2020).
39. Ruzic, L.; Sanfod, J.A. Universal Design Mobile Interface Guidelines (UDMIG) for an Aging Population.
In Mobile e-Health; Springer: Berlin/Heidelberg, Germany, 2017; pp. 17–37.
40. Newell, A.F.; Gregor, P.; Morgan, M.; Pullin, G.; Macaulay, C. User-Sensitive Inclusive Design. Univers. Access
Inf. Soc. 2011, 10, 235–243. [CrossRef]
41. Bhamra, T.; Lofthouse, V. Design for Sustainability: A Practical Approach; Routledge: Abingdon, UK, 2016;
ISBN 1317152352.
42. Bødker, K.; Kensing, F.; Simonsen, J. Participatory IT Design: Designing for Business and Workplace Realities;
MIT Press: Cambridge, MA, USA, 2009; ISBN 0262261790.
43. Davis, J.; Nathan, L.P. Handbook of Ethics, Values, and Technological Design: Sources, Theory, Values and Application
Domains; van den Hoven, J., Vermaas, P.E., van de Poel, I., Eds.; Springer: Berlin/Heidelberg, Germany, 2015;
pp. 12–40. ISBN 9789400769700.
44. Umbrello, S. Beneficial Artificial Intelligence Coordination by Means of a Value Sensitive Design Approach.
Big Data Cogn. Comput. 2019, 3, 5. [CrossRef]
45. Umbrello, S. Atomically Precise Manufacturing and Responsible Innovation: A Value Sensitive Design
Approach to Explorative Nanophilosophy. Int. J. Technoethics 2019, 10, 1–21. [CrossRef]
46. Winkler, T.; Spiekermann, S. Twenty years of value sensitive design: A review of methodological practices in
VSD projects. Ethics Inf. Technol. 2018, 1–5. [CrossRef]
47. van den Hoven, J.; Manders-Huits, N. Value-Sensitive Design. In A Companion to the Philosophy of Technology;
Wiley-Blackwell: Hoboken, NJ, USA, 2009; pp. 477–480. ISBN 9781444310795.
48. Umbrello, S. The moral psychology of value sensitive design: The methodological issues of moral intuitions
for responsible innovation. J. Responsible Innov. 2018, 5, 186–200. [CrossRef]
49. Borning, A.; Muller, M. Next steps for value sensitive design. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, Austin, TX, USA, 5–10 May 2012; p. 1125. [CrossRef]
50. Oosterlaken, I. Applying Value Sensitive Design (VSD) to Wind Turbines and Wind Parks: An Exploration.
Sci. Eng. Ethics 2015, 21, 359–379. [CrossRef]
51. Mouter, N.; de Geest, A.; Doorn, N. A values-based approach to energy controversies: Value-sensitive design
applied to the Groningen gas controversy in the Netherlands. Energy Policy 2018, 122, 639–648. [CrossRef]
52. Woelfer, J.P.; Iverson, A.; Hendry, D.G.; Friedman, B.; Gill, B.T. Improving the Safety of Homeless Young
People with Mobile Phones: Values, Form and Function. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, Vancouver, BC, Canada, 7 May 2011; ACM: New York, NY, USA,
2011; pp. 1707–1716.
53. van den Hoven, J. Architecture and Value-Sensitive Design. In Ethics, Design and Planning of the Built
Environment; Basta, C., Moroni, S., Eds.; Springer Science & Business Media: Berlin, Germany, 2013; p. 224.
ISBN 9400752466.
Appl. Sci. 2020, 10, 4182 25 of 25

54. Friedman, B.; Kahn, P.H., Jr. New Directions: A Value-sensitive Design Approach to Augmented Reality.
In Proceedings of the DARE 2000 on Designing Augmented Reality Environments, Elsinore, Denmark,
1 April 2000; ACM: New York, NY, USA, 2000; pp. 163–164.
55. Timmermans, J.; Zhao, Y.; van den Hoven, J. Ethics and Nanopharmacy: Value Sensitive Design of New
Drugs. Nanoethics 2011, 5, 269–283. [CrossRef] [PubMed]
56. Umbrello, S.; De Bellis, A.F. A Value-Sensitive Design Approach to Intelligent Agents. In Artificial
Intelligence Safety and Security; Yampolskiy, R.V., Ed.; CRC Press: Boca Raton, FL, USA, 2018; pp. 395–410.
ISBN 9780815369820.
57. Calvert, S.C.; Mecacci, G.; Heikoop, D.D.; de Sio, F.S. Full platoon control in Truck Platooning: A Meaningful
Human Control perspective. In Proceedings of the 2018 21st International Conference on Intelligent
Transportation Systems (ITSC), Maui, HI, USA, 4–7 November 2018; pp. 3320–3326.
58. Super, D.E. Work Values Inventory; Houghton Mifflin: Boston, MA, USA, 1968.
59. Super, D.E.; Sverko, B. Life roles, values, and careers: International findings of the Work Importance Study.
In Life Roles, Values, and Careers: International Findings of the Work Importance Study; Jossey-Bass: San Francisco,
CA, USA, 1995.
60. Schwartz, S.H. A Proposal for Measuring Value Orientations across Nations. In Core ESS Questionnaire;
ESS: London, UK, 2003; ISBN 047166782X.
61. Brey, P.A.E. Anticipatory Ethics for Emerging Technologies. Nanoethics 2012, 6, 1–13. [CrossRef]
62. Vallor, S. Technology and the Virtues: A Philosophical Guide for a Future Worth Wanting; Oxford University Press:
New York, NY, USA, 2016; ISBN 9780190498511.
63. La Fors, K.; Custers, B.; Keymolen, E. Reassessing values for emerging big data technologies: Integrating
design-based and application-based approaches. Ethics Inf. Technol. 2019, 21, 209–226. [CrossRef]
64. Yoo, D. Stakeholder Tokens: A constructive method for value sensitive design stakeholder analysis.
In Proceedings of the 2017 ACM Conference Companion Publication on Designing Interactive Systems,
Edinburgh, UK, 6–10 June 2017; ACM: New York, NY, USA, 2017; pp. 280–284.
65. Freeman, B.; Borning, A. YP and urban simulation: Applying an agile programming methodology in
a politically tempestuous domain. In Proceedings of the Agile Development Conference, Salt Lake City,
UT, USA, 25–28 June 2003; pp. 2–11.
66. Nathan, L.P.; Klasnja, P.V.; Friedman, B. Value Scenarios: A Technique for Envisioning Systemic Effects of
New Technologies. In Proceedings of the CHI 07 Extended Abstracts on Human Factors in Computing
Systems, San Jose, CA, USA, 28 April 2007; ACM: New York, NY, USA, 2007; pp. 2585–2590.
67. Friedman, B.; Kahn, P.H., Jr.; Borning, A.; Huldtgren, A. Value Sensitive Design and Information Systems.
In Early Engagement and New Technologies: Opening up the Laboratory; Doorn, N., Schuurbiers, D., van de Poel, I.,
Gorman, M.E., Eds.; Springer: Dordrecht, The Netherlands, 2013; pp. 55–95. ISBN 978-94-007-7844-3.
68. van de Poel, I. Translating Values into Design Requirements BT. In Philosophy and Engineering: Reflections on
Practice, Principles and Process; Michelfelder, D.P., McCarthy, N., Goldberg, D.E., Eds.; Springer: Dordrecht,
The Netherlands, 2013; pp. 253–266. ISBN 978-94-007-7762-0.
69. Umbrello, S. Imaginative Value Sensitive Design: Using Moral Imagination Theory to Inform Responsible
Technology Design. Sci. Eng. Ethics 2019, 25, 1–21. [CrossRef]

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).
 © 2020. This work is licensed under
http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding
the ProQuest Terms and Conditions, you may use this content in accordance
with the terms of the License.