Professional Documents
Culture Documents
Textbook Computer Safety Reliability and Security Safecomp 2018 Workshops Assure Decsos Sassur Strive and Waise Vasteras Sweden September 18 2018 Proceedings Barbara Gallina Ebook All Chapter PDF
Textbook Computer Safety Reliability and Security Safecomp 2018 Workshops Assure Decsos Sassur Strive and Waise Vasteras Sweden September 18 2018 Proceedings Barbara Gallina Ebook All Chapter PDF
https://textbookfull.com/product/computer-safety-reliability-and-
security-39th-international-conference-safecomp-2020-lisbon-
portugal-september-16-18-2020-proceedings-antonio-casimiro/
Computer Vision ECCV 2018 Workshops Munich Germany
September 8 14 2018 Proceedings Part V Laura Leal-Taixé
https://textbookfull.com/product/computer-vision-
eccv-2018-workshops-munich-germany-
september-8-14-2018-proceedings-part-v-laura-leal-taixe/
https://textbookfull.com/product/computer-vision-
eccv-2018-workshops-munich-germany-
september-8-14-2018-proceedings-part-ii-laura-leal-taixe/
https://textbookfull.com/product/computer-security-23rd-european-
symposium-on-research-in-computer-security-
esorics-2018-barcelona-spain-september-3-7-2018-proceedings-part-
i-javier-lopez/
Computer Safety,
Reliability, and Security
SAFECOMP 2018 Workshops
ASSURE, DECSoS, SASSUR, STRIVE, and WAISE
Västerås, Sweden, September 18, 2018, Proceedings
123
Lecture Notes in Computer Science 11094
Commenced Publication in 1973
Founding and Former Series Editors:
Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen
Editorial Board
David Hutchison
Lancaster University, Lancaster, UK
Takeo Kanade
Carnegie Mellon University, Pittsburgh, PA, USA
Josef Kittler
University of Surrey, Guildford, UK
Jon M. Kleinberg
Cornell University, Ithaca, NY, USA
Friedemann Mattern
ETH Zurich, Zurich, Switzerland
John C. Mitchell
Stanford University, Stanford, CA, USA
Moni Naor
Weizmann Institute of Science, Rehovot, Israel
C. Pandu Rangan
Indian Institute of Technology Madras, Chennai, India
Bernhard Steffen
TU Dortmund University, Dortmund, Germany
Demetri Terzopoulos
University of California, Los Angeles, CA, USA
Doug Tygar
University of California, Berkeley, CA, USA
Gerhard Weikum
Max Planck Institute for Informatics, Saarbrücken, Germany
More information about this series at http://www.springer.com/series/7408
Barbara Gallina Amund Skavhaug
•
Computer Safety,
Reliability, and Security
SAFECOMP 2018 Workshops
ASSURE, DECSoS, SASSUR, STRIVE, and WAISE
Västerås, Sweden, September 18, 2018
Proceedings
123
Editors
Barbara Gallina Erwin Schoitsch
Mälardalen University AIT Austrian Institute of Technology
Västerås Vienna
Sweden Austria
Amund Skavhaug Friedemann Bitsch
Norwegian University of Science Thales Deutschland GmbH
and Technology Ditzingen
Trondheim Germany
Norway
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
The SAFECOMP Workshop Day has for many years preceded the SAFECOMP
Conference, attracting additional participants. The SAFECOMP Workshops have
become more attractive since they started generating their own proceedings in the
Springer LNCS series (Springer LNCS vol. 11094, the book in your hands; the main
conference proceedings are LNCS 11093). This meant adhering to Springer’s guide-
lines, i.e., the respective international Program Committee of each workshop had to
make sure that at least three independent reviewers reviewed the papers carefully. The
selection criteria were different from those for the main conference since authors were
encouraged to submit workshop papers, i.e., on work in progress and potentially
controversial topics. In total, 49 regular papers (out of 73) were accepted.
Three of the five workshops are sequels to earlier workshops, which shows conti-
nuity of their relevance to the scientific and industrial community:
• ASSURE 2018 – 6th International Workshop on Assurance Cases for
Software-Intensive Systems, chaired by Ewen Denney, Ibrahim Habli, Ganesh Pai,
and Richard Hawkins
• DECSoS 2018 – 13th ERCIM/EWICS/ARTEMIS Workshop on Dependable Smart
Embedded and Cyber-Physical Systems and Systems-of-Systems, chaired by
Erwin Schoitsch and Amund Skavhaug
• SASSUR 2018 – 7th International Workshop on Next Generation of System
Assurance Approaches for Safety-Critical Systems, chaired by Alejandra Ruiz,
Jose Luis de la Vara, and Tim Kelly
Finally, two new workshops were part of Safecomp for the first time (a third new
proposal was withdrawn in the end):
• STRIVE 2018 – First International Workshop on Safety, securiTy, and pRivacy In
automotiVe systEms, chaired by Gianpiero Costantino and Ilaria Matteucci
• WAISE 2018 – First International Workshop on Artificial Intelligence Safety
Engineering, chaired by Huascar Espinoza, Orlando Avila-García, Rob Alexander,
and Andreas Theodorou;
The workshops provide a truly international platform for academia and industry.
It has been a pleasure to work with the Safecomp chair, Barbara Gallina, and with
the publication chair, Friedemann Bitsch, the workshop chairs, Program Committees,
and the authors. Thank you all for your good cooperation and excellent work!
Committee
EWICS TC7 Chair
Francesca Saglietti University of Erlangen-Nuremberg, Germany
General Chair
Barbara Gallina Mälardalen University, Sweden
Program Co-chairs
Barbara Gallina Mälardalen University, Sweden
Amund Skavhaug The Norwegian University of Science and Technology,
Norway
Workshop Chair
Erwin Schoitsch AIT Austrian Institute of Technology, Austria
Publication Chair
Friedemann Bitsch Thales Deutschland GmbH, Germany
Publicity Chair
Alexander Romanovsky Newcastle University, UK
Workshop Chairs
ASSURE 2018
Ewen Denney SGT/NASA Ames Research Center, USA
Ibrahim Habli University of York, UK
Richard Hawkins University of York, UK
Ganesh Pai SGT/NASA Ames Research Center, USA
VIII Organization
DECSoS 2018
Erwin Schoitsch AIT Austrian Institute of Technology, Austria
Amund Skavhaug NTNU, Norway
SASSUR 2018
Alejandra Ruiz Lopez Tecnalia, Spain
Jose Luis de La Vara Carlos III University of Madrid, Spain
Tim Kelly University of York, UK
STRIVE 2018
Gianpiero Costantino IIT-CNR, Italy
Ilaria Matteucci IIT-CNR, Italy
WAISE 2018
Huascar Espinoza CEA LIST, France
Orlando Avila-García Atos, Spain
Rob Alexander University of York, UK
Andreas Theodorou University of Bath, UK
Organization IX
Supporting Institutions
Co-Engineering-in-the-Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Thomas Gruber, Christoph Schmittner, Martin Matschnig,
and Bernhard Fischer
The Moral Responsibility Gap and the Increasing Autonomy of Systems . . . . 487
Zoë Porter, Ibrahim Habli, Helen Monkhouse, and John Bragg
Contents XV
1 Introduction
This volume contains the papers presented at the 6th International Workshop on
Assurance Cases for Software-intensive Systems (ASSURE 2018), collocated this year
with the 37th International Conference on Computer Safety, Reliability, and Security
(SAFECOMP 2018), in Västerås, Sweden.
As with the previous five editions of ASSURE, this year’s workshop aims to
provide an international forum for presenting emerging research, novel contributions,
tool development efforts, and position papers on the foundations and applications of
assurance case principles and techniques. The workshop goals are to: (i) explore
techniques to create and assess assurance cases for software-intensive systems;
(ii) examine the role of assurance cases in the engineering lifecycle of critical systems;
(iii) identify the dimensions of effective practice in the development/evaluation of
assurance cases; (iv) investigate the relationship between dependability techniques and
assurance cases; and, (v) identify critical research challenges towards defining a
roadmap for future development.
2 Program
ASSURE 2018 kicked off with an invited keynote talk by Robin Bloomfield, Founding
Partner of the engineering consultancy Adelard LLP, and Professor of System and
Software Dependability, Centre for Software Reliability, City University London.
Eight papers were accepted this year covering three assurance case themes: confidence
assessment, patterns and processes, and tools and automation.
Papers under the theme of assurance case confidence assessment frameworks
examined issues such as the relationship between argumentation elements at the same
layer, and the challenges associated with assurance of machine learning-based systems.
The theme of assurance case patterns and processes included papers that dealt with a
new notion of assurance recipe, attack modeling to address safety and security
assurance jointly, and the interoperability of medical systems. Finally, the theme
6th International Workshop on Assurance Cases for Software-Intensive Systems 3
concerning assurance case tools and automation comprised papers providing a survey
of assurance case tools developed over the past two decades, addressing change impact
management in assurance cases, and additional methodological steps in authoring
assurance cases.
3 Acknowledgments
We thank all those who submitted papers to ASSURE 2018 and congratulate the
authors whose papers were selected for inclusion into the workshop program and
proceedings. For reviewing the submissions and providing useful feedback to the
authors, we especially thank our distinguished Program Committee members:
– Simon Burton, Bosch Research, Germany
– Isabelle Conway, ESA/ESTEC, The Netherlands
– Martin Feather, NASA Jet Propulsion Laboratory, USA
– Alwyn Goodleo, NASA Langley Research Center, USA
– Jérémie Guiochet, LAAS-CNRS, France
– Joshua Kaizer, Nuclear Regulatory Commission, USA
– Tim Kelly, University of York, UK
– Yoshiki Kinoshita, Kanagawa University, Japan
– Andrew Rae, Griffith University, Australia
– Philippa Ryan, Adelard LLP, UK
– Mark-Alexander Sujan, University of Warwick, UK
– Kenji Taguchi, CAV Technologies Co. Ltd., Japan
– Sean White, NHS Digital, UK
Their efforts have resulted in an exciting workshop program and, in turn, a suc-
cessful sixth edition of the ASSURE workshop series. Finally, we thank the organizers
of SAFECOMP 2018 for their support of ASSURE 2018.
Research on the Classification of the Relationships
Among the Same Layer Elements in Assurance Case
Structure for Evaluation
1
The Key Laboratory on Reliability and Environmental Engineering Technology,
Beihang University, Beijing, China
{xubiaorms,lmy}@buaa.edu.cn, gutingyang@126.com
2
China Financial Certification Authority, Beijing, China
greatdjz@163.com
1 Introduction
Assurance case has been deeply studied in safety domain for decades, and its applications
in reliability domain and security domain have also made some progress [1]. As argu‐
ments and evidences are practically imperfect, it is difficult to determine that the claim
is true with 100% certainty. The question that “how should the assessor determine
whether the argument and its evidence are sufficient?” leads to the concepts of confi‐
dence (uncertainty) in the argument’s claims [2].
There have been many studies on the evaluation of confidence in assurance case.
These studies can be roughly divided into two types according to their calculation
methods. One kind is based on Bayesian Belief Networks (BBNs) [3–6]. The other kind
is based on Dempster-Shafer theory (D-S theory) [7–10], Jøsang’s opinion triangle [11],
or Evidential Reasoning [12]. However, most of these approaches pay more attention to
the tree structure for calculation, while ignoring the relationships among the supporting
elements of the same level. In [6], if the key evidence of confidence “coverage of hazard
classifications” assigns from 1 to 0, the system safety confidence result changes from 90%
to 86%, which is illogical. Key evidence should have a decisive influence on the system’s
overall confidence [13]. Aiming at this problem, this paper proposes a modified confi‐
dence assessment approach for assurance case based on the existing research [6].
In the next section, the related work and supporting theory are briefly introduced.
After that, we modify the assumption that the same level elements in case structure for
confidence evaluation are independent of each other, discuss the relationships of the
elements and give a relationship classification. In Sect. 4, we compare the results under
the independence assumption or the correlation assumption using the same confidence
evaluation approach, and discuss the causes of differences. The conclusion and future
work to improve this approach are presented in Sect. 5.
2 Related Work
Assurance cases are generally developed to support claims in areas such as safety, reli‐
ability, maintainability, human factors, operability, and security. The assurance case has
one or more top level claims in which confidence is needed and has supporting arguments
connecting the top-level claims with multiple levels of sub-claims. The sub-claims are
in turn supported by evidences or assumptions.
As both arguments and evidences are practically imperfect, we could never say the
claims in our assurance case are 100% true. So we consider the confidence of each claim
in the case should be provided as other elements (e.g. assumptions, context) to facilitate
the decision making.
The studies on the evaluation of confidence in assurance case can be roughly divided
into two types according to their calculation methods. One kind is based on Bayesian
Belief Networks (BBNs). The researchers transform assurance case into BBN [3, 6], or
directly use BBN tools to construct network [4, 5]. The analyst assign probability value
(confidence) for leaf nodes, and The BBN tool could compute the confidence of the non-
leaf nodes, including the root-node representing the top goal. The other kind is based
on D-S theory, Jøsang’s opinion triangle, or Evidential Reasoning. Approaches based
on D-S theory [7–10], differ from Bayesian approaches in that they reason about both
the strength of belief in opinions and the plausibility of those opinions, which means
another dimension of the assessment of each claim in an assurance case. Jøsang’s opinion
triangle [11] reimagines the two dimensional space as three dimensions: belief, disbelief,
or uncertainty. As one of the dimensions grows, the other two dimensions shrink accord‐
ingly. Evidential reasoning [12] is a method to using the D-S theory in multi-attribute
decision problems.
We proposed our previous work [6] to calculate confidence using BBNs. However,
when we re-examined this approach, we found that it produced unreasonable system
safety confidence results when critical evidence was extremely assigned. In other words,
if the confidence of a critical evidence is extremely low (or high), there is no significant
Research on the Classification of the Relationships 7
change in the calculation result of the system’s confidence. This approach needs to be
improved to accommodate this kind of extreme assignment. A similar problem occurs
in [8], which amplifies small doubts about each hazard’s mitigation into large doubt
about system safety. This effect is the result of multiplying together the assessor’s beliefs
in the adequacy of the mitigation of each hazard and will worsen as the number of hazards
increases [13].
Assurance case evaluation approaches based on BBN usually assume that the same level
elements in case structure or case-like structure are independent of each other to simplify
the calculation - in fact this assumption is not always true. In this section, we revise the
independence hypothesis, discuss the classification of relationships among elements
with correlation, and give some suggestions on improving the case structure and leaf
node value assignment.
This section takes GSN [14] as an example. GSN has a typical top-down assurance
case structure, and the confidence of the top-level goal depends on the rationality of the
argument from sub-goal (solution) to top-level goal. Therefore, the first step to determine
the confidence (sufficiency) is to establish the correlations between each isolated sub-
goal and the top-level goal.
A solution (which represents the sub-goal or solution in GSN) can either fully support
the goal or partially support the goal. The support of the solution to the goal can usually
be divided into three types.
Type A represents a solution that supports one goal, and type B represents multiple
solutions that support the same goal. These two types are the main types of assurance
case argument. Type C means that N solutions can support the same goal independently.
Type C and type A are actually the same pattern. Type A can be regarded as a special
case of type C (i.e., (solution) N = 1), or the solutions of type C are coincident which
are completely equivalent for supported goal. Therefore, according to Fig. 1, we get the
solution classification as shown in Fig. 2.
Fig. 1. The types of supports that the solution provides to the goal
8 B. Xu et al.
As shown in Fig. 4, after that type C and type D are partitioned, they can be calculated
as equivalent to the combination of type A and type B. In other words, for the same
assurance case, when two correlative solutions support the goal together, the confidence
of the goal is equivalent to the confidence that the three (or two) sub-solutions offer after
the split.
Research on the Classification of the Relationships 9
Fig. 5. An example “basic BBN for the typical safety argument”, taken from [6]. For
compactness, we represent BBNs nodes as rectangles.
4 Case Study
We use these examples taken from [6] to illustrate that the proposed modified approach is
more sensitive to the critical factors. Thus, we can obtain results that are relatively more reliable.
We created four variants of the original example, namely Independence 1, 2 and
Correlation 1, 2. Independences represent extreme assessments of confidence in the
completeness of hazard identification classification. Correlations modify assessments
of confidence in the completeness of hazard information according to the relationship
that a large part of the node N2 and N3 are shared (Fig. 6).
The node N2 and N3 share a big part of both of them. Node N3 (“Coverage of
Equivalence Partition of Premises”) represents confidence in the completeness of
different equivalence partitions (i.e., whether all classifications of hazards were covered
in the hazard identification process), and Node N2 (“Whether Unprovided Premise is
Obtainable”) represents confidence in that “we provide all obtainable premises we
could”. It is not hard to see that these two claims are not independent, “all obtainable
premises are provided” is not credible without “classification coverage is sufficient”.
We grasp this idea with an example in software testing activity [6]. Assume that we
have tested the target software with 5000 test cases without any failure, so the tester may
have a ‘confirmation bias’ that the software is good enough. But are these 5000 test cases
adequate? The answer depends on the coverage of equivalence partitions of test cases. If the
5000 test cases come from all different equivalence partitions, we have high confidence to
say the software is good enough. In contrary, if the 5000 test cases represent only one
equivalence partition, we are hardly to say the software is good and we need to generate
more test cases.
We briefly name the shared part N0 as “Shared Premise”. According to the classi‐
fication discussed in the third section, the BBN structure should be modified to split the
original two nodes. However, [6] gives the node probability table for N12. Instead of
fixing the BBN structure, an assignment that takes into account correlation is given in
order to control the single variable (Table 1).
Table 2. Node probability table for N12 in Fig. 5, taken from [6].
N3 Complete NotComplete
N2 Obtainable NotObtainable Obtainable NotObtainable
Present 1 1 0.8 1
NotPresent 0 0 0.2 0
Research on the Classification of the Relationships 11
Analysis of Examples. Table 3 gives the calculated confidence in safety for all five
examples. It is not plausible that extreme negative changes in confidence in hazard
classification coverage would produce as small a change in confidence in safety as the
difference between 89% and 86% indicates. After we revise the values, it is somehow
plausible that anyone who completely distrusted a hazard analysis would only have 67%
confidence that the analyzed system is safe. The two optimistic examples do not make
any differences, because the original value is optimistic enough.
Table 3. Calculated value of N11 (“Justified Claim ‘system S is safe’”) for the original version
presented in [6] and our Independence and Correlation variants.
Case Present NotPresent
Original 89% 11%
Independence 1 86% 14%
Correlation 1 67% 33%
Independence 2 90% 10%
Correlation 2 90% 10%
5 Conclusion
is more sensitive to the change of key solution and can better reflect the influence of key
evidence in assurance case.
In practice, the relationships among elements are not limited to the same level. There
may be a cross-layer or cross-block relationship. The approach presented in this article
does not apply to this situation. To improve the approach, database of elements needs
to be developed for different domains. This database contains goals, sub-goals, solutions,
and relationships among these elements within each domain. An assurance case
construction tool with patterns and elements database could also be developed. The
database that stores the potential relationships among different types of elements will
help the developers to build and decompose assurance case for confidence evaluation.
References
1. Weinstock, C.B., Goodenough, J.B.: Towards an Assurance Case Practice for Medical
Devices. CMU/SEI-2009-TN-018 (2009)
2. Bloomfield, R., Littlewood, B., Wright, D.: Confidence: its role in dependability cases for
risk assessment. In: International Conference on Dependable Systems and Networks,
Edinburgh, pp. 338–346 (2007)
3. Denney, E., Pai, G., Habli, I.: Towards measurement of confidence in safety cases. In: 2011
International Symposium on Empirical Software Engineering and Measurement, vol. 9337,
pp. 380–383 (2011)
4. Guo, B.: Knowledge representation and uncertainty management: applying Bayesian Belief
Networks to a safety assessment expert system. In: Proceedings of the 2003 International
Conference on Natural Language Processing and Knowledge Engineering, NLP-KE 2003,
pp. 114–119 (2003)
5. Hobbs, C., Lloyd, M.: The application of Bayesian Belief Networks to assurance case
preparation. In: Dale, C., Anderson, T. (eds.) Achieving Systems Safety, pp. 159–176.
Springer, London (2012). https://doi.org/10.1007/978-1-4471-2494-8_12
6. Zhao, X., Zhang, D., Lu, M., Zeng, F.: A new approach to assessment of confidence in
assurance cases. In: Ortmeier, F., Daniel, P. (eds.) SAFECOMP 2012. LNCS, vol. 7613, pp.
79–91. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33675-1_7
7. Ayoub, A., Chang, J., Sokolsky, O., Lee, I.: Assessing the overall sufficiency of safety
arguments. In: SSS 2013 21st Safety-Critical Systems Symposium (2013)
8. Cyra, L., Gorski, J.: Support for argument structures review and assessment. Reliab. Eng.
Syst. Saf. 96, 26–37 (2011)
9. Guiochet, J., Do Hoang, Q.A., Kaaniche, M.: A model for safety case confidence assessment.
In: Koornneef, F., van Gulijk, C. (eds.) SAFECOMP 2014. LNCS, vol. 9337, pp. 313–327.
Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24255-2_23
10. Zeng, F., Lu, M., Zhong, D.: Using D-S evidence theory to evaluation of confidence in safety
case. J. Theor. Appl. Inf. Technol. 47, 184–189 (2013)
11. Duan, L., Rayadurgam, S., Heimdahl, M.P.E., Sokolsky, O., Lee, I.: Representing confidence
in assurance case evidence. In: Koornneef, F., van Gulijk, C. (eds.) SAFECOMP 2015. LNCS,
vol. 9338, pp. 15–26. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24249-1_2
12. Nair, S., Walkinshaw, N., Kelly, T.: Quantifying uncertainty in safety cases using evidential
reasoning. In: Bondavalli, A., Ceccarelli, A., Ortmeier, F. (eds.) SAFECOMP 2014. LNCS, vol.
8696, pp. 413–418. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10557-4_45
Research on the Classification of the Relationships 13
13. Graydon, P.J., Holloway, C.M.: An investigation of proposed techniques for quantifying
confidence in assurance arguments. Saf. Sci. 92, 53–65 (2017)
14. Kelly, T.: The goal structuring notation-a safety argument notation. In: Workshop on
Proceedings of the Dependable Systems and Networks (2004)
Continuous Argument Engineering:
Tackling Uncertainty in Machine
Learning Based Systems
1 Introduction
Software systems play an increasingly critical role in the real world, as further
investigated in emerging paradigms such as cyber-physical systems (CPSs) and
artificial intelligence (AI). A representative example is an autonomous driving
support system that deals with safety-critical operations in the real world. Assur-
ing the dependability of such systems is highly demanded as the systems have
more direct effects on society and people.
For these emerging systems, there is a notably active effort to apply machine
learning (ML) techniques given their recent advance, specifically in deep learning.
Implementations of advanced functions, such as image recognition, are obtained
in an inductive way from training data sets. This is quite different from clas-
sical software systems where implementations are obtained in a deductive way
by writing down processing rules as programs. The resulting implementations
contain intrinsic uncertainty. We cannot grasp what an implementation can do
exactly, nor can we logically explain the obtained results [6,7,14,15].
Clearly, arguments or assurance cases [9] play a significant role in assuring the
dependability of emerging ML-based systems. On one side, arguments describe
c Springer Nature Switzerland AG 2018
B. Gallina et al. (Eds.): SAFECOMP 2018 Workshops, LNCS 11094, pp. 14–21, 2018.
https://doi.org/10.1007/978-3-319-99229-7_2
Continuous Argument Engineering for Machine Learning 15
goals and how top-level abstract goals are decomposed into concrete goals that
are objectively measurable. On the other side, arguments describe how the sat-
isfaction of the goals is supported by evidences. Arguments can serve as the
foundation for analysis, discussion, and tracing done by development teams as
well as third parties.
However, uncertainty in ML imposes fundamental obstacles against the use of
arguments. It is more difficult or even impossible to be confident of completeness
in arguments. Development teams may still claim completeness to account for
their products in the best way currently known. Nevertheless, it will be increas-
ingly more significant not to stop at releasing arguments (and products) but to
continuously resolve uncertainty and react to expected or unexpected changes.
The development process also has such an incremental nature as the trial-and-
error style is required for uncertainty. For example, ML systems usually require
proof-of-concept (POC) development and evaluation. We believe arguments can
play an essential role in such incremental and continuous activities in addition
to the currently typical role in assurance for release.
In this paper, we discuss the significance of continuous argument engi-
neering in tackling the uncertainty of emerging systems, specifically ML-based
systems that contain intrinsic uncertainty even in an implementation. Our con-
tributions are as follows.
It was characteristic of the man reading the letter that he did not
show his rage by flushing. His nose, however, became a livid, sickly
white, and his thin lips were pressed somewhat more closely
together, causing his mouth to resemble a straight, colorless scar.
His face was that of a most dangerous man who would strike at an
enemy’s back in the dark.
There were other paragraphs that Hutchinson read without
skipping a line: