See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/262035859

First Contact with Safety Cases: a Quick Guide to Common Questions and
Misconceptions

Article · May 2013

CITATIONS READS

0 100

2 authors:

George Despotou Tim Patrick Kelly

The University of Warwick The University of York
65 PUBLICATIONS   287 CITATIONS    222 PUBLICATIONS   4,211 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

C3-Cloud View project

D-MILS View project

All content following this page was uploaded by George Despotou on 05 May 2014.

The user has requested enhancement of the downloaded file.

First Contact with Safety Cases: a Quick Guide to Common Questions and
Misconceptions
George Despotou and Tim Kelly

Safety Critical Systems Club newsletter, May 2013.

Safety cases have been used in the safety domain for a number of years, mostly in the
defence, aerospace and energy domains. Their usefulness, as a tool to improving safety, has
been appreciated by many practitioners. This has resulted in the core concepts of safety cases
to be transferred to other domains (e.g. automotive), and their focus on other system
attributes (e.g. security cases). This article provides concise answers to most common
questions about safety cases, also aiming to explain a number of common (based on the
experience of the authors) misconceptions.

What is a safety case?

Often developers have the onus to defend a position (i.e. make a case) about the safety of
their system. This, usually involves an explanation of how the available information for a
system supports the claim that risks in a system have been acceptably managed. Safety is not
the only system aspect on which this approach has been used; there are other examples, such
as reliability and maintainability cases as well as security cases. The case includes all relevant
reasoning and information of all stakeholders, which are then explicitly captured in some
graphical or textual format, and summarised in safety case reports. There are three
fundamental elements in a safety case: the claim or claims that the developer will make about
the system, the evidence that will support these claims, and the argument that will explain
how the available evidence allows to draw the conclusion conveyed by the claim.

Why is it useful and in some cases necessary to explicitly

document a safety case?
Development of a (safety) case is a requirement in many standards. However, the usefulness
of the safety case has been appreciated in the industry and is used by many organisations as
good practice. The reason for this is that explicitly capturing all reasoning, and information
(about the supported position) such as assumptions and evidence, facilitates assessing the
fitness of the design to meet its (safety) objectives. A manufacturer will design a system
aiming to achieve the required operational attributes. However what is intended is not often
what achieved. Once the reasoning of the developers is explicitly documented as claim and
argument supported by evidence, the gap between what was intended and what achieved
becomes more apparent. Explicitly documenting the safety case will contribute towards the
factual representation of the system, revealing which claims can be supported by evidence
and which, remain intention (for example, because there is no sufficient evidence to support
them). This may not necessarily imply that the latter claims have not been implemented, but
that we are unaware about their achievement as they are not sufficiently supported. There can
be three cases when a claim is not sufficiently supported: a) there is not sufficient evidence to
warrant the claim, b) although there is evidence, there is inadequate explanation (a
problematic argument structure) as to how the evidence supports the claim, and c) the claim
was phrased in a way that is unsupportable.

Why do arguments form a necessary part of a safety case?

Claiming that a system will meet certain safety goals cannot be shown by raw evidence (e.g.
test results). Explanation is needed that explains (argues) how the evidence support one’s
claims. Occasionally, for smaller parts of a system, evidence can directly support the desired
behaviour (e.g. mathematical proofs). But even then we often make or imply arguments that
what has been implemented corresponds to what has been proven. We cannot draw
conclusions with absolute certainty, but the conclusions that we make are only stated to a
degree of confidence, depending on the quality of evidence and the quality of the explanation
(argument) supporting one’s claims. Quality of both evidence and the argument is composite
attribute, encapsulating issues such as the relevance and integrity of evidence, the structure of
the argument and the processes used.

Who owns the safety case?

A safety case is of interest to many stakeholders (e.g. regulator, manufacturer, supplier).
However, by and large, two types of safety case are usually distinguished according to who is
responsible for their development and maintenance. The design safety case, which is
developed by the design authority during development of the system, and the user safety case,
which is owned by the operator of the system. In general, safety cases may be split and
organized according to ownership of arguments and evidence. Nevertheless there are
dependencies as all claims about safety ultimately refer to the same system which has been
designed for an assumed operational context. For example, arguing about safe operation will
depend on assumptions made by the designer (e.g. operational envelope of a
system). Achieving sufficient safety with sufficient assurance will eventually involve
acknowledging these dependencies with the necessary (inter-stakeholder communication)
side-effects. Organising the layout of the safety case in a way to make the dependencies, as
well as the responsibilities of each stakeholder more apparent, can be a way to alleviate (to a
certain) degree this challenge.

What are notations such as GSN?

The reasoning behind the claims in a safety case can be complex; free text often results in
arguments that are difficult to follow, thus making the case unclear and incomprehensible. It
is believed that supporting text by capturing parts of the safety case in a graphical format,
contributes towards the clarity of the safety case. The Goal Structuring Notation (GSN) is a
language containing all the necessary concepts to capture an argument in a structured format.
GSN comes with a graphical notation and a method that can be used to facilitate the
construction of an argument.

Will the safety case make my system safer?

Following the right principles to construct a safety case can help to reveal potential gaps in
the reasoning, and thus will contribute to make the system safer (if these gaps are corrected).
A safety case will help reveal these gaps in a number of ways: a) offering traceability
between claims about the system allowing to explore the steps from the evidence to the
ultimate position that a system is safe, b) by offering transparency to the assumptions made,
the evidence used, and c) by facilitating review, which becomes even more noticeable when
documented in one of the notations. However creating a safety case does not imply a safe
system. A safety case is nothing more but another viewpoint of your system. Specifically a
(GSN) safety case will offer the view of ‘postcondition’ claims made about the overall safety
of the system, the hazards addressed and the contribution of the subsystems to the identified
hazards. A safety case should be thought of as another means towards safety and not as an
objective.

Is a process needed to develop a safety case?

Yes. An argument should not be produced at the end of the system development, as it will
eventually lead to an argument ‘forced’ to fit the already developed system, thus
misrepresenting reality. Safety case development should be seen as the means to evaluate the
fitness of the system to support the the claims made in the argument. The safety case process
will allow interaction between the safety and the design teams, constantly evaluating the
measures in place to manage the risks of the system. Furthermore, the safety case will
provide developers with an opportunity to identify (thus enabling early planning) the
information that will need to be part of the safety case, such as appropriate evidence and
safety or design analyses. Finally, the process of creating the safety case allows the
identification of any counter-evidence and counter-arguments, that may refute the claims of
the safety case. Counter-evidence should not be ignored, as finalising a safety case in
presence of counter-evidence would imply that it is known that a number of the safety case
claims may not be true, thus undermining the entire safety case. The design should be
adapted to resolve the issues raised by the identified counter-evidence.

Is there a need for a safety case when a standard is

followed?
Safety cases and standards serve a different purpose. Standards prescribe a number of steps,
methods, and tools to be used when developing a system, which are considered (i.e. offer
confidence in the safety claims - see confidence argument below), based on extensive
empirical experience and domain expertise, to enable the development of a safe system.
Usually, the rigour (number and depth of activities) required by the prescribed processes,
reflects the estimated criticality of the system (i.e. the more rigorous a followed process the
safer the system). However, it should be noted that they imply and do not explain safety. A
system should not be seen as safe because a standard was followed, but because the tools and
process prescribed in the standard, are believed to result in a convincing claim (and evidence)
about management of the risk. The safety case enables fleshing out this argument, capturing
the explanation for managing risks. Standards provide a very helpful body of knowledge that
can be used to increase the confidence that we have in the claims that we make about a
system. For example, they can contribute to the generation of trustworthy evidence. Safety
cases and standards can be seen as complementary and a number of standards do require the
creation of a safety case.

Why is the purpose of hazard, compliance and confidence

arguments?
Experience has shown that a compelling safety argument will contain claims about issues
tangential to the actual system risk, such as the quality of the evidence produced in the safety
case. These claims have been categorised as belonging to one of these arguments allowing for
separation of concerns, hence facilitating the development and management of the safety
case. The hazard argument contains the core safety argument of the safety case, explaining
how the hazards have been addressed to acceptable risk levels, and what evidence we have to
support this claim. The confidence argument answers the question: why should we believe
the claims made in the hazard argument? It contains claims about issues such as the
trustworthiness, sufficiency, and relevance of evidence, tools and processes used; and
anything the safety engineers may think that will help a reviewer to have confidence in the
hazard argument. Finally, the compliance argument will allow the developer to document
which parts of the safety case offer compliance with associated requirements of an applicable
standard.

Many GSN safety cases start with very similar claims.

This is not something unexpected, as safety critical systems share common goals (i.e.
management of hazards). GSN arguments start from more generic claims that develop to
specific claims, supported by evidence (during development though GSN may be constructed
both top-down and bottom-up). In most systems the top level claims will represent a
statement about the achieved safety of the system. Often the claims will contain the word
acceptable. This represents the fact that there is no absolute safety; but a system can have
residual risk that is acceptable for its operation. In its intended use, the word acceptable
should be associated with other GSN elements (context) explaining what the stakeholders
mean by acceptable safety. Using the word acceptable in a claim without defining it, is an
example of improper application of GSN, suggesting that a step (that identifies and explains
the basis of a claim) of the GSN construction method was missed.
GSN claims are often ambiguous and complex.
This is a phenomenon of incorrect use of GSN. GSN claims should be kept simple and there
is plenty of guidance allowing the formation of cogent and clear claims and arguments.

What is CAE and how it relates to GSN?

The Claims Argument Evidence (CAE) notation, is a language used for the same purpose as
GSN. GSN and CAE, have both influenced the definition of the Object Management Group
(OMG) - the organisation owning standards such as UML - assurance case language
(Structured Assurance Case Metamodel) standard, with the intention to be used in highly
critical domains (safety, security).

What is confirmation bias and how it affects my safety

case?
Confirmation bias is a type of bias, according to which people tend to favour information that
supports their beliefs. This is a natural and expectable cognitive bias that may result in
introduction of weaknesses in the case, such as omitted information, assumptions, big and
unexplained jumps of reasoning and low quality of evidence. Nevertheless, confirmation bias
is present in all human activities. The effects of confirmation bias can be present during the
application of standards (e.g. poor fault trees) and even development (e.g. design that does
not reflect the requirements). To avoid this, organisations usually set up internal assessment
teams independent to the construction of the safety argument, and some standards require the
safety case to be approved by an independent (external) safety assessor. Explicitly
documenting the safety case using a graphical notation can offer transparency to the
underlying reasoning about the safety of a system. This can help to identify the effect of bias,
as this would be more easily identified by inspecting the structure of the argument graph (and
therefore easier to correct).

Why then does the developer make a case about safety

instead of a third party?
Developers are a very valuable asset to the safety case. They are the stakeholders with the
most expertise of the system, and the ones who are able to produce (and interpret) crucial
pieces of evidence, other than black or grey box testing that could at best be performed by a
third party; for example, walkthrough analyses, and proofs. A good safety case construction
process (guided by relevant standards) and training, should provide an organisation the means
 to use this expertise whilst mitigating the effect of bias.

Will capturing the safety case result in significant cost

overhead?
No; the additional cost would be that of the persons responsible with the actual writing of the
safety case report and the argument in one of the notations. Although there is no specific
indication of the man-hours this might require, there is consensus among all organisations,
which have written a safety case, that this overhead is insignificant (when compared to the
entire cost of certification). What is often misunderstood as a safety case overhead is the
additional analyses or testing, or design changes an organisation will need to make, because
of weaknesses that have exposed by writing the safety case. However, these weaknesses are
not caused by the safety case, and they would have been present (or more precisely their
existence would have been unknown) in the system had they not been revealed.

Safety cases have been used in the safety critical domain for a number of years. Although
initially imposed by standards, their usefulness as means of communicating confidence in the
safety of a system has been appreciated. This resulted in safety cases to be considered as
good practice and a number of organisations to adopt the concept, which is increasingly used
in other domains such as security. Although safety cases are considered good practice,
adopting one does not necessarily imply achievement of safety. Ultimately safety cases can
be considered as another tool to be used as means towards achieving safety. Development of
a safety cases requires an appropriate way of approaching the concept. This article provides a
quick introduction to safety cases, offering an initial understanding to the most, based on the
experience of the authors, prevalent questions regarding safety cases

View publication stats