TOLERABLE RISK CRITERIA:

CRITERIA:–
THE ANCOLD GUIDANCE

Len McDonald
Past Convenor,
Convenor ANCOLD
Working Group on Risk
Assessment
OVERVIEW OF PRESENTATION

„ A – ANCOLD AND GUIDELINES

„ B – RISK ASSESSMENT GUIDELINES

„ C – TOLERABLE RISK GUIDELINES

„ D – ISSUES WITH THE GUIDELINES

 PART A

ANCOLD AND GUIDELINES

 ANCOLD
„ A non-profit voluntary association of organizations and
professional people with an interest in dams
„ Formed in June 1937 and in its present form since 1960
„ An active member of the International Commission on
Large Dams [ICOLD]
„ Currently 49 member organizations and 143 associate
members
b
„ Holds an annual conference on dams and study tours
rotating around the states and New Zealand
„ Publishes a technical bulletin three times each year
 GUIDELINES

„ There is currently a suite of 13 technical

guidelines prepared by ANCOLD
„ Th
These guidelines
id li serve as de
d facto
f t
standards for dams in Australia
[Standards Australia does not produce
codes specific to dams]
„ Apart from promoting technical
excellence, a main aim of the guidelines
is consistent practice across the country
DAM SAFETY REGULATION

„ Australia is a federation of six states and two

territories
„ Dam safety is regulated in four states and one
territory
„ The regulators all rely on ANCOLD guidelines,
sometimes
ti with
ith qualification,
lifi ti and
d on recognized
i d
good practice [such as Engineers Australia,
USACE, USBR and FERC]
„ Owners in the un-regulated jurisdictions
generally rely on ANCOLD guidelines and
recognized good practice
 PART B

RISK ASSESSMENT GUIDELINES

 HISTORY
„ Guidelines on risk assessment first commissioned by
ANCOLD IN 1987
„ Interest was triggered by spillway optimization studies and
risk studies on flood provisions during construction of
dams [for example, deciding on the height to which
reinforced rockfill should extend]
„ The first Guidelines on Risk Assessment were published in
January 1994
„ Updated guidelines
gu de es weree e commissioned
co ss o ed around
a ou d 1996
996 in
response to requests for additional guidance
„ The second [current] Guidelines on Risk Assessment were
published in October 2003
p
 THE 1994 GUIDELINES

„ Intended as a review of the state of the art

and an introduction to risk assessment
concepts rather than a guide to
undertaking risk assessment
„ Life safety risk guidelines were a key
concern Some practitioners were –
concern.
controversially - assigning a monetary
value to potential life loss at that time
„ An extensive literature review was
undertaken to try and reach a conclusion
on life
o e sa
safety
ety gu
guidelines
de es
 THE 1994 PROCESS

„ Mainly a literature review

„ A small working group of 4 people [2 dam
owners and d 2 engineering
i i consultants
lt t – one with
ith
experience of dam safety regulation]
„ 2 formal reviewers [Prof. J Stedinger, Cornell
University and Mr D Newton, Tennessee Valley
Authority]
„ Li it d knowledge
Limited k l d and d experience
i off risk
i k
assessment within ANCOLD at that time
„ Limited international network on risk aspects
 THE 2003 GUIDELINES

„ Intended as a guide to the steps required

to undertake risk assessment
„ Life safety risk guidelines remain a key
issue – with significant shifts in emphasis
f
from the
th 1994 guidelines
id li
„ Within the ANCOLD membership, there
was now considerable
id bl experience
i in
i
applying risk assessment
 THE 2003 PROCESS [A]

„ Extensive literature review

„ Working group of 14 people [4 dam owners, 6
d
dam safety
f t regulators,
l t 3 dam
d engineering
i i
consultants and 1 university professor]
„ 2 formal reviewers, from Canada [Mr R Stewart,
BC Hydro] and from USA [Dr J Smart, USBR]
„ Extensive comments by several other reviewers
„ Tentative proposal for tolerable life safety
guidelines released at ANCOLD conference in
August 1998 for guidance and comment
 THE 2003 PROCESS [B]

„ Attendance of at least one working group

member at 9 international meetings [6 of ICOLD
Committee on Dam Safety] devoted to risk
assessment for dams plus 2 extra meetings of
authors of an ICOLD bulletin on risk assessment
„ Several meetings with overseas and Australian
regulators of hazardous industries
„ A risk workshop in New Zealand [November
2001] for
f review
i off the
th draft
d ft guidelines
id li by
b the
th
ANCOLD membership
„ 19 Federal and State agencies responsible for
public safety reviewed the near final draft
guidelines
 PART C

TOLERABLE RISK GUIDELINES

 1994 LIFE SAFETY RISK GUIDELINES

„ Assigning a monetary value to life was rejected

„ Key concepts of individual risk, societal risk and ALARP
were identified
„ A limit and an objective were recognized as boundaries
for three risk zones:
• Unacceptable risks
• Risks subject to ALARP [as low as reasonably
practicable]
• Acceptable risks
„ There was no guidance on meeting the ALARP test
 1994 INDIVIDUAL RISK GUIDELINES

„ For new dams, and upgrading of existing

dams, ensure that the average risk of
death to particular members of the public
from dam failure does not exceed 10-6 per
annum. Do not subject
j anyy person,
p , beingg
a member of the public, to a risk greater
than 10-5 per annum
„ F existing
For i ti dams,
d Individual
I di id l Risks
Ri k up tot
10 times those for new dams could be
tolerable,
to e ab e, subject to application
app cat o ofo the
t e
ALARP Principle
1994 SOCIETAL RISK GUIDELINE
 1994 SOCIETAL RISK GUIDELINE

„ Based on a proposal by Dr J Higson who

had reviewed societal risk criteria from
the time of Farmer [1967] – seemed to
reflect the findings of social
psychologists
p y g on risk aversion
„ New dams and upgradings to meet the
objective risk level
„ Existing dams to meet the limit but
subject to ALARP
 2003 GUIDELINES – KEY
DEVELOPMENTS POST-1994
„ United Kingdom Health and Safety Executive
[HSE] published R2P2 [Reducing Risks,
Protecting People, 2001]
„ HSE ppublished three internal guides
g on
application of life safety criteria, 2001
„ HSE published the Ball and Floyd report Societal
Risks,, 1998
„ United States Bureau of Reclamation [USBR]
published Guidelines for Achieving Public
Protection in Dam Safety Decision Making, 1997
and 2003
„ PlanningNSW had re-published Risk Criteria for
Land Use Planning, 2002
 2003 GUIDELINES - MAIN CHANGES
„ A clearer understanding of ANCOLD
ANCOLD’s s role in proposing life
safety guidelines
„ Shift in focus from acceptable risk to tolerable risk
„ Concept of average individual risk [from Kletz 1976 and
others] abandoned – it had proved unworkable for dams
and had no real meaning as a measure of equity
„ Focus shifted from objective [now broadly acceptable level
after HSE] to limit of tolerability, with ALARP to apply
indefinitely below – ALARP now the key criterion
„ Horizontal truncation of societal risk F-NF N curve – whether to
have a risk averse curve now less of an issue – adopted the
straight risk neutral line at a slope of -1 [log-log plot] based
on Ball and Floyd [1998]
TOLERABLE RISK PRINCIPLES

„ Equity – the right of all individuals to

equal treatment in terms of a minimum
level of safety – achieved by classifying
risks higher than the limit of tolerability
as unacceptable
p
„ Efficiency – society’s need to gain the
greatest benefit from the available
resources – ALARP informs
i f the
th balance
b l
between equity and efficiency with the
balance
ba a ce skewed
s e ed in favour
a ou of o safety
sa ety
HSE RISK REGIONS
BASIS OF INDIVIDUAL RISK

„ A dam or other facility should not impose

on any individual an increment of risk
that is more than a small fraction of their
background risk
„ Wh
Where an individual
i di id l is
i exposed d to
t risks
i k
from multiple facilities, the aggregate
increment of risk needs to be considered
BACKGROUND RISK AUSTRALIA
 2003 INDIVIDUAL RISK GUIDELINES

„ for existing dams

dams, an individual risk to the
person or group, which is most at risk,
that is higher than 10-4 per annum is
unacceptable, except in exceptional
circumstances;
„ for new dams or major augmentations of
existing dams, an individual risk to the
person or group, which is most at risk,
that is higher than 10-5 per annum is
unacceptable except in exceptional
unacceptable,
circumstances
 SOCIETAL RISK GUIDELINES [A]

„ Controversial – not set by HSE [apart from one

point] or the New South Wales planning
regulator – but ANCOLD saw as necessary to
obtain consistent safety practices
„ Usuallyy defined byy F-N pplot [the
[ alternative is
limits on expected value of life loss]
„ F is the probability of failure per dam per year
with expected [means estimated] loss of life ≥ N
„ N is the [estimated] number of fatalities due to
dam failure
 SOCIETAL RISK GUIDELINES [B]

„ Based on F-N charts used by regulators in the

UK, the Netherlands, Hong Kong and NSW [last
one not official policy]
„ E i ti dams
Existing d – as in
i chart
h t on the
th nextt slide
lid
„ New dams and major augmentations [marginal
cost of additional safety similar to a new dam] – a
similar chart with probability of failure one order
lower
„ Not on chart but a broadly acceptable level of risk
two orders lower than the limit of tolerability
[needed for ALARP demonstration]
SOCIETAL RISK – EXISTING DAMS
 THE HORIZONTAL TRUNCATION

„ For some dams, the very low 1994 societal risk probabilities of
failure could not be achieved or demonstrated [e.g., foundations
not fully known, not feasible to fully investigate old dams]
„ If society wants dams it must accept the lowest level of risk that
is achievable
„ The horizontal truncation is to be viewed as the intersection of
two lines a) the societal risk criterion line with a slope of -1
1 and
b) the horizontal truncation line at 10-5 per annum [existing dams]
„ With a horizontal truncation whether there should be a risk
neutral criterion line [[slope
p of -1 on a log-log
g g plot]
p ] or a risk
averse line [slope steeper than -1] becomes less of an issue
 ANCOLD POSITION ON THE EXPECTED
VALUE APPROACH

„ There are two main concerns with expected value of life

loss as a criterion:
• the zero-infinity nature of dams, very low likelihood of
failure but catastrophic consequences should failure
occur. Whether expected value is a suitable measure for
such situations is questionable (NRC, 1985 - page 251).
Impact on psyche of decision
decision-maker;
maker;
• a single expected value number hides information, in that
it can result from an indefinitely large number of different
combinations of event probability and consequence pairs.
The overall expected value number does not tell a
decision-maker anything of the various dam failure
outcomes that could occur.
 ALARP CONCEPT

„ Edwards vs The National Coal Board 1949:

“Reasonably practicable” is a narrower term than
“physically
physically possible”
possible and seems to me to imply that a
computation must be made by the owner in which the
quantum of risk is placed on one scale and the sacrifice
involved in the measures necessary for averting the risk
(whether in money, time or trouble) is placed on the other,
and that if it be shown that there is a gross disproportion
between them – the risk being insignificant in relation to
the sacrifice
f – the defendants
f discharge the onus on them.
 VPF F AND CSSL
VPF,

„ Value in preventing a fatality

[VPF], proportion factor [F],
cost to save a statistical life
[CSSL]
„ CSSL = [F]*[VPF]
„ CSSL is equivalent to the
HSE concept of CPF [cost of
preventing a fatality]
 VALUE IN PREVENTING A
FATALITY [VPF]

„ United Kingdom - GBP 1.0 million

[AUD 2.5
2 5 million then] – HSE 2001
„ United States – USD 2.0 million to
USD
S 6.3 million [AUD 2.4 million to
AUD 7.6 million now] – Brannon 2005
„ Australia – AUD 2.5 million – Abelson
2003
 PROPORTION FACTOR [F] – THE
CONCEPT

„ Edwards vs The National Coal Board 1949:

…..in every case, it is the risk that has to be weighed
against the measures necessary to eliminate the risk. The
greater the risk, no doubt, the less will be the weight to be
given to the factor of cost.[the underline is mine]
„ HSE 2001 [the ALARP guide]:
HSE has not formulated an algorithm which can be used to
determine the proportion factor for a given level of risk.
Theee
extent
te t o
of tthe
ebbias
as must
ust be aargued
gued in tthe
e light
g toof all
a tthe
e
circumstances.
PROPORTION FACTOR VALUE
[A]

„ HSE 2002 [nuclear inspection guide]:

Although this evidence was produced sometime ago, no
subsequent legal proceedings or public inquiries have
countered these views or provided alternatives. In his
evidence John Locke suggested a proportion factor of up
to 3 for workers. For risks to the public the factor would
depend on the level of risk, and where the risks were low
(consequence and likelihood) a factor of about 2 is
suggested, whereas for higher risks the factor would be
about 10 times.
PROPORTION FACTOR VALUE
[B]

„ Rimington et al. 2003:

HSE have suggested in the past a multiplicand of
3 applied
li d to
t the
th estimated
ti t d value
l off an increment
i t
of risk reduction at risk levels near the
tolerability
y limit [[in this context, means the
broadly acceptable level], but higher figures up
to a multiplicand of 10, have also been
suggested at the topmost area of the tolerability
region. [underline is mine]
CSSL – COST TO SAVE A
STATISTICAL LIFE
 PROPORTION FACTORS FOR
ANCOLD CSSL TABLES
„ for risks above the limit of tolerability
y – an infinite
proportion factor;
„ for risks within the upper part of the tolerable
region – a proportion factor of about 10
„ for risks within the lower part of the tolerable
region – a proportion factor of 3
„ f risks
for i k within
ithi the
th middle
iddl off the
th tolerable
t l bl region i
– a proportion factor of say 6 (by interpolation)
„ for risks within the broadly
y acceptable
p region
g –
nobody worries too much about further risk
reduction unless there are some obvious low cost
p
improvements that could be made
 BASIS FOR ANCOLD CSSL TABLES

„ Concept of CSSL as strength of justification to

proceed from Bowles and Anderson [2003];
„ HSE proportion factor multiplied by VPF gives
lower bound result – a floor [HSE 2001];
„ US p
practice [vague
[ g – no defined boundaries]] from
Bowles and Anderson [2003], Kniesner [1997] and
OMB [1992] gives upper bound result – a ceiling;
„ ANCOLD tables fall between floor and ceiling
ceiling.
ANCOLD CSSL TABLE [A]
ANCOLD CSSL TABLE [B]
 OTHER FACTORS IN
DEMONSTRATION OF ALARP

„ CSSL but
b t one consideration
id ti – demonstration
d t ti off ALARP is
i
not an exercise in mathematics
„ economic losses – direct and indirect
„ incommensurable losses – environmental,
environmental public health
impacts
„ intangible losses – grief, societal trauma, loss of public
confidence
„ societal concerns – regional community devastation,
especially vulnerable people at risk, loss of iconic
structures – need community consultation to fully expose
societal concerns
„ recognized good practice – is it established for dams in the
HSE sense?
 ANCOLD ON UNCERTAINTY

„ The estimated risks to be compared with life safety criteria are to

be “best estimates”
„ Estimation of the uncertainty of probability and consequences
p
should be made and reported to the decision-maker
„ Probability is itself a representation of uncertainty [ the
uncertainty of probability is thus the uncertainty of an
uncertainty]
„ Distinguish
g between aleatory y uncertaintyy [[that due to random
variation] and epistemic uncertainty [that due to limited
knowledge] – different impact on conclusions [aleatory often
self-cancelling, epistemic a systematic bias]
„ Monte Carlo simulation is a way of propagating uncertainty
th
throughh the
th analysis
l i
„ Sensitivity testing is valuable in identifying key uncertainties
 SHORT DURATION RISKS

„ …..it is reasonable to recall that the life safety criteria at

G10-1 [tolerable risk guidelines] are not qualified as to
period of exposure.
Consequently, for dams under construction or being
modified, the life safety risks should be estimated, and the
risks reduced ALARP within the tolerability zone, by such
measures as:
• all reasonably practicable measures at the site, such as
provisions for safely passing floods, that will reduce the
life safety risks;
• construction phase Dam Safety Emergency Plans,
including flood warning and evacuation planning.
 PRIORITY AND URGENCY
„ priority (the order in which risk reduction measures are to be
implemented) –give priority to the highest risks. Give priority to
life safety risks over other risks;
„ urgency (how soon the measures should be implemented) –
make urgency proportional to the extent by which risks exceed
tolerable risks, as determined by tolerable risk policies or
criteria. Risks higher
g than the limit of tolerability,
y, in particular
p
life safety risks, need to be reduced to a level below the limit as
soon as practicable;
„ progressive
p g improvement
p –pplan for studies or safety y
improvements in stages, if that would achieve the best
outcomes in reducing risk for the available resources.
 OTHER RISKS

„ ANCOLD did not propose life safety guidelines

for workers because this is covered by
legislation
„ ANCOLD did not propose guidelines for
p
economic loss because these would be specific
to each owner and are affected by insurance
arrangements
„ Guidelines for environmental impacts were not
proposed because this is an area covered by
legislation in most states
 PART D

ISSUES WITH THE GUIDELINES

MINOR ISSUES FROM USE OF GUIDELINES

„ Some minor, but important, flaws have emerged:

• on F-N guideline charts “>N” on the vertical axis
should be “≥N”
≥N [arose from printing problem]
• ANCOLD believes an F-N plot is not a CCDF
[complementary cumulative distribution
function] but some doubt remains [Bedford and
Cooke 2001 – p356]
• some definitions could need adjustment
accordingly
 MAJOR ISSUES FROM USE OF GUIDELINES

„ The horizontal truncation on FF-N

N charts has caused some
difficulty [covered in DSC presentation]
„ There is a mis-match between the CSSL threshold for
ALARP and traditional dam engineering standards [DSC
presentation] – ANCOLD will likely move to seek a
resolution of this issue
„ The short duration provisions [earlier slide] could put
embankment dams out of contention for some sites [DSC
presentation]
„ Analyses of the uncertainty of risks have necessarily been
limited – comprehensive analysis of uncertainty is too
intimidating a task to date