Position

Paper

HSE Workplace Health Expert Committee

(WHEC)
Confidence and Uncertainty

WHEC-08 (2019)
WHEC Report
 Confidence & Uncertainty

HSE Workplace Health Expert Committee

(WHEC)

Confidence and Uncertainty

Figures 1 and 4 republished with permission of BMJ Publishing

Group Ltd from ‘Measuring inconsistency in meta analysis’, by
Higgins et al in British Medical Journal 2003, volume 327, pages 557
– 560 and ‘Effects of intervention on the cardiovascular mortality of
workers exposed to carbon disulphide’, by Nurminen and Hernberg in
British Journal of industrial Medicine 1985, volume 42, page s32 - 35;
permission conveyed through Copyright Clearance Center, Inc.

This report, its contents, including any opinions and/or conclusions

expressed, are those of the committee members alone and do not
necessarily reflect HSE policy.

WHEC
Workplace Health
Expert Comittee

HSE Workplace Health Expert Committee (WHEC) 2

 Confidence & Uncertainty

Foreword

The development of policy in HSE needs to be informed by the best available contemporary
scientific evidence. In 2015, HSE formed the Workplace Health Expert Committee (WHEC) to
provide independent expert advice to them on:

■ New and emerging workplace health issues

■ New and emerging evidence relating to existing workplace health issues

■ The quality and relevance of the evidence base on workplace health issues

Questions about workplace health issues come to WHEC from many sources, which include HSE,
trade unions, employers, interested individuals and members of WHEC. WHEC’s responses to
these questions are published online as reports to HSE, as position papers following investigation,
or as a briefer response where the current evidence is insufficient to warrant further investigation.
In cases where the evidence-base is limited WHEC will maintain a watching brief and undertake
further investigation if new and sufficient evidence emerges.

In its formal considerations, WHEC aims to provide answers to the questions asked based on
the available evidence. This will generally include review of the relevant scientific literature,
identifying the sources of evidence relied on in coming to its conclusions, and the quality and
limitations of these sources of evidence.

The purpose of WHEC reports is to analyse the relevant evidence to provide HSE with an informed
opinion on which to base policy. Where there are gaps in the evidence, which mean that this is
not possible, WHEC will identify these and, if appropriate, recommend how the gaps might be
filled.

HSE Workplace Health Expert Committee (WHEC) 3

 Confidence & Uncertainty

Confidence and Uncertainty while duration, timing of exposures at work and interval
from first exposure can often be accurately assessed,
The primary role of WHEC is, based on the best
estimation of the level, or intensity, of exposure may in
contemporary scientific evidence, to provide advice to
many circumstances be informed guesswork, making
HSE on new and emerging health risks and on new and
accurate estimation of the relevant exposure difficult,
emerging evidence on recognised health risks to the
while information on potential confounding factors may
UK workforce. Two measures of risk are of particular
be absent or limited. All of these factors can serve to
relevance in this context: attributable risk – the additional
threaten the internal validity of a study and undermine
risk an individual incurs as a consequence of an exposure
confidence in the existence of an association, in
at work; and population attributable risk (PAR) – the
estimates of its magnitude or its causal relevance.
burden of illness or disease in the population that is a
consequence of the exposure. PAR will depend on both
Because of the many potential problems, the findings
the attributable risk and the prevalence of exposure in the
of a single observational study alone can only very
population.
rarely be considered sufficient to provide evidence of
cause and effect. Normally, it is necessary to consider
While the regulator’s policy is informed by such evidence,
a body of evidence drawn from all available studies,
it will be one of a number of considerations which
epidemiological, toxicological experiments and other
policy makers will take into account in decision-making.
forms of non-epidemiological research; if these are few,
Other factors may include the affordability and risks of
only cautious inferences can follow.
any proposed intervention in relation to the predicted
benefits, its deliverability, the values of policy makers and
Not uncommonly studies do not agree, sometimes
stakeholders and other constraints.
reaching opposite conclusions. More frequently they
disagree about the size and importance of effects and
The role of WHEC is the consideration and analysis of
the exposure levels that cause them. Circumstances
evidence: what do we know: how confident can we be in
can conspire to force different definitions of exposure
the inferences made and where do the uncertainties lie?
on different investigators. In this situation, an overview
In some circumstances, WHEC can also provide evidence
is needed, focussing on whether a causal relationship
in relation to different options for intervention.
between a putative hazard and adverse outcome is likely
to exist, at least in some circumstances; how big it is
Intervention in the workplace usually requires established
likely to be and in what circumstances; and how much
evidence on occupational causation. Deciding on this
confidence can be placed in the various judgements.
requires a judgement on the relevant scientific evidence.
Beyond this are judgements for the regulator about
Studies to investigate causation in working populations
the feasibility, expediency and effectiveness of control
are usually, necessarily, observational in nature (case­
measures.
referent, cross-sectional or cohort studies), which are
more vulnerable to biases and confounding than well-
Bradford Hill proposed guidelines to assess the likelihood
executed blinded randomised controlled trials. Selection
of an association being causal. Although well-recognised
into, and for remaining in, the workforce is far from
exceptions exist to many of these (other than the absolute
random and blinding of participants to their exposures at
requirement that exposure must precede and not follow
work is not possible, each, potentially, depending on the
the effect), they offer a useful guide, especially given the
study question and design, leading to biased estimates
potential disparities to be found in a body of evidence.
of risk. In addition, an appropriate comparison group in
The most important of the criteria, from the perspective of
studies of working populations can often be difficult to
interpretation of occupational epidemiology, are probably
secure, making comparison of ‘like with like’ difficult.
1) the strength of the association, in terms of relative risk
In these circumstances internal comparisons based on
or risk ratio, the ratio of the risk in the exposed to the
different levels of exposure are often used. However,
risk in the unexposed: the higher the risk ratio, the less

HSE Workplace Health Expert Committee (WHEC) 4

 Confidence & Uncertainty
likely it is to be explained by chance or by unrecognised
confounding; 2) evidence of exposure-response gradients:
causal inference is supported if a bigger ‘dose’ causes a
bigger effect or greater likelihood of disease; 3) repeatability
of the findings: a consistent observation, demonstrated
in different settings, different populations, different eras
by different study designs, is less likely to arise through
bias, confounding or by chance; 4) biological plausibility:
evidence of a plausible underlying biological mechanism
for the particular agent causing the disease: this is
inevitably limited by contemporary knowledge of disease
mechanisms; and 5) the effectiveness of intervention: does
removal of the hazardous exposure makes disease less
likely?
In a few circumstances the strength of an association can
be sufficiently great (say a risk ratio of more than 10) to
have confidence that, (unless very strongly associated with
another risk factor), it is probably causal. An example of
this would be the description by Percival Potts of scrotal
cancer in boys climbing chimneys, in whom Richard Doll
estimated the risk ratio to be 200. Nonetheless, even here
confidence in causation is strengthened by the finding
of an increased risk of scrotal cancer in those in direct
contact in their work with mineral oils also containing
polycyclic aromatic hydrocarbons. A risk ratio in excess of
10 makes confounding by an unrecognised independent
risk factor unlikely. Risk ratios of this level are found in
relationships such as lung cancer with cigarette smoking,
and with asbestos exposure in those exposed before the
1931 Regulations in the UK, and the risk of mesothelioma
in carpenters who worked in the construction industry
in the UK in the 1960’s and 1970’s. Another example
is adenocarcinoma of the nose or para-nasal sinuses in
workers heavily exposed to hardwood dust: relative risks in
the hundreds have sometimes been reported, with many
studies reporting values above 10.
Risk ratios this large are now unusual in studies of
occupational hazards. It can be difficult to make confident
inferences from a relative risk, say of less than 3, although
relative risks of lower magnitude, if true, can be important,
particularly when the population at risk is large, with a
consequent large population attributable risk. Risk ratios
of less than 1.5 are frequently reported in contemporary
studies, such as those investigating the relationship of
HSE Workplace Health Expert Committee (WHEC)
silica and lung cancer, which could easily arise through
confounding and, in the case of lung cancer and silica, be
explicable wholly or in part by cigarette smoking and other
agents such as ionising radiation and arsenic, to which
exposure can occur in the workplace.
In these circumstances, replication of the findings, with
different studies generating similar risk estimates and
consistent evidence of exposure-response relationships,
can provide important supporting evidence of causation.
Where undertaken, evidence for the effectiveness of an
intervention to reduce the level of exposure in reducing the
increased risk of disease, can provide the most powerful
evidence of causation. It is important that replication of
findings comes from studies of high quality undertaken in
different populations in different circumstances of exposure.
Publication bias (the tendency for striking findings to be
published more readily than neutral, negative, or non-
significant ones), for instance, can ensure consistency
of findings, often from small studies which are wrong or
which exaggerate the risks. High quality studies often
generate lower estimates of relative risk than low quality
studies, as is illustrated by estimates of the association of
residential exposure to electromagnetic fields and childhood
leukaemia (Fig 1. From Higgins et al, 2003). Rothman
and Poole (1988) have suggested that weak associations
in epidemiological studies might be strengthened by a)
restricting attention to populations otherwise at low risk and
b) reducing non-differential misclassification of exposure
and of disease which tend to dilute associations.
After
Figure 1. Meta-analysis of six case-control studies of association of
residential exposure to electromagnetic fields and childhood leukaemia
5
 Confidence & Uncertainty
A recent review by WHEC of the relationship between
inhaled respirable crystalline silica (RCS) and lung
cancer identified several well-designed cohort studies,
undertaken in several different circumstances of exposure
(Workplace Health Expert Committee). The great majority
found an increased risk of lung cancer of the order of
20% to 30% (RR 1.2-1.3), where the population at risk
was followed up more than 15 years from initial exposure,
recognised confounders were taken into account, to
the extent possible, and contemporary measurements
of exposure were sufficient to allow categorisation of
exposure (Fig 2).
Figure 2. Risk ratio for lung cancer in cohorts of workers exposed to
crystalline silica
Figure 3. Lung cancer in 10 silica exposed cohorts with good quality
exposure data. Pooled analysis
Such evidence of a repeatable increase in risk of this
order is suggestive but not sufficiently convincing. More
convincing evidence was provided by several different
studies which showed exposure-response relationships,
which are unlikely to be a consequence of confounding.
One important multi-centre study, reported by Steenland
HSE Workplace Health Expert Committee (WHEC)
in 2001, of a pooled exposure-response analysis of 10
cohorts of silica exposed populations of some 44,000
miners and some 22,000 non-miners found clear
evidence of an exposure-response relationship (Fig 3).
Interpretation of epidemiological findings is considerably
helped by evidence of biological plausibility, demonstrated
in human and animal toxicological studies and in vitro
investigations. Such studies often involve the use of
animal species (e.g. rodents from well-characterised
strains), exposed in varying patterns, timeframes and
doses that simulate workplace exposures. Toxicological
studies involving workers or volunteers, although less
common, can also inform on subtle short-term pre-clinical
endpoints, such as irritation of the upper respiratory tract
or neurotransmitter enzyme perturbations. Experimental
control over confounders and effect modifiers can be
greater than in human field studies and this simplifies
interpretation. In addition, a wide range of doses can
be used to explore dose-response and dose-effect, and
substances can be tested in animals and in vitro for
which there is, as yet, no human experience. However,
since animals often have different metabolism and
pathophysiological responses from humans, there may
not be perfect read-across to the human experience.
Collectively, however, toxicological studies can offer
understanding of the molecular and pathophysiological
changes underlying disease-exposure associations
and strengthen confidence in the existence of a causal
relationship. Hazard and risk assessment is now usually
incomplete without the use of adjunctive toxicological
information.
The most convincing evidence of cause and effect is
the reduction in risk of disease following an intervention
shown to reduce the level (intensity) of exposure. Doll and
Bradford Hill’s demonstration of the reducing risk of lung
cancer in cigarette smokers who had stopped smoking
overcame Fisher’s suggestion that the association of
lung cancer and cigarette smoking was the consequence
of a common genetic predisposition to both. Sadly few
formal evaluations of interventions in the workplace have
been reported. Among relatively few published reports
is Hernberg’s demonstration of the reduction in the risk
of ischaemic heart disease over a 15 year period which
followed a measurable reduction in the levels of CS2 in a
Finnish rayon factory (Fig 4. Hernberg et al, 1976).
6
 Confidence & Uncertainty
Figure 4. Rate ratio od deaths from IHD in workers exposed to CS2 in
rayon factory workers v local paper mill workforce
Doll’s studies of lung cancer in an asbestos textile factory in
those first exposed before and after the implementation of
the 1931 Asbestos Regulations also showed a considerable
reduction in the increased risk from 14 to 2 (Doll, 1955,
Peto et al, 1977). Doll also demonstrated how process
change in Welsh nickel refineries had effectively eliminated
a marked increase in risk of nasal cancer (Doll et al, 1977).
Before-after studies from Japan and Finland (Futatsuka
et al, 1989; Koskimies et al, 1992) recorded a marked
reduction in the prevalence of vibration-induced white finger
among forestry workers following industry-wide technical
improvements in their tools and new devices to dampen the
vibration imparted to their hands.
Provision of scientific advice to government requires
evaluation of the totality of the relevant evidence.
Confidence in an association being causal needs to be
based on: (1) the internal validity of the individual studies:
to what extent may the observed associations be due to
chance, bias or confounding? (2) The coherence of the
findings of the different studies: do the different studies
add up to making causation likely? Other questions for the
regulator concern (3) the extent to which the findings will be
generalizable across settings; and (4) the extent to which
the studied exposure conditions are relevant to workplaces
in the UK.
In addressing the second of these questions, a relevant
concern relates to the robustness of the evidence. In
particular, could a new well-designed study with negative
findings change our judgement of cause and effect?
HSE Workplace Health Expert Committee (WHEC)
A single study of a working population exposed to asbestos
which failed to find an increased risk of lung cancer would
not now undermine our confidence in asbestos as a
cause of lung cancer, just as a negative study on cigarette
smoking and lung cancer would not lead us to discount the
large body of evidence in favour of a causal relationship with
smoking. In contrast, a recent report finding no relationship
between shift work and breast cancer, while not negating
previous observations of an increased risk, does give reason
to pause before coming to judgement.
How best to express confidence and uncertainty?
“The scepticism that I advocate amounts only to
this: (1) that when the experts are agreed, the opposite
opinion cannot be held to be certain; (2) that when they
are not agreed, no opinion can be regarded as certain by a
non-expert; and (3) that when they all hold that no sufficient
grounds for a positive opinion exist, the ordinary man would
do well to suspend his judgement”.
Bertrand Russell ‘Sceptical Essays’
Ideas about certainty and uncertainty represent judgements
about available evidence. However, experts do not always
reach the same judgements when faced with the same
information, and opinions may be subject to revision as new
information comes to light. A possible way of expressing
certainty/uncertainty about causal relationships is in terms
of five qualitative descriptors - e.g.:
1. “Almost certain”
2. “Probable”
3. “Possible/uncertain”
4. “Unlikely”
5. “Improbable”
As a guide only, it may be suggested that “almost certain”
could mean >90% likely (i.e. one would expect such
conclusions to turn out to be right >90% of the time); and
that values of >60<90%, 40-60%, >10<40% and <10%
could be assigned to the other descriptors. In practice,
however, and for policy purposes, any assigned percentage
(which would not be quantitative in any rigorous scientific
sense) should matter less than words which successfully
7
 Confidence & Uncertainty
capture the view of a majority of experts who have properly
considered the matter. This should provide sufficient guide as
to general confidence in the scientific evidence and therefore
the scientific grounds for intervention.
The framework we propose above is different from that used
by the International Agency for Cancer (IARC) for the purpose
of identifying human carcinogens. Broadly speaking, the
IARC classifies the available data for humans and animals
separately and then employs a system for bringing the two
strands of evidence together. According to IARC’s system,
human data on carcinogenicity may fall into three categories
of evidence: ‘sufficient’ (meaning very strong); ‘limited’
(meaning plausible and with supporting evidence, but not
proven - “a causal interpretation is considered credible,
but chance, bias or confounding cannot be ruled out with
reasonable confidence”); or ‘inadequate (weak or absent).
Animal data on carcinogenicity are similarly classified into
three categories as ‘sufficient’, ‘limited’ or ‘inadequate’.
According to IARC, Group 1 agents (those deemed definite
human carcinogens) are normally those for which the human
evidence is ‘sufficient’, irrespective of the animal data; group
2A agents (‘probable’ carcinogens) are those with ‘limited’
human data but ‘sufficient’ animal data; while group 2B
agents (‘possible’ carcinogens) are those with ‘limited’ human
evidence and also ‘limited’ animal data. This set of definitions
gives considerable weight to the animal data, as the boundary
between a ‘possible’ and a ‘probable’ carcinogen can shift
according to a difference in the strength of the animal data,
and does not necessarily reflect a view about differences in
the strength of human evidence. In contrast, WHEC takes
differences in the strength of the human data to indicate
potential differences in the likelihood of a causal relationship
in humans, using animal data as one strand of supportive
evidence for inferences in humans.
The IARC framework is an assessment of hazard. Implicit
in the framework proposed by WHEC is the idea of risk at
a given level of exposure. In practice, however, researchers
may have more confidence in the fact that an environmental
agent is hazardous (causal in some circumstances) than in
the relationship between exposure levels and risk, and the
dose levels which are sufficient or insufficient to cause the
disease (‘unsafe’ or ‘safe’). For example, the evidence linking
diesel emissions with cancer is growing and may now be at a
HSE Workplace Health Expert Committee (WHEC)
point when expert opinion would regard a causal association
to be “probable”; but there would be less agreement between
experts over where to set exposure limits. It may be helpful
therefore to contextualise a descriptor of certainty/uncertainty
by reference to defined exposure levels (and when these are
unclear to make this apparent to policy makers).
Figure 5 represents schematically the strengths of association
(risk ratios) and repeatability of findings in epidemiological
studies for several hazard-outcome permutations of interest.
The very high risk ratio of 150 reported in relation to sino­
nasal cancer and work in a nickel refinery prior to the 1950s
makes occupational causation ”almost certain”, even though
the number of studies on the topic, and so the repeatability of
evidence, is limited. For lung cancer and smoking, the subject
of much investigation, the risk ratio is high, on average
10, the repeatability of findings and the case for causation
high – again, almost all experts would agree that a causal
relationship is “almost certain”.
Figure 5. Schematic representation of the strength and consistency of
several putative causal associations between hazard and disease
In comparison, for lung cancer in silica workers, the much lower
risk ratios (around 1.3) are in a range that could easily be explained
by bias or confounding, although, as described above, the
consistency of exposure-response relationships in different studies
in different circumstances of exposure makes causation “probable”,
and the case for intervention is less clear-cut than for the examples
of the nickel refinery worker or the cigarette smoker.
Fig. 6 shows several hazard-outcome permutations where the
risk ratio has been estimated to be less than 3. Confidence
in the association being causal in these examples requires
additional evidence, such as consistent exposure-response
8
 Confidence & Uncertainty
relationships and biological plausibility, before acceptance
that the association is causal. Nonetheless there are
circumstances where risk ratios of this magnitude are
generally considered evidence of a causal relationship;
a good example would be cigarette smoking and
cardiovascular disease, where consistent evidence of a risk
ratio of about 2 has been found in a large body of scientific
reports.
Figure 6. Schematic representation of the strength and consistency
of several putative causal associations between hazard and disease
in which estimates of the risk ratio are <3
Apportioning a probability of causation in a particular
circumstance is ultimately a matter of judgement. Most
experts would judge the associations in the nickel refiners
and asbestos workers to be ‘almost certainly’ causal,
but might debate whether that for lung cancer and silica
exposure is ‘probable’ or ‘possible’, perhaps favouring the
former. Critically, new evidence could sway opinions on this
last judgement.
Such a framework can be used to identify circumstances
in which the regulator can have confidence in the need
to act; others in which further research evidence will be
important to garner; and yet others in which there is too
much uncertainty at present to support enforcement actions
or strongly worded advice to employers and workers.
HSE Workplace Health Expert Committee (WHEC)
References
Doll R. Mortality from lung cancer in Asbestos Workers. Br J
Ind Med 1955;12:81-86.
Doll R, Mathews JD, Morgan LG. Cancers of the lung and
nasal sinuses in nickel workers: a reassessment of the
period of risk. Br J Ind Med 1977; 34: 102-5.
Futatsuka M, Ueno T, Sakurai T. Cohort study of vibration-
induced white finger among Japanese forest workers over
30 years. Int Arch Occup Environ Health 1989; 61: 503-6.
Hernberg S, Tolonen M, Nurminen M. Eight-year follow-up of
viscose rayon workers exposed to carbon disulfide. Scand J
Work Environ Health1976;2:27-30.
Higgins JPT, Thompson SG, Deeks JJ, Altman DG.
Measuring inconsistency in meta-analyses. BMJ 2003; 327:
557–560.
Koskimies K, Pyykko I, Starck J, et al. Vibration syndrome
among Finnish forest workers between 1972 and 1990. Int
Arch Occup Environ Health 1992; 64: 251-6.
Nurminen M, Hernberg S. Effects of intervention on the
cardiovascular mortality of workers exposed to carbon
disulphide: a 15 year follow up. Br J Ind Med 1985; 42:
32–35.
Peto J, Doll R, Howard SV et al. A mortality study among
workers in an English asbestos factory. Br J Ind Med
1977;34:169-173.
Rothman K, Poole C. A strengthening programme for weak
associations. Int J Epidemiol 1988;17:955-959.
Steenland K, Mannetze A, Boffetta P et al. Pooled exposure
analyses and risk assessment for lung cancer in 10 cohorts
of silica-exposed workers: an IARC multicentre study.
Cancer Causes and Control 2001;12:773-784.
Workplace Health Expert Committee. Silica and lung cancer.
http://webcommunities.hse.gov.uk/connect.ti/WHEC/
grouphome
9
 Confidence & Uncertainty

What is WHEC?

The Workplace Health Expert Committee (WHEC) provides

independent expert opinion to HSE by identifying and
assessing new and emerging issues in workplace health.
Working under an independent Chair, WHEC gives HSE
access to independent, authoritative, impartial and timely
expertise on workplace health.

https://webcommunities.hse.gov.uk/connect.ti/WHEC/
view?objectId=235408&exp=c1

WHEC membership

Professor Sir Anthony Newman Taylor (Chair)

Professor Peter Buckle
Professor John Cherrie
Professor Paul Cullinan
Emma Donaldson-Feilder
Professor Len Levy
Professor Keith Palmer
Professor Martie Van Tongeren

HSE Workplace Health Expert Committee (WHEC) 10