Applied Ergonomics 106 (2023) 103906

Contents lists available at ScienceDirect

Applied Ergonomics
journal homepage: www.elsevier.com/locate/apergo

Review article

Are two-person checks more effective than one-person checks for safety
critical tasks in high-consequence industries outside of healthcare? A
systematic review
Ryan D. McMullan a, *, Rachel Urwin a, Mark Wiggins b, Johanna I. Westbrook a
a
Australian Institute of Health Innovation, Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, Australia
b
Centre for Elite Performance, Expertise, and Training, Macquarie University, Sydney, Australia

A R T I C L E I N F O A B S T R A C T

Keywords: Double-checking has been used in high-consequence industries for decades. We aimed to determine the strength
Double-checking of the evidence-base regarding the effectiveness of double-checking which underpins its widespread adoption.
High-consequence industries We searched for quantitative studies of the effectiveness of two-person checking in industry sectors, excluding
Systematic review
healthcare. We performed a systematic literature search across six databases and hand-searched key journals. We
completed a narrative synthesis and quality assessment of the nine studies identified. Most studies were of fair
quality. Two examined the use of two-person checks in aviation, three investigated tasks in chemical
manufacturing, and four studies in psychology involved proofreading and visual search tasks. All studies found
that the performance of two-people checking was not superior to that of one-person in detecting errors. Further
research to compare the effectiveness of different checking processes along with factors which may support
optimisation of safety checks in high-consequence industries is required.

1. Introduction
“Arm doors and cross-check” is a familiar refrain in the standard
operating procedures of modern airline travel. In essence, the procedure
requires flight attendants to check that their colleagues have correctly
armed/disarmed airline doors to ensure that emergency escape slides
are operational/disengaged according to the configuration of the
aircraft. This principle of redundancy, often applied in engineering and
employed in settings as disparate as chemical plants and hospitals, is
intended to reduce the likelihood of system failures and increase reli­
ability by requiring that a number of people or processes independently
perform the same function.
In healthcare, ‘double-checking’ has been a recommended and
mandated practice in many clinical situations for decades. It is often
performed during the administration of medication due to the high
frequency of errors that occur at this stage of the drug treatment process
(Keers et al., 2013). In this context, double-checking involves two nurses
verifying the same information. To improve reliability, it has been rec­
ommended that nurses complete ‘independent double checks’, where
they each independently verify the information (ISMP, 2003). This is
based on the assumption that if these processes are undertaken correctly,
the threat of individual biases (e.g., confirmation bias) will be mini­
mised, such that two nurses are unlikely to make the same error at the
same time (ISMP, 2013).
Double-checking or cross-checking is also used in other high-
consequence industries as a process for building redundancy, a recog­
nised component of high reliability organisations (Hofmann et al., 1995;
Lekka, 2011; Roberts, 1990). In multi-crew cockpits, the pilot moni­
toring will call out a checklist item that will be confirmed or corrected by
the pilot-flying. Similarly, in communicating with air traffic control, a
pilot will read-back a command that will be checked and confirmed by
the air traffic controller. Different operators at chemical plants monitor
and cross-check the same automated processes as a strategy to prevent
errors. This reflects inherent belief that, should one operator fail to
identify an irregularity in the system, another operator will compensate
for this failure by responding appropriately.
The nature of the checking task can differ, such that checking can
involve matching information (e.g., matching flight panel information
against a checklist in an aircraft cockpit) (Pfeiffer et al., 2020; White
et al., 2010), drawing on knowledge and critically analysing information
(e.g., checking to determine the dose of a medication based upon a
child’s body weight) (Pfeiffer et al., 2020; White et al., 2010), or

* Corresponding author. Australian Institute of Health Innovation, Level 6, 75 Talavera Road, Macquarie University, NSW, 2109, Sydney, Australia.
E-mail address: ryan.mcmullan@mq.edu.au (R.D. McMullan).

https://doi.org/10.1016/j.apergo.2022.103906
Received 15 June 2022; Received in revised form 12 September 2022; Accepted 13 September 2022
Available online 20 September 2022
0003-6870/© 2022 Elsevier Ltd. All rights reserved.
R.D. McMullan et al.
monitoring (e.g., monitoring a system that autonomously controls
chemical processes) (Cymek, 2018). Effective checking behaviour re­
quires the allocation of attentional resources to compare the features of
the system to the features expected at the particular point in time.
However, this comparative process is cognitively demanding and where
the process becomes routine, attentional resources may be withdrawn,
leading to checking errors. That is, the completion of routine tasks can
become ritualised, so they are not performed with the degree of atten­
tion necessary to identify deviations (Toft and Mascie-Taylor, 2005).
Using two-people in the checking process is intended to heighten
performance, increasing the likelihood that an anomaly will be detected.
However, there is little good quality evidence to confirm whether this is
the case in medication administration (Koyama et al., 2020). Further,
there is limited research evidence regarding factors that improve or
hinder checking performance. Our systematic review of the effectiveness
of double-checking to reduce medication administration errors identi­
fied a lack of evidence to support the process (Koyama et al., 2020). A
subsequently published, large empirical study of the effects of
double-checking on medication administration errors among 1523
paediatric inpatients found that independent double-checking was
rarely performed, even when mandated by hospital policy (Westbrook
et al., 2020). However, ‘primed double-checking’, where one nurse
shares information or primes a second nurse with information during the
checking process, was highly prevalent. No significant association was
evident between primed, mandated double-checking and the rate or
severity of medication administration error. The results from this large
study raise further questions about the possible safety value of
double-checking and highlights gaps in our understanding of when and
how double-checking may deliver the safety benefits anticipated.
The evidence-base for the effectiveness of double/cross-checking in
industries outside healthcare has not been systematically reviewed.
Thus, the extent to which the safety benefits attributed to such processes
is warranted is unknown. Given the extensive use of double-checking in
industries outside healthcare, an assessment of the extent and quality of
the evidence to support this process is important to both identify evi­
dence gaps and potential recommendations for optimisation of checking
processes.
The objective of this systematic review was to determine the strength
and quality of evidence regarding the effectiveness of double-checking
in safety critical industries outside healthcare including aviation, nu­
clear power, engineering, and chemical manufacturing. We performed a
systematic search for quantitative evidence to address the following
research question: Is two-person checking more effective than one-
person checking in detecting errors and system variations in high-
consequence industries outside of healthcare?
2. Method
This systematic review was conducted in accordance with the
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
(PRISMA) guideline (Page et al., 2021). The protocol for our system­
atic review was not pre-registered.
2.1. Search strategy
We performed a search of PsycINFO, Web of Science, Advanced
Technologies and Aerospace, Health & Safety Science Abstracts, Google
Scholar, and Scopus databases for scientific literature published from
database inception to April 12, 2022. The search strategy (see Appendix
A) was developed with the assistance of a research librarian and
included categories for checking, error, and relevant industries and
disciplines. Synonyms within these categories were linked with the
Boolean operator “OR” and linked across categories using “AND”. The
results of the search were merged using reference management software
(Endnote 20). After the removal of duplicates, two authors indepen­
dently screened titles and abstracts. Studies that were not relevant were
2
Applied Ergonomics 106 (2023) 103906
excluded. The same two authors examined full-text articles to judge
their eligibility in accordance with the inclusion criteria. Disagreement
or uncertainty between the two authors was resolved through discus­
sion. Reference lists and citations of included articles were inspected for
additional studies. The following key journals were hand-searched for
studies to ensure the completeness of the search (Hopewell et al., 2007):
Human Factors, Applied Ergonomics, Journal of Aviation/Aerospace Edu­
cation and Research, and Safety Science.
2.2. Inclusion criteria
Inclusion was contingent on the following: (a) an empirical quanti­
tative study (experimental, simulation, observational); (b) reported
quantitative outcomes related to the effectiveness of two-person
checking forthe detection of errors or system deviations; (c) the study
was conducted outside of healthcare; (d) published in a peer-reviewed
journal; and (e) was written in English. Studies were required to meet
all of the criteria to be included.
2.3. Data extraction
One author extracted data from the studies, comprising: first author;
year of publication; country; sector; study design; participants; nature of
task/s; outcome measures; and key findings. Key findings included re­
sults that related to the detection of errors, error rates, adherence to
checks, and detection of system failures when participants worked
individually versus in pairs. Extracted data were compiled in a table
using Microsoft Excel (data available on request from the authors).
2.4. Quality assessment
One author assessed the quality of the studies by using a quality
assessment tool used
widely to assess the methodological quality and risk of bias in
experimental studies (Hawker et al., 2002). This tool assesses studies on
each of nine categories: abstract and title; introduction and aims;
method and data; sampling; data analysis; ethics and bias; results;
transferability and generalizability; and implications and usefulness.
Each category is scored from good (4 points) to very poor (1 point) using
corresponding criteria (see Appendix B). A global quality grade is
defined based on the total score: good (28–36 points), fair (19–27
points), poor (10–18) and very poor (1–9 points).
2.5. Data synthesis
Due to the heterogeneity of the design and outcome measures of the
included studies, we completed a narrative synthesis of the studies.
3. Results
3.1. Results and study characteristics
The literature search retrieved 1661 articles. After title and abstract
screening, 1651 articles were excluded. One article was excluded after
full-text review. A total of nine studies were included in the qualitative
synthesis (Fig. 1). Study characteristics are outlined in Table 1.
Five studies were conducted in the United States of America (USA)
(Brown, 1963; Brown and Fox, 1965; Contte and Jacobs, 1997; Mosier
et al., 2001; Skitka et al., 2000), three in Germany (Cymek, 2018; Cymek
et al., 2016; Domeinski et al., 2007), and one in Japan (Nihei et al.,
2002). All of the studies were experiments. The year of publication
ranged from 1963 to 2018. Most studies were of fair quality (see Ap­
pendix B) (Brown, 1963; Brown and Fox, 1965; Cymek et al., 2016;
Domeinski et al., 2007; Mosier et al., 2001; Nihei et al., 2002; Skitka
et al., 2000). None of the studies published power calculations. Two of
the studies examined the use of two-person checks involving tasks in
R.D. McMullan et al.
Fig. 1. PRISMA flow diagram of selection process.
aviation (Mosier et al., 2001; Skitka et al., 2000), three studies involved
tasks in chemical manufacturing (Cymek, 2018; Cymek et al., 2016;
Domeinski et al., 2007), and four studies in psychology included
proofreading and visual search tasks (Brown, 1963; Brown and Fox,
1965; Contte and Jacobs, 1997; Nihei et al., 2002).
Of studies of double-checking in aviation (Mosier et al., 2001; Skitka
et al., 2000), one examined the performance of one-person versus
two-person crews in detecting automation errors (e.g., misloads of fre­
quency, altitude, arrival) in a flight simulator (Mosier et al., 2001).
Forty-eight pilots from three major commercial United States carriers
flew under conditions that varied based on crew size (i.e., they either
worked alone or with another participant), type of training, and whether
they received a prompt to verify automated functioning. Six automation
failures were included in the three stages of the simulation. The out­
comes indicated that there was no significant difference in performance
between single pilot and dual-pilot operations.
A second study investigated automation bias by having university
students complete computerised simulation monitoring and tracking
tasks involved in flying commercial aircraft (Skitka et al., 2000).
One-hundred and forty-four participants completed these tasks under
conditions that varied in crew size, type of training, and whether they
received a prompt to verify automated directives. The Automated
3
Applied Ergonomics 106 (2023) 103906
Monitoring Aid (AMA) on the task display provided participants with
notifications about events together with recommendations for specific
courses of action. Participants were advised that this aid was not 100%
accurate in detecting critical events. Across four trials, there were twelve
opportunities for error due to six occasions during which the AMA failed
to notify participants of an event, and six occasions on which the AMA
offered inappropriate directives. The results indicated that individuals
and pairs were equally likely to miss events and fail to respond to system
irregularities.
Three studies investigated redundancy and automation monitoring
in the context of chemical plants (Cymek, 2018; Cymek et al., 2016;
Domeinski et al., 2007). All three involved university students per­
forming a complex task that simulated operations in the control room of
a chemical plant. The first study compared participants who worked
alone (non-redundant group); with a second crewmember who worked
in parallel on the task (redundant group); and those who were informed
that the second crewmember’s performance may be low (informed
redundant group) (Domeinski et al., 2007). The findings indicated that
participants who worked in parallel with a second person (redundant
group) cross-checked the automation significantly less frequently than
the other groups. Subsequent studies reported similar findings, such that
participants who worked redundantly engaged in less cross-checking
R.D. McMullan et al. Applied Ergonomics 106 (2023) 103906

Table 1
Overview of studies that examined two-person checks.
Author/s, Sector Study design Participants Nature of task Key outcome measures Key findings Quality
year, country rating

(Brown, Psychology Experiment: Within- 12 male employees Visual monitoring Number of positive More positive errors in the Fair
1963), subjects design at the research task errors and negative redundant condition for low
USA laboratory errors and intermediate levels of
difficulty. More positive errors
in the nonredundant condition
for high level of difficulty.
(Brown and Psychology Experiment: Within- 16 male employees Visual monitoring Number of positive Fewer negative errors in the Fair
Fox, subjects design at the research task errors and negative redundant compared to the
1965), laboratory errors nonredundant condition.
USA Fewer positive errors in the
nonredundant compared to the
redundant condition.
(Contte and Psychology Experiment: 173 undergraduate Transcript checking Error rate: omission Participants worked faster but Good
Jacobs, Between-subjects students task errors, commission made errors more frequently
1997), design errors, categorisation when in a perceived reliable,
USA errors redundant system compared to
a condition where they worked
alone or in a perceived
unreliable, redundant system.
(β = .16, p < .05).
(Cymek Chemical Experiment: 2 x 12 36 participants (35 Multi-task consisting Monitoring Participants who worked Fair
et al., manufacturing mixed factorial were students) of three subtasks performance: number redundantly with another
2016), design similar to those in a of trials where the participant performed
GER chemical plant automation was fully significantly less cross-checks
control room cross-checked than participants who worked
alone (M = 84.56 vs M =
112.28, p = .03).
(Cymek, Chemical Experiments: Exp 1: Exp 1: 47 university Multi-task consisting Number of trials Exp 1: Participants in the Good
2018), manufacturing 3 x 12 mixed students. Exp 2: 36 of three subtasks where the automation redundant condition detected
GER factorial design. Exp university students similar to those in a was fully cross- significantly less automation
2: 2 x 12 mixed chemical plant checked. Detection of failures compared to
factorial design control room automation failures participants in the
nonredundant condition
(65.6% vs. 89.9%, p = .046).
Exp 1 and Exp 2: Participants
in the redundant condition
cross-checked significantly less
status messages compared to
participants in the
nonredundant condition (Exp
1: 70.8% vs. 86.7%, p = .019;
Exp 2: 55.0% vs. 75.8%, p =
.024).
(Domeinski Chemical Experiment: mixed 36 students Multi-task consisting Number of trials Participants in the redundant Fair
et al., manufacturing factorial design of three subtasks where the automation condition cross-checked the
2007), similar to those in a was fully cross- automation significantly less
GER chemical plant checked than participants in the
control room nonredundant condition (F
(2,33) = 13.05, p < .01).
(Mosier Aviation Experiment: mixed 48 commercial Flight simulator task Number of omission No significant difference for Fair
et al., factorial design glass cockpit pilots errors related to omission error rates between
2001), automation failures one-person and two-person
USA crews (52% vs 43%; p > .05).
(Nihei et al., Psychology Experiment: 48 undergraduate Proofreading task Number of contextual Pairs of participants detected Fair
2002), JAP Between-groups students errors and surface significantly more surface
design errors detected errors than individual
participants (t(30) = 5.296, p
< .001), whereas individual
participants detected
significantly more contextual
errors than the pairs of
participants (t(15) = 2.366, p
< .05).
(Skitka et al., Aviation Experiment: 2 x 3 x 2 144 undergraduate Monitoring and Number of omission No significant differences Fair
2000), between-subjects students tracking tasks errors and number of between one-person and two-
USA design commission errors person crews for omission and
commission errors (p > .05).

than participants who worked in isolation (Cymek, 2018; Cymek et al.,
2016).
There were four studies conducted in psychology. One study
compared participant performance when they either worked alone or in
a redundant system to identify errors in university transcripts (Contte
and Jacobs, 1997). The task involved comparing a checklist with an
official transcript. Participants were randomly allocated to a condition
in which they were told that they would work alone or as part of a
system that involved either a computer, a faculty member, or a faculty
member and a peer. Prior to completing the task, participants in the

4
R.D. McMullan et al.
redundant systems rated their confidence in the performance of the
other versions of the system. These ratings were used to determine
whether the redundant systems were perceived as reliable or unreliable.
The results indicated that participants worked faster but made errors
more frequently when in a perceived reliable, redundant system
compared to a condition where they worked in isolation or in a
perceived unreliable, redundant system.
Another study examined participant performance on a proofreading
task when working alone versus in pairs (Nihei et al., 2002). Participants
read two passages that contained both surface and contextual errors.
Participants were instructed to detect and correct misprints. Those
participants who worked in pairs were each asked to read the passages
and then discuss the errors detected. When participants worked in pairs,
they detected a significantly greater number of surface errors, whereas
individuals detected significantly more contextual errors.
Two studies conducted in 1963 and 1965 examined the effect of
redundancy on speed and reliability on a visual monitoring task (Brown,
1963; Brown and Fox, 1965). In both studies, participants observed a
display on which white circles were illuminated in different patterns.
Participants were provided with a card with ‘critical’ light combinations
and were instructed to release a telegraph key as fast as possible when
these light combinations appeared on the display. These ‘critical’ stimuli
were combined with non-critical stimuli. Stimulus complexity was
manipulated by varying the number of lights displayed for each stim­
ulus. Participants were recruited in pairs and completed the task in both
redundant and nonredundant conditions. In the redundant condition,
participants responded to all of the stimuli as a team. However, in the
nonredundant condition, each participant responded independently to
half of the stimuli. The first of these two studies noted that participants
made more false positive errors (i.e., gave a response to a non-critical
stimulus) in the redundant condition during low and intermediate
levels of difficulty, whereas they made more false positive errors in the
nonredundant condition during high levels of difficulty (Brown, 1963).
The second study found that there were more negative errors (i.e., a
failure to respond to a critical stimulus) in the nonredundant condition
compared to the redundant condition. However, compared to the
nonredundant condition, there were more false positive errors in the
redundant condition (Brown and Fox, 1965).
4. Discussion
Overall, our search revealed few quantitative studies that have
examined the effectiveness of double-checking or similar two-person
monitoring and verification processes across several disciplines. The
studies from aviation, chemical manufacturing, and psychology
demonstrated that the performance of two people checking was not
superior to one person in detecting errors. As it currently stands, the
evidence-base for double-checking as an effective practice is small and
of both mixed quality and findings. Accordingly, it is difficult to
conclude whether double-checking is an effective practice for error
detection.
In both aviation studies, the authors suggest that individual differ­
ences may play a role in a pilot’s interaction with automation (Mosier
et al., 2001; Skitka et al., 2000). For example, factors such as
self-confidence, workload and cognitive load have been found to influ­
ence human performance with automated systems (Parasuraman and
Riley, 1997). Such factors and how they relate to checking performance
and the presence of a second person were not examined in the studies of
double-checking in aviation nor those from chemical manufacturing and
psychology. However, three studies of automation monitoring in the
context of chemical manufacturing did find that double-checking was
undermined by social loafing (Cymek, 2018; Cymek et al., 2016;
Domeinski et al., 2007). That is, participants exerted less effort when
they were aware that they were working with others who they expected
would take greater responsibility during the task. Future research should
extend these findings by investigating how individual, interpersonal,
5
Applied Ergonomics 106 (2023) 103906
and environmental factors influence the effectiveness of
double-checking.
All of the studies except one recruited university students rather than
industry professionals (Mosier et al., 2001). Although Skitka et al.
(2000) suggest that previous studies have not found differences in stu­
dent and pilot samples for error rates and overall response patterns
across automation failures (Mosier and Skitka, 1996; Riley, 1996), the
generalizability of the study findings are limited. Additionally, none of
the studies published power calculations and were most likely under­
powered given both the design of the studies (i.e., between-groups
design, mixed factorial design) and the small sample sizes.
4.1. Types of double-checking
There are different types of two-person checks. Independent double-
checking is often mandated when nurses administer certain types of
medication and specifies that each nurse separately checks medication
details. Collaborative double-checking is another type often performed
in aviation, where one checker calls out information which is then
confirmed by a second checker. It could be the case that collaborative
double-checking is more effective for some safety-critical tasks, whereas
independent double-checking could be effective for others. However,
published studies comparing the factors involved, as well as the effec­
tiveness of these different forms of double-checking, are rare. Impor­
tantly, different types of double-checking should be compared with
single-checking since the interaction between two people may involve
factors that are not present when one person completes the same tasks. If
such factors are not controlled, they may undermine any added safety
benefits of two-person checking practices.
Few studies have investigated the fidelity of checking processes as
performed in real-world settings. Our recent field study found that when
nurses performed double-checks during medication administration, they
almost always failed to independently check as required by hospital
policy (Westbrook et al., 2020). Instead, double-checks involved
collaboration and information priming between the two nurses. This
study also demonstrated that the costs of a second nurse to double-check
medications at one 340-bed hospital was over $7000 per day, or $2.7M
annually. Nurse costs associated with double-checking processes in
Australia, which has over 740 hospitals, are in the order of $5M per day
nationwide.
In a report from NASA (Dismukes and Berman, 2010), flight crew
procedures during 60 airline operations were directly observed to
examine why checklists sometimes fail to catch errors and equipment
malfunctions. Typically, the pilot monitoring reads each checklist item,
and the pilot flying checks the item and verbally responds that it is set
correctly, and the pilot monitoring then cross-checks the item. The
findings indicated that there were 585 failures in this process. In 43
instances, a pilot confirmed without visually inspecting the item, and in
42 instances items from the checklist were omitted or called out incor­
rectly. In 113 instances, the required verification was not obtained. For
example, a first officer set and called out a new altitude, but the captain
was distracted and did not verify the new altitude on the primary flight
display. Both of these studies indicate that checking processes are not
always completed as recommended.
The two direct observational studies of checking processes in real-
world practice are rare. The small evidencebase makes it difficult to
determine whether a failure of double-checking to detect errors is due to
a failure in the fidelity of the process (i.e., whether it was performed as
intended), due to the influence of environmental factors, or because
double-checking itself provides no safety benefits. Further investigation
is required to generate new evidence about why failures in adherence to
checking procedures occur, and how improvements can be made to
optimise checking. The true extent of failures in checking is largely
unknown due to a lack of research evidence. As Dismukes and Berman
(2010) observed, only 18% of observed process failures by aircrew were
corrected, suggesting that many failures go unnoticed.
R.D. McMullan et al.
4.2. Types of checking tasks
The type of checking task must also be identified to evaluate the
significance of differences between tasks, the cognitive resources
required to complete the tasks, and the factors that may influence the
effective completion of tasks. Tasks can be classified as mechanistic (e.
g., comparing a prescription against the label of an IV bag), abstract (e.
g., drawing on knowledge to realise a medication needs to be diluted in a
different solution) (Pfeiffer et al., 2020; White et al., 2010), or moni­
toring tasks (e.g., monitoring a system that autonomously controls
chemical processes) (Cymek, 2018). The distinction between mecha­
nistic and abstract checking tasks is an important one and may be a
useful basis for identifying steps in double-checking that are at greater
risk of disruption. For example, declarative memory retrieval is more
negatively affected by stress than procedural memory retrieval
(Schwabe et al., 2012). Therefore, stress may threaten abstract tasks
more than mechanistic tasks. It is essential to investigate checking
processes considering these factors if we are to understand the processes
conceptually and determine their potential effectiveness to reduce error.
4.3. Future directions
The complexity of high-consequence environments, such as health­
care settings, often makes it difficult and unethical to explore the impact
of various factors on processes in real-world observational studies.
However, simulation provides a safe and controlled environment
whereby researchers can manipulate and control variables to examine
individual behaviour, generate theory, and evaluate interventions
(Lamé and Dixon-Woods, 2020). Greater use of laboratory experiments
and simulation studies to systematically investigate double-checking is
required to determine if and when double-checking strategies are likely
to be effective. Such evidence should then be used as the basis to design
checking policies which are responsive to specific task contexts and
consider the roles of both humans and technology.
There are important factors that cannot be reproduced in the lab,
such as organisational factors (e.g., safety culture). Instead, direct
observational studies and qualitative interviews may be more appro­
priate methods to first explore clinical and technical practice in the local
context, and second to elucidate the impact of organisation level factors
on the effectiveness of checking processes. The ‘work-as-imagined’
versus ‘work-as-done’ distinction is an important one for policy-makers
to acknowledge, because if checking guidelines are disconnected from
real-world work processes, they are unlikely to be followed and will
consequently prove ineffective at improving safety (Clay-Williams et al.,
2015). Additionally, policy for how and when specific checking pro­
cesses should be performed must be supported by strong empirical ev­
idence. However, we can infer from the findings of the present synthesis
that there is a disconnect between policy and empirical evidence, given
that there is currently a limited evidence-base of mixed study quality
and findings for the effectiveness of double-checking. A stronger
evidence-base must be established by researchers, to inform guidelines
for checking practices that are compatible with and effective in
real-world settings.
A further future consideration is the role of technology, which is
increasingly being used to automate checking processes, reducing the
demand for some human checking. However, while technology may be
useful for ‘matching tasks’, higher-order checking tasks may be more
problematic. The integration of technology as part of the checking
process can lead to a false sense of security and automation bias and may
result in an individual placing undue confidence in an automated pro­
cess and failure to adequately question or check (Mosier et al., 2001;
Skitka et al., 2000). For example, Lyell et al. (2017) found staff were
more likely to accept false positive alerts generated by an electronic
prescribing system, such that they assumed the computer ‘must be right’
and failed to check (Lyell et al., 2017). Understanding the types of
checking tasks that can safely be automated within complex workplace
6
Applied Ergonomics 106 (2023) 103906
environments requires improved evidence and understanding of human
checking performance. Thus, future investigations which incorporate
the use of technology in the checking process should be considered.
4.4. Strengths and limitations
This is the first systematic review to synthesise evidence for the
effectiveness of double-checking across high-consequence industries
outside of healthcare. Our review had several limitations. Our search
was limited to English-language studies. Grey literature was not
included. Heterogeneity in the studies limited us to a narrative summary
of the included studies.
5. Conclusion
The nine studies from aviation, chemical manufacturing, and psy­
chology do not support the adage that two people are better than one.
Redundancy in high-consequence industries may be advantageous for
technical systems, but it is less clear whether this is similar for human
redundancy in healthcare, aviation, and chemical engineering. It is
difficult to determine the effectiveness of double-checking based on the
current evidence-base that is both small and of mixed quality. Although
safety checks are an essential part of workflows for high-consequence
tasks across many industries and sectors, there is little empirical evi­
dence to improve their design and application. Therefore, further studies
are required to build a stronger evidence-base from which checking
policies can be optimised for specific task contexts.
Funding
This research did not receive any specific grant from funding
agencies in the public, commercial, or not-for-profit sectors.
Declaration of competing interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Acknowledgements
Johanna Westbrook is supported by a NHMRC Elizabeth Blackburn
Leadership Investigator Grant (1174021).
Appendix A. Supplementary data
Supplementary data to this article can be found online at https://doi.
org/10.1016/j.apergo.2022.103906.
References
Brown, D.W., 1963. The effect of observer redundancy on display monitoring equipment.
J. Psychol. 56, 413–419. https://doi.org/10.1080/00223980.1963.9916656.
Brown, D.W., Fox, G.H., 1965. The effect of observer redundancy and task difficulty on
display monitoring efficiency. J. Psychol. 59, 267–274. https://doi.org/10.1080/
00223980.1965.10544611.
Clay-Williams, R., Hounsgaard, J., Hollnagel, E., 2015. Where the rubber meets the road:
using FRAM to align work-as-imagined with work-as-done when implementing
clinical guidelines. Implement. Sci. 10 (1), 125. https://doi.org/10.1186/s13012-
015-0317-y.
Contte, J.M., Jacobs, R.R., 1997. Redundant systems influences on performance. Hum.
Perform. 10 (4), 361–380. https://doi.org/10.1207/s15327043hup1004_3.
Cymek, D.H., 2018. Redundant automation monitoring: four eyes don’t see more than
two, if everyone turns a blind eye. Hum. Factors 60 (7), 902–921. https://doi.org/
10.1177/0018720818781192.
Cymek, D.H., Jahn, S., Manzey, D.H., 2016. Monitoring and cross-checking automation:
do four eyes see more than two? Proc. Hum. Factors Ergon. Soc. Annu. Meet. 60 (1),
143–147. https://doi.org/10.1177/1541931213601033.
R.D. McMullan et al.
Dismukes, R.K., Berman, B., 2010. Checklists and Monitoring in the Cockpit: Why Crucial
Defenses Sometimes Fail. NASA/TM-2010–216396). Retrieved from. http://huma
n-factors.arc.nasa.gov/publications/NASA-TM-2010-216396.pdf.
Domeinski, J., Wagner, R., Schöbel, M., Manzey, D., 2007. Human redundancy in
automation monitoring: effects of social loafing and social compensation. Proc.
Hum. Factors Ergon. Soc. Annu. Meet. 51 (10), 587–591. https://doi.org/10.1177/
154193120705101004.
Hawker, S., Payne, S., Kerr, C., Hardey, M., Powell, J., 2002. Appraising the evidence:
reviewing disparate data systematically. Qual. Health Res. 12 (9), 1284–1299.
https://doi.org/10.1177/1049732302238251.
Hofmann, D.A., Jacobs, R., Landy, F., 1995. High reliability process industries:
individual, micro, and macro organizational influences on safety performance. J. Saf.
Res. 26 (3), 131–149. https://doi.org/10.1016/0022-4375(95)00011-E.
Hopewell, S., Clarke, M., Lefebvre, C., Scherer, R., 2007. Handsearching versus electronic
searching to identify reports of randomized trials. Cochrane Database Syst. Rev.
2007 (2), Mr000001 https://doi.org/10.1002/14651858.MR000001.pub2.
ISMP, 2003. The virtues of independent double checks—they really are worth your time.
Med. Safety Alert. 8 (5), 1. Retrieved from. https://pdfs.semanticscholar.org/d02a
/90a1d7f64fb931f4af14bd1ebcf1bfa22699.pdf.
ISMP, 2013. Independent Double Checks: Undervalued and Misused: Selective Use of
This Strategy Can Play an Important Role in Medication Safety. Retrieved from. https
://www.ismp.org/resources/independent-double-checks-undervalued-and-misuse
d-selective-use-strategy-can-play.
Keers, R.N., Williams, S.D., Cooke, J., Ashcroft, D.M., 2013. Prevalence and nature of
medication administration errors in health care settings: a systematic review of
direct observational evidence. Ann. Pharmacother. 47 (2), 237–256. https://doi.org/
10.1345/aph.1R147.
Koyama, A.K., Maddox, C.-S.S., Li, L., Bucknall, T., Westbrook, J.I., 2020. Effectiveness of
double checking to reduce medication administration errors: a systematic review.
BMJ Qual. Saf. 29 (7), 595–603. https://doi.org/10.1136/bmjqs-2019-009552.
Lamé, G., Dixon-Woods, M., 2020. Using clinical simulation to study how to improve
quality and safety in healthcare. BMJ Simul. Technol. Enhance Learn. 6 (2), 87–94.
https://doi.org/10.1136/bmjstel-2018-000370.
Lekka, C., 2011. High Reliability Organisations A Review of the Literature. Retrieved
from. https://www.hse.gov.uk/research/rrpdf/rr899.pdf.
Lyell, D., Magrabi, F., Raban, M.Z., Pont, L.G., Baysari, M.T., Day, R.O., Coiera, E., 2017.
Automation bias in electronic prescribing. BMC Med. Inf. Decis. Making 17 (1), 28.
https://doi.org/10.1186/s12911-017-0425-5.
Applied Ergonomics 106 (2023) 103906
Mosier, K., Skitka, L., 1996. Human decision makers and automated decision aids: made
for each other? In. Autmomate. Human Performance.: Theory Appl. 40, 201–220.
Mosier, K.L., Skitka, L.J., Dunbar, M., McDonnell, L., 2001. Aircrews and automation
bias: the advantages of teamwork? Int. J. Aviat. Psychol. 11 (1), 1–14. https://doi.
org/10.1207/S15327108IJAP1101_1.
Nihei, Y., Terashima, M., Suzuki, I., Morikawa, S., 2002. Why are four eyes better than
two? Effects of collaboration on the detection of errors in proofreading. Jpn. Psychol.
Res. 44 (3), 173–179. https://doi.org/10.1111/1468-5884.00020.
Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D.,
Moher, D., 2021. The PRISMA 2020 statement: an updated guideline for reporting
systematic reviews. BMJ 372, n71. https://doi.org/10.1136/bmj.n71.
Parasuraman, R., Riley, V., 1997. Humans and automation: use, misuse, disuse, abuse.
Hum. Factors 39 (2), 230–253. https://doi.org/10.1518/001872097778543886.
Pfeiffer, Y., Zimmermann, C., Schwappach, D.L.B., 2020. What are we doing when we
double check? BMJ Qual. Saf. 29 (7), 536–540. https://doi.org/10.1136/bmjqs-
2019-009680.
Riley, V., 1996. Operator reliance on automation: theory and data. In: Automation and
Human Performance: Theory and Applications. Lawrence Erlbaum Associates, Inc,
Hillsdale, NJ, US, pp. 19–35.
Roberts, K.H., 1990. Some characteristics of one type of high reliability organization.
Organ. Sci. 1 (2), 160–176. Retrieved from. http://www.jstor.org/stable/2635060.
Schwabe, L., Joëls, M., Roozendaal, B., Wolf, O.T., Oitzl, M.S., 2012. Stress effects on
memory: an update and integration. Neurosci. Biobehav. Rev. 36 (7), 1740–1749.
https://doi.org/10.1016/j.neubiorev.2011.07.002.
Skitka, L.J., Mosier, K.L., Burdick, M., Rosenblatt, B., 2000. Automation bias and errors:
are crews better than individuals? Int. J. Aerospace Psychol. 10 (1), 85–97. https://
doi.org/10.1207/s15327108ijap1001_5.
Toft, B., Mascie-Taylor, H., 2005. Involuntary automaticity: a work-system induced risk
to safe health care. Health Serv. Manag. Res. 18 (4), 211–216. https://doi.org/
10.1258/095148405774518615.
Westbrook, J.I., Li, L., Raban, M.Z., Woods, A., Koyama, A.K., Baysari, M.T., White, L.,
2020. Associations between Double-Checking and Medication Administration Errors:
a Direct Observational Study of Paediatric Inpatients. BMJ Quality & Safety. https://
doi.org/10.1136/bmjqs-2020-011473 bmjqs-2020-011473.
White, R.E., Trbovich, P.L., Easty, A.C., Savage, P., Trip, K., Hyland, S., 2010. Checking it
twice: an evaluation of checklists for detecting medication errors at the bedside
using a chemotherapy model. Qual. Health Care 19 (6), 562–567. https://doi.org/
10.1136/qshc.2009.032862.