You are on page 1of 10

ORIGINAL ARTICLE

Validation of a handoff assessment tool: the Handoff CEX


Leora I Horwitz, Janet Dombroski, Terrence E Murphy, Jeanne M Farnan, Julie K Johnson and Vineet
M Arora

Aims and objectives. Test the feasibility and validity of a handoff evaluation tool for nurses.
Background. No validated tools exist to assess the quality of handoff communication during change of shift.
Design. Prospective cohort study.
Methods. A standardised tool, the Handoff CEX, was developed based on the mini-CEX. The tool consisted of seven domains
scored on a 1–9 scale. Nurse educators observed shift-to-shift handoff reports among nurses and evaluated both the provider
and recipient of the report. Nurses participating in the report simultaneously evaluated each other as part of their handoff.
Results. Ninety-eight evaluations were obtained from 25 reports. Scores ranged from 3–9 in all domains except communication
and setting (4–9). Experienced (>five years) nurses received significantly higher mean scores than inexperienced (£five years)
nurses in all domains except setting and professionalism. Mean overall score for experienced nurses was 7Æ9 vs 6Æ9 for
inexperienced nurses. External observers gave significantly lower scores than peer evaluators in all domains except setting.
Mean overall score by external observers was 7Æ1 vs. 8Æ1 by peer evaluators. Participants were very satisfied with the evaluation
(mean score 8Æ1).
Conclusions. A brief, structured handoff evaluation tool was designed that was well-received by participants, was felt to be easy
to use without training, provided data about a wide range of communication competencies and discriminated well between
experienced and inexperienced clinicians.
Relevance to clinical practice. This tool may be useful for educators, supervisors and practicing nurses to provide training,
ongoing assessment and feedback to improve the quality of handoff.

Key words: communication, evaluation, handover, nurses, nursing, nursing education, transfer of care

Accepted for publication: 28 November 2011

(Cox 1994, Baldwin & Mcginnis 1994, Barbera et al. 1998)


Introduction
to bedside report where incoming nurses, off-going nurses
Nursing handoffs at shift changes vary widely in form, and patients mutually discuss the plan of care (Taylor 1993,
content and quality. They range in complexity from taped or Anderson & Mangino 2006). Some are standardised using
written reports left by off-going nurses for incoming nurses one of a variety of templates (Schroeder 2006, Haig et al.

Authors: Leora I Horwitz, MD, MHS, Assistant Professor, Center Johnson, PhD, Associate Professor, Centre for Clinical Governance
for Outcomes Research and Evaluation, Yale-New Haven Hospital, Research, Faculty of Medicine, University of New South Wales,
and Section of General Internal Medicine, Department of Medicine, Sydney, Australia; Vineet M Arora, MD, MAPP, Associate
Yale University School of Medicine, New Haven, CT; Janet Professor, Section of Hospital Medicine, Section of General
Dombroski, PhD, RN, Clinical Nurse Educator, Center for Internal Medicine, Department of Medicine, The University of
Professional Practice Excellence, Yale-New Haven Hospital, New Chicago, Chicago, IL, USA
Haven, CT; Terrence E Murphy, PhD, Research Scientist, Section of Correspondence: Leora Horwitz, Assistant Professor, Section of
Geriatrics, Department of Medicine, Yale University School of General Internal Medicine, PO Box 208093, New Haven, CT
Medicine, New Haven, CT; Jeanne M Farnan, MD, MHPE, 06520-8093, USA. Telephone: +1 (203) 688 5678.
Assistant Professor, Section of Hospital Medicine, Department of E-mail: leora.horwitz@yale.edu
Medicine, The University of Chicago, Chicago, IL, USA; Julie K

 2012 Blackwell Publishing Ltd


Journal of Clinical Nursing, doi: 10.1111/j.1365-2702.2012.04131.x 1
LI Horwitz et al.

2006, No authors listed 2007, Wilson 2007, Block et al. this important skill. It also makes it difficult to assess the impact
2010), most are not. Studies of nursing handoffs have and sustainability of interventions to improve the handoff
identified a variety of problems, including incomplete or process.
inaccurate information, uneven quality, limited opportunities To address this need, we developed a structured handoff
for questions, incorrect judgments and repeated interruptions assessment tool, the Handoff CEX (Farnan et al. 2010),
(Clair & Trussell 1969, Riesenberg et al. 2010, Welsh et al. based on a previously validated educational assessment, the
2010, Calleja et al. 2011). In turn, these may contribute to mini-CEX (Norcini et al. 1995, 2003). The mini-CEX uses a
error through omissions, misunderstandings and delays 9-point scale in several domains and is widely used to
(Anthony & Preuss 2002, Ebright et al. 2004, Sexton et al. evaluate students and trainees. This study was designed to
2004, Pothier et al. 2005, Sharit et al. 2008). Similar test the feasibility and discriminatory power of the Handoff
problems have been noted with handoffs between physicians CEX in real-world practice settings among hospital nurses.
and other providers (Beach et al. 2003, Arora et al. 2005,
2007, Gandhi 2005, Jagsi et al. 2005, Greenberg et al. 2007,
Methods
Borowitz et al. 2008, Horwitz et al. 2008, Kitch et al. 2008).
On the other hand, a well-conducted handoff serves as an
Tool design
opportunity for critical reassessment and error reduction (Lee
et al. 1996, Miller 1998, Lally 1999, Parker & Coiera 2000, Based on expert opinion, clinical guidelines and published
Kerr 2002, Patterson et al. 2004, Paine & Millman 2009, literature, we identified six main domains for handoff
Salerno et al. 2009). Systematic overhauls of nurse handovers assessment: setting, organisation, communication, content,
have been described to reduce adverse events (Alvarado et al. judgment and professionalism. In addition, we added an
2006). Furthermore, nursing handoffs serve important roles assessment of overall competency. We based the format
in acculturation, socialisation and education (Parker et al. and structure of the tool on a previously validated, widely
1992, Ekman & Segesten 1995, Lally 1999, Hays 2002 ). used, real-time educational evaluation tool (the Mini-CEX)
For both these reasons, the World Health Organization (Norcini et al. 1995). Each domain was scored on a 1–9
(WHO Collaborating centre for patient safety solutions point scale and included descriptive anchors at high and
2007) and organisations in many nations, including the low ends of performance to orient the evaluator. The scale
USA (The Joint Commission 2009, Accreditation Council was divided into unsatisfactory (score 1–3), satisfactory (4–
for Graduate Medical Education 2010), UK (British Med- 6) and superior (7–9) sections to further guide the
ical Association, National Patient Safety Agency & NHS evaluator. We designed two tools, one to assess the person
Modernisation Agency 2004) and Australia (Australian providing the handoff and one to assess the handoff
Medical Association 2006, Australian Commission On recipient, each with unique role-based anchors (Figs 1 and
Safety and Quality In Health Care 2010), have focused 2). The recipient evaluation tool did not include a domain
increasing attention on the handoff as a key component of for content.
patient safety. In the USA, standardised handovers are an
accreditation requirement for hospitals (The Joint Com-
Feasibility assessment
mission 2009), and competency in handoff skills is a require-
ment for physicians in training (Accreditation Council for We selected a convenience sampling of routine shift-to-shift
Graduate Medical Education 2010). Likewise, there have been nurse reports both in the morning and the evening on three
widespread calls for standardisation of nursing handovers units (one medicine, one surgical and one cardiovascular) to
(Joint Commission on Accreditation of Healthcare 2005, ensure a range of patient types and nurse experience. Each
Hohenhaus et al. 2006, Riesenberg et al. 2010). Standards for nurse report was observed by an experienced nurse evalu-
evaluation of nursing handoffs, however, have not been ator (either a nurse educator or clinical nurse manager) who
established. had received only a brief (<five minutes) overview of the
To date, there are no established tools for assessing the tool by the study coordinator. For the nurse providing
quality of the verbal handoff, also referred to as ‘sign-out’ report, an evaluation tool was completed by both the nurse
or ‘report (Riesenberg et al. 2009), nor are there tools to evaluator and the nurse receiving report. For the nurse
assess the competency of the handoff participants (Riesen- receiving report, an evaluation tool was completed by the
berg et al. 2009). The lack of validated assessment tools nurse evaluator and by the nurse providing report. Conse-
makes it challenging for hospitals to ensure that their clinical quently, each report included in the study generated four
providers, including nurses and physicians, are competent in evaluations: two of the nurse providing report and two of

 2012 Blackwell Publishing Ltd


2 Journal of Clinical Nursing
Original article Handoff CEX validation

Figure 1 Handoff provider assessment tool.

the nurse receiving report. No training in use of the tool was Statistical analysis
provided to peer evaluators. Feedback of the results of the
We obtained the median and range of scores for each
evaluation was given to each nurse in real time by the
domain. We stratified nurses by years of experience (£five
evaluators; to do this, evaluators were instructed to review
years and >five years) and used the Student’s t-test
the scores on each domain with the nurse with explanations
to compare the effect of participant experience on assess-
for low scores. The tool also included space for open-ended
ment scores. We confirmed the results using the nonpara-
comments about the report or the tool.
metric Wilcoxon test; as the results were the same, we
Each nurse provided verbal informed consent. The study
report the t-test results. We used Spearman’s correlation
was approved by the Yale University Human Investigation
coefficients to describe correlation between domains.
Committee (HIC) and by the Yale New Haven Hospital
We used paired t-tests to compare external evaluator
Research Committee. The HIC granted a HIPAA waiver to
ratings with peer ratings of the same handoff. Finally,
cover patient information discussed during report and a
we tested the inter-rater reliability of the tool by calcu-
waiver of informed patient consent. No patient information
lating a weighted kappa. We described open-ended com-
was recorded during the study.
ments in a narrative fashion as there were too few

 2012 Blackwell Publishing Ltd


Journal of Clinical Nursing 3
LI Horwitz et al.

Figure 2 Handoff recipient assessment tool.

comments to conduct a formal qualitative analysis. Statis- Handoff providers


tical significance was defined by a p value £ 0Æ05, and
A total of 49 evaluations of handoff providers were
analyses were performed using SAS 9.2 (SAS Institute, Cary,
completed for 25 nurses. For each domain except communi-
NC, USA).
cation and setting, scores spanned the full range from
unsatisfactory to superior (Table 1). The highest rated
Results variable on the handoff provider evaluation tool was profes-
sionalism, with a mean of 7Æ7 (SD 1Æ4). The lowest rating was
A total of 25 shift-to-shift nurse reports were observed
for setting, with a mean of 7Æ1 (SD 1Æ4). Handoff providers
between October, 2007 and June, 2008, yielding a total of 98
gave high ratings for their satisfaction with the evaluation, at
evaluations. Participants reported spending a mean of 7Æ3 (SD
a mean of 8Æ1 (SD 1Æ4).
4Æ5) minutes observing report and 2Æ0 (SD 1Æ2) minutes
providing feedback. The evaluators rated their satisfaction
with the tool highly, at a mean of 8Æ2 (0Æ9). Overall, nurses Handoff recipients
received high marks for reports, but there was a wide range of
A total of 49 evaluations of handoff recipients were
scores for both the provider (giving the handoff) and recipient
completed for 25 nurses. The range of scores was narrower
(receiving the handoff).

 2012 Blackwell Publishing Ltd


4 Journal of Clinical Nursing
Original article Handoff CEX validation

Table 1 Mean and range of scores in each domain

Provider of handoff (n = 49) Recipient of handoff (n = 49)

Weighted kappa Weighted kappa


Domain Mean (SD) Range (Adjusted SE) Mean (SD) Range (Adjusted SE)

Setting 7Æ1 (1Æ4) 4–9 0Æ30 (0Æ15) 7Æ1 (1Æ4) 4–9 0Æ36 (0Æ16)
Organisation 7Æ4 (1Æ6) 3–9 0Æ48 (0Æ14) 7Æ6 (1Æ2) 5–9 0Æ30 (0Æ15)
Communication 7Æ4 (1Æ5) 4–9 0Æ29 (0Æ13) 7Æ6 (1Æ3) 4–9 0Æ44 (0Æ14)
Content 7Æ2 (1Æ6) 3–9 0Æ46 (0Æ16) N/A N/A
Judgment 7Æ4 (1Æ6) 3–9 0Æ46 (0Æ13) 7Æ5 (1Æ1) 4–9 0Æ35 (0Æ16)
Professionalism 7Æ7 (1Æ4) 3–9 0Æ39 (0Æ18) 7Æ5 (1Æ6) 1–9 0Æ48 (0Æ13)
Overall 7Æ4 (1Æ5) 3–9 0Æ43 (0Æ13) 7Æ6 (1Æ1) 4–9 0Æ41 (0Æ16)
Satisfaction with evaluation 8Æ1 (1Æ4) 4–9 8Æ2 (1Æ2) 4–9

SD, standard deviation; SE, standard error; N/A, not applicable – content is not a domain on the recipient evaluation tool.

than for the provider assessments, spanning the satisfactory Table 2 Mean and range of scores for providers of handoff, stratified
to superior ranges (Table 1). For the handoff recipient by years of experience
evaluation tool, both organisation and communication >five years
scored the highest with a mean of 7Æ7 (SD 1Æ2 and SD 1Æ3, £five years expe- experience
respectively). The lowest score was also for setting, at 7Æ1 rience (n = 22) (n = 23)
(SD 1Æ4). The overall quality of recipients’ report perfor- Mean Mean
mance was assessed at a mean of 7Æ6 (SD 1Æ1) compared with Domain (SD) Range (SD) Range p-value*
7Æ4 (SD 1Æ5) for providers’ performance. Handoff recipients
Setting 6Æ7 (1Æ4) 4–9 7Æ4 (1Æ5) 4–9 0Æ11
gave high ratings for their satisfaction with the evaluation, at Organisation 6Æ7 (1Æ8) 3–9 8Æ3 (0Æ8) 6–9 <0Æ001
a mean of 8Æ2 (SD 1Æ2). Communication 6Æ7 (1Æ8) 4–9 8Æ2 (1) 6–9 0Æ001
Content 6Æ4 (1Æ9) 3–9 7Æ9 (1) 6–9 0Æ003
Judgment 6Æ7 (1Æ9) 3–9 8Æ1 (1) 6–9 0Æ005
Subgroup analyses Professionalism 7Æ8 (1Æ4) 3–9 7Æ8 (1Æ5) 3–9 0Æ98
Overall 6Æ9 (1Æ8) 3–9 7Æ9 (1Æ2) 6–9 0Æ03
Evaluations were evenly divided among nurses with >five
Satisfaction with 7Æ2 (1Æ8) 4–9 8Æ8 (0Æ4) 8–9 0Æ007
years of experience (n = 23) and those with £five years of evaluation
experience (n = 22). Experienced (>five years) nurses re-
SD, Standard deviation.
ceived significantly higher scores than inexperienced (£five
*p-values based on t-statistics for independent samples.
years) nurses in all domains except setting and professional-
ism (Table 2). For example, experienced nurses received a
mean of 7Æ9 for overall competency, compared with 6Æ9 for the other domains, with correlation coefficients ranging from
inexperienced nurses (mean difference 1Æ0 points, 95% CI 0Æ24–0Æ40. Correlations between individual domains and the
0Æ2–1Æ9, p = 0Æ03). overall competence rating ranged from 0Æ78–0Æ92 for all
External evaluators consistently gave lower marks for domains excluding setting (p < 0Æ001) and was 0Æ40 for
the same report than the peer evaluators did, with the setting (p = 0Æ004).
exception of the setting domain, which was similar in Weighted kappa scores for provider evaluations ranged
both (Table 3). For example, external evaluators gave sub- from 0Æ29–0Æ48, generally considered in the fair–moderate
jects an average score of 7Æ1 for overall quality, whereas peer range (Table 1) (Altman 1991). Weighted kappa scores for
evaluators gave subjects an average score of 8Æ1 (mean recipient evaluations similarly ranged from 0Æ30–0Æ48.
difference 1Æ1 points, 95% CI 0Æ5–1Æ6 points, p < 0Æ001).

Open-ended comments
Inter-rater reliability and domain correlation
Twenty of the evaluations included open-ended comments.
Excluding setting, Spearman’s correlation coefficients among A few were comments about the tool itself (‘Very helpful to
the CEX domains ranged from 0Æ53–0Æ95 (p < 0Æ001 for get started with’, ‘Clear explanation and feedback on each
most correlations). Setting was more weakly correlated with area of evaluation’). However, most were evaluative

 2012 Blackwell Publishing Ltd


Journal of Clinical Nursing 5
LI Horwitz et al.

Table 3 Mean and range of scores for providers and recipients of handoff, stratified by external versus peer evaluator

External observers
(n = 34) Peer evaluators (n = 34)
Mean difference peer –
Domain Mean (SD) Range Mean (SD) Range external score (95% CI) p-value*

Setting 7Æ3 (1Æ4) 4–9 7Æ4 (1Æ4) 4–9 0Æ0 ( 0Æ6 to 0Æ6) 0Æ92
Organisation 6Æ9 (1Æ5) 3–9 7Æ9 (1Æ2) 5–9 1Æ0 (0Æ5 to 1Æ5) <0Æ001
Communication 7Æ0 (1Æ5) 4–9 7Æ9 (1Æ3) 5–9 0Æ9 (0Æ4 to 1Æ4) <0Æ001
Content 6Æ5 (2Æ0) 3–9 7Æ9 (1Æ2) 5–9 1Æ5 (0Æ2 to 2Æ8) 0Æ03
Judgment 6Æ9 (1Æ7) 3–9 8Æ0 (1Æ1) 5–9 1Æ1 (0Æ5 to 1Æ6) <0Æ001
Professionalism 7Æ4 (1Æ2) 3–9 8Æ1 (1Æ3) 3–9 0Æ7 (0Æ3 to 1Æ2) 0Æ004
Overall 7Æ1 (1Æ6) 3–9 8Æ1 (0Æ9) 6–9 1Æ1 (0Æ5 to 1Æ6) <0Æ001

SD, standard deviation; CI, confidence interval.


*p-values based on t-statistics for paired samples.

comments for the participants. Nurses recorded both construct validity and inter-rater reliability of a new evalu-
praise (‘Well-received report/Excellent questions’) and con- ation tool, the Handoff CEX. The tool is designed to be
structive criticism. Negative feedback was provided both to independent of clinical setting and to be used either for nurses
experienced nurses (‘As seasoned nurse should have had a or physicians.
few more clinical questions’) and to inexperienced nurses In this study, handoff evaluations were conducted both by
(‘Scattered report’). Of note, comments captured aspects of external observers – experienced nurse educators or nurse
professionalism (‘Nurse seemed anxious to finish, had to go managers – and by the handoff participants. As is common
home’) in addition to feedback about the content and with evaluation tools, we noted a clustering of scores towards
organisation of report. the higher end of the score range. In an a priori effort to keep the
The comments highlighted the utility of a structured score range wide, we provided descriptive anchors for high and
handoff evaluation in assessing both individual skills and low scores as part of the tool. One approach to increase the
system adequacies. For instance, in this comment, the spread of scores might be to add descriptive anchors to the
evaluator not only noted weaknesses of an inexperienced middle of the range, to help evaluators distinguish satisfactory
nurse, but identified a broader system failure in terms of lack from exemplary performance (French-Lazovik & Gibson
of supervision: 1984, Weng 2004). We will explore this possibility in future
studies. Another means of increasing the spread of scores
Left out a few clinical items. Did not articulate plan of care
would be to formally train users in use of the tool, perhaps by
surrounding a couple of clinical issues. *RN is still on orientation.
having them view standardised videos of handoff encounters
Preceptor did not listen to report.
(Holmboe et al. 2003, 2004). However, videos are cumber-
Finally, the comments illustrated both the potential for some and useful primarily in a research context, while other
error and the potential for error-capture of the handoff educational training sessions have not been found to be
activity: effective (Cook et al. 2009).
Although scores were generally high, we found that the
Overall poor report. Left out major pieces of information. Not up to
external observer scores were consistently lower than peer
date on orders – including DNR/DNI not in as it should be – recipient
evaluations. Similarly, other studies have found peers to be
picked this up.
more lenient than faculty (Hay 1995, Rudy et al. 2001) or
that peer evaluations may differ in their approach from
Discussion
faculty ratings (Kegel-Flom 1975, Risucci et al. 1989). In
As increasing attention is being paid to communication skills fact, although several peer evaluation tools exist for physi-
and handoff competencies, the need for tools to evaluate cians, concern has been raised about their validity (Norcini
handoff skills is growing. A handoff evaluation tool is 2003, Evans et al. 2004). In this study, we postulate several
necessary for assessing staff competency, testing the effect of potential reasons for the differences between faculty and peer
handoff improvements, determining sustainability of inter- reviews in addition to the possibility that peers are influenced
ventions and identifying systematic barriers and gaps in the by their personal relationships with the evaluatees. First, all
handoff process. However, tools should be validated prior to external observers were highly experienced clinicians and
widespread use. This validation study was designed to assess may have been better able to discriminate between high- and

 2012 Blackwell Publishing Ltd


6 Journal of Clinical Nursing
Original article Handoff CEX validation

low-quality handoffs than participants, most of whom had house staff and hospitalist physicians. Finally, this study
less experience. Second, as nurse educators, they are also was not designed to assess test–retest reliability: the
trained in evaluation techniques apart from this tool and likelihood the same observer would give the same report
may therefore have been better primed to provide a range the same score on two separate occasions. These will be
of scores. Third, their sole job was to evaluate the necessary follow-up activities to fully validate the tool for
handoff, as opposed to participants, who had to concen- widespread use.
trate on the actual handoff as well as to consider it criti-
cally from a quality perspective. Thus, although it would be
Conclusion
feasible to use this tool solely in a peer evaluation context, it
will likely prove to be preferable to be completed by an external We designed a brief, structured report evaluation tool that
observer. was well received by participants, was felt to be easy to use
We found a high degree of correlation between individual without training, provided data about a wide range of
domains of the handoff CEX except setting. Very similar communication competencies and discriminated well be-
results were found in the validation of the mini-CEX on which tween experienced and inexperienced clinicians. The tool also
this tool is based (Norcini et al. 1995, 2003). This may be due provided an opportunity for evaluators to identify systems
to an inability of evaluators to distinguish among domains, a failures impeding the handoff process.
‘halo effect’ where high competence in one dimension spills
over into scores given for other dimensions and/or intrinsic
Relevance to clinical practice
correlation of these communication skills. Regardless, as this
tool is intended both as an evaluation method and as a means of The Handoff CEX may prove useful for healthcare organi-
continuing education (by specifying and reinforcing compo- sations seeking to measure and improve the quality of
nents of good communication), we elected to retain all domains handoff communication. In addition, it may be used by
in the final tool. The weighted kappa for individual domains nurse educators to frame initial training in handoff skills and
was fair to moderate, as would be expected from a single by nurse managers to conduct ongoing assessment and
observation by a wide variety of types of observers with no feedback of handoff skills among practicing nurses.
specific training in the tool. Similar scores have been found
in studies of the mini-CEX (Norcini et al. 1995, 2003,
Acknowledgements
Cook & Beckman 2009, Cook et al. 2009) and other
evaluation tools (Kogan et al. 2009). For this reason, we Development and evaluation of the sign-out CEX are
do not recommend single use of the Handoff CEX. As supported by a grant from the Agency for Healthcare
noted in studies of the mini-CEX, repeated observations Research and Quality (1R03HS018278-01). At the time
generate more reliable data (Norcini et al. 1995, 2003). this study was conducted, Dr. Horwitz was supported by
Given the ease and brevity of this evaluation (seven the CTSA Grant UL1 RR024139 and KL2 RR024138 from
minutes per evaluation), it would be feasible to obtain the National Center for Advancing Translational Sciences
multiple observations of the same provider over time for a (NCATS), a component of the National Institutes of Health
more reliable assessment of competency. In addition, we (NIH) and NIH roadmap for Medical Research. Dr.
expect that the kappa score was reduced because we Horwitz is currently supported by the National Institute
compared peer evaluators with experienced nurse educators on Aging (K08 AG038336) and by the American Federa-
and peer evaluators provided systematically higher scores. tion for Aging Research (AFAR) through the Paul B.
Our findings suggest that in future a consistent type of Beeson Career Development Award Program. Dr. Horwitz
evaluator should be employed (Borman 1974). is also a Pepper Scholar with support from the Claude D.
Our study had several limitations. There is no ‘gold Pepper Older Americans Independence Center at Yale
standard’ of handoff quality so we could not determine University School of Medicine (P30AG021342 NIH/NIA).
whether, for instance, external evaluators were systemati- No funding source had any role in the study design; in the
cally over-harsh or peer evaluators over-lenient. We did collection, analysis and interpretation of data; in the
not correlate scores on the handoff CEX to actual writing of the report; or in the decision to submit
clinical outcomes such as problems with the handover. the article for publication. The content is solely the
This study was conducted only on nurses. However, we responsibility of the authors and does not necessarily
have successfully used the tool for medical students represent the official views of the NIA, the NIH, the
(Farnan et al. 2010) and are currently studying its use in NCATS, the AHRQ or AFAR.

 2012 Blackwell Publishing Ltd


Journal of Clinical Nursing 7
LI Horwitz et al.

Author contributions Conflict of interest


Study design: LIH, JD, JMF, JKJ, VMA; data collection and The authors have no conflicts of interest to disclose.
analysis: LIH, JD, TEM and manuscript preparation: LIH,
JD, TEM, JMF, JFJ, VMA.

References
No authors listed (2007) Pass the baton or Baldwin L & Mcginnis C (1994) A com- Education Theory and Practice 14,
NUTS for safer handoff. Healthcare puter-generated shift report. Nursing 655–664.
Risk Management 29, 115–117. Management 25, 61–64. Cook DA, Dupras DM, Beckman TJ, Tho-
Accreditation Council for Graduate Medical Barbera ML, Conley R & Postell M (1998) mas KG & Pankratz VS (2009) Effect
Education (2010) Proposed Standards A silent report. Nursing Management of rater training on reliability and
for Common Program Requirements. 29, 66–67. accuracy of mini-CEX scores: a ran-
Available: http://acgme-2010standards. Beach C, Croskerry P & Shapiro M (2003) domized, controlled trial. Journal of
org/pdf/Proposed_Standards.pdf (ac- Profiles in patient safety: emergency General Internal Medicine 24, 74–79.
cessed 28 August 2010). care transitions. Academic Emergency Cox SS (1994) Taping report: tips to record
Altman DG (1991) Practical Statistics for Medicine 10, 364–367. by. Nursing 24, 64.
Medical Research. Chapman and Hall, Block M, Ehrenworth JF, Cuce VM, Ebright PR, Urden L, Patterson E & Chalko
London. Ng’ang’a N, Weinbach J, Saber SB, B (2004) Themes surrounding novice
Alvarado K, Lee R, Christoffersen E, Fram Milic M, Urgo JA, Sokoli D & Schle- nurse near-miss and adverse-event situ-
N, Boblin S, Poole N, Lucas J & singer MD (2010) The tangible hand- ations. Journal of Nursing Administra-
Forsyth S (2006) Transfer of account- off: a team approach for advancing tion 34, 531–538.
ability: transforming shift handover to structured communication in labor and Ekman I & Segesten K (1995) Deputed
enhance patient safety. Healthcare delivery. Joint Commission Journal on power of medical control: the hidden
Quarterly 9, 75–79. Quality and Patient Safety 36, 282– message in the ritual of oral shift re-
Anderson CD & Mangino RR (2006) Nurse 287, 241. ports. Journal of Advanced Nursing 22,
shift report: who says you can’t talk in Borman WC (1974) The rating of individu- 1006–1011.
front of the patient? Nursing Adminis- als in organizations: an alternate ap- Evans R, Elwyn G & Edwards A (2004)
tration Quarterly 30, 112–122. proach. Organizational Behavior and Review of instruments for peer assess-
Anthony MK & Preuss G (2002) Models of Human Performance 12, 105–124. ment of physicians. British Medical
care: the influence of nurse communi- Borowitz SM, Waggoner-Fountain LA, Bass Journal 328, 1240.
cation on patient safety. Nursing Eco- EJ & Sledd RM (2008) Adequacy of Farnan JM, Paro JA, Rodriguez RM, Reddy
nomics 20, 209–215, 248. information transferred at resident sign- ST, Horwitz LI, Johnson JK & Arora
Arora V, Johnson J, Lovinger D, Humphrey out (in-hospital handover of care): a VM (2010) Hand-off education and
HJ & Meltzer DO (2005) Communi- prospective survey. Quality and Safety evaluation: piloting the observed simu-
cation failures in patient sign-out and in Health Care 17, 6–10. lated hand-off experience (OSHE).
suggestions for improvement: a critical British Medical Association, National Pa- Journal of General Internal Medicine
incident analysis. Quality and Safety in tient Safety Agency & NHS Moderni- 25, 129–134.
Health Care 14, 401–407. sation Agency (2004) Safe Handover: French-Lazovik G & Gibson CL (1984)
Arora V, Kao J, Lovinger D, Seiden SC & Safe Patients. British Medical Associa- Effects of verbally labeled anchor
Meltzer D (2007) Medication dis- tion, London. points on the distributional parameters
crepancies in resident sign-outs and Calleja P, Aitken LM & Cooke ML (2011) of rating measures. Applied Psycho-
their potential to harm. Journal of Information transfer for multi-trauma logical Measurement 8, 49–57.
General Internal Medicine 22, 1751– patients on discharge from the emer- Gandhi TK (2005) Fumbled handoffs: one
1755. gency department: mixed-method nar- dropped ball after another. Annals of
Australian Commission On Safety and rative review. Journal of Advanced Internal Medicine 142, 352–358.
Quality In Health Care (2010) OSSIE Nursing 67, 4–18. Greenberg CC, Regenbogen SE, Studdert
Guide to Clinical Handover Improv- Clair LL & Trussell PM (1969) Nursing DM, Lipsitz SR, Rogers SO, Zinner MJ
ment. Available: http://www.safetyand service. The change of shift report: & Gawande AA (2007) Patterns of
quality.gov.au/internet/safety/publishing. study shows weaknesses, how it can be communication breakdowns resulting
nsf/Content/F72758677DAFA15ECA improved. Hospitals 43, 91–95. in injury to surgical patients. Journal of
2577530017CFC5/$File/ossie.pdf (accessed Cook DA & Beckman TJ (2009) Does scale the American College of Surgeons 204,
15 October 2010). length matter? A comparison of nine- 533–540.
Australian Medical Association (2006). Safe versus five-point rating scales for the Haig KM, Sutton S & Whittington J (2006)
Handover: Safe Patients. Australian mini-CEX. Advances in Health Sciences SBAR: a shared mental model for
Medical Association, Kingston, ACT. improving communication between cli-

 2012 Blackwell Publishing Ltd


8 Journal of Clinical Nursing
Original article Handoff CEX validation

nicians. Joint Commission Journal on on Quality and Patient Safety 34, 563– mnemonics literature. American Jour-
Quality and Patient Safety 32, 167– 570. nal of Medical Quality 24, 196–204.
175. Kogan JR, Holmboe ES & Hauer KE (2009) Riesenberg LA, Leisch J & Cunningham JM
Hay JA (1995) Tutorial reports and ratings. Tools for direct observation and (2010) Nursing handoffs: a systematic
In Evaluation Methods: A Resource assessment of clinical skills of medical review of the literature. American
Handbook (Shannon S & Nocterm G trainees: a systematic review. Journal of Journal of Nursing 110, 24–34.
eds). McMaster University, Hamilton, the American Medical Association 302, Risucci DA, Tortolani AJ & Ward RJ (1989)
ON. 1316–1326. Ratings of surgical residents by self,
Hays MM (2002) An exploratory study of Lally S (1999) An investigation into the supervisors and peers. Surgical Gyne-
supportive communication during shift functions of nurses’ communication at cology and Obstetrics 169, 519–526.
report. Southern Online Journal of the inter-shift handover. Journal of Rudy DW, Fejfar MC, Griffith CH 3rd &
Nursing Research 3(3). Nursing Management 7, 29–36. Wilson JF (2001) Self- and peer assess-
Hohenhaus S, Powell S & Hohenhaus JT Lee LH, Levine JA & Schultz HJ (1996) ment in a first-year communication and
(2006) Enhancing patient safety during Utility of a standardized sign-out card interviewing course. Evaluation and the
hand-offs: standardized communication for new medical interns. Journal of Health Professions 24, 436–445.
and teamwork using the ‘SBAR’ meth- General Internal Medicine 11, 753–755. Salerno SM, Arnett MV & Domanski JP
od. American Journal of Nursing 106, Miller C (1998) Ensuring continuing care: (2009) Standardized sign-out reduces
72A–72B. styles and efficiency of the handover intern perception of medical errors on the
Holmboe ES, Huot S, Chung J, Norcini J & process. Australian Journal of Ad- generalinternalmedicineward.Teaching
Hawkins RE (2003) Construct validity vanced Nursing 16, 23–27. and Learning in Medicine 21, 121–126.
of the miniclinical evaluation exercise Norcini JJ (2003) Peer assessment of compe- Schroeder SJ (2006) Picking up the PACE: a
(miniCEX). Academic Medicine 78, tence. Medical Education 37, 539–543. new template for shift report. Nursing
826–830. Norcini JJ, Blank LL, Arnold GK & Kimball 36, 22–23.
Holmboe ES, Hawkins RE & Huot SJ (2004) HR (1995) The mini-CEX (clinical Sexton A, Chan C, Elliott M, Stuart J, Jay-
Effects of training in direct observation evaluation exercise): a preliminary asuriya R & Crookes P (2004) Nursing
of medical residents’ clinical compe- investigation. Annals of Internal Medi- handovers: do we really need them? Jour-
tence: a randomized trial. Annals of cine 123, 795–799. nal of Nursing Management 12, 37–42.
Internal Medicine 140, 874–881. Norcini JJ, Blank LL, Duffy FD & Fortna Sharit J, Mccane L, Thevenin DM & Barach
Horwitz LI, Moin T, Krumholz HM, Wang GS (2003) The mini-CEX: a method for P (2008) Examining links between sign-
L & Bradley EH (2008) Consequences assessing clinical skills. Annals of out reporting during shift changeovers
of inadequate sign-out for patient care. Internal Medicine 138, 476–481. and patient management risks. Risk
Archives of Internal Medicine 168, Paine LA & Millman A (2009) Sealing Analysis 28, 969–981.
1755–1760. the cracks, not falling through: using Taylor C (1993) Inter-shift report: oral
Jagsi R, Kitch BT, Weinstein DF, Campbell handoffs to improve patient care. Fron- communication using a quality assur-
EG, Hutter M & Weissman JS (2005) tiers of Health Services Management 25, ance approach. Journal of Clinical
Residents report on adverse events and 33–38. Nursing 2, 266–267.
their causes. Archives of Internal Med- Parker J & Coiera E (2000) Improving clinical The Joint Commission (2009) 2010 Hospi-
icine 165, 2607–2613. communication: a view from psychol- tal Accreditation Standards. Joint
Joint Commission on Accreditation of ogy. Journal of the American Medical Commission Resources, Oakbrook
Healthcare (2005) Focus on five. Informatics Association 7, 453–461. Terrace, IL.
Strategies to improve hand-off com- Parker J, Gardner G & Wiltshire J (1992) Welsh CA, Flanagan ME & Ebright P (2010)
munication: implementing a process to Handover: the collective narrative of Barriers and facilitators to nursing
resolve questions. Joint Commission nursing practice. Australian Journal of handoffs: recommendations for rede-
Perspectives on Patient Safety 5, 11. Advanced Nursing 9, 31–37. sign. Nursing Outlook 58, 148–154.
Kegel-Flom P (1975) Predicting supervisor, Patterson ES, Roth EM, Woods DD, Chow Weng L-J (2004) Impact of the number of
peer and self ratings of intern perfor- R & Gomes JO (2004) Handoff response categories and anchor labels
mance. Journal of Medical Education strategies in settings with high conse- on coefficient alpha and test-retest
50, 812–815. quences for failure: lessons for health reliability. Educational and Psycholog-
Kerr MP (2002) A qualitative study of care operations. International Journal ical Measurement 64, 956–972.
shift handover practice and function for Quality in Health Care 16, 125– WHO Collaborating centre for patient
from a socio-technical perspective. 132. safety solutions (2007) Communication
Journal of Advanced Nursing 37, Pothier D, Monteiro P, Mooktiar M & during patient hand-overs. Patient
125–134. Shaw A (2005) Pilot study to show the Safety Solutions 1, 1–4.
Kitch BT, Cooper JB, Zapol WM, Marder loss of important data in nursing Wilson MJ (2007) A template for safe and
JE, Karson A, Hutter M & Campbell handover. British Journal of Nursing concise handovers. Medsurg Nursing:
EG (2008) Handoffs causing patient 14, 1090–1093. Official Journal of the Academy of
harm: a survey of medical and surgical Riesenberg LA, Leitzsch J & Little BW Medical-Surgical Nurses 16, 201–206,
house staff. Joint Commission Journal (2009) Systematic review of handoff 200.

 2012 Blackwell Publishing Ltd


Journal of Clinical Nursing 9
LI Horwitz et al.

The Journal of Clinical Nursing (JCN) is an international, peer reviewed journal that aims to promote a high standard of
clinically related scholarship which supports the practice and discipline of nursing.

For further information and full author guidelines, please visit JCN on the Wiley Online Library website: http://
wileyonlinelibrary.com/journal/jocn

Reasons to submit your paper to JCN:


High-impact forum: one of the world’s most cited nursing journals and with an impact factor of 1Æ228 – ranked 23 of 85
within Thomson Reuters Journal Citation Report (Social Science – Nursing) in 2009.
One of the most read nursing journals in the world: over 1 million articles downloaded online per year and accessible in over
7000 libraries worldwide (including over 4000 in developing countries with free or low cost access).
Fast and easy online submission: online submission at http://mc.manuscriptcentral.com/jcnur.
Early View: rapid online publication (with doi for referencing) for accepted articles in final form, and fully citable.
Positive publishing experience: rapid double-blind peer review with constructive feedback.
Online Open: the option to make your article freely and openly accessible to non-subscribers upon publication in Wiley
Online Library, as well as the option to deposit the article in your preferred archive.

 2012 Blackwell Publishing Ltd


10 Journal of Clinical Nursing