Editorial
Do As We Say, Not As You Do: Using
Downloaded from http://journals.lww.com/simulationinhealthcare by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgb
sIHo4XMi0hCywCX1AWnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023
David M. Gaba, MD
From the Patient Simulation Center of Innovation, VA
Palo Alto Health Care System, Palo Alto, California; and
Center for Immersive and Simulation-based learning,
Stanford University School of Medicine, Stanford,
California.
Reprints: David M. Gaba, MD, Anesthesia Service,
112A VA Palo Alto Health Care System, 3801
Miranda Avenue, Palo Alto, CA 94304 (e-mail:
gaba@stanford.edu).
The author has no conflict of interest to disclose.
Copyright © 2009 Society for Simulation in Healthcare
DOI: 10.1097/SIH.0b013e3181a4a412
Vol. 4, No. 2, Summer 2009
Simulation to Investigate Clinical Behavior
in Action
W hen mannequin-based simulation was first introduced into regular teaching of
clinicians in the late 1980s and early 1990s, a common perception was that the
technology would be good only for early learners, for learning procedures, and
was—at best—a “toy.” Over the years, the applications of the technology have
multiplied greatly. The application of simulation to “nontechnical” or behavioral
skills has become commonplace. This has been extended substantially to also cover
arenas of higher level decision-making concerning issues of professionalism and
indeed of ethics, and these issues are dealt using simulation in many sites, in many
disciplines and domains of healthcare. For example, my simulation colleagues in
Emergency Medicine in 2004 described1 the embedding of “ethical probes” into
nearly all their simulation scenarios in their Emergency Medicine CRM courses.
Although most simulations are still targeted at education and training, the use of
simulation for research on patient care processes and clinician performance has also
grown. In this issue of Simulation in Healthcare, Waisel et al2 describe a research use
of simulation for assessing how clinicians deal with “Do Not Resuscitate” (DNR)
orders and preferences for patients who are facing and then undergoing surgical
procedures. Although many review articles on the topic of DNR in the OR have been
published over more than 15 years (several in the past few years)3– 8 the implemen-
tation of the principles described therein is thought to be patchy at best.
Simulation provides a crucial window on clinical processes and clinician perfor-
mance. I have long described the complementarity of simulation with other meth-
ods of investigating performance. Recently, Mcintosh9 named this concept “Gaba’s
Jigsaw” alluding to the notion that each of the windows on performance is another
piece of a larger jigsaw puzzle. The study of Waisel et al2 adds considerably to our
knowledge of the issues and problems surrounding the way that clinicians handle the
preferences of patients with DNR orders, because it allows the study of what people
actually do (albeit in simulations) rather than what they think should be done, or what is
reported in retrospect. Each method of assessment has strengths and weaknesses.
Prospective nonbiased assessment of clinical process and clinician behavior in
real case situations would clearly be the best. For some questions this can be accom-
plished by making recordings of data (through electronic medical records—the
medical equivalent of aviation’s “black box”) or even sight and sound (through
embedded microphones and cameras—the equivalent of the “cockpit voice re-
corder”) of many cases to capture those of interest for detailed study. Yet, few
patients coming for anesthesia and surgery have DNR orders so one would have to
capture very many cases to get a fair sample of such situations (or else to have either
an expert observers or a camera crew waiting patiently and immediately available for
the few cases that come along). And—I can attest to this personally—the medico-
legal, ethical, and logistical issues raised by filming in the clinical environment are
daunting. Moreover, each such clinical situation is different, making it hard to see
the spectrum of performance of clinicians responding to a “standardized” challenge.
Case reports of real cases also offer a different window on performance. Such cases
are actual clinical situations and only ones of salience would be brought to research-
ers’ attention. However, the retrospective nature of these reports limits the infor-
mation available, and memory recall of complicated cases is often faulty and subject
to various hindsight biases. Moreover, there is likely to be a selection bias. Reporters
are only likely to report cases in which there were significant problems and not those
67
 for which care processes were faulty but whose results were
unremarkable. Even without these limitations, there is still
the problem that all the cases are different making it is hard to
generalize from the reports.
Simulation therefore closes some of the holes left by the
other techniques while possessing some of its own. Each case
scenario is standardized so all clinicians face the same chal-
lenges and conundrums; these can be as difficult as desired.
Observation can be prospective and recorded in a nonbiased
Downloaded from http://journals.lww.com/simulationinhealthcare by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgb
sIHo4XMi0hCywCX1AWnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023
fashion, typically using audio and video capture. Recordings
allow for analysis by multiple observers or in multiple passes.
Observers can be chosen with special expertise or training if
desired. The big limitation is that simulations are not real
cases, and all participants know that this is true. This hole can
be closed— or another piece of the puzzle added if the simu-
lations can be conducted such that the clinicians are unaware
that it is not real. An example of such an approach when
unannounced clinic visits by standardized patient actors are
used for quality assurance or research purposes in the outpa-
tient clinic setting.10 –12 For acute care treatment situations,
where actors cannot be used, such surreptitious simulations
would require an “android” so good that it could be slipped
into the clinical environment as if it was a real patient. This
technology does not exist now and it may never be available.
Thus, a fundamental limitation of acute care simulations for
the foreseeable future is that clinicians know they are simu-
lations. How is this likely to affect their performance? On the
one hand, when they know that the “patient” is not a real
patient, we can imagine that they will not take the situation
seriously and will not perform all the actions as they would
with a real patient. This can sometimes manifest itself as
unusually sloppy or “cavalier” behavior, or clearly acting like
it is all a game. Conversely, we sometimes see “hypervigi-
lance” where participants later admit to doing more than they
usually would do or to jumping too quickly on signs of pos-
sible trouble. Often this is because, knowing it is a simulation,
they were expecting problems to occur and they acted on the
first glimmer of abnormality. Other times they say that it is
because they knew they were being filmed and thus felt that
they should do everything “by the book” even if the book is
usually honored only “in the breach” (as Shakespeare put it).
One thing we learned in our studies of fatigued versus
rested anesthesiologists13 is that it is very hard to reproduce
the “motivational structure” of real work—the incentives
and disincentives and culture of the workplace. This is a real
problem for investigating performance in situations that de-
pend on subtle judgments, negotiation between personnel,
and the concerns for personal advancement, administrative
discipline or risk of litigation. Waisel et al report some behav-
iors that seemed to stem from group interactions (usually,
because of the script, pushing them toward more invasive and
long-term interventions) but it is hard to know whether these
suggestions would have occurred in a real case, or how the
arguments would actually play out in a real operating room
with senior surgeons and anesthesiologists who have their
own interpersonal histories and intra and interdepartmental
politics to consider. Again, without the undetected surrepti-
tious android it is pretty much impossible to study such is-
sues with high generalizability.
68 Using Simulation to Investigate Clinical Behavior
If we assume that this simulation study of DNR situations
does provide accurate data, what are we to make of the re-
sults? It should not be surprising that what people actually do
in complex situations is not quite what we hope or expect that
they would do or what is recommended in the various excel-
lent review articles about the issue. Even for clinical care
issues with widely accepted protocols, the implementation of
the protocols is patchy. For ethical issues for which different
clinicians likely have different personal opinions, high vari-
ability is even more expected. Nonetheless, certain features
stand out as failures of our current system. Even knowing that
they were being filmed, a substantial number of clinicians
never even discussed the DNR issues with the patient to con-
firm what was written in the chart. Questionnaire data
showed that 50% of participants were mostly or totally unfa-
miliar with ASA guidelines on the topic. This is disturbing
especially because the participants in this study were faculty
anesthesiologists at quarternary care hospitals who are re-
sponsible for teaching trainees how to practice, and who also
have a more significant likelihood of encountering complex
care for terminal patients than might be true in community
hospitals. Waisel et al suggest that more education on these
issues will help to increase the frequency and depth of discus-
sions of the issues in the preoperative interviews; certainly for
those who acted out of misapprehension of current protocols
(eg, “automatic suspension of DNR orders”) or of total igno-
rance of the topic, education may at least sensitize them to the
issues or correct their mistaken beliefs. Perhaps, training and
practice in the conduct of difficult conversations, and more
specifically about conducting preoperative interviews about
DNR issues, would be more effective in ensuring thorough
preoperative planning. It has become commonplace to have
patients complete “satisfaction” surveys periodically after
clinical encounters. Should we target some of these at the
process of managing ethical issues including those of DNR in
the perioperative period? Should we set benchmarks for per-
formance to provide incentives to follow established guide-
lines?
Given the nature of the reasoning used in the response to
the dynamically evolving situation in these simulations, it is
clear that establishing a goal-directed plan for patient care is
only a first step, and that many pitfalls exist in actually imple-
menting the plan. The results of a study by Waisel et al high-
light the need for more attention to these issues, and also
suggests the possible benefits of using simulations with both
standardized patient actors (for preoperative interviews) and
multiclinician scenarios of evolving catastrophes to address
the optimum ethical management of patients with clear-cut
goals and preferences for limiting resuscitative efforts. Thus,
I believe that this study provides an important demonstration
of how simulation can be a powerful tool for delineating
strengths and weaknesses of our systems and processes in-
cluding the handling of delicate issues of professionalism and
ethics. It is good for us to know how clinicians really behave
(or at least how they are likely to behave) rather than to
merely assume that our typical “good clinicians” already
know everything about relevant protocols or principles of
ethical care and can readily implement them when things
turn sour.
Simulation in Healthcare
 ACKNOWLEDGMENTS
The author thank Associate Editor Barry Issenberg for rais-
ing some good points that are now embedded in this editorial.
REFERENCES
1. Gisondi MA, Smith-Coggins R, Harter PM, et al. Assessment of
resident professionalism using high-fidelity simulation of ethical
dilemmas. Acad Emerg Med 2004;11:931–937.
2. Waisel DB, Simon R, Truog RD, Baboolal H, Raemer DB.
Anesthesiologist management of perioperative do-not-resuscitate
Downloaded from http://journals.lww.com/simulationinhealthcare by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgb
sIHo4XMi0hCywCX1AWnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 05/01/2023
orders: a simulation-based experiment. Simul Healthc 2009;4:70 –76.
3. Margolis JO, McGrath BJ, Kussin PS, et al. Do not resuscitate (DNR)
orders during surgery: ethical foundations for institutional policies in
the United States. Anesth Analg 1995;80:806 – 809.
4. Truog RD, Waisel DB, Burns JP. DNR in the OR: a goal-directed
approach. Anesthesiology 1999;90:289 –295.
5. Jackson SH, Van Norman GA. Goals- and values-directed approach to
informed consent in the “DNR” patient presenting for surgery: more
demanding of the anesthesiologist? Anesthesiology 1999;90:3– 6.
6. Waisel DB, Burns JP, Johnson JA, et al. Guidelines for perioperative
do-not-resuscitate policies. J Clin Anesth 2002;14:467– 473.
Vol. 4, No. 2, Summer 2009
7. Fallat ME, Deshpande JK. Do-not-resuscitate orders for pediatric
patients who require anesthesia and surgery. Pediatrics
2004;114:1686 –1692.
8. Ewanchuk M, Brindley PG. Perioperative do-not-resuscitate
orders– doing ‘nothing’ when ‘something’ can be done. Crit Care
2006;10:219.
9. Mcintosh C. Lake Wobegone for anestheisa. Where everyone is above
average except those who aren’t: variability in the management of
simulated intraoperative critical incidents (editorial). Anesth Analg
2009;108:6 –9.
10. Beullens J, Rethans J, Goedhuys J, et al. The use of standardized
patients in research in general practice. Fam Pract 1997;14:58 – 62.
11. Franz C, Epstein R, Miller K, et al. Caught in the act? Prevalence,
predictors, and consequences of physician. Health Serv Res 2006;41:
2290 –2302.
12. Krane NK, Anderson D, Lazarus CJ, et al. Physician practice behavior
and practice guidelines: using unannounced standardized patients to
gather data. J Gen Intern Med 2009;24:53–56.
13. Howard S, Gaba D, Smith B, et al. Simulation study of rested
versus sleep deprived anesthesiologists. Anesthesiology 2003;98:
1345–1355.
© 2009 Society for Simulation in Healthcare 69