Professional Documents
Culture Documents
Vol. 11, Number 1, February 2016 * 2016 Society for Simulation in Healthcare 21
Copyright © 2016 by the Society for Simulation in Healthcare. Unauthorized reproduction of this article is prohibited.
TABLE 2. Impact of a Classroom Lecture on the Management patients. Participants can address questions about skin color
of Simulations of Septic Shock and warmth, capillary refill, pupils, neck veins, and mucus
Lecture First Simulation First Significance membranes to the simulation operators and receive the in-
Interns’ technical 9.2 (1.6) 10.3 (1.6) n.s. formation via an overhead speaker. Nonetheless, we grant
score, mean (SD) that some may use these and other clues to form a sub-
Residents’ technical 10.5 (1.6) 12.3 (1.2) P G 0.05 conscious picture of a patient and his clinical course that may
score, mean (SD)
not be possible with a mannequin.
Residents’ nontechnical 23.58 26.95 n.s.
score, mean (SD) Our scoring rubric presents a list of ideal actions and
behaviors, but even executing them perfectly cannot guar-
Phase II data; n.s., not significant.
antee a good outcome in all cases. Similarly, the sequence of
scenarios did not differ between the groups with the highest actions (such as provision of fluids before vasopressors)
(top 5) and lowest (bottom 5 of 23) levels of performance for could be important in real life, whereas our scoring system
both interns or for resident teams. Likewise, responses re- did not distinguish between order of interventions. Although
garding realism did not differ between ‘‘simulation first’’ or we hold the belief that simulation approximates real-life
‘‘lecture first’’ groups. behavior, it is still true that physicians, patients, and the
health care system in general probably behave differently
when real lives are at stake. Although we make great efforts to
DISCUSSION
recreate the ‘‘motivational structure’’ of the real world in
Simulation techniques have created exciting opportu-
simulations, it is impossible to really know whether this has
nities for the enhancement of medical education; however,
been achieved.
their use as a tool to evaluate aspects of traditional medical
education is relatively novel. This is an important goal be- Implications of Findings
cause the ultimate purpose of providing medical knowledge In phase II, all trainees at multiple levels demonstrated
to trainee clinicians is in fact to allow them to conduct pa- an increase in theoretical knowledge on the quiz, so the lack
tient care effectively. of transfer to the simulated patient was completely unex-
Veracity of Findings pected. We assumed that the lecture would have a positive
From the survey data after simulations and voluntary impact on the technical scores of the interns. This as-
comments concerning the quality of lectures, it seems that sumption was based on the idea that interns would have had
the data are a fairly credible representation of participants’ the least experience managing critical illness, and therefore,
behavior. We therefore believe that the results require con- their management of sepsis would be influenced by in-
sideration of a number of factors related to lectures and struction to a degree greater than that of more senior resi-
classroom instruction including quality, context, and cog- dents. Supporting this notion is the fact that quiz scores
nitive processing. The results also prompt consideration of showed the greatest improvement in lower levels of training
our own goals for such instruction and how realistic it is to (Fig. 1). Our finding is not a unique observation. For ex-
expect behavioral change from these lectures. ample, Rodgers et al16 rated performance of nursing stu-
Could the results be explained by a lecture poor in quality dents during simulations of cardiac arrests and found no
or content? All items present on the technical scoring checklist association between simulator performance and score on
were discussed in the corresponding lecture and queried on the
quiz. Residents have repeatedly identified the lectures as being
a high quality, relevant, and valued component of the ICU
rotation. However, it is interesting that no group was able to
achieve an average of 10/10 on the postlecture quiz.
Could temporal trends influence results? Our expectation
that collective knowledge increases over time or increases by
the number of months into internship year seems to be un-
founded (Fig. 2). The latter is somewhat surprising because we
assumed that there was some exposure to the management
concepts under question in other rotations before an intern’s
ICU rotation and that such experiences were likely to accu-
mulate during the course of the academic year.
Is it possible that the performance seen in the simulation
FIGURE 1. Average prelecture and postlecture quiz scores are
underrates true clinical performance? This is certainly a shown for each level of training. Light bars indicate prelecture
concern of many programs offering clinical simulation ex- scores, and dark bars are the postlecture tests. All groups showed
ercises and is the reason we go to great lengths to familiarize improvement in score as a result of the lecture (*P G 0.05 by
the participants with the simulator, to use the same equip- Student t test for paired samples). Intergroup comparisons showed
ment used in the ICU, and to have a ‘‘facilitator nurse’’ who a trend toward higher prelecture scores according to the level of
training, with all postgraduate scores significantly different from
knows where supplies are, so that participants will not be the medical students’ scores (**P G 0.05 by ANOVA with
frustrated by an unfamiliar environment. What we cannot Bonferroni correction for multiple samples). No statistical differ-
replicate are the skin, motor movements, and affects of true ences existed amongst the posttest scores.
previously administered tests. The present study adds to the interval and that more immediate practice and hands-on ex-
latter work by showing a similar disconnection between perience with newly learned material is a requisite for reliable
acquired and deployable knowledge in multiple levels of incorporation into practice. For example, ‘‘just in time’’ and
postgraduate trainees. The degree of standardization of the ‘‘booster’’ training provided immediately before simulations
scenarios for all participants and consistency of content improves the efficacy of chest compressions in simulated
among the lecture, quiz, simulation and rating tool are pediatric arrests.21,22 Unfortunately, it is hard to predict what
unique to this study and improve the credibility of the ex- type of patient problem will present at a given moment, so
perimental findings. pairing training to immediate clinical need is impractical.
We have no clear explanation as to why apparent uptake of Nonetheless, the sequence of instruction followed by guided
clinically relevant material did not manifest itself by higher practice may provide superior training over separation of these
simulation performance. A number of explanations exist and components and deserves further evaluation.
may be reasonable targets for further investigation. Perhaps, We started this experiment hoping to show that the con-
the knowledge gained needs time to be processed, consoli- tent of lectures could be altered to create improvements in
dated, and compared with existing mental models of disease clinical care and ended up questioning the value of lectures in
management before becoming deployable. It is possible that general. Lectures, both in the traditional in-person form and
the conditions presented to the trainees during the simulation in the video-based ‘‘flipped classroom’’ paradigm, have been
were sufficiently stressful that a slower, more deliberate thought to have significant value in conveying ‘‘knows’’ and
method of processing the new information was required but ‘‘knows how’’ information at a rather low cost. Hundreds of
not easily accessible.17Y19 As postcourse surveys and discus- learners can be in the audience in person; an unlimited number
sions do reveal that the trainees consider the simulations in- might view the lecture on video. Yet, if such activities do not
tense and challenging,20 this explanation seems plausible. result in deployable knowledge and skill, this leverage is useless.
A contrasting explanation is that the day or two sepa- If lectures do not work, then what does? Trainees may
rating the lecture and the simulation was too great of an need to be ‘‘shown the way’’ to a greater degree than we are
Vol. 11, Number 1, February 2016 * 2016 Society for Simulation in Healthcare 23
Copyright © 2016 by the Society for Simulation in Healthcare. Unauthorized reproduction of this article is prohibited.
willing to admit. In either event, with resident work hour College of Emergency Physicians; Canadian Critical Care Society;
European Society of Clinical Microbiology and Infectious Diseases;
limitations in place, there is an added premium on under- European Society of Intensive Care Medicine; European Respiratory
standing the impact of educational programs on patient care Society; International Sepsis Forum; Japanese Association for Acute
and finding the most efficient means to increase the effec- Medicine; Japanese Society of Intensive Care Medicine; Society of
Critical Care Medicine; Society of Hospital Medicine; Surgical Infection
tiveness of physicians. Future work in this area should use Society; World Federation of Societies of Intensive and Critical Care
focused interviews and other qualitative techniques to better Medicine. Surviving Sepsis Campaign: international guidelines for
understand the behaviors and cognitive processes that un- management of severe sepsis and septic shock: 2008. Crit Care Med
2008;36:296Y327.
derlie performance at the different ends of the spectrum.
Educational programs should then be shaped around the 5. Dellinger RP, Levy MM, Rhodes A, et al; Surviving Sepsis Campaign
Guidelines Committee including the Pediatric Subgroup. Surviving
factors associated with high performance. Measurement sepsis campaign: international guidelines for management of severe
systems, such as the one presented here, may help evaluate sepsis and septic shock: 2012. Crit Care Med 2013;41:580Y637.
performance of education programs and with further re- 6. Miller GE. The assessment of clinical skills/competence/performance.
finement might predict clinical outcomes. For example, Acad Med 1990;65:S63YS67.
some actions may prove to have greater impact than others 7. Lighthall GK, Barr J, Howard SK, et al. Use of a fully simulated intensive
in managing certain clinical problems, and any rating system care unit environment for critical event management training for
internal medicine residents. Crit Care Med 2003;31:2437Y2443.
would need to weigh these accordingly. In addition, as
mentioned earlier, the sequence of actions is often important 8. Ottestad E, Boulet JR, Lighthall GK. Evaluating the management of
septic shock using patient simulation. Crit Care Med 2007;35:769Y775.
and so might need to be more closely evaluated.
9. Lighthall GK, Harrison TK. A controllable patient monitor for
classroom video projectors. Simul Healthc 2010;5:58Y60.
CONCLUSIONS 10. Gaba DM. Improving anesthesiologists’ performance by simulating
This study used a standardized simulation as a surrogate reality. Anesthesiology 1992;76:491Y494.
for clinical performance and hypothesized that a carefully 11. Flin R, Maran N. Identifying and training non-technical skills for teams
designed lecture would lead to quantitative improvements in acute medicine. Qual Saf Health Care 2004;13(suppl 1):i80Yi84.
in clinical performance. Instead, we showed that perfor- 12. Fletcher G, Flin R, McGeorge P, Glavin R, Maran N, Patey R.
mance scores of intensive care interns and residents man- Anaesthetists’ Non-Technical Skills (ANTS): evaluation of a behavioural
marker system. Br J Anaesth 2003;90:580Y588.
aging a simulated case of septic shock were no better when a
lecture/workshop on the same disease process preceded the 13. Helmreich RL, Merritt AC, Wilhelm JA. The evolution of Crew Resource
Management training in commercial aviation. Int J Aviat Psychol
simulationVdespite demonstrating uptake of the lecture 1999;9:19Y32.
content. Lectures and group discussions are the de facto
14. DeVita MA, Schaefer J, Lutz J, Dongilli T, Wang H. Improving medical
standard for imparting medical knowledge. Despite such crisis team performance. Crit Care Med 2004;32:S61YS65.
status, the transfer of knowledge between lecture and bedside 15. Davis D, Evans M, Jadad A, et al. The case for knowledge translation:
care has not been formally proven. Ironically, proof of effi- shortening the journey from evidence to effect. BMJ 2003;327:33Y35.
cacy is a burden placed on the proponents of simulation and 16. Rodgers DL, Bhanji F, McKee BR. Written evaluation is not a predictor
immersive learning as these techniques attempt to find a for skills performance in an Advanced Cardiovascular Life Support
niche in the instructional repertoire. It is difficult to com- course. Resuscitation 2010;81:453Y456.
pletely discard the value of classroom instruction; however, 17. Stiegler MP, Gaba DM. Decision-making and cognitive strategies.
this and future studies may prove it to be more effective as Simul Healthc 2015;10:133Y138.
a complimentary technique rather than the sine qua non of 18. Croskerry P. Clinical cognition and diagnostic error: applications of a
health care education. dual process model of reasoning. Adv Health Sci Educ Theory Pract
2009;14(Suppl 1):27Y35.
19. Kahneman D. Thinking Fast and Slow. New York: Farrar, Straus and
REFERENCES Giroux; 2011.
1. Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky 20. Lighthall GK, Barr J. The use of clinical simulation systems to train
MR. Epidemiology of severe sepsis in the United States: analysis of critical care physicians. J Intensive Care Med 2007;22:257Y269.
incidence, outcome, and associated costs of care. Crit Care Med
2001;29:1303Y1310. 21. Cheng A, Brown LL, Duff JP, et al; International Network for
Simulation-Based Pediatric Innovation, Research, & Education
2. Rivers E, Nguyen B, Havstad S, et al; Early Goal-Directed Therapy
(INSPIRE) CPR Investigators. Improving cardiopulmonary
Collaborative Group. Early goal-directed therapy in the treatment of
resuscitation with a CPR feedback device and refresher simulations
severe sepsis and septic shock. N Engl J Med 2001;345:1368Y1377.
(CPR CARES Study): a randomized clinical trial. JAMA Pediatr
3. Dellinger RP, Carlet JM, Masur H, et al; Surviving Sepsis Campaign 2015;169:137Y144.
Management Guidelines Committee. Surviving Sepsis Campaign
22. Sutton RM, Niles D, Meaney PA, et al. ‘‘Booster’’ training: evaluation
guidelines for management of severe sepsis and septic shock.
of instructor-led bedside cardiopulmonary resuscitation skill training
Crit Care Med 2004;32:858Y873.
and automated corrective feedback to improve cardiopulmonary
4. Dellinger RP, Levy MM, Carlet JM, et al; International Surviving resuscitation compliance of Pediatric Basic Life Support providers
Sepsis Campaign Guidelines Committee; American Association of during simulated cardiac arrest. Pediatr Crit Care Med
Critical-Care Nurses; American College of Chest Physicians; American 2011;12:e116Ye121.