The Influence of Resident Involvement on

Surgical Outcomes
Mehul V Raval, MD, MS, Xue Wang, MA, Mark E Cohen, PhD, Angela M Ingraham, MD, MS,
David J Bentrem, MD, MS, FACS, Justin B Dimick, MD, MPH, FACS, Timothy Flynn, MD, FACS,
Bruce L Hall, MD, PhD, MBA, FACS, Clifford Y Ko, MD, MS, MSHS, FACS

BACKGROUND: Although the training of surgical residents is often considered in national policy addressing
complications and safety, the influence of resident intraoperative involvement on surgical
outcomes has not been well studied.
STUDY DESIGN: We identified 607,683 surgical cases from 234 hospitals from the 2006 to 2009 American
College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). Out-
comes were compared by resident involvement for all general and vascular cases as well as for
specific general surgical procedures.
RESULTS: After typical ACS NSQIP comorbidity risk adjustment and further adjustment for hospital teaching
status and operative time in modeling, resident intraoperative involvement was associated with
slightly increased morbidity when assessing overall general or vascular procedures (odds ratio [OR]
1.06; 95% CI 1.04 to 1.09), pancreatectomy or esophagectomy (OR 1.26; 95% CI 1.08 to 1.45),
and colorectal resections (OR 1.15; 95% CI 1.09 to 1.22). In contrast, for mortality, resident
intraoperative involvement was associated with reductions for overall general and vascular proce-
dures (OR 0.91; 95% CI 0.84 to 0.99), colorectal resections (OR 0.88; 95% CI 0.78 to 0.99), and
abdominal aortic aneurysm repair (OR 0.71; 95% CI 0.53 to 0.95). Results were moderated
somewhat after hierarchical modeling was performed to account for hospital-level variation, with
mortality results no longer reaching significance (overall morbidity OR 1.07; 95% CI 1.03 to 1.10,
overall mortality OR 0.97; 95% CI 0.90 to 1.05). Based on risk-adjusted event rates, resident
intraoperative involvement is associated with approximately 6.1 additional morbidity events but 1.4
fewer deaths per 1,000 general and vascular surgery procedures.
CONCLUSIONS: Resident intraoperative participation is associated with slightly higher morbidity rates but slightly
decreased mortality rates across a variety of procedures and is minimized further after taking into
account hospital-level variation. These clinically small effects may serve to reassure patients and
others that resident involvement in surgical care is safe and possibly protective with regard to
mortality. (J Am Coll Surg 2011;xx:xxx. © 2011 by the American College of Surgeons)

Teaching hospitals are often thought to provide a high
quality of surgical care based on the complexity of patients
referred, access to specialized technology, experience with
rare diseases, high surgical volumes, education and reputa-
tion of affiliated staff, and participation in clinical re-
Disclosure Information: Nothing to disclose.
Drs Raval and Ingraham participate in the American College of Surgeons Clinical
Scholars in Residence Program. Dr Raval is supported by the John Gray Research
Fellowship and the Daniel F and Ada L Rice Foundation. Dr Bentrem is sup-
ported by a career development award from the Department of Veterans Affairs,
Veterans Health Administration, Health Services Research and Development
Service. Dr Hall assists with the conduct of the ACS NSQIP nationally.
Received August 20, 2010; Revised November 20, 2010; Accepted December
14, 2010.
From the Division of Research and Optimal Patient Care, American College
of Surgeons (Raval, Wang, Cohen, Ingraham, Ko), the Department of
search.1 Many of the top medical centers, as identified by
publicly available sources such as US News and World Re-
port and the Leapfrog Group, are major teaching institu-
tions.2,3 Increasingly, resident and trainee participation in
Surgery, Northwestern University Feinberg School of Medicine (Raval, Ben-
trem), and Jesse Brown VA Medical Center, (Bentrem), Chicago, IL; the
Department of Surgery, University of Cincinnati College of Medicine, Cin-
cinnati, OH (Ingraham); University of Michigan Medical School, Ann Arbor,
MI (Dimick); University of Florida, Gainesville, FL (Flynn); the John Coch-
ran Veterans Affairs Medical Center, Department of Surgery, Washington
University in St Louis, St Louis, MO; Department of Surgery, Barnes Jewish
Hospital; Center for Health Policy and the Olin Business School at Washing-
ton University in St Louis, St Louis, (Hall); and the Department of Surgery,
University of California Los Angeles David Geffen School of Medicine and
the VA Greater Los Angeles Healthcare System, Los Angeles, CA (Ko).
Correspondence address: Mehul V Raval, MD, American College of Sur-
geons, 633 N Saint Clair St, 22nd Floor, Chicago, IL 60611-3211. email:
m-raval@md.northwestern.edu

© 2011 by the American College of Surgeons ISSN 1072-7515/11/$36.00

Published by Elsevier Inc. 1 doi:10.1016/j.jamcollsurg.2010.12.029
2 Raval et al Resident Involvement and Surgical Outcomes
Abbreviations and Acronyms
ACSNSQIP ⫽ American College of Surgeons National
Surgical Quality Improvement Program
OR ⫽ odds ratio
SCR ⫽ surgical clinical reviewer
VA ⫽ Veterans Affairs
the operative and nonoperative management of patients
has come under scrutiny as a potential area for improved
surgical safety and efficiency.4 Concerns of resident inexpe-
rience as a contributing factor to surgical error have long
sparked debate over the existence of a “July phenomenon,”
and resident involvement in surgical patient care is some-
times not preferred by patients.5-9 Resident education using
technologically advanced simulation laboratory training
has been advocated as a mechanism to codify performance
and has been gaining acceptance at most teaching centers.10
Nevertheless, the relationship between surgical outcomes
and intraoperative resident participation has not been well
studied.
Since 2004, the American College of Surgeons National
Surgical Quality Improvement Program (ACS NSQIP) has
provided a validated, outcomes-based, risk-adjusted, and
peer-controlled assessment of surgical quality based on
clinical information.11-15 Reliable 30-day outcomes serve as
a quality improvement catalyst for ACS NSQIP participat-
ing institutions including private-sector academic and
nonacademic hospitals.16,17 The purpose of this study was
to determine whether resident intraoperative involvement
was associated with detectable differences in surgical out-
comes using the ACS NSQIP data set.
METHODS
Data set, variables, and outcomes
Current details of the ACS NSQIP, including developmen-
tal history, sampling strategy, data abstraction procedures,
variables collected, outcomes, and structure have been de-
scribed elsewhere.11,12,18-21 In brief, the program collects
and audits detailed and standardized data on patient demo-
graphics, preoperative risk factors, laboratory values, oper-
ative variables, and postoperative events on a systematic
sample of general and vascular cases (standard ACS
NSQIP), or on a broader group of surgical specialty cases
(ACS NSQIP multispecialty program). Data are gathered
by trained surgical clinical reviewers (SCRs) using standard
ACS NSQIP tools and definitions.22,23 Clinical resources
including continuing education and monthly conference
calls are available to SCRs. From July 1, 2006 to June 30,
2009 (3 complete program years), a total of 607,683 gen-
J Am Coll Surg
eral and vascular surgery cases were entered from 234 par-
ticipating hospitals.
The ACS NSQIP provides participating hospitals with a
variety of risk-adjusted models including mortality and
overall morbidity models. Morbidity is defined as the pres-
ence of at least one of the following ACS NSQIP defined
complications: superficial surgical site infection, deep sur-
gical site infection, organ space surgical site infection,
wound dehiscence, neurologic event (stroke or cerebrovas-
cular accident, coma lasting more than 24 hours, or periph-
eral neurologic deficit), cardiac arrest, myocardial infarc-
tion, bleeding requiring transfusion, deep vein thrombosis,
pulmonary embolism, pneumonia, unplanned intubation,
ventilator dependence more than 48 hours, urinary tract
infection, progressive or acute renal insufficiency, and sep-
sis or septic shock. Patients were excluded from having the
following complications if the condition was documented
preoperatively: wound infection, pneumonia, ventilator
dependence or reintubation, renal failure, stroke, and
coma. To determine mortality, SCRs examine medical re-
cords, attempt to contact patients via telephone or mail,
and query the Social Security Death Index and the Na-
tional Obituary Archives.
In addition to typical ACS NSQIP risk adjustment us-
ing clinically collected preoperative risk factors, laboratory
values, and operative variables, case-mix adjustment vari-
ables included work relative value units (RVU) based on
Current Procedural Terminology (CPT) codes as well as a
134-category CPT procedure group variable that estimates
endogenous risk coefficients for each procedure cate-
gory.24,25 The effects of 3 other factors were also considered.
First, each operation was classified as associated or not as-
sociated with resident intraoperative involvement, as cod-
ified in the ACS NSQIP data fields. Second, hospitals were
identified as teaching centers if their ratio of interns and
residents to hospital beds (as available through the Ameri-
can Hospital Association Annual Survey: n ⫽ 211 catego-
rized in this fashion, 90.2%) was greater than 0.10.1,26 If
the ratio was unavailable, teaching status was assigned
based on hospital affiliation with the Council of Teaching
Hospitals of the Association of American Medical Colleges
and Accreditation Council for Graduate Medical Educa-
tion (n ⫽ 18, 7.7%). If teaching status could not be as-
signed based on these criteria, ACS NSQIP administrative
files were queried for self-reported teaching status (n ⫽ 5,
2.1%). Finally, operative times were assessed for each pro-
cedure grouping and divided into quartiles within proce-
dure groups (shortest, short, medium, and long).
In addition to the ACS NSQIP overall model, which
includes all general and vascular surgery cases at a hospital,
several specific procedures with varying levels of complex-
Vol. xx, No. x, Month 2011
Table 1. Procedure Groups Studied from the ACS NSQIP from July 2006 to June 2009
Cases
Procedures n Total, %
Overall general and vascular surgery 607,683 100
Pancreatectomy and esophagectomy 8,655 1.42
Colorectal surgery 55,798 9.18
Abdominal aortic aneurysm repair 4,218 0.69
Laparoscopic cholecystectomy and open
inguinal hernia repair 92,157 15.17
ACS NSQIP, American College of Surgeons National Surgical Quality Improvement Program.
ity were identified based on CPT codes. Complex general
surgery procedures studied included pancreatectomy and
esophagectomy as well as colorectal cases. Pancreatectomy
and esophagectomy were combined due to low volumes
when considered individually. Open abdominal aortic an-
eurysm (AAA) repair was selected as a complex vascular
case. Laparoscopic cholecystectomy and open inguinal her-
nia repair were selected as simple general surgical cases and
combined due to low event rates. Conclusions were un-
changed when studying the combined procedure groups
together or individually.
Statistical analyses
Chi-square statistics were used to compare morbidity and
mortality rates overall or for each procedure group of inter-
est. Correlation coefficients were calculated between resi-
dent involvement and hospital teaching status and resident
involvement and operative time. Logistic regression models
were constructed for each outcome (morbidity and mortal-
ity) for each procedure grouping. Clinically relevant vari-
ables were entered into the logistic models and additional
ACS NSQIP variables were selected based on a forward
step-wise process, with p for inclusion being 0.05. In addi-
tion, models were sequentially forced to incorporate resi-
dent involvement, resident involvement and hospital
teaching status, or resident involvement, hospital teaching
status, and operative time. Random-effects modeling was
used to account for clustering of patients at hospitals.
Model quality was evaluated through assessment of the
c-statistic and the Hosmer-Lemeshow goodness-of-fit sta-
tistic.27 In addition, fixed-effects hierarchical models were
used to adjust for the hospital-level factors that might affect
outcomes. Fixed-effects models, using hospital identifier
variables, allow for more robust isolation of the effect of
resident involvement on outcomes. If the differences in
outcomes are no longer significant after accounting for
hospital-level variation, then the differences in outcomes
can be assumed to arise from differences between hospitals.
Conversely, if the differences in outcomes remain signifi-
cant after accounting for hospital-level variation, then the
Raval et al Resident Involvement and Surgical Outcomes 3
Unadjusted Unadjusted
Teaching, % Resident, % morbidity, % mortality, %
63.27 61.78 11.27 1.83
76.11 85.58 35.45 2.46
62.60 67.46 26.81 4.17
71.55 70.01 35.44 10.79
58.46 57.62 2.67 0.28
differences can be attributed to variables such as resident
involvement.
Risk standardized morbidity and mortality rates were
generated using the adjusted odds ratios (ORs) on resi-
dent involvement from regressions, in combination with
the equation relating rates to odds ratios: Odds ratio on
resident involvement ⫽ (RateR/[1-RateR])/(RateN/[1-
RateN]); where “RateR” is the risk standardized rate
with resident involvement and “RateN” is the rate with
no resident involvement. Equations were constrained by
requiring the weighted average of rates with and without
resident to equal the observed event rate in question.
Based on these risk standardized rates, estimates for the
predicted increase or decrease in events per 1,000 pa-
tients with resident intraoperative involvement were cal-
culated (multiply each rate ⫻ 1,000, subtract RateN
from RateR). Furthermore, mortality event rates were
calculated for patients with at least 1 morbidity event
and compared by resident involvement. Data manipula-
tion and analyses were done with SAS version 9.2 or
Microsoft Excel 2007 (Microsoft Corp). The study was
performed under exempt status after review of study
protocols by the Northwestern University Institutional
Review Board.
RESULTS
Of the 607,683 general and vascular surgery cases identi-
fied, there were 8,655 pancreatectomy or esophagectomy
cases (1.4%), 55,798 colorectal cases (9.2%), 4,208 open
abdominal aortic aneurysm repairs (0.7%), and 92,157
laparoscopic cholecystectomy or open inguinal hernia cases
(15.2%). Resident intraoperative involvement ranged from
57.6% for laparoscopic cholecystectomy or open inguinal
hernia cases to 85.6% for pancreatectomy or esophagec-
tomy cases (Table 1). Overall morbidity ranged from 2.7%
for laparoscopic cholecystectomy or open inguinal hernia
cases to 35.5% for pancreatectomy and esophagectomy
cases. Mortality ranged from 0.28% for laparoscopic cho-
lecystectomy and open inguinal hernia cases to 10.8% for
open abdominal aortic aneurysm repairs. Within each pro-
4 Raval et al Resident Involvement and Surgical Outcomes J Am Coll Surg

Figure 1. Odds ratios and 95% confidence intervals for resident versus no resident, and for morbidity and mortality, for 5 groupings of surgical
procedures. For each outcome and surgical group, the 5 odds ratios are from models that are based on (from top to bottom): logistic
regression using standard ACS NSQIP variables; this model plus hospital teaching status; the former model plus operative time; the model
analyzed with a random intercepts (hospital) hierarchical model; and a logistic model in which hospital is treated as a fixed effect. Where only
4 models are reported in a cell, the last model is not available due to lack of convergence. ACS NSQIP, American College of Surgeons National
Surgical Quality Improvement Program.

cedure group, resident involvement was correlated with
hospital teaching status (overall general and vascular sur-
gery R ⫽ 0.51, p ⬍ 0.001) and longer operative time
(overall general or vascular surgery R ⫽ 0.21, p ⬍ 0.001).
Resident involvement and outcomes
Figure 1 addresses morbidity and mortality and demon-
strates the effect of sequential addition of resident involve-
ment, hospital teaching status, and operative time variables
to the typical ACS NSQIP risk modeling variables for each
of the procedure groups and modeling techniques. After
controlling for hospital teaching status and operative dura-
tion quartile, resident intraoperative involvement was asso-
ciated with higher morbidity for overall general and vascu-
lar surgery cases (OR 1.06; 95% CI 1.04 to 1.09),
pancreatectomy and esophagectomy (OR 1.26; 95% CI
1.08 to 1.45), and colorectal resections (OR 1.15; 95% CI
1.09 to 1.22). Though results were largely unchanged after
random-effects modeling, after fixed-effects hierarchical
modeling was performed to account for any hospital-level
variation, resident involvement was associated with higher
morbidity events only for the overall general and vascular
surgery cases (OR 1.07 95% CI 1.03 to 1.10).
After controlling for hospital teaching status and opera-
tive duration quartile, resident involvement was associated
with lower mortality for overall general and vascular pro-
cedures (OR 0.93; 95% CI 0.88 to 0.98), colorectal resec-
tions (OR 0.88; 95% CI 0.78 to 0.99), and abdominal
aortic aneurysm repair (OR 0.71; 95% CI 0.53 to 0.95).
After random-effects and fixed-effects hierarchical model-
ing was performed to account for any hospital-level varia-
tion, resident involvement was not associated with lower
Vol. xx, No. x, Month 2011
Figure 2. Risk-adjusted morbidity, risk-adjusted mortality, and mor-
tality rate after morbidity (“failure to rescue”) occurrence stratified
by resident involvement in general and vascular surgical procedures
in the ACS NSQIP. Gray bars, no resident; black bars, resident. ACS
NSQIP, American College of Surgeons National Surgical Quality
Improvement Program.
mortality for the various procedure groups studied (at sig-
nificance of p ⫽ 0.05). Fixed-effects modeling could not be
performed for pancreatectomy and esophagectomy, ab-
dominal aortic aneurysm repair, or laparoscopic cholecys-
tectomy and open inguinal hernia secondary to low event
Figure 3. Morbidity outcomes studied in the ACS NSQIP stratified by resident involvement for general and vascular
surgery procedures (unadjusted rates). Gray bars, no resident; black bars, resident. ACS NSQIP, American College
of Surgeons National Surgical Quality Improvement Program.
Raval et al Resident Involvement and Surgical Outcomes 5
rates and lack of model convergence. Of note, the mortality
rate among patients with at least 1 morbidity event was also
lower with resident involvement (11.0% vs 11.9%, p ⬍
0.001, Fig. 2).
Resident involvement was associated with higher unad-
justed complication rates when studying all ACS NSQIP
outcomes (Fig. 3). Common complications such as surgical
site infections were 21.2% higher when residents were in-
volved (5.34% vs 3.47%, p ⬍ 0.001). Less common com-
plications such as neurologic events including stroke and
coma were only 5.8% higher with resident involvement
(0.36% vs 0.32%, p ⫽ 0.017). Of the 69,215 patients
(11.4%) who had at least 1 complication, 7,792 died
(11.3%).
Quantifying resident influence
Risk standardized morbidity and mortality rates are sum-
marized in Table 2, derived from the results shared in Fig-
ure 1 and as described in the Methods section. When eval-
uating overall general and vascular surgery procedures,
there were approximately 6.1 additional patients with mor-
bidity events but 1.4 lives saved per 1,000 cases associated
6 Raval et al Resident Involvement and Surgical Outcomes J Am Coll Surg

Table 2. Risk-Standardized Morbidity and Mortality Rates for Procedures Studied Using the ACS NSQIP and Predicted
Increased or Decreased Events per 1,000 Patients with Resident Intraoperative Involvement
Risk adjusted odds Increased (or decreased) events per
Resident No ratio with resident 1,000 patients with resident
Morbidity and mortality rates involved, % resident, % involvement involvement
Risk standardized morbidity rate
Overall general and vascular surgery 11.50 10.89 1.063 6.1*
Pancreatectomy and esophagectomy 36.18 31.12 1.255 50.6*
Colorectal surgery 27.71 24.95 1.153 27.6*
Abdominal aortic aneurysm repair 35.11 36.21 0.953 ⫺11.0
Laparoscopic cholecystectomy/open
inguinal hernia repair 2.69 2.65 1.014 0.4
Risk standardized mortality rate
Overall general and vascular surgery 1.78 1.92 0.926 ⫺1.4*
Pancreatectomy and esophagectomy 2.42 2.68 0.903 ⫺2.5
Colorectal surgery 4.00 4.52 0.879 ⫺5.3*
Abdominal aortic aneurysm repair 9.74 13.24 0.707 ⫺35.0*
Laparoscopic cholecystectomy/open
inguinal hernia repair 0.29 0.27 1.092 0.2
*Significant at p ⬍ 0.05 in nonhierarchical logistic regression model.
ACS NSQIP, American College of Surgeons National Surgical Quality Improvement Program.

with resident involvement versus no resident involvement.
In parallel to the results in Figure 1, pancreatectomy and
esophagectomy procedures as well as colorectal opera-
tions demonstrated additional morbidity events with in-
traoperative resident involvement. Resident involve-
ment with colorectal procedures and abdominal aortic
aneurysm repair conferred fewer mortality events. Reit-
erating the results for evaluation of laparoscopic chole-
cystectomy and open inguinal hernia repair, resident
involvement had a nonsignificant and relatively minor
unfavorable effect on absolute numbers of morbidity
and mortality events.
DISCUSSION
Although residents represent only a single component of
the surgical health care team, several recent policy changes
have identified resident participation in the care of patients
as an area of focus for improving the quality of surgical care.
Furthermore, patients often request that residents not be
involved in their operations primarily due to concerns over
safety.8,9 Using ACS NSQIP data, we demonstrate that
resident intraoperative involvement is associated with
lower risk-adjusted mortality rates but slightly higher risk-
adjusted morbidity rates across a variety of general and
vascular surgery procedures. Although these findings were
significant after typical ACS NSQIP comorbidity risk ad-
justment, they were largely minimized after accounting for
hospital-level effects. Our study demonstrates no meaning-
ful clinical differences in surgical outcomes based on resi-
dent involvement.
The role of resident intraoperative involvement on sur-
gical outcomes has been evaluated in the past. Hwang and
colleagues28 used a single center’s experience in 2,293 pa-
tients to evaluate complications, mortality, and costs when
comparing cases with and without resident involvement.
They demonstrated no differences in complications or
mortality but higher costs and increased length of stays
when residents were involved in a variety of common gen-
eral surgery procedures including laparoscopic cholecystec-
tomy, bowel resection, and hernia repair. Our study did not
evaluate length of stay and other potential meaningful clin-
ical outcomes such as patient satisfaction, but focused on
overall morbidity and mortality.
There are 3 key mechanisms by which resident involve-
ment might be associated with increased morbidity. First,
intraoperative technical complications could be more com-
mon when residents are involved in cases and may repre-
sent a source of postoperative morbidity. These technical
issues might also manifest in longer operative times. Pro-
longed operative time has been associated with poor surgi-
cal outcomes in a variety of settings.29-31 Including opera-
tive time in our models moderated the overall effect of
resident intraoperative involvement, and resident involve-
ment was highly correlated with longer operative times for
the various procedures studied. These findings could reflect
an opportunity for quality improvement. As an example,
increased use of surgical simulator training might increase
surgical efficiency in the operating room.10 Alternately,
longer operative times might reflect something endoge-
Vol. xx, No. x, Month 2011
nous to the case but unobserved, which affects risk, a topic
discussed further below.
In addition, a key variable that is not assessed in our
study is the influence of the attending level surgeon in-
volved in the case. Our study accounts for teaching status at
the hospital level but does not take surgeon-level factors
into consideration. At this time, the ACS NSQIP does not
mandate collection of surgeon-specific identifiers preclud-
ing surgeon-level investigation. It is therefore possible that
the outcomes demonstrated might not reflect resident in-
volvement but rather that resident involvement influences
or is correlated with the quality of the surgeon performing
the surgery. However, this mechanism does not explain
how or why resident involvement would have opposite
effects on mortality versus morbidity.
A second, related mechanism to explain the association
between resident involvement and higher morbidity might
be unmeasured case-mix differences between procedures
involving residents and those with no resident involve-
ment. Inherently, resident involvement is higher at teach-
ing centers, which often serve as referral centers for com-
plex patients and procedures. Furthermore, even within a
teaching center, residents might be preferentially assigned
to risky or complex “teaching cases.” This could result in an
unobserved bias. There have been several studies addressing
variation in surgical outcomes at teaching and nonteaching
centers. Brennan and associates6 evaluated the epidemiol-
ogy of adverse events and found that teaching institutions
had 4 times the rates of adverse events compared with rural
hospitals in a sample of 31,000 New York hospitalizations.
Teaching hospitals have been shown to have improved sur-
gical outcomes in regard to specific processes such as
splenic salvage after trauma,32 but similar outcomes are
noted between teaching and nonteaching hospitals in other
surgical specialties including vascular surgery.33 Khuri and
coworkers31 performed a review of Veterans Affairs (VA)
NSQIP data and found that teaching hospitals performed
the vast majority of complex and high-risk procedures
within the VA system with comparable risk-adjusted 30-
day mortality, but cases performed in general surgery, or-
thopaedics, urology, and vascular surgery were associated
with higher risk-adjusted morbidity rates.31 Furthermore,
there was significant discrepancy noted between teaching
and nonteaching hospitals for length of stay depending on
the surgical procedure being performed. For example,
length of stay was longer in teaching hospitals after carotid
endarterectomy, but shorter after abdominal aortic aneu-
rysmectomy.34 Limitations to the VA study include the
definition of teaching versus nonteaching hospital based on
the presence of a dean’s committee overseeing resident is-
sues and at least 1 surgical resident performing surgery in at
Raval et al Resident Involvement and Surgical Outcomes 7
least 1 surgical specialty.31 Our study reaches similar con-
clusions about morbidity, but we believe this work adds
value with use of a more rigorous definition of teaching
hospital based on ratios of residents to beds at the hospital
level. Furthermore, ACS NSQIP hospitals represent a fun-
damentally different cohort of hospitals and patients in the
United States than those represented in the VA NSQIP
study. Whereas teaching hospitals performed roughly 80%
of the total cases in the VA NSQIP, approximately 66% of
cases submitted to the ACS NSQIP were from teaching
hospitals. The majority of hospitals contributing to the
ACS NSQIP are larger, high-volume centers, with 76% of
current hospitals having 300 or more beds. This sampling
of institutions may limit the generalizability of our results
but reduces variation in institutional volumes. Further-
more, Khuri and colleagues31 used a 5-point scale to esti-
mate the complexity of each operation; our study accounts
for case-mix more precisely using a data-driven CPT
grouping risk-score. By representing a more diverse selec-
tion of private sector institutions, strictly defining teaching
status, and using a procedural grouping risk-score, our
study provides a unique and more developed perspective to
evaluate the specific role of resident intraoperative involve-
ment. Despite these intensified efforts to account for dif-
ferences in case-mix, there are still unrecorded procedure-
specific variables that could influence outcomes. Examples
include anatomic location, size, and tortuosity of aortic
aneurysms being repaired or resection of an initial as op-
posed to recurrent cancer. This level of procedure-specific
data is not currently available in the ACS NSQIP and
is being addressed in future program research and
development.
A third mechanism to explain the association between
resident involvement and increased morbidity may be the
increased vigilance toward identifying and recording post-
operative complications and occurrences by residents or in
teaching facilities. This increased clinical vigilance could
even be based on the ordering of additional tests such as
surveillance duplex studies for suspicion of deep vein
thromboses or urinalysis for asymptomatic urinary tract
infections. Residents have been shown to overuse labora-
tory tests and imaging studies based on experience.35,36
Although a combination of intraoperative technical is-
sues, unmeasured case-mix variation, and increased resi-
dent vigilance may partially explain the observed increased
morbidity associated with resident involvement, resident
involvement was also associated with lower mortality. An
increased level of surveillance could assist in the early iden-
tification of complications, leading to prompt intervention
and “rescue” from further adverse developments. The con-
cept of increased surveillance is also supported by the pres-
8 Raval et al Resident Involvement and Surgical Outcomes
ence of higher event rates across all ACS NSQIP morbidity
outcomes (Fig. 1) as well as the fact that morbidity detec-
tion is potentially more variable than mortality detection.
Several studies have demonstrated that early complication
recognition may provide an opportunity to rescue patients
from mortality at teaching centers and with resident in-
volvement.34,37 Our study supports these findings with a
slightly lower mortality rate detected among patients with
at least 1 morbidity event when residents were involved in
the surgery. Rosenthal and associates38 compared 89,851
patients at 30 hospitals in northeast Ohio and found a 19%
lower in-hospital adjusted mortality and 9% lower risk-
adjusted length of stay at teaching hospitals.38 Although
traditional ACS NSQIP modeling does not account for
hospital teaching status, our study evaluated resident intra-
operative involvement after accounting for hospital teach-
ing status. Our data include hospitals from 38 states and
may be more generalizable than either of the reported New
York or Ohio state-wide experiences. Dimick and coau-
thors39 used the Nationwide Inpatient Sample to evaluate
the influence of hospital teaching status for several high risk
procedures including pancreatic, hepatic, and esophageal
resection. That study identified lower unadjusted mortality
rates at teaching hospitals for patients undergoing pancre-
atic and hepatic resections and prolonged length of stay at
nonteaching centers.39 Surgical outcomes for procedures
such as pancreatic, hepatic, and esophageal resections also
have a well documented relationship with volume, with
academic centers often having higher volumes and there-
fore better outcomes.40 Conversely, more common proce-
dures such as colon resections, for which a nonteaching
center may have higher volumes, have been shown to have
lower mortality at nonteaching centers.41 Our study in-
cludes all general and vascular surgical cases sampled
within the ACS NSQIP program including both a spec-
trum of high risk procedures and lower risk, elective, out-
patient procedures.
There are several limitations to this study worth noting.
Lack of surgeon-level information and potential for unob-
served bias have been mentioned. In addition, hospitals
represented in the ACS NSQIP may not be representative
of all hospitals performing surgery in the United States.
Further, risk-adjusted models of morbidity are currently
limited to outcomes being monitored within the ACS
NSQIP. Future studies may delve into the specific morbid-
ity events more commonly associated with resident intra-
operative involvement such as surgical site infections
(which our study notes as being influenced by resident
involvement) or specific procedure-related outcomes such
as long-term inguinal hernia repair recurrence, which has
been previously shown to correlate with junior resident
J Am Coll Surg
involvement.42 Also, we were able to identify resident in-
volvement in the operating room, but the extent to which
residents were involved with the preoperative and postop-
erative care of patients is not quantified and measured.
There are several limitations in ACS NSQIP data collec-
tion that prevent consistent identification of fellow level
trainees as compared with residents, and determination of
when more senior residents may be in a teaching role for
junior level residents, precluding reliable stratification of
trainees by postgraduate years. Finally, we reiterate that no
risk adjustment algorithm is perfect, and the results could
yet reflect inherent unobserved factors that influence risk
and are in some fashion correlated with resident involve-
ment. This is a topic of ongoing investigation.
CONCLUSIONS
Resident intraoperative participation was associated with a
slightly increased morbidity and decreased mortality across
a variety of general and vascular surgery procedures after
taking hospital teaching status as well as operative time into
consideration. These findings were most evident when
studying complex procedures and least evident when eval-
uating low risk procedures such as laparoscopic cholecys-
tectomy and open inguinal hernia repair. The higher mor-
bidity observed might be explained by intraoperative
technical issues, unmeasured case-mix variation, or in-
creased resident vigilance. The lower mortality observed
with resident involvement could reflect surveillance and
rescue functions indicative of a benefit to resident involve-
ment across a variety of general and vascular surgery pro-
cedures. Many of the differences in outcomes were mini-
mized once more robust analyses accounting for hospital-
level variation were used. Ultimately, there appear to be no
major, clinically significant differences in surgical out-
comes based on resident involvement and patients and
other stakeholders can be reassured that resident involve-
ment in surgical care is safe.
Author Contributions
Study conception and design: Raval, Wang, Cohen, Ingra-
ham, Dimick, Flynn, Hall, Ko
Acquisition of data: Raval, Wang, Cohen, Ko
Analysis and interpretation of data: Raval, Wang, Cohen,
Ingraham, Bentrem, Dimick, Flynn, Hall, Ko
Drafting of manuscript: Raval, Wang, Cohen, Ingraham,
Dimick, Hall, Ko
Critical revision: Raval, Wang, Cohen, Ingraham, Ben-
trem, Dimick, Flynn, Hall, Ko
Acknowledgment: The authors thank the ACS NSQIP staff
and the diligent efforts of trained data collectors for providing
Vol. xx, No. x, Month 2011
high quality data for research endeavors aimed at improving
the care of the surgical patient.
REFERENCES
1. Ayanian JZ, Weissman JS. Teaching hospitals and quality of
care: a review of the literature. Milbank Q 2002;80:569–593, v.
2. U.S. News and World Report. America’s best hospitals. Avail-
able at: http://health.usnews.com/health/best-hospitals. Ac-
cessed February 4, 2011.
3. The Leapfrog Group. Evidence-based hospital referral. Available at:
http://www.leapfroggroup.org/media/file/Leapfrog-Evidence-
Based_Hospital_Referral_Fact_Sheet.pdf. Accessed February 4,
2011.
4. Curet MJ. Resident work hour restrictions: where are we now?
J Am Coll Surg 2008;207:767–776.
5. Englesbe MJ, Pelletier SJ, Magee JC, et al. Seasonal variation in
surgical outcomes as measured by the American College of
Surgeons-National Surgical Quality Improvement Program
(ACS-NSQIP). Ann Surg 2007;246:456–462; discussion 463–
465.
6. Brennan TA, Hebert LE, Laird NM, et al. Hospital characteris-
tics associated with adverse events and substandard care. JAMA
1991;265:3265–3269.
7. Englesbe MJ, Fan Z, Baser O, Birkmeyer JD. Mortality in Medi-
care patients undergoing surgery in July in teaching hospitals.
Ann Surg 2009;249:871–876.
8. Cowles RA, Moyer CA, Sonnad SS, et al. Doctor-patient com-
munication in surgery: attitudes and expectations of general
surgery patients about the involvement and education of surgi-
cal residents. J Am Coll Surg 2001;193:73–80.
9. Dutta S, Dunnington G, Blanchard MC, et al. “And doctor, no
residents please!” J Am Coll Surg 2003;197:1012–1027.
10. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR. Virtual
reality training for surgical trainees in laparoscopic surgery. Co-
chrane Database Syst Rev 2009(1):CD006575.
11. Khuri SF, Daley J, Henderson W, et al. The Department of
Veterans Affairs’ NSQIP: the first national, validated, outcome-
based, risk-adjusted, and peer-controlled program for the mea-
surement and enhancement of the quality of surgical care. Na-
tional VA Surgical Quality Improvement Program. Ann Surg
1998;228:491–507.
12. Khuri SF, Daley J, Henderson W, et al. Risk adjustment of the
postoperative mortality rate for the comparative assessment of
the quality of surgical care: results of the National Veterans Af-
fairs Surgical Risk Study. J Am Coll Surg 1997;185:315–327.
13. Fink AS, Campbell DA Jr, Mentzer RM Jr, et al. The National
Surgical Quality Improvement Program in non-Veterans Ad-
ministration hospitals: initial demonstration of feasibility. Ann
Surg 2002;236:344–353; discussion 353– 354.
14. Khuri SF, Henderson WG, Daley J, et al. Successful implemen-
tation of the Department of Veterans Affairs’ National Surgical
Quality Improvement Program in the private sector: the Patient
Safety in Surgery study. Ann Surg 2008;248:329–336.
15. Hall BL, Hamilton BH, Richards K, et al. Does surgical quality
improve in the American College of Surgeons National Surgical
Quality Improvement Program: an evaluation of all participat-
ing hospitals. Ann Surg 2009;250:363–376.
16. Rowell KS, Turrentine FE, Hutter MM, et al. Use of National
Surgical Quality Improvement Program data as a catalyst for
quality improvement. J Am Coll Surg 2007;204:1293–1300.
Raval et al Resident Involvement and Surgical Outcomes 9
17. Khuri SF. The NSQIP: a new frontier in surgery. Surgery 2005;
138:837–843.
18. Daley J, Khuri SF, Henderson W, et al. Risk adjustment of the
postoperative morbidity rate for the comparative assessment of
the quality of surgical care: results of the National Veterans Af-
fairs Surgical Risk Study. J Am Coll Surg 1997;185:328–340.
19. Khuri SF, Daley J, Henderson W, et al. The National Veterans
Administration Surgical Risk Study: risk adjustment for the
comparative assessment of the quality of surgical care. J Am Coll
Surg 1995;180:519–531.
20. Khuri SF, Henderson WG. The Patient Safety in Surgery study.
J Am Coll Surg 2007;204:1087–1088.
21. Khuri SF, Henderson WG, Daley J, et al. The Patient Safety in
Surgery study: background, study design, and patient popula-
tions. J Am Coll Surg 2007;204:1089–1102.
22. ACS-NSQIP. Surgical clinical nurse reviewer training. Available
at: http://acsnsqip.org/main/program_nurse_training.asp. Ac-
cessed February 4, 2011.
23. ACS NSQIP data collection overview. Available at: http://
acsnsqip.org/main/program_data_collection.asp.Accessed Feb-
ruary 4, 2011.
24. Current Procedural Terminology, standard edition. Chicago, IL:
American Medical Association; 2009.
25. Hall BL, Hsiao EY, Majercik S, et al. The impact of surgeon
specialization on patient mortality: examination of a continuous
Herfindahl-Hirschman index. Ann Surg 2009;249:708–716.
26. AHA Data. Survey history and methodology. Available at: http://
www.ahadata.com/ahadata/html/historymethodology.html, 2009.
Accessed February 4, 2011.
27. Iezzoni LI, Shwartz M, Ash AS, et al. Risk adjustment meth-
ods can affect perceptions of outcomes. Am J Med Qual
1994;9:43–48.
28. Hwang CS, Pagano CR, Wichterman KA, et al. Resident versus
no resident: a single institutional study on operative complica-
tions, mortality, and cost. Surgery 2008;144:339–344.
29. Haridas M, Malangoni MA. Predictive factors for surgical site
infection in general surgery. Surgery 2008;144:496–501; dis-
cussion 501–503.
30. Doenst T, Borger MA, Weisel RD, et al. Relation between aortic
cross-clamp time and mortality–not as straightforward as ex-
pected. Eur J Cardiothorac Surg 2008;33:660–665.
31. Khuri SF, Najjar SF, Daley J, et al. Comparison of surgical out-
comes between teaching and nonteaching hospitals in the De-
partment of Veterans Affairs. Ann Surg 2001;234:370–382; dis-
cussion 382–383.
32. Todd SR, Arthur M, Newgard C, et al. Hospital factors associ-
ated with splenectomy for splenic injury: a national perspective.
J Trauma 2004;57:1065–1071.
33. Pearce WH, Parker MA, Feinglass J, et al. The importance of sur-
geon volume and training in outcomes for vascular surgical proce-
dures. J Vasc Surg 1999;29:768–776; discussion 777⫺778.
34. Ghaferi AA, Birkmeyer JD, Dimick JB. Variation in hospital
mortality associated with inpatient surgery. N Engl J Med 2009;
361:1368–1375.
35. Griffith CH 3rd, Wilson JF, Desai NS, Rich EC. Does pediatric
housestaff experience influence tests ordered for infants in the
neonatal intensive care unit? Crit Care Med 1997;25:704–709.
36. Khaliq AA, Huang CY, Ganti AK, et al. Comparison of resource
utilization and clinical outcomes between teaching and non-
teaching medical services. J Hosp Med 2007;2:150–157.
37. Ghaferi AA, Birkmeyer JD, Dimick JB. Complications, failure
10 Raval et al Resident Involvement and Surgical Outcomes
to rescue, and mortality with major inpatient surgery in
Medicare patients. Ann Surg 2009;250:1029–1034.
38. Rosenthal GE, Harper DL, Quinn LM, Cooper GS. Severity-
adjusted mortality and length of stay in teaching and nonteach-
ing hospitals. Results of a regional study. JAMA 1997;278:485–
490.
39. Dimick JB, Cowan JA Jr, Colletti LM, Upchurch GR Jr. Hos-
pital teaching status and outcomes of complex surgical proce-
dures in the United States. Arch Surg 2004;139:137–141.
40. Begg CB, Cramer LD, Hoskins WJ, Brennan MF. Impact of
J Am Coll Surg
hospital volume on operative mortality for major cancer surgery.
JAMA 1998;280:1747–1751.
41. Hayanga AJ, Mukherjee D, Chang D, et al. Teaching hospital
status and operative mortality in the United States: tipping point
in the volume-outcome relationship following colon resections?
Arch Surg 145:346–350.
42. Wilkiemeyer M, Pappas TN, Giobbie-Hurder A, et al. Does
resident post graduate year influence the outcomes of inguinal
hernia repair? Ann Surg 2005;241:879–882; discussion 882–
884.