SURGICAL PERSPECTIVE
A New Era in Graduate Medical Education
A Novel Role for Surgical Subinternships in the Interview Selection Process
Robert Naples, DO,  yY Judith C. French, PhD,  y
Jonah D. Thomas, MS,y and Jeremy M. Lipman, MD, MHPE  y
Downloaded from https://journals.lww.com/annalsofsurgery by 01UGrXh3ipqzR4DKqW7bOJtSxsRVOheLV9OzOeHq2POX2t1GrKUD6m1aBQdlm0lX7bZxxCFBpuxAM3exuxdXARVaPXROhrEIELxLuiE5dAoxEXL1EbBXOmtN5qOe2K0cfOG+czRLAQJzYcB+sBMdZA== on 03/16/2021
Keywords: acting internship, clerkship, interview selection process,
subinternship, surgical boot camp, USMLE
(Ann Surg 2021;273:e16–e18)
D ata from a 2018 program director survey shows that general
surgery residency programs receive an average of 742 appli-
cations with only 36% receiving an in-depth review.1 Historically, a
minimum USMLE Step 1 score has been used as a cutoff to quickly
reduce the applicant pool to a manageable number to review for
potential interview.1 This uses the USMLE in a way that was not
originally intended. Because these scores negatively impact medical
student well-being and have innate biases, a change to a pass/fail
system was announced for USMLE Step 1 scoring.2 This forces the
applicant screening and interview selection process to evolve. The
development of a standardized assessment for surgical subintern-
ships can serve as a practical and efficient method to streamline the
identification of applicants appropriate for a specific surgical
training program.
SHIFTING THE PROBLEM
Although the Step 1 score will be reported as pass/fail, a 3-
digit numeric score will continue to be reported for Step 2 clinical
knowledge (CK).2 Currently, Step 1 is more heavily utilized than
Step 2 CK in the screening process.1,3 Although pass/fail grading can
have significant benefits to the well-being of students by de-empha-
sizing the impact of USMLE Step 1 scores on residency options, it
eliminates the most simplistic method for programs to reduce the
large number of applications received.4 Therefore, a shift toward
utilizing Step 2 CK scores could occur to keep the screening process
simple. Additionally, similar to Step 1, Step 2 CK has been shown to
correlate with the American Board of Surgery In-training Examina-
tion scores, which positively correlate with the American Board of
Surgery pass rates.5,6 However, this again misuses a licensing exam
to stratify applicants. This calls for surgical educators to find a
pragmatic solution now, irrespective of the changes to the
USMLE scoring.
INADEQUATE SOLUTIONS
Surgery clerkship performance could provide a robust replace-
ment for USMLE scores because they are based on a longitudinal
assessment rather than a snapshot in time. However, there are
limitations to this method. First, surgery clerkships are designed
From the Department of General Surgery, Cleveland Clinic, Cleveland, Ohio; and
yCleveland Clinic Lerner College of Medicine of Case Western Reserve
University, Cleveland, Ohio.
naplesr2@ccf.org.
The authors declare no conflicts of interest
Copyright ß 2020 Wolters Kluwer Health, Inc. All rights reserved.
ISSN: 0003-4932/20/27301-0e16
DOI: 10.1097/SLA.0000000000004042
e16 | www.annalsofsurgery.com
Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.
to provide basic surgical knowledge and skills to all medical students
regardless of specialty interests. Second, studies have demonstrated
substantial variation and inflation in medical school grading sys-
tems.7 Because there is no standardization, the comparison of
students across, or even within, institutions based on surgery clerk-
ship grades is problematic.
The lack of data regarding the grading system distribution for
each medical school is even more troublesome. In 2016, the Associ-
ation of American Medical Colleges Medical Student Performance
Evaluation (MSPE) Task Force made recommendations to revise the
MSPE, which included ‘‘a level of standardization and transparency
that facilitates the residency selection process’’ and ‘‘comparative
information on applicants.’’8 However, a recent study showed MSPE
reviewers thought this still required significant improvement.9 The
findings highlight the dichotomy between undergraduate medical
education (UME) and graduate medical education. UME has a
responsibility to help students match into the ‘‘best’’ residency
programs. Therefore, with increased transparency, students could
suffer if their school’s standards are not on par with other medical
schools or the revealed grading system is found to be skewed.
However, this information would be extremely valuable to residency
programs in their selection process. Due to this current lack of UME
transparency, programs must sift through the coded messages buried
in applications to discover those they think are the top applicants for
their residency. A unified grading system would help eliminate this
code so that residency programs can make more informed decisions.
Yet, this seems unlikely given the wide variability in curriculum
structure and assessment between institutions.
USMLE Step 2 clinical skills scores and standardized letters
of recommendation (SLOR) could be substitutes for the USMLE
Step 1 score. Although these may potentially have advantages in
other specialties, general surgery programs would likely not experi-
ence the intended benefits. Patient care scenarios utilized on Step 2
clinical skills are not primarily surgery-specific and emphasize
preventative care, weakening the argument for its use within general
surgery. The American College of Surgeons and Association for
Program Directors in Surgery created a SLOR for general surgery.10
However, there is a lack of widespread acceptance. This highlights
the limitations of SLORs and diminishes its applicability in the
general surgery residency selection process.
Surgical residency preparatory courses (‘‘boot camps’’) are
another potential consideration. Students matching into surgical resi-
dency can participate in these programs and undergo a rigorous and
immersive curriculum that trains them in the necessary skills and
knowledge required for a postgraduate year (PGY)-1 resident. However,
these courses tend to occur in a simulated environment after a student has
matched, thereby providing no value in the selection process.
NOVEL ROLE FOR SURGICAL SUBINTERNSHIPS
One area of untapped potential in general surgery is the
utilization of subinternships (also known as ‘‘acting internships’’),
Annals of Surgery  Volume 273, Number 1, January 2021
Annals of Surgery  Volume 273, Number 1, January 2021 Novel Role for Surgical Subinternships

where students are placed in the role of a PGY-1 resident. The
majority of subinternships are completed before submitting resi-
dency applications, allowing them to be utilized for the selection of
interviewees. Moreover, students can be evaluated on their current
aptitude, offering better insight to their potential in residency. By
identifying specific areas of strength, applicant and program com-
patibility can be established. We propose that subinternships become
a nationally standardized, performance-based process for screening
and comparing potential general surgery residency candidates. We
recognize that the subinternship experience varies at each institution
and reflects the program’s unique values. This component is vital for
applicants to assess their own compatibility with the program.
Therefore, we are not proposing a standardized experience but rather
a standardized evaluation, as subinternships provide students with the
opportunity to be assessed on clinical acumen, basic surgical skills,
and professionalism in the true surgical environment. In this sense,
programs would be able to utilize subinternships as objective mea-
sures for identifying applicants that best fit their residency, exclusive
of other factors. Although bias in completing these evaluations is a
risk, this would be mitigated by standardization.
Systems such as this seem to be effective in other elite selection
processes, such as those used by the Navy Sea, Air, and Land Forces
(SEALs) – basic underwater demolition/SEAL (BUD/S) training. In
BUD/S training, the personnel are placed under the extreme physical
conditions and evaluated in the true environment typically required of
this job. By all accounts, this is exceptionally advantageous for those
involved in the selection process and training, as those candidates who
cannot adequately perform the tasks required of the job are not advanced.
This benefits the trainees, as they recognize they are not appropriately
matched for the position, and the program because they only select those
who can carry out the required responsibilities. In this fashion,
subinternships could help provide students with objective feedback that
is needed in this process to help create compatibility between students
and programs. Although students can only complete a small number of
subinternships before submitting residency applications, this standard-
ized assessment should translate to all programs. Moreover, a rationale
can be presented for students to complete these subinternships at
different institutions (‘‘away rotations’’) to reduce evaluator bias, as
outside faculty would not have incentives to inflate their abilities.
However, with standardization, students will be afforded the opportunity
to complete subinternships at their home institution, alleviating potential
financial concerns or scheduling restrictions. We recognize that BUD/S
training has its own inherent bias and is an extreme example. Further-
more, while surgeons can work in many different environments, such as
private, academic, government, or military institutions, BUD/S only
trains and selects for military personnel. Yet, the Navy SEAL in-situ
selection process through the use of BUD/S training could establish a
precedent for the role of subinternships in the comparative process for
general surgery applicants and produce similar results in regards to
student evaluation and program fit. We are not advocating to place these
students under undue stress. Instead, we are simply calling for an honest
assessment of students’ performance with a standardized grading system
to better determine their projected career path in general surgery. If a
standardized assessment system within surgical subinternships is devel-
oped, it holds much promise in reliably evaluating students applying to
general surgery residency and could play a significant role in the
interview selection process.
We propose that students completing surgical subinternships
be assessed on 3 core standards: clinical practice, surgical practice,
and professionalism (Fig. 1). These standards are fundamental in
trainee assessment within medical school and residency. In this
respect, we can bridge the gap between UME and graduate medical

FIGURE 1. Proposed surgical subinternship core standards. These are the proposed core standards on which to assess students
during their surgical subinternship. They are based on an overlap between the Association of American Medical Colleges (AAMC)
Core EPAs for Entering Residency and the ACGME Surgery Residency Level 1 Milestones 2.0.

ß 2020 Wolters Kluwer Health, Inc. All rights reserved. www.annalsofsurgery.com | e17

Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.

Naples et al
education, providing the transparency needed to critically assess
medical students’ achievement of core entrustable professional
activities for entering residency and potentially predicting their
aptitude as a future PGY-1 surgical resident. These 3 standards
can then be incorporated into an assessment system that can be
implemented by all institutions. This would unify the evaluation
process of medical students applying to general surgery residency
and aid programs in identifying well-suited applicants that warrant an
in-depth review for interview. It is our hope that this perspective can
stimulate the conversation regarding surgical subinternships and
their potential role as a meaningful entity that can benefit both
general surgery residency applicants and programs.
REFERENCES
1. National Residency Matching Program. Results of the 2018 NRMP Program
Director Survey. Published June 2018. Available at: http://www.nrmp.org/wp-
content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-
WWW.pdf. Accessed January 13, 2020.
2. United States Medical Licensing Examination (USMLE). Change to Pass/Fail
Score Reporting for Step 1. 2020. Available at: https://www.usmle.org/incus/
#decision. Accessed March 18, 2020.
3. Schrock JB, Kraeutler MJ, Dayton MR, et al. A cross-sectional analysis of
minimum USMLE step 1 and 2 criteria used by orthopaedic surgery residency
e18 | www.annalsofsurgery.com
Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.
Annals of Surgery  Volume 273, Number 1, January 2021
programs in screening residency applications. J Am Acad Orthop Surg.
2017;25:464–468.
4. Spring L, Robillard D, Gehlbach L, et al. Impact of pass/fail grading on
medical students’ well-being and academic outcomes. Med Educ.
2011;45:867–877.
5. Spurlock DR Jr, Holden C, Hartranft T. Using United States medical
licensing examination (USMLE) examination results to predict later in-training
examination performance among general surgery residents. J Surg Educ.
2010;67:462–466.
6. Shellito JL, Osland JS, Helmer SD, et al. American Board of Surgery
examinations: can we identify surgery residency applicants and residents who
will pass the examinations on the first attempt? Am J Surg. 2010;199:216–222.
7. Fagan R, Harkin E, Wu K, et al. The lack of standardization of allopathic and
osteopathic medical school grading systems and transcripts. J Surg Educ.
2019. doi:10.1016/j.jsurg.2019.06.016. In press.
8. Association of American Medical Colleges (AAMC). Recommendations for
revising the Medical Student Performance Evaluation (MSPE). 2017. Avail-
able at: https://www.aamc.org/download/470400/data/mspe-recommenda-
tions.pdf. Accessed November 18, 2019.
9. Brenner JM, Arayssi T, Conigliaro RL, et al. The revised medical school
performance evaluation: does it meet the need of its readers? J Grad Med
Educ. 2019;11:475–478.
10. American College of Surgeons. Surgery Standardized Letter of Recommen-
dation. 2018. Available at: https://www.facs.org//media/files/about%20acs/
governors/5surgerystandletrec.ashx. Accessed November 5, 2019.
ß 2020 Wolters Kluwer Health, Inc. All rights reserved.