Research Report
Competency Crosswalk: A Multispecialty Review of
the Accreditation Council for Graduate Medical
Education Milestones Across Four Competency Domains
Laura Edgar, EdD, MBA, Sydney Roberts, Nicholas A. Yaghmour, MPP,
Andrea Leep Hunderfund, MD, MHPE, Stanley J. Hamstra, PhD, Lisa Conforti, MPH,
and Eric S. Holmboe, MD
Abstract
Purpose
To identify common and overlapping
themes among the interpersonal and
communication skills (ICS), practice-
based learning and improvement (PBLI),
professionalism (PROF), and systems-
based practice (SBP) milestones of the
transitional year and 26 specialties.
Method
In May 2017, milestones were accessed
from the Accreditation Council for
Graduate Medical Education specialties
website. A thematic analysis of the
ICS, PBLI, PROF, and SBP milestones
was performed to determine unique
and common themes across these
T he Accreditation Council for Graduate
Medical Education (ACGME) began to
move to an outcomes-based accreditation
framework with the creation of the six
core competency domains in 1999.1 The
launch of the Outcomes Project in 2001
was the first effort to incorporate these
domains into educational practice.2 Early
on, programs struggled to implement
the competencies into their curricula
and assessment systems.3–6 In 2009, the
first set of milestones was published
in internal medicine as a possible
framework to help programs implement
the competencies more effectively.7,8
Others also found this developmental
model useful in training.9 On the basis
Please see the end of this article for information
about the authors.
Correspondence should be addressed to Laura
Edgar, 401 N. Michigan Ave., Suite 2000, Chicago, IL
60611; telephone: (312) 755-5029; e-mail: ledgar@
acgme.org.
Acad Med. 2018;93:1035–1041.
First published online November 21, 2017
doi: 10.1097/ACM.0000000000002059
Copyright © 2017 by the Association of American
Medical Colleges
Supplemental digital content for this article is
available at http://links.lww.com/ACADMED/A511.
Academic Medicine, Vol. 93, No. 7 / July 2018
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
competencies and across specialties.
Keywords from the common program
requirements were initially applied as
codes to the milestones. Codes were
then grouped into common themes.
Results
Twenty-two themes were identified: 15
(68%) were unique to a given competency
(3 related to ICS, 4 related to PBLI, 5
related to PROF, and 3 related to SBP), and
7 (32%) appeared in the milestones of
more than one core competency. Eleven
themes (50%) were used by 20 or more
specialties, and 6 themes (27%) by 10 or
fewer specialties. No theme was present
across all specialties.
of these early experiences, milestones
became the outcomes-based assessment
framework for all specialties accredited
by the ACGME in July 2013 as part of the
Next Accreditation System.1
Each specialty, with support from the
ACGME and the American Board of
Medical Specialties (ABMS), coproduced
its own milestones. Specialty working
groups included program directors,
faculty, residents, and other stakeholders
(e.g., medical educators, nurses,
allied health professionals). These
working groups first determined the
subcompetencies most relevant to the
education of residents in their respective
specialties. Working groups then used
program requirements, certification
examination outlines, curricula, national
competency statements, literature
reviews, and results from national
consensus-building exercises to guide the
milestone development process.10,11
Each specialty independently created
milestones for subcompetencies within
each of the six core competencies:
medical knowledge (MK), patient care
(PC), interpersonal and communication
Conclusions
The ICS, PBLI, PROF, and SBP milestones
contain multiple themes with areas of
overlap among these four competencies
and substantial variability across specialties.
This variability may create differential
expectations of residents across specialties,
complicate faculty development, and
make sharing assessment tools difficult.
The thematic analysis provides important
insights into how individual specialties
interpret and operationalize the ICS, PBLI,
PROF, and SBP competency domains and
can inform future revisions of milestones
to enable harmonization and shared
understanding of these competencies
across specialties where appropriate.
skills (ICS), practice-based learning and
improvement (PBLI), professionalism
(PROF), and systems-based practice
(SBP). The rationale for this flexibility
was to create a sense of ownership
and engagement in the milestone
development process and to account for
differences among specialties. During
the design phase, working groups
recognized that specialty milestones
would be reviewed periodically using
current documents (e.g., specialty-
specific competency guidelines, program
requirements, ABMS and American
Osteopathic Association Board
certification documents), research, and
surveys of specialty communities to
ensure that the milestones continue to
meet the needs of residents, training
programs, and society.
As part of the initial rollout of the
milestones, ACGME milestones staff
(L.E., S.J.H., and E.S.H.) also routinely
collected feedback on early stakeholder
experiences with the milestones.12,13 This
was accomplished by conducting focus
groups with program directors, faculty,
designated institutional officials, and
residents at conferences, specialty society
1035
 Research Report
meetings, and institutional visits. These
stakeholders frequently commented
on the high degree of variability across
specialties and overlap in content across
core competency domains. The variability
and overlap complicated efforts to
share assessment tools across programs
and provide comprehensive faculty
development across specialties.
In undertaking this study, we aimed
to identify common and overlapping
themes among the ICS, PBLI, PROF,
and SBP milestones of the transitional
year and 26 core specialties. Identifying
areas of overlap across competencies will
allow the medical education community
to organize themes more consistently
within the core competencies framework
(e.g., whether milestones related to care
coordination should be placed under
ICS or SBP). Identifying areas of overlap
across specialties will highlight potential
opportunities for convergence in order
to reduce variability in milestone content
across training programs. We focused
on the four crosscutting competencies
(ICS, PBLI, PROF, and SBP) because
they are less specialty specific than MK
and PC milestones and hence may offer
more opportunities for meaningful
convergence.
Method
Data source
Our data source was the ICS, PBLI,
PROF, and SBP milestones for the
transitional year and the 26 core
specialties (those leading to an ABMS
member board primary certification).14
Transitional year programs do not
represent a core specialty but provide
education in multiple clinical disciplines
to prepare residents for a specific
specialty. The 26 core specialties were
allergy and immunology; anesthesiology;
colon and rectal surgery; dermatology;
emergency medicine; family medicine;
internal medicine; medical genetics
and genomics; neurological surgery;
neurology; nuclear medicine;
obstetrics–gynecology; ophthalmology;
orthopedic surgery; otolaryngology;
pathology; pediatrics; physical medicine
and rehabilitation; plastic surgery;  
preventive medicine; psychiatry;
radiation oncology; radiology; surgery;
thoracic surgery; and urology. In May
2017, milestones were accessed from the
ACGME specialties website.14
1036
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
Qualitative analysis initial guides to coding.18 Through
We analyzed these milestones using multiple rounds of analysis, the authors
qualitative thematic analysis for several reconciled differences, developed a
reasons.15 First, as described by Braun and final list of codes, and combined codes
Clarke,16 thematic analysis is “a method into groups sharing a common theme
for identifying, analysing and reporting (overarching ideas or concepts that
patterns (themes) within data.” Second, appeared frequently in the milestones of
this study is using existing narratives one or more competency domains).
previously developed by groups through an
iterative process. As such, these narratives To assess the reproducibility of this
constitute the source of the “interview” analysis, two different authors (A.L.H.
data for analysis, unlike a typical qualitative and L.C.) independently applied the
study. Third, although content analysis and coding taxonomy to the milestones of
thematic analysis share many similarities, seven specialties (representing select
thematic analysis typically involves a surgical, medical, and hospital-based
deeper level of interpretation of the data programs). The codes assigned by these
(i.e., both deductive and inductive), a two authors matched those of the initial
necessary step in this project because of coders, supporting the validity and
the complexity of the milestone language reproducibility of the coding scheme.
across the 27 different sets.17
Two members of the study team (L.E. Results
and S.R.) first conducted coding of all Overall, we identified 22 themes among the
ICS, PBLI, PROF, and SBP milestones, ICS, PBLI, PROF, and SBP milestones of the
using keywords derived from the ACGME transitional year and the 26 core specialties.
Common Program Requirements as Fifteen themes (68%) appeared only in
Table 1
Thematic Analysis Results: Unique Themes Among ICS, PBLI, PROF, and SBP
Milestones From the Transitional Year and the 26 Core Specialties, 2017a
Relevant core No. of No. of specialties
competency milestones with milestones
related to this related to this
Theme ICS PBLI PROF SBP theme theme
1: Communication with X 73 24
patients
2: Communication with X 65 21
teams
3: Informed consent X 6 5
4: Self-directed learning X 88 25
5: Evidence-based patient X 20 15
care
6: Scholarly activity X 8 5
7: Teaching X 18 10
8: Attitudes and behaviors X 75 24
9: Accountability X 58 24
10: Ethics X 27 20
11: Limits X 22 17
12: Administrative tasks X 19 12
13: Health economics X 44 22
14: Health care delivery X 13 8
settings
15: Community X 7 2
 Abbreviations: ICS indicates interpersonal and communication skills; PBLI, practice-based learning and
improvement; PROF, professionalism; SBP, systems-based practice.
a
Allergy and immunology, anesthesiology, colon and rectal surgery, dermatology, emergency medicine, family
medicine, internal medicine, medical genetics and genomics, neurological surgery, neurology, nuclear medicine,
obstetrics–gynecology, ophthalmology, orthopedic surgery, otolaryngology, pathology, pediatrics, physical
medicine and rehabilitation, plastic surgery, preventive medicine, psychiatry, radiation oncology, radiology,
surgery, thoracic surgery, urology (accessed May 4, 2017, from http://www.acgme.org/Specialties14).
Academic Medicine, Vol. 93, No. 7 / July 2018
 Research Report

milestones related to one of the four core or educational gap analysis, journal club wellness (physical, emotional, and
competencies (3 related to ICS, 4 related participation, creation and alteration mental), and patient advocacy. The third
to PBLI, 5 related to PROF, and 3 related of learning plans, self-assessment, and unique PROF theme was “ethics,” which
to SBP) and hence were unique to a given literature reviews. The second unique PBLI incorporated general ethical behavior
competency (Table 1). The remaining 7 theme was “evidence-based patient care,” and recognition of ethical violations
themes (32%) appeared in the milestones of which focused on the use of evidence- such as conflict of interest or boundary
more than one core competency (Table 2). based medicine (EBM), scholarly activity breaches. The remaining two unique PROF
We describe each of these 22 themes in with EBM (e.g., applying knowledge), or themes were “limits” and “administrative
more detail below. Chart 1 demonstrates the use of national guidelines and often tasks.” “Limits” incorporated recognizing
which themes were included in each set of related directly to a program requirement personal limits and seeking help, whereas
milestones, and Table 3 provides a specific within the specialty. The third unique “administrative tasks” included topics
example of each theme. PBLI theme was “teaching,” which related to the timely completion of
addressed education of patients, families, administrative responsibilities such as
Interpersonal and communication skills medical students, and peers, either on a medical records and documentation.
Three themes were unique to ICS one-to-one basis or during conferences.
The fourth unique PBLI theme was Systems-based practice
milestones (Table 1). The most common
was “communication with patients, “scholarly activity,” which focused on the SBP milestones contained three unique
families, and the public,” followed by completion of scholarly projects, study themes (Table 1). The most common
“communication with teams,” which design, and other research skills. was “health economics,” which included
included use of proactive communication cost–benefit considerations to treatment,
skills, negotiation, and team building Professionalism proper and complete coding, insurance,
(among others). The third unique ICS Five themes were unique to PROF cost-effective care, utilization and
theme was “informed consent,” which milestones (Table 1). The most review, and practice management. The
often included education of patients and common were “attitudes and behaviors” second most common SBP theme was
families as part of the informed consent and “accountability.” The “attitudes “health care delivery settings,” which
process. and behaviors” theme included the addressed residents’ ability to understand
demonstration of integrity, truthfulness, and navigate various practice settings
Practice-based learning and empathy, sensitivity, compassion, (e.g., clinic vs. community hospital
improvement responsibility, respect, cultural competence, vs. Veterans Affairs facility). The third
Four themes were unique to PBLI responsiveness, and managing personal unique SBP theme was “communities,”
milestones (Table 1). The most common beliefs. The “accountability” theme went which addressed the identification and
was “self-directed learning.” This theme beyond compliance with requirements to understanding of the importance of
included assessment of educational needs also include fatigue management, physician community resources for patients.
Overlapping themes
Table 2 Nearly one-third of the 22 themes
Thematic Analysis Results: Overlapping Themes Among ICS, PBLI, PROF, and SBP (7/22; 32%) appeared in milestones
Milestones From the Transitional Year and the 26 Core Specialties, 2017a of more than one core competency
domain (Table 2). Themes related to
Relevant core No. of No. of specialties SBP demonstrated the greatest degree
competency milestones with milestones
related to this related to this of overlap. Examples of milestones
Theme ICS PBLI PROF SBP theme theme illustrating these overlapping themes are
presented in Table 3.
1: Coordination and X X 32 21
transitions of care (14 ICS and 18 SBP)
2: Feedback X X 27 21
The most common area of overlap was
(19 PBLI and 8 PROF) “medical records and technology.” This
3: Medical records and X X 26 22 theme addressed residents’ ability to
technology (17 ICS and 9 SBP) utilize the electronic medical record and
4: Patient safety and X X 23 21 other technology to provide safe patient
errors (1 PBLI and 22 SBP) care. Other common overlapping themes
5: Quality assurance / X X 18 17 were “coordination and transitions of
improvement (17 PBLI and 1 SBP) care,” “feedback,” and “patient safety
6: Confidentiality X X 16 13 and errors.” The “coordination and
(1 ICS and 15 PROF) transitions of care” theme included
7: Teamwork X X 14 10 coordination within and across health
(6 PROF and 8 SBP) care systems and transitions between
  Abbreviations: ICS indicates interpersonal and communication skills; PBLI, practice-based learning and
health care teams. The “feedback” theme
improvement; PROF, professionalism; SBP, systems-based practice. addressed residents’ ability to solicit and
a
Allergy and immunology, anesthesiology, colon and rectal surgery, dermatology, emergency medicine, family incorporate feedback as well as provide
medicine, internal medicine, medical genetics and genomics, neurological surgery, neurology, nuclear medicine, feedback. The “patient safety and errors”
obstetrics–gynecology, ophthalmology, orthopedic surgery, otolaryngology, pathology, pediatrics, physical
medicine and rehabilitation, plastic surgery, preventive medicine, psychiatry, radiation oncology, radiology, theme addressed topics such as error
surgery, thoracic surgery, urology (accessed May 4, 2017, from http://www.acgme.org/Specialties14). avoidance, error reporting, and practice

Academic Medicine, Vol. 93, No. 7 / July 2018 1037

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Research Report

Chart 1
Themes by Specialty (Transitional Year and the 26 Core Specialties), From a Thematic
Study of ICS, PBLI, PROF, and SBP Milestones, 2017a

Theme TY AIb An CRSb D EM FM IM MG NS N NM OG Oph OS Oto Path Ped PM PMR PS Psy RO R S TS U

Self-directed X X X X X X X X X X X X X X X X X X X X X X X X X
learning
Communication X X X X X X X X X X X X X X X X X X X X X X X X
with patients
Attitudes and X X X X X X X X X X X X X X X X X X X X X X X X
behaviors
Accountability X X X X X X X X X X X X X X X X X X X X X X X
Medical records and X X X X X X X X X X X X X X X X X X X X X X
technology
Health economics X X X X X X X X X X X X X X X X X X X X X
Communication X X X X X X X X X X X X X X X X X X X X X
with teams
Coordination and X X X X X X X X X X X X X X X X X X X X X
transitions of care
Feedback X X X X X X X X X X X X X X X X X X X X X
Patient safety and X X X X X X X X X X X X X X X X X X X X X
errors
Ethics X X X X X X X X X X X X X X X X X X X
Quality assurance / X X X X X X X X X X X X X X X X X
improvement
Limits X X X X X X X X X X X X X X X X X
Evidence-based X X X X X X X X X X X X X X X
patient care
Confidentiality X X X X X X X X X X X X X
Administrative tasks X X X X X X X X X X X
Teamwork X X X X X X X X X X
Teaching X X X X X X X X X
Health care delivery X X X X X X X X
settings
Informed consent X X X X X
Scholarly activity X X X X X
Community X X
  Abbreviations: ICS indicates interpersonal and communication skills; PBLI, practice-based learning and
improvement; PROF, professionalism; SBP, systems-based practice; TY, transitional year; AI, allergy and immunology;
An, anesthesiology; CRS, colon and rectal surgery; D, dermatology; EM, emergency medicine; FM, family medicine;
IM, internal medicine; MG, genetics and genomics; NS, neurological surgery; N, neurology; NM, nuclear medicine;
OG, obstetrics–gynecology; Oph, ophthalmology; OS, orthopedic surgery; Oto, otolaryngology; Path, pathology;
Ped, pediatrics; PM, preventive medicine; PMR, medicine and rehabilitation; PS, plastic surgery; Psy, psychiatry;
RO, radiation oncology; R, radiology; S, surgery; TS, thoracic surgery; U, urology.
a
Accessed May 4, 2017, from http://www.acgme.org/Specialties.14
b
These specialties operationalized the ICS, PBLI, PROF, and SBP competencies in the milestone headers instead of the
milestone anchor statements.

safety (e.g., radiation safety). A fifth
area of overlap was “quality assurance/
improvement.” This theme included
quality improvement principles and
projects, as well as laboratory proficiency
testing. The last two overlapping themes
were “confidentiality” and “teamwork.”
The “confidentiality” theme included
the Health Information Portability and
Accountability Act specifically and patient
confidentiality generally. The “teamwork”
theme addressed the roles of various team
members and being a team player.
Areas of overlap across specialties
As shown in Chart 1, we identified
substantial variability in themes across
specialties. This chart lists themes in
descending order, with themes used by
the most specialties at the top of the chart
and those used by the fewest specialties
at the bottom. Eleven of the 22 themes
we identified (50%) were used by 20 or
more specialties. These are “self-directed
learning,” “communication with patients,”
“attitudes and behaviors,” “accountability,”
“medical records and technology,” “health
economics,” “communication with teams,”
“coordination and transitions of care,”
“feedback,” “patient safety and errors,”
and “ethics.” Six of the 22 themes (27%)
appeared in the milestones of 10 or fewer
specialties: “teaching,” “teamwork,” “health
care delivery settings,” “informed consent,”
“scholarly activity,” and “community.”
Discussion
This study identified 22 themes within
the milestones of four crosscutting

1038 Academic Medicine, Vol. 93, No. 7 / July 2018

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Research Report

Table 3
Examples of Themes Found in Milestones From the Transitional Year and the 26 Core
Specialties, From a Thematic Study of ICS, PBLI, PROF, and SBP Milestones, 2017a

Theme Example Specialty

Interpersonal and
communication skills
 Communication with
patients
 Communication
with teams
 Informed consent
Practice-based learning
and improvement
 Self-directed learning
 Evidence-based patient care
 Scholarly activity
 Teaching
Professionalism
 Attitudes and behaviors
 Accountability
 Ethics
 Limits
 Administrative tasks
Systems-based practice
 Health economics
 Health care delivery settings
 Community
Overlapping themes
 Medical records and
technology: ICS
 Medical records and
technology: SBP
 Coordination and transitions
of care: ICS
 Coordination and transitions
of care: SBP
 Feedback: PBLI
 Feedback: PROF
 Quality assurance/
improvement: PBLI
 Quality assurance/
improvement: SBP
 Confidentiality: ICS
 Confidentiality: PROF
Responds to the social and cultural context of the patient and family to ensure the Plastic surgery
patient understands and is able to participate in health care decision making
Communicates collaboratively with the health care team by listening attentively, Family medicine
sharing information, and giving and receiving constructive feedback
This resident performs clear informed consent discussion for complex procedures Surgery
Develops learning plan independently with accurate assessment of competence and Ophthalmology
areas for continued improvement
Incorporates evidence-based medicine practices into patient management Anesthesiology
Presents a critical appraisal and synthesis of the literature either orally or in writing Allergy and
immunology
Mentors junior colleagues and other team members Urology
Demonstrates consistent understanding of patient- and family-expressed needs and a Pediatrics
desire to meet those needs on a regular basis; is responsive in demonstrating kindness and
compassion
Demonstrates professional accountability, including timely and adequate completion of Transitional year
professional responsibilities, being dressed and groomed appropriately, and being ready
and able to perform expected duties
Consistently and effectively analyzes and manages ethical issues in both clinical and Preventive
population-based medicine medicine
Consistently recognizes limits of knowledge in uncommon and complicated clinical Emergency
situations; develops and implements plans for the best possible patient care medicine
Completes administrative and patient care tasks in a timely manner in accordance with Internal medicine
local practice and/or policy
Gives examples of cost and value implications of care he or she provides (e.g., gives Orthopedic
examples of alternate sites of care resulting in different costs for individual patients) surgery
Advocates for quality care and optimal patient care systems in all clinical situations Radiation
oncology
Coordinates patient access to community and system resources Psychiatry
Consistently ensures that patient records, including outpatient and inpatient consultations Dermatology
and transitions of care, are promptly and accurately documented for routine and complex
situations
Uses technology and other hospital/clinic resources in patient care Otolaryngology
Manages transitions of care and optimizes communication across systems Thoracic surgery
Advocates for and provides high-quality, safe, well-coordinated, patient-centered care Physical medicine
across the health care system and rehabilitation
Solicits feedback from all members of the interprofessional team and patients Internal medicine
Accepts constructive feedback to improve his or her ability to demonstrate compassion, Obstetrics–
integrity, and respect for others gynecology
Continues to develop a quality improvement project, employing methods to measure and Nuclear medicine
analyze the data
Participates in laboratory-specific safety training (e.g., sharps disposal, proper equipment Pathology
utilization)
Utilizes Health Insurance Portability and Accountability Act protection safeguards for Neurosurgery
protected health information and electronic medical record
Demonstrates respect for patient privacy and autonomy Medical genetics
and genomics

(Table continues)

Academic Medicine, Vol. 93, No. 7 / July 2018 1039

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Research Report

Table 3
(Continued)

Theme Example Specialty

 Team: PROF Assumes leadership role among the resident group (e.g., as chief resident, project Dermatology
manager); serves as a role model for junior residents
 Team: SBP Collaborates effectively and respectfully with the patient and family, multiple providers, Physical medicine
and the interdisciplinary team to develop patient-centered goals and rehabilitation
 Patient safety and Magnetic resonance (MR) safety: Applies principles of MR safety including safety zones Radiology
errors: PBLI and pre-MR screening
 Patient safety and errors: SBP Participates in a team-based approach to medical error analysis Neurology
  Abbreviations: ICS indicates interpersonal and communication skills; PBLI, practice-based learning and
improvement; PROF, professionalism; SBP, systems-based practice.
a
Allergy and immunology, anesthesiology, colon and rectal surgery, dermatology, emergency medicine, family
medicine, internal medicine, medical genetics and genomics, neurological surgery, neurology, nuclear medicine,
obstetrics–gynecology, ophthalmology, orthopedic surgery, otolaryngology, pathology, pediatrics, physical medicine
and rehabilitation, plastic surgery, preventive medicine, psychiatry, radiation oncology, radiology, surgery, thoracic
surgery, urology (accessed May 4, 2017, from http://www.acgme.org/Specialties14).

competencies (ICS, PBLI, PROF, and
SBP) for the transitional year and 26
core specialties. References supporting
the relevance and importance of these
themes are available in Supplemental
Digital Appendix 1, at http://links.lww.
com/ACADMED/A511. The results
demonstrate areas of overlap among the
ICS, PBLI, PROF, and SBP milestones and
substantial variability across specialties.
With respect to overlap among
competencies, we found that
approximately one-third of the themes
we identified appeared in the milestones
of more than one of the four crosscutting
competencies. It is not surprising to
find overlapping themes among the
ICS, PBLI, PROF, and SBP milestones
as the competencies do not discreetly
manifest in practice. Specialties have
differing perspectives on the primary
core competency reflected by a given
theme (e.g., the degree to which “giving
and receiving feedback” represents PBLI
vs. PROF) or emphasize different aspects
of a given theme (e.g., the PROF aspect
of teamwork vs. the role of teamwork in
SBP). The results of this analysis highlight
areas where harmonization could
strengthen shared understanding across
specialties to enhance educational and
clinical practice.
We also observed variability in themes
across specialties in that some themes
appeared in the milestones of most
specialties, whereas others appeared only
in a minority. Some of this variability
is not necessarily a problem given that
some milestones may be of particular
importance for select specialties, such
as community in family medicine.
Themes appearing in the milestones of
the smallest number of specialties are
not necessarily less important than those
appearing in the milestones of more
specialties. Rather, they represent themes
that may have the greatest potential for
convergence across specialties. However,
some of the variability is concerning when
it involves abilities considered crucial for
21st-century practice, such as evidence-
based patient care and health care delivery
systems.19 As the specialties prepare for
revision of their milestones, it will be
important for them to reconsider some of
these important competency areas.
The original milestones created a natural
experiment for how each specialty views
and interprets the core competencies in
the context of their discipline.12 Although
our findings are encouraging in terms of
what was included across all specialties,
the differences provide an opportunity
for greater interspecialty work on the
competencies moving forward. For
example, “accountability” and “safety
(patient) and errors” are generalizable
across all specialties. Those responsible
for developing the next version of the
milestones will be able to use and expand
these findings to determine those of most
importance to all specialties. Our findings
suggest that a core set of ICS, PBLI, PROF,
and SBP themes could be similar across
all specialties. Each specialty will need to
customize the milestones to fit specialty-
specific needs (e.g., radiation safety in
radiology), including additional themes
if necessary. Through coproduction with
and inclusion of additional community
and public members, we can improve
our understanding and assessment of
these themes, subcompetencies, and
competency domains.
Another important decision moving
forward is determining which essential,
or core, subcompetency themes programs
should report as part of accreditation
and continuous quality improvement.
The Next Accreditation System has
embraced the need to use meaningful
data for continuous quality improvement
of educational programs. National
milestone reporting has already enabled
analysis of competencies other than
MK within several disciplines and has
provided a rich source of feedback for
those disciplines to guide changes in
curriculum and assessment.20–23
The intent of this work is not to dictate
content to future milestone developers
but, rather, to demonstrate what was
important to each of the specialties
in the original milestones and equip
specialties with this information as they
begin to consider their next steps. The
ACGME has also recently appointed four
interdisciplinary milestone development
groups. These groups will work toward
consensus on a set of draft milestones
for the ICS, PBLI, PROF, and SBP
competencies that each specialty can
consider using in future versions. We fully
expect the application of these consensus
milestones to be variable across
specialties because of practice differences.
This study has some limitations. First,
this study did not include a systematic
review of what important themes might
be missing from existing milestones.

1040 Academic Medicine, Vol. 93, No. 7 / July 2018

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.

Second, we did not perform a gap
analysis between specialties. Third, we
did not perform a specific analysis of
the quality of the actual language and
construction of the milestones. Finally,
this study did not include data on how
specialty programs interpret and use the
milestones.
Future directions for this work include
additional research to more fully explore
each competency domain, harmonize
curricular constructs across specialties,
and determine whether there is sufficient
representation of integral aspects of
medical education and physicians’
expected role in society. Graduate medical
education and the specialties must work
together to determine what is necessary
for effective training in all competencies,
including how milestones can best help
drive meaningful change nationally and
what should be left for institutions and
programs to customize locally. Finally, we
need research for national improvement
in the way we teach and train residents
to ensure that they are equipped to work
in the health care system of today and
tomorrow.
Acknowledgments: The authors wish to thank all
of the volunteers who contributed their time and
efforts to the milestones development.
Funding/Support: None reported.
Other disclosures: None reported.
Ethical approval: Reported as not applicable.
Previous presentations: Preliminary results of the
thematic analysis were presented at the American
Board of Medical Specialties and Accreditation
Council for Graduate Medical Education
Milestones Summit in Chicago, Illinois on
December 3–4, 2015, and December 14–15, 2016.
L. Edgar is executive director for milestone
development, Accreditation Council for Graduate
Medical Education, Chicago, Illinois.
S. Roberts is milestones project manager,
Accreditation Council for Graduate Medical
Education, Chicago, Illinois.
N.A. Yaghmour is research associate for
milestones evaluation, Accreditation Council for
Graduate Medical Education, Chicago, Illinois.
Academic Medicine, Vol. 93, No. 7 / July 2018
Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
A. Leep Hunderfund is assistant professor of
neurology, Mayo Clinic, Rochester, Minnesota.
S.J. Hamstra is vice president for milestone
research and evaluation, Accreditation Council for
Graduate Medical Education, Chicago, Illinois.
L. Conforti is research associate for milestones
evaluation, Accreditation Council for Graduate
Medical Education, Chicago, Illinois.
E.S. Holmboe is senior vice president for milestone
development and evaluation, Accreditation Council
for Graduate Medical Education, Chicago, Illinois.
References
1 Nasca TJ, Philibert I, Brigham T, Flynn TC.
The Next GME Accreditation System—
Rationale and benefits. N Engl J Med.
2012;366:1051–1056.
2 Batalden P, Leach D, Swing S, Dreyfus
H, Dreyfus S. General competencies and
accreditation in graduate medical education.
Health Aff (Millwood). 2002;21:103–111.
3 Swing SR. The ACGME Outcome Project:
Retrospective and prospective. Med Teach.
2007;29:648–654.
4 Lee AG, Oetting T, Beaver HA, Carter K;
Task Force on the ACGME Competencies
at the University of Iowa Department of
Ophthalmology. The ACGME Outcome
Project in ophthalmology: Practical
recommendations for overcoming the
barriers to local implementation of the
national mandate. Surv Ophthalmol.
2009;54:507–517.
5 Schumacher DJ, Lewis KO, Burke AE, et al.
The pediatrics milestones: Initial evidence for
their use as learning road maps for residents.
Acad Pediatr. 2013;13:40–47.
6 Korte RC, Beeson MS, Russ CM, Carter WA,
Reisdorff EJ; Emergency Medicine Milestones
Working Group. The emergency medicine
milestones: A validation study. Acad Emerg
Med. 2013;20:730–735.
7 Green ML, Aagaard EM, Caverzagie KJ,
et al. Charting the road to competence:
Developmental milestones for internal
medicine residency training. J Grad Med
Educ. 2009;1:5–20.
8 Nabors C, Peterson SJ, Forman L, et al.
Operationalizing the internal medicine
milestones—An early status report. J Grad
Med Educ. 2013;5:130–137.
9 Meade LB, Borden SH, McArdle P,
Rosenblum MJ, Picchioni MS, Hinchey KT.
From theory to actual practice: Creation
and application of milestones in an internal
medicine residency program, 2004–2010.
Med Teach. 2012;34:717–723.
10 Holmboe ES, Edgar L, Hamstra SJ. The
Milestones Guidebook. Chicago, IL:
Accreditation Council for Graduate Medical
Education; 2016. http://www.acgme.org/
Research Report
Portals/0/MilestonesGuidebook.pdf. Accessed
October 21, 2017.
11 Swing SR, Beeson MS, Carraccio C, et
al. Educational milestone development
in the first 7 specialties to enter the next
accreditation system. J Grad Med Educ.
2013;5:98–106.
12 Holmboe ES, Yamazaki K, Edgar L, et al.
Reflections on the first 2 years of milestone
implementation. J Grad Med Educ.
2015;7:506–511.
13 Holmboe ES, Call S, Ficalora RD. Viewpoint:
Milestones and competency-based medical
education in internal medicine. JAMA Intern
Med. 2016;176:1601–1602.
14 Accreditation Council for Graduate Medical
Education. Specialty milestones. http://
www.acgme.org/Specialties. Published 2015.
Accessed October 21, 2017.
15 Marks D, Yardley L. Research Methods for
Clinical and Health Psychology. Thousand
Oaks, CA: SAGE; 2004.
16 Braun V, Clarke V. Using thematic analysis in
psychology. Qual Res Psychol. 2006;3:77–101.
17 Vaismoradi M, Turunen H, Bondas T.
Content analysis and thematic analysis:
Implications for conducting a qualitative
descriptive study. Nurs Health Sci.
2013;15:398–405.
18 Accreditation Council for Graduate
Medical Education. Common program
requirements. http://www.acgme.org/What-
We-Do/Accreditation/Common-Program-
Requirements. Published September 2013.
Accessed October 21, 2017.
19 Institute of Medicine. Graduate Medical
Education That Meets the Nation’s Health
Needs. Washington, DC: National Academies
Press; 2014.
20 Beeson MS, Holmboe ES, Korte RC, et al.
Initial validity analysis of the emergency
medicine milestones. Acad Emerg Med.
2015;22:838–844.
21 Hauer KE, Clauser J, Lipner RS, et al. The
internal medicine reporting milestones:
Cross-sectional description of initial
implementation in U.S. residency programs.
Ann Intern Med. 2016;165:356–362.
22 Hauer KE, Vandergrift J, Hess B, et al.
Correlations between ratings on the resident
annual evaluation summary and the internal
medicine milestones and association with
ABIM certification examination scores
among U.S. internal medicine residents,
2013–2014. JAMA. 2016;316:2253–2262.
23 Li ST, Tancredi DJ, Schwartz A, et al.
Competent for unsupervised practice: Use
of pediatric residency training milestones
to assess readiness. Association of Pediatric
Program Directors (APPD) Longitudinal
Educational Assessment Research Network
(LEARN) Validity of Resident Self-
Assessment Group. Acad Med. 2017;92:
385–393.
1041