Acad Psychiatry (2014) 38:255–260
DOI 10.1007/s40596-014-0102-2
SPECIAL ELEMENT: TASK FORCE REPORT
The Milestones for General Psychiatry Residency Training
Christopher R. Thomas & George Keepers
Received: 10 December 2013 / Accepted: 10 March 2014 / Published online: 7 May 2014
# Academic Psychiatry 2014
The Accreditation Council of Graduate Medical Education’s
(ACGME) Milestones project represents the most recent devel-
opment in the evolution of medical training in the USA. Be-
ginning with the establishment of the American Medical Asso-
ciation (AMA) in 1845, medical training in the USA has been
increasingly focused on the development of high caliber phy-
sicians whose practice is scientifically based. The Association
of American Medical Colleges (AAMC) was established in
1876 to reform medical education which had previously been
entirely unregulated but with minimal practical effect. The
AMA, however, through the formation of its Council on Med-
ical Education [5, 6] was able to influence training in medical
schools, hospital internship programs, and specialty training.
Abraham Flexner’s report for the Carnegie Foundation pub-
lished in 1910 essentially established our current system of
training and resulted in the closure of inadequate schools and
curricular reform that banished naturopathy and other alterna-
tive practices from medical education. Flexner strongly en-
dorsed the residency program established by John Hopkins in
1889 that would subsequently become the framework for all
residency training in the USA [4]. Subsequently, the AMA
began a program, in 1914, of evaluation and approval of
hospital internships and published the “Essentials of Approved
Residencies and Fellowships” in 1928. The American Board of
Psychiatry and Neurology (ABPN), established in 1934, was
charged by the AMA, American Psychiatric Association
(APA), and American Neurological Association with the task
of establishing training standards in psychiatry and neurology
as well as the certification of specialists in these areas [5]. In
C. R. Thomas (*)
University of Texas Medical Branch at Galveston,
Galveston, TX, USA
e-mail: crthomas@utmb.edu
G. Keepers
Oregon Health & Science University, Portland, OR, USA
1940, the AMA Council on Medical Education in conjunctions
with the American Board of Internal Medicine and the Amer-
ican College of Physicians established the Conference Com-
mittee on Graduate Training in Internal Medicine, a forerunner
of the ACGME Residency Review Committees (RRC). The
American College of Surgeons and American Board of Surgery
followed suit in 1949. With the establishment of Medicare and
federal funding for residency training, academic educators re-
alized that the multiple councils then in existence with widely
varying standards for approval of training programs required
reform. The need for oversight and uniformity of standards was
thought compelling and was supported by the AMA, other
medical groups, and the government. In 1972, the AMA
brought together the AMA, the American Board of Medical
Specialties (ABMS), the American Hospital Association, the
AAMC, and the Council of Medical Specialty Societies to form
a Liaison Committee for Graduate Medical Education
(LCGME) to coordinate and oversee the activities of the several
independent RRC’s then in existence [7]. A more independent
and streamlined organization was required to accomplish these
tasks, and the ACGME replaced the LCGME in 1981. In the
following years, our current system of RRC’s, program require-
ments, and periodic accreditation site visits was developed.
The milestones for residency training represent the next
step in the outcomes-based accreditation project of the
ACGME [10, 14]. Milestones are based on the six core
competencies for graduate medical education established in
1999 by the ACGME and the ABMS and describe specific
behaviors, attributes, or outcomes to be demonstrated by
residents as they progress through training.
The core competencies focus on educational outcomes, as
opposed to educational process, as a measure of success in
achieving training goals and objects. The US Department of
Education mandated the use of outcome measures for all
educational programs receiving federal funding in the
1980’s, including those for accreditation [1]. The ACGME
256
and ABMS convened the Quadrads [1], composed of educa-
tion leaders from the specialty groups, who developed de-
scriptions of specialty specific competencies within the com-
mon framework of the six core competencies. It was hoped
that this would serve as the basis to define common shared
assessment tools and outcome standards for training. Follow-
ing the ACGME and ABMS model, the ABPN convened the
Invitational Core Competencies Conference in 2001 which
led to the core competencies for psychiatric practice [12] The
American Association of Directors of Psychiatric Residency
Training (AADPRT) began to develop model curriculum and
competency assessment tools [2, 13] that were shared on the
AADPRT website as resource material for programs (www.
aadprt.org). In addition, the creation of the Clinical Skills
Verification exams during training by the ABPN [5] with the
end of the oral examinations for board certification did result
in a uniform assessment of specific competencies across all
psychiatry training programs. While the core competencies
became a common framework for describing the goals and
objectives of training, direct incorporation into resident
assessment with uniform evaluation tools was lacking across
all specialties. Public concerns about the quality and efficacy
of medical education continued and resulted in
recommendations for reform, including the Institute of
Medicine (2000), Council on Graduate Medical Education
(2007), Josiah Macy Jr. Foundation (2011) [8], and Medicare
Payment Advisory Commission (MedPAC) reports (2011)
[9]. The ACGME decided in 2008 that all specialties
must create milestones describing resident progress in
the core competencies in order to uniformly implement
outcomes evaluation and accreditation of programs [10].
Methods
Milestone development began in 2009 for internal medicine,
pediatrics, and surgery. The remaining specialties began de-
velopment in 2011 with the goal that all core specialty mile-
stones would be completed and in use by training programs as
of July 1, 2014. The milestone development process for the
second group differed substantially from specialties that were
in the first phase with a more uniform approach across spe-
cialties incorporating the lessons learned by the initial mile-
stone developers [15]. In addition, the ACGME convened
expert panels that drafted recommendations for the milestone
working groups to serve as models in the shared core compe-
tencies of systems-based practice (SBP), practice-based learn-
ing and improvement (PBLI), interpersonal skills and com-
munication (ICS), and professionalism (P).
The milestones describe the development of resident com-
petencies in the course of training (Fig. 1). These develop-
mental pathways are organized in a five-level framework, with
level 1 representing skills and attributes expected of a
Acad Psychiatry (2014) 38:255–260
beginning resident, levels 2 and 3 as intermediate stages, level
4 those of a graduating resident, and level 5 as advanced or
aspirational goals (Table 3). The first four levels should not be
confused as directly correlating to the year of training as the
skills and competencies do not necessarily develop in a linear
fashion. In addition, as training program differ in design, the
ability to assess milestones will vary greatly depending on the
organization and timing of clinical rotations and experience.
The milestones are grouped by the six core competencies and
are further categorized into sub-competencies that describe the
specific components of the general core competency. The
ACGME suggested that there be no more than 36 sub-
competencies for any specialty. The psychiatry working group
created an additional category called threads, which describe
developmentally related progression of specific skills, knowl-
edge, or attributes. Threads are intended to describe a develop-
mental progression where prior achievements form the founda-
tion for further development, analogous to the developmental
progression of crawling, walking with support, walking inde-
pendently, and then running. The grouping of threads can
potentially assist in the development of assessment tools and
evaluation.
Leaders representing the ABPN, AADPRT, ACGME, and
American Psychiatric Association (APA) selected the psychi-
atry milestone working group and advisory group (Table 1).
The psychiatry milestone working group first met in Decem-
ber 2011. Over the course of the next 2 years with five face-to-
face meetings and numerous conference calls, the working
group crafted the psychiatry milestones and assessment tools.
The group began by establishing the principles that would
guide development of the milestones. It was agreed that the
psychiatry milestones must be measurable, manageable, and
meaningful. The working group reviewed a number of back-
ground documents, including the Core Competencies for Psy-
chiatric Practice (2003), the Core Competencies for Psychiat-
ric Education (2004), the Psychiatry Program Requirements,
and the content outlines for both the ABPN Certification
Exam in Psychiatry and the American College of Psychia-
trists’ Psychiatry Residents in Training Exam (PRITE). The
working group used Bloom’s taxonomy as a guide for termi-
nology and development of milestones describing knowledge,
attitudes, and skills [3]. The working group also reviewed the
milestones from other specialties thought to be useful models
as well as the recommendations from the expert panels on the
four shared core competencies: SBP, PBLI, ICS, and P.
The initial focus for the working group was outlining the
sub-competencies for medical knowledge (MK) and patient
care (PC), the two core competencies considered to be most
specialty specific. Following this, the work moved to the
remaining four shared core competencies. Teams of two to
three members of the working group drafted milestones for
each of the sub-competencies. The entire working group then
reviewed all draft milestones for inclusion, categorization,
Acad Psychiatry (2014) 38:255–260 257

Fig. 1 Components of Psychiatry Milestones

Table 1 Psychiatry milestone

working group and advisory Psychiatry milestone groups
group
Christopher R. Thomas, MD, Chair

Working group Advisory group

Sheldon Benjamin, MD Timothy Brigham, MDiv, PhD (ACGME staff)
Adrienne L. Bentman, MD (AADPRT) Carol A. Bernstein, MD (APA)
Robert Boland, MD Beth Ann Brooks, MD (ABPN)
Deborah S. Cowley, MD Larry R. Faulkner, MD (ABPN)
Jeffrey Hunt, MD Deborah Hales, MD (APA)
George A. Keepers, MD (ABPN) Victor I. Reus, MD (ABPN)
Louise King, MS (ACGME staff) Richard F. Summers, MD (AADPRT)
Gail H. Manos, MD (ACGME RRC)
Donald E. Rosen, MD (ACGME RRC)
Kathy M. Sanders, MD (AADPRT)
Mark E. Servis, MD (ACGME RRC)
Kallie Shaw, MD (ABPN)
Susan Swing, PhD (ACGME staff)
Alik Widge, MD, PhD (Resident representative)
258
wording, and level. Each meeting of the working group was
followed by a conference call during which the advisory
group reviewed the progress of the draft milestones. Based
on their comments and questions, the working group revised
milestones. The first draft of the psychiatry milestones was
posted for public comment and review in November, 2012.
Additionally, the ACGME and AADPRT each conducted their
own survey of program directors. Input was also received
from the Association for Academic Psychiatry (AAP) and
the Group for Advancement of Psychiatry (GAP). Additional
review and comment was sought from the Association of
Directors of Medical Student Education (ADMSEP), particu-
larly on the expectations for residents entering training
outlined by the level one milestones. Members of the working
group made presentations and conducted workshops designed
to elicit feedback on the milestones at the annual meetings of
the ACGME, AADPRT, AAP, APA, the PRITE Editorial
Board, and the American College of Psychiatrists (ACP).
Nineteen training programs agreed to pilot the first draft
psychiatry milestones in January 2013, yielding important
information about its feasibility and validity. The pilot study
required each programs’ Clinical Competency Committee to
review at least two residents in each year of training. A follow-
up pilot study of the second draft of the psychiatry milestones
was conducted by 16 of the 19 initial pilot programs in June
2013, incorporating the revisions and lessons learned from the
first pilot and the surveys. The results from the pilot tests
supported the validity of the milestones with overall resident
performance across the levels conforming to expected pat-
terns. Both pilots helped in adjusting the levels for individual
milestones as well as identifying problematic areas needing
further clarification. Gaps in assessment tools for many of the
milestones were noted.
The surveys, pilot tests, and various professional societies’
comments were critical in guiding the revision of the psychi-
atry milestones. The working group focused on assuring the
milestones’ clarity and specificity as well as their accuracy in
describing a realistic progression in training. The final version
of the psychiatry milestones was completed in October 2013.
Results
There are 22 psychiatry milestone sub-competencies and 66
threads (Table 2). The sub-competencies for MK and PC
account for half of the total. A continuing challenge through-
out the development process was to describe milestones with
enough specificity as to be descriptive and measurable while
keeping the overall number of milestones to a manageable
number that would nevertheless comprehensively cover all of
the core competencies. Another concern, addressed later in the
process, was the potential for redundancies between various
milestones. All related milestones were carefully reviewed
Acad Psychiatry (2014) 38:255–260
Table 2 Psychiatry milestone sub-competencies
Psychiatry milestones sub-competencies
Patient care
Psychiatric evaluation
Psychiatric formulation and differential diagnosis
Treatment planning and management
Psychotherapy
Somatic therapies
Medical knowledge
Development through the life cycle
Psychopathology
Clinical neuroscience
Psychotherapy
Somatic therapies
Practice of psychiatry
Systems-based practice
Patient safety and health care team
Resource management
Community-based care
Consultation to non-psychiatric medical providers and non-medical
systems
Practice-based learning and improvement
Development and execution of life-long learning through constant self-
evaluation, including critical evaluation of research and clinical
evidence
Formal practice-based quality improvement based on established and
accepted methodologies
Teaching
Professionalism
Compassion, integrity, respect, and sensitivity for others; adherence to
ethical principles
Accountability to self, patients, colleagues, and the profession
Interpersonal and communication skills
Relationship development and conflict management
Information sharing and record keeping
with the goal of keeping the final number of milestones to
the fewest necessary. If overlaps were allowed to remain, it
was because they were essential in fully describing the com-
ponent that the affected milestones shared. A good example of
this issue was the overlap between the PC and ICS sub-
competency in psychotherapy. The working group decided
that the key aspects of therapeutic alliance should be described
in the psychotherapy sub-competency, as this is central in
psychiatric practice. Another example of this dilemma is seen
in the repetition of the sub-competencies for psychotherapy
and somatic therapies in both PC and MK core competencies.
In both cases, the working group decided to list skills required
to deliver treatment, under PC, and the knowledge about
treatments under MK. The sub-competencies for MK devel-
opment through the life cycle and PBLI teaching were
Acad Psychiatry (2014) 38:255–260
extensively reorganized and revised based on comments re-
ceived from the surveys. There were also suggestions that MK
clinical neuroscience milestones would be better included
under other MK sub-competencies, but the working group
considered it an important focus of attention in residency
curriculum. Questions were also raised in the surveys regard-
ing the PBLI milestones concerning quality improvement
activities, especially the use of certain new terminology. The
working group considered this an area of growing emphasis in
graduate medical education and revised the wording to clarify
their intent. The initial draft contained 23 sub-competencies,
but two of the SBP sub-competencies were merged, as the
distinction between them was considered insignificant.
Discussion
The central intent of the psychiatry milestones is to provide
the ACGME with aggregate resident progress in training to
enable assessment of residency training program performance
as part of the Next Accreditation System. This represents a
major change not only in the process of program accreditation,
but also in resident evaluation and program review and im-
provement. In contrast to the summative approach of judging
a resident’s performance as satisfactory or unsatisfactory,
milestones provide residents with a concise description of
their developmental progress in attaining the competencies
and with detailed goals for the next phase of their training.
Programs can better assess not only individual resident per-
formance and needs, but also overall program success in
training the core competencies and possible areas for improve-
ment. The milestones provide clear objectives to guide overall
curriculum development and organization.
It is not expected that all residents be evaluated on all
milestones at each opportunity for evaluation. Various rota-
tions and training experiences involve only certain milestones.
In addition, residents at different stages of training would not
be assessed on more advanced milestones nor require evalu-
ation of previously accomplished attainments. The milestones
method offers the opportunity to specifically tailor assess-
ments to the educational setting and focuses supervisors and
residents on the critical aspects of a particular experience. This
is a different approach from previous global assessments of
resident evaluation where individual supervisors rate a resi-
dent across all core competencies rather than focus on specific
components of resident performance. It will be the responsi-
bility of the each residency program’s Clinical Competency
Committee to judge the resident’s performance across the sub-
competencies.
While the milestones will provide useful information re-
garding an individual resident, they are not the only measure
of performance or standing within a program. The decision on
a resident’s continuation, promotion, and graduation is the
259
responsibility of the program director, and there are many
other factors that affect this decision. Level 4 milestones are
defined as a target for training and do not represent graduation
requirement (Table 3).
With the completion of the second draft in April 2013, the
working group also started to develop assessment tools based
on the milestones. These are intended to serve not only as
rating forms for programs, but also models or templates for
assessments developed by programs for specific training ex-
periences. The AADPRT has also formed a task force to
review and adapt existing assessment tools by mapping mile-
stones onto them. The milestone pilots indicated that there are
gaps in training programs’ curriculums and assessments that
will need to be addressed.
The process for developing milestones for the psychiatry
subspecialties has also begun. It will differ from the develop-
ment of the psychiatry milestones in that the subspecialists
will work together on the shared competencies of PBLI, SBP,
ICS, and P and separately for PC and MK. It is expected that
the level 3 and 4 milestones will serve as a guide for the
beginning expectations in the subspecialties. A challenge will
be in adapting the overall developmental concepts of mile-
stone method to fellowships that last just 1 year.
The psychiatry milestones are scheduled to begin as part of
ACGME expected resident assessment July 1, 2014. They
will be an essential part of continuous program accreditation
under the Next Accreditation System [11] along with infor-
mation obtained through the annual resident survey, faculty
survey, and Accreditation Data System. The milestones are a
new process and may require adjustment to assure their feasi-
bility and utility. This will require that several years of data
Table 3 Five-level framework of the developmental pathways
Milestone levels
Has not achieved level 1: The resident does not demonstrate the
milestones expected of an incoming resident.
Level 1: The resident demonstrates milestones expected of an incoming
resident.
Level 2: The resident is advancing and demonstrates additional
milestones, but is not yet performing at a mid-residency level.
Level 3: The resident continues to advance and demonstrate additional
milestones; the resident demonstrates the majority of milestones
targeted for residency in this sub-competency.
Level 4: The resident has advanced so that he or she now substantially
demonstrates the milestones targeted for residency. This level is
designed as the graduation target and does not represent a graduation
requirement. Making decisions about readiness for graduation is the
purview of the residency program director.
Level 5: The resident has advanced beyond performance targets set for
residency and is demonstrating “aspirational” goals, which might
describe the performance of someone who has been in practice for
several years. It is expected that only a few exceptional residents will
reach this level.
260
will be needed with at least one cohort of residents having
completed training with milestones in order to evaluate them
fully. The working group strived to make the psychiatry
milestones flexible and durable, knowing that psychiatry is a
rapidly changing field. Information on the milestone perfor-
mance along with advances in the field will be incorporated in
future revisions.
The psychiatry milestones represent a major paradigm shift
not only in program accreditation but also in our approach to
resident training and assessment. They are founded on and the
next step in the development of the psychiatry core compe-
tencies. A process that encouraged comment and debate and
the information from pilot tests guided their development.
Further work is needed in developing assessments and refin-
ing the individual milestones, but the process is well
established and underway. The psychiatry milestones estab-
lish a new framework for a national discussion of psychiatric
residency training and new opportunities to improve resident
education through outcomes assessment.
Disclosures On behalf of all authors, the corresponding author states
that there is no conflict of interest.
References
1. Adamowski S. The ACGME and ABMS initiatives: toward the
development of core competencies. In: Schreiber S, Kramer T,
Adamowski S, editors. Core competencies for psychiatric practice:
what clinicians need to know. Washington, DC: American
Psychiatric Publishing; 2003. p. 43–54.
Acad Psychiatry (2014) 38:255–260
2. Bienenfeld D, Klykylo W, Knapp V. Process and product: development
of competency-based measures for psychiatry residency. Acad
Psychiatry. 2000;24(2):68.
3. Bloom BS. Taxonomy of educational objectives, handbook I: the
cognitive domain. New York: David McKay Co., Inc.; 1956.
4. Flexner A. Medical education in the United State and Canada.
Boston: The Merrymount Press; 1910. D.B. Updike.
5. Hollender M. The founding of the ABPN. In: Aminoff M, Faulkner
L, editors. The American board of psychiatry and neurology: looking
back and moving ahead. 1st ed. Washington, DC: American
Psychiatric Publishing; 2012. p. 9.
6. Joyner BD. An historical review of graduate medical education and a
protocol of accreditation council for graduate medical education
compliance. J Urol. 2004;172(1):34–9.
7. Kassebaum DG. Origin of the LCME, the AAMC-AMA partnership
for accreditation. Acad Med. 1992;67(2):85–7.
8. Macey Foundation. Ensuring an effective physician workforce for the
United States: recommendations for reforming graduate medical
education to meet the needs of the public: conference summary.
http://josiahmacyfoundation.org/docs/macy_pubs/JMF_GME_
Conference2_Monograph(2).pdf
9. MEDPAC. Graduate medical education financing: focusing on edu-
cational priorities., 2008, from http://www.medpac.gov/chapters/
Jun10_Ch04.pdf
10. Nasca TJ (2008) The CEO’s first column—the next step in the
outcomes-based accreditation project. ACGME Bull. May, 2.
11. Nasca TJ, Philibert I, Brigham T. The next GME accreditation
system—rationale and benefits. N Engl J Med. 2012;366:
1051–6.
12. Schreiber S, Kramer T, Adamowski S, editors. Core competencies for
psychiatric practice: what clinicians need to know. Washington, DC:
American Psychiatric Publishing, Inc; 2003.
13. Swick S, Beresin E. Assessing the ACGME competencies in psychi-
atry training programs. Acad Psychiatry. 2006;30(4):330–51.
14. Swing SR. The ACGME outcome project: retrospective and prospec-
tive. Med Teach. 2007;29(7):648–54.
15. Swing SR, Beeson M, Carraccio C. Educational milestone develop-
ment in the first 7 specialties to enter the next accreditation system. J
Graduate Med Educ. 2013;5(1):98.