Research

JAMA Surgery | Original Investigation

Evaluating the Association of Multiple Burnout Definitions

and Thresholds With Prevalence and Outcomes
D. Brock Hewitt, MD, MPH, MS; Ryan J. Ellis, MD, MS; Yue-Yung Hu, MD, MPH; Elaine O. Cheung, PhD;
Judith T Moskowitz, PhD, MPH; Gaurava Agarwal, MD; Karl Y. Bilimoria, MD, MS

Invited Commentary
IMPORTANCE Physician burnout is a serious issue, given its associations with physician
attrition, mental and physical health, and self-reported medical errors. Burnout is typically
measured in health care by assessing the frequency of symptoms in 2 domains, emotional
exhaustion and depersonalization. However, the lack of a clinically diagnostic threshold to
define burnout has led to considerable variability in reported burnout rates.

OBJECTIVE To estimate the prevalence of burnout using a range of definitions (ie, requiring
symptoms in both domains or just 1) and thresholds (ie, requiring symptoms to occur weekly
vs a few times per year) and examine the strength of the association of various definitions of
burnout with suicidal thoughts and thoughts of attrition among general surgery residents.

DESIGN, SETTING, AND PARTICIPANTS A cross-sectional national survey of clinically active US

general surgery residents administered in conjunction with the 2019 American Board of
Surgery In-Training Examination assessed burnout symptoms, thoughts of attrition, and
suicidal thoughts during the past year. Multivariable logistic regression models were used to
assess the association of burnout symptoms with thoughts of attrition and suicidal thoughts.
Values of R2 and C statistic were used to evaluate multivariable model performance.

EXPOSURES Burnout was evaluated with a 6-item, modified, abbreviated Maslach Burnout
Inventory for 2 burnout domains: emotional exhaustion and depersonalization.

MAIN OUTCOMES AND MEASURES The primary outcome was prevalence of burnout.
Secondary outcomes were thoughts of attrition and suicidal thoughts within the past year.

RESULTS Among 6956 residents (a 85.6% response rate; including 3968 men [57.0%] and
4041 non-Hispanic White individuals [58.1%]) from 301 surgical residency programs, 2329
(38.6%) reported at least weekly symptoms of emotional exhaustion, and 1389 (23.1%)
reported at least weekly depersonalization symptoms. Using the most common definition,
2607 general surgery residents (43.2%) reported weekly burnout symptoms on either
subscale. Subtle changes in the definition of burnout selected resulted in prevalence
estimates varying widely from 3.2% (159 residents; most stringent: daily symptoms on both
subscales) to 91.4% (5521 residents; least stringent: symptoms a few times per year on either
Author Affiliations: Surgical
subscale). In multivariable models, all measures of higher burnout symptoms were associated Outcomes and Quality Improvement
with increased thoughts of attrition (depersonalization: R2, 0.097; C statistic, 0.717; Center (SOQIC), Feinberg School of
emotional exhaustion: R2, 0.137; C statistic, 0.758; both: R2, 0.138; C statistic, 0.761) and Medicine, Department of Surgery,
Northwestern University, Chicago,
suicidal thoughts (depersonalization: R2, 0.077; C statistic, 0.718; emotional exhaustion: R2,
Illinois (Hewitt, Ellis, Hu, Bilimoria);
0.102; C statistic, 0.750; both: R2, 0.106; C statistic, 0.751) among general surgery residents Department of Surgery, Thomas
(all P < .001). Jefferson University Hospital,
Philadelphia, Pennsylvania (Hewitt);
CONCLUSIONS AND RELEVANCE In a national evaluation of general surgery residents, Department of Medical Social
prevalence estimates of burnout varied considerably, depending on the burnout definition Sciences, Northwestern University,
Chicago, Illinois (Cheung,
selected. Frequent burnout symptoms were strongly associated with both thoughts of
Moskowitz); Feinberg School of
attrition and suicide, regardless of the threshold selected. Future research on burnout should Medicine, Department of Psychiatry,
explicitly include a clear description and rationale for the burnout definition used. Northwestern University, Chicago,
Illinois (Agarwal).
Corresponding Author: Karl Y.
Bilimoria, MD, MS, Surgical Outcomes
and Quality Improvement Center
(SOQIC), Feinberg School of
Medicine, Department of Surgery,
Northwestern University,
633 N St Clair St, 20th Floor, Chicago,
JAMA Surg. doi:10.1001/jamasurg.2020.3351 IL 60611 (k-bilimoria@
Published online September 9, 2020. northwestern.edu).

(Reprinted) E1
© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 09/10/2020

 Research Original Investigation Evaluating the Association of Multiple Burnout Definitions and Thresholds With Prevalence and Outcomes
B
urnout is a multifaceted condition of overwhelming ex-
haustion, interpersonal detachment or cynicism to-
ward one’s job, and a sense of reduced professional ef-
ficacy, driven by long-term workplace stress.1,2 Burnout has
garnered attention in the medical community because of its
reported association with physician attrition, mental and physi-
cal health, and self-reported medical errors.2-5 Each year, phy-
sician burnout–driven attrition and reduced clinical hours cost
the US health care system approximately $4.6 billion.6 In a na-
tional study, more than half of practicing physicians reported
at least 1 burnout symptom, almost twice the rate of the gen-
eral population.7 However, exact prevalence estimates of burn-
out are unclear.
Burnout lacks validated clinical cutoffs, and assessment
methods and measurement thresholds vary widely across
studies.8-11 Three commonly used tools for burnout measure-
ment provided by The National Academy of Medicine Action
Collaborative on Clinician Well-Being and Resilience (the
Maslach Burnout Inventory [MBI], the Oldenburg Burnout In-
ventory, and the Physician Work-Life Study’s single item) each
evaluate burnout with different scales and dimensions.12 Re-
garding burnout definitions, a recent systematic review found
142 unique definitions for meeting burnout criteria (ie, being
burned out), and overall prevalence estimates ranged from 0%
to 80.5%, depending on the specialty of medicine examined
and the criteria chosen.8 For general surgery residents alone,
estimates of burnout prevalence range from 43% to 69%.13-15
With the most common tool, the MBI, the frequency is mea-
sured on a 7-point Likert scale as symptoms experienced daily,
a few times a week, once a week, a few times a month, once a
month, a few times per year, or never. Prior studies have set
their burnout threshold at different frequencies and required
symptoms in 1 or multiple domains. The wide variation could
be attributable to the cohorts examined, differing response
rates, and/or these different definitions and thresholds of burn-
out used.
Burnout is typically measured in health care by assessing
the frequency of symptoms in 2 domains, emotional exhaus-
tion and depersonalization. To better understand the varia-
tion in burnout prevalence estimates among surgical resi-
dents and test varying definitions and thresholds of burnout,
a comprehensive national survey of all US general surgery resi-
dents was conducted using an abbreviated MBI.16,17 The ob-
jectives of this study were to (1) estimate the prevalence of
burnout using a range of definitions and thresholds and (2) ex-
amine the strength of the association between various defi-
nitions and thresholds of burnout with 2 important out-
comes, thoughts of attrition and suicidal thoughts, among
clinically active US general surgery residents.
Methods
Study Design and Participants
In collaboration with the American Board of Surgery, a volun-
tary, multiple-choice survey was administered to all examin-
ees from Accreditation Council for Graduate Medical Educa-
tion–accredited general surgery training programs immediately
E2 JAMA Surgery Published online September 9, 2020 (Reprinted)
© 2020 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 09/10/2020
Key Points
Question What is the association of multiple burnout definitions
and thresholds with prevalence and wellness outcomes?
Findings In this national study of 6956 general surgery residents,
burnout prevalence estimates varied from 3.2% to 91.4%,
depending on the burnout definition selected. Frequent burnout
symptoms were significantly associated with thoughts of both
attrition and suicide, regardless of the definition selected.
Meaning Research on burnout should include a clear description
of the burnout definition used and the justification for its use.
following the January 2019 American Board of Surgery In-
Training Examination (ABSITE). The ABSITE is a computer-
based, multiple-choice examination administered annually to
US general surgery residents to measure knowledge and man-
agement of surgical pathology.18 The survey was preceded by
a statement explaining that the purpose of the survey was re-
search, the data would be deidentified prior to analysis, and
program directors or chairs would never have access to indi-
vidual responses. There were no incentives to participate in
the survey. Responses were collected by the American Board
of Surgery and deidentified before being transferred to North-
western University for analysis. Excluded from analysis were
778 residents who were clinically inactive (ie, taking dedi-
cated time to conduct research). After review of this study, in-
cluding the survey content, the Northwestern University in-
stitutional review board office determined that it did not meet
the definition of human subjects research. Survey comple-
tion constituted participant consent.
Survey Instrument
The 2019 survey items were adapted from previously pub-
lished and validated tools.19,20 Pretest cognitive interviews
were conducted with general surgery residents to assess over-
all survey coherence, balance, and clarity. The survey was then
iteratively revised and retested with a larger sample of gen-
eral surgery residents from multiple institutions.
A modified, abbreviated MBI–Human Services Survey for
Medical Personnel was used to assess burnout symptoms.2,16,17
The 6-item instrument assessed 2 burnout domains: emo-
tional exhaustion (3 items) and depersonalization (3 items)
symptoms on a 7-point Likert scale (categorized as never, a few
times a year or less, once a month, a few times a month, once
a week, a few times a week, or every day). Individual sub-
scale and overall burnout frequencies were reported using mul-
tiple criteria. Defining burnout includes 2 main points of clari-
fication: the number of burnout domains involved and the
symptom frequency dichotomization threshold (eg, every day,
once a week, once a month). The full MBI includes 3 do-
mains: emotional exhaustion, depersonalization, and per-
sonal accomplishment. In this study, we examined deperson-
alization and emotional exhaustion, because previous studies
have found that these 2 domains most consistently describe
a clinical burnout syndrome.21 Two categorizations of burn-
out domains were used. First, to meet the criteria for burn-
out, the resident had to report symptoms of only 1 domain,
jamasurgery.com
 Evaluating the Association of Multiple Burnout Definitions and Thresholds With Prevalence and Outcomes Original Investigation Research

either emotional exhaustion or depersonalization. The sec-

Table 1. Characteristics of Residents
ond, more stringent definition required a resident to report From 301 Surgical Residency Programs
symptoms of both domains. Within these categories, mul-
Characteristic Overall, No. (%)
tiple subscale thresholds were examined.
Total No. of residents 6956
The most stringent subscale dichotomization burnout
Sex
threshold was reporting symptoms daily, a threshold that in-
Male 3968 (57.0)
cluded residents reporting burnout symptoms every day vs resi-
Female 2796 (40.2)
dents reporting symptoms a few times a week, once a week, a
few times a month, once a month, a few times a year or less, and Unknown 192 (2.8)

never. The most sensitive subscale threshold was reporting Race/ethnicity

symptoms at least yearly, a dichotomization threshold includ- Non-Hispanic

ing residents reporting symptoms every day, a few times a week, White 4041 (58.1)
once a week, a few times a month, once a month, or a few times Black 316 (4.5)
a year or less vs reporting symptoms as never occurring. Hispanic 584 (8.4)
Finally, each domain was also evaluated as a continuous vari- Asian 1181 (17.0)
able, separately and as a combined overall burnout variable Other/unknown 834 (12.0)
(a continuous variable of emotional exhaustion plus Clinical program year
depersonalization). 1 1809 (26.0)
Residents were also asked if they had considered leaving 2 or 3 2747 (39.5)
their program or taking their own life within the last year. Sui- 4 or 5 2400 (34.5)
cidal thoughts were assessed with the question, “During the Relationship status
past 12 months, have you had thoughts of taking your own Married or in a relationship 5111 (73.5)
life?”22 This question was immediately followed by a webpage No relationship 1635 (23.5)
providing the National Suicide Prevention Lifeline and urg-
Divorced or widowed 125 (1.8)
ing respondents to reach out to their program directors if they
Unknown 85 (1.2)
have had such thoughts. No active outreach was possible be-
Program size (total No. of residents), quartile
cause all data were deidentified, and confidentiality was as-
1 (<24) 1799 (25.9)
sured as a precondition of survey completion.
2 (24-34) 1714 (24.6)
3 (35-49) 1786 (25.7)
Statistical Analysis
4 (≥50) 1657 (23.8)
Raw and cumulative emotional exhaustion and depersonaliza-
Program type
tion symptom frequencies were calculated. Separate multivari-
able logistic regression models were constructed to examine the Academic 4014 (57.7)

association between varying definitions or thresholds of burn- Community or military 2942 (42.3)

out and each resident outcome of interest: thoughts of attrition
and suicidal ideation. Models were adjusted for all available resi-
dent demographics (eg, sex, race/ethnicity, and marital status)
and program characteristics (eg, geographic location, type [aca-
demic or community or military], and size). Values of R2 and C
statistics were used to evaluate multivariable model perfor-
mance. All models were estimated with robust SEs accounting
for residents clustering within programs. Multivariable analy-
ses were limited to individuals with complete survey re-
sponses. All tests were 2-sided, with significance set at .05. Sta-
tistical analyses were performed with Stata 14.1 (StataCorp).
Results
Of the 8129 eligible surgical residents taking the 2019 AB-
SITE, 6956 had at least partial survey responses (response rate,
85.6%). Most residents were male (3968 [57.0%]), non-
Hispanic White (4041 [58.1%]), married or in a relationship (5111
[73.5%]), and training in an academic program (4014 [57.7%])
(Table 1). Additional participant demographics and program
characteristics are listed in Table 1.
Using the most common definition of burnout in the lit-
erature, 2607 general surgery residents (43.2%) reported
Program location
Northeast 2328 (33.5)
Southeast 1352 (19.4)
Midwest 1537 (22.1)
Southwest 799 (11.5)
West 940 (13.5)
weekly burnout symptoms on either subscale. Emotional ex-
haustion symptoms were reported by 510 residents (8.5%) as
daily, 2329 (38.6%) as at least weekly (a cumulative grouping
of symptoms daily, few times a week, and once a week), and
4379 (72.6%) as at least monthly (a cumulative grouping of daily
symptoms to symptoms once a month); 579 residents (9.5%)
reported never experiencing symptoms of emotional exhaus-
tion (Table 2). Depersonalization symptoms were reported by
312 residents (5.2%) as daily, 1389 (23.1%) as at least weekly,
and 3145 (52.3%) as at least monthly; 1474 (24.4%) reported
never experiencing symptoms of depersonalization. Emo-
tional exhaustion symptoms were reported in a more normal
distribution, with a few times a month the most common fre-
quency reported. Depersonalization symptoms were re-
ported less often; almost half of the residents surveyed
reported experiencing symptoms of depersonalization never
(1410 [23.3%]) or a few times a year or less (1474 [24.4%]).

jamasurgery.com (Reprinted) JAMA Surgery Published online September 9, 2020 E3

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 09/10/2020

 Research Original Investigation Evaluating the Association of Multiple Burnout Definitions and Thresholds With Prevalence and Outcomes

Table 2. Burnout Symptom Frequency in General Surgery Residentsa

Participants, No. (%)

Emotional exhaustion subscale Depersonalization subscale Both subscales Either subscale
Burnout symptom Reported Reported Reported Reported
frequency frequency Cumulative frequency Cumulative frequency Cumulative frequency Cumulative
Every day 510 (8.5) 510 (8.5) 312 (5.2) 312 (5.2) 159 (3.2) 159 (3.2) 663 (11.0) 663 (11.0)
A few times a week 1153 (19.1) 1663 (27.6) 529 (8.8) 841 (14.0) 465 (9.3) 624 (12.5) 1217 (20.2) 1880 (31.2)
Once a week 666 (11.0) 2329 (38.6) 548 (9.1) 1389 (23.1) 487 (9.7) 1111 (22.2) 727 (12.0) 2607 (43.2)
A few times a month 1291 (21.4) 3620 (60.0) 974 (16.2) 2363 (39.3) 1019 (20.3) 2130 (42.5) 1246 (20.6) 3853 (63.8)
Once a month 759 (12.6) 4379 (72.6) 782 (13.0) 3145 (52.3) 839 (16.8) 2969 (59.3) 702 (11.6) 4555 (75.4)
A few times a year or 1081 (17.9) 5460 (90.5) 1410 (23.3) 4555 (75.6) 1525 (30.4) 4494 (89.7) 966 (16.0) 5521 (91.4)
less
Never 579 (9.5) 6039 (100) 1474 (24.4) 6029 (100) 516 (10.3) 5010 (100) 519 (8.6) 6040 (100)
a
The reported frequency value was calculated as the most frequent response all reported frequency values at that frequency and responses at greater
given for any of the 3-item subscale questions. For example, if a resident frequencies. For example, the cumulative value for a few times a week is the
reported 1 depersonalization item as every day and the other 2 summation of the reported frequency of residents reporting a few times a
depersonalization items as never, they were counted as every day for the week and the response at the frequency every day.
depersonalization subscale reported frequency. The cumulative value includes

Table 3. Evaluation of Multivariable Model Performance Exploring Associations Between Burnout Symptoms,
Thoughts of Attrition, and Suicidal Ideationa

Thoughts of attritionb Suicidal ideationb

2
Burnout definition Model R C statistic Model R2 C statistic
Dichotomous variables
Once-weekly symptoms
Emotional exhaustion 0.091 0.720 0.051 0.686
Depersonalization 0.082 0.707 0.060 0.700
Both subscales 0.095 0.718 0.053 0.688
Either subscale 0.087 0.717 0.059 0.698 a
All models were adjusted for
Continuous variables resident sex, race/ethnicity,
program year, marital status,
Emotional exhaustion score 0.137 0.758 0.102 0.750
program size, program type
Depersonalization score 0.097 0.717 0.077 0.718 (academic or community/military),
Sum of emotional exhaustion and 0.138 0.761 0.106 0.751 and program geographic location.
depersonalization scores b
P < .001 for all comparisons.

The first categorization used to evaluate burnout was re-
porting symptoms on only 1 subscale—either emotional ex-
haustion or depersonalization. Using the most sensitive thresh-
old of at least yearly burnout symptoms, 5521 residents (91.4%)
reported symptoms of either emotional exhaustion or deper-
sonalization and met criteria for burnout (Table 2). The most
stringent dichotomization threshold requiring daily burnout
symptoms on either domain resulted in a burnout prevalence
rate of 11.0% (in 663 residents). Changing the threshold re-
sulted in an absolute difference of 80.4% in the burnout preva-
lence rate by altering the frequency of symptoms required to
be deemed burnout.
The second burnout categorization required a resident to
experience symptoms from both burnout subscales. When
using the most sensitive threshold of at least yearly burnout
symptoms, 4494 residents (89.7%) reported both emotional
exhaustion and depersonalization symptoms, meeting crite-
ria for burnout. The more specific threshold of daily burnout
symptoms from both subscales resulted in a burnout preva-
lence rate of 3.2% (in 159 residents), an absolute difference of
86.5%.
In multivariable models adjusting for resident demograph-
ics and program characteristics, all definitions of burnout,
including analysis of burnout on a continuous scale, were sig-
nificantly associated with both thoughts of attrition and sui-
cidal thoughts. Using R2 and C statistic to evaluate model fit,
the continuous overall burnout score (ie, summation of both
emotional exhaustion and depersonalization scores) per-
formed better to anticipate resident thoughts of attrition (R2,
0.138; C statistic, 0.761) and suicidal thoughts (R2, 0.106; C sta-
tistic, 0.751), as did the individual continuous subscales of emo-
tional exhaustion (thoughts of attrition: R2, 0.137; C statistic,
0.758; suicidal thoughts: R2, 0.102; C statistic, 0.750) and de-
personalization (thoughts of attrition: R2, 0.097; C statistic,
0.717; suicidal thoughts, R2, 0.077; C statistic, 0.718), com-
pared with any of the dichotomized burnout definitions or
thresholds (thoughts of attrition: R2 range, 0.082-0.095; C sta-
tistic range, 0.707-0.720; suicidal thoughts: R2 range, 0.051-
0.060; C statistic range, 0.686-0.700) (P < .001 for all com-
parisons; Table 3).
The 2 burnout subscales, emotional exhaustion and de-
personalization, were associated with thoughts of attrition and
suicidal thoughts; more frequent symptoms were associated
with higher rates of thoughts of attrition and thoughts of sui-
cide, indifferent to the threshold assessed (Figure). However,
a clear inflection point was not observed for overall burnout,

E4 JAMA Surgery Published online September 9, 2020 (Reprinted) jamasurgery.com

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 09/10/2020

 Evaluating the Association of Multiple Burnout Definitions and Thresholds With Prevalence and Outcomes Original Investigation Research

emotional exhaustion, or depersonalization scores for either

Figure. Percentage of Residents Reporting Thoughts of Attrition
wellness outcome. and Suicidal Thoughts by Burnout Type

A Overall burnout score

80
Discussion Attrition
70
This comprehensive national study of US general surgery resi-
60

Reporting outcome, %
dents demonstrated considerable variability in burnout preva-
50
lence among general surgery residents, depending on the burn-
40
out definition or threshold selected. Available burnout
assessment instruments lack a clinically validated dichotomi- 30
Suicidality
zation threshold to signal the presence of burnout (ie, burned 20

out or not).2,9 Published prevalence cutoffs in the literature are 10

at the discretion of the investigators and vary considerably.8,11 0
Furthermore, although burnout is a multidimensional con- 0 10 20 30 40 50
struct, the requirement to define burnout as frequent symp- Overall burnout score
toms on 1 or multiple subscales is debated, and the decision
to include more than 1 subscale in the definition of burnout is B Emotional exhaustion score

also at the discretion of investigators. The result is wide vari- 70

ability of burnout prevalence estimates, a finding clearly dem- Attrition
60
onstrated in this study. Our results emphasize the need for re-

Reporting outcome, %
50
search on burnout to specify the definition and threshold for
burnout used and justify the rationale for using that specific 40

approach. Moreover, these results make it clear that compar- 30

Suicidality
ing burnout rates among studies requires careful consider-
20
ation of the burnout definition and threshold used.
10
Burnout lacks a consensus definition and clear measure-
ment standard both in medicine and other disciplines; how- 0
ever, the 3-domain definition developed by Maslach et al2 and 0 5 10 15 20 25
the MBI are the most common definition and measurement Emotional exhaustion score

standard. The World Health Organization classifies burn-out

C Depersonalization score
in the International Classification of Diseases, 11th Revision as
a multidimensional occupational phenomenon, not a medi- 50
Attrition
45
cal condition.23 In addition, burnout lacks a Diagnostic and Sta-
40
tistical Manual of Mental Disorders (Fifth Edition) definition.
Reporting outcome, %

35
This diagnostic ambiguity has led to numerous burnout as-
30
sessment tools and published thresholds without a single gold 25
standard. Accepted burnout assessment instruments range Suicidality
20
from simple single-item evaluations in which individuals use 15
their own definition of burnout (eg, the American Medical As- 10
sociation Mini Z) to more complex multidimensional inven- 5

tories, such as the Copenhagen and Oldenburg Burnout In- 0

ventories and the MBI. Furthermore, the intent of these 0 5 10 15 20 25

instruments is to evaluate burnout academically (ie, for sta-
tistical associations) and not clinically (ie, for a diagnosis); by
design, the tools evaluate burnout on a continuous scale and
do not recommend or provide a clinically validated threshold
or cutoff signifying a burnout diagnosis.
Using several published burnout categorizations and
thresholds on the same population, we found that burnout
prevalence estimates ranged widely from 3.2% to 91.4%, with
the most common definition estimating that 43.2% of gen-
eral surgery residents are experiencing weekly burnout symp-
toms. Accepting that burnout is a multidimensional con-
struct and distinct from mental illnesses, such as depression,
there are 2 important considerations with defining burnout:
frequency of symptoms and the association between the burn-
out subscales, emotional exhaustion, and depersonalization
Depersonalization score
A, Overall burnout score. The minimum score was 6, and the maximum score
was 42. B, Emotional exhaustion score. The minimum score was 3, and the
maximum score was 21. C, Depersonalization score. The minimum score was 3,
and the maximum score was 21. For all 3 scales, the score (ie, overall burnout
score, emotional exhaustion score, and depersonalization score) was the
summed score of the reported frequency of burnout symptoms, whereby every
day was a score of 7 and a few times a week was a score of 6, decrementing to
never (a score of 1) for each item response.
in this study. Because there is no accepted frequency that is
considered burned out, researchers must decide where to draw
the line: daily, weekly, monthly, or yearly symptoms. Clearly,
if the occurrence of even 1 symptom in a year is considered

jamasurgery.com (Reprinted) JAMA Surgery Published online September 9, 2020 E5

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 09/10/2020

 Research Original Investigation Evaluating the Association of Multiple Burnout Definitions and Thresholds With Prevalence and Outcomes

burnout, the prevalence is much higher than 1 in which burn-
out is considered to require daily symptoms. In addition, re-
searchers have disagreed about whether burnout requires the
occurrence of symptoms from more than 1 burnout subscale
(eg, both emotional exhaustion and depersonalization) or hav-
ing symptoms of 1 subscale (eg, emotional exhaustion or de-
personalization) is substantial enough to deem someone to be
experiencing burnout. Thus, these differences in how to de-
fine burnout lead to large differences in prevalence estimates
that make comparisons difficult between studies. This work
highlights the need for burnout research to specify and jus-
tify the definition of burnout used.
Previous studies have demonstrated an association be-
tween frequent burnout symptoms and poor wellness
outcomes.3-5 We found that increasing frequency of burnout
symptoms were significantly associated with both thoughts
of attrition and suicidal thoughts among general surgery resi-
dents. Importantly, this association was observed regardless
of the burnout definition selected, and the model diagnostics
were similar among the various definitions or thresholds tested.
Thus, the specific definition or threshold may not be particu-
larly important, but the definition must be specified. In gen-
eral, individuals experience burnout symptoms differently,24
such that weekly symptoms for 1 individual may be signifi-
cant enough to trigger suicidal thoughts, whereas another in-
dividual may experience burnout symptoms daily and not have
suicidal thoughts. More important than the actual preva-
lence estimates are the significant associations between fre-
quent burnout symptoms and poor wellness outcomes.
Future studies examining interventions should focus on re-
ducing the frequency of burnout symptoms instead of the
prevalence of burnout.
Limitations
This study has several potential limitations. First, as a cross-
sectional study, we could only explore associations and could
not identify causes. Second, the timing of the survey imme-
diately following the ABSITE may have affected resident re-
sponses, but the direction of the bias is uncertain because both
examination-associated distress and postexamination relief
may be occurring simultaneously. Third, because this was a
cross-sectional study and survey responses were completely
anonymous, we were unable to follow up or connect actual at-
trition and suicide rates among residents in this study. Fi-
nally, since survey questions involved exposures over the past
year, recall bias may exist. Nonetheless, this study offers the
opportunity to examine burnout measurement in a national
evaluation of surgical residents with a response rate that is sub-
stantially higher than that of prior studies.
Conclusions
In a national evaluation of general surgery residents, burnout
prevalence varied considerably depending on the definition
selected. These results emphasize the need for clear report-
ing of the criteria used for burnout assessment and a justifi-
cation for the rationale for using that approach.

ARTICLE INFORMATION
Accepted for Publication: May 21, 2020.
Published Online: September 9, 2020.
doi:10.1001/jamasurg.2020.3351
Author Contributions: Drs Bilimoria and Hewitt
had full access to all of the data in the study and
take responsibility for the integrity of the data and
the accuracy of the data analysis.
Concept and design: Hewitt, Ellis, Cheung,
Agarwal, Bilimoria.
Acquisition, analysis, or interpretation of data:
Hewitt, Ellis, Hu, Moskowitz, Agarwal, Bilimoria.
Drafting of the manuscript: Hewitt, Ellis, Bilimoria.
Critical revision of the manuscript for important
intellectual content: All authors.
Statistical analysis: Hewitt, Ellis, Bilimoria.
Obtained funding: Bilimoria.
Administrative, technical, or material support:
Ellis, Hu.
Supervision: Hu, Agarwal, Bilimoria.
Conflict of Interest Disclosures: Dr Bilimoria
reported grants from American College of
Surgeons, Association for Graduate Medical
Education, and American Board of Surgery during
the conduct of the study. Dr Agarwal reported
having received honoraria for lectures on burnout
and well-being. No other disclosures were
reported.
Funding/Support: This study is supported by
funding from the American College of Surgeons,
Accreditation Council for Graduate Medical
Education, and the American Board of Surgery. Drs
Hewitt and Ellis were supported by postdoctoral
research fellowships from the Agency for
Healthcare Research and Quality (grant
5T32HS000078). Dr Cheung was supported by a
postdoctoral research fellowship from the National
Science Foundation (grant 1714952).
Role of the Funder/Sponsor: The funders had no
role in the design and conduct of the study;
collection, management, analysis, and
interpretation of the data; preparation, review, or
approval of the manuscript; and decision to submit
the manuscript for publication.
REFERENCES
1. Freudenberger H. Staff burn-out. Journal of
Social Issues. 1974;30(1). doi:10.1111/j.1540-4560.
1974.tb00706.x
2. Maslach C, Jackson S, Leiter M. Maslach Burnout
Inventory Manual 4th edition. MindGarden: 2016.
3. Hu YY, Ellis RJ, Hewitt DB, et al. Discrimination,
abuse, harassment, and burnout in surgical
residency training. N Engl J Med. 2019;381(18):1741-
1752. doi:10.1056/NEJMsa1903759
4. Dyrbye LN, Thomas MR, Power DV, et al.
Burnout and serious thoughts of dropping out of
medical school: a multi-institutional study. Acad Med.
2010;85(1):94-102. doi:10.1097/ACM.
0b013e3181c46aad
5. Shanafelt TD, Balch CM, Bechamps G, et al.
Burnout and medical errors among American
surgeons. Ann Surg. 2010;251(6):995-1000.
doi:10.1097/SLA.0b013e3181bfdab3
6. Han S, Shanafelt TD, Sinsky CA, et al. Estimating
the attributable cost of physician burnout in the
United States. Ann Intern Med. 2019;170(11):784-790.
doi:10.7326/M18-1422
7. Shanafelt TD, Hasan O, Dyrbye LN, et al. Changes
in burnout and satisfaction with work-life balance in
physicians and the general US working population
between 2011 and 2014. Mayo Clin Proc. 2015;90
(12):1600-1613. doi:10.1016/j.mayocp.2015.08.023
8. Rotenstein LS, Torre M, Ramos MA, et al.
Prevalence of burnout among physicians:
a systematic review. JAMA. 2018;320(11):1131-1150.
doi:10.1001/jama.2018.12777
9. Bianchi R, Schonfeld IS, Laurent E. Burnout:
absence of binding diagnostic criteria hampers
prevalence estimates. Int J Nurs Stud. 2015;52(3):
789-790. doi:10.1016/j.ijnurstu.2014.12.008
10. Doulougeri K, Georganta K, Montgomery A.
“Diagnosing” burnout among healthcare
professionals: Can we find consensus? Cogent
Medicine. 2016;3(1):1. doi:10.1080/2331205X.2016.
1237605
11. Heinemann LV, Heinemann T. Burnout research:
emergence and scientific investigation of a
contested diagnosis. SAGE Open. 2017;7(1).
doi:10.1177/2158244017697154
12. National Academy of Medicine. Valid and
reliable survey instruments to measure burnout,
well-being, and other work-related dimensions.
Accessed August 3, 2020. https://nam.edu/valid-
reliable-survey-instruments-measure-burnout-well-
work-related-dimensions/

E6 JAMA Surgery Published online September 9, 2020 (Reprinted) jamasurgery.com

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 09/10/2020

 Evaluating the Association of Multiple Burnout Definitions and Thresholds With Prevalence and Outcomes Original Investigation Research

13. Pulcrano M, Evans SR, Sosin M. Quality of life
and burnout rates across surgical specialties:
a systematic review. JAMA Surg. 2016;151(10):970-
978. doi:10.1001/jamasurg.2016.1647
14. Elmore LC, Jeffe DB, Jin L, Awad MM, Turnbull
IR. National survey of burnout among US general
surgery residents. J Am Coll Surg. 2016;223(3):440-
451. doi:10.1016/j.jamcollsurg.2016.05.014
15. Lebares CC, Guvva EV, Ascher NL, O’Sullivan PS,
Harris HW, Epel ES. Burnout and stress among US
surgery residents: psychological distress and
resilience. J Am Coll Surg. 2018;226(1):80-90. doi:
10.1016/j.jamcollsurg.2017.10.010
16. McManus IC, Winder BC, Gordon D. The causal
links between stress and burnout in a longitudinal
study of UK doctors. Lancet. 2002;359(9323):
2089-2090. doi:10.1016/S0140-6736(02)08915-8
17. McManus IC, Gordon D, Winder BC. Duties of a
doctor: UK doctors and good medical practice. Qual
Health Care. 2000;9(1):14-22. doi:10.1136/qhc.9.1.14
18. American Board of Surgery. ABS in-training
examination. Accessed August 3, 2020. http://
www.absurgery.org/default.jsp?certabsite
19. Bilimoria KY, Chung JW, Hedges LV, et al.
Development of the flexibility in duty hour
requirements for surgical trainees (FIRST) trial
protocol: a national cluster-randomized trial of
resident duty hour policies. JAMA Surg. 2016;151(3):
273-281. doi:10.1001/jamasurg.2015.4990
20. Bilimoria KY, Chung JW, Hedges LV, et al.
National cluster-randomized trial of duty-hour
flexibility in surgical training. N Engl J Med. 2016;
374(8):713-727. doi:10.1056/NEJMoa1515724
21. Schaufeli WB, Bakker AB, Hoogduin K, Schaap
C, Kladler A. On the clinical validity of the Maslach
Burnout Inventory and the burnout measure.
Psychol Health. 2001;16(5):565-582. doi:10.1080/
08870440108405527
22. Kessler RC, Berglund P, Borges G, Nock M,
Wang PS. Trends in suicide ideation, plans,
gestures, and attempts in the United States,
1990-1992 to 2001-2003. JAMA. 2005;293(20):
2487-2495. doi:10.1001/jama.293.20.2487
23. World Health Organization. Burn-out an
“occupational phenomenon”: International
Classification of Diseases. Published May 28, 2019.
Accessed August 3, 2020. https://www.who.int/
mental_health/evidence/burn-out/en/
24. Leiter MP, Maslach C. Latent burnout profiles:
a new approach to understanding the burnout
experience. Burnout Research. 2016;3(4);89-100.
doi:10.1016/j.burn.2016.09.001

jamasurgery.com (Reprinted) JAMA Surgery Published online September 9, 2020 E7

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ by a Weill Cornell Medical Library User on 09/10/2020