Teaching and Learning in Medicine

An International Journal

ISSN: 1040-1334 (Print) 1532-8015 (Online) Journal homepage: http://www.tandfonline.com/loi/htlm20

A Rasch Analysis Validation of the Maslach

Burnout Inventory–Student Survey with Preclinical
Medical Students

Yang Shi, P. Cristian Gugiu, Remle P. Crowe & David P. Way

To cite this article: Yang Shi, P. Cristian Gugiu, Remle P. Crowe & David P. Way (2018): A Rasch
Analysis Validation of the Maslach Burnout Inventory–Student Survey with Preclinical Medical
Students, Teaching and Learning in Medicine, DOI: 10.1080/10401334.2018.1523010

To link to this article: https://doi.org/10.1080/10401334.2018.1523010

Published online: 21 Dec 2018.

Submit your article to this journal

Article views: 45

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=htlm20
TEACHING AND LEARNING IN MEDICINE
https://doi.org/10.1080/10401334.2018.1523010

VALIDATION

A Rasch Analysis Validation of the Maslach Burnout Inventory–Student

Survey with Preclinical Medical Students
Yang Shia , P. Cristian Gugiua , Remle P. Croweb , and David P. Wayc
a
Division of Quantitative Research, Evaluation and Measurement, Department of Educational Studies, The Ohio State University,
Columbus, Ohio, USA; bThe National Registry of Emergency Medical Technicians, Columbus, Ohio, USA; cDepartment of Emergency
Medicine, The Ohio State University Wexner Medical Center, Columbus, Ohio, USA

ABSTRACT KEYWORDS
Construct: Burnout is a psychological construct characterized by emotional exhaustion that Rasch analysis;
arises from an excess of physical, emotional, and social demands over an extended period. undergraduate medical
Symptoms of burnout include withdrawal or disengagement from work. Burnout has education; Maslach Burnout
Inventory–Student Survey;
become an important public health concern due to its association with severe negative con- validity; reliability
sequences across numerous professions. Background: The most widely used instrument for
measuring burnout is the Maslach Burnout Inventory (MBI). An adaptation of the MBI, the
MBI–Student Survey (MBI-SS), was developed for college students. The MBI-SS consists of 15
items covering 3 domains of burnout: exhaustion, cynicism (CY), and professional efficacy
(PE). Although studies have confirmed the validity of the MBI-SS for college student popula-
tions, studies of its use with medical students are limited. The purpose of this study was to
employ the Rasch model to examine the psychometric properties of the MBI-SS when used
with a population of preclinical medical students. Approach: Data were collected from 787
medical students who answered the MBI-SS at the conclusion of their 1st year. A maximum
likelihood exploratory factor analysis for ordinal data confirmed the hypothesized three fac-
tor structure of the MBI-SS. Subsequently, a Rasch analysis was employed to further evaluate
the measurement properties of MBI-SS. We used the Rasch Rating Scale model to investi-
gate the extent to which the three MBI subscales conformed to proper measurement char-
acteristics, including comprehensive coverage of person ability and item difficulty along the
latent continuum. Results: Most of the 15 items on the MBI-SS effectively fit the Rasch
Rating Scale Model, with minimal measurement error. Respondents effectively used the full
range of the rating scale for all 15 items. Two subscales (PE and CY) contained items that
were difficult for respondents to endorse, resulting in significant gaps along the measure-
ment continuum. The CY subscale exhibited a slight floor effect. The 3 subscales showed
good person reliability, good real-item reliability, and good person separation. Conclusions:
The Rasch analysis confirmed that the MBI-SS works well for measuring burnout among pre-
clinical medical students. However, the Rasch analysis was able to identify that additional
items are needed to improve the performance of MBI-SS. New items would be targeted at
reducing the floor effect for the CY subscale and filling the other gaps in measurement
along the latent continuum for the PE and CY subscales.

Introduction musculoskeletal disorders, hypertension, and myocar-

Occupational burnout represents a major public
health concern, affecting both the individual and the
organizations for which they work. Symptoms of
burnout include fatigue, callousness, withdrawal,
cynicism, low work morale, and deterioration of job
performance.1 This condition has also been linked
with more serious individual health problems,
including depression, sleep disturbances, alcoholism,
dial infarction.2–5 Burnout has further been shown to
negatively impact work organizations. Besides reduced
job performance of workers, other problems such as
increased absenteeism and employee turnover affect
organizations.5–7
The term burnout was first coined in the 1970s to
describe the “physical and mental exhaustion that
results from emotionally demanding interactions with

CONTACT David P. Way David.Way@osumc.edu The Ohio State University Wexner Medical Center, Department of Emergency Medicine 778 Prior
Hall, 376 W 10th Avenue, Columbus, OH 43210, USA.
Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/htlm.
ß 2018 Taylor & Francis Group, LLC
2 Y. SHI ET AL.
people.”8,9 As originally conceived, burnout was
thought to be a syndrome affecting primarily frontline
human service workers in occupations such as law
enforcement, healthcare, social work, education, and
so on.9 By the 1990s, a broader definition of burnout
was adopted, which recognized that individuals out-
side of human service occupations could also exhibit
symptoms of burnout.10 This broader definition sug-
gested that burnout could be caused by any individu-
al’s interaction with the stressful demands of work.11
Beyond those in the workforce, burnout was thought
to also impact nonworkers, such as students, who suf-
fer from burnout as a consequence of the stress cre-
ated by their studies.12
Although burnout has been distinguished from
other psychological conditions including depression
and anxiety, as of yet there are no objective clinical
diagnostic tests for burnout outside of self-report
questionnaires.13,14 Numerous questionnaire instru-
ments exist for the measurement of burnout; however,
the most commonly used and studied is the Maslach
Burnout Inventory (MBI).15 By some estimates, the
MBI has been used in more than 90% of empirical
studies of burnout worldwide.16 The original MBI was
introduced in 1981 and was designed for measuring
burnout in individuals who worked in professions
involving high levels of interaction with people. This
instrument consisted of 22 items intended to measure
three domains of burnout: exhaustion, depersonaliza-
tion, and personal accomplishment.17 Of interest, the
MBI has traditionally been considered a classification
tool that assigns individuals to categories of burnout
rather than points on a burnout continuum.7,9
Establishing a continuous measure of burnout would
provide more information about the “degree of
burnout” an individual may be experiencing.18
Since its inception, the MBI has been adapted for
populations beyond the original human service
worker. The MBI General Survey (MBI-GS) was
developed to accommodate the wider scope of burn-
out and was adapted for use with a broader popula-
tion of workers.7 The MBI-GS was reduced to 16
items that measured exhaustion, cynicism, and profes-
sional efficacy.7 Work by Gold and by Balogun et al.
contributed to Schaufeli’s development of a version of
the MBI for measuring burnout in undergraduate col-
lege students.12,19,20 The MBI Student Survey (MBI-
SS) consists of 15 items organized into three sub-
scales.12 As in the MBI-GS, cynicism replaced deper-
sonalization and professional efficacy replaced
personal accomplishment. The MBI-SS uses the same
response set as all other MBI instruments: a 7-point
behavioral frequency scale ranging from 1 (never), 2
(a few times a year or less), 3 (once a month or less), 4
(a few times a month), 5 (once a week), 6 (a few times
a week), to 7 (every day).
Since the establishment of the burnout construct,
effective measurement has been of great interest to
researchers. As of the time this article was written,
more than 3,400 peer-reviewed articles were found
through the search term “Maslach Burnout Inventory”
in the medical and social science databases. A large
proportion of these articles were psychometric evalua-
tions of the various MBI instruments that employed
either exploratory factor analysis (EFA) or confirma-
tory factor analysis to verify dimensionality of the
tools. In a meta-analysis of studies that used con-
firmatory factor analysis to investigate the domain
structure, 18 of the 21 studies concluded that a three-
factor model provided the best fit.21
Other studies have attempted to verify the unidimen-
sionality of each of the three subscales by calculating
the internal consistency coefficients (Cronbach’s alpha).
Estimated alpha reliabilities from a sample of human
service workers reported in the original MBI article by
Maslach and Jackson (1981) were 0.90 for Exhaustion,
0.79 for Depersonalization, and 0.71 for Personal
Accomplishment subscales.22 A meta-analysis of 84
studies published between 1995 and 2006 reported 95%
confidence intervals for Cronbach’s alpha estimates
weighted for sample size as 0.87–0.88 for Exhaustion,
0.77–0.79 for Depersonalization, and 0.77–0.79 for
Personal Accomplishment.23 Of interest, all estimates
of internal consistency reliability reported in the litera-
ture to date have treated MBI response set data as inter-
val, continuous data rather than ordinal,
noncontinuous data.24,25
Very few MBI validation studies provide new infor-
mation regarding the dimensions of the underlying
construct, the burnout, or about the effectiveness of
the measurements. Moreover, it is unknown whether
the MBI can be effectively transformed from a blunt
instrument of categorization into a more informa-
tional instrument, with associated measurement con-
tinuum. Toward this end, Rasch modeling provides an
alternative to investigating the psychometric properties
of the MBI.
The key to the effectiveness of the Rasch model as
an alternative method of evaluating the psychometric
properties of instruments is the transformation of rat-
ings to probabilities. This transformation effectively
converts the rating scale from ordinal (rank-order) to
interval (continuous) level measurement of the latent
trait of interest—in our case, burnout.18,26 Distinct

from dimensional analyses, like EFA, Rasch modeling
allows separate but integrated analyses of both person
and item effects. Person ability is loosely defined as a
person’s relative standing on the latent trait of inter-
est, the probability that he or she will endorse a par-
ticular rating for a specific item.27 The interaction of
the person with the item produces an estimate of the
item difficulty. Estimates of item difficulty allow
researchers to examine whether an instrument con-
tains items the options of which are too difficult (floor
effect) or too easy (ceiling effect) for the persons of a
certain ability level to endorse.28 Comparing the dis-
tributions of person ability and item difficulty cap-
tured by an instrument can reveal gaps in
measurement that potentially deflate effect sizes, reli-
ability estimates, and correlation coefficients.29
Insights such as these are not attainable through clas-
sical item analysis methods.30–33
Although there is a tremendous amount of research
exploring the structural validity of the MBI, the same is
not true for the MBI-SS. Most investigations have used
the MBI-SS to study the relationship of burnout to
other variables, such as professional behavior, academic
performance, or suicide ideation.20,34,35 A few structural
validity studies of the MBI-SS have been conducted but
primarily with students outside of the United States and
none with medical students.36–38 Although we found a
few studies in the medical and medical education litera-
ture that employed Rasch analysis in evaluating meas-
urement instruments, surprisingly we found no
literature on its use with the MBI-SS and its associated
latent trait burnout.39,40 The purpose of this study was
to investigate the psychometric properties of the MBI-
SS using Rasch analysis and to determine whether the
MBI-SS was a practical instrument for measuring burn-
out among preclinical medical students.
Methods
Participants
The data for this analysis were obtained from preclin-
ical, 1st-year medical students enrolled at a large mid-
western medical school. Students completed the MBI-
SS at the end of the academic year as part of a battery
of instruments used for program evaluation. The
MBI-SS was slightly modified from the original MBI-
SS by Schaufeli for use with these medical students by
replacing the term “university” with “medical
school.”12 The final data set for analysis consisted of
complete MBI-SS data for 787 medical students, who
were enrolled in the 1st-year course at the time of
annual data collection in May 2009–2013. These
TEACHING AND LEARNING IN MEDICINE 3
participants represented approximately 75% of the
total population of 1st-year medical students
(N ¼ 1,052) for the same years. Class cohorts (year of
entry) and gender groups were equally represented in
the data set, with participants by gender being 54.2%
male (n ¼ 427) and 45.7% female (n ¼ 360). Our
Institutional Review Board approved the collection of
MBI-SS scores for educational program evaluation
(Protocol 2008B0122). In addition, our Institutional
Review Board approved the use of these MBI-SS
scores for the purposes of instrument validation
research (Protocol 2012B0331).
Data analysis
Dimensionality. To verify the hypothesized dimension-
ality of the MBI-SS (exhaustion, cynicism, and profes-
sional efficacy), we performed a preliminary
maximum likelihood method of EFA using the poly-
choric correlation matrix for input (ordinal ML-EFA).
A parallel analysis was used to identify the number of
factors to extract, and Promax rotation (j ¼ 4) was
used to achieve simple structure.41 The parallel ana-
lysis method has been shown to have an accuracy rate
of higher than 90% in identifying the correct number
of factors to retain.42 An ordinal version of this
method is available that uses polychoric correlations,
which are considered unbiased estimates of the true
correlations had an ordinal response scale not been
used.43–45 SAS Version 9.4 (SAS Institute, Inc., Cary,
NC) was employed to do all statistical calculations.
Rasch analysis. Rasch modeling was performed
using WINSTEPS version 4.1.0 (Linacre JM,
Beaverton, OR) on each of the MBI-SS subscales sep-
arately. The Rating Scale Modeling, which is fixed at
the threshold across all items, was employed to exam-
ine the MBI-SS measurement properties.46
Item fit. A critical step in Rasch analysis is to
evaluate the fit of the data to the model. To this end,
we assessed model fit using the information-weighted
mean square (Infit MNSQ), outlier-sensitive mean
square (Outfit MNSQ), and point-measure correlation.
Outfit MNSQ is sensitive to people with an ability
level far from the item difficulty level, whereas the
Infit MNSQ is sensitive to people with an ability level
close to the item difficulty level.40 Values of Infit and
Outfit MNSQ statistics in the range of .5 to 1.5 were
regarded as satisfactory.47 Items that did not meet this
criterion were dropped from subsequent analyses.
Similarly, good fitting items should exhibit a positive
correlation with the subscale formed by the Rasch
analysis (point-measure or item-Rasch measure
4 Y. SHI ET AL.

Table 1. Item statistics generated from Rasch analysis, their definitions and guide for interpretation.
Rasch term or statistic Meaning/implications
Scale or Subscale Used to identify or verify the existence of mean- Dimensionality from EFA: Interpretable factor load-
ingful groups of items designed to measure
smaller components of a larger more compre-
hensive construct.
Scale Item Items within a subscale
Scale Category Response categories within a subscale
Item Difficulty/Logit Measure of how easy or hard it is for a person to For our study, large positive item difficulties repre-
endorse an item. Hard items are endorsed only
by a person experiencing high levels of burnout.
Easy ones are endorsed only by those experi-
encing low levels of burnout.
Standard Error The quality or precision of the estimate of
item difficulty.
Infit MNSQ: Information Weighted Mean Square Used to identify irregular or unexpected patterns
Outfit MNSQ: Outlier Sensitive Mean Square in the data. Helps to evaluate how well items
fit the Rasch model.
Point-Measure Correlation This is like a point-polyserial correlation between
the item and the total subscale
Item Reliability Indicator of consistency with which persons esti-
mate item and response option difficulties, that
is, the rank order in the item-response
difficulties.
Person Reliability Indicator of consistency with which items estimate
the latent score for persons, that is, the consist-
ency in the rank-order of people’s level
of burnout.
Real and Model Reliabilities Serve as lower and upper bounds of person and
item reliability estimates.
Person Separation Indicator as to how well an item discriminates
between persons suffering low and high levels
of burnout.
Wright-Andrich Map A graphical representation of the distributions of
person measures (scores) and item measures
(difficulties) on the latent trait.
Rasch-Andrich Threshold Most easily understood as part of the Category
Probability Curves: the Andrich Threshold is the
point at which one category probability curve
crosses an adjacent category probability curve.
Category Measure Similar to Item Measure, Category Measures indi-
cate how easy or hard it is for a person to
endorse a particular category or rating on a rat-
ing scale. Difficult to endorse categories are
endorsed only by persons experiencing high lev-
els of burnout. Easy ones are endorsed only by
those experiencing low levels of burnout.
Category Probability Curve A graphical representation of the probability (likeli- Ideally, each probability curve should have a
hood) of the response category being selected
over the range of the latent trait continuum.
Note: EFA ¼ exploratory factor analysis.
Criterion for interpretation/evaluation
ings and a match between hypothesized sub-
scales and factors extracted.
Overall model fit from Rasch: All items consist
of proper fit statistics (.5–1.5).
Absence of gaps in measurement across the
latent trait scale.
Proper fit statistics, positive point-measure correl-
ation, orderly response categories, item
reliability.
Each category is endorsed by at least 10 persons.
Category measure falls into hypothesized order
among other categories. Has Outfit MNSQ <2.0.
sented difficult items and large negative difficul-
ties represented easy items.
The closer the standard error is to zero, the better.
Large positive standard errors means a lack of
precision in estimating item difficulty.
Should fall into this range (.5–1.5) for productive
measurement
Good fitting items should exhibit positive point-
measure correlations.
Reliabilities > .70 are acceptable for research pur-
poses.
Reliabilities > .95 are acceptable when scores
are being used for decisions about individ-
ual persons.
Separation indices >2.0 are satisfactory for
research purposes.
The person distribution on the left side of the map
should align with the item distribution on the
right side of the map.
Threshold values should increase across the
response categories.
For our study, high positive category measures
represented difficult categories and high nega-
tive category measures represented
easy categories.
region along the latent continuum in which it is
the most probable option. Relatively speaking,
category curves should not overlap or be sepa-
rated too much.

correlation). A negative point-measure correlation is
an indication that response categories have been
reversed (i.e., lower response categories have a higher
item measure than subsequent response categories).
Essentially, the latter two statistics test whether
respondents treated the response options as if they
were ordinal in nature. Items that lacked ordinal
properties were also dropped from subse-
quent analyses.
Two indices of reliability were estimated, one for
persons and the other for items. Person reliability, an
equivalent to the traditional “test” reliability, indicates
the consistency with which the items estimate the
latent score for persons (i.e., the frequency with which

Figure 1. Plot of actual versus randomly generated eigenvalues from Ordinal Parallel Analysis.
they experienced symptoms of burnout), whereas item
reliability indicates the consistency with which persons
estimate item and response option difficulties (i.e., the
rank order and relative spacing between items). That
is, person reliability is a measure of the extent to
which items are able to spread people out along the
latent continuum, whereas item reliability is a measure
of the extent to which people are able to spread items
out along the same continuum. The “real” and
“model” person reliabilities serve as lower and upper
bounds to these values, respectively. Although reliabil-
ity is a function of both the instrument and the sam-
ple to which it was administered, standards do exist
for acceptable levels of reliability ranging from .70 to
.95.48,49 Nunnally and Bernstein suggested that reli-
abilities of .70 are acceptable for research,48 whereas
Gugiu and Gugiu illustrated that decisions about per-
sons (interpretation of individual-level scores) require
reliabilities greater than .95 to offset even modest
margins of measurement error.49 Similarly, person
separation is an indication of item discrimination
around ability levels and separation index values of
larger than 2 are satisfactory for clinical research.32
We used a graphical method (Wright-Andrich
Maps) to plot the distributions of items and persons
for each MBI subscale. Ideally, the distribution of
items (or, in our case, response categories) generated
from difficulty scores should align with the
TEACHING AND LEARNING IN MEDICINE 5
distribution of person ability scores. Such an align-
ment or coverage is an indication that the instrument
is well suited for measuring burnout in the population
of preclinical medical students. Gaps in measurement
occur when the person’s and item’s distributions do
not align. Gaps indicate a lack of precision in effective
measurement at that place on the scale. Gaps can
appear anywhere along the latent continuum but are
most commonly observed as either floor effects (not
enough easy items to cover the bottom of the person
distribution) or ceiling effects (not enough difficult
items to cover the top of the person distribution).
Our criteria for evaluating measurement gaps was
inspired by Baghaei.50 Floor and ceiling effects were
defined as the existence of persons with logit scores at
the bottom or top of the persons distribution (left
side of the Wright-Andrich Map) that were at least
one logit from the nearest item measure (right side of
the Wright-Andrich Map). Specifically, we considered
effects to be mild if less than 10% of respondents met
this definition, moderate if 10% to 20% met the defin-
ition, and severe if more than 20% of respondents met
the definition.
Rating scale. The rating scale is considered an indi-
cation that higher ratings are consistently used or pro-
vided by higher functioning individuals, and vice
versa. We evaluated the functionality of the MBI-SS
7-point rating scale (also referred to as response
6 Y. SHI ET AL.

category scale) using criteria proposed by Linacre,

which include the occurrence of more than 10
endorsements per response category, the observation
that both average measures and category thresholds
increase across each response category, and an
observed out-fit mean square residual (MNSQ) value
of less than 2 for each response category.51
Finally, a graphical inspection of the category prob-
ability curves was performed to determine how well
respondents effectively used all seven response catego-
ries. Ideally, every probability curve should have a
region along the latent continuum in which it is the
most probable option. A curve (response categories)
that is buried under an adjacent curve is an indication
that respondents did not effectively discriminate
between these two response options, thereby suggest-
ing that the response options should be reduced or
grouped.52 In contrast, large distances between the
peaks of adjacent curves would indicate that respond-
ents might need an additional response category
between these two options.52 Table 1 provides a sum-
mary of these Rasch terms and statistics along with
their definitions and criteria for interpretation
and evaluation.

Results
Dimensionality
The EFA and parallel analysis procedure generated
eigenvalues from the polychoric correlation matrix
along with the mean eigenvalues and eigenvalues rep-
resenting the 95th percentile based on the Monte
Carlo simulation. This analysis revealed that three
eigenvalues (factors) fell above the 95th percentile
estimates for the simulated eigenvalues (see Figure 1).
In other words, the MBI-SS has three factors consist-
ent with theory and past studies.12 According to the
path diagram (Figure 2) generated by the ML-EFA, all
items loaded on their expected factor. The exhaustion
factor (EX) comprised five items with factor loadings
from .55 to .80, the professional efficacy factor (PE)
consisted of six items with factor loadings from .43 to
.73, and the cynicism factor (CY) consisted of five
items with factor loadings from .32 to .80.
Figure 2. Path diagram for Maslach Burnout Inventory–Student
Survey (MBI-SS).
Overall model fit
All items demonstrated Infit and Outfit MNSQ statis- of measurement because no disordered response cate-
tics within the range of .5 to 1.5 and consisted of gories were found. Because the Rasch model assumes
positive item-Rasch measure correlations, suggesting unidimensionality, these fit statistics further confirm
that all the items fit the model (see Table 2). In add- the dimensionality of the MBI-SS, as does the fact
ition, the response scales functioned as ordinal levels that more than 10 participants selected each response
 TEACHING AND LEARNING IN MEDICINE 7

Table 2. Item statistics including difficulty (in logits) infit and outfit mean square and
point-measure correlations for three MBI-SS subscales dimensions along with separ-
ation and reliability estimates by subscale.
Scale item Item difficulty SE Infit MNSQa Outfit MNSQa Point-measure correlation
EX
EX4 0.94 0.05 1.16 1.17 0.77
EX5 0.09 0.05 0.89 0.89 0.85
EX1 0.23 0.05 0.81 0.81 0.86
EX3 0.29 0.05 1.18 1.18 0.82
EX2 0.33 0.05 0.89 0.89 0.85
rPE
PE5 1.55 0.05 1.24 1.16 0.60
PE4 0.49 0.04 0.96 0.93 0.67
PE1 0.35 0.04 0.95 0.93 0.66
PE3 0.12 0.04 0.81 0.79 0.74
PE6 1.09 0.04 0.78 0.78 0.77
PE2 1.18 0.04 1.32 1.33 0.67
CY
CY4 0.42 0.05 1.02 0.99 0.78
CY1 0.15 0.05 0.98 0.91 0.83
CY3 0.23 0.05 1.03 1.03 0.81
CY2 0.34 0.05 0.91 0.95 0.84
Person Item
Separation Reliability Reliability
EX
Real 2.74 0.88 0.99
Model 3.19 0.91 0.99
rPE
Real 1.93 0.79 1.0
Model 2.23 0.83 1.0
CY
Real 2.14 0.82 0.97
Model 2.44 0.86 0.97
Note: MBI-SS ¼ Maslach Burnout Inventory–Student Survey; Infit MNSQ ¼ information-weighted mean
square; Outfit MNSQ ¼ outlier-sensitive mean square; EX ¼ Exhaustion; rPE ¼ Reversed Professional
Efficacy; CY ¼ Cynicism.
a
Values in the range of .5 to 1.5 indicate a good fit.

Table 3. Response category (rating scale) diagnostics based on Linacre’s criteria involving observations, Outfit
MNSQ, Rasch-Andrich Threshold, and category measure.
Scale category Observed count % of Counts Outfit MNSQ Rasch-Andrich Threshold Category measure
Exhaustion
1 (Never) 156 4 1.33 None 6.42
2 (A few times a year or less) 547 14 .97 5.26 4.03
3 (Once a month or less) 1,340 34 .97 2.75 1.46
4 (A few times a month) 920 23 .91 .09 .49
5 (Once a week) 509 13 .97 1.3 1.82
6 (A few times a week) 361 9 .91 2.2 3.5
7 (Every day) 102 3 1.28 4.6 5.76
Reversed Professional Efficacy
1 (Never) 600 13 1.06 None 4.65
2 (A few times a year or less) 1,347 29 .89 3.47 2.52
3 (Once a month or less) 1,094 23 .88 1.32 1.01
4 (A few times a month) 1,031 22 1.01 .58 .21
5 (Once a week) 478 10 .91 .91 1.38
6 (A few times a week) 125 3 1.12 2.15 2.42
7 (Every day) 47 1 1.60 2.30 3.8
Cynicism
1 (Never) 575 18 1.21 None 5.17
2 (A few times a year or less) 800 25 .9 3.98 3.02
3 (Once a month or less) 1,043 33 .84 2.0 .86
4 (A few times a month) 387 12 .92 .48 .58
5 (Once a week) 179 6 .82 1.12 1.47
6 (A few times a week) 100 3 .91 1.71 2.44
7 (Every day) 64 2 1.7 2.67 4.00
Note: MNSQ ¼mean square.
8 Y. SHI ET AL.

Figure 3. Results of Rasch Analysis, Wright-Andrich Map of Exhaustion subscale with Person distribution on the left of the center
line, and Item distribution on the right. Note: The Measure Scale (7 to þ7) is the logit scale resulting from the Rasch Analysis.

category across all items. Hence, the Andrich
Thresholds and average category measure were stable
(Table 3).
Item and person results by subscale
Exhaustion. Column 2 of Table 2 shows the difficulties
for the items that compose the Exhaustion subscale.
The values presented in this column can be inter-
preted as the average difficulty estimates for the seven
response options that comprise that item. The items
are presented in order from most difficult (high
exhaustion) to easy (low exhaustion). For medical stu-
dents, EX4, “Studying or attending a class is really a
strain for me,” was rated higher as a source of exhaus-
tion, whereas EX 2, “I feel used up at the end of a day
at medical school,” was rated lower. The Exhaustion
subscale was found to have person reliability of .88,
real-item reliability of .99, and real person separation
of 2.61, all of which are satisfactory for research.48
The Wright-Andrich Map (Figure 3) plots the per-
son measures along the left side and the item

Figure 4. Category probability curves for Exhaustion Item 1.
difficulties for each of the EX items along the right
side, allowing one to compare item difficulties to the
person measures. The M found to the left of the cen-
ter line denotes the mean of the person logits on the
left. The M on the right denotes the mean of the
option (item) logits. The S’s and T’s denote 1 and 2
standard deviations from the means, respectively. The
two distributions are roughly aligned, (approximately
.5 logit difference), as demonstrated by the proximity
of the average person measure to the average item
measure. Furthermore, the range of the item measures
covers the vast majority of person measures with no
gaps of notable significance (>1 logit) along
the continuum.
Figure 4 shows the category probability curves for
the Exhaustion response options. By evaluating the
relative position of the response categories to one
another, we see that respondents may have been able
to effectively distinguish between all seven categories
as demonstrated by the fact that no probability curves
were “buried” under other curves. Examination of the
interval between the peaks of the curves, however,
suggests that respondents may have been able to dis-
tinguish two additional response categories: between 1
(never) and 2 (a few times a year) and between
options 2 (a few times a year) and 3 (once a month
or less).
TEACHING AND LEARNING IN MEDICINE 9
Reversed Professional Efficacy. Table 2 shows that
PE5, “I have learned many interesting things during
the course of my studies,” originally had the lowest
ratings, which after reverse coding ended up being
the highest ratings. Students who selected the high
reverse-coded ratings would be higher on the
Burnout Latent Trait Scale. The opposite would be
true for PE2, “I believe that I make an effective
contribution to the classes I attend,” that is, stu-
dents who selected the low reverse-coded ratings for
PE2 would be lower on the Burnout Latent Trait
Scale. The reliability indices for the Reversed
Professional Efficacy (rPE) subscale were also
adequate: person reliability of .79, real-item reliabil-
ity of 1.0, and model person separation of 2.23.
Figure 5 presents the Wright-Andrich Map for
options on the rPE subscale. From this figure, we see
a lack of alignment between the person and item dis-
tributions, which is best characterized as the 1-point
logit difference between the means of the items and
persons. Although the rPE scale attained an acceptable
level of reliability (person reliability ¼ .79 and real-
item reliability ¼ 1.00), the person reliability was con-
siderably lower than those for the EX subscale. The
likely explanation for the lower reliability is the lack
of alignment between the distributions of person and
item measures. Essentially, respondents found the
10 Y. SHI ET AL.

Figure 5. Results of Rasch Analysis, Wright-Andrich Map of Reversed Professional Efficacy subscale with Person distribution on the
left of the center line, and item distribution on the right. Note: The Measure Scale (6 to þ4) is the logit scale resulting from the
Rasch Analysis.

items to be a bit too difficult to endorse. Therefore,
more precise measurement would be achieved by add-
ing a few easier items to the rPE subscale.
Figure 6 shows the response option probability
curves for the rPE subscale. None of the probability
curves were “buried” under other curves, indicating
that respondents were able to effectively distinguish
between the seven response categories. Examination of
the interval gaps between options 6 (a few times a
week) and 7 (every day) indicate the probable need for
more response categories between these two options.
An additional response category may also need to be
inserted between options 1 (never) and 2 (a few times
a year or less).

Figure 6. Category probability curve for Item 1 of reversed professional efficacy.
Cynicism. The Cynicism subscale performed more
poorly when compared to the other two subscales,
perhaps because CY consists of fewer items and the
item difficulties are more closely clustered (Table 2).
Although the overall person and item measure reli-
abilities were good (person, real-item reliability, and
person separation of 0.80, 0.97 and 2.14, respect-
ively), two significant gaps were observed along the
latent continuum (see Figure 7). As was the case
for the rPE subscale, the distributions between the
person measures and the option-item measures did
not align well. The means of the two distributions
were off by about a 1.5 logit difference. In addition,
a mild floor effect was detected, which can be seen
by the lack of overlap between persons and items at
the bottom of the figure. Given the short length of
the subscale, its measurement precision could be
improved with the addition of a few midrange and
easier items.
Similar to the other two subscales, each category in
turn was the modal (most probable) category at some
point on the latent variable, indicating that all catego-
ries performed distinctly different from other catego-
ries (see Figure 8). As was true of the EX subscale,
respondents may have been able to distinguish
between three additional response categories on the
CY subscale: one between options 1 (never) and 2 (a
few times a year or less), one between options 2 (a few
TEACHING AND LEARNING IN MEDICINE 11
times a year or less) and 3 (once a month or less), and
another between options 6 (a few times a week) and 7
(every day).
Discussion
This study aimed to explore the psychometric prop-
erties of the MBI-SS for use with preclinical medical
students and to identify areas of the instrument that
may benefit from further development to enhance
measurement effectiveness. To this end, we
employed Rasch modeling to investigate the struc-
tural and response option validity, along with the
reliability of the instrument. Based on the reliability
estimates and comparison of the distribution of per-
son ability to item difficulty, all three MBI-SS sub-
scales were found to function adequately but not
optimally. We detected a few important areas for
improvement.
The Cynicism subscale exhibited two significant
gaps and a mild floor effect along the latent con-
tinuum, indicating regions in which the amount of
burnout a medical student experienced was not meas-
ured with adequate precision. This signals the need
for improvement by adding items or response options
to fill in the measurement gaps. Because gaps
occurred between the hardest item of Threshold 2
(point demarking the difference between response
12 Y. SHI ET AL.

Figure 7. Results of Rasch Analysis, Wright-Andrich Map of Cynicism subscale with Person distribution on the left of the cen-
ter line, and Item distribution on the right. Note: The Measure Scale (6 to þ5) is the logit scale resulting from the
Rasch Analysis.

categories 1 and 2) and the easiest item of Threshold
3 (point demarking the difference between response
categories 2 and 3) adding three to four items easier
than Item 2 would most effectively close both gaps
and eliminate the floor effect. A similar pattern,
though somewhat milder, was observed with the
EX subscale.
One strategy for improving measurement precision
for all three subscales would be to expand the number
of response categories from seven to nine, such as
1 (never), 2 (once or twice a year), 3 (three to five times
a year), 4 (six to 11 times a year), 5 (once a month), 6
(a few times a month), 7 (once a week), 8 (a few times
a week), and 9 (every day). Another strategy would be

Figure 8. Category probability curve for Item 1 of Cynicism.
to compare the MBI-SS to the original 22 items of the
MBI to see if important items were lost when translat-
ing the measurement of burnout from work to school.
Finally, the medical school context requires some
modification of items originally written for an under-
graduate college population. For instance, because stu-
dents in medical school study independent of their
attendance in class, EX4 becomes a double-barreled
question (i.e., two questions in one). In our medical
school, studying and attending class are competing
endeavors, making this item confusing for the medical
school student.
The reliability of the three subscales averaged
around 0.8. This level of reliability is good for
research purposes at the group level but would not be
adequate should medical schools want to use the
instrument to screen students for burnout and identify
those in need of support. To make decisions at the
individual level, one would need the overall reliability
to exceed .90.49 According to the Spearman-Brown
prophecy formula, doubling the length of the survey
would yield a reliability of about .88 (¼ 2  .80/
(1 þ . 8)).53 This formula highlights the nonlinear rela-
tionship between test length and test reliability coeffi-
cient.54 Hence, if medical schools are interested in
adopting the instrument to identify students in danger
TEACHING AND LEARNING IN MEDICINE 13
of experiencing burnout, a possible solution would be
to increase the number of items within subscales by a
factor of 2 so that each subscale would contain 10 to
15 items, approximately the length of the original
MBI. Future research is needed to improve the MBI-
SS for use with preclinical medical students and to
establish criteria for interpreting scores on the three
MBI-SS measurement scales.
Conclusion
This study employed the Rasch analysis to assess the
performance of the MBI-SS among preclinical 1st-year
medical students. This research provides additional
evidence for the structural validity and reliability of
the MBI-SS. The MBI-SS demonstrated satisfactory
unidimensionality for each of the three subdomains.
Regarding response option validity, we found that the
7-point rating scale functioned adequately; however,
there is evidence to suggest that respondents may be
able to effectively distinguish between 9 points and
may need that many to improve measurement preci-
sion. Although the subscales are appropriately struc-
tured (according to the hypothetical construct), we
found significant gaps in measurement along the
latent scale creating additional precision problems.
14 Y. SHI ET AL.
Suggestions for addressing these measurement gaps
include the addition of items. These efforts would
improve precision but would not likely appreciatively
improve the scale reliability. Nevertheless, if medical
school programs intend to use the instrument to iden-
tify preclinical students in need of support services,
then this would be a necessary step to increase their
person reliability above 0.90.
ORCID
Yang Shi http://orcid.org/0000-0001-9995-4370
P. Cristian Gugiu http://orcid.org/0000-0003-0022-287X
Remle P. Crowe http://orcid.org/0000-0001-9733-9294
David P. Way http://orcid.org/0000-0002-1896-3425
References
1. Bakker AB, Demerouti E, Sanz-Vergel AI. Burnout
and work engagement: The JD–R approach. Annu Rev
Organ Psychol Organ Behav. 2014;1(1):389–411.
2. Madsen IEH, Lange T, Borritz M, Rugulies R.
Burnout as a risk factor for antidepressant treat-
ment–a repeated measures time-to-event analysis of
2936 Danish human service workers. Journal of
Psychiatric Research. 2015;65:47–52.
3. Peterson U, Demerouti E, Bergstr€ om G, Samuelsson
M, Asberg M, Nygren A. Burnout and physical and
mental health among Swedish healthcare workers. J
Adv Nurs. 2008;62(1):84–95.
4. Shanafelt TD, Sloan JA, Habermann TM. The well-
being of physicians. Am J Med. 2003;114(6):513–519.
5. Sorour AS, El-Maksoud MMA. Relationship between
musculoskeletal disorders, job demands, and burnout
among emergency nurses. Advanced Emergency
Nursing Journal. 2012;34(3):272–282.
6. Borritz M, Christensen KB, B€ ultmann U, et al. Impact
of burnout and psychosocial work characteristics on
future long-term sickness absence. Prospective results
of the Danish PUMA Study among human service
workers. J Occup Environ Med. 2010;52(10):964–970.
7. Maslach C, Jackson SE, Leiter MP. Maslach Burnout
Inventory Manual. Palo Alto, California: Consulting
Psychologists Press, 1996.
8. Freudenberger HJ. Staff Burn-Out. Journal of Social
Issues. 1974;30(1):159–165.
9. Milicevic-Kalasic A. Burnout Examination. New York,
NY: Springer Science & Business Media, 2012.
10. Kitaoka-Higashiguchi K, Nakagawa H, Morikawa Y,
et al. Construct validity of the Maslach Burnout
Inventory-General Survey. Stress and Health. 2004;
20(5):255–260.
11. Demerouti E, Bakker AB, Nachreiner F, Schaufeli
WB. The job demands-resources model of burnout. J
Appl Psychol. 2001;86(3):499
12. Schaufeli WB, Martinez IM, Pinto AM, Salanova M,
Bakker AB. Burnout and engagement in university
students: A cross-national study. Journal of Cross-
Cultural Psychology. 2002;33(5):464–481.
13. Leiter MP, Durup J. The discriminant validity of
burnout and depression: A confirmatory factor ana-
lytic study. Anxiety, Stress, and Coping. 1994;7(4):
357–373.
14. Leone SS, Wessely S, Huibers MJ, Knottnerus JA,
Kant I. Two sides of the same coin? On the history
and phenomenology of chronic fatigue and burnout.
Psychology and Psychol Health. 2011;26(4):449–464.
15. Schaufeli WB, Enzmann D, Girault N. Measurement
of Burnout: A Review. Philadelphia, PA: Taylor &
Francis, 1993.
16. Schaufeli W, Enzmann D. The Burnout Companion to
Study and Practice: A Critical Analysis. London;
Philadelphia, PA: Taylor & Francis, 1998.
17. Maslach C, Jackson SE, Leiter MP, Schaufeli W,
Schwab RL. Maslach Burnout Inventory. 2017.
Available at: http://www.mindgarden.com/117-mas-
lach-burnout-inventory. Accessed January 4, 2018.
18. Wright BD, Masters GN. Rating Scale Analysis: Rasch
Measurement. Chicago, IL: MESA Press; 1982:60–89.
19. Gold Y. Does teacher burnout begin with student
teaching. Education 1985;105(3):254.
20. Balogun JA, Hoeberlein-Miller TM, Schneider E, Katz
JS. Academic performance is not a viable determinant
of physical therapy students’ burnout. Percept Mot
Skills. 1996;83(1):21–22.
21. Worley JA, Vassar M, Wheeler DL, Barnes LL. Factor
structure of scores from the Maslach Burnout
Inventory: A review and meta-analysis of 45 explora-
tory and confirmatory factor-analytic studies.
Educational and Psychological Measurement. 2008;
68(5):797–823.
22. Maslach C, Jackson SE. The measurement of experi-
enced burnout. J Organiz Behav. 1981;2(2):99–113.
23. Wheeler DL, Vassar M, Worley JA, Barnes LL. A reli-
ability generalization meta-analysis of coefficient alpha
for the Maslach Burnout Inventory. Educational and
Psychological Measurement. 2011;71(1):231–244.
24. Zumbo BD, Gadermann AM, Zeisser C. Ordinal ver-
sions of coefficients alpha and theta for Likert rating
scales. J Mod App Stat Meth. 2007;6(1):21.
25. Dembe AE, Lynch MS, Gugiu PC, Jackson RD. The
translational research impact scale: development, con-
struct validity, and reliability testing. Eval Health Prof.
2014;37(1):50–70.
26. Tennant A, Conaghan PG. The Rasch Measurement
Model in Rheumatology: What Is It and Why Use It?
When Should It Be Applied, and What Should One
Look for in a Rasch Paper?. Arthritis Rheum. 2007;
Dec57(8):1358–1362.
27. Bond TG, Fox CM. Applying the Rasch Model:
Fundamental Measurement in the Human Sciences,
(3rd ed., pp. 112–39). New York, NY: Routledge,
Taylor & Francis Group, 2015.
28. Hendriks J, Fyfe S, Styles I, Skinner SR, Merriman G.
Scale construction utilizing the Rasch unidimensional
measurement model: A measurement of adolescent
attitudes towards abortion. Amj. 2012;5(5):251.
29. Engelhard G. Jr, Invariant Measurement: Using Rasch
Models in the Social, Behavioral, and Health Sciences.

New York, NY: Routledge, Taylor & Francis Group,
2013.
30. Gugiu MR, Gugiu PC. Utilizing item analysis to
improve the evaluation of student performance.
Journal of Political Science Education. 2013;9(3):
345–361.
31. Wright BD, Stone MH. Best Test Design: Rasch
Measurement. Chicago, IL: Mesa Press, 1979.
32. Boone WJ, Staver JR, Yale MS. Rasch Analysis in the
Human Sciences (pp. 217-The Netherlands: Springer,
2014.
33. Ewing MT, Salzberger T, Sinkovics RR. An alternate
approach to assessing cross-cultural measurement
equivalence in advertising research. Journal of
Advertising. 2005;34(1):17–36.
34. Dyrbye LN, Massie FS, Eacker A, et al. Relationship
between burnout and professional conduct and atti-
tudes among US medical students. JAMA 2010;
304(11):1173–1180.
35. Brazeau CM, Shanafelt T, Durning SJ, et al. Distress
among matriculating medical students relative to the
general population. Academic Medicine. 2014;89(11):
1520–1525.
36. Hu Q, Schaufeli WB. The factorial validity of the
Maslach Burnout Inventory-Student Survey in China.
Psychol Rep. 2009;105(2):394–408.
37. Mostert K, Pienaar J, Gauche C, Jackson L. Burnout
and engagement in university students: A psychomet-
ric analysis of the MBI-SS and UWES-S. South
African Journal of Higher Education. 2007;21(1):
147–162.
38. Yavuz G, Dogan N. Maslach burnout inventory-stu-
dent survey (MBI-SS): a validity study. Procedia-Social
and Behavioral Sciences. 2014;116:2453–2457.
39. Loera B, Molinengo G, Miniotti M, Leombruni P.
Refining the Frommelt Attitude Toward the Care of
the Dying Scale (FATCOD-B) for medical students: A
confirmatoryfactor analysis and Rasch validation
study. Pall Supp Care. 2018;16(01):50–59.
40. Molinengo G, Baiardini I, Braido F, Loera B.
RhinAsthma patient perspective: A Rasch validation
study. J Asthma. 2018; Feb55(2):119–123.
41. Gugiu PC, Coryn C, Clark R, Kuehn A. Development
and evaluation of the short version of the Patient
Assessment of Chronic Illness Care instrument.
Chronic Illn. 2009;5(4):268–276.
42. Hayton JC, Allen DG, Scarpello V. Factor Retention
Decisions in Exploratory Factor Analysis: a Tutorial
on Parallel Analysis. Organizational Research Methods.
2004;7(2):191–205.
43. Drasgow F. Polychoric and Polyserial Correlations. In
Kotz S, Johnson NL (Eds.), Encyclopedia of Statistical
Sciences (pp. 68–74). New York: John Wiley & Sons,
Inc., 1986.
44. J€
oreskog KG. On the estimation of polychoric correla-
tions and their asymptotic covariance matrix.
Psychometrika 1994;59(3):381–389.
45. Holgado-Tello FP, Chacon-Moscoso S, Barbero GI,
Vila-Abad E. Polychoric versus Person correlations
in exploratory and confirmatory factor analysis
with ordinal variables. Qual Quant. 2010;44(1):
153–166.
TEACHING AND LEARNING IN MEDICINE 15
46. Andrich D. A rating formulation for ordered response
categories. Psychometrika 1978;43(4):561–573.
47. Juttner M, Boone W, Park S, Neuhaus BJ.
Development and use of a test instrument to measure
biology teachers’ content knowledge (CK) and peda-
gogical content knowledge (PCK). Educ Asse Eval
Acc. 2013;25(1):45–67.
48. Nunnally JC, Bernstein IH. Psychometric Theory (3rd
ed., pp. 265). New York, NY: McGraw-Hill, 1994.
49. Gugiu C, Gugiu MR. Determining the Minimum
Reliability Standard Based on a Decision Criterion.
The Journal of Experimental Education. 2018;86(3):
458–472.
50. Baghaei P. The Rasch Model as a Construct
Validation Tool. Rasch Measurement Transactions
2008;22(1):1145–1146.
51. Linacre JM. Optimizing rating scale category effective-
ness. J Appl Meas. 2002;3(1):85–106.
52. Linacre JM. A user’s guide to Winsteps: Program
Manual 3.75.0. 2012.
53. Remmers HH, Karslake R, Gage N. Reliability of mul-
tiple-choice measuring instruments as a function of
the Spearman-Brown prophecy formula, I. Journal of
Educational Psychology. 1940;31(8):583.
54. Baumgartner TA. The applicability of the Spearman-
Brown prophecy formula when applied to physical
performance tests. Research Quarterly. American
Association for Health, Physical Education and
Recreation. 1968;39(4):847–856.
APPENDIX
Maslach burnout inventory–student survey
Exhaustion
1. I feel emotionally drained by my studies.
2. I feel used up at the end of a day at medical school.
3. I feel tired when I get up in the morning and I have to
face another day at medical school.
4. Studying or attending a class is really a strain for me.
5. I feel burned out from my studies.
Cynicism
1. I have become less interested in my studies since my
enrollment in medical school.
2. I have become less enthusiastic about my studies.
3. I have become more cynical about the potential useful-
ness of my studies.
4. I doubt the significance of my studies.
Professional efficacy
1. I can effectively solve the problems that arise in my
studies. (R)
2. I believe that I make an effective contribution to the
classes I attend. (R)
3. In my opinion, I am a good student. (R)
4. I feel stimulated when I achieve my study goals. (R)
5. I have learned many interesting things during the
course of my studies. (R)
16 Y. SHI ET AL.

6. During class, I feel confident that I am effective in get- 1 ¼ Never

ting things done. (R) 2 ¼ A few times a year or less
Note: (R) items were reverse coded prior to analyses so that 3 ¼ Once a month or less
scores are consistent with the other two scales, wherein high 4 ¼ A few times a month
values represent more burnout. 5 ¼ Once a week
6 ¼ A few times a week
Related response set 7 ¼ Every day
Less Burnout More Burnout