Research

JAMA Oncology | Original Investigation

Critical Lessons From High-Value Oncology Practices

Douglas W. Blayney, MD; Melora K. Simon, MPH; Beatrice Podtschaske, PhD; Scott Ramsey, MD, PhD;
Margaret Shyu, BS; Craig Lindquist, MD, PhD; Arnold Milstein, MD, MS

Invited Commentary
IMPORTANCE Cancer care is expensive. Cancer care provided by practice organizations varies Author Audio Interview
in total spending incurred by patients and payers during treatment episodes and in quality of
care, and this unnecessary variation contributes to the high cost. Supplemental content

OBJECTIVE To use the variation in total spending and quality of care to assess oncology
practice attributes distinguishing “high value” that may be tested and adopted by others to
produce similar results.

DESIGN, SETTING, AND PARTICIPANTS “Positive deviance” was used in this exploratory
mixed-methods (quantitative and qualitative) analysis of interview results. To quantify value,
oncology practices located near the US Pacific Northwest and Midwest with low mean
insurer-allowed spending were identified. Among those, practices with high quality were
selected. A team then conducted site visits to interview practice personnel from June 2, 2015,
through October 3, 2015, and to probe for attributes of high-value care. A qualitative analysis
of their interview results was performed, and a panel of experienced oncologists was
convened to review attributes occurring uniquely or frequently in low-spending practices for
their contribution to value improvement and ease of implementation. Four positive deviant
(ie, low-spending) oncology practices and 3 oncology practices that ranked near the middle
of the spending distribution were studied.

MAIN OUTCOMES AND MEASURES Thematic saturation in a qualitative analysis of high-value

care attributes.

RESULTS From the 7 oncology practices studied, 13 attributes within the following 5 themes
emerged: treatment planning and goal setting, services supporting the patient journey,
technical support and physical layout, care team organization and function, and external
context. Five attributes (ie, conservative use of imaging, early discussion of treatment
limitations and consequences, single point of contact, maximal use of registered nurses for
interventions, and a multicomponent health care system) most sharply distinguished the
high-value practice sites. The expert oncologist panel judged 3 attributes (ie, early and
normalized palliative care, ambulatory rapid response, and early discussion of treatment
limitations and consequences) to carry the highest immediate potential for lowering
spending without compromising the quality of care.

CONCLUSIONS AND RELEVANCE Oncology practice attributes warranting further testing were Author Affiliations: Stanford Cancer
identified that may lower total spending for high-quality oncology care. Institute, Stanford, California
(Blayney); Clinical Excellence
Research Center, Stanford University,
Stanford, California (Blayney, Simon,
Podtschaske, Shyu, Lindquist,
Milstein); Now with Stanford
Healthcare, Stanford, California
(Podtschaske); Hutchinson Institute
for Cancer Outcomes Research, Fred
Hutchinson Cancer Research Center,
Seattle, Washington (Ramsey); Now
with Feinberg School of Medicine,
Northwestern University, Chicago,
Illinois (Shyu).
Corresponding Author: Douglas W.
Blayney, MD, Stanford Cancer
Institute, 875 Blake Wilbur Dr, Room
JAMA Oncol. doi:10.1001/jamaoncol.2017.3803 CC2219, Stanford, CA 94305
Published online November 16, 2017. (dblayney@stanford.edu).

(Reprinted) E1
© 2017 American Medical Association. All rights reserved.

Downloaded From: on 11/17/2017

 Research Original Investigation
C
ancer care is complex, variably delivered, and usually
conducted in an ambulatory setting by physician-led
teams. Care delivery teams operate at the intersection
of basic biology, clinical medicine, and patient emotional needs.
The complexity and variation in care delivery creates oppor-
tunities to identify attributes of ambulatory oncology prac-
tices that deliver high-value care, that is, high-quality care at
a relatively low total cost. Because value-based payments—
implementation of the Medicare Access and Children’s Health
Insurance Program Reauthorization Act, the Oncology Care
Model, and Accountable Care Organizations—are already re-
warding high-value institutions, a better appreciation of the
attributes of high-value oncology care is timely.1 Combina-
tions of claims data with cancer registry and clinical data can
broadly identify high- and low-value care, but smaller units of
care delivery, such as an ambulatory oncology practice, are not
well studied.2-4
Measuring and defining value includes quantitative mea-
sures of the total cost of care and measures of quality.5 One
measure of ambulatory cancer care is provided by the Ameri-
can Society of Clinical Oncology’s Quality Oncology Practice
Initiative (QOPI)6 and the associated voluntary QOPI certifi-
cation program.7 To identify qualitative attributes of groups
that deliver high-value cancer care, we used the “positive de-
viance” technique. Positive deviance detects superior exist-
ing solutions that can then be adopted to produce similar
results.8 Positive deviance has identified fresh approaches to
patient activation as well as physician engagement and clini-
cal care after myocardial infarction9-11 and has been used pre-
viously by our group.12
Our exploratory hypothesis-generating study started with
quantitative methods to identify positive deviant practices,
meaning practices that ranked favorably compared with their
peers on low mean total spending per treatment episode and
that also scored highly on ambulatory quality measures. We
then applied qualitative methods to identify potentially trans-
ferable attributes of high-value care.
Methods
Quantitative Methods to Select Potential
High-Value Practices
The Hutchinson Institute for Cancer Outcomes Research and
Anthem Inc provided data that enabled the generation of a
sample of oncology practices in western Washington state and
2 Midwestern states. Practices were defined by their unique
federal tax identification numbers; when the same tax iden-
tification number was associated with several sites of care, the
largest site was selected. Sites, practices, practice organiza-
tions, oncologists, and oncology practices are hereinafter re-
ferred to as practices. Participation in the study was volun-
tary. The institutional review board at Stanford University
determined that the present study was exempt from institu-
tional review board review and waived the need for partici-
pant informed consent.
The Hutchinson Institute for Cancer Outcomes Research
developed a measure of quality that included adherence to 4
E2 JAMA Oncology Published online November 16, 2017 (Reprinted)
© 2017 American Medical Association. All rights reserved.
Downloaded From: on 11/17/2017
Critical Lessons From High-Value Oncology Practices
Key Points
Question What are the attributes of “positive deviant” oncology
practices that deliver high-quality cancer care at low total cost?
Findings In this analysis using mixed quantitative and qualitative
methods and site visits to 7 US oncology practices with structured
interview questionnaires, 13 attributes were identified that likely
contributed to high-value cancer care. Five attributes most clearly
distinguished oncology practice sites that ranked favorably on
value.
Meaning Attributes of high-value positive deviant oncology
practices can be implemented in other care systems and their
contribution to value studied.
of the Choosing Wisely treatment recommendations (as an in-
dicator of quality)13 (eTable 1 in the Supplement), spending (in-
cluding health insurance claims data from Premera Blue Cross),
and tumor staging information from tumor registries.3 Data
were used from 3225 patients whose care was attributed to 16
practices from January 2007 through May 2014.3 Potential
high-value practices were in the lowest third of the spending
distribution and were considered “wise choosers,” whereas
comparator practices were in the middle third on spending. As
a marker of quality, QOPI-certified practices were selected.14
In another region of the United States, Anthem Inc iden-
tified 32 973 episodes of care provided by 206 practices in Ohio
and Indiana from July 2010 through July 2012. Anthem Inc
used the Optum Episode Treatment Group, a case-mix adjust-
ment and episode-building system that uses routinely col-
lected inpatient and ambulatory claims data (eg, claims for
physician services, chemotherapy and associated administra-
tion, and imaging) to compare mean total spending per treat-
ment episode among health care practices.15 Based on the Op-
tum Episode Treatment Group output, potential high-value
practices were identified with observed to expected costs be-
low the mean and potential comparator organizations with cost
and CIs overlapping the mean. Because Choosing Wisely per-
formance data on these practices were not available, the Ameri-
can Society of Clinical Oncology’s QOPI certification program
was used such that potential practices were further win-
nowed to those that were QOPI-certified.14
In total, 11 oncology practices remained after excluding
those practices that did not predominantly provide medical
oncology services, those with substantial missing data (eg,
omitted physician codes), or those with too few patients at-
tributed to the practice physicians. Of these, 4 high-value prac-
tices, which were designated positive deviant practices, and
3 comparator practices were assessed in our qualitative study.
Additional details are in the eMethods in the Supplement.
Site Visit Protocol
Among others, two of us (D.W.B., a senior medical oncologist,
and B.P., a qualitative researcher) conducted 2-day site visits
from June 2, 2015, through October 3, 2015. Physicians, ad-
vanced practice providers (including licensed nurse practi-
tioners and licensed physician assistants), nurses, medical sup-
port, administrative office personnel, and practice leaders and
jamaoncology.com
 Critical Lessons From High-Value Oncology Practices
managers were interviewed. A structured interview tool, de-
signed prior to site visits and containing questions covering
clinical, nonclinical, and quality management topics, was used
uniformly. The interview questions were shared with prac-
tice personnel prior to the visit, and the site visitors recorded
extensive interview notes. Questions probed care delivery
methods, staff roles and functions, patient services pro-
vided, cultural norms, and diagnostic and surveillance test-
ing use. The site visit team also solicited interviewee opin-
ions regarding attributes that could affect quality and total
spending. All interviewers and interviewees were blinded to
practice status (ie, high value or comparator).
Qualitative Analysis of Interviews
A separate team debriefed the site-visit team within 48 hours
of a site visit. The site-visit team also prepared a written re-
port summarizing the interviews.
A 5-stage framework approach, developed for applied
qualitative research,16 was used to identify attributes poten-
tially responsible for high value. In the first familiarization
stage, interviewee “mentions” of features contributing to high
value and high-quality care were identified. (Mentions from
all 7 site visits were treated equally regardless of the prac-
tice’s classification.) Based on these mentions, in the second
stage, a thematic framework was identified (eg, “treatment
planning and goal setting”) as given in the first column of
Table 1. For stage 3, site-visit mentions were grouped into
“attributes,” and each attribute was categorized into the
appropriate theme to identify distinctive attributes among
the practices. Each attribute and its implementation details
were charted in stage 4. For the final stage, distinct attri-
butes that might explain the high value were summarized,
synthesized, and identified. Attributes were subdivided into
patient- and practice-centered themes. Attributes found in
low-spending practices and in comparator practices were
then tallied.
Recommendations for theory-based content analysis17
were followed to achieve thematic saturation, that is, identi-
fying all themes from interview data. The practices to study
and the roles for interviewers and interviewees were estab-
lished a priori, and the same structured interview questions
and interview guide were used for all practices. A minimum
of 10 interviewees were included from each practice. The men-
tions and attributes were organized and presented using cu-
mulative frequency graphs to enhance the transparency and
verifiability of the decision that saturation had been achieved
and to address complex or multifaceted descriptions. The
analysis continued until no new themes emerged. The inter-
view results were coded by 3 independent coders (M.K.S., B.P.,
and C.L.) from the debriefing team.
Review of Findings by an Oncology Expert Physician Panel
The results uncovered in the qualitative aspect of this study
were further refined by a recruited expert panel of experi-
enced oncologists. Using a modified Delphi process, the panel
scored each attribute for its potential to lower mean spending
per episode without compromising quality of care. A compos-
ite of their scores was then computed.
jamaoncology.com
© 2017 American Medical Association. All rights reserved.
Downloaded From: on 11/17/2017
Original Investigation Research
Results
Attributes of High-Value Practices
The characteristics of the 7 practices that agreed to partici-
pate and were visited by one of our site-visit teams are given
in eTable 2 in the Supplement. The team identified 13 distinct
practice attributes that may have affected care cost and qual-
ity. These practice attributes were grouped into the following
5 themes and are given in Table 1: (1) treatment planning and
goal setting; (2) support for the patient journey; (3) care team
organized so that members function at the highest level al-
lowed by their competence and license; (4) technical support
and physical layout; and (5) external context. Five “unique”
attributes were found only in high-value practices, that is, none
of these attributes was found in the comparator practices. Five
other “distinguishing” attributes were much more frequent in
high-value practices but were also found in the comparators.
Three “nondistinguishing” attributes occurred at similar fre-
quencies in both high-value and comparator practices.
Unique Attributes of High-Value Practices
Treatment Planning: A Conservative Approach to Diagnostic Testing
High-value practices used remarkably similar language to de-
scribe their approach, such as “[we order tests] to minimize
[inadvertent] repetition and [based] on [the] medical neces-
sity of ‘only if the test result will make a difference in patient
care.’” Oncologists and diagnostic radiologists explicitly dis-
cussed the most efficient testing route to patient goals (eg, using
the same imaging tests for both cancer staging and radio-
therapy planning) and chose those diagnostic tests with the
highest utility. This process was described by clinicians as a
“conservative” or “less is more” approach. Adoption and ad-
herence to this conservative testing approach was reinforced
by routine group discussions and by coordinating test use dur-
ing tumor boards and case conferences.
Treatment Planning: Setting Goals After Explicit Discussion
on the Benefits, Limits, and Consequences
Physicians and other team members emphasized conducting
discussions early after initial diagnosis, during the first or sec-
ond office visit, or after significant clinical events (eg, cancer
recurrence) to incorporate input from the conversations into
the treatment plan and to “set realistic goals.” Sufficient time
was allotted to ensure the patient understood the treatment,
the available patient support services, and the availability of
concurrent palliative care (curative treatment of treatable con-
ditions, including infections; symptomatic treatment of dys-
pnea and pain, as well as end-of-life care). The physician and
other team members revisited these discussions at follow-up
visits to ensure that treatments were continuously aligned with
patient goals.
Support for the Patient Journey
Proactive support for patients during predictably stressful pe-
riods (eg, cancer relapse or unexpected scan results) was also
unique to the high-value practices. A single staff member was
often the “point person” or “go-to person” to assist patients
(Reprinted) JAMA Oncology Published online November 16, 2017 E3
Downloaded From: on 11/17/2017
E4
JAMA Oncology Published online November 16, 2017 (Reprinted)
Table 1. Unique, Distinguishing, and Foundational Attributes of 4 High-Value and 3 Comparator Organizations
Theme
Unique: Found in 3-4 (75%-100%) High-Value Organizations and 0 Comparator Organizations
Treatment planning and goal setting
© 2017 American Medical Association. All rights reserved.
Support for patient journey
Care team functions at the highest
level of competence and license
External context
Distinguishing: Found in Most High-Value and 1 Comparator Organization
Support for patient journey
Care team functions at the highest
level of competence and license
Technical support and physical
layout
jamaoncology.com
External context
Attribute
Conservative use of diagnostic and surveillance imaging
Discussion on treatment limits and consequencesa
Single point of contact (usually a nurse) helps patients
navigate the oncology care system
Maximal use of RNs to make care decisions and appropriate
interventions
Multicomponent health system–affiliated
Proactive and ongoing assessment for signs and symptoms
that trigger further assessment or triage to in-office or higher
level of care
Palliative care incorporated early in the care arc and
normalizeda
Patient issues solved as a team working together
Hospitalists used to maximize medical oncologists’
office-based time
Physical location configured for informal consultation and
collaborative work
Small size
Research Original Investigation
High Value, Comparator,
Example No. (%) No. (%)
Physicians use guidelines as a reference to decide whether to order diagnostic imaging 4 (100) 0
and laboratory tests
Imaging and laboratory tests only ordered if result would influence treatment plan;
decision is based on physician’s clinical judgment or through a mechanism for peer
review (ie, multidisciplinary tumor board)
No imperative to “chase down every lead” and uncover reasons for every finding, in
contradistinction to classic medical training
Care team sets expectations during initial conversations (at first or second office visit) 4 (100) 0
with discussions on treatment prognosis and implications (ie, quality of life, adverse
effects, and symptoms)
Each organization has ≥1 dedicated “go to” person to help patients understand their 3 (75) 0
journey, care team, and available resources
RNs assess patients for adverse effects and initiate symptom management 3 (75) 0
RNs review chemotherapy effects and recommend midcourse corrections; most nurses
are OCNs
Physicians or a physician-owned group employed by a multicomponent health system 3 (75) 0
Organization uses a process to routinely and proactively screen all patients for “red 2 (50) 0
flags” (eg, unexpected disease or treatment complications)
Organization conducts further or more frequent evaluation of high-risk patients
Organization visibly tracks “saves,” including diverting patients from emergency
departments to a more appropriate, often specialized, level of care, such as an urgent
care facility; providing hydration in an infusion center; or securing an urgent office visit
Palliative care is explained to patients and family as an integral part of treatment, is 3 (75) 1 (33)
incorporated early, and is not limited to end-of-life care
A dedicated, specialized resource supports the provision of palliative care rather than
relying solely on the medical oncologist to provide palliative treatment
Regularly scheduled multidisciplinary tumor boards 4 (100) 1 (33)
Informal, unplanned “curbside consults” among care team members encouraged and
Critical Lessons From High-Value Oncology Practices
occur frequently
Organization provides or collaborates with a dedicated oncology-focused hospitalist 3 (75) 1 (33)
Hospitalist’s exclusive responsibility is management of hospital inpatient care
If a nononcologist, the hospitalist closely coordinates hospital care with the patient’s
primary, office-based oncologist
Care team works in a compact, multipurpose physical space that enhances intrateam 4 (100) 1 (33)
verbal and visual communication
Care team is adjacent to other departments (eg, imaging, laboratory, radiation oncology,
and infusion center)
Effective use of EHRs as a communication tool among all members of the care team to
facilitate high-value care; routine incorporation of EHRs into regular workflow (rather
than using email outside of the EHR, text messaging, etc) and using the equivalent of
instant messaging within EHRs to create task lists and reminders
<10 Medical oncologists in organization 4 (100) 1 (33)
(continued)
 Critical Lessons From High-Value Oncology Practices Original Investigation Research
in understanding their disease and learning to navigate the
health care system. Often trained as a nurse, this point per-
Comparator,

son frequently met with the patient (usually at each office visit),
3 (100)

3 (100)

3 (100)

Attributes identified by the expert panel as having the highest potential contribution to cost and quality.
No. (%)

2 (67)

offering tips on self-management and help with a range of

common stressful issues, including financial and transporta-
tion assistance and access to services that provide social and
emotional support.
High Value,

4 (100)

APPs practice collaboratively with oncology physicians but have a high level of autonomy 4 (100)
No. (%)

3 (75)

3 (75)

Care Team Functions at the Highest Level of Competence

and License
Organization provides rapid real-time response by phone and in office to triage patients

The use of experienced oncology nurses and other nononcolo-

As their training and license permit, APPs provide urgent care and “drop-in” visits and
and provides urgent care (eg, hydration) in the office or in an adjacent infusion center

When needs are identified, patient referred to supportive resources and follow-up is

gist care providers was another often-mentioned attribute.

scheduled nonurgent follow-up visits for surveillance and symptom management

EHR tools provide near real-time information and support care coordination and

Nurses worked via protocol to provide clinical assessments and

management suggestions (typically in response to inbound
APPs have their own panels of patients in palliative care and survivorship
Care team has a structured approach to assess patient nonmedical needs

telephone calls); to triage nonscheduled, urgent, or emergent

EHR includes embedded cancer care management tools (eg, protocols)

patient evaluation; and to offer management by nonphysi-

cian clinical staff (usually advanced practice providers, in-
cluding nurse practitioners, advanced practice nurses, or phy-
sician assistants) or to nononcology physicians.
Uniform initial and repeated distress assessment

External Context
Close affiliation with a large and generally hospital-based health
system or with a health plan that employed physicians was also
Tools are used to generate task lists

a feature of high-value practices. However, the oncology prac-

interprofessional communication

tice unit was small and cohesive and retained its distinct iden-
Table 1. Unique, Distinguishing, and Foundational Attributes of 4 High-Value and 3 Comparator Organizations (continued)

tity within a larger system. The affiliated larger systems pro-

vided a broad range of staff to support functions such as quality
measurement, human resources, pharmacy, navigation, and
social work.
a
Example

initiated

Abbreviations: APP, advanced practice provider, including licensed nurse practitioners and licensed physician

Discussion
support proactively identified, offered supportive services,

Themes Unique to High-Value Practices

Patients with a high emotional burden or lacking in social
Ambulatory rapid response for patients with an unstable

In this qualitative analysis of high-value practices, we found

assistants; EHR, electronic health record; OCN, oncology-certified nurse; RN, registered nurse.
APP patient care roles are maximized (“upshifted”)

5 unique attributes, which we categorized within 4 themes. Our

EHR used effectively for communication and care

expert oncology panel ranked 3 of these as highly likely to con-

tribute to a practice’s high quality and low cost of care (Table 2).
and frequently and proactively reassessed

Theme 1: Treatment Planning and Goal Setting

Foundational: Found in Both High-Value and Comparator Organizations

Physician restraint in the use of laboratory testing and imaging

was often expressed as “we don’t order tests if the result won’t
change the treatment plan” or shown by the preplanned use
of 1 imaging procedure for both diagnostic and radiotherapy
planning. In addition, global treatment planning and realistic

coordination
conditiona

goal setting were found to engage patients and families along

Attribute

the cancer journey. Engaged patients and families were con-

sidered under the care of the team. Care teams “added value
by talking more.”18

Care team functions at the highest

level of competence and license

Technical support and physical

Theme 2: Navigation and Palliative Care in Support
Support for patient journey
of the Patient Journey
Oncology care includes multiple procedures, tests, and treat-
ments supervised by different physicians, requires transfers
and exchanges among multiple locations, and necessitates
communication with insurers and pharmacies. Care coordi-

Theme

layout
nation facilitates more efficient use of time and less duplica-
tion. Care coordinator or patient navigator are terms often
jamaoncology.com (Reprinted) JAMA Oncology Published online November 16, 2017 E5
© 2017 American Medical Association. All rights reserved.
Downloaded From: on 11/17/2017
 Research Original Investigation Critical Lessons From High-Value Oncology Practices

Table 2. Expert Panel–Assessed Contribution of Unique Attributes to Cost and Qualitya

Impact Score
Attribute Quality Cost Combinedb
Palliative care incorporated early in the care arc and normalizedc 4.0 4.5 8.5
Ambulatory rapid response provided for patients with an unstable 4.2 4.2 8.4
conditionc
Limits and consequences of treatment discussedc 4.0 4.3 8.3
Signs and symptoms proactively and continually assessed for the need 3.7 4.0 7.7
of further assessment or triage leading to in-office or higher level of
care Abbreviations: APP, advanced
Diagnostic and surveillance imaging used conservatively 3.0 4.2 7.2 practice provider, including licensed
nurse practitioners and licensed
Patient issues solved as a team working together 2.8 4.0 6.8 physician assistants; RNs, registered
Nurses (RNs) maximally used to make care decisions and appropriate 3.0 3.5 6.5 nurses.
interventions a
Attributes of the 7 practices ranked
Hospitalist used to maximize medical oncologists’ office-based time 3.0 3.2 6.2 on a 5-point scale (5 indicates
APP patient care roles maximized (“upshifted”) 2.7 2.8 5.5 greatest impact) based on the
contribution of the attribute to cost
Electronic health record used effectively for communication and care 2.2 3.2 5.4
coordination and quality as judged by an expert
panel.
Physical location configured for informal consultation and collaborative 2.0 3.0 5.0
b
work Sum of the panel’s mean score of
Single point of contact, usually a nurse, provided to help patients 1.8 2.5 4.3 each attribute’s contribution to cost
navigate oncology care system and to quality.
Patients with a high emotional burden or lacking in social support 1.7 2.3 4.0 c
Attributes having the highest
proactively identified, offered supportive services, and frequently and potential contribution to cost and
proactively reassessed
quality.

applied to this function. Although the value of care coordina-
tion has been difficult to demonstrate in broadly focused am-
bulatory care organizations, its value may be greater in oncol-
ogy because of the multiple clinicians engaged in the care of
each patient,19 and its value has been recently demonstrated
in older patients with cancer.20
Early introduction of palliative care services and normal-
izing palliative care—“this is the way we always do it”—is an-
other high-value attribute. Normalizing palliative care miti-
gates the sometimes negative connotations of the end-of-life
or hospice care associated with palliative care. An example of
this successful attribute was the following: “[(O)ur palliative
care team has] taken care of a family member of almost all of
our medical staff, and they have experienced firsthand the ben-
efits we provide.” Our methodology could not distinguish
among the many aspects of palliative care to determine which
had the greatest impact.
Theme 3: Care Team Functions at the Highest Level
of Competence and License
Experienced, well-trained nurses performed clinical assess-
ments and made protocol-based patient self-management rec-
ommendations. If a patient needed urgent care, these nurses
would often direct patients to an on-site ambulatory care fa-
cility. This facility was generally in the chemotherapy infu-
sion area or was to a contracted urgent care facility. The goal
was to avoid unnecessary, expensive, and inconvenient emer-
gency department visits.
In addition, advanced practice providers often provided
chemotherapy symptom management, survivorship care, and
palliative care. This system enabled physicians to use their skills
to focus on complex clinical problems and their time to de-
velop deeper patient relationships and to facilitate shared de-
cision making by patients and families.
Theme 4: External Context
Although our exploratory study focused on attributes of care
delivery, the smaller units of care delivery that surfaced as posi-
tive deviants on value often benefitted from administrative in-
frastructure (eg, a common electronic health record plat-
form, human resources, and compliance personnel) supplied
by an associated health system. The combination of “small
care” and “big administrative support” may enable care teams
to be nimbler in decision making, more open to adoption of
best practices, or better at relational coordination.21
Role of Qualitative Research in Determining Value
Efforts to measure and improve quality and increase the value
in health care, such as through the QOPI, have focused on im-
proving adherence to processes of care. Implicit in this qual-
ity improvement effort is the assumption that clinical trial out-
comes (eg, improved overall survival, disease-free survival
improvements, and reduced toxicity) can be translated into
processes, and adherence to processes will improve care and
lead to better outcomes.22-24 While process measures and
guideline adherence have utility and can be quantified, they
provide little guidance on potentially important and nonob-
vious attributes of care delivery. Qualitative research meth-
ods, such as interviews of positive deviant oncology prac-
tices, can expand our understanding of how value is created
by care teams.
Limitations
The results from our small, hypothesis-generating study are
insufficient to support widespread adoption of the attributes
that we reported. In addition, our results are strictly appli-
cable only to QOPI participants, who represent approxi-
mately 15% of practicing US medical oncologists. High-
quality positive deviant oncology practices who do not

E6 JAMA Oncology Published online November 16, 2017 (Reprinted) jamaoncology.com

© 2017 American Medical Association. All rights reserved.

Downloaded From: on 11/17/2017

 Critical Lessons From High-Value Oncology Practices Original Investigation Research

participate in a QOPI were missed in this study. Although we
achieved thematic saturation, there may be other high-value
attributes that we missed. We did not explore features of pal-
liative care that were perceived as providing the greatest im-
pact. Our methods also did not permit exploration of other po-
tential drivers of cost, including practice consolidation and cost
increases surrounding the implementation of the Patient Pro-
tection and Affordable Care Act.25,26 We also did not explore
the role of increasing costs of anticancer drugs or drug choice,
although both drug costs and negotiated prices were cap-
tured in the claims data and are likely critical components of
episode spending.27 We also did not study low-value prac-
tices nor did we independently rank costs by detailed exami-
nation of claims as reported to Premera Blue Cross or to An-
them Inc. We did not have data from a national database to
compare practices on standard measures of health spending,
necessitating different practice selection methods in the re-
gions we studied. Results from the Oncology Care Model28 may
provide an opportunity to confirm our results on a more uni-
formly selected group of practices. Other studies operating with
different data sources may not reproduce our results, but our
interview and analysis methods should be relevant. Expert
panel physicians ranked attributes associated with physician
involvement highly. A more diversely composed panel may
have led to a different ranking. Finally, our correlation-based
study does not establish causality or the relative importance
of or the relationships between the attributes that correlated
with low spending per episode of care.
Our results can be viewed as those from a “training” data
set; organizations that choose to implement our findings should
carefully study—as a “validation” set—the contribution of our
results in achieving better health, quality care, and low cost.
Validation should preserve the validity of our discriminatory
measures (eg, standard interview questions, interviewers
blinded to practice cost status, multiple practice personnel in-
terviewed across multiple job functions, and thematic satu-
ration) and verify and examine with more granularity the
contribution to costs associated with the practice attribute. Vali-
dation might also include identifying other care practices that
discriminate excellent from good to confirm the validity of the
important attributes provided in Table 1. Our results will in-
form design of larger, confirmatory qualitative studies as well
as of larger value-based data sets.
Conclusions
Organizations facing increased pressure to lower health care
spending and improve quality of oncology care can view this
study as an additional source of insight, until readily transfer-
able attributes of care are tested and available to inform more
refined system designs. Changes to the decision-making cul-
ture of laboratory testing and diagnostic imaging will require
physician leadership and participation and administrative and
implementation science support. Upgrading staff roles and im-
proving care efficiency will require a system redesign. Patient-
centered redesign of care is valuable and should be central to
any change management effort. Our preliminary study has
identified attributes of some of the most valuable care in the
United States.

ARTICLE INFORMATION
Accepted for Publication: September 1, 2017.
Published Online: November 16, 2017.
doi:10.1001/jamaoncol.2017.3803
Author Contributions: Dr Blayney and Ms Simon
had full access to all of the data in the study and
take responsibility for the integrity of the data and
the accuracy of the data analysis.
Study concept and design: Simon, Ramsey, Milstein.
Acquisition, analysis, or interpretation of data: All
authors.
Drafting of the manuscript: Blayney, Simon,
Podtschaske, Shyu, Lindquist.
Critical revision of the manuscript for important
intellectual content: Simon, Podtschaske, Ramsey,
Shyu, Milstein.
Statistical analysis: Shyu, Lindquist.
Obtained funding: Simon, Milstein.
Administrative, technical, or material support:
Simon, Podtschaske, Ramsey, Shyu.
Study supervision: Blayney, Milstein.
Conflict of Interest Disclosures: Dr Blayney
reported serving as a paid consultant for the
Michigan Oncology Quality Consortium, a volunteer
leader of the Quality Oncology Practice Initiative
certification program for the American Society of
Clinical Oncology, and as a paid consultant for and
stockholder in Physician Resource Management,
Inc, and CARET. Ms Simon has an immediate family
member who is employed by and is a shareholder
of Guardant Health, Inc. No other disclosures were
reported.
Funding/Support: This study was supported by a
grant from the Peterson Center on Healthcare to
the Clinical Excellence Research Center, Stanford
University (Dr Milstein, principal investigator).
Role of the Funder/Sponsor: The funder had no
role in the design and conduct of the study;
collection, management, analysis, and
interpretation of the data; preparation, review, or
approval of the manuscript; and decision to submit
the manuscript for publication.
Meeting Presentations: Portions of this work were
presented at the Cancer Quality Symposium of the
American Society of Clinical Oncology; February 27,
2016; Phoenix, Arizona; and at the annual meeting
for the American Society of Clinical Oncology; June
1, 2015; Chicago, Illinois.
Additional Contributions: Jennifer Malin, MD, PhD
(Anthem Inc), provided data and served on the
expert panel along with Lee Newcomer, MD
(UnitedHealthcare), and Gary Lyman, MD, PhD
(Hutchinson Institute for Cancer Outcomes
Research, Fred Hutchinson Cancer Research
Center). Edward Braud, MD, an independent
consultant, served on the expert panel and was
compensated for participation in 3 site visits.
Catherine Fedorenko, MMSc, and Karma L.
Kreizenbeck, BA (Hutchinson Institute for Cancer
Outcomes Research, Fred Hutchinson Cancer
Center), provided technical and logistical support.
No one else was financially compensated. We thank
the physicians, administrators, nurse practitioners,
physician assistants, and staff at the 7 study
practices.
REFERENCES
1. Institute for Healthcare Improvement. IHI Triple
Aim initiative. http://www.ihi.org/engage/initiatives
/TripleAim/Pages/default.aspx. Published January
6, 2017. Accessed August 18, 2017.
2. Cress RD, Chen YS, Morris CR, Chew H, Kizer KW.
Underutilization of gene expression profiling for
early-stage breast cancer in California. Cancer
Causes Control. 2016;27(6):721-727.
3. Ramsey SD, Fedorenko C, Chauhan R, et al.
Baseline estimates of adherence to American
Society of Clinical Oncology/American Board of
Internal Medicine Choosing Wisely initiative among
patients with cancer enrolled with a large regional
commercial health insurer. J Oncol Pract. 2015;11(4):
338-343.
4. Potosky AL, Riley GF, Lubitz JD, Mentnech RM,
Kessler LG. Potential for cancer related health
services research using a linked Medicare-tumor
registry database. Med Care. 1993;31(8):732-748.
5. Porter ME. What is value in health care? N Engl J
Med. 2010;363(26):2477-2481.
6. Blayney DW, McNiff K, Eisenberg PD, et al.
Development and future of the American Society of
Clinical Oncology’s Quality Oncology Practice
Initiative (QOPI). J Clin Oncol. 2014;32(35):3907-3913.

jamaoncology.com (Reprinted) JAMA Oncology Published online November 16, 2017 E7

© 2017 American Medical Association. All rights reserved.

Downloaded From: on 11/17/2017

 Research Original Investigation
7. McNiff KK, Bonelli KR, Jacobson JO. Quality
oncology practice initiative certification program:
overview, measure scoring methodology, and site
assessment standards. J Oncol Pract. 2009;5(6):
270-276.
8. Bradley EH, Curry LA, Ramanadhan S, Rowe L,
Nembhard IM, Krumholz HM. Research in action:
using positive deviance to improve quality of health
care. Implement Sci. 2009;4:25.
9. Greene J, Hibbard JH, Alvarez C, Overton V.
Supporting patient behavior change: approaches
used by primary care clinicians whose patients have
an increase in activation levels. Ann Fam Med.
2016;14(2):148-154.
10. Sinsky CA, Willard-Grace R, Schutzbank AM,
Sinsky TA, Margolius D, Bodenheimer T. In search of
joy in practice: a report of 23 high-functioning
primary care practices. Ann Fam Med. 2013;11(3):
272-278.
11. Curry LA, Spatz E, Cherlin E, et al. What
distinguishes top-performing hospitals in acute
myocardial infarction mortality rates? a qualitative
study. Ann Intern Med. 2011;154(6):384-390.
12. Milstein A, Gilbertson E. American medical
home runs. Health Aff (Millwood). 2009;28(5):1317-
1326.
13. Schnipper LE, Smith TJ, Raghavan D, et al.
American Society of Clinical Oncology identifies five
key opportunities to improve care and reduce
costs: the top five list for oncology. J Clin Oncol.
2012;30(14):1715-1724.
14. American Society of Clinical Oncology Institute
for Quality. QOPI certified practices. http://www
E8 JAMA Oncology Published online November 16, 2017 (Reprinted)
© 2017 American Medical Association. All rights reserved.
Downloaded From: on 11/17/2017
.instituteforquality.org/qcp/certified
-practices. Published December 1, 2014. Accessed
December 9, 2014.
15. Forthman MT, Dove HG, Wooster LD. Episode
Treatment Groups (ETGs): a patient classification
system for measuring outcomes performance by
episode of illness. Top Health Inf Manage. 2000;21
(2):51-61.
16. Pope C, Ziebland S, Mays N. Qualitative
research in health care: analysing qualitative data.
BMJ. 2000;320(7227):114-116.
17. Francis JJ, Johnston M, Robertson C, et al. What
is an adequate sample size? operationalising data
saturation for theory-based interview studies.
Psychol Health. 2010;25(10):1229-1245.
18. Kaplan RS, Haas DA, Warsh J. Adding value by
talking more. N Engl J Med. 2016;375(20):1918-1920.
19. McWilliams JM. Cost containment and the tale
of care coordination. N Engl J Med. 2016;375(23):
2218-2220.
20. Rocque GB, Pisu M, Jackson BE, et al; Patient
Care Connect Group. Resource use and Medicare
costs during lay navigation for geriatric patients
with cancer. JAMA Oncol. 2017;3(6):817-825.
21. Lee CT, Doran DM, Tourangeau AE, Fleshner NE.
Perceived quality of interprofessional interactions
between physicians and nurses in oncology
outpatient clinics. Eur J Oncol Nurs. 2014;18(6):619-
625.
22. Malin JL, Keating NL. The cost-quality trade-off:
need for data quality standards for studies that
Critical Lessons From High-Value Oncology Practices
impact clinical practice and health policy. J Clin Oncol.
2005;23(21):4581-4584.
23. Neuss MN, Desch CE, McNiff KK, et al.
A process for measuring the quality of cancer care:
the Quality Oncology Practice Initiative. J Clin Oncol.
2005;23(25):6233-6239.
24. Blayney DW, McNiff K, Hanauer D, Miela G,
Markstrom D, Neuss M. Implementation of the
Quality Oncology Practice Initiative at a university
comprehensive cancer center. J Clin Oncol. 2009;
27(23):3802-3807.
25. Berenson RA, Ginsburg PB, Christianson JB,
Yee T. The growing power of some providers to win
steep payment increases from insurers suggests
policy remedies may be needed. Health Aff
(Millwood). 2012;31(5):973-981.
26. Martin AB, Hartman M, Benson J, Catlin A;
National Health Expenditure Accounts Team.
National health spending in 2014: faster growth
driven by coverage expansion and prescription drug
spending. Health Aff (Millwood). 2016;35(1):150-160.
27. Saluja R, McDonald E, Arciero VS, Cheng S,
Cheung MC, Chan KK. Examining the relationship
between cost of novel oncology drugs and their
clinical benefit over time. J Clin Oncol. 2017;35(15)
(suppl):6598. doi:10.1200/JCO.2017.35.15_suppl.6598
28. Centers for Medicare and Medicaid Services.
Oncology Care Model. https://innovation.cms.gov
/initiatives/oncology-care/. Published April 10, 2017.
Accessed August 8, 2017.
jamaoncology.com