[ evidence in practice ]

STEVEN J. KAMPER, PhD1

Interpreting Outcomes 1—
Change and Difference:
Linking Evidence to Practice
Downloaded from www.jospt.org at University of Sydney on May 1, 2019. For personal use only. No other uses without permission.

J Orthop Sports Phys Ther 2019;49(5):357-358. doi:10.2519/jospt.2019.0703

S
cientific writing is by nature technical, so getting terminology gets the study treatment, but they are
and wording right is important. When there is inconsistency in different from the treatment effect and
definitions, or misuse of words, problems follow. Inconsistency are not the treatment response.
Copyright © 2019 Journal of Orthopaedic & Sports Physical Therapy®. All rights reserved.

can come about when there is no universally accepted

definition for a concept or simply due to sloppiness on the author’s part.
But this is more than just a problem of
semantics; when the same words are used
to describe different concepts, study re-
sults can be easily and unknowingly
misinterpreted.
Inconsistency is common when de-
scribing outcomes. Outcomes, whether
Journal of Orthopaedic & Sports Physical Therapy®
problem comes when within-group mean
change is called the “treatment effect” or
the “response to treatment.”
As mentioned in a previous Evidence
in Practice article,1 change from base-
line to follow-up includes changes due
to the natural history of the condition,
Difference
Difference requires data from 2 groups of
people. The between-group difference is
the mean score on an outcome measure in
treatment group A minus the mean score
in group B. Typically, it is either the dif-
ference between scores at follow-up or the
difference in change between the 2 groups.
The difference can reasonably be called

assessed at baseline, during or post treat-
ment, or at follow-up, can be measured
in various ways.2 But there is a critical
division that is always relevant, and in-
terpreting the findings of a study is not
possible unless the reader understands
which side of a metaphorical fence the
numbers are on. The 2 sides of this fence
can be called change and difference.
The distinction between change and
difference is critical with respect to inter-
preting the results of studies of treatment
effectiveness. The 2 types of findings give
the reader different kinds of information.
Change
Put simply, change is the score on an out-
come measure (eg, at follow-up) minus
the score on the same measure at an ear-
lier time point (eg, at baseline). This is
change within a person, or mean change
within a group of people over time. The
regression to the mean, nonspecific ef-
fects, and the effect of treatment. This
isn’t the place to go into the intricacies
of all these, but suffice it to say that all
are relevant, regardless of the condition
and the treatment; that is, this applies
to every study! Within-group change
over time is not the same as the treat-
ment effect. Often, a person with a large
change in outcome is called a respond-
er to treatment A, but this language is
misleading, too, because it is very likely
that the same person would also be a
responder to treatment B. This is basic
stuff, but it is extremely tempting to at-
tribute all the observed change to the
treatment, and terms such as treatment
effect, treatment response, and responder
feed the temptation. This is not to say
that change scores do not provide use-
ful information. They are an estimate of
what is likely to happen when a patient
the treatment effect or treatment response,
because (assuming the study is well de-
signed) the size of the difference does not
include natural history, regression to the
mean, and nonspecific effects. Critically,
the “treatment effect” is a comparative ef-
fect. It is what can be expected if a patient
got treatment A compared to what can be
expected if that patient got treatment B.
So, the “effectiveness” of (or response to)
treatment A, as reported in a particular
study, is interpreted in the light of what
treatment B involved. The difference in
outcome scores between groups quanti-
fies the treatment effect (FIGURE).
Things become tricky here for readers
of research because authors sometimes
report conclusions based on within-
group changes in randomized controlled
trials (RCTs). This usually happens when
there is no difference between the groups.
For example, they might conclude that

School of Public Health, University of Sydney, Camperdown, Australia; Centre for Pain, Health and Lifestyle, Australia. t Copyright ©2019 Journal of Orthopaedic & Sports Physical
1

Therapy®

journal of orthopaedic & sports physical therapy | volume 49 | number 5 | may 2019 | 357
 [ evidence in practice
treatment A is effective, or that both
treatment A and treatment B are effec-
tive, based on within-group improve-
ments. Stating that both treatments are
effective is almost never a valid conclu-
sion from an RCT, except in some very
special circumstances. Within-group
change in an RCT is no more the “treat-
ment effect” than are the results from a
Downloaded from www.jospt.org at University of Sydney on May 1, 2019. For personal use only. No other uses without permission.
mean, and nonspecific effects. Interpret-
ing the findings of an RCT in this way de-
feats the purpose of randomization and
having a comparison group.
In Sum
When reading the results of a study,
you need to be able to answer the ques-
tion of whether the authors are talk-
]
includes natural recovery, regression to
the mean, nonspecific effects, and treat-
ment effects. The latter is the treatment
effect.
Most of the time, the information
necessary to answer the question is in
the study methods and requires a work-
ing knowledge of how the study was per-
formed, and what different methods and

single-group (uncontrolled) study: it still ing about a within-group change or a
includes natural history, regression to the between-group difference. The former
Treatment A mean change: 85 – 44 = 41
100 Treatment B mean change: 85 – 19 = 66
Between-group difference
(treatment effect): 44 – 19 = 25
Outcome
analyses can tell us. The important thing
for someone reading an article is not so
much identifying whether an author is
using “correct” language but determin-
ing the true meaning behind the words
used. t

REFERENCES
Copyright © 2019 Journal of Orthopaedic & Sports Physical Therapy®. All rights reserved.

0 2. 
Baseline
FIGURE. Change and difference.
Journal of Orthopaedic & Sports Physical Therapy®
1. K
amper SJ. Engaging with research: linking
evidence with practice. J Orthop Sports Phys
Ther. 2018;48:512-513. https://doi.org/10.2519/
jospt.2018.0701
Kamper SJ. Fundamentals of measurement:
Follow-up linking evidence to practice. J Orthop Sports Phys
Ther. 2019;49:114-115. https://doi.org/10.2519/
jospt.2019.0701

EARN CEUs With JOSPT’s Read for Credit Program

JOSPT’s Read for Credit (RFC) program invites readers to study and analyze
selected JOSPT articles and successfully complete online exams about
them for continuing education credit. To participate in the program:

1. Go to www.jospt.org and click on Read for Credit in the top blue

navigation bar that runs throughout the site.
2. Log in to read and study an article and to pay for the exam
by credit card.
3. When ready, click Take Exam to answer the exam questions for
that article.
4. Evaluate the RFC experience and receive a personalized certificate
of continuing education credits.

The RFC program offers you 2 opportunities to pass the exam. You may
review all of your answers—including your answers to the questions you
missed. You receive 0.2 CEUs, or 2 contact hours, for each exam passed.

JOSPT’s website maintains a history of the exams you have taken and the
credits and certificates you have been awarded in My CEUs and Your Exam
Activity, located in the right rail of the Read for Credit page listing
available exams.

358 | may 2019 | volume 49 | number 5 | journal of orthopaedic & sports physical therapy