You are on page 1of 13

Use of Standardized Outcome Measures in Physical

Therapist Practice: Perceptions and Applications

Diane U Jette, James Halbert, Courtney Iverson, Erin
Miceli and Palak Shah
PHYS THER. 2009; 89:125-135.
Originally published online December 12, 2008
doi: 10.2522/ptj.20080234

The online version of this article, along with updated information and services, can be
found online at:

Online-Only Material

Collections This article, along with others on similar topics, appears
in the following collection(s):
Evidence-Based Practice
Professional Issues
Tests and Measurements
e-Letters To submit an e-Letter on this article, click here or click on
"Submit a response" in the right-hand menu under
"Responses" in the online version of this article.
E-mail alerts Sign up here to receive free e-mail alerts

Downloaded from by guest on March 17, 2015

Research Report

Use of Standardized Outcome

Measures in Physical Therapist
Practice: Perceptions and Applications
Diane U Jette, James Halbert, Courtney Iverson, Erin Miceli, Palak Shah
DU Jette, PT, DSc, is Professor and
Chair, Department of Rehabilita-
Background. Standardized instruments for measuring patients’ activity limita- tion and Movement Science, Uni-
tions and participation restrictions have been advocated for use by rehabilitation versity of Vermont, Rowell 305,
professionals for many years. The available literature provides few recent reports of 106 Carrigan Dr, Burlington, VT
the use of these measures by physical therapists in the United States. 05405 (USA). Address all corre-
spondence to Dr Jette at: diane.
Objective. The primary purpose of this study was to determine: (1) the extent of
the use of standardized outcome measures and (2) perceptions regarding their J Halbert, PT, BS; C Iverson, PT, BS;
benefits and barriers to their use. A secondary purpose was to examine factors E Miceli, PT, BS; and P Shah, PT,
MS, are students at the University
associated with their use among physical therapists in clinical practice.
of Vermont.

Design. The study used an observational design. [Jette DU, Halbert J, Iverson C,
et al. Use of standardized out-
come measures in physical thera-
Methods. A survey questionnaire comprising items regarding the use and per- pist practice: perceptions and
ceived benefits and barriers of standardized outcome measures was sent to 1,000 applications. Phys Ther. 2009;89:
randomly selected members of the American Physical Therapy Association (APTA). 125–135.]

© 2009 American Physical Therapy

Results. Forty-eight percent of participants used standardized outcome measures. Association
The majority of participants (⬎90%) who used such measures believed that they
enhanced communication with patients and helped direct the plan of care. The most
frequently reported reasons for not using such measures included length of time for
patients to complete them, length of time for clinicians to analyze the data, and
difficulty for patients in completing them independently. Use of standardized out-
come measures was related to specialty certification status, practice setting, and the
age of the majority of patients treated.

Limitations. The limitations included an unvalidated survey for data collection

and a sample limited to APTA members.

Conclusions. Despite more than a decade of development and testing of stan-

dardized outcome measures appropriate for various conditions and practice settings,
physical therapists have some distance to go in implementing their use routinely in
most clinical settings. Based on the perceived barriers, alterations in practice man-
agement strategies and the instruments themselves may be necessary to increase their

Post a Rapid Response or

find The Bottom Line:

February 2009 Volume 89 Number 2 Physical Therapy f 125

Downloaded from by guest on March 17, 2015
Standardized Outcome Measures in Physical Therapist Practice

tandardized instruments mea- know of any clinical trials that have cluded questions about use of a vari-
suring various aspects of health demonstrated the direct effects of us- ety of types of outcomes measures;
status have been advocated for ing standardized outcome measures, however, the authors included man-
use by rehabilitation professionals suggested benefits include identify- ual muscle testing and goniometric
for many years, and much has been ing patients who are at risk for poor measurements in their definition of
written about the potential benefits or adverse outcomes,4 facilitating im- outcomes measures. In the 1998
of, and barriers to, the use of such proved continuity of care for pa- study, a high proportion of respon-
measures in practice.1–5 Additionally, tients transitioning from one health dents used manual muscle testing
many such instruments have been care setting to another,11 determin- (88%) and goniometry (90%),
developed for use for patients with ing the most cost-effective settings whereas relatively low proportions
the various conditions managed by for patients to receive rehabilitation used measures such as the Func-
physical therapists. These instru- services,11 assessing practitioner and tional Independence Measure (FIM)
ments have been referred to in the organizational performance,4 and de- (18%) or the Impairment Inventory
literature using different terms such termining the most-effective inter- scale of the Chedoke-McMaster
as “health status measures,” “disabil- ventions for particular conditions.4 Stroke Assessment (35%).
ity measures,” “outcome measures,”
and “quality-of-life measures.” In gen- The need for physical therapists to In 1997, a study examining the use of
eral, they assess the actual or per- use standardized outcome measures outcome measures in rehabilitation
ceived ability of an individual to has been recognized at the national centers in the United Kingdom
carry out activities such as moving in level in the United States. The Cen- showed that 77% of the centers used
an environment or completing per- ters for Medicare & Medicaid Ser- at least one tool; of those centers,
sonal care and to participate in life vices sponsored a report in 2006 to 28% used some measures of general
situations such as work or household determine the possibility of a uni- motor function, and 88% used at
management. The literature, how- form rehabilitation outcomes assess- least one measure of disability.13 In
ever, also includes studies in which ment method for patients leaving 2001, 2 studies were published that
physical therapists have defined acute care.11 The authors proposed examined the use of outcome mea-
these measures to include assess- several purposes for this type of as- sures in Europe.6,14 Haigh et al6
ment of body function.6 –9 Although sessment, including provider deci- found that a few rehabilitation cen-
referred to by different terms and sion making, patient safety, and abil- ters used a large number of tools on
defined at different levels, these mea- ity to determine patients’ health and a small proportion of patients. For
sures, in general, are standardized in function longitudinally.11 On a patients with orthopedic conditions,
that they use closed-ended question- smaller scale, the Commission on the outcomes measured were largely
naire formats or specific protocols Accreditation in Physical Therapy at the body function level. For pa-
for implementation, provide scores Education12 supports the use of stan- tients with neurological conditions,
that allow quantitative assessment dardized outcome measures in prac- disease-specific measures of disabil-
of ability, and have been evaluated tice by requiring all education pro- ity were used more frequently.
for their psychometric properties. grams to demonstrate that their There was minimal use of generic
When they are used to determine the graduates have some experience in measurement tools that can be used
change in ability from before to after using and interpreting them during regardless of condition. Although
an intervention, they may be re- their professional (entry-level) specific data were not reported,
ferred to as outcome measures. education. Torenbeek et al14 noted low overall
satisfaction with outcome measure-
The drive for use of standardized out- The literature provides relatively few ment for patients with stroke and
come measures in practice has been reports of the overall use of standard- low back pain among rehabilitation
motivated to some extent by the rec- ized outcome measures by physical professionals in 5 European coun-
ognition that goals for patients’ im- therapists. Physical therapists in 5 ac- tries. In addition, there was little
provement not only must consider ademically affiliated institutions in consensus about which outcome
the traditionally measured impair- Toronto were surveyed in 19929 and measures to use. In a study of phys-
ments in body function (eg, range of again in 19988 to determine their use ical therapists in outpatient clinics in
motion, strength [force-generating of standardized outcome measures the United States, Russek et al15
capacity]) but also should consider and the perceived obstacles to their found that only 50% of the respon-
patients’ points of view and prefer- use. A second part of the latter study dents used the outcome tools they
ences for daily activities and life par- used qualitative methods to expli- had been provided by their clinics’
ticipation.10 Although we do not cate the findings.7 The studies in- corporate owner.

126 f Physical Therapy Volume 89 Number 2 February 2009

Downloaded from by guest on March 17, 2015
Standardized Outcome Measures in Physical Therapist Practice

A few studies7,8,13,15,16 have exam- their clinical applications, percep- the instruments as “health status
ined perceptions of the benefits of tions of their value, and barriers to questionnaires.” In an attempt to be
and barriers to using standardized their use. Secondarily, we examined consistent with terms used in the
outcome measures among rehabilita- the relationships between practice most recent rehabilitation literature,
tion professionals, and many of the setting and therapist characteristics we use the term “standardized out-
reported barriers were similar across and the use of standardized outcome come measures” throughout this ar-
studies. Perceptions about barriers measures. ticle, recognizing the various terms
include lack of time and inconve- used to identify these measures.
nience; lack of familiarity, know- Method
how, and training; and lack of re- Procedure Approximately 3 weeks after the ini-
sources such as staffing and One thousand potential participants tial mailing, those therapists who did
automation. Attitudes and percep- were randomly selected from the not respond and who had e-mail ad-
tions related to use of outcome mea- membership list of the American dresses listed in the APTA Web site
sures among other health care pro- Physical Therapy Association directory were sent a reminder
viders, including mental health (APTA). The sample size was deter- e-mail, with the survey questionnaire
practitioners, oncologists, general mined based on an estimated 50% and letter as attachments. After an
practitioners (GPs), and nurses, also return rate and a desire for a 95% additional week, another survey
have been reported. Garland et al3 confidence interval of 5 or less if a questionnaire was mailed to those
found variability in attitudes across response was chosen by 50% of the who had not responded to the initial
mental health practitioners, but sample. The random selection pro- mailing or e-mail.
noted that, in general, the responses cess was computer generated and
reflected ambivalence. All of the stratified by geographic area. In Instrument
practitioners interviewed had partic- March 2008, these individuals re- The survey instrument (eAppendix 1
ipated in mandated outcome assess- ceived a survey questionnaire and a available at http://www.ptjournal.
ments, yet they reported being more letter explaining the purpose of the org) was designed by the investiga-
likely to use their own intuition than study and requesting return of the tors. The initial draft was sent to 14
standardized measures to evaluate completed survey questionnaire by clinician colleagues for input. Eight
clients’ progress. Similarly, Taylor et postage-paid return mail. Participa- clinicians in various types of prac-
al17 reported that many oncologists tion was presumed to indicate in- tice, including acute care, outpatient
they interviewed relied on their own formed consent. hospital-based care, and private prac-
impressions and informal assess- tice, responded. They had between
ments of patients’ quality of life to The letter sent to potential partici- 15 and 30 years of practice as phys-
inform their decisions. Most respon- pants noted that the instruments we ical therapists. They were asked to
dents argued that the use of stan- were asking about were “referred to assess the face and content validity
dardized measures made decision by various names and often include of the items in the survey instru-
making more difficult rather than fa- information that is related to pa- ment, to indicate whether there
cilitating it. As in the previously men- tients’/clients’ social, physical, or were important gaps, and to indicate
tioned studies, approximately one psychological status as they relate to whether any items were unclear or
half of GPs and nurses interviewed in daily activities or role participation. confusing. Changes to the survey in-
a study by Meadows et al18 said that Examples include Oswestry Low strument were made based on their
they preferred relying on their own Back Pain Questionnaire, Functional feedback. We also used the previous
clinical judgment in the management Independence Measure (FIM), Ar- literature (cited in the introduction
of their patients. thritis Impact Questionnaire (AIM), of this report) related to health care
and SF-36 [Medical Outcome Study practitioners’ attitudes toward, and
Because of the lack of recent infor- 36-Item Short-Form Health Survey]. use of, standardized outcome mea-
mation about the use of standardized This study asks you to think broadly sures to support the content validity
outcome measures among physical about the measures.” The question- of the instrument. Construct validity
therapists in the United States and naire indicated that in thinking of the parts of the instrument that
the professional and governmental broadly, respondents should con- assessed beliefs about the usefulness
emphasis on the collection and ap- sider instruments “described with of and barriers to using instruments
plication of data from such instru- terms such as ‘health status,’ ‘quality in practice was assessed through fac-
ments, this study was conducted to of life,’ ‘disability,’ ‘functional sta- tor analysis. A principal components
determine the extent of their use, tus,’ or ‘outcomes measures.’” In the factor analysis with varimax rotation
survey questionnaire, we referred to resulted in 5 factors that explained

February 2009 Volume 89 Number 2 Physical Therapy f 127

Downloaded from by guest on March 17, 2015
Standardized Outcome Measures in Physical Therapist Practice

57% of the variance in item re- pendent variables that were signifi- of and problems with using standard-
sponses. Cronbach alpha was deter- cant. We chose one level of each ized outcome measures in practice
mined for each of the factors to variable as a reference group to al- among the participants who used
provide evidence for internal consis- low the most salient interpretation them. More than 90% of the partici-
tency. We interpreted the 5 factors of results. pants who used them agreed that
to support the framework for atti- standardized outcome measures en-
tudes and beliefs provided by the Results hance communication with patients
literature. The factors represented Participants and help to direct a plan of care.
benefits for the management of the Completed questionnaires were re- More than 75% of the participants
patient (7 items, ␣⫽.85), problems ceived from 498 participants, for a who used them agreed that prob-
or limitations for the physical thera- response rate of 49.8%. Three ques- lems with standardized outcome
pist (6 items, ␣⫽.77), problems or tionnaires were returned as undeliv- measures are that they are confusing
limitations for the patient (6 items, erable, 1 questionnaire was returned to patients, difficult for patients to
␣⫽.77), benefits for external com- with no responses, and 38 question- complete, and too time consuming
munication (3 items, ␣⫽.67), and naires were returned with respon- for patients.
limitations due to culture or lan- dents indicating that they did not
guage (2 items, ␣⫽.59). Taken all manage patient care. We, therefore, Implementation of Standardized
together, the internal consistency of had 456 usable questionnaires. Simi- Outcomes Measures in Practice
the items related to beliefs about the lar response rates have been re- Most frequent uses of information
benefits of using standardized out- ported by Haigh et al,6 Russek et al,15 from standardized outcome mea-
come measures was good (␣⫽.84). and Hatfield and Ogles.19 sures were quality assurance, com-
The internal consistency of all items municating with other health care
related to beliefs about problems of Sixty-eight percent of the partici- providers, and determining progress
or barriers to the use of standardized pants were female, and 32% were or outcomes of individual patients
outcome measures was similarly male. The majority (61%) worked in (Tab. 4). Of the participants who
good (␣⫽.83). an outpatient setting. A slim majority used standardized outcome mea-
(53.4%) of participants had postbac- sures, 35.1% responded that they
Data Analysis calaureate professional degrees. were required for all patients in their
Data were analyzed using SPSS sta- Thirty-two percent were certified setting, and 23.8% responded that
tistical software, version 15.0.* Re- clinical specialists. Although not for- they were routinely used for all pa-
sponse frequencies and means or mally tested, the sample seemed to tients but not mandated. The most
medians for the survey items were reflect the demographics of APTA common means of collecting data
determined and displayed in tabular members reported in 2006 and 2007 and analyzing outcome was to have
and graphic formats. After examin- fairly well.20 Our sample had a patients complete paper forms fol-
ing the response frequencies, and slightly greater proportion of those lowed by therapists’ review of the
before examining the associations with postbaccalaureate degrees and raw information (80.6%). That is, the
among variables, some variable cate- less time in practice. Our sample also therapists did not necessarily have
gories were collapsed in order to al- appears to have had slightly more access to scores from the measure-
low further analysis and derive stable therapists working in outpatient and ment tool when seeing the patient
models. acute care settings. It is difficult to and used only their qualitative assess-
determine whether these differences ment of the responses.
Logistic regression analyses were were due to the different time
conducted to examine the associa- frames in which the data were col- Participants were asked to list the
tion of participant and practice char- lected or to bias in the sample. Par- measures that they used in their
acteristics with the use of standard- ticipant and practice characteristics practices and to indicate whether
ized outcome measures. We used a of the sample are shown in Tables 1 the measures were “home grown.”
forward selection process to derive and 2, respectively. The most frequently listed measures
models, requiring P⬍.05 to enter were: Oswestry Low Back Disability
and P⬍.10 to delete. Odds ratios and Overall Perceptions of Index (ODI) (41.3%); facility “home-
their 95% confidence intervals were Standardized Outcome Measures grown” measures (22%); Lower Ex-
recorded for each level of the inde- Of the 456 participants, 218 (47.8%) tremity Functional Scale (LEFS)
indicated that they used standard- (18.8%); Disabilities of the Arm,
* SPSS Inc, 233 S Wacker Dr, Chicago, IL ized outcome measures in practice. Shoulder, and Hand (DASH) (18.3%);
60606. Table 3 shows the perceived benefits and Berg Balance Scale (BBS)

128 f Physical Therapy Volume 89 Number 2 February 2009

Downloaded from by guest on March 17, 2015
Standardized Outcome Measures in Physical Therapist Practice

Table 1.
Participant Characteristics (N⫽456)a
95% CI
National Data20
Variable Percentage Lower Bound Upper Bound N (%)

Sex (1 missing)

Male 31.9 26.9 36.9 145 34.7

Female 68.1 63.1 73.1 310 65.3

Years of physical therapist practice (4 missing)

⬍3 13.7 10.4 17.0 62 11.1 (⬍4 y)

3–5 11.3 8.1 14.3 51 6.5 (4–5 y)

6–10 17.9 14.1 21.7 81 17.2

11–20 24.8 20.4 29.2 112 27.1

⬎20 32.3 27.3 37.3 146 38.5

Professional (entry-level) degree (1 missing)

Certificate 4.4 2.4 6.4 20 6.9

Baccalaureate 42.2 36.7 47.7 192 48.8

Master’s 40.9 35.5 46.3 186 35.6

Doctorate 12.5 9.4 15.6 57 8.1

Highest degree (3 missing)

Professional 72.2 66.6 77.8 327 Unable to determine

Advanced master’s 13.2 9.9 16.5 60 Cannot distinguish from

professional degree
Transitional DPT 10.6 7.7 13.5 48 8.9

Doctorate 4.0 2.2 5.8 18 4

Specialty (could have more than 1) (35 missing)

None 68.0 61.9 74.1 296

Cardiovascular-pulmonary 0.5 ⫺0.1 1.1 2

Geriatric 3.9 2.1 5.7 17

Neurology 1.6 0.8 2.4 7

Orthopaedic 11.5 8.2 14.8 50

Pediatric 2.5 ⫺0.7 5.7 11

Sports 1.8 0.6 3.0 8

Manual therapy 5.5 3.3 7.7 24

Hand therapy 1.1 0.1 2.1 5

Other 3.7 1.9 5.5 15

CI⫽confidence interval, DPT⫽Doctor of Physical Therapy.

(17.9%). The eAppendix 2 (available been shown to be valid and reliable for not using standardized outcome
at com- (64%). measures were: they are too time
prises a list of all measures listed by consuming for patients to complete
the participants. The most frequent Fifty-two percent of participants in- (43%); they are too time consuming
reasons for choosing specific stan- dicated they did not use standardized for clinicians to analyze, calculate,
dardized outcome measures were: outcome measures in practice, and and score (30%); and they are too
they could be completed quickly 49% of them indicated that they did difficult for patients to complete in-
(68.7%), they were easy for patients not plan to implement their use in dependently (29.1%) (Tab. 5).
to understand (68.2%), and they had future. The 3 most common reasons

February 2009 Volume 89 Number 2 Physical Therapy f 129

Downloaded from by guest on March 17, 2015
Standardized Outcome Measures in Physical Therapist Practice

Table 2.
Practice Characteristics (N⫽456)a
95% CI
National Data20
Variable Percentage Lower Bound Upper Bound N (%)

Region (26 missing)

Northeast 19.1 15.0 23.2 82

Midwest 25.3 20.7 29.9 109

South 31.6 26.5 36.7 136

West 23.5 19.1 27.0 101

Guam and Virgin Islands 0.5 ⫺0.1 1.1 2

Type of work facility (21 missing)

Acute care 15.4 11.8 19.0 67 13.1

Inpatient rehabilitation (including subacute care) 6.0 3.8 8.2 26 3.5

Extended care 3.0 1.4 4.6 13 5.6 (including SNF)

Outpatient 61.2 55.2 67.0 266 56

Home health 7.8 3.1 12.5 34 7.9

School system 3.4 1.6 5.2 15 4.1

Other 3.2 1.6 4.8 14 9.8

Age, y (majority of patients) (6 missing)

No majority 70.0 64.2 75.8 315

⬍21 8.7 6.0 11.4 39

21–40 1.8 0.6 3.0 8

41–60 7.1 4.7 9.5 32

61–75 5.1 3.1 7.1 23

⬎75 7.3 4.9 9.7 33

Conditions (majority of patients) (3 missing)

No majority 30.7 25.8 35.8 139

Musculoskeletal 56.1 50.5 62.1 254

Neuromuscular 6.4 4.2 8.6 29

Cardiovascular-pulmonary 1.5 0.3 2.7 7

Women’s health 0.4 ⫺0.2 1.0 2

Integumentary 0.4 ⫺0.2 1.0 2

Other 4.4 2.4 6.4 20

X 95% CI

Treatment sessions per 8-h day 10.9 10.5 11.3

CI⫽confidence interval, SNF⫽skilled nursing facility.

Odds of Using Standardized therapists working in acute care set- use standardized outcome measures
Outcome Measures tings, those working in outpatient than those who did not have a spe-
The type of facility in which the par- settings were nearly 7 times more cialty (Tab. 6).
ticipant practiced, whether or not likely to use standardized outcomes
the participant had a clinical spe- measures and those working in Discussion
cialty certification, and the age of the home care settings were approxi- More than 50% of the respondents in
majority of patients managed in the mately 12 times more likely to use this study reported that they did not
practice were related to the likeli- standardized outcome measures. Par- use standardized outcome measures,
hood of using standardized outcome ticipants with a clinical specialty and only a small proportion of those
measures. Compared with physical were nearly 2 times more likely to indicated that they intended to use

130 f Physical Therapy Volume 89 Number 2 February 2009

Downloaded from by guest on March 17, 2015
Standardized Outcome Measures in Physical Therapist Practice

them in the future. There are no Table 3.

comparable data reported from sim- Perceived Benefits and Problems Among Physical Therapists Who Used Standardized
ilar samples to assist in the interpre- Outcome Measures (n⫽218)
tation of this number. The use of N Percentage
standardized measures was variable Perceived benefits
across settings, and the greater like-
Enhance communication with patient 206 94.5
lihood of use in the outpatient and
Help direct the plan of care 204 93.6
home health care settings compared
with the acute care setting was not Enhances communication with payers 190 87.2

surprising. Abrams et al16 reported Enhance thoroughness of physical therapist examination 190 87.2
that among physical therapists who Improve patient outcomes 184 84.4
participated in their survey, with Help focus the intervention 182 83.5
most managing a majority of patients
Helps to motivate patient 172 78.9
with orthopedic conditions, usage of
Enhance efficiency of physical therapist examination 170 78.0
standardized outcome measures was
fairly high. In the home health care Help to decrease insurance denials 150 68.8

setting, the Outcome Assessment Enhance marketing of practice 115 52.8

and Information Set (OASIS) is man- Other 11 5.0
dated. Huijbregts et al7 reported the Perceived problems
perception that it would be difficult
Confusing to patients 174 79.8
to find suitable measures for patients
Difficult for patients to complete 166 76.1
who might have fluctuating condi-
tions, such as those in intensive care Take too much time for patients 164 75.2

units. Hanekom et al,21 in a 2007 Often are not completed at discharge, so cannot give 144 66.1
systematic review of outcomes mea- information about response to treatment

sures used by physical therapists in Take too much of clinicians’ time 113 51.8
intensive care units, reported that Make patients/clients anxious 110 50.5
only one case study measured func- Are difficult to interpret 102 46.8
tion using the modified Borg scale. Are not culturally sensitive 100 45.9
No other functional measures or
Require too high a reading level 97 44.5
measures of health-related quality of
life were found as outcome measures Provide information that is too subjective 87 39.9

in any of the studies they reviewed. Do not help to direct the plan of care 71 32.6

Items are not relevant for my patients 71 32.6

In our study, the finding that 3 of Require more effort than they are worth 67 30.7
the most frequently used measures English is a language in which many of my patients/clients 58 26.6
are useful in orthopedic conditions are not fluent
is not surprising given the fact that a
majority of the participants prac-
ticed in outpatient settings and only approximately 4% of assess- Science citation index also indicates
approximately 11% had orthopedic ments done for patients with low that the articles in which the LEFS
clinical specialty certification. Among back pain across 418 rehabilitation and DASH were originally reported
Australian physical therapists who centers in Europe in 1998. Toren- have been cited 74 and 431 times
managed mostly patients with ortho- beek et al14 indicated that the ODI since their original publications in
pedic conditions, Abrams et al16 was used in rehabilitation facilities in 1999 and 1996, respectively.24,25
found a relatively high use of the 4 out of 5 European countries; they These data suggest that the measures
ODI; approximately 50% of the ther- reported the highest use in Ireland are fairly well known, at least among
apists indicated that they used the (12.5% of facilities). The date of their those publishing articles in scientific
ODI frequently or always. Measures survey was not reported. The ODI is journals. Many standardized out-
specific to other body regions were available in the public domain, and come measures have been devel-
used less frequently, but the authors the ISI Web of Science citation in- oped within the last decade or so,
did not indicate the specific names dex22 identifies 1,035 citations of the and this timing may explain why the
of most of those measures. Haigh et article in which it was originally re- participants who had been practic-
al6 reported that the ODI was used in ported in 1980.23 The ISI Web of ing for more than 20 years were

February 2009 Volume 89 Number 2 Physical Therapy f 131

Downloaded from by guest on March 17, 2015
Standardized Outcome Measures in Physical Therapist Practice

Table 4. ago, Russek et al15 reported that the

Uses of Information Among Physical Therapists Who Used Standardized Outcome physical therapists in their study
Measures (n⫽218) identified lack of personnel to assist
Use N Percentage in data management as a barrier to
Quality assurance 173 79.4
implementation of these measures.
Kay et al8 reported that approxi-
Communicating with other health care providers 167 76.6
mately 42% of the physical therapists
Determining progress/outcomes of individual patients 163 74.8
they surveyed in 1998 thought that
Determining average patient improvement to examine practice effectiveness 154 70.6 lack of resources was an obstacle.
Determining average patient improvement to examine clinician effectiveness 125 57.3 The study of nurses and GPs indi-
Traditional research 113 51.8 cated that they, too, would be more
Comparing patient outcomes across conditions 108 49.5
willing to use standardized measures
if the data were collected and ana-
Determining case mix 89 40.8
lyzed by someone else.18
Comparing clinicians’ performances 82 37.6

Comparing clinics’ performances 72 33.0 In our study, approximately 7% of

the participants indicated that com-
puters, and not paper, were used for
much more likely than their younger given for not using them among completion and analysis of measures,
colleagues to learn about them from those who did not use them were and slightly fewer than 10% of par-
continuing education workshops fairly similar and included issues that ticipants indicated that they chose
and other therapists than from for- have been discussed in the literature measures based on their ability to
mal, professional education. for more than a decade.13,15,16 Even analyze data electronically. Recent
in the most recent study,16 the ma- literature has suggested that imple-
One surprise is the relatively high jority of participants indicated lack mentation of computerized systems
use (22%) of “home-grown” mea- of familiarity with, lack of training in, is critical to clinical practice in terms
sures. Similarly, Kay et al8 reported and lack of access to measures were of evaluating both individual patients
that 18% of the physical therapists barriers. Practitioners in other health and overall practice performance.
surveyed in their study used depart- care specialties have reported the For example, in 1994, Shields et al26
mentally developed instruments. same types of barriers as those re- described the development of a
This practice seems unnecessary ported by physical therapists. Mead- computer-based clinical database in
given the large number of existing ows et al18 reported that 39% of GPs the acute care setting and urged its
measures that cover all body regions and 28% of nurses indicated having implementation to better measure
and many specific conditions. The insufficient time to discuss health outcomes of physical therapy inter-
finding also is somewhat contradic- outcome data with their patients. Lo- ventions. More recently, Jette et al,27
tory, given that 68% of those who gistical problems such as time, addi- reporting on a new standardized
used standardized outcome mea- tional paperwork, and costs of per- outcome measure that uses a com-
sures indicated that one reason for sonnel were cited as the most puterized adaptive testing format,
choosing an instrument was its doc- important reason for not using the suggested that challenges for imple-
umented validity and reliability. We measures among psychologists.19 mentation included assisting clini-
also found that participants defined Based on our results, it appears that cians in carrying out the testing as
outcome measures broadly to in- many physical therapist practices well as understanding and interpret-
clude not only measures of activity may not yet have determined how ing the data derived from such mea-
and participation but also some mea- best to address these barriers. sures. They stressed the need for
sures of body function such as the training, technical support, and ac-
BBS. This finding is reflected in the Twenty-seven percent of the partici- cess to software.
literature in that previous reports of pants in our study who did not use
use of outcome measures by physical standardized outcome measures In our study, 18% of the participants
therapists have included references cited the lack of a support system in who did not use standardized out-
to measures of body function.6 – 8 terms of technology and staffing as a come measures cited the lack of rel-
reason, and only 11% of those who evance to the plan of care as a rea-
The problems perceived by physical used the measures indicated that of- son. Kay et al8 found that 39% of
therapists who used standardized fice staff helped patients to complete physical therapists surveyed in 1998
outcomes measures and the reasons them. Similarly, more than 10 years thought that outcome measures did

132 f Physical Therapy Volume 89 Number 2 February 2009

Downloaded from by guest on March 17, 2015
Standardized Outcome Measures in Physical Therapist Practice

not meet the needs of their patients. Table 5.

Researchers reporting the percep- Reasons Among Participants Who Did Not Use Standardized Outcome Measures
tions of nurses, GPs, psychologists, (n⫽238, May Indicate More Than 1 Reason)
and oncologists also cite lack of clin- Reason N Percentage
ical relevance as a barrier to use of Take too much time for patients/clients to complete 102 43.0
standardized outcome measures. For
Take too much of clinicians’ time to analyze/calculate/score 71 30.0
example, Hatfield and Ogles19 re-
Are difficult for patients/clients to complete independently 69 29.1
ported that a substantial number of
psychologists felt that standardized Require a support system that I do not have (eg, technology, staffing) 64 27.0

outcome measures could “distort” Often are not completed at discharge, so are not useful in determining 58 24.5
the effects of treatment. General patients’/clients’ response to treatment

practitioners and nurses stated that Do not contain the types of items or questions that are relevant for the 57 24.1
types of patients/clients who I see
they were more likely to use stan-
dardized outcomes measures if they Other reason 54 21.2

helped in the care of the individual Are confusing for patients/clients 48 20.3
patient,18 and oncologists indicated Require more effort than they are worth 47 19.8
that informal collection of data Do not contain information that helps direct the plan of care 43 18.1
seemed a better way to understand
Are difficult to interpret (eg, do not know what norms are, how score relates 40 16.9
individual patient needs than using to severity, or what a clinically important change might be)
standardized outcome measures.17 Require too high a reading level for my patients/clients 27 11.4
Among the physical therapists in our
Make patients/clients anxious 22 9.3
study who used standardized out-
come measures, however, the major- Provide information that is too subjective to be useful 22 9.3

ity believed that these measures Require training that I do not have 18 7.6
could aid in directing the plan of Are in English, a language in which many of my patients/clients are 16 6.8
care and enhancing the thorough- not fluent

ness of their examinations. Similarly, Are not sensitive to the cultural/ethnic concerns of many patients/clients 10 4.2
previous studies7,14 have shown that Cost too much 7 3.0
physical therapists perceived plan- Are really only useful for research purposes 7 3.0
ning of care and monitoring the ef-
Are not relevant because my practice involves consultation, case 6 2.5
fects of treatment as benefits of stan- management, or discharge planning only
dardized outcome measures. Plan to implement?
Although it is likely that many phys-
No 110 49.3
ical therapists are similar to other
health care practitioners in valuing Maybe 93 41.4

and applying the qualitative informa- Yes 20 9.0

tion gathered from patients, differ-
ences in perceptions regarding the
usefulness of standardized outcome they used in different ways. Addi- entire profession of physical ther-
measures may be due to the fact that tionally, the validity and test-retest apy. Given that APTA members may
physical therapists have better tools reliability of our survey data were be more likely than nonmembers to
for measuring the constructs that not tested. We attempted, however, attend national meetings, they may
provide a basis for evaluating the ef- to demonstrate content validity be more likely to have been exposed
fectiveness of their care. through use of previous literature on to issues related to measuring out-
the topic and construct validity comes. Therefore, we might specu-
Limitations through factor analysis. There was late that those who are members
One limitation of our study is that good internal consistency within the would be more likely than nonmem-
our data reflect what has been re- items assessing the perceived bene- bers to use standardized outcome
ported by physical therapists rather fits and barriers to using outcome measures. We considered our re-
than what has been observed, and measures. Another limitation was sponse rate to be adequate in that it
although we provided our partici- that we sent survey questionnaires was comparable to that reported in
pants with a definition of standard- only to members of APTA. There- similar studies; however, there is the
ized outcome measures, they may fore, the results of this study may be possibility that the sample was
have thought about the measures biased and not representative of the biased.

February 2009 Volume 89 Number 2 Physical Therapy f 133

Downloaded from by guest on March 17, 2015
Standardized Outcome Measures in Physical Therapist Practice

Table 6.
Odds of Using Standardized Outcome Measures by Participant and Practice Characteristicsa
95% CI

Factor Odds Ratio Lower Upper Percentage Using


Acute care Reference 16.4

Inpatient rehabilitation 2.63 0.80 8.67 30.8

Extended care 2.21 0.47 10.30 23.1

Outpatient 6.80 2.99 15.48 60.5

Home care 12.56 4.36 36.18 64.7

School system 1.04 0.10 11.07 6.7

Other type of facility 5.46 1.27 23.43 50.0


No Reference 43.2

Yes 1.72 1.03 2.88 59.2

Age of majority (⬎50%) of patients, y

⬍21 Reference 17.9

21–60 3.58 1.16 11.05 57.9

⬎60 2.42 0.74 7.92 35.0

No majority 6.57 2.03 21.31 59.1

CI⫽confidence interval.

Implications report that the measures are not about change in management strate-
Despite more than a decade of devel- used because they are not applicable gies, referral, or discharge from ser-
opment and testing of measures ap- to their patients or that they cannot vices. As noted by Jette et al,27 the
propriate for various conditions and interpret the scores. It appears, essential strategies to improve use
practice settings, the physical ther- therefore, that disseminating infor- of standardized outcome measures
apy profession appears to have some mation through the professional lit- may well require new funding
distance to go in implementing stan- erature may not be an efficient or mechanisms.
dardized outcome measurement rou- effective mechanism. Further in-
tinely in most clinical settings. The struction and enculturation through Given that many of our participants
development of such measures for continuing education as well as pro- believed that standardized outcome
acute care settings may need to be a fessional and graduate professional measures are confusing and difficult
particular focus. Regardless of set- education may increase the use of for patients to complete, efforts
ting, practices will need to help cli- standardized outcome measures. Ed- should be made to ensure readability
nicians to manage time so that col- ucation should include the use of and interpretability by patients.
lection of data can become routine hardware and software to facilitate Reading level, font size, and general
despite productivity expectations. their usage. In addition, software appearance of measurement tools
Given the perceived time-consuming should be made readily available to need to be considered. Language and
nature of standardized outcome mea- provide analyses that assist in the in- cultural concerns were cited by rel-
surement, investment in computer- terpretation of scores. Interpretation atively few of our participants; how-
ized systems for quick data entry and could include comparing patients’ ever, given the changing nature of
analysis may be warranted. scores with norms; using scores to the US population, these concerns
qualify severity of condition or pre- may become magnified and necessi-
Although the content, properties, dict outcome or duration of an epi- tate adaptations to the commonly
and applicability of many standard- sode of care; or categorizing changes used instruments.
ized outcome measures have been in scores as worse, stable, or im-
reported in the literature for more proved. Such data could assist phys-
than a decade, clinicians continue to ical therapists in making decisions

134 f Physical Therapy Volume 89 Number 2 February 2009

Downloaded from by guest on March 17, 2015
Standardized Outcome Measures in Physical Therapist Practice

Conclusion 2 Deyo RA, Patrick DL. Barriers to the use of 16 Abrams D, Davidson M, Harrick J, et al.
health status measures in clinical investi- Monitoring the change: current trends in
Most participants in our study did gation, patient care, and policy research. outcome measure usage in physiotherapy.
not use standardized outcome mea- Med Care. 1989;27:S254 –S268. Man Ther. 2006;11:46 –53.
sures. There was a relationship be- 3 Garland AF, Kruse M, Aarons GA. Clini- 17 Taylor KM, Macdonald KG, Bezjak A, et al.
cians and outcome measurement: What’s Physicians’ perspective on quality of life:
tween specialty certification, age of the use? J Behav Health Serv Res. an exploratory study of oncologists. Qual
patients seen, and practice setting 2003;30:393– 405. Life Res. 1996;5:5–14.
and likelihood of using standardized 4 Lansky D, Butler JBV, Waller FT. Using 18 Meadows KA, Rogers D, Greene T. Atti-
health status measures in the hospital set- tudes to the use of health outcome ques-
outcome measures. Most partici- ting: from acute care to outcomes manage- tionnaires in the routine care of patients
pants who used standardized out- ment. Med Care. 1992;30:MS57–MS73. with diabetes: a survey of general practi-
tioners and practice nurses. Br J Gen
come measures perceived that their 5 Nelson EC, Berwick DM. The measure- Pract. 1998;48:1555–1559.
ment of health status in clinical practice.
use enhanced communication with Med Care. 1989;27:S77–S90. 19 Hatfield DR, Ogles BM. Why some clini-
patients and helped to direct the 6 Haigh R, Tennant A, Biering-Sorensen F, cians use outcomes measures and others
do not. Adm Policy Ment Health Ment
plan of care. More than 70% of the et al. The use of outcome measures in Health Serv Res. 2007;34:283–291.
physical medicine and rehabilitation
participants, however, felt that these within Europe. J Rehabil Med. 2001;33: 20 PT Demographics. Available at: http://
tools could be confusing, difficult, 273–278.⫽
and time consuming for patients. It 7 Huijbregts MPJ, Myers AM, Kay TM, Gavin TaggedPageDisplay.cfm&TPLID⫽101&
TS. Systematic outcome measurement in
appears that outcome measures are clinical practice: challenges experienced ContentID⫽14332. Accessed August 1,
used largely without computerized by physiotherapists. Physiother Can. Win-
ter 2002:25–31, 36. 21 Hanekom SD, Faure M, Coetzee A. Out-
systems for either administration or comes research in the ICU: an aid in de-
8 Kay TM, Myers AM, Huijbregts MPJ. How
analysis. Participants with fewer far have we come since 1992? a compara- fining the role of physiotherapy. Phys-
iother Theory Pract. 2007;23:125–135.
years in practice reported learning to tive survey of physiotherapists’ use of out-
come measures. Physiother Can. Fall 22 ISI Web of Science. Available at: http://apps.
use standardized outcome measures 2001:268 –275.
in their professional education pro- 9 Mayo N, Cole B, Dowler J, et al. Use of⫽WOS&product⫽
grams. Focus on education in the use outcome measurement in physiotherapy: FEile2c4k&search_mode⫽GeneralSearch.
survey of current practice. Can J Rehab.
of outcome measures in professional 1993;7:81– 82. Accessed June 30, 2008.
education programs and continuing 10 Thier S. Forces motivating the use of 23 Fairbank CT, Couper J, Davies JB, O’Brien
JP. The Oswestry low back pain disability
education, as well as alterations in health status assessment measures in clin- questionnaire. Physiotherapy. 1980;66:
ical settings and related clinical research.
practice management strategies, may Med Care. 1992;30:MS15–MS22. 271–273.
lead to increasing use of standard- 11 Kramer AM, Holthaus D. Uniform Patient 24 Binkley JM, Stratford PW, Lott SA, Riddle
DL. The Lower Extremity Functional Scale
ized outcome measures in the future. Assessment for Post-Acute Care. 2006. Avail- (LEFS): scale development, measurement
able at: properties, and clinical application. Phys
InitiativesGenInfo/downloads/QualityPAC Ther. 1999;79:371–383.
FullReport.pdf. Accessed August 1, 2008.
Dr Jette provided concept/idea/research de- 25 Hudak PL, Amadio PC, Bombardier C. De-
12 Commission on Accreditation in Physical
sign, data analysis, and project manage- velopment of an upper extremity outcome
Therapy Education. Evaluative Criteria
ment. All authors provided writing. Mr Hal- measure: the DASH (Disabilities of the
for Accreditation of Education Programs arm, Shoulder and Hand). Am J Ind Med.
bert, Ms Iverson, Ms Miceli, and Ms Shah for the Preparation of Physical Thera- 1996;29:602– 608.
provided data collection. pists. Alexandria, VA: American Physical
Therapy Association; 2006. 26 Shields RK, Leo KC, Miller B, et al. An
The study was approved by the Institutional acute care physical therapy practice data-
13 Turner-Stokes L, Turner-Stokes T. The use base for outcomes research. Phys Ther.
Review Board of the University of Vermont. of standardized outcome measures in re- 1994;74:463– 470.
habilitation centres in the UK. Clin Reha-
This article was received August 3, 2008, and bil. 1997;11:306 –313. 27 Jette AM, Haley SM, Tao W, et al. Prospec-
was accepted October 30, 2008. tive evaluation of the AM-PAC CAT in out-
14 Torenbeek M, Caulfield B, Garrett M, Van patient rehabilitation settings. Phys Ther.
Harten W. Current use of outcome mea-
DOI: 10.2522/ptj.20080234 2007;87:385–398.
sures for stroke and low back pain reha-
bilitation in five European countries: first
results of the ACROSS project. Int J Reha-
References bil Res. 2001;24:95–101.
1 Deyo RA, Carter WB. Strategies for im- 15 Russek L, Wooden M, Ekedahl S, Bush A.
proving and expanding the application of Attitudes toward standardized data collec-
health status measures in clinical settings. tion. Phys Ther. 1997;77:714 –729.
Med Care. 1992;30:MS176 –MS186.

February 2009 Volume 89 Number 2 Physical Therapy f 135

Downloaded from by guest on March 17, 2015
Use of Standardized Outcome Measures in Physical
Therapist Practice: Perceptions and Applications
Diane U Jette, James Halbert, Courtney Iverson, Erin
Miceli and Palak Shah
PHYS THER. 2009; 89:125-135.
Originally published online December 12, 2008
doi: 10.2522/ptj.20080234

References This article cites 21 articles, 6 of which you can access

for free at:
Cited by This article has been cited by 25 HighWire-hosted
Permissions and Reprints
Information for Authors

Downloaded from by guest on March 17, 2015