Patient Reported Outcomes in Clinical Research: Kevin P. Weinfurt, PHD

Patient‐Reported
Outcomes in
Clinical Research
Kevin P. Weinfurt, PhD
Introduction to the Principles and
Practice of Clinical Research
National Institutes of Health
November 30, 2015
IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

Overview
1. Why measure patient‐reported
health status?
2. Diﬀerent types of PROs
3. Development and evaluation of PROs
4. “New” methods using Item Response
Theory
5. Interpreting scores on PRO measures
1
Why measure patient-
reported health status?

How do we assess benefit?

Copies of virus in blood
Tumor size
Blood pressure
PeakVO2

For endpoints to inform
decisions, they must matter to
patients, clinicians, and payers.

PeakV02
r = .53
Flynn et al, 2015
Overactive Bladder Syndrome
Pad weight to
measure leakage
volume
What is a
meaningful
reduction in
volume?
Treatment Benefit
Feel Function Survive

FDA
Patient-Reported Outcome
(PRO)
“A measurement based on a
report that comes directly
from the patient (i.e., study
subject) about the status of
a patient’s health condition
without amendment or
interpretation of the
patient’s response by a
clinician or anyone else.”
®
PROMIS Fatigue Measure

2
Different Types of PROs

Characteristics of
the Individual
1. Dilution of effects of
biological interventions
Biological and General Overall

Symptom Functional
Physiological Health Quality
Status Status
Variables Perceptions of Life
2. Correlation between Nonmedical

Factors
successive boxes
decreases Characteristics of
the Environment
Adapted from Wilson & Cleary, (JAMA 1995)
3
Development and
Evaluation of PRO
Measures

1. Determine what PRO concept
we want to measure and why
“Fatigue”

2. Collect qualitative data to understand
meaning of the PRO concept
Fatigue

3. Write items you think will measure
the concept
Item 1
Item 2
Item 3
Item 4
Fatigue
Item 5
Item 6
Item 7
Item 8

4. Test items for understanding
(cognitive interviews)
Item 1
Item 2
Item 3
Item 4
Item 5
Item 6
Item 7
Item 8
Item 9
Item 10

5. Administer items to a large sample
of people
Item 1
Item 2
Item 3
Item 4
Item 5
Item 6
Item 7
Item 8

6. Use psychometric (statistical) analyses to
see how well items are working and develop
scoring method
How well do items fit
this model?
Item 1
Item 2
Item 3
Item 4
Fatigue Item 5
Item 6
Item 7
Item 8
Fatigue Score

7. Evaluate the reliability and validity of
the measure
Validity Reliability
Measures what it’s Measures with little error
supposed to measure (a.k.a. precision)

Types of Validity
Content
Face
Construct
Convergent/discriminant
Known groups
Predictive
Responsiveness
Convergent Validity:PROMIS
Depression Domain

If I have not changed, I should
get the same score . ..
using different sets of Internal Consistency
items from the same (Cronbach’s alpha)
measure
Test‐Retest
over time
regardless of who Interrater

scores it
4
“New” Methods Using
Item ResponseTheory

Traditional Off-the-Shelf PRO
Measure
Everyone must complete
the same items.
All items are necessary

to obtain a score.
Score might not be on

the same metric as other
measures of the same
thing.
An item bank is a large collection of
items measuring a single domain.
Any and all items can be used to provide

a score for that domain.
Dynamic, not fixed.

0 50 100
Physical Functioning Item Bank
Item Item Item Item Item Item Item Item Item Item
1 2 3 4 5 6 7 8 9 n
Are you able to get in and out of bed?

Are you able to stand without losing your balance for 1 minute?
Are you able to walk from one room to another?
Are you able to walk a block on flat ground?
Are you able to run or jog for two miles?
Are you able to run five miles?
Using aTraditional Off-the-Shelf
PRO Measure

Using a PRO Item Bank

Item Bank
Fixed-Length Computerize
Measures d Adaptive
Tests (CATs)
Ready-made Make-your-own

Item Bank
Fixed-Length Computerize
Measures d Adaptive
Tests (CATs)
Next item administered
depends on answer to
previous item

Fatigue Item Bank
Chemotherapy Osteoarthritis Heart failure trial

trial trial (NYHA Class III)
Items 1‐10 CAT Items 6‐12
Diabetes trial Heart failure trial

(NYHA Class I)
Item 7
Items 1‐5
Same metric, same meaning
Different Sites, Different Measures
Item 1
Measure 1 Item 2
Item 3 Metric 1
(in HCS 1) Item 4
Item 5
Item 1
Measure 2 Item 2
Item 3 Metric 2
(in HCS 2) Item 4
Item 1
Item 2
Measure 3 Item 3
Metric 3
(in HCS 3) Item 4
Item 5
Item 6

Item Bank
Item 1
Item 2
Measure 1 Item 3
Item 4
(in HCS 1) Item 5
Item 1
Item 2
Item 3
Measure 2 Item 4
(in HCS 2) Item 1
One Metric
Item 2
Item 3
Item 4
Item 5
Measure 3 Item 6
(in HCS 3) Item 1
Item 2
Other items Item 3
Item 4
Traditional PRO Measure PRO Item Bank
All items are required to Any and all subsets of items

compute a score can generate a score
Everyone must take same Different people can get

items different items
Use items in bank to create

Use it “off the shelf ” measure for specific use
Scores not easily comparable Cross-walk between scores

to scores from another from different measures in
measure of the same domain the same item bank

Examples of PRO Resources
Based on Item Banks

Adult and pediatric item banks
measuring domains relevant across
chronic diseases
Freely available
www.nihpromis.org

Multidimensional set of brief measures
assessing cognitive, emotional, motor
and sensory function from ages 3-85
Freely available
www.nihtoolbox.org
Core set of instruments for use in
chronic neurological conditions
(supplemental set for specific diseases,
patient subgroups)
Freely available
www.neuroqol.org

Differential Item Functioning

In the past 7 days, did
you cry?
Yes No
(Depression item)

Item behavesdifferently for 2 or

more groups.

Item ResponseTheory
In the past 7 days, did you cry?
1.00
Probability of “YES”
0.75
0.50
0.25
0.00
-3 -2 -1 0 1 2 3
Depression

Item ResponseTheory
1.00
0.75
0.50
0.25
0.00
-3 -2 -1 0 1 2 3
Depression

Item ResponseTheory
1.00
0.90
0.75
0.50
0.25
0.00
-3 -2 -1 0 1 2 3
Depression

Item behavesdifferently for 2 or

more groups.
The “map”between depression and

item is different for 2 or more
groups.
1.00
0.90
Females
0.75
Males
0.50
0.30
0.25
0.00
-3 -2 -1 0 1 2 3
Depression

5
Interpreting Scores on
PRO Measures

Exercise vs Usual care
Changes in
Kansas City
Cardiomyopathy
Questionnaire
(Score Range: 0 -100)

1.93
(95% C.I., 0.84, 3.01)
Exercise arm has statistically greater rate of
change between baseline and 3 months.

Spertus et al., 2005
Mean Diff = 1.93
Exercise = 54%
Control = 29%

Review
1. Why measure patient‐reported
health status?
2. Diﬀerent types of PROs
3. Development and evaluation of PROs
4. “New” methods using IRT
5. Interpreting scores on PRO measures

Questions
kevin.weinfurt@duke.edu

Patient Reported Outcomes in Clinical Research: Kevin P. Weinfurt, PHD

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Patient Reported Outcomes in Clinical Research: Kevin P. Weinfurt, PHD

Uploaded by

Copyright:

Available Formats

Patient‐Reported

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

Feel Function Survive

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

Biological and General Overall

2. Correlation between Nonmedical

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

regardless of who Interrater

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

All items are necessary

Score might not be on

Any and all items can be used to provide

Dynamic, not fixed.

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

Physical Functioning Item Bank

Are you able to get in and out of bed?

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

Chemotherapy Osteoarthritis Heart failure trial

Diabetes trial Heart failure trial

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

All items are required to Any and all subsets of items

Everyone must take same Different people can get

Use items in bank to create

Scores not easily comparable Cross-walk between scores

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

Item behavesdifferently for 2 or

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

Item behavesdifferently for 2 or

The “map”between depression and

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

IPPRC (Weinfurt) 2015 Handouts - November 30, 2015

You might also like