VARIABLES

Variables
Variables: Characteristic that can be manipulated or
observed
• Types of Variables
• Independent or Dependent
• Measurement Scales/Levels
Classification is useful for communication, so that
readers are aware of the author’s hypothesis of what
situation or intervention (independent variable) will
predict or cause a given outcome (dependent
variable)
 3

Variables: Independent or Dependent

• Independent Variable: A variable that is

manipulated or controlled by the researcher,
presumed to cause or determine another
(dependent) variable
• Dependent Variable: A response variable that is
assumed to depend on or be caused by another
(independent) variable
• Extraneous variable: Individual, organizational
or environmental characteristics other than the
factor of interest that may influence the
outcome of study
 4

Variables: Measurement Scales

• Useful to convey information to the reader about

the type of variables observed
• Necessary to determine what statistical analysis
approach should be used to examine relationships
between variables
• From lowest to highest level of measurement,
the scales are nominal, ordinal, interval, and ratio
 5

Variables: Measurement Scales

Nominal Scales (Classification Scale)

• Data, with no quantitative value, are organized
into categories
• Categorizes are based on some criterion
• Categories are mutually exclusive and
exhaustive (each piece of data will be assigned to
only one category)
• Only permissible mathematical operation is
counting (such as the number of items within each
category)
• Examples: Gender, Blood Type, Side of
Hemiplegic Involvement
 6

Variables: Measurement Scales

Ordinal Scales
• Data are organized into categories, which are
rank-ordered on the basis of a defined
characteristic or property
• Categories exhibit a “greater than-less than”
relationship with each other and intervals between
categories may not be consistent and may not be
known
 7

Variables: Measurement Scales

Ordinal Scales, continued
• If categories are labeled with a numerical value,
the number does not represent a quantity, but only
a relative position within a distribution (for
example, manual muscle test grades of 0-5)
• Not appropriate to use arithmetic operations
• Examples: Pain Scales, Reported Sensation,
Military Rank, Amount of Assistance Required
(Independent, Minimal)
 8

Variables: Measurement Scales

Interval Scales
• Data are organized into categories, which are
rank-ordered with known and equal intervals
between units of measurement
• Not related to a true zero
• Data can be added or subtracted, but actual
quantities and ratios cannot be interpreted, due to
lack of a true zero
• Examples: Intelligence testing scores,
temperature in degrees centigrade or Fahrenheit,
calendar years in AD or BC
 9

Variables: Measurement Scales

Ratio Scales
• Interval score with an absolute zero point (so
negative numbers are not possible)
• All mathematical and statistical operations are
permissible
• Examples: time, distance, age, weight
 LEVEL OF MEASUREMENT
LEVEL CLINICAL EXAMPLES
NOMINAL GENDER (males, females)
Race/ethnicity (African, Asian, white, Americans etc.)
Religious affiliation (catholic, Jewish, Muslim, etc)

ORDINAL Weight bearing status (non weight-bearing, toe touch, weight

bearing as tolerated)
Level of assistance required (minimum assist, moderate assist,
maximum assist etc)
MMT grading (trace, poor, fair, good etc.)
Patient Satisfaction (very dissatisfied, some what dissatisfied,
neutral, satisfied, very satisfied)

INTERVAL Temperature
Calendar year
RATIO Height
Wt
Circumference
Bp
Speed
Distance
 11

Variables: Clinical Example

A study investigates how a
strengthening program impacts a child’s
ability to independently walk. In this case,
the strengthening program is the
independent variable and the ability to
independently walk is the dependent
variable.
 12

Measurement Validity
• Measurement Validity examines the “extent to
which an instrument measures what it is intended to
measure”

• For example, how accurate

is a test or instrument at
discriminating, evaluating,
or predicting certain items?
 13

Measurement Validity
Validity of Diagnostic Tests
• Based on the ability for a test to accurately
determine the presence or absence of a condition
• Compare the test’s results to known results, such
as a gold standard.
• For example, a test determining balance
difficulties likely to result in falls could be compared
against the number of falls an individual actually
experiences within a certain time frame. A clinical
test for a torn ACL could be compared against an
MRI.
 14

Measurement Validity
Types

• Face Validity: Examines if an instrument appears

to measure what it is supposed to measure (weakest
form of measurement validity)choice to measure this

‘Does this instrument appears to be the

appropriate choice to measure this variable’ YES-
NO
• Content Validity: Examines if the items within an
instrument adequately comprise the entire content of a
given domain reported to be measured by the instrument

‘A comprehensive balance test should include

activities that challenges an individuals ability to
maintain an upright position while sitting/standing but
it should not include activities that require the subject
to maintain recumbent position’

It is challenging as no external standard or statistician

criterion is available against which an instrument can be
judged
Construct Validity: Degree to which an instrument
can justify the operational definition of the concept or
construct
OR
Examines if an instrument can measure an abstract
concept

NOTE:
Construct Validity differs from content validity as it
emphasize on the definition itself rather than the whole
characteristics

e.g:
Patients Level of Satisfaction
Strength
Balance
Level of Dependence
EXAMPLE
• BALANCE
Defined as ‘Remaining upright while holding a position in
space (static) or moving (dynamic)
• STRENGTH
An ability to lift a certain amount of weight a given number
of times or an ability to move in elation to the gravity
 18

Measurement Validity: Types

• Criterion-related Validity: Degree to which its
scores are related to the gold standard

 Concurrent Validity: Examination of Criterion-

related Validity, when the instrument being examined
and the gold standard are compared at the same time
 Predictive Validity: Examination of Criterion-
related Validity, when the outcome of the instrument
being examined can be used to predict a future
outcome determined by a gold standard
 19

Measurement Validity: Statistics

Ways to Evaluate Usefulness of Clinical Screening or
Diagnostic Tools
• Sensitivity and Specificity
• Positive and Negative Predictive Value
• Positive and Negative Likelihood Ratios
• Receiver Operating Characteristic (ROC)
curve
The above mentioned statistical procedures are often
used when researchers are introducing (and validating)
the test. Hopefully the values from these operations can
be found tool’s testing manual or in articles evaluating the
tool’s validity within certain populations or settings.
 20

Measurement Validity: Statistics

Diagnostic Reference Table

Condition
Present Absent
Test Result
True Positive False Positive
Positive a b
False Negative True Negative
Negative c d
 21

Measurement Validity: Statistics

Table Labels:
(a) True Positive: The subject matter has the
condition, and the test accurately identifies the
presence of the condition
(d) True Negative: The subject matter does not have
the condition, and the test accurately identifies the
absence of the condition
(b) False Positive: The subject matter does not have
the condition, and the test incorrectly identifies the
presence of the condition
(c) False Negative: The subject matter has the
condition, and the test incorrectly identifies the
absence of the condition
 22

Measurement Validity: Statistics

• Positive test result = the test identifies the

condition as being present;
• Negative result = the test identifies the condition
as being absent
(This may or may not be accurate when compared to
the gold standard).

The test’s sensitivity and specificity provide

information about the accuracy of the test.
 23

Measurement Validity: Statistics

Diagnostic Reference Table

Example
+Lung Cancer - Lung Cancer
Test Result
a b
+Smoking
d
-Smoking c
 24

Measurement Validity: Statistics

Sensitivity
• The ability of the test to obtain a positive test when
the condition is present; the ability to detect a true
positive (a) patient with condition who test positive
(a + c) All the patients with the condition

•The proportion that test positive out of those with the

condition

•This information allows the PT to determine which

test may be appropriate to use when a particular
condition is suspected
 25

Measurement Validity: Statistics

Specificity
• The ability of the test to obtain a negative result
when the condition is absent, the ability to detect a
true negative (d)
d patients without the condition who test negative
(b + d) All the patients without the condition

The proportion that test negative out of those without

the condition. Sensitivity and Specificity are often
provided in terms of percents, from 0% to 100% (low
to high)
It indicates which test may be appropriate to use for
a suspected condition
 26

Measurement Validity: Statistics

Helpful Hints to remember
Sensitivity and Specificity
• Sensitivity: SnNout: When a test has a high
sensitivity (Sn), a negative result (N), rules out (out)
the diagnosis.
• Specificity: SpPin: When a test has a high
specificity (Sp), a positive result (P), rules in (in) the
diagnosis
 Measurement Validity: Statistics 27
Clinical Example

Example:

You’re choosing between two tests that screen

participation in school based on physical abilities.

A positive result means that the student’s physical

abilities impact his or her participation.
 Measurement Validity: Statistics 28

Clinical Example

One test has a high sensitivity, but a low specificity. A

high sensitivity means that a negative test will
effectively rule out students whose physical abilities
do not impact participation.

However, with a low specificity, there may be many

false positives, meaning students may test “positive”
for the condition when, in fact, their abilities do not
impact participation.
 Measurement Validity: Statistics 29

Clinical Example

Example:

You’re choosing between two tests that evaluate

participation in school based on physical abilities.

A positive result means that the student’s physical

abilities impact his or her participation.
 Measurement Validity: Statistics 30

Clinical Example

The other test has a low sensitivity, but a high

specificity. A high specificity means that a positive
result will effectively rule in the condition.

However, with a low sensitivity, there may be many

false negatives, meaning that students may test
“negative” for the condition, when, in fact, their
abilities do impact their participation.
 31

Measurement Validity: Statistics

Predictive Values
• Provided in terms of percentages, 0% to 100%,
low to high
• Positive Predictive Value (PV+)
• Probability that a person with a positive test
actually has the condition
• a/(a + b)
• High PV+ desired for screening tools, to
prevent excessive unnecessary future testing
• Negative Predictive Value (PV-)
• Probability that a person with a negative
test does not have the condition
• d/(c + d)
(Portney and Watkins, 2000)
WEEK 2
 33

Measurement Validity: Statistics

Likelihood Ratios
• These are mathematical calculation
• Calculated from the Diagnostic Reference Table
• Requires prior calculation of the pretest
probability of the condition in question
• Easy to use when familiar with the concept, but
requires the use of a probability guide chart or a
nomogram (chart that contains pretest probability
and likelihood ratios, with a ruler connecting those
two points to determine posttest probabilities)
 34

Measurement Validity: Statistics

Likelihood Ratios, continued
• Positive and negative likelihood ratios are
calculated
• Determines the usefulness of a diagnostic test.
If a positive or negative result will change the
posttest probability of having a condition to alter
the clinician and patient’s course of action, it will
be useful. If the likelihood ratios of the test do not
result in a substantial change of knowledge, the
test most likely will not be useful.

(Guyatt and Rennie, 2002)

 35

Measurement Validity: Statistics

Receiver Operating Characteristic (ROC) Curve
• Uses sensitivity and specificity information to find
the probability of correctly choosing between
presence or absence of the condition

For example, a test with an area under the ROC

curve of 0.80, would result in the correct
determination of presence or absence of a
condition 80% of the time.

(Portney and Watkins, 2000)

 36

Measurement Validity: Statistics

Responsiveness to chance statistics evaluate a
measurement tool’s ability to detect change over time
• For example, will administration of the test pre
and post intervention reflect a change in status, if
one actually occurred?
• Evaluated by examining the change in scores in
a pretest-posttest design, or using effect size

(Portney and Watkins, 2000)

 37

Measurement Validity: Statistics

Effect Size
• Effect size (ES) is a measure of the amount of
difference.

For example, experimental group A increased their

score on a coordination measure by an average of
15 points, while experimental group B increased
their score an average of 8 points. The ES
between groups would be 7 points, considering
the groups were homogeneous at the start.

(Portney and Watkins, 2000)

 38

Measurement Validity: Statistics

Effect Size, continued…
• An effect size index is a converted effect size, a
standardized measure of change, so that change
scores across different outcome measures can be
compared.
• ES is often displayed as a correlation coefficient,
r
Portney and Watkins note that considerations of ES
vary based on the interpreting clinician, but review
Cohen’s suggestions of scores <0.4 as small
(treatment had a small effect), 0.5 as moderate,
and >0.8 as large
(Portney and Watkins, 2000)
 39

Measurement Validity: Statistics

Effect Size versus Minimal Clinically Important
Difference
• In addition to numerical data revealed and
statistical significance, the clinician should also
consider what amount of change is clinically
meaningful, such as, how great a gain in strength
or endurance will result in a change in function?
This is often referred to as the minimal clinically
important difference (MCID).

(Jaeschke et al 1989)
 40

Measurement Reliability: Statistics

Reliability examines a measurement’s consistency

and freedom from error
• Can be thought of as reproducibility or
dependability
• Estimate of how observed scores vary from the
actual scores

(Portney and Watkins, 2000)

 41

Measurement Reliability: Statistics

Reliability Coefficient
• Ratio of reliability (many different types with
various symbol representation)
• Range between 0.00 to 1.00;
• 0.00 = no reliability;
• 1.00 = perfect reliability
• Reflection of variance, a measure of the
differences among scores within a sample

(Portney and Watkins, 2000)

 42

Measurement Reliability: Statistics

Correlation
• Comparison of the degree of association
between two variables or sets of data
• Used as a basis for many reliability coefficients

(Portney and Watkins, 2000)

 43

Measurement Reliability: Statistics

Test-Retest Reliability
Examines the consistency of the results of
repeated test administration
• Traditional Analysis
• Pearson product-moment coefficient of
correlation (for interval or ratio data)
• Spearman rho (for ordinal data)
• Current, sometimes considered preferred,
Analysis
• Intraclass correlation coefficient

(Portney and Watkins, 2000)

 44

Measurement Reliability: Statistics

Rater Reliability
• Intrarater reliability
• Reliability of data collection from one
individual over two or more trials
• Interrater reliability
• Reliability of data collection between two or
more raters

(Portney and Watkins, 2000)