CHAPTER 6 VALIDITY TYPES

CONTINUED
• Content-Related Validity
Norm-Referenced Reliability & Validity
• Criterion-Related Validity
• Statistical or correlational
• concurrent
• predictive

• Construct-Related Validity

CONTENT-RELATED VALIDITY CRITERION-RELATED VALIDITY

• Based on determining the relationship between a

• Evidence of truthfulness based on logical criterion and other measures used to estimate the
decision making and interpretation criterion

• Also known as face validity and logical validity • Also known as statistical validity and correlational validity
• Concurrent validity (criterion is measured at about the
• Content experts, expert judges, colleagues,
same time as the alternative measure)
and textbook writers can serve as sources to
• Predictive validity (criterion is measured in the future)
validate instrument content

Chapter 6 1
 EXAMPLE OF CONCURRENT VALIDITY EXAMPLES OF PREDICTIVE VALIDITY

VO2max (criterion: oxygen consumption)
• Distance runs (e.g., 1.0 mi [1.6 km], 1.5 mi [2.4
km]; 2km [1.2 km] ; 9 min, 12 min; 20-m [21.9 yd]
shuttle)
• Submaximal (e.g., cycle, treadmill, swimming)
• Nonexercise models (e.g., self-reported physical
activity)
• Heart disease (criterion: heart disease developed in later
life)
• Present diet, exercise behaviors, blood pressure,
family history of heart disease or related health issues
• Success in graduate school (criterion: grade point
average or graduation status)
• Graduate Record Examination scores
• Undergraduate grade point average

CRITERION-RELATED VALIDITY: CONSTRUCT-RELATED VALIDITY

METHODS OF OBTAINING A CRITERION MEASURE

• Use to validate measures that are unobservable yet

• Actual participation
• e.g., golf, archery
exist in theory
• Perform the criterion • Evidence that combines logical (content) and
• known valid criterion (e.g., treadmill performance) statistical validity procedures
• Expert judges
• If, in theory, the construct is valid, then such-and-such
• panel judges
• Tournament participation should occur . . .
• Round robin • Associated with convergent or discriminant evidence
• Known valid test

Chapter 6 2
 CONSTRUCT-RELATED VALIDITY
(CONTINUED)
An example of known group difference method of
construct validation:
• A student who is poor at golf should score poorly on a golf
test; experienced golfers should obtain higher scores, and
members of the golf team should score the best
• Testing difference of means with ANOVA could provide
construct-related evidence for the golf test
FACTORS INFLUENCING VALIDITY
• A test is valid only for a particular group at a particular time for a
particular purpose. There is no such thing as a generally valid test.
For example, a test that is valid for college students may not be for
elementary students. A test for middle-class white girls may be
invalid for other groups. Thus, the culture or unique characteristics
of a group often affect a test's validity for that group.
• The I.Q. test turned out to be somewhat invalid for persons of
color growing up in low-income households. When tests created
specifically for them were administered to middle-class to upper-
class whites, the results were reversed.
• Other factors are unclear directions, strange patterns of
correct answers on test. For motor tests, novelty may influence the
validity, so a practice round should be available.
• Also, reliability and objectivity. Remember that for a test to be
valid, it must be reliable, but a test can be reliable without having
validity.

STANDARD ERROR OF ESTIMATE STANDARD ERRORS

Standard error
SE of Measurement
Standard error of prediction
SEM  Sy * 1  rxx '
SEE  Sy * 1  r 2 xy SE of Estimate

SEE  Sy * 1  r 2 xy

Chapter 6 3
 FIGURE 6.1
DIAGRAM OF VALIDITY AND Interpreting the “r” you obtain
RELIABILITY TERMS

CONCURRENT VALIDITY CONCURRENT VALIDITY

Error

This square represents variance in The orange color represents ERROR or

performance in a skill (e.g., golf) unexplained variance in the criterion (e.g., golf)

Chapter 6 4
 CONCURRENT VALIDITY CONCURRENT VALIDITY
D – it has the MOST error
and requires 4 tests to
be administered

A B C D A B C D
Consider the Concurrent validity of Which test battery would you be LEAST
the above 4 possible skills test batteries likely to use? Why?

CONCURRENT VALIDITY CONCURRENT VALIDITY

C – it has the LEAST error A or B – requires 1 or 2 tests to
but it requires 3 tests to be administered but you
be administered lose some validity

A B C D A B C D
Which test battery would you Which test battery would you
be MOST likely to use? Why? use if you are limited in time?

Chapter 6 5
 INTERPRET THESE INTERPRET THESE
CORRELATIONS CORRELATIONS
Actual Putting Putting Driving Driving Observer Observer Actual Putting Putting Driving Driving Observer Observer
golf score
Criterion Trial 1 Trial 2 Trial 1 Trial 2 1 2 golf score Trial 1 Trial 2 Trial 1 Trial 2 1 2

Actual golf 1.00 Actual golf 1.00

score score
Putting T1 .78 1.00 Putting T1 .78 1.00
What are these?
Putting T2 .74 .83 1.00 What are these? Putting T2 .74 .83 1.00
Reliability
Driving T1 .58 .21 .25 1.00 Driving T1 .58 .21 .25 1.00
coefficients
Driving T2 .68 .25 .30 Concurrent
.70 1.00 Driving T2 .68 .25 .30 .70 1.00

Observer 1 .48 .34 .40

Validity
.43
coefficients
.38 1.00 Observer 1 .48 .34 .40 .43 .38 1.00

Observer 2 .39 .30 .41 .47 .35 .50 1.00 Observer 2 .39 .30 .41 .47 .35 .50 1.00

INTERPRET THESE
CORRELATIONS SCATTERPLOT
Actual Putting Putting Driving Driving Observer Observer
golf score Trial 1 Trial 2 Trial 1 Trial 2 1 2
Two trials of Leg Press
Actual golf 1.00 Line of identity
score
Putting T1 .78 1.00 Prediction line
What is this?
Putting T2 .74 .83 1.00

Driving T1 .58 .21 .25 1.00 Objectivity

Driving T2 .68 .25 .30 .70 1.00 coefficient

Observer 1 .48 .34 .40 .43 .38 1.00

Observer 2 .39 .30 .41 .47 .35 .50 1.00

Chapter 6 6
 ESTIMATING AGREEMENT BETWEEN
CORRELATION MEASURES: BLAND–ALTMAN METHOD

• Bland and Altman (1986) presented a method of

Two trials of Leg Press estimating the agreement between two measures of
the same attribute
• Correlation is limited in its ability to detect the
magnitude and distribution of differences between
two measures
• Bland–Altman approach is based on the absolute
difference between the measures and plotting those
against the averages

SUMMARY

• Reliability, objectivity, and validity are the most

important concepts
• Reliability and validity results are not typically
generalizable
• Results are specific to the group tested, the
environment of testing, and the testing
procedure

Chapter 6 7