LESSON 3

RELIABILITY AND VALIDITY

Reliability - the consistency of scores from assessment instrument.

Validity - the accuracy of an assessment - whether it measure what it

supposed to measure.

Correlation Coefficient - a key guide for reliability and validity.

Validity Coefficient - describe the relationship between a set of scores

obtained by the same group on some criterion measure.

Reliability Coefficient - describe the relationship between the set of scores

of the same individuals on the same instrument at two different times.
 Pearson Product Moment Correlation
•r  = n(∑xy) – (∑x)(∑y)
∑) – ] [ n(∑) – ]

Where:
r – correlation between x and y (Pearson r)
∑x – sum of test x
∑y – sum of test y
∑xy – sum of the product of x and y
n – number of cases
 Spearman Rho Rank Correlation
•  6∑
r=1- -n

Where:
r - Spearman rho rank coefficient
∑ - sum of the squared differences between ranks
n – total number of cases
Note: for the purpose of the discussion, test/s that gives
reliability and validity coefficient of 0.70 or higher will
be interpreted as reliable and valid respectively.
THE TYPES OF RELIABILITY
 TEST – RETEST RELIABILITY
Involves administering the same test twice to the
same group after a certain time interval has elapsed.

The correlation coefficients can be affected by the

length of time that elapses between the two
administrations of test. The longer the time interval,
the lower the coefficient is likely to be.
 EQUVALENT FORMS RELIABILITY
Also called alternate or parallel

A two different tests but has the same equivalent

which administered at the same time. Though the
questions differ from each other but has the same
coverage.
 EQUIVALENT-FORMS AND TEST-
RETEST RELIABILITY
Giving two different sets of test with the same
coverage and has time interval between its two
administrations.
 EXAMPLE TYPE OF TEST
Test – Retest Reliability Pre-Test and Post-Test
Set A and Set B
Equivalent – Forms
Reliability
Pre-Test (Set A) and
Post-Test (Set B)
Equivalent – Forms and Test
– Retest Reliability
 EXAMPLE
Ten sophomore students in College Algebra were used as sample to test
the reliability of an achievement test. The table below shows the
students’ scores. Use Pearson Product Moment Correlation to find the
two tests reliability.
PRETEST POSTTEST
75 93
81 86
80 78
79 85
65 75
82 89
80 95
69 76
81 91
75 82
 Get the value of xy
PRETEST (x) POSTTEST (y) xy Solution
75 93 6975 75 x 93
81 86 6966 81 x 86
80 78 6240 80 x 78
79 85 6715 79 x 85
65 75 4875 65 x 75
82 89 7298 82 x 89
80 95 7600 80 x 95
69 76 5244 69 x 76
81 91 7371 81 x 91
75 82 6150 75 x 82
   Get the value of
PRETEST (x) POSTTEST (y) xy Solution
75 93 6975 5625 or 75 x 75
81 86 6966 6561 or 81 x 81
80 78 6240 6400 or 80 x 80
79 85 6715 6241 or 79 x 79
65 75 4875 4225 or 65 x 65
82 89 7298 6724 or 82 x 82
80 95 7600 6400 or 80 x 80
69 76 5244 4761 or 69 x 69
81 91 7371 6561 or 81 x 81
75 82 6150 5625 or 75 x 75
   Get the value of
PRETEST POSTTEST xy Solution
(x) (y) or 93 x 93
8649
75
75 93
93 6975
6975 5625
5625
7396 or 86 x 86
81
81 86
86 6966
6966 6561
6561
6084 or 78 x 78
80 78 6240 6400
80 78 6240 6400
79 85 6715 6241 7225 or 85 x 85
79 85 6715 6241
65 75 4875 4225 5625 or 75 x 75
65 75 4875 4225
82 89 7298 6724 7921 or 89 x 89
82 89 7298 6724
80 95 7600 6400 9025 or 95 x 95
80 95 7600 6400
69 76 5244 4761
69 76 5244 4761 5776 or 76 x 76
81 91 7371 6561
81 91 7371 6561 8281 or 91 x 91
75 82 6150 5625
75 82 6150 5625 6724 or 82 x 82
 Get the value of all summations
PRETEST (x) POSTTEST (y) xy ∑x = 75+81+80+79+65+82+
80+69+81+75
75 93 6975 5625 8649
∑y
∑y == 93+86+78+85+75+89+
93+86+78+85+75+89+
81 86 6966 6561 7396 95+76+91+82
95+76+91+82
80 78 6240 6400 6084 ∑xy = 6975+6966+6240+6715+
∑xy = 4875+7298+7600+5244+
6975+6966+6240+6715+
79 85 6715 6241 7225 4875+7298+7600+5244+
7371+6150
65 75 4875 4225 5625 7371+6150
∑ = 5625+6561+6400+6241+
82 89 7298 6724 7921 4225+6724+6500+4761+
6561+5625
80 95 7600 6400 9025
∑ = 8549+7396+6084+7225+
69 76 5244 4761 5776 5625+7921+9025+5776+
8281+6724
81 91 7371 6561 8281
75 82 6150 5625 6724

∑x = 767 ∑y = 850 ∑xy = 65434 ∑ = 59123 ∑ = 72706

 SUBSTITUTING THE VALUES
∑x = 767 ∑y = 850 ∑xy = 65434 ∑ = 59123 ∑ = 72706

•r =
  n(∑xy) – (∑x)(∑y)
∑) – ] [ n(∑) – ]

r= 10(65434) – (767)(850)
59123) – ] [ 10(72706) – ]

r= 10(65434) – (651950)
10(59123) – (588289)] [ 10(72706) – (722500)]
 SUBSTITUTING THE VALUES
•r =
  (654340) – (651950)
(591230) – (588289)] [ (727060) – (722500)]

r= 2390
(2941) (4560)

r= 2390 = 65.26 or 65% (not reliable)

3662.10
 EXAMPLE
Ten sophomore students in College Algebra were used as sample to test
the reliability of an achievement test. The table below shows the
students’ scores. Use Spearman Rho Rank Correlation to find the two
tests reliability.
PRETEST POSTTEST
75 93
81 86
80 78
79 85
65 75
82 89
80 95
69 76
81 91
75 82
  Get the value of
PRETEST (x) SCORE HOW SOLUTION
MANY
75 7.5
82 1 1
81 2.5
81 2 (2+3)/2
80 4.5
80 2 (4+5)/2
79 6 79 1 6
65 10 75 2 (7+8)/2
82 1 69 1 9
80 4.5 65 1 10
69 9
81 2.5
75 7.5
  Get the value of
POSTTEST (y) SCORE HOW SOLUTION
MANY
93 2
95 1 None
86 5
93 1 None
78 8
91 1 None
85 6
89 1 None
75 10
86 1 None
89 4 85 1 None
95 1 82 1 None
76 9 78 1 None
91 3 76 1 None
82 7 75 1 None
 Get the value of d
PRETEST (x) POSTTEST (y) d SOLUTION
75 93 7.5 2 5.5 7.5-2
81 86 2.5 5 -2.5 2.5-5
80 78 4.5 8 -3.5 4.5-8
79 85 6 6 0 6-6
65 75 10 10 0 10-10
82 89 1 4 -3 1-4
80 95 4.5 1 3.5 4.5-1
69
69 76
76 9
9 9
9 0 9-9
81
81 91
91 2.5
2.5 3
3 -0.5 2.5-3
75
75 82
82 7.5
7.5 7
7 0.5 7.5-7
   Get the value of
PRETEST (x) POSTTEST (y) d SOLUTION
75 93 7.5 2 5.5 30.25 or 5.5 x 5.5
81 86 2.5 5 -2.5 6.25 or -2.5 x -2.5
80 78 4.5 8 -3.5 12.25 or -3.5 x -3.5
79 85 6 6 0 0
65 75 10 10 0 0
82 89 1 4 -3 9 or -3 x -3
80 95 4.5 1 3.5 12.25 or 3.5 x 3.5
69
69 76
76 9
9 9
9 0
0 0
81
81 91
91 2.5
2.5 3
3 -0.5
-0.5 0.25 or -0.5 x -0.5
75
75 82
82 7.5
7.5 7
7 0.5
0.5 0.25 or 0.5 x 0.5
 Get the summation then substitute
30.25 •  6∑
6.25 r=1- -n
12.25 SOLUTION
0 30.25+6.25+12.25+0+0+9+
12.25+0+0.25+0.25
0 6 (70.5)
9 r = 1 - - 10
12.25
0
0.25 423
0.25
r = 1 – 1000 - 10
∑= 70.5
 Substitution
423
r=1– 990

r = 1 – 0.43

r = 0.57 or 57% (not reliable)

 Internal Consistency by Split – Half
Reliability
• It involves scoring two halves of a test separately for each
person.
Usually use odd items versus even items.
Spearman – Brown Prophecy Formula

Formula: Where:
= 2 = reliability of odd and even items
1+ = reliability of the whole test
 EXAMPLE
The data below shows students’ scores in a 200-item examination.
Conduct Split-Half Reliability Test.
EVEN ITEMS ODD ITEMS
69 76
81 91
75 80
88 96
86 81
78 80
79 85
65 75
82 89
80 95
 Get the value of xy
EVEN ITEMS (x) ODD ITEMS (y) xy Solution
69 76 5244 69 x 76
81 91 7371 81 x 91
75 80 6000 75 x 80
88 96 8448 88 x 96
86 81 6966 86 x x81
78 80 6240 78 x 80
79 85 6715 79 x 85
65 75 4875 65 x 75
82 89 7298 82 x 89
80 95 7600 80 x 95
   Get the value of
EVEN ITEMS (x) ODD ITEMS (y) xy Solution
69 76 5244 4761
81 91 7371 6561 or 81 x 81
75 80 6000 5625 or 75 x 75
88 96 8448 7744 or 88 x 88
86 81 6966 7396 or 86 x 86
78 80 6240 6084 or 78 x 78
79 85 6715 6241 or 79 x 79
65 75 4875 4225 or 65 x 65
82 89 7298 6724 or 82 x 82
80 95 7600 6400 or 80 x 80
   Get the value of
EVEN ITEMS (x) ODD ITEMS (y) xy Solution
69 76 5244 4761 5776
81 91 7371 6561 8281 or 91 x 91
75 80 6000 5625 6400 or 80 x 80
88 96 8448 7744 9216 or 96 x 96
86 81 6966 7396 6561 or 81 x 81
78 80 6240 6084 6400 or 80 x 80
79
79 85
85 6715
6715 6241
6241 7225 or 85 x 85
65
65 75
75 4875
4875 4225
4225 5625 or 75 x 75
82
82 89
89 7298
7298 6724
6724 7921 or 89 x 89
80 95 7600 6400 9025 or 95 x 95
 Get the summation
EVEN ITEMS (x) ODD ITEMS (y) xy ∑x = 69+811+75+88+86+
69 76 5244 4761 5776 78+79+65+82+80
81 91 7371 6561 8281 ∑y = 76+91+80+96+81+
80+85+75+89+95
75 80 6000 5625 6400
∑xy
∑xy == 52244+7371+6000+
52244+7371+6000+
88 96 8448 7744 9216 8448+6966+6240+
8448+6966+6240+
86 81 6966 7396 6561 6715+4875+7298+
6715+4875+7298+
7600
7600
78 80 6240 6084 6400
∑ = 4761+6561+5626+
79 85 6715 6241 7225 7744+7396+6084+
6241+4225+6724+
65 75 4875 4225 5625 6400
82 89 7298 6724 7921 ∑ = 5776+8281+6400+
80 95 7600 6400 9025 9216+6561+6400+
7225+5625+7921+
9025

∑x = 783 ∑y = 848 ∑xy = 66757 ∑ = 61761 ∑ = 72430

 Substitution
∑x = 783 ∑y = 848 ∑xy = 66757 ∑ = 61761 ∑ = 72430

•=  10(∑xy) – (∑x)(∑y)

∑) – ] [ n(∑) – ]

= 10(66757) – (783)(848)
61761) – ] [ 10(72430) – ]

= 10(66757) – (663984)
10(61761) – (613089)] [ 10(72430) – (719104)]
 Substitution

•=  (667570) – (663984)

(617610) – (613089)] [ (724300) – (719104)]

= 3586
(4521)] [ (5196)]

= 3586
(23491116)
 Substitution
•=  3586
4846.76

= 0.7398 or 0.74 (Reliability for ½ of the test items)

= 2
1+
 Substitution
•=  0.74

= 2
1+

= 1.48 = 0.8505 or 85% (reliable)

1.74
  KUDER – RICHARDSON FORMULA
• It determines the reliability of right versus wrong items.
Can be compute using KR29 formula:
r = ()()
where:
r = reliability coefficient
n = number of items
v = variance of students’ scores which is equal to
pq = the product of proportion of pass and fail for each item
 EXAMPLE
Suppose that 10 students were used as samples to test a reliability of a
10-item test. Results were recorded below. Conduct Kuder Richardson
reliability test.
ITEM STUDENT
1 2 3 4 5 6 7 8 9 10
1 Correct Correct Correct Wrong Correct Correct Wrong Correct Correct Correct
2 Wrong Wrong Wrong Wrong Wrong Correct Correct Correct Correct Correct
3 Correct Wrong Correct Correct Correct Wrong Correct Correct Correct Correct
4 Correct Correct Correct Correct Correct Correct Wrong Wrong Correct Correct
5 Wrong Correct Correct Correct Correct Correct Correct Correct Correct Correct
6 Wrong Correct Wrong Correct Correct Correct Correct Correct Correct Correct
7 Wrong Wrong Correct Wrong Correct Correct Correct Wrong Correct Correct
8 Correct Correct Correct Correct Wrong Wrong Correct Correct Wrong Correct
9 Correct Wrong Wrong Correct Correct Correct Correct Correct Correct Correct
10 Correct Correct Wrong Wrong Correct Correct Wrong Correct Correct Correct
 Let 1 be the correct item and 0 be the wrong
item
ITEM STUDENT
1 2 3 4 5 6 7 8 9 10
1 1 1 1 0 1 1 0 1 1 1
2 0 0 0 0 0 1 1 1 1 1
3 1 0 1 1 1 0 1 1 1 1
4 1 1 1 1 1 1 0 0 1 1
5 0 1 1 1 1 1 1 1 1 1
6 0 1 0 1 1 1 1 1 1 1
7 0 0 1 0 1 1 1 0 1 1
8 1 1 1 1 0 0 1 1 0 1
9 1 0 0 1 1 1 1 1 1 1
10 1 1 0 0 1 1 0 1 1 1
f is the total correct answer of the students
ITEM STUDENT
1 2 3 4 5 6 7 8 9 10 f
1 1 1 1 0 1 1 0 1 1 1 8

2 0 0 0 0 0 1 1 1 1 1 5
3 1 0 1 1 1 0 1 1 1 1 8

4 1 1 1 1 1 1 0 0 1 1 8

5 0 1 1 1 1 1 1 1 1 1 9
6 0 1 0 1 1 1 1 1 1 1 8
7 0 0 1 0 1 1 1 0 1 1 6
8 1 1 1 1 0 0 1 1 0 1 7

9 1 0 0 1 1 1 1 1 1 1 8

10 1 1 0 0 1 1 0 1 1 1 7
Divide each f by the number of the students to
obtain p
ITEM STUDENT
1 2 3 4 5 6 7 8 9 10 f P
1 1 1 1 0 1 1 0 1 1 1 8 0.8

2 0 0 0 0 0 1 1 1 1 1 5 0.5
3 1 0 1 1 1 0 1 1 1 1 8 0.8

4 1 1 1 1 1 1 0 0 1 1 8 0.8

5 0 1 1 1 1 1 1 1 1 1 9 0.9
6 0 1 0 1 1 1 1 1 1 1 8 0.8
7 0 0 1 0 1 1 1 0 1 1 6 0.6
8 1 1 1 1 0 0 1 1 0 1 7 0.7

9 1 0 0 1 1 1 1 1 1 1 8 0.8

10 1 1 0 0 1 1 0 1 1 1 7 0.7
To get q, subtract each value of p to 1.
ITEM STUDENT
1 2 3 4 5 6 7 8 9 10 f P q
1 1 1 1 0 1 1 0 1 1 1 8 0.8 0.2

2 0 0 0 0 0 1 1 1 1 1 5 0.5 0.5
3 1 0 1 1 1 0 1 1 1 1 8 0.8 0.2

4 1 1 1 1 1 1 0 0 1 1 8 0.8 0.2

5 0 1 1 1 1 1 1 1 1 1 9 0.9 0.1
6 0 1 0 1 1 1 1 1 1 1 8 0.8 0.2
7 0 0 1 0 1 1 1 0 1 1 6 0.6 0.4
8 1 1 1 1 0 0 1 1 0 1 7 0.7 0.3

9 1 0 0 1 1 1 1 1 1 1 8 0.8 0.2

10 1 1 0 0 1 1 0 1 1 1 7 0.7 0.3
 Multiply p and q to obtain pq.
ITEM STUDENT
1 2 3 4 5 6 7 8 9 10 f P q pq
1 1 1 1 0 1 1 0 1 1 1 8 0.8 0.2 0.16

2 0 0 0 0 0 1 1 1 1 1 5 0.5 0.5 0.10

3 1 0 1 1 1 0 1 1 1 1 8 0.8 0.2 0.16

4 1 1 1 1 1 1 0 0 1 1 8 0.8 0.2 0.16

5 0 1 1 1 1 1 1 1 1 1 9 0.9 0.1 0.18

6 0 1 0 1 1 1 1 1 1 1 8 0.8 0.2 0.16
7 0 0 1 0 1 1 1 0 1 1 6 0.6 0.4 0.24
8 1 1 1 1 0 0 1 1 0 1 7 0.7 0.3 0.21

9 1 0 0 1 1 1 1 1 1 1 8 0.8 0.2 0.16

10 1 1 0 0 1 1 0 1 1 1 7 0.7 0.3 0.21

 Add all 1’s in each column. Place the sum at the
bottom and label it Total x
ITEM STUDENT
1 2 3 4 5 6 7 8 9 10 f P q pq
1 1 1 1 0 1 1 0 1 1 1 8 0.8 0.2 0.16

2 0 0 0 0 0 1 1 1 1 1 5 0.5 0.5 0.10

3 1 0 1 1 1 0 1 1 1 1 8 0.8 0.2 0.16

4 1 1 1 1 1 1 0 0 1 1 8 0.8 0.2 0.16

5 0 1 1 1 1 1 1 1 1 1 9 0.9 0.1 0.18

6 0 1 0 1 1 1 1 1 1 1 8 0.8 0.2 0.16
7 0 0 1 0 1 1 1 0 1 1 6 0.6 0.4 0.24
8 1 1 1 1 0 0 1 1 0 1 7 0.7 0.3 0.21

9 1 0 0 1 1 1 1 1 1 1 8 0.8 0.2 0.16

10 1 1 0 0 1 1 0 1 1 1 7 0.7 0.3 0.21

Total x 6 6 6 6 8 8 7 8 9 10
   Add all values in pq column and it will be pq.
ITEM STUDENT
1 2 3 4 5 6 7 8 9 10 f P q pq
1 1 1 1 0 1 1 0 1 1 1 8 0.8 0.2 0.16

2 0 0 0 0 0 1 1 1 1 1 5 0.5 0.5 0.10

3 1 0 1 1 1 0 1 1 1 1 8 0.8 0.2 0.16

4 1 1 1 1 1 1 0 0 1 1 8 0.8 0.2 0.16

5 0 1 1 1 1 1 1 1 1 1 9 0.9 0.1 0.18

6 0 1 0 1 1 1 1 1 1 1 8 0.8 0.2 0.16
7 0 0 1 0 1 1 1 0 1 1 6 0.6 0.4 0.24
8 1 1 1 1 0 0 1 1 0 1 7 0.7 0.3 0.21

9 1 0 0 1 1 1 1 1 1 1 8 0.8 0.2 0.16

10 1 1 0 0 1 1 0 1 1 1 7 0.7 0.3 0.21

Total x 6 6 6 6 8 8 7 8 9 10 1.80
  Find the Mean ()
Total x 6 6 6 6 8 8 7 8 9 10

•=  6+6+6+6+8+8+7+8+9+10
10

= 74
10

= 7.4
  Find the sum of , denoted by ∑
STUDENT x SOLUTION
1 6 -1.4 6 – 7.4
2 6 -1.4 6 – 7.4
3 6 -1.4 6 – 7.4
4 6 -1.4 6 – 7.4
5 8 0.6 8 – 7.4
6 8 0.6 8 – 7.4
7 7 -0.4 7 – 7.4
8 8 0.6 8 – 7.4
9 9 1.6 9 – 7.4
10 10 2.6 10 – 7.4

∑ = 18.4
 Solve for v
∑ = 18.4

•v  =∑
N-1
v = 18.4
10 – 1
v = 18.4
9
v = 2.04
 Find r using KR20 formula
n = 10 ∑pq = 1.80 v = 2.04

•r =  ()()

r = ()()

r = ()()

r = (1.11)(0.12) = 0.1332 or 13% (not reliable)

 SUMMARY FOR TYPES OF
RELIABILITY
FORM OF # OF TIMES # OF DIFFERENT # OF INDIVIDUALS
RELIABILITY INSTRUMENT VERSIONS OF WHO PROVIDE
ADMINISTERED INSTRUMENT INFORMATION
Test - retest Twice at different time One version of the test Each student completes
intervals instrument the test twice
Equivalent - forms Each student administered Two different versions of Each student completes
once test of the same concept the test twice
Equivalent - forms and Twice at different time Two different versions of Each student completes
Test – retest intervals test of the same concept the test twice
Internal consistency Each student administered One version of the test Each student completes
once instrument the test twice
THREE MAIN EVIDENCES OF
VALIDITY
 CONTENT – RELATED EVIDENCE OF
VALIDITY
Refers to the content and format of the instrument.

Content validation is partly a matter of determining if the content that the test
contains is an adequate sample of the domain of content it is supposed to
represent. Therefore, a test that measure the learning outcomes and content is
said to possess face validity.

One way to check if the instrument possess face validity is to use Table of
Specification.

Format of instrument – includes clarity of printing, font size, adequacy of

workspace, appropriateness of language, clarity of directions and so on.
 CRITERION – RELATED EVIDENCE OF
VALIDITY
Refers to the relationship between the scores obtained using the test
instrument and scores obtained using one or more test or other
instruments (often called as criterion).

Criterion – is a second test or other instrument presumed to measure

the same content or skills.
TWO FORMS OF CRITERION RELATED
VALIDITY
 PREDICTIVE VALIDITY
Allowing time interval to elapse between the
administration of the instrument and obtaining the
criterion scores.
 CONCURRENT VALIDITY
Instrument data and criterion are gathered at
nearly the same time then the results are
compared.
 CONSTRUCT – RELATED EVIDENCE OF
VALIDITY
Refers to measuring of psychological construct or
characteristic by an instrument.

It is the broadest of the three categories of evidence for

validity that we are considering. There is no single piece of
evidence that satisfies construct – related validity.
END OF LESSON 3