lOMoARcPSD|13649008

FOUR - Essentials OF Reliability

Psychological Assessment (National University (Philippines))

Studocu is not sponsored or endorsed by any college or university

Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

FOUR
ESSENTIALS OF
RELIABILITY

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

LEARNING OBJECTIVES
" Understand the essence of reliability.
" Discuss the diûerent types of reliability and
its implications.

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

WHAT IS RELIABILITY?

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

RELIABILITY
" ÒTrustworthinessÓ
" The consistency and stability of scores over
time, items or other test conditions.
" A quality of test scores that suggests they
are suûciently consistent and free from
measurement error to be useful.

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

RELIABILITY
¥ Error Variance : diûerence in scores resulting
from conditions that are irrelevant to the
purpose of the test

NO TEST IS PERFECTLY RELIABLE

INSTRUMENT

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

VARIABLES WITH ZERO

CORRELATION HAS A
RELATIONSHIP.
YES OR NO?

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

CORRELATION COEFFICIENT
" Expresses the degree of relationship between
two scores.
" Zero correlation indicates the total absence of a
relationship
¥ Pearson Product-Moment Correlation
Coeûcient
" accounts for individualÕs position in the
group (

and the amount of deviation from the mean

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

CORRELATION COEFFICIENT
" Statistical signiÞcance: Whether Þndings in the sample
can be generalized to the population
" SigniÞcant at the .01 level: there is only about 1 out of
100 chance that the Þndings in the sample is wrong (i.e.
only 1 in 100 chance that the correlation is actually 0).
" SigniÞcance level: risk of error weÕre willing to take in
drawing conclusions from our data
" ConÞdence interval: range of score under which the
true score might fall given a speciÞed level of
conÞdence(
"

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

CORRELATION COEFFICIENT
" Reliability coeûcient: Use of correlation
coeûcient for psychometric properties
" Level of acceptable correlation coeûcient =
.80 - .90.
"

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

TYPES OF RELIABILITY

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

TEST-RETEST RELIABILITY
" Repeat same test on the same person on another occasion
" Test for correlation between scores on the two separate testing occasions
" Source of error variance are ßuctuations in performance between the two
testing occasions
" Shows how test can be generalized across situations
" Higher reliability, lower susceptibility to random changes
" Need to specify length of interval
" Interval rarely exceeds 6 months
" Disadvantage: practice eûect
" Can only be applied to tests in which performance is not aûected by
repetition (e.g. sensorimotor, motor) (

"

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

ALTERNATE-FORM RELIABILITY
(PARALLEL FORM)
" Same person is tested with one form on one occasion
and an alternate, equivalent form on another occasion
" Test for correlation of scores on the two forms
" Measure of both temporal stability and consistency of
responses to tw diûerent item samples (

" Source of error variance: content sampling (to what

extent does performance depend on speciÞc items or
arrangement of the test?) (

"

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

ALTERNATE-FORM RELIABILITY
(PARALLEL FORM)
" Parallel Forms must be:
" Be independently constructed;
" Items should be expressed in the same form;
" Same type of content;
" Equivalent range and level of diûculty;
" Instructions, time limits, and sample items
must be equivalent
"

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

ALTERNATE-FORM RELIABILITY
(PARALLEL FORM)
" Disadvantage: reduce but does not
completely eliminate practice eûect
" Questionable: degree of change in the test
due to repetition (e.g. insight tasks)

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

SPLIT-HALF RELIABILITY
" Two scores are obtained by dividing it into equivalent
halves
" Source of error variance: content sampling
" Test for coeûcient of internal consistency (

" Single administration of a single form

" Longer test = more reliable
" Spearman-Brown formula: for estimating the eûect of
shortening or lengthening the test (

" Used because this type of reliability only technically

computes for the reliability of half the test

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

INTER-ITEM CONSISTENCT (KUDER-RICHARDSON

RELIABILITY AND COEFFICIENT ALPHA)
" Single administration of a single form (

" Consistency of all items in the test

" Source of error variance:
" Content sampling
" Heterogeneity of behavior
" More homogenous items, more consistency
" However, is a homogenous test appropriate for a heterogeneous
psychological construct? Is the
(

criterion being predicted homogenous or heterogeneous? (

" Unless items are highly homogenous, the KR coeûcient will be lower
" Used for tests with no wrong or right answers (

"

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

SCORER RELIABILITY
" Factors excluded from error variance:
" True variance (remains in scores)
" Irrelevant factors that can be controlled experimentally
" Correlate results obtained by two separate scorers

STANDARD ERROR OF MEASUREMENT

" Also expresses reliability
" Interval between which the true score may lie (obtained
score ± 1 SEM) (

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)
 lOMoARcPSD|13649008

(c) E.A.Dee
Downloaded by Redgie Gabin (redgabin7@gmail.com)