[SH – PR2 / Practical Research 2]

1
[Validity of Measurement and Reliability]

Validity of Measurement and Reliability

Data about human beings can be obtained from any number of sources,
including observation of individuals or groups in either laboratory or real-
world settings, historical data that have been collected for other purposes,
and data collected by asking questions directly of individuals themselves
regarding their opinions, attitudes, feelings, or past reactions. Data collected
directly from individuals are typically gathered using various paper-and-
pencil measurement instruments (or their electronic equivalents). Some of
the more frequently used instruments of this type include surveys and
questionnaires, personality tests, or even tests of mental ability. As anyone
who has ever participated in a survey or taken a test knows, however, some
data collection instruments are better than others. To be useful for scientific
research, data collection instruments need to have two characteristics: They
must be both reliable and valid.
At the end of the topic the students should be able to:
1. Discuss the types of validity;
2. Discuss the adequacy; and
3. Discuss the objectivity

Validity of Measurement
Validity means that a tool measures what it sets out to measure – for
example, that a pain assessment tool measures pain intensity rather than
anxiety. There are several measures of validity that provide evidence of the
quality of a study. Internal and external validity relate to the overall study
design. Internal validity relates to the extent to which the design of a
research study is a good test of the hypothesis or is appropriate for the
research question (Carter and Porter 2000). External validity, meanwhile,
relates to whether or not research findings can be generalized beyond the
immediate study sample and setting. Measures that are used to assess the
validity of data collection tools are summarized in the box below (these are
adapted from Knapp (1998), Carter and Porter (2000), Peat (2002)). When
reading a research paper, you need to look for evidence that the researcher
has addressed the validity and reliability of the data collection tools. This is
usually discussed in the methodology section of the paper
Measures to assess the validity of data collection tools
Content Validity
Whether a tool appears to others to be measuring what it says it does. Face
validity is a simple form of content validity – the researcher asks a few
people to check the tool covers all areas. A more rigorous way to assess
Course Module
 content validity is to ask recognized experts in the area to give their opinion
on the validity of the tool Concurrent or predictive validity are both
measures of criterion validity.
Criterion Validity
Concurrent validity uses an already existing and well-accepted measure
against which the new measure can be compared – for example, if you were
developing a new pain assessment tool you would compare the ratings
obtained from the new tools with those obtained using a previously validated
tool. Predictive validity measures the extent to which a tool can predict a
future event of interest – for example, does a tool developed to measure the
risk of pressure sores in children in hospital in fact identify the children at
risk? Criterion validity is usually measured using a correlation coefficient –
when the correlation is high, the tool can be considered valid
Construct Validity
This tests the link between a measure and the underlying theory. If a test has
construct validity, you would expect to see a reasonable correlation with
tests measuring related areas. Evidence of construct validity can be provided
by comparing the results obtained with the results obtained using other
tests, other (related) characteristics of the individual or factors in the
individual’s environment which would be expected to affect test
performance. Construct validity is usually measured using a correlation
coefficient – when the correlation is high, the tool can be considered valid.
Reliability
Reliability can be thought of as consistency. Does the instrument consistently
measure what it is intended to measure? It is not possible to calculate
reliability; however, there are four general estimators that you may
encounter in reading research:
Inter-Rater/Observer Reliability: The degree to which different
raters/observers give consistent answers or estimates.
Test-Retest Reliability: The consistency of a measure evaluated over time.
Parallel-Forms Reliability: The reliability of two tests constructed the same
way, from the same content.
Internal Consistency Reliability: The consistency of results across items,
often measured with Cronbach’s Alpha.
Relating Reliability and Validity
Reliability is directly related to the validity of the measure. There are several
important principles. First, a test can be considered reliable, but not valid.
Consider the SAT, used as a predictor of success in college. It is a reliable test
(high scores relate to high GPA), though only a moderately valid indicator of
success (due to the lack of structured environment – class attendance,
parent-regulated study, and sleeping habits – each holistically related to
success).
Second, validity is more important than reliability. Using the above example,
college admissions may consider the SAT a reliable test, but not necessarily a
valid measure of other quantities colleges seek, such as leadership capability,
 [SH – PR2 / Practical Research 2]
3
[Validity of Measurement and Reliability]

altruism, and civic involvement. The combination of these aspects, alongside

the SAT, is a more valid measure of the applicant’s potential for graduation,
later social involvement, and generosity (alumni giving) toward the alma
mater.
Finally, the most useful instrument is both valid and reliable. Proponents of
the SAT argue that it is both. It is a moderately reliable predictor of future
success and a moderately valid measure of a student’s knowledge in
Mathematics, Critical Reading, and Writing.

Glossary
Reliability - s the degree to which an assessment tool produces stable and
consistent results.
Validity - is the extent to which a concept, conclusion or measurement is
well-founded and corresponds accurately to the real world.

References
Baraceros, Esther L. (2017), Practical Research 2, Rex Book Store, Inc., First
Edition
Baraceros, Esther L. (2017), Practical Research 1, Rex Book Store, Inc., First
Edition
Sarno, Emerlita G. (2010), Tips and Techniques in Writing Research, Rex
Book Store, Inc.

Course Module