Quantitative Data

Sherlock
Decoding Clues for Valid Instrumentation

Donald O. Caga
Mathematics Instructor, BISU Candijay
 What is a ● It is a tool used to collect,
research measure, and analyze
data needed to answer a
instrument? research question.
 Characteristics of a Good Research Instrument
Valid and Reliable
Measuring what should be measured and consistency
of results

Based on a Conceptual Framework

The researcher understands the connection of
the variables in the study

Able to Test Hypothesis

Answer proposed research questions
under investigation

Bias-free and Contextualized

Maximized fairness and appropriate for the context,
culture, and diversity
 CONTENT
VALIDITY
- degree to which
FACE VALIDITY items reflect the
- Appearance content universe to
of the questionnaire which the instrument
will be generalized
(Straub et al., 2004).

4 Types of CONSTRUCT
VALIDITY
CRITERION
VALIDITY
Validity - The
operationalization of
- Extent to which a
measure is related to
the instrument an outcome
(convergent & (predictive,
discriminant) concurrent, &
postdictive)
Validity Component Techique Suggested

Face Validity Post hoc theory, expert assessment of items; Cohen’s Kappa
Index (CKI)
Content Validity Literature review; expert opinion; CVRs; Q-sorting

Construct Discriminant Validity MTMM; PCA; EFA/CFA; PLS AVE; Q-sorting

Construct Convergent Validity MTMM; PCA; EFA/CFA; Q-sorting

Criterion Predictive Validity Regression Analysis, Discriminant Analysis

Criterion Concurrent Validity Correlation Analysis

Criterion Postdictive Validity Correlation Analysis

Reliability Internal Consistency Cronbach’s alpha; correlations; SEM reliability coefficients

Comparison of Validities that are Undertaken in Research (Straub et al., 2004;

Netemeyer et al., 2003; Viswanathan, 2005; Engellant et al., 2016)
EFA Validity

Cronbach’s Reliability
Alpha

CFA Validity

Raykov’s Reliability
Rho
What is Factor Analysis?
 2 Types of Factor Analysis

EFA (Exploratory) CFA (Confirmatory)

used to confirm what is

not based on any prior
expected on the basic or
theory
pre-established theory

assumes that each factor

assumes that any
is associated with a
variable may be
specific subset of
associated with any factor
measured variables
Implementing EFA/CFA in R

Ranges from 0 to 1
Zero means no correlation
among the items at all. 1
means perfect correlation.
Benchmark Value
Analysts frequently use 0.7
as the benchmark value.
Values near 0.7 are
minimally acceptable but not
ideal.
Most commonly
used reliability test
Does not measure
validity
It measures consistency of
the items but not the
correctness of the concept.
Too high alpha
Cronbach can be a red flag
Alpha Extremely high values
can indicate that the
questions are redundant.
Increases as the
number of items
increases
 What if the internal consistency index is low?
Item Analysis Item Removal or Revision
Identify problematic items in
the questionnaire.
01 02 Remove or revise an item
or some items to improve
their relevance to the
construct being measured.

Reasses internal Pilot Study

consistency Test the revised
Reasses the internal consistency 04 03 questionnaire to a sample
of participants.
using techniques like Cronbach
alpha, SEM, etc.
Computing Cronbach Alpha in R
Computing Cronbach Alpha in R