Professional Documents
Culture Documents
Reliability and Validity: © Louis Cohen, Lawrence Manion & Keith Morrison
Reliability and Validity: © Louis Cohen, Lawrence Manion & Keith Morrison
Defining validity
Validity in quantitative research
Validity in qualitative research
Types of validity
Triangulation
Validity in mixed methods research
Ensuring validity
Reliability
Reliability in quantitative research
Reliability in qualitative research
BASES OF VALIDITY IN
QUANTITATIVE RESEARCH
Controllability
BASES OF VALIDITY IN
QUALITATIVE RESEARCH
Natural
Thick description
Generalizability
Context-freedom
Fragmentation and atomization
Uniqueness
Context-boundedness
Holism
Randomization of samples
Neutrality
Value-ladenness of observations
Objectivity
Observability
Inference
Confirmability
Observable and non-observable
meanings/ intentions
Description, inference, explanation
Etic research
Observations
Emic research
Meanings
Uniqueness
Emergence, unpredictability
BASES OF RELIABILITY IN
QUANTITATIVE RESEARCH
Reliability
BASES OF RELIABILITY IN
QUALITATIVE RESEARCH
Dependability
Demonstrability
Stability and replicability
Parallel forms
Context-freedom
Objectivity
Coverage of domain
Verification of data and analysis
Answering research questions
Meaningfulness to the research
Parsimony
Internal consistency
Generalizability
Inter-rater reliability & triangulation
Accuracy and precision
Neutrality
Consistency
Alternative forms (equivalence)
Split-half and inter-item correlation
Trustworthiness
Stability and replicability
Parallel forms
Context-specificity
Authenticity and confirmability
Comprehensiveness of situation
Honesty and candour
Depth of response
Meaningfulness to respondents
Richness
Credibility
Transferability
Inter-rater reliability and triangulation
Accuracy and comprehensiveness
Multiple interests represented
Consistency
TYPES OF VALIDITY
Catalytic
Concurrent
Consequential
Construct
Content
Criterion-related
Convergent & discriminant
Cross-cultural
Cultural validity
Descriptive
Ecological
Evaluative
External
Face
Internal
Interpretive
Jury
Predictive
Systemic
Theoretical
Concurrent
Construct
Content
Criterion-related
Convergent & discriminant
Cross-cultural
Evaluative
External
Face
Internal
Jury
Predictive
Theoretical
Representation
Legitimation
Sample integration
Inside-outside
Weakness minimization
Sequential
Conversion
Paradigmatic mixing
Commensurability
Multiple validities
Political
Integration (of methods)
HISTORY
MATURATION
TESTING
DIRECTION
OF CAUSALITY
TYPE 1 AND
TYPE 2
ERRORS
INSTRUMENTATION
THREATS TO
VALIDITY AND
RELIABILITY
EXPERIMENTAL
MORTALITY
OPERATIONALIZATION
CONTAMINATION
REACTIVITY
ESTABLISHING VALIDITY IN
QUALITATIVE RESEARCH
ESTABLISHING VALIDITY IN
QUALITATIVE RESEARCH
THREATS TO VALIDITY IN
QUANTITATIVE RESEARCH
History
Maturation
Statistical regression
Testing
Instrumentation
Selection Bias
Experimental mortality
Instrument reactivity
Selection-maturation interaction
Type I and Type II errors
VALIDITY PROBLEMS IN
CROSS-CULTURAL RESEARCH
Failure to operationalize elements of cultures
Whose construction of culture to adopt: emic/etic
False attribution of causality to cultural factors rather than noncultural factors
Directions of causality
Ecological fallacy
Sampling and instrumentation
Convergent and discriminant validity
Response bias and preparation of participants
Language problems
Problems of equivalence (conceptual, psychological, meaning,
instrument, understanding, significance, relevance,
measurement, linguistic)
Selection effects
Setting effects
History effects
Construct effects
RELIABILITY IN QUANTITATIVE
AND QUALITATIVE RESEARCH
Reliability in quantitative research:
consistency (stability), accuracy,
predictability, equivalence, replicability,
concurrence, descriptive and causal
potential.
Reliability in qualitative research:
accuracy, fairness, dependability,
comprehensiveness, respondent validation,
checkability, empathy, uniqueness,
explanatory and descriptive potential,
confirmability.
TYPES OF RELIABILITY IN
QUANTITATIVE RESEARCH
Reliability as stability:
Consistency over time and samples;
Reliability as equivalence:
Equivalent forms of same instrument;
Inter-rater reliability;
Reliability as internal consistency:
Split half reliability (e.g. for test items)
TRIANGULATION
Methodologies
Instruments
Researchers
Time
Location
Theories
Samples
Participants
Data
SPLIT-HALF RELIABILITY
(Spearman-Brown)
Reliability =
2r
1 r
2 ( 0.85 )
1 0.85
1.70
185
= 0.919
RELIABILITY IN QUALITATIVE
RESEARCH
Credibility
Neutrality
Confirmability
Dependability
Consistency
Applicability
Trustworthiness
Transferability
Address:
Stability of observations
Parallel forms
Inter-rater reliability
Respondent validation
IMPROVING RELIABILITY