INTERNAL VALIDITY

CHAPTER 9
REPORTER: NAWAL B. SADIC
 Internal validity is the confidence that
we can place in the cause and effect
relationship in a study.
WHAT IS
INTERNAL It should unambiguous as to what is
VALIDITY? mean to something else:
• Number of factor (age or ability of the subjects),
the conditions under which the study is conducted,
or type of materials used.
 Subject
SUBJECT Characteristics

CHARACTERISTICS
DEFINITION
The selection of people for a study may result in the individuals or groups differing from
one another unintended ways.

EXAMPLE
Why math students often lose from pulling rope competition?
Subject characteristics differ from the individual’s nutrient consumption, weight and height.

REMEDY
In a particular study, the researcher must decide, based on previous research or
experience, which variables are most likely to create problems, and do his or her best to
prevent or minimize their effects.
 Subject

MORTALITY
Characteristics

Mortality

DEFINITION
The loss of subjects in a study due to attrition, withdrawal, or low participation
rates may introduce bias and affect the outcome of a study.

EXAMPLE
Firstly, there is 100 participants, a week later there are only 88, then where 22 students
go? *maybe they got illness, dying, absent, etc.

REMEDY
This is done by exploring the reasons for such loss and then offering an argument as to
why these reasons are not relevant to the particular study at hand.
Provide evidence that the subjects ost were similar to those remaining on pertinent
characteristics such as age, gender, ethnicity, pretest score, or other variables that
presumably might be related to the study outcomes.
 Subject

LOCATION
Characteristics

Mortality

DEFINITION Location
The particular locations in which data are collected, or in which an
intervention is carried out, may create alternative explanations for results.

EXAMPLE
Doing an interview with students in the classroom and students’ lounge will
give different result.

REMEDY
The best method of control for a location threat is to hold location constant.
 Subject

INSTRUMENTATION
Characteristics

Mortality

DEFINITION Location
The ways in which instruments are used may constitute
an internal validity threat. Instrumentation

Possible instrumentation threats include:

Instrument Decay

Data Collector Characteristics

Data Collector Bias

 Subject

INSTRUMENTATION
Characteristics

Mortality
INSTRUMENT DECAY
DEFINITION Location

Instrumentation can create problems if the nature of this instrument (including the
scoring procedure) is changed in some way or another. Instrumentation

EXAMPLE
Checking a bunch of questionnaires, firstly the researcher is firing up to do checking,
later on he gets tired and messily checking.

REMEDY
To schedule data collection and/or scoring so as to minimize changes of the
instruments or scoring procedures.
 Subject

INSTRUMENTATION
Characteristics

Mortality
DATA COLLECTOR CHARACTERISTICS
DEFINITION Location

Gender, age, ethnicity, language patterns, or other characteristics of the individual

who collect the data in a study may affect the nature of the data obtain. Instrumentation

EXAMPLE
A researcher wants to know the society opinion towards police’s duties, in case the
researcher dressed up like a police. Therefore, people tend to give a positive opinion.

REMEDY
• Use the same data collector(s) throughout
• Analyze data separately for each collector
• Ensure that each collector is used equally with all groups
 Subject

INSTRUMENTATION
Characteristics

Mortality
DATA COLLECTOR BIAS
DEFINITION Location

It is unconsciously distort the data in such a way as to make certain outcomes

Instrumentation
(such as support for the hypothesis) more likely by the data collector.

EXAMPLES
An interviewer unconsciously smiles at certain answers to certain questions
during an interview.

REMEDY
• To standardize all procedures, which usually requires some sort of training
of data collectors
• To ensure that the data collectors lack the information they would need to
distort results.
 Subject

TESTING
Characteristics

Mortality

DEFINITION Location
The use of a pretest in intervention studies may create a “practice effect” that can
affect the results of a study and/or how participants respond to an intervention.
Instrumentation

EXAMPLE Testing
A researcher incorporates items designed to measure self-esteem and achievement
motivation in the same questionnaire. The respondents may figure out what the
researcher is after and react accordingly.

REMEDY
Elimination of pretest
 Subject

HISTORY
Characteristics

Mortality

DEFINITION Location
A history threat is when an unforeseen or unplanned event occurs
during the course of a study. Instrumentation

EXAMPLE Testing

The participant really nervous during the interview because the

History
participant remembered about the previous interview which
gives ”complicated questions”.

REMEDY
Researchers should continually be alert to any such influences
that may occur during the course of a study.
 Subject

MATURATION
Characteristics

Mortality

DEFINITION Location
Naturally participants will have development both cognitive and physical.
Instrumentation
EXAMPLE
Over the course of a semester, very young students, in particular, will change in may Testing
ways simply because of aging and experience.
History
REMEDY
The best way to control for maturation is to include a well-selected comparison group Maturation
in the study.
 Subject

SUBJECT ATTITUDE
Characteristics

Mortality

DEFINITION Location
The way subjects view a study and their participation in it can be considered
a threat to internal validity. Instrumentation
The positive impact of an intervention is known as the “Hawthorne effect”.
Testing

EXAMPLE
A researcher wants to know the effectiveness of suggestopedia. He played a History
music in class A but not in class B. He said to students in class B that music
cannot played in the class. In this case, the researcher does not treat the Maturation
participants properly.
Subject Attitude
REMEDY
To provide the control or comparison group(s) with a special or novel
treatment comparable to that received by the experimental group. In some
cases, make students believe that the treatment is just a regular part of
instruction – that is, not part of an experiment.
 Subject

REGRESSION
Characteristics

Mortality

DEFINITION Location
A regression threat is possible when change is studied in a group with
extreme low or high performance as determined by a pretest. On average, Instrumentation
the group will score closer to the mean on subsequent testing regardless of
the treatment or intervention.
Testing

EXAMPLE History
First test, student A get 10; but in the second test with the same score; but
different days, he gets 5. Maturation

REMEDY Subject Attitude

Like maturation, the use of an equivalent control or comparison group handles
this threat – and this seems to be understood as reflected in published Regression
research.
 Subject

IMPLEMENTATION
Characteristics

Mortality

DEFINITION Location
The possibility that the experimental group may be treated in
ways that are unintended and not necessarily part of the Instrumentation
method. It can occur because:
Testing
First – assigned to implement different methods so the outcomes
will be different. History
Second – have a personal bias in favor of one method over the
other. Maturation

Subject Attitude

Regression

Implementation
 Subject

IMPLEMENTATION
Characteristics

Mortality

EXAMPLE Location

A group of clients who stutter is given a relatively new method of therapy called Instrumentation
generalization training.
Both client and therapist interact with people in the “real world” as part of the Testing

therapy.
History
After six months of receiving therapy, the fluency of these clients is compared with
that of the group receiving traditional in-the-office therapy. Maturation

Speech therapists who use new methods are likely to be more generally competent
than those working with the comparison group. Subject Attitude

If so, greater improvement differ the generalization group may be due not to the Regression
new method but rather to the skill of the therapist.
Implementation
 Subject

IMPLEMENTATION
Characteristics

Mortality

REMEDY Location

First – evaluate the individuals who implement each method on Instrumentation

pertinent characteristics (such has teaching ability) then
try to equate the treatment groups on those dimension
Testing
(for example, by assigning teachers of equivalent
ability to each group).
History
Second – allow individuals to choose the method they wish to
implement, but this creates the possibility of Maturation
differences' in characteristics.
Subject Attitude

Regression

Implementation
ILLUSTRATION OF THREATS TO
INTERNAL VALIDITY
“Hold on —perhaps
private schools are more likely to expel
the poorer students. So it’s the policy,
not the nature of the school, that
makes the difference.”
“Maybe those attending private
schools have higher academic
aptitude – so it not type of schools
that makes the difference.”
(Subject (Loss of
characteristics) subjects)
“Wait a minute.
Private schools may have more
resources (materials,
technologies,
parent support); that could
account
for the differences instead of
the type of school
organization.”
students have more opportunities to
practice taking their tests. This could
(Location)
The teachers in this fictitious
example are discussing the results
of a study which shows that students
who attend private high schools had
higher achievement (as shown by test
scores) than students who attended
public high schools.
“Private school students
may achieve higher scores, not because
of the type of school, but because they are
exposed to a broader range of experiences.
Their parents are more affluent.”
“Maybe private school
account for their higher
performance.”

“Is it likely that the

tests to assess achievement are
biased in favor of the curricula found in
private schools? Could the procedure
used in testing favor the private school
students (testing conditions,
adherence to instructions)?”
(Instrumentation) (Testing) (History)

“Perhaps it is the status and “Maybe there were a lot

self-esteem associated with attending a of students who scored
private school that motivates these students to really low on the pretest
achieve at a higher level, rather than the type in the private schools.”
of school organization.”
“Maybe private
“Perhaps private schools have more experienced Figure 9.10
school students spend more or dedicated teachers and this is Note: We are not implying that any of these statements
years in high school than those the reason for a difference.Ӡ are necessarily true; our guess is that some are and some
in public schools.”*
are not.
* This seems unlikely.
†If these teacher characteristics are a result of the type
of school, then they do not constitute a threat.

(Maturation) (Attitude (Regression) (Implementation)

of subjects)
TWO POINTS TO EMPHASIZE

• First, these various threats to internal validity can be greatly reduced by

planning.
• Second, such planning often requires collecting additional information before
a study begins (or while it is taking place)
QUESTIONS?

THANK YOU