Why Scientific Research Methodology is of Value

The scientific method provides an important way to obtain accurate and reliable information.

Ways of Knowing

There are many ways to obtain information, including sensory experience, agreement with others, expert
opinion, logic, and the scientific method.

The scientific method is considered by researchers the most likely way to produce reliable and accurate
knowledge.

The scientific method involves answering questions through systematic and public data collection and analysis.

Types of Research

Some of the most commonly used scientific research methodologies in education are experimental research,
correlational research, causal-comparative research, survey. research, content analysis research, qualitative
research, and historical research.

Experimental research involves manipulating conditions and studying effects.

Correlational research involves studying relationships among variables within a single group, and frequently
suggests the possibility of cause and effect.

Causal-comparative research involves comparing known groups who have had different experiences to
determine possible causes or consequences of group membership.

Survey research involves describing the characteristics of a group by means of such instruments as interview
schedules, questionnaires, and tests.

Ethnographic research concentrates on documenting or portraying the everyday experience of people using
observation and interviews.

Ethnographic research is one form of qualitative research. Another common form of qualitative research
involves case studies.

A case study is a detailed analysis of one or a few individuals.

Content analysis research involves the systematic analysis of communication.

Historical research involves studying some aspect of the past.

Action research is a type of research by practitioners designed to help improve their practice.

Each of the research methodologies described constitutes a different way of inquiring into reality and is thus a
different tool to use in understanding what goes on in education.

General Research Types

Individual research methodologies can be classified into general research types. Descriptive studies describe a
given state of affairs. Associational studies investigate relationships. Intervention studies assess the effects of
a treatment or method on outcomes.

Quantitative and qualitative research methodologies are based on different assumptions; the purpose of
research, the methods used by researchers, the kinds of studies undertaken, the researcher's role, and the
degree to which generalization is possible.

Research Problems

A research problem is the focus of a research investigation.

Research Questions

Many research problems are stated as questions.

The essential characteristic of a researchable question is that there is some sort of information that can be
collected in an attempt to answer the question.

Characteristics of Good Research Questions

Research questions should be feasible -- that is, capable of being investigated with available resources.

Research questions should be clear -- that is, unambiguous.

Research questions should be significant -- that is, worthy of investigation.

Research questions often (although not always) suggest a relationship to be investigated. The term
"relationship," as used in research, refers to a connection or association between two or more characteristics or
qualities.

Defining Terms in Research

Three commonly used ways to clarify ambiguous or unclear terms in a research question involve the use of
constitutive (dictionary-type) definitions; definition by example, and operational definitions.

A constitutive definition uses additional terms to clarify meaning.

An operational definition describes how examples of a term are to be measured or identified.

Meta-analysis attempts to synthesize the results of all the individual studies on a given topic by statistical
means.

Critical Analysis of Research

Critical analysis of research raises basic questions about the assumptions and implications of educational
research.

The Research Process

Almost all research plans include a problem statement, an exploratory question or hypothesis, definitions, a
literature review, a sample of subjects, instrumentation, a description of procedures to be followed, a time
schedule, and a description of intended data analyses.

The Importance
The Value
of of
Studying
a Literature
Relationships
Review

Identifying
A relationships
literature review
among
helps
variables
researchers
enhances
learn understanding.
what others have written about a topic. It also lets
researchers see what have been the results of other, related studies.

Understanding of relationships helps us to explain the nature of the world in which we live.
A detailed literature review is often required of master's ad doctoral students when they design a thesis.

Variables
Type of Sources for a Literature Review

A variable is any characteristic or quality that varies among the members of a particular group.
Researchers need to be familiar with three basic types of sources (general references, primary sources,
and secondary sources) in doing a literature review.
A constant is any characteristic or quality that is the same for all members of a particular group.

General references are sources a researcher consults to locate other sources.

Several kinds of variables are studied in educational research, the most common being independent and
dependent variables.
Primary sources are publications in which researchers report the results of their investigations. Most
primary source material is located in journal articles.
An independent variable is a variable presumed to affect or influence other variables.

Secondary sources refer to publications in which authors describe the work of others.
A dependent (or outcome) variable a variable presumed to be affected by one or more independent variables.

RIE and CIJE are two of the most frequently used general references in educational research.
A quantitative variable is a variable that varies in amount or degree, but not in kind.

Search terms, or "descriptors" are words researchers use to help locate relevant primary sources.
A categorical variable is a variable that varies only in kind, not in degree or amount.

Steps Involved in a Literature Search

An extraneous variable is an independent variable that may have unintended effects on a dependent variable in a
particular study.
The essential steps involved in a review of the literature include: (1) defining the research problem as
precisely as possible; (2) pursuing the secondary sources; (3) selecting and pursuing an appropriate
Hypotheses
general reference; (4) formulating search terms; (5) searching the general references for relevant
primary sources; (6) obtaining and reading the primary sources, and noting and summarizing key points
the sources.as used in research, usually refers to a prediction of results made before a study
The term in
"hypothesis,"
commences.
Ways to Do a Literature Search

Ethical
Principles
A Basic
significant
hypothesis
is one that is likely to lead, if it is supported, to a greater amount of important knowledge
than a nonsignificant
Today, therehypothesis.
are two ways to do a literature search -- manually, using the traditional paper approach, and
means ofofaright
computer.
The most common and frequently used way, however, is to
electronically,
Ethics refers toby
questions
and wrong.
search
online
via
computer.
Stating a research question as a hypothesis has both advantages and disadvantages.

There are a number of ethical principles that all researchers should be aware of and apply to their
There
are five essential
pointsabout
(problem,
hypothesis,
procedures,
findings,
conclusions)
that researchers
investigations.
A directional
hypothesis
is a prediction
the specific
nature
of a relationship
-- for
example, method
A is more
should
record
when
taking
notes
on
a
study.
effective than method B.
The basic ethical question for all researchers to consider is, "Will any physical or psychological harm
Doing a Computer Search
come tohypothesis
anyone asisa aresult
of my that
research?"
A nondirectional
prediction
a relationship exists without specifying its exact nature -- for
example, there will be a difference between method A and method B (without saying which will be more effective).
Computer searches of the literature have a number of advantages -- they are fast, are fairly inexpensive,
All subjects in a research study should be assured that any data collected from or about them will be
provide printouts, and enable researchers to search using more than one descriptor at a time.
held in confidence.

The steps in a manual search are similar to those in a computer search, though computer searches are
The term "deception," as used in research, refers to intentionally misinforming the subjects of a study as
visually the norm.
to some or all aspects of the research topic.

Researching
the World Wide Web (WWW) should be considered, in addition to ERIK and PsycINFO, in
Research
with Children
doing a literature search.

Children as research subjects present problems for researchers that are different from those of adult
Some
of the
Information
on the
Web is classified
into
directories,
which
group
websites
together
subjects.
Children
are more
vulnerable,
have fewer
legal
rights, and
often
do not
understand
the under
similar
categories.
Yahoo!
is
an
example
of
a
directory.
meaning of "informed consent."

To obtain
specific information, search engines should be used, because they search all of the
The Regulation
of more
Research
contents of a website.

Before any research involving human beings can be conducted at an institution that receives Federal
The Literature
Funds, Review
it must Report
be reviewed by an institutional review board (IRB) at the institution.

The
reviewthat
report
of responsibility
an introduction,
body of the
a summary,
the studies
The literature
federal agency
hasconsists
the major
for the
establishing
thereview,
guidelines
for research
researcher's
conclusions,
andisathe
bibliography.
that
involve human
subjects
Department of Health and Human Services.

Samples and Sampling

The term "sampling," as used in research, refers to the process of selecting the individuals who will participate
(e.g., be observed or questioned) in a research study.

A sample is any part of a population of individuals on whom information is obtained. It may, for a variety of
reasons, be different from the sample originally selected.

Samples and Populations

The term "population," as used in research, refers to all the members of a particular group. It is the group of
interest to the researcher, the group to whom the researcher would like to generalize the results of a study.

A target population is the actual population to whom the researcher would like to generalize; the accessible
population is the population to whom the researcher is entitled to generalize.

A representative sample is a sample that is similar to the population on all characteristics.

Random Versus Nonrandom Sampling

Sampling may be either random or nonrandom. Random sampling methods include simple random sampling,
stratified random sampling, and cluster random sampling. Nonrandom sampling methods include systematic
sampling, convenience sampling, and purposive sampling.

Random Sampling Methods

A simple random sample is a sample selected from a population in such a manner that all members of the
population have an equal chance of being selected.

A stratified random sample is a sample selected so that certain characteristics are represented in the sample
in the same proportion as they occur in the population.

A cluster random sample is one obtained by using groups as the sampling unit rather than individuals.

A two-stage random sample selects groups randomly and then chooses individuals randomly from these
groups.

A table of random numbers is a table of numbers, listed and arranged in no particular order, that is used to
select a random sample.

Nonrandom Sampling Methods

A systematic sample is a sample obtained by selecting every nth name in a population.

A convenience sample is any group of individuals that is conveniently available to be studied.

A purposive sample is a sample selected because the individuals have special qualifications of some sort, or
because of prior evidence of representativeness.

Sample Size

Samples should be as large as a researcher can obtain with a reasonable expenditure of time and energy. A
recommended minimum number of subjects is 100 for a descriptive study, 50 for a correlational study, and 30
in each group for experimental and causal-comparative studies.

External Validity (Generalizability)

The term "external validity," as used in research, refers to the extent that the results of a study can be
generalized from a sample to a population.

What Are Data?

The term "data" refers to the kinds of information researchers obtain on the subjects of their research.

Instrumentation

The term "instrumentation" refers to the entire process of collecting data in a research investigation.

Validity and Reliability

An important consideration in the choice of an instrument to be used in a research investigation is

validity: the extent to which results from it permit researchers to draw warranted conclusions about the
characteristics of the individuals studied.

A reliable instrument is one that gives consistent results.

Objectivity and Usability

Whenever possible, researchers try to eliminate subjectivity from the judgments they make about the
achievement, performance, or characteristics of subjects.

An important consideration for any researcher in choosing or designing an instrument is how easy the
actually be to refers
use. to the extent to which the results of a study can be generalized to the
The term instrument
"populationwill
generalizability"
intended population.

The term "ecological generalizability" refers to the extent to which the results of a study can be generalized to
conditions or settings other than those that prevailed in a particular study.

Replication

When a study is replicated, it is repeated with a new sample and sometimes under new conditions.

Validity

It is important for researchers to use valid instruments for the conclusions they draw are based on the
information they obtain using these instruments.

The term "validity," as used in research, refers to the appropriateness, meaningfulness, correctness, and
usefulness of any inferences a researcher draws based on data obtained through the use of an instrument.

Content-related evidence of validity refers to judgments on the content and logical structure of an instrument
as it is to be used in a particular study.

Criterion-related evidence of validity refers to the degree to which information provided by an instrument
agrees with information obtained on other, independent instruments.

A criterion is a standard for judging; with reference to validity, it is a second instrument against which scores
on an instrument can be checked.

Construct-related evidence of validity refers to the degree to which the totality of evidence obtained is
consistent with theoretical expectations.

A validity coefficient is a numerical index representing the degree of correspondence between scores on an
instrument and a criterion measure.

An expectancy table is a two-way chart used to evaluate criterion-related evidence of validity.

Reliability

The term "reliability," as used in research, refers to the consistency of scores or answers provided by an
instrument.

Errors of measurement refer to variations in scores obtained by the same individuals on the same instrument.

The test-retest method of estimating reliability involves administering the same instrument twice to the same
group of individuals after a certain time interval has elapsed.

The equivalent-forms method of estimating reliability involves administering two different, but equivalent,
forms of an instrument to the same group of individuals at the same time.

The internal-consistency method of estimating reliability involves comparing responses to different sets of
items that are part of an instrument.

Scoring agreement requires a demonstration that independent scorers can achieve satisfactory agreement in
their scoring

The Meaning of "Internal Validity"

When a study lacks internal validity, one or more alternative hypotheses exist to explain the outcomes of the
study. These alternative hypotheses are referred to by researchers as "threats to internal validity."

When a study has internal validity, it means that any relationship observed between two or more variables is
unambiguous as to what it means, rather than being due to something else.

Threats to Internal Validity

Some of the more common threats to internal validity are differences in subject characteristics, mortality,
location, instrumentation, testing, history, maturation, attitude of subjects, regression, and implementation.

The selection of people for a study may result in the individuals or groups differing (i.e., the characteristics of
the subjects may differ) from one another in unintended ways that are related to the variables to be studied.

No matter how carefully the subjects of a study (the sample) are selected, it is common to lose some of them
as the study progresses. This is known as "mortality." Such a loss of subjects may affect the outcomes of a
study.

The particular locations in which data are collected, or in which an intervention is carried out, may create
alternative explanations for any results that are obtained.

The way in which instruments are used may also constitute a threat to the internal validity of a study. Possible
instrumentation threats include changes in the instrument, characteristics of the data collector(s), and/or bias
on the part of the data collectors.

The use of pretest in intervention studies sometimes may create a "practice effect" that can affect the results
of a study. A pretest can also sometimes affect the way subjects respond to all intervention.

On occasion, one or more unanticipated, and unplanned for, events may occur during the course of a study
that can affect the responses of subjects. This is known as a history threat.

Sometimes change during an intervention study may be due more to factors associated with the passing of
time than to the intervention itself. This is known as a maturation threat.

The attitude of subjects toward a study (and their participation in it) can create a threat to internal validity.

When subjects are given increased attention and recognition because they are participating in a study, their
responses may be affected. This is known as the Hawthorne effect.

Whenever a group is selected because of unusually high or low performance on a pretest, it will, on the
average, score closer to the mean on subsequent testing, regardless of what transpires in the meantime. This
is called a regression threat.

Whenever an experimental group is treated in ways that are unintended and not a necessary part of the
method being studied, an implementation threat can occur.

Controlling Threats to Internal Validity

There are a number of techniques or procedures that researchers can use to control or minimize threats to
internal validity. Essentially they boil down to four alternatives: (1) standardizing the conditions under which
the study occurs; (2) obtaining and using more information on the subjects of the study; (3) obtaining and
using more information on the details of the study; and (4) choosing an appropriate design.

What Are Inferential Statistics?

Inferential statistics refer to certain procedures that allow researchers to make inferences about a
population based on data obtained from a sample.
Approaches to Research

The term "probability," as used in research, refers to the predicted relative frequency with which a
A good deal of educational research is done in one of two ways: either two or more groups are compared, or
given
event
will
occur.
variables
within
one
group
are related.
Sampling Error
The data in a study may be either quantitative or categorical.

The term
"sampling
error"
refers to the variations in sample statistics that occur as a result of repeated
Comparing Groups
Using
Quantitative
Data
sampling from the same population.

When comparing two or more groups using quantitative data, researchers can compare them through
The
Distribution
of Sample
Means of averages, and calculation of spreads.
frequency
polygons,
calculation
A samplingtherefore,
distribution
of means isfrequency
a frequency
distribution
frommeans
plotting
means
of a very
We recommend,
constructing
polygons,
usingresulting
data on the
of the
known
groups,
large effect
number
of samples
from the
same population.
calculating
sizes,
and reporting
confidence
intervals when comparing quantitative data from two or more
groups.
The standard error of the mean is the standard deviation of a sampling distribution of means. The
standard error of the difference between means is the standard deviation of a sampling distribution of
differences between sample means.

Relating Variables Within a Group Using Quantitative Data

Confidence Intervals

When researchers examine a relationship between quantitative variables within a single group, the appropriate
techniques
are the scatterplot
the extending
correlationboth
coefficient.
A confidence
interval is aand
region
above and below a sample statistic (such as a sample
mean) within which a population parameter (such as the population mean) may be said to fall with a
specified
probability
of being
Because
a scatterplot
illustrates
all wrong.
the data visually, researchers should begin their analysis of data obtained
from a single group by constructing a scatterplot.
Hypothesis Testing

Therefore, we recommend constructing scatterplots using both scatterplots and correlation coefficients when
relating
variableshypothesis
involving quantitative
dataofwithin
a singlethe
group.
Statistical
testing is a way
determining
probability that an obtained sample statistic
will occur, given a hypothetical population parameter.
Comparing Groups When the Data Involved Are Categorical
A research hypothesis specifies the nature of the relationship the researcher thinks exists in the

When population.
the data are categorical, groups can be compared by reporting either percentages of frequencies in
crossbreak tables.

The null hypothesis typically specifies that there is no relationship in the population.
It is a good idea to report both the percentage and the number of cases in a crossbreak table, as percentages
alone can be
misleading.
Significance
Levels

The term "significance level" (or "level of significance"), as used in research, refers to the probability of
a sample statistic occurring as a result of sampling error.

Therefore, we recommend constructing crossbreak tables and calculating contingency coefficients when
comparing categorical data involving two or more groups.

Relating Variables Within a Group Using Categorical Data

When you are examining relationships among categorical data within one group, we again recommend
constructing crossbreak tables and calculating contingency coefficients.

Two Final Recommendations

When tests of statistical significance can be applied, it is recommended that they be used to evaluate
generalizability only, not to evaluate the magnitude of relationships. Confidence intervals should be reported
in addition to significance levels.

Both parametric and nonparametric techniques should be used to analyze data rather than either one alone.

The Uniqueness of Experimental Research

Experimental research is unique in that it is the only type of research that directly attempts to influence a
particular variable, and it is the only type that, when used properly, can really test hypotheses about causeand-effect relationships. Experimental designs are some of the strongest available for educational researchers
to use in determining cause and effect.

Essential Characteristics of Experimental Research

Experiments differ from other types of research in two basic ways comparison of treatments and the direct
manipulation of one or more independent variables by the researcher.

Randomization

Random assignment is an important ingredient in the best kinds of experiments. It means that every
individual who is participating in the experiment has an equal chance of being assigned to any of the
experimental or control conditions that are being compared.

Control of Extraneous Variables

The researcher in an experimental study has an opportunity to exercise far more control than in most other
forms of research.

Some of the most common ways to control for the possibility of differential subject characteristics (in the
various groups being compared) are randomization, holding certain variables constant, building the variable
into the design, matching, using subjects as their own controls, and the statistical technique of ANCOVA.

Weak Experimental Designs

Three weak designs that are occasionally used in experimental research are the one-shot case study design,
the one-group pretest-posttest design, and the static-group design. They are considered weak because they
do not have built in controls for threats to internal validity.

In a one-shot case study, a single group is exposed to a treatment or event, and its effects assessed.

In the one-group pretest-posttest design, a single group is measured or observed both before and after
exposure to a treatment.

In the static-group comparison design, two intact groups receive different treatments.

True Experimental Designs

Several stronger designs that are more commonly used are true experimental designs, matching designs,
counterbalanced designs, time-series designs, and factorial designs. These designs do have at least some
controls built into the design to control for threats to internal validity.

The randomized posttest-only control group design involves two groups formed by random assignment and
receiving different treatments.

The randomized pretest-posttest control group design differs from the randomized posttest-only control group
only in the use of a pretest.

The randomized Solomon four-group design involves random assignment of subjects to four groups, with two
being pretested and two not.

Matching

To increase the likelihood that groups of subjects will be equivalent, pairs of subjects may be matched on
certain variables. The members of the matched groups are then assigned to the experimental and control
groups.

Matching may be either mechanical or statistical.

Mechanical matching is a process of pairing two persons whose scores on a particular variable are similar.

Two difficulties with mechanical matching are that it is very difficult to match on more than two or three
variables, and that in order to match, some subjects must be eliminated from the study, since no matches can
be found.

Statistical matching does not necessitate a loss of subjects.

Quasi-Experimental Designs

In a counterbalanced design, all groups are exposed to all treatments, but in a different order.

A time-series design involves repeated measurements or observations over time, both before and after
treatment.

Factorial Designs

Factorial designs extend the number of relationships that may be examined in an experimental study.

Essential
Major
The
Nature
Characteristics
Characteristics
of Qualitative
Correlational
Causal-Comparative
of Survey
of
Research
Single-Subject
Research
Research
Research
Research

Single-subject
The
Causal-comparative
Most
major
term
surveys
"qualitative
characteristic
possess
research
research,
research"
three
involves
of correlational
basic
like
refers
the
characteristics:
correlational
extensive
to research
the quality
collection
research,
(1)
is seeking
ofthe
relationships,
of
collection
seeks
data
out to
associations
onidentify
of
one
activities,
information
subject
associations
among
situations,
at (2)
a variables.
time.
form
among
or
a materials.
sample
variables.
(3)
by asking questions, in order to describe some aspects of the population of which the sample is a part.

Purposes
Correlational
Research
The
Anofadvantage
Causal-comparative
natural setting
of single-subject
isresearch
a direct attempts
source
designs
of to
is
data
determine
that
and
they
thecan
the
researcher
be
cause
applied
orisconsequences
a
inkey
settings
part of
where
the
of differences
instrumentation
group designs
that are
The Purpose
of
Survey
Research
difficult to
already
process
exist
in put
qualitative
between
into play.
research.
or among groups of individuals.
Correlational studies are carried out either to help explain important human behaviors or to predict likely
The
major
purpose
all surveys
is toindescribe
the
characteristics
of aand
population.
Single-Subject
outcomes.
Designs
Qualitative
The
basic
causal-comparative
data areof
collected
mainly
approach
the
is toform
begin
of with
words
a noted
or pictures
difference
seldom
between
involve
two groups
numbers.
and then
to look for
Content
analysis
possible
is a
causes
primary
for,method
or consequences
of data analysis.
of, this difference.
Rarely
Single-subject
If
a relationship
is the population
designs
of sufficient
are
as most
a magnitude
whole
commonly
studied,
exists
used
however.
between
to study
Instead,
two
the
variables,
changes
a sample
itinbecomes
isbehavior
surveyed
possible
anand
individual
a to
description
predict
exhibits
a of
after
score
the
population
exposure
on
either
is
to
variable
inferred
a
treatment
if
from
a
score
or
what
intervention
on
the
the
sample
other
of
variable
some
reveals.
sort.
is
known.
Qualitative
There are three
researchers
types ofare
causal-comparative
especially interested
research
in how
(exploration
things occur
ofand
effects,
particularly
exploration
in the
of perspectives
causes,
exploration
of
the subjects
of consequences),
of a study.
which differ in their purposes and structure.
Types
Surveys
ofThe
Single-subject
variable that
researchers
is used toprimarily
make theuse
prediction
line graphs
is called
to present
the predictor
their data
variable.
and to illustrate the effects of
a particular
intervention
or not,
treatment.
Qualitative
When
an experiment
researchers
would
do
takeusually,
a considerable
formulate
length
a hypothesis
of time and
beforehand
be quite and
costly
then
to seek
conduct,
to test
a causalit.
Rather,
There
are
two
major
types
of
surveys:
cross-sectional
surveys
and
longitudinal
surveys.
comparative
they
allow
study
hypotheses
is
sometimes
to
used
emerge
as
an
as
alternative.
a
study
develops.
The variable about which the prediction is made is called the criterion variable.
The basic approach of researchers using an A-B design is to expose the same subject, operating as his
own and
control,
to regression
two conditions
phases.
or
As her
Qualitative
in scatterplots
correlational
quantitative
studies,
relationships
research
differ
can
inbethe
identified
causal-comparative
assumptions
which
study,
underlie
but
the
causation
two
Three
longitudinal
designs
commonly
employed
inphilosophic
survey in
research
are to
trend
studies,
cohort
and
Both
and
linesor
are
used
in
correlational
studies
predict
a
score
on
astudies,
criterion
cannotstudies.
approaches.
be fully established.
panel
variable.
When using an A-B-A design, (sometimes called a reversal design), researchers simply add another
Causal-Comparative
Steps
baseline
inperiod
Qualitative
versus
todifferent
the
Correlational
Research
A-B samples
design.As
Research
Involved
In
a trend
study,
from
a population
whose
members
change
surveyed
at different
A predicted
score
is
never
exact.
a result,
researchers
calculate
an index
of are
prediction
error,
which is
points
known in
astime.
the "standard error of estimate."

The
In the
basic
steps
A-B-A-B
similarity
involved
design,
in
between
conducting
two baseline
causal-comparative
a qualitative
periods are
study
combined
andare
correlational
notwith
as distinct
twostudies
treatment
as they
is that
periods.
are
both
in quantitative
seek to explore
relationships
studies.
They
among
often
overlap
variables.
and
When
sometimes
relationships
even
conducted
are
identified
concurrently.
through
causal-comparative
research
InCorrelational
a cohort study,
different samples from a population whose members do not change are surveyed
at
Complex
Techniques
(or
in
correlational
research),
they
often
are
studied
at
a
later
time
by
means
of
experimental
research.
points
in is
time.
different
The B-A-B
design
used when an individual's behavior is so severe or disturbing that a researcher
All
qualitative
athat
foreshadowed
problem, the
particular phenomenon
researcher
is
cannot
wait
forstudies
a baseline
towith
be established.
Multiple
regression
is abegin
technique
enables a researcher
to determine
a correlation the
between
a
Causal-Comparative
versus
Experimental
Research
in
investigating.
interested
In
a
panel
study,
the
same
sample
of
individuals
is
surveyed
at
different
times
over
the
course
of
the
criterion variable and the best combination of two or more predictor variables.
study.
In the A-B-C-B design, the "C" condition refers to a variation of the intervention in the "B" condition. The
In
experimental
research,
group
membership
variable
is manipulated;
in
causal-comparative
Researchers
who
inthe
a qualitative
study
of some
type
usually
select
a purposive
sample.
Several
intervention
is
changed
during
the "C"
phase
typically
control
forthe
any
extra
attention
the the
subject
may
The
coefficient
of engage
multiple
correlation
(R)
indicates
theto
strength
of
correlation
between
research
the
group
differences
already
exist.
have
types
received
of
purposive
during
samples
the
"B"
exist.
phase.
Surveys
are not
suitable
for all
research
topics,
especially
those that require observation of subjects or
combination
of the
predictor
variables
and
the criterion
variable.
the manipulation of variables.
Steps
Causal-Comparative
Research
Multiple
Baseline
Designs
inThere
is no
in aequation
qualitative
study,on
nor
is thereitany
manipulation
of variables.
The
value
oftreatment
a prediction
depends
whether
predicts
successfully
with a new group of
Steps inindividuals.
Survey Research
The
step in
aused
problem
in
research
is usually
identify and
and return
define to
Multiple-baseline
designs
with
it causal-comparative
is is
not
possible or ethical
to withdraw
a to
treatment
The first
collection
of formulating
data
in aare
qualitative
study
ongoing.
the
particular
phenomena
of
interest,
and
then
to
consider
possible
causes
for,
or
consequences
of,
baseline.
The focus
study in
a survey
is called the
unit of
analysis.
When
the of
criterion
variable
is categorical
rather
than
quantitative, discriminant function analysis (rather
these phenomena.
than multipleare
regression)
must be used.
Conclusions
drawn continuously
throughout the course of a qualitative study.
When
a multiple-baseline
design
used,ofresearchers
doismore
than collect
data on
behavior
for one
As
in other
types of research,
theisgroup
persons that
the focus
of the study
is one
called
the target
The
important
thing in selecting
a sample
for abehaviors
causal-comparative
studyobtaining
is to define
carefullyfor
theeach
subject
in
one
setting;
they
collect
on
several
for
one
subject,
a
baseline
population.
Factor
is aResearch
technique that allows a researcher to determine whether many variables can be
Approaches
to analysis
Qualitative
characteristic
to be
studied
and then to select groups that differ in this characteristic.
during the same
period
of time.
described by a few factors.
There
are four study
basic ways
to collect
data
a survey:
by in
direct life
administration
of the survey instrument
A
biographical
the
story
of
thein
special
events
of
a single individual.
are no limits
totells
the also
kinds
of sometimes
instruments
that
can
be the
used
a causal-comparative
There
Multiple-baseline
designs
are
used
to
collect
datainmethod
on
several
withstudy.
regard to a
a group,
byismail,
by telephone,
by personal
Each
hassubjects
advantages
and
to
Path
analysis
a technique
used toortest
a theory interview.
about the possibility
of causal
connections
among
single
behavior,
or
to
measure
a
subject's
behavior
in
two
or
more
different
settings.
disadvantages.
three
or
more
variables.
A
researcher
studies an individual's
a particular
phenomenon
in a on
phenomenological
study.
The
basic causal-comparative
designreactions
involves to
selecting
two groups
that differ
a particular variable
of
He
or
she
tries
to
describe
an
individual's
experiences
from
the
subject's
perspective.
Threats
interest
to
Internal
and
Validity
then
comparing
in
Single
Subject
them
on
Research
another
variable
or
variables.
Steps
The in
sample
to be surveyed
should be selected randomly if possible.
Basic
Correlational
Research

In
grounded
theory
a researcher
forms
a theory
that is formed
inductively
the data
that is
Threats
to a
Internal
Validity
in study,
Causal-Comparative
Research
Several
threats
to internal
validity
exist with
regard
to single-subject
designs.
Thesefrom
include
the length
The
These
most
include,
common
asof
intypes
most
of
research,
instruments
selecting
used ainproblem,
survey research
choosingare
a sample,
the questionnaire
selecting or
and
developing
the interview
collected
as
part
study.
of the baseline
and the
intervention
conditions, the number of variables changed when moving from one
instruments, determining procedures, collecting and analyzing data, and interpreting results.
schedule.
to another,
the degree and speed
of anyare
change
that
occurs, whether
or not the
condition
Two weaknesses
in causal-comparative
research
lack of
randomization
and inability
to behavior
manipulate an
returns
to
baseline
levels,
the
independence
of
behaviors,
and
the
number
of
baselines.
A
case
study
is
a
detailed
study
of
one
or
(at
most)
a
few
individuals
or
other
social
units,
such as a
independent variable.
Correlation
Questions
Asked
Coefficients
in aSurvey
andResearch
Their
Meaning
classroom,
school,
or a neighborhood.
Controlling
Threats
in Single-Subject
A major
threat
to the internalStudies
validity of a causal-comparative study is the possibility of a subject
The nature
meaning
of of
the
a questions,
given
correlation
and the
coefficient
way theydepends
are asked,
on are
howextremely
it is applied.
important in survey research.
Generalization
in
Qualitative
Research
selection bias. The chief
procedures that a researcher can use to reduce this threat include matching
subjects
Single-subject
designsvariable
are most
in controlling for
subject characteristics,
mortality
testing, and
on a related
or effective
creating homogeneous
subgroups,
and the technique
of statistical
Most
Correlation
surveys
coefficients
some below
form
of
.35
closed-ended
show
only aquestion.
slight
between
history
matching.
threats.
Generalizing
isuse
possible
in qualitative
research,
but itrelationship
is of a different
type variables.
than that found in quantitative
studies. It is more likely it will be done by interested practitioners.
Other
Correlations
The
survey
instrument
between
.40
should
and
.60
bein
pretested
may
have
with
theoretical
a small
and/or
samplepractical
similar
to
value
the depending
potential
respondents.
on thethreats.
context.
They
are
threats
less
effective
to internal
with
validity
location,
causal-comparative
data
collector
characteristics,
studies
include
maturation,
location,
and
instrumentation,
regression
and
Ethics and
Research
lossQualitative
of subjects.
In addition, type 3 studies are subject to implementation, history, maturation, attitude
subjects,
and
testing
threats.
of
Only
A
contingency
when
aregression,
correlation
question
is
ofwhen
a.65
question
or
higher
whose
is
answer is
can
contingent
reasonably
upon
accurate
how abias,
predictions
respondent
be
answers
made. a
They
are especially
weak
it comes
to obtained
instrument
decay,
data-collector
attitude,
and
The
identities
all
participants
in a qualitative
study
shouldWell-organized
be protected, and
should contingency
be treated
prior
question of
tothreats.
which
the contingency
question
is related.
and they
sequenced
implementation
Data Analysis
in Causal-Comparative
Studiesin interview schedules.
questions
with
respect.
are
particularly
important
Correlations over .85 indicate a very strong relationship between the variables correlated.
External Validity and Single-Subject Research
TheLetter
first
step in a and
dataQuantitative
analysis of aResearch
causal-comparative study is to construct frequency polygons.
The Cover
Reconsidering
Qualitative
Evaluating Threats to Internal Validity in Correlational Research

Observer Roles

There are four roles that an observer can play in a qualitative research study, ranging from complete
participant, to participant-as-observer, to observer-as-participant, to complete observer. The degree of
involvement of the observer in the observed situation diminishes accordingly for each of these roles.

Participant versus Nonparticipant Observation

In participant observation studies, the researcher actually participates as an active member of the group in
the situation or setting he or she is observing.

In nonparticipant observation studies, the researcher does not participate in an activity or situation but
observes "from the sidelines."

The most common forms of nonparticipant observation studies include naturalistic observation and
simulations.

A simulation is an artificially created situation in which subjects are asked to act out certain roles.

Observation Techniques

A coding scheme is a set of categories an observer uses to record a person or group's behaviors.

Even with a fixed coding scheme in mind, an observer must still choose what to observe.

A major problem in all observational research is that much that goes on may be missed.

Observer Effect

The term "observer effect" refers to either the effect the presence of an observer can have on the behavior of
the subjects or observer bias in the data reported. The use of audio- and videotapings is especially helpful in
guarding against this effect.

For this reason, many researchers argue that the participants in a study should not be informed of the study's
purpose until after the data have been collected.

Observer Bias

Observer bias refers to the possibility that certain characteristics or ideas of observers may affect what they
observe.

Sampling in Observational Studies

Researchers who engage in observation usually must choose a purposive sample.

Interviewing

A second major technique commonly used by qualitative researchers is in-depth interviewing.

The purpose of interviewing the participants in a qualitative study is not only to find out what they think or
how they feel about something but also to provide a check on the researcher's observations.

Interviews may be structured, semistructured, informal, or retrospective.

The six types of questions asked by interviewers are background or demographic questions, knowledge
questions, experience of behavior questions, opinion or values questions, feelings questions, and sensory
questions.

Respect for the individual being interviewed is a paramount expectation in any proper interview.

Key actors are people in any group who are more informed about the culture and history of the group and who
also are more articulate, than others.

A focus group interview is an interview with a small, fairly homogeneous group of people who respond to a
series of questions asked by the interviewer.

The most effective characteristic of a good interviewer is a strong interest in people and in listening to what
they have to say.

Reliability and Validity in Qualitative Research

An important check on the validity and reliability of the researcher's interpretations in qualitative research is
to compare one informant's description of something with another informant's description of the same thing.

Another, although more difficult, check on reliability/validity is to compare information on the same topic with
different information triangulation.

Efforts to ensure reliability and validity include use of proper vocabulary, recording questions used as well as
personal reactions, describing content, and documenting sources.

What is Content Analysis?

Content analysis is an analysis of the contents of a communication.

Content analysis is a technique that enables researchers to study human behavior in an indirect way by
analyzing communications.

Applications of Content Analysis

Content analysis has wide applicability in educational research.

Content analysis can give researchers insights into problems that they can test by more direct methods.

There are several reasons to do a content analysis: to obtain descriptive information of one kind or another;
to analyze observational and interview data; to test hypotheses; to check other research findings; and/or to
obtain information useful in dealing with educational problems.

Categorization in Content Analysis

Coding (categorizing) by using predetermined categories.

Coding by use of categories that emerge as data is reviewed.

Steps Involved in Content Analysis

In doing a content analysis, researchers should always develop a rationale (a conceptual link) to explain how
the data to be collected are related to their objectives.

Important terms should at some point be defined.

All of the sampling methods used in other kinds of educational research can be applied to content analysis.
Purposive sampling, however, is the most commonly used.

The unit of analysis what specifically is to be analyzed should be specified before the researcher begins
an analysis.

After defining what aspects of the content are to be analyzed, the researcher needs to formulate coding
categories.

Coding Categories

Developing emergent coding categories requires a high level of familiarity with content.

In doing a content analysis, a researcher can code either the manifest or the latent content of a
communication, and sometimes both.

The manifest content of a communication refers to the specific, clear, surface contents: the words, pictures,
images, and such that are easily categorized.

The latent content of a document refers to the meaning underlying what is contained in a communication.

Reliability and Validity as Applied to Content Analysis

Reliability in content analysis is commonly checked by comparing the results of two independent scorers
(categorizers).

Validity can be checked by comparing data obtained from manifest content to that obtained from latent
content.

Data Analysis

A common way to interpret content analysis data is by using frequencies (i.e., the number of specific incidents
found in the data) and proportion of particular occurrences to total occurrences.

Another method is to use coding to develop themes to facilitate synthesis.

Computer analysis is extremely useful in coding data once categories have been determined. It can also be
useful at times in developing such categories.

Advantages and Disadvantages of Content Analysis

Two major advantages of content analysis are that it is unobtrusive and it is comparatively easy to do.

The major disadvantages of content analysis are that it is limited to the analysis of communications and it is
difficult to establish validity

The Nature and Value of Ethnographic Research

Ethnographic research is particularly appropriate for behaviors that are best understood by observing them
within their natural settings.

The key techniques in all ethnographic studies are in-depth interviewing and highly detailed, almost continual,
ongoing participant observation of a situation.

A key strength of ethnographic research is that it provides the researcher with a much more comprehensive
perspective than do other forms of educational research.

Ethnographic Concepts

Important concepts in ethnographic research include culture, holistic outlook, contextualization, emic
perspective, and multiple realities.

Topics That Lend Themselves Well to Ethnographic Research

These include topics that defy simply quantification; those that can best be understood in a natural setting;
those that involve studying individual or group activities over time; those that involve studying the roles that
individuals play and the behaviors associated with those roles; those that involve studying the activities and
behaviors of groups as a unit; and those that involve studying formal organizations in their totality.

Sampling in Ethnographic Research

The sample in ethnographic studies is almost always purposive.

The data obtained from ethnographic research samples rarely, if ever, permit generalization to a population.

The Use of Hypotheses in Ethnographic Research

Ethnographic researchers seldom formulate precise hypotheses ahead of time. Rather, they develop them as
their study emerges.

Data Collection and Analysis in Ethnographic Research

The major means of data collection in ethnographic research are participant observation and detailed
interviewing.

Researchers use a variety of instruments in ethnographic studies to collect data and to check validity. This is
frequently referred to as "triangulation."

Analysis consists of continual reworking of data with emphasis on patterns, key events, and use of visual
representations in addition to interviews and observations.

Fieldwork

Field notes are the notes a researcher in an ethnographic study takes in the field.

They include both descriptive field notes (what he or she sees and hears) and reflective field notes (what he or
she thinks about what has been observed).

Field jottings refer to quick notes about something the researcher wants to write more about later.

A field diary is a personal statement of the researcher's feelings and opinions about the people and situations
he or she is observing.

A field log is a sort of running account of how the researcher plans to spend his or her time compared to how
he or she actually spends it.

Advantages and Disadvantages of Ethnographic Research

A key strength of ethnographic research is that it provides a much more comprehensive perspective than
other forms of educational research. It lends itself well to topics that are not easily quantified. Also, it is
particularly appropriate to studying behaviors best understood in their natural settings.

Like all research, ethnographic research also has its limitations. It is highly dependent on the particular
researcher's observations. Furthermore, some observer bias is almost impossible to eliminate. Lastly,
generalization is practically nonexistent.

The Nature of Historical Research

The unique characteristic of historical research is that it focuses exclusively on the past.

Purposes of Historical Research

Educational researchers conduct historical studies for a variety of reasons, but perhaps the most frequently
cited is to help people learn from past failures and successes.

When well-designed and carefully executed, historical research can lead to the confirmation or rejection of
relational hypotheses.

Steps in Historical Research

There are four essential steps involved in doing a historical study. These include defining the problem or
hypothesis to be investigated; searching for relevant source material; summarizing and evaluating the sources
the researcher is able to locate; interpreting the evidence obtained and then drawing conclusions about the
problem or hypothesis.

Historical Sources

Most historical source material can be grouped into four basic categories: documents, numerical records, oral
statements, and relics.

Documents are written or printed materials that have been produced in one form or another sometime in the
past.

Numerical records include any type of numerical data in printed or handwritten form.

Oral statements include any form of statement made orally by someone.

Relics are any objects whose physical or visual characteristics can provide some information about the past.

A primary source is one prepared by an individual who was a participant in, or a direct witness to, the event
that is being described.

A secondary source is a document prepared by an individual who was not a direct witness to an event, but
who obtained his or her description of the event from someone else.

Evaluation of Historical Source Material

Content analysis is a primary method of data analysis in historical research.

External criticism refers to the genuineness of the documents a researcher uses in a historical study.

Internal criticism refers to the accuracy of the contents of a document. Whereas external criticism has to do
with the authenticity of a document, internal criticism has to do with what the document says.

Generalization in Historical Research

As in all research, researchers who conduct historical studies should exercise caution in generalizing from
small or nonrepresentative samples.

Advantages and Disadvantages of Historical Research

The main advantage of historical research is that is permits the investigation of topics that could be studied in
no other way. It is the only research method that can study evidence from the past.

A disadvantage is that controlling for many of the threats to internal validity is not possible in historical
research. Many of the threats to internal validity discussed in Chapter Nine are likely to exist in historical
studies.

The Nature of Action Research

Action research is research conducted by a teacher, administrator, or other education professional to solve a
problem at the local level.

Each of the specific methods of research can be used in action research studies, although on a smaller scale.

A given research question may often be investigated by any one of several methods.

Some methods are more appropriate to a particular research question and/or setting than other methods.

Assumptions Underlying Action Research

Several assumptions underly action research studies. These are that the participants have the authority to
make decisions, want to improve their practice, and committed to continual professional development, and will
engage in systematic inquiry.

Types of Action Research

Practical action research addresses a specific local problem.

Participatory action research in addition attempts to empower participants or bring about social change.

Level of Participation in Action Research

Participation can vary from information giving to greater and greater involvement in aspects of the study.

Steps in Action Research

There are four. These include identifying the research question or problem, gathering the necessary data,
analyzing and interpreting the data, and sharing the results with the participants.

The participants in a participatory action research study are often referred to as stakeholders.

Advantages of Action Research

There are at least five. It can be done by just about anyone, in any type of school or other institution, to
investigate just about any kind of problem or issue. It can help to improve educational practice. It can help
education and other professionals to improve their craft. It can help them learn to identify problems
systematically, and nit can build up a small community of research-oriented individuals at the local level.

Action research has both similarities to, and differences from formal qualitative and quantitative research.

Sampling in Action Research

Action researchers are most likely to choose a purposive sample.

Threats to the Internal Validity of Action Research

Action research studies suffer especially from the possibility of data-collector bias, implementation, and
attitudinal threats. Most others can be controlled to a considerable degree.

External Validity and Action Research

Action research studies are weak in external validity

Replication is, of necessity, therefore, essential in these studies

Research Proposal versus Research Report

A research proposal communicates a researcher's plan for a study.

A research report communicates what was actually done in a study, and what resulted.

Major Sections of a Research Proposal or Report

The main body is the largest section of a proposal or a report and generally includes the problem to be
investigated (including the statement of the problem or question, the research hypotheses and
variables, and the definition of terms); the review of the literature; the procedures (including a
description of the sample, the instruments to be used, the research design, and the procedures to be
followed; an identification of threats to internal validity; and a description and a justification of the
statistical procedures used); and (in a proposal) a budget of expected costs.

All sections of a research proposal or a research report should be consistent with one another.

Sections Unique to Research Reports

The essential difference between a research proposal and a research report is that a research report
states what was done rather than what will be done and includes the actual results of the study. Thus, in
a report, a description of the findings pertinent to each of the research hypotheses or questions is
presented, along with a discussion by the researcher of what the findings of the study imply for overall
knowledge and current practice.

Normally, the final section of a report is the offering of some suggestions for further research.