III.

Constructing research designs

- Research designs: to avoid as many research errors as possible.
- Five major classes of research design: pre-experimental, true
experimental, quasi-experimental, ex post facto, and time series.
Additionally: factorial design.

a. Pre-experimental designs
- Not really considered model experiments, because they do not
account for extraneous variables, and their internal validity is
questionable.
- But, they are easy and useful ways to get preliminary
information.
- Three most common pre-experimental designs: one-shot case
study, one-group pretest posttest, and intact group comparison.

a.1 The one-shot case study

- No control group
- You give a group a treatment (X), then you test the group on the
treatment (T): X T
- Both internal and external validity questionable.

a.2 One-group pretest posttest design

- Similar to one-shot case, but with a pretest
T1 X T2
- Better than one-shot, because you can measure the gains
- Still without a control group
- Issues with interval validity

a.3 Intact group design

 - Most classroom researchers use
- There’s a control group
- But external validity questionable, because Ss are not randomly
assigned to the two groups (experimental and control).
G1 X T1
-------------
G2 T2
- Internal validity also questionable because the 2 groups might
not have been equivalent to start with

b. True experimental designs

- A control group or groups is present
- Ss are randomly assigned to the groups
- There’s a pretest
=> Avoid most problems related to internal or external validity

b.1 Posttest only control group

- Two groups: experimental & control
- Ss are randomly assigned to the groups
- Initial differences between groups are controlled by the random
assignment of Ss
G1(random) X T1
-------------------------
G2(random) T1

b.2 pretest posttest control group design

- Same with Posttest only, but with a pretest
G1(random)T1 X T2
----------------------------
G2(random)T1 X T2
 - Why would anyone bother not to have the pretest? => To avoid
its impact (Test effect)

c. Quasi-experimental designs
- We can hardly or even cannot conduct a true experimental
design in social sciences, because:
+ It’s hard to find random students.
+ It’s hard to define the numerous variables involved in
a research. They might not even be mutually exclusive (Is Reading
separated from writing?)
+ It’s hard to control the factors. (We choose all
advanced students because we want to control their levels. But
there are advanced students who are good at grammar, and those
who are good at speaking.)
+ It’s hard to ask a group of students to sit our research
if they gain nothing from it.
=> Impossible to create a true experimental design.
- But, doesn’t mean we should give up.
=> Quasi-experimental design:
> control as many variables as possible
> limit the kinds of interpretations we can make
about cause-effect relationships
> hedge the power of generalization statements.

d. Time-series designs
- Great when there’s no control groups
- Several pre- and posttests
T1 T2 T3 X T4 T5 T6
 - Time-sample :

T1 X T2 -> T3 O T4 -> T5 X T6 -> T7 O T8 etc.

- Other ways to deal with no control groups: divide the existing
group into 2 (only works if you have a lot of Ss in that existing
group).

e. Ex post facto designs

- Limit the domain of our claims. Avoid making cause-and-effect
statements
- Look at the type and/or degree of relationship between two
variables rather than a cause-and effect relationship

e.1 Correlation designs: T1 T2

e.2 Criterion group design: two groups are compared on one
measure (the reading speed of Iranian and French students):
G1 T1
G2 T1
e.3 Two-criterion design (factorial design)

f. Factorial designs
- Not a design type in itself
- Simply adding more variables to the other designs, like
moderator variables:

|
|
\/