Lecture 4B:

Quasi-experimental Studies

1
 Objective

 Describe the various forms of quasi-

experimental design and their pros and
cons.

2
 Continuum of Research
Descriptive Exploratory Experimental

--------------------------------------------------------------------------
Describe Find
Cause and Effect
Populations Relationships
True experimental /
randomized controlled
trial (RCT)

Single-subject design

Quasi-experimental design

Adapted from Portney & Watkins (2000), p. 177 3

 Quasi-experimental Design

Major characteristics:

1. No random assignment to groups

OR/AND

2. No control/comparison group

4
 Quasi-experimental Design
1. One group pretest-posttest design

2. Interrupted time series

3. Patient as own control

4. Historical controls

5. Non-equivalent Pretest Posttest Control Group

Design.
5
1. One Group Pretest-Posttest Design

Study sample

Treatment Group

M1 (pre-test)

Treatment Period

M2 (post-test)
6
 1. One Group Pretest-Posttest Design

 Weakest design among all quasi-experimental

designs.

 Disadvantages:
 Lack of control group

 History effect

 Maturation effect

 Testing/Practice effect

7
2. Interrupted Time-Series Design

 One treatment group

 Multiple pretests and Multiple posttests after

intervention.

8
2. Interrupted Time-Series Design

Portney & Watkins, 2009, Fig 11-4 9

2. Interrupted Time-Series Design
 Advantages:

 Lessen the effects of maturation,

testing/practice.

 More confidence in baseline and outcome

data than one group pretest-posttest
design.

10
2. Interrupted Time-Series Design
 Disadvantages:
 More time consuming.

 No control group: no control over history.

 Difficult to get stability of measurements in

the pretest phase in some cases.

 Difficult to apply to subjects with acute

conditions.
11
 3. Patient as Own Control

 The subjects serve as their own control.

 The same group of subjects are measured

on outcome(s) in presence /absence of a
single/various intervention(s).

12
3. Patient as Own Control

Study sample
Measurement

Control Period
Measurement

Treatment 1
Measurement

13
3. Patient as Own Control
Study sample
Measurement

Control period
Measurement

New treatment 1
Measurement

New treatment 2
Measurement

14
 3. Patient as Own Control
 Advantage:
 Have some control over maturation/practice.

 Disadvantages:
 A period of no treatment or delayed treatment
(ethics?)
 Carry-over effect possible.
 No control over history.

15
 4. Historical Controls
 Use of a control group who received a
different intervention during an earlier time
period.

 The protocol in one study acts as a control

for later studies.

16
 4. Historical Controls: Example
Objective: To compare the effects of robotic therapy and
conventional training on gait velocity in patients with
incomplete SCI.

Robotic therapy group:

 100 subjects will receive robotic therapy for the next 3

months.

Conventional training group:

 Check the medical records in the past 2 years and find out

from the rehab notes the information on improvement in

gait velocity before and after conventional training.
17
 4. Historical Controls

 Advantages:
 Having a comparison group.
 Alternative when ethical concerns may
preclude a true control group.

18
 4. Historical Controls
 Disadvantages:
 Different subject characteristics of the
treatment and historical control groups (no
randomization, subjects from the past).
 Concerns with how the data were collected
from historical controls (measurement
error/bias).

19
 4. Historical Controls
 Disadvantages: (Cont’d)
 Exposure to different factors for the two
groups
 Different study environment.
 Different time of the year.

20
 5. Non-equivalent Pretest- Posttest
Control Group Design

Similar to True Experimental Design.

But

The group allocation is NON-RANDOM.

21
 5. Non-equivalent Pretest-Posttest
Control Group Design

Sample
Non-
Random
assignment
Treatment Group Control Group

M1 (pre-test) M1 (pre-test)

Treatment Period Control Period

M2 (post-test) M2 (post-test) 22
 5. Non-equivalent Pretest-Posttest
Control Group Design
 Advantages:
 Strongest quasi-experimental design.
 Some control over history, maturation,
testing/practice, and instrumentation effects.
 An alternative to RCT.

 Disadvantages:
 Comparison groups may not be equivalent
(What are the consequences?).
23
 Summary: Quasi-experimental Design

 Advantages:
 Practical.
 Do not have to deny treatment to one group.
 More generalizable to clinical setting.
 A reasonable alternative when use of
randomization and control groups are not
feasible.

 Disadvantages:
 Less control over factors that influence internal
validity.
24
 Test Yourself #1
 An occupational therapist wanted to determine whether the
OT training is effective in improving UE function for people
with brain injury.

 Each subject received 4 weeks of training.

 Hand function was measured by the same occupational

therapist using the Wolf Motor Function Test once before
and after treatment.

 What is the study design?

25
 Test Yourself #2

 Which of the following is NOT a threat to

internal validity in the previous study?

A. History
B. Maturation
C. Recall bias
D. Placebo
E. Rater/tester bias
26
 References

1. Portney LG, Watkins MP. Foundations of Clinical

Research. Applications to Practice. (3rd
International ed.). Essex: Pearson Education Inc.,
2014.

2. Glass GV, Hopkins KD. Statistical Methods in

Education and Psychology. (3rd ed.). Needham
Heights, MA: Allyn & Bacon, 1996.

27
 Test Yourself #1 (Answer)
 An occupational therapist wanted to determine whether the
OT training is effective in improving UE function for people
with brain injury.

 Each subject received 4 weeks of training.

 Hand function was measured by the same occupational

therapist using the Wolf Motor Function Test once before
and after treatment.

 What is the study design?

28
 Test Yourself #2 (Answer)

 Which of the following is NOT a threat to

internal validity in the previous study?

A. History
B. Maturation
C. Recall bias
D. Placebo
E. Rater/tester bias
29