download full file at http://testbankinstant.

com

Chapter 10
Quasi-Experimental Designs

Learning objectives

To be able to

 Explain how quasi-experimental designs differ from weak and strong experimental
designs.
 Describe the characteristics of each of the quasi-experimental designs.
 Explain the threats to internal validity and how rival hypotheses are ruled out in each of
the quasi-experimental designs.

Multiple choice questions

1) In which of the following circumstances would a researcher be most likely to choose a quasi-
experimental research design?
a) when there are more than three important confounding variables to control
* b) when doing research in an applied or natural setting where control is difficult to
impose
c) when controlling experimenter and participant expectation effects would involve an
unacceptable level of deception
d) when it is difficult to recruit enough participants to make standard statistical testing
valid

2) The most common reason for the use of quasi-experimental research designs is that
a) the participants are maturing too rapidly.
b) the dependent variable cannot be measured reliably.
c) it is unethical to manipulate the independent variable .
* d) participants cannot be randomly assigned to groups.

3) Quasi-experimental designs lack the controls of true experimental designs.

* a) however, causal inferences can be made to the extent that rival hypotheses can be ruled
out
b) as such, causal inferences should never be made
c) however, in most cases such deficiencies can be safely ignored
d) thus, such studies should not be performed by ethical researchers

4) One essential characteristic of quasi-experimental designs that distinguishes them from true
experimental designs is a lack of
a) volunteer participants.
* b) control of extraneous variables.
c) experimenters who can conduct them.
d) an interest in laboratory experimentation.

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

5) When one does not have the ability to , one may consider using quasi-experimental
designs.
a) measure the dependent variable quantitatively
* b) randomly assign participants to groups
c) manipulate the independent variable
d) make multiple measures of the dependent variable

6) You have been asked to evaluate the effectiveness of a new anti-smoking campaign in a local
high school. You are able to collect pretest information on smoking rates before the campaign
begins. You know that the entire school will be required to participate in the campaign and so
random assignment is impossible.You are also aware that a reduction in smoking after the
campaign could be caused by any number of other confounding variables.What can you do in
this situation to enhance your ability to determine if the campaign was effective in reducing
smoking?
a) test the students at the end of the campaign and then again a year later – if reductions
in smoking remain at that time then you can be confident that the campaign was

effective
b) conduct extensive postexperimental interviews to determine the cause of any change in
smoking rates
* c) identify a similar school that is not using the campaign and include them in a
nonequivalent comparison group design
d) perform a median split among the smokers in the school and perform a regression
discontinuity study

7) Although quasi-experimental designs do not meet all of the criteria for true experimental
designs, they may still
a) be used by experimenters without much training.
b) be performed in a fairly short time frame.
* c) allow one to make statements about causation.
d) control for potential effects of extraneous variables.

8) The non-equivalent comparison group design is a quasi-experimental design in which, for

reasons of practicality, we cannot insure that the control and experimental groups are equivalent
to each other when the experiment begins. The major interpretational difficulty imposed by this
design is
a) knowing whether the two groups are different from each other on the dependent
measure once the experiment is complete.
b) deciding how much each group has to gain on the posttest compared to the pretest to
be sure that the differences are reliable.
c) determining when we have collected enough data points to make a statement about the
experiment's outcome.
* d) being sure that any differences between groups at the end of the experiment result
from the independent variable's influence alone, and not related to preexisting
differences.

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

9) What is the main distinction between the nonequivalent comparison group design and the
pretest-posttest design?
a) the use of pretesting in the latter
b) the use of statistical data analysis in the latter
* c) the random assignment of participants to groups in the latter
d) more than one independent variable in the latter

10) Which of the following designs is the “best” in terms of eliminating potential confounds?
a) nonequivalent comparison group
b) posttest-only design with nonequivalent groups
* c) pretest-posttest control-group design
d) interrupted time series design

11) Dr. Johannsen wants to test whether recovering alcoholics perform worse on tests of motor
coordination than a group of control participants. Unbeknownst to him, the control group meets
at an amusement park three hours before testing and goes on as many rides as possible. They
perform worse than the alcoholic group. This is an example of a(n)
a) regression effect.
b) selection-maturation effect.
c) regression artifact.
* d) selection-history effect.

12) The selection-history effect is particularly a problem in which quasi-experimental design?

* a) non-equivalent comparison group
b) time series
c) interrupted time series
d) regression discontinuity

13) The nonequivalent comparison group design can yield several possible outcomes. In one
scenario, the experimental group scores higher than the control group at the start of the
experiment, and only the experimental group's scores increase from pretesting to posttesting.
This could reflect an influence of the independent variable. However, it could also reflect
a) a local instrumentation effect.
b) a attrition effect in the control group.
c) simply that the experimental group was studied for a longer time period than the
control group.
* d) selection-maturation effect.

14) One strategy for dealing with preexisting differences between experimental and control
groups in the nonequivalent comparison group design is to match the participants.
a) this procedure is as effective as randomly assigning participants from the start
b) this procedure eliminates local history as a confounding factor
* c) this procedure, however, may introduce a regression-artifact phenomenon as extreme
scores are frequently needed to match
d) this procedure, however, confounds the research even more than the preexisting

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

differences

15) The nonequivalent comparison group design can yield several possible outcomes. A possible
outcome of the non-equivalent comparison group design is that the groups differ on the pretest
and both groups improve on the posttest, but to different degrees. This could reflect the fact that
the independent variable had an effect on the experimental group. It could also reflect
a) an unreliable dependent measure.
b) the fact that the control group experienced less of the independent variable than the
experimental group did.
c) a participant-expectation effect.
* d) a selection-maturation effect.

16) The nonequivalent comparison group design can yield several possible outcomes. A possible
outcome of the non-equivalent comparison group design is that the control group scores higher
than the experimental group on the pretest and posttest but the experimental group improves
more on posttesting. This could reflect the fact that the independent variable had an effect on the
experimental group. It could also reflect
a) an unreliable dependent measure.
b) the fact that the control group experienced less of the independent variable than the
experimental group did.
* c) regression-artifact effect if the experimental group had been selected for their low
initial scores.
d) a selection-maturation effect if the experimental group had been selected for their

older age.

17) In the physical education department, Dr. Shanz asks two of his strength training classes to
participate in an experiment. One class will follow a traditional program of index finger strength
enhancement, while the other class will try a new, experimental procedure. At the beginning of
the semester, the class that will use the experimental method starts out with, on average, weaker
index fingers than the other class. At the end of the semester, the class using the traditional
method shows no significant increase in finger strength, while the other experimental class now
has fingers that are, on average, significantly stronger than those of the control class. This is an
example of a(n) effect.
a) interaction
* b) crossover
c) intragroup regression
d) selection-maturation

18) The presence of a crossover effect in one's data can usually rule out the influence of which
rival hypothesis?
a) history
* b) regression-artifact
c) local history
d) attrition

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

19) Dr. Kaiser, a sports psychologist, wants to know if he can improve the cardiovascular
endurance in elementary school children. He is allowed to study two physical education classes
at a local school, one boys’ class and one girls’ class. On the pretest, the girls on average have
slightly better cardiovascular endurance as measured on the step test. He administers the year-
long aerobic dancing program to the girls’ class. The posttests revealed that the girls’
cardiovascular endurance improved much more than the boys.’ Dr. Kaiser must now decide if the
program really worked or
a) if there was a selection-history effect.
* b) if there was a selection-maturation effect.
c) if girls always do better in aerobic exercises.
d) if the boys just didn’t want to compete with the girls.

20) The most common of the quasi experimental designs is the nonequivalent comparison group
design. What is the dependent variable most commonly used in this design?
a) latency measure
* b) difference scores (Pretest-posttest)
c) number or percent correct
d) total correct

21) Which of the following is NOT a bias that commonly exits in nonequivalent comparison
group designs?
a) selection bias
b) attrition bias
c) selection-regression bias
* d) experimenter expectancy bias

22) According to Glass, Wilson and Gottman, you need about data points to make a
statistical assessment of the data from a time-series design.
a) 25
* b) 50
c) 75
d) 100

23) ___________ is a statistical technique designed to assess the effectiveness of an interrupted

time series design.
a) Reliability adjusted analysis of covariance (ANCOVA)
b) Propensity score matching
c) Selection modeling
* d) Autoregressive integrated moving average (ARIMA)

24) Making a causal inference from quasi-experimental evidence requires all but which of the
following:
a) the suspected cause must covary with the effect
b) the suspected cause must precede the effect
c) rival hypotheses must be ruled out or be highly implausible.
* d) Bayesian moving average statistical analyses.

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

25) When looking at interaction effects, it is really good to find a crossover effect because it can
rule out various rival hypotheses by its very nature. Which of the following rival hypotheses is
not ruled out by the presence of a crossover effect?
a) statistical regression effect
b) maturation effect
* c) selection-history effect
d) none of the above

26) In a nonequivalent comparison group study,

* a) the experimental group gets a pretest, then the experimental treatment, then a posttest.
b) participants are assigned to the control group by means of a matching procedure.
c) to interpret the results, the pretest scores have to be identical for the two groups.
d) random assignment can be used if sample size is large enough.

27) In a study comparing two nonequivalent groups, a selection-history confound occurs when
* a) some event intervenes between pretest and posttest and affects just one of the groups.
b) some event intervenes between pretest and posttest and affects both selected groups
equally.
c) selection influences one of the groups and history influences the other.
d) some historical event causes a failure of the random assignment procedure.

28) Which of the following threats to internal validity is most likely to be a problem in the
nonequivalent comparison group design?
a) history
b) maturation
* c) selection bias
d) all of the above are potential problems

29) Imagine a hypothetical study using a nonequivalent comparison group designed to assess the
effectiveness of an anti-drug, “Just Say No,” campaign. In this study students at school X receive
a semester-long series of “Just Say No” lectures. Students at school Y across town do not receive
the intervention. Unfortunately, during the semester a student at school Y died after an accidental
drug overdose. This incident may have influenced drug use at school Y (but not school X) and
consequently make it impossible to interpret our research findings. Which of the following
threats to internal validity is present in this study?
a) selection-instrumentation
b) selection-maturation
* c) selection-history
d) selection-attrition

30) Differential drop out rates in your experimental and control groups (selection-attrition) is
most likely to be a problem in which of the following designs:
a) time series
b) regression discontinuity
c) pretest-posttest control group

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

* d) nonequivalent comparison group

31) The ___________ is to ___________ as nonequivalent comparison group design is to
posttest-only design with nonequivalent groups.
a) one group pretest-posttest design; interrupted time series design
* b) interrupted time series design; one group pretest-posttest design
c) pretest-posttest control group design; interrupted time series design
d) interrupted time series design; regression discontinuity design

32) Sometimes it is difficult if not impossible to find a control group that is sufficiently
comparable to an experimental group to allow for a traditional two-group experimental design. In
these cases, one may want to use a time-series design
* a) which is a quasi-experimental design involving multiple measures of the dependent
variable on a single group, both before and after treatment.
b) which is an experimental design involving multiple measures of the dependent variable
on a single group, both before and after treatment.
c) which is an experimental design using multiple dependent measures on the
experimental group, then taking measures on control participants over time, as he or
she can find them.
d) staggers the time of presentation of the independent variable across participants.

33) Dr. Toris wants to test an incentive program to increase attendance in her classes. For the
first 8 weeks of the semester she does not use the program, and records attendance rates each
week. Then she institutes the incentive program for the last 8 weeks and continues to record
attendance rate each week. She detects an increase in attendance over the last 8 weeks. This is an
example of which experimental design?
a) one group pretest-posttest
* b) interrupted time series
c) non-equivalent control group
d) multiple baseline

34) Visual inspection of the pre- and post-treatment pattern of dependent measure scores is
typical of which quasi-experimental design?
* a) interrupted time series
b) non-equivalent control group
c) before-after
d) factorial

35) Visual inspection of the data in the interrupted time series design is not always sufficient.
With enough data points, the patterns of the dependent measures before and after can be
analyzed with
* a) an autoregressive integrated moving average model.
b) Cohen’s d.
c) an analysis of covariance.
d) correlation coefficients.

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

36) Of the major extraneous variables discussed in previous chapters, which is most likely a
source of confounding in an interrupted time-series design?
a) participant expectations
* b) history
c) attrition
d) regression

37) In using an interrupted time-series design, how does one determine whether the independent
variable had an effect?
* a) by looking for changes in the response pattern from pre- to post-treatment
b) by using the chi-square statistic
c) by comparing pre-treatment mean performance to post-treatment mean performance
d) by comparing the treatment group's data to a single observation of a control group

38) In a(n) design, the experimenter takes multiple measures on the dependent variable
both before and after treatment.
a) non-equivalent control group
* b) interrupted time series
c) multivariate
d) before-after

39) What is the "interruption" in an interrupted time series design?

* a) introduction of the independent variable
b) instrumentation failure
c) a period of at least two weeks
d) a brief period when measurements are taken.

40) A small town decides to ban cell phone texting while driving. Before putting the law into
effect, officials measure the amount of cell phone use among drivers by placing observers at a
busy intersection and recording the percentage of drivers using phones. They do this over 6
consecutive one-week intervals, each week being the middle week of a month (i.e., six months of
pretesting). Right after the law is passed, officials repeat the observational study for another six
months. Which of the following is true about the design of this study?
* a) it is an interrupted time series design
b) it is an example of a nonequivalent control group design
c) without a control group, there is no way to evaluate trends with this design
d) because there are multiple observations this would qualify as a regression
discontinuity design

41) The National Science Foundation has a policy of, each year, selecting the best young
scientists and giving them a 5 year grant to allow them to work on their program of research.
These scientists are selected based on the number of publications they have produced in the prior
year. All scientists who have published 4 or more articles in the prior years receive these grants.
The National Science Foundation wants to find out if this program is effective. The research
design that could be used to test the effect of this program is
a) posttest-only design.

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

* b) regression-discontinuity design.
c) time series design.
d) pretest-posttest design.

42) Which of the following is NOT one of the requirements of the regression-discontinuity
design.
a) group assignment must be based on a cutoff score
* b) the cutoff score should be located at one of the extreme ends of the distribution
c) the experimenter must control the assignment of participants to groups
d) all participants must be from the same population

43) If you have conducted a study using the regression-discontinuity design, how would you tell
if your treatment had an effect?
a) by looking at the data to see if the regression line of the posttest scores differs from the
regression line of pretest scores.
b) by statistically analyzing the data using Analysis of Variance.
c) by comparing the pretest scores with the posttest scores to see if there was a significant
change.
* d) by determining if there is a discontinuity in the regression line for the people above
and below the cutoff score.

44) What is the primary threat to internal validity that could confound the results of a regression
discontinuity study?
a) maturation
* b) history
c) attrition
d) testing

45) In a study designed to examine the effects of an intensive reading program, Vaughn et. al
(2009) screened 2nd grade students and separated good readers from poor readers. The poor
readers were given the intervention treatment. At the end of 26 weeks of intervention
significant improvement in the poor readers was noted. What type of experimental design
was employed in this study?
* a) regression discontinuity
b) nonequivalent comparison group
c) interrupted time series
d) median split design

46) Which of the following is NOT a quasi-experimental design?

a) interrupted time series
b) nonequivalent comparison group design
* c) pretest-posttest control group design
d) regression discontinuity design

Vocabulary

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

Define the following in psychological terms:

quasi-experimental design design components

nonequivalent comparison group design
selection-maturation effect
selection-instrumentation effect
selection-regression effect
second increasing treatment effect
interrupted time-series design
assignment measure
increasing treatment and control groups
selection-history effect
selection-attrition effect
first increasing treatment effect
crossover effect
regression discontinuity design

Essay questions

1) Distinguish between experimental and quasi-experimental research designs. When would a

researcher be more likely to use a quasi-experimental design?

2) Discuss the principles used to rule out rival explanation in quasi-experimental designs as
identified by Shadish et. al (2002) and presented in you text.

3) Describe a hypothetical example of a study using the nonequivalent comparison group design.
Be sure to describe all of the essential features of the design.

4) Explain how the nonequivalent comparison group design controls most of the threats to
internal validity (e.g., history, maturation, instrumentation etc) as compared to the posttest only
design. Explain why selection bias is still a problem with the nonequivalent comparison group
design.

5) Describe the differences between a nonequivalent comparison group design and a one-group
pretest-posttest design. For each, give an example of a situation in which one might have to use
that design rather than a true experimental design. What is the critical aspect of your example
that makes it impossible to use a true experimental design.

6) Discuss several different possible outcomes in a study using a nonexperimental comparison

group design. Describe the rival hypotheses that could explain these outcomes.

7) Describe the essential features of an interrupted time series design. Explain why this is a better
design than the one-group pretest-posttest design.

8) Draw two graphs of hypothetical outcomes for interrupted time series, one clearly showing an
effect of the treatment and the other no effect of the treatment. Explain how your hypothetical
data support your assertion of an effect and of no effect.

9) Describe the regression discontinuity design and when you would use it.

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

10) When using a regression discontinuity design, what kind of outcome or data would have to
exist to enable you to infer that the treatment was effective? What potential threats to internal
validity exist when using this design?

Classroom exercise suggestions

1) Quasi-experimental designs are often used to evaluate large scale programs that are designed
to benefit groups of individuals or entire communities. Examples include the DARE program to
reduce drug abuse or the Head Start program designed to help prepare children of poverty for
kindergarten. There is increasing pressure on these types of programs to demonstrate positive
outcomes. According to Rossi, Freeman, and Lipsey (1999) there are five different types of
program evaluation – 1) evaluation of need (are there problems that need to be addressed?); 2)
assessment of program theory (does the proposed program impact the identified needs?); 3)
program monitoring; 4) outcome evaluation (an assessment of whether or not the goals of the
program are being met); 5) efficiency assessment (are the outcomes worthwhile?). Although any
of the experimental methodologies may be used to evaluate programs it is likely that quasi
designs will be used because random assignment is typically not possible. More information
about program evaluation can be found in the sources below.

Campbell, D.T. (1969). Reforms as experiments. American Psychologist, 24, 409-429.

Posavac, E., & Carey, R. (1997). Program evaluation: Methods and case studies (5th

ed.). Englewood Cliffs, NJ: Prentice-Hall, Inc.

Shadish, W.R., Cook, T.D., & Campbell, D.T. (2002). Experimental and quasi-
experimental designs for generalized causal inference. Boston: Houghton Mifflin.

As suggested by Shaughnessy, Zechmeister, & Zechmeister (2009) one way to make program
evaluations more relevant to students is to relate them to institutional changes at your college or
university. Below is a brief summary of an activity suggested by Shaughnessy et. al.

Ask students to consider recent changes that have been made on your campus (e.g.,
changes in graduation requirements or student life issues). What evidence was used to
determine that a change needed to be made (e.g., comparison with other schools,
consultation with experts)? Did the administration use experiments or quasi-experiments
to determine whether or not to make these changes? Why do you think institutions do not
make use of experiments or quasi-experiments? Can you think of situations in which an
experimental approach could have been used?

2) You can extend the text discussion of interrupted time series by discussing several variations
on the basic design. These include reversal time-series designs which involves multiple removals
and re-introductions of the treatment – similar to single subject reversal designs. If similar
changes in behavior are seen each time the treatment is introduced then we can be more
confident in its effectiveness. Present examples to your students with the caveat that in many
instances it may unethical or simply not feasible to employ this type of design.

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

Another variation of the interrupted time series design involves the addition of a (nonequivalent)
control group that does not receive the experimental treatment. An example of this time of design
is a study conducted by Asbridge et. al (2009) who used a time series design to evaluate the
effect of a change in Ontario law that required the suspension of driver’s license for anyone
driving with a blood alcohol level over the legal limit. Monthly traffic fatalities were monitored
before and after the introduction of the new law. Additionally, two other provinces served as
controls are were monitored as well. The results of the study indicate that the introduction of the
new law resulted in a significant decrease in traffic fatalities. The inclusion of control provinces
rules out the influence of simple history effects as an explanation for changes in the traffic
fatalities. However it should be noted that these types of designs would not eliminate the
possibility of selection-history interactive effects.

Asbridge, M., Mann, R., Smart, R., Stoduto, G., Beirness, D., Lamble, R., et al. (2009).

The effects of Ontario's administrative driver's licence suspension law on total

driver fatalities: A multiple time series analysis. Drugs: Education, Prevention &

Policy, 16(2), 140-151. 

A third variation that improves the basic design is called interrupted time series with switching
replications. This type of design involves repeating the intervention at a different point in time
with a different sample. For example, a study by Palmgreen et. al (2001) assessed the
effectiveness of public service announcements (PSA) on reducing marijuana use among high
users. An interrupted time series design was used with the PSA introduced at different points in
time for two different counties in Kentucky. Similar reductions in marijuana use for the two
counties provides strong evidence for the efficacy of the PSAs.

Palmgreen, P., Donohew, L., Lorch, E., Hoyle, R., & Stephenson, M. (2001). Television

campaigns and adolescent marijuana use: Tests of sensation seeking

targeting. American Journal of Public Health, 91(2), 292-296. 

3) Describe the massed vs. distributed practice effects to the class. Tell them how these
differences could be applied to improve learning in school. Then challenge them to design a field
experiment in a real school. Let them come to grips with the idea that random assignment is not
always feasible. Then let them design the study as a quasi-experiment.

4) If this material is introduced early enough in the semester, students may wish to collect time
series data over the course of several weeks or at least several days. The important point of this
exercise is to challenge students to decide how to determine whether pre- and post-treatment data
are significant. Although a thorough treatment of time series statistical analyses are most likely
beyond the scope of the course, students can at least gain an appreciation of the issues involved.
This will be a particularly valuable introduction to the next chapter on single-case research
designs, as many of these designs are time-series based.

download full file at http://testbankinstant.com

 download full file at http://testbankinstant.com

5) Have students identify a treatment or a program that should be investigated using the
regression-discontinuity design. After they have identified this program, have them design the
study selecting such variables as the pretest and posttest measures and the cutoff score.

6) Investigate the effect of midterms on number of people in the library. Choose a study area of
your campus library and visit it at roughly the same time every day; be sure to use a popular
study time. Count the number of people in the study area (be sure to define the area's boundaries
clearly so that each observer is taking the same kind of data). Pretreatment days = days before
midterms, post-treatment = days during and after midterms. Do midterms have a temporary or
long-lasting effect on the number of people studying in the library?

download full file at http://testbankinstant.com