In-Depth: Quasi-Experimental Designs

Introduction to Research Methodology & Inleiding Methodenleer

Dr. Michèle B. Nuijten

A. The city of Ottawa, Canada, passed a law banning smoking in bars and restaurants. Along
with the bar and restaurant owners, a group of researchers wanted to know if the smoking
ban would hurt sales figures (Luk, Ferrence, & Gmel, 2006). They kept track of bar and
restaurant sales every month, starting two years before the smoking ban and continuing
for ten months after the smoking ban. They noticed that although sales fluctuated greatly
from month to month, there was no overall change in the sales trend after the smoking ban
started.

a. Determine whether the study is a nonequivalent control group design, an interrupted

time-series design, or a nonequivalent groups interrupted time-series design.
b. Graph the results of the study, according to the results in the description. (You may not
have exact values to plot, but you can indicate which groups are higher or lower.)
c. What causal statement is the researcher trying to make, if any? Is it appropriate? If the
researcher is making a causal statement, use the results and design to interrogate the
study’s internal validity. If you notice an internal validity flaw, can you redesign the study
to remove the flaw?
d. If the researcher has a null effect, ask questions to interrogate the null effect.
e. Ask one question to address construct validity and one to address external validity.

B. How are interrupted time-series designs similar to and different from stable-baseline
designs? How are nonequivalent groups interrupted time-series designs similar to
multiple-baseline designs? In addition to other factors, be sure to think about how each
design establishes internal validity.

C. Researchers Langer and Rodin (1976) were interested in improving the quality of life for
nursing home residents. They reasoned that in many nursing homes, residents had very little
say in how they spent their days. The researchers suspected that having more control over
their daily schedules would improve the quality of life for residents. They studied two floors
of a nursing home: Floor 6 and Floor 3. Administrators at the nursing home assured them
that people on the two floors had very similar levels of medical problems and were of similar
ages.

For a baseline period, Langer and Rodin asked nurses and doctors to record the health and
activity levels of the residents on both floors. Then they introduced an intervention:
Residents on Floor 6 listened to a “pep talk” about all of the activities and services they could
take advantage of if they chose to. In addition, residents on this floor were offered a potted
plant and were told that they could take care of it however they wanted to. And they were
offered a new movie night option: They could choose either Tuesday or Friday, or no
movie, depending on their preference. In contrast, residents on Floor 3 listened to a speech,
too; while this speech was positive, it did not emphasize taking personal control. In addition,
residents on this floor were offered a potted plant, but they were told that the nurses would
take care of it. And residents on this floor were offered a new movie night as well, but they
were told which night they would be watching the movie.

Eighteen months later, the doctors and nurses rated the residents’ health again. Although
residents of the two floors were equal at the baseline measurement, the residents of Floor 3
showed dramatic decreases in health and activity level at the follow-up. In addition, residents
of Floor 6 actually showed an improvement in mental alertness and activity level. Twice as
many residents of Floor 3 had died after 18 months, compared with the residents of Floor 6.
a. Determine whether the study is a nonequivalent control group design, an interrupted
time-series design, or a nonequivalent groups interrupted time-series design.
b. Graph the results of the study, according to the results in the description. (You may not
have exact values to plot, but you can indicate which groups are higher or lower.)
c. What causal statement is the researcher trying to make, if any? Is it appropriate? If the
researcher is making a causal statement, use the results and design to interrogate the
study’s internal validity.
d. If you notice any internal validity flaws, can you redesign the study to remove the flaw?
e. Ask one question to address construct validity and one to address external validity.