Bahir Dar University

Faculty of Humanities

Department of English Language and Literature

Course Title: Advanced Research Methods (TEFL 701)

A term paper on quasi experimental research design

By: Dagninet Gebey

Submitted to: Birhanu S. (PhD)

Yenus N. (PhD)

November, 2022

Bahir Dar, Ethiopia

Table of Contents
1. Quasi experimental research design....................................................................................................1
1.1 Introduction.......................................................................................................................................1
1.2 Definitions of quasi experimental research.......................................................................................1
1.3 Why quasi-experiments?...................................................................................................................4
1.4 Types of quasi experimental research design....................................................................................4
1.4.1 One group designs (posttest only and pretest – posttest)..........................................................4
1.4.1.1 One group posttest design only…………………………………………………………………………………………6

1.4.1.2 One group pretest- posttest design……………………………………………………………………………………7

1.4.2 Nonequivalent Groups Design....................................................................................................6

1.4.3 Interrupted Time Series Design..................................................................................................7
1.4.4 Combination Designs..................................................................................................................8
1.5 Differences between quasi-experiments and true experiments........................................................8
1.6 Advantages and Disadvantages of Quasi-Experimental Design.........................................................9
1.7 Conclusion.........................................................................................................................................9
References.............................................................................................................................................11
1. Quasi experimental research design
1.1 Introduction
Researchers within the field of applied linguistics have long used experiments to investigate
cause–effect relationships regarding the use and learning of second languages. In experimental
research, one or more variables are altered and the effects of this change on another variable are
examined. This change or experimental manipulation is usually referred to as the treatment.
Researchers typically draw upon either experimental or quasi-experimental research designs to
determine whether there is a causal relationship between the treatment and the outcome.

This paper outlines key features and provides examples of common quasi-experimental research
designs. It also makes recommendations for how experimental designs might best be applied and
utilized within applied linguistics research. Thus, this paper will discuss about Quasi-
experimental designs are commonly used in research but knowing how to use them can be a
challenging.

1.2 Definitions of quasi experimental research

Many writers define quasi experimental research design in different ways. For instance,
according to Cook & Campbell as cited in Price, P., Jhangiani, R., & Chiang, I. (2015) , the
prefix quasi means resembling. Thus quasi-experimental research is research that resembles
experimental research but is not true experimental research. Although the independent variable is
manipulated, participants are not randomly assigned to conditions or orders of conditions

In addition, Quasi-experiments are studies that aim to evaluate interventions but that do not use
randomization. Similar to randomized trials, quasi-experiments aim to demonstrate causality
between an intervention and an outcome. Quasi-experimental studies can use pre intervention
and post intervention measurements as well as non-randomly selected control groups.

John F. Stevenson (2020) stated that a quasi-experimental research design is the use of methods
and procedures to make observation in a study that is structured similar to an experiment, but the
conditions and experiences of the participants lack some control because the study lacks random
assignment, includes a preexisting factor (i.e. a variable that is not manipulated), or does not
include a comparison or control group.

In addition, according to John F. Stevenson (2020), the quasi experimental design is structured
similar to an experiment, except that is the design does one or both of the following:

 It includes a quasi-independent variable

1
  It lacks an appropriate or equivalent comparison or control group.

Moreover, John F. Stevenson (2020) reported that a quasi-independent variable is a preexisting

variable that is often characteristic inherent to an individual, which differentiates the groups or
conditions being compared in a research study. Because the levels of variable are preexisting, it
is not possible to randomly assign participants to groups. Thus, a quasi-experimental research
resembles an experiment but includes a quasi-independent variable and/or lacks a control group.

Another definition is given by William R. Shadish, Thomas D. Cook, and Donald T. Campbell
(2002). According to them, quasi-experiments share with all other experiments a similar
purpose-to test descriptive causal hypothesis about manipulable causes-as well as many
structural details, such as the frequent presence of control groups and pretest measures, to
support a counterfactual inference about what would have happened in the absence of treatment.
But, by definition, quasiexperiments lack random assignment. Assignment to conditions is by
means of self-selection, by which units choose treatment for themselves, or by means of
administrator selection, by which teachers, bureaucrats, legislators, therapists, physicians, or
others decide which persons should get which treatment.

William et al. (2002) also stated that in quasi-experiments, the cause is manipulable and occurs
before the effect is measured. However, quasi-experimental design features usually create less
compelling support for counterfactual inferences. For example, quasi-experimental control
groups may differ from the treatment condition in many systematic (nonrandom) ways other than
the presence of the treatment many of these ways could be alternative explanations for the
observed effect, and so researchers have to worry about ruling them out in order to get a more
valid estimate of the treatment effect. By contrast, with random assignment the researcher does
not have to think as much about all these alternative explanations. If correctly done, random
assignment makes most of the alternatives less likely as causes of the observed treatment effect
at the start of the study.

Moreover, according to Aek Phakiti (2014), Quasi-experimental research is classified under

experimental research because it aims to examine causal-like effects. The term quasi is Latin for
almost. We consider doing quasi-experimental research when we cannot achieve complete
control over potential confounding variables that can be threats to the internal validity of the
study. As discussed above, a true experimental research design randomly assigns participants
into groups. However, we cannot do random assignments in quasi-experimental research. There
are numerous real-life situations in language learning where random assignment is impossible.
For example, there are intact classes that cannot be rearranged since the governing institution
may have a policy to put students doing the same academic majors together in one class. Another
situation may be that it is not ethical or practical to mix disadvantaged learners with high-ability
learners. Yet another would be that girls and boys in some cultures may not study in the same
classroom. It is therefore not possible to reassign them randomly. Since random assignment

2
cannot be done in quasi-experimental research, it is important to recognize that several potential
threats from existing confounding variables (e.g. the characteristics of learners,
disciplinaryspecific knowledge, time of day, and teachers) are present. These threats make it
difficult to make valid causal-like inferences because this can be achieved only when there are
no other rival explanations (i.e. other plausible alternative explanations of the same finding). Yet
quasi-experimental research can still yield some useful insights into a causal-like relationship.
Usually such findings are treated as suggestive and prompt a more sophisticated randomized
experimental research design.

Finally, according to Shadish, Cook, & Campbell as cited in Charles S. Reichardt (2019),
Comparisons used to estimate the effects of treatments can be partitioned into two types:
randomized experiments and quasi-experiments. The difference has to do with how people (or
other observational units such as classrooms, schools, or communities) are assigned to treatment
conditions. In randomized experiments, study units are assigned to treatment conditions at
random. Assigning treatment conditions at random means assigning treatments based on a coin
flip, the roll of a die, the numbers in a computer-generated table of random numbers, or some
equivalently random process. In quasi-experiments, units are assigned to treatment conditions in
a nonrandom fashion, such as by administrative decision, self-selection, legislative mandate, or
some other nonrandom process. For example, administrators might assign people to different
treatment conditions based on their expectations of which treatment would be most effective for
people with different characteristics. Alternatively, people might self-select treatments based on
which treatment appears most desirable or most convenient.

To summarize, quasi-experimental research designs examine whether there is a causal

relationship between independent and dependent variables. Simply defined, the independent
variable is the variable of influence and the dependent variable is the variable that is being
influenced (Loewen & Plonsky as cited in John Rogers and Andrea Révész (2019)). In other
words, the independent variable is expected to bring about some variation or change in the
dependent variable. For example, in a study examining the impact of oral corrective feedback on
grammatical development, corrective feedback will serve as the independent variable and
grammatical development as the dependent variable.

In general, quasi-experiments are most likely to be conducted in field settings in which random
assignment is difficult or impossible. They are often conducted to evaluate the effectiveness of a
treatment.

3
1.3 Why quasi-experiments?
Randomized experiments are generally considered the gold standard for estimating effects, with
quasi-experiments being relegated to second-class status (Boruch et al. as cited in Charles S.
Reichardt (2019)). So why do we need quasi-experiments when randomized experiments hold
such an exalted position? One answer is that randomized experiments cannot always be
implemented, much less implemented with integrity (West, Cham & Liu as cited in Charles S.
Reichardt (2019)). Implementing a randomized experiment would often be unethical. For
example, it would be unethical to assess the effects of HIV by assigning people at random to be
infected with the virus. Nor would it be ethical for researchers to randomly assign children to be
physically abused or for couples to divorce. Even if we were to randomly assign children to be
physically abused, we might not be sufficiently patient to wait a decade or two to assess the
effects when the children become adults. Similarly, it is impractical, if not impossible, to assess
the effects of such massive social interventions as recessions or wars by implementing them at
random. Even when random assignment would be both ethical and physically possible, it can be
difficult to convince both administrators and prospective participants in a study that randomized
experiments are desirable. Or funding agencies might require that investigators serve all of those
most in need of a presumed ameliorative intervention, so that none of those most in need could
be relegated to a presumed less effective comparison condition. People sometimes perceive
random assignment to be unfair and therefore are unwilling to condone randomized experiments.
Sometimes, too, data analyses are conducted after the fact when a randomized experiment has
not been implemented or when a randomized experiment was implemented but became degraded
into a quasi-experiment.

1.4 Types of quasi experimental research design

According to William et al. (2002), there are four categories of quasi experimental research
design.

1. One group designs (posttest only and pretest – posttest)

2. Nonequivalent Groups Design
3. Interrupted Time Series Design
4. Combination Designs

1.4.1 One group designs (posttest only and pretest – posttest)

When only one group observed, the study lacks comparison group and so does not demonstrate
cause. These designs may also be referred to as “pre-experimental” designs. Two types of one
group designs are the following:

1.4.1.1 One group posttest design only

A One group posttest design only is a quasi-experimental research design in which a dependent
variable is measured for one group of participants following a treatment. For example, after a
teacher provides instruction on the steps of effective paragraph writing (the treatment), she or he

4
may record the number of paragraphs wrote effectively on a practice worksheet (the dependent
variables) to test their learning.
The major limitation of this design is that it lacks a comparison or control group. Consider, for
example, the number of paragraphs wrote effectively on a practice worksheet.
If many learners wrote effective paragraph following the instruction, can we conclude the
instruction is effective? How can we know for sure if the learners write effective paragraph even
without the instruction? We cannot know this because we have nothing to compare this outcome
to; we have no comparison/ control group. Hence, the design is susceptible to many threats to
internal validity, such as history effects (unanticipated events that can co-occur with the exam)
and maturation effects (natural changes in learning).

1.4.1.2 One group pretest- posttest design

A One group pretest- posttest design is a quasi-experimental research design in which the same
dependent variables are measured in one group of participants before (pretest) and after (posttest)
a treatment is administered.

In a pretest-posttest design, the dependent variable is measured once before the treatment is
implemented and once after it is implemented. Imagine, for example, a researcher who is
interested in the effectiveness of an antidrug education program on elementary school students’
attitudes toward illegal drugs. The researcher could measure the attitudes of students at a
particular elementary school during one week, implement the antidrug program during the next
week, and finally, measure their attitudes again the following week. The pretest-posttest design is
much like a within-subjects experiment in which each participant is tested first under the control
condition and then under the treatment condition. It is unlike a within-subjects experiment,
however, in that the order of conditions is not counterbalanced because it typically is not possible
for a participant to be tested in the treatment condition first and then in an “untreated” control
condition.

If the average posttest score is better than the average pretest score, then it makes sense to
conclude that the treatment might be responsible for the improvement. Unfortunately, one often
cannot conclude this with a high degree of certainty because there may be other explanations for
why the posttest scores are better. Other things might have happened between the pretest and the
posttest. Perhaps an antidrug program aired on television and many of the students watched it, or
perhaps a celebrity died of a drug overdose and many of the students heard about it. Another
category of alternative explanations goes under the name of maturation. Participants might have
changed between the pretest and the posttest in ways that they were going to anyway because
they are growing and learning. If it were a yearlong program, participants might become less
impulsive or better reasonable and this might be responsible for the change.

5
1.4.2 Nonequivalent Groups Design
In nonequivalent group design, the researcher chooses existing groups that appear similar, but
where only one of the groups experiences the treatment. In a true experiment with random
assignment, the control and treatment groups are considered equivalent in every way other than
the treatment. But in a quasi-experiment where the groups are not random, they may differ in
other ways—they are nonequivalent groups.

Creswell (2014) describes that in this design, a popular approach to quasi-experiments, the
experimental Group A and the control Group B are selected without random assignment. Both
groups take a pretest and posttest. Only the experimental group receives the treatment.

Group A O ____________X ____________O

_______________________

Group B O–_________O

When using this kind of design, researchers try to account for any confounding variables by
controlling for them in their analysis or by choosing groups that are as similar as possible. This is
the most common type of quasi-experimental design.

Imagine, for example, a researcher who wants to evaluate a new method of teaching vocabulary
to third graders. One way would be to conduct a study with a treatment group consisting of one
class of third-grade students and a control group consisting of another class of third-grade
students. This design would be a nonequivalent groups design because the students are not
randomly assigned to classes by the researcher, which means there could be important
differences between them. For example, the teachers’ styles, and even the classroom
environments, might be very different and might cause different levels of achievement or
motivation among the students. If at the end of the study there was a difference in the two
classes’ knowledge of vocabulary, it might have been caused by the difference between the
teaching methods—but it might have been caused by any of these confounding variables.

Of course, researchers using a nonequivalent groups design can take steps to ensure that their
groups are as similar as possible. In the present example, the researcher could try to select two
classes at the same school, where the students in the two classes have similar scores on a
standardized English test and the teachers are the same sex, are close in age, and have similar
teaching styles. Taking such steps would increase the internal validity of the study because it
would eliminate some of the most important confounding variables. But without true random
assignment of the students to conditions, there remains the possibility of other important
confounding variables that the researcher was not able to control.

For example, there is a research on this type which is entitled “A Quasi-Experimental Study On
Using Short Stories: Statistical And Inferential Analyses On The Non-English Major University

6
Students’ Speaking And Writing Achievements”. This research was conducted to find out
whether or not using short stories significantly improve the speaking and writing achievements.
A quasi-experimental study of non-equivalent pretest-posttest control group design or
comparison group design was used in this research. The population of this research was the all
first semester undergraduate students of urban and regional planning study program of Indo
Global Mandiri University. Forty students were selected as the sample by using purposive
sampling technique in which each group consisted of 20 students, respectively. The treatment
was given for 14 meetings. This research was primarily concerned on the quantitative data in the
form of the students’ speaking and writing scores. Rubrics were used to measure the students’
speaking and writing achievements. The findings showed that (1) there was a significant
improvement on the non-English major university students’ speaking and writing achievements
after being taught by using short stories than those who were not, (2) there was also a significant
mean difference on the non-English major university students’ speaking and writing
achievements between the experimental and control groups, (3) the aspect of speaking and
writing skill gave high contribution on the students’ speaking and writing achievements in the
experimental group. The highest contribution of speaking skill (aspects) toward the speaking
achievement (total) was fluency. Meanwhile the highest contribution of writing skill (aspects)
toward the writing achievement (total) was vocabulary. Hence, it could be concluded that using
short stories significantly improve the students’ speaking and writing achievements (Jaya Nur
Iman, 2017).

1.4.3 Interrupted Time Series Design

Interrupted time series quasi experimental research designs involve many observations made
before and after a treatment. In other words, time series quasi experimental research design is a
set of measurements taken at intervals over a period of time. For example, a manufacturing
company might measure its workers’ productivity each week for a year. In an interrupted time
series-design, a time series like this one is “interrupted” by a treatment. In one classic example,
the treatment was the reduction of the work shifts in a factory from 10 hours to 8 hours (Cook &
Campbell as cited in Price et al (2015)). Because productivity increased rather quickly after the
shortening of the work shifts, and because it remained elevated for many months afterward, the
researcher concluded that the shortening of the shifts caused the increase in productivity. Notice
that the interrupted time-series design is like a pretest-posttest design in that it includes
measurements of the dependent variable both before and after the treatment. It is unlike the
pretest-posttest design, however, in that it includes multiple pretest and posttest measurements.

The dependent variable is the number of student absences per week in a research methods
course. The treatment is that the instructor begins publicly taking attendance each day so that
students know that the instructor is aware of who is present and who is absent. There are a
consistently high number of absences before the treatment, and there is an immediate and
7
sustained drop in absences after the treatment. On average, the number of absences after the
treatment is about the same as the number before. This figure also illustrates an advantage of the
interrupted time-series design over a simpler pretest-posttest design. If there had been only one
measurement of absences before the treatment at Week 7 and one afterward at Week 8, then it
would have looked as though the treatment were responsible for the reduction. The multiple
measurements both before and after the treatment suggest that the reduction between Weeks 7
and 8 is nothing more than normal week-to-week variation.

1.4.4 Combination Designs

A type of quasi-experimental design that is generally better than either the nonequivalent groups
design or the pretest-posttest design is one that combines elements of both. There is a treatment
group that is given a pretest, receives a treatment, and then is given a posttest. But at the same
time there is a control group that is given a pretest, does not receive the treatment, and then is
given a posttest. The question, then, is not simply whether participants who receive the treatment
improve but whether they improve more than participants who do not receive the treatment.

Imagine, for example, that students in one school are given a pretest on their attitudes toward
drugs, then are exposed to an antidrug program, and finally are given a posttest. Students in a
similar school are given the pretest, not exposed to an antidrug program, and finally are given a
posttest. Again, if students in the treatment condition become more negative toward drugs, this
change in attitude could be an effect of the treatment, but it could also be a matter of history or
maturation. If it really is an effect of the treatment, then students in the treatment condition
should become more negative than students in the control condition. But if it is a matter of
history (e.g., news of a celebrity drug overdose) or maturation (e.g., improved reasoning), then
students in the two conditions would be likely to show similar amounts of change. This type of
design does not completely eliminate the possibility of confounding variables, however.
Something could occur at one of the schools but not the other (e.g., a student drug overdose), so
students at the first school would be affected by it while students at the other school would not.

1.5 Differences between quasi-experiments and true experiments

There are several common differences between true and quasi-experimental designs. This table
tries to summarize the difference between true experimental design and quasi experimental
design.

True experiments Quasi-experiments

Participants are assigned randomly to the Participants and not randomly assigned to the
experimental groups. experimental groups.
Participants have an equal chance of getting Participants are categorized and then put into a
into any of the experimental groups. respective experimental group.

Researchers design the treatment participants Researchers do not design a treatment.

8
will go through.
There are no various groups of treatments.
Includes control groups and treatment groups
It does not include a pre-test
Researchers study the existing groups of
treatments received
Does not necessarily require control groups,
apart from the fact they are generally used
It includes a pre-test

1.6 Advantages and Disadvantages of Quasi-Experimental Design

Quasi-experimental design has some advantages and disadvantages you’ll need to consider when
designing your research.
The followings are the advantages of quasi-experimental design:
● has a higher external validity than true experimental design, as it usually involves real-world
scenarios
● allows you to control for unexpected, confounding variables, resulting in a higher internal
validity than other non-experimental methods of research
● enables the study of cause-and-effect relationships without the ethical issue of denying a
treatment to those who may benefit from it
● does not require access to large-scale funding and other practical concerns, as the treatment
has already been issued by others
The disadvantages of quasi-experimental design, however, include:
● Lower internal validity than found in true experiments, as it’s more difficult to account for
all confounding variables without using random assignment
● The necessary data required for research potentially being inaccurate, outdated, or difficult to
access

1.7 Conclusion
Generally, Quasi-experimental research involves the manipulation of an independent variable
without the random assignment of participants to conditions or orders of conditions. Quasi-
experimental research eliminates the directionality problem because it involves the manipulation
of the independent variable. It does not eliminate the problem of confounding variables,
however, because it does not involve random assignment to conditions. For these reasons, quasi-
experimental research is generally higher in internal validity than correlational or other non-
experimental studies but lower than true experiments.

Quasi experimental research design has different categories. Among the important types are one
group designs (posttest only and pretest – posttest), nonequivalent groups design, interrupted
time series design, and combination designs.

Hence, both randomized and quasi-experiments estimate the effects of treatments. Some writers
have believed that only randomized experiments can satisfactorily fulfill that purpose. Such
writers fail to recognize both the frequent limitations of randomized experiments and the
potential benefits of quasi-experiments. While randomized experiments are to be preferred to

9
quasi-experiments in many instances, randomized experiments cannot always be implemented,
especially in field settings—in which case quasi-experiments are the only option. Even when
randomized experiments can be implemented, there can be benefits to quasi-experiments. Hence,
researchers need to understand how to conduct quasi-experiments if they are to be able to
estimate treatment effects when confronted with the diverse array of research needs and
circumstances.

10
References
1. Aek Phakiti (2014). Research methods in linguistics: experimental research methods in
language learning. London, Bloomsbury publishing plc.
2. Charles S. Reichardt (2019). Quasi experimentation: A guide to design and analysis.
New York, the Guilford press.
3. Creswell, J. W. (2014). Research design: qualitative, quantitative, and mixed methods
approach (4th ed.). SAGE Publications, Inc.
4. John F. Stevenson (2020). Quasi experiment designs. Sage publication, Inc.
5. John Rogers and Andrea Révész (2020). The Routledge Handbook of Research Methods
in Applied Linguistics. https://www.researchgate.net.
6. Price, P., Jhangiani, R., & Chiang, I. (2015). Research Methods of Psychology – 2nd
Canadian Edition. Victoria, B.C.: Campus. Retrieved from https://opentextbc.ca/res
7. William R. Shadish, Thomas D. Cook and Donald T. Campbell (2002). Experimental
and quasi experimental designs for generalized casual inference. Boston, Houghton
Mifflin Company.
8. Jaya Nur IMAN (2017). A Quasi-Experimental Study on Using Short Stories:
Statistical and Inferential Analyses on the Non-English Major University Students’
Speaking and Writing Achievement: International Journal of Languages’ Education
and Teaching, 5 (1). https://www.researchgate.net.
9. Yassine Benhadj (2021). A Quasi-experimental Study on the Impact of Blended
Learning on EFL Students’ Language Proficiency: International Journal of Language
and Literary Studies, 3 (3). https://ijlls.org.

11