Non-Experimental Research

Lecture Outline
‣ Overview of Nonexperimental Research
‣ Correlational Research
‣ Quasi-Experimental Research
‣ Qualitative Research

Chapter 7
Psyc 201 — Research Methods in Psychology — Spring 2019 Psyc 201 — Research Methods in Psychology — Spring 2019

Nonexperimental Research When to Use Nonexperimental Research

‣ Nonexperimental research is defined as research that lacks a
manipulated independent variable, or lacks random sampling/
assignment to conditions, or both.
‣ Because of this, nonexperimental research can never establish
cause and effect.
‣ But, nonexperimental research is still very important.
‣ Milgram’s (1963) initial study; Loftus & Pickrell’s (1995) study on
implanting false memories; Cacioppo & Petty’s (1982) Need for
Cognition validation study; Rosenham’s (1973) “voices” study.
‣ We use nonexperimental research when it is NOT possible,
feasible, or ethical to
‣ manipulate an independent variable, and/or
‣ randomly assign participants to conditions or to orders of conditions.
‣ We also use it in cases where
‣ We study a single variable (as opposed to two variables)
‣ We study non-causal relationships between variables
‣ There may be a causal relationship, but we cannot manipulate the IV
‣ The research question may be broad and exploratory

Psyc 201 — Research Methods in Psychology — Spring 2019 Psyc 201 — Research Methods in Psychology — Spring 2019

131 • RESEARCH METHODS IN PSYCHOLOGY

some problems, but the lack of random assignment and experimental control fails to ad
for example, that a researcher finds two similar schools, starts an antibullying program
fewer bullying incidents in that “treatment school” than in the “control school.” There is no
Types of Nonexperimental Research Let’s Revisit Internal Validity
because clearly
‣ Nonexperimental research falls into three broad categories the number of bullying
‣ Internal incidents did not determine which school got the p
validity is the extent to which the independent variable
‣ Single-variable research lack of random assignment of children to observed
caused the schoolschangecould still
in the meanvariable.
dependent that students in the tre
‣ Milgram’s original research was just on obedience ‣ Internal validity tends to be highest with experimental research,
‣
from students in the control schoolfollowed
in some other way thatresearch,
by quasi-experimental couldandexplain thenon-
then finally difference in b
Correlational research
‣ Looking at the relationship between two variables without any attempt to experimental research, although there is some overlap
control extraneous variables
‣ Quasi-experimental research
‣ The experimenter manipulates the IV, but there is no random assignment to
conditions or orders of conditions.

Psyc 201 — Research Methods in Psychology — Spring 2019 Psyc 201 — Research Methods in Psychology — Spring 2019

Figure 7.1 Internal Validity of Correlation, Quasi-Experimental, and Experimental Studies. Experiments are gener
validity, quasi-experiments lower, and correlation studies lower still.
Advantages of the Non-Experimental Method
‣ Real life situations
‣ Prediction
‣ Study aspects of people such as personality characteristics
‣ Can study variables that would be unethical to manipulate in
a true experiment
Psyc 201 — Research Methods in Psychology — Spring 2019
Causal Statement Problems
Psyc 201 — Research Methods in Psychology — Spring 2019
What is Correlational Research?
‣ Correlational research is when two variables are measured, and the
statistical relationship between them is assessed.
‣ There is little or no effort to control extraneous variables
‣ Correlational research is often conducted when there is no reason to
believe there is a causal relationship between two variables
‣ Look at the relationship between a brief version of an extraversion test and a
long version of the test to see if the results are correlated with one another…
the short version does not cause the long version!
‣ The terms “independent variable” and “dependent variable” do not
really apply to correlational research.
Psyc 201 — Research Methods in Psychology — Spring 2019
Problems
‣ Direction of cause and effect
‣ Does X cause Y?
‣ Or does Y cause X?
‣ Uncontrolled third variables
‣ Perhaps Z causes both X and Y
‣ If so X and Y will appear to be related
Psyc 201 — Research Methods in Psychology — Spring 2019
Lecture Outline
‣ Overview of Nonexperimental Research
‣ Correlational Research
‣ Quasi-Experimental Research
‣ Qualitative Research
CORRELATIONAL RESEARC
Psyc 201 — Research Methods in Psychology — Spring 2019
idea that hassles cause symptoms, it is also consistent with the idea that symptoms cause hassles or that so
variable (e.g., neuroticism) causes both.
A common misconception among beginning researchers is that correlational research must involve tw
titative variables, such as scores on two extraversion tests or the number of hassles and number of sy
people have experienced. However, the defining feature of correlational research is that the two variables
sured—neither one is manipulated—and this is true regardless of whether the variables are quantitativ
egorical. Imagine, for example, that a researcher administers the Rosenberg Self-Esteem Scale to 50 A
university students and 50 Japanese university students. Although this “feels” like a between-subjects exp
it is a correlational study because the researcher did not manipulate the students’ nationalities. The same i
the study by Cacioppo and Petty comparing professors and factory workers in terms of their need for cog
is a correlational study because the researchers did not manipulate the participants’ occupations.
Figure 7.2 shows data from a hypothetical study on the relationship between whether people make a dai
things to do (a “to-do list”) and stress. Notice that it is unclear whether this design is an experiment or a
tional study because it is unclear whether the independent variable was manipulated. If the researcher ra
assigned some participants to make daily to-do lists and others not to, then it is an experiment. If the re
Types of Correlations
simply asked participants whether they made daily to-do lists, then it is a correlational study. The distin
important because if the study was an experiment, then it could be concluded that making the daily to
‣ A common misconception is that correlational research can
reduced participants’ stress. But if it was a correlational study, it could only be concluded that these varia
related. Perhaps being stressed has a negative effect on people’s ability to plan ahead (the directionality p
only be done with variables that are quantitative in nature
Or perhaps people who are more conscientious are more likely to make to-do lists and less likely to be
(e.g., ordinal or interval/ratio).
(the third-variable problem). The crucial point is that what defines a study as experimental or correlation
the variables being studied, nor whether the variables are quantitative or categorical, nor the type of grap
‣ This is not the case, as we can look at the correlation between
tistics used to analyze the data. It is how the study is conducted.
any two variables.
‣ Correlational research is defined

by the method, not by the 

variables.
Figure 7.2 Results of a Hypothetical Study on Whether People Who Make Daily To-Do Lists Experience Less Stress Than People W
Psyc 201 —
Not Make Research
Such Lists Methods in Psychology — Spring 2019
Data Collection in Correlational Research
Relationships Between Quantitative Variables Positive relationships
‣ Positive ‣ Increases in the values of one
‣ Negative variable are associated with
increases in the second variable
‣ Curvilinear
‣ Higher speech rates are associated
‣ No Relationship with higher levels of attitude
change.

Psyc 201 — Research Methods in Psychology — Spring 2019 Psyc 201 — Research Methods in Psychology — Spring 2019

Negative relationships Classify the Relationship

‣ Increases in the values of ‣ Are the following relationships positive or negative?
one variable are associated ‣ GPA and Alcohol Consumption
with decreases in the ‣ # of sexual partners and # of STDs
second variable
‣ SAT scores and first year GPA
‣ Increasing group size is ‣ # hrs TV viewed and # hrs studying
associated with lower
amounts of noise

Psyc 201 — Research Methods in Psychology — Spring 2019 Psyc 201 — Research Methods in Psychology — Spring 2019

Monotonic Functions Curvilinear Relationships

‣ Linear relationships are a more ‣ Increases in the values of one
specific example of a general variable are associated with
class of functions known as both increases and decreases of
monotonic functions. the second variable
‣ These functions do not change ‣ These are non-monotonic
the direction of their slope. functions
‣ Examples include log functions ‣ Anxiety can help increase
performance until it gets too high –
and exp functions.
then it decreases performance

Psyc 201 — Research Methods in Psychology — Spring 2019 Psyc 201 — Research Methods in Psychology — Spring 2019
 No Relationship Data Collection in Correlational Research
‣ When the two variables are ‣ The defining feature of correlational research is that neither
not related to one another, variable is manipulated, they are only measured.
then there is no relationship. ‣ Correlational research can be carried out in a variety of
‣ That is, the variables vary settings
independently from one ‣ Studies can be carried out in labs, correlating two different measures, or
another. can be carried out in more natural situations.
‣ The produces a graph with a ‣ Three alternative ways (i.e., other than a lab setting) of
flat line collecting data for correlational research are
‣ Naturalistic Observation; Archival Data; and Surveys

Psyc 201 — Research Methods in Psychology — Spring 2019 Psyc 201 — Research Methods in Psychology — Spring 2019

Naturalistic Observation Description and Interpretation of Data

‣ Naturalistic observation is sometimes called field work or
field observations.
‣ Researcher makes observations in a natural setting, over a
period of time, using a variety of techniques to collect
information.
‣ Used to describe and understand how people in a social or
cultural setting live, work, and experience the setting
‣ Usually qualitative data.
‣ Researcher must interpret what occurred.
‣ A form of coding must be developed to analyze results
‣ Generate hypotheses that help explain the data.
‣ Final report of results organized around a structure developed by the
researcher.

Psyc 201 — Research Methods in Psychology — Spring 2019 Psyc 201 — Research Methods in Psychology — Spring 2019

Participation in Naturalistic Observation Concealment in Naturalistic Observation

‣ Do you become and activate participant in the group, or do
you observe from the outside?
‣ Nonparticipant observer does not become an active part of the setting
(outsider)
‣ Participant observer assumes an active role (insider)
‣ Problem: Objectivity
‣ Should the researcher remain concealed or be open about the
research purposes?
‣ Concealed observation may be preferable to minimize reactivity
‣ Nonconcealed observation may be preferable from an ethics viewpoint.
‣ In practical terms, there are degrees of participation and
concealment.
‣ Researchers must decide on their role and how it will affect
the data collection

Psyc 201 — Research Methods in Psychology — Spring 2019 Psyc 201 — Research Methods in Psychology — Spring 2019
Defining the Scope of the Observation
‣ In naturalistic observation, there is an incredible wealth of
information.
‣ It is tempting to want to study it all, in case you miss
something important.
‣ In practical terms, we must limit the scope of the observations
to focus on one or two behaviours that are relevant to the
central issues of the study.
Psyc 201 — Research Methods in Psychology — Spring 2019
Archival Data
‣ Archival research involves using previously compiled
information to answer research questions.
Psyc 201 — Research Methods in Psychology — Spring 2019
Content Analysis
‣ Just like other methods of research, archival research requires
the operational definition of variables.
‣ In other words, it requires a coding system.
‣ The systematic analysis of existing documents is known as
content analysis.
Psyc 201 — Research Methods in Psychology — Spring 2019
Limits of Naturalistic Observation
‣ Cannot be used to study all issues.
‣ Less useful when studying well-defined hypotheses under
precisely specific conditions.
‣ Field research is difficult to do.
‣ A negative case analysis may be necessary.
‣ Examine the reasons why an observation is inconsistent with the
general observations.
Psyc 201 — Research Methods in Psychology — Spring 2019
Types of Archival Research
‣ Three types of archival research data:
‣ Statistical records
‣ Daily temperatures, sports records, crime data
‣ Survey archives
‣ Inter-university Consortium for Political and Social Research (ICPSR)
‣ Written and mass communication records
‣ Newspapers, Social Media
Psyc 201 — Research Methods in Psychology — Spring 2019
Problems with Archival Research
‣ There are two main problems with archival research:
‣ The desired records may be difficult to obtain.
‣ They may have been lost or destroyed
‣ The accuracy of the information may be suspect.
Psyc 201 — Research Methods in Psychology — Spring 2019
 Lecture Outline
‣ Overview of Nonexperimental Research
‣ Correlational Research
‣ Quasi-Experimental Research
‣ Qualitative Research
Psyc 201 — Research Methods in Psychology — Spring 2019
Nonequivalent Groups Design
‣ A nonequivalent group design is a between groups design in
which participants are not randomly assigned to groups
‣ A Patient vs. Control group study is a good example of a nonequivalent
group design.
‣ Researchers try to minimize the differences between groups
(and thus increase internal validity), but there are likely
important confounding variables that cannot be controlled.
Psyc 201 — Research Methods in Psychology — Spring 2019
Problems with Interpretations
‣ There are several problems when trying to interpret pretest-posttest designs
1. History
‣ Things may have happened between the pretest and posttest that the experimenter has
no control over
2. Maturation
‣ People grow, mature, and learn between the pretest and posttest
3. Regression to the Mean
‣ Extreme measures tend to move towards the mean with repeated measurements
4. Spontaneous Remission
‣ Many things improve over time without any intervention
Psyc 201 — Research Methods in Psychology — Spring 2019
Quasi-Experimental Research
‣ Quasi-experimental research resembles true experimental
research (the independent variable is manipulated), but
participants are not randomly assigned to conditions or orders
of conditions.
‣ When people form their own groups, it is sometimes referred to as an
ex post facto design
‣ Quasi-experimental removes the directionality issue, but
confounding variables are not removed, so cause and effect
cannot be established.
Psyc 201 — Research Methods in Psychology — Spring 2019
Pretest-Posttest Design
‣ In a pretest-posttest design, the dependent variable is
measured once before the treatment is implemented, and then
after the treatment.
‣ This helps establish a baseline response, and then allows
researchers to see changes in the scores (are the posttest
scores higher, lower, or the same as the pretest)
‣ This is not an experiment, however, because the order of
conditions is not randomized QUASI-EXPERIMENTAL RESEARCH • 14
Interrupted Time Series Design
A variant of the pretest-posttest design is the interrupted time-series design. A time series is a set of measur
Psyc 201ments
— Research
taken atMethods
intervalsin Psychology
over —time.
a period of SpringFor2019
example, a manufacturing company might measure its worker
productivity each week for a year. In an interrupted time series-design, a time series like this one is “interrupted
by a treatment. In one classic example, the treatment was the reduction of the work shifts in a factory from 1
5
hours to 8 hours (Cook & Campbell, 1979) . Because productivity increased rather quickly after the shortenin
of the work shifts, and because it remained elevated for many months afterward, the researcher concluded th
the shortening of the shifts caused the increase in productivity. Notice that the interrupted time-series design
like a pretest-posttest design in that it includes measurements of the dependent variable both before and after th
treatment. It is unlike the pretest-posttest design, however, in that it includes multiple pretest and posttest me
surements.
Figure 7.3 shows data from a hypothetical interrupted time-series study. The dependent variable is the numb
of student absences per week in a research methods course. The treatment is that the instructor begins public
taking attendance each day so that students know that the instructor is aware of who is present and who is absen
The top panel of Figure 7.3 shows how the data might look if this treatment worked. There is a consistently hig
number of absences before the treatment, and there is an immediate and sustained drop in absences after the trea
ment. The bottom panel of Figure 7.3 shows how the data might look if this treatment did not work. On averag
the number of absences after the treatment is about the same as the number before. This figure also illustrates a
advantage of the interrupted time-series design over a simpler pretest-posttest design. If there had been only on
measurement of absences before the treatment at Week 7 and one afterward at Week 8, then it would have looke
Interrupted Time Series Design
as though the treatment were responsible for the reduction. The multiple measurements both before and after th
treatment suggest that the reduction between Weeks 7 and 8 is nothing more than normal week-to-week variatio
‣ An extension of the pretest-
posttest design is called the
interrupted time series
design.
‣ In this study, participants are
measured multiple times
prior to the treatment, and
then multiple times after the
treatment.
Figure 7.3 A Hypothetical Interrupted Time-Series Design. The top panel shows data that suggest that the treatment caused a reduction in
absences. The bottom panel shows data that suggest that it did not.
Psyc 201 — Research Methods in Psychology — Spring 2019
5. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues in field settings. Boston, MA: Houghton Mifflin.
 Combination Designs
‣ A combination design uses both the pretest-posttest design
and the non-equivalent groups design.
‣ One group (the treatment group) is given the pretest, the
treatment, and then the posttest
‣ Another group (the control group) is given the pretest, no
treatment or a placebo, and then the posttest.
‣ The important measure is whether or not the treatment group
shows more improvement than the control group.
Psyc 201 — Research Methods in Psychology — Spring 2019
Quantitative Approaches
‣ Quantitative approaches
‣ Assign numerical values to responses and measures and then subject
the data to quantitative statistical analyses.
‣ Often uses such measures as percentages and averages.
Psyc 201 — Research Methods in Psychology — Spring 2019
The Purpose of Qualitative Research
‣ Quantitative research is very good at providing precise
answers to specific questions, and for drawing general
conclusions about behaviour.
‣ It is not so good at generating novel and interesting research
questions, or for describing detailed experiences of a specific
group at a specific time (a “lived experience”).
‣ Qualitative research fills this gap… it provides a rich and
detailed description of human behaviour in real world
contexts (a “thick description”).
Psyc 201 — Research Methods in Psychology — Spring 2019
Lecture Outline
‣ Overview of Nonexperimental Research
‣ Correlational Research
‣ Quasi-Experimental Research
‣ Qualitative Research
Psyc 201 — Research Methods in Psychology — Spring 2019
Qualitative Approaches
‣ Qualitative approaches
‣ Describe behaviour or findings based on themes that emerge from the
data.
‣ Data are nonnumerical and expressed in language and images.
‣ Generally less concerned with drawing general conclusions about
human behaviour, and more concerned with the detailed understanding
of the experience of the participant.
Psyc 201 — Research Methods in Psychology — Spring 2019
Data Collection in Qualitative Research
‣ Data collection in Qualitative research can involve naturalistic
observations, archival data, artwork, and many other types of data
collection
‣ One of the most common types of data collection (especially in
Psychology) is the interview.
‣ Interviews can either be conducted individually or within a focus group.
‣ Another form of data collection is called participant observation,
where the researcher becomes an active participant in the group or
situation they are studying.
‣ There are issues with objectivity and possible ethical issues with this approach.
Psyc 201 — Research Methods in Psychology — Spring 2019
Data Analysis in Qualitative Research
‣ The difference between quantitative and qualitative research is not
so much in the methods used to collect the data, but in the methods
used to analyze the data.
‣ If we collect data using unstructured interviews, but then code the responses
and provided descriptive and inferential statistics, then it is quantitative
research.
‣ Analysis of qualitative research often uses techniques such as
grounded theory.
‣ The data is collected, and then a theory is developed based on the data.
‣ Repeated ideas are identified and theoretical narrative is developed
Psyc 201 — Research Methods in Psychology — Spring 2019
Quantitative and Qualitative Approaches
‣ Often use both approaches in the same study.
‣ This is called mixed-methods research
‣ One approach is to use qualitative research to generate
hypotheses, and then use quantitative research to test the
hypotheses.
‣ A second approach is to use triangulation, which uses both
techniques simultaneously to try to come to the same conclusion.
‣ Both approaches are valuable and provide us with different ways
of understand or interpreting the same situation.
Psyc 201 — Research Methods in Psychology — Spring 2019