280 NON‐EXPERIMENTAL DESIGN I: SURVEY METHODS

themselves predict college success, as do SAT scores. Together, however, they predict better
than either one by itself (Sprinthall, 2000). As students, you might find the following Research
Example interesting. It is a good example of both simple, linear regression and multiple regres
sion and it might give you some insight into how the passion and motivation you may feel for
studying psychology can predict how engaged or overwhelmed you feel.

Research Example 26 – Regression and Multiple Regression

Stoeber, Childs, Hayward, and Feast (2011) used several different questionnaires to examine the
relationships between passion for studying, academic engagement, and burnout in college stu
dents. First, using simple, linear regression, they wanted to see if degrees of passion could
predict scores on the measures of engagement and burnout. Thus, passion was their predictor
variable, and engagement and burnout were criterion variables. They used two separate linear
regression analyses for each criterion variable. Passion was conceptualized as two different
types: harmoni ous passion and obsessive passion. Harmonious passion is characterized as the
passion one feels when an individual engages in activities freely and thus controls the passion.
Unlike harmonious passion, obsessive passion occurs when the individual feels personal
pressure to engage in activi ties and thus feels less control of the passion. As you might guess,
unlike harmonious passion, obsessive passion tends to create conflict with other life domains
(such as life with family and friends). It seems easy to imagine situations in which students may
feel either harmonious pas sion or obsessive passion about studying for psychology. For
example, harmonious passion might be thought of as loving psychology and wanting to study it
freely, without pressure to do so – studying psychology for the pure love it. Obsessive passion
might be thought of as still loving psychology, but feeling internal pressure to get good grades
or go to graduate school or the like – studying psychology to be the best student possible, at
whatever cost.
Stoeber et al. (2011) wanted to predict students’ feelings of engagement in their studies and
academic burnout from their passion. To do so, they asked 105 college students to complete
three questionnaires that measured passion, engagement, and burnout. Items on the surveys
asked stu dents to rate their level of agreement on various items on 7‐point Likert scales from 1
(strongly disagree) to 7 (strongly agree). For the criterion variables of engagement and burnout,
the sur veys allowed Stoeber et al., to subdivide academic engagement into three components:
how dedi cated one was to their work (dedication), how vigorously one worked (vigor), and how
absorbed one became in one’s work (absorption). Burnout was also subdivided into three
components: feeling exhausted about studying (exhaustion), feeling pessimistic or skeptical
about studying (cynicism), and feeling unable to produce the effects they want from their efforts
(inefficacy). They first ran correlations between all their measures and after finding significant
correlations between passion and academic engagement and burnout, they ran two separate
simple, linear regression models, one for each criterion variable, engagement and burnout. They
also found some degree of correlations between harmonious and obsessive passion, as their
overall measure of passion tapped into both forms of passion. The regression analysis, however,
enables research ers to examine the unique contributions (i.e., unique variance) of a variable by
controlling for shared variance across variables. Thus, by using regression, Stoeber et al.
controlled for shared variance (remember r2?) across the two types of passion and tested if
harmonious and obsessive passion would uniquely predict academic engagement and burnout.
And, they did! Harmonious passion uniquely predicted all three measured aspects of academic
engagement: dedication, vigor, and absorption. Obsessive passion uniquely predicted only vigor
and absorption but not dedication. For academic burnout, harmonious passion was a unique
negative predictor of all three aspects of burnout, where higher levels of harmonious passion
predicted lower levels of exhaustion, cynicism, and inefficacy. In contrast, obsessive passion did
not predict exhaustion and cynicism features of burnout, but it did predict inefficacy, in which
higher levels obsessive passion predicted lower levels of inefficacy.
 Analyzing Data from Non‐Experimental Methods 281

You might be wondering if motivation played some sort of role in students’ passion for study
ing. Maybe students who are more motivated also felt more passionate about studying. Well,
Stoeber et al., (2011) also controlled for motivation using multiple regression. Thus, passion and
motivation were predictor variables and engagement and burnout were criterion variables. Like
the linear regression analyses, they had to run separate multiple regression analyses – one for
each criterion variable.8 To measure motivation, students wrote down two personal goals they
wished to achieve when studying. They then rated on a 7‐point Likert scale their level of
agreement of the reason for each goal. For example, students rated to how much they agreed
that their goal was intrinsically motivated or extrinsically motivated, which was then coded by
Stoeber et al. as autonomous or controlled motivation, respectively. The results from the
multiple regression analyses were virtually identical to the linear regression analyses, meaning
that neither type of motivation factored into the relationships between passion, achievement,
and burnout demon
strated by the first set of regression analyses.
Stoeber et al.’s (2011) results were the first that demonstrated the relationships between pas
sion, engagement, and burnout in a college setting, and their work serves as an extension of
applied research in the field of industrial‐organizational psychology on employee burnout. The
external validity of the study should be considered, as the authors caution that their results
should not be generalized to all college students, in large part because they tested a lot more
women (n = 93) than men (n = 12). Such research certainly opens the door for more “what’s
next” type of thinking, including asking empirical questions about gender differences in passion
and moti vation, and whether passion and engagement are related to academic success. For you
as stu dents, harmonious passion is probably a healthier form of passion than obsessive passion.
Obsessive passion relates to more pressures to perform and burnout, which may sour you on
studying psychology. Harmonious passion for psychology can be better for you in terms of feel
ing more involved and in control your love of psychology.
One final point about a regression analysis is both procedural and ethical. In general, predic
tions should be made only for people who fall within the range of scores on which the correla
tion is based. For example, if a regression equation predicting college success is based on a
study using middle‐class suburban Caucasians with SAT scores ranging from 1000 to 1400,
then the equation should not be used to predict success for any future applicant not part of that
population.

SELF TEST
9.3
1. Consider the relationship between depression and exercise. Do you think it is a positive
or a negative correlation? Explain.
2. Suppose you wish to predict academic success in college by looking at high school
grades. Which is the criterion variable and which is the predictor variable?
3. How does simple, linear regression differ from multiple regression?

8
Stoeber et al. (2011) actually ran what is called a hierarchical linear regression model, which is a form of multiple regression
which allows the researcher to enter multiple predictors in a step-wise fashion, rather than adding all predictors into the model at
once. Technically, the hierarchical regression allows the research to statistically control for one predictor before another
predictor is entered into the model. In Stoeber et al.’s case, they controlled for the unique contributions of motivation before
testing the passion predictor.
282 NON‐EXPERIMENTAL DESIGN I: SURVEY METHODS

Interpreting Correlational Results

Both correlation and regression are statistical techniques that are used in non‐experimental
methods, including survey research. Such techniques are also used in many other types of
non‐experimental methods, which you will encounter in Chapters 10 and 11. For now, it is
impor tant to keep some things in mind when interpreting data from research that uses
correlations and regression analyses. In an experimental study with a manipulated independent
variable, we’ve already seen that cause‐and‐effect conclusions can be drawn with some degree
of confidence. The independent variable of interest is manipulated and, if all else is held
constant (i.e., no confounds), the results can be attributed directly to the independent variable.
With non‐experimental research using correlations, the “all else held constant” feature is
missing, however, and this lack of control makes it impossible to conclude anything about cause
and effect from a simple correlation. Let’s consider two specific ways in which interpretation
problems can occur with correlations. These are the directionality problem and the third
variable problem (Neale & Liebert, 1973).

Directionality
If there is a correlation between two variables, A and B, it is possible that A is causing B to
occur (A → B), but it also could be that B is causing A to occur (B → A). That the causal
relation could occur in either direction is known as the directionality problem. The existence
of the correlation by itself does not allow one to decide about the direction of causality. For
instance, consider a study described in the New York Times in 2008, in which researchers
examined the research pro
ductivity and beer consumption of ornithologists in the Czech Republic. They found a negative
correlation: The more beer consumed by ornithologists, the less productive they were as scien
tists, a finding not likely to be taken well by those scientists who often claim that their best ideas
occur to them in bars or pubs. The Times article emphasized the interpretation that probably
occurred to you, that spending too much time drinking beer might cause the scientists to have
little time left for research. But one researcher, thinking in terms of directionality, suggested that
perhaps “those with poor publication records are drowning their sorrows” (For Scientists, 2008).
So it is conceivable that drinking lots of beer causes Czech ornithologists to fail in their publish
ing efforts (A → B), but it is also possible that failing to publish causes Czech ornithologists to
drink more beer (B → A). It is also worth noting the article also illustrated external validity, a
concept you learned about in Chapter 5. One non‐ornithologist critic of the study suggested that
perhaps the results were limited to scientists who studied birds. Another suggested the results
were limited individuals from the Czech Republic, which, the article claimed, has the highest
rate of beer consumption on Earth.
Research on the relationship between TV watching and children’s aggression typifies the
directionality problem. Some of these studies are correlational and take the following general
form. Some measure (variable A) of TV watching is made, the number of hours per week per
haps. For the same children, a second measure (variable B) of aggressive behavior is taken. It
might be a combined score of teacher ratings of the aggressiveness of those in the study.
Suppose this study yields a correlation of +.58, which is found to be significantly greater than
zero. What can be concluded?
One possibility, of course, is that watching large amounts of TV inevitably exposes the child
to a great deal of violence, and we know children learn by observation; thus, it would follow that
a large dose of TV watching causes children to become aggressive—that is, A → B. Even if a
regression analysis is conducted and watching TV (A) is a significant predictor of aggression
(B), this still doesn’t mean that watching TV causes aggression. Why? Because the regression
 itself is based on a correlation, and the research design is non‐experimental. But could causality
be work
ing in the reverse direction? Could it be that children who are already aggressive for some other
Interpreting Correlational Results 283

reason simply like to watch more TV than their nonaggressive peers? Knowing that much of
television involves violent programming, perhaps aggressive children choose to watch more of
the things that really interest them. In short, perhaps being aggressive causes children to watch
more TV—that is, B → A.
Choosing the correct causal direction is not possible based on an existing correlation.
However, the directionality problem can be addressed to some extent. The approach derives
from the crite ria for determining causality first described in Chapter 1. As you recall, research
psychologists are generally satisfied with attributing causality between A and B when they
occur together with some regularity, when A precedes B in time, when A causing B makes
sense in relation to some theory, and when other explanations for their co‐occurrence can be
ruled out.
For the TV and aggressiveness study, all we have is A and B occurring together and the fact
that A causing B makes some sense from what is known about observational learning theory
(remember Bandura’s Bobo doll study from Chapter 5—Box 5.2?). However, using a procedure
called a cross‐lagged panel correlation, it is possible to increase one’s confidence about direc
tionality. In essence, this procedure investigates correlations between variables at several points
in time. Hence, it is a type of longitudinal design, adding the causal element of A preceding B.
The following Research Example illustrates the procedure.

Research Example 27—Correlations and Directionality

In a famous study, Eron, Huesman, Lefkowitz, and Walder (1972) looked at the same
relationship between TV watching and aggression that we’ve been using as a hypothetical
example.9 In par ticular, they measured (a) preference for watching violent television programs
and (b) peer rat ings of aggressiveness. The participants were 875 third graders from a rural area
of New York State, first studied in 1960; a modest but significant correlation of +.21 between
preference for violent TV and aggressiveness was found. What made the study interesting,
however, was that Eron’s team returned 10 years later, found 427 of the same students (now
arbitrarily labeled “thirteenth graders”), and reassessed the same two variables. By measuring
the two variables at two points in time, six correlations could be calculated. These correlations,
as they occurred in the Eron et al. study, are displayed in Figure 9.5.
Preference for
violent TV in the 1
.05
3th grade (TVVL1
Preference for 3)
violent TV in the 3rd
grade (TVVL3) .01

FIGURE 9.5
.21 −.05 .31 Results of a cross‐lagged panel study of

grade (AGG1 3) on later aggression (

Aggression in the 3rd
.38 the efects of preference from Eron et al., 1972).
grade (AGG3)
Aggression in the 1 3th for violent TV programs

9
Actually, the study began with another purpose: to see if parental child rearing practices led to aggressiveness in children. But
in a questionnaire for parents, Eron et al. included “filler” questions, items designed to disguise the study’s true purpose. One
asked parents to list their child’s three favorite TV programs. When looking at the data, Eron et al. were surprised to see a
relation ship emerge: Children who preferred violent programs tended to be more aggressive. The entire focus of the study then
 changed (Huesman & Dubow, 2008). You might recognize this as an example of how serendipity (Chapter 3) can alter the
direction of research.
284 NON‐EXPERIMENTAL DESIGN I: SURVEY METHODS

Of special interest are the diagonal or cross‐lagged correlations because they measure the
relationships between two main variables but separated in time. If third‐grade aggressiveness
caused a later preference for watching violent TV (B → A), then we would expect a fair‐sized
correlation between aggressiveness at time 1 and preference at time 2; in fact, the correlation is
virtually zero (+.01). On the other hand, if an early preference for viewing violent TV programs
led to a later pattern of aggressiveness (A → B), then the correlation between preference at time
1 and aggressiveness at time 2 should be substantial. As you can see, this correlation is + .31,
not terribly large but significant. Based on this finding, as well as on other indications in their
study, Eron and his colleagues concluded that an early preference for watching violent TV is at
least partially the cause of later aggressiveness.
Cross‐lagged panel correlations must be interpreted cautiously, however (Rogosa, 1980). For
one thing, if you examine the overall pattern of correlations in Figure 9.8, you will notice the cor
relation of +.31 may be partially accounted for by the correlations of +.21 and +.38—that is,
rather than a direct path leading from 3rd‐grade preference to 13th‐grade aggression, perhaps
the path is an indirect result of the relationship between preference for violent TV and
aggression in the 3rd grade and between the two measures of aggression. A child scoring high
on preference for violent TV in 3rd grade might also be aggressive in 3rd grade and still be
aggressive (or even more so) in 13th grade. Alternatively, it could be that aggressiveness in third
grade produced both (a) a prefer ence for watching violent TV in third grade and (b) later
aggressiveness. Thus, cross‐lagged panel correlations help with the directionality dilemma, but
problems of interpretation remain. More generally, interpretation difficulties take the form of
the third variable problem.

Third Variables
The June 4, 2000, issue of the New York Times Magazine, contained a playful article entitled
“Greens Peace” (Plotz, 2000). In it the author addressed the weighty issue of why some
countries seem to be always at war, while others remain relatively peaceful. His answer was
golf: Countries where a substantial portion of the population plays golf are less belligerent than
countries without golf. As Plotz put it:

Every peaceful European nation loves golf. But Russia, at war in Chechnya, doesn’t hit
the links. Non‐golf Greece and non‐golf Turkey have long warred over non‐golf Cyprus.
The former Yugoslavia has fragmented into five states. Only peaceful Slovenia swings
the sticks. Do India or Pakistan golf? Of course not. Algerians shoot one another;
Moroccans next door shoot par. (p. 32)

Although the slogan “make par, not war” (p. 37) might have merit, I think you can see the
absurdity of the argument that golf causes peace. And it is only slightly more likely that the
reverse is true—that peace causes golf. Rather, if there really is a correlation between peace and
golf on a national level, and Plotz doesn’t present a Pearson’s r, of course, its existence is an
exaggerated example of what researchers call the third variable problem. Because
correlational research may not attempt to control extraneous variables directly, these variables
often provide an explanation for the correlation found—that is, rather than A causing B or B
causing A, an unknown third variable, C, might be causing both A and B to happen. C is an
uncontrolled third variable (or variables—it is often the case that more than one uncontrolled
variable lies behind a correlation). Can you think of third variables that could produce the
alleged golf‐peace correla tion? Economic prosperity perhaps? Highly prosperous countries
might be more likely to be peaceful and also have more time for leisure, including golf.
 The relationship between watching violent TV programming and children’s aggressiveness
provides a clear example of the third variable problem. As we’ve already seen, it is possible that