This article was downloaded by: [Loughborough University]

On: 24 November 2014, At: 06:59

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954
Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,
UK

Multivariate Behavioral
Research
Publication details, including instructions for
authors and subscription information:
http://www.tandfonline.com/loi/hmbr20

Commentary: Mediation
Analysis, Causal Process, and
Cross-Sectional Data
a
Patrick E. Shrout
a
New York University
Published online: 17 Oct 2011.

To cite this article: Patrick E. Shrout (2011) Commentary: Mediation Analysis, Causal
Process, and Cross-Sectional Data, Multivariate Behavioral Research, 46:5, 852-860,
DOI: 10.1080/00273171.2011.606718

To link to this article: http://dx.doi.org/10.1080/00273171.2011.606718

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the
information (the “Content”) contained in the publications on our platform.
However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness,
or suitability for any purpose of the Content. Any opinions and views
expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the
Content should not be relied upon and should be independently verified with
primary sources of information. Taylor and Francis shall not be liable for any
losses, actions, claims, proceedings, demands, costs, expenses, damages,
and other liabilities whatsoever or howsoever caused arising directly or
indirectly in connection with, in relation to or arising out of the use of the
Content.
 This article may be used for research, teaching, and private study purposes.
Any substantial or systematic reproduction, redistribution, reselling, loan,
sub-licensing, systematic supply, or distribution in any form to anyone is
expressly forbidden. Terms & Conditions of access and use can be found at
http://www.tandfonline.com/page/terms-and-conditions
Downloaded by [Loughborough University] at 06:59 24 November 2014
 Multivariate Behavioral Research, 46:852–860, 2011
Copyright © Taylor & Francis Group, LLC
ISSN: 0027-3171 print/1532-7906 online
DOI: 10.1080/00273171.2011.606718

Commentary: Mediation Analysis,

Causal Process, and
Downloaded by [Loughborough University] at 06:59 24 November 2014

Cross-Sectional Data

Patrick E. Shrout
New York University

Maxwell, Cole, and Mitchell (2011) extended the work of Maxwell and Cole
(2007), which raised important questions about whether mediation analyses based
on cross-sectional data can shed light on longitudinal mediation process. The
latest article considers longitudinal processes that can only be partially explained
by an intervening variable, and Maxwell et al. showed that the same general
conclusions are obtained, namely that analyses of cross-sectional data will not
reveal the longitudinal mediation process. While applauding the advances of the
target article, this comment encourages the detailed exploration of alternate causal
models in psychology beyond the autoregressive model considered by Maxwell
et al. When inferences based on cross-sectional analyses are compared to alternate
models, different patterns of bias are likely to be observed. I illustrate how different
models of the causal process can be derived using examples from research on
psychopathology.

The article by Maxwell, Cole, and Mitchell (2011) extends the important con-
tributions of Cole and Maxwell (2003) and Maxwell and Cole (2007), which
made the critical point that it cannot be assumed that mediation analysis car-
ried out on cross-sectional data will be informative about longitudinal causal
processes. Although the previous papers were generally convincing, they left

Correspondence concerning this article should be addressed to Patrick E. Shrout, Department of

Psychology, New York University, 6 Washington Place, New York, NY 10003. E-mail: pat.shrout@
nyu.edu

852
 MEDIATION ANALYSIS, CAUSAL PROCESS, AND CROSS-SECTIONAL DATA 853

open the possibility that the findings might be restricted to longitudinal pro-
cesses that represented complete mediation rather than more general models
that include direct and indirect paths (i.e., partial mediation models). Maxwell
et al. (henceforth MCM) shut the door on that possibility through a careful
analysis of two versions of the more general longitudinal mediation model.
They found that the estimates from a cross-sectional mediation model would
be biased relative to the assumed causal processes, and that the bias can be in
either direction, depending on the structure of the assumed causal model. Most
strikingly, MCM provided an example in which a cross-sectional analysis would
Downloaded by [Loughborough University] at 06:59 24 November 2014

suggest complete mediation when the actual causal model involves no mediation
whatsoever.
Since the first paper by Cole and Maxwell (2003) appeared, I have cited it in
numerous journal reviews and suggested to editors that mediation claims made
on the basis of cross-sectional survey data should be seriously questioned. In
some cases I have heard from editors that this critique is too strict and runs
counter to common practice in psychological research. I can only hope that
the MCM article and the new results drive home to psychological researchers
that it is time to move beyond uncritical application of the Baron and Kenny
(1986) mediation steps. If researchers want to understand causal processes in
psychology, it will not be enough to simply list some caveats about possible
bias from cross-sectional analyses. Instead, it will be necessary to think deeply
about the causal processes that are of interest. In this commentary I attempt to
push further some of the issues raised by the MCM article.

ASSUMPTIONS OF MEDIATION ANALYSIS

It is tempting to blame Baron and Kenny (1986) or Judd and Kenny (1981) for
the naive use of mediation analysis in psychology’s best journals, but as MCM
pointed out, Kenny and his colleagues have been quite clear about how viola-
tions of mediation assumptions can bias the analysis. Blame for the noncritical
application of the mediation framework needs to rest on the broader community
of scientists who have submitted, reviewed, and printed analyses that explicitly
ignore assumptions required for mediation analyses. Although there is some
variation in the assumptions methodologists list for making clear inferences from
structural models including mediation (for a thorough discussion, see Holland,
1988), there are some core assumptions that are on nearly all lists for mediation
in the Baron and Kenny tradition. For example, Judd and Kenny stated that the
causal order of variables needs to be established, that there should be no omitted
variables in the analysis, and that all the variables need to be measured without
error. These important points were repeated in Baron and Kenny and have been
elaborated in Hoyle and Kenny (1999), MacKinnon (2008), and others.
 854 SHROUT

The assumptions of a properly specified model, correct functional form, no

omitted variables, and reliable measures allow the residuals in the mediation
model to be represented as uncorrelated. MCM represented this assumption in
Figure 3 in their article by not showing a double-headed arrow connecting ©M
and ©Y . If the assumptions listed previously are not met, then the assumption
of uncorrelated errors is particularly suspect. For example, if there are omitted
variables that explain the association between M and Y other than the causal
path represented by b 0 , then the residuals should be considered to be correlated.
Although correlation of residuals cannot be estimated from the usual three-
Downloaded by [Loughborough University] at 06:59 24 November 2014

variable mediation analysis, it is sometimes possible to estimate (and adjust

for) these effects by the introduction of instrumental variables in the cross-
sectional analysis (e.g., Martens, Pestman, de Boer, Belister, & Klungel, 2006).
However, instrumental variable (IV) adjustments involve their own assumptions,
and many methodologists (e.g., Morgan & Winship, 2007) have warned that
these assumptions are difficult to justify. Elimination of bias might be nearly
impossible, but reduction of bias is sometimes possible. For example, when
the effect being studied in a mediation model is a randomized variable (X),
adjustments for correlated residuals can be accomplished by including preran-
domization assessments of the M and Y constructs (e.g., Shrout, 2011).
One important advantage of the analytic approach taken by MCM is that
they were able to show that under the models represented in Figures 1 and
2 the bias cannot be represented as a simple additive component that can be
adjusted by modeling correlated residuals. This means that even if a well-justified
instrumental variable were identified for the cross-sectional analysis, it would
not eliminate the bias identified by MCM in the cross-sectional analysis. Rather
than being an additive source of bias, MCM showed that a multiplicative factor
of bias is a function of the stability of the X, M , and Y longitudinal processes
in Figures 1 and 2.

LONGITUDINAL MEDIATION PROCESSES

As informative as the MCM analysis is about potential limitations of cross-

sectional mediation analyses, their representation of longitudinal process models
is limited by the models in Figure 1 and 2, which they use as the gold standard
in evaluating cross-sectional analyses. For example, if we consider a process
such as how maternal depression has an effect on depression of children (the
example used by MCM), the models in Figures 1 and 2 could fall short in at least
three ways. One is the important omitted variable problem of ignoring trait-level
individual differences in the levels of the three processes (maternal depression,
parenting behaviors, and child depression in the MCM example). Another is the
assumption that the behavioral process is stationary and in equilibrium (issues
 MEDIATION ANALYSIS, CAUSAL PROCESS, AND CROSS-SECTIONAL DATA 855

nicely discussed in Cole and Maxwell, 2003). A third is the assumption that all
of the variables are measured perfectly, with no measurement error.1
There are a number of ways that individual differences can be incorporated
into autoregressive models such as the MCM Figures 1 and 2 (e.g., McArdle,
2009). An especially promising model is the autoregressive latent trajectory
(ALT) model that was described by Bollen and Curran (2004). It incorporates
person-level latent variables to represent individual differences in the level and
trajectory of the process variables. According to the ALT model, the tendency
for maternal depression one day to predict maternal depression the next day is
Downloaded by [Loughborough University] at 06:59 24 November 2014

due in part to individual differences in levels of maternal depression, regardless

of whether a given day was a good or bad depression day. Such individual
differences may be due to genetic risk, chronic social adversity, or response
effects related to literacy or openness for disclosure. The ALT models can adjust
for these individual differences, but the MCM models in Figures 1 and 2 would
not. In fact, if a scientist adopted the models in Figures 1 and 2 for an analysis
of mediation of maternal depression effects, he or she would obtain biased
estimates of the effects that MCM call x, m, and y, even if the data were
measured without error and the processes were stationary and in equilibrium.
The calculations of the extent of the bias from cross-sectional analysis would
themselves be biased.
MCM acknowledge that their approach emphasizes between-individual vari-
ation rather than within-individual patterns (p. 7 in manuscript). However,
between-individual variation is only informative about causal processes under
special circumstances. Holland (1986) and Rubin (1974) showed how between-
individual information in randomized studies allows scientists to learn about
within-individual causal process, and how causal processes can also be inferred
from a careful analysis of within-individual variation. Longitudinal models are
well suited to study within person processes even when randomization is not
possible, but it is time to move beyond the arbitrary mixing of within- and
between-individual variation in the analysis of longitudinal data. A number
of authors are challenging methodologists and substantive scientists alike to
consider the strong assumptions that must be made to use between-individual
information to learn about within-individual process (e.g., McArdle, 2009;
Molenaar & Campbell, 2009).
When thinking about within-individual causal models, it is important to
match the parameters with psychological processes of interest. For example,
what can be said about the x, m, and a effects in the MCM Equation (1) and

1 In fairness, MCM said that they chose the models for Figures 1 and 2 because of historical

precedence and their popularity, and they also say that they assumed that all measures were latent
variables or measured perfectly, but readers could be excused for thinking that there are stronger
reasons for defining these models to be gold standards.
 856 SHROUT

the y, b, and c parameters in Equation (2)? MCM assumed that there is a

causal lag whereby variations in maternal depression at time t (Xi t ) lead to
a unit changes in parenting behaviors at time t C 1 (Mi;tC1 ). As Gollob and
Reichardt (1987) pointed out many years ago, the timing of this is critical. If a
mother wakes up depressed, perhaps she will be neglectful that day. However,
the effect of today’s depression might not be evident on tomorrow’s parenting,
particularly after adjusting for the today’s parenting (e.g., neglectfulness). MCM
also assumed that the same cross-day lag is important for updating the maternal
depression. Taken literally, the autoregressive model posits that a change in
Downloaded by [Loughborough University] at 06:59 24 November 2014

depression at time t causes a change in depression at time t C 1. If the mother

wakes up less depressed one day, she may be able to engage in cognitive
strategies that promote further coping, and this may have an effect on subsequent
improvements in depression at the next time point. It is possible to imagine
cognitive processes that are affected by diminished depression that make it
possible to engage in further coping. Such causal processes are worth thinking
about when writing about adjusting for the causal effects (x) of Xt on XtC1
and specifying the proper temporal interval. Similar thinking is necessary to
represent the adjustment for previous parenting effects on subsequent parenting.
By the way, even if the timing is well worked out, the process might not
be the same in all depressed mothers. For example, one mother might take
medication for her depression, while another might not. The medication could
affect the maternal depression lagged effect as well as its consequences on
parenting. If researchers want to talk about bias in parameter estimates, they
should be thinking about estimates of the average of distributions of individual
causal effects.
MCM also assumed that the effect of parenting on childhood depression is tied
to the same temporal unit as the link between maternal depression and parenting.
This poses a challenging theoretical and empirical question: How quickly does
depression in the child follow from a day of neglect? The mother is only one
influence on the child’s well-being, and the impact of daily fluctuations of
maternal depression on the affect of the child may be reduced by family, school,
and peer contexts. However, unrelenting fluctuations in parenting behavior may
have cumulative effects. Whether these can be represented in a neat temporal
package shown in Figures 1 and 2 is not a trivial question.
A whole host of other issues are raised by MCM’s assumption that the
longitudinal autoregressive models are stationary longitudinal processes that
have reached equilibrium. In the context of their motivating example, their
assumption would mean that there is not an effect of season, school year, or
holiday breaks on maternal depression, parenting, child outcomes, or the relation
among these variables. Of course, growth and development are also excluded.
Substantive scientists may think that the models in Figures 1 and 2 have been
endorsed by respected methodologists and therefore should be given special
 MEDIATION ANALYSIS, CAUSAL PROCESS, AND CROSS-SECTIONAL DATA 857

credence, but I encourage such scientists to consider alternate models, including

less elegant models that do not assume a stationary process in equilibrium. One
type of model that has been underused is the class of multivariate models of
change scores, including latent change scores adjusted for measurement error
(see McArdle, 2009).

SYSTEMATIC THINKING ABOUT MEDITATIONAL

(CAUSAL) PROCESSES AND ANALYSIS OF
Downloaded by [Loughborough University] at 06:59 24 November 2014

CROSS-SECTIONAL DATA

As I said at the beginning, the article by MCM is important in shaking the

confidence of those who believe that a routine application of the Baron and
Kenny (1986) can expose the causal pathways by which X has an effect on Y .
I appreciate that MCM did not intend to present the models in Figures 1 and 2
as definitive causal models for the analysis of the impact of maternal depression
on child depression through parenting behaviors, but I can report that after
having recommended Cole and Maxwell (2003) and Maxwell and Cole (2007),
I have had scientists come to me trying to estimate what they interpreted to be
the correct model. Frankly, science is complicated and statistical models can
be difficult to specify, so it is natural for productive researchers to look for
heuristics rather than engaging in bottom-up systematic thinking about causal
processes. The goal of my comment is to dissuade psychologists from walking
away from the MCM analysis with an overly simplistic view.
One simplistic take on the MCM contribution is that a strong mediation claim
based on the analysis of cross-sectional data cannot be justified. It is a simplistic
take because of the absolute (heuristic) of CAN NOT, which means can never.
What MCM showed is that if the true model represents partial mediation in a
stationary longitudinal autoregressive process that has reached equilibrium, and
if the same causal time frame holds for each of the causal processes, then the
analysis of a cross-sectional slice of that process is likely to be misleading.
MCM were quite clear that they do not claim that this finding generalizes to all
mediation models, but this qualification merits amplification.
In my opinion, there still will be occasional analyses of cross-sectional
correlations that can reveal possible causal mechanisms. These situations are
ones in which there are well-founded theories that describe the causal direction
of the processes, and for which the interpretation of the cross-sectional mea-
sures is informative about the temporal process. One such example is based on
the fascinating longitudinal findings of Costello, Compton, Keller, and Angold
(2003) on the relation of poverty to childhood psychopathology. Their study was
in fact longitudinal—it tracked rates of psychopathology of children in rural
families with new income and compared those rates to a natural comparison
 858 SHROUT

group of families with no new income. Their analyses suggested that reduced
time demands on the less impoverished families were a possible mediator of
the benefits of new income on child conduct problems. What made their study
remarkable is that the new income was not confounded with family charac-
teristics or risk for child conduct problems. The income was attributable to
membership in the Cherokee Nation, which opened a casino and distributed
its profits to the members of the tribe. Before the distribution of profits the
Cherokee members were not different from their neighbors in level of poverty
or risk of psychopathology. The payments started at a known time and eventually
Downloaded by [Loughborough University] at 06:59 24 November 2014

amounted to $6,000 per year.

If Costello et al. (2003) had not been collecting longitudinal data on devel-
opmental problems in this rural population when the casino opened, would it
have been possible to obtain information about this mediation process with data
collected in a cross-sectional survey? If that survey found that 5 years after the
casino opened the Cherokee mothers were able to spend more time at home after
school relative to their non-Cherokee neighbors, and that those spending more
time at home had children with fewer behavior problems than those spending
more time working away from home, then I believe a tentative mediation case
could be made—especially if retrospective information were collected that more
time was spent at home because of financial resources rather than because
of concern about behavior problems in their latchkey children. In this case
the causal process is revealed by gradual changes of the home environment
and awareness of parental monitoring rather than a daily interaction of mother
and child.
Even if a case can be made for publishing a mediation analysis based on
cross-sectional data, it is likely that all of the parameter estimates will be biased
to some extent. If the bias does not affect the sign, then the biased analysis
can still advance knowledge. However, the possibility of bias will always make
claims of complete mediation suspect. Complete mediation is said to occur when
the direct effect (after adjusting for the mediator) is exactly zero. Any bias in
the analysis precludes being able to make strong statements about the precise
value of the direct effect.

CONCLUSION

MCM showed that even when longitudinal causal models represent partial rather
than complete mediation with the autoregressive structure previously examined
by Maxwell and Cole (2007), the analysis of between-individual information
from a cross-sectional survey cannot provide information about the underlying
mediation process. It can be imagined that if they extended their analysis to
include the ALT models of Bollen and Curran (2004), they would obtain similar
 MEDIATION ANALYSIS, CAUSAL PROCESS, AND CROSS-SECTIONAL DATA 859

conclusions. This line of work should challenge the norm that routine application
of Baron and Kenny (1986) steps, without attention to assumptions that the
model is properly specified, the causal direction is known, the measures are
without error, and that all relevant predictors are adjusted, should be avoided. It
is my hope that this line of work stimulates more detailed consideration of the
assumed causal model that affects individuals and special consideration of the
timing of effects. Those proposing to analyze cross-sectional data to reveal causal
processes should be required to argue systematically against the importance of
biases in their analyses using systematic thinking about the causal processes
Downloaded by [Loughborough University] at 06:59 24 November 2014

under study.

REFERENCES

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social
psychological research: Conceptual, strategic and statistical considerations. Journal of Personality
and Social Psychology, 51, 1173–1182.
Bollen, K. A., & Curran, P. J. (2004). Autoregressive latent trajectory (ALT) models: A synthesis
of two traditions. Sociological Methods and Research, 32, 336–383.
Cole, D. A., & Maxwell, S. E. (2003). Testing mediational models with longitudinal data: Questions
and tips in the use of structural equation modeling. Journal of Abnormal Psychology, 112, 558–
577.
Costello, E. J., Compton, S. N., Keeler, G., & Angold, A. (2003). Relationships between poverty
and psychopathology: A natural experiment. Journal of the American Medical Association, 290,
2023–2029.
Gollob, H. F. & Reichardt, C. S. (1987). Taking account of time lags in causal models. Child
Development, 58(1), 80–92.
Holland, P. W. (1986). Statistics and causal inference (with discussion). Journal of the American
Statistical Association, 81, 945–970.
Holland, P. W. (1988). Causal inference, path analysis, and recursive structural equation models. In
C. Clogg (Ed.), Sociological methodology (pp. 449–484). Washington, DC: American Sociological
Association.
Hoyle, R. H., & Kenny, D.A. (1999). Sample size, reliability, and tests of statistical mediation. In
R. H. Hoyle (Ed.), Statistical strategies for small sample research (pp. 195–222). Thousand Oaks,
CA: Sage.
Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating mediation in treatment evaluations.
Evaluation Review, 5, 602–619.
MacKinnon D. P. (2008). Introduction to statistical mediation analysis. New York: LEA.
Martens, E., Pestman, W., de Boer, A., Belister, S., & Klungel, O. (2006). Instrumental variables:
Applications and limitations. Epidemiology, 17, 260–267.
Maxwell, S. E., & Cole, D. A. (2007). Bias in cross-sectional analyses of longitudinal mediation.
Psychological Methods, 12, 23–44.
Maxwell, S. E., Cole, D. A., & Mitchell, M. A. (2011). Bias in cross-sectional analyses of longi-
tudinal mediation: Partial and complete mediation under an autoregressive model. Multivariate
Behavioral Research, 46, 816–841.
McArdle, J. J. (2009). Latent variable modeling of differences and changes with longitudinal data.
Annual Review of Psychology, 60, 577–605.
 860 SHROUT

Molenaar, P. C. M., & Campbell, C. G. (2009). The new person-specific paradigm in psychology.
Current Directions in Psychology, 18, 112–117.
Morgan, S. L., & Winship, C. (2007). Counterfactuals and causal inference. New York, NY:
Cambridge University Press.
Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized
studies. Journal of Educational Psychology, 66, 688–701.
Shrout, P. E. (2011). Integrating causal analysis into psychopathology research. In P. E. Shrout,
K. Keyes, & K. Ornstein (Eds.), Causality and psychopathology: Finding the determinants of
disorders and their cures (pp. 3–24), New York, NY: Oxford University Press.
Downloaded by [Loughborough University] at 06:59 24 November 2014