Why Ineffective Psychotherapies Appear

Why Ineffective Psychotherapies Appear to Work: A Taxonomy of Causes of Spurious
Therapeutic Effectiveness
Author(s): Scott O. Lilienfeld, Lorie A. Ritschel, Steven Jay Lynn, Robin L. Cautin and
Robert D. Latzman
Source: Perspectives on Psychological Science , July 2014, Vol. 9, No. 4 (July 2014), pp.
355-387
Published by: Sage Publications, Inc. on behalf of Association for Psychological Science
Stable URL: https://www.jstor.org/stable/44290141
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms
Sage Publications, Inc. and Association for Psychological Science are collaborating with JSTOR
to digitize, preserve and extend access to Perspectives on Psychological Science
This content downloaded from

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
All use subject to https://about.jstor.org/terms
aps MĒĒĒĒĒĒĒĒĒ
MĒĒĒĒĒĒĒĒĒ I ASSOCIATION FOR I ASSOCIATION FOR
PSYCHOLOGICAL SCIENCE
Perspectives on Psychological Science

2014, Vol. 9(4) 355-387
Why Ineffective Psychotherapies Appear © The AuthoKs) 2014
Reprints and permissions:
to Work: A Taxonomy of Causes of sagepub.com/journalsPermissions.nav

DOI: 10.1177/1745691614535216
Spurious Therapeutic Effectiveness <§)SAGE

pps.sagepub.com
Scott O. Lilienfeld1, Lorie A. Ritschel2,3, Steven Jay Lynn4,

Robin L. Cautín5, and Robert D. Latzman6
department of Psychology, Emory University; department of Psychiatry, University of
North Carolina at Chapel Hill; 33C Institute, Cary, NC; department of Psychology,
Binghamton University; department of Psychology, Manhattanville College; and
department of Psychology, Georgia State University
Abstract
The past 40 years have generated numerous insights regarding errors in human reasoning. Arguably, clinical practice
is the domain of applied psychology in which acknowledging and mitigating these errors is most crucial. We address
one such set of errors here, namely, the tendency of some psychologists and other mental health professionals to
assume that they can rely on informal clinical observations to infer whether treatments are effective. We delineate
four broad, underlying cognitive impediments to accurately evaluating improvement in psychotherapy - naive realism,
confirmation bias, illusory causation, and the illusion of control. We then describe 26 causes of spurious therapeutic
effectiveness (CSTEs), organized into a taxonomy of three overarching categories: (a) the perception of client change
in its actual absence, (b) misinterpretations of actual client change stemming from extratherapeutic factors, and
(c) misinterpretations of actual client change stemming from nonspecific treatment factors. These inferential errors
can lead clinicians, clients, and researchers to misperceive useless or even harmful psychotherapies as effective. We
(a) examine how methodological safeguards help to control for different CSTEs, (b) delineate fruitful directions for
research on CSTEs, and (c) consider the implications of CSTEs for everyday clinical practice. An enhanced appreciation
of the inferential problems posed by CSTEs may narrow the science-practice gap and foster a heightened appreciation
of the need for the methodological safeguards afforded by evidence-based practice.
Keywords
psychotherapy, effectiveness, science-practice gap, confirmation bias, illusory correlation, spontaneous remission,
placebo effect, regression to the mean, effectiveness, efficacy
A clinically depressed client obtains psychotherapy; 2 the error of concluding that a treatment worked when
months later, she is free of serious symptoms. Was her the evidence for this inference is insufficient. They can
improvement due to the treatment? commit this mistake when evaluating the effectiveness of
The correct answer is "We don't know." On the one treatment for a given client, the effectiveness of a specific
hand, ample data demonstrate that scientifically sup- school or modality of psychotherapy, or both.
ported psychotherapies can alleviate many mental health This error in reasoning can be found in published
difficulties (Barlow, 2004), so the client's improvementresearch as well. In numerous articles, authors have inter-
may well stem at least partly from the intervention. On preted client improvement following an intervention -
the other hand, as most mental health professionals
even in the absence of differences from a no-treatment
know, we cannot draw valid conclusions regarding control
a group - as evidence for treatment efficacy (e.g.,
treatment's effectiveness in the absence of methodologi-
cal safeguards against errors in inference, such as well-
Corresponding Author:
validated outcome measures, randomized control groups, Scott O. Lilienfeld, Department of Psychology, Room 473, Emory
and blinded observations (Gambrill, 2012). Yet even sea-University, 36 Eagle Row, Atlanta, GA 30322
soned clinicians and researchers can easily fall prey toE-mail: slilien@emory.edu

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
356 Lilienfeld et al.
Leins et al., by2007).

Division 12 (Society of Clinical
For Psychology) of the
exam
cal treatment American Psychological
for Association
outpatie(see http://www
research team randomized
.divl2.org/empirically-supported-treatments/).
The research leg, which is the component
cognitive-behavioral or of evidence-
int
broadly equivalent
based practice most pertinent to ourimprov
arguments, incorpo-
the absence ratesof control groups,
a within-subject
no-treatm designs, blinding,
tion, the randomization, and other methodological
authors conclud bulwarks
equally against inferential mistakes.
effective for In ways that have often not
depress
see also p. been adequately appreciated
500). More or articulated, theserec
trolled study research safeguards
comparingare frequently nonintuitive. When p
cognitive-behavioral viewed in this light, the much decried therap
science-practice
also contained no control condition - the authors con- gap (Baker, McFall, & Shoham, 2008; Lilienfeld, Lynn, &
cluded that "Both treatments had substantial effects on Lohr, 2003; Tavris, 2003) and the resistance to evidence-
global eating disorder psychopathology and general psy- based practice that often accompanies it (Lilienfeld,
chopathology" (Poulsen et al., 2014, p. 114). Ritschel, Lynn, Cautin, & Latzman, 2013) are not entirely
In this article, we explain why the error of inferringsurprising.
that a treatment is effective on the basis of inadequate Although there are multiple sources of the science-
evidence is widespread, understandable, and problem- practice gap (for discussions, see Lilienfeld et al., 2013;
atic for clinical inference. We contend that a number of Shafran et al., 2009; Ritschel, 2005; and Stewart,
mental health professionals are insufficiently cognizant Chambless, & Baron, 2011), we focus on one key con-
of the manifold reasons why ineffective or even harmful tributor here: the myriad reasons why individuals can be
treatments can appear effective to the unaided eye.led to conclude that psychotherapy is effective even
Because of this inadequate recognition, some clinicians when it is not. We term these sources of inferential error
and researchers may dismiss or minimize the need for causes of spurious therapeutic effectiveness (CSTEs).
evidence-based practice (Sackett, Rosenberg, Gray, Because of an insufficient recognition of CSTEs, psychol-
Haynes, & Richardson, 1996; Straus et al., 2010). ogists may assume that they can rely on informal clinical
observations of client change during and after treatment
Evidence-Based Practice and Causes of to gauge whether interventions are effective.
We do not contend that informal clinical observations
Spurious Therapeutic Effectiveness of client improvement are never accurate; they frequently
Evidence-based practice is a threefold frameworkare. forNor do we argue that such observations are useless
clin-
or should be disregarded, as they are at times helpful
ical practice that is often conceptualized as a three-legged
stool. These legs comprise (a) research findingssignposts
regardingof change in treatment. As noted earlier, sub-
stantial
the efficacy and effectiveness of psychotherapies, evidence attests to the efficacy and effectiveness
(b) clin-
of a broad swath of psychotherapies for many mental
ical expertise, and (c) client values and preferences
(Norcross, Beutler, & Levant, 2007; Spring, 2007).
health conditions, including mood, anxiety, sleep, sexual,
Evidence-based practice is not synonymous with andempiri-
eating disorders, as well as some personality disor-
cally supported therapies (ESTs), which are merely ders, such
oneas borderline personality disorder (Roth &
Fonagy,
set of operationalizations of the research leg of the evi- 2005; Wampold, 2001; Weisz, Weiss, Han,
dence-based practice stool (Westen, Novotný, & Granger, & Morton, 1995). Hence, clinicians' inferences
Thompson-Brenner, 2005). ESTs are interventions that of client improvement during and after psychological
have been demonstrated to work better than no treatment treatment are surely correct in many instances. Moreover,
(or an alternative treatment) for specific disorders in inde-the social cognition literature demonstrates that numer-
pendently replicated (a) controlled between-subjectous forms of intuitive thinking, such as heuristic process-
designs or (b) controlled single-subject designs, namely,ing, are often adaptive in real-world settings (Gigerenzer
those in which participants serve as their own controls & Gaissmaier, 2011).
(Barlow, Hayes, & Nelson, 1984; Chambless & Hollon, At the same time, the histories of medicine and psy-
1998). Although the scientific status of ESTs is controver- chology demonstrate that subjective inferences of change
sial (for diverse viewpoints, see Beutler, 2004; Castelnuovo, in treatment, subjectively compelling as they may be, are
2010; Chambless & Ollendick, 2001; Herbert, 2003; and often mistaken (Garb, 1998; Grove & Meehl, 1996). Our
Westen, Novotný, & Thompson-Brenner, 2004), accep- overarching message is that because of CSTEs, unsystem-
tance of the need for the research prong of evidence- atic clinical observations of client change are rarely trust-
based practice does not hinge on agreement with theworthy guides by themselves for inferring treatment
criteria for or specific lists of ESTs, such as those proposedeffectiveness.

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Why Ineffective Psychotherapies Appear to Work
Goals of the Article accurately perceive and evaluate therapeutic change. In

addition, we delineate four broad obstacles to scientific
Numerous articles have canvassed the magnitude and
thinking - naive realism, confirmation bias, illusory cau-
sources of the science-practice gap (e.g., Baker sation,al.,
et and the illusion of control - that underpin many
2008; McHugh & Barlow, 2010; Stewart et al., 2011). We
or most CSTEs. The distinction between these overarch-
do not intend to retread that well-traveled ground here.
ing cognitive impediments and specific CSTEs themselves
Instead, in light of relatively recent developments con-
may not be entirely clear-cut. Nevertheless, we posit that
cerning (a) the implications of heuristics and biases thesefordomain-general impediments lay the cognitive
clinical practice (e.g., Crumlish & Kelly, 2009; Kahneman,
groundwork for more specific errors in inferring the exis-
2011; Stanovich & West, 2008), (b) iatrogenic (i.e., psycho-
tence or meaning of changes in treatment.
logically harmful) effects in psychotherapy (e.g., Bootzin
Second, we present a taxonomy of 26 CSTEs, divided
& Bailey, 2005; Dimidjian & Hollon, 2010; Lilienfeld, 2007),
into three categories, that can contribute to the appear-
and (c) challenges to the dissemination of evidence-based
ance of therapeutic effectiveness in its objective absence.
practice (Lilienfeld et al., 2013; Stewart et al., 2011), we
These three classes of CSTEs comprise influences that
address the more specific and largely neglected question
generate (a) the perception of client change in its actual
of what kinds of inferential errors in psychotherapy ren-
absence, (b) misinterpretations of actual client change
der evidence-based practice imperative.
stemming from extratherapeutic factors, and (c) misinter-
The movement toward evidence-based practice has
pretations of actual client change stemming from non-
been contentious in many quarters, in part because some
specific treatment factors. Some CSTEs operate at the
authors have taken issue with the premise that evidence
level of individual clients, others at the level of groups of
derived from randomized controlled trials, controlled
clients, and still others at both levels. Several writers in
single-subject experiments, and other systematic research
the medical literature have provided partial lists of arti-
designs should be accorded higher priority than facts
clinical
that can make ineffective medical treatments seem
experience when selecting treatments. Indeed, some
effective (e.g., Beyerstein, 1997; Hall, 2011; Hartman,
scholars have proposed that "practice-based evidence,"
2009; Kienle & Kiene, 1997), but no comparable list exists
namely, therapeutic practice informed by thoughtful clin-
for psychotherapies; nor have previous authors provided
ical observations, should be accorded roughly equal
a taxonomy of these artifacts.
weight to traditional evidence-based practice (Barkham,
Third, we outline how specific research methods help
Hardy, & Mellor-Clark, 2010; Green & Latchford, 2012;
to control for, although not necessarily eliminate, CSTEs
Strieker, 2003). For example, Chwalisz (2003) lobbied for of erroneous conclusions in treatment.
as sources
expanding the definition of evidence to encompass clini-
Although these research methods are by no means new
cal observations and clinical consensus (see also
to our knowledge their role in helping to rule out differ-
Hoshmand & Polkinghorne, 1992). Similarly, while
ing CSTEs has not been explicitly articulated. We also
acknowledging that "practical knowledge" (viz., knowl-
discuss how certain CSTEs continue to pose unresolved
edge acquired from clinical observations of what does
challenges to psychotherapy researchers and point to
and does not work in treatment) is fallible, Bohartfruitful
(2005) areas for further research on CSTEs and methods
maintained that it is "evidence-based" (p. 46) and should
for attenuating their influence. In this respect, our analy-
be valued as a legitimate source of inferences for thera-
sis has heuristic value in that it points to gaps to be filled
peutic effectiveness.
in extant psychotherapy methodology to minimize CSTEs.
We view these assertions with decided ambivalence.
Just as important, we discuss how a better appreciation
On the one hand, clinical observations can sometimes (a)
of CSTEs can inform everyday clinical practice. By pro-
usefully guide therapists' choices of interventions during
moting thoughtful consideration of explanations for cli-
treatment, (b) serve as springboards for the development
ent change above and beyond improvement due to
of new models of treatment, and (c) inform the feasibility
therapy itself, CSTEs can assist clinicians with becoming
and transportability of scientifically based interventions
better clinical scientists. Finally, we demonstrate that our
to real-world settings. On the other hand, for reasons that
discussion of CSTEs bears important implications for
we will explicate, the proposition that practice-based
models of clinical training.
observations should be accorded comparable weight to
the results of controlled clinical trials in treatment selec-
tion underestimates the inferential dangers stemming
Overarching Cognitive Impediments to
from CSTEs. With this background in mind, our goals are
threefold. First, we aim to demonstrate that inadequate
Evaluating Therapeutic Change
appreciation of the inferential threats posed by CSTEs is We submit that the principal reason why some mental
partly a by-product of natural cognitive processes that health professionals may not appreciate sufficiently the
render it difficult for clinicians, clients, and researchers to problems posed by CSTEs is that scientific thinking does

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
not come that dovetails with naive realism. This heuristic

naturally to (me
2011; shortcut) leads us to focus
Wolpert, on what is most obvious in Suc
1992).
practiced assiduously,
our environments while ignoring subtler background b
question our information. commonsen
pensity to Naive realism is erroneous because the world mean
perceive is not
absence (Bloom exactly as we perceive & it, a pointWeisbe
illustrated vividly by
errors in visual illusions (Chabris & Simons, 2010) arisin
judgment and enshrined
rapid and in the time-honored psychological distinction between
intuitively pla
tations that sensation
do and perception
not (Coren, 2003). What we per-
suffici
nations of client
ceive is constrained by external change.
reality, but it is also influ-
One telltale encedsignby our expectations, of biases, and interpretationsco
the
entific thinking ("apperceptions"; Morgan is & Murray, the 1935). To a substan-
his
trol group.tial extent, Contrary "believing is seeing" as much as the converse
to
assume, this (Gilovich, 1991).
concept is a r
scientific history
Naive realism bears important implications (Bul for the
Lemak, 1994). evaluation of psychotherapy
Examples outcome. It can lead clini- o
surfaced only cians, researchers, as and others to assume that they can
recently
first arguably rely on their intuitiveconducte
judgments - "I saw the change with
famously divided sailors
my own eyes" - to infer that an intervention is effective
ship into (Ghaemi, 2009; Lilienfeld, Lohr,
several & Olatunji, 2008). As a
groups
received citrusconsequence, these individuals
juice may (a) misperceive
imp
Lind's discovery change when it does not occur, was (b) misinterpret it when
app
Navy waited it does, or a (c) both.
full half-
juice on its Onevessels example of the overreliance on naive realism
(Bull,
and Angelicomes from Arnold Shapiro, producer
(1907) advoca of the 1978
groups in evaluating
Academy Award-winning documentary Scared Straight tra /,
psychology who responded to scientific
that a criticisms
formal of Scared Straight
concept in interventions.
social These interventions attempt to frighten
science
of the randomized adolescents at high risk for crime out of criminalcontr careers
emerging in by bringing the them to prisons and introducing them to
publishe
2005). Moreover, inmates. Shapiro defended Scared it Straight
wasprograms by n
nent authors insisting that(e.g.,
"I'm seeing it [the change following Scared
Eysen
call for randomized
Straight programs] with my own eyes, I'm there con
for every
(Cautín, 2008). one of those shoots" (Harrison, 2011, p. 2). However,
As noted data from controlled studieswe
earlier, suggest that Scared
conte Straight
impediments is not merely ineffective
underlie but probably harmful, in that it m
view as specific produces a heightened risk for antisocial behavior
instanti
the context (Petrosino,of Turpin-Petrosino,
psychoth & Buehler, 2005). In another
impediments, example, Healy (2002) wrote in an article, subtitled
followed b
implications "Evidence-Based
for Psychiatry,"
therapist that "When treatments
work, the condition being treated vanishes, and we don't
need randomized controlled trials to see this happening"
Naive realism
(p. 1). Yet the condition being treated may disappear for
Naive realism (Ross & Ward, 1996; Segali, Campbell,
a plethora of&reasons other than the intervention. Contra
Herskovits, 1966) is a concept imported into Healy's implication, randomized controlled trials and
psychology
from philosophy. Also termed commonsense other rigorous
realism or designs are indeed needed to exclude
rival
direct realism , naive realism is the ubiquitous hypotheses for observed change.
assumption
Naive realism
that the world is precisely as we see it. A plethora of also reminds us óf an easily forgotten
principle:
phrases in everyday life attest to the power of naive real- Change following therapy is not equivalent to
ism in our thinking: "Seeing is believing," "I change
saw because
it withof therapy, a logical error known as the
my own eyes," "I'll believe it when I see it," post hoc,"What
and ergo propter hoc (after this, therefore because of
you see is what you get." In a related vein, this) fallacy (Finocchiaro, 1981). Conversely, this error
Kahneman
can also lead
(2011) referred to a core principle of intuition as individuals to equate deterioration follow-
"WYSIATI": What You See Is All There Is, an ing a treatment with deterioration because of the
assumption

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Why Ineffective Psychotherapies Appear to Work 359
treatment (Lilienfekl, 2007), thereby overestimating

rebuttals to Bourne's the by
article: "Their tone was typified
iatrogenic effects of certain interventions. Theexperience
remarks such as 'it is clinical post that
hoc , here,
counts
ergo propter hoc fallacy underscores the
despite all figures point
to the that
contrary'" (p. 148). pre-
By the late
post studies of interventions are
1950s, problematic
insulin coma therapy had been all(T. D.
but abandoned
Wilson, 2011). Fortunately, as we will
(Shapiro discover,
& Shapiro, 1997). there are
multiple ways to compensate forPrefrontal
the lobotomy,
limitations of
which earned its pre-
principal devel-
post designs. Investigations using
oper ofsuch designs
the procedure in humans, are espe-
Portuguese neurosur-
cially suspect when the "pre" data derive from geon Egas Moniz, the Nobel Prize in Medicine or
retrospective assessments. For example, the much bally- Physiology in 1949, offers another telling example. One
hooed Consumer Reports study (Seligman, 1995) of 4,100 practitioner of this technique insisted that "I am a sensi-
magazine subscribers who had participated in psycho- tive observer, and my conclusion is that a vast majority of
therapy revealed that most felt that they had been helped my patients get better as opposed to worse after my treat-
by it. Yet, as numerous critics (e.g., Jacobson & ment" (see Dawes, 1994, p. 48), a view echoed by many
Christensen, 1996; Mintz, Drake, & Crits-Christoph, 1996) of his contemporaries (Diefenbach, Diefenbach,
pointed out, these data are difficult to interpret, because Baumeister, & West, 1999). Later research, however,
the study neglected to control for many potential con- revealed lobotomy to be essentially worthless and to be
founds that may have led to improvement even without associated with many disastrous psychological and neu-
therapy. rological side effects (Valenstein, 1986).
The history of medicine offers a powerful cautionary
tale regarding the hazards of naive realism (Bigby, 1998).
Most medical scholars agree that the history of physical
Confirmation bias
treatments administered prior to about 1890 is essentially A second cognitive impediment to appreciating the need
tantamount to the history of the placebo effect. Along for controls in psychotherapy research is confirmation
with ineffective medications, such interventions as blood- bias. Confirmation bias is the deeply ingrained and com-
letting, blistering, purging, and leeching were routinely monly exercised tendency to seek out evidence consis-
prescribed and presumed to be beneficial based on little tent with one's hypotheses and to deny, dismiss, or distort
more than informal clinical observations (Grove & Meehl, evidence that is not (Lilienfeld, Ammirati, & Landfield,
1996; see Belofsky, 2013, for a review of bizarre but 2009; Nickerson, 1998). Although confirmation bias is a
widely accepted medical practices through the ages). cognitive phenomenon, it can be fueled by desires to
Even today, medicine has its share of ineffective interven- find supportive evidence for our beliefs, a propensity
tions. A recent meta-analysis estimated that 40% of widely termed "motivated reasoning" (Kunda, 1990). Because
used medical procedures (e.g., intensive glucose lower- clinicians want their clients to improve, they can be
ing in Type 2 diabetes, induction of hypothermia for driven to perceive change in its absence.
intracranial aneurysms) are useless or harmful (Prasad Confirmation bias can foster a propensity toward illu-
et al., 2012). sory correlation (not to be confused with illusory causa-
The history of psychiatry is similarly replete with a tion; see next section), which is the perception of a
litany of useless or harmful interventions, many of which statistical association in its absence (Chapman &
were endorsed by experts of the era yet that strike us as Chapman, 1967; Hamilton & Gifford, 1976). Specifically,
inhumane today. Such "treatments" as spinning chairs, confirmation bias can predispose clinicians to attend to
tranquilizing chairs, and cold water were ubiquitous in the "hits" and forget the "misses" (Garb, Lilienfeld, &
early American psychiatry. As another example, insulin Fowler, 2008; Gilovich, 1991) and thereby overestimate
coma therapy, introduced by Manfred Sakel in 1933, was the extent to which their interventions are associated
used widely to treat schizophrenia throughout the 1930s with subsequent improvement. Imagine a therapist who
and 1940s. This procedure involved administering engages from time to time in confrontational tactics with
increasingly high doses of insulin to induce a hypoglyce- a client. Even though these tactics are ineffective for his
mic state, followed by a coma and sometimes convul- client's presenting problem, the therapist may attend to
sions. Early clinical reports described encouraging results. and recall the sessions in which the client was doing bet-
Its high morbidity and mortality rates notwithstanding, ter and neglect and forget the sessions in which the client
insulin therapy spread rapidly throughout Europe, the was not doing better or doing worse. As a consequence,
United States, Japan, and Australia (James, 1992). This the therapist may conclude that his use of confrontation
wave was unceremoniously interrupted by an article in was consistently followed by client improvement, even
the Lancet by Bourne (1953), who concluded there was though it was not. In contrast, if the therapist were to
no evidence that insulin coma therapy was effective. As monitor his clients' symptoms systematically, this errone-
Jones (2000) noted, many psychiatrists published ous inference would presumably be less likely

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
360 Lilienfeld et al
Illusory causation 1975). For example, when money is at stake, most people
prefer to select a lottery ticket or roll a die themselves
Scottish philosopher David Hume (1748) maintained that
rather than leave these actions to others, even though the
humans are prone to perceiving causal relations in their outcomes in all scenarios do not exceed chance. This
absence. Two centuries later, Michotte (1945) argued that
illusion may predispose therapists to believe that they
our propensity to perceive causal relations between possess more causal power over client outcomes than
events, even those that are causally unrelated, comes to
they do. The illusion of control is especially likely when
us as naturally as does our propensity to perceive color.
the individual in question (a) is personally involved in
Research on illusory causation , or the propensity to per- the behaviors, (b) is familiar with the situation at hand,
ceive a spurious causal relation between two associated (c) is aware of the desired outcome, and (d) has a history
variables, bears out these contentions.
of previous success at the task (Thompson, 1999). Most
Laboratory evidence for illusory causation dates at
or all of these criteria presumably apply to the modal
least to the work of Koffka (1935), who showed observ-
psychotherapist. Indeed, when interventions are consis-
ers two points of light in a dark room. When the points
tently followed by improvement, treatment providers
moved apart, perceivers tended to attribute causality to may conclude that they are the active causal agents when
the dot on which they happened to be focusing, even if they are not (Matute, Yarritu, & Vadillo, 2011).
it was stationary. Koffka's findings suggest that we are
more likely to attach causal significance to the object of
our attention while ignoring competing evidence. Later Implications of cognitive impediments
research demonstrated that illusory causation extends tofor clinicians 9 self-perceptions and
social interactions. When observers are positioned physi-
predictions
cally so as to attend primarily to one partner in a two-
person conversation, they regard him or her as more These four broad cognitive impediments may help to
interpersonal^ influential than the other partner (Taylor explain why some therapists overestimate their positive
& Fiske, 1975; see also McArthur & Solomon, 1978). client outcomes. In this respect, they appear to be no dif-
There are two potential, nonmutually exclusive expla- ferent from professionals in many other fields, including
nations for illusory causation (McArthur, 1980). The first college professors (Cross, 1977), physicians (Hodges,
is perceptual: Individuals tend to attribute causality to Regehr, & Martin, 2001), and political pundits (Tetlock,
whatever stimulus is most vivid and prominent in their 2005), all of whom tend to hold an overly charitable view
visual fields and to accord less causal import to what lies of their effectiveness (Dunning, Heath, & Suis, 2004). In
in the visual background (Lassiter, Geers, Munhall, a sample of 129 therapists in private practice (26.4% psy-
Ploutz-Snyder, & Breitenbecher, 2002). The second is chologists), the average clinician rated him- or herself at
cognitive: Individuals recall more information about stim- the 80th percentile of all therapists in terms of effective-
uli that are prominent in their visual foregrounds than in ness and skills; 25% of respondents placed themselves at
their visual backgrounds (Taylor & Fiske, 1978). With the the 90th percentile. None rated themselves as below
aid of an availability heuristic, by which we gauge the average. Moreover, the typical therapist in the sample
probability of an event by using its accessibility in mem- estimated the rate of client deterioration in his or her
ory (Tversky & Kahneman, 1974), we come to view the caseload to be 3 7% (Walfish, McAlister, O'Donnell, &
former stimuli as more influential. Lambert, 2012). In fact, numerous studies have indicated
Because of illusory causation, therapists, researchers, that about 10% of clients become worse following psy-
clients, and external observers may leap to the conclu- chotherapy (Boisvert & Faust, 2002; Lilienfeld, 2007).
sion that a treatment exerted a causal effect on the client Other evidence dovetails with these results. In a sam-
when it did not (Sloman, 2005). The client's improvement ple of 49 psychotherapists in college counseling centers,
clinicians markedly overestimated their rates of positive
within therapy sessions is plainly visible to the clinician,
whereas rival explanations for this improvement (e.g.,client outcomes (91%) relative to their actual positive out-
events occurring to the client outside of sessions, placebocomes (40%), as ascertained by a standardized symptom
effects, changes in cognitive biases over the course ofmeasure. Furthermore, although therapists predicted that
treatment) rarely are. As a consequence, these explana- only 3 out of a total of 550 clients (0.5%) in their collec-
tions may be assigned less weight. tive caseloads would deteriorate, outcome data revealed
that 40 (7.3%) did so (Hannan et al., 2005). Taken
together, these findings suggest that many or most psy-
Illusion of control chotherapists perceive improvements in clients in their
A related error is the illusion of control , or the propensityabsence and fail to perceive deterioration in their
to overestimate our ability to influence events (Langer,presence.

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Summary inclusive, because it does not distinguish among a myriad

of sources of erroneous therapeutic effectiveness.
In summary, four overarching cognitive biases - naive We stake no claim to our list's comprehensiveness, but
realism, confirmation bias, illusory causation, and illu-
it provides a helpful starting point for conceptualizing the
sion of control - probably contribute to the difficulty of numerous challenges that confront clinicians, research-
accurately evaluating change in psychotherapy, as well ers, and clients when gauging psychotherapeutic effec-
as to an insufficient appreciation of the inferential diffi-
tiveness. Although all of the CSTEs we describe have
culties posed by CSTEs, which we view as more specific been the subject of research on perceptions of change
instantiations of these four broad biases within the
following interventions or experimental manipulations,
domain of psychotherapy. These broad biases may alsoof these CSTEs (e.g., response-shift bias) have
several
contribute to clinician overconfidence, inadvertent
not, to our knowledge, been investigated with respect to
neglect of adverse client outcomes, and an undue reli-
psychotherapy per se. Nevertheless, there is no a priori
ance on unguided clinical experience (see also reason why these latter CSTEs cannot produce the illu-
Groopman, 2007).
sion of change following psychological treatment as well.
Causes of Spurious Therapeutic Overview of the taxonomy of CSTEs

Effectiveness: A List and Taxonomy
We divide our proposed CSTEs into three overarching
categories (see Table 1). The distinctions between these
As noted earlier, we refer to the manifold ways in which
people can be misled into believing that a treatmentcategories
is are conceptual, not empirical. First, some
working when it is not as causes of spurious therapeutic
CSTEs, which we term Category 1 CSTEs, can lead indi-
viduals, including clinicians, researchers, and other
effectiveness (CSTEs). We next briefly describe 26 CSTEs
that can deceive individuals into concluding that ineffec-
observers, to misperceive change in its actual absence. In
tive or even harmful psychotherapies are effective. We these cases, clients are not changing, although individu-
regard this list of CSTEs as provisional and subject toals erroneously perceive them to be changing. The prob-
improvement pending further research. Hence, for heu- lem of Category 1 CSTEs is underscored by a recent
ristic purposes, we adopt a "splitting" rather than a "lump-
quotation from eminent psychiatrist Robert Spitzer, who
ing" approach (see Mayr, 1981, for a discussion of the 9 years earlier (Spitzer, 2003) had endorsed the effective-
splitting-lumping dichotomy in classification) toward
ness of "conversion therapies" for homosexuality on the
CSTEs, electing to subdivide them into distinct categoriesbasis of self-reported improvement from clients. In a
when there is research support for doing so. The advan-
widely publicized retraction of his conclusions, Spitzer
(2012) acknowledged that there was no way to deter-
tage of a splitting approach is that certain CSTEs can later
be combined into broader categories if evidence demon-
mine whether these perceptions of change were accu-
strates that they are merely variants of the same inferen-
rate. As Spitzer told a reporter (Carey, 2012, p. Bl), "I
tial error. knew this was a problem, a big problem, and one I
In distinguishing among CSTEs, we part company withcouldn't answer. How do you know someone has really
authors who have placed most or all CSTEs under the changed?"
overarching rubric of placebo effects (e.g., Offit, 2010;Category 1 CSTEs are highly heterogeneous, as some
Shapiro & Shapiro, 1997). For example, Novella (2008, (e.g., CSTE Numbers 1 through 4; see following section)
2010) defined placebo effects as "including everythingprobably exert their initial effects primarily on clients'
other than a physiological response to a biologically
perception of change, whereas others (e.g., CSTE
active treatment" (p. 33) and operationalized it as "the
Numbers 7 through 9 and 11) probably exert their initial
treatment effect measured in the placebo arm of a clinical
effects primarily on clinicians' perceptions of change. Still
trial" (p. 33). Under placebo effects, Novella included others (e.g., CSTE Numbers 10 and 13) probably exert
such artifacts as regression to the mean, observer biases,
their initial effects on both clients' and clinicians' percep-
demand characteristics, and expectancy effects. There are
tions. Nevertheless, because psychotherapy is a process
two shortcomings with this expansive conceptualization. of bidirectional influence between client and clinician
First, it conflates the response following a placebo (the
(Marmar, 1990), most or all Category 1 CSTEs can eventu-
placebo response) with the response to a placebo (the ally come to deceive both treatment recipient and treat-
placebo effect) and thereby runs afoul of the post hoc, ment provider. Hence, these distinctions are unlikely to
ergo propter hoc error (Ernst & Resch, 1995; Kirsch, 2013).
be clear-cut, and they will require empirical corrobora-
Many of the symptomatic changes that occur in a study's tion and potential revision.
placebo arm can arise from variables other than the pla-Category 2 CSTEs can lead individuals, in most cases
cebo itself. Second, this conceptualization is overly both clinicians and clients, to misattribute actual client

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Table 1. Causes of Spurious Th
CSTE Description Research safeguards

Category 1 CSTEs: Erroneous All Category 1 CSTEs: Well-validated
perceptions of client change outcome indicators
in its absence
1. Illusory placebo effects Perceived improvement occurring in the absence of (none additional)
genuine improvement
2. Palliative benefits Feeling better about one's signs and symptoms (none additional)
without tangible improvements in them
3. Confusing insight with Mistaking apparent understanding of one's problem (none additional)
improvement with improvement in that problem
4. Retrospective rewriting of Belief that one has improved arising from a tendency Measures of pretreatment functioning
pre treatment functioning to remember one's pretreatment functioning as
worse than it was
5. Response-shift bias Change in one's evaluation standard with respect Measures of pretreatment functioning
to an outcome dimension as a consequence of
treatment
6. Reduction in cognitive biases Declines in cognitive biases tied to pretreatment Measures of cognitive biases throug
reporting of symptoms treatment
7. Demand characteristics Tendency of clients to report improvement in accord Outcom
with what they believe to be the therapist's or
researcher's hypotheses
8. The therapist's office error Confusion of client's in-session behavioral Out-of-
presentation with out-of-session improvement improvement
9. Test-retest artifacts Tendency of scores on psychopathology measures to Outco
decline spuriously on their second administration skip-out structu
10. Unknowable outcomes in Lack of information regarding what would have Comparis
the control condition occurred had the treatment not been administered condition
11. Selective attrition Tendency of clients who drop out of therapy to Intent-to-treat analyses. Measur
improve less than other clients of pretreatment differences between
treatment completers versus dropouts
12. Compliance bias Tendency for client adherence to treatment Measures of treatment compliance (e.g.,
recommendations to be confounded with variables completion of homework assignments),
that predict improvement Examination of compliance in the
control condition
13. Selective attention to client Tendency of individuals to unwittingly "cherry- Blinding of observers. Explicit a priori
outcomes pick" the outcome variables on which clients are predictions concerning client outcomes
improving
14. Selective memory for client Tendency of individuals to preferentially recall Blinding of observers
outcomes indications of improvement as opposed to those of
no improvement or worsening
15. Selective interpretation of Tendency of individuals to interpret ambiguous Blinding of observers
client outcomes changes in signs or symptoms as indications of
improvement
Category 2 CSTEs: All Category 2 CSTEs: Randomization to
Misinterpretations of actual treatment conditions
client change stemming from
extratherapeutic factors
16. Spontaneous remission Tendency of some psychological conditions to
improve on their own
17. History Widely shared events transpiring outside of Repeated measurements throughout
treatment treatment. Tracking of life events
outside of treatment
18. Cyclical nature of some Tendency of some psychological conditions to go up Long-term follow-ups
disorders and down
19. Self-limiting nature of Tendency of individuals w
disorder episodes conditions to improve once episodes have run
their natural course
(continued)

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Table 1. (continued)
CSTE Description Research safeguards
20. Regression to the mean Tendency of extreme

on retesting high reliability. Minimizing use of
extreme group designs. Estimating
controlling for measurement error
21. Maturation Improvement arising from naturally occurring (none additional)
psychological growth
22. Multiple treatment Tendency of individuals who obtain one treatment Use of adjunctive treatments as covari
interference to obtain others at the same time in analyses
23. Initial misdiagnosis Errors in the diagnosis of a condition, resulting in Use of well-valida
erroneous inferences of improvement and well-trained diagnostician
out medical causes of psychological
signs and symptoms
Category 3 CSTEs: All Category 3 CSTEs: Common factor
Misinterpretations of actual control groups
client change stemming from
nonspecific treatment factors
24. Placebo effects Improvement stemming from the expect
improvement mediators. Measurement of expectancies
25. Novelty effects Improvement owing to enthusiasm regarding the Inclusion of m
prospects of receiving a new intervention mediators. Measurem
26. Effort justification Clients who invest substantial time, energy, effort, Inclusion
and resources in an intervention often feel a mediators
psychological need to justify this commitment
Note: CSTEs in each category have one safeguard in common and then, usually, additional specific safeguards.
change stemming from extratherapeutic factors to the

Whether one regards Category 3 CSTEs as artifacts or
active treatment per se. These factors include life
as events
active agents of therapeutic change hinges largely on
that occur outside of treatment and changesone's in the
hypotheses regarding the mechanisms of improve-
client's psychological condition that are causallyment.
inde- If one believes that a given psychotherapy works
pendent of treatment. In the case of Category 2because
CSTEs,of specific processes that are not shared with
clients are improving, but their improvement bearsother notreatments, Category 3 CSTEs are best regarded as
relation to either the specific or nonspecific effectsartifacts
of the that can predispose to spurious inferences
treatment. Instead, the intervention is incidental toregarding
client the causes of change. In contrast, if one
change. believes that a given psychotherapy works because of
Category 3 CSTEs can lead individuals, again usually common factors that are shared with most or all effective
both clinicians and clients, to misattribute actual client interventions (e.g., Frank & Frank, 1961; Wampold, 2001),
change stemming from nonspecific effects of the treat- then the sources of change comprising Category 3 CSTEs
ment (e.g., provision of hope) to the specific effects of are best regarded as valid causes of improvement in their
this treatment (see Wampold, 2001). In the case of own right. Indeed, the long-standing interest in psycho-
Category 3 CSTEs, clients are improving, as they are in therapy integration largely reflects a desire to identify
Category 2 CSTEs. In Category 3 CSTEs, however, this cross-cutting mechanisms that operate across many treat-
change is a consequence of common factors shared with ments (Goldfried, 2010). Hence, we caution readers
most or all effective psychological treatments; little or against regarding Category 3 CSTEs as extraneous influ-
none of the improvement is attributable to the specific ences to be automatically minimized or eliminated in
treatment. Category 3 CSTEs are readily overlooked research, as from the standpoint of scholars who argue
because they are highly correlated with the provision of for the primacy of common factors in psychotherapy,
the treatment. As a consequence of these CSTEs, clini- such influences play a pivotal role in treatment effective-
cians and researchers may conclude that their hypothe- ness (e.g., Messer & Wampold, 2002).
sized mechanisms of therapeutic effectiveness are A fourth category of inferential errors that we do not
corroborated when they are not, as these mechanisms explicitly address comprises erroneous inferences regard-
(e.g., placebo effects) are shared by most if not all effec- ing the mechanisms of change in a given psychotherapy
tive treatments.
As a consequence of this class of errors, researchers and

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
therapists may enter therapy distressed about his repeated marital

conclude infi-
that
specific mechanism delity and leave therapy less distressedX but with whe
an unal-
specific mechanism tered risk for future infidelity. As AlpertY (see
(2012) observed,
the clinical "Therapy sessions can work like spa appointments: They ar
improvements
of the treatment can be relaxing but don't
but necessarily not
help solve prob- to
ited by the lems" (p. SR5).
treatment's prop
continue to One could justifiably contend
debate that palliative changes
whether
works by modifying can themselves be therapeutic in some instances, espe-
cognitio
proponents cially (Hofmann,
if distress regarding one's behaviors is a treatment200
nisms, such target. Yet
as especially for clients whose behaviors
increases in rou- re
tion of maladaptive though
tinely engender interpersonal distress for other individu-
Christensen, als, such1996; Longmo
as those with narcissistic or antisocial personality
the inferential errors in this fourth class involve an (a) disorders (American Psychiatric Association, 2013), the
erroneous inference regarding the specific cause(s) of problem behaviors themselves are often the foci of the
treatment effectiveness rather than (b) an erroneous intervention. In these cases, alleviating client distress
inference of treatment effectiveness, we do not catego- may actually be countertherapeutic. For example, some
rize them as CSTEs. In this respect, these errors differ authors have argued that psychological treatment often
makes psychopaths worse (Rice, Harris, & Cormier,
from Category 3 CSTEs, which involve the error of attrib-
1992), although the research support for this contention
uting specific effectiveness to a treatment that does not
contain specific active ingredients. is admittedly equivocal (D'Silva, Duggan, & McCarthy,
2004).
Category 1 CSTEs: Erroneous perceptions of client
change in its actual absence 3. Confusing insight with improvement. Some clients
1. Illusory placebo effects. Illusory placebo effectsmay achieve greater insight into their difficulties over
arise when expectations of improvement lead clients to the course of therapy. Although such insight may not be
believe that an attribute or condition improves in the linked to improvements in objective treatment outcomes,
absence of genuine changes on specified outcome mea- clients may believe that they have achieved progress
merely because they can now conceptualize and verbal-
sures (Wechsler et al., 2011). Illusory placebo effects dif-
fer from placebo effects in that the former do not involveize their problems in greater richness and detail. In this
genuine change (hence, individuals harbor the illusionexample, insight is unrelated to improvement and thus
that they have improved when they have not), whereas constitutes a CSTE. If, however, the acquisition of insight
the latter do. per se were a therapeutic goal, then acquiring insight
In a clever study (Greenwald, Spangenberg, Pratkanis, (even in the absence of change in signs and symptoms)
would not constitute a CSTE.
& Eskenazi, 1991), experimenters switched audiotapes
containing subliminal messages so that people who There are two separable issues here, both of which
bear on the veracity of insight as opposed to its clinical
thought they listened to audiotapes designed to enhance
memory actually listened to audiotapes designed to
utility. First, the insights obtained in psychotherapy may
enhance self-esteem, and vice versa. Participants came sometimes be illusory, reflecting subjectively compelling
away believing that their memory or self-esteem, as the but erroneous causal stories (Taleb, 2007). To the extent
case may be, had improved in response to the tape they that humans are "meaning-making" beings (Bruner,
believed they had heard rather than in response to the 1990), insight may at times prove helpful in constructing
tape they had actually heard. In fact, on objective tests ofa framework within which to better comprehend them-
selves and others. Indeed, some specious insights
memory and self-esteem, all of the tapes were ineffective.
The illusory placebo effect demonstrates that expecta- acquired in treatment may improve clients' mood or
tions and implicit theories can lead people to perceive, behavior,
or at least in the short term, by affording them a
at least report, imaginary changes in their behaviors,sense of understanding and control over their problems
thoughts, and feelings (see also Nisbett & Wilson, 1977). (see Jopling, 2001, for a discussion of "placebo insights"
in treatment), but others may be therapeutically inert or
2. Palliative benefits. Psychotherapy sometimes makes harmful (Jopling, 2008).
clients feel better about their difficulties but exerts little or Second, even if the insights accrued in therapy are
no effect on these difficulties (Beyerstein, 1997). Echoing veridical, they may not guarantee or even facilitate
this point, Albert Ellis (2003) underscored the importance improvement. For example, a client with a specific pho-
of distinguishing "feeling better" from "getting better" inbia of dogs may come to recognize that his fears origi-
psychotherapy. For example, an antisocial client maynated with a frightening dog attack and that he is now

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
negatively reinforcing these fears by change

avoidingin long-term
dogs. Yet if therapy may be suspe
he is unwilling to confront his fears ifduring therapy
they reflect by
implicit beliefs regarding the
engaging in systematic in vivo exposure nature of tochange
dogs, his (Ross, 1989).
symptoms are unlikely to abate (Wachtel, 1987). Nor is
insight always necessary for improvement 5. Response-shift
(Bloom, bias. A related phenomenon,
1994).
response-shifthalf
In one study of psychoanalytic treatment, bias , occurs
of 42 when an intervention leads
patients were rated as better adjusted individuals to change "their
at the conclusion evaluation standard with
of
therapy although few were judged regard to exhibit increased
to the dimension measured" (G. S. Howard, 1980,
insight into their "core conflicts" p. 93; see also Bray,
(Bachrach, Maxwell, & Howard, 1984; G. S.
Galatzer-
Levy, Skolnikoff, & Waldron, 1991). Howard & Dailey, 1979). This shift, which is of particu-
lar concern for researchers or clinicians using self-report
measures, canfunction-
4 . Retrospective rewriting of pretreatment occur when an intervention leads clients
ing. In some cases, clients may persuade themselves
to reconceptualize their initial levels of a specific psycho-
that they have improved by misremembering
logical trait. Intheir
contrastinitial
to retrospective rewriting of pre-
level of functioning as worse than treatment
it wasfunctioning,
(Ross, 1989).
which reflects a memorial change,
Such biased memories may stem from this CSTEclients'
reflects an implicit
alteration in one's "implicit scale" for
expectations of change during therapy.measuring In one
a trait study,
(McLeod, 2001). Response-shift bias can
cause individuals
researchers randomly assigned university to either underestimate
students to or overestimate
either a study skills course designed the to improve
effects their intervention, depending on
of a psychological
grades or to a no-intervention controlthe direction of the shift.and
condition
measured their study skills and grades Forbefore
example, an and after self-critical spouse may
excessively
the intervention. The study skills class was therapy
enter couples apparentlyconcerned that she is to blame for
useless, as it failed to improve students'
problems grades. Yet stu-
in her marriage; on self-report and interview
dents in the experimental conditionmeasures,
perceived she initially
the inter-rates herself as narcissistic and
vention as effective, because they misremembered their she may come to realize
anger-prone. During treatment,
initial study skills as worse than they
that herwere (Conway
verbally abusive and& overbearing husband is pri-
Ross, 1984). Similarly, evidence suggests that at
marily responsible forleast
their marital conflicts and that her
some of the change commonly attributed to "posttrau-
levels of self-centeredn ess and resentment are no higher
matic growth" - psychological improvement
than the average following
person would experience in a similarly
trauma - may actually be due to derogation of individu-
trying situation. Even though her levels of these two
als' pretrauma selves (Frazier et al.,problematic
2009; McFarland
traits have not &changed over the course of
Alvaro, 2000). Retrospective rewriting of pretreatment
treatment, her trait scores on standardized measures may
functioning may sometimes also occur declineduring
from pretest psycho-
to posttest, leading the therapist
therapy, especially when clients harbor strong
(and often expecta-
the client herself) to conclude erroneously
tions of improvement. that the treatment has lowered her self-centeredn ess and
Such retrospective rewriting may transpire
hostility. In aeven
sense, when
the treatment has exerted an impact -
individuals are asked to evaluate their long-standing
on the per- of her traits but not on
client's conceptualization
sonality traits. In an elegant series of these traits themselves.1
studies, A. E. Wilson
and Ross (2001) found that individuals frequently
described their current selves more favorably
6. Reduction in than their
cognitive biases. Successful treatment
past selves, largely because they derogated
for depressiontheir past
and similar conditions may attenuate cer-
selves. This tendency was especially pronounced
tain cognitive biases, when
such as those tied to self-criticism
participants cared about the traits and being judged.
perception of one's These
level of impairment (Whisman,
results dovetail with longitudinal dataMiller, Norman, & Keitner,
demonstrating that 1991). Although a reduction
the correlations between actual and in perceived
such distortionschange
is often in
a legitimate treatment tar-
personality traits are only modest (Robins,
get per se, it may Noftle,
engender the spurious appearance of
Trzesniewski, & Roberts, 2005). A studyimprovementof 290 under-
on other measures. For example, depres-
graduates tracked across 4 years ofsion college
is often found
marked bythatoverreporting of the features
participants retrospectively overestimated the extent
of associated psychopathology. to As a consequence, an
which they had become more extraverted overthat
intervention time, per-the intensity of the cognitive
diminishes
haps consistent with the cultural narrative
biases oftenthat students
associated with depression (e.g., magnifica-
become more outgoing and socially adept
tion of in college
one's weaknesses) may lead to decreases in the
(Robins, Fraley, Roberts, & Trzesniewski, 2001).
reported severity of Such
co-occurring problems (e.g., social
findings suggest that retrospective self-evaluations
adjustment), even when theseofproblems have remained

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
objectively unchanged
adequately functioning clients use psychotherapy ses- (M
Widlocher, 1991).
sions as opportunities to express their pent-up negative
emotions (see Nichols & Efran, 1985).
7. Demand characteristics. Demand characteristics In other cases, however, the therapist's office error
may contribute to overestimates of treatment effective-
occur when clients or research participants adjust their
behavior, including self-reported behavior, in accordness. For example, clients with social anxiety disorder
(social phobia) involving apprehension of interpersonal
with what they believe to be the therapists' or investiga-
tors' hypotheses (Orne, 1962). The treatment rationale
rejection who are initially anxious in treatment may grow
provided by clinicians can convey potent demand char-more comfortable with the therapist over time, leaving
the therapist (and perhaps clients themselves) with the
acteristics to patients regarding treatment and thereby
shape their attributions, expectations, emotions,misleading
and impression that they are experiencing
improvement in social anxiety symptoms. Yet these cli-
actions (Addis & Carpenter, 2000; McReynolds & Tori,
ents may merely be exhibiting stimulus discrimination,
1972). In one study, participants informed that thoughts
precede affect in response to images (i.e., a cognitive
learning to respond less anxiously to the psychotherapist
or others who provide them with unconditional accep-
therapy rationale) were more likely to report thoughts
first compared with participants informed that affecttance
pre-but not to the very people they find most interper-
cedes thoughts. Differences between the two rationales sonal^ threatening. Indeed, studies of behavior therapy
were especially apparent in response to highly arousingfor anxiety disorders sometimes point to a stimulus gen-
images (Kanter, Kohlenberg, & Loftus, 2004) and were eralization gradient from the therapist's office to the out-
side world, reflecting marked improvements in the former
maintained at a 1-week follow-up (Busch, Kanter, Sedivy,
& Leonard, 2007). setting followed by decrements upon treatment termina-
tion (Gruber, 1971; see Lynch, Chapman, Rosenthal, Kuo,
Moreover, clients are often motivated to tell their ther-
apists what they believe their therapists want to hear;& Linehan, 2006, for a discussion of real-world general-
they may also be motivated to persuade themselvesization that strategies in dialectical behavior therapy). These
they have improved. Hathaway (1948) referred to findings
the underscore the need to ensure that the client's
anxiety-provoking behaviors are addressed in real-world
"hello-goodbye" effect as clients' propensity to present
settings during treatment.
themselves as worse than they actually were at the outset
The therapist's office error may pose a particular chal-
of treatment and better than they actually are at the con-
clusion of treatment. As a consequence of this phenom-lenge for psychoanalytic therapies, which rely heavily on
enon, therapists and other observers may conclude the therapist-client transference as the engine of change.
that
client improvement occurred in its absence. In many respects, one can conceptualize transferences as
reflecting interpersonal expectancies (Westen, 1998).
Similarly, hypnosis researchers have identified a "hold-
Accordingly,
back effect" when participants are tested sequentially in if clients do not generalize their positive
transference reactions toward the therapist to others, their
nonhypnosis and hypnosis conditions. One of the implicit
demands of hypnosis is to behave as a "good" hypnoticlong-term improvements may be limited (Holmes, 1971).
subject should, or at least as this role is understood by
9. Retest artifacts. The retest artifact (Loranger,
the participant (Orne, 1962). The holdback effect can
Lenzenweger, Gartner, & Susman, 1991) is the tendency
arise when participants are not hypnotized during an ini-
tial baseline trial but know they will be hypnotized in of
thescores on psychopathology indices to decline spuri-
ously upon their second administration. This artifact may
following trial. In such cases, they may deliberately "hold
back" from fully responding when they are not hypno- be especially likely with measures characterized by a
tized to demonstrate gains on the later hypnosis trial, skip-out structure, such as many structured and semis-
thereby presenting themselves as good hypnotic subjects tructured interviews. Clients may realize that if they say
(Braffman & Kirsch, 1999; Zamansky, Scharf, & Brightbill, "no" to many questions, they will have a much briefer
1964). and less emotionally distressing experience than if they
say "yes" to them, generating the false appearance of
8. The therapist's office error. What we term the ther- improvement. In other cases, clients may deny more
apist's office error is the mistake of confusing clients' symptoms during the second assessment if they learn
in-session behavioral presentation with out-of-session that the questions concern sensitive behaviors, like drug
improvement. Clients' behavior within the cloistered use or antisocial activities. Indeed, evidence suggests that
confines of the therapist's office may not reflect their this artifact may be especially pronounced for measures
behavior or functioning outside of treatment (Holmes, of socially undesirable characteristics (Jorm, Duncan-
1971; Magaret, 1950). This error may sometimes lead cli- Jones, & Scott, 1989). Although the test-retest artifact has
nicians to underestimate treatment effectiveness, as when not received the research attention it merits, data suggest

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Why Ineffective Psychotherapien Appear to Work 367
investigators evaluated the 3-year outcome of 61 patients

who had experienced traffic accidents; some had been
randomly assigned to receive CISD and others to receive
no intervention. Among other measures, participants
completed the Impact of Events Scale (IES; M.J. Horowitz,
Wilner, & Alvarez, 1979), an index measure of posttrau-
matic stress symptoms. As is evident from Figure 1, high-
scoring IES participants who received CISD improved
between the pretreatment baseline and the 3-year follow-
up. Yet remarkably, high-scoring IES participants who
received no intervention at all improved even more.
These findings suggest that CISD can impede natural
healing processes (McNally et al., 2003). They also help
us to understand why so many people are persuaded that
CISD is efficacious even though it is not. Specifically,
trauma-exposed individuals who receive CISD do
improve, but not because of the treatment. To the con-
trary, they probably would have improved even more
had they received no treatment at all.
11. Selective attrition. This CSTE differs from others

we have described in that it operates not at the level
Fig. 1. The effects of critical incident stress debriefing on posttraumatic
of individual clients but at the level of all clients in a
stress symptoms among traffic accident victims. Note the striking differ-
clinician's
ence in trajectories between high scorers whocaseload. Selective
did and didattrition refers to the fact
not receive
the intervention. Both groups improved, that
but clients who drop
the group thatout of therapy
received are
not a random
the
intervention would have improved more had they
subsample of all received no inter-
clients. Research demonstrates
that cli-
vention at all. From Mayou et al. (2000). Reprinted with permission.
ents who are not improving are especially likely to leave
psychotherapy (Garfield, 1994; Tehrani, Krüssel, Borg,
that it may be more of a threat&to
Munk-j0rgeńsen,
the validity 1996;of
see short-
also Swift & Greenberg,
term than long-term assessments ofAspersonality
2012). disorder
a result, therapists may conclude erroneously
features (Lenzenweger, 1999; Samuel
that their et al, 2011).
treatments are effective merely because their
remaining clients are those that have improved. One
10. Unknowable outcomes in the control condition. problem that has long bedeviled the evaluation of Alco-
A largely unappreciated reason for erroneous inferences
holics Anonymous and similar 12-step interventions for
of therapeutic effectiveness is the absence of informa-
substance disorders is the high level of client dropout
tion regarding the "hypothetical counterfactuaP (Dawes,
from this intervention, often approaching 40% following
1994): our inability to know what would have occurred1 year (Kelly & Moos, 2003). The clients who remain in
had we not intervened. Because clinicians in routine these treatments after several years are generally faring
practice settings are necessarily unaware of how their
better than when they began, but they are unrepresenta-
clients would have fared in a control condition, they can-
tive of the clients who initially enrolled. The clients who
not gauge the extent to which the improvement dropped
they out may not have been helped or may have
observed might have occurred in the absence ofeven treat-
been harmed by the intervention.
ment or in the presence of an alternative treatment. Cli-
12. Compliance bias. A cognate problem of selec-
ents are certainly subject to the same epistemic limitation.
An illustrative example derives from research on tion bias can arise even among clients who remain in
criti-
cal incident stress debriefing (CISD), which is widely
treatment. Compliance bias occurs when differences
used to decrease the risk of posttraumatic stressamongsymp-clients in their adherence to treatment recom-
mendations are confounded with variables that predict
toms among trauma-exposed victims. Controlled research
demonstrates that CISD is ineffective and perhaps iatro- such as motivation to improve or conscien-
outcome,
tiousness (Grodstein, Clarkson, & Manson, 2003; Petitti,
genic (Litz, Gray, Bryant, & Adler, 2002; McNally, Bryant,
& Ehlers, 2Ö03). Yet many people who have undergone 1994). One well-known case of such bias comes from
CISD are convinced that it was effective (Carlier, Voerman,
the 1970s Coronary Drug Project, which examined the
& Gersons, 2000). A study by Mayou, Ehlers, and Hobbs
effects of Clofibrate, a cholesterol-lowering medication,
(2000) offers intriguing insights into this paradox. on These
heart disease (Coronary Drug Project Research Group,

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
1975). When the

the same error: They investigat
may engage in "selective symp
effect of monitoring" (Pennebaker & Skelton,
Clofibrate versus 1981), focusin
comes, theysymptoms conducted that they expect to change whileinte neglecting
sus irregular underattending Clofibrate to others. users
that only 15% of regular Cl
taken 80% or more
14. Selective memory forof thei
client outcomes. The p
ease compared several decades with of psychological 25% research leave of sc
suggest a positive doubt that memory is effect fallible (Loftus, 1993; Lynn of & Na
researchers 1994) compared
and that most of us preferentiallyregu recall informa
the placebo, consistent the with our hunches and desires (Walker,
results we Sk
2001; Taubes, roński, 2007).& Thompson, 2003). AsPresum a consequence, clinici
as health consciousness,
may be more likely to recall positive than negative acc
diligent adherence signs and symptoms during to and after phys
treatment, po
(b) better cardiovascular outcomes. tially resulting in overestimates of treatment effect
Research on cognitive-behavior therapy reveals that ness.3
clients who comply with extrasession homework assign-
ments display better treatment outcomes than those who 15. Selective interpretation of client outcomes. Confir-
do not (Mausbach, Moore, Roesch, Cardenas, & Patterson, mation bias and similar cognitive errors may predispose
2010). Similarly, evidence suggests that clients who prac- to selective interpretation of the clients' difficulties during
tice meditation regularly in studies of compassion-based and after treatment. The more ambiguous the outcome
meditation training exhibit better outcomes than clients variables rated by clinicians, the larger the potential for
who do not (Pace et al., 2009). Because of compliance biases in their ratings (Markin & Kivlighan, 2007; Westen
bias, unwary psychotherapists may notice that some of & Weinberger, 2005). Hence, clinicians who are moti-
their clients comply with their prescribed interventions vated to perceive improvement in their clients may inter-
more than do others, find that the former clients display pret ambiguous symptoms (e.g., increased anger toward
superior treatment outcomes, and conclude that these a spouse in marital therapy, heightened emotional pro-
interventions were effective. Yet individual differences in cessing of painful childhood memories) as evidence of
client treatment adherence may merely be a proxy fortreatment success.
another variable, such as treatment motivation or emo-
tional resilience, which in turn is linked to enhanced psy- Category 2 CSTEs: Misinterpretations of actual c
chological health.2 Moderator analyses, which examine ent change stemming from extratherapeutic
whether interventions are especially beneficial for certain factors
clients (Kazdin, 2007; Kraemer, Wilson, Fairburn, & Agras, 16. Spontaneous remission. Spontaneous remission
2002), may be helpful in this regard, as levels of compli-refers to the tendency for disorders to resolve on their
ance can be treated as continuous moderators of own (Beyerstein, 1997). Early reports by Eysenck (1952)
outcome. of spontaneous remission rates of 70% or more among
psychiatric patients were almost surely overestimates
13. Selective attention to client outcomes.(Rachman, Confirma- 1973). Nevertheless, later data point to nontriv-
tion bias (Nickerson, 1998), illusory correlation ial rates of spontaneous remission in outpatient samples
(Chap-
man & Chapman, 1967), and allied cognitive(Chadwell errors & mayHowell, 1979; Lambert, 1976). For example,
lead clinicians to attend selectively to certain outcome Zimmerman (2000) reported a spontane-
Posternak and
variables while ignoring or minimizing others. ous remission
Specifi- rate of 52% among patients with major
cally, psychotherapists may unwittingly "cherry-pick" the depressive disorder. The rates of spontaneous remission
outcome variables on which clients are improving. among children For and adolescents with psychopathology,
example, because of diagnostic overshadowing including behavioral problems, approach or exceed 40%
(Garb,
1998), therapists may focus unduly on apparent improve- (Harrington, Whittaker, Shoebridge, & Campbell, 1998;
ment in dramatic client signs and symptoms, such Jacobson & Christensen, 1996; McCullough, 2000).
as psy-
chotic features or aggressive behaviors, while The longer people remain in therapy, the greater the
neglecting
lack of improvement or deterioration in less overt signs opportunity for extratherapeutic factors, including natu-
and symptoms, such as depressed mood, anxious rumi- ral healing processes, coping, social support, and posi-
nations, or anger (Zimmerman & Mattia, 1999). As a con- tive experiences in everyday life, to contribute to observed
sequence, they may conclude that clients have improved or perceived improvement (Jacobson & Christensen,
when they have exhibited no change across multiple 1996). Moreover, when frequent spontaneous remissions
clinically important domains. Clients may fall victim to happen to coincide with the administration of specific

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
interventions, client and clinician alike

of this may fall prey
condition (e.g.,toaggression, self-injur
social that
illusory causation, coming to believe interaction
the difficulties)
interven- often wax and wane over
tions are producing the spontaneous briefremissions (Blanco,
time periods (Romanczyk, Arnstein, Soorya, & Gillis,
Barberia, & Matute, 2014). 2003), leading observers to mistake short-term behavioral
Spontaneous remission may be partly
changes foraccounted for
beneficial treatment effects.
by what Alexander and French (1946) termed the "cor-
19. Self-limiting
rective emotional experience," a positive affective natureoccur-
of disorder episodes. Like the
rence that ameliorates the detrimental impact
acute exacerbations of ofmanyearly
physical disorders, the epi-
negative life events (Bridges, 2006). Although
sodes Alexander
of some psychological disorders tend to be self-
and French emphasized the role oflimiting.
correctiveA treatment may appear to exert a beneficial
emotional
experiences in psychotherapy, such events
effect (e.g.,
on a disorder finding
episode that has run its natural course
a loving partner in the aftermath (Beyerstein,
of an abusive 1997). Forrelation-
example, the median duration of
ship) surely occur in everyday lifea depressive
as well.episodeIn the is approximately
words 13 weeks (Solo-
of psychoanalyst Karen Horney (1945),
mon et al.,"life itself
2010), and still
some untreated episodes remit or
remains a very effective psychotherapist" (p. 240).without any intervention (Kirsch &
improve substantially
Sapirstein, 1998). In other cases, certain disorders them-
1 7. History. A related extratherapeutic factor
selves may thatFor
be short-lived. can
example, short-term drug-
induced psychiatric
contribute to the erroneous inference conditions, such as amphetamine
of a therapeutic
effect is what Campbell and Stanley (1963)
intoxication termed
or alcohol his-
withdrawal delirium (American
tory : widely shared events transpiring
Psychiatric outside of the
Association, 2013), wane in intensity once the
treatment setting. A client who is experiencing
active physiologicalsevere
effects oflife
the substance (or the with-
stressors due to a poor economy ordrawal
recenteffectsnatural
of the substance)
disas- have subsided.
ter may improve when the impact of these events on
his (and his friends' and loved ones')20. financial and
Regression to the mean.per-
It is a statistical fact of life
sonal life has dissipated. The clinician mayscores
that extreme erroneously
tend to become less extreme upon
attribute improvement during therapy to
retesting, the treatment
a phenomenon known as regression toward the
itself rather than to the salubrious changes
mean (Kruger,inSavitsky,
the client's
& Gilovich, 1999). By mathemati-
everyday life. cal necessity, regression to the mean will occur whenever
the correlation between pretest and posttest scores is less
18. Cyclical nature of some disorders.
than Another extrath-
unity (Salsburg, 2001); such regression will be espe-
cially
erapeutic factor that can be linked to pronounced when
short-term measures are of low reliability.
improve-
ment is the cyclical nature of many Ifdisorders (Beyerstein,
a patient presents to therapy as severely depressed,
1997). In contrast to spontaneous remission, whichhigh
chances are reasonably refers
that he or she will be less
to substantial amelioration in or disappearance
depressed (or at leastof a con-
report lower levels of depression
on standardized
dition per se, this CSTE refers to a transient outcome
shift measures)
into the in a few weeks,
even in the absence
benign phase of a condition characterized by a of recurrent
treatment.
course. Like many medical conditions, such as multiple
Regression to the mean is an especially thorny prob-
sclerosis, arthritis, and gastrointestinal problems, many of psychotherapy in
lem in evaluating the effectiveness
psychological disorders have theirreal-world
"ups and downs."
settings, because mostIn patients enter treatment
disorders that are cyclical, peoplewhenoftentheir improve,
symptoms are most peri-extreme and hence when
regression
odically or over the long term, without effects are maximized
intervention. For (Gilovich, 1991).
example, in cyclothymic and bipolar disorders, which
Similarly, antisocial children and adolescents may not be
referred to treatment
are characterized by affective, interpersonal, and until their behaviors become
behav-
ioral lability, an ineffective treatment implemented
unbearable to teachers or parentsover (Costello & Janiszewski,
a lengthy period will have ample opportunities to coin-
1990). Some authors have conjectured that most of the
cide with upticks that likely would have
varianceoccurred
commonly regard-
attributed to placebo effects in con-
less of treatment. Accordingly, clinicians may
trolled trials infer that
of medication is actually due to regression
therapy is responsible for improvement whenMazzuca,
effects (McDonald, positive & McCabe, 1983). Moreover,
changes are instead induced by fluctuations in the disor-
regression effects may sometimes be misinterpreted as
ders' natural course. One likely reason for the
spontaneous popularity
remission (Campbell & Kenny, 1999).
of unvalidated and fringe treatments for autism
Rendering this CSTEspectrum
especially problematic are find-
disorders, such as secretin (a polypeptide hormone
ings that humans are pronesyn-to nonregressive predictions
thesized from the intestines of pigs)(Dawes,
and 1986; Nisbett & Ross, 1980). That is, we do not
sensory-motor
integration therapy, is the fact that the corollary
sufficiently symptoms
compensate for regression to the mean when

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
predicting undergoing behavior fro

temporary life stressors are at times mistak-
because ofenlythe diagnosed as psychopathological;
represen when they are
shortcut later examined, they have improved but not necessarily by
characterized
with like" (Tversky
because of the treatment. The same may hold for clients&
behavior at with Time
acute medical disorders that
A are misdiagnosed
and as B
we detect a difference between Time A and Time B psychiatric conditions. For example, acute intermittent
behavior, we tend to attribute this change to spurious
porphyria has been called "a great imitator" (Morrison,
extraneous factors, such as the effects of treatment,1997,
ratherp. 155) and is occasionally mistaken for bipolar
than to statistical regression (an error knowndisorder as the and other cyclical emotional conditions. If this
regression fallacy, Kahneman, 1965). As Campbell medical
and condition resolves on its own, which it some-
times does (Loftus & Arnold, 1991), an unwary clinician
Kenny (1999) commented, "it seems likely that regression
may mistakenly conclude that a treatment targeted for a
toward the mean leads people to believe in the efficacy
manic aepisode was beneficial.
of the scientifically unjustified regimens. . . . Many
quack has made a good living from regression toward
the mean" (p. 48). Category 3 CSTEs: Misinterpretations of actual cli-
ent change stemming from nonspecific treatment
21. Maturation. A source of erroneous inferences factors
of therapeutic efficacy, especially among children and24. Placebo effects. The omnipresent placebo effect
adolescents, is maturation: improvement owing to has been defined in multiple ways, but it is traditionally
natu-
regarded as improvement resulting from the mere expec-
rally occurring psychological growth (Cook & Campbell,
1979). For example, children and young adolescents tation
with of improvement (Beecher, 1955; S. Horowitz, 2012;
high levels of what appear to be certain pre-psycho- Steer & Ritschel, 2010). By instilling hope and the convic-
tion that one can rise above life's challenges, virtually any
pathic features, such as poor impulse control, low frustra-
tion tolerance, and defiance, may improve on their credible
own treatment can be at least somewhat helpful for
because levels of these characteristics often diminishcombating demoralization (Frank & Frank, 1961), which
is a central component of many psychological disorders
with the passage of time, especially when they are early
appearing (Edens, Skeem, Cruise, & Cauffman, 2001). (Tellegen et al., 2003). Admittedly, importing the placebo
Such maturation can mislead clinicians into concludingconcept into the domain of psychotherapy is fraught with
that their treatment was responsible for declines in complexities
the given that at least some of the efficacy of
psychological treatment probably derives from expec-
levels of these and other externalizing problems. Psycho-
logical growth may be a source of mistaken therapeu- tancies of improvement (Kirsch, 2005; Lambert, 2005).
tic conclusions even among adult clients. For example,Nevertheless, because such expectancies presumably
some patients with borderline personality disorder maycut across most or all effective psychotherapies, they can
improve over long stretches of time without treatment lead clinicians and researchers to conclude that the spe-
(Shea et al., 2009). cific ingredients of a treatment are efficacious when they
are inert.
22. Multiple treatment interference. When clients seek In the case of medication, some research suggests that
out a treatment, they often obtain other interventions up to 80% of the effects of antidepressants on clinical
simultaneously (Kendall, Butcher, & Holmbeck, 1999), depression,
a especially when it mild or moderate, may be
confound known as multiple treatment interferenceattributable
or to placebo effects (Kirsch, 2005; Kirsch &
co-intervention bias. Some of these adjunctive interven-Sapirstein, 1998; but see Coyne, 2012; Klein, 1998, for dif-
tions may be formal treatments, such as antidepressants ferent views). Placebos generally exert their most potent
or marital therapy. Others may be informal "treatments,"effects on subjective reports, such as depression, pain,
such as exercise, which has generally been found in con-and nausea, rather than on largely objective indices, such
as assays of cancer, heart disease, or other organic ill-
trolled studies to be effective for alleviating depression
(Fremont & Craighead, 1987; Penedo & Dahn, 2005), or nesses (Hróbjartsson & Gotzsche, 2001).
confiding in trusted friends or religious figures. Multiple Placebo effects appear to play an important role in the
treatment interference renders it difficult or impossibleefficacy
to of psychotherapy, too. Estimates of placebo
effects in psychotherapy, typically obtained by compar-
attribute client change conclusively to the active ingredi-
ents of the intervention of choice. ing treatment outcomes from attention-placebo control
groups with those of wait-list control groups, are on the
order of d = 0.40, or about half of the typical effect size
23 ■ Initial misdiagnosis. Even the best trained diagnos-
yielded by active therapies (Grissom, 1996; Lambert,
ticians are fallible (Beyerstein, 1997; Garb, 1998; Groop-
man, 2007). For example, relatively normal individuals 2005; Lambert & Ogles, 2004). Moreover, meta-analyses

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
including major depression

indicate that the estimated and eating disorders,
efficacy ofperhaps
psy
60% to 80% of clinical
considerably smaller when it is compared w improvement occurs by the fourth
tion-placebo controlsession (Ilardi & Craighead,
group than 1999; G. T. Wilson, 1999;a
with but wai
group (Baskin, Tierney,
see Tang & DuRubeis,Minami, & Wam
1999). Much of the early change in
Bowers & Cium, 1988),psychotherapy may suggesting
similarly reflect clients' reactionsthat
to a
new intervention
potency of psychological that offers the promise
treatment of change,
derives fr
although some of it may also stem from placebo or
specific effects of expectancies.
Still, ascertaining regression
theeffects. precise magnitude
Novelty effects may account in part for
meta-analytic findings
effects in psychotherapy that the effect sizes for and
is difficult the efficacyargu
of some psychotropic
sible given the absence of a medications,
perfect includingpsycho
second-
ment analogue to a generation
pill antipsychotics
placebo (Kirsch,
and antidepressants, have been 200
even control conditions designed
highest shortly following to
their introduction, be
only to dis- act
sipate with
may not control fully fortime (Lehrer,
the 2010;effects
Leucht, Arbter, Engel,of ex
Kissling, & Davis,
these placebos are often less2009), although other factors (e.g.,
plausible than
interventions against which
enrollment they
of progressively arein pitted
milder patients medication (B
Stothart, & Stutts, 2013).
studies, publication bias) may also be at play.
Placebo effects should not be confused with other
nonspecific effects of treatment (Kienle & Kiene, 1997; 26.
cf. Effort justification. Because clients often devote
Novella, 2010), such as those of empathy and support substantial time, energy, effort, and money to treatment,
(Nathan, Stuart, & Dolan, 2003). The causal role of they
thesemay feel a need to justify this investment. They may
do so
nonspecific factors is controversial. On the one hand, theby persuading themselves that the therapy was
beneficial, a phenomenon termed effort justification
therapeutic alliance is modestly and positively associated
(Cooper, 1980; Cooper & Axsom, 1982). In one study,
(average r = .22) with therapeutic improvement (Baldwin,
college students with snake phobic symptoms improved
Wampold, & Imel, 2007; Orlinsky, R0nnestad, & Willutzki,
2004). This finding has led some scholars to contendequally
that when receiving exposure therapy and when
the therapeutic alliance is a causal agent in psychothera-
performing strenuous physical exercises (e.g., running
peutic change. On the other hand, relatively few therapy
quickly in place). The latter "treatment" required consid-
outcome studies account for the temporal relation erable effort and presumably led to a need to rationalize
between the alliance and improvement, precludingthis rela-
effort (Axsom & Cooper, 1985). Effort justification
tively clear-cut inferences of causality (Kazdin, 2007).
may be a particularly challenging interpretative problem
Several investigations that have incorporated assessments
for long-term insight-oriented therapies, especially those
of therapeutic alliance and symptom change at multiple
lasting decades, because of the enormous financial, time,
and emotional investment involved.
therapeutic time points suggest that a positive alliance
typically follows symptom change, not vice versa
(DeRubeis, Brotman, & Gibbons, 2005; DeRubeis &
Feeley, 1990); but other studies have arrived at different
Summary
conclusions (Horvath, Del Re, Flückiger, & Symonds, These 26 CSTEs are a helpful springboard for examining
2011; Norcross & Lambert, 2006). In light of this mixed why certain inert or harmful treatments (or treatment
evidence, we do not class these nonspecific factors as ingredients) may appear to be effective. Our list is only a
CSTEs. starting point, however, because CSTEs almost surely
comprise only one set of sources for incorrect inferences
25. Novelty effects. Clients may improve, especially regarding treatment effectiveness. Other sources include
at the outset of treatment, because they are excited by the fact that clinicians are often extremely busy and are
the prospect of receiving a new intervention (Fraenkeltherefore
& forced to make rapid decisions in complex and
Wallen, 1993; Marino & Lilienfeld, 2007). Novelty effectsinformation-rich environments. Moreover, as noted ear-
probably overlap with placebo effects in some cases, but lier, another source of erroneous inferences comprises
the former typically operate largely or exclusively during
incorrect hypotheses regarding the specific mechanisms
the initiation of treatment. of a treatment. We encourage additional research on
Psychotherapy outcome data suggest that about 15% other potential CSTEs, as well as on shared processes that
of patients improve between the initial phone call from
may underpin superficially different CSTEs.
the clinician and the first session (K. I. Howard, Kopta,Like many rival hypotheses in psychology (Huck &
Sandler, 1979), CSTEs are readily overlooked because
Krause, & Orlinsky, 1986). At least some of this improve-
ment probably stems from the anticipation of receipt they
of are nonintuitive. In addition, they are less perceptu-
a novel treatment. Moreover, for many conditions, ally obvious than the easily observed impact of client

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
change and exposition

therefore is
ar
background only on the
(Lilienfeld e
some crucial
clinicians mayCSTEs
assu
dispense methodologic
with the resear
tice and replace it with in
Protecting
Research Methods
WelUvalidated outcome indicators . Well-validated
Against Causes of
and largely objective outcome measures help to rule out
Therapeutic Effect
all Category 1 CSTEs, because these CSTEs can engender
A key pointthe false appearance of change
that is in its not
absence. For exam-
em
tion in ple, well-validated indicators
clinical psycholoof depression or anxiety
systematic help research
to exclude - although not eliminate - illusory
des pla-
subject and cebo effects and palliative effects in controlled trials of
single-subje
mize CSTEs major as
depression rival
and anxiety disorders. To be effective
hypo
(Lilienfeld et
safeguards al,
against Category 2008;
1 CSTEs, well-validated out-
respects, come indicators
the should be sensitive not only to client
existence o
raison symptoms for
d'être but also to client impairment.
evidenc Such indicators
knowledge are also useful as protections
this crucialagainst Category 1 CSTEs inp
itly (but controlled single-subject
see Lilienfeld designs. In contrast, demand e
2011, for characteristics can be especially difficult to rule
discussions of out as
dence-based sources of erroneous clinical inference. Nevertheless,
practice).
outcome without
Specifically, measures that are low in reactivityrand (Weiss &
controlled Weisz, 1990), such as extrasession behavioral data or
quasi-experim
subject unobtrusive behavioral observations,
designs, and are at least partial
oth
safeguardsantidotes against this CSTE. Collateral reports
against CSTE from out-
whether side informants (e.g., friends,
client change significant others), which w
opposed tocan supply
a "social
wealth validation" (Kazdin, 1977), can
of be use- e
controlled ful in ruling out the confusion
trials are of insight
notwith improve- s
not ment, retrospective
remove all rewriting of pretreatment functioning,
potential
response shift
Nevertheless, bias, the therapist's office error, and
analyses of similar t
treatment CSTEs. Specifically, these reports can assist
designs based clinicians and
yield more investigators
replicable with excluding the hypothesisresu that client-
perceived change
experimental orin symptoms is (a) limited to behaviors
natural
probably atwithin therapy sessions, (b)
least in illusory, or (c) both.
part b
nate more CSTEs as rival e
As a Pretreatment measures . Collecting
consequence of measures of client
thei
randomized psychological status at pretreatment is especially helpfult
controlled
designs for ruling
justifiably out one specific Category 1 CSTE, namely, ret-
occup
tiary rospective
certainty rewriting of pretreatment
in functioning.
the Specifi-ev
(Ghaemi, cally, such measures can Neverthe
2009). assist in excluding the hypothesis
archy, suchthat clients
as are merely misremembering their initial
quasi-expe
ods, can adjustmentplay
also as worse than it actually was, thereby leading
valuabl
they help to
to spurious inferences
protect of improvement. If these measures
inv
(Wachtel, do not rely exclusively on Moreove
2010). self-report ratings, they can
pensable in also help
the to eliminate response-shift
early biases as explana-
ph
as they tions
allow for apparent improvement.
researchers
can shape the developm
turn, Blinding
these of observers. Blinded observations in con-
intervention
empiricallytrolled clinical trials control partially for several additional c
promising,
ously Category
controlled 1 CSTEs, especially
trials.those stemming from con-
In the firmationsection,
next bias and illusory correlation (i.e., selective
we
attention, memory, and interpretation
methodological procedu of client outcomes).
research When external evaluators
help to are fully blinded,
elimi they cannot

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
controlled
subtly and selectively perceive, recall, trials do not eliminate
or interpret ambig- Category 2 CSTEs,
uous symptom changes as a function whichof
still arise in these assign-
treatment investigations and can deceive
observers in the
ment. For example, blinded observers inabsence of randomized controlled
a randomized
controlled trial of cognitive-behavior therapy
groups. Nevertheless, theversus a
randomization process helps to
wait-list control for generalizedexclude
anxiety disorder
Category 2 CSTEs asare
rivalless
explanations for thera-
peutic effectiveness,
likely to differentially elicit or cherry-pick because of
indicators these CSTEs are equally
improvement (e.g., reports of less frequent
likely worrying)
in sizeable experimental and in
control groups. Given
the treatment condition. the law of large numbers, these CSTEs should no longer
Nevertheless, these Category 1 CSTEs may be difficult account for between-group differences in randomized
to eliminate entirely. Because therapy outcome studies controlled trials provided that clinical trials are adequately
cannot be strictly double-blinded (i.e., clients and clini- powered (Hsu, 1989). For example, in a randomized con-
cians know who is receiving treatment), confirmation trolled trial, spontaneous remission, history, regression to
bias can still affect ratings of improvement by clients and the mean, maturation, and multiple treatment interfer-
clinicians. Moreover, even the blinding of external observ- ence occur frequently among individuals assigned ran-
ers in psychotherapy trials is rarely infallible, as these domly to both treatment and no-treatment (or alternative
evaluators can often surmise treatment assignment at treatment) conditions. Nevertheless, proper randomiza-
above-chance levels (Carroll, Rounsaville, & Nich, 1994). tion ensures that these CSTEs tend to be equalized across
Assessing potential violations of blinding by asking eval-the active treatment and comparison arms.
uators to guess treatment conditions and using this vari-
able as a covariate in analyses can be a helpful safeguard Repeated measurements. In both between-subject
against selective perception, memory, and interpretation and controlled single-subject experiments, repeated mea-
of client change. Nevertheless, such covariate analyses surements across the course of treatment can help to rule
may underestimate treatment differences (especiallyout history and other extratherapeutic influences as
when based on guesses made at the conclusion of treat- sources of improvement in therapy (Laurenceau, Hayes,
ment), because above-chance guessing could stem from & Feldman, 2007). If one observes changes in treatment
evaluators' accurate observations of differential improve-at multiple time points rather than at only one time point
ment across conditions (Carroll et al., 1994; Rickeis, following an extratherapeutic event (e.g., initiation of a
Lipman, Fisher, Park, & Uhlenhuth, 1970). romantic relationship), ¿he likelihood that such events -
rather than the therapeutic intervention - are contribut-
Intent-to-treat analyses. Intent-to-treat (ITT) analyses ing to improvement is minimized (such observations are
(Hollis & Campbell, 1999) help to rule out one key Cat- also useflil for ruling out novelty effects, a Category 3
egory 1 CSTE, namely, selective attrition. By examiningCSTE). In the context of single-subject designs, multiple
outcomes of all participants enrolled in clinical trials, baseline designs - especially those in which the interven-
including dropouts, ITT analyses minimize erroneoustion is applied to different behaviors in a temporal
inferences of improvement stemming from the fact that sequence - can help to rule out history and other extra-
clients who leave treatment prematurely are often unrep- therapeutic factors as rival explanations for change dur-
resentative of those who initially enrolled (Tehrani et al., ing treatment (Engel & Schutt, 2012; Nock, Michel, &
1996). In contrast to clients who remain in treatment, Photos, 2007). If one consistently observes change in dif-
those who drop out of treatment tend to be lower func-ferent behaviors at different time points, the likelihood
tioning and more psychologically disturbed (Swift &that extratherapeutic factors account for the improve-
Greenberg, 2012), although in a minority of cases they ment is minimized. Finally, long-term follow-up measure-
comprise clients who have improved and no longer per- ments can be helpful in excluding CSTEs arising from the
ceive themselves as requiring treatment (Baekeland & cyclical and self-limiting nature of certain disorders, as
Lundwall, 1975; Tehrani et al., 1996). As a consequence such assessments can ensure that improvements in signs
of the selection biases introduced by client dropout, ITTand symptoms are not transient.
analyses help to avoid misestimating - and typically over-
estimating - treatment effects. Minimizing and estimating measurement error.
The use of pre- and posttreatment indicators with high
reliability will minimize regression to the mean, as this
Protecting against Category 2 CSTEs
statistical phenomenon is most probable when measures
Randomization to treatment conditions. Random- contain substantial amounts of nonsystematic (random)
ization to treatment conditions helps to address the measurement
infer- error. Particularly in quasi-experimental
ential errors generated by Categoiy 2 CSTEs, treatmentwhich studies, investigators should be circumspect in
produce changes stemming from extraneous factors out-
their use of extreme-groups designs (in which partici-
side of treatment. To be clear, well-executed randomized
pants are selected on the basis of very high pretreatment

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
scores), as are especially suited for

such excluding different CSTEs. The
designs ar
levels of need to minimize CSTEs using between-subject
regression effects and sin-
and control gle-subject research designs offers the most compelling for
statistically
outcome studies
rationale for evidence-based(see Bar
practice. Our discussion
2005, for a alsodiscussion).
points to important gaps in methodology for atten-
uating the influence of CSTEs as well as fruitful direc-
tions for future research. As is evident from our analysis,
Protecting against
Category 3 CSTEs are especially difficult to eliminate as
C
Common factor
erroneous sources of improvement,
contro
because equating
for active treatment
common and attention-placebo control condi-
therapeutic fa
gory 3 tions on expectancies
CSTEs, whichand treatment credibility isinvo
often
change to difficult or impossible. Hence, onetherap
specific key direction for
factors future psychotherapy
(e.g., expectanciesresearch will be the development
operative. of placebo conditions
For that are closely matched to treat-
example, a
placebo ment conditions on credibility
control groups (see Boot et al., 2013). (P
tancies for Moreover, because psychotherapyas
change trials cannot
an be con- ex
in ducted in a Neverthele
outcome. genuinely double-blind fashion, certain
groups are Category 1 CSTEs, especially selective to
unlikely attention, mem-
elim
by Category 3 ofCSTEs,
ory, and interpretation client outcomes, are difficult b
common factor to eradicate, particularlycontrol
for client and clinician reports c
as active treatment
of improvement. The development of largely condi
objective
et al., 2013; measures that are less susceptible to these &
O'Leary and otherBo
culty and observer biases is therefore an important
perhaps imposs direction for
tor control conditions with active treatment conditions on future psychotherapy outcome research. In the case of
expectancies for change, hope, and treatment credibility, all CSTE categories, researchers are well advised to
researchers and clinicians should ideally measure these heed the methodological maxim that if one cannot
factors at different points in treatment (in the case of remove a source of error, one should attempt to mea-
novelty effects, at the outset of treatment). These vari-sure it. For example, by systematically assessing expec-
ables can be treated as covariates in analyses, again bear-tancies during treatment, investigators can strive to rule
ing in mind that such statistical controls can underestimateout rival hypotheses concerning client improvement
treatment effects if expectancies and treatment credibilityand thereby draw more valid inferences regarding treat-
in part contribute to treatment efficacy. ment effects.
Inclusion of measures of proposed mediators. The Conclusions and Future Directions

demonstration that a proposed mediator of treatment
outcome accounts statistically for client improvement The oft-lamented gap between science and practice in
supports, although does not prove, the contention that clinical psychology is in large measure a clash of episte-
this mediator is the underlying mechanism of change mologies (McHugh, 1994). In particular, this schism
(see Kazdin & Nock, 2003, for conditions in which medi-reflects deep-seated differences of opinion regarding th
ation offers especially compelling evidence for change place of controlled research versus intuition in clinical
mechanisms). In this regard, mediational tests can be decision making (Lilienfeld et al., 2013; Tavris, 2003). Ou
helpful for excluding Category 3 CSTEs. Specifically, con- central thesis is that the science-practice gap and the
verging findings that a given psychotherapy appears to accompanying reluctance of some psychologists -
operate via a hypothesized mediator that is largely spe- clinicians and researchers alike - to adopt evidence-
cific to that intervention (e.g., changes in maladaptive based practices rarely reflect a willful disregard of
cognitions, cognitive defusion, increase in social rein- evidence per se. Instead, this reluctance stems largely
forcement) minimizes the likelihood that this intervention from an erroneous belief that the evidence supplied by
is operating exclusively via common mechanisms sharedinformal clinical observations of client change tends to
by most or all treatments, such as placebo effects. be as trustworthy as the evidence supplied by the meth
odological safeguards comprising the research prong o
evidence-based practice (Spring, 2007). When viewe
Summary through this lens, the science-practice gap is not funda-
Methodological techniques in psychotherapy outcome mentally a disagreement about whether evidence is
research help to control for CSTEs, and certain methods important in ascertaining therapeutic effectiveness: It is a

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
difference of opinion about which

need to waitkinds of evidence
weeks or months before discovering whether
a client improved
should be accorded priority in clinical followingmaking.
decision an intervention). Hence, the
literature on expertise effects provides scant reason to
expect the accuracy of intuitions concerning therapeutic
Implications for the roleeffectiveness
of intuition
to improve with experience. Research on
in clinical decision making the relation between the amount of therapeutic experi-
Clinical intuitions and informal observations play invalu-
ence and accuracy of clinical judgments offers few
able roles in psychotherapy, especially in hypothesis (Spengler et al., 2009) or virtually no (Garb, 1998, 2005)
generation (Chambless, in press). For example, the spark additional grounds for optimism.
that ignited Aaron Beck's seminal theorizing regarding
cognitive-behavioral therapy originated from his observa- Implications for prioritizing sources
tions of a client who seemed anxious during sessions.
of evidence
After Beck, who was trained psychoanalytically, sug-
gested to her that her anxiety reflected discomfort with These points bear noteworthy implications for the weight-
unconscious sexual impulses, she replied politely that ing of the three legs of evidence-based practice: research
she felt nervous because she was concerned she was evidence, clinical expertise, and client preferences and
boring him. This experience inspired Beck to explore his (Spring, 2007). Although some authors contend or
values
imply that these three prongs should be accorded approx-
clients' unstated thoughts and assumptions, culminating
imately equal weight in clinical decision making (e.g.,
in the development of what he initially termed cognitive
American
therapy (Smith, 2009). Moreover, clinical impressions of Psychological Association Task Force on
change during treatment are sometimes accurate and Evidence-Based Practice, 2006), our analysis suggests that
this
should be regarded as fallible but potentially informative ecumenical approach may be misguided. As we have
signposts to be corroborated by more systematic evi- seen, controlled research on treatment efficacy is better
dence. At the same time, our analysis is a remindersuited
thatthan unguided clinical judgment to ruling out man-
ifold rival explanations for improvement, a finding that
clinical observations are often poorly suited to detecting
accords with the superior replicability of medical findings
and evaluating the sources of improvement in treatment.
The evidence we have reviewed demonstrates that derived from randomized controlled trials compared with
(a) throughout history, ineffective and harmful less rigorously controlled trials (Ioannidis, 2005). Hence,
mental
when well-replicated treatment outcome data conflict
health treatments have routinely been perceived as effec-
tive; (b) psychotherapists frequently overestimate withsub-
clinical impressions of improvement, we should gen-
stantially the rates of positive outcomes in theirerally default to the former (Baker et al., 2008).
clients
(Hannan et al., 2005); and (c) many sources can contrib-
ute to the erroneous impression of therapeutic effective-
ness in its absence.
Implications for everyday clinical
practice
One potential response to our arguments is that CSTEs
are less of an impediment for highly experienced psy- Our arguments point to useful suggestions for everyday
chotherapists, who can gradually learn to distinguish clinical practice as well (see Table 1). Because of Category
accurate from inaccurate inferences of treatment effec- 1 CSTEs, clinicians can be led to conclude that client
tiveness. Nevertheless, research across multiple domains change has occurred when it has not. One underutilized
reveals that the conditions for the acquisition of intuitive corrective to this problem is the periodic administration
expertise are highly constrained. Intuitive expertise tends of outcome measures, such as the Outcome
to emerge only in "high-validity environments" - those in Questionnaire-45 (Lambert, Lunnen, Umphress, Hansen,
which feedback is relatively objective, consistent, and & Burlingame, 1994), throughout treatment. These mea-
immediate (Dawes, 1994; Kahneman & Klein, 2009; sures can alert clinicians to instances in which they may
Tracey, Wampold, Lichtenberg, & Goodyear, 2014). None be erroneously perceiving improvement in its absence or
of these conditions apply to typical psychotherapy, a overlooking deterioration. Some Category 1 CSTEs, espe-
"low-validity environment" in which feedback to clini- cially those stemming from confirmation bias on the part
cians is often ambiguous (e.g., detecting whether a client of both clinician and client, can be minimized by collect-
is less anxious than in the previous session can be chal- ing systematic data from outside informants. In some
lenging, and detecting whether such change is due to the cases, as in the treatment of anxiety disorders, these
intervention itself is even more so), inconsistent (e.g., a CSTEs can also be minimized by collecting psychophysi-
client may appear improved in one session but not in the ological data (e.g., autonomic responsivity to anxiety-
succeeding session), and delayed (e.g., clinicians may provoking events) over the course of treatment.

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
376 Lilienfeld et al
Although In our view, the inferential

many Category problems posed by CSTEs, 2
ruleout in and, equally important, the ways in which
everyday research safe-
clinica
enhance the accuracy
guards compensate of
for them, should be mandatory com-
ponents of by
effectiveness training for all mental health professionals.
inquirin
cally. For Exposing students to the long and
example, by sordid history
mon of failed
adjunctive but widely espoused treatments in psychology and psy-
"interventions,
edies, and confiding
chiatry may be especially helpful as a didactic indevice. In
become more addition, a cognizant
thoughtful consideration of CSTEs shouldof be
ence as a potential explan
integrated routinely into clinical supervision and case pre-
ents' clinical status. sentations. For example, when reviewing client improve-
Our analysis also reminds clinicians to be attuned toment over the course of treatment, supervisors should
the possibility that some client characteristics may serveencourage trainees to carefully consider (a) rival explana-
as moderators of certain CSTEs, thereby affecting theirtions for such improvement other than, or in addition to,
likelihood. For example, as noted earlier, people with the intervention itself, (b) cognitive biases that may lead to
depression may be especially prone to symptom over- false inferences in this regard, and (c) safeguards against
reporting and hence to the false appearance of improve-drawing erroneous inferences concerning the existence
ment on adjunctive symptoms following treatmentand sources of improvement. Nevertheless, because it is
(Morgado et al., 1991). Similarly, because individualsnot known whether instruction in CSTEs enhances thera-
with high levels of negative emotionality, especiallypeutic outcomes, we call for research on this question. The
trait anxiety, are prone to attend selectively to psycho-absence of such evidence notwithstanding, education
logical symptoms (Suis & Howren, 2012), cliniciansregarding CSTEs may diminish resistance to evidence-
should be alert to the possibility that declines in nega-based practice among future and current clinicians, as
tive emotionality over the course of treatment could such knowledge provides a persuasive rationale for reli-
predispose to spurious inferences of declines in other ance on research designs to gauge therapeutic efficacy
psychological symptoms. In addition, although effortsand effectiveness (Lilienfeld et al., 2013).
to identify a "placebo-prone personality" have met with Our arguments also bear implications for training
mixed success, some evidence raises the possibility that models in clinical psychology. For example, the local
optimists are more likely than pessimists to respond to clinical scientist model (Strieker & Trierweiler, 1995),
positive expectancies (Geers, Helfer, Koskab, Weiland, adopted by many or most scholar-professional (Psy.D.)
& Landry, 2005) and hence may be especially prone to programs, encourages clinicians to operate as scientists
engendering certain CSTEs, especially placebo and within the miniature laboratory of the clinical setting,
novelty effects. carefully observing client behaviors in response to inter-
ventions, generating hypotheses about them, and testing
these hypotheses with additional interventions. In prin-
Implications for clinical psychology
ciple, these are laudable goals. Nevertheless, CSTEs raise
education and training largely unappreciated challenges for the implementation
Our analysis implies that CSTEs and the research safe- of the local clinical scientist model, because they render
guards against them that we have delineated should be it difficult to draw reasonably clear-cut conclusions
emphasized in the education and training of all would-regarding treatment effectiveness for individual clients.
be psychologists and other mental health professionals, Hence, although local ("idiographic") clinical science cer-
as well as in the continuing education of current mentaltainly has its merits, it cannot substitute for nomothetic
health professionals. Although we are unaware of sur- clinical science derived from randomized controlled tri-
vey data on how often CSTEs are discussed in graduateals, single-subject designs, and other systematic research
courses in the mental health professions, there is rea- methods.
son to believe that such coverage is often minimal. At the same time, idiographic clinical science is hardly
Most standard psychotherapy handbooks (e.g., Corsini a dead end, so clinicians need not despair. Although cli-
& Wedding, 2010; Koocher, Norcross, & Hill, 2005; nicians operating in the context of individual clients can-
Meyer & Deitsch, 1996) accord scant attention to thenot exclude many alternative explanations for client
overarching problem of CSTEs or to specific CSTEs improvement, especially Category 3 CSTEs, they can
themselves, such as placebo effects, spontaneous remis- nonetheless evaluate client change through the prism of
sion, and regression to the mean. Moreover, to our CSTEs. In this way, they can become more alert to alter-
knowledge, no continuing education course approvednative explanations for change. For example, as noted
by the American Psychological Association has everearlier, clinicians can monitor client change systemati-
focused on CSTEs. cally across sessions, thereby permitting them

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Why Ineffective Psychotherapies Appear to Work òli
demoralization"
to minimize illusory placebo effects; (Cook can
they & Campbell,
solicit 1979) as a conse-
information from informants quence
regarding clients'
of not receiving out-of-
treatment afforded to other indi-
viduals,
session behaviors to minimize the or both. Nevertheless,
therapist's office the evidence for this
error;
they can attend diligently to all relevant
assertion client
is mixed (e.g., outcomes
S. A. Elliott & Brown, 2002). Still
other authors
to avoid inadvertent cherry-picking of argue that "treatment
signs and symp- as usual" conditions,
toms; they can be alert to the which
fact that
often serve client
as control improve-
groups in psychotherapy
ments over time may reflect outcome
regression
designs, are to the mean,
best conceived of as "intent to fail"
conditions,
history, and other artifacts; they candefined
attend (perhaps
totendentiously)
potential as "pseudo-
treatments designed
client characteristics that may moderate the specifically
likelihood as control
of groups to
CSTEs; and so on. In this respect,
prove they canofadopt
the superiority a scien-
the investigator's preferred treat-
ment and that
tific mind-set while bearing in mind that have various
no theoretical sources
rationale or are deliv-
of inferential error cannot be ered by graduate students
completely who know they are
eliminated.
Hence, the local clinical scientist model,
administering although
treatment not
that is not an to work"
supposed
adequate substitute for scientist-practitioner or If
(Westen & Bradley, 2005, p. 267). clinical
these critics are cor-
science models of training (see Baker
rect, et al.,
some standard 2008),
psychotherapy is adesigns may
outcome
helpful reminder that clinicians should
overestimate continually
the efficacy oper- or gen-
of beneficial treatments
ate as "detectives" who strive toerate identify
the mistaken conclusion that inefficacious
potential rival treat-
sources of improvement and who
ments (a) minimize these
are efficacious.
sources when they can and (b) bear
Third, them
some readers mightin mind
contend that ouras
core argu-
ments
inferential constraints when they are rendered effectively moot by the Dodo Bird
cannot.
verdict, named after the Dodo Bird in Lewis Carroll's
Adventures of Alice in Wonderland , who proclaimed fol-
Limitations of our analysis
lowing a race that "Everybody has won, and all must
Our analysis is limited in at least three respects. First, our
have prizes." This verdict posits that all psychotherapies
review leaves unresolved the question of how often each are (a) effective and (b) equivalent in their effectiveness,
of the 26 CSTEs we have identified contributes to errone- both overall and for all disorders (Luborsky, Singer, &
ous inferences in actual clinical practice. As in many Luborsky, 1975; Shedler, 2010; Wampold et al., 1997). If
domains of psychology, one must distinguish "can" from the Dodo Bird verdict is correct, the reasoning continues,
"does" in discussions of causality (McCall, 1977). The fact CSTEs are of little or no concern because all treatments
that a CSTE can lead to incorrect inferences of therapeu- work, and work equally well (see Stewart et al., 2011).
tic effectiveness does not tell us how often it does so. Nevertheless, the Dodo Bird verdict has historically
Research examining therapists' knowledge of and under-referred to a rough equivalence in the effectiveness of
standing of CSTEs, both in the abstract and in real-world different schools of therapy (e.g., psychodynamic, cogni-
practice, would be a useful starting point in addressingtive-behavioral) rather than to a precise equivalence in
this question. the efficacy of all specific treatments (e.g., Smith, Glass,
Second, we have focused only on inferential errors & Miller, 1980). Moreover, the assertion that all therapies
that apply to everyday clinical practice. We have not are of equal efficacy, either overall (a main effects hypoth-
examined the many methodological decisions that can esis) or for all conditions (an interactional hypothesis), is
generate spurious inferences of treatment effectiveness in difficult to sustain (Lilienfeld, 2014; but see Wampold,
research studies of all kinds. For example, the file-drawer 2001, for a more sanguine perspective on the Dodo Bird
effect (Rosenthal, 1979), which is the bias against submit- verdict). For example, well-replicated data indicate that
ting negative results for publication, and outcome report- exposure-based therapies are more efficacious than other
ing bias (Chan & Altman, 2005), which is the propensitytreatments for at least some anxiety-related disorders
to cherry-pick data on dependent measures that yield (e.g., obsessive-compulsive disorder) and that behavioral
positive results, can lead to overestimates of treatmenttherapies are more efficacious than nonbehavioral thera-
efficacy. Recent data also raise the possibility of a discon- pies for child and adolescent behavioral problems
certingly high prevalence of "p-hacking," that is, analyz- (Chambless & Ollendick, 2001; Hunsley & Di Giulio,
ing data - or peeking repeatedly at already collected2002). A meta-analysis by Tolin (2010) similarly revealed
data - until alpha levels fall just below .05 (Masicampo &that behavioral and cognitive-behavior therapies are
Lalande, 2012). More directly relevant to clinical practice, more efficacious than other therapies for anxiety and
some researchers also contend that the use of wait-list mood disorders. Further calling into question the Dodo
control groups contributes to overestimates of psycho- Bird verdict are findings that at least some interventions,
therapy efficacy, because clients in these groups maysuch as CISD, are at best ineffective and perhaps harmful
deteriorate as they await treatment, experience "resentful(Lilienfeld, 2007; McNally et al., 2003). Even Bruce

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Wampold, Notes
a prominent p
dict, acknowledges
1. Note that in a case in which the client'sthatdistress concerning
"bona-fide" therapies,
her self-centeredness and hostility was itself a treatmentna focus,
chological principles,
the response-shift bias would not be considered a CSTE. del
therapists,2. A and
related cognitive error
laidis the subset fallacy
out (Dawes, 2001).
publicationsA treatment may not
(Wampold differ in its effects from those of a control
&
treatment;
Furthermore, yet on a post
our hoc basis, investigators
discu may identify
a subset of clients within the treatment group who displayed
only to "schools" of psy
positive outcomes, leading them to conclude that the treatment
therapeutic techniques,
was efficacious for that subset. Yet if there was no mean differ-
m
treatment modalities. Th
ence between the treatment and control groups, mathematically
to false
inferences regardi
the individuals in the treatment group outside of the subset
techniques must
delivered with
have become worse following the intervention.
interpretation of
3. Nevertheless, especially a with
among clients clien
marked anxiety
imparted to a
and depression, client,
memory or
biases may run in the opposite direc-
ent and clinician. Hence,
tion, predisposing them to selectively recall threatening or sad
Dodo Bird information
verdict, (Coles & Heimberg, 2002; R. Elliott,
the Rubinsztein,
i
CSTEs in Sahakian, &psychother
the Dolan, 2002).
References
Closing thoughts
Addis, M. E., & Carpenter, K. M. (2000). The treatment rationale
The challenges posed
in cognitive-behavioral by
therapy: Psychological mechanisms
simism, letand clinical
alone guidelines. Cognitivenihilis
Therapy and Research ,
practice or 24, 313-326.
research sett
associated with them are to some extent surmountable.
Alexander, F. F., & French, T. M. (1946). Psychoanalytic ther-
Nevertheless, CSTEs underscore the pressing need to apy: Principles and application. Oxford, England: Ronald
Press.
inculcate humility in clinicians, researchers, and students
(McFall, 1991). We are all prone to neglecting CSTEs, notAlpert, J. (2012, April 22). In therapy forever? Enough already.
The New York Times , p. SR5.
because of a lack of intelligence but because of inherent
American Psychiatric Association. (2013). Diagnostic and sta-
limitations in human information processing (Kahneman,
tistical manual of mental disorders (5th ed.). Washington,
2011). As a consequence, all mental health professionals DC: American Psychiatric Association.
and consumers should be skeptical of confident procla-American Psychological Association Task Force on Evidence-
mations of treatment breakthroughs in the absence of rig- Based Practice. (2006). Report of the 2005 Presidential
orous outcome data (Dawes, 1994; Lilienfeld et al., 2003). Task Force on Evidence-Based Practice. Washington, DC:
CSTEs are potent reminders that although our intuitions American Psychological Association.
are at times accurate, they can be misleading. When eval-Axsom, D., & Cooper, J. (1985). Cognitive dissonance and
uating treatment effectiveness, our intuitions may fail to psychotherapy: The role of effort justification in inducing
account for numerous rival hypotheses for change that weight loss. Journal of Experimental Social Psychology , 21 ,
149-160. doi: 10. 1016/0022-1031(85)90012-5
are difficult or impossible to detect without the aid of
Bachrach, H. M., Galatzer-Levy, R., Skolnikoff, A., & Waldron,
finely honed research safeguards. As a consequence,
S. (1991). On the efficacy of psychoanalysis. Journal of the
CSTEs highlight the inherent limits of our knowledge as American Psychoanalytic Association, 39 , 871-916.
applied to the individual client and should impel us to beBaekeland, F., & Lundwall, L. (1975). Dropping out of
mindful of our propensities toward overconfidence. treatment: A critical review. Psychological Bulletin , 82,
Science, which is a systematic approach to reducing 738-783.
uncertainty in our inferences (McFall & Treat, 1999;Baker, T. B., McFall, R. M., & Shoham, V. (2008). The current
O'Donohue & Lilienfeld, 2007), is ultimately our best pre- status and future of clinical psychology: Towards a scien-
scription against being deceived by inadequate evidence. tifically principled approach. Psychological Science in the
Public Interest, 9, 67-103.
Acknowledgments Baldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007). Untangling
the alliance-outcome correlation: Exploring the relative
The authors thank Sean Carey and Ben Johnson for their valu-
importance of therapist and patient variability in the alli-
able assistance with compiling references.
ance. Journal of Consulting and Clinical Psychology , 75,
842-852 . doi : 10. 1037/0022-006X.75 .6.842
Declaration of Conflicting Interests Barkham, M., Hardy, G. E., & Mellor-Clark, J. (2010). Developing
The authors declared that they had no conflicts of interest with and delivering practice-based evidence. Chichester,
respect to their authorship or the publication of this article. England: Wiley.

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Barlow, D. H. (2004). PsychologicalPsychological

treatments. American
Measurement , 44, 781-804. doi: 10. 1177/
Psychologist , 59, 869-878. 0013164484444002
Bridges, M.
Barlow, D. H., Hayes, S. C, & Nelson, R.R. O.
(2006). Activating
(1984). the corrective
The sci- emotional
experience. Journal of in
entist-practitioner: Research and accountability Clinical Psychology, 62, 551-568.
clinical
and educational settings. New York, NY: Pereamon. doi: 10; 1002/jclp. 20248
Barnett, A. G., van der Pols, J. C., & Dobson, A. J. (2005). Bruner, J. S. (1990). Acts of meaning. Cambridge, MA: Harvard
Regression to the mean: What it is and how to deal with it. University Press.
International Journal of Epidemiology. 34 , 215-220. Bull, J. P. (1959). The historical development of clinical thera-
Baskin, T. W., Tierney, S., Minami, T., & Wampold, B. E. (2003). peutic trials. Journal of Chronic Diseases , 10 , 218-248.
Establishing specificity in psychotherapy: A meta-analysis Busch, A. M., Kanter, J. W., Sedivy, S. K., & Leonard, J. L. (2007).
of structural equivalence of placebo controls. Journal A follow-up analogue study on the effectiveness of the cogni-
of Consulting and Clinical Psychology , 71, 973-979. tive rationale. Cognitive Therapy and Research, 31 , 805-815.
doi: 10. 1037/0022-006X.71 .6.973 Campbell, D. T., & Kenny, D. A. (1999). A primer on regression
Beecher, H. K. (1955). The powerful placebo. Journal of the artifacts. New York. NY: Guilford Press.
American Medical Association, 159 , 1 602-1606. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-
Belofsky, N. (2013). Strange medicine: A shocking history of experimental designs for research on teaching. In N. L.
real medical practices through the ages. New York, NY: Gage (Ed.), Handbook of research on teaching (pp. 171-
Penguin Books. 246). Chicago, IL: Rand McNally.
Beutler, L. E. (2004). The empirically supported treatments Carey, B. (2012, May 18). Psychiatry giant sorry for backing gay
movement: A scientist-practitioner's response. Clinical "cure." The New York Times. Available from http://www
Psychology: Science and Practice, 11, 225-229. .nytimes.com
Beyerstein, B. L. (1997). Why bogus therapies seem to work. Carlier, I. E., Voerman, A. E., & Gersons, B. R. (2000). The
Skeptical Inquirer , 21 , 29-34. influence of occupational debriefing on post-traumatic
Bigby, M. (1998). Evidence-based medicine in a nutshell. stress symptomatology in traumatized police offi-
Archives of Dermatology, 123 , 1609-1618. cers. British Journal of Medical Psychology, 73 , 87-98.
Blanco, F., Barberia, I., & Matute, H. (2014). The lack of side doi: 10. 1348/00071 1200160327
effects of an ineffective treatment facilitates the develop- Carroll, K. M., Rounsaville, B. J., & Nich, C. (1994). Blind man's
ment of a belief in its effectiveness. PLoS ONE , 9(1), e84084. bluff: Effectiveness and significance of psychotherapy and
Bloom, P. B. (1994). Is insight necessary for successful treat- pharmacotherapy blinding procedures in a clinical trial.
ment? American Journal of Clinical Hypnosis, 36, 172-174. Journal of Consulting and Clinical Psychology, 62, 276-
Bloom, P., & Weisberg, D. (2007, May 18). Childhood ori- 280. doi:10.1037/0022-006X.62.2.276
gins of adult resistance to science. Science, 316, 996-997. Castelnuovo, G. (2010). Empirically supported treatments
doi: 10. 1 126/science. 1 133398 in psychotherapy: Towards an evidence-based or evi-
Bohart, A. C. (2005). Evidence-based psychotherapy means dence-biased psychology in clinical settings? Frontiers in
evidence-informed, not evidence-driven. Journal of Psychology, 1, 27-38.
Contemporary Psychotherapy, 35, 39-53. Cautin, R. L. (2008). David Shakow and schizophrenia research
Boisvert, C. M., & Faust, D. (2002). Iatrogenic symptoms in at Worcester State Hospital: The roots of the scientist-
psychotherapy: A theoretical exploration of the potential practitioner model. Journal of the History of the Behavioral
impact of labels, language and belief systems. American Sciences, 44, 219-237. doi:10.1002/jhbs.20312
Journal of Psychotherapy, 56, 244-259. Chabris, C., & Simons, D. (2010). The invisible gorilla: And other
Boot, W. R., Simons, D. J., Stothart, C., & Stutts, C. (2013). The ways our intuitions deceive us. New York, NY: Crown.
pervasive problem with placebos in psychology why active Chadwell, B. H., & Howell, R. J. (1979). Research note: A self-
control groups are not sufficient to rule out placebo effects. report measure of spontaneous remission and psycho-
Perspectives on Psychological Science, 8, 445-454. therapy outcome. Journal of Community Psychology, 7,
Bootzin, R. R., & Bailey, M. T. (2005). Understanding placebo, 69-71. doi: 10. 1002/1520-6629(197901)7: l<69::AID-JCOP
nocebo, and iatrogenic treatment effects. Journal of Clinical 22900701 13>3.0.CO;2-6
Psychology, 61, 871-880. Chambless, D. L. (in press). Can we talk? Fostering interchange
Bourne, H. (1953). The insulin myth. Lancet, 262, 964-968. between scientists and practitioners. Behavior Therapy.
doi:10.10l6/S0l40-6736(53)91232-X Chambless, D. L., & Hollon, S. D. (1998). Defining empirically
Bowers, T. G., & Cium, G. A. (1988). Relative contribu- supported therapies. Journal of Consulting and Clinical
tion of specific and nonspecific treatment effects: Meta- Psychology, 66, 7-18.
analysis of placebo-controlled behavior therapy research. Chambless, D. L., & Ollendick, T. H. (2001). Empirically sup-
Psychological Bulletin, 103 , 315-323. ported psychological interventions: Controversies and
Braffman, W., & Kirsch, I. (1999). Imaginative suggestibility and evidence. Annual Review of Psychology, 52, 685-716.
hypnotizability: An empirical analysis .Journal of Personality doi : 1 0. 1 1 46/annurev. psych .52.1 .685
and Social Psychology, 77, 578-587. doi: 10. 1037/0022- Chan, A., & Altman, D. G. (2005). Identifying outcome report-
3514.77.3.578 ing bias in randomised trials on PubMed: Review of publi-
Bray, J. H., Maxwell, S. E., & Howard, G. S. (1984). Methods cations and survey of authors. British Medical Journal, 330,
of analysis with response-shift bias. Educational and 753-759. doi:10.1136/bmj.38356.424606.8F

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Chapman, L.American
J., Psychologist,& 55, 264-268. doi: 10. 1037/0003-
Chapma
but erroneous066X.55.2.264 psychodiag
Abnormal Dehue, T. (2005). History of the control group. In B. Everitt
Psychology, 72 & ,
Chwalisz, K. D. Howell (Eds.), Encyclopedia of statistics Evide
(2003). in the behav-
for ioral science (do. 829-836). Chichester, England: Wilev.
twenty-first-century
Counseling DeRubeis, R. J., Brotman, M. A., & Gibbons, C. J. (2005). A
Psychologist, 3
Coles, M. E., &
conceptual Heimberg
and methodological analysis of the nonspecifics
the anxiety argument. Clinical Psychology: Science and Practice, 12,
disorders: Cur
Review, 22 ,
174-183. 587-627.
doi: 10. 1093/clipsy/bpi022 doi:
Conway, M.,DeRubeis,& R. J., & Feeley,
Ross, M. (1990). DeterminantsM.of change (1
revising what in cognitive therapy
you for depression. Cognitive
had. Therapy and Jo
Psychology , Research,
47, 14, 469-482. doi: 10. 1007/BF01 172968
738-748. d
Cook, T. D.,Diefenbach,
& G. J., Diefenbach,
Campbell, D., Baumeister, A., & West, M. D
Design and (1999). Portrayal of lobotomy in the popular press:
analysis 1935-
for
McNally. 1960. Journal of the History of the Neurosciences, 8, 60-69.
Cooper, J. (1980). Reducing fears and increasing assertiveness: doi: 10. 1076/jhin.8. 1 .60. 1766
The role of dissonance reduction. Journal of Experimental Dimidjian, S., & Hollon, S. D. (2010). How would we know if
Social Psychology, 16, 199-213. doi: 10. 1016/0022-1031 psychotherapy were harmful? American Psychologist, 65,
(80)90064-5 21-33.
Cooper, J., & Axsom, D. (1982). Effort justification in psycho- D'Silva, K., Duggan, C., & McCarthy, L. (2004). Does treatment
therapy. In G. Weary & H. Mireis (Eds.), Integrations of really make psychopaths worse? A review of the evidence.
clinical and social psychology (pp. 98-121). New York, NY: Journal of Personality Disorders, 18, 163-177. doi: 10. 1521/
Oxford University Press. pedi. 18.2. 163.32775
Coover, J. E., & Angell, F. (1907). General practice effect of spe- Dunning, D., Heath, C., & Suis, J. M. (2004). Flawed self-
cial exercise. American Journal of Psychology, 18, 328-340. assessment: Implications for health, education, and the
doi: 10.2307/1412596 workplace. Psychological Science in the Public Interest, 5,
Coren, S. (2003). Sensation and perception. In D. K. Freedheim 69-106. doi:10.1111/j.l529-1006.2004.00018.x
(Ed.), Handbook of psychology: History of psychology (Vol. Edens, J. F., Skeem, J. L., Cruise, K. R., & Cauffman, E. (2001).
1, pp. 85-108). Hoboken, NT: Wiley. Assessment of "juvenile psychopathy" and its association
Coronary Drug Project Research Group. (1975). Clofibrate and with violence: A critical review. Behavioral Sciences & the
niacin in coronary heart disease. Journal of the American Law, 19, 53-80. doi:10.1002/bsl.425
Medical Association, 231, 360-381. Elliott, R., Rubinsztein, J. S., Sahakian, B. J., & Dolan, R. J.
Corsini, R. J., & Wedding, D. (2010). Current psychotherapies. (2002). The neural basis of mood-congruent processing
Pacific Grove, CA: Wadsworth. biases in depression. Archives of General Psychiatry, 59,
Costello, E. J., & Janiszewski, S. (1990). Who gets treated? 597-604. doi: 10. 1001/archpsyc.59.7.597
Factors associated with referral in children with psy- Elliott, S. A., & Brown, J. L. (2002). What are we doing to wait-
chiatric disorders. Acta Psychiatrica Scandinavica , 81, ing list controls? Behaviour Research and Therapy, 40,
523-529. 1047-1052. doi:10.10l6/S0005-7967(01)00082-l
Coyne, J. (2012). The antidepressant wars, a sequel: How theEllis, A. (2003). Helping people get better rather than merely
media distort findings, and do harm to patients. PLOSBlogs. feel better. Journal of Rational-Emotive & Cognitive-
Retrieved from http://blogs.plos.org/mindthebrain/tag/ Behavior Therapy, 21, 169-182.
placebo-washout/ Engel, R. J., & Schutt, R. K. (2012). The practice of research in
Cromer, A. (1993). Uncommon sense: The heretical nature of social work. New York, NY: Sage.
science. New York, NY: Oxford University Press. Ernst, E., & Resch, K. L. (1995). Concept of true and perceived
Cross, K. P. (1977). Not can, but will college teaching be placebo effects. British Medical Journal, 311 , 551-553.
improved? New Directions for Higher Education, 19 77(17),Eysenck, H. J. (1952). The effects of psychotherapy: An eval-
1-15. uation. Journal of Consulting Psychology, 16, 319-324.
Crumlish, N., & Kelly, B. D. (2009). How psychiatrists think. doi:10.1037/h0063633
Advances in Psychiatric Treatment, 15, 72-79. Finocchiaro, M. (1981). Fallacies and the evaluation of reason-
Dawes, R. M. (1986). Representative thinking in clini- ing. American Philosophical Quarterly, 18, 13-22.
cal judgment. Clinical Psychology Review , 6, 425-441. Fraenkel, J. R., & Wallen, N. E. (1993). How to design and
doi: 10. 1016/0272-7358(86)90030-9 evaluate research in education (2nd ed.). New York, NY:
Dawes, R. M. (1994). House of cards: Psychology and psycho- McGraw-Hill.
therapy built on myth. New York, NY: Free Press. Frank, J. D., & Frank, J. B. (1961). Persuasion and healing.
Dawes, R. M. (2001). Everyday irrationality: How pseudo-scien- Baltimore, MD: Johns Hopkins University Press.
tists, lunatics , and the rest of us systematically fail to thinkFrazier, P., Tennen, H., Gavian, M., Park, C., Tomich, P., &
rationally. Boulder, CO: Westview Press. Tashiro, T. (2009). Does self-reported posttraumatic growth
Dehue, T. (2000). From deception trials to control reagents: reflect genuine positive change? Psychological Science , 20,
The introduction of the control group about a century ago. 912-919.

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Why Ineffective Psycbotherapies Appear to Work
clinical-statisticalL.
Fremont, J., & Craighead, controversy.
W. Psychology,
(1987).Public Policy , Aerobi
cognitive therapy in and
the Law, 2, 293-323.
treatmentdoi: 10. 1037/1076-8971. 2.2.293
of dyspho
Gruber, R.Research,
Cognitive Therapy and P. (1971). Behavior therapy: Problems
11 in,general- 241-251. d
BF01 183268 ization. Behavior Therapy, 2, 361-368. doi:10.10l6/S0005-
Gambrill, E. (2012). Critical thinking in clinical prac- 7894(71)80070-9
tice: Improving the quality of judgments and decisions. Hall, H. (2011). Evidence-based medicine, tooth-fairy science,
Hoboken, NJ: Wiley. and Cinderella medicine. Skeptic , i Xl), 4-5.
Garb, H. N. (1998). Studying the clinician: Judgment research Hamilton, D. L., & Gifford, R. K. (1976). Illusory correlation in
and psychological assessment. Washington, DC: American interpersonal perception: A cognitive basis of stereotypic
Psychological Association, doi: 10. 1037/10299-002 judgments. Journal of Experimental Social Psychology, 12,
Garb, H. N. (2005). Clinical judgment and decision making. 392-407. doi:10.10l6/S0022-1031(76)80006-6
Annual Review of Clinical Psychology , 1, 67-89. Hannan, C., Lambert, M. J., Harmon, C, Nielsen, S., Smart, D.
Garb, H. N., Lilienfeld, S. O., & Fowler, K. A. (2008). Psychological W., Shimokawa, K., & Sutton, S. W. (2005). A lab test and
assessment and clinical judgment. In J. E. Maddux & B. A. algorithms for identifying clients at risk for treatment failure.
Winstead (Eds.), Psychopathology: Foundations for a con- Journal of Clinical Psychology, 61, 155-163. doi: 10. 1002/
temporary understanding (2nd ed., pp. 103-124). New jclp.20108
York, NY: Routledge. Harrington, R., Whittaker, J., Shoebridge, P., & Campbell, F.
Garfield, S. (1994). Research on client variables in psychother- (1998). Systematic review of efficacy of cognitive behaviour
apy. In A. E. Bergin & S. Garfield (Eds.), Handbook of psy- therapies in childhood and adolescent depressive disorder.
chotherapy and behavior change (4th ed., pp. 190-228). British Medical Journal, 316, 1559-1563.
Oxford, England: Wiley. Harrison, S. (2011, August 15). "Beyond scared straight" producer
Geers, A. L., Weiland, P. E., Kosbab, K., Landry, S. J., & Heifer, shares secret s of A&E's hit series. Channel Guide Magazine.
S. G. (2005). Goal activation, expectations, and the pla- Available from http://www.channelguidemagblog.com
cebo effect. Journal of Personality and Social Psychology , Hartman, S. E. (2009). Why do ineffective therapies seem help-
89, 143-159. ful? A brief review. Chiropractic and Osteopathy, 17, 1-7.
Gehan, E. A., & Lemak, N. A. (1994). Statistics in medical doi: 10. 1 186/1746-1340-17-10
research: Developments in clinical trials. New York, NY: Hathaway, S. R. (1948). Some considerations relative to nondi-
Plenum. rective counseling as therapy. Journal of Clinical Psychology,
Ghaemi, S. N. (2009). A clinician's guide to statistics and epi- 4, 226-231. doi:10.1002/1097-4679(194807)4:3<226::AID-
demiology in mental health: Measuring truth and uncer- JCLP2270040303>3.0.CO;2-V
tainty. Cambridge, England: Cambridge University Press. Healy, D. (2002). Randomized controlled trials: Evidence biased
Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic decision psychiatry. Retrieved from http://www.ahrp.org/COI/
making. Annual Review of Psychology, 62, 451-482. healy0802.php
Gilovich, T. (1991). How we know what isn't so: The fallibility Herbert,
of J. D. (2003). The science and practice of empirically
human reason in everyday life. New York, NY: Free Press. supported treatments. Behavior Modification, 27, 412-430.
Goldfried, M. R. (2010). The future of psychotherapy inte- Hodges, B., Regehr, G., & Martin, D. (2001). Difficulties in
gration: Closing the gap between research and practice. recognizing one's own incompetence: Novice physicians
Journal of Psychotherapy Integration, 20, 386-396. who are unskilled and unaware of it. Academic Medicine,
Green, D., & Latchford, G. (2012). Practice-based evidence. In 76C Suppl. 10), S87-S89.
D. Green & G. Latchford (Eds.), Maximising the benefits Hofmann, S. G. (2008). Cognitive processes during fear acquisi-
of psychotherapy: A practice-based evidence approach (pp. tion and extinction in animals and humans: Implications for
87-108). Chichester, United Kingdom: Wiley-Blackwell. exposure therapy of anxiety disorders. Clinical Psychology
Greenwald, A. G., Spangenberg, E. R., Pratkanis, A. R., & Review, 28, 199-210.
Eskenazi, J. (1991). Double-blind tests of subliminal Hollis, S., & Campbell, F. (1999). What is meant by intention to
self-help audiotapes. Psychological Science, 2, 119-122. treat analysis? Survey of published randomised controlled
doi: 10. 1 1 1 1/j. 1467-9280. 1991 .tbOOl 12 .x trials. British Medical Journal, 319, 610-61 A. doi: 10. 1136/
Grissom, R. J. (1996). The magical number .7±.2: Meta-meta- bmj. 3197211.670
analysis of the probability of superior outcome in compari- Holmes, D. S. (1971). Round Robin therapy: A technique
sons involving therapy, placebo, and control. Journal of for implementing the effects of psychotherapy. Journal
Consulting and Clinical Psychology, 64, 973-982. of Consulting and Clinical Psychology, 37, 324-331.
Grodstein, F., Clarkson, T. B., & Manson, J. E. (2003). doi : 1 0 . 1 037/h003 1 968
Understanding the divergent data on postmenopausal hor- Horney, K. (1945). Our inner conflicts. Oxford, England: W.W.
mone therapy. New England Journal of Medicine, 348, Norton.
645-650. Horowitz, M. J., Wilner, N., & Alvarez, W. (1979). Impact of
Groopman, J. (2007). How doctors think. Boston, MA: Houghton Event Scale: A measure of subjective stress. Psychosomatic
Mifflin. Medicine , 41, 209-218.
Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency Horowitz, S. (2012). New perspectives on the placebo effect:
of informal (subjective, impressionistic) and formal Implications for research and clinical practice. Alternative
(mechanical, algorithmic) prediction procedures: The and Complementary Therapies , 18, 130-135.

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Horvath, A.symptoms
O., Del
and personality. Re,
Psychological Medicine, 19, A
(2011). 487-493. doi: 10.
Alliance in 1017/S003329170001 25 14
individuai
48 , 9-16. Kahneman, D. (1965). Control of spurious association and the
Hoshmand, L. T., & Polkinghorne, D. E. (1992). Redefining reliability of the controlled variable. Psychological Bulletin,
the science-practice relationship and professional training. 64, 326-329. doi:10.1037/h0022529
American Psychologist, 47 , 53-66. Kahneman, D. (2011). Thinking, fast and slow. New York, NY:
Howard, G. S. (1980). Response-shift bias: A problem in evalu- Farrar, Straus and Giroux.
ating interventions with pre/post self-reports. Evaluation Kahneman, D., & Klein, G. (2009). Conditions for intuitive
Review, 4 , 93-106. doi: 10. 1177/0193841X8000400105 expertise: A failure to disagree. American Psychologist, 64,
Howard, G. S., & Dailey, P. R. (1979). Response-shift bias: A 515-526. doi: 10. 1037/a00 16755
source of contamination of self-report measures. Journal Kanter, J. W., Kohlenberg, R. J., & Loftus, E. F. (2004).
of Applied Psychology , 54, 144-150. doi: 10. 1037/002 1- Experimental and psychotherapeutic demand characteris-
9010.64.2.144 tics and the cognitive therapy rationale: An analogue study.
Howard, K. I., Kopta, S., Krause, M. S., & Orlinsky, D. E. (1986). Cognitive Therapy and Research, 28, 229-239.
The dose-effect relationship in psychotherapy. American Kazdin, A. E. (1977). Assessing the clinical or applied importance
Psychologist , 159-164. doi:10.1037/0003-066X.4l.2.159 of behavior change through social validation. Behavior
Hróbjartsson, A., & Gotzsche, P. C. (2001). Is the placebo pow- Modification , 1, 427-452. doi: 10. 1177/014544557714001
erless? An analysis of clinical trials comparing placebo Kazdin, A. E. (2007). Systematic evaluation to improve the qual-
with no treatment. New England Journal of Medicine, 344 , ity of patient care: From hope to hopeful. Pragmatic Case
1594-1602. doi: 10. 1056/NEJM200105243442 106 Studies in Psychotherapy, 3(4), 37-49.
Hsu, L. M. (1989). Random sampling, randomization, and equiv- Kazdin, A. E., & Nock, M. K. (2003). Delineating mechanisms
alence of contrasted groups in psychotherapy outcome of change in child and adolescent therapy: Methodological
research. Journal of Consulting and Clinical Psychology , issues and research recommendations. Journal of Child
57, 131-137. doi: 10. 1 037/002 2-006X. 57. 1.131 Psychology and Psychiatry, 44, 1116-1129.
Huck, S. W., & Sandler, H. M. (1979). Rival hypotheses: Kelly, J. F., & Moos, R. (2003). Dropout from 12-step self-help
Alternative interpretations of data based conclusions. New groups: Prevalence, predictors, and counteracting treat-
York, NY: Harper & Row. ment influences. Journal of Substance Abuse Treatment, 24,
Hume, D. (1748). Enquiry concerning human understanding. 241-250. doi: 10. 10l6/S0740-5472(03)0002 1-7
London, England: P. F. Collier & Son. Kendall, P. C., Butcher, J. N., & Holmbeck, G. N. (1999).
Hunsley, J., & Di Giulio, G. (2002). Dodo bird, phoenix, or Handbook of research methods in clinical psychology (2nd
urban legend? The question of psychotherapy equiva- ed.). Hoboken, NJ: Wiley.
lence. Scientific Review of Mental Health Practice: Objective Kienle, G. S., & Kiene, H. (1997). The powerful placebo effect:
Investigations of Controversial and Unorthodox Claims in Fact or fiction? Journal of Clinical Epidemiology , 50, 1311-
Clinical Psychology, Psychiatry, and Social Work , 1, 11-22. 1318.
Ilardi, S. S., & Craighead, W. (1999). Rapid early response, Kirsch, I. (2005). Placebo psychotherapy: Synonym or oxymo-
cognitive modification, and nonspecific factors in cogni- ron? Journal of Clinical Psychology, 61, 791-803.
tive behavior therapy for depression: A reply to Tang and Kirsch, I. (2013). The placebo effect revisited: Lessons learned to
DeRubeis. Clinical Psychology: Science and Practice, 6, date. Complementary Therapies in Medicine, 21, 102-104.
295-299. Kirsch, I., & Sapirstein, G. (1998). Listening to Prozac but
Ioannidis, J. A. (2005). Contradicted and initially stronger hearing placebo: A meta-analysis of antidepressant medi-
effects in highly cited clinical research. Journal of the cation. Prevention & Treatment, 7(2), Article 0002a.
American Medical Association, 294, 218-228. doi:10.1001/ doi: 10. 1037/1 522-3736. 1 . 1 . 12a
jama.294.2.218 Klein, D. F. (1998). Listening to meta-analysis but hearing bias.
Jacobson, N. S., & Christensen, A. (1996). Studying the effec- Prevention & Treatment, 1(2), Article 6c. doi: 10. 1037/1522-
tiveness of psychotherapy: How well can clinical tri- 3736.1.1.16c
als do the job? American Psychologist, 51, 1031-1039. Koffka, K. K. (1935). Principles of Gestalt psychology. Oxford,
doi: 10. 1037/0003-066X.5 1 . 10. 103 1 England: Harcourt, Brace.
James, F. E. (1992). Insulin treatment in psychiatry. History ofKoocher, G. P., Norcross, J. C., & Hill, S. S. (Eds.). (2005).
Psychiatry, 3, 221-235. Psychologists' desk reference (2nd ed.). New York, NY:
Jones, K. (2000). Insulin coma therapy in schizophrenia. Journal Oxford University Press.
of the Royal Society of Medicine, 93, 147-149. Kraemer, H. C., Wilson, G. T., Fairburn, C. G., & Agras, W. S.
Jopling, D. A. (2001). Placebo insight: The rationality of insight- (2002). Mediators and moderators of treatment effects in
oriented psychotherapy. Journal of Clinical Psychology , randomized clinical trials. Archives of General Psychiatry,
57, 19-36. doi: 10. 1002/1097-4679(200101)57 : 1<19:: AID- 59, 877-883.
JCLP4>3.0.CO;2-Z Kruger, J., Savitsky, K., & Gilovich, T. (1999). Superstition and
Jopling, D. A. (2008). Talking cures and placebo effects. Oxford, the regression effect. Skeptical Inquirer, 23, 24-29.
England: Oxford University Press. Kunda, Z. (1990). The case for motivated reasoning.
Jorm, A. F., Duncan-Jones, P. P., & Scott, R. R. (1989). An analy- Psychological Bulletin, 108, 480-498. doi: 10. 1037/0033-
sis of the re-test artefact in longitudinal studies of psychiatric 2909.108.3.480

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Lambert, M. J. (1976).Lilienfeld, S. O., Lynn, S. J., & Lohr, J.

Spontaneous M. (2003). Science and
remission in
disorders: A revision and pseudoscience in clinical psychology.
summary. New York, NY:
Psychologica
S3, 107-118. Guilford Press.
Lilienfeld,
Lambert, M. J. (2005). Early S. O., Ritscheł,
response L. A., Lynn, S. J.,
in Cautin, R. L., &
psychoth
evidence for the importanceLatzman, R. D. (2013).
ofWhy many clinical psychologists
common factors
"placebo effects." Journal are resistant of
to evidence-based
Clinical practice: Root causes and
Psycholog
869. constructive remedies. Clinical Psychology Review , 33,
Lambert, M. J., Lunnen, K., Umphress, V., Hansen, N., & 883-890.
Burlingame, G. M. (1994). Administration and scor- Litz, B. T., Gray, M. J, Bryant, R. A., & Adler, A. B. (2002).
ing manual for the Outcome Questionnaire (OQ-45.1). Early intervention for trauma: Current status and future
Salt Lake City, UT: IHC Center for Behavioral Healthcare directions. Clinical Psychology: Science and Practice , 9,
Efficacy. 112-134. doi: 10. 1093/clipsy/9.2. 1 12
Lambert, M. J., & Ogles, B. M. (2004). The efficacy and effec- Loftus, E. F. (1993). The reality of repressed memories. American
tiveness of psychotherapy. In M. J. Lambert (Ed.), Bergin Psychologist , 48 , 518-537. doi:10.1037/0003-066X.48.5.518
and Garfield's handbook of psychotherapy and behavior Loftus, L. S., & Arnold, W. N. (1991). Vincent van Gogh's illness:
change (5th ed., pp. 139^193). New York, NY: Wiley. Acute intermittent porphyria? British Medical Journal , 303,
Langer, E.J. (1975). The illusion of control. Journal of Personality 1589-1591.
and Social Psychology , 32 , 311-328. doi: 10. 1037/0022- Longmore, R. J., & Worrell, M. (2007). Do we need to challenge
3514.32.2.311 thoughts in cognitive behavior therapy? Clinical Psychology
Lassiter, G., Geers, A. L., Munhall, P., Ploutz-Snyder, R. J., & Review , 27, 173-187.
Breitenbecher, D. L. (2002). Illusory causation: Why it Loranger, A. W., Lenzenweger, M. F., Gartner, A. F., & Susman,
occurs. Psychological Science , 13 , 299-305. doi: 10.11 11/ V. L. (I99I). Trait-state artifacts and the diagnosis of per-
Í.0956-7976.2002.X sonality disorders. Archives of General Psychiatry , 48 , 720-
Laurenceau, J. P., Hayes, A. M., & Feldman, G. C. (2007). Some 728. doi: 10. 1001/archpsyc. 1991. 01810320044007
methodological and statistical issues in the study of change Luborsky, L., Singer, B., & Luborsky, L. (1975). Comparative
processes in psychotherapy. Clinical Psychology Review , studies of psychotherapies: Is it true that "everyone has won
27, 682-695. and all must have prizes"? Archives of General Psychiatry ,
Lehrer, J. (2010, February 25). Depression's upside. The New 32, 995-1008. doi: 10. 1001/archpsyc. 1975.01760260059004
York Times. Available from http://www.nytimes.com Luty, S. E., Carter, J. D., McKenzie, J. M., Rae, A. M., Frampton,
Leins, U., Goth, G., Hinterberger, T., Klinger, C., Rumpf, N., & C. M., Mulder, R. T., & Joyce, P. R. (2007). Randomised
Střehl, U. (2007). Neurofeedback for children with ADHD: controlled trial of interpersonal psychotherapy and cogni-
A comparison of SCP and Theta/Beta protocols. Applied tive-behavioural therapy for depression. British Journal of
Psychophysiology and Biofeedback , 32. 73-88. Psychiatry , 190, 496-502.
Lenzenweger, M. F. (1999). Stability and change in personal- Lynch, T. R., Chapman, A. L., Rosenthal, M., Kuo, J. R., & Linehan,
ity disorder features: The longitudinal study of personality M. M. (2006). Mechanisms of change in dialectical behavior
disorders. Archives of General Psychiatry , 56, 1009-1015. therapy: Theoretical and empirical observations. Journal of
doi: 10. 1001/archpsyc.56. 1 1 . 1009 Clinical Psychology, 62 , 459-480. doi:10.1002/jclp.20243
Leucht, S. S., Arbter, D. D., Engel, R. R., Kissling, W. W., & Lynn, S. J., & Nash, M. R. (1994). Truth in memory: Ramifications
Davis, J. M. (2009). How effective are second-generation for psychotherapy and hypnotherapy. American Journal of
antipsychotic drugs? A meta-analysis of placebo-controlled Clinical Hypnosis, 36(3), 194-208.
trials. Molecular Psychiatry , 14 , 429-447. doi:10.1038/ Magaret, A. (1950). Generalization in successful psychotherapy.
sj.mp.4002136 Journal of Consulting Psychology, 14, 64-70. doi: 10. 1037/
Lilienfeld, S. O. (2007). Psychological treatments that cause h0053633
harm. Perspectives on Psychological Science , 2, 53-70. Manzi, J. (2012). Uncontrolled: The surprising payoff of trial-
doi: 10. 1 1 1 1/j. 1745-6916.2OO7.OOO29.X and-error in business, politics and society. New York, NY:
Lilienfeld, S. O. (2014). The Dodo Bird verdict: Status in 2014. Basic Books.
Behavior Therapist, 37 , 91-95. Marino, L., & Lilienfeld, S. O. (2007). Dolphin-assisted ther-
Lilienfeld, S. O., Ammirati, R., & Landfìeld, K. (2009). Giving apy: More flawed data and more flawed conclusions.
debiasing away: Can psychological research on correct- Anthrozoös , 20, 239-249. doi: 10.2752/089279307X224782
ing cognitive errors promote human welfare? Perspectives Markin, R. D., & Kivlighan, D. (2007). Bias in psychotherapist
on Psychological Science , 4, 390-398. doi: 10.11 1 1/j. 1745- ratings of client transference and insight. Psychotherapy:
6924.2009.01144.x Theory, Research, Practice, Training, 44, 300-315.
Lilienfeld, S. O., Lohr, J. M., & Olatunji, B. O. (2008). doi: 10. 1037/0033-3204.44.3300
Encouraging students to think critically about psycho- Marmar, C. R. (1990). Psychotherapy process research: Progress,
therapy: Overcoming naïve realism. In D. S. Dunn, J. S. dilemmas, and future directions. Journal of Consulting and
Halonen, & R. A. Smith (Eds.), Teaching critical think- Clinical Psychology, 58, 265-272.
ing in psychology: A handbook of best practices (pp. 267- Masicampo, E. J., & Lalande, D. R. (2012). A peculiar preva-
271). Chichester, United Kingdom: Wiley-Blackwell. lence of p values just below .05. The Quarterly Journal of
doi:10.1002/9781444305173.ch26 Experimental Psychology, 65, 2271-2279.

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Matute, H., McReynolds,

Yarritu, I.,
W. T., & Tori, C. (1972). A further assessment &
of V
causality at attention-placebo
the effects heart and demand characteristics
of in p
Psychology , studies
102, 392-405.
of systematic desensitization. Journal of Consulting
Mausbach, B. T., Moore, R., Roesch, S., Cardenas, V., & and Clinical Psychology, 38, 261-264.
Patterson, T. L. (2010). The relationship between home- Meehl, P. E. (1955). Psychotherapy. Annual Review of Psychology,
work compliance and therapy outcomes: An updated meta- 63, 57-378. doi: 10. 1 l46/annurev.ps.06.0201 55. 002041
analysis. Cognitive Therapy and Research , 34, 429-438. Messer, S. B., & Wampold, B. E. (2002). Let's face facts: Common
doi: 10. 1 007/s 1 0608-0 1 0-9297-z factors are more potent than specific therapy ingredients.
Mayou, R. A., Ehlers, A. A., & Hobbs, M. M. (2000). Psychological Clinical Psychology: Science and Practice , 9, 21-25.
debriefing for road traffic accident victims: Three-year fol-Meyer, R. G., & Deitsch, S. E. (1996). The clinician's handbook:
low-up of a randomised controlled trial. British Journal of Integrated diagnostics, assessment , and intervention in
Psvchiatrv . 176, 589-593. doi: 10. 1192/bio. 176.6.589 adult and adolescent psychopathology (4th ed.). Needham
Mayr, E. (1981, October 30). Biological classification: Toward Heights, MA: Allyn & Bacon.
a synthesis of opposing methodologies. Science , 214 Michotte,
, A. (1945). The perception of causality. New York, NY:
510-516. Basic Books.
McArthur, L. Z. (1980). Illusory causation and illu- Mintz, J., Drake, R. E., & Crits-Christoph, P. (1996). Efficacy
sory correlation: Two epistemological accounts. and effectiveness of psychotherapy: Two paradigms,
Personality and Social Psychology Bulletin, 6 , 507-519- one science. American Psychologist, 51, 1084-1085.
doi: 10.1 177/0l46l6728064003 doi: 10. 1037/0003-066X.51 . 10. 1084
McArthur, L. Z., & Solomon, L. K. (1978). Perceptions of anMorgado, A., Smith, M., Lecrubier, Y., & Widlocher, D. (1991).
aggressive encounter as a function of the victim's salience Depressed subjects unwittingly overreport poor social
and the perceiver's arousal. Journal of Personality and adjustment which they reappraise when recovered .Journal
Social Psychology , 36, 1278-1290. doi: 10. 1037/0022- of Nervous and Mental Disease , 1 79, 614-619.
3514.36.11.1278 Morgan, C. D., & Murray, H. A. (1935). A method for investigat-
McCall, R. B. (1977). Challenges to a science of developing fantasies: The Thematic Apperception Test. Archives of
mental psychology. Child Development , 48, 333-344. Neurology and Psychiatry, 34, 289-306.
doi: 10.2307/1 128626 Morrison, J. R. (1997). When psychological problems mask medi-
McCauley, R. (2011). Why religion is natural and science is not. cal disorders: A guide for psychotherapists. New York, NY:
Oxford. England: Oxford University Press. Guilford Press.
McCullough, J. P. (2000). Treatment for chronic depression: Nathan, P. E., Stuart, S. P., & Dolan, S. L. (2003). Research on
Cognitive Behavioral Analysis System of Psychotherapy psychotherapy efficacy and effectiveness: Between Scylla
(CBASP). New York, NY: Guilford Press. and Charybdis? In A. E. Kazdin (Ed.), Methodological issues
McDonald, C. J., Mazzuca, S. A., & McCabe, G. P. (1983). How & strategies in clinical research (3rd ed., pp. 505-546).
much of the placebo "effect" is really statistical regression? Washington, DC: American Psychological Association.
Statistics in Medicine, 2, 417-427. Nichols, M. P., & Efran, J. S. (1985). Catharsis in psychother-
McFall, R. M. (1991). Manifesto for a science of clinical psychol- apy: A new perspective. Psychotherapy: Theory , Research,
ogy. The Clinical Psychologist , 44(6), 75-88. Practice, Training, 22, 46-58.
McFall, R. M., & Treat, T. A. (1999). Quantifying the information Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phe-
value of clinical assessments with signal detection theory. nomenon in many guises. Review of General Psychology, 2,
Annual Review of Psychology, 50, 215-241. doi:10.1l46/ 175-220. doi: 10. 1037/1089-2680.2.2.175
annurev.psych.50.1.215 Nisbett, R. E., & Ross, L. (1980). Human inference: Strategies
McFarland, C., & Alvaro, C. (2000). The impact of motivation and shortcomings of social judgment. Englewood Cliffs, NJ:
on temporal comparisons: Coping with traumatic events Prentice-Hall.
by perceiving personal growth. Journal of Personality and Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can
Social Psychology, 79, 327-343. know: Verbal reports on mental processes. Psychological
McHugh, P. M. (1994). Psychotherapy awry. Amerìcan Scholar , Review, 84, 231-259.
63, 17-30. Nock, M. K., Michel, B. D., & Photos, V. I. (2007). Single-case
McHugh, R. K., & Barlow, D. H. (2010). The dissemination and research designs. In D. McKay (Ed.), Handbook of research
implementation of evidence-based psychological treatments: methods in abnormal and clinical psychology (pp. 337-
A review of current efforts. American Psychologist, 65, 73-84. 350). New York, NY: Sage.
McLeod, J. (2001). An administratively created reality: Some Norcross, J. C., Beutler, L. E., & Levant, R. F. (2007). Evidence-
problems with the use of self-report questionnaire mea- based practices in mental health. Washington, DC:
sures of adjustment in counselling/psychotherapy outcome American Psychological Association.
research. Counselling & Psychotherapy Research, 1, 215-226. Norcross, J. C., & Lambert, M. J. (2006). The therapy relation-
McNally, R. J., Bryant, R. A., & Ehlers, A. (2003). Does early psy- ship. In J. C. Norcross, L. E. Beutler, & R. F. Levant (Eds.),
chological intervention promote recovery from posttrau- Evidence-based practices in mental health: Debate and
matic stress? Psychological Science in the Public Interest , 4, dialogue on the fundamental questions (pp. 208-218).
45-79. doi: 10. 1 1 1 1/1529-1006.0142 1 Washington, DC: American Psychological Association.

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Novella, S. E. (2008). The placeboRachman,

effect. Science-Based
S. J. (1973). The effects of psychological treatments.
Medicine. Retrieved from http://www.sciencebased
In H. Eysenck (Ed.), Handbook of abnormal psychology
(pp. 805-819). New York, NY: Basic Books.
medicine.org/index.php/the-placebo-effect/
Rice, M. E., Harris, G. T., &placebo.
Novella, S. E. (2010). The poor, misunderstood Cormier, C. A. (1992). An evalu-
Skeptical Inquirer , 34(6), 33-34. ation of a maximum security therapeutic community for
O'Donohue, W., & Lilienfeld, S. O. (2007).
psychopathsThe epistemological
and other mentally disordered offenders. Law
and ethical dimension of clinical science. In T.16,
and Human Behavior, A. Treat,
399-412. doi:10.1007/BF02352266
R. R. Bootzin, & T. B. Baker (Eds.), Psychological
Rickeis, clinical
K., Lipman, R. S., Fisher, S., Park, L. C., & Uhlenhuth,
science: Papers in honor of Richard E. H.
M. (1970). Is a double-blind
McFall clinical trial really dou-
(pp. 29-52).
New York, NY: Psychology Press. ble-blind? A report of doctors' medication guesses.
Psychopharmacologia,
Offit, P. A. (2010). Autismes false prophets: 16, 329-336. risky
Bad science,
medicine, and the search for a cure
Ritschel, . (2005).
L. A. New York,
Reconciling the NY:
rift: Improving clinical
Columbia University Press. psychology graduate training in the twenty-first century.
O'Leary, K., & Borkovec, T. D. (1978). Conceptual,
Journal of Clinical Psychology, method-
61, 1111-1114. doi: 10. 1002/
ological, and ethical problems of placebo
jclp.20147 groups in psy-
Robins, R. W., Fraley, R.,
chotherapy research. American Psychologist , Roberts, B. W., & Trzesniewski,
33, 821-830.
doi: 10. 1037/0003-066X.33.9.82 1 K. H. (2001). A longitudinal study of personality change
Orlinsky, D. E., Ronnestad, M. H., in& Willutzki,
young adulthood. Journal of U. (2004).
Personality, 69 , 617-640.
Fifty years of psychotherapy process-outcome
doi: 10. 1 1 1 1/1467-6494.694157 research:
Robins, R. W., Noftle,
Continuity and change. In M. Lambert (Ed.), E. E., Trzesniewski,and
Bergin K. H., & Roberts,
Garfield's handbook of psychotherapy B. W. (2005).
andDo behavior
people know how changetheir personality has
(5th ed., pp. 307-390). New York, NY: changed? Wiley.
Correlates of perceived and actual personality
change in young
Orne, M. T. (1962). On the social psychology of adulthood. Journal of Personality, 73,
the psychologi-
cal experiment: With particular reference
489-521 . doi: 10.to1 1 1 demand char-
l/j. 1467-6494.2005.00317.X
acteristics and their implications. American
Romanczyk, Psychologist
R. G., Arnstein, , J. (2003).
L., Soorya, L. V., & Gillis,
17, 776-783. doi:10.1037/h0043424 The myriad of controversial treatments for autism: A critical
Pace, T. W., Negi, L., Adame, D. D., Cole, S. P.,
evaluation Sivilli,
of efficacy. In S. T. I., Brown,
O. Lilienfeld, S. Lynn, & J. M.
T. D., . . . Raison, C. L. (2009). Effect of compassion
Lohr (Eds.), Science and pseudosciencemedi-
in clinical psychol -
tation on neuroendocrine, innate immune ogy (pp. 363-395). and behavioral
New York, NY: Guilford Press.
responses to psychosocial stress. Psychoneuroendocrinology, Rosenthal, R. (1979). The file drawer problem and toler-
34, 87-98. doi:10.10l6/j.psyneuen.2008.08.011 ance for null results. Psychological Bulletin, 86, 638-641.
Paul, G. L. (1966). Insight vs. desensitization doi: 10. 1037/0033-2909.86.3.638
in psychotherapy;
an experiment in anxiety reduction. Ross, Stanford,
L., & Ward, A. (1996).CA: NaiveStanford
realism in everyday life:
University Press. Implications for social conflict and misunderstanding. In
Penedo, F. J., & Dahn, J. R. (2005). Exercise E. S. Reed, E. Turiel,
and & T. Brown (Eds.), Values and
well-being: A knowl-
review of mental and physical health edge (pp. benefits
103-135). Hillsdale,associated
NJ: Erlbaum.
with physical activity. Current Opinion Ross, M. (1989). in Psychiatry
Relation , the
of implicit theories to 18, construction
189-193. doi: 10. 1097/00001 504-200503000-00013 of personal histories. Psychological Review, 96, 341-357.
Pennebaker, J. W., & Skelton, J. A. (1981). doi:10.1037/0033-295X.96.2.34l
Selective monitoring
of physical sensations. Journal of Personality
Roth, A., and
& Fonagy, P. (2005). What worksSocial
for whom? A critical
Psychology, 41, 213-223. doi:10.1037/0022-35l4.4l.2.213 review of psychotherapy research. New York, NY: Guilford
Petitti, D. B. (1994). Meta-analysis, decision Press. analysis and cost-
effectiveness analysis. New York, NY:
Sackett, D. Oxford
L., Rosenberg, University
W. M. C., Gray, J. A. M., Haynes, R. B.,
Press. & Richardson, W. S. (1996). Evidence based medicine: What
Petrosino, A., Turpin-Petrosino, C., & Buehler, J. (2005). Scared it is and what it isn't. British Medical Journal, 312, 71-72.
Straight and other juvenile awareness programs for pre- Salsburg, D. (2001). The lady tasting tea: How statistics revolu-
venting juvenile delinquency. The Scientific Review of tionized science in the 20th century. New York, NY: W.H.
Mental Health Practice, 4, 48-54. Freeman.
Posternak, M. A., & Zimmerman, M. (2000). Short-term spon- Samuel, D. B., Hopwood, C. J., Ansell, E. B., Morey, L. C.,
taneous improvement in depressed outpatients. Journal of Sanislow, C. A., Markowitz, J. C., . . . Grilo, C. M. (2011).
Nervous and Mental Disease, 188, 799-804. Comparing the temporal stability of self-report and inter-
Poulsen, S., Lunn, S., Daniel, S. I., Folke, S., Mathiesen, B. B., view assessed personality disorder. Journal of Abnormal
Katznelson, H., & Fairburn, C. G. (2014). A randomized Psychology, 120, 670-680. doi:10.1037/a0022647
controlled trial of psychoanalytic psychotherapy or cog- Segali, M. H., Campbell, D. T., & Herskovits, M. J. (1966). The
nitive-behavioral therapy for bulimia nervosa. American influence of culture on visual perception. Oxford, England:
Journal of Psychiatry , 1 71, 109-1 16. Bobbs-Merrill.
Prasad, V., Cifu, A., & Ioannidis, J. P. (2012). Reversals of estab- Seligman, M. P. (1995). The effectiveness of psychotherapy:
lished medical practices. Journal of the American Medical The Consumer Reports study. American Psychologist, 50,
Association , 30 7, 37-38. 965-974. doi: 10. 1037/0003-066X.50. 12.965

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Shafran, R., Clark,

training in empirically supported D. M.,
treatments, fournal of Fa
D. H., Ehlers, Clinical Psychology
A., , 68, 8-23. & Wilson,
Improving Straus, S. E., Tetroe,
the J., Graham, I. D., Zwarenstein, M.,
dissemination
and Therapy Bhattacharyya,
, 47 O.,, & Shepperd,
902-909. S. (2010). Monitoring
Shapiro, A. K., use of knowledge
& and evaluating outcomes. Canadian E.
Shapiro, (
about nothing? Medical Association
In fournal,A. 182, e94-e98. doi: 10. 1503/
Harrin
An interdisciplinary cmaj.081335 explor
MA: Harvard University
Strieker, G. (2003). The many faces of self-disclosure, fournal Presof
Shea, M., Edelen, Clinical Psychology,M. 59 , 623-630. O., Pinto
G., Skodol, Strieker, G.,E.,
A. & Trierweiler, & S. J. (1995).
.The . local clinical
. sci-
Mor
in borderline entist:personality
A bridge between science and practice. American di
Acta Psychiatrica Psychologist, 50, 995-1002. doi:Scandinav
10. 1037/0003-066X. 50. 12.995
j.l600-0447.2008.01 Suis, J., & Howren, M. B. (2012). Understanding
274.x the physical-
Shedler, J. (2010). The
symptom experience: The distinctive contributionseffic
of anxi-
therapy. American ety and depression. Current DirectionsPsycholo in Psychological
a0018378 Science, 21, 129-134.
Sloman, S. (2005). Causal models : How people think aboutSwift, J. K., & Greenberg, R. P. (2012). Premature discontinu-
the world and its alternatives. New York, NY: Oxford ation in adult psychotherapy: A meta-analysis, fournal
University Press. of Consulting and Clinical Psychology, 80, 547-559.
Smith, D. B. (2009, September). The doctor is in. The American doi:10.1037/a0028226
Scholar , 75(4). Available from http://theamericanscholarTaleb, N. N. (2007). The black swan: The impact of the highly
.org improbable. New York, NY: Random House.
Tang, T. Z., & DeRubeis, R. J. (1999). Sudden gains and criti-
Smith, M. L., Glass, G. V., & Miller, T. I. (1980). The benefits of
psychotherapy. Baltimore, MD: Johns Hopkins University cal sessions in cognitive-behavioral therapy for depression.
Press. fournal of Consulting and Clinical Psychology, 67, 894-904.
Solomon, D. A., Leon, A. C., Coryell, W. H., Endicott, J., Li, C.,Taubes, G. (2007, September 16). Do we really know what
Fiedorowicz, J. G., . . . Keller, M. B. (2010). Longitudinal makes us healthy? The New York Times. Available from
course of bipolar I disorder: Duration of mood episodes. http://www.dcscience.net
Archives of General Psychiatry , 67, 339-347. doi: 10. 1001/ Tavris, C. (2003). The widening scientist-practitioner gap. In
archeenpsychiatry . 20 10. 1 5 S. O. Lilienfeld, S. J. Lynn, & J. M. Lohr (Eds.), Science
Spengler, P. M., White, M. J., ^Egisdóttir, S., Maugherman, A. and pseudoscience in clinical psychology (pp. ix-xvii). New
S., Anderson, L. A., Cook, R. S., & Rush, J. D. (2009). The York, NY: Guilford Press.
Meta-Analysis of Clinical Judgment Project: Effects of expe- Taylor, S. E., & Fiske, S. T. (1975). Point of view and perceptions
rience on judgment accuracy. The Counseling Psychologist , of causality, fournal of Personality and Social Psychology,
37, 350-399. 32, 439-445. doi:10.1037/h0077095
Spitzer, R. L. (2003). Can some gay men and lesbians change their Taylor, S. E., & Fiske, S. T. (1978). Salience, attention, and attri-
sexual orientation? 200 participants reporting a change from bution: Top of the head phenomena. In L. Berkowitz (Ed.),
homosexual to heterosexual orientation. Archives of Sexual Advances in experimental social psychology (Vol. 11, pp.
Behavior , 32, 403-417. doi: 10. 1023/A: 1025647527010 249-288). New York, NY: Academic Press.
Tehrani, E. E., Krüssel, J. J., Borg, L. L., & Munk-J0rgensen, P.
Spitzer, R. L. (2012). Spitzer reassesses his 2003 study of repara-
tive therapy of homosexuality. Archives of Sexual Behavior , P. (1996). Dropping out of psychiatric treatment: A pro-
41 , 747. doi:10.1007/sl0508-012-9966-y spective study of a first-admission cohort. Acta Psychiatrica
Spring, B. (2007). Evidence-based practice in clinical psychol- Scandinavica, 94, 266-271. doi: 10. 11 11/j. 1600-0447.1996.
ogy: What it is, why it matters: What you need to know. tb09859.x
fournal of Clinical Psychology , 63, 611-631. doi: 10. 1002/ Tellegen, A., Ben-Porath, Y. S., McNulty, J. L., Arbisi, P. A.,
jclp. 20373 Graham, J. R., & Kaemmer, B. (2003). MMPI-2 Restructured
Stanovich, K. E., & West, R. F. (2008). On the relative indepen- Clinical (RC) scales: Development , validation , and interpre-
dence of thinking biases and cognitive ability, fournal of tation. Minneapolis: University of Minnesota Press.
Personality and Social Psychology , 94, 672-695. Tetlock, P. E. (2005). Expert political judgment: How good is
Steer, R. Y., & Ritschel, L. A. (2010). Placebo. In I. Weiner & it? How can we know? Princeton, NJ: Princeton University
W. E. Craighead (Eds.), Corsini encyclopedia of psychology Press.
(pp. 1252-1254). New York, NY: Wiley. Thompson, S. C. (1999). Illusions of control: How we overestimate
Stewart, R. E., & Chambless, D. L. (2007). Does psychotherapy our personal influence. Current Directions in Psychological
research inform treatment decisions in private practice? Science , 8 , 187-190. doi:10.1 11 1/1467-872 1.00044
fournal of Clinical Psychology , 63, 267-281. doi: 10. 1002/ Tolin, D. F. (2010). Is cognitive-behavioral therapy more
jclp. 20347 effective than other therapies?: A meta-analytic review.
Stewart, R. E., Chambless, D. L., & Baron, J. (2011). Theoretical Clinical Psychology Review, 30, 710-720. doi: 10. 1016/
and practical barriers to practitioners' willingness to seek j.cpr.2010.05.003

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00
Tracey, T. J., Wampold, B. E., Lichtenberg, Weisz, J. R., Weiss,

J. B., Han, S.
W., &S.,Goodyear,
Granger, D. A., & Morton, T.
R. K. (2014). Expertise in psychotherapy: (1995). Effects
An of psychotherapy
elusive goal?with children and adoles-
American Psychologist , 69, 218-229.cents revisited: A meta-analysis of treatment outcome stud-
Tversky, A., & Kahneman, D. (1974, September ies. Psychological Bulletin, 117, 450-458.
27). Judgment
under uncertainty: Heuristics and Westen, D. (1988). Science
biases. Transference ,
and185
information
, processing.
1124-1131. doi:10.1126/science.l8S.4lS7.1124 Clinical PsvcholoQV Review, 8, 161-179.
Valenstein, E. S. (1986). Great and desperate cures: TheWesten,
rise andD., & Bradley, R. (2005). Empirically supported com-
decline of psychosurgery and other radical treatmentsplexity:
for Rethinking evidence-based practice in psycho-
mental illness. New York, NY: Basic Books. therapy. Current Directions in Psychological Science, 14,
Wachtel, P. L. (1987). Action and insight. New York, 266-271
NY: . doi: 10. 1 1 1 l/j.0963-72 14.2005.00378.x
Guilford Press. Westen, D., Novotný, C. M., & Thompson-Brenner, H. (2004).
Wachtel, P. L. (2010). Beyond "ESTs": Problematic assumptions The empirical status of empirically supported psycho-
in the pursuit of evidence-based practice. Psychoanalytic therapies: Assumptions, findings, and reporting in con-
Psychology , 27, 251-263. trolled clinical trials. Psychological Bulletin, 130, 631-663.
Walfìsh, S., McAlister, B., O'Donnell, P., & Lambert, M. J. doi: 10. 1037/0033-2909. 130.4.631
(2012). An investigation of self-assessment bias in men- Westen, D., Novotný, C. M., & Thompson-Brenner, H. (2005).
tal health providers. Psychological Reports , 110 , 639-644. EBP^EST: Reply to Crits-Christoph et al. (2005) and Weisz
doi: 10.2466/02.07. 17.PR0. 110.2.639-644 et al. (2005). Psychological Bulletin, 131 , 427-433.
Walker, W., Skowroński, J. J., & Thompson, C. P. (2003). Life is Westen, D., & Weinberger, J. (2005). In praise of clinical
pleasant - and memory helps to keep it that way! Review judgment: Meehl's forgotten legacy. Journal of Clinical
of General Psychology , 7, 203-210. doi: 10. 1037/1089- Psychology, 61, 1257-1276. doi:10.1002/iclp.20181
2680.7.2.203 Whisman, M. A., Miller, I. W., Norman, W. H., & Keitner,
Wampold, B. E. (2001). The great psychotherapy debate: Models, G. I. (1991). Cognitive therapy with depressed inpatients:
methods , and findings. Hillsdale, NJ: Erlbaum. Specific effects on dysfunctional cognitions. Journal of
Wampold, B. E., & DeFife, J. (2010, February 6). How psycho- Consulting and Clinical Psychology , 59, 282-288.
therapy works. Psychology Today. Retrieved from http:// Wilson, A. E., & Ross, M. (2001). From chump to champ:
www.psychologytoday.com/blog/the-shrink-tank/201002/ People's appraisals of their earlier and present selves.
how-psychotherapy-works Journal of Personality and Social Psychology, 80, 572-584.
Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, doi: 10. 1037/0022-3514.80.4.572
K., & Ahn, H. (1997). A meta-analysis of outcome stud- Wilson, G. T. (1999). Rapid response to cognitive behavior ther-
ies comparing bona fide psychotherapies: Empirically, "all apy. Clinical Psychology: Science and Practice, 6, 289-292.
must have prizes." Psychological Bulletin , 122, 203-215. Wilson, T. D. (2011). Redirect: The surprising new science of
doi: 10. 1037/0033-2909. 122.3.203 psychological change. Boston, MA: Little, Brown.
Wechsler, M. E., Kelley, J. M., Boyd, I. O., Dutile, S., Wolpert, L. (1992). The unnatural nature of science. Cambridge,
Marigowda, G., Kirsch, I., . . . Kaptchuk, T. J. (2011). MA: Harvard University Press.
Active albuterol or placebo, sham acupuncture, or no Zamansky, H. S., Scharf, B., & Brightbill, R. (1964). The
intervention in asthma. New England Journal of Medicine, effect of expectancy for hypnosis on prehypnotic
365, 119-126. performance. Journal of Personality, 32, 236-248.
Weiss, B., & Weisz, J. R. (1990). The impact of methodologi- doi: 10. 1 1 1 1/j. 1467-6494. 1964.tb01338.x
cal factors on child psychotherapy outcome research: A Zimmerman, M., & Mattia, J. I. (1999). Is posttraumatic stress dis-
meta-analysis for researchers. Journal of Abnormal Child order underdiagnosed in routine clinical settings? Journal
Psychology , 18, 639-670. doi:10.1007/BF01342752 of Nervous and Mental Disease, 187, 420-428.

180.254.104.26 on Sun, 11 Feb 2024 16:30:13 +00:00

Why Ineffective Psychotherapies Appear

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Why Ineffective Psychotherapies Appear

Uploaded by

Copyright:

Available Formats

Why Ineffective Psychotherapies Appear to Work: A Taxonomy of Causes of Spurious

Stable URL: https://www.jstor.org/stable/44290141

This content downloaded from

Perspectives on Psychological Science

to Work: A Taxonomy of Causes of sagepub.com/journalsPermissions.nav

Spurious Therapeutic Effectiveness <§)SAGE

Scott O. Lilienfeld1, Lorie A. Ritschel2,3, Steven Jay Lynn4,

This content downloaded from

Leins et al., by2007).

This content downloaded from

Goals of the Article accurately perceive and evaluate therapeutic change. In

This content downloaded from

not come that dovetails with naive realism. This heuristic

This content downloaded from

treatment (Lilienfekl, 2007), thereby overestimating

This content downloaded from

This content downloaded from

Summary inclusive, because it does not distinguish among a myriad

Causes of Spurious Therapeutic Overview of the taxonomy of CSTEs

This content downloaded from

Table 1. Causes of Spurious Th

CSTE Description Research safeguards

This content downloaded from

CSTE Description Research safeguards

20. Regression to the mean Tendency of extreme

change stemming from extratherapeutic factors to the

This content downloaded from

therapists may enter therapy distressed about his repeated marital

This content downloaded from

negatively reinforcing these fears by change

This content downloaded from

This content downloaded from

investigators evaluated the 3-year outcome of 61 patients

11. Selective attrition. This CSTE differs from others

This content downloaded from

1975). When the

This content downloaded from

interventions, client and clinician alike

This content downloaded from

predicting undergoing behavior fro

This content downloaded from

including major depression

This content downloaded from

change and exposition

This content downloaded from

This content downloaded from

scores), as are especially suited for

Inclusion of measures of proposed mediators. The Conclusions and Future Directions

This content downloaded from

difference of opinion about which

This content downloaded from

Although In our view, the inferential

This content downloaded from

This content downloaded from

This content downloaded from

Barlow, D. H. (2004). PsychologicalPsychological

This content downloaded from

This content downloaded from

This content downloaded from

This content downloaded from

Lambert, M. J. (1976).Lilienfeld, S. O., Lynn, S. J., & Lohr, J.

This content downloaded from

Matute, H., McReynolds,

This content downloaded from