Weisz Et Al 2023 What Four Decades of Meta Analysis Have Taught Us About Youth Psychotherapy and The Science of

Annual Review of Clinical Psychology
What Four Decades of

Access provided by Pontificia Universidade Catolica do Rio Grande do Sul on 01/24/24. See copyright for approved use.
Meta-Analysis Have Taught Us

About Youth Psychotherapy
and the Science of Research
Annu. Rev. Clin. Psychol. 2023.19:79-105. Downloaded from www.annualreviews.org
Synthesis
John R. Weisz,1 Katherine E. Venturo-Conerly,1
Olivia M. Fitzpatrick,1 Jennifer A. Frederick,2
and Mei Yi Ng2
1
Department of Psychology, Harvard University, Cambridge, Massachusetts, USA;
email: john_weisz@harvard.edu
2
Department of Psychology, Florida International University, Miami, Florida, USA
Annu. Rev. Clin. Psychol. 2023. 19:79–105 Keywords

First published as a Review in Advance on
youth psychotherapy, meta-analysis, randomized controlled trial, effect
February 7, 2023
size, evidence-based psychotherapy, treatment benefit
The Annual Review of Clinical Psychology is online at
clinpsy.annualreviews.org Abstract
https://doi.org/10.1146/annurev-clinpsy-080921-
Intervention scientists have published more than 600 randomized controlled
082920
trials (RCTs) of youth psychotherapies. Four decades of meta-analyses have
Copyright © 2023 by the author(s). This work is
been used to synthesize the RCT findings and identify scientifically and
licensed under a Creative Commons Attribution 4.0
International License, which permits unrestricted clinically significant patterns. These meta-analyses have limitations, noted
use, distribution, and reproduction in any medium, herein, but they have advanced our understanding of youth psychotherapy,
provided the original author and source are credited.
revealing (a) mental health problems for which our interventions are more
See credit lines of images or other third-party
material in this article for license information. and less successful (e.g., anxiety and depression, respectively); (b) the benefi-
cial effects of single-session interventions, interventions delivered remotely,
and interventions tested in low- and middle-income countries; (c) the as-
sociation of societal sexism and racism with reduced treatment benefit in
majority-girl and majority-Black groups; and, importantly, (d) the finding
that average youth treatment benefit has not increased across five decades
of research, suggesting that new strategies may be needed. Opportunities
for the future include boosting relevance to policy and practice and using
meta-analysis to identify mechanisms of change and guide personalizing of
treatment.
79
Contents
1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
1.1. The Stakeholder’s Dilemma: What to Make of Diverse Findings
from Studies that Differ in Design, Sample Characteristics,
Methodological Quality, and Statistical Power? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
1.2. Historical Roots of Meta-Analysis and Its Eventual Application
to Psychotherapy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
2. LIMITATIONS OF META-ANALYSIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
2.1. Any Summary Based on Heterogeneous Studies Can Provide Only
a Broad-Strokes Picture; Finer-Grained Analyses Are Often Underpowered . . 82
2.2. Confounding Among Candidate Moderators Makes Conclusions Tentative
and Typically Rules Out Causal Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

2.3. Decisions Related to Lumping Versus Splitting Treatment, Control,
and Measure Types Rely on the Subjective Judgment of the Meta-Analyst
and Will Affect Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
2.4. Publication Bias May Inflate Estimates of Treatment Benefit . . . . . . . . . . . . . . . . 83
2.5. Other Forms of Bias Can Affect Study Quality, Which in Turn Can
Affect the Validity of Meta-Analytic Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
2.6. Search Strategies, Databases Selected, and Inclusion/Exclusion Criteria
May Have a Major Impact on Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
2.7. A Critical Question: What Was the Control Group? . . . . . . . . . . . . . . . . . . . . . . . . 84
2.8. Effect Sizes from Multiple Measures or Group Comparisons Within
Studies Are Not Independent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
3. ALTERNATIVE APPROACHES TO META-ANALYSIS . . . . . . . . . . . . . . . . . . . . . . . 85
3.1. Meta-Analysis of Single-Subject Experimental Design Studies . . . . . . . . . . . . . . . 85
3.2. Network Meta-Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.3. Individual Patient Data Meta-Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4. WHAT META-ANALYSES HAVE (AND HAVE NOT) SHOWN ABOUT
YOUTH PSYCHOTHERAPY: TEN EXAMPLES OF KEY FINDINGS
AND NEW APPROACHES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.1. How Much Bang for the Therapy Buck? Mean (Pooled) Effects
and Variations Across Treated Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.2. Dodo Bird Versus Different Effects for Different Treatments . . . . . . . . . . . . . . . . 88
4.3. Are There Developmental/Age Differences in the Benefits of Therapy? . . . . . . 88
4.4. How Fast Can Youth Psychotherapy Work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.5. How Effective Is Youth Psychotherapy When Delivered Remotely,
as in Pandemic Conditions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.6. Can Youth Psychotherapies (Most of Which Are from High-Income
Countries) Work in Low- and Middle-Income Countries? . . . . . . . . . . . . . . . . . . . 90
4.7. Therapy Effects and Their Social Context: Are Outcomes Worse
in Sexist and Racist Environments? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.8. Psychotherapy Effects and Principles of Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.9. Searching for Psychotherapy’s Holy Grail: Can Meta-Analysis Reveal
Mechanisms of Change? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
4.10. Are Youth Therapies Becoming More (or Less) Effective over Time? . . . . . . . . 94
80 Weisz et al.
5. CHALLENGES AND OPPORTUNITIES FOR THE DAYS AHEAD . . . . . . . . . 94
5.1. Tackling the Tower of Babel Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.2. Evaluating Hot Trends in Therapy Design and Implementation . . . . . . . . . . . . . 96
5.3. Expanding Our Understanding of How Treatments Work
(When They Do) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.4. Collaborating with Stakeholders to Make Meta-Analysis Relevant
to Real-World Policy and Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.5. Finding Funding for High-Quality, Timely Meta-Analyses . . . . . . . . . . . . . . . . . . 98
5.6. Using Meta-Analysis to Guide Treatment Personalization . . . . . . . . . . . . . . . . . . . 98

6. CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
1. INTRODUCTION
1.1. The Stakeholder’s Dilemma: What to Make of Diverse Findings
from Studies that Differ in Design, Sample Characteristics,
Methodological Quality, and Statistical Power?
We are living in an era of evidence-based mental health care for children and adolescents
(herein, “youths”). Legislators, policy makers, school personnel, service program administrators
and providers, and caregivers want to know which youth interventions are evidence based—
supported by sufficient scientific evidence to be regarded as effective—and how the evidence
compares for different interventions (Ng et al. 2021). Treatment outcome studies with youths
are designed to answer these questions, and many such studies have been published (more than
600, by our count), but using them to guide policy or clinical practice can be challenging. The
studies differ widely in interventions tested, sample characteristics, experimental design, statistical
power, and multiple dimensions of methodological quality, so their findings and conclusions dif-
fer markedly. Synthesizing studies subjectively by reviewing some of them and forming a general
impression may be risky because conclusions would depend heavily on which particular subset
of studies one finds to review, and different reviewers might use very different criteria. These
challenges have been addressed by meta-analysis, a method that aims to systematically identify
and organize treatment outcome studies and synthesize their findings to identify patterns of the-
oretical or practical importance, using transparently reported statistical procedures. The period
called youth is operationally defined in different ways in different contexts; in this article, we refer
to individuals younger than 18. Youth meta-analyses, including our own, often encompass study
samples ranging from approximately age 4 (when children may play relatively active roles in their
treatment) through 17.
1.2. Historical Roots of Meta-Analysis and Its Eventual Application

to Psychotherapy
Some experts believe that seventeenth-century astronomers’ efforts at data synthesis were an
initial step toward meta-analysis, but there is a clearer connection with a 1904 article in the
British Medical Journal that pooled data from multiple studies of typhoid inoculation (Pearson
1904) and a 1940 publication that combined 145 reports of extrasensory perception experiments
(Pratt et al. 1940). The term “meta-analysis” was coined by statistician Gene Glass and used, with
his colleague Mary Smith, to provide a synthesis of 375 controlled trials of psychotherapy and
counseling (Smith & Glass 1977). They concluded that the typical treated individual is better off
www.annualreviews.org • Youth Psychotherapy Meta-Analyses 81

after treatment than approximately 75% of the untreated comparison individuals. The methods
used by Smith and Glass have been refined by quantitative experts and applied widely—including
to youth psychotherapy (for early examples, see Casey & Berman 1985, Kazdin et al. 1990, Weisz
et al. 1987)—to estimate pooled effects; identify moderators and mediators of treatment benefit;
track trends over time within and across studies; address an array of theoretical and practical
questions; and highlight accomplishments, gaps, questions, and hypotheses for future research.
2. LIMITATIONS OF META-ANALYSIS
In this review, we illustrate some of the payoffs generated by meta-analyses. To place these into an
appropriate context, we begin by noting some limitations and challenges that warrant attention
when interpreting meta-analytic findings.
2.1. Any Summary Based on Heterogeneous Studies Can Provide Only

a Broad-Strokes Picture; Finer-Grained Analyses Are Often Underpowered
Meta-analyses inherently require combining estimates from somewhat heterogeneous studies.
Therefore, combining these estimates can provide only a general estimate of an overall ef-
fect based on these heterogeneous studies. This overall estimate may be biased in certain ways
and will likely better reflect some subgroups of studies than others (e.g., males versus females,
Whites versus minoritized youths). Finer-grained analyses (e.g., moderator analyses) designed to
estimate and compare effect sizes for various subgroups are helpful in principle but are often
underpowered.
2.2. Confounding Among Candidate Moderators Makes Conclusions Tentative

and Typically Rules Out Causal Interpretation
Moderator analyses can also be complicated by correlations among proposed moderators. For
example, family income is correlated with the number of youth psychotherapy sessions attended
by the family (e.g., Miller et al. 2008). Thus, if both variables were tested as moderators in a
meta-analysis, it might be difficult to determine the extent to which these analyses offer unique
information and, if significant effects were identified, which variable (or variables) drove these
effects. In our experience, it has been quite difficult to collect large enough study pools to conduct
properly powered tests of interactions among moderators.
2.3. Decisions Related to Lumping Versus Splitting Treatment, Control,

and Measure Types Rely on the Subjective Judgment of the Meta-Analyst
and Will Affect Findings
Meta-analytic teams must decide how to combine or split apart different categories of key vari-
ables, such as treatment type (e.g., behavioral, cognitive, other), control condition type (e.g., usual
care, wait list), and measure type (e.g., self-report, behavioral observation). Such decisions may
be based partly on precedent in the literature, theoretical and practical considerations, and the
number of studies with different characteristics, but they are at least partly subjective, and many
require balancing the ideal against the feasible. For example, fine-grained approaches that involve
creating many subgroups might provide the most precisely nuanced information, but breaking
studies into many subgroups can result in underpowered analyses. Even decisions about how to
define, combine, or distinguish between such categories can be challenging, requiring still more
subjective judgment. For example, a team seeking to learn whether evidence-based treatments
82 Weisz et al.
outperform other treatments must first define “evidence-based,” and not all experts agree on how
that should be done.
2.4. Publication Bias May Inflate Estimates of Treatment Benefit

Journal editors and reviewers, as well as study authors, may be more eager to publish news that a
treatment has worked well than news that a treatment has flopped or tanked. Thus, disappointing
findings may be less likely than exciting ones to be submitted to journals, and less likely to be
published when they are submitted. Likelihood of publication bias can be detected via funnel
plots—graphs that display the effect size estimates in a study pool plotted against study sample
size or precision in order to check for deviations from the symmetry that would be expected if
there were no relation between study precision and effect size (Egger et al. 1997). Another tool,
the funnel asymmetry test, can be used to test whether the study sample is influenced by significant
publication bias (Rosenberger & Stanley 2009). However, while these tools are helpful in detecting
publication bias, and thus signaling the need for caution in interpreting meta-analytic findings,
they do not eliminate the impact of the bias on those findings.
2.5. Other Forms of Bias Can Affect Study Quality, Which in Turn Can
Affect the Validity of Meta-Analytic Findings
The validity of a conclusion is influenced by the quality of the studies synthesized, and those stud-
ies inevitably vary in susceptibility to bias in various forms—for example, whether the assessors
who collected the outcome data were aware of whether participants were in treatment or control
groups. In some cases, study quality may be assessed using resources such as the Cochrane Risk of
Bias tool (e.g., Cuijpers et al. 2020, Venturo-Conerly et al. 2022). In many cases, however, study
quality is difficult to gauge based on the descriptions in published articles. In these cases, meta-
analytic teams often try to obtain this information from the authors of included studies. Whether
this process is successful will depend on the extent to which authors are reachable (authors of older
articles may be retired or even deceased) and willing and able to share the needed information.
When the needed information cannot be obtained, the risk of bias may be coded as unclear, and
meta-analysts may compare findings for studies with high, low, and unclear risk of bias. Increas-
ingly, particularly with the advent of more systematic reporting guidelines such as the CONSORT
checklist (Schulz et al. 2010), information relevant to assessing risk of bias is becoming easier to
obtain through published articles.
2.6. Search Strategies, Databases Selected, and Inclusion/Exclusion Criteria

May Have a Major Impact on Findings
Decisions about which databases to search, which search terms to use in the process, and what
inclusion/exclusion criteria to employ can significantly influence meta-analytic findings. One
common search strategy decision is to include only published peer-reviewed articles. While this
approach may help promote study quality, it can also increase susceptibility to the publication
bias noted above. A middle ground can be inclusion of dissertations, which have some degree
of quality control via faculty review; dissertation findings may well differ from those of pub-
lished studies (McLeod & Weisz 2004). Many meta-analyses, for feasibility reasons, do not include
studies published in languages other than English, and do not search databases that specialize in
indexing studies conducted in certain parts of the world, such as low- and middle-income coun-
tries (LMICs). These decisions are relevant to how representative the meta-analysis is and how
generalizable its conclusions may be. Even when teams conduct searches can affect which studies

might emerge and what their findings are, as findings discussed below (Weisz et al. 2019) reveal
temporal trends across the years.
2.7. A Critical Question: What Was the Control Group?

An important question to ask of any randomized controlled trial (RCT) finding is: To what control
condition was the treatment compared? Passive control conditions, such as clinical monitoring or
wait list, may control for the natural time course of recovery for some mental health problems—
but not for the participants’ investment of time and energy in treatment, or for their expectation
that the investment will lead to improvement. Psychological placebos, which appear to be treat-
ment but are not actually designed to produce mental health benefit, can do a better job of
controlling for investment and expectation. That said, creating a true psychological placebo can
be challenging, because it can be difficult to gauge in advance whether an intervention designed
as a placebo might actually produce mental health benefit. The most robust and clinically valu-
able control condition, in our view, is treatment as usual (i.e., usual care): treatment by providers
who are genuinely trying to address participants’ mental health problems and to reduce symp-
toms and suffering. Our reasoning is that whether a new treatment represents value added rests
in part on whether it produces better outcomes than interventions that are currently available and
being used, in the contexts for which the new treatment is intended. That said, the term “usual
care” is sometimes used in a rather lax way, and arguably inappropriately, to refer to situations
in which the usual is, in fact, little or even no intervention. In such cases, usual care provides
an ostensibly strong but actually weak comparison. Youth meta-analyses have rather consistently
found larger effects for treatments that are compared with inert or otherwise weak control con-
ditions (e.g., Weisz et al. 2017). Another challenge of the usual care comparison is that the nature
and quality of usual care, even when it is an active intervention, may differ across settings and
providers and, thus, across RCTs. Thus, for several reasons, it is important for meta-analysts to
read studies carefully when characterizing and coding the control conditions in included RCTs, to
routinely test whether type of control group is a moderator of effect size, and to interpret findings
accordingly.
2.8. Effect Sizes from Multiple Measures or Group Comparisons Within

Studies Are Not Independent
Effect sizes and group comparisons drawn from the same study are not independent, and this needs
to be accounted for in meta-analyses to improve accuracy of estimates. Traditionally, dependence
among effect sizes was handled by either averaging effect sizes within studies or removing all but
one effect size per study from the study pool; however, both of these approaches lose information
and limit power and accuracy (Becker 2000). Several alternative methods may be used to account
for these dependencies, including robust variance estimation (Hedges et al. 2010) and multilevel
meta-analysis (Assink & Wibbelink 2016). Multilevel meta-analysis allows for inclusion of mul-
tiple effect sizes within studies by explicitly modeling dependence at multiple levels (e.g., effect
sizes within studies, within-study variation, and between-study variation; Becker 2000, Weisz et al.
2017); however, it also requires relatively large sample sizes, particularly for moderator analyses.
An alternative meta-analytic approach for addressing dependency between effect sizes, robust vari-
ance estimation (Fisher et al. 2017, Hedges et al. 2010), which adjusts study standard errors, may
be used to conserve power in the context of smaller sample size; but this newer approach still
carries limitations in the ways groups can be compared, at least in its current form. There is no
perfect solution to the inevitable dependency among effect sizes.
84 Weisz et al.
3. ALTERNATIVE APPROACHES TO META-ANALYSIS
The most widely used approach to meta-analysis involves pooling group comparison findings
from multiple RCTs in which active treatments are tested against control conditions. The find-
ings we report in this article are primarily from meta-analyses of that type. However, much can
also be learned from meta-analyses in which treatment–treatment RCTs are pooled. Some addi-
tional variations in approach warrant attention, along with the special functions to which they are
especially well suited.
3.1. Meta-Analysis of Single-Subject Experimental Design Studies

Single-subject experimental designs (SSEDs) include multiple baseline, alternating treatment
(e.g., ABAB), and other single-case designs—approaches often used for relatively rare conditions,
for other situations where the large samples needed for RCTs are not available, and in circum-
stances where random assignment of groups is not feasible. The SSED literature includes, for
example, studies of interventions for attention-deficit/hyperactivity disorder (ADHD) (Evans et al.
2018, Pelham & Fabiano 2008), autism spectrum disorder (Smith & Iadarola 2015), and special
education (Maggin et al. 2011). Although SSEDs can be used both to establish causal effects and
as evidence that treatments are empirically supported (Chambless & Hollon 1998), they are often
excluded from conventional meta-analyses. One reason is the difficulty of synthesizing SSED and
RCT effect sizes, given the unique qualities of repeated measures data (e.g., autocorrelation; Van
den Noortgate & Onghena 2008). A challenge for consumers of SSED research can be discern-
ing how representative the findings of one particular published SSED study are of all the SSED
analyses—with other individuals and groups—that were conducted by the same team; it can also
be unclear how much the findings with one individual or small group may generalize to the rel-
evant reference group (e.g., other youths with the same diagnosis). Meta-analyses of SSEDs can
help address these concerns by synthesizing substantial numbers of SSED studies, thereby in-
creasing generalizability of conclusions. In one example, Miller & Lee’s (2013) synthesis of 82
SSED studies, encompassing 168 participants diagnosed with ADHD, found that interventions
guided by functional analysis—that is, assessment aimed at identifying and addressing the func-
tion served by problem behavior—produced larger treatment effects than interventions not thus
guided.
Treatment effects in meta-analyses of SSEDs are estimated by comparing data from the
treatment phase with data from the baseline or control phase(s). One common metric involves
computing the percentage of data points in the treatment phase that exceed either the highest
or the lowest point in the baseline phase, termed the percentage of nonoverlapping data points
(PND) ( Jenson et al. 2007, Maggin et al. 2011). There is also a multilevel extension of meta-
analysis of SSEDs which nests data by participant and by study and accounts for variation within
participants, between participants of the same study, and between studies (Van den Noortgate &
Onghena 2008). The multilevel approach can provide more accurate estimates of treatment ef-
fects compared with the PND approach because it can account for autocorrelations among the
data ( Jenson et al. 2007).
3.2. Network Meta-Analysis

Clinicians and patients often need to choose among multiple treatments, but the vast majority
of RCTs compare only two conditions—a treatment and either an alternative treatment or a
control. Similarly, most meta-analyses focus on treatment versus control conditions. Network
meta-analyses (NMAs) can answer questions about the comparative effectiveness of multiple

treatments by using hierarchical or multivariate models to compute relative effect estimates
between all pairs of treatments included in the network (Rouse et al. 2017). As long as two
treatments from different trials have both been directly compared with the same condition,
they can be indirectly compared with each other using an NMA (Rouse et al. 2017), sometimes
with intriguing results. For example, an NMA of 101 youth psychotherapy trials compared 11
different therapies for anxiety and 4 different control conditions, and produced a surprising
finding: Only group cognitive behavioral therapy (CBT) was significantly more effective than the
other therapies and all control conditions (Zhou et al. 2019).
NMA permits quantitative assessment and comparison of interventions that have not been
directly compared with one another in trials. While conclusions based on indirect comparisons,
across trials that differ on many dimensions, raise questions for some, others see NMA as a valu-
able complement to more widely used approaches. For example, in many meta-analyses, power
considerations limit the number of treatment types or control conditions that can be tested in a
moderation analysis. In contrast, NMAs synthesize data from RCTs with various combinations of
treatment–treatment and treatment–control comparisons, thereby tapping into a larger propor-
tion of the evidence base, estimating relative effects of a larger number of treatment and control
conditions, and facilitating a more precise ranking of all the conditions involved in the network.
NMAs have strict assumptions that must be met (e.g., transitivity, heterogeneity, and consistency
of comparisons) and entail complex analyses (Rouse et al. 2017), but if all assumptions are met,
NMA can be a powerful tool for comparing interventions.
3.3. Individual Patient Data Meta-Analysis

Unlike more widely used approaches to meta-analysis, which use summary statistics from studies,
individual patient data (IPD) meta-analysis uses data from individual patients to estimate treat-
ment effects. The use of patient-level data greatly expands the set of predictor and moderator
variables that can be examined, making it possible to include multiple individual difference char-
acteristics for which subgroup summary statistics are not usually reported by study authors and,
thus, are often not available for moderator testing in a typical meta-analysis. For example, one
IPD meta-analysis found that parental involvement moderated outcomes for youths who received
CBT for anxiety, such that higher degrees of parental involvement were associated with greater
symptom reduction (Manassis et al. 2014). Even when summary statistics are available at the study
or group level (e.g., percentage of sample that is female), the use of patient-level data greatly in-
creases power to detect predictors and moderators of outcome (Cuijpers et al. 2022, Tierney et al.
2015). Conversely, nonsignificant findings despite increased power provide stronger evidence that
the tested variables are truly not predictors or moderators. One IPD meta-analysis of CBT trials
for anxious youths found that several demographic and clinical characteristics (i.e., age, sex, base-
line anxiety severity, comorbid diagnosis of depression) failed to moderate outcomes, suggesting
that CBT may work equally well for anxious youths regardless of their status on any of these
characteristics (Bennett et al. 2013).
A major challenge of IPD meta-analysis is that it is extremely time and resource intensive to
gather and manage the data from multiple trials. Furthermore, its advantage over other forms of
meta-analysis in the number of candidate predictors and moderators it can examine is limited by
the number of variables that were coded and recorded in multiple included trials. The future of
IPD meta-analysis will depend partly on researchers’ willingness to share data from trials; what is
and is not shared may result in selection bias. However, there is increasing interest in IPD meta-
analysis because of the hope that using patient-level findings could translate into more useful
clinical applications for individual patients (Cuijpers et al. 2022).
86 Weisz et al.
4. WHAT META-ANALYSES HAVE (AND HAVE NOT) SHOWN ABOUT
YOUTH PSYCHOTHERAPY: TEN EXAMPLES OF KEY FINDINGS
AND NEW APPROACHES
While these variations are promising, most meta-analyses of youth psychotherapy have simply
pooled effect sizes from RCTs involving treatment versus control comparisons. In this section, we
draw from these studies to illustrate some particularly interesting contributions of meta-analysis
to our understanding of psychotherapy effects with young people. An important caveat is that the
methodological and interpretive challenges described above are relevant, to varying degrees, in
each of the reports we cite here; these challenges have been addressed to examine the robustness
of findings in a variety of ways via moderator analyses and sensitivity tests, and we encourage
readers to access the articles and review the details. Another caveat is that evidence continues
to accumulate on each topic and question. So, the findings noted here may be best regarded as
patterns in the evidence to date, subject to change in the future but offering useful hypotheses for
further study.
4.1. How Much Bang for the Therapy Buck? Mean (Pooled) Effects
and Variations Across Treated Problems
Experts have conducted meta-analyses designed to explore pooled effects of treatment targeting
various problem domains. We focus on those domains that account for a majority of clinically re-
ferred problems—anxiety, depression, conduct problems, and ADHD/attention problems (Weisz
& Kazdin 2017). Some meta-analyses have examined pooled effects of a range of evidence-based
treatments across problem domains, as well as subanalyses for each problem domain (e.g., Weisz
et al. 1987, 1995). The most recent such meta-analysis we know of synthesized findings of 447
studies (including 30,431 youths) and reported a pooled effect size of g = 0.46 across treatment–
control comparisons (Weisz et al. 2017). This is just below a widely used benchmark for a medium
effect (0.5, following Cohen 1988). This value can be converted to what is called common language
effect size—that is, the probability that a youth in the treatment condition would be better off af-
ter treatment than a youth in the control condition (McGraw & Wong 1992). That probability
across the 447 studies in Weisz et al. (2017) was 63%, only 13% better than chance expectancy of
50%. In other words, the overall pooled effect of youth psychotherapy might be best described as
moderate, not dramatically high, leaving plenty of room for improvement. The pooled effect size
was strongest for treatment of anxiety (g = 0.61), followed by conduct problems (0.46), ADHD
(0.34), and depression (0.29) (Figure 1).
Other meta-analyses have exclusively targeted specific problem domains, and specific ques-
tions within those domains. For example, Baker et al. (2021), focusing on treatments for youth
anxiety disorders, found a pooled effect size of 0.45, and Whiteside et al. (2020), focusing only on
CBT for youth anxiety, found that a greater dose of in-session exposure and response prevention
was associated with larger effect sizes, relative to control conditions. In a recent meta-analysis
of youth depression treatment trials, Eckshtain et al. (2020) found that the pooled effect size of
psychotherapy for youth depression was 0.36 at posttreatment and 0.21 at follow-up, similar to
the disappointing findings for youth depression 14 years earlier (Weisz et al. 2006). Focusing
on conduct problem treatment, Comer et al. (2013) found that psychotherapies for youth dis-
ruptive behaviors produced a pooled effect size of 0.82. Zooming in on one especially successful
approach, Ward et al. (2016) found that the mean effect size of parent–child interaction therapy
was an impressive 1.39. Meta-analyses of ADHD treatment studies have shown mixed results,
with the strongest effects for behavioral interventions. For example, Fabiano et al. (2021) found
pooled effect sizes of 0.70 for behavioral parent training and 0.66–0.72 for behavioral school-based
interventions on youth symptoms and functioning, respectively.
0.90
Posttreatment
Large
0.80 Follow-up
0.70
0.60
Medium
0.50 Mean posttreatment ES = 0.46
Mean effect size

0.40 Mean follow-up ES = 0.36
0.30 0.61
0.55
0.46 0.44
Small 0.20
0.34
0.29
0.10 0.22 0.22
0.15
0.02
0.00
Anxiety Conduct ADHD Depression Multiple

–0.10 problems
–0.20
–0.30
Figure 1
Mean effect size (ES) at posttreatment and follow-up by target problem. Dashed horizontal lines represent
mean effects for the full sample of studies reporting posttreatment assessments and for the full sample of
studies reporting follow-up assessments. Error bars represent standard error. Effect sizes at posttreatment
and follow-up for anxiety, conduct, attention-deficit/hyperactivity disorder (ADHD), and depression have
error bars that do not cross the x axis, indicating that all these effect sizes are significantly greater than zero.
Figure adapted with permission from Weisz et al. (2017); copyright 2017 American Psychological
Association.
4.2. Dodo Bird Versus Different Effects for Different Treatments

The dodo bird verdict (Luborsky et al. 2002) is the view that all bona fide therapies have very
similar effects (in part, some argue, because common factors such as the therapeutic relationship
matter most). Proponents predict that effect sizes will not differ from one treatment to the next,
and there is some support for this view: Quite a number of therapy–therapy comparisons in youth
meta-analyses either are difficult to interpret with confidence or do not differ significantly in effect
size. Findings of such broad overall meta-analytic therapy–therapy comparisons are difficult to
interpret because therapy methods are virtually always confounded with treated problems, age
of treated youths, and other potentially influential factors. Reducing confounding by focusing on
very specific subgroups of studies (e.g., treatment versus treatment comparisons for depression
among youths aged 6 to 10 years) usually generates underpowered tests. This is an area in which
the boosts in power provided by network meta-analysis (described above) can be especially helpful,
and such meta-analyses have begun to show treatment versus treatment differences. For example,
Zhou et al. (2015) found both CBT and Interpersonal Therapy for Adolescents to be significantly
superior to Problem Solving Therapy in the treatment of youth depression.
4.3. Are There Developmental/Age Differences in the Benefits of Therapy?

Most meta-analyses of youth psychotherapy have found that age does not moderate effect size,
and the few age-related differences that have been found have not been consistent in direction.
This has been the case for our own meta-analyses, which have generally spanned ages 4 through
17. The absence of consistent age effects may reflect, to some extent, the tendency of treat-
ment developers and investigators to select and adapt treatments to fit the developmental level of
88 Weisz et al.
Age category Number Effect size (95% CI)
Children 15 0.35 (0.15–0.55)
Adolescents 28 0.55 (0.34–0.75)
Young adults 27 0.98 (0.79–1.16)
Middle-aged adults 304 0.77 (0.67–0.87)

Older adults 69 0.66 (0.51–0.82)
Older old 10 0.97 (0.42–1.52)
All studies 453 0.75 (0.67–0.82)

0.0 0.5 1.0 1.5

Effect size (95% CI)
Figure 2
Effect sizes of psychotherapies for depression in different age groups. Abbreviation: CI, confidence interval.
Figure adapted with permission from Cuijpers et al. (2020); copyright 2020 American Medical Association.
participants. That said, the question remains whether, even with the best efforts of developers and
researchers, there might still be developmental differences in treatment effectiveness, particularly
if we include studies across a broad age range. To our knowledge, the broadest range encom-
passed in any psychotherapy meta-analysis is a life-span synthesis by Cuijpers et al. (2020). These
authors pooled 366 RCTs (with 36,702 participants), with age groups ranging from preadolescent
(≤13 years) to “older old” (≥75). The results (Figure 2) revealed several significant age differences.
Most relevant to our focus is that combined studies of children and adolescents showed signifi-
cantly lower mean effects than studies of young adults and middle-aged adults combined, and also
lower than the two older adult groups combined. Concerns were raised about the quality of many
of the studies and the heterogeneity of findings across studies, but most of the primary findings
were robust across sensitivity analyses. The overall pattern of the results is consistent with findings
reported elsewhere in this article that raise concern about the effectiveness of current depression
treatments with young people. One valuable function of meta-analyses can be identifying gaps
that need to be addressed in future research. Multiple findings, including those of Cuijpers et al.
(2020), suggest that there may be a need to rethink our strategies for treating pediatric depression.
4.4. How Fast Can Youth Psychotherapy Work?

The average number of sessions of youth psychotherapy tested in RCTs is 16 (Weisz et al. 2017),
but the average number of sessions actually attended by young people in real-world clinical care is
evidently around 3 to 4 (Harpaz-Rotem et al. 2004, McKay & Bannon 2004). The modal number
may be as few as 1 to 2 (Abel et al. 2022), and a very large percentage drop out of therapy before
completing it. This state of affairs raises the question of what can be accomplished if therapy is
markedly briefer than the more ambitious therapies that are so well represented in the RCTs of
our field. Stated another way: How fast can therapy work? One answer to that question comes
from the growing number of RCTs testing single-session therapies—treatments in which all the
therapy content is delivered in one sitting. A meta-analysis pooling 50 RCTs of single-session
youth therapies (with 10,508 participants; Schleider & Weisz 2017) found what, to many, may
have been a surprising result: The mean posttreatment effect size across all treated problems was

a quite respectable 0.32, with effects especially strong for treatments of anxiety (0.56) and conduct
problems (0.54). This finding suggests that there may be a future in our field for efficient, dis-
seminable interventions for young people that do not even require that they return for treatment
after an initial session. Many youths may well need more than what can be packed into a single
session, but an important question for the future is whether these brief approaches may provide
a useful complement to the traditional, multisession treatments that are so often started but not
completed by clinically referred youths. Single-session treatments might be considered an early
step in a stepped-care approach and as a way to relieve pressure on clinics that are coping with long
wait lists. Recent evidence (e.g., Schleider & Weisz 2016, 2018; Schleider et al. 2021) continues
to be encouraging; future meta-analyses can help us monitor the accumulating evidence on these
highly scalable interventions.
4.5. How Effective Is Youth Psychotherapy When Delivered Remotely,

as in Pandemic Conditions?
One effect of the COVID-19 pandemic was to essentially halt the practice of in-person therapy
in many settings and nations. This focused attention on the question of how effective the array
of approaches to remotely delivered therapy may be for young people. Even before the pandemic
surged in 2020, remotely delivered forms of youth psychotherapy, including tele–mental health
care (Am. Acad. Child Adolesc. Psychiatry 2017, Hailey et al. 2008), had grown in popularity.
Various systematic reviews had supported the feasibility and acceptability of remote psychotherapy
when used with youths and families (Am. Acad. Child Adolesc. Psychiatry 2017, Hailey et al. 2008,
Hilty et al. 2013), and a meta-analysis of computerized CBT interventions for youths with anxiety
and depression had found a medium to large effect size (Ebert et al. 2015).
Recently, a meta-analysis of 37 RCTs (with 43 treatment–control comparisons) of diverse forms
of remotely delivered youth psychotherapies (e.g., phone, email, text, prerecorded video, computer
programs) targeting anxiety, depression, conduct problems, and ADHD found a pooled overall ef-
fect size of 0.47 at posttreatment and 0.44 at follow-up (Venturo-Conerly et al. 2022), surprisingly
similar to the pooled effects found for RCTs of mainly traditional in-person youth psychother-
apies for those problems (Weisz et al. 2017). Importantly, treatment effects were substantial for
anxiety (0.52) and conduct problems (0.78) but small and nonsignificant for ADHD (−0.03) and
depression (0.09), suggesting the possibility that not all problems are equally amenable to re-
mote intervention. In line with a previous meta-analysis of adult computerized psychotherapies
(Andersson & Cuijpers 2009), Venturo-Conerly et al. (2022) also found that remotely delivered
youth psychotherapies that included provider support for skill building far outperformed those
that did not have therapeutic provider contact (0.68, 0.18). As more RCTs accumulate, it will be
valuable to conduct a similar meta-analysis with a sufficiently large study pool to permit properly
powered tests of moderators, including some for which certain levels have to date been completely
unrepresented (e.g., Venturo-Conerly et al. found no studies of therapy via video calls).
4.6. Can Youth Psychotherapies (Most of Which Are from High-Income

Countries) Work in Low- and Middle-Income Countries?
Monitoring by global entities such as the World Health Organization (WHO 2018) reveals a
massive need for effective mental health care in LMICs. Many tested psychotherapies for young
people might make a difference, but the great majority of them were developed and tested in
high-income, Western countries, so it has not been clear whether the treatment development and
testing done to date will be very helpful for youths in LMICs. Encouraging news comes from a
recent meta-analysis of 33 RCTs (with 42 treatment–control comparisons and 3,763 participants)
90 Weisz et al.
that tested youth psychotherapies in LMICs of five continents (Venturo-Conerly et al. 2023). The
treated problems encompassed anxiety, depression, conduct problems, and ADHD. The authors
reported a pooled posttreatment effect size (1.01) that was nearly double that found in the most
recent comparable meta-analysis of RCTs in mainly high-income countries (Weisz et al. 2017).
This large effect may seem surprising, but it is actually consistent with findings of a meta-
analysis comparing effects of adult psychotherapies for depression delivered in high-income
countries versus LMICs (Cuijpers et al. 2018). The reasons for these findings are unclear; it could
be that the psychotherapies found to be most effective in high-income countries are the ones most
likely to be selected for testing in LMICs. Alternatively, perhaps the need is so great in LMICs,
and previous professional mental health service access so unlikely, that the stage is set for carefully
developed and implemented treatments to have a powerful impact. Proper testing of these and
other explanations, and testing of moderators of treatment benefit, will require a massive boost in
the number of LMIC trials. Currently, although 90% of the world’s youths live in LMICs, fewer
than 10% of youth RCTs have been conducted in these settings (Weine et al. 2020). This dis-
crepancy underscores the need for more funding for youth psychotherapy trials in LMICs, and
subsequently for better-powered meta-analyses of those trials.
4.7. Therapy Effects and Their Social Context: Are Outcomes Worse
in Sexist and Racist Environments?
It is common to think of psychotherapy outcomes as the result of what happens during therapy.
That must certainly be true, but what if there is more to the story—could the outcomes of therapy
also be influenced by the environment in which youths exist and to which youths return after
therapy sessions (Weisz et al. 2013)? One way to explore that possibility is by using spatial meta-
analysis—an approach that involves geolocating the studies included in meta-analyses in order
to characterize the social environment in which the included studies were conducted. Doing so
permits spatial meta-analysts to examine associations between contextual–environmental variables
and study outcomes. This approach has been used in recent meta-analyses that suggest important
hypotheses about the impact of sexism and racism on treatment outcomes. In one such meta-
analysis, Price et al. (2021) coded US states and counties in which each of 314 RCTs (with 19,739
participants) of youth psychotherapies for internalizing and externalizing disorders and problems
had been conducted. The locations were coded for degree of sexist attitudes toward females, using
19 indicators drawn from (a) four decades of data from a public opinion survey (e.g., about whether
it is better for women to stay at home and care for children and whether men are better suited to
politics) and (b) Implicit Association Test data relating to implicit attitudes about gender in relation
to careers and science. The findings showed that higher levels of cultural sexism, thus assessed,
were associated with lower effect sizes for studies with majority-girl samples, and the greater the
proportion of girls in the study samples was, the stronger the association became. In contrast,
antifemale sexism was unrelated to treatment effect sizes for studies that had majority-boy samples.
In a second meta-analysis, Price et al. (2022) used spatial meta-analysis to investigate whether
there might be a connection between cultural racism and treatment outcomes. Some 194 youth
psychotherapy RCTs (with 14,081 participants) were coded with regard to level of anti-Black
racism in the settings where the studies were conducted, using 31 items from an implicit atti-
tudes test and two public opinion surveys (sample item: “I would rather not have Black people
live in the same apartment building I live in.”). As Figure 3 illustrates, treatment benefit in sam-
ples with majority-Black samples grew weaker as level of anti-Black racism in the environment
increased, but the level of racism was unrelated to treatment outcome in majority-White samples.
The findings of these two meta-analyses raise an important question about whether the impact of

Majority Black youths (k = 385) Majority White youths (k = 2,348)
1.0
Effect size (g)

0.5
–2 –1 0 1 –2 –1 0 1
Cultural racism
Figure 3
Anti-Black cultural racism and effect size across subsets by majority race. Abbreviation: g, type of effect size,
specifically Hedges g. Figure adapted with permission from Price et al. (2022, p. 759); copyright 2022
Elsevier.
psychotherapy on young people may be affected, perhaps markedly, by the social environment in
which youths exist and to which they return after therapy sessions. This is a good example of how
provocative findings from meta-analyses may set the stage for hypothesis testing in controlled tri-
als and for increased attention to the potential impact of young people’s social environments on
the benefit they derive from psychotherapy.
4.8. Psychotherapy Effects and Principles of Change

For more than 30 years, prominent intervention scientists have proposed a focus on empirically
supported principles of change (ESPCs) in psychotherapies, with some suggesting that these
might replace or complement name-brand therapies (e.g., Castonguay & Beutler 2006; Davison
2019; Goldfried 1980, 2009; Hofmann & Hayes 2019; Oddli et al. 2016; Rosen & Davison
2003; Tolin et al. 2015; Weisz & Bearman 2020; Weisz et al. 2021). ESPCs have been defined
as a middle ground between more abstract theoretical orientations (e.g., CBT, psychodynamic
therapy) and more concrete, specific treatment procedures (e.g., cognitive restructuring, exposure,
transference assessment; Goldfried 1980). This framework is in line with historical and recent
92 Weisz et al.
recommendations that psychologists focus attention on the processes of change that may be
activated during effective psychotherapy (e.g., Bandura 1969, Conway et al. 2019).
Recently, several authors (Fitzpatrick et al. 2023, Weisz & Bearman 2020) used a review of hun-
dreds of RCTs to identify five candidate ESPCs that had been tested as solo interventions in RCTs
and found to outperform control and comparison conditions, making them significant candidates
for ESPC status. Fitzpatrick et al. (2023) then carried out a review and meta-analysis of the youth
treatment outcome evidence base, focusing on these five candidate ESPCs: calming, increasing
motivation for desired behaviors, changing unhelpful thoughts, solving problems, and practicing
“positive opposites” of undesired behaviors. Fitzpatrick et al. synthesized 348 treatment–control

comparisons from 263 RCTs (with 19,004 participants) spanning six decades, testing treatments
for anxiety, depression, ADHD, and conduct problems. They found that (a) ESPCs could be reli-
ably identified and distinguished by independent coders and (b) psychotherapies usually included
fewer than three ESPCs, but that (c) across the entire study pool and the anxiety subsample,
controlling for dose, treatments with all five ESPCs showed effects about twice as large as treat-
ments with fewer ESPCs. These findings suggest that ESPCs are reliably identifiable, that most
treatments combine multiple ESPCs, and that treatments containing more ESPCs may produce
greater therapeutic benefit. The finding regarding the number of ESPCs within treatments relates
to what might be called the conceptual density of youth psychotherapies. Future meta-analyses
might complement this focus by examining procedural density (i.e., the number of specific treat-
ment procedures included in treatments). A perennial question for treatment developers is: How
much treatment content is optimal—neither too little to provide what youths need nor too much
for them to assimilate? Meta-analyses may help answer that question for both conceptual and
procedural content.
4.9. Searching for Psychotherapy’s Holy Grail: Can Meta-Analysis Reveal

Mechanisms of Change?
For many who study psychotherapy, the holy grail is the mechanism of change—the internal switch
that, when flipped, makes psychotherapy work. Theories have proliferated, but the requirements
for identification of a genuine mechanism are stringent (Kazdin 2007), and no proposed mecha-
nism has satisfied those requirements to date. The search for that holy grail may be accelerated by a
recently developed twist on conventional meta-analysis that can power the search for mediators of
change, a critical step on the path to identifying mechanisms. The conventional approach focuses
on the relationship between two variables—such as treatment–outcome or process–outcome—
with treatment type, participant, and other characteristics tested as moderators of the relationship.
Meta-analytic structural equation modeling (MASEM; Cheung & Hafdahl 2016) allows one to
examine complex models that encompass relationships among multiple variables—for example, a
treatment–process–outcome model, with multiple mediators, covariates, and moderators—even if
these models were not tested by authors of the original RCTs. Each study need only contribute cor-
relations between some of the variables in the model to be included in analyses. In this approach,
fixed- or random-effects meta-analysis is used to synthesize effect sizes from different studies into
a pooled correlation (or covariance) matrix, which is then used to fit the models to be examined
(Cheung 2015). A recent MASEM report (M. Ng, K. DiVasto, N. Gonzalez, S. Cootner, M. Lipsey
& J. Weisz, manuscript submitted) investigated candidate change mechanisms of CBT and inter-
personal psychotherapy (IPT) for youth depression, using a collection of 34 CBT and IPT RCTs
(with 3,891 participants) that spanned three decades. The RCTs had measured various combina-
tions of seven candidate mechanisms for youth depression. The MASEM analyses showed that
reducing negative cognition was a robust mediator of depression reduction, evident in 24 RCTs;
however, surprisingly, that mediation was evident across treatments, not specific to trials of CBT.
Consistent with IPT theory, improved social engagement and family functioning mediated depres-
sion reduction more strongly in IPT than in CBT. Contradicting CBT theory, the MASEM found
no evidence of treatment or mediation effects involving problem-solving or reframing. MASEM
poses practical challenges for authors—for example, creating, coding, and managing potentially
massive correlation matrices—but these early findings supporting some proposed mechanisms
and discrediting others illustrate the rich potential of this potent new method.
4.10. Are Youth Therapies Becoming More (or Less) Effective over Time?
A valuable function of meta-analyses—especially when they span long periods of time—can be

big-picture evaluation of progress in the field. In some fields, medical science for example, meta-
analyses have shown dramatic improvements in treatment benefit; consider childhood cancer, for
which 50 years of research has boosted recovery rates from less than 30% to more than 80%
(Weisz et al. 2019). To find out whether research on youth psychotherapy has similarly produced
gains over time, Weisz et al. (2019) carried out a meta-analysis of 453 RCTs (with 31,933 partici-
pants), tracking temporal trends in pooled effect sizes across five decades (1963–2016). There was
a significant interaction between study year and treated problem/disorder, so separate analyses
were conducted with studies treating anxiety, depression, ADHD, and conduct problems. These
analyses showed no significant improvement across the five decades in the effect sizes produced
by treatments for any of these four problem domains. In fact, treatment effects actually grew
significantly weaker over time for depression and for conduct problems (Figure 4).
It is interesting to consider these findings in combination with those of a meta-analysis by Jones
et al. (2019). Jones and colleagues used data from 502 RCTs of youth psychotherapy (with 38,055
participants) to generate a bivariate copula model from which they predicted changes in effect
size as therapy quality improves, approaching “infinitely” good quality. The findings, based on
the current array of therapies available for youths, indicated that even a therapy of perfect quality
would produce an estimated Hedges g of only 0.83. The findings suggested that other kinds of
therapies, and potentially other ways of implementing therapies, may be needed to break through
a relatively low ceiling of potential therapy benefit from currently available approaches. The Jones
et al. findings, together with the temporal trend findings of Weisz et al. (2019) (Figure 4), suggest
that our field’s approach to psychotherapy research over the past half-century has not generated
the kinds of improved benefits produced by research on pediatric interventions in other fields. We
may need to consider whether changes are needed in our field’s approach to youth psychotherapy
development and evaluation.
5. CHALLENGES AND OPPORTUNITIES FOR THE DAYS AHEAD

Even as meta-analyses have challenged intervention science, including the very trajectory of youth
psychotherapy research, they have spurred fresh thinking and innovation. This has created new
opportunities to advance the field in potentially significant ways (summarized in Table 1).
5.1. Tackling the Tower of Babel Problem

While evidence synthesis can be valuable in identifying trends and central tendencies across nu-
merous individual trials, the specific trends and central tendencies identified are very much a
function of the ground rules set by the synthesizers—including which databases will be searched,
which study inclusion/exclusion criteria will be applied, how study effect sizes will be synthesized
and dependencies among them addressed, how risk of bias will be assessed and addressed, and many
others. As a simple illustration, consider the contrast between several published meta-analyses of
94 Weisz et al.
a Anxiety b Depression
1.0 1.0
0.8 0.8
Mean effect size (g)
0.6 0.6
0.4 0.4
0.2 0.2
0 0
1970 1980 1990 2000 2010 1990 2000 2010
c ADHD d Conduct problems

1.0 1.0
0.8 0.8
Mean effect size (g)
0.6 0.6
0.4 0.4
0.2 0.2
0 0
1970 1980 1990 2000 2010 1960 1970 1980 1990 2000 2010
Study year Study year
Figure 4
Estimated change in mean effect size over time for treatment of (a) anxiety, (b) depression, (c) attention-deficit/hyperactivity disorder
(ADHD), and (d) conduct problems in the mixed-effects model. The lines indicate the mean Hedges g, and the shading around lines
represents the 95% confidence interval. There was a significant interaction between study year and target problem. There was no
significant change in mean effects across the years for anxiety and ADHD, but there was a significant decline for depression and
conduct problems. Figure adapted with permission from Weisz et al. (2019); copyright 2019 Association for Psychological Science.
youth depression treatment RCTs, on the one hand, and a recent US Agency for Healthcare Re-
search and Quality (AHRQ) report (Viswanathan et al. 2020), on the other. The AHRQ systematic
review included only RCTs with youths who had a “confirmed diagnosis” of a depressive disorder,
and only RCTs from countries with a very high Human Development Index (HDI). Applica-
tion of these and other inclusion criteria led to a pool of 23 RCTs testing nonpharmacological
treatments. The only such treatment clearly identified in the AHRQ report as possibly reducing
depressive symptoms is CBT; family-based IPT is noted as having only one supportive RCT. Most
meta-analyses of youth depression RCTs that are done by investigators outside of AHRQ con-
tracts include studies with participants who have either elevated scores on standardized depression
symptom measures or diagnoses, and most do not require study countries to have high HDI. For
example, a recent youth depression treatment meta-analysis by Eckshtain et al. (2020) included
55 RCTs (versus 23 in AHRQ), five RCTs of IPT (versus one in AHRQ), and the important find-
ing that IPT produced significantly larger effects than CBT—quite a different takeaway than the
pro-CBT conclusion readers would derive from the AHRQ report.

Table 1 Challenges and opportunities for meta-analysis in youth psychotherapy
Challenges Opportunities
Tower of Babel problem. Diverse approaches to evidence Liberal cross-referencing and reviews of reviews, identifying
synthesis lead to differing findings and conclusions about common and differing findings, could help readers locate the
the same question. findings that fit their population and mission.
Good idea or not? Appealing ideas and hot trends in youth Timely meta-analyses, as RCTs accumulate, can gauge which new
psychotherapy may or may not lead to improved directions are and are not supported by the evidence.
treatment outcomes.
What makes therapies work? Relatively little is known about Synthesize the very large quantity of unused evidence on putative
how youth psychotherapies work, when they do. mediators and change mechanisms.
Translation needed. Meta-analyses for scientists (noting Collaborate with stakeholders to identify the questions and
effect size, heterogeneity, risk of bias) can be hard for reporting formats needed for their work; complement scientific
stakeholders in policy and practice to assimilate and use. publications with reports for policy and practice communities.
Funding orphans. Unlike RCTs and other empirical studies, Create competitive funding opportunities with peer review for
meta-analyses are typically done with no funding, by timely production of meta-analyses judged by experts to be most
volunteers; quality and timeliness may suffer. urgent and valuable for the field.
Fit or misfit. Individually personalized treatment may Combine evidence on outcome predictors from hundreds of RCTs,
produce the best outcomes, but how do we identify the using combined synthesis tools (e.g., network and individual
best-fit treatment for one individual? patient data meta-analyses) to create online tools that use
multiple person variables to identify best-fit interventions.
Abbreviation: RCT, randomized controlled trial.
What should policy makers, clinic administrators, clinicians, or parents conclude about which
of these two treatments to use to help depressed youths? The fact that different ways of synthe-
sizing evidence produce different conclusions has implications that go beyond science, affecting
public policy, clinical practice, and even family decision-making. Eliminating such differences
might require a higher authority who dictates exact rules for synthesis that all must follow; that’s
not happening, and probably shouldn’t! At least two more realistic options could be informative
and useful. First, it would be helpful if the authors of any evidence synthesis made a vigorous effort
to identify, describe, and cite other syntheses on the same topic, so that readers have access to other
findings derived from other approaches. This would be a significant change; to date, relevant pub-
lished meta-analyses are often not cited in AHRQ reports, and relevant AHRQ reports are often
not cited in published meta-analyses. Second, given the diversity of methods and findings we now
confront, the time seems right for meta-syntheses, using something like the systematic review of
reviews methodology that is increasingly evident in medical fields (Smith et al. 2011). Summary
reports on what meta-analyses and systematic reviews have been conducted, with what scope and
methods, and what their findings have been could help stakeholders (a) identify common findings
across the various methods and, (b) where findings differ, determine which evidence syntheses and
findings are most appropriate to the population they serve and the needs they seek to address.
5.2. Evaluating Hot Trends in Therapy Design and Implementation

Youth psychotherapy developers and researchers are continually creating new models and ap-
proaches, as well as launching evaluative trials. By synthesizing the trials as they accumulate,
meta-analysts can help us gauge what the growing evidence base reveals about the potential of
these innovations. Sometimes, findings may reveal significant potential, as with the surprisingly
positive pooled effects of 50 RCTs of single-session therapies, discussed above (Schleider & Weisz
2017). At other times, the findings may be disappointing, as in a meta-analysis of 23 RCTs of
cognitive bias modification (CBM), which led the authors to conclude: “Our results cast serious
96 Weisz et al.
doubts on CBM interventions having any clinical utility for nonadult populations” (Cristea et al.
2015, p. 723). Carefully done syntheses of the evidence on hot trends in the field can be extremely
valuable guides, helping clinical scientists identify those that are, and are not, supported by the
accumulation of early evidence.
Other recent developments in youth intervention are gaining steam and in need of such
scrutiny in the days ahead. Figure 1 shows an especially modest effect size for youth treatments
that have targeted multiple problems, but an increasing number of trials have tested new transdi-
agnostic interventions that use modular approaches (e.g., Marchette & Weisz 2018, Weisz et al.
2015); those trials need to be pooled to tell us what early evidence is showing. Other examples
include mental health apps and other purely digital interventions, which have proliferated in the
past decade; stepped-care models of treatment; and the rich variety of innovations highlighted in
Kazdin & Blase’s (2011) widely cited survey of disruptive interventions and delivery models.
5.3. Expanding Our Understanding of How Treatments Work (When They Do)
Maximizing treatment gains and minimizing duration will require a thorough understanding of
how treatments produce beneficial effects via change processes (i.e., what the therapist and client
do in session or as a direct result of sessions) and change mechanisms (i.e., changes in client skills
and characteristics; Doss 2004). Unfortunately, progress has been stymied by a low proportion
of trials that have examined candidate change processes and mechanisms, small study samples
resulting in low power to detect mediation effects, and conflicting findings within and between
studies—which call into question whether different findings stem from variation across studies or
from lack of power (Forehand et al. 2014; Ng et al. 2020, 2021). Meta-analysis can help, addressing
mixed findings and sample size issues by estimating mean effects while accounting for differences
in precision due to sample size, identifying sources of heterogeneity (by assessing moderators),
minimizing variance due to differences in analytic method, and increasing power to detect treat-
ment and moderation effects by pooling multiple studies. Yet, to our knowledge, meta-analyses of
change processes and mechanisms are uncommon in psychotherapy research.
A rare example is a meta-analysis by Chu & Harrison (2008). They found that CBT for youth
anxiety had medium to large significant effects on cognitive, behavioral, physiological, and coping
candidate mechanisms, but that CBT for youth depression had only small effects on cognitive
candidate mechanisms and nonsignificant effects on other candidate mechanisms. Another ex-
ample is a multilevel meta-analysis by Leijten et al. (2018), who examined the specific effects
of experimentally manipulated change processes in behavioral parent training (see also Leijten
et al. 2021). They reported that time out, selective attention, and verbal reprimands—but not
praise—increased child compliance with caregivers. There is untapped potential for many more
contributions like these. A recent analysis (Ng et al. 2020) found that nearly 75% of CBT and
IPT trials for youth depression measured candidate change processes or mechanisms, but fewer
than 20% reported any mediation tests. There is clearly an enormous pool of data awaiting the
enterprising meta-analyst who wants to help us all understand how youth psychotherapies work,
when they do.
5.4. Collaborating with Stakeholders to Make Meta-Analysis Relevant

to Real-World Policy and Practice
Meta-analyses are typically designed and written up for an audience of clinical researchers,
following reporting guidelines (e.g., PRISMA) that emphasize methodological details and such
metrics as effect size heterogeneity and risk of bias ratings. The reports are also typically evaluated
by researchers who serve as journal editors and reviewers and decide whether the findings get

disseminated. There is some question about how accessible and useful these reports are for those
who actually make policy and practice decisions about youth psychotherapy—for instance, clinic
and school mental health administrators who plan services for youths as well as local, state, and na-
tional policy makers who decide which problem areas and treatments to prioritize for services and
funding. Health care managers and policy makers in Canada and the USA have reported in surveys
(Marquez et al. 2018, White et al. 2017) that the traditional format of meta-analysis reports is
difficult to use; that the technical language, statistics, and plots can be difficult to understand; and
that the time required to carry out and eventually publish meta-analyses results in dated findings.
Respondents have reported preferences for information on cost, implementation, comparison of

the efficacy of different treatments (not just comparison to control groups), formats that highlight
key messages and brief summaries with links to full reports, and ease of access through email or
common search engines. Such findings and recommendations suggest a need for meta-analysts to
connect with stakeholders as well as with professional associations and journal editors to identify
ways to make meta-analyses both accessible and useful to decision makers. And what about the
most proximal decision makers—caregivers? There could be genuine real-world value in translat-
ing meta-analytic findings into a form that is accessible to lay readers and presenting the findings,
thus translated, via the media to which family members are most likely to turn for guidance.
5.5. Finding Funding for High-Quality, Timely Meta-Analyses

A major challenge in conducting broad-based meta-analyses that encompass multiple problems
and treatments is that the process is time consuming and labor intensive, with little financial sup-
port available. To illustrate the time required, our group’s meta-analysis data set development
process—encompassing a range of treatments and treated problems—includes iterative literature
searches, organization and cleaning of search results, double-screening of articles yielded from the
search, extensive and detailed coding of each included article (e.g., with double-coding, checking,
and revising for coders to reach our reliability standard), and a synthesis of new data with the large
existing database that extends back to the 1960s. Because grant opportunities are rarely available
for such work, teams that conduct meta-analyses must rely on gratis contributions from students
and other collaborators who are already fully employed. So, available time is limited, there is
frequent turnover in the “workforce,” and the time required to complete a meta-analysis is so ex-
tended that it may not be possible to ever be completely current. In our experience, executing the
entire process of updating our meta-analysis data set in a careful, accurate way has required about
1.5 years for each additional year of published articles. Essentially, we never catch up. This may
be a challenge for other meta-analytic teams, as well, and a challenge for our field. Dated findings
were also raised as a concern in the decision-maker surveys noted in the previous subsection. If it
is important to identify and synthesize the RCT evidence base to characterize the science of youth
psychotherapy, and to do so in a timely way, then it also seems important to create more funding
opportunities—for example, through government agencies and foundations—to make that pos-
sible. Peer review of funding applications, similar to that for individual RCT applications, could
help ensure that meta-analyses deemed most valuable to the field are appropriately staffed and
timely enough to truly reflect the current state of knowledge.
5.6. Using Meta-Analysis to Guide Treatment Personalization

Heather is a clinician preparing for treatment with a new client, Keisha, who is struggling with
depression. Keisha is a 16-year-old math whiz who identifies as Black, female, and gay; is troubled
by conflict with some of her valued peers; and has a family history of depression. Heather follows
meta-analyses closely; she sees support for youth depression treatments that focus on cognitive
98 Weisz et al.
restructuring, others that use behavioral activation, and others that use IPT. Which should she use
with Keisha, and are there adjustments she should make to fit Keisha’s age, talents, and identity
more precisely? Herein lies the clinical challenge of meta-analysis, and indeed of RCTs: The find-
ings refer to groups, not individuals. Individually personalized treatment may have the potential
to optimize outcomes (Huibers et al. 2021, Ng & Weisz 2016), but the evidence base and strate-
gies needed to inform personalizing are very much works in progress. An important step will be
to identify client characteristics that are linked to superior outcomes of specific interventions.
Emerging research with adults (Cohen & DeRubeis 2018, DeRubeis et al. 2014) and youths
(Young et al. 2021) suggests, as we might expect, that optimal treatments differ depending on per-
sonal and clinical characteristics. Treatment personalization needs to be informed by evidence on
both the relative efficacy of multiple treatments and variations in efficacy associated with multiple
client characteristics. This approach is nicely illustrated by a recent meta-analysis of internet-
based treatments for depressed adults. Karyotaki et al. (2021) combined a network data and IPD
meta-analysis with data from 8,107 participants in 39 RCTs. Their analyses identified baseline
depression severity as the most important prognostic factor for determining outcome, but the in-
dividual client data were used to provide a kind of personalizing tool, complete with sliders, to
show variations in outcomes depending on client age, gender, level of baseline severity, relation-
ship status, and employment status. This tool suggests a possible future in which participant data
from multiple RCTs are synthesized using advanced meta-analytic methods to inform decisions
about treatments for individual patients. The utility of this approach will depend on the array of
moderators tested in RCTs, and the effectiveness of decision tools will need to be tested in clini-
cal care contexts; but the research suggests one path to inform clinicians as they make treatment
decisions for individual clients.
6. CONCLUSION
Youth psychotherapy researchers have carried out more than 600 RCTs in the past six decades,
and meta-analysts have been busy synthesizing the findings for four decades. Both the diversity
and the rigor of the meta-analytic tools they have used have accelerated over the years, sharpening
the precision with which it has been possible to estimate pooled effects, identify moderators and
mediators of treatment benefit, track trends over time within and across subgroups of studies,
highlight accomplishments and gaps, and articulate questions and hypotheses for future research.
To be sure, there are persistent limitations. Confounding among variables of interest rules out
clear causal inference, and the moderator analyses needed to address this challenge are typically
underpowered. Included studies may have significant risk of bias in multiple forms, and publication
bias may even affect which studies the meta-analyst finds. Diverse decisions about methods are
judgment calls, which may differ from one meta-analyst to the next. These limitations, together
with the burgeoning evidence from new RCTs, mean that—as is true of any individual RCT—
findings are best regarded as patterns in the evidence to date, subject to change in the future but
offering useful hypotheses for further study.
That said, the findings reported in this article offer a useful sample of what evidence synthesis
can tell us about the state of knowledge about psychotherapy with children and adolescents. The
findings suggest that the effect of psychotherapy for the most commonly treated problems, averag-
ing across more than 400 RCTs, is moderate, generating a 63% probability that the average treated
youth will be better off after treatment than the average youth assigned to a control group—in
other words, a 13% advantage over the chance probability of 50% (Weisz et al. 2017). This ob-
servation suggests that the many tested youth psychotherapies have been producing measurable
benefit, but that there is a great deal of room for improvement. Notably, meta-analyses of youth
depression treatment have shown persistently modest effects (e.g., Eckshtain et al. 2020, Weisz

et al. 2006), declining effect sizes across decades of research (Weisz et al. 2019), and markedly
smaller effects than those of adult depression treatment (Cuijpers et al. 2020). One suggestion
emerging from these various strands of research is that our approaches to treating youth depres-
sion need close examination, together with some creative thinking about what is missing and what
new approaches may produce different results.
The heads-up for our field may be broader still, because the meta-analytic evidence also showed
declining effects for treatment of youth conduct problems and a lack of significant increase in
effects for anxiety or ADHD treatment across time (Weisz et al. 2019). These results mesh with
the finding in the Jones et al. (2019) meta-analytic copula analysis suggesting that the upper limit
of effect sizes that can be achieved with the current array of youth treatments is relatively modest.
Together, these findings highlight the question of whether the scientific strategy for development
and improvement of youth treatments needs to be reevaluated, and perhaps restructured, to fuel
the gains in treatment benefit observed in numerous other domains of pediatric health care.
These worrisome findings coexist with some encouraging news on other fronts. For example,
findings have shown beneficial effects of remarkably brief youth psychotherapy—even therapy
that is completed in a single session (Schleider & Weisz 2017). Other meta-analyses have found
that effects of remotely delivered psychotherapy—as during the COVID-19 pandemic—are very
similar to those of therapy delivered in person (Venturo-Conerly et al. 2022), and that the effects
of therapies that have been developed and tested largely in high-income Western countries have
produced effects about twice as large, on average, when delivered in LMICs (Venturo-Conerly
et al. 2023).
Although it is common to think of psychotherapy effects as being generated by activities during
therapy, recent meta-analyses have highlighted the potential impact of the environment outside
of therapy. Reports based on the geolocation method called spatial meta-analysis have shown that
therapy effects for majority-girl groups were reduced when treatment was delivered in states and
counties rated high on sexism (Price et al. 2021). Similarly, therapy effects for majority-Black
youth groups were reduced when treatment was implemented in locations rated high on anti-
Black racism (Price et al. 2022). When we look for reasons why youth psychotherapy effects might
be less than optimum in the meta-analyses cited above, perhaps we should look not only to the
treatments and their implementation but also to the social context in which treatment took place.
Finally, we have emphasized the value of making meta-analyses useful to those who serve youths
in the worlds of policy and practice. There is much that can be done, we suspect, to design, frame,
and report meta-analyses in ways that, while retaining their scientific value, will have practical
utility for leaders of youth-focused policy-making bodies, clinical administrative leaders, clinicians,
and perhaps even families seeking the care that is best supported by the evidence. We look forward
to the day that meta-analyses can inform decisions in ways that make youth mental health care
evidence based, individually personalized, effective, and accessible.
SUMMARY POINTS
1. Youth psychotherapy meta-analyses have limitations but also reveal key patterns.
2. Mean effects are in the medium range, strongest for anxiety and weakest for depression.
3. Benefits are evident for single-session therapies and those delivered remotely.
4. Sexism and racism are linked to reduced treatment benefit for girls and Black youths.
5. Treatment benefit has not increased across five decades; new methods may be needed.
100 Weisz et al.

DISCLOSURE STATEMENT
J.R.W. is funded by the National Institute of Mental Health (R01MH123591), the Manton Foun-
dation, the Marriott Foundation, School Mental Health Ontario (SMHO), and the Templeton
World Charity Foundation (TWCF0633 and TWCF0509); he receives royalties from Cambridge
University Press, Guilford Press, and PracticeWise. K.E.V.-C. receives funding from the Tem-
pleton World Charity Foundation (TWCF0633 and TWCF0509), Harvard University Dean’s
Competitive Fund for Promising Scholarship, and SMHO. O.M.F. receives funding from the Na-
tional Institute of Mental Health (F31MH127862), Harvard University Dean’s Competitive Fund
for Promising Scholarship, and SMHO. M.Y.N. and J.A.F. are funded by a grant from the National
Institute of Mental Health (R21MH126394), and M.Y.N. has a Child Intervention, Prevention,
and Services (CHIPS) Fellowship funded by the National Institute of Mental Health.
ACKNOWLEDGMENTS
We are grateful to the many colleagues who have collaborated with us in meta-analyses described
in this article, and to the many other meta-analysts whose diligence and innovations in data synthe-
sis have enriched our understanding of youth psychotherapy. We are also grateful to the funders
noted in the Disclosure Statement above for the resources they have provided in support of our
research.
LITERATURE CITED
Abel M, Bianco A, Gilbert R, Schleider JL. 2022. When is psychotherapy brief? Considering sociodemo-
graphic factors, problem complexity, and problem type in U.S. adolescents. J. Clin. Child Adolesc. Psychol.
51(5):740–49
Am. Acad. Child Adolesc. Psychiatry. 2017. Telepsychiatry with children and adolescents. J. Am. Acad. Child
Adolesc. Psychiatry 56(10):875–93
Andersson G, Cuijpers P. 2009. Internet-based and other computerized psychological treatments for adult
depression: a meta-analysis. Cogn. Behav. Ther. 38(4):196–205
Assink M, Wibbelink CJM. 2016. Fitting three-level meta-analytic models in R: a step-by-step tutorial. Quant.
Methods Psychol. 12(3):154–74
Baker HJ, Lawrence PJ, Karalus J, Creswell C, Waite P. 2021. The effectiveness of psychological therapies for
anxiety disorders in adolescents: a meta-analysis. Clin. Child Fam. Psychol. Rev. 24(4):765–82
Bandura A. 1969. Principles of Behavior Modification. New York: Holt, Rinehart & Winston
Becker BJ. 2000. Handbook of Applied Multivariate Statistics and Mathematical Modeling. San Diego, CA:
Academic
Bennett K, Manassis K, Walter SD, Cheung A, Wilansky-Traynor P, et al. 2013. Cognitive behavioral ther-
apy age effects in child and adolescent anxiety: an individual patient data metaanalysis. Depress. Anxiety
30(9):829–41
Casey RJ, Berman JS. 1985. The outcome of psychotherapy with children. Psychol. Bull. 98:388–400
Castonguay LG, Beutler LE. 2006. Principles of therapeutic change: a task force on participants, relationships,
and techniques factors. J. Clin. Psychol. 62(6):631–38
Chambless DL, Hollon SD. 1998. Defining empirically supported therapies. J. Consult. Clin. Psychol. 66(1):7–18
Cheung MW. 2015. metaSEM: an R package for meta-analysis using structural equation modeling. Front.
Psychol. 5:1521
Cheung MW, Hafdahl AR. 2016. Special issue on meta-analytic structural equation modeling: introduction
from the Guest Editors. Res. Synth. Methods 7:112–20
Chu BC, Harrison TL. 2008. Disorder-specific effects of CBT for anxious and depressed youth: a meta-
analysis of candidate mediators of change. Clin. Child Fam. Psychol. Rev. 10(4):352–72
Cohen J. 1988. Statistical Power Analysis for the Behavioral Sciences. Mahwah, NJ: Erlbaum
Cohen ZD, DeRubeis RJ. 2018. Treatment selection in depression. Annu. Rev. Clin. Psychol. 14:209–36

Comer JS, Chow C, Chan P, Cooper-Vince C, Wilson LAS. 2013. Psychosocial treatment efficacy for dis-
ruptive behavior problems in young children: a meta-analytic examination. J. Am. Acad. Child Adolesc.
Psychiatry 52(1):26–36
Conway CC, Forbes MK, Forbush KT, Fried EI, Hallquist MN, et al. 2019. A hierarchical taxonomy of
psychopathology can transform mental health research. Perspect. Psychol. Sci. 14(3):419–36
Cristea IA, Mogoase C, David D, Cuijpers P. 2015. Cognitive bias modification for mental health problems in
children and adolescents: a meta-analysis. J. Child Psychol. Psychiatry 56(7):723–34
Cuijpers P, Ciharova M, Quero S, Miguel C, Driessen E, et al. 2022. The contribution of “individual par-
ticipant data” meta-analyses of psychotherapies for depression to the development of personalized
treatments: a systematic review. J. Pers. Med. 12(1):93

Cuijpers P, Karyotaki E, Eckshtain D, Ng MY, Corteselli KA, et al. 2020. Psychotherapy for depression across
different age groups: a meta-analysis. JAMA Psychiatry 77(7):694–702
Cuijpers P, Karyotaki E, Reijnders M, Purgato M, Barbui C. 2018. Psychotherapies for depression in low- and
middle-income countries: a meta-analysis. World Psychiatry 17(1):90–101
Davison GC. 2019. A return to functional analysis, the search for mechanisms of change, and the nomothetic-
idiographic issue in psychosocial interventions. Clin. Psychol. Sci. 7(1):51–53
DeRubeis RJ, Cohen ZD, Forand NR, Fournier JC, Gelfand LA, Lorenzo-Luaces L. 2014. The personal-
ized advantage index: translating research on prediction into individualized treatment recommendations.
A demonstration. PLOS ONE 9:e83875
Doss DB. 2004. Changing the way we study change in psychotherapy. Clin. Psychol. Sci. Pract. 11:368–86
Ebert DD, Zarski AC, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M. 2015. Internet and computer-
based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized
controlled outcome trials. PLOS ONE 10(3):e0119895
Eckshtain D, Kuppens S, Ugueto A, Ng MY, Vaughn-Coaxum R. 2020. Meta-analysis: 13-year follow-up of
psychotherapy effects on youth depression. J. Am. Acad. Child Adolesc. Psychiatry 59(1):45–63
Egger M, Smith GD, Schneider M, Minder C. 1997. Bias in meta-analysis detected by a simple, graphical test.
BMJ 315:629–34
Evans SW, Owens JS, Wymbs BT, Ray AR. 2018. Evidence-based psychosocial treatments for children and
adolescents with attention deficit/hyperactivity disorder. J. Clin. Child Adolesc. Psychol. 47(2):157–98
Fabiano GA, Schatz NK, Aloe AM, Pelham WE, Smyth AC, et al. 2021. Comprehensive meta-analysis
of attention-deficit/hyperactivity disorder psychosocial treatments investigated within between group
studies. Rev. Educ. Res. 91(5):718–60
Fisher Z, Tipton E, Zhipeng H. 2017. robumeta: robust variance meta-regression (2.0). Software Package. https://
cran.R-project.org/package=robumeta
Fitzpatrick OM, Cho E, Venturo-Conerly KE, Ugueto AM, Ng MY, Weisz JR. 2023. Empirically supported
principles of change in youth psychotherapy: exploring codability, frequency of use, and meta-analytic
findings. Clin. Psychol. Sci. 11(2):326–44
Forehand R, Lafko N, Parent J, Burt KB. 2014. Is parenting the mediator of change in behavioral parent
training for externalizing problems of youth? Clin. Psychol. Rev. 34:606–19
Goldfried MR. 1980. Toward the delineation of therapeutic change principles. Am. Psychol. 35(11):991–99
Goldfried MR. 2009. Searching for therapy change principles: Are we there yet? Appl. Prev. Psychol. 13:32–34
Hailey D, Roine R, Ohinmaa A. 2008. The effectiveness of telemental health applications: a review. Can. J.
Psychiatry 53(11):769–78
Harpaz-Rotem I, Leslie D, Rosenheck RA. 2004. Treatment retention among children entering a new episode
of mental health care. Psychiatr. Serv. 55(9):1022–28
Hedges LV, Tipton E, Johnson MC. 2010. Robust variance estimation in metaregression with dependent effect
size estimates. Res. Synth. Methods 1:39–65
Hilty DM, Ferrer DC, Parish MB, Johnston B, Callahan EJ, Yellowlees PM. 2013. The effectiveness of
telemental health: a 2013 review. Telemed. e-Health 19(6):444–54
Hofmann SG, Hayes SC. 2019. The future of intervention science: process-based therapy. Clin. Psychol. Sci.
7(1):37–50
Huibers MJH, Lorenzo-Luaces L, Cuijpers P, Kazantzis N. 2021. On the road to personalized psychotherapy:
a research agenda based on cognitive behavior therapy for depression. Front. Psychiatry 11:607508
102 Weisz et al.

Jenson WR, Clark E, Kircher JC, Kristjansson SD. 2007. Statistical reform: evidence-based practice, meta-
analyses, and single subject designs. Psychol. Sch. 44(5):483–93
Jones PJ, Mair P, Kuppens S, Weisz JR. 2019. An upper limit to youth psychotherapy benefit? A meta-analytic
copula approach to psychotherapy outcomes. Clin. Psychol. Sci. 7(6):1434–49
Karyotaki E, Efthimiou O, Miguel C, Bermpohl FMG, Furukawa TA, et al. 2021. Internet-based cognitive
behavioral therapy for depression: a systematic review and individual patient data network meta-analysis.
JAMA Psychiatry 78(4):361–71
Kazdin AE. 2007. Mediators and mechanisms of change in psychotherapy research. Annu. Rev. Clin. Psychol.
3:1–27
Kazdin AE, Bass D, Ayers WA, Rodgers A. 1990. Empirical and clinical focus of child and adolescent
psychotherapy research. J. Consult. Clin. Psychol. 58:729–40
Kazdin AE, Blase SL. 2011. Rebooting psychotherapy research and practice to reduce the burden of mental
illness. Perspect. Psychol. Sci. 6(1):21–37
Leijten P, Gardner F, Melendez-Torres GJ, Knerr W, Overbeek G. 2018. Parenting behaviors that shape child
compliance: a multilevel meta-analysis. PLOS ONE 10:e0204929

Leijten P, Weisz JR, Gardner F. 2021. Research strategies to discern active psychological therapy components:
a scoping review. Clin. Psychol. Sci. 9(3):307–22
Luborsky L, Rosenthal R, Duguer L, Andrusyna TP, Berman J et al. 2002. The dodo bird verdict is alive and
well—mostly. Clin. Psychol. Sci. Pract. 9(1):2–12
Maggin DM, O’Keeffe BV, Johnson AH. 2011. A quantitative synthesis of methodology in the meta-analysis
of single-subject research for students with disabilities: 1985–2009. Exceptionality 19(2):109–35
Manassis K, Lee TC, Bennett K, Zhao XY, Mendlowitz S. 2014. Types of parental involvement in CBT with
anxious youth: a preliminary meta-analysis. J. Consult. Clin. Psychol. 82(6):1163–72
Marchette LK, Weisz JR. 2018. Empirical evolution of youth psychotherapy toward transdiagnostic
approaches. J. Child Psychol. Psychiatry 58(9):970–84
Marquez C, Johnson AM, Jassemi S, Park J, Moore JE, et al. 2018. Enhancing the uptake of systematic reviews
of effects: What is the best format for health care managers and policy makers? A mixed-methods study.
Implement. Sci. 13(1):84
McGraw KO, Wong SP. 1992. A common language effect size statistic. Psychol. Bull. 111:361–65
McKay MM, Bannon WM. 2004. Engaging families in child mental health services. Child Adolesc. Psychiatr.
Clin. N. Am. 13(4):905–21
McLeod BD, Weisz JR. 2004. Using dissertations to examine potential bias in child and adolescent clinical
trials. J. Consult. Clin. Psychol. 72:25–51
Miller FG, Lee DL. 2013. Do functional behavioral assessments improve intervention effectiveness for
students diagnosed with ADHD? A single-subject meta-analysis. J. Behav. Educ. 22(3):253–82
Miller LM, Southam-Gerow MA, Allin RB. 2008. Who stays in treatment? Child and family predictors of
youth client retention in a public mental health agency. Child Youth Care Forum 37(4):153–70
Ng MY, DiVasto KA, Cootner S, Gonzalez N, Weisz JR. 2020. What do 30 years of randomized trials tell us
about how psychotherapy improves youth depression? A systematic review of candidate mediators. Clin.
Psychol. Sci. Pract. https://doi.org/10.1111/cpsp.12367
Ng MY, Schleider JL, Horn RL, Weisz JR. 2021. Psychotherapy for children and adolescents: from efficacy
to effectiveness, scaling, and personalizing. In Bergin and Garfield’s Handbook of Psychotherapy and Behavior
Change, ed. M Barkham, W Lutz, L Castonguay, pp. 625–70. Hoboken, NJ: Wiley. 7th ed.
Ng MY, Weisz JR. 2016. Building a science of personalized intervention for youth mental health. J. Child
Psychol. Psychiatry 57(3):216–36
Oddli HW, Nissen-Lie HA, Halvorsen MS. 2016. Common therapeutic change principles as “sensitizing
concepts”: a key perspective in psychotherapy integration and clinical research. J. Psychother. Integr.
26(2):160–71
Pearson K. 1904. Report on certain enteric fever inoculation statistics. BMJ 3:1243–46
Pelham WE, Fabiano GA. 2008. Evidence-based psychosocial treatments for attention-deficit/hyperactivity
disorder. J. Clin. Child Adolesc. Psychol. 37(1):184–214
Pratt JG, Rhine JB, Smith BM, Stuart CE, Greenwood JA. 1940. Extra-Sensory Perception After Sixty Years: A
Critical Appraisal of the Research in Extra-Sensory Perception. New York: Holt

Price M, McKetta S, Weisz JR, Ford J, Lattanner MR, et al. 2021. Cultural sexism moderates efficacy of
psychological therapy for girls: results from a spatial meta-analysis. Clin. Psychol. Sci. Pract. 28(3):299–312
Price M, Weisz JR, McKetta S, Lattanner MR, Reid AE, et al. 2022. Meta-analysis: Are psychotherapies less
effective for Black youth in communities with higher levels of anti-Black racism? J. Am. Acad. Child
Rosen GM, Davison GC. 2003. Psychology should list empirically supported principles of change (ESPs) and
not credential trademarked therapies or other treatment packages. Behav. Modif. 27(3):300–12
Rosenberger RS, Stanley TD. 2009. Publication selection of recreation demand price elasticities: a meta-analysis.
Work. Pap., Oregon State Univ./Hendrix Coll.
Rouse B, Chaimani A, Li T. 2017. Network meta-analysis: an introduction for clinicians. Intern. Emerg. Med.
12(1):103–11
Schleider JL, Mullarkey MC, Fox KR, Dobias ML, Shroff A, et al. 2021. A randomized trial of online single-
session interventions for adolescent depression during COVID-19. Nat. Hum. Behav. 6:258–68
Schleider JL, Weisz JR. 2016. Reducing risk for anxiety and depression in adolescents: effects of a single-session
intervention teaching that personality can change. Behav. Res. Ther. 87:170–81
Schleider JL, Weisz JR. 2017. Little treatments, promising effects: meta-analysis of single-session interventions
for youth psychiatric problems. J. Am. Acad. Child Adolesc. Psychiatry 56(2):107–15
Schleider JL, Weisz JR. 2018. A single-session growth mindset intervention for adolescent anxiety and
depression: nine-month outcomes of a randomized trial. J. Child Psychol. Psychiatry 59(2):160–70
Schulz KF, Altman DG, Moher D, CONSORT Group. 2010. CONSORT 2010 statement: updated guidelines
for reporting parallel group randomised trials. BMJ 8:18
Smith ML, Glass GV. 1977. Meta-analysis of psychotherapy outcome studies. Am. Psychol. 32(9):752–60
Smith T, Iadarola S. 2015. Evidence base update for autism spectrum disorder. J. Clin. Child Adolesc. Psychol.
44(6):897–922
Smith V, Devane D, Begley CM, Clarke M. 2011. Methodology in conducting a systematic review of systematic
reviews of healthcare interventions. BMC Med. Res. Methodol. 11:15
Tierney JF, Vale C, Riley R, Smith CT, Stewart L, et al. 2015. Individual participant data (IPD) metaanalyses
of randomised controlled trials: guidance on their use. PLOS Med. 12(7):e1001855
Tolin DF, Mckay D, Forman EM, Klonsky ED, Thombs BD. 2015. Empirically supported treatment:
recommendations for a new model. Clin. Psychol. Sci. Pract. 22(4):317–38
Van den Noortgate W, Onghena P. 2008. A multilevel meta-analysis of single-subject experimental design
studies. Evid.-Based Commun. Assess. Interv. 2(3):142–51
Venturo-Conerly KE, Eisenman D, Wasil A, Singla D, Weisz JR. 2023. Effectiveness of youth psychotherapy
interventions in low- and middle-income countries (LMICs): a meta-analysis. J. Am. Acad. Child Adolesc.
Psychiatry. In press
Venturo-Conerly KE, Fitzpatrick OM, Horn RL, Ugueto AM, Weisz JR. 2022. Effectiveness of youth
psychotherapy delivered remotely: a meta-analysis. Am. Psychol. 77(1):71–84
Viswanathan M, Kennedy SM, McKeeman J, Christian R, Coker-Schwimmer M, et al. 2020. Treatment of
depression in children and adolescents: a systematic review. Comp. Effect. Rev. 224, Agency Healthc. Res.
Qual., Rockville, MD
Ward MA, Theule J, Cheung K. 2016. Parent–child interaction therapy for child disruptive behaviour
disorders: a meta-analysis. Child Youth Care Forum 45:675–90
Weine S, Marques AH, Singh M, Pringle B. 2020. Global child mental health research: time for the children.
J. Am. Acad. Child Adolesc. Psychiatry 59(11):1208–11
Weisz JR, Bearman SK. 2020. Principle-Guided Psychotherapy for Children and Adolescents: The FIRST Treatment
Program for Behavioral and Emotional Problems. New York: Guilford
Weisz JR, Fitzpatrick OM, Venturo-Conerly K, Cho E. 2021. Process-based and principle-guided approaches
in youth psychotherapy. World Psychiatry 20(3):378–80
Weisz JR, Kazdin AE. 2017. Evidence-Based Psychotherapies for Children and Adolescents. New York: Guilford.
3rd ed.
Weisz JR, Krumholz LS, Santucci L, Thomassin K, Ng MY. 2015. Shrinking the gap between research and
practice: tailoring and testing youth psychotherapies in clinical care contexts. Annu. Rev. Clin. Psychol.
11:139–63
104 Weisz et al.

Weisz JR, Kuppens S, Ng MY, Eckshtain D, Ugueto AM, et al. 2017. What five decades of research tells us
about the effects of youth psychological therapy: a multilevel meta-analysis and implications for science
and practice. Am. Psychol. 72(2):79–117
Weisz JR, Kuppens S, Ng MY, Vaughn-Coaxum RA, Ugueto AM, et al. 2019. Are psychotherapies for young
people growing stronger? Tracking trends over time for youth anxiety, depression, ADHD, and conduct
problems. Perspect. Psychol. Sci. 14(2):216–37
Weisz JR, McCarty CA, Valeri SM. 2006. Effects of psychotherapy for depression in children and adolescents:
a meta-analysis. Psychol. Bull. 132:132–49
Weisz JR, Ugueto AM, Cheron DM, Herren J. 2013. Evidence-based youth psychotherapy in the mental
health ecosystem. J. Clin. Child Adolesc. Psychol. 42(2):274–86

Weisz JR, Weiss B, Alicke MD, Klotz ML. 1987. Effectiveness of psychotherapy with children and adolescents:
a meta-analysis for clinicians. J. Consult. Clin. Psychol. 55:542–49
Weisz JR, Weiss B, Han S, Granger DA, Morton T. 1995. Effects of psychotherapy with children and
adolescents revisited: a meta-analysis of treatment outcome studies. Psychol. Bull. 117:450–68
White CM, Sanders-Schmidler GD, Borsky A, Butler M, Wang Z, et al. 2017. Understanding health-systems’ use
of and need for evidence to inform decision-making. Res. White Pap., Univ. Conn./Duke Evid.-Based Pract.
Cent.
Whiteside SPH, Sim LA, Morrow AS, Farah WH, Hilliker DR, et al. 2020. A meta-analysis to guide the
enhancement of CBT for childhood anxiety: exposure over anxiety management. Clin. Child Fam. Psychol.
Rev. 23:102–21
WHO (World Health Organ.). 2018. Mental Health Atlas 2017. Geneva: WHO. https://www.who.int/
publications-detail-redirect/9789241514019
Young JF, Jones JD, Gallop R, Benas JS, Schueler CM, et al. 2021. Personalized depression prevention: a
randomized controlled trial to optimize effects through risk-informed personalization. J. Am. Acad. Child
Zhou X, Hetrick SE, Barth J, Qin B, Whittington CJ, et al. 2015. Comparative efficacy and acceptability of
psychotherapies for depression in children and adolescents: systematic review and network meta-analysis.
World Psychiatry 14(2):207–22
Zhou X, Zhang Y, Furukawa TA, Cuijpers P, Pu J, et al. 2019. Different types and acceptability of psychother-
apies for acute anxiety disorders in children and adolescents: a network meta-analysis. JAMA Psychiatry
76(1):41–50

CP19_TOC ARjats.cls March 25, 2023 15:49
Annual Review of
Clinical Psychology
Volume 19, 2023
Contents
A Clinical Psychologist Who Studies Alcohol

Kenneth J. Sher p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 1
Community Mental Health Services for American Indians
and Alaska Natives: Reconciling Evidence-Based Practice
and Alter-Native Psy-ence
Joseph P. Gone p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p23
Culturally Responsive Cognitive Behavioral Therapy for Ethnically
Diverse Populations
Stanley J. Huey Jr., Alayna L. Park, Chardée A. Galán, and Crystal X. Wang p p p p p p p p p p p51
What Four Decades of Meta-Analysis Have Taught Us About Youth
Psychotherapy and the Science of Research Synthesis
John R. Weisz, Katherine E. Venturo-Conerly, Olivia M. Fitzpatrick,
Jennifer A. Frederick, and Mei Yi Ng p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p79
Evaluation of Pressing Issues in Ecological Momentary Assessment
Arthur A. Stone, Stefan Schneider, and Joshua M. Smyth p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 107
Machine Learning and the Digital Measurement of
Psychological Health
Isaac R. Galatzer-Levy and Jukka-Pekka Onnela p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 133
The Questionable Practice of Partialing to Refine Scores on and
Inferences About Measures of Psychological Constructs
Rick H. Hoyle, Donald R. Lynam, Joshua D. Miller, and Jolynn Pek p p p p p p p p p p p p p p p p p p p p 155
Eating Disorders in Boys and Men
Tiffany A. Brown and Pamela K. Keel p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 177
Mental Health of Transgender and Gender Diverse Youth
Natalie M. Wittlin, Laura E. Kuper, and Kristina R. Olson p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 207
Behavioral Interventions for Children and Adults with Tic Disorder
Douglas W. Woods, Michael B. Himle, Jordan T. Stiede, and Brandon X. Pitts p p p p p p p p p 233
CP19_TOC ARjats.cls March 25, 2023 15:49
The Garrett Lee Smith Memorial Act: A Description and Review

of the Suicide Prevention Initiative
David B. Goldston and Christine Walrath p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 261
Racism and Social Determinants of Psychosis
Deidre M. Anglin p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 277
Developmental Consequences of Intimate Partner Violence
on Children
G. Anne Bogat, Alytia A. Levendosky, and Kara Cochran p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 303

Psychoneuroimmunology: An Introduction to Immune-to-Brain
Communication and Its Implications for Clinical Psychology
Julienne E. Bower and Kate R. Kuhlman p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 331
Racial, Ethnic, and Cultural Resilience Factors in African American

Youth Mental Health
Enrique W. Neblett Jr. p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 361
Acculturation and Psychopathology
Gail M. Ferguson, José M. Causadias, and Tori S. Simenec p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 381
Posttraumatic Stress Disorder in Refugees
Richard A. Bryant, Angela Nickerson, Naser Morina, and Belinda Liddell p p p p p p p p p p p p p p 413
Risk and Resilience Among Children with Incarcerated Parents:
A Review and Critical Reframing
Elizabeth I. Johnson and Joyce A. Arditti p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 437
Supernatural Attributions: Seeing God, the Devil, Demons, Spirits,
Fate, and Karma as Causes of Events
Julie J. Exline and Joshua A. Wilt p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 461
Errata
An online log of corrections to Annual Review of Clinical Psychology articles may be

found at http://www.annualreviews.org/errata/clinpsy

Weisz Et Al 2023 What Four Decades of Meta Analysis Have Taught Us About Youth Psychotherapy and The Science of

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Weisz Et Al 2023 What Four Decades of Meta Analysis Have Taught Us About Youth Psychotherapy and The Science of

Uploaded by

Copyright:

Available Formats

Annual Review of Clinical Psychology

What Four Decades of

Meta-Analysis Have Taught Us

Annu. Rev. Clin. Psychol. 2023. 19:79–105 Keywords

and Typically Rules Out Causal Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

5.6. Using Meta-Analysis to Guide Treatment Personalization . . . . . . . . . . . . . . . . . . . 98

1.2. Historical Roots of Meta-Analysis and Its Eventual Application

www.annualreviews.org • Youth Psychotherapy Meta-Analyses 81

2.1. Any Summary Based on Heterogeneous Studies Can Provide Only

2.2. Confounding Among Candidate Moderators Makes Conclusions Tentative

2.3. Decisions Related to Lumping Versus Splitting Treatment, Control,

2.4. Publication Bias May Inflate Estimates of Treatment Benefit

2.6. Search Strategies, Databases Selected, and Inclusion/Exclusion Criteria

www.annualreviews.org • Youth Psychotherapy Meta-Analyses 83

2.7. A Critical Question: What Was the Control Group?

2.8. Effect Sizes from Multiple Measures or Group Comparisons Within

3.1. Meta-Analysis of Single-Subject Experimental Design Studies

3.2. Network Meta-Analysis

www.annualreviews.org • Youth Psychotherapy Meta-Analyses 85

3.3. Individual Patient Data Meta-Analysis

Mean effect size

Anxiety Conduct ADHD Depression Multiple

4.2. Dodo Bird Versus Different Effects for Different Treatments

4.3. Are There Developmental/Age Differences in the Benefits of Therapy?

Children 15 0.35 (0.15–0.55)

Adolescents 28 0.55 (0.34–0.75)

Young adults 27 0.98 (0.79–1.16)

Middle-aged adults 304 0.77 (0.67–0.87)

Older adults 69 0.66 (0.51–0.82)

Older old 10 0.97 (0.42–1.52)

All studies 453 0.75 (0.67–0.82)

0.0 0.5 1.0 1.5

4.4. How Fast Can Youth Psychotherapy Work?

www.annualreviews.org • Youth Psychotherapy Meta-Analyses 89

4.5. How Effective Is Youth Psychotherapy When Delivered Remotely,

4.6. Can Youth Psychotherapies (Most of Which Are from High-Income

www.annualreviews.org • Youth Psychotherapy Meta-Analyses 91

Effect size (g)

4.8. Psychotherapy Effects and Principles of Change

“positive opposites” of undesired behaviors. Fitzpatrick et al. synthesized 348 treatment–control

4.9. Searching for Psychotherapy’s Holy Grail: Can Meta-Analysis Reveal

A valuable function of meta-analyses—especially when they span long periods of time—can be

5. CHALLENGES AND OPPORTUNITIES FOR THE DAYS AHEAD

5.1. Tackling the Tower of Babel Problem

c ADHD d Conduct problems

www.annualreviews.org • Youth Psychotherapy Meta-Analyses 95

Abbreviation: RCT, randomized controlled trial.

5.2. Evaluating Hot Trends in Therapy Design and Implementation

5.4. Collaborating with Stakeholders to Make Meta-Analysis Relevant

www.annualreviews.org • Youth Psychotherapy Meta-Analyses 97

Respondents have reported preferences for information on cost, implementation, comparison of

5.5. Finding Funding for High-Quality, Timely Meta-Analyses

5.6. Using Meta-Analysis to Guide Treatment Personalization

www.annualreviews.org • Youth Psychotherapy Meta-Analyses 99

100 Weisz et al.

www.annualreviews.org • Youth Psychotherapy Meta-Analyses 101

treatments: a systematic review. J. Pers. Med. 12(1):93

102 Weisz et al.

compliance: a multilevel meta-analysis. PLOS ONE 10:e0204929

www.annualreviews.org • Youth Psychotherapy Meta-Analyses 103

104 Weisz et al.

health ecosystem. J. Clin. Child Adolesc. Psychol. 42(2):274–86

www.annualreviews.org • Youth Psychotherapy Meta-Analyses 105