Choosing Effective Youth-Focused Prevention Strategies: A Practical Guide for Applied

Family Professionals
Author(s): Jonathan R. Olson
Source: Family Relations, Vol. 59, No. 2 (April 2010), pp. 207-220
Published by: National Council on Family Relations
Stable URL: https://www.jstor.org/stable/40663954
Accessed: 27-01-2020 08:06 UTC

REFERENCES
Linked references are available on JSTOR for this article:
https://www.jstor.org/stable/40663954?seq=1&cid=pdf-reference#references_tab_contents
You may need to log in to JSTOR to access the linked references.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

National Council on Family Relations is collaborating with JSTOR to digitize, preserve and
extend access to Family Relations

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
 Jonathan R. Olson The Pennsylvania State University-Altoona

Choosing Effective Youth-Focused Prevention

Strategies: A Practical Guide for Applied Family
Professionals

Advances in the field of prevention science and well-evaluated programs have been pack-
have resulted in strategies that target various aged for widespread dissemination (Catalano,
youth outcomes. In recent years, numerous Berglund, Ryan, Lonczak, & Hawkins, 2004;
' 'best practices ' ' lists have been developed to Hawkins, Catalano, & Arthur, 2002; Wanders-
help professionals identify such strategies. Someman & Florin, 2003). Furthermore, a variety
scholars have questioned the quality of these listsof "best practices" lists have been developed
and cautioned that there are flaws in evaluationsto help applied professionals identify effective
of many prevention strategies. The purpose of strategies to implement in their own schools,
this paper is to review these criticisms and neighborhoods, towns, and cities (Gandhi,
provide applied family scholars with suggestionsMurphy-Graham, Petrosino, Chrismer, & Weiss,
regarding how to identify effective strategies. 2007; Gorman & Conde, 2007; Small, 2005).
Despite the positive trends mentioned above,
several scholars have recently cautioned that the
Over the past several decades, applied familyfield of prevention science still has much work
professionals have been increasingly involved to do. Specifically, scholars have questioned
in developing, implementing, and evaluatingthe quality of some outcome evaluations of
strategies aimed at preventing a variety ofpopular prevention strategies and have indicated
negative outcomes among youth (Bogenschnei- that many suffer from methodological flaws
der, 2006; Kumpfer, Alvarado, & Whiteside, (Gorman, 1998; Gorman & Conde, 2007;
2003; Spoth, Kavanagh, & Dishion, 2002). Wandersman & Florin, 2003; Weiss, Murphy-
Since the 1980s, significant advances have Graham, Petrosino, & Gandhi, 2008). In
been made in the field of prevention science addition, there is a growing concern that
with the goal of creating high-quality initia- although the popular "best practices" lists
tives aimed at preventing outcomes such as typically include examples of strategies that
delinquency, substance use, and school failure have demonstrated effectiveness in high-quality
among youth. Indeed, numerous well-designed evaluations, they also tend to include some
strategies that have not been properly evaluated,
are not ready for widespread dissemination,
and/or fail to demonstrate long-term effects on
The Pennsylvania State University - Altoona, 128 Hawthorn
youth outcomes (Gandhi et al., 2007; Gorman,
Building, 3000 Ivyside Park, Altoona, PA 16601 2002; Hallfors, Pankratz, & Hartman, 2007;
(jrol0@psu.edu). Wandersman & Florin; Weiss et al.). As a
Key Words: best practices, evidence-based practice, result, users of such lists must be cautious when
prevention. selecting strategies for their own communities as

Family Relations 59 (April 2010): 207-220 207

DOI:10.1 1 1 1/J.1741-3729.2010.00596.X

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
208 Family Relations

it may be difficult to assess their overall quality History of the Field of Prevention
from the limited amount of information included Science
in the lists.
During the 1960s, programs and policies
In light of the above-mentioned criticisms, designed to address youth outcomes such as
the primary purposes of this paper are to substance use and delinquency began to appear
highlight current concerns related to popular in the United States. Such initiatives ini-
"best practices" lists and to propose specific tially involved responding to existing crises
criteria that family professionals can use to and tended to be treatment-oriented in their
help guide the selection of effective prevention approach. For instance, residential programs
strategies. In addition, this discussion is meant to were designed to promote "character develop-
stimulate a more general dialogue among applied ment" among delinquent youth, and social poli-
family scholars regarding how we can best cies were enacted to help substance using youth
receive services rather than entering the justice
identify and disseminate high-quality strategies.
system (Catalano, Hawkins, Berglund, Pollard,
As can be seen in the following sections of
& Arthur, 2002; Jenson & Fraser, 2006). In the
this paper, family professionals may be in a
years that followed, such approaches gradually
unique position that can help move the field gave way to more prevention-oriented strate-
of prevention science forward. Indeed, many gies. By the 1980s, common prevention efforts
of the most promising prevention strategies focused largely on public service announce-
either directly or indirectly target family-level ments designed to raise awareness of issues
variables. As family professionals, our expertise and to educate the public about how and why
and understanding of how these variables to avoid risky behaviors. Examples of such ini-
interact to impact developmental outcomes of tiatives included the "Just Say No" campaign,
individual family members can help facilitate the "brain on drugs" commercials, and car-
the identification, and ultimately dissemination, toons featuring characters such as McGruff the
of effective strategies. Crime Dog. Such early approaches to preven-
This paper begins with a brief overview of the tion tended to rely on simple strategies aimed at
history of the field of prevention science and dis-
demonstrating consequences of targeted behav-
iors and often were based on instilling concern
cusses some of the progress as well as problems
and fear among those tempted to engage in such
that have developed within the field. Next is a
behaviors (Jenson & Fraser).
discussion of six areas of concern that are asso-
By the mid-1980s, more comprehensive
ciated with current practices of evaluating and
school and community-based strategies began
disseminating prevention strategies. After each to appear (Jenson, 2006). These strategies,
area of concern are various implications for fam-including popular programs such as D.A.R.E.,
ily scholars and practitioners that can help pushtended to target a single problem behavior,
the field toward improved identification and dis-although they slowly evolved to become
semination of high-quality prevention strategies.more sophisticated over the next several
Throughout this article, prevention strategies are decades. Despite the popularity of these early
referred to at a general level. In practice, current initiatives, outcome evaluations suggested that
prevention strategies encompass a wide vari-they had limited effectiveness at best (Catalano
ety of approaches, which commonly includeet al., 2002; Jenson; Weiss et al., 2008). One
packaged programs, governmental policies, andsignificant positive effect of early prevention
various educational initiatives. The examplesefforts, however, is that they served as a starting
point for the evolution of the field of prevention
provided in this paper focus primarily on pack-
science. Indeed, since the 1980s, efforts to design
aged programs that are included on popular
and evaluate prevention strategies have become
"best practices" lists, although the basic con-
increasingly sophisticated, and the number of
cepts contained within can be applied to policiesprevention strategies has increased dramatically
and educational initiatives as well. Note that
(Catalano et al.; Greenberg, 2004; Hawkins
the examples used in this paper are for illus-
et al., 2002; Wandersman & Florin, 2003).
trative purposes only - they are not intended as professionals from a wide variety of
Today,
an endorsement of any particular program disciplines
over are active in the field of prevention
another. science, and a variety of stakeholders have a

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
Choosing Effective Prevention Strategies 209

vested interest in the outcomes of prevention family members. For example, the Functional
strategies. Family Therapy (FFT) program is a short-term
Prevention scholars typically classify current indicated family therapy program intended to
prevention strategies by the level of risk within reduce future offences among youth who are
the population they target. Universal prevention currently in the juvenile justice system. Evalua-
strategies are the most general, as they are tions of this program have demonstrated that
intended for everyone, regardless of their level it not only reduces recidivism rates among
of risk. Such strategies tend to be rather targeted youth but also prevents their sib-
broad in nature and are commonly implemented lings from engaging in similar delinquent acts
within schools or community settings. An (Alexander, Pugh, Parsons, & Sexton, 2000;
example of a current universal strategy is Sexton & Alexander, 2000).
the Guiding Good Choices Program, which In recent years, prevention science schol-
is training program for parents of school-age ars have noted that increasing numbers of
children. Program activities are designed to teach prevention strategies, such as those described
parents how to set clear family rules related to above, have been implemented with fidelity, and
substance use and to encourage healthy family have been subject to rigorous evaluations that
processes that empower all family members. have demonstrated positive effects on partici-
Evaluations of this program have indicated that pants (Catalano et al., 2004; Greenberg, 2004;
participants are less likely than members of a Hawkins et al., 2002; Wandersman & Florin,
comparison group to use substances or engage 2003). In light of these advances, government
in delinquent behaviors (Mason, Kosterman, agencies have begun to take notice of the
Hawkins, Haggerty, & Spoth, 2003; Spoth, growing body of evidence in support of the effec-
Redmond, Shin, & Azevedo, 2004). tiveness of many prevention strategies. Since the
Selective prevention strategies specifically 1990s, a variety of "best practices" lists have
target participants who are at elevated risk for appeared. Strategies that are included on such
engaging in problem behaviors. For example, lists typically have demonstrated positive effects
the Nurse-Family Partnership program was on common youth problems through outcome
designed to help low-income pregnant women evaluations. In recent years, much of the fed-
experience healthy pregnancy outcomes and to eral grant money that has been made available
develop positive parenting skills. Trained nurses to support prevention programming has been
visit participants' homes throughout pregnancy tied to selecting strategies that appear on one or
and during the first 2 years of their children's more of these lists (Gandhi et al., 2007; Hallfors
lives. The nurses encourage the women to et al., 2007; Weiss et al., 2008). Table 1 includes
receive proper prenatal care, teach them about information about several of the most popular
child development, and demonstrate how to "best practices" lists. As can be seen, there is
effectively parent their children in positive ways. considerable overlap among them in terms of
Outcome evaluations have demonstrated a wide criteria for inclusion and information provided,
variety of positive effects for both mothers andalthough each also has some unique character-
their children, some of which have persisted for istics. As such, they largely complement each
at least 15 years (Olds, 1997). other as resources.
The most intensive prevention strategies typ- In light of the above discussion, there seems to
ically are referred to as indicated strategies.be significant enthusiasm among scholars in the
These strategies target youth who not only prevention science field regarding how far we
experience multiple risk factors for particular have come in such a short time frame. Although
outcomes but also have begun to experiment most acknowledge that there is still room for
with certain behaviors such as delinquency and improvement, various scholars have noted that
substance use. The line between indicated pre- in recent years, increasing numbers of prevention
vention strategies and treatment is blurry. In strategies have been based on a strong theoretical
fact, some scholars do not make a distinction and empirical foundation (Catalano et al., 2002;
between the two types of intervention (Chan- Flay et al., 2005; Greenberg, 2004). In addition,
ning Bete Company, 2004). Regardless, manythere is mounting evidence that current strategies
indicated strategies also have preventive effects, are at least somewhat effective in preventing
as they may prevent associated problems from problem behaviors, and government officials
developing in both targeted youth and otherhave tied funding to selecting strategies from

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
 2 1 0 Family Relations

•Ills ss :ö s 1 ¡
* £ "g =» | "1 .a 'S f s •= I §• g"
^1«
•I ! ■§*! s"g
* I =» I »
s | ■§ ¡Iing
s r
.a||slg »
o rf 11. s

'»HÜ fill lHflill

* .5 j £ *zw já 1

î II lililí |, IMifh II 1 1
üsü? {trill »11 lilîtflf jîi
3 |
_ " ^ || Ä fi fil g -g ¿I
!!!! il i, î«l !J II i î
¿3
•S

ì alii î| l| fili Si 1 | II si
I 3n" lî II itîï 11 t Ml l Ì
I
1 11 ë eî ÌÌ lì î i
! 111 fi 1 ¿I |C 11 ï i s
ut n i iiiiiSfì 1 1 ii i
's-
.8

'S
Ì
il 8 l-a -a ïlîillSi ? ff lì B 11
1 Ia! S"ft>l'6 I I 2 B |1
5
CD

1 ^ ì| ili 1 1 M % 111 î 1
tit ff ìli 1SL íí Í,, IH I f
l!1 if î« Mil i ¡i lî ! i
^ !> -a 1 S i I ! I I * -s 1 1 t s
t i i | i I -i i 1 2 1 £ 1 1 11-8 Ê g
lüiííniuii sii ¡n m i ¡
|a|Sa--g<s^El«2^Ë. I s o¿¡z;pu
e g.Js | a 1 .2 |
c^hS u cu ^s »^ o

1 <2.2«u sil ¿1
2- il 'i I« Il, foi 1. 1 f"! i §3
il il ¡ili rii muí! ip
HCtfO X & £ £ Cu

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
Choosing Effective Prevention Strategies 211

"best practices lists," suggesting that they might vary depending on the purposes of the
are actively considering the results of research evaluation, the nature of the strategy being
(Flay et al.; Greenberg; Kellam & Lange vin, evaluated, and the degree to which the strategy
2003; Weiss et al., 2008). Such connections will be implemented in unique populations
between research and practice are no doubt (McCall & Green, 2004; Nathan, 2008).
exciting to family scholars who have an In an excellent review of these issues,
interest in strengthening ties among researchers, McCall and Green (2004) suggested that while
policymakers, and family practitioners (e.g., well-designed randomized experiments make a
Bogenschneider, Olson, Linney, & Mills, 2000; positive contribution to our knowledge about
Myers- Walls, 2000; Small, 1995, 2005). the effectiveness of specific interventions, their
Despite these positive trends, however, some status as a "gold standard" may be overstated.
have begun to question if we are celebrating a More specifically, the authors argued that while
bit too much, a bit too soon. During the past randomized experiments have the potential to
several years, a handful of scholars have stepped increase the internal validity of a particular
back to critically examine the current state of study, they run the risk of decreasing the external
the field of prevention science (e.g., Gandhi validity. That is, although we might be confident
et al., 2007; Gorman, 2008; Gorman & Conde, that the results are due to the strategy, we would
2007; Hallfors et al., 2007; Sherman, 2003; be less confident that similar results can be
Weiss et al., 2008). Although most of these replicated within "real-world" conditions in
scholars agree that examples of high-quality, which participants are not assigned to groups,
effective prevention strategies currently exist, but rather choose whether to participate or not.
McCall and Green further argued that even
there have been rather consistent complaints
that too many flawed, under-evaluated, and randomized designs include a variety of threats
sometimes ineffective strategies appear on many to internal validity such as differential attrition,
of the popular "best practices" lists. As a result, diffusion of treatment effects to the control

some scholars have directly criticized specific group, and an inability to find a "no treatment"
characteristics of many current "best practices" control group, among others. In sum, the authors
lists. In the following sections of this paper, I argued that although randomized designs are
review six basic concerns that are associated a valuable tool and have taught us much
with various aspects of these lists. Three of the about program effectiveness, they should not
concerns are related to evaluations that are used be considered the only high-quality evaluation
to place strategies on the lists, two are related design.
to characteristics of the strategies appearing on Indeed, other scholars have argued that
the lists, and one is related to issues associated various evaluation designs may be appropriate
with the lists themselves. After commenting when assessing the quality of prevention
on how each of the six concerns impacts our strategies (Cook, Shadish, & Wong, 2008;
attempts to disseminate high-quality prevention Nathan, 2008). For example, a variety of quasi-
programming, I draw out specific implications experimental designs attempt to approximate
for family professionals interested in identifying the effects of random assignment when such
effective prevention strategies. an assignment is not possible. In such designs,
researchers attempt to statistically control for
potential differences across groups. Recent
Areas of Concern advances in statistical methods have resulted
in a wide variety of sophisticated techniques
Concern #/: Limited View of What Constitutes
that are intended to minimize threats to both
a High-Quality Evaluation
internal and external validity. For example,
propensity
Historically, many social scientists have agreed analyses have been introduced
that random experimental designs are as a way
the goldof controlling for factors that
standard for assessing the effectiveness ofselection into experimental versus
influence
social policies and programs (Flay et al.,
control
2005;groups. Hierarchical modeling designs
Moskowitz, 1993; G. C. Patton, 2008; are M.intended
Q. to control biases and systematic
Patton, 2008; Rossi & Freeman, 1993). In similarities that result from the clustering of
recent years, however, some scholars have begun groups of individuals (such as what happens
to suggest that the optimal evaluation design in classrooms, teams, or clubs). Regression

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
212 Family Relations

discontinuity designs enable researchers to experimental designs. As noted in Table 1 , some

control various confounding variables while lists are restricted to only strategies that have
assigning individuals who need treatment to the been subjected to such evaluations. In light of the
treatment groups (Cook et al.; Nathan). preceding discussion, this practice may be short-
Other scholars have argued that there is a need sighted and overlook some high-quality inter-
for alternatives to both experimental and quasi- ventions in which random assignment would
experimental methodologies when evaluating be unethical or otherwise impossible. Given the
the effectiveness of social interventions. For strengths and limitations associated with each
example, Shaw (2003) argued that various of the above-mentioned designs, it may be best
qualitative designs are particularly good at to choose strategies on the basis of their per-
uncovering unintended program effects and
formance on multiple designs (Gilgun, 2006).
individualized outcomes that may not generalize Although Shaw (2003) cautioned that this may
to the entire population. He suggested that lead to conflicting data and lack of consensus,
qualitative designs are well poised to help us such conflict can help guide future research with
understand the "micro-processes," or more the goal of gaining a deeper understanding of
subtle aspects of particular interventions thatfactors that impact program effectiveness. As a
result in particular participant outcomes. In rich trail of data related to a particular strategy
addition, qualitative methods enable evaluators begins to accumulate, we can have much more
to gain a much deeper understanding of confidence regarding if and why a strategy is
contextual factors that influence outcomes, such effective.
as culture, values, and personal meanings that If such an accumulation of evidence is not
participants attribute to their experiences. Gilgun available for a particular strategy, the best way
(2006) echoed many of these ideas as she to assess the effectiveness of a particular inter-
suggests that the rich contextual background vention may be to consider the degree to which
uncovered by well-designed qualitative studies existing evaluation designs are appropriate for
serves as an invaluable resource upon which their intended use. As outlined in the preceding
successful interventions can be tailored to meet discussion, a wide variety of methodologies are
the needs of individual participants. available. In some cases, a randomized exper-
Another general area of evaluation research imental design may be the preferred method
that has relied on both quantitative and quali- to gauge program effectiveness, particularly if
tative designs is process evaluation. In process the user is interested in making causal con-
evaluations, researchers typically are interested clusions. In cases when randomization is not
in gathering information about program imple- possible, or not ethical, there are a variety
mentation (M. Q. Patton, 2008). Such data of quasi-experimental designs such as regres-
can complement outcome evaluations by help- sion discontinuity designs that approximate the
ing evaluators determine not only if a strategy effects of true experiments (Cook et al., 2008;
worked, but also why it succeeded or failed. For Nathan, 2008). Alternately, evaluation users
example, process data may reveal that a strategy may be interested in implementation issues or
was not implemented fully because participants want rich contextual data, in which case pro-
did not enjoy a certain part of the curriculum. cess evaluations or qualitative designs may be
Such data can help developers revise the strategy appropriate.
in order to be more effective in the future. Given Most of the popular "best practices" lists
the strong contextual components of qualitative can serve as a starting point for potential users
research, such designs lend themselves well to of evaluations. Indeed, as noted in Table 1,
process evaluation (Gilgun, 2006; Shaw, 2003). most lists include references to evaluations of
In practice, however, a variety of methodologies each strategy. Such reference lists, however, are
can furnish such data (M. Q. Patton; Rossi & not comprehensive generally, and only some
Freeman, 1993). include unpublished studies. In addition, some
lists, such as the Coalition for Evidence-Based
Implications related to Concern #7 - Consider Policy list, reference only randomized trials. As
multiple evaluation designs. Despite the avail- such, family professionals interested in objec-
ability of the wide variety of methodologies tively assessing the accumulated evidence of
described above, most of the current "best prac- effectiveness for a strategy would be wise to con-
tices" lists are heavily biased in favor of true duct an independent literature review, including

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
Choosing Effective Prevention Strategies 213

library research and contacting program devel- adolescents in general, and differential subject
opers to seek out unpublished work. Accessing attrition can result in differences across compari-
unpublished studies is particularly important, as son groups (Flay et al., 2005; Moskowitz, 1993).
many professional journals are more focused on A second common problem associated with all
publishing evaluations that have demonstrated types of quantitative designs is that evaluators
program effectiveness than those that have not. sometimes conduct an exceedingly large number
As a result, reviews of published literature may of group comparisons without correcting for the
be biased such that prevention strategies appear increased likelihood of a Type I error (Gandhi
more effective than they actually are. In practice, et al., 2007; Gorman, 2008; Moskowitz). In such
program developers seem to be very receptive cases, "positive" results could be because of
to requests for research reports, as I found when the number of statistical analyses rather than the
working with a team of researchers on a com- effects of the program (commonly referred to as
prehensive review of more than 300 prevention family-wise error). Specifically, when evaluators
strategies. Indeed, most developers were happy use a common p < .05 criterion for statistical
to provide published and unpublished studies significance, we would expect 1 in 20 analyses to
that employed a wide variety of research designs be significant as a result of sampling error alone.
(Channing Bete Company, 2004). In some cases, evaluators conduct far more than
Conducting a comprehensive literature review 20 cross-group comparisons without adjusting
such as the one described above does take time. their criterion for significance. As such, some
As such, family professionals who are evaluating significant findings may simply be an artifact of
the evidence supporting a strategy's effective- the large number of statistical comparisons in
ness will need to weigh the benefits against the study (Weiss et al., 2008). A similar prob-
potential time constraints. If a particular strat- lem results when researchers report statistically
egy looks like a good fit for a community, but the significant findings that are so small, they are
"best practices" lists provide references to only clinically meaningless. This is most common in
a limited number of evaluations, such a com- studies with large samples, in which even very
prehensive review may be worth the investment. small effects reach conventional levels of sta-
If a preferred strategy already has substantial tistical significance (McCartney & Rosenthal,
documentation of effectiveness across multiple 2000; Small, 2005).
types of research designs, then such a review In addition to the above-mentioned problems
may be unnecessary. with internal validity, many studies are designed
such that their results cannot be readily
generalized to the "real world." Such problems
Concern #2: Inconsistent Quality of Evidence
with external validity are oftentimes caused
Cited on ' 'Best Practices ' ' Lists
by samples that are not representative of the
When family professionals are interested in population to which the evaluators wish to
selecting a particular intervention from one ofgeneralize their findings. External validity can
the "best practices" lists, it is important to also be compromised when evaluators fail
assess not only the type of evaluation designto consider the many contextual factors that
but also the quality of the research that placedlikely impact how a strategy works for a
it on the list. Unfortunately, there are a fair particular group of individuals (Gilgun, 2006;
number of evaluations of strategies that appear Shaw, 2003).
on one or more "best practices" lists that Of course, not all strategies appearing on
suffer from some very basic methodological"best practices" lists have been evaluated with
flaws that threaten both internal and external flawed evaluations. It is very difficult, however,
validity (Gandhi et al., 2007; Gorman, 2008; to have confidence in the effectiveness of
Moskowitz, 1993). The following flaws relate strategies that were evaluated with a poorly
specifically to quantitative outcome evaluations, designed evaluation. Indeed, although it is
because to date, the "best practices" lists have entirely possible that a poorly evaluated strategy
tended not to include qualitative research or is effective, it is equally possible that positive
process evaluations. outcomes uncovered by a poor evaluation are
One example of a fundamental flaw in many merely artifacts of methodological flaws (Flay
true- and quasi-experimental designs is that et al., 2005; Gorman, 2008; M. Q. Patton, 2008;
samples oftentimes are not representative ofRossi & Freeman, 1993; Skager, 2007).

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
214 Family Relations

Implications related to Concern #2 - Evaluate are often characterized by a higher degree of

the evaluations. In light of the impact that external validity. As so few of these designs are
poorly designed evaluations can have on internal included in "best practices" lists, however, it is
validity (Flay et al., 2005; Moskowitz, 1993; important to consider external validity issues as
Rossi & Freeman, 1993; Sherman, 2003), it they relate to the commonly used experimental
is essential that family professionals carefully and quasi-experimental designs. The following
evaluate various aspects of the quality of questions can help make this assessment:
evaluations. Although a complete discussion of
1 . Was the sample representative of adolescents
internal validity can be found elsewhere (e.g.,
in general?
Campbell & Stanley, 1963; Cook & Campbell,
1979), some of the more common threats can be 2. Is it representative of the population to which
uncovered by asking the following questions: you wish to generalize?
3. Do the researchers take into account contex-
1 . Are there attempts to statistically control for tual or demographic factors or both?
existing group differences? 4. Do the results have "real-world" signifi-
2. Did group differences evolve as a result of cance?
differential subject attrition?
3. Was there evidence of family- wise error?
In light of the common threats to external
4. Were there events during the study that validity, some professionals may prefer to
influenced the results? develop novel approaches that are tailored
5. Were the measures of high quality?
to the unique characteristics of their local
6. Could the results be because of repeated communities. Indeed, program developers may
in fact overcome some of the above-mentioned
testing of participants?
7. Did the experimental group include only threats by developing a unique strategy that
very high-risk children, such that behaviors is specifically designed to fit with local
would appear to improve on their own by demographics. There are problems, however,
"regressing" to the mean? associated with developing new strategies.
8. Did treatment effects spill over to any of the For example, such efforts typically are time
members of the comparison group? consuming and expensive. In addition, the
effectiveness of new strategies is unknown. It
In addition to considering these threats to takes a long period of time and significant
internal validity, family professionals should be effort to properly evaluate new strategies to
aware of issues related to external validity. That determine long-term effectiveness (Jenson &
is, they need to determine if the effects of the Fraser, 2006).
strategy are likely to be replicated in the unique Fortunately, family professionals will find
contexts of their own communities, schools, and that there are a number of currently available
neighborhoods. Family professionals may be at strategies whose effectiveness has been docu-
a unique advantage in terms of understanding mented by high-quality evaluations with low
some of the factors that commonly impact the threats to both internal and external validity. For
external validity of a study. Indeed, in light of example, each of the three programs reviewed
the strong ecological focus in the work of many early in this paper have a long record of exper-
family science professionals, we commonly are imental and quasi-experimental evaluations that
aware of how factors such as family structure,support the effectiveness claims. Indeed, the
Guiding Good Choices Program, the Nurse-
socioeconomic status, cultural factors, extended
family, and community resources interact to Family Partnership, and FFT are examples of
influence family processes. Such processes programs that have repeatedly demonstrated
in turn impact the behavioral, psychological, positive effects across various types of eval-
and emotional outcomes of individual family uation methodologies when implemented in a
members. As such, it is important to determine variety of settings (e.g., Alexander et al., 2000;
if specific prevention strategies have different Mason et al., 2003; Olds, 1997). Such results
effects for families and family members support the conclusions of prevention scholars
depending on their unique contexts and family who have suggested that there are common fac-
processes. tors that predict a variety of risky outcomes
As mentioned earlier, given their attention among
to youth, and that strategies designed to
contextual factors, qualitative research designs address such factors can be effective across

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
Choosing Effective Prevention Strategies 215

multiple contexts (Catalano et al, 2004; Green- Implications for Concern #3 - Look for
berg, 2004; Hawkins et al., 2002; Jenson & effectiveness trials. In light of potential biases
Fraser, 2006). associated with preliminary efficacy trials,
As suggested in the discussion of Con- applied family professionals may be more confi-
cern #1, family professionals can refer to the dent in strategies that have demonstrated positive
"best practices" lists as a starting point when effects in high-quality effectiveness trials. In the
assessing the quality of evaluations. As noted event that such evidence is not available, the
in Table 1, most lists include references to following questions may help determine how to
evaluations, and some include links to electronic maximize the likelihood of success when only
copies of reports. Of course, the resources on preliminary data are available:
these lists may not be representative of all
1 . Has the strategy shown preliminary effects in
research conducted on a particular strategy.
As such, users of these lists should consider high-quality efficacy trials?
2. Do the efficacy trials limit the threats to
conducting a more extensive literature review as
external validity mentioned earlier?
outlined earlier in this paper.
3. Does your community have the resources
necessary to implement the strategy?
Concern #3: Problems Associated With Going 4. Do program distributors provide implemen-
to Scale tation support?
A related concern associated with some "best
Communities may need to assume some risk
practice" strategies has to do with whether a
if they select strategies that have only shown
strategy shown to be effective in a highly con- effects in preliminary efficacy trials. The above
trolled setting can be effectively replicated in questions,
the however, can help determine if the
"real-world" setting. In recent years, a growing
trials are "good enough," and if there is enough
number of strategies have shown positive effects
implementation support to put the strategy in
in high-quality efficacy trials, which are initial
place in a way that is true to the underlying
evaluations of strategies conducted in settingsprogram theory. Fortunately, resources are
characterized by a high degree of experimenter available to help family professionals identify
control (Kellam & Langevin, 2003). Prevention such support. Indeed, as noted in Table 1, most
researchers, however, have not had as much suc-of the "best practices" lists include information
cess in demonstrating effects in effectiveness tri-
about available resources related to individual
als, in which strategies are implemented in "real-
strategies.
world" settings (Gandhi et al., 2007; Greenberg,
2004; Kellam & Langevin; Skager, 2007).
Concern #4: Lack of Comprehensive
Prevention scholars have suggested several
Prevention Strategies
possible explanations for this trend. First, effi-
cacy trials often are implemented with signif- As described in the beginning of this paper,
icant financial and personnel resources. As a prevention strategies were relatively sim-
early
result, the strategy can be implemented with highplistic in their approach. They tended to focus
fidelity to the program model, which increases on only one problem behavior, and they typi-
the likelihood of positive effects. Similarly,callyas attempted to address only one or a few
efficacy trials oftentimes are implemented of bythe many factors that place an adolescent at
program developers, there is substantial exper- risk for engaging in such behavior (Catalano
tise in terms of how to implement the strategy et al., 2002; Jenson & Fraser, 2006). Given the
and how to troubleshoot should unexpected many advances in our understanding of the eti-
problems occur. Again, these factors are likely ology of many youth outcomes, we now know
to increase the chance of positive findings (Gor-that it is important to address a wide variety
man & Conde, 2007; Lipsey, 1995; Petrosino of factors within multiple ecological contexts.
& Soydan, 2005; Small, 2005). Some have also Indeed, various reviews of the literature con-
clude that the most effective prevention strate-
argued that positive results in efficacy trials may
be encouraged by conflicts of interest and biasesgies typically target factors within individual,
that result from program developer participation peer, school, family, and community contexts
in the research process. Such potential biases (Bogenschneider,
are 2006; Catalano et al., 2002,
addressed later under Concern #6. 2004; Jenson & Fraser). Unfortunately, although

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
216 Family Relations

many of the "best practices" lists include such
comprehensive strategies, some also continue to
include simple approaches that target only one
or a few risk factors for a single outcome.
One trend that may be of particular interest to
family professionals is that many prevention
strategies with a strong family focus tend
to be particularly effective (Kumpfer et al.,
2003; Spoth et al., 2002). This is probably
because of two factors. First, many of the
most effective family programs, such as FFT,
are extremely comprehensive. Indeed, while
FFT does emphasize the importance of family
relations, it also focuses on the ways in which
a particular family interacts with and is affected
by other ecological contexts, such as school,
work, and peer groups, among others (Alexander
et al., 2000).
In addition, the strong effects of family
strategies are likely influenced by the fact
that the proximal processes associated with
the family exert a particularly strong influence
on human behavior (Bronfenbrenner, 1986a).
Individuals are imbedded within a particular
family system from the moment they are born.
Family members typically provide the earliest
social interactions that an infant experiences,
and the primacy of family influences generally
continues throughout childhood, despite the
introduction of multiple influences such as
peers, teachers, and others within the larger
community. Given the significant influence
of the family context, scholars have noted
its importance in influencing, guiding, and
supporting social and emotional development.
Indeed, as Bronfenbrenner (1986b) noted, "the
family is the most powerful, the most humane,
and by far the most economical system known
for building competence and character."
Implications for Concern #4 - Look for a broad
contextual focus. In light of the preceding dis-
cussion, family professionals who are interested
in selecting highly effective prevention strate-
gies should be sure to look for comprehensive
programs that target risk and protective factors
within multiple contexts. Most of the current
"best practices" lists help facilitate this pro- difficult to change. Furthermore, when changes
cess by providing rich descriptions of program
activities.
Family professionals can take several
approaches when considering comprehensive
approaches to prevention. First, they can select for such difficulties in maintaining program
specific strategies that have a broad focus and effects is that program participants typically
target risk or protective factors or both that occur
within multiple ecological contexts. Some fam-
ily therapy programs such as FFT fall into this
category (Alexander et al., 2000). Alternatively,
they can engage in a community-wide effort in
which multiple smaller, more targeted programs
are combined to address the underlying causes of
problem behaviors. Several strategies have been
developed that help guide communities through
a needs assessment, selection of effective pre-
vention strategies, and evaluation of the results
(Bogenschneider, 2006; Hawkins et al., 2002).
One example of such a strategy is the Com-
munities That Care system, which is currently
available through the Substance Abuse and Men-
tal Health Services Administration (SAMHSA).
In addition to focusing on such compre-
hensive strategies, we, as family professionals,
should pay close attention to strategies that target
family relationships and interactions. Although
a family focus certainly does not guarantee that
a strategy will be effective, there are many
examples of successful family-based preven-
tion strategies. Indeed, each of the effective
programs used as examples throughout this
paper has a strong family component. The
Guiding Good Choices Program specifically
encourages parents to develop healthy fam-
ily processes (Mason et al., 2003; Spoth et al.,
2004). The Nurse-Family Partnership intervenes
with high-risk mothers before their children
are even born (Olds, 1997). Among the most
effective indicated prevention strategies are a
variety of family therapy models, including the
FFT program described above (Alexander et al.,
2000).
Concern #5: Few Long-Term Results
Although some high-quality evaluations have
identified effective prevention strategies, very
few have demonstrated lasting impact. Even
the "best" strategies tend to have effects that
last only for several months. Typically, program
effects wear off unless follow-up, or "booster"
sessions are conducted. Many scholars have
noted that human behavior is extraordinarily
are made, it is equally difficult to maintain such
changes (Catalano et al., 2004; Flay et al., 2005;
Gandhi et al., 2007; Greenberg, 2004).
Perhaps one of the most significant reasons

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
Choosing Effective Prevention Strategies 217

go back to their prior ecological contexts: (Alexander et al., 2000; Spoth et al., 2004). In
their same families, peer groups, schools, and addition, the Nurse-Family Partnership program
workplaces. As discussed earlier, these contexts has demonstrated effects on children whose
have a significant influence on an individual's mothers had participated in the program 1 5 years
behavior (Bronfenbrenner, 1986a). Short-term earlier (Olds, 1997). Such effects are impressive,
educational sessions that are commonly part of and are not typical among most prevention
prevention strategies simply cannot be expected strategies.
to overcome the intense influence of being
immersed in one's typical ecological settings.
Concern #6: Political Influences on ' 'Best
As described in the preceding section, many
Practices ' ' Lists
prevention strategies are rather simplistic in
nature. As described below, however, there are As mentioned earlier in this paper, many
some notable examples of more comprehensive funding agencies currently tie dollars for
strategies that do show lasting positive effects prevention programming to selecting strategies
(Catalano et al., 2002, 2004; Hawkins étal., from one of the popular "best practices"
2002; Kellam & Langevin, 2003). lists. Although such a practice has increased
incentives for communities to use "evidence-
Implications for Concern #5 - Do positive effects based" practices, it also has had an unfortunate
persist? Clearly, long-term effectiveness of a side effect of encouraging conflicts of interest
prevention strategy is an important consideration among program developers. More specifically,
for anyone interested in investing resources in given that schools and community groups
a particular strategy. A few basic questions can are spending money on packaged prevention
help guide family professionals as they identify programs, there is a profit that can be
strategies that are likely to result in long-term made from developing, packaging, and selling
effects:
such programs. By tying prevention dollars
to appear on "best practices" lists, flinders
1 . Did the program evaluations include follow-
have inadvertently placed immense pressure on
up assessments?
program developers and distributors to get their
2. If so, how long were program effects
maintained? programs on at least one of these lists. This
obviously introduces a serious conflict of interest
3. Does the strategy include follow-up or
booster sessions? that can result in selective reporting of evaluation
results to "best practices" review panels
4. If so, were they evaluated for effectiveness?
(Borman, Hewes, Overman, & Brown, 2003;
5. Is the strategy comprehensive enough to
Gorman & Conde, 2007; Moskowitz, 1993;
expect long-term effects?
Petrosino & Soydan, 2005). Indeed, Gorman and
Most of the popular "best practices" lists Conde have noted that many program developers
can help family professionals find answers to have a financial relationship with the distributor
these types of questions. Indeed, most include of their packaged programs that may encourage
at least basic descriptions of the findings ofthem to emphasize positive findings.
evaluations, and these descriptions typically Of course, the above-mentioned relationships
discuss any long-term follow-up evaluations.among program developers and distributors
One of the lists, the Blueprints Program list, certainly do not guarantee overt unethical
actually requires that its "model programs" behavior among individuals within each group.
have demonstrated effectiveness at least 1 year Such relationships, however, have the potential
following program termination (see Table 1). to influence research findings in subtle ways,
Although there are not many strategies that especially if the program developers also act
demonstrate long-term effectiveness, there are as program evaluators or if they have close
some that show effects for at least several ties with outside evaluators. For example, while
years following the end of the program. Indeed, one researcher may focus on two significant
each of the three programs I have used as
positive findings and conclude that a strategy is
examples in this paper have demonstrated lastingeffective, another may look at the same data, see
effects. Both the Guiding Good Choices Program six nonsignificant findings and deem the strategy
and FFT have demonstrated effects lasting at
ineffective. In this example, two researchers may
least several years after program termination interpret the same data in very different ways

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
218 Family Relations

depending on their own personal views of criteria such, it is often helpful to use the lists as a
for effectiveness. When program evaluators have preliminary source of information to narrow
a vested interest in seeing positive outcomes, choices and then conduct a more comprehensive
they may be more likely to emphasize the two literature review as outlined earlier in this paper.
significant findings than the six that did not reach The following questions may help guide this
statistical significance. process:
This discussion does not mean to imply
1. Are there articles not cited within the lists
that program developers are purposefully biased
or in some other way unethical. Although that are related to the strategy?
there may be examples of program developers 2. Will program developers share unpublished
data?
being influenced by the possibility of increasing
program sales, many scholars have noted that 3. Have colleagues had success with the
true objectivity in research is difficult to maintain strategy?
in any empirical investigation, regardless of the 4. Are there any unpublished studies available
from other sources?
field of study. Our past experiences, education,
and general world views influence everything 5. Have there been any independent evaluations
from the way we phrase our research questions, of the strategy?
to the types of statistical analyses we employ, to
the types of conclusions we draw from a set of Conclusion
data (Sherman, 2003; Small, 1995). Given the
close ties between a program developer and his Taken together, the suggestions outlined in th
or her program, it may be inherently difficult paper provide a starting point for applied fami
for him or her to step back and make a truly professionals who are interested in assessin
unbiased judgment regarding overall program the quality of existing prevention strategie
quality. Of course, following these suggestions doe
In addition to biases of program developers, not guarantee success. I hope, however, th
Weiss et al. (2008) have suggested that conflicts the advice contained within will encourag
of interest might arise from members of personal reflections and further dialogue that w
review panels. They specifically suggest thathelp family professionals avoid some common
decisions made about whether or not to rec- missteps when assessing the nature and qualit
of specific evaluation designs. Ultimately,
ognize particular programs may be influenced by
hope such a reflection results in the selection
panel members' own past histories, biases, and
possible misperceptions of specific programs.of strategies that demonstrate true effectivene
Taken together with the above discussion on when implemented in the "real world."
potential conflicts of interest, these points The discussion outlined in this article also
encourages family professionals to be cautious
suggest that inclusion on ''best practices" lists
may be at least partially influenced by multiplewhen using the popular "best practices"
biases stemming from the priorities, values, andlists. Although such lists may serve as an
goals of developers, distributors, and reviewers.
excellent starting point for identifying effective
strategies, it is up to the individual user of
the information to obtain objective outside
Implications for Concern #6 - Can the source be
trusted? As mentioned throughout this paper, information to help assess the true potential
many of the popular "best practices" listsof such strategies. Although collecting outside
include valuable information that can help
information may take considerable time and
family professionals select effective prevention effort, such an investment would increase
strategies. In light of potential biases described
the likelihood that a chosen strategy is truly
above, however, it is essential that users ofeffective. As stated throughout this paper,
these lists critically and objectively examine the
applied family professionals may have unique
evidence in support of program effectiveness. insights into identifying effective strategies.
While some lists include a comprehensive
Because our professional training and work
review of the literature (see Table 1), more often
strongly emphasize the important influences of
contextual variables on individual outcomes, we
the cited research represents only a small portion
of the findings related to a particular strategy may be well positioned to assess the degree to
(Gandhi et al., 2007; Weiss et al., 2008). As which particular strategies adequately address

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
Choosing Effective Prevention Strategies 219

Catalano,
factors related to risk and R. F., Berglund,
resiliency M. L., Ryan, J. A. M.,
within
Lonczak, H. S., & Hawkins, J. D. (2004). Positive
multiple ecological contexts.
youth
By examining the history ofdevelopment in the United
the field of States: Research
prevention science, it is easyfindings
to see on evaluations of positive youth develop-
that we have
ment programs. Annals of the American Academy,
made significant progress over the past several
597,98-124.
decades. Today, there are examples of well-
Catalano, R. F., Hawkins, J. D., Berglund, M. L.,
designed, comprehensive prevention strategies
Pollard, J. A., & Arthur, M. W. (2002). Prevention
that appear to have lasting positive
science andeffects on
positive youth the
development: Com-
developmental trajectories of young
petitive people.
or cooperative As Journal of
frameworks?
Adolescent relatively
suggested in this paper, however, Health, 31, 230-239.few
strategies have convincinglyChanning
and Beteconsistently
Company. (2004). Prevention strate-
demonstrated such positive gies guide. South Deerfield,
effects. As such, MA: Author.
it
Cook,T.
is up to individuals interested inD., implementing
& Campbell, D. T. (1979). a
Quasi-
experimentation: Design and analysis issues for
specific prevention strategy to gain the skills
field settings. Boston: Houghton-Mifflin.
necessary to assess the quality of evidence
Cook, T. D., Shadish, W. R., & Wong, V. C. (2008).
supporting its effectiveness. With a critical
Three conditions under which experiments and
eye for inferior quality research
observationalandstudies program
produce comparable causal
designs, a focus on comprehensive approaches,
estimates: New findings from within-study com-
and realistic expectations regarding
parisons. Journal potential
of Policy Analysis and Manage-
program effects, we, as family professionals,
ment, 27,724-750.
can help youth within our Flay, B. R., Biglan, A., Boruch,
communities R. F., Castro, F. G.,
grow
Gottfredson,
and develop into happy, healthy, and D., Kellam, S., Moscicki, E. K.,
productive
members of adult society. Schinke, S., Valentine, J. C, & Ji, P. (2005).
Standards of evidence: Criteria for efficacy, effec-
tiveness and dissemination. Prevention Science, 6,
151-175.
References
Gandhi, A. G., Murphy-Graham, E., Petrosino, A.,
Alexander, J., Pugh, C, Parsons, B., & Sexton, T.
Chrismer, S. S., & Weiss, C. H. (2007). The devil
(2000). Blueprints for violence protection: Func- is in the details: Examining the evidence for
tional Family Therapy. Boulder, CO: Institute "proven"
of school-based drug abuse prevention
Behavioral Science, University of Colorado. programs. Evaluation Review, 31, 43-74.
Bogenschneider, K. (2006). Family policy matters:Gilgun, J. F. (2006). The four cornerstones of
How policymaking affects families and what qualitative research. Qualitative Health Research,
professionals can do (2nd ed.). Mahwah, NJ: 76,436-443.
Lawrence Erlbaum Associates. Gorman, D. M. (1998). The irrelevance of evidence in
Bogenschneider, K., Olson, J. R., Linney, K. D., &the development of school-based drug prevention
Mills, J. (2000). Connecting research and policy- policy, 1986-1996. Evaluation Review, 22,
making: Implications for theory and practice from 118-146.
Gorman, D. M. (2002). Defining and operationaliz-
the Family Impact Seminars. Family Relations, 49,
327-339. ing "research-based" prevention: A critique (with
Borman, G. D., Hews, G. M., Overman, L. T., & case studies) of the US Department of Educa-
Brown, S. (2003). Comprehensive school reform tion's Safe, Disciplined and Drug-Free Schools
and achievement: A meta-analysis. Review of Exemplary Programs. Evaluation and Program
Educational Research, 73, 125-230. Planning, 25,295-302.
Bronfenbrenner, U. (1986a). Ecology of the family Gorman, D. M. (2008). Science, pseudoscience and
as a context for human development: Research the need for practical knowledge. Addiction, 103,
perspectives. Developmental Psychology, 22, 1752-1753.
723-742. Gorman, D.M., & Conde, E. (2007). Conflict of
Bronfenbrenner, U. (1986b, May). A generation interest in the evaluation and dissemination
in jeopardy: America's hidden family policy. of "model" school-based drug and violence
Testimony presented at a hearing of the preventionSenate programs. Evaluation and Program
Committee on Rules and Administration on a Planning, 30, 422-429.
resolution to establish a Select Committee on Greenberg, M. T. (2004). Current and future chal-
Families, Youth and Children, Washington, DC. lenges in school-based prevention: The researcher
Campbell, D. T., & Stanley, J. C. (1963). Experimen-
perspective. Prevention Science, 5, 5- 13.
tal and quasi-experimental designs for research.
Hallfors, D. D., Pankratz, M., & Hartman, S. (2007).
Chicago: Rand-McNally. Does federal policy support the use of scientific

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms
 220 Family Relations

evidence in school-based
Patton, pr
G. C. (2008). Weighing an ounce of preven
Prevention tion. Journal of Adolescent
Science, 8, 75Health, -
43,3-5.
8 1
Hawkins, D.,
Patton, M. Catalano, J. R.
Q. (2008). Utilization-focused evaluatio
(2002). Promoting science-
(4th ed.). Thousand Oaks, CA: Sage.
communities. Addictive
Petrosino, A., & Soydan, H. (2005). TheBehimpact
Jenson, J. program
M. developers as evaluators
(2006). Advan on crimina
preventing childhood and
recidivism: Results from meta-analyses of exper-
behavior. Social
imental andWork
quasi-experimentalResea
research. Journ
Jenson, J. M., & Fraser,
of Experimental M.
Criminology, 1, 435-450.
for children Rossi,
andH. E., & Freeman, P. H. (1993). Evaluation
families:
perspective. A systematic approach (5th ed.).Oaks,
Thousand Thousand Oaks
Kellam, S. CA: Sage.
G., & Langevin,
work for Sexton, T., & Alexander, J. F. (2000, December).
understanding "evi
research and programs.
Functional Family Therapy. Office of Juvenile Jus- Pr
137-153. tice and Delinquency Prevention Juvenile Justice
Kumpfer, K. L., Al varado, R., & Whiteside, H. Bulletin
O. (OJJDP Publication No. NCJ 184743).
(2003). Family-based interventions for substance Rockville, MD: Juvenile Justice Clearinghouse.
use and misuse prevention. Substance Use Shaw,
andI. (2003). Qualitative research and outcomes
Misuse, 38, '1 59 -mi. in health, social work and education. Qualitative
Lipsey, M. W. (1995). What do we learn from 400 Research, 3,51-71.
research studies on the effectiveness of treatment Sherman, L. W. (2003). Misleading evidence and
with juvenile delinquents? In J. McGuire (Ed.), evidence-led policy: Making social science more
What works? Reducing reoffending: Guidelines experimental. Annals of the American Academy,
from research and practice (pp. 63-68). Oxford, 589,6-19.
England: John Wiley & Sons. Skager, R. (2007). Replacing ineffective early alco-
Mason, W. A., Kosterman, R., Hawkins, J. D., hol/drug education in the United States with age-
Haggerty, K. P., & Spoth, R. L. (2003). Reduc- appropriate adolescent programmes and assistance
ing adolescents' growth in substance use and to problematic users. Drug and Alcohol Review,
delinquency: Randomized trial effects of a parent- 26,577-584.
training prevention intervention. Prevention Sci- Small, S. A. (1995). Action-oriented research: Mod-
ence, 4, 203-212. els and methods. Journal of Marriage and the
McCall, R. B., & Green, B. L. (2004). Beyond the Family, 57,941-955.
methodological gold standards off behavioral Small, S. A. (2005). Bridging research and practice in
research: Considerations for practice and policy. the family and human sciences. Family Relations,
Social Policy Report, 18, 3-19. 54,320-334.
McCartney, K., & Rosenthal, R. (2000). Effect size, Spoth, R., Redmond, C, Shin, C, & Azevedo, K.
practical importance, and social policy for children. (2004). Brief family intervention effects on ado-
Child Development, 71, 173-180. lescent substance initiation. School-level growth
Moskowitz, J. M. (1993). Why reports of outcome curve analyses 6 years following baseline. Jour-
evaluations are often biased or uninterpretable: nal of Consulting and Clinical Psychology, 72,
Examples from evaluations of drug abuse preven- 535-542.
tion programs. Evaluation and Program Planning, Spoth, R. L., Kavanagh, K. A., & Dishion, T. J.
76,1-9. (2002). Family-centered preventive intervention
Myers- Walls, J. (2000). An odd couple with promise: science: Toward benefits to larger populations of
Researchers and practitioners in evaluation set- children, youth, and families. Prevention Science,
tings. Family Relations, 49, 341-347. 3,145-152.
Wandersman, A., & Florin, P. (2003). Community
Nathan, R. P. (2008). Point/counterpoint: The role
of random assignment in social policy research. interventions and effective prevention. American
Journal of Policy Analysis and Management, 27, Psychologist, 58, 441 -448.
401-415. Weiss, C. H., Murphy-Graham, E., Petrosino, A.,
Olds, D. L. (1997). The Prenatal/Early Infancy Gandhi, A. G. (2008). The fairy godmother- And
Project: Fifteen years later. In G. W. Albee her &warts: Making the dream of evidence-based
T. Gullotta (Eds.), Primary prevention works policy come true. American Journal of Evaluation,
(pp. 41 -67). Thousand Oaks, CA: Sage. 29,29-41.

This content downloaded from 41.89.10.241 on Mon, 27 Jan 2020 08:06:53 UTC
All use subject to https://about.jstor.org/terms