Dissemination and implementation of

evidence-based practices in child and

adolescent mental health First Issued
As An Oxford University Press
Paperback Edition Rinad S. Beidas
Visit to download the full and correct content document:
https://textbookfull.com/product/dissemination-and-implementation-of-evidence-based
-practices-in-child-and-adolescent-mental-health-first-issued-as-an-oxford-university-p
ress-paperback-edition-rinad-s-beidas/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

The Civil Sphere First Issued As An Oxford University

Press Paperback Edition Jeffrey C. Alexander

https://textbookfull.com/product/the-civil-sphere-first-issued-
as-an-oxford-university-press-paperback-edition-jeffrey-c-
alexander/

This life of sounds evenings for new music in Buffalo

First Issued As An Oxford University Press Paperback
Edition Levine Packer

https://textbookfull.com/product/this-life-of-sounds-evenings-
for-new-music-in-buffalo-first-issued-as-an-oxford-university-
press-paperback-edition-levine-packer/

Americanizing Britain the rise of modernism in the age

of the entertainment empire First Issued As An Oxford
University Press Paperback Edition Abravanel

https://textbookfull.com/product/americanizing-britain-the-rise-
of-modernism-in-the-age-of-the-entertainment-empire-first-issued-
as-an-oxford-university-press-paperback-edition-abravanel/

DSM 5 Pocket Guide for Child and Adolescent Mental

Health First Edition American Psychiatric Association

https://textbookfull.com/product/dsm-5-pocket-guide-for-child-
and-adolescent-mental-health-first-edition-american-psychiatric-
association/
Primary Child and Adolescent Mental Health A Practical
Guide Volume 1 Quentin Spender

https://textbookfull.com/product/primary-child-and-adolescent-
mental-health-a-practical-guide-volume-1-quentin-spender/

Child Adolescent and Family Refugee Mental Health A

Global Perspective Suzan J. Song

https://textbookfull.com/product/child-adolescent-and-family-
refugee-mental-health-a-global-perspective-suzan-j-song/

Dignity its history and meaning First Harvard

University Press Paperback Edition Rosen

https://textbookfull.com/product/dignity-its-history-and-meaning-
first-harvard-university-press-paperback-edition-rosen/

Adolescent Mental Health Prevention and Intervention

Terje Ogden

https://textbookfull.com/product/adolescent-mental-health-
prevention-and-intervention-terje-ogden/

Latin story of a world language First Harvard

University Press Paperback Edition Kronenberg

https://textbookfull.com/product/latin-story-of-a-world-language-
first-harvard-university-press-paperback-edition-kronenberg/
 Dissemination and
Implementation of
Evidence-Based Practices
in Child and Adolescent
Mental Health
 Dissemination and
Implementation of
Evidence-Based Practices
in Child and Adolescent
Mental Health

EDIT ED BY RINAD S. BE I DAS

and PH IL IP C. KE NDA LL

1
 1
Oxford University Press is a department of the University of
Oxford. It furthers the University’s objective of excellence in research,
scholarship, and education by publishing worldwide.

Oxford New York

Auckland Cape Town Dar es Salaam Hong Kong Karachi
Kuala Lumpur Madrid Melbourne Mexico City Nairobi
New Delhi Shanghai Taipei Toronto

With offices in
Argentina Austria Brazil Chile Czech Republic France Greece
Guatemala Hungary Italy Japan Poland Portugal Singapore
South Korea Switzerland Thailand Turkey Ukraine Vietnam

Oxford is a registered trademark of Oxford University Press

in the UK and certain other countries.

Published in the United States of America by

Oxford University Press
198 Madison Avenue, New York, NY 10016

© Oxford University Press 2014

All rights reserved. No part of this publication may be reproduced, stored in

a retrieval system, or transmitted, in any form or by any means, without the prior
permission in writing of Oxford University Press, or as expressly permitted by law,
by license, or under terms agreed with the appropriate reproduction rights organization.
Inquiries concerning reproduction outside the scope of the above should be sent to the
Rights Department, Oxford University Press, at the address above.

You must not circulate this work in any other form

and you must impose this same condition on any acquirer.

Library of Congress Cataloging-in-Publication Data

Dissemination and implementation of evidence-based practices in child and adolescent
mental health / edited by Rinad S. Beidas, Philip C. Kendall.
pages cm
Includes bibliographical references and index.
ISBN 978–0–19–931162–0
1. Child mental health. 2. Teenagers—Mental health. 3. Evidence-based psychiatry. 4. Clinical competence.
I. Beidas, Rinad S., editor of compilation. II. Kendall, Philip C., editor of compilation.
RJ499.D53 2014
618.92′89—dc23
2014001817

1 3 5 7 9 8 6 4 2
Printed in the United States of America
on acid-free paper
To my husband, Karl, and my son, Miles, for cheering me on to follow my dreams
and making every day a brighter one and to my parents, Sary and Amal, for
teaching me the value of life-long learning.
—Rinad

To those who are willing to look at the data, be influenced by the findings, and
use what is learned to benefit others.
—Phil
 Contents

About the Editors ix 4 Capturing Fidelity in Dissemination and

Contributors xi Implementation Science 44
Sonja K. Schoenwald, Jason E. Chapman,
and Ann F. Garland
PART ONE: INTRODUCTION
5 The Role of Organizational Culture
1 The Promise of Evidence-Based Practices
and Climate in the Dissemination and
in Child and Adolescent Mental
Implementation of Empirically Supported
Health 3
Treatments for Youth 61
Philip C. Kendall and Rinad S. Beidas
Nathaniel J. Williams and Charles Glisson
6 Leadership and Strategic Organizational
PART TWO: BACKGROUND
Climate to Support Evidence-Based
2 Guiding Theory for Dissemination and Practice Implementation 82
Implementation Research: A Reflection Gregory A. Aarons, Lauren R. Farahnak,
on Models Used in Research and and Mark G. Ehrhart
Practice 9
7 Dissemination and Implementation in
David A. Chambers
Children’s Mental Health: Closing the
3 Measurement in Dissemination and Research to Training Gap 98
Implementation Science 22 Stacy L. Frazier, Sarah Kate Bearman,
Enola K. Proctor, Byron J. Powell, Ann F. Garland, and Marc S. Atkins
and Megan A. Feely

• vii
PART THREE: SYSTEM-LEVEL
APPROACHES TO IMPLEMENTING
EVIDENCE-BASED PRACTICES FOR
YOUTH IN COMMUNITY MENTAL
HEALTH SETTINGS
8 Practice–Research Partnerships That
Scale-up, Attain Fidelity, and Sustain
Evidence-Based Practices 127
Patricia Chamberlain and Lisa Saldana
9 From Experience to Experiment: Using
State Systems as Laboratories for
Implementation of Evidence-Based
Practices for Children 143
Erum Nadeem, S. Serene Olin, Alissa
Gleacher, Ka Ho Brian Chor, Dara C. Weiss,
Andrew F. Cleek, Mary M. McKay, and
Kimberly E. Hoagwood
10 Transformation of Mental Health
Services for Children and Young People
in England 158
Roz Shafran, Peter Fonagy, Kathryn Pugh, and
Pamela Myles
11 Global Dissemination and
Implementation of Child
Evidence-Based Practices in Low
Resources Countries 179
Laura K. Murray, Shannon Dorsey,
and Eric Lewandowski
12 Building and Advancing an
Evidence-Based Service System in
Hawaii 204
Brad J. Nakamura, Lesley Slavin, Scott
Shimabukuro, and Scott Keir
PART FOUR: APPROACHES TO
IMPLEMENTING EVIDENCE-BASED
PRACTICES FOR YOUTH IN SCHOOLS
13 Dissemination and Implementation
of Empirically Supported Treatments
for Anxious Youth in Community
Settings 223
Chiaying Wei, Colleen Cummings, Joanna Herres,
Kendra L. Read, Anna Swan, Matthew M.
Carper, Alexandra Hoff, Vijaita Mahendra,
and Philip C. Kendall
viii • C ontents
14 Dissemination and Implementation
of Treatments for Disruptive Disorders
in Schools 243
Caroline L. Boxmeyer, John E. Lochman, Nicole
P. Powell, Rachel Baden Sherrill, Sara Stromeyer,
and Meghann Kelly Sallee
15 Implementation of Evidence-Based
Practices for Children with Autism
Spectrum Disorders in Public
Schools 261
Jill Locke, Hilary E. Kratz, Erica M. Reisinger,
and David S. Mandell
16 Dissemination and Implementation of
Cognitive Therapy for Depression in
Schools 277
Courtney L. Benjamin, Kristin Pontoski Taylor,
Samantha M. Goodin, and Torrey A. Creed
17 Cognitive Behavioral Intervention for
Trauma in Schools: Dissemination and
Implementation of a School-Based
Intervention 294
Sheryl H. Kataoka, Catherine DeCarlo
Santiago, Lisa H. Jaycox, Audra K. Langley,
Bradley D. Stein, and Pamela Vona
PART FIVE: APPROACHES TO
IMPLEMENTING EVIDENCE-BASED
PRACTICES FOR YOUTH USING
TECHNOLOGY
18 Internet-Based Dissemination and
Implementation of Cognitive Behavioral
Therapy for Child Anxiety 313
Muniya S. Khanna, Connor Morrow Kerns, and
Matthew M. Carper
19 Internet-Based Implementation:
Broadening the Reach of Parent-Child
Interaction Therapy for Early Child
Behavior Problems 336
R. Meredith Elkins and Jonathan S. Comer
Index 357
 About the Editors
Rinad S. Beidas is a successful researcher and cli-
nician with expertise in implementation science
and the assessment and treatment of child and ado-
lescent anxiety disorders. Dr. Beidas is currently
a senior fellow in the Leonard Davis Institute,
as well as a fellow in the National Institute of
Mental Health (NIMH) funded Implementation
Research Institute (IRI). She is also an alumna
fellow of the National Institutes of Health
funded Training Institute in Dissemination and
Implementation Research in Health (TIDIRH),
and the NIMH-funded Child Intervention and
Prevention Services (CHIPS) Fellowship.
Dr. Beidas’s research centers on the dissemi-
nation and implementation of evidence-based
practices (EBPs) for youth in community set-
tings. Dr. Beidas is particularly interested in
understanding how to most effectively support
therapists, organizations, and systems in the
implementation of EBPs. Current work involves
an NIMH funded K23 MH099179 project that
prospectively investigates the impact of a policy
mandate on implementation of EBPs in outpa-
tient mental health services for youth in the pub-
lic sector (Policy to Implementation; P2I).
Clinically, Dr. Beidas’s expertise is in the
cognitive-behavioral treatment of child and ado-
lescent anxiety.
Dr. Beidas holds a bachelor of arts in psychol-
ogy from Colgate University and a doctorate of
philosophy in psychology from Temple University.
Dr. Beidas is known for her clinical acumen,
her passion for improving the quality of care for
underserved youth through the implementation
of evidence-based practices, and strong commit-
ment for mentoring others.
Philip C. Kendall has been a productive
researcher, scholar, and clinician. His CV lists
more than 450 publications, including over
30 books and over 20 treatment manuals and
workbooks. His treatment programs have been
• ix
translated into dozens of languages, and he has
had 30 years of uninterrupted grant support from
various agencies. Having received many thou-
sands of citations per year, he placed among an
elite handful of the most “Highly-Cited” individ-
uals in all of the social and medical sciences. In
a recent quantitative analysis of the publications
by and citations to all members of the faculty in
the 157 American Psychological Association
approved programs in clinical psychology,
Dr. Kendall ranked fifth.
Dr. Kendall has garnered prestigious
awards: Fellow at the Center for Advanced Study
in the Behavioral Sciences, inaugural Research
Recognition Award from the Anxiety Disorders
Association of America, “Great Teacher” award
from Temple University, identified as a “top
therapist” in the tri-state area by Philadelphia
Magazine, and a named chair and Distinguished
University Professorship at Temple University.
He is Board Certified by the American Board
of Professional Psychology in (1) Cognitive
and Behavioral Psychology and (2) Child and
Adolescent Clinical Psychology. He has been
president of the Society of Clinical Child and
x • A bout the E ditors
Adolescent Psychology (Division 53) of APA
as well as President of the Association for the
Advancement of Behavior Therapy (AABT, now
ABCT). Recently, ABCT recognized and awarded
him for his “Outstanding Contribution by an
Individual for Educational/Training Activities.”
Dr. Kendall has contributed as a basic sci-
entist, theorist, teacher, and administrative
leader. His contributions include seminal work
on the treatment of anxiety disorders in youth,
cognitive-behavioral theory, assessment, and
treatment, research methodology, and in the
conceptualizing and understanding of the psy-
chopathology and treatment of children and
adolescents.
Dr. Kendall’s doctorate in clinical psychol-
ogy is from Virginia Commonwealth University
where his work emphasized research in clinical
child and adolescent psychology. He has been
honored with the Outstanding Alumnus Award
from this institution.
Dr. Kendall is known for his clinical sensitiv-
ity, rigorous research methodology, creative and
integrative approaches, and commitment to pro-
fessional development and graduate mentoring.
 Contributors
Gregory A. Aarons, PhD
University of California, San Diego
Center for Organizational Research on
Implementation and Leadership
Child and Adolescent Services Research Center
Department of Psychiatry
Marc S. Atkins, PhD
University of Illinois at Chicago
Department of Psychiatry
Sarah Kate Bearman, PhD
Ferkauf Graduate School of Psychology
Rinad S. Beidas, PhD
University of Pennsylvania
Department of Psychiatry
Courtney L. Benjamin, PhD
University of Pennsylvania
Department of Psychiatry
Caroline L. Boxmeyer, PhD
The University of Alabama
Department of Psychiatry and Behavioral
Medicine
Matthew M. Carper, MA
Temple University
Department of Psychology
Patricia Chamberlain, PhD
Oregon Social Learning Center
David A. Chambers, D.Phil
The National Institutes of Health
Dissemination and Implementation Research
Jason E. Chapman, PhD
Medical University of South Carolina
Department of Psychiatry and Behavioral
Sciences
• xi
Ka Ho Brian Chor, PhD Ann F. Garland, PhD
New York University University of San Diego
Department of Child and Adolescent Psychiatry Department of School, Family, & Mental Health
Professions
Andrew F. Cleek, PsyD
New York University Alissa Gleacher, PhD
McSilver Institute for Poverty Policy & Research New York University
Department of Child and Adolescent
Jonathan S. Comer, PhD
Psychiatry
Florida International University
Department of Psychology Charles Glisson, PhD
University of Tennessee
Torrey A. Creed, PhD
Children’s Mental Health Services
University of Pennsylvania
Research Center
Department of Psychiatry
College of Social Work
Colleen Cummings, PhD
Samantha M. Goodin, MsED
Temple University
University of Pennsylvania
Department of Psychology
Department of Psychiatry
Shannon Dorsey, PhD
Joanna Herres, PhD
University of Washington
Temple University
Department of Psychology
Department of Psychology
Mark G. Ehrhart, PhD
Kimberly E. Hoagwood, PhD
San Diego State University
New York University
Department of Psychology
Department of Child and Adolescent
Center for Organizational Research on
Psychiatry
Implementation and Leadership
Alexandra Hoff, MA
R. Meredith Elkins, MA
Temple University
Boston University
Department of Psychology
Department of Psychology
Lisa H. Jaycox, PhD
Lauren R. Farahnak, MS
RAND Corporation
University of California, San Diego
Department of Psychiatry Sheryl H. Kataoka, MD, MSHS
Center for Organizational Research on UCLA Semel Institute
Implementation and Leadership Division of Child & Adolescent Psychiatry
Child and Adolescent Services Research Center Center for Health Services and Society
Megan A. Feely, MSW Scott Keir, PhD
Washington University in St. Louis. State of Hawaii Child and Adolescent Mental
Brown School of Social Work Health Division
Research, Evaluation, and Training
Peter Fonagy, PhD
University College, London Philip C. Kendall, PhD
Research Department of Clinical, Educational Temple University
and Health Psychology Department of Psychology
Stacy L. Frazier, PhD Connor Morrow Kerns, PhD
Florida International University Drexel University
Department of Psychology A.J. Drexel Autism Institute

xii • C ontributors
Muniya S. Khanna, PhD Erum Nadeem, PhD
The OCD & Anxiety Institute New York University
Department of Child and Adolescent Psychiatry
Hilary E. Kratz, PhD
University of Pennsylvania Perelman School Brad J. Nakamura, PhD
of Medicine University of Hawaii at Manoa
Department of Psychiatry Department of Psychology
Child & Adolescent OCD, Tic, Trich &
S. Serene Olin, PhD
Anxiety Group
New York University
Audra K. Langley, PhD Department of Child and Adolescent Psychiatry
UCLA Semel Institute
Byron J. Powell, AM
Division of Child & Adolescent Psychiatry
Washington University in St. Louis
Eric Lewandowski, PhD Brown School of Social Work
New York University
Nicole P. Powell, PhD, MPH
Department of Child and Adolescent Psychiatry
The University of Alabama
John E. Lochman, PhD, ABPP Center for the Prevention of Youth Behavior
The University of Alabama Problems
Department of Psychology
Enola K. Proctor, PhD
Jill Locke, PhD Washington University in St. Louis
University of Pennsylvania Perelman School of Brown School of Social Work
Medicine
Kathryn Pugh, MA
Department of Psychiatry
National Health Service England
Center for Mental Health Policy and Services
MACantab Medical Directorate
Research
Kendra L. Read, MA
Vijaita Mahendra, MPhil
Temple University
Temple University
Department of Psychology
Department of Psychology
Erica M. Reisinger, MSEd
David S. Mandell, ScD
University of Pennsylvania Perelman School
University of Pennsylvania Perelman School
of Medicine
of Medicine
Department of Psychiatry
Department of Psychiatry
Center for Mental Health Policy and Services
Center for Mental Health Policy and Services
Research
Research
Lisa Saldana, PhD
Mary M. McKay, PhD
Oregon Social Learning Center
New York University
McSilver Institute for Poverty Policy & Research Meghann Kelly Sallee, MA
Silver School of Social Work The University of Alabama
Department of Psychology
Laura K. Murray, PhD
Johns Hopkins Bloomberg School of Catherine DeCarlo Santiago, PhD
Public Health Loyola University Chicago
Department of Mental Health Department of Psychology
Pamela Myles, MPhil Sonja K. Schoenwald, PhD
University of Reading, Berkshire, UK Medical University of South Carolina
School of Psychology and Clinical Language Department of Psychiatry and Behavioral
Sciences Sciences, Charleston, South Carolina

Contributors • xiii
Roz Shafran, PhD
University College, London
Institute of Child Health
Rachel Baden Sherrill, PhD
University of Pennsylvania
Perelman School of Medicine
Department of Psychiatry
Scott Shimabukuro, PhD
State of Hawaii Child and Adolescent Mental
Health Division
Clinical Services Office
Lesley Slavin, PhD
State of Hawaii Child and Adolescent Mental
Health Division
Clinical Services Office
Bradley D. Stein, MD, PhD
RAND Corporation and
University of Pittsburgh
Department of Psychiatry
Sara Stromeyer, MA
The University of Alabama
Department of Psychology
xiv • C ontributors
Anna Swan, MA
Temple University
Department of Psychology
Kristin Pontoski Taylor, PhD
University of Pennsylvania
Department of Psychiatry
Pamela Vona, MA
UCLA
Center for Health Services and Society
Department of Psychiatry
Chiaying Wei, MA
Temple University
Department of Psychology
Dara C. Weiss, MA
New York University
Department of Child and Adolescent Psychiatry
Nathaniel J. Williams, MSW
University of Tennessee
Children’s Mental Health Services
Research Center
College of Social Work
PART ONE
Introduction
 1
The Promise of Evidence-Based
Practices in Child and Adolescent
Mental Health
P H I L I P C . K E N DA L L A N D R I N A D S . B E I DA S
THE PUSH for evidence-based practices has
dominated the mental health and health care are-
nas for more than a decade. Conversations among
professionals who provide mental health services
for youth have included both support and oppo-
sition to this position. On the one hand, there
is a plethora of discourse indicating widespread
support for the need to provide the best avail-
able services for youth in need, delivered through
the provision of evidence-based practices. On
the other hand, there are also opponents to this
viewpoint, primarily arguing that evidence-based
practices developed in research settings may not
fit the context of community providers. A gap
already existed between research and practice,
and the push for evidence-based practices has
further widened the rift between divisions in the
mental health field. These divisions include type
of provider (e.g., psychologist, social worker) and
theoretical orientation (e.g., cognitive-behavioral,
psychodynamic). These divisions, coupled with a
lack of infrastructure providing guidance on best
practices, create a thorny challenge for the field.
For example, pharmacotherapy (medications)
receives support from the pharmaceutical indus-
try—an enterprise that provides advertisements,
sales representatives, and free samples. No such
comparable infrastructure exists for stakehold-
ers examining and finding evidence in support of
psychological treatments: Where do stakeholders
turn when they want to identify and promote best
practices? For decades, the findings of the benefi-
cial effects of psychological treatments for youth
were published in scholarly journals read by other
researchers, often of like mind, but did not reach
those stakeholders who could implement these
treatments in community settings.
The evidence-based practice movement in
mental health can be attributed to several forces,
including the emphasis on evidence-based medi-
cine in health care, an American Psychological
Association (APA) task force formed to identify
• 3
evidence-based practices, and even insurance
companies that, with financial interest, sought
to identify the effective and efficient ways to
provide coverage for their insured with mental
health needs. These efforts gave new and needed
airtime to highlight the benefits of psychological
treatments that work. Mental health researchers,
concurrently, took a more visible stand when
espousing the benefits of psychological treat-
ments, the relative merits of psychological as
compared with pharmacological treatments and,
based on the data, the relative merits of some psy-
chological treatments as compared with others.
Further, they began to consider how to dissemi-
nate the message of the promise of evidence-based
practices to a wider audience.
Gathering the data of which treatments work,
sharing the information, and taking a more pro-
nounced stand was a first step. But like-minded
mental health professionals speaking to each
other were only modestly successful in bringing
evidence-based practices to those in need. The
emergence of dissemination and implementation
(DI), as a focus and as a topic of scientific study,
takes the next step, and allows us to achieve the
promise of evidence-based practices.
A number of evidence-based mental health
treatments for youth have been developed
(see Kendall, 2012) and the prior work docu-
ments that laudable progress has been made.
Unfortunately, the promise of these treatments
will remain largely unrealized if the majority of
youth in the community cannot access these ser-
vices (President’s New Freedom Commission on
Mental Health, 2003). Estimates suggest that it
can take up to 17 years for evidence-based treat-
ments to make their way from research to practice
(Balas & Boren, 2000). This lag is unacceptable.
One of the biggest challenges facing the mental
health field is the dissemination and implementa-
tion of evidence-based practices from the world
of research to the community settings where the
services are provided (McHugh & Barlow, 2010).
Fortunately, a growing interest in implementation
research, the “scientific study of methods to pro-
mote the systematic uptake of research findings
and other evidence-based practices into routine
practice . . . to improve the quality and effective-
ness of health services” (Eccles & Mittman,
4 • I nt r o d uction
2006), sparks a new energy to realize the promise
of evidence-based practices.
The field of implementation science is still
in its infancy, and one of the initial tasks is to
achieve some terminology clarity (McKibbon
et al., 2010). Given this need, we define a few key
terms that are used frequently within this book.
Evidence-based practices (EBPs) refer to the
provision of psychosocial treatments supported
by the best scientific evidence while also taking
into account clinical experience and client pref-
erence (American Psychological Association,
2005). Empirically supported treatments (ESTs)
refer to specific psychological interventions
that have been evaluated scientifically with real
patients and independent evaluators (e.g., a ran-
domized controlled trial [RCT]) and then rep-
licated by others (Chambless & Hollon, 1998).
Dissemination refers to the purposeful distribu-
tion of relevant information and materials to
mental health providers and implementation refers
to the adoption and integration of EBPs (hereaf-
ter including ESTs) into practice (Lomas, 1993).
Dissemination and implementation are best
when they occur in tandem: Both are needed to
influence systemic change (Proctor et al., 2009).
The promise of the DI of EBP can be realized
for a number of stakeholders, including policy
makers, researchers, clinicians, administrators,
and consumers. Policy makers can benefit by
enacting system-level changes that transform
community mental health care. For example,
in the city of Philadelphia, exemplar efforts are
underway to implement EBPs on a large scale (see
Beidas et al., 2013). Researchers and treatment
developers benefit when their work is dissemi-
nated and implemented in community settings
rather than languishing on shelves. Clinicians
benefit by improving their practice, and adminis-
trators benefit by having agencies that provide the
most effective services. Finally, consumers benefit
as they receive the treatments that are most likely
to provide the greatest rate of improvement in
symptoms and return to adaptive functioning. All
of these stakeholders comprise the intended audi-
ence for this book.
We set out to provide cutting-edge knowledge
in DI as it relates to EBPs in child and adolescent
mental health. We arranged this book in four
sections: (a) background information relevant to
DI, (b) system-level approaches to implementing
EBPs for youth in community mental health set-
tings, (c) approaches to implementing EBPs for
youth in schools, and (d) approaches to imple-
menting EBPs for youth using technology.
In Part Two, general issues and themes rel-
evant to DI of EBPs for youth are discussed. The
models, theories, and frameworks that guide
DI are considered first (Chambers, ­chapter 2).
Measurement issues (Proctor, Powell, & Feeley,
­chapter 3) and capturing fidelity in community
settings (Schoenwald, Chapman, & Garland,
­chapter 4) are discussed next. Organizational
(e.g., culture and climate; Williams & Glisson,
­chapter 5) and leadership considerations (Aarons,
Farahnak, & Ehrhart, ­chapter 6) are addressed,
followed by considerations of how best to close
the research to training gap in DI science (Frazier,
Bearman, Garland, & Atkins, c­ hapter 7).
We next highlight exemplars of implement-
ing EBPs for youth in various settings (i.e.,
community mental health, schools) and inno-
vative methodologies (i.e., technology). In
Part Three, exemplars of implementation from
system-level approaches are provided, including
practice-research partnerships (Chamberlain &
Saldana, c­hapter 8), implementation of EBPs
for youth using state-wide systems as a labo-
ratory (Nadeem and colleagues, ­ chapter 9),
implementation of EBPs for youth in England
(Shafran, Fonagy, Pugh, & Myles, ­chapter 10),
global DI in low- and middle-income countries
(Murray, Dorsey, & Lewandowski, c­ hapter 11),
and Building and Sustaining an Evidence-Based
Service System in Hawaii (Nakamura, Slavin,
Shimabukuro, & Keir, ­chapter 12).
In Part Four, exemplars of implementation
of EBPs for anxiety for youth in schools (Wei
and colleagues, c­hapter 13), disruptive disor-
ders (Boxmeyer and colleagues, c­hapter 14),
autism (Locke, Kratz, Reisinger, & Mandell,
chapter 15), depression (Benjamin and col-
­ evidence-based psychological treatments. A review
leagues, ­chapter 16), and trauma (Kataoka and
colleagues, ­chapter 17) are highlighted.
In Part Five, the use of technology takes the
forefront, with exemplars of implementation
of EBPs for youth using technology for anxi-
ety (Khanna, Kerns, & Carper, ­chapter 18) and
The Promise of Evidence-Based Practices in Child and Adolescent Mental Health
early child behavior problems (Elkins & Comer,
­chapter 19).
The field of implementation science is alive
and flourishing. With a special eye toward the DI
of EBPs for youth, we hope that this book will
guide the next needed steps toward progress and
will be consumed by a wide range of stakeholders.
REFERENCES
American Psychological Association. (2005).
American Psychological Association
statement: Policy statement on evidence-based
practice in psychology. Retrieved from http://
www.apapracticecentral.org/ce/courses/
ebpstatement.pdf
Balas, E. A., & Boren, S. A. (2000). Managing
clinical knowledge for healthcare improvement.
In J. Bemmel & A. T. McCray (Eds.), Yearbook
of Medical Informatics: Patient-Centered Systems
(pp. 67–70). Stuggart, Germany: Schattauer
Verlagsgesellschaft mbH.
Beidas, R. S., Aarons, G. A., Barg, F., Evans, A., Hadley,
T., Hoagwood, K., . . . Mandell, D. S. (2013). Policy
to implementation: Evidence-
based practice in community mental health-
study protocol. Implementation Science, 8, 38.
doi:10.1186/1748-5908-8-38
Chambless, D. L., & Hollon, S. D. (1998). Defining
emipirically supported therapies. Journal of
Consulting and Clinical Psychology, 66(1), 7–18.
Eccles, M. P., & Mittman, B. S. (2006). Welcome to
implementation science. Implementation Science, 1,
1. doi:10.1186/1748-5908-1-1
President’s New Freedom Commission on Mental
Health. (2003). Report of the President’s New Freedom
Commission on Mental Health. Washington, DC.
Kendall, P. C. (Ed). (2012). Child and Adolescent
Therapy: Cognitive-Behavioral Procedures (4th ed.).
New York, NY: Guilford Press.
Lomas, J. (1993). Diffusion, dissemination, and
implementation: Who should do what? Annals of
the New York Academy of Sciences, 703, 226–237.
doi:10.1111/j.1749-6632.1993.tb26351.x
McHugh, R. K., & Barlow, D. H. (2010). The
dissemination and implementation of
of current efforts. American Psychologist, 65(2),
73–84. doi:10.1037/a0018121
McKibbon, K. A., Lokker, C., Wilczynski, N. L.,
Ciliska, D., Dobbins, M., Davis, D. A., . . . Straus, S.
E. (2010). A cross-sectional study of the number
and frequency of terms used to refer to knowledge
translation in a body of health literature in
• 5
 2006: A tower of Babel? Implementation Science, 5,
16. doi:10.1186/1748-5908-5-16
Proctor, E. K., Landsverk, J. A., Aarons, G. A.,
Chambers, D., Glisson, C., & Mittman, B. S.
(2009). Implementation research in mental health
6 • I nt r o d uction
services: An emerging science with conceptual,
methodological, and training challenges.
Administration and Policy in Mental Health and
Mental Health Services Research, 36, 24–34.
doi:10.1007/s10488-008-0197-4
PART TWO
Background
 2
Guiding Theory for Dissemination
and Implementation Research
A Reflection on Models Used in Research and Practice
DAV I D A . C H A M B E R S
IN THE past several decades, the field of dissemi-
nation and implementation (DI) research has
gone from an amorphous and disparate collection
of observations of the inability of effective inter-
ventions to be successfully implemented within
clinical and community settings to an identifiable
advancement of knowledge on tools, strategies,
and measures effectively improving the uptake of
a range of scientific discoveries within real-world
contexts. The growth of DI research has come
in the form of targeted funding opportunity
announcements from federal agencies, states, and
foundations (e.g., Glasgow et al., 2012), work-
shops and conferences (e.g., Chambers, 2008;
Office of Behavioral and Social Sciences Research
websites; Seattle Implementation Research
Collaborative), training opportunities (e.g.,
Gonzales, Handley, Ackerman, & O’Sullivan,
2012; Meissner et al., 2013), the founding of
the Implementation Science journal, and recog-
nition among many in the biomedical research
community that innovation without uptake yields
limited benefit to public health (Woolf, 2008).
The progress of the field can be seen not
only through empirical contributions of a range
of observational and experimental studies, but
through advances in theory and conceptual
frameworks mapping processes key to effective
DI. This chapter provides guidance as to how
researchers can use theories, models, and concep-
tual frameworks (words often used interchange-
ably) to support studies of dissemination and
implementation research in health. As the field
has advanced, the use of theory to guide research
questions, designs, and measures has expanded.
TERMS AND DEFINITIONS
Language remains a particular challenge within
DI, wherein investigators often choose disparate
terms to cover similar constructs and may use the
same terms to convey clearly distinct concepts.
• 9
To avoid adding to this confusion, it is important
to lay out working definitions for the key con-
cepts that are employed. In National Institutes
of Health (NIH) program announcements, we
define our use of the terms “dissemination” and
“implementation” as follows:
“Dissemination is the targeted distribu-
tion of information and intervention materials
to a specific public health or clinical practice
audience. The intent is to spread (scale up)
and sustain knowledge and the associated
evidence-based interventions.”
“Implementation is the use of strategies to
adopt and integrate evidence-based health inter-
ventions and change practice patterns within
specific settings.” (Department of Health and
Human Services, 2013)
For this chapter, the key issue is what term best
encapsulates the range of theoretical and concep-
tual approaches to best represent key dissemina-
tion and implementation processes. Investigators
in the field use a range of terms, with “theory” and
“conceptual framework” frequently employed, as
defined here:
Theory: A plausible or scientifically
acceptable general principle of body of
principles offered to explain phenomena
(Merriam-Webster, 2013)
Conceptual Framework: A type of interme-
diate theory that attempt to connect to all aspects
of inquiry; can act like maps that give coherence
to empirical inquiry (Wikipedia, 2013).
Classically, one may consider that a theory is
more likely to specify relationships between dif-
ferent concepts (typically represented by arrows)
whereas frameworks may be more likely designed
as a guide for what concepts to consider. In our
prior search this distinction was at best artificial,
wherein there seemed to be limited consistency
as to how the developers chose which term to use.
Good arguments can be made for the use of either
of these terms, as well as the hybrid term of “theo-
retical framework.” For this chapter, as in other
places, we use the term “model’ (Tabak, Khoong,
Chambers, & Brownson, 2012, 2013).
10 • B ac k gro u nd
Model: A description of analogy used to
help visualize something that cannot be directly
observed (Merriam-Webster, 2013)
“Model” is used to enable the casting of a
wide net in terms of strategies to identify and
link different concepts to explain, predict, and
investigate a range of dissemination and imple-
mentation research issues. An initial search of
over 100 models (with a final tally of 61) found
that not all models directly arose out of theory;
some came from observation, and each had differ-
ences in focus and in scope (Tabak et al., 2013).
This chapter extends upon previous work
(Tabak et al., 2012) that reviewed a range of mod-
els that are specifically relevant to the conduct of
dissemination and implementation research. The
chapter complements other formative papers that
more extensively and exhaustively review a range
of theoretical approaches (e.g., Damschroder
et al., 2009; Greenhalgh, Robert, MacFarlane,
Bate, & Kyriakidou, 2004) and presents examples
of how dissemination and implementation mod-
els are useful to specific research questions. The
examples are not meant to advocate the use of spe-
cific models—indeed, comparative analysis of the
relative merits of models is not the intent of this
chapter. Rather, the examples are a summary of
frequently used models, offering some context
of their utility in studying the implementation of
psychosocial interventions for children and ado-
lescents, and encourage the readers to ensure con-
vergence between the phenomenon under study,
the research questions targeted within, and helpful
theoretical grounding for the investigation.
The chapter also reflects on the utility of mod-
els not solely for use in research study design and
execution, but as a way of understanding the com-
plex processes of dissemination and implementa-
tion among a variety of stakeholders. The models
can help decision makers within practice and
community settings understand the multiple fac-
tors that may affect the success of introducing and
sustaining a new intervention with the local con-
text. In addition, and distinct from other reviews
of multiple models, the chapter presents a brief
discussion of a potential heuristic for selecting an
appropriate model for a research study or for use
in local decision making and evaluation, that may
be helpful as investigators and practitioners frame
research and practice initiatives.
CAVEATS RELATED TO THE
USE OF DISSEMINATION
AND IMPLEMENTATION
MODELS
Prior to the discussion of specific models used
in DI research and practice, it is helpful to recog-
nize the limitations of the models to effectively
govern an implementation strategy, or to predict
implementation outcomes on the basis of key
constructs included within a specific model (see
Powell et al., 2012; Proctor et al., 2009). Although
it would be helpful if the available models pro-
vided a “turn-key” solution to DI challenges in
research and practice, the complexity of these
processes and the variation that exists within
child and adolescent service systems outpace
our ability to parsimoniously represent the field
optimally. Rather, one must keep in mind several
caveats, which should not detract from the util-
ity of the models but instead ensure appropriate
expectations for their use. The following are nec-
essary to keep in mind throughout the remainder
of the chapter:
1. There is no comprehensive model suffi-
ciently appropriate for every study or program.
Some of the models can be described as
“meta-models” (e.g. Damschroder et al., 2009,
Greenhalgh et al., 2004), in that they attempt to
pull in the features of a range of models to portray a
more complete picture of the complex field of dis-
semination and implementation. Although there
are strengths to this approach, the true complexity
of the field means that even these more compre-
hensive approaches have limitations. In addition,
the varied research questions asked exist within
unique contexts, making models created for one
purpose fall short of universality. A quote from
George Box, is apt: “All models are wrong; some
are useful” (Box & Draper, 1987). This chapter
reflects on the utility of different models for dif-
ferent studies and is an aid in the selection process.
2. Not all models are well operationalized.
Guiding Theory for Dissemination and Implementation Research
As the field has matured, new measures have
taken on the operationalization of constructs pre-
viously investigated through more exploratory
methods. Specific measures (e.g., Evidence Based
Practice Attitude Scale, Aarons, 2004; Stages
of Implementation Completion, Chamberlain,
Brown, & Saldana, 2011) have emerged to add
robustness, rigor, and reproducibility to DI
research. However, many components of mod-
els remain without valid and reliable scales. This
chapter notes that model selection should take
into account the degree to which the components
of the model can be sufficiently studied within a
given research project.
3. Models should be considered dynamic.
The current state of the field is of consistent
growth and new insights. For every existing
model, one can appropriately question whether
the specific form of the model effectively and
comprehensively accounts for the complexity of
the phenomenon under investigation. There is
great scope for refinement and extension of most
models that have been proposed. This chapter
suggests broad opportunity to view chosen mod-
els as dynamic, with hopes that each study can
bring both empirical and theoretical contribu-
tions to grow the field.
CONNECTING
DISSEMINATION AND
IMPLEMENTATION MODELS
TO PSYCHOTHERAPEUTIC
INTERVENTIONS
There are meaningful parallels between psycho-
logical therapy and DI research. Both are predi-
cated on change mechanisms. Psychological
therapy, whether focused on cognition, behavior,
or some combination, can be seen as an organized
approach that results in change for the betterment
of an individual’s mental health. DI strategies are
also about trying to change cognitive processes
as well as behavior, both at the individual and
system level. One may target individual demand
for a given intervention, be it a client or a prac-
titioner. One may be focusing on changing clini-
cal practice of a specific therapist, seeking to shift
• 11
from the use of a prior therapeutic approach with
questionable benefit to one with more evidence
to support it.
Behavior change in DI may occur at the sys-
tem level. Models may focus on how to change
an organizational level construct (e.g., culture,
climate, structure; see Chapter 5), or to more
comprehensively enable a group of clinicians
and staff to deliver a new intervention. Many of
the models focus on this level of change, with the
model referencing an aggregation of individual
behavior change approaches at an organizational
or systemic level.
Clearly, the expertise of many clinicians and
researchers related to behavior change forms both
the basis of the evidence-based interventions and
the DI strategies that researchers are develop-
ing and testing. Child and adolescent therapy
researchers are in an advantageous position to
adapt their knowledge of behavior change to the
complex problems of DI research.
THE RELEVANCE OF MODELS
TO RESEARCH, PRACTICE,
AND POLICY
Models are often assumed to be for the research
community to guide the development and exe-
cution of a research study, but models can be
particularly helpful across the boundaries of
research, policy, and practice. Researchers can
anchor key research questions, data collection,
and analytical strategies to the components of a
model, and demonstrate the value of a proposed
study on both empirical and theoretical grounds.
Empirically, models can be used to identify rele-
vant streams of data that allow inference of causal
relationships between components of a model
and the key health or system outcomes that the
effort seeks to improve. Theoretically, the model
may be validated, extended, or even disproved,
offering necessary testing and refinement of con-
cepts key to understanding the field. The assump-
tion that models are solely useful for research is
inaccurate.
Models are relevant to those engaged in
implementation practice, particularly for psy-
chotherapy interventions. Data exist to sug-
gest the generally underwhelming degree to
12 • B ac k gro u nd
which psychotherapy training leads to long-term
delivery of effective interventions (Beidas &
Kendall, 2010), and underscore the limitations
of traditional models of evidence dissemina-
tion. From early metaanalyses of clinical practice
change (e.g., Bero et al., 1998; Davis, Thompson,
Oxman, & Haynes, 1995; Grol, 1997), we have
learned of the limited effectiveness of passive
implementation efforts, like didactic training,
and the poor impact of any single change strat-
egy (Oxman, Thomson, Davis, & Haynes, 1995).
Implementers of psychotherapy can use mod-
els to help develop comprehensive strategies to
install interventions within settings, and gain a
sharper understanding of the components of the
local context that should be attended to and how
additional influences will likely impact the suc-
cess of their efforts.
Practitioners use models to understand the
likely investments needed to support the intro-
duction of new interventions within their service
delivery systems. Rather than making assump-
tions that the decision to adopt is the major deter-
minant of implementation success, practitioners
can find models helpful in providing a template
for what to expect well into implementation and
sustainability. Practitioners can also make great
contributions to the evolution of models, iden-
tifying areas in which models do not effectively
represent key components of implementation in
their settings, and clarifying the utility of different
assumptions that models intuit.
Models are relevant to decision maker at every
level of the political system. Local, state, and fed-
eral policy makers can use models to identify
characteristics of initiatives that could best pro-
mote the uptake of effective practices. Models
can be used to predict the impact of different
policy options on meaningful outcomes to those
whom the decision maker represents, and models
can identify key data to be collected in an ongo-
ing basis to inform planning and decision making
over time. The long-term nature of some models
may be of particular value to some decision mak-
ers in justifying multiyear initiatives, whereas
other models may more readily inform the initial
rollout of a specific program. Like practitioners,
policy makers can also serve to assess and improve
the existing models. Feedback on the relevance of
models to decisions of scale-up, adaptation, and
discontinuation of evidence-based practices can
improve the rigor, relevance, and salience of the
models.
Although models can be helpful to a range of
stakeholders, it will likely be the case that an indi-
vidual model will be of varying interest to a given
audience. Indeed, in the Tabak et al. paper, which
concentrated primarily on models for research
use, we found a number of additional mod-
els that were specifically intended for program
implementers, offering a step-by-step “how-to”
approach to guide implementation. These mod-
els were developed for specific efforts to imple-
ment evidence-based interventions and could be
particularly helpful for practitioners and other
decision makers. The majority of the models in
our broadest search did seem specifically tailored
for research, but it is worth recognizing that even
among those models there is variable applicabil-
ity to a given situation.
A SAMPLING OF REACH, EFFECTIVENESS,
DISSEMINATION AND ADOPTION,
IMPLEMENTATION MODELS IMPLEMENTATION,
Although we do not intend to provide a ranking
MAINTENANCE
system for the use of specific models, we do offer
guidance around the selection and use of models
in the context of DI research for child and adoles-
cent psychotherapeutic interventions. A few com-
monly used models are selected and described in
terms of the constituent parts and how they can
be used in framing different DI questions.
Models can be grouped, as in Tabak et al.
(2012): construct flexibility, dissemination
and/or implementation (D/I), and the lev-
els of the socioecological framework construct
flexibility describe whether each model can be
categorized as being loosely defined (broad)
or detailed as in a step-by-step process (opera-
tional). Dissemination and/or implementation
refers to whether the model is primarily targeting
the spread of interventions via planned commu-
nication strategies (dissemination) or focused on
integrating interventions within a setting (imple-
mentation). The socioecological framework
(SEF) criteria, which stems from a multilevel
framework ranging from individual, organization,
Guiding Theory for Dissemination and Implementation Research
community, system, or policy, determines at
which level of the framework the model operates.
Of the 61 models examined, we found that
each of the five categories for the construct flexi-
bility and DI scales were populated by at least four
models. In addition, we found the models spread
across all levels of the SEF, with the majority of
the models including community and/or orga-
nizational levels, and every model operating at
more than one level. Only eight models touched
on the policy level of the SEF (Tabak et al., 2012).
The models included here are chosen to
exemplify the varied characteristics of the mod-
els according to the sorting criteria we used. The
specific examples are extracted from a more com-
prehensive table in a previous paper (Tabak et al.,
2012). The inclusion of the fifth model offers a
representative “emergent” model that was spe-
cifically developed within child and adolescent
mental health services research. The criteria of the
models included are provided in Table 2.1.
The RE-AIM model, first conceptualized by
Glasgow, Vogt, and Boles (1999), conceptual-
izes five different factors as determining the
public health impact of a specific intervention.
The five factors—reach, effectiveness, adoption,
implementation, maintenance—are posited as
dimensions that inform both the scale through
which an intervention has been used as is ben-
eficial (“reach,” “effectiveness”) and the success
and durability with which it is embedded within
specific settings (“adoption,” “implementation,”
and “maintenance”). It is presented as an evalua-
tive framework, in which each dimension is mul-
tiplied to derive an overall population or public
health impact.
RE-AIM is frequently used by DI research-
ers as an organizing principle for specific ques-
tions that focus on the process of implementing
a specific intervention within multiple settings,
focusing on the pathway from initial decision
to adopt, early implementation of the interven-
tion, and the degree to which the intervention is
• 13
Table 2.1 A Listing of Sample Models According to the Tabak et al. Review Categories

MO D EL D AND/OR I CO NST RUCT SE F L E VE L * * R E FE R E NCE

FL E X I BI L IT Y*

The RE-AIM*** D and I equally 4 C, O, I Glasgow et al., 1999

Framework
Diffusion of D-only 1 C, O, I Rogers, 1995
Innovations
Consolidated I-Only 4 C, O Damschroder et al., 2009
Framework for
Implementation
Research
Interactive Systems D and I Equally 2 S, C, O, I Wandersman et al., 2008
Framework
The EPIS*** Model I-Only 3 S, C, O, I Aarons et al., 2011
* Construct flexibility is a five point scale from Broad (1), which suggest loosely outlined and defined constructs to Operational (5), which provides
detailed, step-by-step instructions for completion of D/I research processes.
** The Socio-Ecological Framework levels include: Individual (I), Organization (O), Community (C), and System (S).
*** RE-AIM, Reach, Effectiveness, Adoption, Implementation, Maintenance; EPIS, Exploration, Adoption/Preparation, Implementation, Sustainment.
More details from this categorization process are presented in extended format at: www.ajpmonline.org.

continued to be delivered as designed at a speci-
fied time point after implementation (referred to
in the model as “Maintenance”). The developers
of the framework have created a large suite of tools
to support researchers using RE-AIM (http://
www.re-aim.org; Dzewaltowski, Glasgow, Klesges,
Estabrooks, & Brock, 2004), and multiple articles
have published accounts of its use to study a range
of interventions within a wide variety of service set-
tings (e.g., Glasgow, Nelson, Strycker, & King, 2006;
Jilcott, Ammerman, Sommers, & Glasgow, 2007).
In particular, RE-AIM is very helpful in think-
ing about therapy interventions because it offers
an easy-to-interpret depiction of the trade-off
between adherence to a specific training manual
and flexibility often represented in practice around
type of practitioner, access and engagement of cli-
ents, and ability of manual-based therapies to be
consistently delivered over time. With concerns
about the concept of “program drift,” whereby
protocol adherence is assumed to degrade among
practitioners (Chambers, Glasgow, & Stange,
2013) over time, the RE-AIM model allows a
clear mapping of the potential degradation of
impact as one assesses each dimension. See Table
2.2 for an example of how RE-AIM can be used to
demonstrate decline in impact.
RE-AIM is identified as an operational frame-
work, in that it specifically outlines a process
through which an intervention is installed within
a given context, whereas its dimensions are flexi-
bly defined to fit most prospective, observational,
and retrospective studies of implementation. It
expects the operationalization of each of the five
dimensions, and provides tools to aid its use.
Furthermore, it allows the inclusion of other
models at different points in process. For exam-
ple, the “I” in RE-AIM could be studied employ-
ing other models that map influences on the initial
implementation of psychotherapy within clinics,
enabling theoretical and empirically driven pre-
dictors of implementation to be investigated.
DIFFUSION OF INNOVATIONS
One of the earliest models to emerge in the DI
research field is Everett Rogers’s Diffusion of
Innovations (DOI). Though it arose out of the
study of seed corn innovation and spread, the
model has been used increasingly to frame the

14 • B ac k gro u nd
Another random document with
no related content on Scribd:
1.C. The Project Gutenberg Literary Archive Foundation (“the
Foundation” or PGLAF), owns a compilation copyright in the
collection of Project Gutenberg™ electronic works. Nearly all the
individual works in the collection are in the public domain in the
United States. If an individual work is unprotected by copyright
law in the United States and you are located in the United
States, we do not claim a right to prevent you from copying,
distributing, performing, displaying or creating derivative works
based on the work as long as all references to Project
Gutenberg are removed. Of course, we hope that you will
support the Project Gutenberg™ mission of promoting free
access to electronic works by freely sharing Project
Gutenberg™ works in compliance with the terms of this
agreement for keeping the Project Gutenberg™ name
associated with the work. You can easily comply with the terms
of this agreement by keeping this work in the same format with
its attached full Project Gutenberg™ License when you share it
without charge with others.

1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside
the United States, check the laws of your country in addition to
the terms of this agreement before downloading, copying,
displaying, performing, distributing or creating derivative works
based on this work or any other Project Gutenberg™ work. The
Foundation makes no representations concerning the copyright
status of any work in any country other than the United States.

1.E. Unless you have removed all references to Project

Gutenberg:

1.E.1. The following sentence, with active links to, or other

immediate access to, the full Project Gutenberg™ License must
appear prominently whenever any copy of a Project
Gutenberg™ work (any work on which the phrase “Project
Gutenberg” appears, or with which the phrase “Project
Gutenberg” is associated) is accessed, displayed, performed,
viewed, copied or distributed:

This eBook is for the use of anyone anywhere in the United

States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it
away or re-use it under the terms of the Project Gutenberg
License included with this eBook or online at
www.gutenberg.org. If you are not located in the United
States, you will have to check the laws of the country where
you are located before using this eBook.

1.E.2. If an individual Project Gutenberg™ electronic work is

derived from texts not protected by U.S. copyright law (does not
contain a notice indicating that it is posted with permission of the
copyright holder), the work can be copied and distributed to
anyone in the United States without paying any fees or charges.
If you are redistributing or providing access to a work with the
phrase “Project Gutenberg” associated with or appearing on the
work, you must comply either with the requirements of
paragraphs 1.E.1 through 1.E.7 or obtain permission for the use
of the work and the Project Gutenberg™ trademark as set forth
in paragraphs 1.E.8 or 1.E.9.

1.E.3. If an individual Project Gutenberg™ electronic work is

posted with the permission of the copyright holder, your use and
distribution must comply with both paragraphs 1.E.1 through
1.E.7 and any additional terms imposed by the copyright holder.
Additional terms will be linked to the Project Gutenberg™
License for all works posted with the permission of the copyright
holder found at the beginning of this work.

1.E.4. Do not unlink or detach or remove the full Project

Gutenberg™ License terms from this work, or any files
containing a part of this work or any other work associated with
Project Gutenberg™.
 1.E.5. Do not copy, display, perform, distribute or redistribute
this electronic work, or any part of this electronic work, without
prominently displaying the sentence set forth in paragraph 1.E.1
with active links or immediate access to the full terms of the
Project Gutenberg™ License.

1.E.6. You may convert to and distribute this work in any binary,
compressed, marked up, nonproprietary or proprietary form,
including any word processing or hypertext form. However, if
you provide access to or distribute copies of a Project
Gutenberg™ work in a format other than “Plain Vanilla ASCII” or
other format used in the official version posted on the official
Project Gutenberg™ website (www.gutenberg.org), you must, at
no additional cost, fee or expense to the user, provide a copy, a
means of exporting a copy, or a means of obtaining a copy upon
request, of the work in its original “Plain Vanilla ASCII” or other
form. Any alternate format must include the full Project
Gutenberg™ License as specified in paragraph 1.E.1.

1.E.7. Do not charge a fee for access to, viewing, displaying,

performing, copying or distributing any Project Gutenberg™
works unless you comply with paragraph 1.E.8 or 1.E.9.

1.E.8. You may charge a reasonable fee for copies of or

providing access to or distributing Project Gutenberg™
electronic works provided that:

• You pay a royalty fee of 20% of the gross profits you derive from
the use of Project Gutenberg™ works calculated using the
method you already use to calculate your applicable taxes. The
fee is owed to the owner of the Project Gutenberg™ trademark,
but he has agreed to donate royalties under this paragraph to
the Project Gutenberg Literary Archive Foundation. Royalty
payments must be paid within 60 days following each date on
which you prepare (or are legally required to prepare) your
periodic tax returns. Royalty payments should be clearly marked
as such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4, “Information
 about donations to the Project Gutenberg Literary Archive
Foundation.”

• You provide a full refund of any money paid by a user who

notifies you in writing (or by e-mail) within 30 days of receipt that
s/he does not agree to the terms of the full Project Gutenberg™
License. You must require such a user to return or destroy all
copies of the works possessed in a physical medium and
discontinue all use of and all access to other copies of Project
Gutenberg™ works.

• You provide, in accordance with paragraph 1.F.3, a full refund of

any money paid for a work or a replacement copy, if a defect in
the electronic work is discovered and reported to you within 90
days of receipt of the work.

• You comply with all other terms of this agreement for free
distribution of Project Gutenberg™ works.

1.E.9. If you wish to charge a fee or distribute a Project

Gutenberg™ electronic work or group of works on different
terms than are set forth in this agreement, you must obtain
permission in writing from the Project Gutenberg Literary
Archive Foundation, the manager of the Project Gutenberg™
trademark. Contact the Foundation as set forth in Section 3
below.

1.F.

1.F.1. Project Gutenberg volunteers and employees expend

considerable effort to identify, do copyright research on,
transcribe and proofread works not protected by U.S. copyright
law in creating the Project Gutenberg™ collection. Despite
these efforts, Project Gutenberg™ electronic works, and the
medium on which they may be stored, may contain “Defects,”
such as, but not limited to, incomplete, inaccurate or corrupt
data, transcription errors, a copyright or other intellectual
property infringement, a defective or damaged disk or other
medium, a computer virus, or computer codes that damage or
cannot be read by your equipment.

1.F.2. LIMITED WARRANTY, DISCLAIMER OF DAMAGES -

Except for the “Right of Replacement or Refund” described in
paragraph 1.F.3, the Project Gutenberg Literary Archive
Foundation, the owner of the Project Gutenberg™ trademark,
and any other party distributing a Project Gutenberg™ electronic
work under this agreement, disclaim all liability to you for
damages, costs and expenses, including legal fees. YOU
AGREE THAT YOU HAVE NO REMEDIES FOR NEGLIGENCE,
STRICT LIABILITY, BREACH OF WARRANTY OR BREACH
OF CONTRACT EXCEPT THOSE PROVIDED IN PARAGRAPH
1.F.3. YOU AGREE THAT THE FOUNDATION, THE
TRADEMARK OWNER, AND ANY DISTRIBUTOR UNDER
THIS AGREEMENT WILL NOT BE LIABLE TO YOU FOR
ACTUAL, DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE
OR INCIDENTAL DAMAGES EVEN IF YOU GIVE NOTICE OF
THE POSSIBILITY OF SUCH DAMAGE.

1.F.3. LIMITED RIGHT OF REPLACEMENT OR REFUND - If

you discover a defect in this electronic work within 90 days of
receiving it, you can receive a refund of the money (if any) you
paid for it by sending a written explanation to the person you
received the work from. If you received the work on a physical
medium, you must return the medium with your written
explanation. The person or entity that provided you with the
defective work may elect to provide a replacement copy in lieu
of a refund. If you received the work electronically, the person or
entity providing it to you may choose to give you a second
opportunity to receive the work electronically in lieu of a refund.
If the second copy is also defective, you may demand a refund
in writing without further opportunities to fix the problem.

1.F.4. Except for the limited right of replacement or refund set

forth in paragraph 1.F.3, this work is provided to you ‘AS-IS’,
WITH NO OTHER WARRANTIES OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR
ANY PURPOSE.

1.F.5. Some states do not allow disclaimers of certain implied

warranties or the exclusion or limitation of certain types of
damages. If any disclaimer or limitation set forth in this
agreement violates the law of the state applicable to this
agreement, the agreement shall be interpreted to make the
maximum disclaimer or limitation permitted by the applicable
state law. The invalidity or unenforceability of any provision of
this agreement shall not void the remaining provisions.

1.F.6. INDEMNITY - You agree to indemnify and hold the

Foundation, the trademark owner, any agent or employee of the
Foundation, anyone providing copies of Project Gutenberg™
electronic works in accordance with this agreement, and any
volunteers associated with the production, promotion and
distribution of Project Gutenberg™ electronic works, harmless
from all liability, costs and expenses, including legal fees, that
arise directly or indirectly from any of the following which you do
or cause to occur: (a) distribution of this or any Project
Gutenberg™ work, (b) alteration, modification, or additions or
deletions to any Project Gutenberg™ work, and (c) any Defect
you cause.

Section 2. Information about the Mission of

Project Gutenberg™
Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new
computers. It exists because of the efforts of hundreds of
volunteers and donations from people in all walks of life.

Volunteers and financial support to provide volunteers with the

assistance they need are critical to reaching Project
Gutenberg™’s goals and ensuring that the Project Gutenberg™
collection will remain freely available for generations to come. In
2001, the Project Gutenberg Literary Archive Foundation was
created to provide a secure and permanent future for Project
Gutenberg™ and future generations. To learn more about the
Project Gutenberg Literary Archive Foundation and how your
efforts and donations can help, see Sections 3 and 4 and the
Foundation information page at www.gutenberg.org.

Section 3. Information about the Project

Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-
profit 501(c)(3) educational corporation organized under the
laws of the state of Mississippi and granted tax exempt status by
the Internal Revenue Service. The Foundation’s EIN or federal
tax identification number is 64-6221541. Contributions to the
Project Gutenberg Literary Archive Foundation are tax
deductible to the full extent permitted by U.S. federal laws and
your state’s laws.

The Foundation’s business office is located at 809 North 1500

West, Salt Lake City, UT 84116, (801) 596-1887. Email contact
links and up to date contact information can be found at the
Foundation’s website and official page at
www.gutenberg.org/contact

Section 4. Information about Donations to

the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission
of increasing the number of public domain and licensed works
that can be freely distributed in machine-readable form
accessible by the widest array of equipment including outdated
equipment. Many small donations ($1 to $5,000) are particularly
important to maintaining tax exempt status with the IRS.

The Foundation is committed to complying with the laws

regulating charities and charitable donations in all 50 states of
the United States. Compliance requirements are not uniform
and it takes a considerable effort, much paperwork and many
fees to meet and keep up with these requirements. We do not
solicit donations in locations where we have not received written
confirmation of compliance. To SEND DONATIONS or
determine the status of compliance for any particular state visit
www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states

where we have not met the solicitation requirements, we know
of no prohibition against accepting unsolicited donations from
donors in such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot

make any statements concerning tax treatment of donations
received from outside the United States. U.S. laws alone swamp
our small staff.

Please check the Project Gutenberg web pages for current

donation methods and addresses. Donations are accepted in a
number of other ways including checks, online payments and
credit card donations. To donate, please visit:
www.gutenberg.org/donate.

Section 5. General Information About Project

Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could
be freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose
network of volunteer support.

Project Gutenberg™ eBooks are often created from several

printed editions, all of which are confirmed as not protected by
copyright in the U.S. unless a copyright notice is included. Thus,
we do not necessarily keep eBooks in compliance with any
particular paper edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg
Literary Archive Foundation, how to help produce our new
eBooks, and how to subscribe to our email newsletter to hear
about new eBooks.