Research

JAMA Pediatrics | Original Investigation

Effectiveness of the Extension for Community Health Outcomes

Model as Applied to Primary Care for Autism
A Partial Stepped-Wedge Randomized Clinical Trial
Micah O. Mazurek, PhD; Robert A. Parker, ScD; James Chan, MA; Karen Kuhlthau, PhD; Kristin Sohl, MD; for the
ECHO Autism Collaborative

Supplemental content
IMPORTANCE The Extension for Community Health Outcomes (ECHO) model is a widely
adopted technology-based model for training primary care physicians and practitioners
(PCPs) to care for patients with complex conditions. Despite its popularity, to our knowledge,
direct effects of ECHO on clinical practice have not been tested in a large-scale study.

OBJECTIVE To test the effectiveness of the ECHO model as applied to primary care for autism
and whether it resulted in improved clinical practice, knowledge, and self-efficacy regarding
autism screening and comorbidity management.

DESIGN, SETTING, AND PARTICIPANTS Primary care physicians and practitioners were recruited
to participate in a 6-month ECHO Autism program delivered by 1 of 10 academic medical
center sites. A sequential, staggered rollout of ECHO Autism was delivered to 5 cohorts of
participants (15 per site; 2 sites per cohort). Sites were randomized after recruitment to
cohort/start time. Cohorts launched every 3 months. The ECHO Autism program used
videoconferencing technology to connect community-based PCPs with interdisciplinary
expert teams at academic medical centers. There were 148 participants (PCPs [family practice
physicians, pediatricians, nurse practitioners, and physician assistants] providing outpatient
services to underserved children) studied between December 2016 and November 2018.

INTERVENTIONS The 6-month ECHO Autism program included twelve 2-hour sessions
connecting PCP participants with an interdisciplinary expert team. Sessions included
didactics, case-based learning, guided practice, and discussion.

MAIN OUTCOMES AND MEASURES Coprimary outcomes were autism screening practices and
comorbidity management (assessed by medical record review). Secondary outcomes were
knowledge (assessed by direct testing) and self-efficacy (assessed by self-report survey).
Assessments were conducted at baseline, mid-ECHO, post-ECHO, and follow-up (3 months
after ECHO).

RESULTS Ten sites were randomized to 1 of 5 cohorts. Participants were 82% female
(n = 108), 76% white (n = 100), and 6% Hispanic or Latino (n = 8); the median age was 46
years (interquartile range, 37-55 years). Significant changes in autism screening and
treatment of comorbidities in children with autism were not observed. Participants
demonstrated significant improvements in knowledge (9%; 95% CI, 4-13; P < .001) and
self-efficacy (29%; 95% CI, 25-32; P < .001).

CONCLUSIONS AND RELEVANCE The ECHO model was developed to increase access to
high-quality health care for underserved patients with complex conditions. Study results
provide support for the model in improving clinician knowledge and confidence but little
support for achieving practice change.

TRIAL REGISTRATION ClinicalTrials.gov Identifier: NCT03677089 Author Affiliations: Author

affiliations are listed at the end of this
article.
Group Information: The ECHO
Autism Collaborative members
appear at the end of the article.
Corresponding Author: Micah O.
Mazurek, PhD, Curry School of
Education and Human Development,
University of Virginia, 417 Emmet St
JAMA Pediatr. 2020;174(5):e196306. doi:10.1001/jamapediatrics.2019.6306 S, PO Box 400267, Charlottesville, VA
Published online March 9, 2020. 22904 (mazurekm@virginia.edu).

(Reprinted) 1/9
© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ on 11/11/2022

 Research Original Investigation Effectiveness of Extension for Community Health Outcomes for Improving Primary Care for Autism
D
espite rapid advancements in medical knowledge, pa-
tients with complex conditions from rural and under-
served areas face significant barriers in accessing spe-
cialty care.1 The Extension for Community Health Outcomes
(ECHO) model was developed to address this gap by infusing
specialty knowledge into local primary care practices.2 Through
ECHO, groups of community-based primary care physicians
and practitioners (PCPs) connect with a team of experts using
secure videoconferencing and receive best-practice knowl-
edge through didactics, case-based learning, comanage-
ment, and a virtual community of practice.3,4 Since 2003, the
framework has been adopted by more than 500 programs glob-
ally. Originally developed to improve care of hepatitis C, the
model has now been applied to more than 100 complex con-
ditions (ranging from diabetes to addiction).5,6 Despite wide-
spread adoption of the model, evidence of effectiveness re-
mains preliminary. Evidence to date derives from tests of
single-project ECHO clinics rather than from more robust mul-
tisite designs and primarily from self-reported outcomes rather
than direct measures of practice change, as was extensively
summarized for Congress.5 More rigorous tests of the model
are needed to assess its effectiveness in improving the qual-
ity of health care for patients with complex conditions. Au-
tism represents one such increasingly common and complex
condition.
Autism is a neurodevelopmental disorder associated with
high rates of medical and psychiatric comorbidities. 7-9
Increased autism prevalence over the past 2 decades10 has re-
sulted in health care demands that greatly exceed the capac-
ity of specialty centers.11 Children with autism experience
greater unmet health care needs, higher health care costs, and
worse access to medical homes and specialty care than chil-
dren with other special health care needs.12-16 Children with
autism from underserved populations face disproportionate
health care barriers owing to clinician shortages, high costs of
services, and language or cultural barriers.17-19 Poor access to
care results in diagnostic and intervention delays, unman-
aged comorbid conditions, poor health, and reduced quality
of life for children with autism.20-22
Community-based PCPs are ideally positioned to address
these disparities. Primary care, by definition, consists of “in-
tegrated, accessible health care services by clinicians who are
accountable for addressing a large majority of personal health
care needs, developing a sustained partnership with pa-
tients, and prac tic ing in the context of family and
community.”23 Unfortunately, many PCPs lack training in au-
tism and feel unprepared to provide best-practice care.24,25 Al-
though the American Academy of Pediatrics issued clear rec-
ommendations for developmental surveillance and screening
for autism,26,27 physician compliance has been strikingly
poor.28-31 Many PCPs lack knowledge about autism20,28,32 and
lack confidence in managing common co-occurring condi-
tions in children with autism.33 Equipping PCPs with the skills
to provide effective community-based screening and manage-
ment of autism has the potential to enhance outcomes for chil-
dren with autism.
A 2017 pilot study34,35 was conducted to examine the
application of the ECHO model to enhance primary care for
2/9 JAMA Pediatrics May 2020 Volume 174, Number 5 (Reprinted)
© 2020 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ on 11/11/2022
Key Points
Question Is the Extension for Community Health Outcomes
(ECHO) telementoring model effective in improving clinical
practice among primary care clinicians caring for children with
autism?
Findings In this large-scale, multisite partial stepped-wedge study
of the ECHO model as applied to autism, significant changes in
autism screening or treatment of comorbidities were not
demonstrated. However, primary care clinicians demonstrated
significant improvements in knowledge and self-efficacy
immediately following and 3 months after completing the ECHO
program.
Meaning The results provide support for the ECHO model in
improving clinician knowledge and confidence in caring for
patients with autism in primary care practices, but measurable
changes in practice were not observed.
autism. The study consisted of a small sample of 14 PCPs
who attended 12 biweekly ECHO Autism clinics focused on
best-practice medical care for children with autism. The
results were promising, showing significant improvements
in participant self-efficacy and self-reported practice
change. However, objective measures of improvement were
not collected.34 Thus, a more rigorous examination of the
model is needed.
This study was conducted to test the effectiveness of ECHO
Autism in a large and geographically diverse randomized
sample using objective measures of improvement in clinical
practice, while delivering a potentially important educa-
tional intervention to all participants. The specific hypoth-
eses were that participating in ECHO Autism would result in
significant improvements among PCPs in (1) clinical practice,
(2) knowledge, and (3) self-efficacy regarding autism screen-
ing and management of comorbidities.
Methods
Study Design
Ten academic medical centers (eAppendix in Supplement 1)
participating in the Autism Treatment Network and the Autism
Intervention Research Network on Physical Health served as
individual ECHO Autism clinic sites (each training a target
sample of 15 PCP participants). A sequential, staggered roll-
out of the ECHO Autism program was studied in 5 cohorts of
participants (2 sites every 3 months) between December 2016
and November 2018. Sites were randomized to startup order
after recruitment into study. This design allowed us to distrib-
ute limited resources to assess the effect of the ECHO Autism
program across a broad range of sites. Assessments were com-
pleted at 4 points separated by 3-month intervals: Baseline/
pre-ECHO, mid-ECHO, post-ECHO, and a post-ECHO fol-
low-up to assess persistence of effect. Figure 1 shows a
CONSORT diagram of participation in the study. Figure 2 de-
scribes the timeline for data collection. The study was regis-
tered at ClinicalTrials.gov (NCT03677089), and the formal trial
protocols are in Supplement 2.
jamapediatrics.com
 Effectiveness of Extension for Community Health Outcomes for Improving Primary Care for Autism Original Investigation Research

Figure 1. CONSORT Flow Diagram

10 Sites recruited

Cohort 1 Cohort 2 Cohort 3 Cohort 4 Cohort 5

December 2016 to June 2017 March 2017 to September 2017 June 2017 to December 2017 September 2017 to March 2018 December 2017 to June 2018
2 Sites recruited 2 Sites recruited 2 Sites recruited 2 Sites recruited 2 Sites recruited
28 PCPs assessed for 33 PCPs assessed for 34 PCPs assessed for 32 PCPs assessed for 22 PCPs assessed for
eligibility eligibility eligibility eligibility eligibility
28 Enrolled 33 Enrolled 34 Enrolled 32 Enrolled 22 Enrolled

148 Total PCP participants in study

16 Did not complete demographics

132 Completed demographics

15 Unable to arrange clinic visit

117 Clinic visits for baseline medical

record reviews
99 Reviews with available autism
medical records
92 Reviews with available 18- or
24-mo WCV medical records

13 Did not complete any follow-up forms

104 Completed any follow-up forms at

6 mo (end of ECHO training)
74 Reviews with available autism
medical records
90 Reviews with available 18- or
24-mo WCV medical records

PCP indicates primary care physician or practitioner; ECHO, Extension for Community Health Outcomes; WCV, well-child visit.

Figure 2. Planned Data Collection Times Across Cohorts

Cohort 1 Cohort 2 Cohort 3 Cohort 4 Cohort 5

December 1, 2016 T1
3 mo ECHO
March 1, 2017 T2 T1
3 mo ECHO ECHO
June 1, 2017 T3 T2 T1
3 mo ECHO ECHO
September 1, 2017 T4 T3 T2 T1
3 mo ECHO ECHO
December 1, 2017 T4 T3 T2 T1
3 mo ECHO ECHO
March 1, 2018 T4 T3 T2
3 mo ECHO
June 1, 2018 T4 T3
3 mo
ECHO indicates Extension for
September 1, 2018 T4
Community Health Outcomes; T,
time.

Participants geographic region. Inclusion criteria were current practice as

Each of the 10 ECHO Autism sites (9 US and 1 Canadian site) a PCP (ie, family practice physician, pediatrician, nurse prac-
planned to recruit a sample of 15 PCPs from their respective titioner, or physician assistant), providing outpatient ser-

jamapediatrics.com (Reprinted) JAMA Pediatrics May 2020 Volume 174, Number 5 3/9

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ on 11/11/2022

 Research Original Investigation Effectiveness of Extension for Community Health Outcomes for Improving Primary Care for Autism
vices to children, and providing care to underserved popula-
tions (at least 50% of patients were underserved). Only 1 PCP
from a single-practice location was allowed to participate in
the study. Recruitment methods included professional distri-
bution lists, professional meetings, and face-to-face and word-
of-mouth recruitment. Participants received no-cost continu-
ing medical education credit for attendance at each ECHO
Autism clinic, and modest financial incentives were provided
for completion of study measures at each assessment point.
Informed consent followed procedures dictated by the insti-
tutional review board at the specific site; generally verbal con-
sent was considered adequate because the study involved no
more than minimal risk. The study was approved by the insti-
tutional review board at each site and at the Network Coordi-
nating Center.
ECHO Autism Implementation and Procedures
ECHO Autism expert teams were formed at each of the 10 par-
ticipating sites, consisting of at least 5 autism specialists (phy-
sician/autism medical specialist, psychologist, family re-
source specialist, dietician, and parent expert). One site
included an additional Family Resource Specialist because par-
ticipants were practicing in 2 different states. Teams were
trained to fidelity in the ECHO model and in the ECHO Autism
curriculum.34 Training included in-person training and orien-
tation at the ECHO Institute, regular learning network video-
conference meetings, attendance and observation of at least
3 ECHO Autism clinics facilitated by the leadership team, and
facilitation of at least 2 mock clinics with live coaching and feed-
back prior to clinic launch.
Each ECHO autism clinic site provided twice-monthly
2-hour ECHO Autism clinics during a 6-month period. Clinics
were conducted using high-quality secure Health Insurance
Portability and Accountability Act–compliant videoconferenc-
ing technology (Zoom), enabling screen-sharing (for docu-
ment viewing) and real-time interactions among participants
and ECHO Autism expert team members. All clinics followed
the established ECHO Autism curriculum, which included a
standard set of didactic presentation slides and content, case
presentation forms, and materials.34 Each clinic consisted of
a brief 15-minute didactic presentation, 2 PCP-generated case
presentations, and collaborative discussion among experts and
participants. Didactic content focused on guidelines and al-
gorithms for evidence-based best-practice medical care for chil-
dren with autism, with particular emphasis on autism screen-
ing and identification and on medical management of comorbid
conditions 27,36-41 (didactic materials are available on re-
quest). The intent of the program was for participants to de-
velop new clinical skills through guided practice and collab-
orative case-based learning while maintaining responsibility
for care of their patients. Implementation fidelity was as-
sessed by study team members at 2 randomly selected ses-
sions per site using a form developed by the University of New
Mexico Project ECHO Team to assess fidelity; adequate fidel-
ity was defined as “agree” or “strongly agree” for at least 80%
of the relevant items.
Unconstrained random allocation was generated in SAS
(SAS Institute Inc) by the senior study statistician after sites
4/9 JAMA Pediatrics May 2020 Volume 174, Number 5 (Reprinted)
© 2020 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ on 11/11/2022
were enrolled by principal investigator. Sites were allocated to
a cluster based on the random allocation.
Some operational problems occurred during the study.
Owing to scheduling issues, 2 sites started 1 month after the
scheduled start of their cohort. This was ignored for the
analysis. Because Canadian guidelines for screening for
autism were not consistent with US guidelines, autism
screening for the Canadian site was excluded from the pri-
mary analysis. Full details of all operational problems are
provided in section 12 of the statistical analysis plan in
Supplement 3.
Outcome Measures
Clinical practice/behavior was assessed through medical rec-
ord review conducted by study staff in each PCP practice. Ap-
propriate autism screening was assessed at 18-month and 24-
month well-child visits for the 30 days prior to the scheduled
date of medical record review. Medical record reviews were
done at baseline (before the start of ECHO, but on rare occa-
sions after the first but before the second ECHO session), at mid-
ECHO (between the sixth and seventh ECHO session), post-
ECHO (within 30 days after the end of the ECHO program), and
for long-term effect (between 3 and 4 months after the end of
the ECHO program). Appropriate autism screening was de-
fined as administering an autism screening tool at 18 and 24
months, as recommended by the American Academy of
Pediatrics27 for the 9 US sites (all other measures included both
US and Canadian sites). Treatment of 4 common autism co-
morbidities (constipation, sleep disturbance, behavior prob-
lems, and anxiety) was assessed for all visits for children with
autism seen within the 60 days prior to the scheduled date of
medical record review. We considered any treatment address-
ing a recognized problem (from a predefined list of appropri-
ate treatment options for each condition, which are shown in
eTable 1 in Supplement 1) as appropriately addressing the prob-
lem. Secondary outcome measures (knowledge, self-
efficacy, and barriers to care) are described further in the trial
protocol in Supplement 2. Self-reported demographic infor-
mation (including age, sex, and race/ethnicity) was collected
to characterize the sample (Table 1).
Power Calculations
Given the complexity of the proposed analysis, power
calculations were based on simulations. The data generation
process allowed for random effects for site, PCP within site,
and nominal period. There was no time trend in the data, al-
though a potential time trend as a fixed effect was included
in the model. Simulations were done for 10 randomly se-
lected seeds (from several different websites and different ran-
dom number tables) at 1000 simulations per seed. The data-
generating process allowed for approximately a 50% intraclass
correlation for the PCP within-group effect, reflecting the pos-
sibility that the effect of ECHO would be correlated within each
site, even with good fidelity to the intervention program. Simu-
lations allowed for varying numbers of patients per PCP
practice.
If PCPs on average had 15 patients with well-child visits
seen in the past month, we would have more than 90% power
jamapediatrics.com
 Effectiveness of Extension for Community Health Outcomes for Improving Primary Care for Autism Original Investigation Research

Table 1. Demographics for mITT Sample

No. (%)
Variable Overall Cohort 1 Cohort 2 Cohort 3 Cohort 4 Cohort 5
PCP participants, No. 132 27 33 29 28 15
Age, median (IQR), y 46 (37-55) 43 (36-55) 49 (45-60) 45 (37-53) 46 (36-55.5) 39 (36-50)
Sex
Female 108 (82) 21 (78) 26 (79) 23 (79) 23 (82) 15 (100)
Race
White 100 (76) 21 (78) 23 (70) 23 (79) 20 (71) 13 (87)
Black or African American 7 (5) 2 (7) 2 (6) 2 (7) 1 (4) 0
Asian 21 (16) 3 (11) 8 (24) 3 (10) 6 (21) 1 (7)
Other/multiracial 4 (3) 1 (4) 0 1 (3) 1 (4) 1 (7)
Ethnicity
Hispanic or Latino 8 (6) 2 (7) 0 2 (7) 4 (14) 0
Specialty
General pediatrician 100 (76) 18 (67) 30 (91) 23 (79) 20 (71) 9 (60)
Internal medicine–pediatrics physician 6 (5) 3 (11) 1 (3) 1 (3) 1 (4) 0
Family medicine physician 7 (5) 1 (4) 1 (3) 2 (7) 1 (4) 2 (13)
Nurse practitioner 15 (11) 5 (19) 1 (3) 2 (7) 3 (11) 4 (27)
Physician assistant 2 (2) 0 0 1 (3) 1 (4) 0
Other 2 (2) 0 0 0 2 (7) 0
Years in practice, median (IQR) 13.5 (6-22.5) 13 (2.5-23) 19 (11-25) 12 (7-17) 13.5 (5.5-25.5) 7 (5-15)
Children seen at practice per y, median (IQR) 3000 3500 3000 2880 2600 1504
(1302-4000) (2079-4500) (1000-4000) (1000-3600) (2000-4200) (700-3700)
Additional training in autism 62 (47) 18 (67) 13 (39) 11 (38) 14 (50) 6 (40)
Type of additional traininga
Content within a graduate course 13 (21) 6 (33) 2 (15) 1 (9) 1 (7) 3 (50)
(not autism-specific)
Specific graduate course on autism 5 (8) 1 (6) 1 (8) 2 (18) 0 1 (17)
Workshop 23 (37) 5 (28) 4 (31) 4 (36) 7 (50) 3 (50)
Conference presentation 17 (27) 4 (22) 3 (23) 2 (18) 6 (43) 2 (33)
Continuing medical education seminar 30 (48) 9 (50) 7 (54) 6 (55) 6 (43) 2 (33)
Webinar 6 (10) 0 1 (8) 4 (36) 0 (0) 1 (17)
Residency rotation during residency 23 (37) 5 (28) 7 (54) 3 (27) 6 (43) 2 (33)
Medical school rotation 7 (11) 3 (17) 2 (15) 1 (9) 1 (7) 0
Practice setting
Academic medical center 21 (16) 7 (26) 2 (6) 3 (10) 5 (18) 4 (27)
Federally qualified health center 13 (10) 3 (11) 1 (3) 1 (3) 2 (7) 6 (40)
Multiple settings 10 (8) 0 4 (12) 4 (14) 2 (7) 0
Solo practice 20 (15) 1 (4) 12 (36) 2 (7) 5 (18) 0
Private group practice 50 (38) 10 (37) 11 (33) 13 (45) 14 (50) 2 (13)
Other 18 (14) 6 (22) 3 (9) 6 (21) 0 3 (20)

Abbreviations: IQR, interquartile range; mITT, modified intent-to-treat; PCP, primary care physician or practitioner.
a
Percentages are of those with any additional training in autism.

to detect an increase from 50% to 60% in autism screening
(α = .025; all tests were 2-sided). If PCPs on average had seen
5 patients with autism in the last 60 days, we would have more
than 90% power to detect an increase from 50% to 65% in
appropriate comorbidity management (α = .025; all tests were
2-sided).
Statistical Methods
Data are summarized as median (interquartile range [IQR]) or
counts. The primary analysis used a generalized linear mixed
model with data from baseline, mid-ECHO, post-ECHO, and
9-month follow-up to assess program effect, with fixed ef-
fects for period (time since start of study) and time in study
(categorical time since PCP participant entered study) includ-
ing site-specific random intercept for each PCP participant. We
accounted for repeated measures within PCP participants with
unstructured covariance. This is a modified version of the stan-
dard model for a stepped-wedge design put forward by Hussey
and Hughes.42 Model estimates for baseline, month 3, month
6, and month 9 are presented are presented with the primary
analysis test of change over 6 months of study. Additional re-
sults test changes from month 6 to month 9 to assess any

jamapediatrics.com (Reprinted) JAMA Pediatrics May 2020 Volume 174, Number 5 5/9

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ on 11/11/2022

 Research Original Investigation Effectiveness of Extension for Community Health Outcomes for Improving Primary Care for Autism

Table 2. Model Estimates at Each Time

Mean (95% CI)

Outcome Month 0 Month 3 Month 6 Month 9
Primary outcomes
Proportion of children screened 0.65 (0.40-0.84) 0.69 0.71 0.63 (0.37-0.84)
appropriately for autism (0.51-0.83) (0.52-0.84)
Proportion of comorbid medical 0.81 (0.67-0.90) 0.76 0.77 0.83 (0.70-0.91)
problems appropriately addressed (0.65-0.85) (0.66-0.86)
in children with autism
Secondary outcomes
Proportion of children screened 0.90 (0.55-0.98) 0.88 0.88 0.90 (0.58-0.98)
appropriately for general (0.64-0.97) (0.64-0.97)
development
Proportion of comorbid medical 0.14 (0.09-0.22) 0.16 0.16 0.18 (0.12-0.26)
conditions identified for each child (0.12-0.22) (0.12-0.22) a
with autisma Among 4 prespecified conditions
Knowledge Score, % 56 (52-61) 62 (58-65) 65 (62-68) 66 (61-70) (anxiety, behavioral problems,
constipation, and sleep disorders).
Self-efficacy Total Scoreb 42 (38-45) 61 (58-63) 70 (68-73) 71 (68-74) b
Ranging from 0 to 100, with higher
Total perceived barriers to care 4.8 (4.3-5.3) 3.7 (3.3-4.1) 2.6 (2.2-3.0) 2.7 (2.2-3.1) scores meaning higher perceived
(of 10 possible)
self-efficacy.

regression after ECHO is completed. Results are reported for
a modified intent-to-treat population that includes all partici-
pants who consented and provided basic demographics at base-
line. Although we initially planned an analysis restricted to par-
ticipants who more closely followed protocol, we agreed with
a reviewer to present mITT analyses because this was better
suited to the study design; results were almost identical be-
tween the 2 analyses. The SAP called for baseline data to be
imputed if more than 5% of data were missing for a primary
outcome. Because less than 5% of baseline data were miss-
ing, no baseline data were imputed. Missing follow-up data had
the baseline carried forward as a conservative estimate (5 PCPs
had autism screening follow-up information imputed, and 15
PCPs had follow-up information imputed for the proportion
of comorbidities addressed outcome). Because we had 2 copri-
mary end points, we used P < .025 to determine statistical sig-
nificance for proportion screened appropriately and propor-
tion of comorbidities addressed appropriately at 6 months;
otherwise P < .05 was considered statistically significant. All
tests were 2-sided.
Role of the ECHO Institute
The study team received training and consultation from ECHO
Institute members on ECHO implementation. Members of the
ECHO Institute were not involved in study design, analyses,
interpretation of results, or decision to publish.
Results
Population, Participation, and Implementation
A total of 148 participants were enrolled across the 10 ECHO
Autism clinic sites, ranging from 11 to 22 per site. See Table 1
for demographic data by recruitment cohort. Of the 129 par-
ticipants (87%) who attended at least 1 session, the median at-
tendance was 10 of 12 sessions (interquartile range [IQR], 8-11).
Regarding implementation, all sessions assessed during ran-
dom fidelity checks demonstrated fidelity to the ECHO model.
Median fidelity at individual clinics was 96% (range, 82%-
100%).
Some demographic information was provided by 132 par-
ticipants (89%), and baseline medical record review was
completed for 118 participants (80%). The number of
medical records available for review at each point varied across
participants and visit type. For the baseline medical record re-
view window, participants had a median of 5 well-child visits
(IQR, 2-9) reviewed for autism screening (18-month or
24-month visits) and 3 visits (IQR, 1-6) with children with au-
tism. Of the 118 participants having a medical record review
attempted at baseline, 25 had no relevant well-child visits
within the 30-day review period, 18 had no visits with a child
with autism within the 60-day review period, and 24 pro-
vided a medical record for a child with autism but did not iden-
tify any of the listed problems. Baseline outcome measures by
cohort are presented in eTable 2 in Supplement 1.
Efficacy of ECHO Autism
Model-based estimates of the outcomes are presented in
Table 2, while effect estimates and statistical significance are
shown in Table 3. Our primary analysis models estimated a
small 6-month increase in autism screening rates from 65%
(95% CI, 40%-84%) to 71% (95% CI, 52%-84%) (P = .63) and a
small 6-month decrease in mean percentage of comorbidities
addressed from 81% (95% CI, 67%-90%) to 77% (95% CI, 66%-
86%) (P = .59). There was substantial improvement in knowl-
edge (9% increase; 95% CI, 4%-13%; P < .001) and overall self-
efficacy (29% increase; 95% CI, 25%-32%; P < .001) and a
substantial decrease in the number of barriers identified by par-
ticipants (2.2 decrease; 95% CI, −2.7 to −1.7; P < .001) (Table 3).
Long-term Effects of ECHO Autism
The significant effects seen in knowledge, overall self-
efficacy, and barriers identified did not subside by month 9,
and there were no significant differences observed from the
6-month post-ECHO visit to the 9-month follow-up visit
(Table 3).

6/9 JAMA Pediatrics May 2020 Volume 174, Number 5 (Reprinted) jamapediatrics.com

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ on 11/11/2022

 Effectiveness of Extension for Community Health Outcomes for Improving Primary Care for Autism Original Investigation Research

Table 3. Model-Based Estimated Treatment Effects

Persistence Between
Change at 6 mo 6 and 9 mo
Treatment Effect, Difference, OR
Outcome OR (95% CI) P Value (95% CI) P Value
Primary outcomes
Proportion of children screened appropriately for 1.29 (0.46 to .63 0.72 (0.42 to .23
autism 3.57) 1.23)
Proportion of comorbid medical problems 0.80 (0.35 to .59 1.40 (0.82 to .22
appropriately addressed in children with autism 1.82) 2.40)
Secondary outcomes
Proportion of children screened appropriately for 0.80 (0.13 to .81 1.18 (0.44 to .73
general development 5.04) 3.16)
Proportion of comorbid medical conditions identified 1.16 (0.69 to .57 1.11 (0.82 to .49
for each child with autisma 1.94) 1.50) Abbreviation: OR, odds ratio.
a
Knowledge Score, Δ (95% CI) 8.54 (4.09 to <.001 0.62 (−2.24 to .67 Among 4 prespecified conditions
12.98) 3.48) (anxiety, behavioral problems,
Self-efficacy Total Score, Δ (95% CI)b 29 (25 to <.001 1 (−1 to .29 constipation, and sleep disorders).
32) 4) b
Ranging from 0 to 100, with higher
Total perceived barriers to care (of 10 possible), −2.20 (−2.74 to <.001 0.02 (−0.37 to .92 scores meaning higher perceived
Δ (95% CI) −1.67) 0.41)
self-efficacy.

population of PCPs. However, their screening rates at base-

Discussion
This study was designed to evaluate the effectiveness of a
widely adopted technology-based continuing education model
(project ECHO) in improving direct clinical care. In what is to
our knowledge the first large-scale prospective multisite study
of this model, we directly tested the effects of ECHO as ap-
plied to primary care for autism. Contrary to predictions, par-
ticipants did not demonstrate statistically significant improve-
ments in either autism screening or in treatment of
comorbidities in children with autism after participation in the
6-month ECHO autism program. However, consistent with pre-
vious studies,5 participants did demonstrate significant in-
creases in both knowledge and self-efficacy as well as signifi-
cant reductions in perceived barriers to caring for children with
autism in their practices.
The ECHO model was developed to increase access to high-
quality health care for underserved patients with complex con-
ditions. The results of this study do not provide strong sup-
port for the model in achieving direct practice change among
PCPs, at least as it relates to the autism screening and comor-
bidity management. However, marked improvements in cli-
nician knowledge and confidence in treating children with au-
tism were observed. Although less direct than the specific
practice behaviors we assessed through medical record re-
view, these outcomes may ultimately enhance overall care and
patient-clinician interactions for a population at high risk for
poor health care experiences and outcomes.11-15
Strengths and Limitations
This study benefitted from a large, multisite, and geographi-
cally diverse sample of PCPs. Additional strengths included ran-
domization at the level of cohort, staggered implementation
of the program across cohorts, and tests of effectiveness un-
der baseline, intervention, and follow-up conditions. There are
also several limitations that should be noted. Participants in
this study had some motivation to improve their care of chil-
dren with autism and may not be representative of the larger
line were generally consistent with 2016/2017 clinician-
reported estimates,43,44 suggesting that they did not differ sub-
stantially from their peers with regard to screening practices
prior to participation in the program.
The ECHO Autism curriculum was delivered for a pre-
defined period (6 months) for the purposes of the research
study. However, typical ECHO implementation is ongoing, with
no clearly defined stopping point. We also prohibited mul-
tiple participants from the same practice to minimize con-
founds; however, traditional ECHO models encourage partici-
pation from multiple clinicians from within a single practice.
We also observed a relatively high rate of dropout, which may
indicate issues with feasibility or acceptability of the model for
some practicing PCPs.
Finally, our measure of comorbidity management may
have been too simplistic. We were unable to assess whether
PCPs were actively screening for each condition or whether
they were noted only when concerns were raised by care-
givers. As such, we were unable to determine the extent to
which comorbid conditions were accurately screened and iden-
tified. Because the medical record review process did not al-
low for determination of best treatment approach for an indi-
vidual patient (because this would have required thorough
clinical review of each case), our measure simply noted
whether the PCP administered 1 or more potentially appropri-
ate treatments when a comorbid condition was observed.
In future studies, a more fine-grained analysis of treatment
strategies would be more helpful in determining whether a
clinician followed a best-practice treatment approach or
algorithm.
Conclusions
In conclusion, the results of this large-scale study provide some
support for the effectiveness of the ECHO model in improv-
ing primary care knowledge and self-efficacy regarding treat-
ment of children with autism but little evidence of practice

jamapediatrics.com (Reprinted) JAMA Pediatrics May 2020 Volume 174, Number 5 7/9

© 2020 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ on 11/11/2022

 Research Original Investigation
change in specific screening and management domains. Fu-
ture research is needed to further evaluate the cost, benefit,
and effectiveness of ECHO. A more comprehensive assess-
ARTICLE INFORMATION
Accepted for Publication: December 12, 2018.
Published Online: March 9, 2020.
doi:10.1001/jamapediatrics.2019.6306
Author Affiliations: Curry School of Education and
Human Development, Department of Human
Services, University of Virginia, Charlottesville
(Mazurek); Biostatistics Center, Department of
Medicine, Massachusetts General Hospital, Boston
(Parker, Chan); Department of Medicine, Harvard
Medical School, Boston, Massachusetts (Parker);
Department of Pediatrics, Harvard Medical School,
Boston, Massachusetts (Kuhlthau); Division of
General Academic Pediatrics, Massachusetts
General Hospital, Boston, Massachusetts
(Kuhlthau); Department of Child Health, University
of Missouri, Columbia (Sohl).
Author Contributions: Dr Parker and Mr Chan had
full access to all of the data in the study and take
responsibility for the integrity of the data and the
accuracy of the data analysis.
Concept and design: Mazurek, Parker,
Kuhlthau, Sohl.
Acquisition, analysis, or interpretation of data:
All authors.
Drafting of the manuscript: Mazurek, Parker, Chan.
Critical revision of the manuscript for important
intellectual content: All authors.
Statistical analysis: Parker, Chan.
Obtained funding: Kuhlthau, Sohl.
Administrative, technical, or material support:
Mazurek, Parker, Kuhlthau, Sohl.
Supervision: Mazurek, Parker, Kuhlthau.
Conflict of Interest Disclosures: Dr Mazurek
reported grants from Health Resources and
Services Administration (HRSA) of the US
Department of Health and Human Services during
the conduct of the study. Dr Parker reported other
support from HRSA and grants from Autism Speaks
during the conduct of the study. Since completing
the study, Dr Parker has become a consultant to the
ECHO Institute, University of New Mexico.
Dr Kuhlthau reported grants from Autism
Intervention Research Network on Physical Health
grant from MCHB HRSA and funding from the
Autism Speaks Foundation during the conduct of
the study. Dr Sohl reported other support from
Quadrant Biosciences, Autism Navigator, the
National Institutdes of Health/National Institute of
Neurological Disorders and Stroke; and Cognoa and
personal fees from university-based grand rounds;
in addition, Dr Sohl had a patent to Autism Ready
Communities pending. No other disclosures were
reported.
Funding/Support: This project was supported by
the Health Resources and Services Administration
of the US Department of Health and Human
Services under cooperative agreement UA3
MC11054–Autism Intervention Research Network
on Physical Health.
Role of the Funder/Sponsor: The funding source
had no role in the design and conduct of the study;
collection, management, analysis, and
interpretation of the data; preparation, review, or
8/9 JAMA Pediatrics May 2020 Volume 174, Number 5 (Reprinted)
© 2020 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ on 11/11/2022
Effectiveness of Extension for Community Health Outcomes for Improving Primary Care for Autism
ment of screening and management practices, patient-
clinician interactions, and patient outcomes would be particu-
larly informative in future studies.
approval of the manuscript; and decision to submit
the manuscript for publication.
Group Information: The members of the ECHO
Autism Collaborative are Evdokia Anagnostou, MD,
University of Toronto, Toronto, Ontario, Canada;
Amanda Bennett, MD, MPH, Children’s Hospital of
Philadelphia, Philadelphia, Pennsylvania; Kristin
Dalope, MD, University of Pittsburgh, Pittsburgh,
Pennsylvania; Susan Hyman, MD, University of
Rochester, Rochester, New York; Heather Johnson,
PsyD, Cincinnati Children’s Hospital Medical Center,
Cincinnati, Ohio; Maya Lopez, MD, University of
Arkansas for Medical Sciences, Little Rock; Beth
Malow, MD, MS, Vanderbilt University Medical
Center, Nashville, Tennessee; Ann Neumeyer, MD,
Lurie Center for Autism, Lexington, Massachusetts;
Karen Ratliff-Schaub, MD, Nationwide Children’s
Hospital, Columbus, Ohio; Robin Steinberg-Epstein,
MD, University of California, Irvine.
Disclaimer: This information or content and
conclusions are those of the author and should not
be construed as the official position or policy of, nor
should any endorsements be inferred by Health
Resources and Services Administration of the US
Department of Health and Human Services or the
US government.
Data Sharing Statement: See Supplement 4.
Additional Contributions: We thank the members
of the Clinical Coordinating Center and Data
Coordinating Center at Massachusetts General
Hospital, and the project staff and hub team
members at each ECHO Autism Collaborative site.
We also gratefully acknowledge all the primary care
clinicians who participated in this study.
Additional Information: This work was conducted
through the Autism Speaks Autism Treatment
Network serving as the Autism Intervention
Research Network on Physical Health.
REFERENCES
1. Cook NL, Hicks LS, O’Malley AJ, Keegan T,
Guadagnoli E, Landon BE. Access to specialty care
and medical services in community health centers.
Health Aff (Millwood). 2007;26(5):1459-1468.
doi:10.1377/hlthaff.26.5.1459
2. Arora S, Thornton K, Murata G, et al. Outcomes
of treatment for hepatitis C virus infection by
primary care providers. N Engl J Med. 2011;364(23):
2199-2207. doi:10.1056/NEJMoa1009370
3. Arora S, Kalishman S, Thornton K, et al.
Expanding access to hepatitis C virus treatment:
Extension for Community Healthcare Outcomes
(ECHO) project: disruptive innovation in specialty
care. Hepatology. 2010;52(3):1124-1133.
doi:10.1002/hep.23802
4. Arora S, Kalishman S, Dion D, et al. Partnering
urban academic medical centers and rural primary
care clinicians to provide complex chronic disease
care. Health Aff (Millwood). 2011;30(6):1176-1184.
doi:10.1377/hlthaff.2011.0278
5. Office of Health Policy; Office of the Assistant
Secretary for Planning and Evaluation (ASPE); US
Department of Health and Human Services. Report
to Congress: current state of technology-enabled
collaborative learning and capacity building models.
report to congress. https://aspe.hhs.gov/pdf-
report/report-congress-current-state-technology-
enabled-collaborative-learning-and-capacity-
building-models. Published March 2019. Accessed
February 6, 2020.
6. Project ECHO. https://echo.unm.edu/. Accessed
January 8, 2019.
7. Simonoff E, Pickles A, Charman T, Chandler S,
Loucas T, Baird G. Psychiatric disorders in children
with autism spectrum disorders: prevalence,
comorbidity, and associated factors in a
population-derived sample. J Am Acad Child
Adolesc Psychiatry. 2008;47(8):921-929.
doi:10.1097/CHI.0b013e318179964f
8. Myers SM, Johnson CP; American Academy of
Pediatrics Council on Children With Disabilities.
Management of children with autism spectrum
disorders. Pediatrics. 2007;120(5):1162-1182.
doi:10.1542/peds.2007-2362
9. American Psychiatric Association. Diagnostic
and Statistical Manual of Mental Disorders. 5th ed.
Washington, DC: American Psychiatric Association;
2013.
10. Baio J, Wiggins L, Christensen DL, et al.
Prevalence of autism spectrum disorder among
children aged 8 years: autism and developmental
disabilities monitoring network, 11 sites, United
States, 2014. MMWR Surveill Summ. 2018;67(6):1-
23. doi:10.15585/mmwr.ss6706a1
11. Bisgaier J, Levinson D, Cutts DB, Rhodes KV.
Access to autism evaluation appointments with
developmental-behavioral and neurodevelop-
mental subspecialists. Arch Pediatr Adolesc Med.
2011;165(7):673-674. doi:10.1001/
archpediatrics.2011.90
12. Kogan MD, Strickland BB, Blumberg SJ, Singh
GK, Perrin JM, van Dyck PC. A national profile of the
health care experiences and family impact of autism
spectrum disorder among children in the United
States, 2005-2006. Pediatrics. 2008;122(6):e1149-
e1158. doi:10.1542/peds.2008-1057
13. Croen LA, Najjar DV, Ray GT, Lotspeich L, Bernal
P. A comparison of health care utilization and costs
of children with and without autism spectrum
disorders in a large group-model health plan.
Pediatrics. 2006;118(4):e1203-e1211. doi:10.1542/
peds.2006-0127
14. Liptak GS, Stuart T, Auinger P. Health care
utilization and expenditures for children with
autism: data from U.S. national samples. J Autism
Dev Disord. 2006;36(7):871-879. doi:10.1007/
s10803-006-0119-9
15. Krauss MW, Gulley S, Sciegaj M, Wells N. Access
to specialty medical care for children with mental
retardation, autism, and other special health care
needs. Ment Retard. 2003;41(5):329-339. doi:10.
1352/0047-6765(2003)41<329:ATSMCF>2.0.CO;2
16. Sheldrick RC, Perrin EC. Medical home services
for children with behavioral health conditions. J Dev
Behav Pediatr. 2010;31(2):92-99. doi:10.1097/DBP.
0b013e3181cdabda
17. Newacheck PW, Hung YY, Wright KK. Racial and
ethnic disparities in access to care for children with
jamapediatrics.com
 Effectiveness of Extension for Community Health Outcomes for Improving Primary Care for Autism
special health care needs. Ambul Pediatr. 2002;2
(4):247-254. doi:10.1367/1539-4409(2002)
002<0247:RAEDIA>2.0.CO;2
18. Skinner AC, Slifkin RT. Rural/urban differences
in barriers to and burden of care for children with
special health care needs. J Rural Health. 2007;23
(2):150-157. doi:10.1111/j.1748-0361.2007.00082.x
19. Gresenz CR, Rogowski J, Escarce JJ.
Dimensions of the local health care environment
and use of care by uninsured children in rural and
urban areas. Pediatrics. 2006;117(3):e509-e517. doi:
10.1542/peds.2005-0733
20. Mandell DS, Novak MM, Zubritsky CD. Factors
associated with age of diagnosis among children
with autism spectrum disorders. Pediatrics. 2005;
116(6):1480-1486. doi:10.1542/peds.2005-0185
21. Daniels AM, Mandell DS. Explaining differences
in age at autism spectrum disorder diagnosis:
a critical review. Autism. 2014;18(5):583-597. doi:
10.1177/1362361313480277
22. Kuhlthau KA, McDonnell E, Coury DL,
Payakachat N, Macklin E. Associations of quality of
life with health-related characteristics among
children with autism. Autism. 2018;22(7):804-813.
23. Institute of Medicine. Defining Primary Care: An
Interim Report. Washington, DC: The National
Academies Press; 1994.
24. Golnik A, Scal P, Wey A, Gaillard P.
Autism-specific primary care medical home
intervention. J Autism Dev Disord. 2012;42(6):
1087-1093. doi:10.1007/s10803-011-1351-5
25. Carbone PS. Moving from research to practice
in the primary care of children with autism
spectrum disorders. Acad Pediatr. 2013;13(5):390-
399. doi:10.1016/j.acap.2013.04.003
26. Council on Children With Disabilities; Section
on Developmental Behavioral Pediatrics; Bright
Futures Steering Committee; Medical Home
Initiatives for Children With Special Needs Project
Advisory Committee. Identifying infants and young
children with developmental disorders in the
medical home: an algorithm for developmental
surveillance and screening. Pediatrics. 2006;118(1):
405-420. doi:10.1542/peds.2006-1231
27. Johnson CP, Myers SM; American Academy of
Pediatrics Council on Children With Disabilities.
jamapediatrics.com
© 2020 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ on 11/11/2022
Identification and evaluation of children with
autism spectrum disorders. Pediatrics. 2007;120(5):
1183-1215. doi:10.1542/peds.2007-2361
28. Dosreis S, Weiner CL, Johnson L, Newschaffer
CJ. Autism spectrum disorder screening and
management practices among general pediatric
providers. J Dev Behav Pediatr. 2006;27(2)(suppl):
S88-S94. doi:10.1097/00004703-200604002-
00006
29. King TM, Tandon SD, Macias MM, et al.
Implementing developmental screening and
referrals: lessons learned from a national project.
Pediatrics. 2010;125(2):350-360. doi:10.1542/peds.
2009-0388
30. Self TL, Parham DF, Rajagopalan J. Autism
spectrum disorder early screening practices:
a survey of physicians. Comm Disord Q. 2015;36:
195-207. doi:10.1177/1525740114560060
31. Arunyanart W, Fenick A, Ukritchon S, Imjaijitt W,
Northrup V, Weitzman C. Developmental and
autism screening: a survey across six states. Infants
Young Child. 2012;25:175-187. doi:10.1097/IYC.
0b013e31825a5a42
32. Fenikilé TS, Ellerbeck K, Filippi MK, Daley CM.
Barriers to autism screening in family medicine
practice: a qualitative study. Prim Health Care Res Dev.
2015;16(4):356-366. doi:10.1017/
S1463423614000449
33. Carbone PS, Murphy NA, Norlin C, Azor V,
Sheng X, Young PC. Parent and pediatrician
perspectives regarding the primary care of children
with autism spectrum disorders. J Autism Dev Disord.
2013;43(4):964-972. doi:10.1007/s10803-
012-1640-7
34. Mazurek MO, Brown R, Curran A, Sohl K. ECHO
autism. Clin Pediatr (Phila). 2017;56(3):247-256.
doi:10.1177/0009922816648288
35. Sohl K, Mazurek MO, Brown R. ECHO autism:
using technology and mentorship to bridge gaps,
increase access to care, and bring best practice
autism care to primary care. Clin Pediatr (Phila).
2017;56(6):509-511. doi:10.1177/0009922817691825
36. Malow BA, Byars K, Johnson K, et al; Sleep
Committee of the Autism Treatment Network. A
practice pathway for the identification, evaluation,
and management of insomnia in children and
adolescents with autism spectrum disorders.
(Reprinted) JAMA Pediatrics May 2020 Volume 174, Number 5
Original Investigation Research
Pediatrics. 2012;130(suppl 2):S106-S124. doi:10.
1542/peds.2012-0900I
37. Mahajan R, Bernal MP, Panzer R, et al; Autism
Speaks Autism Treatment Network
Psychopharmacology Committee. Clinical practice
pathways for evaluation and medication choice for
attention-deficit/hyperactivity disorder symptoms
in autism spectrum disorders. Pediatrics. 2012;130
(suppl 2):S125-S138. doi:10.1542/peds.2012-0900J
38. Vasa RA, Mazurek MO, Mahajan R, et al.
Assessment and treatment of anxiety in youth with
autism spectrum disorders. Pediatrics. 2016;137
(suppl 2):S115-S123. doi:10.1542/peds.2015-2851J
39. Furuta GT, Williams K, Kooros K, et al.
Management of constipation in children and
adolescents with autism spectrum disorders.
Pediatrics. 2012;130(suppl 2):S98-S105. doi:10.1542/
peds.2012-0900H
40. Buie T, Campbell DB, Fuchs GJ III, et al.
Evaluation, diagnosis, and treatment of
gastrointestinal disorders in individuals with ASDs:
a consensus report. Pediatrics. 2010;125(suppl 1):
S1-S18. doi:10.1542/peds.2009-1878C
41. Maglione MA, Gans D, Das L, Timbie J, Kasari C;
Technical Expert Panel; HRSA Autism Intervention
Research – Behavioral (AIR-B) Network.
Nonmedical interventions for children with ASD:
recommended guidelines and further research
needs. Pediatrics. 2012;130(suppl 2):S169-S178. doi:
10.1542/peds.2012-0900O
42. Hussey MA, Hughes JP. Design and analysis of
stepped wedge cluster randomized trials. Contemp
Clin Trials. 2007;28(2):182-191. doi:10.1016/j.cct.
2006.05.007
43. Moore C, Zamora I, Patel Gera M, Williams ME.
Developmental screening and referrals: assessing
the influence of provider specialty, training, and
interagency communication. Clin Pediatr (Phila).
2017;56(11):1040-1047. doi:10.1177/
0009922817701174
44. Porter S, Qureshi R, Caldwell BA, Echevarria M,
Dubbs WB, Sullivan MW. Developmental
surveillance and screening practices by pediatric
primary care providers. Infants Young Child. 2016;
29:91-101. doi:10.1097/IYC.0000000000000057
9/9