Research in Developmental Disabilities 132 (2023) 104392

Contents lists available at ScienceDirect

Research in Developmental Disabilities

journal homepage: www.elsevier.com/locate/redevdis

Measuring the association between behavioural services and

outcomes in young children with autism spectrum disorder
Kate Tsiplova a, Wendy J. Ungar a, b, *, Peter Szatmari c, d, e, Katherine Cost e,
Eleanor Pullenayegum a, f, Eric Duku g, h, Joanne Volden i, Isabel M. Smith j, k,
Charlotte Waddell l, Lonnie Zwaigenbaum m, Teresa A. Bennett g, h,
Mayada Elsabbagh n, Stelios Georgiades g, h, Anat Zaidman-Zait o, p
a
Child Health Evaluative Sciences, The Hospital for Sick Children, Peter Gilgan Centre for Research and Learning, 11th floor, 686 Bay Street,
Toronto, Ontario, M5G 0A4, Canada
b
Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College Street, Toronto, Ontario M5T 3M6, Canada
c
Centre for Addiction and Mental Health, 250 College Street, Toronto, Ontario M5T 1R8, Canada
d
Department of Psychiatry, University of Toronto, 250 College Street, Toronto, Ontario M5T 1R8, Canada
e
Department of Psychiatry, Hospital for Sick Children, 555 University Avenue, Toronto, Ontario M5G 1×8, Canada
f
Dalla Lana School of Public Health, University of Toronto, 155 College Street, Toronto, Ontario M5T 3M7, Canada
g
Offord Centre for Child Studies, McMaster University, 1280 Main St. W. - MIP 201A, Hamilton, Ontario L8S 4K1, Canada
h
Department of Psychiatry and Behavioural Neuroscience, McMaster University, St. Joseph’s Healthcare Hamilton, West 5th Campus,
Administration B3, 100 West 5th Street, Hamilton, Ontario L8N 3K7, Canada
i
Faculty of Rehabilitation Medicine, University of Alberta, 8205 114 Street, 3-48 Corbett Hall, Edmonton, Alberta T6G 2G4, Canada
j
Department of Pediatrics, Dalhousie University, IWK Health Centre, 5850 University Avenue, P. O. Box 9700, Halifax, Nova Scotia B3K 6R8,
Canada
k
Autism Research Centre, IWK Health Centre, 4th Floor Link Building, 5850/5980 University Avenue, P.O. Box 9700, Halifax, Nova Scotia B3K
6R8, Canada
l
Children’s Health Policy Centre, Faculty of Health Sciences, Simon Fraser University, Room 2435, 515 West Hastings Street Vancouver, British
Columbia V6B 5K, Canada
m
Department of Pediatrics, University of Alberta, Edmonton Clinic Health Academy, 11405-87 Avenue Edmonton, Alberta T6G 1C9, Canada
n
Montreal Neurological Institute, McGill University, 3801 University Street, Montreal, Quebec H3A 2B4, Canada
o
Department of School Counseling and Special Education, Constantiner School of Education, Tel Aviv University, Tel Aviv University, P.O. Box
39040, Tel Aviv 6997801, Israel
p
The School of Population and Public Health, Faculty of Medicine, University of British Columbia, 2206 East Mall, Vancouver, BC V6T 1Z3, Canada

A R T I C L E I N F O A B S T R A C T

Keywords: Background: Children with autism spectrum disorder (ASD) receive a wide range of services.
Autism spectrum disorder Aims: To examine the association between behavioural services received by children with ASD
Early intervention between ages 2 and 5 years and outcomes during primary school years.
Service use
Methods: A total of 414 preschool-aged children diagnosed with ASD were enrolled at five Ca­
Treatment outcomes
nadian sites and were assessed within four months of diagnosis (T1), six months later (T2), 12
months later (T3), at school entry (T4), and then annually (T5-T8) to 11 years of age. The as­
sociation between the receipt of behavioural services during T1 to T3 and T8 outcomes related to
adaptive behaviour and behavioural problems was modelled using linear regressions adjusted for

* Corresponding author at: Child Health Evaluative Sciences, The Hospital for Sick Children, Peter Gilgan Centre for Research and Learning, 11th
floor, 686 Bay Street, Toronto, Ontario M5G 0A4, Canada.
E-mail address: wendy.ungar@sickkids.ca (W.J. Ungar).

https://doi.org/10.1016/j.ridd.2022.104392
Received 26 November 2021; Received in revised form 28 November 2022; Accepted 30 November 2022
Available online 6 December 2022
0891-4222/© 2022 Elsevier Ltd. All rights reserved.
K. Tsiplova et al. Research in Developmental Disabilities 132 (2023) 104392

immigrant status, family income, child’s age at diagnosis, site, sex assigned at birth, and baseline
(T1) outcome.
Results: Children who received behavioural services during at least one time period from T1 to T3
did not have significantly different outcomes at T8 than children who did not receive any
behavioural services.
Implications: Pre-school use of behavioural services was not found to affect outcomes during later
childhood. Numerous challenges accompany studies of the association between pre-school service
use and later outcomes in a heterogeneous ASD sample. Recommendations for study design are
provided.

What this paper adds

Children with autism spectrum disorder (ASD) receive a wide range of behavioural and other services yet few researchers have
studied the relationship between service use and functional outcomes during later years. This study contributes to the ASD
literature by examining the complex association between parent-reported community-based behavioural services received
during preschool years and outcomes during primary school years across multiple jurisdictions. The study also examined which
factors related to the child and family may affect the association between receipt of behavioural services and outcomes. Children
who received any behavourial service during preschool years did not have significantly different outcomes than children who did
not receive any behavioural services. This suggests that some pre-school behavioural interventions may not be effective or may
not demonstrate a sustained effect above and beyond any effect of developmental services and other general services received by
almost all children. Although the findings were negative, this study illustrates the challenges of conducting ASD services research
in a highly heterogeneous population using prospectively collected observational data and highlights opportunities for future
research. For example, the results illustrate the importance of well-designed studies that collect long-term data on intensity and
duration of ASD interventions alongside outcomes, and that capture key demographic and clinical characteristics that affect
access to and use of services. The recommendations made in this study will enhance the quality of evidence used to inform public
ASD policy and service provision.

Data Availability

The authors do not have permission to share data.

1. Introduction

Autism spectrum disorder (ASD) is a lifelong condition characterized by impairments in social communication, and by restricted
and repetitive behaviour (Diagnostic & statistical manual of mental disorders (5th Edition), 2013). Individuals with ASD exhibit wide
heterogeneity in their symptoms, impairments and outcomes, with implications for treatment planning (Georgiades, Szatmari, &
Boyle, 2013; Georgiades et al., 2013; Munson et al., 2008; Szatmari et al., 2015). In the long-term, outcomes are variable and difficult
to predict (Howlin, Moss, Savage, & Rutter, 2013; pp. 5, 1211; Howlin, Savage, Moss, Tempier, & Rutter, 2014; pp. 5, 1211).
Behavioural interventions are widely used for ASD (Hyman, Levy, & Myers, 2020; Sandbank et al., 2020). Most are based on the
methods of applied behaviour analysis (ABA); one such program is early intensive behavioural intervention (EIBI) popularized by
Lovaas (Lovaas, 1987), which often uses discrete trial training (DTT). Traditional EIBI programs are generally characterized by 20–40
h per week of one-to-one intervention administered by trained providers to preschool-aged children for one to three years. Less
structured naturalistic developmental behavioural interventions (NDBI) (Schreibman et al., 2015) such as Pivotal Response Treatment
(PRT) (Koegel, O’Dell, & Koegel, 1987) are another option. NDBI programs are also provided at an early age and are typically less
intensive and have a shorter duration than EIBI and may incorporate parent coaching (I.M. Smith, 2019). In addition to behavioural
approaches, individuals with ASD and their caregivers often receive a wide range of developmental services, such as speech language
therapy, occupational therapy and other social, educational and community services (Lai & Weiss, 2017; Tsiplova et al., 2019; Volden
et al., 2015). Many jurisdictions in Canada and the United States fund some form of early intervention services for young children with
ASD (Hyman et al., 2020; Shepherd & Waddell, 2015).
Systematic reviews found that EIBI improves cognitive and language skills and adaptive behaviour in some children compared to
community care; however, the evidence was considered weak due to a lack of high quality randomized controlled trials (RCTs)
(Reichow, Hume, Barton, & Boyd, 2018; Rodgers et al., 2020; Sandbank et al., 2020; Warren et al., 2011; Weitlauf et al., 2014).
Retrospective studies in children and young adults with autism reported that early intervention may optimize outcomes (Anderson,
Liang, & Lord, 2014; pp. 8, 1217; Fein et al., 2013; pp. 8, 1217; Pellicano, 2012; pp. 8, 1217; Perry, Koudys, Prichard, & Ho, 2017; pp.
8, 1217). The effectiveness of early intervention programs may depend on whether they are provided in the community, a specialized
clinic or research-based settings (McGrew, Ruble, & Smith, 2016; pp. 3, 1307; Nahmias, Pellecchia, Stahmer, & Mandell, 2019; pp. 3,
1307). Community-based early intervention programs vary depending on location or provider type and may have uncertain treatment
fidelity (Bryson et al., 2007). In addition to the variation in the structure and intensity of early behavioural programs, pre-school
children may use additional interventions that become sources of confounding when estimating a specific treatment effect (Sterne
et al., 2016). A meta-analysis (Nahmias et al., 2019, pp. 13073) showed that early intervention programs developed in association with

2
K. Tsiplova et al. Research in Developmental Disabilities 132 (2023) 104392

universities and hospitals generally had significantly better outcomes than entirely community-based programs. The authors hy­
pothesized that better outcomes in an academic environment were possibly due to greater access to ASD experts who could advise on
evidence-based best practices (Nahmias et al., 2019, pp. 13073).
Another challenge in evaluating early ASD interventions delivered in “real-world” community settings is that follow-up periods in
many studies seldom extend past two years (Dawson et al., 2010; Eldevik, Titlestad, Aarlie, & Tønnesen, 2019; Hayward, Eikeseth,
Gale, & Morgan, 2009; Reed, Osborne, & Corness, 2007; Zachor & Ben Itzchak, 2010). While a two-year follow-up is useful for many
observational studies, for chronic conditions in children who are undergoing developmental change, it may be insufficient and
long-term effects remain unclear (Rodgers et al., 2020). Long-term evidence suggests that gains in cognitive, language and adaptive
behaviour scores are greatest after the first year of treatment, and although scores typically failed to increase, early gains were
maintained at follow-ups ranging from 3 to 10 years (Cohen, Amerine-Dickens, & Smith, 2006; pp. 8, 1217; Howard, Stanislaw, Green,
Sparkman, & Cohen, 2014; pp. 8, 1217; Magiati, Moss, Charman, & Howlin, 2011; pp. 8, 1217; Perry et al., 2017; pp. 8, 1217; Smith,
Hayward, Gale, Eikeseth, & Klintwall, 2019; pp. 8, 1217).
Finally, there is variation in early intervention programs across jurisdictions which may result in variation in outcomes. In Canada,
the funding and delivery of programs, children’s eligibility for treatment, and diagnostic guidelines vary among provinces and ter­
ritories. In Nova Scotia, the early intervention program follows a treatment model based on PRT and is administered as part of the
public healthcare system (I.M. Smith et al., 2019). In other provinces, more traditional ABA-based programs are predominantly
provided. Quebec provides therapy for children aged 2–5 years under its 2017–2022 action plan for autism spectrum disorder
(Ministère de la Santé et des Services sociaux, 2017). In Ontario, as part of recent funding reform to address a long waitlist for
needs-based autism services (Financial Accountability Office of Ontario, 2020), the government provides funding directly to families
based on the severity of a child’s needs and age to purchase core services (Ministry of Children Community & Social Services, 2021).
The New Brunswick Department of Education funds a private agency that delivers intervention services. Both British Columbia and
Alberta provide individualized funding directly to families to obtain eligible private interventions (Shepherd & Waddell, 2015).
Parents in all provinces are also free to purchase private supplemental EIBI services.
To improve the understanding of the development of children with ASD across Canada, including identifying predictors of out­
comes, the Pathways in ASD (Pathways in Autism Spectrum Disorder, 2021) (hereafter Pathways), prospective longitudinal cohort
study was initiated in 2005. The study recruited children aged 2–4 years at the time of their ASD diagnosis at five Canadian academic
autism centers. Children were seen repeatedly from diagnosis to age 11 years.
The objective of this study was to examine the association between the use of community-based behavioural intervention services
by preschool-age children as reported by parents and level of functional outcomes during later school years, and to determine which
factors affect this association. The process and challenges inherent in conducting this type of analysis, including issues related to
handling missing data and addressing patient heterogeneity arising in a multi-site observational longitudinal ASD study characterized
by attrition are also described.

2. Methods

Parents of Pathways study participants provided written informed consent and the protocol was approved by Research Ethics
Boards at all sites. The present analysis was approved by the Hospital for Sick Children Research Ethics Board.

2.1. Recruitment

Eligible Pathways participants were aged 2–4 years, 11 months and had a clinical ASD diagnosis confirmed by the Autism Diag­
nostic Observation Schedule (ADOS) (Lord et al., 2000) and the Autism Diagnostic Interview-Revised (ADI-R) (Lord, Rutter, & Le
Couteur, 1994). Exclusion criteria were neuromotor disorder interfering with study assessments; known genetic abnormalities or
severe vision or hearing impairment.
The study enrolled 421 children at five Canadian sites: Vancouver, British Columbia (BC); Edmonton, Alberta (AB); Hamilton,
Ontario (ON); Montreal, Quebec (QC); and Halifax, Nova Scotia (NS). In Phase I, participants were recruited and assessed at four time
points until 6 years of age: within four months of diagnosis (T1); six months later (T2); 12 months later (T3); and at school entry (T4).
Attrition occurred at each time point during the study. Seven families withdrew at T1 without completing any measures. These families
were excluded from the present analysis resulting in a sample size of n = 414. In Phase II, participants who were not lost to follow-up
(n = 300) were followed from 7 to 11 years of age and were assessed annually (T5-T8).

2.2. Intervention and comparison groups

Data on all forms of service use were collected from parents at each Phase I time point in an open-ended questionnaire. The research
team created the Pathways Autism Services Log (PASL) to categorize these data subsequently. Service use data were coded to service
types and broad service categories as previously described (Volden et al., 2015). PASL services were assigned to four categories: (1) no
services, (2) behavioural, (3) developmental, and (4) general services. Behavioural services were defined as receiving one-to-one
structured behavioural interventions, based on use of descriptors such as EIBI, ABA, DTT or PRT. Developmental services were
defined as either (1) attending an integrated mainstream group program with an aide or specialized support; (2) attending a segre­
gated, specialized group program such as social skills program, a preschool for children with special needs; (3) receiving speech
language communication therapies; (4) receiving occupational or physical therapy; or (5) receiving community mental health services.

3
K. Tsiplova et al. Research in Developmental Disabilities 132 (2023) 104392

General services were defined as either (1) attending an integrated mainstream group program without aide or other specialized
support; (2) receiving other services such as a relationship development intervention, music therapy or therapeutic horseback riding;
or (3) attending a mainstream community recreation program such as community play group with or without an aid or other
specialized support (Volden et al., 2015). A receipt of a service was defined as at least one occurrence of a child receiving one or more
services from a provider during the time period for which parents were asked to recall their service use. While the frequency of receipt
of each service was recorded, accurate data on the intensity of a service, e.g., number of visits or number of hours per week, were not
available. Therefore, any instance of service use during an assessment period would qualify as a receipt of service and no distinction
was made between participants who received a service once and those receiving a service multiple times. For example, at T2, during
the six months covered by that assessment period, a child could have received EIBI multiple times a week and could have also have seen
another ABA specialist, as well as receiving speech language services. For this child, receipt of both behavioural and development
services was recorded at T2. There were 68 children who received no services (18.8 %) at T1, 30 (9.14 %) at T2 and 16 (5.11 %) at T3.
Only 59 (21.7 %) of children received no services during T1 to T3 and therefore they were not used as a comparator.
Reflecting the focus on volume of behavioural services received, comparisons were defined as follows:

1. Children who received behavioural services during at least one time period from T1 to T3 were compared to children who did not
receive these services during these time points. Both groups could have also received developmental or general services. Those who
received no services were included in the comparator.
2. Children who received behavioural services at each assessment from T1 to T3 (i.e., receipt of services at all three assessments) were
compared to children who did not receive these services at all three assessments, but could have received behavioural services at
one or two time points. Both groups could have also received developmental or general services.

2.3. Outcomes

Outcomes related to adaptive behaviour and behavioural problems were measured at the last available time point, T8 (mean age =
129 months). Since baseline abilities are significant predictors of a child’s later outcomes (Zachor & Ben Itzchak, 2010), outcome
scores at baseline (T1) were included as covariates in each analysis of that primary outcome measured at a later time point. The
Vineland Adaptive Behavior Scales, 2nd Edition (VABS-II) (Sparrow, Cicchetti, & Balla, 2005) is a measure of child adaptive
behaviour. In this analysis, VABS-II Communication, Socialization, Daily Living Skills and Adaptive Behaviour Composite (ABC)
standard scores at T8 were used as outcomes. Higher VABS-II scores indicate better functioning. The Child Behaviour Checklist (CBCL
1.5–5) (Achenbach & Rescorla, 2000) was completed by parents at T1 to T4 and used for evaluating a wide range of internalizing and
externalizing problems. The CBCL 6–18 Externalizing and Internalizing Problems was completed by parents or teachers at T5, T6, T7
and T8. CBCL 6–18 t scores at T8 were also used as outcomes. Higher CBCL scores indicate greater problems.

2.4. Confounders and effect modifiers

Heterogeneity in ASD symptoms was addressed by identifying important potential confounders and effect modifiers. The following
variables were included as co-variates: child age at the time of ASD diagnosis as a continuous variable, site (NS, reference category; QC;
ON; BC; AB), sex assigned at birth (male, reference category; female), family income (<$20,000, reference category; >=$20,000 &
<$50,000; >=$50,000 & <$80,000; >= $80,000) and parental immigrant status (born in Canada, reference category; first-generation
immigrant). The Autism Diagnostic Observation Schedule, 2nd edition (ADOS-2) is a semi-structured assessment of social and
communication behaviour related to the diagnosis of ASD (Lord et al., 2000). The ADOS-2 calibrated severity score at T1 was used as a
covariate in each model. Categorical variables with more than two levels were included in the model using simple effects coding (i.e.
each category was compared to a reference category). The continuous variables, child age at the time of ASD diagnosis, ADOS-2 score
and outcome variable scores at T1, were not mean centered.
The following effect modifiers measured at baseline were identified as most relevant either clinically or for designing ASD
programs:

1. Verbal vs. minimally verbal: Using the overall language-level question of the Autism Diagnostic Interview – Revised (ADI-R), a
standardized measure used for the diagnosis of ASD (Lord et al., 1994), children were identified as either verbal (“functional use of
spontaneous, echoed or stereotyped language that, on a daily basis, involves phrases of three words or more that, at least some­
times, include a verb and that is comprehensible to other people”) or minimally verbal (“no functional use of three-word phrases in
spontaneous, echoed or stereotyped speech, but using speech on a daily basis with at least five different words in the last month” or
“fewer than five words total and/or speech not used on a daily basis”). Dichotomization of this variable was performed in
accordance with the American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders (Diagnostic & sta­
tistical manual of mental disorders (5th Edition), 2013) and to be consistent with previous analyses of this cohort (Bennett et al.,
2012; Georgiades et al., 2013).
2. Non-cognitively disabled vs. cognitively disabled: Using the M-P-R Developmental Index standard score at baseline, children were
identified as either cognitively disabled if the score was below 70 (two standard deviations or more below the population mean)
(Diagnostic & statistical manual of mental disorders, 4th Edition, 2000) or otherwise as non-cognitively disabled. This dichoto­
mization has been used in previous analyses of this cohort (Flanagan et al., 2015) and aids in interpretation of regression results.

4
K. Tsiplova et al. Research in Developmental Disabilities 132 (2023) 104392

3. Immigrant vs. non-immigrant caregiver: As new immigrants may have more difficulty navigating health, behavioural and social
services, the primary caregiver was identified as an immigrant (if not born in Canada) or as a non-immigrant.
4. Province: Delivery of services differs provincially, therefore, a variable indicating child’s province of residence (NS, ON, QC, AB,
BC) was included as an interaction term.

2.5. Missing data

In Phase I, 414 participants completed at least one measure at T1. Of these, 114 (27.5 %) withdrew or declined to continue to Phase
II. Missing data included participants who withdrew from the study as well incomplete responses from enrolled participants. At
baseline, 3–11 % of demographic and outcome data were missing. At the end of Phase II at T8, 47–58 % of outcome data were missing.
Across T1 to T3, 13–24 % of behavourial service use data were missing (Table 1). A multiple imputation approach using predictive
mean matching was considered to impute missing data for 414 participants. However, a complete case approach (i.e., a dataset with no
missing values for any of the variables that were used in the analysis) was ultimately used due to the extent of missing data at T8 and
since it was not possible to impute values for raw service use variables which were combined to create service categories.

2.6. Statistical models

Independent sample t-tests and chi-square tests were run to determine whether there were differences in baseline covariates be­
tween children who received behavioural services and those who did not using complete data. The histograms and quantile-quantile
plots of outcome measures were examined and shown to mostly follow a normal distribution. A series of linear regressions (six in total,
one for each outcome) were developed to model the relationship between the outcomes and behavioural service use. Unadjusted
models and models adjusted for covariates identified above were fitted in R (Team, 2019). A significance level was set to 0.05. All six
outcomes were considered primary and therefore, p-values were adjusted for multiple comparisons. VABS-II Communication, So­
cialization, Daily Living Skills and Adaptive Behaviour Composite (ABC) standard scores at T8 were highly correlated with each other
with a coefficient exceeding 0.5. Similarly, CBCL 1.5–5 Externalizing and Internalizing Problems t scores were highly correlated with
each other. However, VABS and CBCL scores were only slightly negatively correlated. Hommel method (Hommel, 1988) is recom­
mended for mildly correlated outcomes (Blakesley et al., 2009), is easy to implement and was used to adjust the p-values in this
analysis.
Measuring the effect of continued behavioural therapy on outcomes over time was also of interest. To this end, an exploratory
generalized estimating equation (GEE) analysis of repeated outcomes as well as continued service use, with services lagged behind
outcomes by one time period over the 8 time periods, was conducted. No effect of ongoing behavioural service use on adaptive
functioning over time was found. Ultimately, the lagged service use-outcomes analysis was not included due to the extent of missing
Phase II service use data and the difficulties linking disparately structured Phase I and Phase II service use questionnaires.

3. Results

Table 1 shows summary statistics for service use over time using the complete case data. One hundred and eighty-six (68.4 %) of
participants received behavioural services during at least one of three assessment time periods and 58 (21.3 %) during all three
assessment periods. Among children who received preschool behavioural services during one or more time periods (n = 186), almost
all (173, 93.0 %) also received developmental services and 144 (77.4 %) received general services during at least one assessment
period. Among children who did not receive any behavioural services during the three assessment periods (n = 86), 74 (86.1 %)
received developmental services and 60 (69.8 %) received general services.
As seen in Table 2, receipt of behavioural services was not related to the age of diagnosis. Girls were reported to receive behavioural
services less frequently than boys, although the differences did not reach statistical significance. The receipt of behavioural services
was significantly different between provinces. NS and BC had the highest proportions of children receiving behavioural services. This
difference was particularly pronounced for the receipt of behavioural services at all time points, with a range of 26.7–53.6 % of
participants receiving continuous services across three assessment periods for NS and BC sites, compared to a range of 6.5–10.8 % for
other sites. As household income increased, the proportion of respondents receiving services increased; however, differences between
income groups did not reach statistical significance. Baseline VABS II Communication domain, Socialization domain and ABC scores,
CBCL (1.5–5) externalizing problems t-score and ADOS severity score were not significantly different at the 0.05 level between the

Table 1
Summary statistics for behavioural service use, n = 414.
Receipt of services, n (%) Missing, n (%)

T1 108 (29.9) 53 (12.8)

T2 158 (48.2) 86 (20.8)
T3 187 (59.7) 101 (24.4)
Receipt of service at one or more time points (T1-T3) 186 (68.4) 142 (34.3)
Receipt of service at every time point (T1-T3) 58 (21.3) 142 (34.3)

Note: Assessment periods: T1, within four months of diagnosis; T2, six months later; T3, 12 months later.

5
K. Tsiplova et al. Research in Developmental Disabilities 132 (2023) 104392

Table 2
Baseline characteristics of participants by level of behavioural services received during preschool years, n = 414.
Total Sample Receipt of Behavioural Services Receipt of Behavioural Services
(1 or more time points) (all time points)

Yes (n = 186) No (n = 86) Yes (n = 58) No (n = 214)

Child’s age at ASD diagnosis, months (M, SD) 38.3 (8.8) 37.9 (8.3) 38.5 (8.6) 38.7 (8.6) 37.9 (8.3)
missing n = 0 p = 0.58 p = 0.56
Child’s sex (n, %)
Male 350 (84.5 %) 162 (69.8 %) 70 (30.2 %) 53 (22.8 %) 179 (77.2 %)
Female 64 (15.5 %) 24 (60.0 %) 16 (40.0 %) 5 (12.5 %) 35 (87.5 %)
missing n = 0 p = 0.294 p = 0.21
Site (n, %)
NS 56 (13.5 %) 28 (93.3 %) 2 (6.7 %) 8 (26.7 %) 22 (73.3 %)
QC 134 (32.4 %) 66 (61.7 %) 41 (38.3 %) 7 (6.5 %) 100 (93.5 %)
ON 68 (16.4 %) 14 (37.8 %) 23 (62.2 %) 4 (10.8 %) 33 (89.2 %)
AB 66 (15.9 %) 13 (44.8 %) 16 (55.2 %) 2 (6.9 %) 27 (93.1 %)
BC 90 (21.7 %) 65 (94.2 %) 4 (5.8 %) 37 (53.6 %) 32 (46.4 %)
missing n = 0 p < 0.001 p < 0.001
Household income (n, %)
< $20,000 48 (12.4 %) 15 (60.0 %) 10 (40.0 %) 3 (12.0 %) 22 (88.0 %)
> =$20,000 & < $50,000 79 (20.5 %) 33 (68.8 %) 15 (31.2 %) 5 (10.4 %) 43 (89.6 %)
> =$50,000 & < $80,000 112 (29.0 %) 57 (67.1 %) 28 (32.9 %) 22 (25.9 %) 63 (74.1 %)
> = $80,000 147 (38.1 %) 79 (71.2 %) 32 (28.8 %) 27 (24.3 %) 84 (75.7 %)
missing n = 28 p = 0.73 p = 0.09
Immigrant status (PMK) (n, %)
Born in Canada 278 (71.6 %) 132 (67.3 %) 64 (32.7 %) 46 (23.5 %) 150 (76.5 %)
Immigrant 110 (28.4 %) 53 (70.7 %) 22 (29.3 %) 12 (16.0 %) 63 (84.0 %)
missing n = 26 p = 0.70 p= 0.24
VABS II Communication score, T1 (M, SD) 73.3 (16.0) 73.8 (15.5) 75.2 (17.5) 74.1 (14.7) 74.3 (16.5)
missing n = 13 p = 0.52 p= 0.93
VABS II Socialization score, T1 (M, SD) 72.2 (9.1) 72.4 (8.8) 74.0 (10.0) 71.0 (8.2) 73.4 (9.4)
missing n = 13 p = 0.20 p= 0.06
VABS II Daily Living Skills score, T1 (M, SD) 76.2 (9.1) 75.5 (11.0) 79.4 (12.2) 76.2 (10.5) 76.9 (11.8)
missing n = 13 p = 0.01 p= 0.69
VABS II ABC, T1 (M, SD) 72.8 (10.1) 72.7 (9.7) 75.1 (11.2) 72.9 (9.2) 73.6 (10.5)
missing n = 15 p = 0.09 p= 0.60
CBCL Internalizing Problems t-score, T1 60.2 (9.3) 58.9 (9.0) 59.7 (9.2) 56.9 (8.1) 59.8 (9.2)
missing n = 49 p = 0.52 p= 0.02
CBCL Externalizing Problems t-score, T1 56.1 (10.4) 55.5 (10.3) 54.9 (9.7) 55.4 (10.3) 55.3 (10.0)
missing n = 49 p = 0.64 p= 0.99
ADOS Severity score, T1 (M, SD) 7.6 (1.7) 7.6 (1.7) 7.8 (1.7) 7.9 (1.6) 7.6 (1.7)
missing n = 8 p = 0.60 p= 0.28

Abbreviations: ASD, autism spectrum disorder; NS, Nova Scotia, QC, Quebec, ON, Ontario; AB, Alberta; BC, British Columbia; PMK, person most
knowledgeable about the child; VABS II, Vineland Adaptive Behaviour Scales - Second Edition; ABC, Adaptive Behaviour Composite; CBCL, Child
Behaviour Checklist (1.5–5). ADOS, Autism Diagnostic Observation Schedule, 2nd edition.
Note. Demographic characteristics which were missing at baseline were replaced with data at T2. If T2 data were missing, then T3 data were used and
so on.

groups. The VABS II Daily Living Skills score at the baseline assessment T1 was significantly lower (i.e. worse) in children who received
behavioural services during at least one assessment period compared to children who did not receive any behavioural services during
T1-T3. In contrast, the CBCL Internalizing Problems t-score at T1 was significantly lower (i.e. better) in children who received
behavioural services at each time point during T1-T3 compared to children who did not.
The estimated effects of receipt of behavioural services during T1-T3 on are shown in Table 3 using unadjusted and adjusted
regression models. The full regression results are shown in Supplementary Tables 1 and 2. The greatest improvement was observed for
the VABS-II Daily Living Skills score at T8. After adjustment for co-variates, children who received behavioural services during at least
one assessment period had a T8 Daily Living Skills score that was higher by a mean of 4.32 than children who did not. However, in both
unadjusted and adjusted models, none of the behavioural service coefficients were significant when p-values were adjusted to account
for multiple comparisons. None of the effect modifiers were significant apart from the province. For the T8 VABS-II Daily Living Skills
and ABC scores, the interaction between service use at each time point during T1 and T3 and receiving services in QC relative to NS was
significant at 0.05 level. In other words, the effect of behavioural services on the outcome was moderated by whether these services
were received in QC compared to NS. However, once these p-values were adjusted for multiple comparisons, the significant effects
disappeared.

4. Discussion

This study investigated the relationship between the receipt of public and private community-based parent-reported behavioural

6
K. Tsiplova et al. Research in Developmental Disabilities 132 (2023) 104392

Table 3
Estimated coefficients measuring the effect of receipt of behavioural services during T1-T3 on select outcomes, unadjusteda and adjustedb linear
models, n = 414.
Outcome Unadjusted Adjusted
c
Coef SE P Coef SE Pc

Receipt of services during at least one time period (T1-T3)

Model 1 VABS-II Communication score, T8 (missing = 246) 1.50 2.77 0.78 0.03 2.48 0.99
Model 2 VABS-II Socialization score, T8 (missing = 248) 3.62 2.54 0.48 2.93 2.48 0.96
Model 3 VABS-II Daily Living Skills score, T8 (missing = 247) 3.16 2.29 0.51 4.32 2.27 0.35
Model 4 VABS-II ABC, T8 (missing = 247) 2.78 2.36 0.72 2.81 2.14 0.76
Model 5 CBCL externalizing problems t score, T8 (missing = 248) -0.55 1.91 0.78 -0.06 2.10 0.99
Model 6 CBCL internalizing problems t score, T8 (missing = 248) -2.49 1.80 0.50 -0.19 1.88 0.99
Receipt of services at each time period during T1-T3
Model 1 VABS-II Communication score, T8 (missing = 246) -1.19 3.05 0.86 -2.33 2.85 0.90
Model 2 VABS-II Socialization score, T8 (missing = 248) 1.00 2.79 0.86 1.08 2.84 0.90
Model 3 VABS-II Daily Living Skills score, T8 (missing = 247) 1.00 2.53 0.86 -0.33 2.62 0.90
Model 4 VABS-II ABC, T8 (missing = 247) 0.45 2.60 0.86 -0.71 2.47 0.90
Model 5 CBCL externalizing problems t score, T8 (missing = 248) 0.85 2.13 0.86 1.04 2.46 0.90
Model 6 CBCL internalizing problems t score, T8 (missing = 248) -0.65 2.01 0.86 1.27 2.21 0.90

Abbreviations: VABS II, Vineland Adaptive Behaviour Scales – 2nd edition; ABC, Adaptive Behaviour Composite; CBCL, Child Behaviour Checklist
(1.5–5); ADOS, Autism Diagnostic Observation Schedule, 2nd edition.
a
Unadjusted models exclude covariates.
b
Adjusted models include the following covariates: income, parental immigrant status, age at the time of ASD diagnosis, site and sex.
c
p-values were adjusted for multiple comparisons using the Hommel method

services in preschoolers and distal outcomes for these children 4–5 years later. The receipt of behavioural services at some point during
the preschool years (as a categorical variable) was not significantly associated with outcomes at T8 when adjusted for multiple
comparisons. Few studies have examined long-term effects of early intervention. Some studies found that early behavioural in­
terventions showed larger effects for proximal compared to distal outcomes (Landa & Kalb, 2012; Sandbank et al., 2020; Yoder,
Bottema-Beutel, Woynaroski, Chandrasekhar, & Sandbank, 2013). In a study of children with ASD who received intervention prior to
age 3 years, significant improvements in IQ and communication scores were observed at age 6 years, but the study lacked a
comparative control group (Landa & Kalb, 2012). In a well-designed RCT comparing a parent-mediated social communication
intervention to treatment as usual in children aged 2–4 years with autism, no significant differences between groups were found in
symptom severity, communication or language skills at age 10 years. However, there were significant improvements in symptom
severity and communications skills when examined over the course of the whole study period compared to controls (Pickles et al.,
2016). In an individual patient data meta-analysis, Rodgers et al. found that early improvements in IQ observed in children receiving
EIBI compared to eclectic or treatment as usual were not observed at 7 years post-treatment (Rodgers et al., 2021). That paper nicely
outlines the risk of bias in the design elements of published short- and long-term studies. In order for early interventions to show a
long-term or sustained effect after ending treatment, they must be effective and also positively improve a child’s developmental
trajectory (Sandbank et al., 2020).
In the present study, mean standard scores declined over time, regardless of whether children received behavioural services. This is
a feature of standardized scores, since the developmental gap between individuals with ASD and age-adjusted norms may increase over
time (Nahmias et al., 2019, pp. 13073). Additionally, the standard scores may not have been sensitive enough to measure clinically
significant change in some children with ASD (Magiati et al., 2011). Another possible explanation of the decline is that children did not
continue to receive necessary or effective services. Although service use was collected during school years, a large proportion of
observations were missing (~60 %). Nevertheless, for those families for whom data were available, there was a considerable decrease
in the receipt of community-based behavioural services (at T8, 30.1 % of children received behavioural services) and school-based
behavioural services (at T8, 12.9 % of children received behavioural services) during school years.
During data exploration and analysis, it was found that participants used a variety of services that changed over time. Although it
would be valuable to compare the receipt of behavioural services with the receipt of only eclectic or general services, there were too
few children who only received such services during T1 to T3. Most children in both groups received developmental and general
services. Testing hypotheses regarding a relationship between continuous service use (after T3 and up to T8) and outcomes using
longitudinal methods were considered. However, due to a large extent of missing data for Phase II service use (~60 %) and difficulties
with linking Phase I and Phase II services since Phase II used two different questionnaires, only service use during the developmentally
critical preschool years was included in the analysis.
The present study indicated that girls were less likely to receive behavioural services compared to boys, however the difference was
not significant. Other studies have reported that more boys than girls received EIBI compared to a treatment as usual group (Cohen
et al., 2006; Eldevik et al., 2019). There were significant provincial differences in the receipt of behavioural services, with some sites
having substantially higher rates of children receiving services. Province appears to be an effect modifier of the association between
service use and outcomes for a few select outcomes and provinces; however, this was true only when no adjustment for multiple
comparisons was made. As expected, higher household income was associated with receiving services. Although behavioural services
are publicly subsidized in all provinces, families with higher household income may be able to access private services more easily, as

7
K. Tsiplova et al. Research in Developmental Disabilities 132 (2023) 104392

well as allocate more time and resources to enable the child to access ASD services. A similar trend was observed for parental education
in other studies (Cohen et al., 2006; Lai & Weiss, 2017; Magiati, Charman, & Howlin, 2007).

4.1. Study limitations

There were several study-specific limitations. The intensity and duration of interventions likely varied between provinces and
participants. Children may have received different versions of similar ABA-based interventions, but due to the lack of data on intensity,
duration, fidelity and type of interventions children received, it was not possible to differentiate among them (Volden et al., 2015). As
often seen in children with ASD, children accessed a wide range of services and every child received some type of service, which made
it difficult to standardize the intervention and comparison groups. The PASL was developed from unstructured data self-reported by
parents, who at the time the T1 interventions were administered had only recently received an ASD diagnosis for their children and
may have not reported services with the necessary specificity. As a structured comprehensive resource use questionnaire was added to
a subsequent phase of the Pathways cohort, future analysis will include an examination of how outcomes relate to range of concomitant
services, duration of use, frequency and intensity (Ungar, Tsiplova, Millar, & Smith, 2018, pp. 0000226). Another source of potential
bias was a large proportion of missing data (~35–50 % of observations were missing).
Although efforts were made to identify and adjust for potential confounders, unmeasured confounders likely affected the analysis.
It is not clear whether the analysis adequately controlled for disease severity, given the seemingly conflicting findings at T1 for the
association between preschool service use and the baseline VABS II Daily Living Skills and the baseline CBCL Internalizing Problems.
Potential bias from confounders could have been reduced by using propensity score matching; however, given the large extent of
missing data, this method was not utilized. Although the effect modifiers included in the present analysis were not significant, it is
possible that the full extent of heterogeneity in participants was not captured.

4.2. Recommendations for services research in ASD

There are challenges that apply to all health and non-health services research studies that aim to estimate the effects of ASD in­
terventions, since individuals are heterogeneous in symptoms and abilities, as are their families’ capacities to access a wide range of
services in home, community and school settings. Moreover, treatments administered to children may differ in fidelity and may be
based on ASD presentation and co-morbidities, individual provider preferences and provincial programs. Longitudinal effectiveness
research is complicated by several layers of heterogeneity in ASD interventions. In addition to wide variation in the way early
intervention services are designed, there are differences between regions regarding eligibility and modes of delivery and access, as well
as different mechanisms for funding and oversight. For example, a recent study compared publicly-funded behavioural intervention
programs in NS and New Brunswick (NB), two adjacent Maritime provinces. Due to differences in provincial service delivery, children
who accessed PRT program in NS were older, with more impaired adaptive functioning and more severe ASD symptoms (I.M. Smith
et al., 2019) compared to the children receiving programs in neighbouring NB. Given substantial program differences across Canada,
any multi-jurisdictional study must collect detailed data on eligibility, type, intensity and duration of the intervention of interest, as
well as demographic data and baseline symptoms and functioning, so that similar types of interventions could be compared between
groups that are matched on or adjusted for demographic and baseline differences. It is challenging for parents to accurately recall and
report the type, dose and intensity of interventions received by their children, particularly when children are receiving many serv­
ices/and or services change over time. To improve the accuracy of reporting, parent questionnaires should be designed to capture
intervention intensity in a structured and uniform manner; be administered by interviewers using memory aids as opposed to asking
parent to self-complete; and have relatively short recall periods.
Children with ASD typically receive multiple interventions in addition to the intervention being evaluated. Concomitant similar
services may confound the intervention effect. If it is not possible to limit the receipt of potentially confounding services, then these
services should be recorded and an effort should be made to account for them in the analysis. There may also be a relationship between
outcomes and interactions between services. Increasing sample size and taking steps to reduce attrition would facilitate examining the
interactions between service types and effect modifiers and would also facilitate latent class analysis to identify groups of children
displaying similar patterns of response to services. In addition, studies of ASD interventions should be well designed RCTs and consider
long-term effectiveness of continuous service exposure. Although conducting such studies may be costly, allocating public or
household funds toward ineffective or inadequate services is clinically and ethically unjustified. Further, there is yet no consensus on
definitions of “good” outcomes for individuals with ASD in late childhood or adolescence. Developing and validating a core set of
outcomes would enable a standardized approach to measure progress and to guide clinical care (P. Szatmari et al., 2021).

5. Conclusion

This study examined how the receipt of community-based parent-reported behavioural services in preschool-aged children affected
outcomes for these children 4–5 years later, compared to children who did not receive these services. The receipt of services differed
significantly based on the province, which can be explained by provincial differences in service funding and delivery policies. Children
who received behavioural services at some point during the preschool years did not have statistically significant differences in out­
comes during primary school years compared to children who did not receive these services, although almost all children in the cohort
received some form of developmental and general services. Major limitations of this study were a lack of information on intensity and
duration of intervention and significant data attrition. Recommendations are made for the design of studies in ASD to enhance the

8
K. Tsiplova et al. Research in Developmental Disabilities 132 (2023) 104392

quality and rigour of evidence used to inform public ASD policy and service provision.

CRediT authorship contribution statement

Kate Tsiplova: Conceptualization, Methodology, Formal analysis, Writing – Original Draft, Visualization Wendy J. Ungar:
Conceptualization, Methodology, Writing – Review & Editing, Supervision, Resources, Project administration, Funding acquisition
Peter Szatmari: Conceptualization, Methodology, Writing – Review & Editing Katherine Cost: Methodology, Writing – Review &
Editing Eleanor Pullenayegum: Methodology, Writing – Review & Editing Eric Duku: Methodology, Writing – Review & Editing
Joanne Volden: Conceptualization, Writing – Review & Editing Isabel M. Smith: Conceptualization, Writing – Review & Editing
Charlotte Waddell: Writing – Review & Editing Lonnie Zwaigenbaum: Writing – Review & Editing Terry Bennett: Writing – Review
& Editing Mayada Elsabbagh: Writing – Review & Editing Stelios Georgiades: Writing – Review & Editing Anat Zaidman-Zait:
Writing – Review & Editing.

Declarations of Competing Interest

None.

Data Availability

The authors do not have permission to share data.

Acknowledgements

This study was supported by an operating grant from the Canadian Institutes of Health Research (CIHR) (Grant #151528). Wendy
Ungar is supported by a Canada Research Chair in Economic Evaluation and Technology Assessment in Child Health. The Pathways
study was supported by CIHR, Autism Speaks, Government of British Columbia, Alberta Innovates Health Solutions, Kids Brain Health
Network (formerly NeuroDevNet), Joan & Jack Craig Chair in Autism Research and the Sinneave Family Foundation. The authors
thank all the families who participated in the Pathways in ASD study. The authors remain grateful to the Pathways in ASD Study Team,
members working with participants.

Appendix A. Supporting information

Supplementary data associated with this article can be found in the online version at doi:10.1016/j.ridd.2022.104392.

References

Achenbach, T. M., & Rescorla, L. A. (2000). Manual for the ASEBA preschool forms and profiles. Burlington, VT: University of Vermont.
Anderson, D. K., Liang, J. W., & Lord, C. (2014). Predicting young adult outcome among more and less cognitively able individuals with autism spectrum disorders.
Journal of Child Psychology and Psychiatry, 55(5), 485–494. https://doi.org/10.1111/jc
ANON. (2000). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association.
ANON. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Association Publishing.
Bennett, T., Boyle, M., Georgiades, K., Georgiades, S., Thompson, A., Duku, E., & Szatmari, P. (2012). Influence of reporting effects on the association between
maternal depression and child autism spectrum disorder behaviors. Journal of Child Psychology and Psychiatry, 53(1), 89–96. https://doi.org/10.1111/j.1469-
7610.2011.02451.x
Blakesley, R. E., Mazumdar, S., Dew, M. A., Houck, P. R., Tang, G., Reynolds, C. F., 3rd, & Butters, M. A. (2009). Comparisons of methods for multiple hypothesis
testing in neuropsychological research. Neuropsychology, 23(2), 255–264. https://doi.org/10.1037/a0012850
Bryson, S. E., Koegel, L. K., Koegel, R. L., Openden, D., Smith, I. M., & Nefdt, N. (2007). Large scale dissemination and community implementation of pivotal response
treatment: Program description and preliminary data. Research & Practice for Persons with Severe Disabilities, 32(2), 142–153. Retrieved from 〈http://search.
ebscohost.com/login.aspx?direct=true&db=eue&AN=507988911&site=ehost-live〉.
Cohen, H., Amerine-Dickens, M., & Smith, T. (2006). Early intensive behavioral treatment: replication of the UCLA model in a community setting. Journal of
Developmental & Behavioral Pediatrics, 27(2), S145–S155. Retrieved from 〈https://journals.lww.com/jrnldbp/Fulltext/2006/04002/Early_Intensive_Behavioral_
Treatment_Replication.13.aspx〉.
Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., & Varley, J. (2010). Randomized, controlled trial of an intervention for toddlers with autism:
The Early Start Denver Model. Pediatrics, 125(1), e17–e23. https://doi.org/10.1542/peds.2009-0958
Eldevik, S., Titlestad, K. B., Aarlie, H., & Tønnesen, R. (2019). Community implementation of early behavioral intervention: Higher intensity gives better outcome.
European Journal of Behavior Analysis, 21(1), 92–109. https://doi.org/10.1080/15021149.2019.1629781
Fein, D., Barton, M., Eigsti, I.-M., Kelley, E., Naigles, L., Schultz, R. T., & Tyson, K. (2013). Optimal outcome in individuals with a history of autism. Journal of Child
Psychology and Psychiatry, 54(2), 195–205. https://doi.org/10.1111/jc
Financial Accountability Office of Ontario. (2020). Autism services: A financial review of autism services and program design considerations for the new Ontario
autism program. Retrieved from 〈https://www.fao-on.org/web/default/files/publications/FA1901%20Autism/AutismServices-EN.pdf〉.
Flanagan, H. E., Smith, I. M., Vaillancourt, T., Duku, E., Szatmari, P., Bryson, S., & Georgiades, S. (2015). Stability and change in the cognitive and adaptive behaviour
scores of preschoolers with autism spectrum disorder. Journal of Autism and Developmental Disorders, 45(9), 2691–2703. https://doi.org/10.1007/s10803-015-
2433-6
Georgiades, S., Szatmari, P., & Boyle, M. (2013). Importance of studying heterogeneity in autism. Neuropsychiatry, 3(2), 123–125. https://doi.org/10.2217/npy.13.8

9
K. Tsiplova et al. Research in Developmental Disabilities 132 (2023) 104392

Georgiades, S., Szatmari, P., Boyle, M., Hanna, S., Duku, E., Zwaigenbaum, L., … A. S. D. S. T. (2013). Investigating phenotypic heterogeneity in children with autism
spectrum disorder: A factor mixture modeling approach. Journal of Child Psychology and Psychiatry, 54(2), 206–215. https://doi.org/10.1111/j.1469-
7610.2012.02588.x
Hayward, D., Eikeseth, S., Gale, C., & Morgan, S. (2009). Assessing progress during treatment for young children with autism receiving intensive behavioural
interventions. Autism: The International Journal of Research and Practice, 13(6), 613–633. https://doi.org/10.1177/1362361309340029
Hommel, G. (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika, 75(2), 383–386. https://doi.org/10.1093/biomet/
75.2.383
Howard, J. S., Stanislaw, H., Green, G., Sparkman, C. R., & Cohen, H. G. (2014). Comparison of behavior analytic and eclectic early interventions for young children
with autism after three years. Research in Developmental Disabilities, 35(12), 3326–3344. https://doi.org/10.1016/j.ridd.2014.08.021
Howlin, P., Moss, P., Savage, S., & Rutter, M. (2013). Social Outcomes in mid- to later adulthood among individuals diagnosed with autism and average nonverbal IQ
as children. e571 Journal of the American Academy of Child & Adolescent Psychiatry, 52(6), 572–581. https://doi.org/10.1016/j.jaac.2013.02.017.
Howlin, P., Savage, S., Moss, P., Tempier, A., & Rutter, M. (2014). Cognitive and language skills in adults with autism: A 40-year follow-up. Journal of Child Psychology
and Psychiatry, 55(1), 49–58. https://doi.org/10.1111/jc
Hyman, S. L., Levy, S. E., & Myers, S. M. (2020). Identification, evaluation, and management of children with autism spectrum disorder. Pediatrics, 145(1), Article
e20193447. https://doi.org/10.1542/peds.2019-3447
Koegel, R., O’Dell, M., & Koegel, L. (1987). A natural language teaching paradigm for nonverbal autistic children. Journal of Autism and Developmental Disorders, 17(2),
187–200. https://doi.org/10.1007/BF01495055
Lai, J. K. Y., & Weiss, J. A. (2017). Priority service needs and receipt across the lifespan for individuals with autism spectrum disorder. Autism Research, 10(8),
1436–1447. https://doi.org/10.1002/aur.1786
Landa, R. J., & Kalb, L. G. (2012). Long-term outcomes of toddlers with autism spectrum disorders exposed to short-term intervention. Pediatrics, 130(Supplement_2),
S186–S190. https://doi.org/10.1542/peds.2012-0900Q
Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavore, P. C., & Rutter, M. (2000). The autism diagnostic observation schedule—generic: A standard
measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30(3), 205–223. https://doi.
org/10.1023/a:1005592401947
Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism diagnostic interview-revised: A revised version of a diagnostic interview for caregivers of individuals with
possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24(5), 659–685. https://doi.org/10.1007/bf02172145
Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology,
55(1), 3–9. https://doi.org/10.1037/0022-006X.55.1.3
Magiati, I., Charman, T., & Howlin, P. (2007). A two-year prospective follow-up study of community-based early intensive behavioural intervention and specialist
nursery provision for children with autism spectrum disorders. Journal of Child Psychology and Psychiatry, 48(8), 803–812. https://doi.org/10.1111/j.1469-
7610.2007.01756.x
Magiati, I., Moss, J., Charman, T., & Howlin, P. (2011). Patterns of change in children with Autism Spectrum Disorders who received community based comprehensive
interventions in their pre-school years: A seven year follow-up study. Research in Autism Spectrum Disorders, 5(3), 1016–1027. https://doi.org/10.1016/j.
rasd.2010.11.007
McGrew, J. H., Ruble, L. A., & Smith, I. M. (2016). Autism spectrum disorder and evidence-based practice in psychology. Clinical Psychology: Science and Practice, 23
(3), 239–255. https://doi.org/10.1111/cpsp.12160
Ministère de la Santé et des Services sociaux. (2017). Plan d’action sur le trouble du spectre de l’autisme 2017–2022 – Des actions structurantes pour les personnes et
leur famille. Retrieved from 〈https://publications.msss.gouv.qc.ca/msss/document-001828/〉.
Ministry of Children Community and Social Services. (2021). New core services for children in the Ontario Autism Program. Retrieved from 〈https://news.ontario.ca/
en/backgrounder/60221/new-core-services-for-children-in-the-ontario-autism-program〉.
Munson, J., Dawson, G., Sterling, L., Beauchaine, T., Zhou, A., Elizabeth, K., & Abbott, R. (2008). Evidence for latent classes of IQ in young children with autism
spectrum disorder. American Journal of Mental Retardation: AJMR, 113(6), 439–452. https://doi.org/10.1352/2008.113:439-452
Nahmias, A. S., Pellecchia, M., Stahmer, A. C., & Mandell, D. S. (2019). Effectiveness of community-based early intervention for children with autism spectrum
disorder: a meta-analysis. Journal of Child Psychology and Psychiatry, 60(11), 1200–1209. https://doi.org/10.1111/jc
Pathways in Autism Spectrum Disorder. (2021). The Pathways in ASD study examines how children with ASD grow and develop. Retrieved from 〈http://www.
asdpathways.ca〉.
Pellicano, E. (2012). Do Autistic symptoms persist across time? Evidence of substantial change in symptomatology over a 3-year period in cognitively able children
with autism. American Journal on Intellectual and Developmental Disabilities, 117(2), 156–166. https://doi.org/10.1352/1944-7558-117.2.156
Perry, A., Koudys, J., Prichard, A., & Ho, H. (2017). Follow-up study of youth who received EIBI as young children. Behavior Modification, 43(2), 181–201. https://doi.
org/10.1177/0145445517746916
Pickles, A., Le Couteur, A., Leadbitter, K., Salomone, E., Cole-Fletcher, R., Tobin, H., & Green, J. (2016). Parent-mediated social communication therapy for young
children with autism (PACT): long-term follow-up of a randomised controlled trial. The Lancet, 388(10059), 2501–2509. https://doi.org/10.1016/S0140-6736
(16)31229-6
Reed, P., Osborne, L. A., & Corness, M. (2007). The Real-world effectiveness of early teaching interventions for children with autism spectrum disorder. Exceptional
Children, 73(4), 417–433. https://doi.org/10.1177/001440290707300402
Reichow, B., Hume, K., Barton, E. E., & Boyd, B. A. (2018). Early intensive behavioral intervention (EIBI) for young children with autism spectrum disorders (ASD.
CD009260-CD009260 The Cochrane Database of Systematic Reviews, 5(5). https://doi.org/10.1002/14651858.CD009260.pub3.
Rodgers, M., Marshall, D., Simmonds, M., Le Couteur, A., Biswas, M., Wright, K., & Hodgson, R. (2020). Interventions based on early intensive applied behaviour
analysis for autistic children: a systematic review and cost-effectiveness analysis. Health Technology Assessment, 24(35), 1–306. https://doi.org/10.3310/hta24350
Rodgers, M., Simmonds, M., Marshall, D., Hodgson, R., Stewart, L. A., Rai, D., & Couteur, A. L. (2021). Intensive behavioural interventions based on applied behaviour
analysis for young children with autism: An international collaborative individual participant data meta-analysis. Autism, 25(4), 1137–1153. https://doi.org/
10.1177/1362361320985680
Sandbank, M., Bottema-Beutel, K., Crowley, S., Cassidy, M., Dunham, K., Feldman, J. I., & Woynaroski, T. G. (2020). Project AIM: Autism intervention meta-analysis
for studies of young children. Psychological Bulletin, 146(1), 1–29. https://doi.org/10.1037/bul0000215
Schreibman, L., Dawson, G., Stahmer, A. C., Landa, R., Rogers, S. J., McGee, G. G., & Halladay, A. (2015). Naturalistic developmental behavioral interventions:
Empirically validated treatments for autism spectrum disorder. Journal of Autism and Developmental Disorders, 45(8), 2411–2428. https://doi.org/10.1007/
s10803-015-2407-8
Shepherd, C. A., & Waddell, C. (2015). A qualitative study of autism policy in Canada: Seeking consensus on children’s services. Journal of Autism and Developmental
Disorders, 45(11), 3550–3564. https://doi.org/10.1007/s10803-015-2502-x
Smith, D. P., Hayward, D. W., Gale, C. M., Eikeseth, S., & Klintwall, L. (2019). Treatment gains from early and intensive behavioral intervention (EIBI) are maintained
10 years later. Behavior Modification. https://doi.org/10.1177/0145445519882895
Smith, I. M., Flanagan, H. E., Ungar, W. J., D’Entremont, B., Garon, N., den Otter, J., & Murray, P. (2019). Comparing the 1-year impact of preschool autism
intervention programs in two Canadian provinces. Autism Research, 12(4), 667–681. https://doi.org/10.1002/aur.2072
Sparrow, S., Cicchetti, D., & Balla, D. (2005). Vineland adaptive behavior scales (Second ed.). (Vineland II), Survey interview form/caregiver rating form. Livonia, MN:
Pearson Assessments.
Sterne, J. A. C., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., & Higgins, J. P. T. (2016). ROBINS-I: A tool for assessing risk of bias in
non-randomised studies of interventions. BMJ, 355, i4919. https://doi.org/10.1136/bmj.i4919
Szatmari, P., Cost, K. T., Duku, E., Bennett, T., Elsabbagh, M., Georgiades, S., & Zwaigenbaum, L. (2021). Association of child and family attributes with outcomes in
children with autism. JAMA Network Open, 4(3), Article e212530. https://doi.org/10.1001/jamanetworkopen.2021.2530

10
K. Tsiplova et al. Research in Developmental Disabilities 132 (2023) 104392

Szatmari, P., Georgiades, S., Duku, E., Bennett, T. A., Bryson, S., & Fombonne, E. (2015). Developmental trajectories of symptom severity and adaptive functioning in
an inception cohort of preschool children with autism spectrum disorderpreschool children with autism spectrum disorderpreschool children with autism
spectrum disorder (Team, f. t. P. i. A. S) JAMA Psychiatry, 72(3), 276–283. https://doi.org/10.1001/jamapsychiatry.2014.2463.
Team, R. C. (2019). R: A Language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-
project.org.
Tsiplova, K., Ungar, W. J., Flanagan, H. E., den Otter, J., Waddell, C., Murray, P., & Smith, I. M. (2019). Types of services and costs of programs for preschoolers with
autism spectrum disorder across sectors: A comparison of two Canadian provinces. Journal of Autism and Developmental Disorders, 49(6), 2492–2508. https://doi.
org/10.1007/s10803-019-03993-3
Ungar, W. J., Tsiplova, K., Millar, N., & Smith, I. M. (2018). Development of the Resource Use Questionnaire (RUQ–P) for families with preschool children with
neurodevelopmental disorders: Validation in children with autism spectrum disorder. Clinical Practice in Pediatric Psychology, 6(2), 164–178. https://doi.org/
10.1037/c
Volden, J., Duku, E., Shepherd, C., Georgiades, S., Bennett, T., Di Rezze, B., et al., Pathways in ASD Study Team. (2015). Service utilization in a sample of preschool
children with autism spectrum disorder: A Canadian snapshot. Paediatrics & Child Health, 20(8), e43–e47. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/
articles/PMC4699532/.
Warren, Z., McPheeters, M. L., Sathe, N., Foss-Feig, J. H., Glasser, A., & Veenstra-VanderWeele, J. (2011). A systematic review of early intensive intervention for
autism spectrum disorders. Pediatrics, 127(5), Article e1303. https://doi.org/10.1542/peds.2011-0426
Weitlauf, A., McPheeters, M., Peters, B., Sathe, N., Travis, R., Aiello, R.,. Warren, Z. (2014). Therapies for children with autism spectrum disorder: Behavioral
interventions update. Comparative effectiveness review No. 137. (Prepared by the Vanderbilt Evidence-based Practice Center under Contract No. 290–2012-
00009-I.) Retrieved from Rockville, MD: 〈www.effectivehealthcare.ahrq.gov/reports/final.cfm〉.
Yoder, P. J., Bottema-Beutel, K., Woynaroski, T., Chandrasekhar, R., & Sandbank, M. (2013). Social communication intervention effects vary by dependent variable
type in preschoolers with autism spectrum disorders. Evidence-Based Communication Assessment and Intervention, 7(4), 150–174. https://doi.org/10.1080/
17489539.2014.917780
Zachor, D. A., & Ben Itzchak, E. (2010). Treatment approach, autism severity and intervention outcomes in young children. Research in Autism Spectrum Disorders, 4(3),
425–432. https://doi.org/10.1016/j.rasd.2009.10.013

11