You are on page 1of 13

Curr Dev Disord Rep (2016) 3:33–45

DOI 10.1007/s40474-016-0068-8

AUTISM SPECTRUM (A RICHDALE, SECTION EDITOR)

Assessing Spoken Language Outcomes in Children


with ASD: a Systematic Review
David Trembath 1 & Marleen Westerveld 2 & Leanne Shellshear 3

Published online: 1 February 2016


# Springer International Publishing Switzerland 2016

Abstract Comprehensive early intervention programs based Keywords Comprehensive intervention . Early intensive
on the principles of applied behaviour analysis can promote behavioural intervention (EIBI) . Language . Autism spectrum
social communication development in children with autism disorder
spectrum disorder (ASD). However, it is not clear to what
extent these positive results represent progress towards help-
ing children achieve the ultimate goal of spontaneous, flexi- Introduction
ble, directed and effective spoken language for a range of
purposes across all life contexts. The aim of this systematic The ultimate communication goal for all children in the
review was to document the intervention targets and assess- preschool years is the development and use of spontane-
ment tools used to measure spoken language outcomes for ous, flexible, directed and effective spoken language for a
children with ASD receiving comprehensive intervention pro- range of purposes across all life contexts. This requires the
grams. A total of 23 studies met the inclusion criteria. coordinated acquisition of a sophisticated range of recep-
Although social communication functioning was the most fre- tive and expressive language skills across the domains of
quently targeted aspect of development, assessment of these morphology (word structure), syntax (sentence structure),
skills relied predominantly on the use of standardised assess- semantics (meaning), phonology (organisation of sounds),
ments. No studies included sampling and analysis of the chil- and pragmatics (interaction) [1]. For children with autism
dren’s spoken language in daily activities. The lack of com- spectrum disorder (ASD) who, by definition, have perva-
prehensive measures with high ecological validity limits our sive impairments in social communication skills and be-
ability to draw conclusions regarding spoken language out- haviour, the path to spoken language development is com-
comes from the studies to date. promised [2]. Accordingly, a range of evidence-based in-
terventions have been developed with some promising out-
comes for communication reported [3••]. In this article, we
review the skills targeted and assessment tools used to
measure intervention outcomes for children with ASD
This article is part of the Topical Collection on Autism Spectrum
and consider the suitability of these tools for measuring
* David Trembath changes in spoken language.
D.Trembath@Griffith.edu.au
Comprehensive Versus Focused Interventions

1
Menzies Health Institute Queensland, Griffith University, Odom, Collet-Klingenberg [4] used the terms ‘comprehensive
Gold Coast, Australia treatment models’ (CTMs) and ‘focused intervention practices’
2
Griffith Institute of Education Research, Griffith University, (FIPs) to categorise the range of interventions developed for
Brisbane, Australia children with ASD. Comprehensive treatment models comprise
3
School of Allied Health Sciences, Griffith University, a set of practices designed to address the core impairments of
Gold Coast, Australia ASD across multiple developmental domains, often delivered
34 Curr Dev Disord Rep (2016) 3:33–45

through a manualised program (e.g. Early Start Denver Model; standardised assessments to evaluate changes in intellectual
[5]). Focused intervention practices include a set of targeted functioning and adaptive behaviour. Five studies included
strategies designed to change a specific behaviour or to achieve standardised assessment of speech and language development.
a specific developmental outcome (e.g. the picture exchange Although standardised assessments are an essential tool for eval-
communication system (PECS) targeting requesting, uating outcomes [13], there are several challenges associated
commenting and response to questions; [6]). Based on a system- with their use with children with ASD. Matson [14], for exam-
atic review of research evidence, Maglione et al. [7] concluded ple, noted that the commonly used Vineland Adaptive
that children with ASD should have access to intensive compre- Behaviour Scales [15] was not designed for use with very young
hensive interventions, which may also be supplemented with children with ASD presenting with complex learning and lan-
focused interventions to meet their individual needs. Here, we guage difficulties, resulting in an artificial ceiling effect leading
focus on comprehensive early intensive interventions based on to low capacity to discriminate differences in skill level.
the principles of applied behaviour analysis. Furthermore, it is likely that when assessing young children with
ASD, changes in IQ or adaptive behaviour over time may often
Evidence for Comprehensive Interventions reflect changes in compliance and test-taking ability [14].
There is also concern that many commonly used
There is growing evidence that comprehensive interventions standardised assessment tools may not adequately capture chil-
can facilitate receptive and expressive language development dren’s level of functioning in everyday activities. Neisworth
in children with ASD. Reichow et al. [8], for example, com- and Bagnato [16], for example, argued that standardised assess-
pleted a Cochrane review of five studies in which early inten- ments conducted in highly controlled clinical settings with
sive behavioural interventions (EIBIs) were compared to treat- strict administration guidelines are unlikely to provide an accu-
ment as usual in the community for 203 children with ASD rate representation of each child’s strengths and difficulties
under 6 years of age. Based on the principles of applied behav- across a range of settings. McConachie et al. [11••], following
iour analysis, EIBIs target multiple developmental domains and their systematic review of assessment tools used in ASD re-
are delivered to young children at an intensity of between 20– search, concluded that (a) only a small number of tools had
40 h per week. Reichow et al. reported that EIBIs resulted in been shown to be valid measures for children with ASD and
superior gains in adaptive behaviour, social skills, intelligence, (b) that the information provided by commonly used assess-
communication and language, ASD symptoms and quality of ment tools aligned poorly with the information parents reported
life compared to treatment as usual in the community. to be valuable in judging outcomes. With regard to spoken
However, they noted that only a small number of studies were language, the parent-report McArthur Bates Communicative
of sufficient scientific quality to warrant inclusion and called Development Inventories (MB-CDI) [17] was the only assess-
for further research. Other researchers (e.g. [9]) have observed ment tool that was featured in the list of 12 tools presented in
that even within well-controlled studies yielding positive group the ‘summary of quality tools’ indicating that it met criteria for
results, considerable individual variability in outcomes is com- three or more quality indicators used in the review (i.e. used in
mon. This lack of high quality research studies and variability high quality study, blinding of assessors, can be used for chil-
in child outcomes presents major challenges for parents, edu- dren in the 0–6 years range and had been used by more than
cators, and researchers when attempting to infer the likely ben- one research group). While the MB-CDI has excellent face
efits of an intervention for a particular child, including the validity, provides a detailed measure of vocabulary, is relatively
possible impact on his or her receptive and expressive language quick to administer and has been shown to be a sensitive mea-
development [10]. sure of changes in child language (e.g. [18, 19]), it does not
provide an objective and comprehensive measure of children’s
The Challenge of Assessing Outcomes spoken language outcomes. McConachie et al. [11••] also not-
ed that although parent-report measures have excellent external
The challenge of interpreting findings is compounded by the validity, they are at greater risk of bias than clinician-delivered
selection of skills to be targeted (i.e. dependent variables) and direct assessments. These findings illustrate the challenge of
the manner in which outcomes are measured and reported in selecting appropriate assessment tools in ASD research, and
intervention studies. Parents, educators, researchers, and policy the need for parents, educators and researchers to not only
makers require data collected from outcome measures that are critically appraise the findings of research but also the assess-
sensitive, objective and ecologically and socially valid in order ment tools from which the outcome data have been derived.
to judge the effectiveness of a particular intervention [11••].
However, there is evidence to suggest that such measures are Measuring Spoken Language Outcomes
not always used. Matson and Rieske [12], for example, docu-
mented outcome measures used in EIBI research published from Given that supporting children to develop spoken language is a
1987 to 2013. Of the 25 studies presented, 22 utilised key goal of intervention, and considering the concerns regarding
Curr Dev Disord Rep (2016) 3:33–45 35

commonly used assessment tools, it seems prudent to examine and play; [30]) based on the principles of applied behaviour
the assessment tools commonly used in intervention studies. analysis (e.g. EIBI, discrete trial training), (c) the study contained
But, what might be a valid measure of spoken language? Key a control group, (d) participants had a diagnosis of ASD (i.e.
practice guidelines (e.g. (20–22]) for the assessment of commu- autism, autistic disorder, PDD-NOS) and (d) the majority of
nication skills in individuals at risk for, or diagnosed with ASD, participants commenced intervention prior to formal schooling.
include a thorough examination of (a) communicative forms
(e.g. gestures, vocalisations, eye gaze, physical actions, use of Search Procedures
augmentative communication, idiosyncratic behaviours includ-
ing behaviours of concern); (b) communicative functions (e.g. We completed searches of Medline, Psycinfo, and ERIC using
sharing information, requesting, protesting); (c) frequency, ef- the search terms ‘autis*’ AND ‘child*’ AND [‘intensive behav*
fectiveness and social quality of verbal and non-verbal commu- intervention’ OR ‘intensive’ OR ‘discrete trial training’ and ‘ap-
nication strategies; (d) the extent to which communication strat- plied behave* analysis’]. A total of 1077 articles were returned
egies are coordinated and (e) the use of atypical communication from initial searches. Article titles and abstracts from databases
strategies. While standardised assessments form an essential part were reviewed and 82 articles were short-listed for further anal-
of the recommended battery for assessing this broad range of ysis to determine eligibility against inclusion criteria. Interrater
skills, they do not necessarily provide a comprehensive picture reliability was calculated by comparing the papers for the
of each child’s skills and needs in everyday settings. Medline database independently selected by the first and second
Kasari et al. [23], following a review of assessment tools authors for short-listing, yielding agreement of 100 %. From the
for minimally verbal children with ASD, noted that ‘…most 82 short-listed articles, 32 were found to be reviews, meta-
of the measures have serious limitations for use with minimal- analyses or more general discussions of behavioural intervention
ly verbal children, which have severely impeded progress in practices; 27 articles were deemed not to meet inclusion criteria
both research and clinical practice’ (p. 12). They recommend- and 23 articles were progressed for data extraction. Reasons for
ed a comprehensive approach to measuring children’s com- exclusion included the following: (a) the study did not contain a
munication skills and outcomes, including the collection and control group, (b) the intervention was not based on principles of
analysis of language samples in the children’s everyday envi- applied behaviour analysis, (c) the study was continuing longi-
ronments. Kasari et al. noted that language samples are highly tudinal and therefore participants were beyond the desired age
valid for use with children with ASD, and provide useful range or the outcome measures had already been included from
measures of expressive language in both verbal and minimally another study and (d) the study did not include child-outcome
verbal children. Indeed, as Heilmann et al. [24] claimed, ‘lan- variables. We also completed a hand search of the reference lists
guage sampling and analysis has unparalleled validity for of review articles identified during the search and those of in-
measuring language use’ and has long been considered the cluded studies, yielding five additional articles that met the in-
gold standard approach for evaluating spoken language skills clusion criteria.
in children [25–27]. Given the pressing need to support, mea-
sure and report the development of spoken language in chil- Data Extraction
dren with ASD receiving comprehensive intervention pro-
grams, the aim of this review was to document (a) the skills The following information was extracted for each study: (a)
targeted and (b) assessment tools used to measure spoken number of participants and participants’ ages and diagnoses;
language outcomes for children with ASD receiving compre- (b) treatment type, frequency, duration and personnel respon-
hensive intervention programs in controlled studies. sible for implementation; (c) comparison intervention/s; (d)
dependent variables targeted and (e) assessment tools used
Method to measure dependent variables.

We conducted a systematic review of the literature according Results


to the methods outlined by Pickering and Byrne [28].
As presented in Table 1, 504 children with ASD were included
Inclusion Criteria in treatment groups across the 23 studies. Substantial variabil-
ity in methods used to confirm diagnosis is evident, ranging
The inclusion criteria were as follows: (a) articles published be- from inclusion based on community-based diagnosis of ASD
tween 1987 (year in which Lovaas published original article on to community diagnosis plus confirmation as part of the study
comprehensive applied behaviour analysis intervention for chil- using ASD-specific diagnostic tools. Although the breakdown
dren with ASD [29]) to July 2015 in peer-reviewed journals, (b) of subgroups (where possible under DSM-IV [52]) was pro-
the study included comprehensive intervention (i.e. addressing vided for a small number of studies (e.g. Dawson et al. [5]), in
core difficulties in autism including language, social, cognition most cases, readers are reliant on assessment results at Time 1
36 Curr Dev Disord Rep (2016) 3:33–45

to infer the children’s abilities. The children received a range language ability (e.g. number of recognisable words, verbal
of comprehensive interventions delivered by trained clini- imitation, expressive object labelling). However, none of the
cians, volunteers and/or parents based on clinician engage- studies in this review included the collection and analysis of
ment of between 1–40 h per week, over a period of between language samples in natural environments.
12 weeks to approximately 3 years. The two most common
comparison interventions were treatment as usual (seven stud-
ies) and eclectic interventions (seven studies), followed by Discussion
programs delivered in special education settings (five studies).
Our aim was to document the intervention targets and assess-
Dependent Variables Assessed ment tools used to measure spoken language outcomes for
children with ASD receiving comprehensive intervention pro-
The dependent variables (DVs) targeted in each study are listed grams. Our systematic review, spanning the 28 years since
in Table 1, using the terms reported in each study. To summarise Lovaas’ original study of applied behaviour analysis [29] for
the variables, we categorised them according to the skills children with ASD, yielded 23 studies that met our inclusion
targeted, resulting in the seven categories presented in Fig. 1. criteria. The results indicate that although supporting social
Social communication skills, intellectual functioning, and daily communication development is consistently a primary target
living skills were the three most frequently targeted aspects of of intervention, the dependent variables selected and the as-
development. The social communication category included sessment tools used to measure progress focus on broad mea-
broad measures of ‘communication’ (42 DVs), ‘expressive’ sures of communication development, predominantly through
and or ‘receptive’ language (20 DVs), following instructions (6 the use of standardised assessments in clinical settings and
DVs), object labelling (6 DVs), vocabulary (8 DVs), requesting parent report. No studies used language sampling and analy-
(2 DVs), response to name (2 DVs) and ‘words’ (2 DVs). sis, despite its validity for measuring language use in everyday
settings [24], thus limiting our capacity to draw comprehen-
Assessment Tools Utilised sive conclusions regarding the extent to which participants
achieved the ultimate communication goal of spontaneous,
As illustrated in Fig. 2, the five most frequently used assess- flexible, directed and effective spoken language for a range
ment tools included measures of intellectual functioning and of purposes across all life contexts.
receptive and expressive language, as well as a set of
customised assessment tools designed to measure a range of Communication as a Dependent Variable
adaptive and maladaptive behaviours. Note that only assess-
ment tools that were used to assess dependent variables in In most studies, children’s development of spoken language
each study are presented in Table 1 and Fig. 2, not tools used was assessed under a broad dependent variable such as ‘com-
for the sole purpose of participant characterisation. Although munication’ or ‘expressive language’. This approach is under-
assessments of intellectual functioning (e.g. Mullen Scales of standable, given the statistical imperative to preserve power
Early Learning [53]) and adaptive behaviour (e.g. Vineland by minimising the number of dependent variables in trials
Adaptive Behaviour Scales [15]) include expressive and re- featuring relatively small numbers of participants, as is com-
ceptive language domains, few dedicated language assess- mon in ASD research. This issue is particularly pertinent in
ments were included. Of the assessment tools designed spe- evaluations of comprehensive intervention programs targeting
cifically to measure language, the Reynell Developmental multiple developmental domains, with each requiring mea-
Language Scales (norm-referenced clinician-administered re- surement. Nevertheless, there is a risk that in selecting a broad
ceptive and expressive language scales; [54]) was the most dependent variable, important information regarding chil-
commonly used (nine studies), followed by the MacArthur dren’s functional skills across the domains of morphology,
Bates–Communicative Development Inventory (norm-refer- semantics, syntax, phonology and pragmatics will be lost or
enced parent-report tool assessing expressive and receptive masked. The alternative is to add to these broader dependent
vocabulary; two studies), Peabody Picture Vocabulary Test variables with more specific and descriptive language targets.
(norm-referenced receptive vocabulary assessment; two stud- Lovaas [29], for example, included a video-recorded play-
ies), the Clinical Evaluation of Language Fundamentals based observation measure of ‘recognizable words as part of
(norm-referenced clinician-administered broad spectrum re- the pretreatment assessment battery, defined to include any
ceptive and expressive language assessment; one study), and recognizable word, independent of whether the subject used
the Receptive-Expressive Emergent Language Scale (norm- it in a meaningful context or for communicative purposes^
referenced parent-interview receptive and expressive lan- (pp.50). Although the operationalisation of the dependent var-
guage subtests; one study). Four customised assessment tools iable is at odds with the definition for spoken language used in
(e.g. Early Learning Measure) included specific measures of this review (spontaneous, flexible, directed, etc), its inclusion
Table 1 Summary of dependent variables and outcome measures used in evaluation of comprehensive interventions

Study Design ASD treatment groupab Treatment Comparison Dependent variable/s Outcome measure
treatment

Lovaas 1987 [29] Non-RCT n = 19 Applied Behaviour analysis Low intensity Applied Intellectual functioning Wechsler Intelligence Scale for
Age: mean 34.6 months delivered 1:1 by trained behaviour analysis Children-Revised
Diagnosis: Autism (DSM-III) therapists. Approximately Stanford-Binet Intelligence Scale
40 h per week for 2 or Merrill-Palmer Preschool Performance
more years Test
Wechsler Preschool Scale
Vineland Social Maturity Scale
Gesell Infant Development Scale
Curr Dev Disord Rep (2016) 3:33–45

Bayley Scales of Infant Development


Cattell Infant Intelligence Scale
Peabody Picture Vocabulary Test
Educational placement Parent report of educational placement
Birnbrauer and Leach Non-RCT n=9 Murdoch early intervention Treatment as usual Communication, daily living, social, Vineland Adaptive Behaviour Scales
1993 [31] Age: 39 months (range 32–47) program, delivered 1:1 by and motor skills composite score
Diagnosis: Autistic disorder or parents and volunteers. Intellectual functioning Bayley Scales of Infant Development
PDD-NOS (DSM-III-R) Mean 18.72 h per week Stanford-Binet Intelligence Scale
for 24 months Leiter International Performance Scale
Peabody Picture Vocabulary Test
Wechsler Preschool and Primary Scale
of Intelligence
Wechsler Intelligence Scale for
Children
Receptive and expressive language Reynell Developmental Language
Scale
Receptive-Expressive Emergent
Language Scale
Emotional, behavioural, social and Personality Inventory for Children
cognitive adjustment
Play, self-stimulatory behaviour, Customised video assessment
withdrawal, compliance and involving structured play interaction
imitation under controlled with clinician
conditions
ASD symptoms Customised rating scale based on
DSM-III-R criteria
Smith, Groen, and RCT n = 15 UCLA early intensive Parent-administered Intellectual functioning Bayley Scales of Infant Development
Wynn 2000 [32] Age: 36.07 (6.0) Diagnosis: intervention, delivered 1:1 UCLA early Stanford Binet Intelligence Scale
Autism/PDD-NOS by clinicians. Mean intensive Merrill-Palmer Scale of Mental Tests
24.52 h per week in the intervention Receptive and expressive language Reynell Developmental Language
first the 12 months, with a Scales
reduction in hours
thereafter Communication, daily living, social Vineland Adaptive Behaviour Scales
and motor skills domain and
composite scores
Verbal imitation, nonverbal imitation, Early Learning Measure
following verbal instructions and
expressive object labelling
37
38

Table 1 (continued)

Study Design ASD treatment groupab Treatment Comparison Dependent variable/s Outcome measure
treatment

Eikeseth, Smith, Jahr, Non-RCT n = 13 Behavioral intervention Eclectic intervention Intellectual functioning Wechsler Preschool and Primary Scale
and Eldevik 2002 Age: 66.31 (11.31) delivered 1:1 by teachers, of Intelligence
[33] Diagnosis: Childhood autism aides, and parents. Mean Wechsler Intelligence Scale for
(ICD-10) 28.52 h per week for Children—revised
12 months Bayley Scales of Infant Development
Merrill-Palmer Preschool Performance
Test
Receptive and expressive language Reynell Developmental Language
Scale
Communication, daily living, social Vineland Adaptive Behaviour Scales
and motor skills composite score
Sallows and Graupner RCT n = 13 EIBI delivered 1:1 by EIBI delivered by Intellectual functioning Bayley Scales of Infant Development
2005 [34] Age: 33.23 (3.89) clinicians in clinical trained parents Wechsler Preschool and Primary Scale
Diagnosis: ASD (DSM-IV) setting. Mean 38.6 h per of Intelligence
week for 2 years Wechsler Intelligence Scale for
Children (3rd Ed.)
Bayley Scales of Infant Development
(2nd Ed.)
Receptive and expressive language Reynell Developmental Language
Scales
Clinical Evaluation of Language
Fundamentals (3rd Ed.)
Communication, daily living, and Vineland Adaptive Behaviour Scales
social skills domain and composite
scores
Autism symptoms Autism Diagnostic Interview—revised
Verbal imitation, nonverbal imitation, Early Learning Measure
following verbal instructions and
expressive object labelling
Howard, Sparkman, Non-RCT n = 29 Intensive behaviour analytic Public school Intellectual functioning Bayley Scales of Infant Development
Cohen, Green, and Age: 30.86 (5.16) treatment, delivered 1:1 classrooms for Wechsler Preschool and Primary Scale
Stanislaw 2005 [35] Diagnosis: ASD (DSM-IV) delivered by clinicians. children with ASD of Intelligence Differential Abilities
25–40 per week for Special education Scale
approximately 12 months classrooms for Stanford-Binet Intelligence Scale (4th
children with a Ed.)
range of disabilities Merrill-Palmer Scale of Mental Tests
Receptive and expressive language Reynell Developmental Language
Scales
Communication, daily living, social, Vineland Adaptive Behaviour Scales
and motor skills domain and
composite scores
Cohen, Amerine- Non-RCT n = 21 EIBI delivered 1:1 by Special education Intellectual functioning Bayley Scales of Infant Development
Dickens, and Smith Age: 30.2 (5.8) Diagnosis: clinicians. 35–40 h per classes at local Wechsler Preschool and Primary Scale
2006 [36] Autistic disorder or PDD- week for 3 or more years schools. of Intelligence—Revised
NOS Visual-spatial skills Merrill-Palmer Scale of Mental Tests
Expressive and receptive language Reynell Developmental Language
Scales
Curr Dev Disord Rep (2016) 3:33–45
Table 1 (continued)

Study Design ASD treatment groupab Treatment Comparison Dependent variable/s Outcome measure
treatment

Communication, daily living, and Vineland Adaptive Behaviour Scales


social skills domain and composite
scores
Eldevik, Eikeseth, Non-RCT n = 13 Behavioural intervention Eclectic intervention Intellectual functioning Bayley Scales of Infant Development
Jahr, and Smith Age: 53 (9.5) Diagnosis: delivered 1:1 by clinicians. Stanford-Binet Intelligence Scale
2006 [37] Childhood autism (ICD-10) Mean 15 h per week for Wechsler Preschool and Primary Scale
2 years of Intelligence Stanford-Binet
Intelligence Scale
Curr Dev Disord Rep (2016) 3:33–45

Wechsler Intelligence Scale for


Children—revised
Merrill-Palmer Preschool Performance
Test
Receptive and expressive language Reynell Developmental Language
Scale
Psycho Educational Profile-Revised
Adaptive behaviour: words, affection, Customised questionnaire and scoring
toy play, peer play, stereotypical system based on archival data, parent
behaviours, severe tantrums, toilet report or direct observation
training
‘Degree of mental retardation’ ICD-10 classifications for ‘mental
retardation’
Eikeseth, Smith, Jahr, Non-RCT n = 35 Applied behaviour analysis Treatment as usual Communication, daily living, social Vineland-II
and Eldevik 2007 Age: 4 years; 5 months (range delivered by school staff. skill, and motor skills composite
[38] 20–74) Approximately 23 h per score
Diagnosis: Autism (ICD-10) week for 1 year Autism symptoms Childhood Autism Rating Scale
Zachor, Ben-Itzchak, Non-RCT n = 45 Applied Behavior Analysis Eclectic intervention Autism symptoms Autism Diagnostic Observation
Rabinovich, and Age: 25.1 (3.9) delivered 1:1 by clinicians. Schedule
Lahat 2007 [39] Diagnosis: Autism (DSM-IV) Approximately 35 h per Intellectual functioning Bayley Scales of Infant Development
week for 12 months Stanford Binet Intelligence Scale (4th
Ed.)
Reed, Osborne, and Non-RCT n = 12 Applied Behavior Analysus Special nursery, Autism symptoms Gilliam Autism Rating Scale
Corness 2007 [40] Age: 40 months (range 32–47) delivered 1:1 by trained Portage program Imitation, perception, fine and gross Psycho-Educational Profile
Diagnosis: ASD tutors. Mean 30.4 h per motor skills, eye-hand coordination
week for 24 months and nonverbal and verbal conceptual
ability domain and composite
scores)
Intellectual functioning (verbal British Abilities Scale (2nd Ed.) Early
comprehension, early number Years Battery
concepts, picture matching and
naming vocabulary subtests)
Communication, daily living, social, Vineland Adaptive Behavior Scales
and motor skills domain and
composite scores
Educational achievement British Abilities Scale (2nd Ed.) Early
Years Battery
39
40

Table 1 (continued)

Study Design ASD treatment groupab Treatment Comparison Dependent variable/s Outcome measure
treatment

Reed, Osborne, and Non-RCT n = 14 ‘High intensity’ applied ‘low intensity’ applied Autism symptoms Gilliam Autism Rating Scale
Corness 2007 [41] Age: 42.9 (14.8) Diagnosis: behaviour analysis behavior analysis Imitation, perception, fine and gross Psycho-Educational Profile
ASD (community delivered 1:1 by trained motor skills, eye-hand coordination
diagnosis) tutors. Approximately 20– and nonverbal and verbal conceptual
40 h per week for 9– ability domain and composite
10 months scores)
Intellectual functioning (verbal British Abilities Scale (2nd Ed.) Ed
comprehension, early number Early Years Battery
concepts, picture matching and
naming vocabulary subtests)
Communication, daily living, social, Vineland Adaptive Behaviour Scales
and motor skills domain and
composite scores
Magiati, Charman, and Prospective n = 25 EIBI delivered 1:1 by Eclectic intervention Autism symptoms Autism Diagnostic Interview—revised
Howlin 2007 [42] follow-up Age: 38 (7.2) clinicians. Mean 32.4 h Intellectual functioning Bayley Scales of Infant Development
study Diagnosis: Autism per week for 24 months Wechsler Preschool and Primary Scale
(community diagnosis) of Intelligence—revised
Merrill-Palmer Scale of Mental Tests
Communication, daily living, social, Vineland Adaptive Behaviour Scales
and motor skills domains and
composite scores
Receptive vocabulary British Picture Vocabulary Scale-II
Expressive vocabulary Expressive One-Word Picture
Vocabulary Test—revised
Play skills Symbolic Play Test—II
Test of Pretend Play
Remington et al. 2007 Non-RCT n = 23 Early Intensive Behavioural Treatment as usual Intellectual functioning Stanford Binet Intelligence Scale (4th
[43] Age: 35.7 (4.0) Intervention delivered 1:1 Ed.)
Diagnosis: Autism, based on by trained tutors and Expressive and receptive language Reynell Developmental Language
community diagnosis and parents. Mean 25.6 h per Scales (3rd Ed.)
ADI-R week for 2 years
Communication, social, and daily Vineland Adaptive Behaviour Scales
living skills domain and composite
scores
Behaviour Nisonger Child Behaviour Rating Form
Developmental Behaviour Checklist
Autism symptoms Autism Screening Questionnaire
Joint attention (initiation and response) Early Social Communication Scales
Fava et al. 2011 [44] Non-RCT n = 12 EIBI delivered 1:1 and small Eclectic intervention Autism symptoms Autism Diagnostic Observation
Age: 52 (19.5) group across settings by Schedule
Diagnosis: Autism or PDD- staff and parents. Intellectual functioning Griffith Mental Developmental Scales
NOS Approximately 26 h per for ages 2–8
week for 12 months
Receptive and expressive language MacArthur Bates—Communicative
Development Inventories
Frequency and function of challenging Customised video-based challenging
behaviours behaviours assessment
Curr Dev Disord Rep (2016) 3:33–45
Table 1 (continued)

Study Design ASD treatment groupab Treatment Comparison Dependent variable/s Outcome measure
treatment

Communication, daily living, social, Vineland Adaptive Behaviour Scales


and motor skills domain and
composite scores
Psychopathological comorbidities Child Behaviour Checklist
Parenting stress Parenting Stress Index—Short Form
Hayward, Eikeseth, Non-concurrent n = 23 UCLA applied behaviour UCLA applied Intellectual functioning Bayley Scales of Infant Development
Gale, and Morgan multiple Age: 35.7 (6.2) analysis delivered 1:1 by behaviour analysis Wechsler Preschool and Primary Scale
Curr Dev Disord Rep (2016) 3:33–45

2009 [45] baseline Diagnosis: Autism (ICD-10) clinicians. Approximately delivered by parent- of Intelligence—revised
36 hours per week for organised clinicians Merrill-Palmer Scale of Mental Tests
12 months Receptive and expressive language Reynell Developmental Language
Scales
Communication, daily living, social, Vineland Adaptive Behaviour Scales
and motor skills domain and
composite scores
‘Non-vocal imitation’, ‘receptive Session treatment data
instruction’, ‘vocal imitation’ and
‘expressive labels’
Dawson et al. 2010 [5] RCT n = 24 Early Start Denver Model Treatment as usual Autism symptoms Autism Diagnostic Observation Scale
Age: 23.1 (3.9) delivered 1:1 by trained Intellectual functioning Mullen Scales of Early Learning
Diagnosis: 21 ASD, 3 PDD- clinicians and parents.
NOS (DSM-IV) Mean 15.2 clinician hours Communication, daily living, social Vineland-II
per week for 24 months skill, and motor skills composite
score
Parent questionnaire on repetitive Repetitive Behaviour Scale
behaviours
Peters-Scheffer, Non-RCT n = 12 Discrete trial training, Regular eclectic early Intellectual Functioning Bayley Scales of Infant Development
Didden, Mulders, Age: 53.5 (5.52) delivered 1:1 by intervention (Dutch version)
and Korzilius 2010 Diagnosis: ASD (DSM-IV) supervised preschool staff. program SON 2.5-7
[46] Mean 6.29 h per week for Communication, daily living, and Vineland Adaptive Behaviour Scales
8 months, in addition to social skills domain and composite (Dutch version)
regular eclectic early scores
intervention program Emotional and behaviour problems Child Behaviour Checklist (Dutch
version)
Communication, social behaviour, and Scale of Pervasive Developmental
stereotyped behaviour Disorder in Mentally Retarded
Persons (Dutch version)
Zachor and Ben Non-RCT n = 45 Applied behaviour analysis Eclectic intervention Autism symptoms Autism Diagnostic Observation
Itzchak 2010 [47] Age: 25.1 (3.9) delivered 1:1 by therapists Schedule
Diagnosis: Autism (DSM-IV) with certificate in ABA. Communication, daily living, social Vineland Adaptive Behaviour Scales
Approximately 20 h per skill, and motor skills
week for 12 months Visual reception, fine motor, receptive Mullen Scales of Early Learning
language, and expressive language
skills
Eldevik, Hastings, Non-RCT n = 31 Early Intensive Behavioural Treatment as usual Intellectual functioning Bayley Scales of Infant Development
Jahr, and Hughes Age: 42.2 (9.0) Intervention delivered 1:1 (2nd or 3rd Edition)
2012 [48] Diagnosis: Autism or PDD- by school staff trained by Stanford-Binet Intelligence Scales
NOS (ICD-10) psychologist. Mean 13.6 h
41
42

Table 1 (continued)

Study Design ASD treatment groupab Treatment Comparison Dependent variable/s Outcome measure
treatment

of treatment per week for Wechsler Preschool and Primary Scale


2 years Intelligence-Revised
Communication, daily living, and Vineland Adaptive Behaviour Scales or
social skills domains and composite Vineland-II
Reed and Osborne Non-RCT n = 14 Applied behaviour analysis, Special nursery Cognitive development Psycho-Educational Profile
2012 [49] Age = 39 (6.9) delivered 1:1 by trained Program, Educational achievement British Abilities Scale (2nd Ed.) Early
Diagnosis: Autistic disorder or tutors. Approximately Portage Program, Years Battery
PDD-NOS (DSM-IV) 30 h per week for Local authority
9 months approach Communication, daily living, social Vineland-II
skill, and motor skills composite
score
Rogers et al. 2012 [50] RCT n = 49 Early Start Denver Model Treatment as usual Social affect, restricted and repetitive Autism Diagnostic Observation
Age: 12–24 Based parent intervention. behaviours Schedule
Diagnosis: ‘At risk for ASD’ Parents coached to deliver Intellectual functioning Mullen Scales of Early Learning
treatment during 12 × 1 h
training sessions over Expressive and receptive vocabulary Macarthur-Bates Communicative
12 weeks Development Inventory
Communication, daily living, social, Vineland-II
and motor skills
Reitzel et al. 2013 [51] RCT n=8 Functional Behavioural Skills Treatment as usual Communication, daily living, social, Vineland-II
Age: 57.9 (13.1) Training delivered by a and motor skills
Diagnosis: Autistic Disorder teacher, a parent coach and Severity of problem behaviours Developmental Behaviour Checklist
child prompters. Weekly Adaptive behaviour (requesting an item Customised Functional Behaviour
2 h group sessions for or activity, hand washing, eating at Skills Assessment
4 months table, response to name, toileting)

a
Age in months (mean/SD) unless otherwise indicated
b
Table presents diagnostic terminology used in each article
Curr Dev Disord Rep (2016) 3:33–45
Curr Dev Disord Rep (2016) 3:33–45 43

demonstrates the potential value of including descriptive, eas- ultimate goal for children with ASD to become successful com-
ily interpreted, variables in intervention research. Notably, a municators in everyday contexts, the importance of spontaneous
brief scan of Table 1 reveals that earlier studies more often language sampling and analysis cannot be ignored (see also
included dependent variables related to specific communica- [56]). Condouris et al. [13] raised concerns that language sam-
tive behaviours (e.g. vocal imitation, expressive object label- pling and analysis is generally time consuming and presented
ling, use of words) than later studies. Given the increasing evidence that standardised language assessments may correlate
awareness of the need to study individual differences in inter- well with measures derived from language samples. However,
vention outcomes, and the calls for the development of more the finding was based on a study involving children who were all
sensitive and socially valid outcome measures [11••], it may able to complete the selected standardised assessments within
be timely to consider what can be learned from these earlier their age level, rather than the broader spectrum of children with
endeavours to document intervention outcomes. ASD, including those with intellectual disability or those who
show limited verbal skills. Results from a study of minimally
verbal children with ASD [57] showed that these children pro-
Assessing Communication Outcomes duced more spontaneous communication than elicited commu-
nication, indicating that standardised tests may at times underes-
The majority of studies used standardised assessments to mea- timate the children’s expressive language abilities. Furthermore,
sure intervention outcomes. These assessment tools have clear results from standardised tests will fail to yield the descriptive
benefits including standardised administration procedures, the detail needed for careful, individualised intervention planning or
capacity to compare children’s functioning with peers of similar progress monitoring [58]. Technology has the potential to assist
age, and the opportunity to compare the findings across different with both the collection (e.g. Language Environment Analysis;
intervention studies [13]. The Reynell Developmental Language LENA) and analysis (e.g. Systematic Analysis of Language
Scales [54], for example, was the third most commonly used Transcripts; SALT, [59]) of language samples and thus help to
assessment tool across the studies reviewed. It is a norm- reduce the time demands. Therefore, language sampling need
referenced assessment, appropriate for young children (2 to not be considered an alternative to standardised assessments,
7 years, 6 months), which includes a mix of play-based activities but rather a crucial complementary method designed to provide
using a range of stimulus activities designed to assess both re- a comprehensive picture of children’s intervention outcomes.
ceptive and expressive language skills, including understanding Language sampling and analysis can help to confirm the results
and production of selected vocabulary and grammatical features.
Kjellmer et al. [55] suggested that the use of concrete objects and Wechsler
pictures may be appealing to children with neurodevelopmental Bayley
disorders. However, there are documented limitations with the Reynell
use of standardised assessments, including the contexts in which Stanford-Binet
behaviours are assessed and the potential confound of test-taking ADOS
ability on test performance, which may limit their capacity to Merrill-Palmer
capture each child’s functional capacity across a range of BAS
contexts. PEP/PEP-R
Several authors have advocated for greater use of language Merrill-Palmer…
sampling and analysis in ASD research, but this approach was MSEL
not featured in any of the studies reviewed. Considering our Custom
Other
0 10 20 30 40
Frequency
Fig. 2 Assessment tools used to measure treatment outcomes across
studies. a Assessment tools published in multiple versions/editions have
been combined to yield frequency count, b ‘custom’ refers to outcome
measures developed for use in the study, rather than published assessment
tools, c ‘other’ refers to assessment tools that were used in fewer than
three studies included in the review, d standardised tests, including
original and revised editions where relevant: Wechsler Intelligence
Scale for Children, Wechsler Preschool and Primary Scale of
Intelligence, Wechsler Preschool Scale; Bayley Scales of Infant
Development; Reynell Developmental Language Scales; Stanford-Binet
Intelligence Scale; Autism Diagnostic Observation Schedule; Merrill-
Palmer Scale of Mental Tests British Abilities Scale; Psycho-
Educational Profile; Merrill-Palmer Preschool Performance Test;
Fig. 1 Summary of dependent variables targeted across the 23 studies Mullen Scales of Early Learning
44 Curr Dev Disord Rep (2016) 3:33–45

of standardised assessments, provide rich descriptive informa- 4. Odom SL et al. Evidence-based practices in interventions for chil-
dren and youth with autism spectrum disorders. Prev Sch Fail:
tion for intervention planning, and contribute to detailed moni-
Altern Educ Child Youth. 2010;54(4):275–82.
toring of children’s response to intervention [27, 58]. 5. Dawson G et al. Randomized, controlled trial of an intervention for
Furthermore, evaluating children’s language in everyday inter- toddlers with autism: the Early Start Denver Model. Pediatrics.
actions with parents, teachers and peers would also help to 2010;125(1):e17–23.
address McConachie et al.’s [11] call for the use of ecologically 6. Bondy A, Frost L. The picture exchange communication system.
Behav Modif. 2001;25(5):725–44.
valid outcome measures in ASD research. 7. Maglione MA et al. Nonmedical interventions for children with
ASD: recommended guidelines and further research needs.
Pediatrics. 2012;130 Suppl 2:S169–78.
8. Reichow B et al. Early intensive behavioral intervention (EIBI) for
Conclusion young children with autism spectrum disorders (ASD). Cochrane
Database Syst Rev. 2012;10:CD009260.
Given that promoting spoken language development in chil- 9. Tonge BJ et al. A review of evidence-based early intervention for
dren with ASD is a primary goal of comprehensive interven- behavioural problems in children with autism spectrum disorder:
the core components of effective programs, child-focused interven-
tions, it is imperative that the way it is targeted (i.e. in terms of
tions and comprehensive treatment models. Curr Opin Psychiatry.
dependent variables) and measured (i.e. in terms of assess- 2014;27(2):158–65.
ment tools) is the subject of careful consideration when criti- 10. Trembath D, Vivanti G. Problematic but predictive: individual dif-
cally appraising the results of studies to date and designing ferences in children with autism spectrum disorders. Int J Speech
future studies. The results of this review indicate that there is Lang Pathol. 2014;16(1):57–60.
11.•• McConachie H et al. Systematic review of tools to measure out-
an opportunity to move the field forward by supplementing comes for young children with autism spectrum disorder. Health
the use of standardised assessments with language sampling Technol Assess. 2015;19(41):1–506. The most comprehensive re-
and analysis in children’s everyday environments. Such an view of assessment tools for individuals with ASD currently
approach would afford parents, educators and researchers available.
12. Matson JL, Rieske RD. Are outcome measures for early intensive
more accurate and detailed insights into the real-world im- treatment of autism improving? Res Autism Spectr Disord.
pacts of comprehensive interventions for children with ASD. 2014;8(3):178–85.
13. Condouris K, Meyer E, Tager-Flusberg H. The relationship be-
tween standardized measures of language and measures of sponta-
Compliance with Ethical Guidelines neous speech in children with autism. Am J Speech-Lang Pathol.
2003;12(3):349–58.
Conflict of Interest David Trembath reports grants from the National 14. Matson JL. Determining treatment outcome in early intervention
Health and Medical Research Council (GNT1071881), during the con- programs for autism spectrum disorders: a critical analysis of mea-
duct of the study. surement issues in learning based interventions. Res Dev Disabil.
Marleen Westerveld and Leanne Shellshear declare that they have no 2007;28(2):207–18.
conflict of interest. 15. Sparrow SS, Cicchetti DV, Balla DA. Vineland Adaptive Behavior
Scales—2nd edition (Vineland—II). Livonia: Pearson
Assessments; 2005.
Human and Animal Rights and Informed Consent This article does
not contain any studies with human or animal subjects performed by any 16. Neisworth JT, Bagnato SJ. The mismeasure of young children—the
of the authors. authentic assessment alternative. Infants Young Child. 2004;17(3):
198–212.
17. Fenson L, Dale PS, Reznick JS et al. Technical manual for the
MacArthur communicative development inventories. San Diego:
San Diego State University; 1991.
18. Green J et al. Parent-mediated communication-focused treatment in
children with autism (PACT): a randomised controlled trial. Lancet.
References 2010;375(9732):2152–60.
19. McConachie H et al. A controlled trial of a training course for
Papers of particular interest, published recently, have been parents of children with suspected autism spectrum disorder. J
Pediatr. 2005;147(3):335–40.
highlighted as:
20. Filipek PA et al. The screening and diagnosis of autistic spectrum
•• Of major importance disorders. J Autism Dev Disord. 1999;29(6):439–84.
21. Raising Children Network. Parent guide to therapies. 25/05/15];
1. Paul R, Norbury CF. Language disorders from infancy through Available from: http://raisingchildren.net.au/parents_guide_to_
adolescence. Listening, speaking, reading, writing, and communi- therapies/parents_guide_to_therapies.html
cating. 4th ed. Baltimore: Mosby Elsevier; 2012. 22. NIASA, N.I.f.A.S.a.A. National Autism Plan for Children (NAPC):
2. Howlin P et al. Adult outcome for children with autism. J Child plan for the identification, assessment, diagnosis and access to early
Psychol Psychiatry. 2004;45:212–29. interventions for pre-school and primary school aged children with
3.•• National Autism Center. National Standards Report. Randolph, autism spectrum disorders (ASD). London: The National Autistic
MA: National Autism Centre; 2015. This report presents a com- Society; 2003.
prehensive review of the evidence-base for ASD interventions 23. Kasari C et al. Assessing the minimally verbal school-aged child
based on research conducted in the period 2007-2012. with autism spectrum disorder. Autism Res. 2013;6(6):479–93.
Curr Dev Disord Rep (2016) 3:33–45 45

24. Heilmann JJ, Miller JF, Nockerts A. Using language sample data- 43. Remington B et al. Early intensive behavioral intervention: out-
bases. Lang Speech Hear Serv Sch. 2010;41(1):84–95. comes for children with autism and their parents after two years.
25. Tager-Flusberg H, Cooper J. Present and future possibilities for Am J Ment Retard. 2007;112(6):418–38.
defining a phenotype for specific language impairment. J Speech 44. Fava L et al. The effectiveness of a cross-setting complementary
Lang Hear Res. 1999;42(5):1275–8. staff- and parent-mediated early intensive behavioral intervention
26. Dunn M et al. The use of spontaneous language measures as criteria for young children with ASD. Res Autism Spectrum Disord.
for identifying children with specific language impairment: an at- 2011;5(4):1479–92.
tempt to reconcile clinical and research incongruence. J Speech 45. Hayward D et al. Assessing progress during treatment for young
Hear Res. 1996;39(3):643–54. children with autism receiving intensive behavioural interventions.
27. Miller JF. Progress in assessing, describing, and defining child lan- Autism. 2009;13(6):613–33.
guage disorder. In: Cole CN, Dale PS, Thal DJ, editors. Assessment 46. Peters-Scheffer N et al. Low intensity behavioral treatment
of communication and language. Baltimore: Paul Brookers supplementing preschool services for young children with autism
Publishing Co.; 1996. p. 309–324. spectrum disorders and severe to mild intellectual disability. Res
28. Pickering CM, Byrne J. The benefits of publishing systematic quan- Dev Disabil. 2010;31(6):1678–84.
titative literature reviews for PhD candidates and other early career 47. Zachor DA, Itzchak EB. Treatment approach, autism severity and
researchers. High Educ Res Dev. 2014;33:534–48. intervention outcomes in young children. Res Autism Spectrum
29. Lovaas OI. Behavioral treatment and normal educational and intel- Disord. 2010;4(3):425–32.
lectual functioning in young autistic children. J Consult Clin 48. Eldevik S et al. Outcomes of behavioral intervention for children
Psychol. 1987;55(1):3–9. with autism in mainstream pre-school settings. J Autism Dev
30. Rogers SJ, Vismara LA. Evidence-based comprehensive treatments Disord. 2012;42(2):210–20.
for early autism. J Clin Child Adolesc Psychol. 2008;37(1):8–38. 49. Reed P, Osborne L. Impact of severity of autism and intervention
31. Birnbrauer JS, Leach DJ. The Murdoch early intervention project time-input on child outcomes: comparison across several early in-
after 2 years. Behav Chang. 1993;10(2):63–74. terventions. Br J Spec Educ. 2012;39(3):130–6.
50. Rogers SJ et al. Effects of a Brief Early Start Denver Model
32. Smith T, Groen AD, Wynn JW. Randomized trial of intensive early
(ESDM)-based parent intervention on toddlers at risk for autism
intervention for children with pervasive developmental disorder.
spectrum disorders: a randomized controlled trial. J Am Acad
Am J Ment Retard. 2000;105(4):269–85.
Child Adolesc Psychiatry. 2012;51(10):1052–65.
33. Eikeseth S et al. Intensive behavioral treatment at school for 4- to 7-
51. Reitzel J et al. Pilot randomized controlled trial of a functional behav-
year-old children with autism. A 1-year comparison controlled
ior skills training program for young children with Autism Spectrum
study. Behav Modif. 2002;26(1):49–68.
Disorder who have significant early learning skill impairments and
34. Sallows GO, Graupner TD. Intensive behavioral treatment for chil-
their families. Res Autism Spectrum Disord. 2013;7(11):1418–32.
dren with autism: four-year outcome and predictors. Am J Ment
52. American Psychiatric Association. Diagnostic and statistical man-
Retard. 2005;110(6):417–38.
ual of mental disorders. Washington, DC: American Psychiatric
35. Howard JS et al. A comparison of intensive behavior analytic and Association; 1994.
eclectic treatments for young children with autism. Res Dev 53. Mullen E. Mullen scales of early learning. Cicle Pines, MN:
Disabil. 2005;26:359–83. American Guidance Service; 1985.
36. Cohen H, Amerine-Dickens M, Smith T. Early intensive behavioral 54. Reynell J, Huntley M. Reynell developmental language scales—
treatment: replication of the UCLA model in a community setting. J revised. 3rd ed. Windsor: National Foundation for Educational
Dev Behav Pediatr. 2006;27(2):S145–55. Research Nelson; 1987.
37. Eldevik S et al. Effects of low-intensity behavioral treatment for 55. Kjellmer L et al. Language comprehension in preschoolers with
children with autism and mental retardation. J Autism Dev autism spectrum disorders without intellectual disability: use of
Disord. 2006;36(2):211–24. the Reynell Developmental Language Scales. Res Autism
38. Eikeseth S et al. Outcome for children with autism who began Spectrum Disord. 2012;6(3):1119–25.
intensive behavioral treatment between ages 4 and 7: a comparison 56. Kim SH, Junker D, Lord C. Observation of Spontaneous Expressive
controlled study. Behav Modif. 2007;31(3):264–78. Language (OSEL): a new measure for spontaneous and expressive
39. Zachor DA et al. Change in autism core symptoms with interven- language of children with autism spectrum disorders and other com-
tion. Res Autism Spectrum Disord. 2007;1(4):304–17. munication disorders. J Autism Dev Disord. 2014;44(12):3230–44.
40. Reed P, Osborne LA, Corness M. Brief report: Relative effective- 57. Chiang HM. Naturalistic observations of elicited expressive com-
ness of different home-based behavioral approaches to early teach- munication of children with autism: an analysis of teacher instruc-
ing intervention. J Autism Dev Disord. 2007;37(9):1815–21. tions. Autism. 2009;13(2):65–78.
41. Reed P, Osborne LA, Corness M. The real-world effectiveness of 58. Westerveld M. Spontaneous language sampling and analysis in
early teaching interventions for children with autism spectrum dis- children: from research to practice. ACQuiring Knowledge
order. Except Child. 2007;73(4):417–33. Speech Lang Hearing. 2011;13(2):63–7.
42. Magiati I, Charman T, Howlin P. A two-year prospective follow-up 59. Miller JF, Andriacchi K, Nockerts A et al. Assessing language
study of community-based early intensive behavioural intervention production using SALT software. A clinician’s guide to language
and specialist nursery provision for children with autism spectrum sample analysis. New Zealand version. Middleton: SALT Software;
disorders. J Child Psychol Psychiatry. 2007;48(8):803–12. 2012.

You might also like