683887

research-article2017
AUT0010.1177/1362361316683887AutismLaw et al.

Original Article

Autism

The use of behavior modeling training 1­–16

© The Author(s) 2017
Reprints and permissions:
in a mobile app parent training program sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/1362361316683887
https://doi.org/10.1177/1362361316683887

to improve functional communication journals.sagepub.com/home/aut

of young children with autism spectrum

disorder

Gloria C Law, Maureen Neihart and Anuradha Dutt

Abstract
Communication intervention in early life can significantly impact long-term outcomes for young children with autism.
Parents can be vital resources in the midst of the current manpower shortage. Map4speech is a new mobile application
developed for parents of children with autism spectrum disorder. It is specially designed to provide high-quality, interactive
learning, coupled with frequent feedback and live coaching to train parents in a naturalistic language intervention. A
multiple-baseline single-case experimental design was conducted across three parent–child dyads. Results indicate that
procedural integrity of parents’ intervention techniques was above 85% during post-training intervention, and their
respective children showed increases in spontaneous word/gesture use. The results show that mobile applications with
feedback can be a promising means for improving efficiency and effectiveness in disseminating evidence-based practices
for autism intervention.

Keywords
autism intervention, behavior modeling training, mobile technology, naturalistic developmental and behavioral
intervention, parent training

Parent training (PT) has long been viewed as an impor-
tant component of early intervention programs for chil-
dren with autism spectrum disorders (ASD) (Harris,
1994). It contributes to the effectiveness of behavioral
treatments by therapists (Schreibman, 2000; Schreibman
and Anderson, 2001), enhances functional communica-
tion in young children with ASD (Moes and Frea, 2002),
and improves parental coping and mental health (Tonge
et al., 2006). Most parent-mediated intervention pro-
grams have used behavioral or naturalistic intervention
strategies to improve young children’s communication
outcomes because they have demonstrated efficacy and
effectiveness and allow treatment delivery in natural con-
texts within everyday routines (Brookman-Frazee et al.,
2006; Nevill et al., 2016; Patterson et al., 2012;
Schreibman et al., 2015; Schultz et al., 2011).
Most PT programs using naturalistic developmental
and behavioral interventions (NDBIs) rely on face-to-face,
live sessions by trained therapists to provide feedback to
parents. Such feedback is considered essential to help par-
ents achieve fidelity in intervention. Attempts to leverage
advances in technology to expand training modalities have
been increasing, but there are few randomized controlled
trial (RCT) studies investigating the outcomes of such
efforts (Vismara et al., 2013; Wainer and Ingersoll, 2014).
Wainer and Ingersoll’s (2014) web-based, interactive
training program combined self-directed learning compo-
nents with a remote live feedback component to train five
parent–child dyads in naturalistic imitation training. Results
were promising but highly variable. Small increases in chil-
dren’s spontaneous imitation were observed during the
self-directed learning condition, but imitation levels varied
considerably during the coaching condition. Two children
demonstrated decreasing trends in their imitation levels.
National Institute of Education, Nanyang Technological University,
Singapore
Corresponding author:
Maureen Neihart, Psychological Studies Academic Group, National
Institute of Education, Nanyang Technological University, 1 Nanyang
Walk, Singapore 637616.
Email: maureen.neihart@nie.edu.sg
2 Autism
All but one parent achieved fidelity. Parents reported satis-
faction with the program but noted limitations and barriers
such as difficulties accessing the video conferencing pro-
gram and expressed a desire for more coaching sessions
and additional video demonstration clips.
Similarly, Vismara et al. (2013) used video conferenc-
ing and a self-guided website to train eight parents in a
naturalistic intervention in their homes over a 12-week
intervention with 3-month follow-up. The curriculum con-
sisted of 10 intervention topics from the Early Start Denver
Model (ESDM; Rogers and Dawson, 2010). Individualized
parent feedback sessions occurred once a week for 90 min
via a secure video conferencing portal. Within each 90-min
feedback session, parents were invited to participate in a
10-min parent–child interaction of chosen play activities
over the two-way video call. These interactions provided
the data for weekly outcome measures. Results indicated
that parents made steady progress from baseline to follow-
up with their skill development and needed about 7 weeks
on average to reach fidelity. Children’s rates of functional
verbal utterances and non-verbal joint attention initiations
improved overall across conditions, but rates of joint atten-
tion did not change until follow-up. Most parents said they
thought the feedback sessions were very important to their
learning.
The initial results of these telehealth efforts are exciting
and they highlight some of the benefits and challenges to
providing parent-mediated NDBI training via technology.
Studies are needed to further investigate the extent to
which it is possible to make PT even more individualized,
flexible, portable, and accessible. A major concern and
important limitation of most telehealth PT programs to
date is their lack of a conceptual framework for adult
learning. Learning frameworks are important because they
illuminate the nature of training, the factors affecting the
delivery of training, and the constructs influencing the
transfer of training (Ford et al., 2014). Learning via tech-
nology platforms is not only just about content and compo-
nents but also about learning across contexts (e.g. Elias,
2011; Gedik et al., 2012; Park, 2011). Park (2011) states,
“The most serious issue faced by mobile learning is the
lack of a solid theoretical framework which can guide
effective instructional design …” (p. 3). Identifying a con-
ceptual framework for learning is especially important for
telehealth and mHealth where the design of components
and features must consider scalability and feasibility in
real-life contexts and sustainability over time (Park, 2011).
Behavior modeling training (BMT) was chosen as the
conceptual framework for the app’s design because it is
one of the best researched and most widely used adult
training interventions. BMT is based on Bandura’s (1976)
social learning theory and has been used to train both psy-
chomotor and interpersonal skills (e.g. Decker, 1980;
Roessger, 2012). Bandura (1976) proposed that modeling
occurs with four sub-processes: (a) attention, (b) retention,
(c) production, and (d) motivation. In essence, learners
need to observe the modeled behavior, remember what the
model did, do what the model has done, and desire to apply
what they have learned. These four sub-processes are evi-
dent in the five components in BMT, that is, (a) modeling,
(b) retention, (c) behavioral rehearsal, (d) feedback, and
(e) transfer of training (Decker and Nathan, 1985).
Extensive evaluations of BMT (Taylor et al., 2005) have
concluded that BMT is an effective training intervention,
producing sustainable improvements across a variety of
skills and post-training behaviors. It has the potential to be
a good fit for PT and as an instructional design for interac-
tive mobile technology. The app’s components were
designed to address each of these sub-processes at every
stage of parents’ learning.
The purpose of this study was to build on the strengths
of these early efforts in telehealth parent-mediated inter-
vention to investigate how a specially designed mobile
app, in conjunction with video feedback sessions, could
serve as a device for parents of children with ASD to learn
NDBI and thereby increase the accessibility of evidence-
based services. The primary purpose of the study was to
examine the effectiveness of a new mobile app,
Map4speech, which uses NDBI techniques in highly
interactive features to increase feedback to parents so that
they improve both their intervention skills and the func-
tional communication of their child with ASD. The study
contributes to the literature in three ways. First, it pro-
vides a rationale and demonstration for why and how a
learning framework can be used to guide the development
of PT. Second, it illustrates how frequency and individu-
alization of feedback in PT can be provided easily via a
mobile app. Most importantly, it provides evidence that
such training can achieve sustainable, positive outcomes
for both parents and their children with ASD. Details of
the app’s curriculum development are provided in the
‘Methods’ section to follow and in Law (2016).
Methods
Design
A multiple-baseline single-case design with multiple set-
tings was conducted across three parent–child dyads
(Kazdin, 2011). There were five phases in this experi-
ment: baseline, PT, post-training intervention (PTI), novel
settings, and 1-month follow-up. Phases 1, 2, and 3 (i.e.
baseline, PT, and PTI) used a concurrent design and were
conducted in the children’s homes with the same toy
materials within each parent–child dyad. Phases 4 and 5
(i.e. novel settings and 1-month follow-up) used a non-
concurrent design and were conducted to learn how par-
ents generalized their new skills in different contexts. The
study was approved by the Nanyang Technological
University Institutional Review Board.
Law et al. 3
Table 1.  Participants’ characteristics.
Dyad Dyad Parent’s Parent’s Child’s age
ethnicity education occupation (months)
Dyad 1 Chinese University Homemaker 52
degree
Dyad 2 Malay Post- Homemaker 44
secondary
Dyad 3 Indian Post- Allied 30
secondary educator
CDI: Communicative Development Inventories.
aSparrow et al. (2005).
bThe scores were acquired from an earlier diagnostic assessment administered at 4 years of age.
cThe MacArthur-Bates Communicative Development Inventories: words and gestures/words and sentences (Fenson et al., 2007).
Participants
Three young children with ASD (30–52 months old) and
their mothers participated in the study. Those children
who were actively involved in early intervention and
speech/occupational therapy sessions and parents who
were not English speakers were excluded. Participants
were recruited from wait lists for intervention services at
two local intervention centers that had agreed to distribute
information and invitations about the study to parents
waiting for services for their children. In total, 17 parents
responded to the invitation and of those, 4 met the inclu-
sion criteria. First, all participating children met the cut-
off for autism on the Autism Diagnostic Observation
Schedule: ADOS-2 (Lord et al., 2012) and the criteria for
ASD in the Diagnostic and Statistical Manual of Mental
Disorders (5th ed.; DSM-V; American Psychiatric
Association (APA), 2013). Second, all child participants
exhibited limited verbal repertories (i.e. non-verbal to sin-
gle words). Child 1 usually requested items by pulling his
mother’s hand or pointing to things he wanted. He labeled
things using single words. Child 2 and 3 were non-verbal.
They pulled their mother’s hand to indicate what they
wanted. They did not use any pointing for request at the
intake. Parents completed the Vineland Adaptive Behavior
Scales (Sparrow et al., 2005) and MacArthur-Bates
Communicative Development Inventories (CDI) (Fenson
et al., 2007) to determine the adaptive functioning and
language age of each child participant. Each parent gave
written consent for their participation. See Table 1 for the
functioning and language levels.
Materials
Each parent was loaned an iPad containing the Map4speech
mobile app which was developed by the authors. The inter-
vention program introduces naturalistic language interven-
tion (NLI; Charlop-Christy and Carpenter, 2000; Koegel
et al., 1987), which is also considered as naturalistic devel-
opmental behavioral interventions (NDBI). The current
Child Vineland Adaptive Language age Communication goals
gender Behavior (MacArthur for request
Compositea CDI)c
Male 66 (low)b 25 months Use two-word phrases
Male 63 (low) 11 months Point with index finger
or use single words
Female 70 (low) 14 months Point with index finger
or use single words
program adopted and utilized a compact curriculum from
Project imPACT (Ingersoll and Dvortcsak, 2010) with a
key goal of eliciting functional communication from
young children with ASD. The selection and adoption of
key naturalistic strategies for the curriculum was a thought-
ful process which involved consideration of not only what
was minimally essential to be effective but also implemen-
tation feasibility and local customization (Neta et al.,
2015). The intervention adopted developmental elements
by conducting intervention within the context of the child’s
daily routine and environment, setting goals with reference
to typical child development, and increasing parent’s
responsiveness to the behaviors of children with ASD. It
also utilized techniques based on applied behavior analysis
which emphasizes prompting language via direct prompt-
ing with the use of natural reinforcement embedded in
children’s choice for activity or object. See Table 2 for the
curriculum outline. In addition to the training components,
the app was designed to collect usage data automatically.
This data included time and date and frequency and dura-
tion of use for every feature for every skill taught. These
data were automatically uploaded into a downloadable
excel sheet in the server.
Map4speech is a mobile app training program that uses
the framework of BMT to teach the user to conduct natu-
ralistic intervention with young children with ASD to
improve their functional communication (see Figure 1 for
selected screenshots of the mobile app). The mobile train-
ing includes eight learning stages, each of which teaches
one to two intervention skills under four curriculum con-
tent areas (i.e. (a) follow your child’s lead, (b) imitate and
animate, (c) make moments for togetherness, and (d)
prompt, reward, and expand). In each training stage, there
are five application features that correspond to the BMT
sequential process, that is, (a) instructional videos (mode-
ling and retention), (b) quick check (retention), (c) interac-
tive game (retention), (d) practice video (behavioral
rehearsal), and (e) feedback (feedback and transfer of
training) (see Table 3). Each iPad was also installed with
Skype™, an application software for free video chat and
4 Autism

Table 2.  The catalog of the mobile app interface.

Curriculum Mobile app learning stages

Introduction Stage 1: Introductiona
Part 1: Follow your child’s lead Stage 2: Follow your child’s lead
Part 2: Imitate and animate Stage 3: Imitate and animate
  Stage 4: Expand your child’s language
Part 3: Make moments for togetherness Stage 5: Make moments for togetherness
Part 4: Prompt, reward, and expand Stage 6: Prompt, reward, and expand 1
  Stage 7: Prompt, reward, and expand 2
Summary Stage 8: Putting it altogethera
aAll
stages are supported with five application features, that is, instructional videos, quick check, interactive game, practice video, and feedback
except for Stages 1 and 8 with no interactive game.

Figure 1.  Selected screenshots of the main pages of mobile app interface.

Table 3.  BMT learning component and the corresponding app features and functions.

BMT learning component Representing app icon Mobile app features and functions
Modeling “Instructional Video” Videos with highlighted images, voiceovers, and written statements
Retention “Instructional Video” Videos with highlighted images, voiceovers, and written statements
  “Quick Check” Online quizzes—multiple choice questions related to the content
of the Instructional Video
  “Interactive Game” Digital game presented situated intervention opportunities
which prompted for responses from learners after watching the
Instructional Video and the Quick Check
Behavioral rehearsal “Practice Video” Video recording function; upload feature for sending videos
Feedback “Feedback” Skype™ video chat and text chat feature for feedback

BMT: behavior modeling training.

Law et al. 5
voice call. Parents were to work through the training com-
ponents and the stages sequentially. They were prevented
from accessing other components or stages by locked
icons until they had passed a certain component/stage.
Following a toy preference assessment, parents were also
provided with a unique set of high-preferred toys that were
developmentally appropriate and stimulating for each
child participant. Six to eight toys were provided for each
family. These included soap bubbles, building blocks, a
train set, toy cars, and a spinning toy, among others.
Procedure
Three parent–child dyads attended intake assessment ses-
sions at their homes. The intake consisted of collecting
parent and child demographic data, conducting toy prefer-
ence assessments with each child to identify highly pre-
ferred toys using Reinforcer Assessment for Individuals
with Severe Disabilities (RAISD, Fisher et al., 1992), and
collecting baseline data and instructing parents in the use
of the mobile app program.
In Phase 1 baseline, participants were assigned to differ-
ent baseline periods according to their time preferences for
home appointments. The shortest baseline period lasted 1
day with 5 sessions, and the longest baseline period lasted
7 days with 14 sessions. During baseline, parents were
video-recorded in free play with their child for 2 min in
each session. Parents were instructed to play with their
child as they usually do with the set of highly preferred
toys. No specific guidelines were given for how they should
play. Researchers determined a communication goal for
each child after discussion with each respective parent.
Table 1 records the communication goals for each child.
In Phase 2 PT, parents learned to implement NLI in a
progressive and sequential learning sequence using the
mobile app. The four curriculum content areas (i.e. PT1,
PT2, PT3, and PT4) were organized into eight mobile app
learning stages (see Table 2 for the curriculum and the cor-
responding learning stage). Each stage utilizes the same
mobile app components—instructional videos, quick check,
interactive game, practice video, and feedback. Parents
completed the training stages at their own pace. The app
was designed so that they could not skip any learning com-
ponent until they had completed each requirement (e.g.
watched the entire instructional video in order to unlock the
quick check icon; attained 100% accuracy in quick check in
order to access the interactive game). Parents were led by
the app to practice their learned intervention skills for each
stage. For 5 days a week, they were asked to spend 15 min
to practice the learned intervention skills with their child.
They used the app to take two or three 2-min video clips of
themselves practicing the skills with their child and uploaded
them to a secure server via the app. The effects of the mobile
app training were evaluated moment-by-moment. They
were encouraged to use the intervention techniques
throughout the day in their daily routines; however, only the
uploaded practice videos were evaluated. Uploaded videos
were scored by the researchers using a fixed protocol (see
Table 4 for behavioral variables and definitions). Once the
researcher received parent’s practice videos for that day, she
scored and provided feedback over Skype to improve and
strengthen parent’s intervention skills. The researcher fol-
lowed a fixed procedure in giving feedback to each parent
after each submission of practice videos. The key compo-
nents of a feedback session included using the “share
screen” function in Skype to watch the practice videos
together with parents, affirming parent’s rightful application
of skills, and highlighting and demonstrating the missing
skills. All feedback sessions over Skype were video-
recorded for scoring and for maintaining the researcher’s
treatment integrity on feedback administration (see
Appendix 1 for the criteria of procedural integrity for psy-
chologist’s feedback). The researcher who provided the
feedback was a psychologist with 15 years of experience in
autism assessment and intervention. Parents were required
to attain 90%–100% of the intervention skills in two con-
secutive practice videos in order to advance to the next stage
of the app where they learned new skills while continuing to
use the previously learned skills. Parents were expected to
attain all 10 parent intervention behaviors by the end of
Phase 2 PT. Parents needed to achieve at least 90% fidelity
of implementation for the intervention over in two consecu-
tive videos before proceeding to PTI. Each parent pro-
gressed at her own pace. The average PT period was around
6.7 weeks or 33 weekdays (see Table 5 for additional
details). The average feedback sessions in Phase 2 were 19
sessions across three parents. Parent 1 received 22 feedback
sessions (M = 20 min; range = 10–36 min). Parent 2
received 18 feedback sessions (M = 16 min; range = 6–25
min). Parent 3 received 17 feedback sessions (M = 14 min;
range = 7–22 min).
In Phase 3 PTI, parents continued to implement the
intervention procedure with their child and video-
recorded their sessions on the iPad. Videos were scored
by the researcher, but feedback was provided only when
their intervention skills fell below 80% fidelity instead of
being given after each submission of practice videos as in
Phase 2. The average post-training period was around 9
days across three parents. The average feedback sessions
in Phase 3 were two sessions across three parents. Parent
1 received three feedback sessions (M = 26 min; range =
18–38 min). Parent 2 received two feedback sessions (M
= 12 min; range=10–14 min). Parent 3 received one
feedback session of 12 min.
Phase 4 of the experiment involved conducting the
intervention in novel settings. All parents participated
immediately after Phase 3 except for dyad 3, which took a
1-month break to attend to family demands. There were
two novel settings—community playgrounds and family
dining areas. At the playground, parents were free to
6 Autism

Table 4.  Parent behavior variables and definitions.

Parent variables Definitions

1 Wait and follow the child Parent waits for the child to choose his activity as long as the child is taking time to choose his
toys or activities. Parent must wait for child to start playing with his preferred toy or activity
first before he joins in, especially at the beginning of the play session.
2 Stay face-to-face with your Parent stays within child’s line of sight. Parent may need to sit at child’s level and orientate
child himself toward the child so that he may make eye contact with him occasionally for at least 45 s.
3 Join in child’s play Child initiates to play with the preferred item or activity and parent joins in and facilitates child’s
play as he or she engages in these items/activities. For example, parent can hand play items to
child or add something to his play.
4 Give comment Parent says something related to the toys child is playing (e.g. say “bubbles”), child’s actions (e.g.
“up”), comments on his or her own actions (e.g. “daddy stacks”), or the situation (“We are
having so much fun”)for at least once in 2 min. It can be a sentence.
5 Imitation Parent fulfills 5a or 5b or 5c.
 5a Imitate child’s play Parent imitates what child is doing with a toy (excluding imitating inappropriate behavior) for at
least 12 s intermittently (note: not the child imitating the parent).
 5b Imitate child’s gestures/body Parent imitates and exaggerates child’s gestures or body movements (excluding imitating
movements inappropriate behavior) for at least once in 2 min.
 5c Imitate child’s word or vocal Parent imitates child’s vocalizations or words (excluding imitating inappropriate behavior) at
sounds least once in 2 min.
6 Animate in play Parent exaggerates her gestures (e.g. point with a bigger movement), facial expressions (e.g. make
smiles more obvious and bigger), or vocal quality (e.g. by increasing the volume or the tone) to
attract child’s attention, for at least 12 s intermittently or four distinctive instances (e.g. say “uh-
oh,” “oh-no,” “Wow” sound effects, audible gasps).
7 Expand child’s language Parent uses the pre-determined set goals and words which are suggested by the psychologist to
model and expand child’s language for at least once in 2 min.
8 Communication temptations Parent fulfills 8a or 8b or 8c.
 8a Use playful obstruction While child is actively playing with a toy, parent triggers child’s communication using the
following steps:
1. Use the same anticipatory phrase (e.g. “1-2-3-stop”).
2. “Playfully” obstruct his play.
3. Pause and provide a target word before letting child resume his play.
 8b Use balance turn taking While child is actively playing with a toy, parent triggers child’s communication using the
following steps:
1. Say “Child name’s turn” while child is playing with the toy.
2. Use an anticipatory phrase to get a turn (e.g. “Mummy’s turn”).
3. Take a turn for a few seconds.
4. Pause and provide a target word before returning the toy to child (e.g. it can be Johnny’s
turn).
 8c Use other communicative Parent triggers child’s communication using the following steps:
temptations 1. Put child’s favorite item within his visual sight but out of his reach (out of reach) OR withhold
parts of toys (e.g. fruits without knife)(missing portion) OR hold toys up in front of her
own face (control access) OR provide materials/opportunities child needs to help to play
(assistance), then;
2. Pause, provide a target word before giving the child the favorite item/activity.
9 Prompt Parent fulfills 9a or 9b or 9c.
 9a Verbal prompt Parent prompts using words or phrases for child to imitate for request. Verbal prompts can
include verbal modeling (e.g. “say bubbles”), verbal delay (e.g. parent pauses, gasps, and delays
in saying the target word, “I want … bubbles”), verbal choice (e.g. “do you want blue or red?”),
verbal instruction (e.g. “point!”), or verbal question (e.g. what do you want?).
This is different from giving comments as parent will continue to give prompts (e.g. verbal
prompt, physical prompt) to pursue child’s response to use the target word or gesture.
 9b Gesture prompt Parent prompts by showing a gesture to child (within his line of sight) so that he will imitate for
request (e.g. pointing to an object).
 9c Physical prompt Parent prompts by holding the child’s hand to produce a gesture for request (e.g. shape his hand
to point to an object).
10 Reward or praise Parent fulfills 10a or 10b.
 10a Reward child Parent rewards child immediately with what he asks for (i.e. give the tangible item within 5 s).
 10b Praise child Parent praises the child immediately (within 5 s) after he has made a request (prompt or without
prompt) with much enthusiasm and animation, like “yay!”, “well done.” Praise can be given to
spontaneous requests of the child.

choose any playground equipment that was highly pre- parents were free to choose any food items that were
ferred by their child at this stage. In family dining areas, highly preferred. Parents implemented the intervention
Law et al. 7

Table 5.  Number of practice videos produced and the time spent by each parent across phases.

Number of practice Days/weeks used Average number

videos produced of videos per day
Phase 2: Parent training
  Parent 1 52 23 days/4.4 weeks 2.26
  Parent 2 43 36 days/7.4 weeks 1.19
  Parent 3 48 40 days/8.2 weeks 1.2
Phase 3: Post-training intervention
  Parent 1 21 10 days/2 weeks 2.1
  Parent 2 20 13 days/2.4 weeks 1.54
  Parent 3 15 4 days/0.6 weeks 3.75
Phase 4: Novel setting—playground
  Parent 1 20 12 days/2.1 weeks 1.66
  Parent 2 17 7 days/1.3 weeks 2.43
  Parent 3 20 8 days/1.9 weeks 2.5
Phase 4: Novel setting—snack time
  Parent 1 18 8 days/1.4 weeks 2.25
  Parent 2 20 9 days/1.7 weeks 2.22
  Parent 3 15 7 days/1.3 weeks 2.14
Phase 5: 1-month follow-up
  Parent 1 3 2 days 1.5
  Parent 2 3 1 day 3
  Parent 3 3 2 days 1.5

with their child and recorded their sessions using the app
in the respective sites. Feedback was provided only when
parent fidelity fell below 80%, or at the parent’s request,
instead of being given after each submission of practice
videos as in Phase 2. Only two to four feedback sessions
were given to each parent across two novel settings. The
average intervention period on the playground and the
snack time setting were 8 and 7 days, respectively. The
average feedback sessions for two novel settings were 3.3
sessions across three parents. Parent 1 received four feed-
back sessions (M = 16 min; range = 4–26 min). Parent 2
received four feedback sessions (M = 15 min; range =
12–17 min). Parent 3 received two feedback sessions
(M = 10 min; range = 8–12 min).
During the 1-month follow-up, parents were asked to
record three intervention sessions using the mobile app with
their child at the same home settings and upload them to the
server. The researcher did not provide feedback in these
follow-up sessions, but all uploaded videos were scored.
At the conclusion of the mobile app training program,
parents completed the Treatment Acceptability Rating
Form-Revised (TARF-R; Reimers et al., 1991) for social
validity measures. They also participated in a 1-h semis-
tructured interview to better understand their experience
using the app.
Dependent measures
Parent procedural fidelity.  In total, 10 parent behavior var-
iables were measured in discrete categorization of
occurrence/non-occurrence in each 2-min practice video
(i.e. yes/no). See Table 4 for behavioral definitions.
Scores of behavior occurrences were converted into a
summary score in percentage (0%–100%) and indicated
the extent of application of the 10 parent intervention
behaviors in each practice video.
Child’s language and pointing gestures (spontaneous/
prompted).  Spontaneous utterances, prompted utterances,
spontaneous pointing gestures, and prompted pointing
gestures were scored in frequency counts in each 2-min
practice video. See Table 6 for child behavior variables
and definitions.
Social validity.  Social validity and treatment acceptability
were measured with the TARF-R (Reimers et al., 1991).
TARF-R is commonly used to measure treatment accepta-
bility in clinical settings and consists of 21 items, using a
Likert-type seven-point scale. It has an internal consist-
ency of 0.92 (Carter, 2007).
Inter-observer agreement
The inter-observer agreement (IOA) for parents’ interven-
tion behaviors and child functional communication were
obtained for at least 33% of all practice videos by a trained
observer in each phase. The trained observer was a psy-
chologist for children with special needs. The IOA on the
performance criteria for feedback sessions was obtained
for at least 25%. The percentages of IOA ranged between
8 Autism

Table 6.  Child behavior variables and definitions.

Child variables  
1 Spontaneous Child uses a meaningful speech sequence consisting of one or more words spontaneously
utterances (mands/ without parental prompts to request an object or an activity or to label an action, an emotion,
tacts) or an object. Must be in context and meaningful. Examples can be “give me,” “please,” “(name of
object).” Exclude echolalia.
2 Prompted utterances Child uses a meaningful speech sequence consisting of one or more words within 5 s after
(mands/tacts) parental prompts to request an object that will be delivered by the parent. Or child uses a
meaningful speech sequence consisting of one or more words within 5 s after parental prompts to
label an action, an emotion, or an object. Must be in context and meaningful. Exclude echolalia.
3 Spontaneous Child uses his or her index finger to point to a preferred object/activity spontaneously for the
pointing gestures purpose to request the object/activity. It must be self-initiated without parental prompts. Exclude
gestures such as giving an object to an adult, grabbing a toy, taking things from others, reaching,
pulling adult’s hand to a certain direction, and showing.
4 Prompted pointing Child uses his or her index finger to point to a preferred object/activity with parental prompts
gestures for the purpose to request the object/activity. The prompted pointing gesture must be done after
one of the parental prompts (physical prompt, gesture prompt, verbal prompt (e.g. Johnny, point
to bubbles)). Exclude gestures such as giving an object to an adult, grabbing a toy, taking things
from others, reaching, pulling adult’s hand to a certain direction, and showing.

90% and 90.6% for parents’ intervention behaviors across
parents, between 88.3% and 93% for child’s functional
communication across children, and between 85% and
97% for the performance criteria for feedback sessions
across participants.
Data analysis
Behavior data.  The behavior data were analyzed by visual
inspection of graphs to determine the functional relations
between the variables. This is a standard and commonly
accepted method for multiple-baseline single-case designs
(Kazdin, 2011). In addition, the non-overlap techniques of
percentage of non-overlapping data (PND) and non-over-
lap of all pairs (NAP) were used as indices of the amount
of improvement by participants (Parker et al., 2011).
Data from intake and end of program assessment. Simple
descriptive statistics were used to analyze data from the
social validity questionnaire.
Results
Parents’ intervention behaviors
Results indicated that parents significantly improved their
intervention behaviors and maintained high fidelity to the
intervention using the mobile app. In addition, substantial
improvements in children’s functional communication
were also observed. Parents’ intervention behaviors in all
intervention phases (Phases 2–5) improved substantially
in comparison to baseline (Phase 1) with a positive upward
trend. See Figure 2 for a summary graph of percentage of
parents’ intervention behaviors. The PND for all three
participants indicate that the overall intervention program
was fairly effective to highly effective (i.e. Parent 1’s
PND = 88%, effective; Parent 2’s PND = 91%, very
effective; Parent 3’s PND = 95%, very effective) (Scruggs
and Mastropieri, 1998). The NAP results indicate inter-
vention phases had a significant effect on all intervention
skills across three parent participants with high probabil-
ity scores of data non-overlap (Parent 1’s NAP = 0.97, p
< 0.05; Parent 2’s NAP = 0.98, p < 0.05; Parent 3’s NAP
= 0.98, p < 0.05).
In Phase 1 baseline, average intervention behaviors
were at moderately low levels (Range of means 35%–
38.2%) across the three participants. All parents demon-
strated some parent target behaviors across baseline
sessions but consistently lacked some parent target behav-
iors which are derived from naturalistic/behavioral
approaches, such as “animation,” “playful obstruction/bal-
ance turn-taking/communication temptations,” “verbal/
gesture/physical prompts,” and “reward/praise.”
In Phase 2 PT, the range of percentage of parent inter-
vention behaviors rose from 40% to 100% across the three
participants, indicating that they had learned to use six or
seven new intervention behaviors during the course of PT.
The gradual increment of behaviors suggest that all three
participants needed to learn these behaviors across multi-
ple trials with the feedback support from the researcher in
order to apply the appropriate behaviors for their child and
settings. Parent 1 had 52 practice videos with a highly vari-
able upward trend (M = 67.9%; range = 30%–100%).
Parent 2 had 43 practice videos with a highly variable
upward trend (M = 71.2%; range = 40%–100%). Parent 3
had 48 practice videos with a stable substantial upward
trend (M = 69%; range = 30%–100%).
In Phase 3 PTI, results indicated that parents contin-
ued to maintain a high level of procedural fidelity across
all 10 parent target behaviors after the PT phase, although
Law et al. 9

Figure 2.  Frequency of child’s prompted utterances/pointing gestures across three child participants and percentage of parent’s
intervention behaviors across all phases.
Bar: frequency of child’s prompted utterances; line of open circles: frequency of child’s prompted pointing gestures; line of diamonds: parent’s inter-
vention behaviors in percentage; PT1: part 1: follow your child’s lead; PT2: part 2: imitate and animate; PT3: part 3: make moments for togetherness;
PT4: part 4: prompt, reward and expand; //: 1-month break.

fluctuating levels in intervention behaviors were observed
across the three participants, with outlying low scores
occurring at least once (range of scores in PTI—Parent 1:
50%–100%, Parent 2: 60%–100%, Parent 3: 60%–100%).
These fluctuating levels suggested that parents still
needed time to become more fluent and consistent in
applying the target intervention behaviors. At this stage,
all three participants maintained high procedural fidelity
levels (mean scores in PTI: Parent 1 = 88.6%, Parent 2 =
91.5%, Parent 3: 88%), compared to the average level
displayed during PT (mean scores in PT: Parent 1 =
67.9%, Parent 2 = 71.2%, Parent 3: 69%) and at baseline
(mean scores at baseline: Parent 1 = 35%, Parent 2 =
38.2%, Parent 3: 37.1%).
In Phase 4 novel settings, parents spent 17–20 sessions
in the first novel setting—playground. Both Parent 1 and
Parent 2 maintained a high level of intervention behaviors
in this phase (Parent 1: M = 87.5%, range = 80%–100%;
Parent 2: M = 86.5%, range = 70%–100%). Parent 3’s
intervention behaviors at the playground deteriorated after
the 1-month break but still maintained a moderately high
level of procedural fidelity (i.e. above 80% on average;
M = 82.5%, range = 60%–100%). Her behaviors at the
playground were also variable, but Parent 1 and Parent 2’s
intervention behaviors at the playground were rather stable
with a slightly upward trend.
Parents spent 15–20 sessions in the second novel set-
ting—snack time. All three parents continued at high lev-
els of procedural fidelity in this novel setting with a stable
and slightly upward trend (Parent 1: M = 91.1%, range =
80%–100%; Parent 2: M = 96.5%, range = 80%–100%;
Parent 3: M = 94.7%, range = 70%–100%).
In Phase 5: 1-month follow-up, all three parents main-
tained high levels of procedural fidelity across the three
sessions conducted without any follow-up Skype session.
Procedural fidelity of Parent 1, Parent 2, and Parent 3’s
intervention behaviors at 1-month follow-up ranged
between 80% and 100%.
In summary, the three parents were able to maintain a
high level of implementation fidelity. They achieved 88%–
92% fidelity for intervention implementation during the
PTI, 83%–88% during novel setting—playground, 91%–
97% during novel setting at snack time, and 90%–93%
during the 1-month follow-up.
Child’s prompted/spontaneous language or
gestures
Prompted language or gestures. Figure 2 shows the fre-
quency of prompted utterances (bars) and prompted point-
ing (lines of open circles) for each child participant. Child
1’s communication goal was to use two-word phrases for
10 Autism
request, while Child 2 and Child 3’s communication goal
were to point their index finger for request. Child 2 and 3
gave prompted utterances sparingly throughout interven-
tion phases. For instance, Child 2 had one prompted utter-
ance in three sessions among 20 sessions in Phase 3 and
Child 3 had one prompted utterance in four sessions among
15 sessions in Phase 3. Therefore, only prompted language
for Child 1 and prompted gestures for Child 2 and 3 are
elaborated in details.
Child 1 produced more prompted utterances during
the intervention phases in comparison to baseline. The
result of NAP indicated the intervention phases had a sig-
nificant effect on the production of Child 1’s prompted
utterances as it showed a high probability score of data
non-overlap (NAP = 79%, p < 0.05). During Phase 1
baseline, Child 1’s prompted utterances were variable
with a slightly downward trend, ranging from 0 to 7
prompted utterances (M = 1.1; range = 0–7). In inter-
vention phases 2–5, there was a variable but steady
increase in more prompted utterances. Child 1 increased
his prompted utterances to an average of three instances
in each 2-min session (M = 3.41; range = 0–11).
Both Child 2 and Child 3 produced more prompted
pointing during the intervention than in baseline. The
results of NAP indicated the intervention phases had a sig-
nificant effect on Child 2 and Child 3’s prompted pointing
as the figures showed high probability scores of data non-
overlap (Child 2’s NAP = 80%, p < 0.05; Child 3’s NAP
= 76%, p < 0.05). Child 2 and Child 3 had no prompted
pointing during Phase 1 baseline and during PT 1, 2, and 3
of Phase 2 PT because the intervention curriculum did not
require parents to use gesture prompts yet. During the last
part of PT4, Child 2 and Child 3 pointed with prompts 2.2
and 1.5 times per session, respectively. In intervention
phases 2–5, both Child 2 (M = 1.31; range = 0–6) and
Child 3 (M = 1.02; range = 0–4) on average increased at
least one prompted pointing in each 2-min session com-
pared to their baseline sessions. These results indicated
that parents’ interventions were effective in increasing
Child 2 and Child 3’s functional communication.
Spontaneous language or gestures.  Figure 3 shows the fre-
quency of spontaneous utterances (bars) and prompted
pointing (lines of open circles) for each child participant.
Child 1 consistently presented spontaneous utterances but
Child 2 and Child 3 gave spontaneous utterances sparingly
throughout the intervention phases. For instance, Child 2
had spontaneous utterances in 7 sessions among 43 ses-
sions in Phase 2, and Child 3 had spontaneous utterances
in 4 sessions among 48 sessions in Phase 2. Only Child 2
showed a substantial increase in spontaneous pointing to
request an object or an activity. Child 3 used spontaneous
pointing to request infrequently. Therefore, only results of
spontaneous language for Child 1 and results of spontane-
ous pointing for Child 2 are presented.
Child 1 produced more spontaneous utterances during
the intervention phases than in baseline. Results of the
NAP indicated that the intervention phases had a signifi-
cant effect on the production of Child 1’s spontaneous
utterances as it showed a high probability score of data
non-overlap (NAP = 82%, p < 0.05). During Phase 1
baseline, Child 1’s spontaneous utterances were at a low
level with a stable zero trend (M = 0.1; range = 0–1). He
did not utter any spontaneous words/phrases in all base-
line sessions except one session where he said “bubbles.”
However, he made 8–10 spontaneous utterances in the last
two sessions of Phase 2 PT. On average, he presented
more than two spontaneous utterances (M = 2.45; range
= 0–13) of three to five words in each of the 2-min ses-
sions during intervention phases 2–5.
Child 2 learned to request by pointing spontaneously
to an object across home, playground, and snack time set-
tings and maintained this skill at a consistent level. The
results of the NAP indicated the intervention had a sig-
nificant effect on the production of his spontaneous
pointing as there was a high probability score of data
non-overlap (NAP = 72%, p < 0.05). During Phase 1
baseline and Phase 2 PT, Child 2 did not use any sponta-
neous pointing. Parent 2 started to use physical or gesture
prompts during PT part 4; however, Child 2 did not make
any spontaneous pointing until Phase 3 PTI. On average,
Child 2’s spontaneous pointing was variable and more
than one instance (M = 1.34; range = 0–10) in each of
the 2-min sessions during intervention phases 2–5.
Social validity
The three parent participants completed the TARF-R
pre- and post-intervention. All three participants
reported good levels of program acceptability following
the short introduction of the program (80.7%–88.2%),
and these TARF-R scores increased by 7%–10% across
all parent participants to scores ranging between 90.8%
and 96.6% after completion of the program. These find-
ings indicate that the Map4speech was highly accepta-
ble to parents.
Automated usage data
All usage data were uploaded to the server except the fre-
quency and duration of viewing of instructional videos.
This appeared to be a design error. A preliminary review of
the data indicated there was considerable variability in
both frequency and duration of usage across participants
for each feature of the app. For example, while one parent
was able to successfully complete every quiz on the first
try, another parent usually required three or four tries.
Similarly, there was considerable variability across all
users in the number of tries needed to pass each game. For
example, the range was one to six tries for the first game
Law et al. 11

Figure 3.  Frequency of child’s spontaneous utterances/pointing gestures across three child participants and percentage of parent’s
intervention behaviors across all phases.
Bar: frequency of child’s spontaneous utterances; line of open circles: frequency of child’s spontaneous pointing; line of diamonds: parent’s interven-
tion behaviors in percentage; PT1: part 1: follow your child’s lead; PT2: part 2: imitate and animate; PT3: part 3: make moments for togetherness;
PT4: part 4: prompt, reward and expand; //: 1-month break.

and three to four for game 6. A thorough analysis of this
large data set is not possible here but is under preparation.
Discussion
The aim of this research was to evaluate the effectiveness
of using a mobile app PT program to improve the func-
tional communication of children with autism. Results
demonstrated that parents gained significant NLI skills
over a few months with the help of a newly developed
mobile app that included performance feedback on use of
strategies based on video, Map4speech, and their children
with ASD also substantially improved their functional
communication. Child participants used more functional
words/phrases or gestures and attempted more frequently
to request their favorite objects/activities.
These findings have four important implications. First,
they provide further evidence that naturalistic intervention
is effective in improving functional communication in
young children with autism. Second, they support the
effectiveness of PT to improve functional communication
in young children with autism. Third, they suggest that
mobile applications, based on adult learning theory that
include opportunities for practice and professional feed-
back, have potential as effective training and intervention
tools, and fourth, they provide evidence that BMT may be
an effective pedagogical framework for the design of
future training apps. These implications are discussed fur-
ther in the sections that follow.
Effectiveness of naturalistic intervention
Two child participants (Child 1 and Child 2) in the study
improved their functional communication, not only in their
immediate home settings but also in two other settings.
Child 1 increased his use of prompted and spontaneous
phrases when requesting objects in the 2-min sessions. His
speech utterances also lengthened substantially, up to four
to six words per spontaneous phrase. Child 2 learned to
point spontaneously to request for his favorite toys/activi-
ties/food items. Child 3 started to use pointing, aided by
her parent’s physical/gesture prompts for requests.
These results suggest that young, non-verbal, or mini-
mally verbal children can learn to use spoken language or
functional gestures to communicate after receiving NLI.
This is consistent with most of the early interventions and
communication literature for children with ASD (e.g.
Ingersoll and Dvortcsak, 2006; Koegel et al., 1987;
Stadnick et al., 2015). Differential results for the three
children were expected, given their differences in age and
12 Autism
abilities at baseline. Child 1, for example, demonstrated
gestural joint attention at baseline, while Child 2 and 3 did
not and was therefore stronger in the developmental foun-
dations needed for speech than the other two participants
at the start of the program.
Effectiveness of PT
The results of this study further demonstrate that parents
can learn intervention through a short-term and intensive
partnership with therapists, and that they can achieve fidel-
ity levels high enough to generate positive, sustained com-
munication outcomes in children with ASD (Vismara
et al., 2009; Warren et al., 2011). The National Research
Council (2001) has stated that inclusion of parents as treat-
ment providers for their own children is an essential aspect
of autism intervention. There is strong evidence that PT
can improve children’s verbal communication skills,
reduce problem behavior, and develop appropriate play
skills. In addition, intervention delivered by trained par-
ents has been shown to increase generalization and main-
tenance of treatment gains over time (Koegel et al., 1982).
However, despite demonstrations that PT programs are
effective in changing children’s behaviors, many chal-
lenges have remained in the implementation of such pro-
grams. Some require heavy time commitments that are not
possible for many parents who work full-time. Other
obstacles include long waiting lists for access to validated
interventions and financial costs which are not usually
covered by insurance.
In an effort to address these challenges, Vismara et al.
(2009) demonstrated that a 1 h a week, parent-mediated
intervention for 12 weeks could lead to sustained growth
in children’s social communication. Their non-current,
multiple-baseline study demonstrated that a PT program
that provided 1 h a week for 12 weeks of individualized
parent–child education program to eight toddlers newly
diagnosed with autism could achieve lasting outcomes for
children. Parents were trained in naturalistic therapeutic
techniques from the ESDM and achieved high fidelity of at
least 85% by the fifth or sixth intervention session and
maintained at follow-up for several parents. Gains contin-
ued for six of the children at 3-month follow-up and high
fidelity was maintained at follow-up for several parents.
Vismara et al. (2009) noted several limitations to their
study, including the non-concurrent design, which limited
control to external threats to validity, the reliance on sub-
jective measures for children’s outcomes, and the clinic-
based delivery, which they said economizes the clinicians’
time but limits accessibility for parents. They suggested
adding home delivery in addition to clinic delivery to
strengthen the model.
This study similarly provided brief PT and improved
upon earlier studies by employing a concurrent design,
objectively measuring children’s outcomes, providing
home-based delivery, and rigorously measuring fidelity
for both the parents’ implementation of the intervention
and for the psychologist’s adherence to the feedback pro-
tocol (Wainer and Ingersoll, 2013). Results demonstrated
that parents achieved high levels of integrity during the
PTI and could maintain this integrity across several natural
environments. Implementation of intervention with high
fidelity in real-world contexts has been considered a chal-
lenge, and many PT studies do not either report parent
fidelity data or demonstrate low or variable fidelity rates
(e.g. Meadan et al., 2009; Rakap and Rakap, 2014). In this
study, parents’ adherence to the intervention was essen-
tially forced by the design of the mobile app, which did not
allow them to advance to the next stage until they achieved
at least 90% fidelity. It is possible that some users would
find this requirement frustrating or stressful and may sub-
sequently give up using the app. Scaling studies are needed
to evaluate the app’s effectiveness and social validity to
determine its greater potential as a training tool for large
numbers of parents across populations. The effectiveness of
the Map4speech program in developing and maintaining
users’ fidelity has implications for further professional train-
ing and development in autism intervention. Monitoring
fidelity of implementation in intervention over time may
improve retention of intervention staff, possibly because
their self-efficacy improves as they receive emotional sup-
port and targeted coaching (Aarons et al., 2009).
Potential of mobile applications
The positive results of this study are encouraging because
they imply that mobile applications have good potential as
an intervention training tool. They may provide a viable
solution to long-standing problems with manpower short-
ages and could significantly reduce the waiting time for
intervention of children with ASD and their families. This
small success using a mobile app tool to train parents using
an evidence-based practice has important economic impli-
cations as it suggests that intervention costs could poten-
tially be greatly decreased (Neta et al., 2015). In this study,
each parent and their child spent less than 2 h per week on
average with this PT program to achieve their intervention
goal. The psychologist coach spent an average of 60–90
min giving feedback to each parent weekly. Further evalu-
ation is needed to determine whether this feedback time
might be reduced. Training apps may serve as a platform to
improve access to evidence-based intervention for the ris-
ing numbers of children with ASD and their families, espe-
cially in remote, rural areas or regions where services are
severely limited.
BMT enhances training effectiveness
The gradual increments in parents’ intervention skills sug-
gest that using BMT as an instructional design for the
Law et al. 13
mobile app is a potent pedagogical framework to help
users gain and apply intervention skills. The five mobile
app components (i.e. instructional video, quick check,
interactive game, practice video, and live feedback) were
designed to represent the sequential learning components
in BMT (i.e. modeling, retention, behavioral rehearsal,
feedback, and transfer of training). Collectively, these
components were designed to maintain high fidelity in
administration of a training program across parents as well
as to achieve high fidelity of parents’ intervention over
time. The innovative use of an adult training framework
enhanced training effectiveness for distance learning. The
use of BMT forges a new conceptual framework to be used
for training on a mobile device. BMT was supported to be
an effective training approach in a face-to-face (F2F) envi-
ronment (Taylor et al., 2005), yet this study advances the
use of BMT framework beyond a face-to-face environ-
ment to a mobile learning environment. Within the litera-
ture of parent-implemented NLI (Rakap and Rakap, 2014),
we found no studies which employ a systematic, theoreti-
cal framework for PT as in this study. Results suggest
important implications for the use of the BMT model. The
BMT model has significant implications for implementa-
tion practice in autism intervention, over and above inter-
vention practice. Implementation practice concerns the
methods and procedures used by implementation agents,
such as therapists and teachers to promote interventionists’
use of evidence-based intervention practices (Dunst and
Trivette, 2012). The substantial improvement in parents’
intervention skills in this study indicates an initial success
in the use of an innovative method for PT. The innovation
effectively trained parents to implement evidence-based
naturalistic developmental behavioral interventions
(NDBI). There is little discussion in the literature about
conceptual models to inform the design and development
of technologies that use evidence-based interventions to
change behavior or cognition (e.g. behavioral intervention
technologies (BITs); Mohr et al., 2014). The current pro-
gram demonstrates the potential of using the BMT frame-
work as a conceptual model in mobile technology for this
type of early childhood intervention implementation.
Limitations and recommendations for future
research
This study has several limitations and weaknesses. First,
the experiment was conducted with only three parent–
child dyads. Although three participants in a multiple-
baseline single-case experimental design is considered to
be acceptable (Kratochwill et al., 2013), larger samples
would better support the findings and their generalization,
especially across diverse populations and settings. Second,
there is clearly response bias among participants. Several
parents who initially expressed interest in the project
declined to participate when they heard what would be
required of them. This is also true of live, face-to-face PT
programs, however. Findings from this study suggest that
mobile apps may improve access to those who want to
develop their skills and desire more flexibility and con-
venience to access the training. Training apps may also be
a valuable adjunctive tool for face-to-face training pro-
grams. Research that helps to explore the reasons behind
parents’ choices to participate in training programs or not
would improve efforts to engage them as intervention
agents for their children. More studies are needed to evalu-
ate who can benefit the most and who can successfully
deliver such programs.
Furthermore, similar to many studies in the PT litera-
ture, all participating parents in this study were mothers
(Patterson et al., 2012). Thus, the results are more repre-
sentative of mothers than of fathers and may not be gener-
alizable to other caregivers. Replication studies with
fathers, grandparents, teachers, therapists, and so on, are
needed to strengthen the findings. Mothers in this study
were well-educated, and two did not work outside their
home. This program may not be as acceptable to parents
with less education or who must cope with greater work
demands. Studies are needed that compare the app’s
acceptability and effectiveness across samples of parents
who are more diverse in educational background and occu-
pational status.
Also, it was not possible in this study to evaluate the
relative contribution of each mobile app component to par-
ents’ intervention skills development. Qualitative feed-
back from participants in this study did not point to any
particular consensus regarding the perceived value of each
component of the app or of any particular set of skills.
Future research with comparison groups should make it
possible to determine what components and skills contrib-
ute most to the desired outcomes.
Finally, this study did not collect any data on parents’
stress or confidence levels before and after the mobile app
PT program. While there is good evidence that parents can
be effective agents of change in the communication and
behavior of their young children with autism, we also
know that parents of children with ASD are very much
stressed (Karst and Van Hecke, 2012; Koegel et al., 2002).
Participation in a training program may elevate parental
stress because of the demands it places on parent, but there
is also evidence that training reduces symptoms of depres-
sion and stress in parents (Vismara et al., 2009). Future
studies should measure parental stress pre- and post-inter-
vention to evaluate this potential impact.
Many more studies are needed to determine the suita-
bility of mobile platforms for intervention training. Studies
will be needed to compare outcomes and costs of programs
delivered in different formats and to replicate interven-
tions under different conditions so that long-term feasibil-
ity and sustainability can be better evaluated (Vismara
et al., 2013). Scaling the project to scores of parents will
14 Autism
first require developing a training protocol for the clini-
cians who interact with users via the app. The next phase
of this project will focus on developing the skills therapists
need to effectively use the app to achieve desired out-
comes. Map4speech demonstrates potential as a prototype
for additional apps that could be developed to address
other needs of young children with ASD, such as improv-
ing joint attention in children under 2 years of age.
Summary
Results of this experimental study suggest that mobile
technology may be a promising platform to deliver inter-
vention for children with ASD. Investigation of mobile
technologies to deliver evidence-based practices for chil-
dren with ASD and their families is still in its infancy
(Meadan and Daczewitz, 2015; Vismara et al., 2009), and
few studies provide robust empirical data on parent-led
intervention programs delivered via technologies. This
study filled an important gap by demonstrating the effec-
tiveness of a specially developed PT program that is deliv-
ered via a mobile app called Map4Speech. The intervention
program included components that are common in PT:
modeling or demonstrations, opportunities for rehearsal,
and personalized feedback. Parents learned 1–2 different
intervention skills in each of eight learning stages. In each
stage, they watched an instructional video of the skill, took
a short quiz about the skill, practiced the skill in a game
simulation, and then recorded themselves using the skill
with their child. Individualized feedback was provided by
the app for every quiz and game attempt. Live feedback
was provided via multiple brief coaching sessions con-
ducted with the app at six of the learning stages. Three
parents achieved and maintained high fidelity to the natu-
ralistic intervention they learned. In addition, two children
substantially improved in their functional communication.
It is conceivable that the design features of the app could
serve as a prototype for future applications to address
problems of concern other than speech, such as joint atten-
tion. Given the chronic shortage of resources available for
timely intervention, it is important to explore whether
mobile technology can increase the availability of evi-
dence-based intervention without compromising the qual-
ity of service (Stirman et al., 2004). Scalability, feasibility
in real-life contexts and sustainability over time are perti-
nent issues worth investigating when expansion is consid-
ered for the wider use with clinical populations.
Acknowledgements
The authors would like to thank the professional team at
Simulation Software and Technology, especially Mr Leon Liang
(mobile app programmer) and Mr Christopher Pau (graphic
designer), and the staff of the National Institute of Education
(NIE) Center for User Support for their patient assistance with
the development and maintenance of the mobile app. They also
thank Ms Marilyn Tan who assisted with the inter-observer reli-
ability for the experiment. The paper was based on Gloria
Law’s doctoral dissertation at National Institute of Education.
G.L. conceived of the study, participated in its mobile app and
single-case experimental design, conducted the experiment and
coordination of the study, performed the statistical analysis, and
drafted the manuscript. M.N. conceived of the study and par-
ticipated in its mobile app and single-case experimental design,
participated in revising the manuscript. A.D. participated in its
single-case experimental design, helped with analysis and inter-
pretation of data and participated in revising the manuscript.
All authors read and approved the final manuscript. The views
expressed in this paper are the authors’ and do not necessarily
represent the views of NIE.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of this
article.
Funding
The author(s) disclosed receipt of the following financial support
for the research, authorship, and/or publication of this article:
This research was supported by a grant (OER 63/12 MFN) from
the Office of Educational Research at the National Institute of
Education, Singapore. This paper refers to data from the research
project “The use of a mobile app training program to improve
functional communication in children with autism” (OER 63/12
MFN), funded by the Education Research Funding Programme,
National Institute of Education (NIE), Nanyang Technological
University, Singapore.
Informed consent
Informed consent was obtained from all individual participants
included in the study.
References
Aarons GA, Sommerfeld DH, Hecht DB, et al. (2009) The
impact of evidence-based practice implementation and
fidelity monitoring on staff turnover: evidence for a protec-
tive effect. Journal of Consulting and Clinical Psychology
77: 270–280.
American Psychiatric Association (APA) (2013) Diagnostic
and Statistical Manual of Mental Disorders: DSM-5.
Washington, DC: APA.
Bandura A (1976) Social Learning Theory. Englewood Cliffs,
NJ: Prentice-Hall.
Brookman-Frazee LI, Stahmer A, Baker-Ericzén M, et al. (2006)
Parenting interventions for children with autism spectrum
and disruptive behavior disorders: opportunities for cross-
fertilization. Clinical Child and Family Psychology Review
9: 181–200.
Carter SL (2007) Review of recent treatment acceptabil-
ity research. Education and Training in Developmental
Disabilities 42: 301–316.
Charlop-Christy MH and Carpenter MH (2000) Modified inci-
dental teaching sessions. Journal of Positive Behavior
Interventions 2: 98–112.
Law et al. 15
Decker PJ (1980) Effects of symbolic coding and rehearsal in
behavior-modeling training. Journal of Applied Psychology
65: 627–634.
Decker PJ and Nathan BR (1985) Behavior Modeling Training:
Principles and Applications. New York: Praeger.
Dunst CJ and Trivette CM (2012) Meta-analysis of implemen-
tation practice research. In: Kelly B and Perkins DF (eds)
Handbook of Implementation Science for Psychology
in Education. Cambridge: Cambridge University Press,
pp.68–91.
Elias T (2011) Universal instructional design principles for
mobile learning. The International Review of Research in
Open and Distributed Learning 12(2): 78–102.
Fenson L, Marchman VA, Thal DJ, et al. (2007) Macarthur-
Bates Communicative Development Inventories: User’s
Guide and Technical Manual. 2nd ed. Baltimore, MD: Paul
H. Brookes Publishing Company.
Fisher WW, Piazza CC, Bowman LG, et al. (1992) A compari-
son of two approaches for identifying reinforcers for per-
sons with severe to profound disabilities. Journal of Applied
Behavior Analysis 25: 491–498.
Ford JK, Kozlowski SWJ, Kraiger K, et al. (2014) Improving
Training Effectiveness in Work Organizations. New York:
Psychology Press.
Gedik N, Hanci-Karademirci A, Kurson E, et al. (2012) Key
instructional design issues in a cellular phone-based mobile
learning project. Computers & Education 58(4): 1149–1159.
Harris SL (1994) Treatment of family problems in autism. In:
Scholpler E and Mesibov G (eds) Behavioral Issues in
Autism. New York: Springer, pp.161–175.
Ingersoll B and Dvortcsak A (2006) Including parent training in
the early childhood special education curriculum for chil-
dren with autism spectrum disorders. Journal of Positive
Behavior Interventions 8: 79–87.
Ingersoll B and Dvortcsak A (2010) Teaching Social
Communication to Children with Autism (2 book set): A
Practitioner’s Guide to Parent Training and a Manual for
Parents. New York: Guilford Press.
Karst JS and Van Hecke AV (2012) Parent and family impact of
autism spectrum disorders: a review and proposed model
for intervention evaluation. Clinical Child and Family
Psychology Review 15: 247–277.
Kazdin AE (2011) Single-Case Research Designs: Methods for
Clinical and Applied Settings. New York: Oxford University
Press.
Koegel RL, O’Dell MC and Koegel LK (1987) A natural language
teaching paradigm for nonverbal autistic children. Journal of
Autism and Developmental Disorders 17: 187–200.
Koegel RL, Schreibman L, Britten KR, et al. (1982) A com-
parison of parent training to direct child treatment. In:
Koegel RL, Rincover A and Egel AL (eds) Educating and
Understanding Autistic Children. San Diego, CA: College
Hill Press, pp.260–279.
Koegel RL, Symon JB and Koegel L (2002) Parent education
for families of children with autism living in geographically
distant areas. Journal of Positive Behavior Interventions 4:
88–103.
Kratochwill TR, Hitchcock JH, Horner RH, et al. (2013) Single-
case intervention research design standards. Remedial and
Special Education 34: 26–38.
Law GC (2016) The use of a mobile app parent training pro-
gram to improve functional communication of children with
autism spectrum disorder. PhD Thesis, National Institute of
Education, Nanyang Technological University, Singapore.
Lord C, Rutter M, DiLavore PC, et al. (2012) Autism Diagnostic
Observation Schedule: ADOS-2. Torrance, CA: Western
Psychological Services.
Meadan H and Daczewitz ME (2015) Internet-based intervention
training for parents of young children with disabilities: a
promising service-delivery model. Early Child Development
and Care 185: 155–169.
Meadan H, Ostrosky MM, Zaghlawan HY, et al. (2009)
Promoting the social and communicative behavior of
young children with autism spectrum disorders: a review
of parent-implemented intervention studies. Topics in Early
Childhood Special Education 29: 90–104.
Moes DR and Frea WD (2002) Contextualized behavioral sup-
port in early intervention for children with autism and their
families. Journal of Autism and Developmental Disorders
32: 519–533.
Mohr DC, Schueller SM, Montague E, et al. (2014) The behavio-
ral intervention technology model: an integrated conceptual
and technological framework for eHealth and mHealth inter-
ventions. Journal of Medical Internet Research 16(6): e146.
National Research Council (2001) Committee on Educational
Interventions for Children with Autism: Educating Children
with Autism. Washington, DC: National Academy Press.
Neta G, Glasgow RE, Carpenter CR, et al. (2015) A framework
for enhancing the value of research for dissemination and
implementation. American Journal of Public Health 105:
49–57.
Nevill RE, Lecavalier L and Stratis EA (2016) Meta-analysis
of parent-mediated interventions for young children with
autism spectrum disorder. Autism. Epub ahead of print 14
November 2016. DOI: 10.1177/1362361316677838.
Park Y (2011) A pedagogical framework for mobile learning:
categorizing applications of mobile technologies into four
types. The International Review of Research in Open and
Distributed Learning 12(2): 78–102.
Parker RI, Vannest KJ and Davis JL (2011) Effect size in sin-
gle-case research: a review of nine nonoverlap techniques.
Behavior Modification 35: 303–322.
Patterson SY, Smith V and Mirenda P (2012) A systematic
review of training programs for parents of children with
autism spectrum disorders: single subject contributions.
Autism 16: 498–522.
Rakap S and Rakap S (2014) Parent-implemented naturalistic
language interventions for young children with disabilities:
a systematic review of single-subject experimental research
studies. Educational Research Review 13: 35–51.
Reimers TM, Wacker DP and Cooper LJ (1991) Evaluation of
the acceptability of treatments for children’s behavioral dif-
ficulties: ratings by parents receiving services in an outpa-
tient clinic. Child and Family Behavior Therapy 13: 53–71.
Roessger KM (2012) Toward an interdisciplinary perspective: a
review of adult learning frameworks and theoretical models
of motor learning. Adult Education Quarterly 62: 371–392.
Rogers SJ and Dawson G (2010) Early Start Denver Model
for Young Children with Autism: Promoting Language,
Learning, and Engagement. New York: Guilford Press.
16 Autism

Schreibman L (2000) Intensive behavioral/psychoeducational psychotherapies: a synthesis of dissemination theory. Clinical

treatments for autism: research needs and future directions.
Journal of Autism and Developmental Disorders 30: 373–
378.
Schreibman L and Anderson A (2001) Focus on integration:
the future of the behavioral treatment of autism. Behavior
Therapy 32: 619–632.
Schreibman L, Dawson G, Stahmer A, et al. (2015) Naturalistic
developmental behavioral interventions: empirically vali-
dated treatments for autism spectrum disorder. Journal of
Autism and Developmental Disorders 45: 2411–2428.
Schultz TR, Schmidt CT and Stichter JP (2011) A review of
parent education programs for parents of children with
autism spectrum disorders. Focus on Autism and Other
Developmental Disabilities 26: 96–104.
Scruggs TE and Mastropieri MA (1998) Summarizing sin-
gle-subject research issues and applications. Behavior
Modification 22: 221–242.
Sparrow S, Cicchetti D and Balla D (2005) Vineland Adaptive
Behavior Scales: Survey Interview Form/Caregiver Rating
Form. 2nd ed. Livonia, MN: Pearson Assessments.
Stadnick NA, Stahmer A and Brookman-Frazee L (2015)
Preliminary effectiveness of project imPACT: a parent-
mediated intervention for children with autism spectrum
disorder delivered in a community program. Journal of
Autism and Developmental Disorders 45: 2092–2104.
Stirman SW, Crits-Christoph P and DeRubeis RJ (2004)
Achieving successful dissemination of empirically supported
Psychology: Science and Practice 11: 343–359.
Taylor PJ, Russ-Eft DF and Chan DWL (2005) A meta-analytic
review of behavior modeling training. Journal of Applied
Psychology 90: 692–709.
Tonge B, Brereton A, Kiomall M, et al. (2006) Effects on paren-
tal mental health of an education and skills training program
for parents of young children with autism: a randomized
controlled trial. Journal of the American Academy of Child
and Adolescent Psychiatry 45: 561–569.
Vismara LA, Colombi C and Rogers SJ (2009) Can one hour per
week of therapy lead to lasting changes in young children
with autism? Autism 13: 93–115.
Vismara LA, McCormick C, Young GS, et al. (2013) Preliminary
findings of a telehealth approach to parent training in
autism. Journal of Autism and Developmental Disorders
43: 2953–2969.
Wainer A and Ingersoll B (2013) Intervention fidelity: an essen-
tial component for understanding ASD parent training
research and practice. Clinical Psychology: Science and
Practice 20: 335–357.
Wainer A and Ingersoll B (2014) Increasing access to an ASD
imitation intervention via a telehealth parent training pro-
gram. Journal of Autism and Developmental Disorders 45:
3877–3890.
Warren Z, McPheeters ML, Sathe N, et al. (2011) A systematic
review of early intensive intervention for autism spectrum
disorders. Pediatrics 127(5): e1303–e1311.

Appendix 1.  Procedural integrity for psychologist’s feedback.

Criteria Observed/not observed

1. Score the two videos and locate the skills that are fulfilled and those that are not fulfilled.  
2. Use share screen function to show parent’s video and comment on their skills.  
3. A
 ffirm and praise parent for specifically what they have done right in the fulfilled criteria.  
Encourage them to keep practicing the skills.
4. H
 ighlight and tell parent specifically what he can do (describe in words/demonstrate over Skype  
what they need to do).
5. Give the rationale why doing a specific skill is important when necessary.  
6. P
 rioritize and focus on giving feedback on most important skills (1–2) if parent misses three or  
more skills.
7. Help parent resolve any technology issues relating to the mobile app as they arise.  
8. U
 se a responsive interaction style with parent throughout the session.  
Total fidelity = Observed/(Observed + Not Observed) × 100