2017 IEEE International Conference on Big Data (BIGDATA)
Machine learning for early detection of autism (and other conditions) using a
parental questionnaire and home video screening
Halim Abbas Eric Glover
and Ford Garberson Email: eric.g@ericglover.com
Cognoa Inc
Palo Alto, CA, USA
Email: halim@cognoa.com
Abstract—Existing screening tools for early detection of autism
are expensive, cumbersome, time-intensive, and sometimes fall
short in predictive value. In this work, we apply Machine
Learning to gold standard clinical data obtained across thou-
sands of children at risk for autism spectrum disorders to
create a low-cost, quick, and easy to apply autism screening
tool that performs better than most widely used standardized
instruments. This new tool combines two screening methods
into a single assessment, one based on short, structured parent-
reported questionnaires and the other based on tagging key
behaviors from short, semi-structured home videos of children.
We demonstrate a significant accuracy improvement over stan-
dard screening tools in a clinical study sample of 162 children.
We further discuss the challenge of extending machine learning
algorithms to conditions beyond autism, and we propose a
generalized framework for using machine learning algorithms
to simultaneously search for the presence of many different
conditions.
1. Introduction
Diagnosis within the first few years of life dramati-
cally improves the outlook of children with autism, as it
allows for treatment while the child’s brain is still rapidly
developing [1], [2]. Unfortunately, in the United States
autism is typically not diagnosed earlier than age 4, with
approximately 27% of cases remaining undiagnosed at age
8 [3]. This delay in diagnosis is driven primarily by a lack
of effective screening tools and a shortage of specialists to
evaluate at risk children.
Most autism screeners in use today are based on score-
sheets with questions for the parent or the medical practi-
tioner, that produce results by comparing summation scores
to predetermined thresholds. Notable examples are the
Modified Checklist for Autism in Toddlers, Revised (M-
CHAT) [4], a checklist-based screening tool for autism
that is intended to be administered during developmental
screenings with children between the ages of 16 and 30
months, and the Child Behavior Checklist (CBCL) [5] which
is a parent-completed screening tool.
978-1-5386-2715-0/17/$31.00 ©2017 IEEE 3558
Dennis P Wall
Stanford University
Stanford, CA, USA
We have developed two new machine learning autism
screeners that are reliable, cost-effective, easy enough to
complete in minutes, and achieve higher accuracy than
existing screeners. We have deployed these screeners using
the Cognoa [6] App. To date, Cognoa has been used by over
250,000 parents in the US and internationally using web and
native smartphone mobile App tools. One screener is based
on a short questionnaire about the child which is answered
by the parent. The other screener is based on identification
of specific behaviors by trained analysts after watching two
or three short videos of the child at their home environment
captured by parents using a mobile device.
The screening algorithms were trained using behavioral
patterns keyed off of features that are probed by established
clinical instruments. These instruments, the ADI-R [7] and
ADOS [8], are considered gold standard instruments that
have high accuracy and high consistency across practitioners
and are frequently used for at risk children. They both,
however, are costly and can require hours to administer.
We will briefly discuss our screening algorithms and
show a snapshot of their performance in a clinical trial that
was conducted last year (more details are available in a draft
that is currently under review [9]).
We further investigate the problem of applying a clas-
sifier that can simultaneously classify multiple conditions
using chained machine learning algorithms. We discuss the
major limitations inherent in training individual algorithms
that can be trained to classify the presence or absence of all
conditions in all children, and we propose a more practical
approach.
2. Autism screener data and methodology
The autism machine learning based screeners were
trained using data compiled from multiple clinical repos-
itories of ADOS and ADI-R score-sheets of nearly seven
thousand children between 18 to 84 months of age, sup-
plemented by data collected from the parents of children
answering screening questions on Cognoa’s website. Both
screeners were applied using Cognoa’s App in a clinical trial
to a sample of 162 at-risk children between the ages of 18
and 84 months who have undergone full clinical examination
and received a clinical diagnosis.
2.1. Parental questionnaire screener methodology
The parental questionnaire screener keys on behavioral
patterns typically probed in a standard autism diagnostic
instrument, the Autism Diagnostic Interview Revised (ADI-
R) [7]. This clinical tool consists of a parent interview of
93 multi-part questions with multiple choice and numeric
responses which are delivered by a trained professional in a
clinical setting. While this instrument is considered a gold-
standard and gives consistent results across examiners, the
cost and time to administer it can be prohibitive. These ques-
tions are therefore keyed on by machine learning algorithms
when building the parental questionnaire screener for the
Cognoa application.
A variety of machine learning approaches were studied
before settling on an optimal approach for the parental
questionnaire screener. Each of the instrument’s 155 data
columns was encoded using in a manner similar to a one hot
encoding approach but which preserves information about
the severity of the response. Two machine learning algo-
rithms using random forests were designed, one to screen
children between 18 months and three years of age, and
the other to screen children between four and six years of
age. To make the screening questionnaire easier for parents,
the number of questions asked was reduced. From each
age group, detailed studies were performed to identify an
optimal subset of questions to include for the final machine
learning algorithm (17 were chosen for the model applied
to younger children, and 21 for the model applied to older
children). Questions similar to these, but simplified and
rephrased in order to be easily understood by parents were
chosen to be presented in the Cognoa application. Aggrega-
tions of the average, extremes, and most common responses
were found to significantly increase accuracy. Further details
are discussed in [9].
2.2. Video screener methodology
The video screener keys on behavioral patterns typically
probed in another diagnostic tool, the Autism Diagnostic
Observation Schedule (ADOS) [8]. ADOS consists of an
interactive, highly standardized examination of the child by
trained clinicians in a tightly controlled setting. ADOS is
widely considered a gold standard and is one of the most
common behavioral instruments used to aid in diagnosis of
autism [10], however the cost and time to administer it can
be prohibitive. For the Cognoa video screening algorithm,
a subset of ADOS questions were identified as probing
features that can realistically be observed in home videos.
Separate algorithms were trained for use with pre-verbal
children or verbal children, and for each an optimal subset
of ten questions were identified as the most effective for
identification of autism in that age group. Questions based
3559
on these features were answered by a minimally trained
analyst after watching two or three one minute home videos
of the child’s behavior taken by their parent.
A variety of machine learning approaches were stud-
ied before settling on an optimal approach for the video
screener. Random forests were then trained to determine
the screener output. In order to reduce the impact of bias
from non-observable features, missing observations were
randomly injected into the training data at a rate that was
calculated to minimize the decision making impact of a
missing feature in the trees of the forests. Further details
are discussed in [9].
3. Autism screening inconclusive results
Patients with more complex symptom presentation are
known to pose challenges to developmental screening. These
children often screen as false positives or false negatives,
resulting in an overall degradation of screening accuracy that
is observed by all standard methods and has become accept-
able in the industry. Given that our low-cost instruments do
not rely on sophisticated observations to differentiate com-
plex symptom cases, our approach was to avoid assessing
them altogether, and to try instead to spot and label them
as “inconclusive”.
For both the video and the parental questionnaire based
algorithms, multiple methods to implement this were de-
vised of varying sophistication. In the end the simplest
method performed as well as the others and was chosen
to report inconclusive results for the clinical study. Instead
of using the output score of the random forest to choose
an optimal threshold to separate autism from non-autism
cases, the score was passed to a grid search to determine an
optimal cutoff range, subject to a constraint on the maximum
number of subjects which could be determined inconclusive.
Subjects with a score above the lowest threshold but below
the highest were considered inconclusive.
4. Autism screening combination
The combination of the questionnaire and video screen-
ers is made more challenging by the fact that no training
samples are available for children that are known to have
both ADI-R and ADOS results. As a consequence, the
clinical data itself must be used to build the model to
perform the combination.
The numerical response of each of the parent ques-
tionnaire and video classifiers were combined using l2-
regularized logistic regression. While some overfitting is
expected, this is minimal due to the fact that the logistic
regression is highly constrained, with only three free pa-
rameters. Since screening models were trained for young
children and old children, separate combination algorithms
were trained per age group. For each combination algorithm,
optimal inconclusive output criteria were chosen using the
logistic regression response.
5. Autism screening results in the clinical trial
ROC curves in Figure 1 show the performances of
the parental questionnaire-based screener, the video-based
screener and the combined screener. The performances of
the industry-standard M-CHAT and CBCL autism screen-
ers are also compared. Operating near the commonly used
threshold of 80% sensitivity, the combined and video-based
screeners presented here have a better specificity than both
the M-CHAT or the CBCL at a 95% confidence level, while
the questionnaire alone has a better specificity than the
CBCL screener at a 95% confidence level but is not quite
better in specificity than the M-CHAT screener at a 95%
confidence level.
6. Simultaneous screening for multiple condi-
tions
In addition to autism there are many other important
childhood conditions that could potentially be screened for
using machine learning algorithms, including ADHD, lan-
guage disorders, and intellectual disabilities, among others.
Building one or more high performing machine learning
algorithms to screen multiple conditions for all children is a
daunting task for several reasons. First, each of these condi-
tions has unique signatures that depend upon very different
types of features from one another. Asking a sufficiently
thorough set of questions to be able to effectively classify
all of these conditions would be prohibitive on any parent’s
time. Secondly, a large sample of training data would be
needed of children with every one of these conditions which
contain features that are relevant not just to their particular
condition, but also to all other conditions that the classifier
will attempt to identify. Such data is typically not available.
For cost considerations, children in clinical settings are
typically not evaluated using instruments unless they are
deemed to be at risk of a particular condition that is probed
by the instrument.
Therefore, instead of building a single master machine
learning classifier that classifies all conditions simultane-
ously, we propose a tree of condition classification algo-
rithms. Figure 2 shows what such a screening tree might
look like. This tree would start with an algorithm to de-
termine whether the child is at risk of any of a number of
broad categories of conditions, and if so, presents them with
a new particular machine learning algorithm that is targeted
to determine more precisely which kind of condition they
have. Further classifiers can then be trained to identify a
more specific condition if possible. The machine learning
algorithms that are run at each node can be individual
algorithms, or composite algorithms such as discussed in
Section 4 of this paper.
The algorithms at each node of the tree can further come
to an inconclusive determination at each node in the manner
discussed in Section 3 of this paper. The functionality to
return inconclusive results is especially useful in a tree of
3560
algorithms due to the fact that it allows for much more
flexible tuning of the algorithms. For example, if there is
an algorithm that is unable to accurately classify 80% of
children but which can determine a very accurate classi-
fication on the remaining 20%, such an algorithm would
be infeasible to use in a standalone screening application.
However, in a tree of algorithms the inconclusive parameter
of the algorithm at that node can be tuned to reach an
inconclusive determination in the vast majority of subjects
without losing relevance (such subjects can still be informed
of the results of the upstream classifier determinations) while
giving a more precise determination for the minority of
subjects for which an accurate conclusive determination is
possible.
The chained tree of machine learning algorithms has
further advantages. Parents would only have to spend time
answering questions that are relevant to their particular
child. Further, this approach more closely mirrors what
happens in an actual clinical setting. Thus the data that is
available in real clinical settings for at-risk children would
become all that is needed to train a classifier to discriminate
between the relevant conditions at a particular node of the
algorithm tree.
7. Conclusion
We have briefly presented two new machine learning
algorithms, one of which operates based on a parental ques-
tionnaire and the other of which operates on home cell phone
videos, as well as their combination. We have also shown
that their performance exceeds that of the industry standard
screening tools M-CHAT and CBCL in a clinical trial of
162 at risk children. Further details on these algorithms and
this clinical trial are available elsewhere [9]. We have also
proposed an engine to handle machine learning classification
of many conditions simultaneously with clear advantages
over individual algorithms that are designed to classify all
conditions for all children. Work is in progress to apply this
tool to conditions beyond autism.
References
[1] MS Durkin, MJ Maenner, FJ Meaney, SE Levy, C DiGuiseppi,
JS Nicholas, et al. Socioeconomic inequality in the prevalence of
autism spectrum disorder: evidence from a U.S. cross-sectional study.
PloS One, 5, 2010.
[2] DL Christensen, J Baio, KV Braun, et al. Prevalence and characteris-
tics of autism spectrum disorder among children aged 8 years autism
and developmental disabilities monitoring network, 11 sites, United
States, 2012. MMWR Surveill Summ, 65 (No. SS-3)(No. SS-3):1–23,
2016.
[3] L Zwaigenbaum, S Bryson, C Lord, S Rogers, A Carter, et al. Clinical
assessment and management of toddlers with suspected autism spec-
trum disorder: insights from studies of high-risk infants. Pediatrics,
123:1383–1391, 2009.
[4] R Bernier, A Mao, and J Yen. Diagnosing autism spectrum disorders
in primary care. Practitioner, 255(1745):27–30, 2011.
[5] TM Achenbach and LA Rescorla. Manual for the ASEBA preschool
forms & profiles. 2000.
Figure 1. ROC curves on the clinical sample for the questionnaire and the video based algorithms, separately and in combination. Inconclusive determination
is allowed for up to 25% of the cases. The established screening tools MCHAT and CBCL are included as baselines.

Figure 2. A hypothetical illustration of a chained tree of machine learning algorithms. At each node in the tree a three way classifier (such as the one
discussed in this paper) is run to determine whether a condition is present, or whether the determination is inconclusive given the available data.

[6] Cognoa, Inc. 2390 El Camino Real St 220, Palo Alto, CA 94306, [10] C Lord, E Petkova, V Hus, W Gan, F Lu, et al. A multisite study of
https://www.cognoa.com/. the clinical diagnosis of different autism spectrum disorders. Archives
of General Psychiatry, 69:306–313, 2012.
[7] C Lord, M Rutter, and A Le Couteur. Autism diagnostic interview-
revised: a revised version of a diagnostic interview for caregivers of
individuals with possible pervasive developmental disorders. Journal
of Autism and Developmental Disorders, 24:659–685, 1994.
[8] C Lord, M Rutter, S Goode, J Heemsbergen, H Jordan, et al. Autism
diagnostic observation schedule: a standardized observation of com-
municative and social behavior. J Autism Dev Disord, 19:185–212,
1989.
[9] Halim Abbas, Ford Garberson, Eric Glover, and Dennis Wall. Ma-
chine learning approach for early detection of autism by combining
questionnaire and home video screening. Journal of the American
Medical Informatics Association (submitted, under review), 2017.

3561