2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI)

April 3-7, 2020, Iowa City, Iowa, USA

A NOVEL FRAMEWORK FOR GRADING AUTISM SEVERITY USING TASK-BASED FMRI

Reem Haweel1,2 , Omar Dekhil 1 , Ahmed Shalaby1 , Ali Mahmoud1

Mohammed Ghazal1,3 , Ashraf Khalil3 , Robert Keynton1 , Gregory Barnes4 , and Ayman El-Baz1
1
Bioengineering Department, University of Louisville, Louisville, KY, USA.
2
Faculty of Computer and Information Sciences, University of Ain Shams, Cairo, Egypt.
3
Department of Electrical and Computer Engineering, Abu Dhabi University, Abu Dhabi.
4
Department of Neurology, University of Louisville, Louisville, KY, USA.

ABSTRACT (GLM) [11]. In recent literature, group analysis is applied to address

Autism is a developmental disorder associated with difficulties in
communication and social interaction. Currently, the gold standard
in autism diagnosis is the autism diagnostic observation schedule
(ADOS) interviews that assign a score indicating the level of sever-
ity for each individual. However, current researchers investigate de-
veloping objective technologies to diagnose autism employing brain
image modalities. One of such image modalities is task-based func-
tional MRI which exhibits alterations in functional activity that is
believed to be important in explaining autism causative factors. Al-
though autism is defined over a wide spectrum, previous diagnosis
approaches only divide subjects into normal or autistic. In this paper,
a novel framework for grading the severity level of autistic subjects
using task-based fMRI data is presented. A speech experiment is
used to obtain local features related to the functional activity of the
brain. According to ADOS reports, the adopted dataset of 39 sub-
jects is classified to three groups (13 subjects per group): mild, mod-
erate and severe. Individual analysis with the general linear model
(GLM) is used for feature extraction for each 246 brain areas ac-
cording to the Brainnetome atlas (BNT). Our classification results
are obtained by random forest classifier after recursive feature elim-
ination (RFE) with 72% accuracy. Finally, we validate our selected
features by applying higher level group analysis to prove how in-
formative they are and to infer the significant statistical differences
between groups
Index Terms— Task-based fMRI; ASD; FSL; Machine learning
1. INTRODUCTION
Autism spectrum disorder (ASD) is a neuronal developmental dis-
order associated with a range of symptoms as social, sensorimotor,
communicative deficits that differ in severity [1]. Autism is typi-
cally diagnosed at the age of three, however, some characteristics
can be observed as early as 12 months old, especially with the ad-
vent of medical imaging modalities and the recent state of the art ma-
chine learning algorithms [2]. Structural magnetic resonance imag-
ing (sMRI), functional magnetic resonance imaging (fMRI) and dif-
fusion tensor imaging (DTI) are widely adopted for analysing brain’s
structural and functional characteristics [3–8].
In general, task-based fMRI is used for addressing evoked blood
oxygen level-dependent (BOLD) signals in all brain regions with
response to certain tasks belonging to a range of different task do-
mains [9]. Basic fMRI tasks include, motor tasks, visual process-
ing tasks, auditory and language tasks, and basic social processing
tasks [10].
Studies in literature statistically examined the hemodynamic
effects associated with task-control using the general linear model
commonly activated brain areas of both groups of ASD and nor-
mal controls in response to auditory and language tasks. Gomot et
al. [12] used a conical HRF and SPM2 package to examine extreme
repetitive behavior. They found that the left inferior parietal regions
and the right prefrontal-premotor of autistic subjects were more
activated than neurotypicals. Lombardo et al. [13] made a study on
three groups of typically developed toddlers, ASD toddlers of good
outcome and those of relatively lower language outcome. Results
showed same activation in the first two groups in the language sensi-
tive superior temporal cortices, whereas less activation was detected
for those of lower language outcome.
Brain image modalities have been a powerful tool for autism di-
agnosis. Structural MRI and resting state fMRI are the commonly
incorporated image modalities used for classification. Although task
fMRI provides indicative information about brain functional impair-
ment and biomarkers, few attempts for classification using task fMRI
were recorded in literature. Guillaume Chanel in [14] applied multi-
variate pattern analysis (MVPA) to two different fMRI experiments
with social stimuli (faces and bodies). They used support vector
machines (SVMs) and recursive feature elimination (RFE) for clas-
sification between 15 ASD and 14 control subjects and gained ac-
curacy range between 69% and 92.3%. Another attempt in [15]
demonstrated the effectiveness of different methods to train general-
izable recurrent neural networks from small datasets to classify chil-
dren with ASD (N = 21) vs. typical control subjects (N = 19) from
task-fMRI scans. The classification accuracy ranged from 51.8% to
69.8%.
The focus of previous studies was to analyze or diagnose autism
disorder in two groups of ASD or normal, which is not sufficient for
addressing differences among subjects across the wide autism spec-
trum. To overcome such limited diagnosis, we have developed an
autism severity grading approach to diagnose each autistic subject
as mild, moderate or severe autistic. Moreover, while [14] applied
experiments on adults between 19 and 53 years and the age for sub-
jects in [15] is (6.05 ± 1.24 years), we include toddlers from 12 to
27 months old in our study aiming for early detection of autism.
The goal of this paper is to utilize brain processing and analysis
tools as well as machine learning algorithms for developing more
specialized objective computer aided diagnosis (CAD). Hence, pro-
viding a better early diagnosis and treatment plan for each autistic
subject individually having different autism severity.
2. MATERIALS
The task-based fMRI images were recorded in a speech experiment
that included four audio stimuli: complex forward speech, simple

978-1-5386-9330-8/20/$31.00 ©2020 IEEE 1404

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on May 27,2020 at 11:21:43 UTC from IEEE Xplore. Restrictions apply.
 that n has zero mean (E(n ) = 0). By adding the second level of
analysis on the group level we have:

β = XG βG + η (2)
Where XG is the group-level design matrix, and it separates the
two groups (controls and patients), βG is the vector of the group level
parameters and η is the group level residuals. Also η is considered
to have zero mean, E(η) = 0. Substituting with (2) in (1) we get:

Fig. 1. Illustration of the speech experiment that includes four audio Y = XXG βG + Xη +  (3)
stimuli: Complex forward speech, Simple forward speech, backward Let
speech and silence which are alternating repeatedly along 6 min 20 γ = Xη +  (4)
sec. then,
forward speech, backward speech and silence which were alternating Y = XXG βG + γ (5)
repeatedly along 6 min 20 seconds. Fig 1 illustrates the fMRI exper-
iment design. In order to divide subjects in our study, we have used Let V is the covariance of  and VG is the covariance of η, then
the calibrated severity score (CSS) for the toddler module which in- by using the general least squares approach [21], the first level pa-
dicates the level of autism severity. The CSS is obtained from raw rameters could be estimated as:
total domain scores of the autism diagnostic observation schedule β̂ = (X T V −1 X)X T V −1 Y (6)
(ADOS) [16] and varies from 0 to 10. It is divided into 3 classes: (i)
mild (CSS 1-4), (ii) moderate (CSS 5-7), and (iii) severe (CSS 8-10). and
Three matched sets, one for each autistic group are included in this ˆ = (X T V −1 X)−1
cov(β) (7)
study, each with the size of 13 subjects. In the similar way with the group level, the group parameters are
given by:
T −1
3. METHODS βˆG = (XG VG XG )−1 X T V −1 Y (8)
and
Fig 2 shows the general three-stage framework of our proposed sys- T −1
cov(βˆG ) = (XG VG XG )−1 . (9)
tem for grading ASD subject’s severity level using task-based fMRI
images. The following sections explain each step in detail: After estimating the parameters for each voxel, it is required to
define some contrasts and check which voxels are significant with
respect to these conditions. The most common technique for testing
3.1. Data preprocessing
the voxel significance is the paired z-test [22, 23]. The statistical
The preprocessing pipeline is performed using fMRI expert analysis analysis result could be expressed as a map of corrected P-values at
tool (FEAT) [17] included in fMRIB’s software library (FSL) [18] each voxel. For more details about GLM parameter estimates, the
accordingly applying the following steps:(i) interleaved slice timing reader is referred to [22].
correction to correct the effect of recording slices at different points In this study, we utilize the output z-stats of each subject to ex-
in time within the volume, (ii) motion correction using MCFLIRT tract features. The Brainnetome atlas (BNT) [24] is applied to map
to correct the effect of subjects motion in the scanner by applying the brain to 246 areas. A histogram with 6 bins is constructed to
rigid-body transformations with 6 degrees of freedom (DOF) [19], define the percent of z-stat intensity value within each interval, for
(iii) spatial smoothing using Gaussian window with full width at each brain area. Since the z-stat map varied between 0 and 6, the
half maximum (FWHM) of 5mm, to improve the signal to noise ra- used intervals are (0 <= |z| < 1, 1 <= |z| < 2, 2 <= |z| <
tio. (iv) high pass temporal filtering (100s) to remove low frequency 3, 3 <= |z| < 4, 4 <= |z| < 5, |z| >= 5). This creates a feature
artefacts and scanner drifts, (v) brain extraction using BET, to re- vector of 246 areas by 6 features per subject. We apply a feature
move the scalp and all non-brain data in sMRI images, and (vi) Reg- selection algorithm prior to classification to reduce the feature space
istration to standardize the fMRI brain images by applying 2 steps, dimensionality and to detect the significant features.
the first step is to register the functional volume to its high resolution
anatomical scan, the second step is to register the anatomical scan to 3.3. Classification and severity grading
MNI-152 space with 12 DOF [20].
As mentioned above, the speech task has 4 conditions, complex for-
ward speech, simple forward speech, backward speech, and silence.
3.2. Feature extraction with multi level General Linear Model
In this study, we apply both the first level analysis and the higher
(GLM)
level analysis to model these four regressors in GLM. The first level
Consider an experiment with N subjects, and for each subject Yn analysis is used to quantify the activation differences between the
there is a vector of T time points, where n = 1,...,N. The first level subjects for the classification purpose, while the higher level analy-
GLM is defined as: sis is applied to have more insightful analysis about the overall group

Y1
 
X1 0 ... 0 0
 
β1
  
1
differences. Hence, we study the relation between the selected brain
 Y2   0 X2 ... 0   β2    2  areas with discriminating features for classification and the com-
 . = . . . .  ∗  .  +  .  = Xβ + (1) monly activated significant areas in each group. As shown in Fig
       
..
 ..   .. .
. . .
.
.   .   . 
. . . 3, the algorithm used for feature selection is RFE [25]. This algo-
YN 0 0 ... XN βN N
rithm uses random forest classifier to fit a model for the data and sort
Where Xn is the design matrix,  is the residual error, and βn the feature importance, then start removing the least features recur-
are the parameters to be estimated. To estimate βn , we will assume sively.

1405

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on May 27,2020 at 11:21:43 UTC from IEEE Xplore. Restrictions apply.
 Fig. 2. Block diagram of the proposed framework for grading autistic subject’s severity level using task-based fMRI images
Table 1. The classification confusion matrix of the 3 groups
Predicted label
True label* Mild Moderate Severe
9 2 2
1 10 2
2 2 9
4. EXPERIMENTAL RESULTS AND DISCUSSION
This study is conducted on ”Biomarkers of Autism at 12 Months:
From Brain Overgrowth to Genes” dataset obtained by the National
Database for Autism Research (NDAR: http://ndar.nih.gov). The
data selection criteria is designed to include the subjects that have
ADOS toddler module available. We have included 39 (13 mild, 13
moderate and 13 severe) with ages ranging from 12 to 27 months
(mean : 20.13 months and std: 5.07 months). Each subject has a
(T1) structural MRI and speech fMRI scans that were acquired us-
ing a Signa HDxt, 1.5T GE Healthcare scanner. Each fMRI scan
consists of 154 volumes with TR=2.5 seconds and TE=30 milli sec-
onds. Each volume is constructed by 31 slices with alternating in the
plus direction slice acquisition pattern. Results are reported for both
the statistical inference for each group and between all groups (mild,
moderate and severe) as well as the classification accuracies.
4.1. Classification results
In this experiment, we use random forest classifier in both RFE and
classification. We used a 10-folds cross validation technique and
calculated the accuracy of classification. The achieved accuracy is
71.79%. In addition to calculating the accuracy, the confusion matrix
between the 3 classes is also calculated. Table 1 shows the confu-
sion matrix. The model hyperparameters for both RFE and random
forest classifier are both selected using a grid search. The optimal
performance is obtained when using 11 trees with maximum depth
of 9 for RFE and 238 with maximum depth of 28 for random for-
est. Table 2 shows the areas with the most selected features by RFE.
Different classifiers other than random forest were tested including:
(i) linear SVM (accuracy = 54%), (ii) SVM with RBF kernel (accu-
racy 59%) and neural network (accuracy =64%). The random forest
outperformed the other tested classifiers.
4.2. Validation with higher level analysis
We have validated our selected features in our classification by com-
paring them with the most activated areas in the group analysis. Two
1406
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on May 27,2020 at 11:21:43 UTC from IEEE Xplore. Restrictions apply.
Fig. 3. The pipeline of the proposed classification and severity grad-
ing approach
types of group analysis, voxel and area wise are applied. The higher
level voxel-wise modeling is done using fMRIB’s local analysis of
mixed effects (FLAME). Group analysis reveals activation patterns
between the 3 groups but with different significance levels.
Table 2. Brain areas having maximum selected features by RFE
1 2 3 4 5 6
7 8 9 10 11 12
13 14 15 16 17 19
20 21 22 23 24 32
To show how the histogram features per area are matching the
group activation, we extracted the mean percent of activated voxels
out of the most significant histogram bins (|z| > 3) per group for
each area. Table 3 shows the top 5 areas per group. The intersections
between the top activated areas and the FLAME group analysis is
shown in fig 4. It is obvious that our top informative areas match
the group analysis results. Moreover, such analysis gives insightful
information about significant activation over brain areas rather than
independent voxels as in FLAME group analysis.
It is important to check the quality of feature representation and
selection. Consequently, comparing table 2 and table 3 reveals the
high intersection between the selected areas by RFE, in the feature
selection step, with the top activated areas in each group.
5. CONCLUSION AND FUTURE WORK
In this study, we introduce a machine learning based approach for
autism severity grading on the autism spectrum. To the best of our
knowledge, this is the first effort to utilize task-based fMRI for this
goal. With a limited number of subjects (n = 39), our algorithm
achieved accuracy of 72% using random forest classifier following
 [8] Omar Dekhil et al., “Using resting state functional mri to build
a personalized autism diagnosis system,” PLOS ONE, vol. 13,
no. 10, pp. 1–22, 10 2018.
[9] John D Van Horn et al., “The functional magnetic resonance
imaging data center (fmridc): the challenges and rewards of
large–scale databasing of neuroimaging studies,” Philosophi-
cal Transactions of the Royal Society of London B: Biological
Sciences, vol. 356, no. 1412, pp. 1323–1339, 2001.
[10] Ayman El-Baz et al., Imaging the brain in autism, Springer,
2013.
[11] Karl J Friston et al., “Statistical parametric maps in functional
imaging: a general linear approach,” Human brain mapping,
vol. 2, no. 4, pp. 189–210, 1994.
Fig. 4. A cross section showing top 5 informative areas, for each [12] Marie Gomot et al., “Brain hyper-reactivity to auditory novel
group, in terms of percent of significant voxels in each area, inter- targets in children with high-functioning autism,” Brain, vol.
secting with the FSL group level analysis results (orange). 131, no. 9, pp. 2479–2488, 2008.
[13] Michael V Lombardo et al., “Different functional neural sub-
Table 3. The top 5 areas having percent of significant voxels for
strates for good and poor language outcome in autism,” Neu-
each group
Mild Moderate Severe percent of significant voxels ron, vol. 86, no. 2, pp. 567–577, 2015.
Mild Moderate Severe [14] Guillaume Chanel et al., “Classification of autistic individuals
First area 14 13 14 25.5 24.8 32.1
Second area 13 5 19 20.7 23.5 31.5 and controls using cross-task characterization of fmri activity,”
Third area 19 11 20 19.9 23.3 30.9 NeuroImage: Clinical, vol. 10, pp. 78–88, 2016.
Forth area 20 19 13 18.5 23.1 28.6
Fifth area 32 20 24 14.5 22.1 24.7
[15] Nicha C Dvornek et al., “Learning generalizable recurrent neu-
ral networks from small task-fmri datasets,” in International
recursive feature elimination. We also applied group analysis to vali- Conference on Medical Image Computing and Computer-
date selected features and study common group brain activation. Our Assisted Intervention. Springer, 2018, pp. 329–337.
future work will focus mainly on integrating more data from differ- [16] Katherine Gotham et al., “Standardizing ADOS scores for a
ent experiments and modalities to get more comprehensive under- measure of severity in autism spectrum disorders,” Journal of
standing of how brain activation abnormalities in response to dif- autism and developmental disorders, vol. 39, no. 5, pp. 693–
ferent tasks explains autism. Such significant differences will be 705, 2009.
investigated to correlate with ADOS reports to develop personalized
diagnosis and treatment. [17] Mark W aoolrich Woolrich, “Temporal autocorrelation in uni-
variate linear modeling of fmri data,” Neuroimage, vol. 14, no.
6, 2001.
6. REFERENCES [18] Mark Jenkinson, Christian F Beckmann, Timothy EJ Behrens,
Mark W Woolrich, and Stephen M Smith, “Fsl,” Neuroimage,
[1] David G. Amaral aand others, “Neuroanatomy of autism,”
vol. 62, no. 2, pp. 782–790, 2012.
Trends in Neurosciences, vol. 31, no. 3, pp. 137 – 145, 2008.
[19] Mark Jenkinson et al., “Improved optimization for the robust
[2] Manuel F Casanova et al., Autism Imaging and Devices, CRC and accurate linear registration and motion correction of brain
Press, 2017. images,” Neuroimage, vol. 17, no. 2, pp. 825–841, 2002.
[3] Marwa Ismail et al., “Studying autism spectrum disorder with [20] Jack L. Lancaster, Diana Tordesillas-Gutiérrez, Michael Mar-
structural and diffusion magnetic resonance imaging: a sur- tinez, Felipe Salinas, Alan Evans, Karl Zilles, John C. Mazz-
vey,” Frontiers in human neuroscience, vol. 10, 2016. iotta, and Peter T. Fox, “Bias between mni and talairach coor-
[4] R. Haweel et al., “Functional magnetic resonance imaging dinates analyzed using the icbm-152 brain template,” Human
based framework for autism diagnosis,” in 2019 Fifth Inter- Brain Mapping, vol. 28, no. 11, pp. 1194–1205, 2007.
national Conference on Advances in Biomedical Engineering [21] Shayle Searle et al., Variance components, vol. 391, John
(ICABME), Oct 2019, pp. 1–4. Wiley & Sons, 2009.
[5] Y. ElNakieb et al., “Towards accurate personalized autism di- [22] Christian F Beckmann, Mark Jenkinson, and Stephen M Smith,
agnosis using different imaging modalities: smri, fmri, and “General multilevel linear modeling for group analysis in
dti,” in 2018 IEEE International Symposium on Signal Pro- fmri,” Neuroimage, vol. 20, no. 2, pp. 1052–1063, 2003.
cessing and Information Technology, Dec 2018, pp. 447–452. [23] Yufeng Zang et al., “Regional homogeneity approach to fmri
[6] Omar Dekhil et al., “Identifying personalized autism related data analysis,” Neuroimage, vol. 22, no. 1, 2004.
impairments using resting functional mri and ados reports,” in [24] Lingzhong Fan et al., “The human brainnetome atlas: a new
Medical Image Computing and Computer Assisted Interven- brain atlas based on connectional architecture,” Cerebral cor-
tion – MICCAI 2018, 2018, pp. 240–248. tex, vol. 26, no. 8, pp. 3508–3526, 2016.
[7] O. Dekhil et al., “A novel cad system for autism diagnosis [25] Pablo M Granitto et al., “Recursive feature elimination with
using structural and functional mri,” in 2017 IEEE 14th Inter- random forest for ptr-ms analysis of agroindustrial products,”
national Symposium on Biomedical Imaging (ISBI 2017), April Chemometrics and Intelligent Laboratory Systems, vol. 83, no.
2017, pp. 995–998. 2, pp. 83–90, 2006.

1407

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on May 27,2020 at 11:21:43 UTC from IEEE Xplore. Restrictions apply.