Received: 22 May 2020 Revised: 17 June 2020 Accepted: 17 June 2020 Published on: 19 August 2020

DOI: 10.1002/sam.11480

RESEARCH ARTICLE

Multiclass machine learning classification of functional

brain images for Parkinson’s disease stage prediction

Guan-Hua Huang1 Chih-Hsuan Lin1 Yu-Ren Cai1 Tai-Been Chen2

Shih-Yen Hsu3 Nan-Han Lu2,4,5,6 Huei-Yung Chen7 Yi-Chen Wu7

1
Institute of Statistics, National Chiao
Tung University, Hsinchu, Taiwan Abstract
2
Department of Medical Imaging and We analyzed a data set containing functional brain images from 6 healthy con-
Radiological Sciences, I-Shou University, trols and 196 individuals with Parkinson’s disease (PD), who were divided into
Kaohsiung, Taiwan
five stages according to illness severity. The goal was to predict patients’ PD
3
Department of Information Engineering,
I-Shou University, Kaohsiung, Taiwan
illness stages by using their functional brain images. We employed the fol-
4
Department of Pharmacy, Tajen lowing prediction approaches: multivariate statistical methods (linear discrimi-
University, Pingtung, Taiwan nant analysis, support vector machine, decision tree, and multilayer perceptron
5
Department of Radiology, E-Da Hospital, [MLP]), ensemble learning models (random forest [RF] and adaptive boosting),
I-Shou University, Kaohsiung, Taiwan
and deep convolutional neural network (CNN). For statistical and ensemble
6
School of Medicine, College of Medicine,
I-Shou University, Kaohsiung, Taiwan
models, various feature extraction approaches (principal component analysis
7
Department of Nuclear Medicine, E-Da [PCA], multilinear PCA, intensity summary statistics [IStat], and Laws’ tex-
Hospital, I-Shou University, Kaohsiung, ture energy measure) were employed to extract features, the synthetic minority
Taiwan
over-sampling technique was used to address imbalanced data, and the opti-
Correspondence mal combination of hyperparameters was found using a grid search. For CNN
Guan-Hua Huang, Institute of Statistics, modeling, we applied an image augmentation technique to increase and balance
National Chiao Tung University, 1001
data sizes over different disease stages. We adopted transfer learning to incorpo-
University Road, Hsinchu 30010, Taiwan.
Email: ghuang@stat.nctu.edu.tw rate pretrained VGG16 weights and architecture into the model fitting, and we
also tested a state-of-the-art machine learning model that could automatically
Funding information
generate an optimal neural architecture. We found that IStat consistently out-
Ministry of Science and Technology,
Taiwan, Grant/Award Numbers: MOST
performed other feature extraction approaches. MLP and RF were the analytic
105-2118-M-009-004-MY2, MOST approaches with the highest prediction accuracy rate for multivariate statistical
107-2118-M-009-005-MY2 and ensemble learning models, respectively. Overall, the deep CNN model with
pretrained VGG16 weights and architecture outperformed other approaches;
it captured critical features from imaging, effectively distinguished between
normal controls and patients with PD, and achieved the highest classification
accuracy.

KEYWORDS
deep neural network, functional brain image, machine learning, supervised classification

Stat Anal Data Min: The ASA Data Sci Journal. 2020;13:508–523. wileyonlinelibrary.com/sam © 2020 Wiley Periodicals LLC 508
HUANG et al.
1 I N T RO DU CT ION
Parkinson’s disease (PD) is a degenerative neurologi-
cal disorder related to striatal dopamine deficiency, with
symptoms such as slow movement, muscle stiffness, and
shaking.1 PD has a prevalence of 1–2 per 1000 in the
general population, but the rate is up to 2% in people
aged over 65 years.2 In Taiwan, the prevalence of PD per
100,000 was 84.8 in 2004 and 147.7 in 2011, representing
a 7.9% yearly increase.3 PD illness severity can be classi-
fied into five stages, which are based on the level of clinical
disability.4,5 The type and severity of PD-related symptoms
vary between individuals and according to the different
stages of the disease.
Because PD is degenerative, early detection, which can
limit expenditure and improve a patient’s quality of life,6
is critical. The diagnosis of PD is based on clinical crite-
ria, but misdiagnosis is reportedly as high as 25% of cases,
according to anatomic-pathologic studies.7 Functional
imaging techniques such as positron-emission tomogra-
phy (PET), single photon-emission computed tomography
(SPECT), and functional magnetic resonance imaging can
elucidate the pathophysiology and evolution of PD and aid
in the differential diagnosis of the disease. PET and SPECT
have enabled noninvasive, in vivo visualization of the
progression of striatal neuronal function in patients with
PD.8 Unlike for PET, no on-site cyclotron or radiochem-
istry facilities are required for SPECT imaging because of
its longer half-life. SPECT studies also benefit from the
industrial production of tracers. The lower cost of radio-
tracer synthesis enables the investigation of more patients
through SPECT than through PET.
In clinical practice, SPECT images are typically evalu-
ated visually or through region-of-interest (ROI) analysis.9
ROI techniques involve outlining or positioning the ROI
over the striatum (target region) and the occipital cortex
(reference region) and computing a quantitative measure
termed the background-subtracted striatal uptake ratio.9
An alternate approach is to perform shape and intensity
distribution (surface profile) analysis and use pattern
recognition techniques for differentiation.10,11
Numerous studies have attempted to design a
computer-aided diagnosis system for PD detection. Most
of them have focused on the pattern recognition approach.
In general, features can be extracted from voxels of the
complete brain12,13 or of the striatum.14–16 Feature extrac-
tion from voxels of the complete brain is typically followed
by dimensional reduction, such as principal component
analysis (PCA)15 or singular value decomposition (SVD).13
Feature extraction from voxels of the striatum is typically
followed by feature selection16 or the use of the striatal
binding ratio14,17 as features. Classification into different
509
PD disease stages is performed after feature selection.
Classifiers can be support vector machine (SVM),12,15,16,18
linear or quadratic discriminant analysis,17,18 naïve
Bayes,13 and so on. Deep learning, such as a convolutional
neural network (CNN) framework, is another suitable
method for predicting PD stages.19
Researchers have developed several methods for clas-
sifying PD diagnosis. However, these methods can only
classify individuals as having PD or being healthy. Meth-
ods that enable the classification of multiple PD stages
are rare. Therefore, this study investigated the optimal
multiclass classification for predicting patients’ PD illness
stages by using their functional brain images. The data
analyzed in this study were derived from SPECT imag-
ing with 99m Tc-TRODAT-1 ligands.7,20 We used data from
6 healthy controls and 196 patients with PD (with iden-
tification of disease stage from 1 to 5). We developed
an analytic system for the multiclass classification of PD
stages. This system includes a series of methods for image
preprocessing, feature extraction, imbalanced data adjust-
ment, and three types of classification models: multivari-
ate statistical methods, ensemble learning models, and
deep CNN.
2 MATERIALS
Patients’ imaging and diagnostic reports, collected
between March 2006 and August 2013, were extracted
from the picture archiving and communication system
at E-Da Hospital, I-Shou University, Taiwan. After we
excluded those who were ineligible, 202 patients remained
for the analysis. Each patient underwent 99m Tc-TRODAT-1
SPECT imaging. The image matrix was 128 × 128, and
a total of 64 images were captured at a collection rate
of 25 s per image. These SPECT images were stored in
the Digital Imaging and Communications in Medicine
format.
According to the diagnostic reports, 6 patients had
healthy brain function (3 men and 3 women, with a
median age of 47.5 years) and 196 patients had PD (80 men
and 116 women, with a median age of 69.13 years). The
physical symptoms of PD were classified into five differ-
ent stages using the Hoehn and Yahr Scale (HYS),4 with
stages I and V indicating the mildest and most severe ill-
ness, respectively. The number of patients in stages I to V
was 22, 27, 53, 87, and 7, respectively.
More details on the exclusion criteria, imaging instru-
ment, and experimental design can be found in Hsu
et al..21 This clinical study was approved by the Medical
Ethics Committee of E-Da Hospital. All patients signed
written informed consent before participating.
510
FIGURE 1 Proposed analytic system for multiclass machine learning classification of functional brain images for Parkinson’s disease
stage prediction
3 M ET H ODS
The goal of this study was to predict patients’ PD ill-
ness stage by using their SPECT functional brain images.
All SPECT images were preprocessed before any analy-
sis to remove noise or correct for errors. We used the
following machine learning methods for PD classifica-
tion: multivariate statistical methods (linear discriminant
analysis [LDA], SVM, decision tree [DT], and multilayer
perceptron [MLP]), ensemble learning models (random
forest [RF] and adaptive boosting [AdaBoost]), and deep
CNN. For statistical and ensemble models, various fea-
ture extraction approaches (PCA, multilinear PCA, inten-
sity summary statistics [IStat], and Laws’ texture energy
measure [LTEM]) were employed to extract features. The
synthetic minority over-sampling technique (SMOTE) was
used to address imbalanced data concerns, and the opti-
mal combination of hyper-parameters was obtained using
a grid search. For CNN modeling, we applied an image
augmentation technique to increase and balance data
sizes across different disease stages, adopted the trans-
fer learning concept to incorporate pretrained VGG16
weights and architecture into the model fitting, and used a
state-of-the-art machine learning model that can generate
an optimal neural architecture automatically (AutoML).
HUANG et al.
The overall structure of our analytic system is illustrated
in Figure 1. The Python programming language22 was the
analytic tool employed in the implementation of these
methods.
We used 5-fold cross-validation for model evalua-
tion: the data were divided into five parts, with four of
them serving as the training set and one as the test set,
with a total of five different training-test combinations
(five cross-validation rounds). Our analytic system, which
includes a series of methods for image preprocessing,
feature extraction, imbalanced data adjustment, and clas-
sification, was applied to the training set. When tuning
the model hyperparameters, the training data were fur-
ther divided into five folds: four folds were fitted with
different parameter settings and one fold was for vali-
dation. The optimal parameter setting was determined
using the average accuracy of five validations. The perfor-
mance of various approaches was evaluated in terms of the
training accuracy, test accuracy, and F1 score. Accuracy
is defined as the proportion of images that are correctly
classified. Training accuracy denotes the average accu-
racy of the training sets during the five cross-validation
rounds. Test accuracy refers to the accuracy on the five
test parts. In binary classification, F1 score is the har-
monic mean of the true positive rates over truths (ie, recall)
HUANG et al.
F I G U R E 2 Preprocessing procedure for three-dimensional (3D) single photon-emission computed tomography (SPECT) images. Step
1 involved selecting a single slice from the original 3D stereo image that contained the clearest striatum shape as the target data for
subsequent analysis. Step 2 involved trimming the image from 128 × 128 to 50 × 50 to obtain a cropped image that contained a complete brain
image with only a small portion of the black background
and over positives (ie, precision). For our multiclass clas-
sification, we adopted the macro-F1 score, which is the
equal-weighted average of the F1 scores of each class. We
calculated the macro-F1 score according to the five test
parts.
3.1 Image preprocessing
For SPECT images, statistical parametric mapping (SPM)
software23,24 can be used for preprocessing. In the early
stages of the study, we attempted to use SPM to prepro-
cess the collected SPECT images, but the results were of
insufficient quality. Our SPECT brain images were col-
lected through general clinical pipelines. Compared with
the images obtained in typical research projects, our brain
image collection procedure was less standardized; there-
fore, the image noise induced by human or environmen-
tal factors was larger. We speculated that these noises
caused the violation of SPM model assumptions, resulting
in implementation results that were poorer than expected.
Our preprocessing, therefore, proceeded as follows:
from the original three-dimensional (3D) stereo image,
we first selected a single slice that contained the clearest
striatum shape as the target data for subsequent anal-
ysis. PD diagnosis is mainly based on the characteriza-
tion of the striatum, which only occupies a small part of
the brain. Complete brain images may contain too much
unnecessary information, leading to an increase in ana-
lytic burden. By focusing on the striatum, we reduced our
data from the original 3D stereo image (128 × 128 × 64)
to a two-dimensional (2D) planar image (128 × 128), as
depicted in Figure 2. Because most of the image consisted
511
of a black background, and only the middle section con-
tained the brain image, we further trimmed the image
from 128 × 128 to 50 × 50 pixels. The cropped image con-
tains a complete brain image with only a small portion of
the black background (Figure 2).
3.2 Feature extraction
For multivariate statistical methods and ensemble learn-
ing models, we used pixel-based features (ie, every pixel
represents a feature) for analysis. Our input image had
2500 (50 × 50) pixels (features). Conventional machine
learning models could not be directly applied because the
number of features was too large. We thus applied the fol-
lowing approaches to extract vital features for later PD
classification analysis.
3.2.1 Principal component analysis
PCA is a commonly used dimensional reduction tech-
nique that involves explaining the variance–covariance
structure of a set of variables through a few linear com-
binations of these variables (ie, principal components
[PCs]) and then using these PCs to replace the original
variables.25 Our 50 × 50 input image was first flattened into
a one-dimensional vector with a length of 2500 pixels. We
then performed PCA on the training data with 2500 vari-
ables and extracted the first k (≤2500) PCs, which were
selected using the proportion of total variance explained or
the scree plot.25 The PCA function in the sklearn Python
library was used to perform the PCA.
512
3.2.2 Multilinear PCA
When analyzing data in matrix format (eg, images),
PCA first vectorizes these matrix data and then pro-
ceeds with a large variance–covariance matrix, which
can be inefficient and unstable.26 MPCA is a modifica-
tion of PCA. It preserves the natural matrix structure
of the data in searching for PCs. For our 50 × 50 input
image X i , i = 1, … , n, MPCA will find A1 (50 × p)
and A2 (50 × q) with p ≤ 50 and q ≤ 50 that minimize
∑n
i=1 ||(Xi − X) − A1 A1 (Xi − X)A2 A2 || , where X is the
T T 2
average of the input images and ||⋅|| is the matrix Frobenius
norm. In the solution, the low-dimensional core tensors
AT1 (Xi − X)A2 , i = 1, … , n are used to replace the original
images for further analysis; thus, we reduced the num-
ber of features to pq (≤2500). Notably, PCA performs SVD
for a 2500 × 2500 variance-covariance matrix, whereas in
MPCA, the SVD is performed for only two 50 × 50 matri-
ces, which is much more parsimonious in terms of param-
eter usage. MPCA was implemented with the Python pro-
gramming language.
3.2.3 Intensity summary statistics
Given the flattened vector of pixel values from an image
(x1 , x2 , … , x2500 ), we calculated its mean, variance, kur-
tosis, skewness, maximum, minimum, and Shannon
entropy. In addition, to create the histogram representing
the distribution of pixel values, we standardized the vec-
(xj −min xi )
tor components as yj = i
, j = 1, … , 2500, and
(max xi −min xi )
i i
calculated the proportions
[ ) [ of )yj ’s that[ fell in each
) [ of the ]
1 1 2
following b bins 0, b , b , b , … , b , b , b−1
b−2 b−1
b
, 1
Our intensity summary statistics consisted of these sum-
mary statistics and the histogram’s bin proportions, a fea-
ture vector of length 7 + b. Python functions were adopted
for calculating summary statistics.
3.2.4 Laws’ texture energy measure
An image contains information not only on the bright-
ness of the image on each unit pixel but also on the
interaction among units. Image texture is a set of metrics
that quantify the spatial arrangement of color or intensi-
ties in an image.27 Image texture can be defined in two
manners: the structured approach considers image texture
as a set of primitive texels in some regular or repeated
patterns, whereas the statistical approach defines texture
as quantitative measures of the arrangement of intensi-
ties in a region. Compared with the structured approach,
the statistical approach is more general and simpler to
HUANG et al.
compute and is used more often in practice. LTEM are tex-
ture features generated through the statistical approach
and consist of mine full images that are created by apply-
ing nine types of texture filters to the original image.
These nine filtered images measure level-, edge-, spot-, and
ripple-related contents of the original image (Figure 3).
Here, we applied the aforementioned PCA, MPCA, and
IStat to the nine texture images from the LTEM in addition
to the original image. Thus, by including texture informa-
tion, we expanded the feature set generated by each feature
extraction by 10 times. A Python implementation of LTEM
was used to create texture images.
3.3 Class imbalance adjustment
The distribution of our data over multiple PD stages
(classes) was relatively imbalanced, which could result in
a distortion of the classification model, biasing the predic-
tion results toward the majority class at the expense of the
minority class. To address the class imbalance matter, we
adopted the following approaches.
3.3.1 Synthetic minority over-sampling
technique
SMOTE is an over-sampling approach in which the minor-
ity class is over-sampled by creating “synthetic” exam-
ples rather than over-sampling with replacements.28 We
applied SMOTE to the extracted features used by the mul-
tivariate statistical and ensemble learning classification
analysis. We selected three nearest neighbors for creat-
. ing synthetic minority samples and ensured that made all
classes in the training set contained the same number of
samples. The SMOTE function in the imblearn Python
package was used to perform SMOTE.
3.3.2 Image augmentation
To construct a powerful image classifier using little train-
ing data, image augmentation is typically required to
boost the performance of deep neural network mod-
els. The image augmentation process uses a combi-
nation of affine transformations to generate duplicate
images that are rotated, shifted, flipped, or zoomed in
or out. We augmented our training images by using the
ImageDataGenerator function in the Keras Python
library: images from each PD class were randomly selected
to perform the transformations until the preset sample
size for the class was reached. Our augmentation process
not only increased the sample size but also attempted to
HUANG et al. 513

F I G U R E 3 Texture feature images generated by Laws’ texture energy measures. The first image on the left is the original image; the
nine pictures on the right contain feature images, from left to right and top to bottom, generated using the following nine texture filters,
respectively: L5E5/E5L5, L5R5/R5L5, E5S5/S5E5, S5S5, R5R5, L5S5/S5L5, E5E5, E5R5/R5E5, and S5R5/ R5S5

balance classes in the training data set. Then, both original
and duplicate images were entered into our deep neural
network models.29
3.4 Machine learning classification
The machine learning classification used in this study con-
sisted of multivariate statistical methods, ensemble learn-
ing models, and the deep CNN.
3.4.1 Multivariate statistical methods
LDA adopts multivariate normal distributions to model
input features and assigns objects according to their
posterior probabilities of class membership.30 The
LinearDiscriminantAnalysis function in the
sklearn Python library was used to perform LDA. No
hyperparameter required setting.
An SVM produces a separating hyperplane that has the
largest distance to the nearest training data point and has
nonlinear boundaries by constructing a linear boundary in
a large, transformed version of the feature space.31 We used
the SVC function in the sklearn Python library for SVM.
The one-vs-one strategy was adopted for multiclass classi-
fication (ie, decision_function_shape = “ovo”).
In this study, we tested linear, polynomial, radial basis
function, and sigmoid kernels for mapping inputs
into high-dimensional feature spaces. Parameter val-
ues for kernels were set to be exponentially growing
sequences, following the suggestion in ref. 32. The opti-
mal kernel-parameter combination was selected through
a grid search using 5-fold cross-validation, which was
implemented through the GridSearchCV function in
the sklearn Python library.
514
DT is a nonparametric supervised learning method
that partitions the feature space into a set of rectangles
and then fits a simple model (such as a constant) in each
rectangle.33 We used the DecisionTreeClassifier
function in the sklearn Python library for DT. We con-
sidered hyperparameters, including different strategies for
choosing the split, maximum depth of the tree, and num-
ber of features to scrutinize, when searching for the most
suitable split.
The central concept of an MLP (also known as the
neural network model) is to extract linear combinations
of the inputs as derived features and then model the tar-
get as a nonlinear function of these features. We used
the MLPClassifier function in the sklearn Python
library for MLP. We selected quasi-Newton methods as the
solver for weight optimization (ie, solver = “lbfgs”)
because of our small dataset. We considered hyperparam-
eters, including different hidden layer sizes and activation
functions.
3.4.2 Ensemble learning models
Ensemble learning constructs a classification model by
combining the strengths of a collection of simpler base
models.34 Two families of ensemble methods are typically
distinguished. Bagging fits the same classification function
many times to bootstrap-sampled versions of the training
data and averages the results. The objective is to aver-
age many noisy but approximately unbiased models and
thus reduce the variance. Boosting sequentially applies
the weak (base) classifier to the bootstrap-sampled train-
ing data with repeatedly modified sampling weights; the
weights are increased for those observations that are mis-
classified at the previous classification and the weights are
decreased for those correctly classified.
RF is a substantial modification of bagging
that constructs a large collection of “de-correlated”
trees and then averages them.35 We used the
RandomForestClassifier function in the sklearn
Python library for RF. We considered hyper-parameters,
including different number of trees in the forest, max-
imum depth of the tree, and number of features to
scrutinize, when searching for the most suitable split.
AdaBoost, proposed by Freund and Schapire,36 is
the first practical boosting algorithm. We used the
AdaBoostClassifier function in the sklearn
Python library for AdaBoost. We employed SVM and DT as
the base classifiers. The optimal hyperparameter for each
base classifier was determined first. Then, we selected the
boosting hyperparameters for this optimal base classifier,
including different maximum numbers of base classifiers
HUANG et al.
at which boosting was terminated and learning rates that
shrank the contribution of each classifier.
3.4.3 Deep convolutional neural
network
Deep learning is part of a broader family of machine learn-
ing methods based on neural network models. A CNN
is a deep learning architecture that constructs a series of
hidden layers to progressively extract higher level features
from the raw input. Hidden layers in CNN typically consist
of convolutional layers for feature extraction, activation
layers (functions) for nonlinear transformation, pooling
layers for dimension reduction, and fully connected layers
to flatten the matrix to a one-dimensional form. In con-
volutional layers, a learnable filter (small matrix) is con-
volved across the width and height of one slice of the input
volume, computing the dot product between the entries of
the filter and the input and producing a 2D activation map
for a specific type of feature that the filter intends to learn.
Stacking the activation maps for all filters along the depth
dimension of the input volume forms the output volume of
the convolution layer.37 The pooling procedure partitions
the input image into a set of nonoverlapping rectangles
and, for each such subregion, outputs the maximum or the
average. The pooling layer progressively reduces the size
of the image to avoid rapid retraining, alleviate computa-
tional burden and hence also control overfitting. Typically,
the pooling layer is used after the convolution operation
is performed. Activation functions can generate nonlinear
mapping from inputs to complex outputs. When apply-
ing CNNs for image classification, the rectified linear unit
(ReLU) activation functions are commonly used after con-
volutional layers, and the softmax function is used in the
final fully connected layer. In contrast to other image clas-
sification algorithms that rely on prior knowledge and
human effort in feature design, CNNs can automatically
extract input features through learned filters.
Transfer learning is a machine learning method that
utilizes knowledge learned from one task as the starting
points on related ones. Implementing transfer learning in
the CNN involves reusing the first several layers of the net-
work for a similar task with much more data. By using
large amounts of data, hidden layers earlier in the net-
work learn to recognize a variety of base elements that
are typically not specific to the exact task that the net-
work may be used for; therefore, transfer learning enables
a CNN to have a broader application. VGG1638 is cur-
rently one of the highest-quality CNN architectures. We
used the VGG16 function in the keras Python library to
adopt the pretrained VGG16 model whose weights were
HUANG et al.
trained from ImageNet, which is a large and publicly avail-
able image database with a total of 14 million images and
22,000 visual categories.39 VGG16 requires input images
with three channels (eg, color). In our preprocessing pro-
cedure, we had used the original SPECT 3D stereo image
to select a single slice that contained the clearest striatum
shape as the target data for subsequent analysis. Our input
data were grayscale medical imaging, and we thus repre-
sented three channels for VGG16 inputs, either with the
same selected (2D) image or with the three consecutive
images above and below the selected one. We used weights
from the first 18 layers of VGG16 on ImageNet and fur-
ther trained three fully connected layers with sizes 128,
64, 6 and activations ReLU, ReLU, softmax, respectively.
The RMSprop and Adam optimizers in the keras Python
library, at their default hyper-parameter values, were used
to obtain weight estimation.
The VGG16 architecture was fine-tuned on ImageNet
data that consist of common images in daily life; there-
fore, it may not be applicable to our medical imaging data.
Neural architecture search (NAS)40 aims to identify the
optimal neural network architecture for the given learning
task and data set, which is the key component of AutoML.
NAS adopts reinforcement learning (RL) to train a recur-
rent neural network controller to produce a child network
by combining a set of building blocks of the network. Then,
we can use the child network’s prediction accuracy as the
reward signal to update the controller, and as a result,
produce a new child network with a higher accuracy in
the next iteration of RL. Although NAS is powerful, it is
computationally expensive and time consuming. The effi-
cient neural architecture search (ENAS)41 accelerates the
training process of NAS by forcing all child networks to
share weights to avoid training each child network from
the beginning to convergence. Auto-Keras42 is a Python
library for ENAS. We used the ImageClassifier func-
tion in the autokeras library to automatically search for
the architecture and hyperparameters of the deep CNN
most suitable to our PD illness stage prediction by using
SPECT images.
4 R E S U LTS
4.1 Results from multivariate
statistical classification
To determine the number of PCs to retain in fitting the
PCA, we calculated the cumulative proportions of the total
variance explained by k PCs, k = 1, 2, 3, … on the
basis of four training folds for each cross-validation round
(Figure 4). We selected 50 components because more than
97% of the total variance was consistently explained by
515
F I G U R E 4 Cumulative proportions of total variance
explained by k principal components (PCs), k = 1, 2, 3, … based on
four training folds for each cross-validation round. In the table on
the right, the values represent the cumulative percentages of the
total variance explained by the first 50 PCs. The x-axis of the plot
has only 160 rather than 2500 PCs because the explained variance
ratio was nearly zero after PC 160
these PCs in each cross-validation round. For compari-
son, MPCA was thus fitted on the numbers of image bases
p = q = 7 (ie, 7 × 7 = 49).
After features were extracted using the PCA, MPCA,
or IStat methods, we first applied SMOTE to ensure that
all classes contained the same number of samples; sub-
sequently, machine learning classification based on mul-
tivariate statistical methods was used for PD illness stage
prediction. Table 1 presents the accuracy of the differ-
ent feature extraction methods according to the 5-fold
cross-validation. IStat outperformed the other extraction
methods (in both training and test accuracy) regardless
of the multivariate statistical classification used. When
adopting a specific feature extraction method, MLP gen-
erally resulted in higher test accuracy than other multi-
variate statistical classifications did. When SVM or MLP
adopted features from IStat, we obtained the highest test
accuracy (52%). However, when DT adopted features from
MPCA, the test accuracy was as low as 26.2%. Performance
results based on F1 scores (Table 1) were generally sim-
ilar to the aforementioned findings. The performance of
MLP was an exception: when MLP adopted features from
IStat, it did not have the highest F1 score. The MLP with
IStat features incorrectly classified all images represent-
ing healthy brains and most illness stages. This approach
appeared to bias the prediction results toward majority
classes and ignore the two minority classes, which led to
higher accuracy but poorer F1 scores.
We also applied PCA, MPCA, and IStat to the nine
texture images from LTEM in addition to the original
image and thus expanded the feature set generated by
each feature extraction by 10 times (Table 2). The addi-
tional texture information from LTEM did not appear to
alter the performance ranking of various approaches; how-
ever, it improved the training accuracy and reduced the
test accuracy and F1 score, implying that these textures
516 HUANG et al.

T A B L E 1 Results of Parkinson’s Feature Number of Training Test Test

disease stage prediction using Model extraction features accuracy accuracy F1 score
multivariate statistical methods
LDA PCA 50 0.883 0.450 0.314
MPCA 49 0.884 0.351 0.269
IStat 50 0.911 0.500 0.320
SVM PCA 50 0.999 0.421 0.279
MPCA 49 0.997 0.426 0.289
IStat 50 0.991 0.520 0.370
DT PCA 50 1.000 0.347 0.221
MPCA 49 0.995 0.262 0.180
IStat 50 1.000 0.470 0.365
MLP PCA 50 1.000 0.450 0.355
MPCA 49 1.000 0.450 0.308
IStat 50 1.000 0.520 0.298

Abbreviations: DT, decision tree; IStat, intensity summary statistics; LDA, linear discriminant analysis; MLP,
multilayer perceptron; MPCA, multilinear principal component analysis; PCA, principal component analysis;
SVM, support vector machine.

T A B L E 2 Results of Feature Number of Training Test Test F1

Parkinson’s disease stage Model extraction features accuracy accuracy score
prediction using multivariate
LDA LTEM + PCA 500 (50 × 10) 0.985 0.347 0.261
statistical methods with additional
features from Laws’ texture energy LTEM + MPCA 490 (49 × 10) 0.988 0.297 0.255
measure (LTEM) images LTEM + IStat 500 (50 × 10) 0.963 0.475 0.294
SVM LTEM + PCA 500 (50 × 10) 1.000 0.431 0.238
LTEM + MPCA 490 (49 × 10) 1.000 0.421 0.187
LTEM + IStat 500 (50 × 10) 1.000 0.406 0.146
DT LTEM + PCA 500 (50 × 10) 0.999 0.317 0.272
LTEM + MPCA 490 (49 × 10) 0.991 0.262 0.192
LTEM + IStat 500 (50 × 10) 0.997 0.460 0.250
MLP LTEM + PCA 500 (50 × 10) 1.000 0.416 0.283
LTEM + MPCA 490 (49 × 10) 1.000 0.431 0.298
LTEM + IStat 500 (50 × 10) 1.000 0.470 0.267

Abbreviations: DT, decision tree; IStat, intensity summary statistics; LDA, linear discriminant analysis; LTEM,
Laws’ texture energy measure; MLP, multilayer perceptron; MPCA, multilinear principal component analysis;
PCA, principal component analysis; SVM, support vector machine.

from LTEM did not generate additional useful features for
predictions.
4.2 Results from ensemble learning
classification
The results obtained from applying feature extraction
methods on the original image and on the 10 texture
images (including the original one) from LTEM are
presented in Tables 3 and 4, respectively. IStat outper-
formed PCA and MPCA regardless of the ensemble learn-
ing classification used. When adopting a specific feature
extraction method, RF resulted in higher test accuracy and
F1 score than did AdaBoost. When RF adopted features
from IStat, we obtained the highest test accuracy (54.5%)
and F1 score (38.5%). Additional texture information from
LTEM did not appear to improve the test accuracy and F1
score of various approaches.
HUANG et al. 517

TA B L E 3 Results of Parkinson’s disease stage prediction using ensemble learning models

Feature Number of Training Test Test F1
Model extraction features accuracy accuracy score
RF PCA 50 1.000 0.470 0.282
MPCA 49 1.000 0.406 0.235
IStat 50 1.000 0.545 0.385
AdaBoost + DT PCA 50 1.000 0.272 0.152
MPCA 49 1.000 0.347 0.229
IStat 50 1.000 0.446 0.315
AdaBoost + SVM PCA 50 0.617 0.322 0.234
MPCA 49 0.578 0.268 0.182
IStat 50 0.823 0.515 0.350

Abbreviations: AdaBoost, adaptive boosting; DT, decision tree; IStat, intensity summary statistics; MPCA, multilinear
principal component analysis; PCA, principal component analysis; RF, random forest; SVM, support vector machine.

TA B L E 4 Results of Parkinson’s disease stage prediction using ensemble learning models with additional
features from Laws’ texture energy measure (LTEM) images
Feature Number of Training Test F1
Model extraction features accuracy Test accuracy score
RF LTEM+PCA 500 (50 × 10) 1.000 0.406 0.202
LTEM+MPCA 490 (49 × 10) 1.000 0.455 0.233
LTEM+IStat 500 (50 × 10) 1.000 0.564 0.341
AdaBoost + DT LTEM + PCA 500 (50 × 10) 1.000 0.327 0.193
LTEM + MPCA 490 (49 × 10) 1.000 0.307 0.201
LTEM + IStat 500 (50 × 10) 1.000 0.421 0.301
AdaBoost + SVM LTEM + PCA 500 (50 × 10) 0.762 0.381 0.176
LTEM + MPCA 490 (49 × 10) 0.789 0.401 0.113
LTEM + IStat 500 (50 × 10) 0.766 0.371 0.153

Abbreviations: AdaBoost, adaptive boosting; DT, decision tree; IStat, intensity summary statistics; MPCA, multilinear principal
component analysis; PCA, principal component analysis; RF, random forest; SVM, support vector machine.

4.3 Comparison of feature extraction
approaches
To compare the performance of PCA and MPCA on our
SPECT image data, in each cross-validation round, PCA
or MPCA was applied on four training folds to produce
PC loadings or image bases (A1 , A2 ), which could then be
used to reconstruct images in the training and test folds.
Figures 5 and 6 illustrate the reconstruction of one random
training image and one random test image, respectively.
For the training image (Figure 5), the component-wide
sum of absolute difference between the original and recon-
structed images (the “Diff” value in the figure) for PCA
was much smaller than that for its MPCA counterpart
(ie, k ≈ p × q). The absolute differences for both PCA and
MPCA decreased as the number of features increased.
For the test image (Figure 6), MPCA could progressively
improve the reconstruction through the addition of more
features, but PCA appeared not to make much difference
after 100 PCs. These results exemplify the tendency of
overfitting in PCA. Notably, PCA experienced the “large
p and small n” problem, where the sample size for the
training set (161) was much smaller than the length of the
input vector (2500). This problem had a smaller impact on
MPCA because it is much more parsimonious in parame-
ter usage.
IStat consistently outperformed other feature extrac-
tion approaches regardless of the multivariate statistical
or ensemble learning classification used for prediction
(Tables 1 and 3). We observed no clear difference in predic-
tion accuracy and F1 score between PCA and MPCA. Addi-
tional texture information from LTEM did not appear to
518 HUANG et al.

F I G U R E 5 Reconstructions of one randomly selected training image by using principal component analysis (PCA) and multilinear
PCA (MPCA). The first image on the left is the original image; the first and third layers of the 12 images on the right are the results of PCA
reconstruction, and the second and fourth layers are the results of MPCA reconstruction. The Diff value in the figure is the component-wide
sum of absolute difference between the original and reconstructed images. PCA_nc denotes the number of principal components used for
PCA, and MPCA_p,q refers to the number of image bases used for MPCA

improve the test accuracy of various approaches (Tables 2
and 4).
We further investigated the effect of the number of fea-
tures retained in the three feature extraction methods. To
facilitate the comparison, here we only used LDA for clas-
sification. Figure 7 illustrates the training and test accu-
racies as the number features increased for the different
extraction methods. All three extraction methods per-
formed fairly well in the training set, with the increased
feature numbers improving the training accuracy. For the
test set, IStat consistently exhibited the highest test accu-
racy for a given number of features. The prediction in
the test set, in general, were initially more accurate but
then progressively worsened with the increased number of
HUANG et al. 519

F I G U R E 6 Reconstructions of one randomly selected test image by using principal component analysis (PCA) and multilinear PCA
(MPCA). The first image on the left is the original image; the first and third layers of the 12 images on the right are the results of PCA
reconstruction, and the second and fourth layers are the results of MPCA reconstruction. The Diff value in the figure is the component-wide
sum of absolute difference between original and reconstructed images. PCA_nc refers to the number of principal components used for PCA,
and MPCA_p, q denotes the numbers of image bases used for MPCA

features. Notably, we observed a considerable increase in 4.4 Results from the deep
IStat’s test accuracy when the number of features was 70 convolutional neural network
or 75. With an appropriately selected bin size for the his-
togram, IStat can generate features that benefit prediction We used Python’s VGG16 function to fit the VGG16 model
accuracy. with pretrained weights from ImageNet. VGG16 requires
520 HUANG et al.

FIGURE 7 Effect of the number of features retained in the three feature extraction methods—principal component analysis (PCA)
and multilinear PCA (MPCA), and intensity summary statistics (IStat). We only used linear discriminant analysis (LDA) for classification.
The left plot illustrates the training accuracies as the number of features increases for different extraction methods; the right plot reveals the
test accuracies

TA B L E 5 Results of Parkinson’s disease stage prediction using a deep convolutional neural network
Training Test Test F1
Model The 3 channels for input data accuracy accuracy score
Deep CNN—VGG16 Repeat the image three times 0.922 0.649 0.576
Use three consecutive images 0.912 0.653 0.606
Deep CNN—Auto-Keras Repeat the image three times 0.700 0.431 0.357

three-channel images as its input. We thus repeated our
grayscale images three times to obtain a three-channel
input. The expanded three-channel images in the training
set were augmented to ensure that each PD illness label
contained 250 images. They were then input into VGG16
for modeling. The VGG16 deep CNN model achieved a
64.9% test accuracy and a 57.6% test F1 score (Table 5), rep-
resenting an improvement of approximately 20% and 50%,
respectively, compared with the results from multivariate
statistical and ensemble learning classifications.
We also used the three consecutive images above and
below the selected slice in our preprocessing as VGG16’s
input. We did observe an improvement in test accuracy
and F1 score; nevertheless, the two types of inputs yielded
very similar prediction results (Table 5). We speculate that
the difference between the three consecutive images was
not obvious; thus, the model construction was not greatly
affected by the input data changes.
When autokeras in Python was used to automati-
cally search for the architecture and hyperparameters of
the deep CNN, the three channels of the input data were
formed by the same grayscale image, and the augmented
training set had 250 images for each PD illness class.
The performance of the model automatically generated by
autokeras was not only inferior to that of the VGG16
model but also poorer than that of some multivariate sta-
tistical and ensemble learning models (Table 5). The five
deep CNN architectures generated by autokeras for the
five cross-validation rounds differed substantially; they
contained 50, 32, 87, 93, and 83 blocks, respectively. Our
data appeared to be insufficient for autokeras to train
a stable and robust model, which ultimately resulted in a
relatively poor prediction ability.
To examine how the sample size and the number of
augmented images affected the performance of the VGG16
deep CNN model, we executed the model on all of the orig-
inal samples, a random selection of two thirds of it, or a
random selection of half of it. The random sampling was
stratified by PD illness stages, except for the normal and
fifth stages, where the number of images (six and seven,
respectively) were limited; therefore, all images for those
two groups were selected for analysis. For each sample
type, the repeated three-channel images of the training set
were augmented to create balanced, 150, 250, 500, or 700
images for each illness stage. Here, “balanced” indicates
that all stages had the same number of images as that of
the largest stage. The training and test accuracies from
various combinations of sample and augmented sizes are
HUANG et al.
F I G U R E 8 Effects of the sample size and number of augmented images on the performance of the VGG16 deep convolutional neural
network (CNN) model. The left plot illustrates the training accuracies from various combinations of sample sizes and augmented image
numbers; the right plot reveals the test accuracies
revealed in Figure 8. Within each sample type, as the num-
ber of augmented images increased, the training accuracy
increased, whereas the test accuracy first improved and
then became poorer. However, when adopting the same
number of augmented images, the increased sample size
appeared to reduce the training accuracy and increase the
test accuracy.
5 DISCUSSION
In the classification of high-dimensional data (eg, imag-
ing, microarray, or genome sequencing), feature extraction
is typically first performed to achieve dimension reduc-
tion; then, machine learning classification is applied to
predict the data class labels. This study compared the
performance of various combinations of feature extrac-
tion and machine learning methods in classifying medical
imaging data. We considered dimension reduction, hand-
crafted, and deep learning feature extraction approaches.
We employed the following machine learning classifica-
tions: multivariate statistical methods, ensemble learning
models, and the deep convolutional neural network. For
multivariate statistical methods and ensemble learning
models, we observed the highest accuracy when MLP and
RF, respectively, adopted features from IStat. Overall, the
deep CNN model with pretrained VGG16 weights and
architecture outperformed other approaches and achieved
a greater than 20% and 50% improvement in test accuracy
and F1 score, respectively, compared with the multivariate
statistical and ensemble learning classifications.
The deep learning model tends to require con-
siderably more data to achieve its full capacity than
conventional machine learning algorithms. When ana-
lyzing “not-so-big data,” the conventional wisdom favors
521
carefully constructed multivariate statistical or ensem-
ble learning models over complicated deep learning
approaches. When MLP and RF adopted features from
IStat, we observed average test accuracies of 49.5% and
55.5%, respectively, whereas the AutoML deep CNN model
only reached a test accuracy of 43.1%. These findings were
consistent with our expectations. However, transfer learn-
ing that used weights trained from large and publicly
available data on our tasks compensated for the short-
age of data in applying deep learning models. Our VGG16
deep CNN model with pretrained weights from ImageNet
achieved a test accuracy of 64.9%. Medical image data are
rare and their labeling is expensive; therefore, having a
small to medium amount of data is not unusual. The trans-
fer learning approach can thus be applied to enhance the
deep CNN model’s capacity in analyzing these small- or
medium-sized data. Moreover, different criteria may be
employed for certain diseases’ diagnosis. The diagnosis of
PD can serve as an example: in addition to the HYS used
in the current study, the Unified Parkinson’s Disease Rat-
ing Scale43 is another commonly used rating scale. When
several SPECT image datasets with class labels that were
generated through different diagnostic criteria are avail-
able, transfer learning can use the knowledge obtained
from one data set as the starting points for another set and
thus combine these data sets to form a better classifier.
Our analysis revealed that the number of augmented
images and the sample size can affect the performance
of the VGG16 deep CNN model. Image augmentation not
only increased the amount of data but also introduced
some noise to enhance the robustness of the model. Addi-
tion of more augmented images can thus improve both
training and test accuracies. However, excessively expand-
ing training images is equivalent to repeating the training
on the same data, leading to more measurement errors
522
and less prediction improvement. When the same number
of augmented images is used, the larger data can gener-
ate an analytic set with rich variation, whereas the smaller
data have many similar images with small transforms; as a
result, classifiers for large size data are superior in predict-
ing unknown future, whereas models constructed for less
variable data have a better goodness of fit. Applying trans-
fer learning in the VGG16 deep CNN model is more appro-
priate when the source task (eg, ImageNet object classifica-
tion) is sufficiently related to the target task (eg, functional
brain images for PD stage prediction). Our results indi-
cated that the increased sample size appeared to reduce the
training accuracy, which indirectly support the aforemen-
tioned assertion because less variable target data are more
similar to the source data than are data with richer vari-
ation. We thus offer the following suggestions for apply-
ing deep CNN models with transfer learning. First, the
retained number of augmented images can be determined
using cross-validation, as described in Section 4. Second,
the larger the target data are, the higher the predictive
ability of the model is. Third, to obtain the largest bene-
fit from transfer learning, the source data must be vastly
larger than and sufficiently similar to the target data.
The handcrafted feature extraction approach IStat
consistently outperformed the dimensional reduction
approaches of PCA and MPCA. PCA and MPCA solely
use first- and second-order moments (ie, mean and
variance–covariance) to generate summary features for
classification, whereas IStat includes extra higher order
moments that appear necessary for predicting class labels.
The deep CNN model constructs convolutional and acti-
vation layers to form non-linear combinations of input
variables and can thus learn flexible and useful features for
class prediction.
To include as much information as possible in the anal-
ysis, we examined the effects of including extra texture
information from LTEM. These efforts have the poten-
tial to create additional features that are useful in class
prediction, but they did not appear to improve the perfor-
mance of the classification models. One explanation might
be that our data were of insufficient size to observe some
extreme values of created features that have larger effects
on the prediction of class labels. Enlarging the sample size
might increase the likelihood of observing these extreme
values and thus help verify their effects on prediction. Con-
ventional approaches are constructed around the mean
and focus on common event prediction, whereas extreme
values are on the tail of the distribution, with a rare occur-
rence. Therefore, as is common in extreme value analysis44
or in genetic studies for testing the contribution of rare
variants to the disease,45,46 new approaches are required
that are based on collapsing extreme values in different fea-
tures into a single variable (eg, block maxima), followed
HUANG et al.
by adopting the generalized extreme value distribution for
analysis.
ACKNOWLEDGMENTS
This research was partially supported by grants
from the Ministry of Science and Technology, Tai-
wan (MOST 105-2118-M-009-004-MY2, and MOST
107-2118-M-009-005-MY2). We are grateful to the National
Center for High-performance Computing, Taiwan for
computer time and facilities. This manuscript was edited
by Wallace Academic Editing.
ORCID
Guan-Hua Huang https://orcid.org/0000-0002-1802-
3855
REFERENCES
1. A. H. Schapira et al., Perspectives on recent advances in the under-
standing and treatment of Parkinson’s disease, Eur. J. Neurol.
16(10) (2009), 1090–1099.
2. L. M. de Lau and M. M. Breteler, Epidemiology of Parkinson’s
disease, Lancet Neurol. 5(6) (2006), 525–535.
3. W. M. Liu et al., Time trends in the prevalence and incidence of
Parkinson’s disease in Taiwan: A nationwide, population-based
study, J. Formosan Med. Assoc. 115(7) (2016), 531–538.
4. M. M. Hoehn and M. D. Yahr, Parkinsonism: onset, progression
and mortality, Neurology 17(5) (1967), 427–442.
5. Parkinson’s Foundation: Stages of Parkinson’s, (2020),
Miami, FL, Parkinson’s Foundation: available at https://
www.parkinson.org/Understanding-Parkinsons/What-is-
Parkinsons/Stages-of-Parkinsons
6. F. L. Pagan, Improving outcomes through early diagnosis of
Parkinson’s disease, Am. J. Managed Care 18(7) Suppl (2012),
S176–S182.
7. L. Wang et al., SPECT molecular imaging in Parkinson’s disease,
J. Biomed. Biotechnol. 2012 (2012), 412486.
8. J. Booij et al., Imaging of the dopaminergic neurotransmission sys-
tem using single-photon emission tomography and positron emis-
sion tomography in patients with parkinsonism, Eur. J. Nuclear
Med. 26(2) (1999), 171–182.
9. C. Scherfler and M. Nocker, Dopamine transporter SPECT: How
to remove subjectivity? Mov. Disord. 24(S2) (2009), S721–S724.
10. R. Prashanth et al., High-accuracy classification of Parkin-
son’s disease through shape analysis and surface fitting in
123I-Ioflupane SPECT imaging, IEEE J. Biomed. Health Inf.
21(3) (2017), 794–802.
11. R. T. Staff et al., Shape analysis of 123i-n-ω-fluoropropyl-2-
β-carbomethoxy-3β-(4-iodophenyl) nortropane single-photon
emission computed tomography images in the assessment of
patients with Parkinsonian syndromes, Nuclear Med. Commun.
30(3) (2009), 194–201.
12. I. A. Illan et al., Automatic assistance to Parkinson’s disease diag-
nosis in DaTSCAN SPECT imaging, Med. Phys. 39(10) (2012),
5971–5980.
13. D. J. Towey, P. G. Bain, and K. S. Nijran, Automatic classifica-
tion of 123I-FP-CIT (DaTSCAN) SPECT images, Nuclear Med.
Commun. 32(8) (2011), 699–707.
HUANG et al.
14. R. Prashanth et al., Automatic classification and prediction mod-
els for early Parkinson’s disease diagnosis from SPECT imaging,
Expert Syst. Appl. 41(7) (2014), 3333–3342.
15. A. Rojas et al., Application of empirical mode decomposition
(EMD) on DaTSCAN SPECT images to explore Parkinson disease,
Expert Syst. Appl. 40(7) (2013), 2756–2766.
16. F. Segovia et al., Improved parkinsonism diagnosis using a par-
tial least squares based approach, Med. Phys. 39(7) (2012),
4395–4403.
17. N. A. Bhalchandra et al., Early detection of Parkinson’s disease
through shape based features from 123 I-Ioflupane SPECT imaging,
in IEEE 12th International Symposium on Biomedical Imaging
(ISBI), New York, NY , IEEE, New York, NY, Vol 2015, 2015,
963–966. doi: 10.1109/ISBI.2015.7164031.
18. W. Caesarendra et al., A pattern recognition method for stage
classification of Parkinson’s disease utilizing voice features,
in IEEE Conference on Biomedical Engineering and Sciences
(IECBES), Kuala Lumpur, 2014, pp. 87–92.
19. H. Choi et al., Refining diagnosis of Parkinson’s disease with deep
learning-based interpretation of dopamine transporter imaging,
NeuroImage Clin. 16 (2017), 586–594.
20. W. S. Huang et al., Evaluation of early-stage Parkinson’s disease
with 99mTc-TRODAT-1 imaging, J. Nuclear Med. 42(9) (2001),
1303–1308.
21. S. Y. Hsu et al., Feasible classified models for Parkinson disease
from 99m Tc-TRODAT-1 SPECT Imaging, Sensors 19(7) (2019),
1740.
22. Python, available at https://www.python.org/
23. K. J. Friston et al., Statistical parametric maps in functional imag-
ing: A general linear approach, Human Brain Map. 2(4) (1994),
189–210.
24. SPM—Statistical Parametric Mapping, available at https://www.
fil.ion.ucl.ac.uk/spm/
25. R. A. Johnson and D. W. Wichern, Applied multivariate statistical
analysis, 6th ed., Prentice Hall, Upper Saddle River, NJ, 2007,
430–459.
26. T. L. Chen et al., An introduction to multilinear principal compo-
nent analysis, J. Chin. Stat. Assoc. 52(1) (2014), 24–43.
27. L. G. Shapiro and G. C. Stockman, Computer vision, Prentice
Hall, Upper Saddle River, New Jersey, 2001, 235–245.
28. N. V. Chawla et al., SMOTE: Synthetic minority over-sampling
technique, J. Artif. Intell. Res. 16(1) (2002), 321–357.
29. L. Perez and J. Wang, The effectiveness of data augmentation
in image classification using deep learning, arXiv 1712.04621,
(2017).
30. R. A. Johnson and D. W. Wichern, Applied multivariate statisti-
cal analysis, 6th ed., Prentice Hall, Upper Saddle River, NJ, 2007
Chapter 11.
31. T. Hastie, R. Tibshirani, and J. Friedman, The elements of statis-
tical learning, 2nd ed., Springer, New York, NY, 2009 Chapter
12.
523
32. C. W. Hsu, C. C. Chang, and C. J. Lin, A practical guide to support
vector classification, Technical Report, Department of Computer
Science and Information Engineering, National Taiwan Univer-
sity, Taipei, 2016.
33. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of
Statistical Learning, 2nd ed., Springer, New York, NY, 2009,
389–409.
34. T. Hastie, R. Tibshirani, and J. Friedman, The elements of statis-
tical learning, 2nd ed., Springer, New York, NY, 2009 Chapter
16.
35. T. Hastie, R. Tibshirani, and J. Friedman, The elements of statis-
tical learning, 2nd ed., Springer, New York, NY, 2009 Chapter
15.
36. Y. Freund and R. E. Schapire, A decision-theoretic generalization
of on-line learning and an application to boosting, J. Comput.
Syst. Sci. 55(1) (1997), 119–139.
37. Wikipedia: Deep learning, available at https://en.wikipedia.org/
wiki/Deep_learning
38. K. Simonyan and A. Zisserman, Very deep convolutional net-
works for largescale image recognition, arXiv (2014), 1409.1556.
39. ImageNet, available at http://image-net.org/
40. B. Zoph and Q. V. Le, Neural architecture search with reinforce-
ment learning, arXiv (2017), 1611.01578v2.
41. H. Pham et al., Efficient neural architecture search via parameter
sharing, arXiv 1802.03268v2, (2018).
42. H. Jin, Q. Song, and X. Hu, Auto-Keras: an efficient neu-
ral architecture search system, arXiv preprint:1806.10282v3,
2019.
43. S. Fahn, R. L. Elton, and UPDRS Program Members, Uni-
fied Parkinson’s Disease Rating Scale, in Recent developments
in Parkinson’s Disease, Vol 2, S. Fahn et al., Eds., Macmillan
Healthcare Information, Florham Park, NJ, 1987, 153–163.
44. S. Coles, An Introduction to Statistical Modeling of Extreme Val-
ues, Springer-Verlag, London, 2001 Chapter 3.
45. S. Morgenthaler and W. G. Thilly, A strategy to discover genes
that carry multi-allelic or mono-allelic risk for common diseases:
a cohort allelic sums test (CAST), Mutat. Res. 615(1–2) (2007),
28–56.
46. M. C. Wu et al., Rare-variant association testing for sequencing
data with the sequence kernel association test, Am. J. Human
Genetics 89(1), (2011), 82–93.
How to cite this article: Huang G-H, Lin C-H,
Cai Y-R, et al. Multiclass machine learning
classification of functional brain images for
Parkinson’s disease stage prediction. Stat Anal Data
Min: The ASA Data Sci Journal. 2020;13:508–523.
https://doi.org/10.1002/sam.11480