You are on page 1of 7


Journal of Biomechanics 43 (2010) 720726

Contents lists available at ScienceDirect

Journal of Biomechanics
journal homepage:

Comparison among probabilistic neural network, support vector machine

and logistic regression for evaluating the effect of subthalamic stimulation in
Parkinson disease on ground reaction force during gait
A.M.S. Muniz a, H. Liu b, K.E. Lyons c, R. Pahwa c, W. Liu b, F.F. Nobre a, J. Nadal a,n
Biomedical Engineering Program, Federal University of Rio de Janeiro, COPPE, P.O. Box 68510, 21941-972, Rio de Janeiro, RJ, Brazil
Department of Physical Therapy and Rehabilitation Sciences, University of Kansas Medical Center, Kansas City, KS, USA
Department of Neurology, University of Kansas Medical Center, Kansas City, KS, USA

a r t i c l e in f o a b s t r a c t

Article history: Deep brain stimulation of the subthalamic nucleus (DBS-STN) is an approved treatment for advanced
Accepted 6 October 2009 Parkinson disease (PD) patients; however, there is a need to further evaluate its effect on gait. This study
compares logistic regression (LR), probabilistic neural network (PNN) and support vector machine
Keywords: (SVM) classiers for discriminating between normal and PD subjects in assessing the effects of DBS-STN
Parkinson disease on ground reaction force (GRF) with and without medication. Gait analysis of 45 subjects (30 normal
Deep brain stimulation and 15 PD subjects who underwent bilateral DBS-STN) was performed. PD subjects were assessed under
Logistic regression four test conditions: without treatment (mof-sof), with stimulation alone (mof-son), with medication
Probabilistic neural network alone (mon-sof), and with medication and stimulation (mon-son). Principal component (PC) analysis
Support vector machine
was applied to the three components of GRF separately, where six PC scores from vertical, one from
Gait analysis
anteriorposterior and one from mediallateral were chosen by the broken stick test. Stepwise LR
analysis employed the rst two and fth vertical PC scores as input variables. Using the bootstrap
approach to compare model performances for classifying GRF patterns from normal and untreated PD
subjects, the rst three and the fth vertical PCs were attained as SVM input variables, while the same
ones plus the rst anteriorposterior were selected as PNN input variables. PNN performed better than
LR and SVM according to area under the receiver operating characteristic curve and the negative
likelihood ratio. When evaluating treatment effects, the classiers indicated that DBS-STN alone was
more effective than medication alone, but the greatest improvements occurred with both treatments
& 2009 Published by Elsevier Ltd.

1. Introduction 2008). Approaches that capture features of the entire waveform

instead of a few parameters may improve the effectiveness of the
Parkinson disease (PD) is a neurodegenerative disorder leading analysis (Chester et al., 2007). Additionally, the correlations
to difculty in motor function, including gait and balance. Deep among variables must be considered to accurately evaluate the
brain stimulation of the subthalamic nucleus (DBS-STN) is a extent of gait abnormalities and to assess the changes resulting
treatment for advanced PD patients with disabling motor from a specic treatment (Schutte et al., 2000).
uctuations, allowing a signicant reduction in dopaminergic A clinical challenge is to understand the disease process as well
medications (Ferrarin et al., 2005). Various studies have evaluated as outcomes of potential interventions. Logistic regression (LR) is
the effects of DBS-STN using clinical motor scores (Krack et al., commonly used as a linear predictive model for diagnostic and
2003; Ostergaard and Sundae, 2006), while only a few have prognostic tasks. Recently, computational intelligence techniques
quantitatively assessed the gait of PD patients (Liu et al., 2005; such as articial neural networks (ANN) and support vector
Ferrarin et al., 2005). Gait speed is shown to be the variable most machines (SVM) have played an important role in gait classica-
affected by the DBS-STN; however, it does not take into account tion and the diagnosis of diseases (Lai et al., 2009). Studies have
atypical waveforms and therefore does not provide enough compared the predictive ability of LR and ANN (Dreiseitl and
information about the gait pattern (Schwartz and Rozumalski, Ohno-Machado, 2002; Song et al., 2005). ANN modeling has been
used in gait analysis focusing on pattern recognition (Hahn et al.,
2005), as well as for classifying normal and pathological patterns
Corresponding author. Tel.: +55 21 25628577; fax: + 55 21 25628591.
(Lafuente et al., 1997; Su and Wu, 2000). SVM has recently been
E-mail address: (J. Nadal). used for automated identication of gait pathologies (Begg et al.,

0021-9290/$ - see front matter & 2009 Published by Elsevier Ltd.

A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726 721

2005; Lai et al., 2009). However, none of the past studies (Schumacher et al., 1996):
compared LR, probabilistic neural network (PNN) and SVM in 1
Px Pn 1
classifying gait patterns or evaluated the effect of therapeutic 1 ebo i 1
bi Pci

interventions on ground reaction force (GRF) of PD patients. where bo is the intercept and bi is the coefcient associated with the explanatory
This study evaluated LR, PNN and SVM models for discriminat- variable PCi. The maximum likelihood estimation method is used to estimate the b
ing between normal and PD subjects using principal components coefcients. For subjects classication, the PC scores were selected as the
derived from the GRF as input variables. For performance independent variables for computing the natural logarithm of the odds ratio; the
ratio between the probabilities that an event will or will not occur (1controls
evaluation, the accuracies (ACC) and the areas under the receiver
and 0PD subjects). The classication threshold was set to 0.5.
operating characteristic (ROC) curves (AUC) based on 1000
bootstrap runs of the classiers were compared. The effects of
2.4. Probabilistic neural networks
DBS-STN on GRF with and without medication were also
evaluated with both the models.
PNN is a feedforward ANN developed by Specht (1990), in which the response
to an input pattern is processed from one layer to the next, without feedback paths
to previous layers. A typical PNN has four layers: input, pattern, summation and
2. Materials and methods
output. The input units supply the same values to all pattern units. The pattern
units form a dot product between the input pattern vector x and a weight vector wi
2.1. Studied groups (zi = xwi), which is followed by the nonlinear neuron activation function:
w  x0 wi  x
The subjects (n= 45) consisted of 30 healthy controls (20 women) and 15 PD gzi exp  i 2
patients (three women). The healthy subjects, recruiting by advertisements, were
without neurological illness, degenerative conditions or any general disease that This Bayesian function takes into account the relative likelihood of events and
might interfere with gait (Table 1). A screening questionnaire was completed to uses a priori information to improve the prediction (Specht and Romsdahl, 1994).
guarantee that subjects were suitable for the study. The PD subjects were recruited The summation units simply sum the inputs from the pattern units, corresponding
from the Parkinson Disease and Movement Disorder Center of the University of to the category from which the training patterns were selected. Repeating this
Kansas Medical Center. All PD subjects had undergone bilateral DBS-STN and were procedure for each class, the un-normalized density functions gk (x), for k= 1,
stable when the study was conducted. Each subject signed an informed consent 2, y, K were estimated. The Bayesian probability that the case was from class k is
approved by the local Institutional Review Board. as follows:

gi x
Px A k Pk 3
2.2. Experimental protocol and signal processing i1 gi x
The output units have a competitive transfer function that picks the maximum
For each PD subject, quantitative gait measurement was obtained on two of the probabilities and produces 1 for one class (normals) and 0 (PD patients) for
different days. In the rst session, the subject had taken the usual dose of PD the other.
medication and stimulators were turned on. The gait assessment was rst
conducted with both medication and stimulation (mon-son). After turning the
2.5. Support vector machine
stimulators off for 30 min, the measurements were repeated (mon-sof). In the second
session, the subjects were without medication for at least 12 h. Gait analysis was rst
conducted with stimulation (mof-son), and repeated after 30 min without stimula- The SVM estimates a function for classifying data into two classes (Vapnik,
tion (mof-sof). Due to technical problems, some subjects did not complete all tests. 2000). Using a nonlinear transformation ^ that depends on a regularization
Therefore, 13 subjects were evaluated in mof-sof, 12 in mof-son, 14 in mon-sof and 11 parameter C (Begg et al., 2005), the input vectors are placed into a high-
in mon-son conditions. Subjects from the control group were evaluated only once. dimensional feature space, where a linear separation is employed. To construct a
The quantitative analysis for the controls and PD subjects in the mof-sof condition nonlinear support vector classier, the inner product (x,y) is replaced by a kernel
was used to develop the classier models. The other three PD conditions were function K(x,y)
included in the developed models to evaluate the DBS-STN effect in PD treatment. 
Two force platforms (AMTI, USA) were mounted in series at the middle of a f x sgn ai yi Kxi ; x b 4
walkway. All subjects practiced the walking trial ve times before the experiment. i1
The subjects walked barefoot at their self-selected speed and repeated the walking
where f(x) determines the membership of x. In this study, the normal subjects
trial ve times. The GRF from both force platforms were collected for 10 s at a
were labeled as  1 and PD subjects as +1. The SVM has two layers. During the
sampling frequency of 100 Hz, ltered using a low-pass Butterworth lter, with a
learning process, the rst layer selects the basis K(xi,x), i= 1,2, y, N from the given
cut-off frequency of 30 Hz, and normalized by subject body weight.
set of kernels, while the second layer constructs a linear function in the ^ space.
The averaged vertical, anteriorposterior and mediallateral components of
This is equivalent to nding the optimal hyperplane in the corresponding feature
GRF from ve walking trials were interpolated with cubic splines and re-sampled
space. The SVM algorithm can construct a variety of learning machines using
with 101 sample points according to the stance phase duration of each foot. Thus,
different kernel functions.
202 GRF samples were analyzed for each GRF component separately.
Each vertical, anteriorposterior and mediallateral GRF waveform was stored
in a matrix E with 43 rows (# of subjects) and 202 columns (# of GRF samples for 2.6. Variable selection
both right and left limbs). Principal component analysis (PCA) was applied to the
covariance matrices S (202  202) from each E, separately (Jollife, 2002).
The broken stick criterion (Jolliffe, 2002) was used for choosing the signicant
PCs of vertical, anteriorposterior and mediallateral GRF components for the
2.3. Logistic regression analysis. Moreover, to build a more accurate classier model, it was necessary to
evaluate which scores contributed to improvement in the task (Chang, 1983).
LR is a statistical modeling technique that estimates the probability of a In the LR model, a stepwise approach was used to select the input variables by
dichotomous outcome event being related to a set of explanatory variables the Akaike information criterion (AIC), followed by w2 tests to contrast with a full
model including all PC scores selected by the broken stick criterion or with subsets
of variables close to the nal model (Krzanowski, 1998).
PNN requires the selection of the optimal value for the width (s2) of the radial
Table 1 basis function. For optimizing s2 and selecting the relevant input variables, PNN
Subjects characteristics. models were trained and evaluated by the bootstrap method, considering each
possible combination of scores and varying the values s2 in the interval [0.1, 1].
Control group PD patients Briey, bootstrapping generates training sets drawing samples with replacement
from the original data set.
Age (years) 50.17 7.8 56.4 7 8.3 For the SVM, the appropriate kernel function, the number of PC scores to be
Mass (kg) 90.56 7 15.7 50.52 7 8.02 used as input variables and the parameter C were evaluated using the same
Height (m) 1.73 70.08 1.67 7 0.09 bootstrap approach applied to the PNN. The input set was determined considering
Duration of PD (years) 12.27 4.3 N.A. each possible combination of scores. All SVM models were trained over the range
Time since surgery (months) 15.17 9.5 N.A. C= {0.1, 1, 10, 100, 1000} using linear, polynomial and Gaussian kernels (Lai et al.,
722 A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726

Fig. 1. Averaged vertical ground reaction force from controls (bold line) and PD (dotted line): (a) mof-sof, (b) mof-son, (c) mon-sof and (d) mon-son conditions.

Fig. 2. Averaged anteriorposterior ground reaction force from controls (bold line) and PD (dotted line): (a) mof-sof, (b) mof-son, (c) mon-sof and (d) mon-son conditions.

2.7. Performance evaluation indices for each classier represented the average and standard error over all 1000
tests with different bootstrap samples. Comparisons among models were
performed using the one-way ANOVA, with a = 0.05 followed by the post-hoc
The models performances were assessed using the AUC and ACC. With small
Bonferroni test. The agreement between each model pair was compared using the
sample sizes, the application of a resampling technique such as the bootstrap
percent agreement and the Cohens kappa coefcients, using the algorithm
approach is recommended to estimate the performance of the classier (Sahiner et
proposed by Cardillo (2007) and interpreted according to the standard criteria
al., 2008). Both ACC and AUC were obtained using the set of samples not included
(Landis and Koch, 1977).
in the bootstrap sample that were obtained using the 0.632+ bootstrap method
(Efron and Tibshirani, 1997).
The predictive power of a classier to ascertain if a PD patient under treatment
has achieved a normal gait prole was evaluated with the negative likelihood ratio
(NLR) between false and true negatives. This ratio gives the odds that a subject 2.8. Effect of treatments
does not have symptoms of the disease, in this case indicating if the treatment has
a positive effect. For quantifying the effect of the treatments, the PC scores from PD subjects in
To perform a comparison among LR, PNN and SVM models, the same bootstrap the mon-sof, mof-son and mon-son conditions were calculated and used as inputs
samples were used for developing and testing the models. The AUC, ACC and NLR in the developed classiers.
A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726 723

Fig. 3. Averaged mediallateral ground reaction force from controls (bold line) and PD (dotted line): (a) mof-sof, (b) mof-son, (c) mon-sof and (d) mon-son conditions.

Fig. 4. (a) Area under ROC (AUC) and accuracy (ACC) of probabilistic neural network classier; and (b) average and 95% condence bands for AUC and ACC for one thousand
bootstrap training and testing samples as a function of the spread constants for the selected inputs.

3. Results normal pattern (Figs. 13), especially when both medication and
stimulation were used (Figs. 1d, 2d, 3d).
Visual inspection of three averaged GRF components from The Broken Stick test indicated that the rst six PCs from
normal and PD subjects showed the PD pattern moving toward a vertical GRF, the rst PC from anteriorposterior GRF and the rst
724 A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726

PC from mediallateral GRF, which respectively explained 91.1%, The best SVM performance was obtained for the linear kernel
69.1% and 62.1% of the total variation, should be considered in the with C = 1 and four inputs (the rst three and the fth vertical GRF
analysis. PC scores), for both AUC and ACC criteria (Table 2).
The stepwise selection of LR variables identied the rst, The resulting values of AUC, ACC and NLR for the 1000
second and fth PC scores from vertical GRF as the signicant bootstrap samples indicate that all methods presented high
inputs. Contrasting with the LR model including all eight scores performance indexes (Fig. 5). The ANOVA showed a signicant
under a w2 distribution, this model yielded a higher P value difference among these values (Table 3), indicating that PNN
(P40.05), indicating a non-signicant difference between mod- presented better AUC and NLR and the SVM a better ACC.
els, which validates the selected PC scores as input variables in the According to the standard criteria for interpreting Cohens kappa
LR classier. coefcients, the classication agreement was substantial in the
The highest PNN performance occurred with the rst three comparison between PNN and SVM, and just moderate between
and the fth vertical PCs, and the rst anteriorposterior PC LR and PNN and between LR and SVM (Table 4).
(Fig. 4a). The nal model including these ve variables was
thus maintained for comparison among model and treatment
assessment. The AUC indicated higher average performance Table 3
values within the whole range of spread constants, reaching its Average and standard error and P value for AUC, ACC and NLR.
maximum with s2 =0.8, while the ACC has the maximum with
s2 = 0.1 (Fig. 4b). To make a compromise between AUC and ACC, a LR PNN SVM P
0.1 spread was chosen. The AUC ( 7Standard Error) and the ACC at
AUC 0.9813 70.0314 0.9867 7 0.0195 0.9463 7 0.0622 o 0.001n, + ,y
this spread constant were 0.98 ( 70.019) and 0.92 ( 70.058), ACC 0.9185 70.0711 0.9273 7 0.0729 0.94607 0.0556 o 0.001n, + ,y
respectively. NLR 0.1216 70.1761 0.0354 7 0.0942 0.0569 7 0.1172 o 0.001n, + ,y

Post-hoc P o 0.05: n LR different from PNN, + LR different from SVM, y PNN

different from SVM.

Table 2
Table 4
The best SVM bootstrap training and test set area under ROC (AUC), accuracy (ACC)
Comparison between models using the Cohens kappa coefcient.
and the selected scores inputs.

Agree (%) Disagree (%) Cohens kappa

Kernel C AUC ACC Variables

LR and PNN 70 30 0.4537 Moderate agreement

Linear 1 0.9463 0.9460 1, 2, 3 and 5a
LR and SVM 67 33 0.4127 Moderate agreement
Gaussian (s = 1) 10 0.8907 0.9115 1, 4 and 7a
PNN and SVM 86.5 13.5 0.8649 Substantial
Polynomial (d =3) 1 0.8979 0.9291 1, 5 and 7a

Variables 15 refer to the rst to fth PC scores of the vertical GRF, while LRlogistic regression, PNNprobabilistic neural network, and SVM support
variable 7 is the rst PC score of the anteriorposterior horizontal GRF. vector machine

Fig. 5. Box plot of performance indexes of LR, PNN and SVM models obtained with 1000 bootstraps: (a) AUC; (b) ACC; and (c) NLR.
A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726 725

Table 5
Comparison among logistic regression (LR), probabilistic neural network (PNN) and support vector machine (SVM) models.


Normal PD Normal PD Normal PD

Mon-sof 4 (28.6%) 10 (71.4%) 4 (28.6%) 10 (71.4%) 2 (14.3%) 12 (85.7%) 14 (100%)

Mof-son 10 (83.3%) 2 (16.7%) 4 (33.3%) 8 (66.7%) 4 (33.3%) 8 (66.7%) 12 (100%)
Mon-son 10 (90.9%) 1 (9.1%) 7 (63.3%) 4 (36.7%) 7 (63.3%) 4 (36.7%) 11 (100%)

When assessing treatments (Table 5), the classiers ranked studies that compared only the LR and ANN (Nguyen et al., 2002;
treatments in the same order, with DBS-STN presenting better Song et al., 2005).
results than just medication, and further GRF improvements with Following Landis and Koch (1977) method for describing the
combined treatments. strength of agreement associated with the Cohens kappa statistics
(Table 4), the present results indicate that PNN and LR had a
moderate agreement and PNN and SVM presented a substantial
agreement. These ndings are also in agreement with Dhana-
4. Discussion lakshmi et al. (2009), who demonstrated that SVM and ANN can
be effectively used as classication approaches. Additionally,
This is the rst study that used force platforms for assessing the these results are in line with other research (Begg et al., 2005;
effects of the DBS-STN on PD gait. Although most DBS-STN Lai et al., 2009), indicating that SVM had good performance in
assessments show an improvement in gait speed (Liu et al., 2005; pathological gait pattern recognition.
Ferrarin et al., 2005), none of them quantied its effect by analyzing When applied for PD treatment comparison, the three models
the entire waveform. GRF is one important indicator of joint moments ranked results from testing conditions in the same order
and muscle activities (Zajac et al., 2003), and is partially explained by indicating that DBS-STN alone was more effective than medication
subjects velocity or cadence (Winter, 1991). Its improvement with alone in moving PD patients to a normal GRF pattern. The best
treatment as measured by force peaks represents only a preliminary results occurred with the combination of treatments (Table 2).
step in gait analysis (Loslever and Barbier, 1998). Indeed, it does not Similarly, past studies (Ferrarin et al., 2005; Xie et al., 2001) have
allow a clear classication of subjects in different conditions. reported enhancements in gait performance with stimulation
Conversely, PCA coefcients account for information from the entire alone and further improvement when combined with medication.
GRF waveforms, rather than discrete parameters as in most past Those authors suggested a synergistic effect of DBS-STN and
studies (Li and Hamill, 2002; Hsiang and Chang, 2002), which levodopa for PD symptoms. PNN classied fewer PD subjects as
disregard the high correlation among those parameters (Chester et al., normal, presenting reduced classication difference between
2007). Additionally, PCA provided a substantial dimension reduction DBS-STN and medication alone (Table 2), agreeing with past
of classier inputs and simplied the use of the bootstrap technique, studies (Ferrarin et al., 2005; Xie et al., 2001). Moreover, PNN had
which are recommended for small sample sizes (Sahiner et al., 2008). lower NLR or odds of a subject having a normal gait pattern;
Not all PCs selected by the broken stick method are necessarily therefore, being more reliable to classify the PD subjects as
important for classication, since the best trade-off between normal when the GRF pattern is improved.
variance and classication should be taken into account (Chang,
1983; Jolliffe, 2002). Dreiseitl and Ohno-Machado (2002) also
recommend determining which variables are relevant to reach a 5. Conclusion
good model performance. The stepwise and bootstrap approaches
allowed for the selection of the PCs that maximized performance The LR, PNN and SVM models presented high performance
of the three tested models, using an adequate criteria. The indexes for classifying GRF pattern of normal subjects and
comparison using AUC and ACC also allowed for the selection of untreated PD. When using the bootstrap approach, PNN per-
the best spread value in PNN, as recommended by Hernandez- formed better according to AUC and NLR criteria and the SVM
Caraballo et al. (2005), as well as the best kernel and C parameter showed the best ACC.
in SVM. When evaluating the effect of treatments, the three classiers
The bootstrap is recognized as a robust approach for estimat- indicated DBS-STN alone was more effective than medication
ing classier performance under a limited sample size (Sahiner et alone, with the greatest improvement with combined treatments.
al., 2008). According to Sahiner et al. (2008), the 0.632+ bootstrap However, PNN was more restrictive for accepting the patients GRF
approach provides the lowest bias with the greatest accuracy. In as normal.
the present study, with 1000 bootstrap samples, the PNN
presented the highest AUC and the lowest NLR values compared
to LR and SVM, while SVM had the highest ACC value (Table 3). Conict of interest statement
AUC has been recommended as better performance index, since it
presents higher convergence than ACC (Brandley, 1997), and None of the authors have any nancial or personal relationship
represents the average sensitivity across all possible specicities that could inappropriately inuence the work submitted for
(Glas et al., 2003). On the other hand, ACC is still the most publication.
commonly used index, although dependent on the prevalence of
the disorder in the studied group (Glas et al., 2003). The NLR
describes the discriminatory properties of negative test results. Acknowledgements
According to Deeks (2009), the NLR values correspond to strong
diagnostic evidence. Thus, PNN was more rigorous than LR and This work was partially supported by the Brazilian Research
SVM in accepting a pattern as normal, which agreed with other Council (CNPq) and by CAPES and FAPERJ foundation.
726 A.M.S. Muniz et al. / Journal of Biomechanics 43 (2010) 720726

References Lai, D.T., Levinger, P., Begg, R.K., Gilleard, W.L., Palaniswami, M., 2009. Automatic
recognition of gait patterns exhibiting patellofemoral pain syndrome using a
support vector machine approach. IEEE Transactions on Information Technol-
Brandley, A.P., 1997. The use of the area under the roc curve in the evaluation of ogy in Biomedicine 13 (4), 810817.
machine learning algorithms. Pattern Recognition 30 (7), 11451159. Landis, J.R., Koch, G.G., 1977. The measurement of observer agreement for
Begg, R.K., Palaniswami, M., Owen, B., 2005. Support vector machines for categorical data. Biometrics 33 (1), 159174.
automated gait classication. IEEE Transactions on Biomedica Engineering 53 Li, L, Hamill, J., 2002. Characteristics of the vertical ground reaction force
(5), 828838. component prior to gait transition. Research Quarterly for Exercise and Sport
Cardillo G., 2007. Cohens kappa: compute the Cohens kappa ratio on a 2x2 matrix. 73 (3), 229237. Liu, W., Mcintire, K., Kim, S.H., Zhang, J., Dascalos, S., Lyons, K.E., Pahwa, R., 2005.
Chang, W.C., 1983. On using principal components before separating a mixture of Quantitative assessments of the effect of bilateral subthalamic stimulation on
two multivariate normal distribuitions. Applied Statistics 32 (3), 267275. multiple aspects of sensoriomotor function for patients with Parkinsons
Chester, W.L., Tingley, M., Biden, E.N., 2007. An extended index to quantify disease. Parkinsonism and Related Disordens 11, 503508.
normality of gait in children. Gait and Posture 25, 549554. Loslever, P., Barbier, F., 1998. Multivariate graphical presentation for gait
Dhanalakshmi, P., Palanivel, S., Ramalingam, V., 2009. Classication of audio rehabilitation study. Gait and Posture 7, 3944.
signals using SVM and RBFNN. Expert Systems with Applications 36, 6069 Nguyen, T., Malley, R., Inkelis, S., Kuppermann, N., 2002. Comparison of prediction
6075. models for adverse outcome in pediatric meningococcal disease using articial
Deeks, J.J., 2009. Systematic reviews in health care Systematic reviews of neural netweork and logistic regression analyses. Journal of Clinical Epide-
evaluations of diagnostic and screening tests. British Medical Journal 323, miology 55, 687695.
157162. Ostergaard, K., Sundae, N.A., 2006. Evolution of Parkinsons disease during 4 years
Dreiseitl, S., Ohno-Machado, L., 2002. Logistic regression and articial neural of bilateral deep brain stimulation of the subthalamic nucleus. Movement
network classication models: a methodology review. Journal of Biomedical Disorders 21 (5), 624631.
Informatics 35, 352359. Sahiner, B., Chan, H., Hadjiiski, L., 2008. Classier performance prection for
Efron, B., Tibshirani, R., 1997. Improvements on cross-validation: The .632 + computer-aided diagnosis using a limited dataset. Medical Physics 35 (4),
bootstrap method. Journal of the American Statistical Association 92 (438), 15591570.
548560. Schumacher, M., Robner, R., Wemer, V., 1996. Neural networks and logistic
Ferrarin, M., Rizzone, M., Bergamasco, B., Lanotte, M., Recalcati, M., Pedotti, A., regression. Part I Computational Statistics and Data Analysis 21, 661682.
Lopiano, L., 2005. Effects of bilateral subthalamic stimulation on gait Schutte, L.M., Narayanan, U., Stout, J.L., Selber, P., Gage, J.R., 2000. An index for
kinematics and kinetics in Parkinsons disease. Experimental Brain Research quantifying deviations from normal gait. Gait and Posture 11, 2531.
160, 517527. Schwartz, M.H., Rozumalski, A., 2008. The gait deviation index: A new
Glas, A.S., Lijmer, J.G., Prins, M.H., Bonsel, G.J., Bossuyt, P.M.M., 2003. The diagnostic comprehensive index of gait pathology. Gait and Posture 28, 351357.
odds ratio: a single indicator of test performance. Journal of Clinical Song, J.H., Venkatesh, S.S., Conant, E.A., Arger, P.H., Sehgal, Chandra M., 2005.
Epidemiology 56, 11291135. Comparative analysis of logistic regression and articial neural network
Hahn, M.E., Farley, A.M., Lin, V., Chou, L., 2005. Neural network estimation of for computer-aided diagnosis of breast masses. Academic Radiology 12,
balance control during locomotion. Journal of Biomechanics 38, 717724. 487495.
Hernandez-Caraballo, E.A., Rivas, F., Perez, A.G., Marco-Parra, M., 2005. Evaluation Specht, D.F., 1990. Probabilistic neural networks. Neural Networks 3, 109118.
of chemometric techniques and articial neural networks for cancer screening Specht, D., Romsdahl, H., 1994. Experience with adaptive probabilistic neural
using Cu, Fe, Se and Zn concentrations in blood serum. Analytica Chimica Acta networks and adaptive general regression neural networks. Neural Networks
533, 161168. 12031208.
Hsiang, S, Chang, C., 2002. The effect of gait speed and load carrying on the Su, F., Wu, W., 2000. Design and testing of a genetic algorithm neural network in
reliability of ground reaction forces. Safety Science 40, 639657. the assessment of gait patterns. Medical Engineering and Physics 22, 6774.
Jolliffe, I.T., 2002. In: Principal Component Analysis second ed. Springer, New York. Vapnik, VN., 2000. In: The Nature of Statistical Learning Theory second ed.
Krack, P., Batir, A., Van Blercom, N., Chabardes, S., Fraix, V., Ardouin, A., et al., 2003. Springer, New York.
Five-year follow-up of bilateral stimulation of the subthalamic nucleus in Winter, D.A., 1991. In: The Biomechanics and Motor Control of Human Gait:
advanced parkinsons disease. The New England Journal of Medicine 340 (20), Normal, Elderly and Pathological second ed. University of Waterloo Press,
19251934. Waterloo.
Krzanowski, W.J., 1998. In: An Introduction to Statistical Modelling. Arnold, Xie, J., Krack, P., Benabid, A., Pollak, P., 2001. Effect of bilateral subthalamic nucleus
London. stimulation on parkinsonian gait. Journal of Neurology 248, 10681072.
Lafuente, R., Belda, J.M., Sanchez-Lacuesta, J., Soler, C., Prat, J., 1997. Design and test Zajac, F.E., Sachin, R.R., Kautz, S.A., 2003. Biomechanics and muscle coordination of
of neural networks and statistical classiers in computer-aided movement human walking part II: lessons from dynamical simulations and clinical
analysis: a case study on gait analysis. Clinical Biomechanics 13 (3), 216229. implications. Gait and Posture 17, 117.