Downloaded from https://iranpaper.
ir
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 54, NO. 7, JULY 2007
A Feature Selection Method for Multilevel Mental
Fatigue EEG Classification
Kai-Quan Shen, Chong-Jin Ong, Xiao-Ping Li*, Zheng Hui, and Einar P. V. Wilder-Smith
Abstract—Two feature selection approaches for multilevel
mental fatigue electroencephalogram (EEG) classification are
presented in this paper, in which random forest (RF) is com-
bined with the heuristic initial feature ranking scheme (INIT)
or with the recursive feature elimination scheme (RFE). In a
“leave-one-proband-out” evaluation strategy, both feature selec-
tion approaches are evaluated on the recorded mental fatigue
EEG time series data from 12 subjects (each for a 25-h duration)
after initial feature extractions. The latter of the two approaches
performs better both in classification performance and more
importantly in feature reduction. RF with RFE achieved its lowest
test error rate of 12.3% using 24 top-ranked features, whereas
RF with INIT reached its lowest test error rate of 15.1% using
64 top-ranked features, compared to a test error rate of 22.1%
using all 304 features. The results also show that 17 key features
(out of 24 top-ranked features) are consistent between the subjects
using RF with RFE, which is superior to the set of 64 features as
determined by RF with INIT.
Index Terms—Electroencephalogram (EEG), feature selection,
mental fatigue, random forests.
I. INTRODUCTION
M ENTAL fatigue, defined as a state of reduced mental
alertness that impairs performance [1], [2], has become
one of the most significant causes of accidents throughout the
modern society [3]–[6]. Electroencephalogram (EEG) might
be the most predictive and reliable physiological indicator
of mental fatigue [7]–[9]. In recent years, there has been an
increasing interest in the EEG-based mental fatigue tracking
technology [10]–[13], with the widespread hope that it will be
invaluable in the prevention of mental fatigue related accidents.
A major obstacle for this “most wanted” technology [14] is the
challenge of selecting key EEG features from a large number
of available EEG features.
Feature selection is important for improving generalization,
meeting system specifications or constraints (such as run-
ning time and storage) and enhancing system interpretability
[15]–[17]. However, an exhaustive search for the optimal
feature subset from all possible feature combinations is com-
putationally expensive. To overcome this, many methods for
feature selection have been proposed and a good review of
Manuscript received March 7, 2006; revised October 29, 2006. Asterisk indi-
cates corresponding author.
K.-Q. Shen, C.-J. Ong, and Z. Hui are with the Department of Mechanical
Engineering, National University of Singapore 117576, Singapore (e-mail:
shen@nus.edu.sg, mpeongcj@nus.edu.sg, zhenghui@nus.edu.sg).
*X.-P. Li is with the Department of Mechanical Engineering & Division of
Bioengineering, National University of Singapore, Singapore 117576, Singa-
pore (e-mail: mpelixp@nus.edu.sg).
E. P. V. Wilder-Smith is with the Department of Medicine, National Univer-
sity of Singapore, Singapore 119074, Singapore (e-mail: mdcwse@nus.edu.sg).
Digital Object Identifier 10.1109/TBME.2007.890733
0018-9294/$25.00 © 2007 IEEE
Authorized licensed use limited to: Carleton University. Downloaded on June 21,2023 at 11:00:48 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
1231
these can be found in the paper by Guyon and Elisseeff [18].
It is generally recognized that feature selection methods are
classified into two categories: filter-based and wrapper-based
approaches with the latter being superior in performance but
normally carrying a heavier computational load [19].
Most of the feature selection methods in the literature have
been originally designed for selecting features for two-class
classification [18]. As a result, past attempts at feature selection
in biomedical applications have extensively followed the idea
in which the multiclass classification problem is decomposed
into several two-class classification problems and the feature
selection is carried out for each independently [18], [20]–[23].
However, it may be advantageous to treat the multiclass clas-
sification problem directly rather than decomposing it into
combinatorial two-class classification problems, where the fea-
ture selection has to be carried out in the context of multiclass
classification [18].
This paper investigates the use of RF [24], [25] to select the
key EEG features from a broad pool of available features, for the
purpose of multilevel mental fatigue EEG classification. Unlike
some other methods, the feature ranking criterion of RF pro-
vides a direct relationship between the ranking criterion and the
feature’s influence on the performance of the multiclass classi-
fication. This feature ranking criterion of RF is combined with
RFE approach [21] or a heuristic INIT approach to yield an
overall feature selection schemes. The performances of both
schemes are compared by using EEG data recorded from 12 sub-
jects during 25-h mental fatigue experiments.
This paper is organized as follows. EEG experimental setup
and data preparation are described in Section II. The feature se-
lection method using RF is detailed in Section III. The numerical
experiments are reported in Section IV, followed by the discus-
sion in Section V and conclusions in Section VI.
II. EXPERIMENTAL SETUP AND DATA PREPARATION
A. Subjects
In total 12 right-handed healthy young adults (age range:
19–25 years old; years of education: 11–15 years) were re-
cruited from the local tertiary institutions. These subjects were
selected from volunteers who fulfilled the inclusion criteria
of not being on any medication, no history of severe disease
and no history of a sleep disorder with regular sleep hygiene
verified by a one-week sleep diary documenting nightly sleep
periods of no less than 8 h. Each person was asked to document
that over a period of one week, they did not go to bed later
than 1 am, and did not wake later than 9 am. Additionally, the
presence of habitual daytime napping resulted in exclusion.
Informed consent was obtained and the protocol was approved
by the local ethics committee.
 Downloaded from https://iranpaper.ir
1232
B. Auditory Vigilance Task (AVT)
The details of AVT test have been reported previously [26],
which was designed to catch the cortical deactivation of the
functional lobes of the brain which is believed to be the physi-
ological basis of the mental fatigue [27]. Four audio commands
(left, right, up, down) with 500 ms duration each were randomly
ordered in one command set. Each test session had 50 com-
mand sets in about 3 min (1.5 s interset interval). Subjects were
required to constantly concentrate and press the corresponding
prespecified buttons within 1.5 s once they heard each complete
command set. For every AVT session, the AVT score was cal-
culated in terms of percentage of correct responses.
C. Experimental Protocol
Subjects were monitored in a temperature-controlled labora-
tory (23 –25 from 8:30 a.m. until 9:30 a.m. the next day.
This 25-h design ensures all phases of circadian-induced mental
fatigue were sampled. Caffeine, tea, smoking were prohibited
for 24 h before and throughout the experiment. Subjects were
required to perform one AVT session (with eyes closed) once
an hour throughout the experiment. They were allowed to en-
gage in nonstrenuous activities in the period when they were
not required to perform AVT session.
D. EEG Acquisition
Multichannel EEG data were recorded simultaneously during
every AVT session according to the International 10–20 system
[28] (A1 or A2 as the reference channels for the right or left
brain, respectively), with sampling rate of 167 Hz (or 6 ms sam-
pling interval) and a 0.02–35 Hz pass band using a customized
bandpass filter implemented in LabView (version 6.1, National
Instruments, USA).
E. EEG Data Labeling
The EEG data were labeled to 5-level mental fatigue ac-
cording to AVT performance score. Specifically, for every
subject, the individual performance span (the highest AVT
score to the lowest AVT score) was evenly divided into five
segments corresponding to fatigue level 1 to 5, respectively.
The label (i.e., mental fatigue in 5 levels) of the EEG data
recorded in an AVT session was determined by which segment
the corresponding AVT performance score fell into.
F. Feature Extraction
In the present study, EEG feature extraction can be carried
out in both online and offline manner for the purposes of offline
feature selection and online multilevel mental fatigue classifica-
tion. EEG data were passing through the feature extraction win-
dows, with one window for each EEG channel. The length of
each feature extraction window was 2 s (or 334 EEG samples),
with half second (or 84 EEG samples) time interval between
two adjacent calculations of feature extraction. The power spec-
tral density for the EEG data segment falling into the fea-
ture extraction window was calculated by fast Fourier transform
with Hann window [30]. Four features were chosen to charac-
terize the power spectral density . Their definitions are as
follows.
Dominant Frequency (DF): Among all the peaks in a spec-
trum segment corresponding to a frequency band, the peak
Authorized licensed use limited to: Carleton University. Downloaded on June 21,2023 at 11:00:48 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 54, NO. 7, JULY 2007
with the largest average power in its full width half max-
imum band [31] is defined as the dominant peak. The corre-
sponding frequency of this dominant peak is the dominant
frequency.
Average Power of Dominant Peak (APDP): This is defined
as the average power on the full width half maximum band
of the dominant peak.
Center of Gravity Frequency (CGF): It is defined as
(1)
where is frequency and is the estimated power
spectral density.
Frequency Variability (FV): It is defined as
(2)
As a result, in total 304 quantitative EEG features (4 kinds of
features 19 channels 4 frequency bands, i.e., standard , ,
, frequency bands [29]) were extracted. Combining with the
EEG labels as described in Section II-E, the feature selection
results in a data subset for each subject. Specifically, for the th
subject, a data subset is formed in terms of ,
where is a 304-dimentional feature vector for the th sample
and the corresponding mental fatigue level. Small portion of
the EEG data (less than 20 s EEG data) in the beginning of each
recording was discarded to ensure the well-balanced samples
for all the five classes.
III. FEATURE SELECTION USING RF
Random forest [24], [25] is an ensemble of many random-
ized decision-trees so that its output is the plurality vote of
all these decision-trees. Each decision-tree is randomized by
both bootstrapping (a statistical resampling technique) and
random variable selection. Specifically, each decision-tree
of RF is constructed using a bootstrapped set (only about
two-third of the original training data). The other one-third is
left out as out-of-bag (OOB) cases. During the tree growing, a
random selection of a subset of features (with a fixed size of
, , where denotes the dimensionality of feature
vector) is used for the determination of the split at each node.
The randomization by both bootstrapping and random variable
selection assures low correlation of the individual trees. Mean-
while, each decision-tree is grown to maximum depth to keep
the bias low (but with high variance). The ensemble of these
weak decision-trees (with low correlation, low bias but high
variance) yields a strong classifier, i.e., RF (with both low bias
and low variance).
Random forest is, a rare feature for nonlinear classifiers, very
benevolent to train. Even without extensive tuning of the few
RF hyper-parameters ( and another hyper-parameter ,
i.e., the number of decision-trees to grow), it usually yields
good classification results comparable to or better than other
 Downloaded from https://iranpaper.ir
SHEN et al.: FEATURE SELECTION METHOD FOR MULTILEVEL MENTAL FATIGUE EEG CLASSIFICATION
TABLE I
PROCEDURE OF RF-INIT FEATURE SELECTION SCHEME
state-of-the-art classifiers on many data sets [25], [32], [33]. Be-
sides its superior classification performance, another feature of
RF is that it can give an importance measure for each feature, ac-
cording to that feature’s contribution to the overall classification
performance. The contribution of a feature is calculated by com-
paring the performance of the predictors (decision-trees) on the
original OOB cases and that obtained from the permuted OOB
cases with the content of that feature having been masked off
(through random permuting the values of that feature). In more
detail, let or - denote the count of the number
of votes cast for the correct class by th tree for all the OOB
cases before and after permuting the values of the th feature,
respectively. The raw importance score for the th feature is
the average decrease in the number of votes for the correct class
due to permuting the th feature, which is defined as
- this new ranking is stored as the one but last position in , and
(3)
As suggested by Breiman [32], raw importance score of (3) is,
however, too unstable to be used as the feature ranking criterion.
The correlation of the decreases in the number of votes for the
correct class due to permuting the th feature, from tree to tree,
has been shown to be quite low. The standard error of these
decreases can be computed in the classical way and the standard
error of the corresponding raw importance score is . removes one feature from the data set at a time. Obviously, more
A more robust feature ranking criterion can be defined by the
-score of the raw importance score, which is computed by di-
viding the raw importance score by its standard error assuming
normality as follows:
(4)
The remainder of this section presents two approaches using
this criterion in an overall scheme for feature selection: the
RF-based initial feature ranking (RF-INIT) approach and the
RF-based recursive feature elimination (RF-RFE) approach.
Both approaches assume that the feature ranking criteria ’s
(for each feature) are available from RF solution.
Let denote the dimensionality of feature vector (
for the present case). The RF-INIT approach has as its inputs
data set and the index set . The output
of RF-INIT is a ranked list of the features in the form of an index
set with and . culated for each of them in a sliding window (sampling at 0.5 s
The steps involved in this approach are summarized in Table I.
The RF-RFE approach is similar to the one given by Guyon
[21] but with the of (4) used as the ranking criterion. The
steps involved in this approach are summarized in Table II. The
inputs are the same as RF-INIT, with the output being the ranked
list of features . One starts with full feature set . RF-INIT
is called, which builds a random forest and gets a ranked list
Authorized licensed use limited to: Carleton University. Downloaded on June 21,2023 at 11:00:48 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
1233
TABLE II
PROCEDURE OF RF-RFE FEATURE SELECTION SCHEME
of the features containing all elements in . Next, the last
element of (with the smallest ) is removed from , and
stored in the last position of , meaning it is the least important
feature. RF-INIT is then called again on the reduced . As it
now builds a random forest using a reduced set of features, it
will give a different random forest and, therefore, a possibly
different ranking of the features still in . The last feature in
removed from . This process continues, each time removing
the least important feature from and storing it in the rightmost
free position of , until is empty.
RF-INIT is in fact the existing RF feature selection method
proposed by Breiman [25], while RF-RFE is a combination of
RF and a backward feature elimination scheme motivated by
SVM-RFE algorithm [21]. The implementation of RF-RFE only
requires some slight modifications from the standard random
forest package [34]. Steps 3 and 4 of RF-RFE ( , ) in Table II
than one feature can be removed at one time with slight mod-
ifications to steps 3 and 4 in Table II. The current description
of RF-RFE does not involve the determination of parameters
and for each of the inner loop through a model selec-
tion process. Such a process is possible albeit with higher com-
putational costs.
IV. NUMERICAL EXPERIMENTS
A. Methods
Both RF-INIT and RF-RFE feature selection approaches are
applied on the multilevel mental fatigue EEG data obtained in
Section II. These two feature selection approaches are compared
quantitatively by the predictive performance in the classification
of multilevel mental fatigue.
The EEG data were recorded in 19 different channels in 4
frequency bands (Section II). Four kinds of features were cal-
intervals, with a 2 s sliding window) along the time series, re-
sulting in a vector of features ( features), at
a total of samples. The data are available in
subject-wise data subsets for (12 subjects).
The down-sampling scheme using sliding window with overlap-
ping makes the features of adjacent samples highly correlated.
This violates, together with the high correlation which is natural
 Downloaded from https://iranpaper.ir
1234
to most time series data such as EEG, the assumption of an inde-
pendent and identical distribution of the data. As a consequence,
a random sampling of a testing set will not be appropriate to test
the predictive performance on this kind of data.
To estimate the predictive performance, a block resampling
scheme “leave-one-proband-out” [35] is used, where each sub-
ject corresponds to one proband. Samples from 11 subjects are
used to form a training set , and the samples from the
left-out subject are used to form a testing set . Practically,
this resampling procedure results in 12 pairs of and
in total. In the numerical experiments, for each pair of and
, is used by feature selection approaches RF-INIT
(Table I) and RF-RFE (Table II), producing a ranked feature
list and , respectively. For comparison purposes, random
forest is iteratively fit on , at each iteration building a new
random forest after discarding the least important features ac-
cording to and , respectively. To estimate the predictive
performance of the selected features, the test errors are obtained
by putting (with the least important features discarded cor-
respondingly) down the random forest and calculating the per-
centage of misclassifications on . In order to save the com-
putational time, a three-tier feature removal scheme was used, in
which 20 features were removed at each recursion until 44 fea-
tures left, and then 5 features were removed at each recursion
until 24 features were left, followed by two features removed at
each recursion.
The most important hyper-parameter of random forest to
tune, i.e., the number of input variables to split on at each
node, has been shown not sensitive over a wide range of values
and the default value is often a good choice [36]. In addition,
as earlier mentioned, the re-tuning of the hyper-parameters is
computationally very expensive in the inner loops for RF-RFE.
Therefore, is set to be the integer nearest to the square root
of the feature vector dimensionality, i.e.,
for RF-INIT and a decreasing for RF-RFE.
Another hyper-parameter is set to 1000 where the OOB
error rates already converge with the setting of above-men-
tioned hyper-parameter .
B. Results
Fig. 1 shows the mean test error rates of RF on unseen testing
sets ’s with the decreasing number of top-ranked features,
where the top-ranked features were selected by either RF-INIT
or RF-RFE approach. The mean test error rates are the average
values over 12 test error rates corresponding to 12 pairs of
and . Error bars of standard deviation have not been
plotted for the sake of clarity. However, the results show that
for RF-RFE approach the standard deviation is rather stable
with regards to the number of the top-ranked features used for
classification (about 5%), whereas for the RF-INIT approach it
varies between 5% and 7%.
Fig. 1 shows that the performances of RF-RFE and RF-INIT
are similar in terms of the trends of mean test error rate in
variation against the number top-ranked features, but RF-RFE
performs better. The difference in performance is confirmed
by a one-tailed paired -test [37] using a significance level of
. The curve for RF-RFE in Fig. 1 shows that the mean
test error rate varying against the number of top-ranked features
Authorized licensed use limited to: Carleton University. Downloaded on June 21,2023 at 11:00:48 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 54, NO. 7, JULY 2007
Fig. 1. Mean test error rate against the number of top-ranked features where
the top-ranked features are selected by either RF-RFE (solid curve) or RF-INIT
(dashed curve). The test error rates are obtained by averaging 12 test error rates
on all resampled subsets D ’s.
TABLE III
MEAN CONFUSION MATRIX FOR THE OPTIMAL CLASSIFICATION (USING 24
KEY FEATURES CHOSEN BY RF-RFE FOR EACH PAIR OF RESAMPLED D
AND ) D
has two stages. In the stage 1, the mean test error rate monotoni-
cally decreased, as the number of top-ranked features decreased
from 304 to 24. The lowest mean test error rate (approximately
12.3%) was obtained using 24 top-ranked features, compared
to the mean test error rate of 22.1% with 304 features without
feature selection. In stage 2, however, as the number of features
was further reduced from 24, the mean test error rate increased.
The transition of mean test error rate in variation against number
of top-ranked features from stage 1 to stage 2 is a clear indica-
tion of the effectiveness of RF-RFE feature selection approach.
The curve for RF-INIT in Fig. 1 shows a similar trend, however,
the result is not as appealing. Compared to RF-RFE, RF-INIT
generally gave a higher mean test error rate for using the same
number of top-ranked features. The lowest mean test error rate
of 15.1% was obtained when using 64 top-ranked features, com-
pared to 12.3% using 24 top-ranked features for RF-RFE. It
should also be noted that there were some fluctuations with the
mean test error rate in the stage 1 for RF-INIT, in contrast to the
clear monotonic decrease in the case of RF-RFE.
Equal costs for any misclassification are assumed in the
present study. Table III shows the mean confusion matrix for
the optimal classification (using 24 key features chosen by
RF-RFE), by averaging 12 confusion matrices for 12 ’s.
It shows that gross errors (such as mental fatigue level 1 being
misclassified to mental fatigue level 5) did not occur heavily.
The feature rankings and obtained by RF-INIT and
RF-RFE are significantly different, which can be easily shown
by a scatter plot (not shown). RF-RFE achieves optimal classi-
fication at 24 features, which is superior to the set of 64 features
(less variability, easier to interpret) as determined by RF-INIT.
 Downloaded from https://iranpaper.ir
SHEN et al.: FEATURE SELECTION METHOD FOR MULTILEVEL MENTAL FATIGUE EEG CLASSIFICATION
Fig. 2. Distribution of the 17 features which are important in all of the re-
sampled subsets at optimal classification (selected by RF-RFE). The number
in bracket following the channel name is the number of key features deriving
from that channel.
TABLE IV
LIST OF THE 17 FEATURES WHICH ARE IMPORTANT IN ALL OF THE RESAMPLED
SUBSETS AT OPTIMAL CLASSIFICATION (SELECTED BY RF-RFE)
It may be useful to examine the features which are within the
24 top-ranked features in all of the resampled subset ’s by
RF-RFE. There are 17 such features in total (shown in Fig. 2
and Table IV). The neurophysiological interpretation of this
common feature set is discussed in the next.
C. Neurophysiological Interpretation of Key Features
Benefiting from the increased system interpretability as a re-
sult of feature selection by RF-RFE, the identified key features
also give some important information on the critical EEG chan-
nels and the critical EEG features for discriminating mental fa-
tigue in 5 levels. Fig. 2 indicates that the frontal and occipital
regions of the brain were important for classification of mental
fatigue at different levels, which is consistent with the neuro-
physiologic basis of the mental fatigue [27]. However, there was
no key feature derived from channel Fp1 and Fp2, which is the
region of the prefrontal lobe known to be involved in the com-
plex effect induced by mental fatigue [38]. This could be ex-
plained by the existence of the unavoidable EOG and EMG arti-
Authorized licensed use limited to: Carleton University. Downloaded on June 21,2023 at 11:00:48 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
1235
facts, which are generated by eye movements and facial muscle
movements. They typically have larger amplitudes compared
to normal EEG. The locations of Fp1 and Fp2 are close to the
sources of EOG and EMG. Thus, channel Fp1 and Fp2 failed in
contributing to the classification of mental fatigue at different
levels.
In addition, as shown in Table IV, an interesting finding is
implied, in which the four standard frequency bands (i.e., , , ,
frequency bands) are all important to adequately capturing the
subtle changes in EEG in relationship to mental fatigue, whereas
only some of the four frequency bands were mentioned to be
relevant in the literature [6], [27], [39]. APDP seemed to be an
important feature from Table IV, which means that the power
over the standard frequency bands (i.e., , , , frequency
bands) in EEG is an important measure to mental fatigue.
V. DISCUSSION
Two different random forest feature selection approaches
(i.e., RF-INIT and RF-RFE) are evaluated quantitatively in this
paper for the classification of multilevel mental fatigue EEG.
In the literature, only few applications of RF feature selection
were reported [36], [40]. Most of them are related to gene
selection using microarray data sets, where the ill-posedness
of the classification tasks demands for a strong dimension
reduction. Other examples of a successful application of the
random forest feature selection remain rare.
In the evaluation of generalization performance, a block
resampling scheme “leave-one-proband-out” [35] is used.
The benefits of “leave-one-proband-out” resampling scheme
are manifold: 1) It is assured that the temporal overlapping
neighbours and of a test sample are not part of
the training set. 2) Multiple testing sets are obtained to evaluate
the error and a confidence can be assessed. This confidence
is always obligatory when discussing the significance in the
comparison between the performances of two proposed feature
selection approaches. 3) Each testing set is an independent
data set of a left-out proband. One considers RF being trained
on a leave-one-proband-out subset. It is natural to simulate its
generalization performance on other unseen subjects by the test
error on the corresponding testing set of a left-out proband, and
to seek for key features which are repeatedly selected in all of
the resampled subsets.
The better performance of RF-RFE over RF-INIT (Fig. 1)
is most interesting and deserves attention. RF-INIT is in fact
the existing RF feature selection method proposed by Breiman
[25], while RF-RFE is a feature selection method motivated by
SVM-RFE feature selection method [21] by inserting RF in-
stead of SVM into the backward feature selection procedure.
It is also worthy to acknowledge that RF-INIT approach is vir-
tually the same as the gene selection method proposed by Diaz-
Uriarte and Andres [36], whereas RF-RFE is similar to a “gene-
shaving” approach proposed by Jiang et al. [40]. In their papers,
good performances of variable selection using random forests
for microarray data were reported. However, they used raw im-
portance score as in (3) rather than a more reliable Z-score as
in (4) as the feature ranking criterion. More importantly, their
methods for the evaluation of test error are suitable for mi-
croarray data but not appropriate for a time series data such as
 Downloaded from https://iranpaper.ir
1236
EEG data. The following paragraphs highlights the reasons why
the proposed approach RF-RFE, together with the “leave-one-
proband-out” resampling scheme, can be well suited for anal-
ysis of neuronal (and other biomedical) data.
Firstly, RF-RFE is naturally a feature selection method for
both two-class and multiclass classification, in contrast to most
of existing methods (such as SVM-RFE) which are designed
for binary classification only. Many biomedical applications re-
quire feature selection for multiclass classification. One may
argue that a multiclass classification problem can be always de-
composed into combinatorial binary classification problems and
feature selection can be carried out for each binary classification
independently. However, it remains an open issue on the mixing
strategy which allows to combine multiple ranked feature lists
resulting from feature selection for each binary classification
problem. Meanwhile, it may be advantageous to carry out the
feature selection directly in the context of multiclass classifica-
tion because the larger the number of the classes, the less likely a
“random” set of features can provide a good classification [18].
Second, RF-RFE inherits every appealing characteristic of
random forest. They usually yield good classification results,
comparable to or better than other state-of-the-art classifiers on
a number of data sets [33]. In addition, they are very benevolent
to train, with a little need to fine-tune the few hyper-parame-
ters. Random forest can also be used in extremely ill-posed clas-
sification problems when there are many more variables than
observations.
Thirdly, RF-RFE runs efficiently on large data sets (e.g., the
mental fatigue EEG data set used in the present study). The
little need to fine-tune the hyper-parameters may account for the
computational advantage of RF-RFE. In addition, more than one
feature can be removed at one time in RFE procedure, which
makes computation cost more affordable. Besides the major
computation cost of re-growing random forest at each iteration,
re-calculation of feature importance measure only slightly in-
creases the computation cost. The feature importance measure
of RF is calculated through internal out-of-bag estimates [25].
Its computation cost is running of the OOB and permuted OOB
cases on each decision-tree which is very fast.
The present study also gives some insights into the consistence
of the random forest importance over multiple resampled sub-
sets, which is an issue that still seems largely ignored in the liter-
ature. In the present study, there are 17 features (Table IV) which
are within the 24 top-ranked features in all of the resampled sub-
sets (i.e., ’s) using RF-RFE feature selection approach. It is a
bit disappointing with regards to the consistence of feature selec-
tion. The slight inconsistence could be due to individual depen-
dence of mental fatigue EEG from subject to subject, since the
samples in the present study were divided subject-wise according
to the “leave-one-proband-out” resampling scheme. Therefore, it
is natural to seek for features which are repeatedly selected over
multiple resampled subsets. This concern is taken into account
in the present study in the attempt of relating key features with
the biological basis of mental fatigue. The consistence between
those common key features over multiple resampled subsets
with the anatomical regions known to be involved in mental fa-
tigue is also a strong indication of the good performance of the
proposed RF-RFE feature selection approach.
Authorized licensed use limited to: Carleton University. Downloaded on June 21,2023 at 11:00:48 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 54, NO. 7, JULY 2007
VI. CONCLUSIONS
A method which allows to identify relevant features for the
classification of multilevel mental fatigue EEG has been pro-
posed. This is done by transforming this search into a multi-
class classification experiment and to use a feature selection
strategy. Two different feature selection approaches are evalu-
ated, both based on random forest and its feature importance
measures. The first is in fact a filter-based approach proposed
by Breiman, i.e., random forest with the heuristic initial feature
ranking scheme; the other, following Guyon’s machine learning
advice, is a wrapper-based approach using an iterative feature
elimination scheme. In the “leave-one-proband-out” evaluation
strategy, both feature selection approaches are tested on the EEG
time series data after initial feature extractions. The latter of the
two approaches performs better, both in classification perfor-
mance and more importantly in feature reduction. Results show
that 17 (out of 304) features are consistently important between
subjects, which is superior to the set of 64 features (less vari-
ability, easier to interpret) as determined by Breiman’s standard
filter-based approach. From these 17 key features, it was found
that the electrode locations in the frontal and occipital regions of
the brain are most important to multilevel mental fatigue classi-
fication, which is consistent with the anatomical regions known
to be involved in mental fatigue. These key features also show
that the four standard frequency bands ( , , , ) are all impor-
tant to the multilevel mental fatigue classification.
ACKNOWLEDGMENT
The authors are grateful to L. Breiman and A. Cutler who
made their random forest code available through the Internet.
The implementation for the present study grew from that Fortran
package. The authors would also like to thank the reviewers for
their valuable comments and suggestions.
REFERENCES
[1] E. Grandjean, Fitting the Task to the Man. London, U.K.: Taylor &
Francis, 1981.
[2] D. J. Mascord and R. A. Heath, “Behavioral and physiological indices
of fatigue in a visual tracking task,” J. Safety Res., vol. 23, pp. 19–25,
1992.
[3] K. Idogawa, “On the brain wave activity of professional drivers during
monotonous work,” Behaviormetrika, vol. 30, pp. 23–34, 1991.
[4] D. F. Dinges, “An overview of sleepiness and accidents,” J. Sleep Res.,
vol. 4, no. 2, pp. 4–14, 1995.
[5] J. C. Stutts, J. W. Wilkins, and B. V. Vaughn, Why do People Have
Drowsy Driving Crashes? Input From Drivers Who Just did 1999, AAA
Foundation for Traffic Safety.
[6] S. K. L. Lal and A. Craig, “A critical review of the psychophysiology
of driver fatigue,” Biological Psychology, vol. 55, pp. 173–94, 2001.
[7] O. G. Okogbaa, R. L. Shell, and D. Filipusic, “On the investigation of
the neurophysiological correlates of knowledge worker mental fatigue
using the EEG signal,” Appl. Ergonomics, vol. 25, pp. 355–365, 1994.
[8] J. A. Horne and L. A. Reyner, “Driver sleepiness,” J. Sleep Res., vol.
4, pp. 23–29, 1995.
[9] S. K. L. Lal and A. Craig, “Driver fatigue: Electroencephalography
and psychological assessment,” Psychophysiology, vol. 39, no. 3, pp.
313–321, 2002.
[10] P. Artaud, S. Planque, C. Lavergne, H. Cara, P. de Lepine, C. Tarriere,
and B. Gueguen, “An on-board system for detecting lapses of alertness
in car driving,” in Proc. 14th Int. Conf. Enhanced Safety of Vehicles,
Munich, Germany, 1994, vol. 1.
[11] A. Gevins, H. Leong, R. Du, M. E. Smith, J. Le, D. DuRousseau,
J. Zhang, and J. Libove, “Towards measurement of brain function in
operational environments,” Biological Psychol., vol. 40, no. 1–2, pp.
169–186, 1995.
 Downloaded from https://iranpaper.ir
SHEN et al.: FEATURE SELECTION METHOD FOR MULTILEVEL MENTAL FATIGUE EEG CLASSIFICATION
[12] D. Dinges and M. Mallis, , L. R. Hartley, Ed., Managing Fatigue in
Transportation: Selected Papers From the 3rd Fatigue in Transporta-
tion Conference. Oxford, U.K: Elsevier, 1998, pp. 209–229.
[13] S. K. L. Lal, A. Craig, P. Boord, L. Kirkup, and H. Nguyen, “Develop-
ment of an algorithm for an EEG-based driver fatigue countermeasure,”
J. Safty Res., vol. 34, pp. 321–328, 2003.
[14] National Transportation Safety Board (NTSB), Most Wanted Trans-
portation Safety Improvements [Online]. Available: http://www.ntsb.
gov/Recs/mostwanted/index.htm
[15] B. Boser, I. Guyon, and V. N. Vapnik, “A training algorithm for op-
timal margin classifiers,” in Proc. 5th Annu. Workshop Computational
Learning Theory, Pittsburgh, PA, 1992, pp. 144–152.
[16] C. Cortes and V. N. Vapnik, “Support vector networks,” Mach. Learn.,
vol. 20, pp. 273–297, 1995.
[17] V. N. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
[18] I. Guyon and A. Elisseeff, “An introduction to variable and feature
selection,” J. Mach. Learn. Res., vol. 3, pp. 1157–1182, 2003.
[19] R. Kohavi and G. John, “Wrappers for feature subset selection,” Artif.
Intell., vol. 97, pp. 273–324, 1997.
[20] E. Yom-Tov and G. F. Inbar, “Feature selection for the classification
of movements from single movement-related potentials,” IEEE Trans.
Neural Syst. Rehabil. Eng., vol. 10, no. 3, Sep. 2002.
[21] I. Guyon, J. Weston, S. Barnhill, and V. Vapnik, “Gene selection for
cancer classification using support vector machines,” Mach. Learn.,
vol. 46, no. 1–3, pp. 389–422, 2002.
[22] A. Rakotomamonjy, “Variable selection using SVM-based criteria,” J.
Mach. Learn. Res., vol. 3, pp. 1357–1370, 2003.
[23] T. N. Lal, M. Schroder, T. Hinterberger, J. Weston, M. Boqdan, N.
birbaumer, and B. Scholkopf, “Support vector channel selection in
BCI,” IEEE Trans. Biomed. Eng., vol. 51, no. 6, pp. 1003–1010,
Jun. 2004.
[24] L. Breiman, “Bagging predictors,” Mach. Learn., vol. 26, pp. 123–140,
1996.
[25] L. Breiman, “Random forests,” Mach. Learn., vol. 45, pp. 5–32, 2001.
[26] Y. Y. Pang, X. P. Li, K. Q. Shen, H. Zheng, W. Zhou, and E. P. V.
Wilder-Smith, “An auditory vigilance task for mental fatigue detec-
tion,” presented at the 27th Annu. Int. Conf. EMBC, Shanghai, China,
2005.
[27] R. D. Ogilvie, “The process of falling asleep,” Sleep Med. Rev., vol. 5,
no. 3, pp. 247–270, 2001.
[28] H. Jasper, “The 10–20 electrode system of the international federa-
tion,” Electroencephalogr. Clin. Neurophysiol., vol. 10, pp. 371–375,
1958.
[29] E. Niedermeyer and F. L. D. Silva, Electroencephalography: Basic
Principles, Clinical Applications, and Related Fields. Baltimore,
MD: Lippincott Williams & Wilkins, 1999.
[30] A. V. Oppenheim and R. W. Schafer, Discrete-Time Signal Pro-
cessing. Upper Saddle River, NJ: Prentice-Hall, 1989.
[31] E. W. Weisstein, Full Width at Half Maximum MathWorld—A
Wolfram Web Resource [Online]. Available: http://mathworld.wol-
fram.com/FullWidthatHalfMaximum.html
[32] L. Breiman, RF/Tools: A Class of Two-Eyed Algorithms—Expository
Notes 2003, Univ. California Berkeley, Statist. Dept., .
[33] D. Meyer, F. Leisch, and K. Hornik, “The support vector machine under
test,” Neurocomputing, vol. 55, no. 1–2, pp. 169–186, 2003.
[34] L. Breiman and A. Cutler, Random Forests. Version 5.1 [Online].
Available: http://www.stat.berkeley.edu/~breiman/RandomForests/
cc_software.htm
[35] S. N. Lahiri, Resampling Methods for Dependent Data, ser. Statistics.
New York: Springer, 2003.
[36] R. Diaz-Uriarte and S. A. de Andres, “Gene selection and classification
of microarray data using random forest,” BMC Bioinformatics, vol. 7,
no. 3, 2006.
[37] G. E. P. Box, W. G. Hunter, and J. S. Hunter, Statistics for Ex-
perimenters: An Introduction to Design, Data Analysis, and Model
Building. New York: Wiley, 1978.
[38] A. Muzur, E. F. Pace-Schott, and J. A. Hobson, “The prefrontal cortex
in sleep,” Trends Cognit. Sci., vol. 6, pp. 475–481, 2002.
[39] S. Makeig and T. Jung, “Tonic, phasic, and transient EEG correlates
of auditory awareness in drowsiness,” Cognit. Brain Res., vol. 4, no. 1,
pp. 15–25, 1996.
[40] H. Y. Jiang, Y. P. Deng, H.-S. Chen, L. Tao, Q. Y. Sha, J. Chen,
C.-J. Tsai, and S. L. Zhang, “Joint analysis of two microarray gene-ex-
pression data sets to select lung adenocarcinoma marker genes,” BMC
Bioinformatics, vol. 5, no. 81, 2004.
Authorized licensed use limited to: Carleton University. Downloaded on June 21,2023 at 11:00:48 UTC from IEEE Xplore. Restrictions apply.
https://www.tarjomano.com https://www.tarjomano.com
1237
Kai-Quan Shen received the B.S. degree from the
University of Science and Technology of China,
Hefei, China. He is currently working toward the
Ph.D. degree under the supervision of Prof. X.-P.
Li, Associate Professor E. P. V. Wilder-Smith and
Associate Professor C.-J. Ong at the Department of
Mechanical Engineering, the National University of
Singapore.
His research interests focus on feature selection
methods, support vector machines, brain signal pro-
cessing, blind signal separation, and the investigation
of neurophysiological mechanisms of human brain using functional MRI.
Chong-Jin Ong received the B.Eng. (Hons) and
M.Eng. degrees in mechanical engineering from
the National University of Singapore, Singapore,
in 1986 and 1988, respectively, and the M.S.E. and
Ph.D. degrees in mechanical and applied mechanics
from University of Michigan, Ann Arbor, in 1992
and 1993, respectively.
He joined the National University of Singapore
in 1993 and is now an Associate Professor with
the Department of Mechanical Engineering. His
research interests are in algorithms for machine
learning, feature selection and robust control.
Xiao-Ping Li received received the Ph.D. degree in
mechanical and manufacturing engineering from the
University of New South Wales, Sydney, Australia,
in 1991.
He joined the National University of Singapore in
1992, where he is currently a Full Professor with the
Department of Mechanical Engineering and Division
of Bioengineering. His current research interests in-
clude neurosensors and nanomachining. He is a guest
editor of the International Journal of Computer Ap-
plications in Technology, an editorial board member
of the International Journal of Abrasive Technology and Engineering, an edito-
rial advisor of the Chinese Journal of Mechanical Engineering, and a regular re-
viewer of 14 international journals. His research achievements include 6 patents
granted, 2 patents in pending, and 230 technical publications, of which 118 are
international refereed journal papers. He supervised 6 postdoctoral research fel-
lows, 13 Ph.D. degree students and 23 masters degree students.
Dr. Li is a member of the American Society of Mechanical Engineers
(ASME), a senior member of Society of Manufacturing Engineers (SME), a
senior member of the North American Manufacturing Institute of SME.
Zheng Hui received the B.S. degree in electrical and
computer engineering in 2004 and spent two years
doing research in signal processing and pattern recog-
nition for EEG applications as a master student at the
National University of Singapore, Singapore.
He is currently a Research Scientist with the Cog-
nitive Neuroscience Laboratory, Singapore.
Einar P. V. Wilder-Smith received the M.B.B.S.
(equivalent) and M.D. degrees from Heidelberg
University, Heidelberg, Germany, in 1986 and 1989,
respectively.
He joined the National University of Singapore,
Singapore, in 2001 as Associate Professor in the De-
partment of Medicine were he heads the Neurology
Diagnostic Laboratory and is director of Research for
the Department of Medicine. His research interests
are in the filed of clinical neuropathysiology and in-
clude EEG changes in relation to emotions and neu-
rological disorders.