2018 IEEE 16th Int. Conf. on Dependable, Autonomic & Secure Comp., 16th Int. Conf.

on Pervasive Intelligence &

Comp., 4th Int. Conf. on Big Data Intelligence & Comp., and 3rd Cyber Sci. & Tech. Cong.

Feature Selection For Machine Learning-Based

Early Detection of Distributed Cyber Attacks
Yaokai Feng
Faculty of Advanced Information Technology,
Kyushu University, Japan.
Email: fengyk@ait.kyushu-u.ac.jp
Hitoshi Akiyama Liang Lu1
Department of Informatics, Department of Informatics,
Kyushu University, Japan. Kyushu University, Japan.
Email: ru78toy6x hitc@icloud.com Email: lu.liang@jp.fujitsu.com

Kouichi Sakurai
Faculty of Informatics,
Kyushu University, Japan.
Email: sakurai@inf.kyushu-u.ac.jp

Abstract—It is well known that distributed cyber attacks
simultaneously launched from many hosts have caused the most
serious problems in recent years including problems of privacy
leakage and denial of services. Thus, how to detect those attacks
at early stage has become an important and urgent topic in the
cyber security community. For this purpose, recognizing C&C
(Command & Control) communication between compromised
bots and the C&C server becomes a crucially important issue,
because C&C communication is in the preparation phase of
distributed attacks. Although attack detection based on signature
has been practically applied since long ago, it is well-known that
it cannot efficiently deal with new kinds of attacks. In recent
years, ML(Machine learning)-based detection methods have been
studied widely. In those methods, feature selection is obviously
very important to the detection performance. We once utilized
up to 55 features to pick out C&C traffic in order to accomplish
early detection of DDoS attacks. In this work, we try to answer
the question that ”Are all of those features really necessary?”
We mainly investigate how the detection performance moves as
the features are removed from those having lowest importance
and we try to make it clear that what features should be
payed attention for early detection of distributed attacks. We
use honeypot data collected during the period from 2008 to
2013. SVM(Support Vector Machine) and PCA(Principal Com-
ponent Analysis) are utilized for feature selection and SVM and
RF(Random Forest) are for building the classifier. We find that
the detection performance is generally getting better if more
features are utilized. However, after the number of features has
reached around 40, the detection performance will not change
much even more features are used. It is also verified that, in
some specific cases, more features do not always means a better
detection performance. We also discuss 10 important features
which have the biggest influence on classification.
Keywords-distributed cyber attacks; DDoS attacks; machine
learning; feature selection; early detection;
I. D ISTRIBUTED C YBER ATTACK AND I TS E ARLY
D ETECTION
The problems and losses caused by cyber attacks have been
increasing greatly in recent years, despite many works for
avoiding and detecting cyber attacks have been done and a
1 Presently with Fujitsu Co. ltd, Japan.
huge amount of money has been invested on cyber security.
The main reason for this is that attackers have also been more
and more sophisticated.
Distributed attacks are those launched cooperatively by
many compromised hosts. Such attacks are referred to as next-
generation cyber-attacks in Xu’s work [1] and it is well known
that such attacks are one kind of the most sophisticated attacks.
According to many reports, distributed attacks have caused
the most serious problems/losses in these years. Thus, many
researchers and developers in the cyber security community
have been working on how to detect and avoid such attacks.
In general, the attacker prepares or hijack a C&C server,
which is used to send attack instructions to the compromised
hosts (bots). Then, the bots launch an actual distributed attack
to the victim(s). Thus, the C&C communication is prepara-
tion phase for distributed attack. If such communication is
recognized, the upcoming actual distributed attack might be
blocked. Thus, the detection of the C&C communication is
regarded as early detection of distributed attacks.
There have been many cases of distributed attack. The non-
profit anti-spam organization Spamhaus [2] was suffering a
large DDoS attack against their website. The peak of that
DDoS attack was up to about 300 Gbps on March 19, 2013
[3]. That means the data in about 64 full DVDs of 4.7GB were
poured on the server of Spamhaus’s server in every second,
which finally Knocked Spamhaus Offline. Another large DDoS
attack peaking at around 400Gbps was reported by Cloudflare
on March 3, 2016 [4]. All 13 DNS root servers suffered from
a distributed attack on June 25, 2016 and this wasn’t the first
time that critical DNS infrastructure was aimed at. At the end
of 2015, several root servers encountered a DDoS attack, and
they also experienced a DDoS attack in the middle of May of
the same year that made services like Yelp and Alexa stop. The
DDoS attack on June 25, 2017 lasted for around three hours
during which the average availability across all root servers
dropped to around 50% of the normal [5].
Thus, the early detection of distributed attacks which can
stop the upcoming attacks obviously becomes critically sig-

978-1-5386-7518-2/18/$31.00 ©2018 IEEE 173

DOI 10.1109/DASC/PiCom/DataCom/CyberSciTec.2018.00040
nificant. In this study, we try to predict possible upcoming
distributed attacks during their preparation phase by picking
out its C&C communication.
There have been many ML (Machine Learning)-based stud-
ies on detecting distributed attacks including how to ac-
complish early detection. Those methods tried to find some
particular features of the abnormal traffic to distinguish them
from the others. Also, we once used up to 55 traffic features
to implement early detection of distributed attacks. In this
study, we discuss what will happen if the number of features
is decreased gradually. That is, how the detection performance
changes as the features are decreased from those having low
importance. Of course, the appropriate number of features is
different according to different machine learning algorithms.
Honeypot data collected during the period from 2008 to 2013
and the OSS Weka are used in our study. SVM and PCA
are used for feature selection and SVM and RF (Random
Forest) are for building classifier. We find that, the detection
performance is generally getting better if more features are
used. However, after the number have reached around 40,
the detection performance will not change much even more
features are used. Moreover, in some specific cases, it is also
verified that more features do not always lead a better detection
performance. Also, we will discuss the top-10 important
features for detection of C&C communication.
II. E XISTING D ETECTION T ECHNOLOGIES
Many approaches have been proposed to detect cyber
attacks [6], [7], including signature-based methods [8],
histogram-based methods [10], [11], volume-based methods
[9] and information theory-based methods [13], [14]. However,
it is well known that signature-based methods cannot effi-
ciently deal with new kinds of attacks and new variants. This
is because they can only detect the anomalies stored in a pre-
defined database of signatures. Many statistic histograms are
built in histogram-based methods using clean traffic data and
all the histograms are mapped into a high-dimensional space.
Such methods are easy to understand. But, their false negative
rates are often very high [12]. Volume-based methods need to
thresholds that must be determined in advance, which is not
easy for most situations. Information theory-based methods
also suffer from the following problems. Their performance
is greatly dependent on the information theoretic measure;
they behavior well only when a significantly large number
of anomalies are present in the data and, moreover, it is
difficult to associate an anomaly score with a test instance
[15]. To detect scan attacks, the session information based
on the actions of communication protocols (such as TCP and
UDP) was also used in the work [17] . Change-points based
methods also have been proposed. For example, the work [16]
proposed a change-point based method to detect TCP SYN
flood attacks and scan attacks by computing the characteristics
of the packets.
Signature-based detection is a direct method to detect botnet
in a network and payload analysis is a common technique
174
of signature-based detection. Gu et al. proposed a botnet de-
tection framework based on signature-based detection, called
BotHunter [21], which is an alert system when some bot be-
haviors are detected by Snort [18] intrusion detection system.
Bot activity can also be detected by picking out C&C session
that occurs before actual distributed attacks. Several methods
have been proposed to pick out C&C sessions using some
features of those sessions. For example, the C&C server only
send commands whose packets are of small size. The work
[22] analyzes network traffic and uses machine learning to
detect IRC-based C&C communication. In that work, it was
found that visible differences between regular sessions and
C&C sessions are the size of time-interval of packets. That is,
C&C sessions show more normalized and small in packet size
and small periodicity of interval time. The work [22] uses three
different feature vectors focusing on total, first 16 packets and
histogram separately.
The HTTP-based C&C communication has been confirmed
in around 2005. According to the investigation in the work
[23], a periodicity occurs among the operation obtaining the
command and the standard deviation in the time interval of the
communication between the C&C server and HTTP-based bots
is smaller compared with the frequent communication. How-
ever, because of the network interception or an un-receivable
command for some reason, a large number of exceptions may
appear.
Some methods to detect C&C session based on multiple
protocols also have been proposed. The work [24] proposed a
C&C traffic detection approach based on analysis of network
traffic using seven features (e.g. the standard deviation of
access time and access time). That study claims that their
approach is able to effectively detect C&C traffic even in
multiple protocols. Our previous research [25] adopts 55
features for the same purpose, which is explained in the next
section. In that study, adding some new features to the work
[24], we defined a 55-dimensional feature vector to improve
the detection performance.
Generally, an actual detection system based machine learn-
ing algorithms may be online or offline. An online system
must be able to report alerts for possible attacks/anomalies and
the learning result should updated by invoking the machine
learning algorithm repeatedly while an offline system can
use a training dataset and a machine learning algorithm to
obtain the learning result, which will be used to pick out the
attacks from the dataset that we want to detect the possible
attacks/anomalies in it. In order to evaluate the learning result
and to know if it can be used for actual detection, a dataset
for testing the learning result is also used to tune parameters
or/and to decide what learning algorithm is suitable for the
current application. Figures 1 and 2 depict the online detection
and the offline detection, respectively.
III. O UR D ETECTION U SING 55 F EATURES
In our previous work [25], all the related features we
could find, totally 55 features, were used and some simple
experimental results for verifying detection performance were
 Fig. 1. The image of Online Detection.

Fig. 3. 55 Features Used in Our Detection.

Fig. 2. The image of Offline Detection.

IV. F EATURE S ELECTION AND I TS I NFLUENCE ON

D ETECTION P ERFORMANCE
presented. The total size and total amount of sent-packets
Good feature selection is critically important for many
and received-packets, respectively, are selected. The minimum
actual machine learning-based attack detections. At first, it can
packet size was also chosen as a feature because we consider
speed up the detection process and it can also perform feature
many C&C sessions only send commands to bots, which
discovery, that is, finding what features are really important.
means the payload of packets is almost empty. Besides, some
Many researches have indicated that SVMs may perform badly
statistics features of packet size are also selected including the
if many irrelevant features being used [26], [27], [28], [29].
average size and the size variance in a session. The interval
In the last section, our work using 55 features of traffic
time of successive packets is also contained in our feature
to pick out the C&C traffic is briefly explained. An obvious
vector because it was reported meaningful for picking out
question here is that ”Are all of them necessary?” and ”What
C&C traffic in the work [22]. Similar to packet size, total
if some of them are removed”. To answer these questions, in
interval time, minimum interval time, average interval time
this paper, after our proposal using 55 features is investigated
and the variance of interval times in a session are also used.
in more details, all the 55 features are ranked according to
Furthermore, because C&C sessions just issue some similar
their important degrees, and then the features will be removed
commands to the bots, the size of most C&C packets should
gradually to examine the change of the detection performance.
be concentrated. Thus, in our research, some scale features of
PCA is able to select a number of important individuals
packet size are also used. Specifically, the possible range of
from all the feature components and has the potential to per-
packet size is split into 15 intervals from 0-99 bytes, 100-199
form feature selection [30] and the work [31], [32] confirmed
bytes, ... . And, the proportion of the packets in each interval
that SVM can perform feature weighting. In this paper, PCA
is utilized. The Figure 3 shows all the 55 features.
and SVM are employed for ranking the features. The ranked
In this paper, after the behavior of our proposal using features are removed 5 features at one time from the ones of
55 features are investigated in more details, we will try to the least importance and the change of detection performance
decrease some features to see how the detection performance are examined using SVM and RF (Random Forest) . As shown
changes. In this way, we can know what features are most below, the TP (Ture Positive) rate, the FP (False Positive)
important to detection C&C communication. rate, Presion and F-Measure are used to evaluate the detection

175
performance. The detailed examination result will be presented
in the next section.
TP
T P rate = Recall = ;
TP + FN

FP
F P rate = ;
FP + TN

TP
P recision = ;
TP + FP

2 × P recision × Recall
F − M easure = ;
P recision + Recall

where T P =the number of the true positives; T N =the

number of the true negatives; F P =the number of false
positives; F N =the number of false negatives. Fig. 4. Distribution of average packet sizes. The X-axis is for packets sent;
The Y-axis packet received; Reds are C&C sessions and blues normal.
V. E XPERIMENT
A. Data
In our experiment, as C&C session data we used the CCC
datasets [33] including C08, C09, C10 and C13 datasets
(https://www.telecom-isac.jp/ccc/). CCC datasets were col-
lected by honeypots in a project supported by Japanese govern-
ment and managed by Cyber Clean Center. The normal session
data is collected in our lab for two months from Aug. 2012 to
Sept. 2012. We collected traffic packets in port No. 6667 used
by IRC and port No. 80 used by HTTP protocol. Because a bot
need to establish a connection with the C&C server, so we only
take sessions having bidirectional communication. Totally1162
C&C traffic sessions and 894 normal sessions were extracted
successfully.
B. Verification of availability of different features (examples) Fig. 5. Distribution of packet numbers. The X-axis is for packets sent; The
Y-axis packet received; Reds are C&C sessions and blues normal.
In order to investigate the impact of each feature on attack
detection, we need to know the distribution of the normal
sessions and the C&C sessions in the feature space. Here are
some results of our investigation. Figure 4 is the distribution • Step1: session split. All the sessions are extracted from
of average packet sizes in the feature space. The X-axis is the packet datasets.
for packets sent, The Y-axis is for packet received; The reds • Step2: feature extraction. 55 features are computed from
are C&C sessions and The blues normal. Figure 5 shows the the session datasets.
distribution of packet numbers. Figure 6 is the distribution of • Step3: feature ranking. the 55 features are ranked using
the variance of packet sizes. PCA and SVM, respectively.
From these distribution figures, we can observed that avail- • Step4: decision of test data and training data.
ability of different features is very different from each other • Step5: building classifier using SVM and RF, respec-
when they are used for detecting C&C traffic. tively.
• Step6: testing the classifier
C. Experiment process for feature selection • Step7: looping to Step 4 for 10 times.
After the packet lists in our datasets are split into sessions, • Step8: looping to Step 4 with a decreased number of
the features of each session are computed. In our experiment, features.
10-fold cross-validation [34] is used. That is, each of our
session datasets is split into 10 folds, one of which is used D. Experiment result about the influence of different feature
as test data and the other nine are used as training data. selections
Afterwards, select the next fold as test data. In this way, this
Two machine learning algorithms, SVM and RF, are used
process will be conducted 10 times and get the average as
and two algorithms, PCA and SVM, are for ranking the
the final result. In summary, our experiment has the following
features by their importance. Thus, totally, the influence of
eight steps, which is also depicted in Figure 7. This process
different feature selection on the detection performance is in-
is also simply shown in Figure 8.
vestigated in the following four different cases. As mentioned

176
 packet
datasets

session
feature 55 features
split
extraction

feature
ranking

classifier

Fig. 6. Distribution of the variance of packet sizes. The X-axis is for packets
sent; The Y-axis packet received; Reds are C&C sessions and blues normal. removing features
gradually
result
10-fold
cross-validation
Fig. 8. Our experiment process.
start changing
test data

session splitting training

the number of features increased up to 40. In the case of
feature selection using PCA and classifying by SVM, however,
feature extracting testing
the irregularity appears (see Figure 12) , which is thought to
feature ranking lp-1
be due to the fact that PCA immediately deleted important
features for classification for SVM. Anyway, Figure 11 shows
fn=55; lp=10; N lp=0 that PCA feature selection also lead to a good detection
performance if RF is used for classification. Generally, the
Y detection performance is getting better if more features are
removing some features utilized. However, after the number of features has reached
(updating fn) around 40, the detection performance will not change much
even more features are used. It is also verified that, in some
fn<1 N specific cases, more features do not always means a better
detection performance. That is, some features may have bad
Y
influence to the detection performance. Thus, our experiment
end
verified again that the importance of feature selection. It
is necessary to investigate in detail why the results are so
Fig. 7. The flow of our experiment process.
different for different algorithms of machine learning and
feature selection, which will be our future work.
Moreover, from our experiment, the most important 10
in Section IV, the TP rate, the FP rate, Precision and F- features for picking out the C&C traffic were found and listed
Measure are evaluated for each case. in Table I.
1) SVM feature selection and SVM classifier: The experi-
TABLE I
ment result is shown in Figure 9. T HE MOST IMPORTANT FEATURES FOR DETECTING C&C TRAFFIC .
2) SVM feature selection and RF classifier: The experiment Feature Description
result is shown in Figure 10. Spc amount of packets sent
3) PCA feature selection and RF classifier: The experi- FlagR proportion of packets with flag R in a session
ment result is shown in Figure 11. Sip1 Proportion of the sent packets having the size of 100-199B
Sip0 Proportion of the sent packets having the size of 0-99B
4) PCA feature selection and SVM classifier: The experi-
FlagS proportion of packets with flag S in a session
ment result is shown in Figure 12. Svar Variance of sent packets
RITmax Maximum interval time in received package
E. Observation RITvar Variance of interval time of received packages
Sip5 Proportion of the sent packets having the size of 500-599B
From Figures 9 and 10, a slow improvement on the detection Rvar Variance of packet size of the received packets
performance is observed after the number of features increased
up to 20. From Figure 11, a slow improvement appears after

177

(a) the TP rate
(a) the FP rate
(a) the Precision
(a) the F-Measure
Fig. 9. Detection performance based on SVM feature selection and SVM
classifier.
VI. C ONCLUSION AND F UTURE W ORK
In this paper, we focused on the issue of feature selection
for early detection of distributed cyber attacks. We imple-
mented the early detection by detecting C&C communication
of distributed attacks because those communication is at the
preparation phase of distributed attacks. Based on our previous
research using 55 features to detect C&C communication, in
this paper we investigated what if we remove the features
from those of the least importance. We did this for the
purpose of finding that what features are actually critical for
early detection of distributed attacks. From our experiment
using traffic data collected by honeypots, we observed that
the detection performance is generally getting better if more
features are utilized. However, after the number of features
has reached around 40, the detection performance will not
change much even more features are used. We also found
the top-10 important features for detecting C&C traffic. We
again verified that some ”bad” features would deteriorate the
detection performance.
As future work, we will analyze the experiment result in
more detail and find the detailed reason why the detection
performance changes in such ways. Also, we will verify our
observation using other traffic datasets.
178
(a) the TP rate
(a) the FP rate
(a) the Precision
(a) the F-Measure
Fig. 10. Detection performance based on SVM feature selection and RF
classifier.
ACKNOWLEDGE
This work was partially supported by JSPS KAKENHI
Grant Numbers JP17K00187 and JP18K11295. This work is
also partially supported by Strategic International Research
Cooperative Program, Japan Science and Technology Agency
(JST).
R EFERENCES
[1] S. Xu S., ”Collaborative Attack vs. Collaborative Defense,” in Proc. the
4th International Conference on Collaborative Computing (Collaborate-
Com), pp. 217–228, 2009.
[2] The non-profit anti-spam organization Spamhaus,
https://www.spamhaus.org/ (accessed on April 19, 2018).
[3] ”The DDoS That Knocked Spamhaus Offline,” reported by Cloudflare
on March 30, 2013. https://blog.cloudflare.com/the-ddos-that-knocked-
spamhaus-offline-and-ho/ (accessed on April 19, 2018).
[4] ”400Gbps: Winter of Whopping Weekend DDoS Attacks,” reported by
Cloudflare on March 3, 2016. https://blog.cloudflare.com/a-winter-of-
400gbps-weekend-ddos-attacks/ (accessed on April 19, 2018).
[5] ”DDoS Attack Has Varying Impacts on DNS Root Servers,” reported by
ThousandEyes on July 19th, 2016. https://blog.thousandeyes.com/ddos-
attack-varying-impacts-dns-root-servers/ (accessed on April 19, 2018).
[6] Y. Feng, Y. Hori, and K. Sakurai, ”A Proposal for Detecting Distributed
Cyber-Attacks Using Automatic Thresholding,” in Proc. the 10th Asia
Joint Conference on Information Security (AsiaJCIS), pp. 152–159,
2015.
[7] Y. Feng, Y. Hori, K. Sakurai, J. Takeuchi, ”A Behavior-Based Method for
Detecting Distributed Scan Attacks in Darknets,” Journal of Information
Processing, Vol.21, No.3, page 527-538, 2013.
 (a) the TP rate
(a) the FP rate
(a) the Precision
(a) the F-Measure
Fig. 11. Detection performance based on PCA feature selection and RF
classifier.
[8] Y. Tang, ”Defending against Internet Worms: a Signature-based Ap-
proach,” in Proc. the 24th IEEE Annual Joint Conference of the
Computer and Communications Societies (INFOCOM), pp. 1384–1394,
2005.
[9] I. Yazid, A. Hanan and M. Aizaini, ”Volume-based Network Intrusion
Attacks Detection,” Advanced Computer Network and Security, UTM
Press, pp. 147–162, 2008.
[10] A. Kind, M. P. Stoecklin and X. Dimitropoulos, ”Histogram-Based
Traffic Anomaly Detection,” IEEE Transactions on Network Service
Management, Vol. 6, No. 2, pp. 1–12 (2009).
[11] E. Eskin and W. Lee, ”Modeling System Call for Intrusion Detection
with Dynamic Window Sizes,” in Proc. DARPA Information Survivabil-
ity Conference and Exposition (DISCEX), pp. 165–175, 2001.
[12] Y. Feng, Y. Hori, K. Sakurai and J. Takeuchi, ”A Behavior-based Method
for Detecting Outbreaks of Low-rate Attacks,” in Proc. 3rd Workshop
on Network Technologies for Security, Administration and Protection
(NETSAP), pp. 267–272, 2012.
[13] Y. Xiang, K. Li and W. Zhou, ”Low-Rate DDoS Attacks Detection and
Traceback by Using New Information Metrics,” IEEE Transactions on
Information Forensics and Security, Vol.6, No.2, pp. 426–437, 2011.
[14] W. Lee and D. Xiang, ”Information-theoretic Measures for Anomaly
Detection,” in Proc. IEEE Symposium on Security and Privacy, pp. 130–
143, 2001.
[15] V. Chandola, A. Banerjee and V. Kumar, ”Anomaly Detection: a Survey,”
ACM Computing Survey, Vol.41, No.3, pp. 1–72, 2009.
[16] M. S. Kim, H. J. Kang and S. C. Hong; ”A Flow-based Method for
Abnormal Network Traffic Detection,” in Proc. IEEE/IPIP Network
Operations and Management Symposium, pp. 599–612, 2004.
[17] J. Treurniet, ”A Network Activity Classification Schema and its Appli-
cation to Scan Detection,” IEEE/ACM Transactions on Networking, Vol.
19, No. 5, pp. 1396–1404, 2011.
179
(a) the TP rate
(a) the FP rate
(a) the Precision
(a) the F-Measure
Fig. 12. Detection performance based on PCA feature selection and SVM
classifier.
[18] Snort Users Manual, http://www.snort.org/docs (accessed on June 20,
2018).
[19] C. Gates, ”The Modeling and Detection of Distributed Port Scans: a
Thesis Proposal,” Technical Report CS-2003-01, Dalhousie University,
2003.
[20] V. Yegneswaran, P. Barford and J. Ullrich, ”Internet Intrusions: Global
Characteristics and Prevalence,” in Proc. 2003 ACM Joint International
Conference on Measurement and Modeling of Computer Systems, pp.
138–147, 2003.
[21] G. Gu, P. Porras, V. Yegneswaran, M. Fong, W. Lee, ”BotHunter:
Detecting Malware Infection Through IDS-driven Dialog Correlation,”
in Proc. the 16th USENIX security symposium, pp. 167–182, 2007.
[22] S. Kondo and N. Sato, ”Botnet traffic detection techniques by C&C
session classification using SVM,” in Proc. 2nd international conference
on Advances in information and computer security, pp. 29–31, 2007.
[23] D. Ashley, ”An algorithm for http bot detection,” University of Texas at
Austin - Information Security Office, 2011.
[24] S. Yamauchi, J. Kawamoto and K. Sakurai, ”Evaluation of Machine
Learning Techniques for C&C traffic Classification,” (in Japanese) IPSJ
Journal, Vol. 56, No. 9 : pp. 1745–1753, 2015.
[25] L. Lu, Y. Feng and K. Sakurai, ”C&C session detection using random
forest,” in Proc. the 11th ACM International Conference (IMCOM) ,
2017.
[26] J. Weston et al. , ”Feature Selection for SVMs, ”
http://www.cs.columbia.edu/ jebara/6772/papers/weston01feature.pdf
(accessed on April 29, 2018).
[27] Y. W. Chen and C. J. Lin, ”Combining SVMs with various feature
selection strategies,” Feature extraction, pp. 315–324, Springer Berlin
Heidelberg, 2006.
[28] I. Guyon, J. Weston, S. Barnhill and V. Vapnik, ”Gene selection for
 cancer classification using support vector machines,” Machine learning,
46(1-3), pp. 389–422, 2002.
[29] V. Sugumaran, V. Muralidharan and K. I. Ramachandran, ”Feature
selection using decision tree and classification through proximal support
vector machine for fault diagnostics of roller bearing,” Mechanical
systems and signal processing, Vol. 21, No. 2, pp. 930–942, 2007.
[30] F. Song, Z. Guo and D.Mei, ”Feature Selection Using Principal Compo-
nent Analysis,” in Proc. the International Conference on System Science,
Engineering Design and Manufacturing Informatization (ICSEM), pp.
27–30, 2010.
[31] T. Grtner, P. A. Flach, ”WBCSV M : Weighted Bayesian classification
based on support vector machines,” in Proc. the 18th International
Conference on Machine Learning (ICML), pp. 154161, 2001.
[32] J. Brank, M. Grobelnik, N. M. Frayling and D. Mladeni, ”Feature
Selection Using Linear Support Vector Machines,” Technical Report
MSR-TR-2002-63, Microsoft Research, Microsoft Corporation, 2002.
[33] O. Takata, et al., ”MWS Datasets 2016, Anti-Malware Engineer-
ing Workshop (MWS),” www.iwsec.org/mws/2016/20160714-takata-
dataset.pdf (2016) (accessed on April 29,2018).
[34] K. Ron, ”A study of cross-validation and bootstrap for accuracy es-
timation and model selection,” in Proc. the 14th international joint
conference on Artificial intelligence (IJCAI) , pp. 1137–1143 , 1995.

180