Comparative Study of CNN and RNN

for Deep Learning Based Intrusion

Detection System

Jianjing Cui1(B) , Jun Long1 , Erxue Min1 , Qiang Liu1 , and Qian Li2
1
Department of Computer Science, National University of Defense Technology,
Changsha 410005, China
cuijianjing16@nudt.edu.cn
2
Faculty of Engineering and IT, University of Technology Sydney,
Sydney, NSW 2007, Australia

Abstract. Intrusion detection system plays an important role in ensur-

ing information security, and the key technology is to accurately identify
various attacks in the network. Due to huge increase in network traffic
and different types of attacks, accurately classifying the malicious and
legitimate network traffic is time consuming and computational inten-
sive. Recently, more and more researchers applied deep neural networks
(DNNs) to solve intrusion detection problems. Convolutional Neural Net-
work (CNN) and Recurrent Neural Network (RNN), the two main types
of DNN architectures, are widely explored to enhance the performance
of intrusion detection system. In this paper, we made a systematic com-
parison of CNN and RNN on the deep learning based intrusion detection
systems, aiming to give basic guidance for DNN selection.

Keywords: Intrusion detection system · Deep neural networks

Convolutional neural network · Recurrent neural network

1 Introduction

An Intrusion Detection System (IDS), a significant research achievement in the

information security field, can identify an invasion, which could be an ongo-
ing invasion or an intrusion that has already occurred. In fact, intrusion detec-
tion is usually equivalent to a classification problem, such as a binary (i.e. nor-
mal/attack) or a multi-class (i.e. normal and different attack types) classification
problem. Follow this line of thinking, machine learning methodologies have been
widely used in the intrusion detection systems and achieved high performances.
However, the performance of traditional machine learning methods is highly
dependent on the manually designed features. It’s also hard to extract effective
features automatically and costs lots of time. Deep neural networks (DNNs)
[1,2] applying deep learning approach can automatically extract high-level
features from low-level ones and gain powerful representation and inference.
c Springer Nature Switzerland AG 2018
X. Sun et al. (Eds.): ICCCS 2018, LNCS 11067, pp. 159–170, 2018.
https://doi.org/10.1007/978-3-030-00018-9_15
160 J. Cui et al.

As a result, many researchers proposed different kinds of deep learning based

intrusion detection methods. Their experiments showed deep learning highly
enhanced the performance of intrusion detection systems.
There are two main DNN architectures: convolutional neural network (CNN)
[3] and recurrent neural network (RNN) [4]. Many researchers have been strug-
gling with how to choose proper one to solve intrusion detection tasks. Generally
speaking, CNN is good at classification tasks which needs only key components;
while RNN performs better at sequence modeling tasks which requires flexible
modeling of context dependencies [5]. But to our best knowledge, there is still not
a conclusion which one is better for intrusion detection tasks. Therefore, we com-
pares CNNs, and RNNs systematically on the NLP tasks. Experimental results
demonstrate that CNNs are better at binary classification tasks and RNNs per-
form better in detecting some sophisticated attacks for multi-class classification
tasks.
The remainder of this paper is organized as follows. Section 2 describes related
work. Section 3 introduces the models of CNNs and RNNs used in our experi-
ments. Section 4 shows details of experiments and analyzes the results. Finally,
Sect. 5 presents conclusions and future work.

2 Related Work

Many machine learning techniques were used for developing IDS. Singh et al.
[6] and Kishorwagh et al. [7] did a survey on each technique and discuss clearly
with their pros and cons. Out of these surveys, one promising machine learning
technique for IDS is neural networks. Qiu et al. [8] used supervised learning
techniques based multi-layer perception (MLP). Neural network is an efficient
way to improve the performance of IDS which are based on the misuse detection
model and the anomaly detection model [9].
Due to the great success of deep learning in the fields of computer vision
[10] and natural language processing [11], some research using deep learning
approach for intrusion detection have recently emerged. Studies have shown that
deep learning completely surpasses traditional methods. Among those studies,
CNNs and RNNs are the two most widely used deep neural network models;
they are capable of learning effective spatial and temporal features, respectively.
There are lots of deep learning based intrusion detection work for either CNNs
or RNNs.
CNNs are capable of learning effective spatial features from hierarchical struc-
tures. In fact, network traffic has an obvious hierarchical structures: TCP con-
nection, flow, session, service and host [12]. As a result, many researchers use
CNNs as deep learning architecture in intrusion detection systems. Wang et al.
[13] used a CNN to learn the spatial features of network traffic and achieved
malware traffic classification using the image classification method. Jia et al.
[14] built a CNN model with multiple “convolution-downsampling” layer to
learn deep features representing the normal and abnormal user behavior. Their
experiments improved the classification accuracy in the intrusion detection and
 Comparative Study of CNN and RNN for Deep Learning 161

recognition tasks. Qian et al. [15] used CNN on NSL-KDD dataset and increased
the classification accuracy.
RNNs are capable of learning temporal features from sequence data. Consid-
ered about the fact network traffic is actually the sequences of binary numbers
“0” and “1”, lots of work have been done using RNNs in intrusion detection sys-
tems. Torres et al. [16] first transformed network traffic features into a sequence
of characters and then used RNNs to learn their temporal features, which were
further applied to detect malware traffic. Yuan et al. [17] chose 20 features at the
preprocessing stage and ran their RNN model on them to detect DDoS attack on
the ISCX2012 dataset. Sheikhan et al. [18] considered RNNs as reduced-size neu-
ral networks. In that paper, they proposed a three-layer RNN architecture with
41 features as inputs and four intrusion categories as outputs, and for misuse-
based NIDS. However, the nodes of layers were partially connected, the reduced
RNNs did not show the ability of deep learning to model high-dimensional fea-
tures, and they did not study the performance of the model in the binary clas-
sification. Yin et al. [19] used the RNN model to perform classification directly
and researched both the binary and multiclass classification.
These work show the popularity and ability of CNNs and RNNs for deep
learning based intrusion detection systems. But as mentioned above, there is
still no comparison of these two methods which one is more suitable for intrusion
detection tasks. That’s the motivation of our work.

3 Models

There are lots of popular variants of CNNs and RNNs. To solve the hardness of
training, the inception architecture CNN [20] is proposed and successfully applied
in GoogLeNet. To alleviate some limitations of the basic RNN, long short-term
memory (LSTM) [21] and gated recurrent unit (GRU) [22] are proposed using
gating mechanisms. This section gives a brief introduction of the models we
used in our experiments: basic CNN, the inception architecture CNN, LSTM
and GRU.

3.1 Basic CNN

The basic CNN model showed in Fig. 1 can be divided into 3 layers:
• Input layer: Suppose the input sequences has n elements. Each element is a
d dimension vector. Then the input x is a matrix of shape d × n.
• Convolution layer: Suppose the filter width is w. Let vector ci ∈ Rwd be the
concatenated embeddings of w elements xi−w+1 , ..., xi . W ∈ Rd×wd denotes
the convolution weights, f denotes the activation function and b ∈ Rd denotes
the bias. Then the definition of the output pi ∈ Rd is:

pi = f (W · ci + b) (1)
162 J. Cui et al.

• Maxpooling layer: Suppose the output of convolution layer is p1 , ..., ps , then

the result of maxpooling is:

xj = max (p1,j , ..., ps,j ) , where j = 1, ..., d (2)

In practical application, the convolution layer and maxpooling layer are usu-
ally added for several times. This can enhance the performance of CNN model
effectively but also cause dramatic increase in the number of parameters and
hardness of training.

Fig. 1. The basic CNN.

3.2 Inception Architecture CNN

To solve the problem of large number of parameters and speed up the training
of CNN, Szegedy et al. [20] propose the inception architecture CNN, which was
successfully applied in GoogLeNet. The details of inception architecture CNN
we used are shown in Fig. 2.

3.3 Long Short-Time Memory (LSTM)

LSTM aims to overcome vanishing gradient problem of RNN and uses a memory
cell to present the previous timestamp. The details of the memory cell is shown
in Fig. 3.
Current modified LSTM usually includes three gates in each cell: input, for-
get, and output. They are calculated as follows:

it = σ (Wi · [ht−1 , xt ] + bi ) (3)

C̃t = tanh (WC · [ht−1 , xt ] + bC ) (4)

 Comparative Study of CNN and RNN for Deep Learning 163

ft = σ (Wf · [ht−1 , xt ] + bf ) (5)

Ct = ft · Ct−1 + it · C̃t (6)
ot = σ (Wo · [ht−1 , xt ] + bo ) (7)
ht = ot · tanh (Ct ) (8)
where xt is the input at time t, Wi , WC , Wf , Wb are weight matrices, bi , bC , bf ,
bo are biases, Ct , C̃t are the new state and candidate state of memory cell, ft ,
ot are forget gate and output gate.

3.4 Gated Recurrent Unit (GRU)

GRU combines the forgetting gate and the input gate to synthesize a single
update gate. It also mixed the cell state and the hidden state, and made some
other changes. The final model is simpler than the standard LSTM model.

Fig. 2. The inception architecture CNN.

Fig. 3. The memory cell in LSTM.

164 J. Cui et al.

Fig. 4. The memory cell in GRU.

GRU is also a very popular variant. Details of GRU are shown in Fig. 4. The
gates in GRU are calculated as follows:
zt = σ (Wz · [ht−1 , xt ]) (9)
rt = σ (Wr · [ht−1 , xt ]) (10)

ht = tanh (W · [rt ∗ ht−1 , xt ]) (11)
ht = (1 − zt ) ∗ ht−1 + zt ∗ ht (12)

4 Experiments
This section introduces details of our experiments and analyse the results. Specif-
ically, it can be divided into 2 parts:
• Experimental methodology (data preprocessing and evaluation metrics)
• Comparing the performance of the 4 models on binary classifications and
multi-class classification tasks.

4.1 Experimental Methodology

1. Data preprocessing

(a) Dataset
We did experiments on ISCX2012 dataset [23] because it’s not out of
date and has anonymous raw data. ISCX2012 concluded 7 days’ network
traffic (3 legitimate and 4 malicious). It’s not necessary to use all the
data because of the imbalance of different classes of data. As a result,
we choose the whole attack data and the normal data of “12/6/2010”.
We divided the new dataset into training and test datasets using a ratio
of 60% to 40%, respectively. Table 1 shows the composition of the our
dataset.
 Comparative Study of CNN and RNN for Deep Learning 165

(b) Preprocessing
In this stage, the raw network traffic data are transformed into text
sequences. We chose 10 features and then added the source payload and
destination payload. The length of the payload is 1000, which means that
we choose 1000 bytes from each packet’s payload. If the length of a pay-
load is less than 1000, zeroes are padded. Correspondingly, the extra part
is truncated. Table 2 shows the features we used.
2. Evaluation metrics
Four metrics are used to evaluate the experimental results: accuracy (ACC),
precision (P), recall rate (R) and F1 score. Accuracy is used to evaluate the
overall performance of the system. Recall rate and precision are used to eval-
uate the system’s performance with respect to its malware traffic detection.
F1 score is used to evaluate performance of every class of traffic, which takes
into account both the precision and recall of the classification model. The
definitions of these metrics are presented below.
TP + TN
Accuracy (ACC) = (13)
TP + FP + FN + TN
TP
P recision (P ) = (14)
TP + FP
TP
RecallRate (R) = (15)
TP + FN
P R · DR
F1 score (F1 ) = 2 · (16)
P R + DR
where T P is the number of instances correctly classified as X, T N is the num-
ber of instances correctly classified as Not-X, F P is the number of instances
incorrectly classified as X, and F N is the number of instances incorrectly
classified as Not-X.

Table 1. Composition of our dataset

Dataset Training Test Total

Normal 78,664 52,443 131,107
Attack BFSSH 3,131 2,088 5,219
Infiltrating 12,214 8,144 20,358
HttpDoS 2,265 1,511 3,776
DDoS 22,475 14,984 37,459
Total 40,085 26,727 66,812
Total 118,749 79,170 197,919
166 J. Cui et al.

Table 2. Features used in our experiments

Feature name Description

src length Total source bytes
dst length Total destination bytes
dst num Total destination packets
src num Total source packets
direction The direction of the packets (L2R, R2L, etc.)
protocol The protocol of the packets (TCP, UDP, ICMP)
src port Source port for TCP or UDP packets, 0 for ICMP packets
dst port Destination port for TCP or UDP packets, 0 for ICMP
packets
src TCP flags Value of flags (i.e. S) for source TCP packets; 0 for UDP
and ICMP packets
dst TCP flags Value of flags (i.e. S, R) for destination TCP packets; 0 for
UDP and ICMP packets

4.2 Comparing the Performance of the 4 Models

Researchers have proposed many deep learning based intrusion detection sys-
tems, most of which are built by CNNs or RNNs. However, there is still no
research on which one is better for intrusion detection tasks. As a result, we
compare the 4 models (basic CNN, inception architecture CNN, LSTM and
GRU) on their performance during our experiments.

Table 3. Comparison among the 4 models for binary classification

Dataset ACC P R F1
Basic CNN Normal 96.14% 94.91% 99.63% 0.97
Attack 92.37% 92.64% 83.59% 0.88
Overall 94.26% 93.78% 91.61% 0.93
Inception CNN Normal 97.05% 96.23% 99.56% 0.98
Attack 92.43% 91.33% 85.12% 0.88
Overall 94.74% 93.78% 92.34% 0.93
LSTM Normal 96.31% 95.13% 99.69% 0.97
Attack 91.12% 90.27% 81.78% 0.86
Overall 93.72% 92.70% 90.74% 0.91
GRU Normal 97.89% 97.23% 99.78% 0.98
Attack 90.70% 87.65% 83.13% 0.85
Overall 94.30% 92.44% 91.46% 0.92
 Comparative Study of CNN and RNN for Deep Learning 167

Table 3 shows a comparison of the experimental results. From the results,

we could find that the inception architecture CNN got the highest overall ACC
and overall recall rate. Besides, the two CNN models surpassed the two RNN
models on both the overall precision and overall recall rate. Although the GRU
model gets the highest normal ACC, precision, recall rate and f1 score, it failed
apparently on those on attack data. The proper explanation is that RNNs (both
LSTM and GRU) tried a lot on the whole sequence comprehension, while a
binary classification task might only need some key information. As a result,
CNNs could extract the key information more quickly. It can be concluded that
if one only wants to classify the network traffic as normal or attack, CNN model
will be a better choice.
The results of multi-class classification are shown in Figs. 5 and 6. We found
that all 4 models had good performance on the normal data and DDoS data.
But for Infiltrating data and HttpDoS data, basic CNN and inception archi-
tecture CNN failed on the precision and F1 score. What’s more, the incep-
tion architecture CNN model performed worse than the other 3 models on
recall rate. The result could be possibly explained by these two attack types
is harder to detect if the model can’t analyse the inside features. This requires a

Fig. 5. The ACC and Recall rate of multi-class classification.

168 J. Cui et al.

comprehensive understanding of the network traffic, actually the sequence data,

which the RNNs are famous for.

Fig. 6. The Precision and F1 score of multi-class classification.

5 Conclusions and Future Work

This work compared the four most widely used DNNs, basic CNN, inception
architecture CNN, GRU and LSTM in deep learning based intrusion detec-
tion systems. From the above experiments, we conclude that CNNs are better
for binary classification (normal/attack) and RNNs perform better in detecting
some sophisticated attacks for multi-class classification tasks. This gives a guid-
ance of DNN selection for researchers studying deep learning based intrusion
detection systems.
Two problems require further study in future work. The first involves the
influence of hidden size on CNNs and DNNs when applied in intrusion detec-
tion. The second problem involves the influence of different data preprocessing
methods for intrusion detection systems.

Acknowledgement. This research work is supported by National Natural Science

Foundation of China under grant number 61702539 and 60970034.
 Comparative Study of CNN and RNN for Deep Learning 169

References
1. Liu, X., Yin, J., Wang, L.: An adaptive approach to learning optimal neighborhood
kernels. IEEE Trans. Syst. Man Cybern. Part B Cybern. A Publ. IEEE Syst. Man
Cybern. Soc. 43(1), 371–384 (2012)
2. Ming, Y., Zhao, Y., Wu, C., et al.: Distributed and asynchronous stochastic gra-
dient descent with variance reduction. Neurocomputing 281, 27–36 (2017)
3. Lecun, Y., Bottou, L., Bengio, Y.: Gradient-based learning applied to document
recognition. Proc. IEEE 86(11), 2278–2324 (1998)
4. Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)
5. Yin, W., Kann, K., Yu, M., et al.: Comparative Study of CNN and RNN for Natural
Language Processing (2017)
6. Singh, J., Nene, M.J.: A survey on machine learning techniques for intrusion detec-
tion systems. Int. J. Adv. Res. Comput. Commun. Eng. 2(11), 4349–4355 (2013)
7. Kishorwagh, S., Pachghare, V.K., Kolhe, S.R.: Survey on intrusion detection sys-
tem using machine learning techniques. Int. J. Comput. Appl. 78(16), 30–37 (2013)
8. Qiu, C., Shan, J.: Research on intrusion detection algorithm based on BP neural
network. Int. J. Secur. Appl. 9(4), 247–258 (2015)
9. Planquart, J.P.: Application of neural networks to intrusion detection. Sans Insti-
tute (2001)
10. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep con-
volutional neural networks. In: International Conference on Neural Information
Processing Systems, pp. 1097–1105. Curran Associates Inc. (2012)
11. Mikolov, T., Yih, W.T., Zweig, G.: Linguistic regularities in continuous space word
representations. In: HLT-NAACL (2013)
12. Dainotti, A., Pescape, A., Claffy, K.C.: Issues and future directions in traffic clas-
sification. IEEE Netw. 26(1), 35–40 (2012)
13. Wang, W., Zhu, M., Zeng, X., et al.: Malware traffic classification using convolu-
tional neural network for representation learning. In: International Conference on
Information Networking, pp. 712–717. IEEE (2017)
14. Jia, F., Kong, L.Z.: Intrusion detection algorithm based on convolutional neural
network. Beijing Ligong Daxue Xuebao/Trans. Beijing Inst. Technol. 37(12), 1271–
1275 (2017)
15. Qian, T., Wang, Y., Zhang, M., et al.: Intrusion detection method based on deep
neural network. Huazhong Keji Daxue Xuebao 46(1), 6–10 (2018)
16. Torres, P., Catania, C., Garcia, S., et al.: An analysis of Recurrent Neural Networks
for Botnet detection behavior. In: Biennial Congress of Argentina (ARGENCON),
pp. 1–6. IEEE (2016)
17. Yuan, X., Li, C., Li, X.: DeepDefense: identifying DDoS attack via deep learning.
In: 2017 IEEE International Conference on Smart Computing (SMARTCOMP),
pp. 1–8. IEEE (2017)
18. Sheikhan, M., Jadidi, Z., Farrokhi, A.: Intrusion detection using reduced-size RNN
based on feature grouping. Neural Comput. Appl. 21(6), 1185–1190 (2012)
19. Yin, C., Zhu, Y., Fei, J.: A deep learning approach for intrusion detection using
recurrent neural networksl. IEEE Access 5, 21954–21961 (2017)
20. Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: IEEE
Conference on Computer Vision and Pattern Recognition, pp. 1–9. IEEE Computer
Society (2015)
21. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8),
1735–1780 (1997)
170 J. Cui et al.

22. Cho, K., Van Merrienboer, B., Bahdanau, D., et al.: On the properties of neural
machine translation: encoder-decoder approaches. Computer Science (2014)
23. Shiravi, A., Shiravi, H., Tavallaee, M.: Toward developing a systematic approach
to generate benchmark datasets for intrusion detection. Comput. Secur. 31(3),
357–374 (2012)