ADAPTIVE SECURITY IN COMPUTER NETWORKS

by

Benfano Soewito

Master of Science in Electrical and Computer Engineering

W
Southern Illinois University, 2004

IE
EV
PR

A Dissertation
Submitted in Partial Fulfillment of the Requirements for the
Doctor of Philosophy Degree in Electrical and Computer Engineering

Department of Electrical and Computer Engineering

in the Graduate School
Southern Illinois University Carbondale
August, 2009
 UMI Number: 3372567

INFORMATION TO USERS

The quality of this reproduction is dependent upon the quality of the copy
submitted. Broken or indistinct print, colored or poor quality illustrations and

W
photographs, print bleed-through, substandard margins, and improper
alignment can adversely affect reproduction.
IE
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
copyright material had to be removed, a note will indicate the deletion.
EV
PR

______________________________________________________________

UMI Microform 3372567

Copyright 2009 by ProQuest LLC
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
_______________________________________________________________

ProQuest LLC
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106-1346
 DISSERTATION APPROVAL

ADAPTIVE SECURITY IN COMPUTER NETWORKS

By

Benfano Soewito

A Dissertation Submitted in Partial

W
Fulfillment of the Requirements
IE
for the Degree of

Doctor of Philosophy
EV
in Electrical and Computer Engineering
PR

Approved by:

Dr. Ning Weng, Chair

Dr. Nazeih M. Botros

Dr. Haibo Wang

Dr. Ramanarayanan Viswanathan

Dr. Garth Crosby

Graduate School
Southern Illinois University Carbondale
June 19, 2009
 AN ABSTRACT OF THE DISSERTATION OF

Benfano Soewito, for the Doctor of Philosophy degree in Electrical and Computer

Engineering, presented in June 19, 2009, at Southern Illinois University Carbondale.

TITLE: ADAPTIVE SECURITY IN COMPUTER NETWORKS

MAJOR PROFESSOR: Dr. Ning Weng

Even-changing attacks require computer networks to continuously adapt to provide

W
confidentiality, availability, and integrity for diverse system connected through them. If

network security system could be dynamically reconfigured for new attacks, techniques
IE
and algorithms, computer networks would be able to provide better protection. The key

challenge to achieve this adaptability is due to the lacking adaptive framework which can
EV
simultaneously consider underlying hardware platform, algorithms processing

complexity, and integrating them together.

This dissertation presents a adaptive framework for Network Intrusion Detection
PR

System with the capability to detect unknown attacks, reconfigurable to new

technologies, adaptable to network traffic, and easily updating. The key idea is by

adding network analyzers and interdisciplinary several network security techniques into

a system. The network analyzers analyze network traffic to know the characteristic of the
network traffic over the time by analyzing the packets header and payload. Using this
network characteristic information, a suitable intrusion detection system will be

constructed.

In this dissertation, I introduce the implementation of intrusion detection system on

FPGA, multiprocessor, and memory with packet pre-filtering base on part of pattern. In

FPGA, the methodology to construct a high performance string matching engine was
introduced. Various techniques, including multi-threading FSM design, partitioning

ii
FSM, and a novel high-speed FSM interface circuit, are developed to improve the

performance of the string matching circuits. In multiprocessor, I present a workload

mapping methodology for multi-core (Network Processor) security systems using a

comprehensive simulation framework. With this methodology, the network engineers

can be utilize the multi-core without knowledge the complicated of multi-core

programming. In memory-based, I present the technique to reduce the huge memory

requirement for finite state machine using state coding.

Using this framework methodology, network engineers can evaluate run-time

characteristic network traffic and integrate security technologies to NIDS. Without this

W
methodology, it will be very difficult if not impossible.

IE
EV
PR

iii
 ACKNOWLEDGMENTS

I would like to thank Dr. Ning Weng for providing excellent guidance and opportunities

to engage in intense discussions and especially his great assistance and insights leading

to the writing of this dissertation. I am grateful to Dr. Haibo Wang‘s co-supervision for

one part of the research on this dissertation and serving as members in my dissertation
committee. My sincere thanks also goes to Dr. Nazeih M. Botros, Dr. Ramanarayanan

Viswanathan, and Dr. Garth Crosby for being a member of my dissertation committee.
I am very thankful to my parents for providing me this level of education, support

W
and for their immense confidence in me. Finally, I would like to acknowledge Lucas

Vespa and all my friends who helped me directly or indirectly to complete my

dissertation.
IE
EV
PR

iv
 TABLE OF CONTENTS

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Network Application . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Network Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Attack Signature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

W
1.4 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.5 Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6
IE
Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.7 contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
EV
1.8 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1 Signatured based IDS . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

PR

2.1.1 String matching algorithm . . . . . . . . . . . . . . . . . . . . 9

2.1.2 Implementation signatured based IDS . . . . . . . . . . . . . . 10

2.2 Encryption Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3 Optimization IDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.4 Memory Efficient Exact Pattern Matching . . . . . . . . . . . . . . . . 15

2.5 Network Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 Adaptive IDS Design Methodology . . . . . . . . . . . . . . . . . . . . . . . 19

3.1 Analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2 Pre-filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2.1 Filter Architecture . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2.2 Pattern Representatives Selection . . . . . . . . . . . . . . . . 21

v
 3.2.3 Filter Cost Characterization . . . . . . . . . . . . . . . . . . . 22

3.3 High-speed String Matching on FPGA . . . . . . . . . . . . . . . . . . 24

3.3.1 Multi-threading FSM . . . . . . . . . . . . . . . . . . . . . . . 24

3.3.2 Partitioning Algorithm for Multi-Threading FSM . . . . . . . . 29

3.4 Memory Efficient IDS . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.4.1 Deterministic Finite Automata . . . . . . . . . . . . . . . . . . 31

3.4.2 Memory efficient Architecture . . . . . . . . . . . . . . . . . . 33

3.4.3 FSM State Encoding . . . . . . . . . . . . . . . . . . . . . . . 36

3.5 Multiprocessor Workload Mapping Technique . . . . . . . . . . . . . 39

W
3.6 Encryption/Decryption . . . . . . . . . . . . . . . . . . . . . . . . . . 41

4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.1
IE
Multi-cores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

4.1.1 Uni-processor Processing Cost . . . . . . . . . . . . . . . . . 45

EV
4.1.2 Storage Requirement . . . . . . . . . . . . . . . . . . . . . . . 46

4.1.3 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.1.4 Algorithm Represented Using ADAG . . . . . . . . . . . . . . 50

PR

4.1.5 Case Study: IDSs Evaluation in a Router . . . . . . . . . . . . 50

4.2 Pre-filter using part of pattern . . . . . . . . . . . . . . . . . . . . . . . 53

4.3 High Performance IDS on FPGA . . . . . . . . . . . . . . . . . . . . . 58

4.4 Memory-efficient for exact string matching . . . . . . . . . . . . . . . 64

4.4.1 Architecture Validation . . . . . . . . . . . . . . . . . . . . . . 64

4.4.2 System Throughput . . . . . . . . . . . . . . . . . . . . . . . . 65

4.4.3 Storage Requirements . . . . . . . . . . . . . . . . . . . . . . 66

4.5 Encryption/Decryption . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

5.1 Multi-cores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

5.2 Pre-filter using part of pattern . . . . . . . . . . . . . . . . . . . . . . . 75

vi
 5.3 High Performance IDS on FPGA . . . . . . . . . . . . . . . . . . . . . 76

5.4 Memory-efficient for exact string matching . . . . . . . . . . . . . . . 76

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Vita . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

W
IE
EV
PR

vii
 LIST OF TABLES

3.1 Transition State Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.2 Clustering Group Graph . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.3 Encoding for each Group . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.4 State Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.1 Packet Byte-Average . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4.2 Storage Requirement in Best and Worst Cases (byte) . . . . . . . . . . . . 46

4.3 Router using Different Matching Algorithm . . . . . . . . . . . . . . . . . 53

W
4.4 False Positive Special Snort Rules . . . . . . . . . . . . . . . . . . . . . . 54

4.5 Filter Processing Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

4.6
IE
Empirical Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

4.7 Number of Patterns Matched with Different Trace in Defcon10 . . . . . . . 58

EV
4.8 Comparison Memory Efficiency . . . . . . . . . . . . . . . . . . . . . . . 65

4.9 Energy, Delay and Storage Requirements for Encryption . . . . . . . . . . 71

4.10 Energy, Delay and Storage Requirements for Encryption . . . . . . . . . . 72

PR

viii
 LIST OF FIGURES

1.1 Network Attack Pattern Increase from 2000 to 2008. . . . . . . . . . . . . 4

1.2 Analyzer block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.1 Approximate Packet Filter . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2 Finite State Machine Diagram: SHE, HERS and HIS . . . . . . . . . . . . 24

3.3 Conventional FSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.4 Pipelined FSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.5 Multi-threading FSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

W
3.6 Levelization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.7 Sample graph partitioned into two threads . . . . . . . . . . . . . . . . . . 31

3.8
IE
FSM Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.9 Group Graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

EV
3.10 Simulation Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.11 Encryption/Decription Experiment Block Diagram . . . . . . . . . . . . . 41

4.1 Processing Cost Depending on Packet Size and Rule Size. . . . . . . . . . 47

PR

4.2 Memory Access Depending on Packet Size and Rule Size. . . . . . . . . . 48

4.3 Storage Requirement Depending on Packet Size and Rule Size. . . . . . . 49

4.4 Minimum Pattern Length Impacting Matching Algorithm Processing Cost . 49

4.5 Annotated Directed Acyclic Graphs (ADAGs) for Workload Applications. 51

4.6 Sample Applications of a Router. . . . . . . . . . . . . . . . . . . . . . . 52

4.7 Increasing Throughput Normalized to Non Filter Approach . . . . . . . . . 56

4.8 Percentage of Representative to Cover All Patterns . . . . . . . . . . . . . 56

4.9 False Positive Rate Decreasing with Representative length Increasing . . . 57

4.10 FSM Clock Frequency Versus Number of Threads. . . . . . . . . . . . . . 60

4.11 Interconnect Delay with Different FSM Sizes. . . . . . . . . . . . . . . . . 61

4.12 DFF Utilization in Multi-threading FSMs. . . . . . . . . . . . . . . . . . . 62

ix
4.13 FPGA Resource Utilization for Different FSM Design. . . . . . . . . . . . 63

4.14 Delay of FSM Input Path. . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

4.15 Storage Requirement Comparison with Related Work . . . . . . . . . . . . 66

4.16 Energy Used for Encryption Relative to RC6 . . . . . . . . . . . . . . . . . 67

4.17 Energy used for decryption relative to RC6 . . . . . . . . . . . . . . . . . 68

4.18 Number of instructions per byte in encryption processing . . . . . . . . . . 69

4.19 Number of instructions per byte in decryption processing . . . . . . . . . . 69

4.20 Number of memory accesses per byte in encryption processing . . . . . . . 70

4.21 Number of memory accesses per byte in decryption processing . . . . . . . 70

W
4.22 Storage requirement in encryption processing . . . . . . . . . . . . . . . . 71

4.23 Storage requirement in decryption processing . . . . . . . . . . . . . . . . 72

IE
EV
PR

x
 CHAPTER 1

INTRODUCTION

The information technology based on internet has been used widely all over the world.
The internet has evolved to include a simple application such as information system to

complex applications such as e-commerce, teleconference, internet banking, and voice

over internet. With an explosive increase of internet application, unfortunately it has also
provided the means for malicious attacks on information systems. Therefore, it is

important to balancing the security system in the future network. The future network

W
security system should have flexibility which is easy to update, adaptable to network

traffics, changeable to the attack, reconfigurable to the new security technology, can
IE
detect new attacks, and energy efficient. The question is how do we construct it.

However, to construct the NIDS that considers the requirements above is very difficult
EV
for network engineer. This dissertation will present a methodology framework to address

this challenge.
PR

1.1 NETWORK APPLICATION

A Network application is a function that utilize the Internet or other network

hardware infrastructure to perform useful work. Below is a list of network applications

as examples.

• Firewall. An application that provides security according to a policy defined by an

administrator

• Virtual Private Network (VPN). An application that uses encryption to provide

private communication over the Internet

• Network Address Translation (NAT). An application that allows multiple

computers at a site to share a single, globally valid IP address

1
 • Content Delivery Networks. An application that overcomes the inadequacies of

existing networks by introducing intelligence into the network in order to enhance

performance of services and delivery of contents to the consumer

• IPv4-radix. An application that performs RFC1812-compliant packet forwarding

and uses a radix tree structure to store entries of the routing table

• Flow Classification. An application that classifies the incoming packets into the
flows based on packet protocol fields

• IPSec Encryption. An application that encrypts the packet payload using the

W
Rijndael algorithm

• Top-hashed subtree-replicated anonymization (TSA). An application that

IE
performs the prefix-preserving anonymization of IP address of the packet
EV
• Packet Intrusion Detection. An application that searches the whole packet for

multiple patterns according to the rules specified by an administrator

PR

1.2 NETWORK TRAFFIC

Network traffic is composition of network application flow. By monitoring network

traffic in real-time, it will allow us to quickly and easily examine particular network and
take a necessary action if needed. Monitoring network traffic base on header and

payload packet. Packets are the fundamental unit of information transport in all modern

computer networks, and increasingly in other communications networks as well. They

can be a fixed size or variable sizes, depending on the system. Regardless of their size,

each packet consists of three main parts: a header, the body, also called the payload, and
a trailer. Below are some of the fields in the packet that can be inputs to monitor network

traffic.

2
 • Version. The first header field in an IP packet is the four-bit version field. For

IPv4, this has a value of 4 (hence the name IPv4).

• IP Address. The source and destination address of the computer that sent and

receive packet frames. (This address is a unique hexadecimal (or base-16) number

that identifies that computer on the network.)

• Port Number. The channel for network communication. Port numbers allow
different applications on the same computer to utilize network resources without

interfering with each other.

W
• Total Length. This 16-bit field defines the entire datagram size, including header

and data, in bytes. The minimum-length datagram is 20 bytes (20-byte header + 0

IE
bytes data) and the maximum is 65,535 the maximum value of a 16-bit word. The
minimum size datagram that any host is required to be able to handle is 576 bytes,
EV
but most modern hosts handle much larger packets.

• Protocol. The standard rules for communication between two devices. This field
defines the protocol used in the data portion of the IP datagram. The Internet
PR

Assigned Numbers Authority maintains a list of Protocol numbers and were

originally defined in RFC 790

• Data. Data is packet payload. This is the last field that is not a part of the header

and, consequently, not included in the checksum field. The contents of the data

field are specified in the protocol header field and can be any one of the transport

layer protocols.

1.3 ATTACK SIGNATURE

The heart of almost every Intrusion Detection System (IDS) is a string matching

algorithm, which is a very computational intensive task. The basic operation of string

3
 32
%& !'$"
Number of Unique Patterns
Number of Characters

16
8
8 !"#$
78
*5 6
*4 8
3,
)/2
.
*1
0 4
,/
- ).
+,
*
() 2

1                   
                  


 
 
     

 
 


W
Figure 1.1: Network Attack Pattern Increase from 2000 to 2008.

matching is to search for predefined patterns (attack signatures) in the packet. However,
IE
this simple operation can be easily the bottleneck of IDSs. This is caused by three
EV

factors: tens to hundreds Gigabit per second traffic rate, hundreds to thousands possible

attack patterns and the starting position of predefined pattern is probabilistic, hence it is
PR

necessary to scan every byte of a packet. I utilize SNORT for the attack signature

database.

1.4 CHALLENGES

The problem with current intrusion detection system is that it does not efficiently
utilize the available signature database, and the updating yet manually done by a network
system administrator. For example, in a network that does not have a UDP packet, we do

not need to include the security rule for UDP packet into the NIDS. If in some other time
the characteristic of network changed and we find the UDP packet in the network then
the security rule for UDP need to be included in NIDS. Basically, we only included the

security rule in NIDS base on the characteristic of network traffic and automatically add

4
or take out the security rule base on the characteristic of the network traffic. This concept

will reduce workload and increase the performance of NIDS. There are several challenge
to solve this problem.

• Number of Signature. The number of signature attack growth very fast, and

many new signature attack frequently have to be added to the database . Figure 1.1
shows how the size of the Snort rule database [58] has grown over time. It can be

seen that the number of unique patterns is increasing dramatically, and we are
expecting this trend will continue. Therefore, NIDS must easily update to new

attack patterns.

W
• Dynamic Network Traffic. The characteristic of network traffic will not the same

between one and others networks. One of the most key issue in network security is
IE
the characteristic of network traffic never stay the same over the time.
EV
• Hardware Reconfigurable. Beside many new signatures have to be added, also

many new optimization algorithms or new technologies have to adopt by NIDS.

Therefore, the NIDS must have capability to easily to adopt the new security
PR

technologies.

Moreover, it is very difficult to integrate and dynamically change the intrusion

detection system base on network traffic.

1.5 OBSERVATIONS

In this dissertation, I present a framework methodology for NIDS by adding network

analyzer and integrate several security techniques as shown in figure 1.2 This can be
obtained by the following observations:

• Regularity of Network Traffic. The behavior network traffic in particular

network is normally only has small variation. This make it shorter time to

reconfigure the system.

5
 • Available Reconfigurable Hardware. There are several promising reconfigurable

hardware for NIDS: FPGA, Network Processor, and memory-based. The

performance of NIDS, not only depend on the hardware itself, but also depend on

managing to fully utilize them.

1.6 PROBLEM STATEMENT

Given a dynamically network traffics, a database of signature, reconfigurable

hardware, and a performance target.

• The methodology for dynamic workload characterization.

W
• The methodology to evaluate string matching algorithms on multi-core.
IE
• The hardware accelerator for string matching.
EV
1.7 CONTRIBUTION

The main contribution is a framework methodology to build NIDS with capability

easy to update, adaptable, and reconfigurable. The major contribution include:

PR

• Run-time Workload Adaption. The system is able to adapt the network traffic

variation by re-mapping the workload. Also, the system can easily update new

attack pattern for signature-based intrusion detection system.

• Automatic Methodology Workload Mapping. A methodology to profiling and

characterize the network applications, mapping network applications onto network

processor in automated fashion.

• Pattern Matching Accelerator. A novel scheme to accelerate pattern matching

by adding a pre-filter to the exact pattern matching engines.

6
 Security
Technologies
Attacks Packets

Attack pattern Traffic Analyzer Application

analyzer analyzer

Part of signature, String matching,

DFA, etc encryption, etc

Mapping Mapping

Hardware: NP, Memory, FPGA, ASSIC, TCAM, etc

W
Performance:
Throughput, memory
size, delay

IE
Figure 1.2: Analyzer block diagram
EV

1.8 ORGANIZATION

The remainder of this dissertation is organized as follows. Chapter 2 discusses

PR

related work on signatured based IDS, optimization IDS, memory efficient for pattern

matching, and network processor. Chapter 3 describes the proposed framework

methodology in this dissertation. Chapter 4 shows the results and few observations made
in this work and finally Chapter 5 presents conclusion.

7
 CHAPTER 2

RELATED WORK

NIDS have been studied in different forms since Dennings classic statistical analysis of
host intrusions [38]. The main task of NIDS are to detect and thwart attacks in computer

systems and networks. The traditional NIDS rely on signature detection to find attack
pattern in incoming packets. A string matching is one of technique to detect signature in
the packet. The traditional NIDS can not detect new patterns attack or unknown patterns.

In order to prevent attack from unknown pattern, the modern NIDS include the anomaly

W
detection to detect unknown signature.

In this chapter I have reviewed some of previous works in detection signature,

IE
anomaly detection, implementation and optimization techniques, and memory utilization

for pattern matching.

EV

2.1 SIGNATURED BASED IDS

String matching has been one of the major operations in network processing
PR

applications: flow classification, IP address lookup, Intrusion Detection Systems

(signatured based IDSs), layer-7 switches, packet filtering, transformation systems and

etc. Intrusion detection system (IDS) are more difficult than other applications in that it

has to scan the packet payload and packet header. In IDS, the basic operation of string
matching is to search for known attack (predefined patterns) in the packet. However, this
simple operation can be easily the bottleneck of IDSs. This is caused by three factors:

tens to hundreds Gigabit per second traffic rate, hundreds to thousands possible attack

patterns and the starting position of predefined pattern is probabilistic, hence it is

necessary to scan every byte of a packet. Therefore it is vital to design an efficient string
matching based IDSs.