(IJCSIS) International Journal of Computer Science and Information Security,Vol. 9, No. 1, 2011
ANNaBell Island: A 3D Color Hexagonal SOMfor Visual Intrusion Detection
Chet Langin, Michael Wainer, and Shahram Rahimi
Computer Science DepartmentSouthern Illinois University CarbondaleCarbondale, Illinois, USA
Abstract
—
Self-Organizing Maps (SOM) are considered by manyto be
black boxes
because the results are often non-intuitive. Ourresearch takes the multidimensional output from a successfulintrusion detecting SOM and displays it in novel full color and 3Dformats, with landscape features similar to an island, that assistin understanding the SOM results. This paper describes thevisual data mining from the map and explains the methodology inobtaining the full color and 3D maps.
Keywords: Data Mining; Forensics; Intrusion Detection; Modeling; Self-Organizing Map (SOM); Visualization.
I.
I
NTRODUCTION
Self-Organizing Maps (SOM) have been researched foryears as being possible methods of intrusion detection. SOMcan display multidimensional data in lower dimensions, but theresults are often not intuitive, resulting in SOM sometimesbeing called a
black box
method, meaning that the innerworkings are not visible. Security technicians appear to bereluctant to use methods that they do not understand.SOM methods are actually programmed by design and thecreators know exactly what is
inside the box
. The resultsmystify many technicians, though, resulting in the
black box
epithet. Our visual approach attempts to present the output of asuccessful SOM intrusion detector in a way that is morecomprehensible to people that need to understand how this typeof intrusion detection works.Compare SOM to a hound dog with a good sense of smell.One knows when the dog uses this sense of smell to findsomething, even if the exact smell is not known. Likewise withSOM, the method can be successfully used, even if the innerworkings are not exactly understood by the technicians. Thevalue of our research is that it can help to convince techniciansto use SOM as a valid means of intrusion detection instead of dismissing it as being something that cannot be understood,thus helping to find more intrusions.Previously existing methods of showing SOM in hexagonalformats are discussed in Section II, Related Works. Thebackground of our research leading up to this paper isexplained in Section III, Background. How the full color and3D maps were created is given in Section IV, Methodology,and Section V is the Conclusion.II.
R
ELATED
W
ORKS
A.
Intrusion Detection Development
Amoroso [1] said intrusion detection is the process of identifying and responding to malicious activity targeted atcomputing and networking sources. Applied intrusiondetection was first notably methodized in 1986 by Denning [2].One of the first published systems was reported by Lunt [3] inthe late 1
980’s and was called the
Intrusion Detection ExpertSystem (IDES). It used expert systems and statistics.The following papers summarized the development of computational intelligent methods in intrusion detection. Theuse of soft computing methods in intrusion detection was notedby Garcia [4] in 2000. A comprehensive survey of intrusiondetection systems was written by Lazarevic [5] in 2005. Acomprehensive summary of unsupervised learning algorithmsfor intrusion detection systems was written by Zanero [6] in2008. The state of the art of using soft computing methods forintrusion detection was written in 2010 by Langin [7].
B.
Using Visual SOM for intrusion detection
The idea for Self-Organizing Maps was developed over aperiod of years by Kohonen starting in 1976 with his currentform conceived in 1982 [8]. (See Kohonen [8] for detailedinformation about the SOM.) SOM was suggested as apossible method for intrusion detection in 1990 by Fox [9].Graphical representations of SOM are data mining in thesense that the multidimensional data needs to be organized anddisplayed in ways that are clearer and can be interpreted.Fig. 1 shows an early method of visually displaying theinformation contained in a SOM. It is a 4x4 sample cutout of an 8x8 SOM from Girardin [10] in 1998 which was trainedwith firewall logs. Each SOM node is represented by a squarewhich is subdivided into four triangular parts with colors andtextures to indicate characteristics of that node, resulting in a non-intuitivecryptic display. (A key was notprovided for the colors and textures).An alternative layout from the samepaper labels each node with an acronymof its primary characteristic, such as httpor udp. For example, the upper leftnode in Fig. 1 was subsequently labeled
Figure 1,Pre-Hexagonal
1http://sites.google.com/site/ijcsis/ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 9, No. 1, 2011
with http as being the primary characteristic. See that paper forthe entire graphic and an explanation of it.Fig. 2 shows an 11x3 sample cutout of an 18x14 SOM fromHoglund [11] in a hexagonal format representing userbehaviors such as CPU times, characters transmitted, andblocks read in a U-matrix display, meaning that every otherhexagon is a node (marked in Fig. 2 with either a dot or anumerical label) and that the intervening hexagons are in a greyscale indicating the distances between the neighboring nodes,darker meaning a larger distance and lighter meaning a closerdistance. The labels indicate a user number and the number of Best Matching Node (BMN) hits in a node for that user. Forexample, 127_8 means that User 127 had 8 BMN hits on thenode with that label. A single user as reported in this paper canhave hits on nodes in numerous areas of the map. Thehexagons provide better representation than a standard 2Dlayout, but the rectangular layout of the hexagons limits thispotential. The U-matrix display is an advantage in that itvisually highlights clusters of nodes. The researchers on thisproject probably have a good idea of the characteristics of various clusters, but these characteristics are not readilyapparent from the displayed map. See that paper for the entiregraphic and explanation of it. Rectangular U-matrix hexagonalmaps were also used by Cho [12] in 2002.Fig. 3 is a sample cutout of a SOM from Kayacik [13] in2003 based on network traffic where each hexagon is a nodeand the amount of filling in the hexagon represents how manyBMN hits the corresponding node has (the more hits, the largerthe filling). This can create different patterns for differenttypes of traffic, attack vs. normal traffic, for example. See thatpaper for the entire graphic and explanation of it. A similarhistogram map was used by Yeloglu [14] in 2007. This type of map produces useful visual patterns, but does not indicatedistances between nodes nor characteristics of nodes.Fig. 4 is a sample cutout of a U-matrix SOM from Kayacik [15] in 2006 which has been labeled with acronyms and withboundaries drawn to enclose clusters. MHP, for example,stands for
multihop
, and is in a region in this cutout called
host-based attack group
. See that paper for the full graphic and anexplanation of it. This type of map provides more informationthat previous ones, but is still somewhat cryptic.III.
B
ACKGROUND
This research evolved from a one dimensional SOM, nowcalled 1D ANNaBell, reported by Langin [16 and 17]. This 1DANNaBell has discovered numerous real life instances of malicious network traffic, being the first self-trainedcomputational intelligence to find feral malware, as far as theauthors know, on March 29, 2008, and is still in productionafter more than two years.The 1D ANNaBell was only conceptually a map becausethere was very little visually, just a jagged line, to observe. Aparticular jag in the line represented the BMN only for the IPaddresses of two local computers infected with a certain kind of bot. This node was used to find additional infected computers.The SOM
hound dog
had the scent, but it was not clear totechnicians what the scent was---thus, the
black box
effect.So 1D ANNaBell was redesigned, using the same data, as ahexagonal map with the intent of producing something visualwhich would aid technicians in understanding the SOMprocess. Some of the methodology, using grey scale, for thishexagonal ANNaBell was described by Langin [18 and 19].This paper continues the methodology by showing howcolorization influenced the map, and this paper also shows themap as a 3D
island
. Look ahead to Fig. 14 for the full colormap and Fig. 23 for the 3D island to see where this is leading.The source data for ANNaBell Island is from firewall logsand is in the form of a six dimensional vector for each local IPaddress---these are the pertinent features, given here as areference for the rest of this paper:1
tot_norm:
Total normalized
. The total number of logentries in a 24 hour period, normalized. The lowestnumber of entries in the source data for a local IPaddress was 0 and the highest number was 2,020,349.These counts were normalized to a range of 0 to 1.2
src_rat:
Source ratio
. The ratio of unique source(external) IP addresses to the total number of logentries.3
port_rat:
Port ratio
. The ratio of unique destination(local) ports to the total number of log entries.4
lo_norm:
Lowest port normalized
. The lowestattempted destination (local) port, normalized from 0to 1, with the lowest possible port being 0 and thehighest possible port being 65,535.5
hi_norm:
Highest port normalized
. The highestattempted destination (local) port, normalized from 0to 1, with the lowest possible port being 0 and thehighest possible port being 65,535.6
udp_rat:
UDP ratio
. The ratio of UDP network traffic to all network traffic.For example, a local IP address with 1,548 log entries in a24-hour period, from 139 external IP addresses, directed at 58local ports, from Port 22 to Port 61,123, with 1,345 of the logentries being for UDP traffic would have a vector of 0.000766204, 0.089793282, 0.0374677, 0.000335698,
Figure 3,Histogram
Figure 4,AcronymsFigure 2, U-matrix
2http://sites.google.com/site/ijcsis/ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 9, No. 1, 2011
0.932677195, 0.868863049.Over 6,000 (out of 65,536) IP addresses on our localnetwork had no entries in the input data and these were givenvectors of 0, 0, 0, 0, 0, 0. This was enough IP addresses towarrant their own node in the SOM, and so a special node wascreated in the SOM with the vector 0, 0, 0, 0, 0, 0. Since thisvector represents the origin in a graph of multidimensionalspace, this node was called the
Origin
. All other node vectorsin the SOM were created with random vectors (as describedshortly).Fig. 5 shows how a meta-hexagonal layout was usedinstead of a rectangular one because this would allow the SOMnodes to spread out in more directions. It is a large hexagonmade of 919 smaller hexagons, with each smaller hexagonrepresenting a node on the SOM. Thus, there is a one-to-onerelationship between small hexagons and nodes in this layout.The nodes have numbered labels in a spiral fashion from thecenter towards the edge. The node with the largest numberedlabel is Node 918 and is located at the very top. The
island
in
ANNaBell Island
refers to this meta-hexagon. Other parts of this paper will refer to other features in Fig. 5 later.Fig. 6 is an enlarged cutout from the center of Fig. 5 andshows how the smaller hexagons, each representing a SOMnode, were labeled inside the meta-hexagon. Node 0, theOrigin, was placed in the center and the other numbersincreased in a clockwise spiral from the Origin. Hexagons 1-6in yellow indicate nodes with a distance of 1 from Node 0.Likewise, hexagons 7-18 in blue indicate which nodes are adistance of 2 from Node 0 and hexagons 19-36 in greenindicate which nodes are a distance of 3. (The colors yellow,blue, and green in this graphic are not related to how thesesame colors are used in other graphics in this paper.) Forcomparison, these nodes are a distance of 1 from Node 10: 23,24, 25, 11, 2, and 9; and, these nodes are a distance of 2 fromNode 3: 1, 9, 10, 25, 26, 27, 28, 29, 14, 15, 5, and 6.Some 918 random vectors were created and sorted by theirEuclidean distance from the Origin. The closest vector inmultidimensional space to Node 0 was assigned to Node 1, thesecond closest to Node 2, the third closest to Node 3, and soforth up to Node 918, which then had the vector which wasfurthest away from Node 0 in multidimensional space.The reason that these vector assignments were made to thenodes in this sorted order was to speed the training time of theSOM by placing at least some of the neighboring nodes inmultidimensional space closer to each other in the SOM.Node movement in multidimensional space was monitoredduring the SOM training and the training was terminated whenthe movement stabilized after approximately a week of processing. Referring again to Fig. 5, the Origin moved fromNode 0 in the middle to Node 850 in the lower right (yellow).The node furthest from the Origin changed from Node 918 atthe top to Node 827 in the upper right (blue arrow and redasterisk). The Best Matching Nodes for the two local bot IPaddresses moved from various areas of the map to Nodes 819and 820 in the upper right (red). (Note that the SOM did notknow these were the IP addresses of the bots during thetraining.) Nodes 819 and 820 were also the BMN for other IPaddresses in addition to the IP addresses for the bots. (Thecolors red, yellow, and blue in Fig. 5 have no relation to howthese same colors are used in other graphics in this paper.)A couple of specific issues and a general issue arose as aresult of this training. One specific issue was that 1DANNaBell did a better job mathematically of isolating the bots(even though ANNaBell Island provided more visualinformation). Can the hexagonal method be refined to produceas good alerts as the 1D method? The second specific issuewas how to represent that nodes (hexagons) 827 and 850 werefurthest apart in multidimensional space when they were notfurthest apart on the meta-hexagonal island? The general issuewas how does one extract other meaningful information fromthe island? Attempts to answer these questions sparked themethodology reported on below.
Figure 5, Meta-Hexagonal LayoutFigure 6, Node Label Numbering Scheme
3http://sites.google.com/site/ijcsis/ISSN 1947-5500
Search History:
Result 00 of 00
00 results for result for
p.
ANNaBell Island: A 3D Color Hexagonal SOM for Visual Intrusion Detection
Self-Organizing Maps (SOM) are considered by many to be black boxes because the results are often non-intuitive. Our research takes the multidimensional output from a successful intrusion detecting SOM and displays it in novel full color…
(More)
174
Reads
0
Readcasts
0
Embed Views
More From This User
Notes
Load more
