CHAPTER 19

System Security
LEARNING OBJECTIVES
Upon completion of this chapter, you should be able to
LO1 Memorize a brief descriptionof the system and
.s Otline a detailed define the
components of the system
malicious programs
overview of buffer
overflows and show howthey may be
exploited for writing
LO3 List and explain the
commonly known malicious programs
LO4 Relate to the concept of
technologies Intrusion Detection Systems (1DS) and explain the
types of IDS
LOS Define firewalls and explain their
different architectures with their respective advantages and
disadvantages

he development of computer networks has lead to a world full of

The ubiquitous nature of networks has provided human society withconnected computational resources.
the capability of solving probiems
In a distributed but yet collective manner. However, the boon of
communication is accompanied with
e concern for security. For an end to end security, it is thus not only sufficient to secure the network, but
bO mperative to consider threats which may exist because of the loop-holes existing in the computational
systems. Building Correct and fault-free systems is an extremely difficult task. With the complexity of modern
Computers and systems, abugfree design is impossible. Designers try to ensure that the bugs do not hamper
normal functionality. Thus, although we know systems have bugs, they are mostly adequate for normal
functions. However, security has more stringent requirements. Asingle bug may compromise all the security
Measures adopted, as a system is as strong as its weakest defence. In spite of theimportance of security, it has
Temained an additional.Along
even moreechallenging. andthe
featurewith after thought. This approach makes the job of the security designers
an needed security,the designers are pressed with the additional challenge
of maintaining transparency to the user and also not increasing the cost of the service signiticantly.

LO1 DESCRIPTION OF THE SYSTEM

environment
A the communications
Oversystwhich
em is a vague entity of the computing and
that comprisesthe totality boundary demarcates between the protected
and the
system
the developers have some control. A
 Security
Cryptography and Network
526

system and the

components of a system, It detines the interface between the outside
unprotected
long as the detinition of the boundary is not changed. system security principles try to provide world.
A
informationwithin the system from threats
that originate in the
canalso be divided into
world
two
outside
distinct
the boundar.
categories. security
of
The componentsinside the system
Security Relevant These components are crucial to the security. A malfunction or penetration in
these components can leadto security violations. The Operating System(0S) and the computer hardware its

examples.
J Others These are obËects that the system controls and protects. Examples are programs (note no
processes). data, terminals, modenms, etc.
components and others is called
The line of demarcation between the security relevant
perimeter.
system sors
A common misconception which must be avoided is that encryption provides
Encryption, as we have seen in the previous discussions in the book, provides data-confidentiality, Houo
for system security it is needed to have trust in the security-related components inside the security perimeter
For example, the OS needs to be trusted. If the OS is found to be untrustworthy then the result of an encminti
program which runs on the OS cannot also be relied upon. Hence, encryption alone cannot solve the problem
other principles and methods are necessary.
Users, Trust, and Trusted Systems
A user is a person whose information the system protects and whose access to information is controlled
by the system. A user is in general trusted to keep his secret, often in the form of a password, confidential
from other users who do not have access to the trusted user's documents. In spite of this, a user may be
deceived to manipulate his data in afashion which compromises the security of the system. In order to prote:
against such athreat, atrusted user should be warned when he accidentally gives away valuable intomaton
However. if a user betrays this trust intentionally, then system security cannot handle such a scenario.
The system identifies a user through a unique identifier (ID), which is public information like namt
or account number. The identifier must be unique and unforgetable. The act of associating a user
win
unique identifier is called authentication. The identification number is used by the
system to associale d
process (arunning program) with a user. Trust in systems is built using the techniques of
authentication. Systems in a network authentication as other systems, or programs identification a
impersonating is u
prograns is a serious security threat. Trusted system enforces a given security policy based on this trust
Trusted systems classify programs based on the level of trust on them. Trusted programs are those upon
which the security of the system relies. These programs are responsible for the security of the system.The
outputs of these programs are verified and are believed to be correct even if they interact with untrusted
programs. On the other hand, benign programs are believed to be benign and obtained generally fromtrusted
vendors, like OS providers. They are believed lo be harmless and are trustworthy as long as they ddo notcone
into play with untrusted programs. Untrusted programs are obtained from unknown or untrusted sources,ike
adownloaded file from an
unknown web-page.
LO2 BUFFER OVERFLOW AND
MALICIOUS
Buffer overftow is a commonly known mistake that exist inSOFTWARE
some Cimplementations. These classes ofbugs
are extremely dangerous, as they write past the end of a buffer or array and hence corrup the processsack
Often they change the return
address of a process after a location where
function callto a secret memory
 System Security 527

malcioIscode s planted. This calls for the understanding of these kinds of errors to develop secure coding
principlestolessen these sccurity threats in systems.
Abuffer. often visualized as an array is a contiguous space of related variables of the same data type
C++there are n0 automatiC checks on the buffer, which means a user may write past a buffer. This
Cor
pheomCnonis known
In as buffer overflow. Arrays, like all variables in C, can he either static or dynamic
Stalic variables are allocated at load tme on data segment. Dynamic variables are allocated at run time on
shall diseuss on the
the stack. We overflow of dynamnic buffers, commonly referred as stack-based hbuffer
overflowS,
For exanple. consider the following Ccode snippet as a simple example of buffer overflow.
main( ) {
int int buffer [10]:
buffeI [20] =5;

Most compilers will comp1le the abOve program without errors. This seemingly simple mistake can beto
In order
used by the authors of malicious softwares to trigger their codes and thus compromise security.
dorstand the principle behind these attacks let us look at the organization of the process in memory
text, data and stack. as shown in
ADrocess, which is a program inexecution, is divided into three regions:
binary instructions and read-onlv data.
Eio 19.1. The text region is decided by the programn and is made of uninitialized data region stores
Anveffort to write in this area leads to asegmentation fault. The initialized or
the process is blocked and rescheduled with
static variables. If the available memory is rendered unavailable,
a larger memory space between the
text and stack segments.
Lower
memory
Text
addresses

Data

Higher
Stack memory
addresses

Process memory reqions

Fig. 19.1

containing data, which grows trom higher memory address to

of memory which means that the last object
ASlack is a contiguous block the property of LIFO (Last In First Out),
are the
OWer ones. A stack of object has removed. Two of the most important stack operationselement
object to be while POP removes the last
K'aed on the stack is the first the top of the stack,
adds an element at use stacks to implement
E and the POP. PUSH the last element from the stack. High level languages towards the function
he stack. This removes afunction, the program flow is changed immediately after
call of
procedure or function calls. After thefinishes its task, the control returnstotheinstruction the local variables
used in
being called. But after the function the return address and
the call instruction. The parameters of the function call, called the stack pointer(sp) pointstothe top to
register
the located in the stack. A while the sp is updated by the kernel at run timelike
function
of the are dynamically
stack. The bottom of the stack is a
fixed address,
the function. In general
architectures
required by the local variables of
higher memory address to lower
accommodate for extra space stack grows down,
that is from
Intel, SPARC and MIPS processors, the
 LO3 MALICIOUS PROGRAMs

Malicious programs,as already clarified, are those programs which try to subvert the expected operation of
secured and benign codes. As mentioned in the previous section, buffer overflow attacks give the adversary
the power to change the returm address. Typically, the changed return address points back to the buffer itselt.
where amalicious exploit is written. The exploit couldbe aprogram to run ashell in the victim computer. The
various types of
ability to open up ashell in a target computer, gives in turn the ability to the attacker to run
harnful programs. The two most commonly known categories of malicious programs are worms and viruses.
Although often worms are also referred to as viruses, it would be appropriate to mention at this point the
difference between these two types.

O Worms These are programs that can run independently. It can propagate a full working version of
itself to other machines. The term tries to draw an analogy of these programs with that of parasites which live
inside ahost and uses its resources for its existence. The concept of aworm program that spreads itself among
machines was first mentioned in the classic science fiction, named "The Shockwave Rider' by John Brunner in
1975. Thereafter researchers at Xerox PARC reported their experiments on the worm programs in an article
in the Communications of the ACM.
Interestingly, the initial worm programs were designed to do "good" work in a distributive way. and not
break systems. However as of now, worms are used only as a special category of harmful programs.
 (ryptography ornd Network Security
536

Viruseshee programs on the other hand, cannot run independently.not It requires tho
biological viruses, which are alive themselves,.
run andactivate them)|Thesc are analogous to word VIRUS was used to indiccate
and coupt host cellk. The first time when
the
fiction, named "When Harlie Was
a
One': program infienctvals e a
The first
hut
that
computeT Was by David Gerold in a science
thatformcanal insapefect
1983. He defined computer virus to be. a program
of the term was made bv FredCohen around
itself.
other programs by modifying them toinclude a copy of
adwares
Some other vpes of malicioIs progranns are logic bombs. Trojans. sPywares and

J Logic Bombs 4logic bomb is a malicious program which has typically two parts: payload and
ingger 1he payload tvpically is a malicious piece of code, and the trigger is usually a Boolean the
triggers the malicies code when the condition is satisfied )The trigger Is usually developed hat
conditions like date.
Logie bombs are inseted stealthily into a big program. They often have an objective of causing inancial
harm. Thus, the logic bomb creates violation of security when some external events occur.
Historically. there are several examples of such codes. Recently, a person fired from his job.
alogic bomb in the system such that it will be fired on a particular date after he left the
company. imTheplanted
bomb would delete al! the files inthe system. The man was ultimately detained in jail for several yearsIogcfor
this misdoing. Such examples of logic bombs which fires on a particular time or date. say April 010
known as time bombs.
Once. a program was posted in the USENET news network to make system administration easy. In the
program which was to be compiled and run with root privilege. dangerously there were lines to changeto
root directory and remove all files.

J Trojans (Trojans are malicious programs that perform some harmless activities in addition o
some malicious activities. A Trojan horse is a program with some known or documented effect and some
undocumented or unexpected effects.
Aclassic example is password grabbing programs. Afake login prompt asks the user to enter the
The program then obtains the password and displays 'an error message password.
showing incorrect
actual login prompt is displayed, thus making the user believe that he typed in the password. Then the
he enters the system with correct password, but password incorrectly.Now
meanwhile he has given avway his password.
A propagating Trojan horse or a replicating Trojan horse is an
a copy of itself. One of the carliest examples, called the example of Trojan horse which can create
create an extra copy of itself. animal, was an example of a Trojan which would
J Spyware Spyware is software that is used to
another computerThe information which the spywarecollect information from a computer and transmit it 0
exports the another system
to
is that spywares do not replicate. could be of the same iyp
as done by viruses, but the essential
difference
Trojans, in the fact that the spywares do not deceive the user They are also diferent rou
downloaded when weare viewing some webpage, the but does its malice passively. Spywares olel
Examples of phenomenon being called drive by downloa0
information gathered include the following:
Passwords Akey logger is often used to retrieve the
during login. Ause of a virtual key by the user
Uses the key strokes to board can often be a information typed in as password which
ascertain the content of the typed way to subvert actions of these key loggers
(redit Card Numbers and message.
benefits. Bank Secrets These are special subjects financial
of for their
spywares
 System Security
537

softwareLicense Keys They can be Ised for piracy of

softwares.
Adwares Adwares have much similaity with
objetive is more Irom thc marketngspywares, Ihey are also not self replicating like the
Their
perspective Examples of them could include the
unntentionally depending on
(o inCrcase the salc of some products. They can
the content or wehsite we are
also transmit information which viewing Their
the narkeing pont of view. may he usetul
tom
neCN tew sections, we try to provde an
Inthe overview on the working principles of the more important
ous programs, namely wOrMN and wiruses.
mal
Worms
The worm programs exposedthe security flaws of the standardfunctions provided by UNIX) We shall discuss
some problems specitic to some UNIX utilities. Afamily of routines in the standardC ibrary takes inputs
obufters without checking for bounds. Examples include gets, scanf. fscanf. sscanf and other such routines.
thus be used to overrun buffers unless the user explicitly takes care of the nun.ber of
characters
Careless usage of the routines like sprintf and usage of strcat and strcpy instead of strncat and strncpy may
so overflow buffers. The problem wih these codes is they are functionally correct and seemingly harmless
uowever. the usage ot such codes in netWorking or trusted programs can cause exploits to work by carefully
doctoring inputs.
In order to cireumvent this issue patches were developed. The first measure adopted was to replace all
aach codes by bounded versions.These versions accept values for bounds on their butfer arguments. Next.
l servers and trusted applications should be checked for usage of these bounded versions of codes. For
example. consider the fingerd program which runs as adaemon to service remote requests using the hinger
Drotocol, an utility which allows users to obtain information about other users. The fingerd servers used the
gets call. and thus patches had to be developed for these programs. These revised versions of programs do
buffers without
not make use of the original gets commands and were devoid of function calls which fill
checking for bounds.
program is used
The other way of running a wormis by targeting the sendmail program. The sendmail
operates as a daemon process, the
to route mail in a heterogeneous network. When the sendmail program Transfer Protocol
Simple Message
program "listens on aTCP port 25 in order to deliver mails using
the program. The worm would specify the
(SMTP). The worm workedby exploiting the debugging option in instead of a user address as the recipient
debug command to sendmail and then specify a set of commands casy testing of the complicated sendmail
kept to allow
ofa message. Normally, this is not allowed but it is where the debug option has been disabled. Or, it
of this program have been released
program. New versions
avail of the debug option.
Tcquests the user to enter the root password in order to
attempt to determine the passwords which the users provide. The Unix passwords
Ine worms also The system protected the passwords by controlling
plaintext in a 'password file". However.
Peviously were stored as system administrators and privileged users. which
passwords were visible only to
access, Such that the programming, this practice can be dangerous. There is
an instance
u to unintentional errors or wrong editing the password tile and the
System administralors at MIT were
ured in the early 1960s. when two during login. However due to a soltware error,
both
terminal
daily message that gets printed on every ones tile was printed on every terminal when it was logged in.
the files got swapped andfor atime the passwordnot keep actua! passwords anywhere on the systen. Instead.
does password. The result of the
Order to avoid such problems. UNIX 0 bits with the user
encrypt a block of decrypted (it
UNIX uses a program called crypt( )to file. When a user tries to login, the password is not
encryption is stored in the letclpasswd and compares the newly
the login program takes a block of zeros
Was actually never encrypted). Instead
 Network Security
Cryptography and
538
Iransformed block with the block stored in etepasswd ile. If there is a match the user is allowed in by the

The algorithmthat CIpOnsesis bascd onthe DataEneryption Standard(DES). developed by Nationgl

Svstem
Institutc of Standards and Technologv(NIST), The crypt )functionsttakes the user password as the encryplion
key and uses it to enervpt a64 bit block of 0's. The resulting 64 bit ciphertext is then encrypted again with
the password, the process beingrepeated 25 times. The final 64 bits are unpackedinto a string of |l printable
resist against
ability of DES function to
has lcadtothat
Characters theare
factstored

Adietionarvattack
or adictionarv
in the
that the etcpassud file. The
cryptanalysis,
only way of breaking the UNIX password security isto do a brute force search

attack is launched bythe adversary by recording possible passwords chosen from say English
verified with
by the UNIX crypt function and the
and numbers in afile.
file for any matches.
Then they are encrvpted

Atthe time of design of DES, the sccurity of

56 bits brute force search was considered to be high.
letclpasswd
in todav's world with the progress of VLSI designs and parallel
processors systems have
cracker named COPACOBANA was
Hooweopedver,
been devel
which can quite cfficientlv do such asearch. The recent DES in gate built 2006
by teams from Universities of Bochunm and Kiel, COPACOBANA consists of 120 Field-programmable
arrays (FPGAs) of type XILINX Spartan3-I000 running in parallel. The cost of the machine is
S10,000, which is areduction by afactor of 25 over previous such machines. Since 2007. approximGmbH.
SciEngines ately
aspin-off company of the two project partners of COPACOBANA has enhanced and developed
of COPACOBANA. In 2008,their COPACOBANA RIVYERA reduced the time to break DES to lesssuccessors
than
day. using 128 Spartan-3 5000's.
In order to reduce such attacks, an approach was taken long back, which was to shadow the
passwords.
The hashed passwords (they are not really encrypted) are stored in a file. which is visible to only the su
administrator. and a privileged call performs encryptions and comparisons with an appropriate delay
Additionally, often a threshold is set, which exceeded an alarm is raised by the system.
In addition the password is often "salted" to ensure
that the result of the DES encryption gets changed
The DES salt is a 12 bit number., between 0 and 4095,
When the password is changed with the /bin/passwd which slightly changes the result of the DES function.
the day. The salt is converted into a program, a salt is selected depending on the time of
two-character string and is stored in the letclpasswd
encrypted password. The purpose of the salt is thus that the same file along with the
etclpasswd file and thus increase the complexity of a password can be stored in 4096 wavs in the
Worms can be classified by the primary dictionary attack 4096 times.
following types: method they use for transport. They can be
1. IM worms: divided into the
Worms using instant messaging is
Worms using email as a means ofcalled IM woms.
2. email worms:

3.2
spreading are called email worms.
Viruses
The number of
computer
experience an attack everyviruses
is on the rise.
39 Literature says that a computer
building viruses has evolved. Asseconds. The art of using new connected to the Internel hey
told vulnerabilities
in
computer without permission or above,ofviruses are malicious programs thatsystems anditselfusingandthem for

J Infection
knowledge the user. Acomputer virus has can copy infect a
Mechanism/ three parts:
a copy of the virus,The This part involves how a
exact means through virus
vector of the virus. which the virus spreads isby modifying other code to contain
spreads commonly known as the infection
 System Security
539
Payload This part
Jintenti0nal damage. details what the virus does. hesides spreading The
or Accidental damage may result from unk nown payload may create accidental
systems. hugs in the virus code, etc.
The pavload is triggered on some
J Trigger conditions.
The parts 'pavload and 'rigger are optional
infect. Alogic bomb is atem according to the definition of a virus. By
hast o
devoidofthe infection step. attributed to a program, which has definition, a virus
the trigger and the payload. but is
Apscudocodefor the virus is as shown below:
infect();
if trigger( )
then payload ( ) :

The vius by definition targets a locally accessible target code, which

The virus does not select the same target code always, but includes codes in shared network.
useless. However, it should be kept in mind that
rather varies its target as infecting the
being able to detect whether a code is infected orsame code is
bat the virus can also be detected by an anti-virus. not. means
In next section we will discuss detail Thus this has to be done cleverly.
method adopted by the virus.
classifications of the virus based on the target of the virus and the

3:3 Classification by the Target

The important classes of virus depending on the type of the target
fle infectors and the data file infectors, also called code are the boot sector virus. the executable
macro virus.
O Boot Sector Virus
The basic boot sequence has two important steps after power on the
following:
Primary Boot The ROM based instructions first run, and then after a self test
The boot block is then read, and the control is transferred to the loaded code. identifies the boot device.
Secondary Boot The code loaded during the primary boot loads a program that
device's file system structure. This is called the secondary boot and the understands the boot
operating system kernel is loaded
during this step.
A boot sector virus, or Boot Sector Infector (BS) is a virus that
infects by copying itself into the master
boot block. The content of the former boot block is copied else where in the
disk. Thus, the virus after
completing the infection and other tasks can complete the booting process.
The advantage of a boot sector virus is that it launches betore any anti-virus starts or the
IS loaded. However BSI's are now rare. Machines are not booted with
operating system
boot floppies, and operating systems
prevent writing to the disk's boot block.
Examples of boot virus were Michelangelo and Stone.
 Network Security
ryptogr ophyond
544

(IDS)
INTRUSION DETECTION SYSTEMS ocCurring in a computer system
LO4
Signs of violations 0 compiter seiy moitoiDg
polcies, the events use
acceptable or
policies, or standard securty practices are
Detecton s the Drocess ot
Intrusion
analvzed Intruson P'revention is theDess of detecting the sgns of itrusion and attempting to op he
intrusive eflorts. Collectivell the sIstem is known as Intusion Detection and Prevention System (1bPs
netwurk
inftrastruIcturc of nearly every
DPS have beome a neCssaIN
aditionto the
CCUity organization
types.
dided into three different
Intruders canbe boadh
Masquerador They are vDCallk utsiders from the Irusted users and are not authorized \0 5e he
COmputet STSems. These intuders penetratethe system protection by way of legitimate user acCOunts
MisteasOr Thev are Ivpically insiders and legitimate userS Who aCCeSses resources that theey are no

Or, thev may be authorized but misuses her privileges.

authorized usc.
to Users They can be both insiders and outsiders. These type of intruders gain supervisor
Clandestine

access othe svstem.

4.1 Types of IDS Technologies

An Intrusion Detection System (IDS) is software that automates the intrusion detection process, while the
Intrusion Prevention System (|PS) is software with all the properties of IDS, with the additional feature that
1l stops the intrusions.
monitor and a
The types of IDS technologies are differentiated primarily by the types of events that thev
ways by which the features are achieved. Broadly there are four types of IDS technologies:
Network-based These monitor the network traffic for a segment of the network. It also analvzes the
network and application protocol activity to identify suspicious activity.
Wireless These IDS monitor wireless network traffic. Its analysis is to identify suspicious activities
involving the wireless protocols.
" Network Behaviour Analysis The network traffic is again analyzed to identify threats that create unusual
Iraffic flows, Distributed Denial of Service (DDoS) attacks, malwares and policy violations.
" Host-based These IDS monitors the host and the events
that occur within the host.
4.2 Usage of IDS
Apart from identifying suspicious incidents, there are some
other usages of IDSs also:
ldentifying Security Policy Problems An
policy implementalions. This can include IDScan provide some amount of quality control tor setuythe
network traffic is not blocked by firewall duplicating firewalls also
and that
because of configuration errors. raising
alerts when it sees

detect.
Documenting the Existing Threat to an Organization They maintain logs about the threats that they
Deterring
makes them lessIndividuals from
likely to commilViolating Security Policies The fact that the users are monitored by
IDS.
violations.
Preventive Actions of the
perform security allacks on thelDS The IDS uses several response prevent the attempts t0
systems. Some of techniques to
the ways of how this can be
achieved are by
terminating
 System Security 545

thenetwork connection or user sessions that are being used in the attack, block access to the target from the
IPS
lendingaccouNts or .The preventive step could be a drastic
(arget
host. measure like blocking all accesses to the
importantfunctions of the IPSinvolvethe
Other
TheelDSChangethe Security Environment The
following
IDS changes security
include recontiguration of the network firewalls. environment to stop an atack. This
ould
susspectedto have vulnerabilities. application of patches onto the host
computers which
are
ID. can Change the Contenttof the Attack TheIDS often acts like a
The1DS proxy which does
That meansthey
unpackage the payloads of the request, remove the
headers. normalization.
This step nullifies certain
They often
remove malicious attachments from incoming files and pass the cleaned email to the attacks.
recipient.
43 False Positives and Negatives
DPS technology adopts statistical methods to comprehend the threats to the
companying attribute of false positives and
system. Thus, this has an
negatives. They arise because of the fact that the IDPS cannot
providecomplete and accurate detection. The false alarms are defined as follows:
Calse Positive When the IDPS incorrectly identifies a benign (harmless) activity as
occurred. malicious, a false
nositive is said to have
.Ealse Negative When the IDPS fails to identify a malicious activity, a false negative is said to have
occurred.
I is not possible to eliminate both false positives and negatives, as reducing one of the errors has
a
consequence of increasing the other. It is intuitive that organizations prefer to reduce the false negatives at the
cost of an increased false positive. Thus more malicious events are detected. However, extra work needs to
be done to be sure (or surer) that they are realy malicious. Altering the configurations of an IDPS to improve
the detection accuracies is technically known as tuning of the IDPS.
The malicious activities adopt several evasive techniques to bypass the IDPS. For example, the atack
can encompass changing the encoding of characters, hoping that while the target will be able to understand
the encoding, the IDPS would not. Thus the IDPS needs to take measures to compensate these evasive
techniques. The thumb rule is that if the IDPS is capable to see the activity the same way as the target, then
most of the evasive techniques will fail.

4.4 Intrusion Detection Techniques

Next we highlight the common detection methodologies of the lDPS technology. The classes of detection
methodologies are: signature-based, anomaly-based and stateful protocol analysis. Most of the IDPS uses a
Combination of these techniques to reduce the error of its detections.

J Signature Based Detection Asignature is apatterm that coresponds to aknown threat. Signature
based detection is the process of comparing the signature, which signities aknown threat against the events
email
eoDServed. Examples of signatures could be a telnet attempt with roOt as the username. Or, an
Ua subject name as Free xyz" or an attachment, "picture.jpg" are other examples of signatures of
malicious events.
 Security
546 Cryptography and Network

methods which use string matching as

Signature based detection schemes are simple a Iist ot signatures. Although (he
lechnique. The current packet
against nown threats, signature based
1.
or loo

They are ineffective against unknown

matched to
entry is
schemes have several disadvantages:
simple and
threats: Simply moditying the subject name to
undeletreycnvge
They signature
cannotthepair
will change a request
undetected.
go the
andwith coresponding response, like knowing that a request
Fre xyN
403.
generated a response status code of
Server tOr a particular page conprise multiple events if none of the events alone
Ihev cannot detect attacks that
indication of an attack. This limitation is because of the inherent incapability of
processing the current one.
this Cometntainhsod an
temember previous requests when
J Anomaly Based Detection Anomaly based detection is the process of comparing
activities which are supposed to be normal
against observed events to identity deviations An
Uses anomaly based detection techniques has
represent the normal
profiles that behaviors of
dIeDfiPStionswhichof
network connections. or applications. For
example, a normal profile could include the users,
fact that web hosts
Is the most commonly done activity during day hours.
activit
effective in detecting previously
major benefit of anomaly detection is that they can be very
The
threats. For example. the power consumption of a computer may increase drastically Compared to norma
characteristic due to an infection from a malware. An initial profile is generated over a period of time, this
unknown
period being called a training period. Profiles can be either static or dynamic. Static profles are not change
for a long period of time, unless the IDPS is specifically directed to obtain a new profile. Adynamic profile
dvnamic hek
on the other hand, constantly gets updated with additional events. Because of the inherent
of networks and systems, static profiles are not suitable as they get out-dated soon. Dynamic profles do
suffer from this deficiency. But they suffer from the fact that the attackers can adopt evasive technianex .
fool such an IDPS. For example, the attacker can slowly increase its activity. The IDPS may think that the
rate of change is quite less and the small increase in activity may get included in the present proñile. Te
malicious programs then further increase its activity and thus incrementally evade the IDPS.
Another problem with IDPS technology implemented with anomaly detection methods have the probie
of false positives.They often treat benign activities as malicious events. For example, a system administrator's
may include backup of large files. Such large file transfers may not be specified during the training penod
and thus this perfectly benign activity may raise an alarm. It is often quite difficult to decide whether araisd
alarm is false, due to the complexity and the number of events that have resulted in the alarm.
9 Stateful Protocol Analysis This is the process of comparing predetermined profiles of genera
accepted definitions of benign protocol activity for each protocol state against observed events to idenui
deviations. Unlike anomaly based detection, which uses host or network specific profiles, staterui pous
analysis relies on vendor-developed universal profiles that specify how protocols should work. The saicu
in this protocol analysis means that the IDPS is capable of checking networks,
applications and Pi
protocols that have the notion of state in them. For example, the FTP (File Transfer
be visualized to consist of two states: unauthenticated and
Protocol) sess
authenticated.
consist of several operations, there are few "benign' operations While the authenticaeu
in the unauthenticated nt
state, viz. providiny
user names and passwords and seeing help manuals. Thus, the IDPS considers operations benignl or malicioU

depending on the present state of the protocol.

Stateful protocol analysis can identify unexpected sequences of commands. Such Suspicious sequence
could be repeated issues of same commands or issuing of a command without first issuing a commandupon
which it is dependent on.
 System Security
drawback of these 547
complexityinvolved analysis
methods is that they are
of states extremelv
n
that these methods do tracking the
not capture Iransitios for severalresource intensive This is
dprotoco|behavior. These may be attacks that do not violate the sessions Another serious
several henign requests to Create a characteristics
denal of
of generally
service attack
CLREWALLS: DEFINITIONS, CONSTRUCTION, AND
is a
firewall point
single of defense between two networks, Itis an WORKING PRINCIPLES
tremendousgrowth of
industry nthe recent days. Access to extremely important field of study with
th
tost(ransalon. which is neceSsary tor the steep internet is a great source of
competitionfaced in information and
internetaeSs, scomes with a
cautionary note. Not only does it bring the modern industrial era However.
ISersofa company, but it has the cerie fact of opening the information from the outside world to the
underslandingand design of proper firewalls is inside information to the external world Thus
extremely
firewalll can be simply arouter that is used o important.
4 filter the packets or a complex
router
solution that performs filtering of
packets along with applicaion level proxy
multi-computer. multi-
NNentially a router Or a group of routers and computers to enforce services A firewall is
access control between two networks.
firewall can be through of as a pair of mechanisms: allow,
4 which permits traffic and deny. which blocks
ratic. There are some firewalls which emphasize on blocking traffic, while
others emphasize on permiting
raffic.
standard corporate network topology has a hierarchy, often referred to as the security perimeters. Broadly
ternal perimeter of network and the internal network perimeter are separated by a DMZ
Zone). When information moves from the Internet to the
(Demilitarized
internal world, integrity of data is a greater concern
Annthe confidentiality of data. Suitable guards are enforced between the Internet and the DMZ and between
che DMZ and the internal network to ensure that messages which can cause servers to function incorrectly or
crash are not accepted. However when information travels fromthe internal network to the exernal world.
hoth integrity and confidentiality are concerns. Guards placed have to check further that no conidential
information is leaked and the data is not altered (or spoofed) while traveling from the internal computers to
the Internet. If such changes are found in the packets being transmitted either while traveling into or out of
the internalnetwork, it is assumed that the network is attacked. Thus the job of the guards is reduced to allow
rdeny access to the external network of internalsystems selectively. These guards are technically known as
frewalls and are hence essential for the integrity and confidentiality of the intormation present in the intenal
network.
around 1985
The first generation of firewall architectures were essentially packet filters and appeared first
ot packet
and came out of Cisco's 10S software division. However the first paper describing the principles
Sinee then there has been
lers was published in 1988 by Jeff Mogul from Digital Equipment Corporation.application layer hrewalls,
elolutions of the firewall industry. After packet filters, circuit level firewalls,
provide ditferent capabilities of auditing
yiamic firewalls etc were developed. Most firewalls technologies lead to the tiring of auditing events. With
Onmunication events. The firewalls record the causes which
network packet information, use more sophislicaled
evolutionof the firewalls, they inspect additional network packets at more network
information, and inspect the
Puon algorithms, maintain more state firewall architectures.
we present an overview on the
s. Inhe next sections

Packet Filters at thetransport protocol

that analyze network trafticwhich
Apacket filter is firewalltechnologies detines the nature
one of the foremost it matches one of a set of rules
layer. Each IP see
network packetis examined to
if
 Cryptogrophy and Network Security
548

of allowable data flow. These ules specify the allowable data flow and also the direction of data
internal to external network or vice versa
Following factors allow or deny the data flow through the packet filters:
How, ie.
1. The physical network interface (network adaptor) that the packet arrives on.
The address the data is coming from.
The address the data is going to.
4 The type of transport laver, TCP, UDP.
5. The transport layer source pot.
The transport layer destinationport.
Packet filters generally do not understand the application layer protocols used in the
packets. The rules are instead kept in the TCPAP kernel and applicd to any packet. The actions communicmayationbe
either deny or permit the packet. For a network packet to be routed to its destination, it has totaken
pemit listnule maintained in the kemel.If apacketmatches with adeny rule, then it is dropped. match
Hoe with a
apacket does not match with either an allow rule or adeny rule. then also it is dropped. There are
somepacket fillers which allow packets, if it does not match with a deny rule. however
Commands in packet filters check the source and destination port numbers on the TCP and UDP Iransn
layer protocols. The combinations of ports and protocols which are to be allowed are mentioned in the all
list.
Because packet filters work in the network layer, they are unable to process protocols with state
The packet filters do not also inspect the application layer data in the packets. This information
least secured. However they are the fastest firewall makes these filters the
technologies and often a part of the IP routers. The packet
filters often do network address translation so that the topology of the
the network is hidden to untrusted or network and the addressing scheme of
external network.
The advantages of the packet firewalls are
summarized below:
Advantages
1. Packet filters are faster than other techniques.
2 Less complicated, in the sense that a single rule
3. They do not require client controls deny or allow of packets.
4. They shield the internal IP computers to be configured specially.
address from the external world.
J Disadvantages
1. Packet filters do not understand
services, such andPUT and GETapplication layer protocols and hence cannot restrict access to Flr
2. They are stateless, and hence not commands.
3. Packet filters have almost no auditsuitable for appication layer protocols.
Next we present an event generation and alerting
an example of a cheap example of a packet filter. This example is writtenmechanisms.
of the syntax) can be packet filering lools. It is a kernel based tool on specifically for ipivadm, whichis
applied for other kernel Linux. The principles (and even thus
There are four basiC
-A
interfaces for packet
categories covered by the ipfwadm rules: filtering on open source Unix systems.
Packet Accouning
Input firewall
-0
 System Security S49

Outputfirewall

Forwardingtìrewall
pfisadmalsohas masquerading (-M) capabilities. For more information on the
iipfradm manual page.
switches and options, see
theImaginethat the organization uses a private network 192. 168.10. The Internet Service Provider has
asgnedtheaddress 201123. 102.32 as the gateway and 20L123.102.33 as the mail server. The
orgganization are as follows: policies oft
the
To allow all outgoing TCP connections
To allow incoming SMTP and DNS to external mail server
To block all other traffic
The following block of commands can be placed in a system boot file (perhaps rc local on Unix
Svstems).

ipfwadm -F-f
ipfwadn -F-p deny
infwadm -F-im-b-Ptcp -S 0.0.0.00 1024:65535 -D 201.123.102.33 25
infwadm -F -im-b -P tcp -S 0.0.0.0/0 1024:65535 -D 201.123.102.33 53
infwadm -F -im-b -Pudp -S 0.0.0.0/0 1024:65535 -D 201.123.102.33 53
infwadm -F -am-S 192.168.1.0/24 -D 0.0.0.0/0 -W eth0
sbin/route add -host 201.123.102.33 gw 192.168.1.2
The lines can be explained as follows: the first line flushes (-f) all forwarding (-F) rules. The second line
setsthe default policy (-p) to deny, while the line three to five are input rules (-i) in the following format:
ipfwadm -F (forward) -i (input) m(masq.) -b (bi-directional) -P (protoco)[protocol]-S (source)[subnet/
mask] [originating ports]-D (destination) [subnet/maskJ[port)
The sixth line appends (-a) a rule that permits all internal IP addresses output to all external addresses
on all protocols, all ports. The seventh line adds a route so that the traffic going to 201.123.102.33 will be
directed to the internal address 192.168.1.2.
In the next section we present the working principle of circuit level firewalls, which are second generation
firewalls.

5.2 Circuit-Level Firewalls

Circuit-level firewalls are similar inoperation to packet-filtering firewalls, but they operate at the transport
and session layers of the OSI model. The biggest difference between a packet-filtering firewall and a circuit
level firewall is that a circuit-level firewallvalidates TCP and UDP sessions before opening aconnection, or
CIrcuit, through the firewall. When the session is established, the firewall maintains a table otf valid connections
and lets data pass through when session information matches an entry in the table. The table entry is removed.
and the circuit is closed when the session is terminated.
To validate a session, these kinds of firewall thus examine each connection set up to ensure that it follows
ålegitimate handshake for the transport layer protocol.TCP is a widely used protocol in the transport layer
Which uses handshake. The firewall maintains a virtual circuit table, which stores the connection details,
Manely the session state and the sequencing information, of the successtul connections. When a connection
IS set up, the circuit level firewall typicallystores the following:
I. Aunique session identifier for the connection.
2. The state of the connection, namely handshake, established, or closing.
dThe sequencing information.
550 Cryptogrophy ond Network Security

4 The source |P address, fromwhere the datahas arrived.

5. The destination |P address, where the data is to he delivered.
The physical network interface throughwhich the data arrives.
7. The phvsical network interface. through which the packet goes out.
The circuit level firewall checks the header infomation contained in the network packet
has the necessarv permissions to be transmitted. These firewalls have a limited understandingto see whether it
of
Used in network packets. Thev can only detect one transport layer protocol. TCP. Like packet the
sets are kcpt in the TCP kemel filters.prottheocolrulse
These firewalls pertom a minimal security check compared to the application layer
network
hirewalls. Only the
connections
that are assoCiated with existing ones are allowed. But once a connection is allowod
packets associatedwith the connection are allowed without further security checks. The method is hence fo
and performs a limited amount of state checking.but is less secured. However, they perform
limited ches.
to deteet whether the packet data has been modified or spoofed. They
check that the data contained in tha
transport protocol header complies with the definition of the corresponding protocol. ike packet filters, theso
firew alls also perfom network address translation to hide the internal addresses from the external world
We summarize the advantages and disadvantages of the circuit level firewalls:

J Advantages
They are faster than application layer firewalls.
They are more secured than packet filter firewalls.
3. Thev maintain limited state information of the protocols.
4 They protect against spoofing of packets.
5 They shield internal IP addresses from
external networks by network address translation.
J Disadvantages
They cannot restrict access to protocol subsets other than TCP.
2 Thev have limited audit event generation capabilities.
3. They cannot perform security checks on higher level protocols.

5-3 Application-Layer Firewalls

An application-layer firewall is a third generation firewall
valid data at the application layer before allowing a technology that evaluates network packets for
connection. It examines the data in all network packets at
the application layer and maintains a complete list of
connection states and sequencing information. Further.
other security items that appear only in the application layer
requests are validated.
protocols, like user passwords and service
Application layer firewalls use special purpose program, called
through a firewall for a specificservice such as fip or htp. Proxy proxy services to manage data transter
protocol and provide increased securitychecks, access controls andservices are thus dedicated to a particular
Proxy services do not allow direct connection between the real generate appropriate audit records.
between the user and the real server and handles and inspects every service and the user. They sit transparently
service has two cOmponents that are lypically implemented as asingle comnmunication between them. A proxy
client. executable: prory server and pro4Y
When a real client wants to cOmmunicatelo an external
is directed to the proxy server, beCause the service in the internet, like fp or telnet, the reques
user's default gateway is set to the proxy server. The
then evaluates the request and decides lo deny or allow it, proxy serv
depending on a set of rules that are
the network servICe. PrOxy servers are aware of the protocols, and managed To
thus allow only complying packets with
 System Security S51

thcprotoco/ knitions. They also pertom auditing, ser authentication and caching. services which were
nM/ertormed,bv the packet filters or the cirCuitthelevel
Onthe other band. 0nce the packet from realfirewalls
client is allowed hy the proxy server the packet is
andedto;anroxy client who conacIs the actual server
information sent hy the actual server to the provid1ng the service [he proxy clhent
ntiormationto .be actual cient. proxV server. who decides wwhethersuhsequeniy
to send the
Thus the ptoxy service is
MnmunicatingArectlv with the service n
the Internet. However. transparent to a 1ser. who heleves that he is
firevaall hosts network stack and the proxy
topofthe operate only in the application services
laver of
are implemented on
the
level protocols in the kernel hefore heing operating syste
the
cach nacket must pass through the low:
Henc
thestackktothe application layer for athorough analysis by the proxy services Then thepassed packetto must travelof
the top
downthe stack and then be distributed by the low level
hak
protooBs are verv slow. protocols in the kernel Hence the application
laver
Theadvantages and disadvantages of the application layer protocols are summarized helow

J Advantages
They enforce and understand high level protocols, like HTTP and FIP.
Thoy maintain information about the
communication passing through the firewall server partial
ommunication derived state information, full application derived state information. partial session
information.
They can be used to deny access to certain network services, while allowing others.
4 They are capable of processing and manipulating packet data.
They do not allow direct communication between external servers and internal svstems, thus
shielding internal IP addresses from the outside network.
6. They are transparent between the user and the external network.
They provide features like HTTP object caching, URL filtering, and user authentication.
8. They are good at generating auditing records, allowing administrators to monitor threats to the
firewall.

D Disadvantages
1. They require replacing the native network stack on the firewall server.
2. They do not allow network servers to run on the firewall servers, as the proxy servers use the same
port to listen.
They are slow and thus lead to degradation in performance.
4. They are not scalable, as each new network service adds onto the number of proxV services
required.
J. Proxy services require modifications to client procedures.
in the system. Thus bugs
O. They rely on operating system support and thus are vulnerable to bugs
In NDIS, TCP/IP. WinSock, Win32 or the standard C library can cause security concems in the
security provided by the application layer firewalls.
modifications of the security
ynamic packet filter firewalls are a fourth generation firewalls that allow protocol.
nules onthe fiy. This technology is most suitable for providing limitedsupport for the UDP transport
network or vice-
This sfirewall associates to the
network external
all UDP packets that cross fromthe internal
generated and sent back to the original requester.
then a Virtual connection. If aresponse packet is
then a pass the firewall server. The intormation
virtual connection is established and the packet is allowed to is received
for asmall unit of time. If no response packet
Comes ponding
within this to)a virtual
time frame then
connection
the
is
virtual
remembered
connection is invalidated. The response packet that is allowed back
 Security
Cryptography and Network
552

matches the original source address, a transport layer

must contain a destination
address
source
that
port, and the same transport layer protocol type. This
that matches the original protocos ike Domain Name System(DNS). Aninternal DNS
des
feature t
isination
allowing application laver address information for unknown hosts. Theseserver usefreuqlues
must foK
other DNS servers on the
Internet to
be made bv TCP or UDP vitual connections.
obtain

are the most secured among

the firewal Connections
Thus to summarize
inareturn
morenore
secure
application
the dvnamic
laver firewalls
firewalls, which are more securethan the
circuit
technologiewhis.chThe
level firewalls,
secure than the packet level filters. However pertormmance wise. the application layer filters
than
often faster than the
are
slowest. A point to be notecd is that the
circuit level firewalls are packet level
wifthilters.a
whether a network packet is
checks. other than
they do not perform extensive security which have a large set of allow and deny rules.
assoCiated a
connection incontrast to packet filters violations
valh,
only detect and warn about security A
An lDS (Intrusion Detection System) may may simply block the attack or action firewal.
violation but
the other hand. may not notify a security
security firewall.
policy itof isthegOod
In practice,
the
to have both an IDS and a firewall, because the IDS warns us and afirewall block
violating
IDS are combined into one security program. like
attacks on the systems security. Some firewall and Noton
Internet Security.