Intrusion Detection

System
 Intrusion Detection System
 Intruders: Intruder is a person who trying to
access system, resources or network without
authorization.
 IDS is developed to provide early warning of
an intrusion so that defensive action can be
taken to prevent it.
 Intrusion detection involves detecting unusual
patterns of activity.
April 22, 2024 2
 Intrusion Detection System
 Masquerader: not authorized to use the computer and
who penetrates a system’s access controls to exploit a
legitimate user’s account
 The masquerader is likely to be an outsider;
 Misfeasor: A legitimate user who accesses data,
programs, or resources for which such access is not
authorized, or who is authorized for such access but
misuses his or her privileges
 the misfeasor generally is an insider;
 Clandestine user: An individual who seizes supervisory
control of the system and uses this control to evade
auditing and access controls or to suppress audit
Collection
 clandestine user can be either an outsider or an insider.
April 22, 2024 3
 Intrusion Techniques
 Objective of the intruder is to gain access to a system
 Most initial attacks use system/software
vulnerabilities
 The intruder attempts to acquire information that
should have been protected. (eg. Password)
 Intruder can log in to a system using password and
exercise all the privileges accorded to the legitimate
user
 The password file can be protected in one of two
ways: One Way Function & Access Control.
April 22, 2024 4
 Intrusion Detection System
 IDS monitors network traffic and computer system
for suspicious activity and alerts the system or
network.
 IDS is same as Burglar alarm system.
 Detect based on looking for specific signatures of
known threats.
 Approach of detecting suspicious traffic in two
different ways.
 Host Based IDS
 Network Based IDS
April 22, 2024 5
 Critical Signature
Files Database

Network Traffic Analysis User

Traffic Collector Engine Interface

Log Alarm Reports

Files Storage

Components of IDS
April 22, 2024 6
 Components of IDS
Traffic Collector: Collects activity or events from the
IDS to examine.
Analysis Engine: examines the collected network
traffic & compares it to known patterns of
suspicious or activity stored in the signature
database.
Signature database: collection of patterns &
definitions of known suspicious activity.
User Interface & Reporting: interfaces with the
human element, providing alerts.
April 22, 2024 7
 Host Based IDS
 Performs logging and analyzing data plus
events of single host to identify possible
threats.
 HIDS checks log files, critical files, audit
trails and network traffic entering into or
leaving a host.
 HIDS operate in real time or operate in batch
mode.
 Focuses on log files or audit trails produced
by OS.
April 22, 2024 8
 Host Based IDS
 They are installed on a single system
 It captures all traffic and incidents perform on
host and analyzes it.
 Each packet coming and going from the host is
scanned against defined rules and signatures
 Alerts are configured to notify security team

April 22, 2024 9

 Critical Signature
Files Database

Traffic Analysis User

Collector Engine Interface

Log Alarm Reports

Files Storage

Components of Host Based

IDS
April 22, 2024 10
 Host Based IDS
 HIDS is looking for certain activity in log file
 Login at odd hours
 Login authentication failure
 Adding new user account
 Modification or access to critical files
 Modification or removal of binary files
 Starting and stopping processes
 Use of certain program

April 22, 2024 11

 Host Based IDS
Advantage
 Operating System specific and detailed signature
 Examine data after it has been encrypted
 Application specific
Disadvantage
 Secure only single host, cannot relate activities around
it
 Uses local system resources
 High cost of ownership and maintenance
 If logged locally, could be compromised or disable
April 22, 2024 12
 Network Based IDS
 Performs packet sniffing and analyze network
traffic to identify possible threats in network traffic
 Some times they are deployed as in-line or out of
path in network
 Switch ports traffic is mirrored to IDS.
 Each packet coming and going from the network is
scanned against defined rules and signatures
 Alerts are configured to notify security team
whenever it detects any anomaly.
April 22, 2024 13
 Network Based IDS
 Network based IDS focuses on network traffic
 Bits and bytes traveling along the cables.
 NIDS having capability to analyze traffic
according to protocol, source, dest, content,
traffic already seen.
 The IDS must be able to handle traffic at any
speed the network operates.

April 22, 2024 14

 Signature
Database

Network Traffic Analysis User

Traffic Collector Engine Interface

Alarm Reports
Storage

Components of Network Based

IDS
April 22, 2024 15
 Network Based IDS
 Network based IDS looks for certain activites
 Denial of service attack
 Port scans
 Malicious content in the data payload of packet
 Vulnerability scanning
 Trojan horse, viruses and worm
 Tunneling
 Brute force attack

April 22, 2024 16

 Network Based IDS
Advantages
 Examine content of packet
 Lower cost of deployment, maintenance and upgrade
 Examine all network traffic and correlate attacks
Disadvantage
 Ineffective when traffic is encrypted
 Don’t know the activity on the hosts
 Cant check traffic which does not pass it

April 22, 2024 17

 Honey pots
 “Honeypot is a trap set to detect, deflect, or in some
manner counteract attempts at unauthorized use of
information systems."
 "Honeypot is an information system resource whose
value lies in unauthorized use of that resource."
 Honeypots are designed to
 Divert an attacker from accessing critical systems
 Collect information about the attacker’s activity
 Encourage the attacker to stay on the system long enough for
administrators to respond
April 22, 2024 18
 Honey pot
 Honey pot contains information designed to
appear valuable
 But a legitimate user of the system wouldn’t
access.
 Thus any access to honey pot is suspect.
 Contains sensitive monitors and event loggers
 Any attack against honeypot is made successful,
administrators have time to mobilize and log and
track the attacker.
April 22, 2024 19
 Honey pots
Low-Interaction Honeypot
 Have limited interaction
 They normally work by emulating services and
operating systems
 Attacker activity is limited to the level of emulation
by the honeypot.
 Easier to deploy and maintain, with minimal risk.
 Examples include Specter, Honeyd, and KFSensor.

April 22, 2024 20

 Honey pots
 High-Interaction Honeypot
 Are usually complex solutions as they involve real
operating systems and applications
 Nothing is emulated; we give attackers the real thing.

April 22, 2024 21

E-Mail Security
 Email Security
 Email is one of the most widely used and
regarded network services
 Currently message contents are not secure
 May be inspected either in transit
 Or by suitably privileged users on destination
system

April 22, 2024 23

 Email Security
 For text email transmission, the message is
considered as two portions :
 Contents
 Header
 An Email message consist of a number of
header lines followed by actual message
contents
 Header keywords are From, To, Subject -
April 22, 2024 24
 Email Security
Enhancements
 Confidentiality
 Protection from disclosure
 Authentication
 of sender of message
 Message integrity
 Protection from modification
 Non-repudiation of origin
 Protection from denial by sender

April 22, 2024 25

 SMTP Simple Mail Transfer
Protocol
 Used for email communication.
 SMTP is request/response based, which means
email client & SMTP server.
 This server transfer message to receivers
SMTP server
 SMTP’s job is to carry email messages
between sender & receiver.

April 22, 2024 26

Email communication consist of following steps:
1. At the senders end, SMTP server takes message
sent by a users computer.
2. The SMTP server at the senders then transfers
message to SMTP server of the receiver.
3. The receiver computer drags the email message
from SMTP server at receiver end, using protocol
POP or IMAP

1 2 3

SENDER RECEIVER
April 22, 2024 SENDER RECEIVER 27
SMTP SMTP
 Post Office Protocol (POP)
 POP is used to retrieve (download) email
 Uses plaintext to communicate between sender
and receiver
 To denote success POP server sends plus (+) at
beginning of response
 To denote failure POP server sends minus (-)

April 22, 2024 28

 IMAP
 Internet Message Access Protocol
 It is plaintext mail protocol that combines aspect of
both POP and SMTP.
 It allows user to send and retrieve email
 But to send outgoing mail it require SMTP server
 User has to connect IMAP server, authenticate itself
and then start working
 IMAP works in two modes
 It can store all data on the server or.
 Allow user to work offline by storing data locally.
April 22, 2024 29
 Privacy Enhanced Mail
 PEM is internet standard to provide email
security
 Employ cryptographic techniques for
confidentiality, sender authentication, and
message integrity
 Message integrity ensures user that message is not
modified during transit.
 Sender authentication verifies received message is
originated from person who claim to have sent it.
 Confidentiality allows a message to kept secret

April 22, 2024 30

Origin Authentication
 Certificates – digital signature algorithm, Subject
Name, Issuer name, Validity, subjects public key
Message Confidentiality
 Implemented using standardized cryptographic algo.
 RFC 1423 defines both symmetric (DES) and
asymmetric (RSA) encryption
Data Integrity
 PEM implements concept of message digest
 PEM uses RSA MD2 and RSA MD5

April 22, 2024 31

 Pretty Good Privacy (PGP)
 Widely used de facto secure email
 Developed by Phil Zimmermann
 Selected best available crypto algs to use
 Integrated into a single program
 On Unix, PC, Macintosh and other systems
 Originally free, now also have commercial versions
available

April 22, 2024 32

 Pretty Good Privacy (PGP)
 Provides confidentiality and authentication service for
email & file storage
 Combines features of conventional & public key crypto.
 When a user encrypts plaintext with PGP, PGP first
compress plaintext
 Then PGP creates a session key, one time only secret
key
 Plaintext is encrypted with session key.
 Once the data is encrypted, session key is encrypted
with recipients public key
April 22, 2024 33
 Pretty Good Privacy (PGP)
 Public key encrypted session key is transmitted along
with ciphertext to receiver.

Fig. PGP Encryption

April 22, 2024 34
 Pretty Good Privacy (PGP)
 Receiver uses his/her private key to recover session key,
 Session key is used to decrypt the ciphertext.

Fig. PGP Decryption

April 22, 2024 35
 PGP Operation –
Authentication
 Sender creates message
 Make SHA-1 160-bit hash of message
 Attached RSA signed hash to message
 Receiver decrypts & recovers hash code
 Receiver verifies received message hash

April 22, 2024 36

 PGP Operation –
Confidentiality
 Sender forms 128-bit random session key
 Encrypts message with session key
 Attaches session key encrypted with RSA
 Receiver decrypts & recovers session key
 Session key is used to decrypt message

April 22, 2024 37

 PGP Operation –
Confidentiality &
Authentication
 Can use both services on same message
 Create signature & attach to message
 Encrypt both message & signature
 Attach RSA/ElGamal encrypted session key

April 22, 2024 38

 PGP Operation –
Compression
 By default PGP compresses message after signing
but before encrypting
 So can store uncompressed message & signature for
later verification
 Uses ZIP compression algorithm

April 22, 2024 39

 S/MIME (Secure/Multipurpose
Internet Mail Extensions)
 Security enhancement to MIME email
 Original Internet RFC822 email was text only
 MIME provided support for varying content types and
multi-part messages
 With encoding of binary data to textual form
 S/MIME added security enhancements
 Have S/MIME support in many mail agents
 eg MS Outlook, Mozilla, Mac Mail etc

April 22, 2024 40

 S/MIME Functions
 Enveloped data
 Encrypted content and associated keys
 Signed data
 Encoded message + signed digest
 Clear-signed data
 Cleartext message + encoded signed digest
 Signed & enveloped data

April 22, 2024 41

 S/MIME Cryptographic
Algorithms
 Digital signatures: DSS & RSA
 Hash functions: SHA-1 & MD5
 Session key encryption: ElGamal & RSA
 Message encryption: AES, Triple-DES, RC2/40
and others
 MAC: HMAC with SHA-1
 Have process to decide which algos to use

April 22, 2024 42

 S/MIME Messages
 S/MIME secures a MIME entity with a signature,
encryption, or both
 Forming a MIME wrapped PKCS object
 Have a range of content-types:
 Enveloped data
 Signed data
 Clear-signed data
 Registration request
 Certificate only message

April 22, 2024 43

IP Security
 IP Security
 Internet Architecture Board (IAB) included
authentication and encryption necessary features in
IPv6.
 These features can also applied with IPv4.
 IP packets consist of IP header and payload.
 IP payload contains data in plaintext form.
 IPSec is a set of protocols with sit on top of IP layer.
 Provide security by authenticating and encrypting
each IP packet.
April 22, 2024 45
 IP Security
 IPSec uses cryptographic security to protect
comm.
 IPSec supports network level
 Peer authentication,
 Data origin authentication,
 Data integrity,
 Confidentiality and
 Reply protection.

April 22, 2024 46

 IPSec Uses

April 22, 2024 47

 Benefits of IPSec
 In a firewall/router provides strong security to all traffic
crossing the perimeter
 Is resistant to bypass
 Is below transport layer, hence transparent to applications
 Can be transparent to end users
 Can provide security for individual users if desired
 Additionally in routing applications:
 Assure that router advertisements come from authorized routers
 Neighbor advertisements come from authorized routers
 Insure redirect messages come from the router to which initial
packet was sent
April 22, 2024 48
 IPSec Architecture
• IPSec mechanism uses a Security Policy
Database to determine how to handle messages.
• Legal actions are discarding message, applying
security services & forward with no change.
• Action depends on information in the IP &
transport layer headers.
• When packet arrives, IPSec mechanism consult
the SPD for relevant n/w interface.

April 22, 2024 49

 • A security association (SA) is a set of security
enhancement to a channel along with packets are
sent.
• Defined uniquely by dest. address, security
protocol (AH or ESP) & a unique 32 bit security
parameter Index.
• Each SA uses either AH or ESP, but not both.
• If both required, two SAs are created.
• When IPSec services are to be applied, SPD entry
identifies one or more security associations and
parameters

April 22, 2024 50

 IPSec
• IPSec is a set of protocols developed by IETF.
• Developed for exchange of packets at n/w layer.
• Overall idea of IPSec is to encrypt & seal the
transport & application layer data during
transmission
• This protocol only works in combination with IP
• Once an IPSec connection is established it is
possible to tunnel across other n/w

April 22, 2024 51

 Original Original
Message Message

Application Application
Transport Transport
IPSec IPSec
Internet Internet
Physical Physical

Transmission
Media
April 22, 2024 52
IPSec in TCP/IP protocol
 Transport Mode
• Encrypts only data portion of packet,
• Thus enabling outsiders to see source & dest. IP
address.
• This protects data being transmitted, but allows
knowledge of transmission.
• IPSec takes transport layer payload adds IPSec
header & then adds the IP header
• Thus the IP header in not encrypted.
• Protection of data portion of packet is referred as
content protection
April 22, 2024 53
 Transport Mode

April 22, 2024 54

 Transport Mode

Transport Transport
Layer Layer Payload

April 22, 2024 55

 Transport Mode

Transport Transport
Layer Layer Payload

IPSec
IPSec Payload

April 22, 2024 56

 Transport Mode

Transport Transport
Layer Layer Payload

IPSec IPSec IPSec

IPSec H Payload T

April 22, 2024 57

 Transport Mode

Transport Transport
Layer Layer Payload

IPSec IPSec IPSec

IPSec H Payload T

IP IP Payload

April 22, 2024 58

 Transport Mode

Transport Transport
Layer Layer Payload

IPSec IPSec IPSec

IPSec H Payload T

IP IP H IP Payload

April 22, 2024 59

 Transport Mode

Transport Transport
Layer Layer Payload

IPSec IPSec IPSec

IPSec H Payload T

IP IP H IP Payload

IPSec Transport Mode

April 22, 2024 60
 Tunnel Mode
• Provides encryption of source & dest. IP
addresses, as well as of the data itself.
• It can only be done between IPSec servers because
final dest. Needs to be known for delivery.
• Protection of header information known as
context protection.
• It takes IP datagram, including IP header.
• It adds IPSec header & trailer & encrypts whole
thing.
April 22, 2024 61
 Tunnel Mode
• It then adds new IP header to this encrypted
datagram
• It is possible to use both methods at the same
time
• Such as using transport within ones own n/w to
reach an IPSec server
• Which then using the transport method from
the target n/w IPSec server to target host.
• Has three connections- host to server, server to
server & host to host.
April 22, 2024 62
 Tunnel Mode

April 22, 2024 63

 Tunnel Mode
Transport Transport
Layer Payload

April 22, 2024 64

 Tunnel Mode
Transport Transport
Layer Payload

Network
IP Payload
Layer

April 22, 2024 65

 Tunnel Mode
Transport Transport
Layer Payload

Network
IP H IP Payload
Layer

April 22, 2024 66

 Tunnel Mode
Transport Transport
Layer Payload

Network
IP H IP Payload
Layer

IPSec IPSec Payload

April 22, 2024 67

 Tunnel Mode
Transport Transport
Layer Payload

Network
IP H IP Payload
Layer

IPSec IPSec H IPSec Payload IPSec T

April 22, 2024 68

 Tunnel Mode
Transport Transport
Layer Payload

Network
IP H IP Payload
Layer

IPSec IPSec H IPSec Payload IPSec T

New IP payload

April 22, 2024 69

 Tunnel Mode
Transport Transport
Layer Payload

Network
IP H IP Payload
Layer

IPSec IPSec H IPSec Payload IPSec T

New
New IP payload
IP H

April 22, 2024 70

 IPSec Configuration
 Host-to-host
 Internet is not part of SA between two machines
 Both the parties who want communication, agree on the use
of the protocols that are available & this agreement is
known as Security association

Internet

Host A Host B

Host to host IPSec connection

April 22, 2024 71

 IPSec Configuration
 Keeps two devices for security within the stream, relieving
hosts of calculating & encapsulating duties
 These two gateways have a SA between them, but n/w is
assume to be secure from each machine to its gateway

Internet

Host A Gateway 1 Gateway 2 Host B

IPSec between machine using gateway security devices

April 22, 2024 72

 IPSec Configuration
 Third level combines the previous two methods
 A separate SA exist between the gateway devices
 Additionally a SA exist between hosts and can be considered
as a tunnel inside a tunnel.

Internet

Host A Gateway 1 Gateway 2 Host B

Separate IPSec tunnels gateway to gateway & host to host

April 22, 2024 73

 IPSec Configuration
 A remote user connects to organizations network through
internet.
 N/w has security gateway by which it secures traffic to &
from its server & authorized user
 User establishes a SA with security gateways

Internet

Host A Gateway 2 Host B

April 22, 2024 Remote connection across the internet using IPSec 74
 IPSec Security
• IP packets consist of 2 portions
– IP header
– Actual data
• IPSec features implemented in the form of
additional IP header to standard default IP
Header
• IPSec offers 2 main services
– Authentication
– Confidentiality
April 22, 2024 75
 IPSec Security
• IPSec defines two IP extension headers one for
authentication & other for confidentiality.
• IPSec consists of 2 main protocols

IPSec

Authentication Encapsulating
Header (AH) Security Payload (ESP)

April 22, 2024 76

Authentication Header (AH)
• The AH, when added to an IP datagram
ensures
– The integrity of the data
– The authenticity of the data’s origin
– Optional anti replay service
• Protects non changing elements in IP header
• AH protects the IP address, which enables
data origin authentication.

April 22, 2024 77

Authentication Header (AH)
• IPSec AH contains a cryptographic checksum
for the content of packet.
• The AH is simply inserted between IP header
& any subsequent packet contents
• No changes are required to the data contents
of the packet
• Security resides completely in the contents of
the AH.

April 22, 2024 78

Encapsulating Security Payload
(ESP)
 Provides security for the higher level portion
of packet not the IP header.
 Provides data confidentiality
 Defines a new header, inserted into IP packet
 Transform data into unreadable encrypted
form.
 The ESP will be inside AH i.e encryption
happens first & then authentication.

April 22, 2024 79

• Both AH & ESP can work in transport & tunnel mode
AH Transport Mode
 In transport mode position of AH is between the
original IP header & the original TCP header of
the IP packet
AH Tunnel Mode
 In tunnel mode, entire original IP packet is
authenticated And the AH is inserted between the
original IP header & new outer IP header.
 Inner IP header contains ultimate source & dest.
address
 Outer IP header possibly contains different IP address
April 22, 2024 80
 IP TCP Original
Header Header Data
Before applying AH

IP TCP Original
AH
Header Header Data
After applying AH in transport mode

New IP Original IP TCP Original

AH
Header Header Header Data

After applying AH in tunnel mode

April 22, 2024 81

ESP Transport Mode
 Used to encrypt and optionally authenticate data
carried by IP.
 ESP is inserted into the IP packet immediately
before the transport layer header & ESP trailer is
inserted after the IP packet.
 If authentication is also used, ESP authentication
field is added after ESP trailer.
 The entire transport layer segment & ESP trailer
are encrypted.

April 22, 2024 82

 IP TCP
Original Data
Header Header

Before applying ESP

IP ESP TCP Original ESP ESP

Header Header Header Data trailer Auth

After applying ESP in transport mode

NEW IP ESP ORI IP TCP Original ESP ESP

Header Header Header Header Data Trailer Auth

After applying ESP in Tunnel mode

April 22, 2024 83