Unity University

Faculty of Engineering, Technology, and Computational Sciences

Department of Computer Science

Course Title: Computer System Security (COSC 4082)

Topic : IP Security
After completing this topic and solving the review questions, you will be able to:

 Define IPSecurity.
 List down applications of IPSec ,
 Explain why IPSec has been designed (or introduced)?
 Describe the general benefits of IP Security .
 Discuss how IPSec provides security?
 Compare IPSec operation modes (Tunnel mode and transport mode),
 Identify the IPSec policy applied to each IP packet,
 Explain about IPSec authentication header (AH) format when used in IPv4 ,
 Describe IPSec Encapsulating Security Payload (ESP) format when used in IPv4 ,
 Discuss security issues with IPSec
 Comparing IPsec versus SSL/TLS

 Internet Protocol security (IPSec or IP security) defined:

 (IPsec or IP security) is a set of protocols and algorithms used to secure IP data at the network

layer with the use of cryptographic techniques.

 By means of additional headers, it can be incorporated to internet protocol (1Pv4 or IPv6).

 IPsec may be implemented by integrating it with the native IP stack or by implementing an

existing implementation of an IP protocol stack .

 IPsec provides secure communications over LAN, private and public WANs and the

Internet.

Source: Feleke Merin (Dr.Engr.) Page 1

 Unity University

1. IP-level security

IP level security is designed to provide the following services:

 Data origin authentication,
 integrity,
 confidentiality,
 protection against replays (a form of partial sequence integrity), and
 key management.

 The authentication mechanism ensures that the source mentioned in the received packet
header and the original transmitted source are the same.
 The integrity mechanism guarantees (or assures) that in the journey of the packet, it has not
been modified. That means, the received packet has not been modified in transit.
 Confidentiality permits (or enables) communicating nodes to encrypt messages and thereby
preventing eavesdropping by third parties.
 The replay protection ensures that a third party cannot seize a datagram and play it back
sometime later.
 The key management facility is concerned with the exchange of keys, used for
encrypting/decrypting messages, in a secure manner.

Source: Feleke Merin (Dr.Engr.) Page 2

 Unity University

2. Applications of IPSec
The application of IPsec varies in the capability of providing a secure communication across a
LAN, across private and public WANs, and across the Internet. Examples of its use include:
1. Secure connection between different branch offices of the same company over the Internet:
A VPN (Virtual private network) can be erected by a company to have a secure connection
between the branch offices over the Internet. This enable the company to:
 Rely (or depend) heavily on the Internet),
 Reduce its need for private networks,
 Saving costs and network management overhead.
2. Secure remote access to a distant Intranet over an insecure medium (Internet): With the
help of a system which is outfitted with IP security protocols, an employee can make a local
call to an Internet Service Provider (ISP) and gain secure access to a company's intranet.
This reduces the cost of toll charges for traveling employees and telecommuters.
3. Set up secure connection between peers (or partners): IPsec can be used to establish secure
communication within and outside network connectivity with associates of other
institutions.
4. Ensuring security for E-commerce applications: IPsec ensures that all communication
selected by the network administrator is both encrypted and authenticated, adding an extra
layer of security.

3. General benefits of IPSec

• Confidentiality
– By encrypting data
• Integrity
– Routers at each end of a tunnel calculates the checksum or hash value of the data
• Authentication
– Signatures and certificates
– All these while still maintaining the ability to route through existing IP networks.
• Data integrity and source authentication
– Data “signed” by sender and “signature” is verified by the recipient
– Modification of data can be detected by signature “verification”
– Because “signature” is based on a shared secret, it gives source authentication

Source: Feleke Merin (Dr.Engr.) Page 3

 Unity University

• Anti-replay protection
– Optional; the sender must provide it but the recipient may ignore
• Key management
– IKE – session negotiation and establishment
– Sessions are rekeyed or deleted automatically
– Secret keys are securely established and authenticated
 In a firewall/router IPSec provides strong security to all traffic crossing the perimeter
 Is below transport layer, hence transparent to applications
 Can be transparent to end users
 Can provide security for individual users

4. How IPSec applies to network traffic?

For each packet, an IPSec implementation decides whether to:
 discard that packet
 bypass IPSec security services, or
 afford IPSec security services

In other words, applying IPSec security services is largely a network management decision.

5. IP Security Architecture
Regarding the IPSec specification, lot of documents have been published by the IP Security

Protocol Working Group set up by the Internet Engineering Task Force (IETF). The most

important of these are as follows:

1. RFC 4301: Security architecture for the IP

2. RFC 4302: IP Authentication Header (AH)

3. RFC 4303: IP Encapsulating Security Payload (ESP)

4. RFC 4308: Cryptographic suites for IPsec

5. RFC 4835: Cryptographic algorithm implementation requirements for ESP and AH

Source: Feleke Merin (Dr.Engr.) Page 4

 Unity University

6. How IPSec provides security?

IPsec uses two protocols to provide traffic security:
1. Authentication Header (AH) provides
 integrity of datagram ,
 data origin authentication, and
 an optional anti-replay service.

2. Encapsulating Security Payload (ESP) provides

 confidentiality (encryption), and limited traffic flow confidentiality, and
 optionally data origin authentication, and an anti-replay service

 Summary of mapping of services to protocols

7. IPSec Transport and tunnel modes

Both AH and ESP hold up to two modes of use, namely transport and tunnel mode when used
between end-stations and between gateways, respectively.
Both AH and ESP can function in either:
 transport mode provides security for upper-level protocols (such as TCP or UDP) by
authenticating and/or encrypting the payload
 tunnel mode provides security for the whole IP packet by encapsulating (tunneling)

that packet into another IP packet.

 Transport mode
 Modify the original header
 Tunnel mode
 Keep the old header and put a new one on top

Source: Feleke Merin (Dr.Engr.) Page 5

 Unity University

In Figure above, IPSec tunnel mode only protects traffic between two IPSec – Gateways at
different sites. These gateways send traffic securely through the Internet between Themselves.
Tunnel mode creates a site-to-site VPN.

Source: Feleke Merin (Dr.Engr.) Page 6

 Unity University

 AH Transport vs Tunnel mode

 Transport mode for AH
 For transport mode AH, the IPSec authentication header is inserted after the original
IP header and before the IP payload of the packet, and only the payload of the packet
is encrypted or authenticated.

 Tunnel mode for AH

 For tunnel mode AH, the original IP packet is encapsulated within another IP

packet. The IPsec authentication header authenticates the entire original IP packet.

Source: Feleke Merin (Dr.Engr.) Page 7

 Unity University

 ESP Transport vs Tunnel mode

 Transport mode for ESP
 The ESP packet is inserted after the original IP header.
 It offers end-to-end security and having lower overhead than tunnel mode

 Before ESP

Source: Feleke Merin (Dr.Engr.) Page 8

 Unity University

 Tunnel mode for ESP

 For tunnel mode ESP, the original IP packet is encapsulated within another IP packet as an
encrypted and (optionally) authenticated ESP.

 Before ESP

Note:
When using tunnel mode , a new packet is constructed with IPSec header information, and the
entire original packet , including its header, is encapsulated as the payload of the new packet.
Tunnel mode is commonly used to create “Virtual private networks” (VPNs).

Source: Feleke Merin (Dr.Engr.) Page 9

 Unity University

 Compare Tunnel vs Transport modes

 Compare IPSec protocols (AH vs ESP)

Authentication Header vs Encapsulating Security Payload

Source: Feleke Merin (Dr.Engr.) Page 10

 Unity University

8. IP Security Policy
 An IPsec policy is applied to each IP packet that traverses between the source and the
destination, which is decided by the interaction of two databases, namely, the Security
Association Database (SADB) and the Security Policy Database (SPD).

Source: Feleke Merin (Dr.Engr.) Page 11

 Unity University

Question: What are the information contained in SAs?

 Answer: SAs contain encryption keys, information on which algorithms are to be used, and

other parameters related to communication.

 SAs are uni-directional, so each party must create an SA for inbound and outbound traffic.

 Communicating parties store SAs in a “Security Association Database (SADB).

 IPSec provides protection for outgoing packets and verifies or decrypts incoming packets

by using a “Security Parameter Index (SPI) “ field stored in the IPSec packet header, along

with the destination or source IP address, to index into the SADB and perform actions

based on the appropriate SA.

 Whether it is using AH or ESP protocol, the sender and the receiver must agree on a key

for authentication or encryption algorithm.

 This set of agreement between the hosts constitutes the Security Association (SA).

 Security associations are selected on the basis of the security policy.

 The SA separates the key management and the security mechanisms from each other.

 An association is a one-way connection, and so for a peer-to-peer communication two SAs

are used, one for each direction.

 The SA is identified by the following parameters:

1. Security Parameters Index (SPI): An arbitrary 32-bit value having only logical significance
is transmitted with an AH or ESP packet to enable the receiving system to select the SA
under which a received packet will be processed.
2. IP destination address: A 128-bit IPv6 or IPv4 address value of the destination end-point
of the SA.
3. Security Protocol Identifier: This field indicates whether the association is an AH or
ESP SA.

Source: Feleke Merin (Dr.Engr.) Page 12

 Unity University

9. Internet Key Exchange (IKE) protocol

 IPSec uses the Internet Key Exchange protocol to handle the negotiation of SAs.
 IKE operates in two stages:
 First, an initial security association is established to encrypt subsequent IKE
communications, and
 Second, this encrypted channel is used to define the SAs for the actual IPSec traffic.
 To establish the initial SA, a secure-key exchange algorithm is used to establish a shared
secret key between the two parties.
 Once this encrypted channel is established, the parties exchange information to define their
SAs, including:
 an encryption algorithm,
 a hash algorithm, and
 an authentication method such as preshared keys.
 Once these SAs have been created, the two parties can communicate using IPSec protocols
to provide:
 Confidentiality,
 Authentication, and
 Data integrity.

Source: Feleke Merin (Dr.Engr.) Page 13

 Unity University

10. Components of the IPsec Authentication Header (AH)

The authentication header (AH) protocol is used to authenticate the origin and guarantee the
data integrity of IPSec packets. Please refer fig. below.

The authentication header (AH) protocol, shown in fig. below, is added to an IPSec packet
before the payload, which either contains the original IP payload or the entire encapsulated IP
packet, depending on whether the transport or tunnel mode is used.
The IPsec AH format contains the following fields and illustrated in Figure below:

 Next header: It is used to link the headers and contain a header number.
 Payload length: This field mentions the length of the AH in 32-bit units with 2 subtracted
for consistency.
 Reserved: It is not in use, so it is set as zero by default.
 SPI: Security Parameter Index is a 32-bit value which identifies the SAs used for the
datagram.
 Sequence number: This number uniquely identifies each datagram.
 Authentication data: This field contains ICV (Integrity Check Value)

Source: Feleke Merin (Dr.Engr.) Page 14

 Unity University

11. IPsec Encapsulating Security Payload (ESP)

 Authentication header (AH) has functional limitations during communication as it

protects only the intermediate devices and not the message communication.

 That means, AH provides integrity and origin authentication, it does nothing to guarantee

confidentiality ― packets are still unencrypted.

 To satisfy this additional security requirement, the “Encapsulating Security Payload”

header, depicted in figure below, can be used.

 While AH places a header before the payload (please refer the diagram on page 12), ESP

encapsulates its payload by providing a header and a “trailer”.

 To provide encryption, ESP uses a specified “block cipher” (typically AES, 3DES, or

Blowfish) to encrypt either the entire original IP packet or just its data, depending on

whether the tunnel or transport mode is used.

 ESP also provides optional authentication in the form of “authentication data” field in the

ESP trailer.

 Unlike AH, ESP authenticates the “ESP header and payload” , but not the IP header.

 This provides slightly “weak security” in that it does not protect the IP header from

tampering, but allows NAT(network address translation) devices to successfully rewrite

source IP addresses.

Source: Feleke Merin (Dr.Engr.) Page 15

 Unity University

 ESP Format
 ESP has various fields which are divided into the following three components:
1. ESP header: The placement of ESP Header is based on the types of modes it uses and works
as in the AH Header. In front of the encrypted data, ESP header has two fields, one is the
SPI and other is the sequence number.
2. ESP trailer: This contains padding and pad length field for the alignment ofthe encrypted
data that is placed after the encrypted data and it has the next header field for ESP.
3. ESP authentication data: The ICY is calculated and placed as it is in the AH protocol.

In the encryption phase, ESP holds the encrypted data and the padding field is used to authenticate
the encrypted data. The following are the fields that are included in ESP.

 SPI: This field is a 32-bit value and when combined with the destination address and
security protocol type, it determines the SA of this datagram.
 Sequence number: This sequence number is used to give protection against replay attacks.

 Payload data: It has higher layer message or encapsulated IP datagram.

 Padding: This extra padding field is used for encryption alignment.

 Pad length: It denotes the size of the preceding padding field.

 Next header: It is used to link the headers and contains a header number.

 ESP authentication data: This field contains ICVs.

Source: Feleke Merin (Dr.Engr.) Page 16
 Unity University

12. Security issues with IPsec

The common types of attacks in the computer network which bring forth the need of IPsec are as
follows:
1. Eavesdropping: Commonly the data communication happens in plain text format which
gives place for the attackers to tamper, watch and modify the data by gaining unauthorized
access in several network paths and routes. Eavesdropping means sniffing or snooping
which is the most challenging security issue. Therefore, the data needs strong encryption
techniques; otherwise, the data traversing across the network is unsecure.

2. Data modification: Once the data is read by an attacker, the contents may be modified
without the knowledge of the data originator or the receiver. The data which is not
confidential lacks the value.

3. Identity spoofing (IP address spoofing): Every system is identified in the network by the
valid IP addresses and in some cases the IP addresses are falsely generated by some
organizations intranet using special algorithms which are pretended to be valid and get the
identity in the network. This is called IP spoofing. Using this IP addresses, the attackers
gain access to the network and modifies the data and the routing paths and makes the
system exhausted to inconvenience.

4. Password-based attack: Generally in network and operating systems, security is

provided by password-based access control where the access control is determined by
user name and password. The login account is not commonly protected by the operating
systems and it sends the identity data in plain text format across the network
communication to validate it. In the meantime, the attackers hack such identity
information and gain unauthorized access to the network. After gaining access to a
network with a valid account, an attacker can get all user accounts and domain names
or change the network parameters like configuration file type access permissions and
routing information or even modify the data.

5. Man-in-the-middle attack: This shows how an intruder enters, listens, tampers and
controls the communication between two parties exchanging sensitive information which
is unknown by other parties.

Source: Feleke Merin (Dr.Engr.) Page 17

 Unity University

6. Denial-of-service attack: This attack will exhaust the total bandwidth of the system
and make the entire service unavailable to its intended users. After gaining access to a
network, an attacker may attack the applications and make the functions abnormal or
send a flood of sync messages and exhaust the available bandwidth or block the access
gain and network resources for the genuine users.

7. Sniffer attack: The sniffer tool is available to monitor the packet exchanges between the
users. It shows the data encapsulated in a packet and can be opened and read if the packets
are not provided with security mechanisms. Using a sniffer, an attacker may determine the
access permissions and the related information and corrupt the network or read private
data.

13. Summary

Source: Feleke Merin (Dr.Engr.) Page 18

 Unity University

14. IPSec versus SSL/TLS

Review Questions
1. Define the term IPSecurity.
2. Write down the applications of IPSec.
3. Name the general benefits of IPSec .
4. What do you mean by IP Security policy?
5. Compare between IPSec transport mode and IPSec tunnel mode.
6. Write short notes on Security Association Database (SAD) and the Security Policy Database (SPD).
7. Briefly explain IPsec Authentication Header (AH).
8. Write a short note on AH format.
9. Discuss on AH datagram placement.
10. Discus about IPsec Encapsulating Security Payload (ESP).
11. Compare AH with ESP.
12. With the help of a neat diagram, explain the ESP format.
13. Write about ESP field placement.
14. List down the security issues of IPSec (where IPSec is need).

============ The End! ============

Source: Feleke Merin (Dr.Engr.) Page 19