You are on page 1of 15

IPSec VPN Fundamentals Page 1 of 15

EE Times Home > Design >Communications DesignLine

Design Article
82 Tweet

IPSec VPN Fundamentals

Pradosh Kumar Mohapatra and Mohan Dattatreya
9/19/2002 12:00 AM EDT

With the explosive growth and popularity of the

Internet, more and more enterprises are
looking towards building their network
infrastructure across the Internet without
having to spend a lot on private leased lines.
However, with the increasing usage of the
Internet to build networks, more and more "evil
ways" of breaking into the network to gather
sensitive information are also evolving.
Therefore, security on the Internet has been a
main concern for each enterprise. Enter IP
Security or IPSec. IPSec provides the
necessary infrastructure to extend an enterprise's private network across the I
to customers and business partners, in other words, to build what is called a
network (VPN)".
IPSec is a set of protocols and standards developed by the Internet Engineeri
(IETF) to support secure communication at the IP layer. Used widely to implem 6/27/2013
IPSec VPN Fundamentals Page 2 of 15

a full-fledged security gateway sitting between the corporate network and the
the Internet, or as part of the router itself, IPSec and an
complete cryptographic security.

VPN Applications
There are three basic flavors of IPSec VPNs, each with an associated set of b
requirements (Figure 1):
Remote-Access VPNs: These let individual users, such as telecommuter
corporate network. The user's laptop usually contains a VPN client that cr
tunnel to the security gateway at the corporate headquarters. Another flav
application is offered via creating an L2TP/PPTP session that is

Intranet VPNs: This type connects branch offices to the corporate headq
creating a transparent Intranet.

Extranet VPNs: These let companies connect with their business partner
suppliers, customers, and joint ventures).

Figure 1: IPsec VPN applications

Basics of Cryptography
Cryptography is a set of mathematical functions that forms the foundation of a
With respect to IPSec, the following functions are 6/27/2013
IPSec VPN Fundamentals Page 3 of 15

Encryption/Decryption: This forms the basis

taking a plaintext M and converting it to a ciphertext C of equal length. Th
transmitted on the wire that protects the message M from any eavesdropp
the communication. On the receiving end, decryption is performed
C back to plaintext M.

There are two kinds of cryptographic transforms available that differ on th

used to encrypt/decrypt:

1. Symmetric-Key Cryptography: Here only one shared key K is know

but it is not known to any third party. Encryption is the function of two
and decryption is the function of (C, K). Many
popular ones are DES , 3DES, and AES.
block of data at a time and transform it into a ciphertext block. These
functions can be operated in different modes to add extra security. O
mode is cyclic block chaining (CBC), where each processed ciphertex
binary arithmetic) to the next plaintext block before it is input to the
function. An initialization vector (IV), random data of
data for this mode. describes these algorithms and various modes

2. Asymmetric-Key Cryptography (also called public

keys is assigned per peer: a public key K
known only to the owner of the key pair. A text M
by K2 and vice versa, but not by any other means. The most popular
is RSA. Refer to Mohapatra for an introduction to public

Note that symmetric key algorithms are computationally much faster than
algorithms. For efficiency, symmetric-key algorithms like DES or 3DES are
preserving confidentiality. Public-key algorithms are used in a "hybrid" or
mode to achieve authentication and non-repudiation. 6/27/2013
IPSec VPN Fundamentals Page 4 of 15

Message Digest or Integrity Checksum Value (ICV):

A message digest is a small "fingerprint" of a long message M that is obta
one-way hash function, H - a function that is easy to compute but for whic
computationally not feasible to find two messages M and M' such that H(M
message digest is included in the IPSec packet, the receiving
digest on the received message and compare it with
message has been compromised. Popular hash functions are MD5 and S

Digital Signature: This forms the basis of non

like RSA, is combined with a one-way hash function, like MD5 or SHA1. F
message digest H(M) is computed and encrypted (signed) by the
called the digital signature for message M that
owner knows the private key, the sender cannot repudiate the fact that he

Certificate: This forms the basis of authentication.

format that binds a person's identity to his/her public key and is signed by
authority." This certificate is "self-authenticating" because both the parties
authority" and have its public key available.

IPSec Modes of Operation

IPSec provides two different modes to exchange protected data across the dif
Transport Mode: This mode is applicable only for
extends to the payload of IP data. The IP addresses of the hosts must be

Tunnel Mode: This mode is used to provide data security between two n
protection for the entire IP packet and is sent by adding an outer IP head
the two tunnel end-points. The unprotected packets generated by hosts tr
protected "tunnel" created by the gateways on both ends. The outer IP
corresponds to these gateways. Both intranet and 6/27/2013
IPSec VPN Fundamentals Page 5 of 15

mode. Since tunnel mode hides the original IP header, it facilitates securi
with private IP address space.

Figure 2: IPsec modes of operation

tunnel and transport

IPSec Architecture
IPSec is an open, standards-based security architecture.
following concepts that are the building blocks of the

Security Association Database (SAD)

A security association (SA) is a unidirectional (simplex) logical
systems, uniquely identifiable by a tuple. Figure 3

Figure 3: Security association structure

An SA indicates all the security services to the traffic carried by it (pointed to b

authentication transform, encryption transform, and the replay protection
denial of service attack where an eavesdropper saves already traversed packe
at a later point of time). It also has a lifetime associated with it, by
becomes expired. An expired SA can no longer be
tuple has to be created. Because SAs are simplex, for bi
IPSec systems, there must be two SAs defined, one for each direction. SAs c
manually or through an automatic key-exchange protocol, IKE. 6/27/2013
IPSec VPN Fundamentals Page 6 of 15

Security Parameter Index (SPI) is a 32-bit value used to identify an SA. The S
IPSec header of the protected packet and has local significance
that are either created manually or automatically through negotiation, indexed

Security Policy Database (SPD)

SPD contains what security services (policies) are to be offered to the IP traffi
set of fields of the IP packet, called a "selector." The selector for IPSec opera
Each entry in the SPD is indexed by the selector and
to be performed for an IP packet if it matches the selector:

1. Discard the packet

2. Process by the IPSec module, in which case the SPD

3. Pass through (pass the packet to the IP stack for normal forwarding).

Interaction Between the Building Blocks

Figure 4 describes the overall IPSec architecture.

Figure 4: IPsec architecture

The "Policy Manager" module is the interface between the user adding a secu
SPD. The "IKE Daemon" module does the automatic SA negotiation between
The "Certificate Manager" verifies and enrolls certificates for
In short, a typical packet flow inside this architecture proceeds as follows: 6/27/2013
IPSec VPN Fundamentals Page 7 of 15

A packet is received through the receive queue and passed to the IPSec

The IPSec packet processing module extracts the

SPD for a policy. If the policy is "discard", the packet is discarded. If the p
through", the forwarding engine forwards the packet normally.

If the policy is "IPSec", the SPD entry should point to an SA in SAD. The
fetches the corresponding SAD entry and checks for validity. If the SA sta
module indicates the IKE Daemon for another SA

All the transforms depicted in the SA are performed on the packet with th
"cryptography" module.

The transformed packet is sent to the "transmit queue"

IPSec Protocols
IPSec standards have defined three main protocols: AH, ESP,

Authentication Header (AH)

AH provides data integrity and replay protection for the whole IP datagram
measure against IP-spoofing and session-hijacking attacks. AH uses the modi
hash function (such as, MD5 or SHA1) that takes an authentication
integrity checksum value (ICV). The ICV is included
computes the ICV on the received packet and checks for equality. Any modific
to the packet during transmission can be discovered, as the ICVs will not matc
specifies a set of mutable IP header fields that should not be used to
Fragment offset and flags, TTL, Checksum). Figure

Encapsulating Security Payload (ESP)

ESP provides data confidentiality, data integrity, and replay protection for th
datagram. It uses a symmetric key algorithm (like 3DES
and, like AH, uses a secure hash algorithm to compute ICV.
protected against data-integrity attacks. Figure 5b 6/27/2013
IPSec VPN Fundamentals Page 8 of 15

Figure 5: IPsec (a)

ESP format

Figure 6 shows how AH and ESP are used in tunnel mode to protect an IP pa

Figure 6: AH and ESP in tunnel mode

Internet Key Exchange (IKE) 6/27/2013
IPSec VPN Fundamentals Page 9 of 15

IKE defines the mechanism to establish SAs required to secure the packets
IPSec peers. As defined in Figure 3, the main components of an SA are the
(the algorithm and the key) that will be used to protect data.
way of negotiating these details between the two peers.

IKE uses the Internet Security Association and Key

framework and incorporates Oakley and SKEME as the key
values to be negotiated are defined in the IPSec domain of interpretation (DO
IKE Daemon runs as an UDP application with UDP port number 500. The prot
two phases:

1. Phase I (Authentication Phase)

When two peers over the Internet wish to communicate, it is assumed tha
exists. Therefore, the objective of "phase I" is to establish a secure
negotiating parties, and generate shared keys to
are available for use in "phase I": main mode and aggressive mode. They
the same thing"establishing an IKE SA that will be used to protect the neg
of phase II. Aggressive mode is a little faster, but it does not
the negotiating parties. Also, for some authentication methods used (like t
encryption), aggressive mode is known to be vulnerable to denial

Main Mode
Figure 7 shows the basic main-mode message exchanges. In main m
negotiating parties use six messages: the first two messages to nego
policy that will be used to protect the phase II messages. The next tw
perform a Diffie-Hellman key exchange and pass nonces (random nu
signing) to each other. The last two messages are used to authentica

There are four methods specified in the IKE standard to

peers. They are: 6/27/2013
IPSec VPN Fundamentals Page 10 of 15

Pre-Shared Keys (PSK): As the name suggests, a shared

of-band to the peers. The peers use this
a hash that is used to authenticate messages.

Digital Signatures (RSA or DSS): Here, certificates of the peers

the last two messages and hashes are calculated over these cer
authenticate each other. There are many protocols and standard
ease the process of certificate enrollment, certificate request, and
checking. Some popular ones are RSA Labs' PKCS #7, PKCS
Certificate Enrolment Protocol (SCEP), and Online

RSA Public Key Encryption: With this, each negotiating party h

key-pair that they use to encrypt/decrypt messages starting from
Nonces exchanged are secured through this and the parties calc
these nonces to authenticate each other.

Revised RSA Public Key Encryption:

but reduces the number of public key operations from four to two
incorporates two symmetric-key operations.

The authentication data payload shown in

described modes. For example, for PSK mode the previously created
an RSA digital-signature mode, the certificate and the digital signatur
authentication data. 6/27/2013
IPSec VPN Fundamentals Page 11 of 15

Figure 7: IKE main mode message


Aggressive Mode
In aggressive mode, you need only three messages to establish the I
the identities of the parties involved are revealed.

Figure 8: IKE aggressive mode message

exchange 6/27/2013
IPSec VPN Fundamentals Page 12 of 15

2. Phase II (Key Exchange)

Phase II, also called as the "Quick Mode," is used to
new keying material. A full Diffie-Hellman key exchange may be done to p
forward secrecy (PFS), otherwise the keys are derived from the phase I k
basic quick-mode message exchanges are illustrated in

Figure 9: IKE quick mode message


IPsec Packet Processing

The following two flowcharts describe the IPSec operation on the security dev
outbound packets.

Inbound Packet Processing

Figure 10 shows the functional flowchart for inbound packets, in other words,
packets that arrive at the security gateway. These are the packets
public network, like the Internet to the private network. The packets have to be
(decrypted), authenticated, and forwarded to the private network. As the selec
goes deep into the IP protocol fields, such as the port numbers, the packet ma
through the IP reassembly process before the IPSec processing

Outbound Packet Processing

Figure 11 shows the functional flowchart for packets that arrive from the
to another private network across the Internet. These
does affect the MTU, as it adds more bytes with headers, so the packets may
fragmented after the IPSec processing. IPSec is not affected by fragmentation
security device does have to participate in the PMTU discovery process becau
headers. 6/27/2013
IPSec VPN Fundamentals Page 13 of 15

IPSec and NAT Interaction

IPSec and NAT are inherently not compatible protocols, as IPSec protects
integrity, whereas NAT, as a protocol, changes the IP header and TCP/UDP h
and NAT operations are supported in the same security device, then the probl
by performing the NAT operation before doing IPSec and making sure that
are in the public address space.
For scenarios where there is a NAT device in-between (NAT
standards group at IETF has proposed a new protocol called "NAT traversal
the IPSec packet is encapsulated within a UDP packet using the IKE UDP por
of NAT-T support of the peers as well as detection of NAT presence in the pa
IKE phase-I.

Engineering A High-Performance Security Gateway

Encryption and authentication are extremely compute
particular DES implementation in C optimized for the
a single 64-bit block. This only corresponds to a speed of 28 Mbps on a 20
platform. Phil Karn has written highly tuned assembly code for the Intel Pen
that achieves around 24 Mbps with 3DES on a Pentium II running at 400 MHz
gateway that must perform at wire speed with 64-byte packets cannot scale m
throughput range with a software-only implementation. Specialized ASICs that
cryptographic functions, including encryption and authentication,
These devices are necessary to scale to higher throughput rates. There are d
crypto accelerators available in the market. At this point of time, these crypto
to fall into three general categories:
Processors with Basic Algorithm Support
These processors perform basic symmetric-key operations such
and hash operations such as SHA1, MD5, and

Packet Processors
These take in a packet along with an SA and do the complete packet pro
example, the addition of the AH or ESP header, as required) in addition to 6/27/2013
IPSec VPN Fundamentals Page 14 of 15

prior functionality.

Inline Security Coprocessors

These processors can handle SA lookup and packet handling, as well as
You can choose the right kind of crypto accelerators with the right price/perfor
for your system. For example, if you are building a 1
all of the security processing from your main network processor, you should u

IPSec is a thorough and complete solution for protecting IP traffic. IPSec prote
against unauthorized modification and eavesdropping, and also securely
communicating parties. Because of the cost-savings potential
interest in security, the VPN market is still growing strongly despite a nearly tw
economic downturn. Infonetics Research has forecasted
expenditures to grow 117%, from $21.3 billion to $46.2 billion between 2002 a
certain problem areas that need to be addressed are as follows.
The protocol is as strong as the underlying algorithms it employs, so proper a
important to network security. Furthermore, for smooth operation, IPSec requi
infrastructure (PKI). Such infrastructures are still in their infancy, and
infrastructures are just emerging on the Internet. All
access policies is a complicated field and a thorough knowledge of several co
to properly administer these policies. Finally, as IPSec and IKE are evolving s
interoperability between different vendors is still a problem.
Still, the overall impact of IPSec will be tremendous in the years to come. IPS
to securely connect offices, users, and partners to the network and safely tran
a very cost-effective solution. IPSec does this in a manner
users. As the networks migrate to Ipv6, IPSec will become an integral compon
networks as well. 6/27/2013
IPSec VPN Fundamentals Page 15 of 15

MORE EE TIMES Subscriptions Newsletters Editorial Calendar RSS Feeds

GLOBAL NETWORK EE Times Asia EE Times China EE Times Europe EE Times India EE Times Japan
EDN Asia EDN China EDN Japan TechOnline India ESC Brazil

FEATURED UBM TECH SITES: EE Times | EBN | EDN | | Design News | Embedded | TechOnline
OUR MARKETS: Business Technology | Channel | Electronics | Game & App Development
Working With Us: Advertising Contacts | Event Calendar | Tech Marketing Solutions | Corporate Site | Contact Us / Feedback
Terms of Service | Privacy Statement | Copyright 2013 UBM Tech, All rights reserved 6/27/2013