ISEC 311: Network Security

Virtual Private Network (VPN)

College of Information Technology (CIT)
United Arab Emirates University
Spring 2023
 The Concern: How to Keep Data Secure
During Transmission?

Hacker Internet

Hacker

The solution: Virtual Private Networking (VPN)

 Outline
Basic concepts of how a VPN works
Basic encryption technologies that a VPN uses
Introduction to Internet Security (IPSec)
Protocol (a standard for VPN networking)
Other protocols for virtual networking
Example of VPN configuration
 VPN Basics
• A VPN is a connection
• that is established over an existing “public” or shared
infrastructure
• using encryption or authentication technologies to
secure its payload.
• This creates a “virtual” segment between any two
entities that have access
• A powerful tool that can be used to create a secure
communication channel.
• VPN can be categorized into three basic
configuration types:
Host-to-Host
Host-to-Gateway
Gateway-to-Gateway
 Basic VPN Configuration Types
Host-to-Host

Host-to-Gateway

VPN gateway
(Router or firewall)
LAN

Gateway-to-Gateway

VPN gateway VPN gateway

(Router or firewall) VPN Tunnel (Router or firewall)
 Basic VPN Methodology
• The basic concept behind a VPN is securing
communication channel with encryption.
• The communication can be safeguarded through
encryption at many different layers of the network,
such as the following:

Application (FTP…)

Transport (TCP,UDP)
Network (IP)
Data Link (Ethernet)
 Application Layer Encryption
Encryption can be applied
Application with programs such as
Pretty Good Privacy
(PGP), or through
channels such as Secure
Encrypted Shell (SSH)
 Transport Layer Encryption
• Protocols such as Secure Sockets Layer
(SSL) can be used to protect the contents
Transport (TCP,UDP) (Application + Transport) of a specific
communication between two parties.
• However, IP packets that carry this
information are available for inspection (i.e.,
Encrypted not encrypted).
 Network Layer Encryption
• Protocols such as IPSec can be used to
encrypt the payload and the TCP/IP
information.
Network (IP) • True endpoint IP addresses can be hidden
if a gateway device such as a router, or a
firewall is doing the encryption using
concept called tunneling.
Encrypted
 Data Link Layer Encryption
Layer 2 Tunneling Protocol (L2TP) is an
addition to Point-to-Point Protocol (PPP),
Data Link (Ethernet) which allows the encryption of packets sent
over PPP on the Data Link layer.

Encrypted
 VPN Tunneling
Tunneling is the process of encapsulating one type of packet
inside another to facilitate some sort of transport advantage

Gateway-to-Gateway

IP_2 IP_3
VPN gateway VPN gateway
(Router or firewall)
VPN Tunnel (Router or firewall)

Not encrypted IP_4

IP_1 Not encrypted Encrypted packet
packet
packet

Source IP = IP_1 Source IP = IP_2 Source IP = IP_1

Destination IP = IP_4 Destination IP = IP_3 Destination IP = IP_4
 VPN Tunneling (CONTD)

 The hosts have no knowledge of the fact that the

packets are being encrypted.

 No special software or configuration are required

for the hosts.

 However, we may still know which network is

communicating with which network.
© 2017 Pearson Education, Ltd., All rights reserved.
Protocols That Offer VPN

 IPSec
 PPTP
 L2TP
IPSec & VPN

15
 IPSec Basics
• The security measures that were incorporated into Ipv6
• Also available at Ipv4 as an optional protocol suite.
• This set of protocols is known as IPSec Protocol Suite.
• Goal is to facilitate the confidentiality, integrity, and
authentication of information communicated using IP.
• This is accomplished through the use of several protocols,
including:
• Internet Key Exchange (IKE)
• Encapsulating Security Payload (ESP)
• Authentication Header (AH)
• These three protocols combine to allow secure exchange
of information without fear of outside attack/tempering.
 IPSec Basics (contd)
To understand how two parties communicate using
IPSec, we need to study the followings:
Security Association (SA): An agreement between the
communicating parties about encryption, mode, protocol
Communication modes: Transport or Tunnel mode
Protocols used:
 IKE – Authenticator and negotiator for the connection
 AH – Authentication and integrity protection
 ESP – Authentication, integrity and confidentiality protection

17
 IPSec Basics (contd)
Security Association (SA):
Communication modes:
Protocols used:

18
 Security Association (SA)
• An SA is an agreement between two entities
• on how they will securely transmit information
• IPSec supports
• multiple protocols and communications modes, and
• various encryption and hash types.
• All of these details (i.e., which protocol/mode etc.)
• must be pre-negotiated
• before the secure exchange of user’s data can begin.
• The resultant agreement is an SA.
• Each communication session has two SAs
• one for each communication partner.
• one per direction
• Each partner negotiates a new SA
• for every IPSec connection it makes.
 Databases for SA: SPD and SAD
• Before an SA is negotiated,
• the particulars that an IPSec partner is going to
support must be configured for it locally.
• These settings are stored in what is known as a
security policy database (SPD).
• After the SA has been negotiated,
• it (i.e., the SA) is stored in a security association
database (SAD).
• Different communication rules can be configured
• for each of the sessions initiated by a host or device
Security Association (SA) Example
 Security Association (SA) Example
• Cisco PIX Firewall can be set up to allow
• Data Encryption Standard (DES) or
• 3DES as the encryption algorithm for VPN tunnel.
• Host 1 might only support DES
• and would negotiate an SA with DES encryption.
• Host 2 might require a 3DES tunnel
• because of their own business and security requirements.
• Each of these negotiated connections would require
• its own SA entry in the SAD
• listing all the specific details of what was negotiated for each.

 Encryption algorithm negotiated (DES, 3DES, and so on)

 VPN mode (Transport or Tunnel modes)
 Security protocol negotiated (ESP or AH)
 Hash algorithm negotiated (MD5 or SHA-1), and so on.
 Security Parameter Index (SPI)
• Each SA session has its own singular identifier: Security
Parameter Index (SPI)
• SPI tells:
• which SA database entry is used for the connection
• the destination address of the connection
• the protocol identifier for ESP (50) or AH (51) protocols
 IPSec Basics (contd)
Security Association (SA):
Communication modes:
Protocols used:

24
 IPSec Modes: Transport & Tunnel
 Encryption algorithm negotiated (DES, 3DES, and so on)
 VPN mode (Transport or Tunnel modes)
 Security protocol negotiated (ESP or AH)
 Hash algorithm negotiated (MD5 or SHA-1), and so on.
IPSec connection has two basic modes:

Transport Tunnel

• Host-to-Host communication • Gateway-to-gateway communication

• Encryption of the packet’s
payload only
• Software needs to be loaded on
all communicating hosts -
Administrative overhead
• Encryption of the entire packet
• Partially or completely hides source &
destination addresses
• No additional software/setup needed
for hosts
 IPSec Transport Mode
• This VPN mode is well-suited for
• encrypted communications between hosts on the
same network,
• or in situation where it is important to be able to
differentiate hosts by their IP addresses since
headers are not encrypted.
• Drawback of this is mode is:
• Hosts IP addresses can be seen.
 IPSec Tunnel Mode
• This is the method of choice for most VPNs because
• it encrypts the entire original packet including payload
• partially or completely hides the source and destination addresses
of the communicating systems.
• It is a Gateway-to-Gateway form of communication
• Allows network-to-network communication
• Setup is easier than transport mode as
• Only gateway devices, such as Firewalls or routers need to be setup
• no special setup / software needed for communicating hosts

Gateway-to-Gateway
IP_2 IP_3
NetScreen NetScreen
gateway
VPN Tunnel gateway

IP_4
IP_1 Encrypted Not encrypted
Not encrypted
packet
packet packet
 IPSec Basics (contd)
Security Association (SA):
Communication modes:
Protocols used:

28
Internet Key Exchange (IKE) Protocol
• IKE protocol is the authenticator and the negotiator of IPSec
• It verifies that you are someone who
• should be allowed to start encrypted communication with the device
in question
• and then negotiates the type of encryption that will be used.

IKE transaction

Produce

Creation of an SA between
communication partners
 IPSec Security Protocols
• The security protocols used by IPSec are:
• Authentication Header (AH) and
• Encapsulating Security Payload (ESP)
• When building an IPSec-based VPN
• you can elect to apply either one of these
• or use both AH and ESP at the same time
• In practical applications, ESP is more popular.
 IPSec Security Protocol: AH
• Offers :
• authentication
Payload
• Integrity-checking capabilities
• NO confidentiality for the packet’s payload AH header
• provides authentication and integrity protection
• by adding an additional header to the IP packet. IP header
• This header contains a digital signature
• called an integrity check value (ICV)
• that is a hash value verifying that the packet hasn’t been
changed in transit.
• The IP information in the packet
• is guaranteed to be correct, but it is not hidden
• We can be sure that
• the source IP address on the packet is authentic
• and that the packet came from where it is been claimed.
 IPSec Security Protocol: ESP
• Offers :
• Full confidentiality by completely encrypting the payload of
IP packet.
• authentication as needed
• Integrity-checking capabilities ESP packet

Original Original
packet packet

• ESP protocol can use

• any symmetric encryption algorithms to encrypt its
payload: DES, 3DES, AES.
 IPSec Security Protocol: ESP (contd)
In Transport mode:
• ESP simply add its own header after the IP header
• and encrypt the rest of the packet information from layer 4
(Transport) up.
• If ESP’s authentication service is specified
• during the initial negotiation of the IPSec connection,
• ESP then adds a trailer that contains ICV information to
confirm packet integrity and authentication.

In Tunnel mode:
• ESP encapsulates the entire original packet
• encrypting it fully and
• creating a new IP header and ESP header at the tunneling device
• A trailer is also added for authentication purposes if ESP’s
authentication service is chosen.
Applied to the entire
IP packet including
certain fields of the
IP header
© 2017 Pearson Education, Ltd., All rights reserved.
© 2017 Pearson Education, Ltd., All rights reserved.
Other VPN Protocols: PPTP and L2TP
• PPTP and L2TP are both popularly implemented VPN
protocols At layer 2.
• PPTP and L2TP are both included with MS Windows OS.
• PPTP and L2TP do not have encryption capabilities.
• Encryption must be added to make either a true VPN protocol.
• L2TP replaces PPTP as the VPN protocol of choice included
with Microsoft Windows OS.

L2TP
combines the best attributes of both

PPTP Cisco’s Layer Two Forwarding

(L2F) protocol