Internet Routing

Jennifer Rexford
Princeton University
http://www.cs.princeton.edu/~jrex/bgp-tutorial

Local Control vs. Global Properties

The Internet is a network of networks

~40,000 separately administered networks

Competitive cooperation for e2e reachability

Local Control
Intradomain routing,
interdomain policies

Global Properties
Performance, security,
reliability, scalability

Outline of the Tutorial

Internet addressing and routing architecture
IP address allocation and packet forwarding
Two-tiered Internet routing architecture

Border Gateway Protocol (BGP)

Policy-based path-vector routing on IP prefixes

BGP routing policy and example applications of BGP

BGP security

Security vulnerabilities and examples

Anomaly detection and secure extensions to BGP

BGP convergence

Path exploration and convergence delay

Protocol oscillation and the influence of routing policy

Internet Addressing
and Routing Architecture

Goals of This Section

Internet addressing and forwarding
Hierarchical addressing
Hierarchical address allocation
Longest prefix match forwarding
Growth in number of prefixes over time

Two-tiered Internet routing architecture

Autonomous Systems and AS topology
Interdomain vs. intradomain routing
Classes of routing protocols

Hierarchical Addressing

IP Address (IPv4)
A unique 32-bit number
Identifies an interface (on a host, router, )
Represented in dotted-quad notation
12

34

158

00001100 00100010 10011110 00000101

7

Grouping Related Hosts

The Internet is an inter-network

Used to connect networks together, not hosts

Needs to address a network (i.e., group of hosts)
host

host ...

host

host

host ...

host

LAN 2

LAN 1
router

WAN

LAN = Local Area Network

WAN = Wide Area Network

router

WAN

router

Scalability Challenge
Suppose hosts had arbitrary addresses
Every router would need a lot of information
to direct packets toward every host
1.2.3.4

5.6.7.8

host

host ...

2.4.6.8
host

1.2.3.5

5.6.7.9

host

host ...

2.4.6.9
host

LAN 2

LAN 1
router

WAN

router

WAN

router

1.2.3.4
1.2.3.5

forwarding table

The solution:
Introduce hierarchy
9

Hierarchical Addressing: IP Prefixes

Divided into network & host portions (left and right)
12.34.158.0/24 is a 24-bit prefix with 28 addresses
12

34

158

00001100 00100010 10011110 00000101

Network (24 bits)

Host (8 bits)

10

IP Address and a 24-bit Subnet Mask

Address
12

34

158

00001100 00100010 10011110 00000101

11111111 11111111 11111111 00000000

Mask

255

255

255

0
11

Scalability Improved: Smaller Tables

Number related hosts from a common subnet
1.2.3.0/24 on the left LAN
5.6.7.0/24 on the right LAN

1.2.3.4

1.2.3.7 1.2.3.156
host ...

host

5.6.7.8 5.6.7.9 5.6.7.212

host

host

host ...

host

LAN 2

LAN 1
router

WAN

router

WAN

router

1.2.3.0/24
5.6.7.0/24

forwarding table

12

Scalability Improved: Fewer Updates

No need to update the routers

E.g., adding a new host 5.6.7.213 on the right

Doesnt require adding a new forwarding-table entry

1.2.3.4

1.2.3.7 1.2.3.156
host ...

host

5.6.7.8 5.6.7.9 5.6.7.212

host

host

host ...

host

LAN 2

LAN 1
router

WAN

router

WAN

router

host

5.6.7.213
1.2.3.0/24
5.6.7.0/24

forwarding table

13

Hierarchical Address Allocation

14

Classful Addressing
In the olden days, only fixed allocation sizes
Class A: 0*

Very large /8 blocks (e.g., MIT has 18.0.0.0/8)

Class B: 10*

Large /16 blocks (e.g,. Princeton has 128.112.0.0/16)

Class C: 110*

Small /24 blocks (e.g., AT&T Labs has 192.20.225.0/24)

Class D: 1110*

Multicast groups

Class E: 11110*

Reserved for future use

This is why we use dotted-quad notation!

15

Classless Inter-Domain Routing (CIDR)

Use two 32-bit numbers to represent a network.
Network number = IP address + Mask

IP Address : 12.4.0.0
Address

Mask

IP Mask: 255.254.0.0

00001100 00000100 00000000 00000000

11111111 11111110 00000000 00000000
Network Prefix
Written as 12.4.0.0/15

for hosts

16

CIDR: Hierarchal Address Allocation

Prefixes are key to Internet scalability

Address allocated in contiguous chunks (prefixes)

Routing protocols and packet forwarding based on prefixes
Today, routing tables contain ~300,000 prefixes
12.0.0.0/16
12.1.0.0/16
12.2.0.0/16
12.3.0.0/16
12.0.0.0/8

:
:
:
12.254.0.0/16

12.3.0.0/24
12.3.1.0/24

:
:

:
:
:

12.3.254.0/24
12.253.0.0/19
12.253.32.0/19
12.253.64.0/19
12.253.96.0/19
12.253.128.0/19
12.253.160.0/19

17

Obtaining a Block of Addresses

Separation of control

Prefix: assigned to an institution

Addresses: assigned by the institution to their nodes

Who assigns prefixes?

Internet Corporation for Assigned Names and Numbers

Allocates large address blocks to Regional Internet Registries

Regional Internet Registries (RIRs)

E.g., ARIN (American Registry for Internet Numbers)
Allocates address blocks within their regions
Allocated to Internet Service Providers and large institutions

Internet Service Providers (ISPs)

Allocate address blocks to their customers
Who may, in turn, allocate to their customers

18

Figuring Out Who Owns an Address

Address registries

Public record of address allocations

Internet Service Providers (ISPs) should update
when giving addresses to customers
However, records are notoriously out-of-date

Ways to query
UNIX: whois h whois.arin.net 128.112.136.35
http://www.arin.net/whois/
http://www.geektools.com/whois.php

19

Example Output for 128.112.136.35

OrgName: Princeton University
OrgID: PRNU
Address: Office of Information Technology
Address: 87 Prospect Avenue
City: Princeton
StateProv: NJ
PostalCode: 08544-2007
Country: US
NetRange: 128.112.0.0 - 128.112.255.255
CIDR: 128.112.0.0/16
NetName: PRINCETON
NetHandle: NET-128-112-0-0-1
Parent: NET-128-0-0-0-0
NetType: Direct Allocation
RegDate: 1986-02-24

20

Scalability: Address Aggregation

Provider is given 201.10.0.0/21

Provider

201.10.0.0/22

201.10.4.0/24

201.10.5.0/24

201.10.6.0/23

Routers in the rest of the Internet just need to know

how to reach 201.10.0.0/21. The provider can direct the
IP packets to the appropriate customer.
21

But, Aggregation is Not Always Possible

201.10.0.0/21

Provider 1

Provider 2

201.10.0.0/22 201.10.4.0/24 201.10.5.0/24 201.10.6.0/23

Multi-homed customer with 201.10.6.0/23 has two

providers. Other parts of the Internet need to know how
to reach these destinations through both providers.
22

Load Balancing and Backup Routes

Provider 1

Provider 2

201.10.6.0/23
201.10.7.0/24

201.10.6.0/23
201.10.6.0/24
201.10.6.0/23

Multi-homed customer deaggregates its address block

for more control over load balancing and backup routes
23

CIDR Makes Packet Forwarding Harder

Hierarchical addressing vs. fast packet forwarding
CIDR allows efficient use of the limited address space
But, CIDR makes packet forwarding much harder

Forwarding table may have multiple matches

E.g., table entries for 201.10.0.0/21 and 201.10.6.0/23

The IP address 201.10.6.17 would match both!
201.10.0.0/21

Provider 1

201.10.0.0/22 201.10.4.0/24 201.10.5.0/24 201.10.6.0/23

Provider 2

24

Longest Prefix Match Forwarding

Forwarding tables in IP routers

Maps each destination IP prefix to next-hop link(s)

Destination-based hop-by-hop forwarding

Packet has a destination address
Router identifies longest-matching prefix
Cute algorithmic challenge: very fast lookups
forwarding table
destination
201.10.6.17

4.0.0.0/8
4.83.128.0/17
201.10.0.0/21
201.10.6.0/23
126.255.103.0/24

outgoing link

Serial0/0.1
25

Scalability Through Hierarchy

Hierarchical addressing
Critical for scalable system
Dont require everyone to know everyone else
Reduces # of updates when things changes

Non-uniform hierarchy
For heterogeneous networks of different sizes
Initial class-based addressing was far too coarse
Classless InterDomain Routing (CIDR) helps

Yet, many practical needs are leading to a

proliferation of prefixes

26

Growth in the Number of

Globally-Visible Prefixes

27

Pre-CIDR (1988-1994): Steep Growth

Growth faster than improvements in equipment capability

28

CIDR Deployed (1994-1996): Much Flatter

Efforts to aggregate (even decreases after IETF meetings!)

29

CIDR Growth (1996-1998): Roughly Linear

Good use of aggregation, and peer pressure in CIDR report

30

Boom Period (1998-2001): Steep Growth

Internet boom and increased multi-homing

31

Long-Term View (1989-2005): Post-Boom

32

Prefix Scalability Challenges

Rapid increase in the number of prefixes

New ASes coming online

Existing ASes acquiring new address blocks
Single-homed ASes becoming multi-homed
ASes doing load balancing and backup routes

Now up to around 300,000 prefixes

Challenge for forwarding IP data packets
Challenge for storing and computing routes

Ongoing research and standards work

Separation of identity and location

33

Running out of IP Addresses

Not all that many unique addresses

232 = 4,294,967,296 (just over four billion)

Plus, some are reserved for special purposes
And, addresses are allocated in larger blocks

And, many devices need IP addresses

Computers, PDAs, routers, tanks, toasters,

Long-term solution: a larger address space

IPv6 has 128-bit addresses (2128 = 3.403 1038)

Short-term solutions: limping along with IPv4

Private addresses
Network address translation (NAT)
Dynamically-assigned addresses (DHCP)

34

Internet Routing Architecture

35

Goals of This Section

Internet structure

Autonomous Systems (ASes)

Business relationships between ASes
Structure of the AS-level topology

Routing architecture

Two-tiered routing architecture

Intradomain: among cooperating routers
Interdomain: among competing ASes

Classes of routing protocols

Link-state routing, distance-vector routing,

source routing, and path-vector routing

36

Internet Structure

37

Autonomous Systems (ASes)

Divided into Autonomous Systems
Distinct regions of administrative control
Routers/links managed by a single institution
Service provider, company, university,

Hierarchy of Autonomous Systems

Large, tier-1 provider with nationwide backbone
Medium-sized regional provider
Small network for a company or university

But they must cooperate for e2e reachability

38

Autonomous System Numbers (ASNs)

AS Numbers are 16 bit values.
Currently around 40,000 in use.

Level 3: 1
MIT: 3
Harvard: 11
Yale: 29
Princeton: 88
AT&T: 7018, 6341, 5074,
UUNET: 701, 702, 284, 12199,
Sprint: 1239, 1240, 6211, 6242,

39

AS-Level Topology
Node: Autonomous System
Edge: Two ASes that connect to each other

4
3
5
2

40

What is an Edge, Really?

Edge in the AS graph
At least one connection between two ASes
Some destinations reached from one via other
d
d
AS 1

AS 1

Exchange Point
AS 2

AS 2

AS 3

41

Business Relationships Between ASes

Neighboring ASes have business contracts
How much traffic to carry
Which destinations to reach
How much money to pay

Common business relationships

Customer-provider
Peer-peer
Backup
Sibling
42

Customer-Provider Relationship
Customer needs to be reachable from everyone

Provider ensures all neighbors can reach the customer

Customer does not want to provide transit service

Customer does not let its providers send traffic through it

Traffic to the customer

Traffic from the customer

d

provider

provider
traffic

customer

customer

43

Peer-Peer Relationship
Peers exchange traffic between customers
AS lets its peer reach (only) its customers
AS can reach its peers customers
Often the relationship is settlement-free (i.e., no $$$)
Traffic to/from the peer and its customers

peer

traffic

peer

44

AS Structure: Tier-1 Providers

Top of the Internet hierarchy

Has no upstream provider of its own

Typically has a large (inter)national backbone
Around 10-12 ASes: UUNET, AT&T, Level 3,
peer-peer

peer-peer

peer-peer

peer-peer

45

AS Structure: Other ASes

Lower-layer providers (tier-2, )

Provide transit service to downstream customers

But need at least one provider of their own

Typically have national or regional scope

E.g., Minnesota Regional Network

Includes a few thousand ASes

Stub ASes

Do not provide transit service

Connect to upstream provider(s)
Most ASes (e.g., 85-90%)
46

Routing Architecture

47

Two-Tiered Routing Architecture

Goal: distributed management of resources
Internetworking of multiple networks
Networks under separate administrative control

Intradomain: inside a region of control

Routers configured to achieve a common goal
Okay for routers to share topology information
Different ASes can run different protocols

Interdomain: between regions of control

ASes have different (maybe conflicting) goals
Routers only share reachability information

48

Intradomain Routing: Shortest Path

Routers belong to the same institution
Share a common, network-wide goal

Metric-based routing protocols

Typically shortest-path routing
With configurable link weights
2
3
2

3
5

49

Intradomain Routing: Tunneling

Routers belong to the same institution
Share a common, network-wide goal

Tunneling based solutions

Pinning path(s) between ingress-egress routers
Chosen based on load, reliability, delay,

50

Interdomain Routing: Path-Based

Routers belong to different institutions

No common goal, reluctant to share information

But must cooperate to reach remote destinations

Policy-based path selection

AS selects a path through one of its neighbors
Optionally makes the path available to others
4

5
2

51

Classes of Routing Protocols

52

Forwarding vs. Routing

Forwarding: data plane

Directing a data packet to an outgoing link

Individual router using a forwarding table

Routing: control plane

Computing paths the packets will follow
Routers talking amongst themselves
Individual router creating a forwarding table

53

Shortest-Path Routing
Path-selection model

Destination-based
Load-insensitive (e.g., static link weights)
Minimum hop count or sum of link weights

Used mainly for intradomain routing

Routers share common goal

Main approaches
Link-state routing
Path-vector routing

3
2

1
1
4

4
5
3

54

Shortest-Path Problem
Compute: path costs to all nodes
From a given source u to all other nodes
Cost of path through each outgoing link
Next hop along the least-cost path to s
2
3
u

2
6

4
s

3
55

Link-State Routing
Flooding of topology information

Routers share complete topology information

Shortest-path computation

Routers compute shortest paths to all dests

Running Dijkstras algorithm on full topology

Next-hop forwarding
Router forwards packets to
next hop in (shortest) path

Examples: OSPF and IS-IS

3
2

1
1
4

4
5
3

56

Distance-Vector Routing
Dissemination of path-cost information

Routers share only path costs with neighbors

Shortest-path selection

Routers add link cost to compute new path cost

Bellman-Ford algorithm to select shortest paths

Next-hop forwarding
Router forwards packets to
next hop in (shortest) path

Examples: RIP and EIGRP

3
s

5
d

3
57

Source Routing
Flooding of topology information

Routers share complete topology information

End host or edge router computes path

Potentially any path through the network

Maximizes flexibility for the host or edge router

Forwarding along the chosen path

Packets carry the list of hops in the path

Examples: IP source routing, s

RSVP to establish tunnel

58

Path-Vector Routing
Extension of distance-vector routing
Support flexible routing policies
Avoid count-to-infinity problem

Key idea: advertise the entire path

Distance vector: send distance metric per dest d
Path vector: send the entire path per dest d
1

Next-hop forwarding
Forward packets to next hop

Example: BGP

2
3

s
5

d
6

59

Intradomain vs. Interdomain

Intradomain routing

Amongst the routers inside an AS

Cooperating to optimize a common objective
Shortest-path routing, optimization of tunnels,
Different ASes can use different protocols

Interdomain routing
Between different ASes
Cooperating only for end-to-end reachability
Policy-based path selection
Different ASes need to run a common protocol
60

Conclusions
IP address

A 32-bit number
Allocated in prefixes
Non-uniform hierarchy (for scalability & flexibility)

Scalability challenges
Overhead of 300,000 prefixes on IP routers
Running out of IPv4 addresses

Internet routing architecture

Intradomain: routers share a common goal
Interdomain: ASes have different objectives

61