You are on page 1of 335

Introduction to MPLS

Gary Day

MPLS Training - Basic

2005 Cisco Systems, Inc. All rights reserved. Version 2.0 Oct-2005

Cisco Confidential

Business Drivers for MPLS

2003 Cisco Systems, Inc. All rights reserved.

Changing Telecom Landscape


Old World Infrastructure Traffic Services Focus Private Networks Business Networks OSS Circuit-Switched Voice-Centric Transport FR-Based VPNs In-House Network-Based New World Packet-Switched Data-Centric IP Value-Added IP-Based VPNs Outsourced Service-Based

MPLS Training - Basic

Customer Requirements

IP Intranet

IP Extranet

Remote Offices

Customers Suppliers Partners

MPLS Training - Basic

Telecommuters Mobile Users

Service Provider Requirements


Content Hosting Private Voice Networks

Managed Intranets Service Portfolio

Multimedia

MPLS Training - Basic

The Barriers

Frame Relay and ATM services are available: They provide connectionoriented service They have inflexible point-topoint bandwidth guarantees But they have good privacy

Carriers customers want IP services: They need connectionless IP services They need more flexible IP quality of service guarantees They need more privacy than the Internet provides
6

MPLS Training - Basic

The Solution - MPLS


MULTI-PROTOCOL LABEL SWITCHING A mechanism that delivers the best of both worlds:
PRIVACY and QOS of ATM, Frame Relay FLEXIBILITY and SCALABILITY of IP

Foundation for IP business services


Flexible grouping of users and value-added services

Low cost managed IP services


scales to large and small private networks

MPLS Training - Basic

MPLS Concepts

2003 Cisco Systems, Inc. All rights reserved.

MPLS concepts
Packet forwarding is done based on labels Labels assigned when the packet enters the network Labels inserted between layer 2 and layer 3 headers MPLS nodes forward packets based on the label Separates ROUTING from FORWARDING
Routing uses IP addresses Forwarding uses Labels

Labels can be stacked

MPLS Training - Basic

MPLS Capabilities

2003 Cisco Systems, Inc. All rights reserved.

10

Relevant MPLS Capabilities


The ability to FORWARD on and STACK LABELS allows MPLS to provide some useful features including: IP+ATM Integration
Provides Layer 3 intelligence in ATM switches

Virtual Private Networks


Layer 3 Provider has knowledge of customer routing Layer 2 Provider has no knowledge of customer routing

Traffic Engineering
Force traffic along predetermined paths

MPLS Training - Basic

11

Traditional IP over ATM

Put routers around the edge of an ATM network Connect routers using Permanent Virtual Circuits This does not provide optimal integration of IP and ATM
MPLS Training - Basic 12

MPLS VPN Layer 3


Private, connectionless IP VPNs Outstanding scalability Customer IP addressing freedom Multiple QoS classes
VPN A VPN B VPN C VPN A VPN C VPN B

Connection-Oriented VPN Topology

Secure support for intranets and extranets Easy to provide Intranet/Extranet/ 3rd Party ASP Support over any access or backbone technology
Determines VPN on PE Router

VPN A

VPN B VPN C VPN A VPN B VPN C

Connectionless VPN Topology

VPN B
VPN C

VPN A VPN C VPN B

VPN A

Determines PE Router

VPN A VPN B VPN C VPN A VPN B VPN C

IP Packet
MPLS Training - Basic

VPN Label

IGP Label

13

Why Providers like MPLS VPN


vs

MPLS VPN Network

Build once, Sell once

Build once, Sell many

MPLS Training - Basic

14

MPLS VPN Layer 2


Additional Capabilities:
Virtual leased line service Offer PVC-like Layer 2-based service
L2 Pseudowire/Emulated VC L2 Frames Attachment Circuit Attachment Circuit

Reduced costconsolidate multiple core technologies into a single packet-based network infrastructure Simpler provisioning of L2 services Attractive to Enterprise that wish keep routing private
Determines VC inside the tunnel

Determines PE Router end point

L2 Frame
MPLS Training - Basic

VC Label

Tunnel Label
15

Traffic Engineering
Why traffic engineer?
Optimise link utilisation Specific paths by customer or class Balance traffic load
Route chosen by IP routing protocol Route specified by traffic engineering

Traffic follows pre-specified path Path differs from normally routed path Controls packet flows across a L2 or L3 network
Determines LSP next hop contrary to IGP

IP Packet
MPLS Training - Basic

VPN Label

IGP Label

TE Label
16

MPLS Components

2003 Cisco Systems, Inc. All rights reserved.

17

MPLS Components
Edge Label Switching Routers (ELSR or PE)
Label previously unlabeled packets - at the beginning of a Label Switched Path (LSP) Strip labels from labeled packets - at the end of an LSP

Label Switching Routers (LSR or P)


Forward labeled packets based on the information carried by labels

MPLS Training - Basic

18

MPLS Components
CE PE LSR P LSR PE CE

ELSR

ELSR

ELSR

ELSR

LSR

LSR

C Network (Customer Control)

P Network (Provider Control)

C Network (Customer Control)

MPLS Training - Basic

19

Functional Components
Forwarding component:
Uses label information carried in a packet and label binding information maintained by a Label Switching Router to forward the packet

Control component:
Responsible for maintaining correct label binding information among Label Switching Routers

MPLS Training - Basic

20

Forwarding Component
Label Forwarding Information Base (LFIB) Each entry consists of:
incoming label outgoing label outgoing interface outgoing MAC address

LFIB is indexed by incoming label LFIB could be either per Label Switching Router or per interface

MPLS Training - Basic

21

Forwarding Component
IOS Label Forwarding Code is based on Cisco Express Forwarding (CEF)
Maintenance of label rewrite structures in LFIB Recursive route resolution IP to label switching (label imposition) path

MPLS Training - Basic

22

Forwarding Component
Forwarding algorithm:
Extract label from a packet Find an entry in the LFIB with the INCOMING LABEL equal to the label in the packet Replace the label in the packet with the OUTGOING LABEL (from the found entry) Send the packet on the outgoing interface (from the found entry)

MPLS Training - Basic

23

Label Header (Shim)


Bit 1 2 3 4 5 6 7 8 1 2 EXP TTL Label EXP S TTL Label Value (20 bits) Class of Service (3 bits) Bottom of Stack (1 bit) Time to Live S 3 4
Byte

Label

Can be used over Ethernet, 802.3, or PPP links Ethertype 0x8847 Four octets per label in stack

MPLS Training - Basic

24

Label Encapsulation
Packet over SONET/SDH Ethernet Frame Relay PVC ATM PVCs Subsequent cells

PPP Ethernet Frame Relay ATM Header ATM Header

Label Label Label Label Data

IP header IP Header IP Header IP Header

Data Data Data Data

F R A M E

Label
ATM label switching Subsequent cells

GFC VPI GFC VPI

VCI VCI Label

PTI CLP HEC IP Header PTI CLP HEC Data

Data

C E L L

MPLS Training - Basic

25

Control Component
Labels can be distributed by several protocols
TDP/LDP from IGP routes RSVP for traffic engineering paths BGP for VPN routes

Responsible for binding between labels and routes: Create label binding (local) Distributing label binding information among Label Switching Routers

MPLS Training - Basic

26

MPLS Forwarding Decisions


Packets are forwarded based on the label value IP header and forwarding decision have been decoupled for better flexibility No need to strictly follow unicast destination based routing Allows to have distinct forwarding decision based on different control component
Destination unicast routing, Traffic Engineering Multicast, VPN, QoS

MPLS Training - Basic

27

Basic MPLS Forwarding

2003 Cisco Systems, Inc. All rights reserved.

28

MPLS: Forwarding

MPLS Training - Basic

29

MPLS: Forwarding
Existing routing protocols (e.g. OSPF, IGRP) establish routes

MPLS Training - Basic

30

MPLS: Forwarding
Label Distribution Protocol (e.g., LDP) establishes label to routes mappings

MPLS Training - Basic

31

MPLS: Forwarding
Label Distribution Protocol (e.g., LDP) creates LFIB entries on LSRs
IN OUT I/F MAC 16 32 S0/0 aa-00-bb 18 27 S0/0 aa-00-cc IN OUT Null Null I/F MAC E0/0 aa-00-bb E0/1 aa-00-cc

IN OUT I/F MAC 32 64 S0/0 aa-00-bb 27 18 S0/1 aa-00-cc

IN OUT 64 POP 65 POP

I/F MAC S0/0 aa-00-bb S0/1 aa-00-cc

MPLS Training - Basic

32

MPLS: Forwarding
Ingress edge LSR receives packet, performs Layer 3 value-added services, and label packets
IN OUT I/F MAC 16 32 S0/0 aa-00-bb 18 27 S0/0 aa-00-cc IN OUT Null Null I/F MAC E0/0 aa-00-bb E0/1 aa-00-cc

IN OUT I/F MAC 32 64 S0/0 aa-00-bb 27 18 S0/1 aa-00-cc

IN OUT 64 POP 65 POP

I/F MAC S0/0 aa-00-bb S0/1 aa-00-cc

MPLS Training - Basic

33

MPLS: Forwarding
LSRs forward labelled packets using label swapping
IN OUT I/F MAC 16 32 S0/0 aa-00-bb 18 27 S0/0 aa-00-cc

IN OUT Null Null -

I/F MAC E0/0 aa-00-bb E0/1 aa-00-cc

IN OUT I/F MAC 32 64 S0/0 aa-00-bb 27 18 S0/1 aa-00-cc

IN OUT 64 POP 65 POP

I/F MAC S0/0 aa-00-bb S0/1 aa-00-cc

MPLS Training - Basic

34

MPLS: Forwarding
Edge LSR at egress removes remaining label* and delivers packet
IN OUT I/F MAC 16 32 S0/0 aa-00-bb 18 27 S0/0 aa-00-cc IN OUT Null Null I/F MAC E0/0 aa-00-bb E0/1 aa-00-cc

IN OUT I/F MAC 32 64 S0/0 aa-00-bb 27 18 S0/1 aa-00-cc

IN OUT 64 POP 65 POP

I/F MAC S0/0 aa-00-bb S0/1 aa-00-cc

* Pentulimate hop popping actually occurs. There may may not necessarily be a label in the packet at the ultimate or egress LSR.
MPLS Training - Basic 35

Basic Application Framed Based MPLS

2003 Cisco Systems, Inc. All rights reserved.

36

Traditional Routing
Route Distribution

1 1 0

128.89

You Can Reach 128.89 thru Me You Can Reach 128.89 and 171.69 thru me 2 171.69

Routing Updates (OSPF, EIGRP)


MPLS Training - Basic

You Can Reach 171.69 thru Me

37

Traditional Routing
Packet Routing

1 1
Data | 128.89.25.4 Data | 128.89.25.4

0
Data | 128.89.25.4

128.89

Data | 128.89.25.4

2 171.69

Packets Forwarded Based on IP Address


MPLS Training - Basic 38

MPLS Forwarding
In/Out Label Fields
Out Label Out Label Out Label

1 1 0

128.89

2 171.69

MPLS Training - Basic

39

Frame Based MPLS


Assigning Labels
Out Label Out Label Out Label

1 1 0

128.89

Pop Label for 128.89 Use Label 27 for 128.89 Use Label 29 for 171.69 2 171.69 Use Label 22 for 171.69

Unsolicited Downstream Label Allocation


MPLS Training - Basic

40

Frame Based MPLS


Packet Forwarding
Out Label Out Label Out Label

1 1
Data Data Data Data 128.89.25.4 171.69.21.7 Data 171.69.21.7 22 128.89.25.4 27 171.69.21.7 29

0
Data 128.89.25.4 Data

128.89
128.89.25.4

Penultimate Hop (Pop the label)


171.69

MPLS Training - Basic

41

Basic Application Hierarchical Routing

2003 Cisco Systems, Inc. All rights reserved.

42

Internet Scalability
Out Label Out Label Out Label

1 1 0

Loopback 150.10.1.1 EBGP I can reach 128.89,136.50 156.50,119.10 via the BGP next hop 150.10.1.1 using only label 18!
MPLS Training - Basic

128.89 136.50 156.50 119.10

EBGP
171.69 127.18 204.162
43

Loopback 150.10.1.2

Basic Application Cell Based MPLS (IP+ATM)

2003 Cisco Systems, Inc. All rights reserved.

44

MPLS and ATM


Label Switching Steps:
Make forwarding decision using fixed-length Label Rewrite label with new value Similar to ATM cell switching

Key differences:
Label set up: LDP vs ATM Forum Signaling Label granularity: Per-prefix

MPLS Training - Basic

45

MPLS and ATM


Common forwarding paradigm
label swapping = ATM switching

Use ATM user plane


use VPI/VCI for labels Label is applied to each cell, not whole packet

Replace ATM Forum control plane with the MPLS control component:
Network Layer routing protocols (e.g., OSPF, BGP, PIM) + Label Distribution Protocol (e.g., LDP)

MPLS Training - Basic

46

Label Distribution for ATM


Uses LDP in Downstream on Demand mode Referred to as Cell Based MPLS (rather than Frame Based MPLS) Label Virtual Circuit (LVC) labels are requested when topology changes Precedence can be associated with Label Virtual Circuit (LVC) Some LDP extensions for negotiation of ATM specific parameters

MPLS Training - Basic

47

Summary and Benefits

2003 Cisco Systems, Inc. All rights reserved.

48

Summary
MPLS allows flexible packet classification and network resources optimisation Labels are distributed by different protocols
LDP, RSVP, BGP

Different distribution protocols may co-exist in the same LSR Labels have local (LSR) significance
No need for global (domain) wide label allocation/ numbering

MPLS Training - Basic

49

Benefits of MPLS
De-couples IP packet forwarding from the information carried in the IP header of the packet Provides multiple routing paradigms (e.g., destination-based, explicit routing, VPN, multicast, CoS, etc) over a common forwarding algorithm (label swapping) Facilitates integration of ATM and IP - from control plane point of view an MPLS-capable ATM switch looks like a router

MPLS Training - Basic

50

LDP

2003 Cisco Systems, Inc. All rights reserved.

51

LDP

2003 Cisco Systems, Inc. All rights reserved.

52

Label Distribution Protocol (LDP)


The fundamental concept in MPLS based networks is the meaning of the label The Label Distribution Protocol (LDP) provides a set of methods that allow an Label Switch Router (LSR) to share a particular label and its association with other LSRs

MPLS Training - Basic

53

LDP Overview
IETF standard protocol RFC 3036
Distributes <label, prefix> bindings for MPLS forwarding along normally routed paths

Runs in parallel with routing protocols Neighbor discovery with UDP (646) Incremental updates over TCP (646) Other label distribution mechanisms can run in parallel Descendent of Cisco proprietary Tag Distribution Protocol (TDP)
54

MPLS Training - Basic

LDP Introduction
LDP is not the only protocol that can share knowledge about labels:
TDP (Cisco specific)

And other protocols have been extended to support label distribution:


BGP RSVP PIM (rfc3107) (draft-ietf-mpls-rsvp-lsp-tunnel-09.txt ) Under development

MPLS Training - Basic

55

Terminology Upstream and Downstream

Label Switch Path (LSP) direction ! (Packet ow) ! Source Destination IP-Prefix

Upstream! platform!

Downstream! platform!

Label binding {Label, IP-Prex}!

MPLS Training - Basic

56

Terminology
Label Information Base (LIB)
A data structure that holds locally assigned labels and labels learned from LDP peers

Label Forwarding Information Base (LFIB)


A data structure and way of managing forwarding in which destinations and incoming labels are associated with outgoing interfaces and labels. The LFIB can be updated by routing changes and label advertisements from peers

Forwarding Equivalence Class (FEC)


Groups of packets that are forwarded over the same Label Switch Path

MPLS Training - Basic

57

LIB and LFIB structures


156.50.20.0 156.50.20.0 156.50.20.0

Label Distribution!

Label Distribution!
S0/2! S0/1! S0/0!

Label Distribution!

Label Information Base (LIB)!


Destination In Label (Peer, Out Label)

Routing Information Base (RIB)!


Destination Interface

156.50.20.0/24

27

(R2:0, 32), (R3:0, 56), (R4:0, 85)

156.50.20.0/24

S0/0

Label Forwarding Information Base (LFIB)!


Destination In Label Out Label Interface

156.50.20.0/24

27

85

S0/0

MPLS Training - Basic

58

Basic Configuration
ip cef mpls ip mpls label protocol ldp mpls ldp router-id loopback0 interface e0/0 ip address 10.10.20.0 255.255.255.0 mpls ip Enables LDP on this interface Use LDP protocol as opposed to TDP

Use loopback when establishing LDP session

MPLS Training - Basic

59

Label Space

2003 Cisco Systems, Inc. All rights reserved.

60

Concepts
LSRs must be able to distinguish between labelled packets
A label corresponds to a particular Forwarding Equivalence Class (FEC)

LSR can distribute the same label/FEC mapping to different neighbours Same label can be assigned to different FECs if and only if the LSR can distinguish the interface from which the packet will arrive
That is, the LSR can identify who the upstream neighbour that inserted the label

MPLS Training - Basic

61

Classes of Label Space


There are two classes of label spaces:
INTERFACE LABEL SPACE the label is specific to a particular interface. This is generally found (but not restricted to) in ATM interfaces in MPLS cell mode which uses the VPI/VCI fields as labels. PLATFORM LABEL SPACE the label value/meaning is not specific to an interface, but can be understood by a number of interfaces on the same box. This is generally found in frame mode (This is the Cisco implementation for Frame Mode)

MPLS Training - Basic

62

Per Interface Label Space


Per interface label space
Label are unique in a per interface base Used over ATM interfaces Label = VCs With interface label space, an LSR will accept labelled packets from upstream neighbours only if the labels have been previously advertised to that neighbour. No label spoofing Useful when interconnecting MPLS domains

MPLS Training - Basic

63

Per Interface Label Space


LFIB on Router C
Destination 156.50.4.0/24 156.50.4.0/24 Incoming I/F ATM 0/0 ATM 1/0 IN VPI/VCI 1/73 1/73 Outgoing I/F ATM 1/3 ATM 1/3 OUT VPI/VCI 1/339 1/342

ATM 0/0 ATM 1/0


C

ATM 1/3

D 156.50.4.0/24

LFIB on an LSR contains incoming interface.! Labels have to be assigned for individual interfaces.! The same label can be reused (with a different meaning) on different interfaces.! Label allocation is secure LSRs cannot send packets with labels that were not assigned to them.!
MPLS Training - Basic 64

Per Platform Label Space


LFIB on Router C
Destination X
A

IN Label X = 25! OUT Label Next Hop 25


C

38

Router D
D

X=25!

E X

X=38!
B

LFIB on a LSR does not contain an incoming interface.! The same label can be used on any interface and is announced to all adjacent LSRs.! The label is announced to adjacent LSRs only once and can be used on any link.! Per-platforms label-space is less secure than per-interface label space.!
MPLS Training - Basic 65

LDP Identifier & Sessions

2003 Cisco Systems, Inc. All rights reserved.

66

LDP Identifier
a! b! c! d! LSR ID! LSR ID
The LSR ID is a four byte number that identifies a specific LSR. These four bytes must be unique in the network. Generally they are derived from an interface on the LSR. In IOS (by default) this is the highest IP address, or highest IP address of a loopback if it is available.

n! Label Space ID!

Label Space ID
A two byte number that identifies a specific label space on the LSR. The label space id 0x00 is reserved for the platform label space (This is the Cisco default for Frame based MPLS)

LDP Identifier
The six byte concatenation of the LSR ID and LABEL SPACE ID results in the LDP Identifier. This uniquely identifies the label space.

Example: 156.50.10.1:0
MPLS Training - Basic 67

LDP Identifier IOS Commands


router#show mpls ldp discovery detail Local LDP Identifier: Local LSR ID, global space 200.200.200.200:0 Discovery Sources: Remote LSR ID discovered Interfaces: Ethernet0/0 (ldp): xmit/recv LDP Id: 10.10.10.10:0 Src IP addr: 100.50.0.2; Transport IP addr: 10.10.10.10

router(config)#mpls ldp router-id loopback0 force Force will change the LSR ID immediately, rather than waiting for reload or current ID being removed

MPLS Training - Basic

68

LDP Session
Each LDP identifier has a separate LDP session per neighbour
Each LSR label space has its own distinct LDP session Multiple links between adjacent routers use the same session

Each session has its own TCP (646) connection and discovery process.

MPLS Training - Basic

69

LDP Sessions and Label Space


Single LDP Session !
1.0.0.1:0! POS! POS! 1.0.0.1:0! 1.0.0.1:0! POS!

Per Platform Label Space!

Two LDP Sessions !

Ethernet! 1.0.0.1:0!

1.0.0.1:10! ATM! ATM! 1.0.0.1:20!

Per Platform Label Space!

Per Interface Label Space!

One LDP session is established for each announced LDP identifier (Router ID + Label Space). The number of LDP sessions is determined by the number of different label spaces.
MPLS Training - Basic 70

LDP Neighbor Discovery

2003 Cisco Systems, Inc. All rights reserved.

71

LDP Neighbor Discovery


Basic Discovery
Directly connected LSRs Discovered through hello packets Sent to multicast all-routers-in-subnet address

Extended discovery
Non-directly connected LSRs (e.g., across TE path) Targeted hello packets to specific address Discovery is asymmetric (one in each direction)

Once discovery is done, LDP sessions are established over TCP (646)
MPLS Training - Basic 72

Basic LDP Discovery


UDP: Hello! (1.0.0.2:1064 224.0.0.2:646)!
TCP (1.0.0.4:1066 1.0.0.2:646)!

B MPLS_B!

1.0.0.2!
NO MPLS !
C NO_MPLS_C!

TCP

43 1.0 (1.0.0.2:10

.0.1:646)!

A MPLS_A!

UDP: Hello! (1.0.0.1:1050 224.0.0.2:646)!

1.0.0.1!
TCP (1. 0 .0.4:106 5 1.0 .0.1:646 )!

1.0.0.3!

UDP: Hello! (1.0.0.4:1033 224.0.0.2:646)!

D MPLS_!

1.0.0.4!
LDP Session is established from the LSR with higher transport address. The establishing router is called the Active LSR.
MPLS Training - Basic 73

Extended LDP Discovery


LDP neighbor discovery of non adjacent neighbors
Differs from normal discovery only in the addressing of hello packets

Targeted hello packets use unicast IP address


Instead of multicast address

Extended discovery is asymmetric Once a neighbor is discovered, the mechanism to establish a session is the same.

MPLS Training - Basic

74

LDP Sessions - Non directly connected LSR


Normally routed path
133.0.0.33

R7!
R1!
118.1.1.1

R6!

R5!

R8!

R9!
Targeted LDP session

R2!

R3 !

R4!
Traffic Engineered Path R1 R8

UDP: Hello! (118.1.1.1:1052 133.0.0.33)! UDP: Hello! (133.0.0.33:1052 118.1.1.1)!


MPLS Training - Basic 75

LDP Identifier IOS Commands


Router# show mpls ldp discovery Local LDP Identifier: 118.1.1.1:0 Discovery Sources: Interfaces: Targeted Hello being sent POS2/0 (ldp): xmit/recv LDP Id: 155.0.0.55:0 Tunnel1 (ldp): Targeted -> 133.0.0.33 Targeted Hellos: 118.1.1.1 -> 133.0.0.33 (ldp): active, xmit/recv LDP Id: 133.0.0.33:0 Targeted LDP session is active across the tunnel interface

MPLS Training - Basic

76

Targeted Configuration
ip cef mpls ip mpls label protocol ldp mpls ldp router-id loopback0 interface tunnel0 tunnel destination 10.20.10.1 mpls ip Enables LDP with target of 10.20.10.1 mpls ldp discovery targeted-hellos accept

If this command is entered then it means that the router will accept and LDP hellos from other end and establish session

MPLS Training - Basic

77

Label Stacking across tunnel interface


R7!
R1! R2! R3 ! R4!

R6!

R5!

R8!

R9!

TE ! Labels LDP ! Packet !

TE ! LDP ! Packet !

TE ! LDP ! Packet !

LDP ! Packet !

MPLS Training - Basic

78

LDP Session Establishment

2003 Cisco Systems, Inc. All rights reserved.

79

LDP Session Negotiation


A MPLS_A! B MPLS_B!

1.0.0.1!

1.0.0.2!

Peers first exchange initialization messages. The session is ready to exchange label mappings after receiving the first keepalive.

MPLS Training - Basic

80

LDP Session Negotiation


A MPLS_A! B Establish TCP session! Initialization message! MPLS_B!

1.0.0.1!

1.0.0.2!

Peers first exchange initialization messages. The session is ready to exchange label mappings after receiving the first keepalive.

MPLS Training - Basic

81

LDP Session Negotiation


A MPLS_A! B Establish TCP session! Initialization message! Initialization message! Keepalive! MPLS_B!

1.0.0.1!

1.0.0.2!

Peers first exchange initialization messages. The session is ready to exchange label mappings after receiving the first keepalive.

MPLS Training - Basic

82

LDP Session Negotiation


A MPLS_A! B Establish TCP session! Initialization message! Initialization message! Keepalive! Keepalive! Address message .! MPLS_B!

1.0.0.1!

1.0.0.2!

Peers first exchange initialization messages. The session is ready to exchange label mappings after receiving the first keepalive.

MPLS Training - Basic

83

LDP Session Maintenance


LSRs maintain their session by:
Continued periodic transmission of discovery Hello packets to indicate willingness to label switch on link Periodic transmission of keepalive messages on session TCP connection to monitor integrity of TCP connection

In session establishment, if there is a Init fatal notification, there is an backoff starting at less than 15 seconds and exponentially increasing to 2 minutes. Only the active LSR does this. Hello configuration TLV could be used to speed up session establishment.
84

MPLS Training - Basic

LDP Neighbours IOS command


Unsolicited downstream label allocation router#show mpls ldp neighbor Peer LDP Ident: 10.13.1.52:0; Local LDP Ident 10.13.1.59:0 TCP connection: 10.13.1.52.646 - 10.13.1.59.12331 State: Oper; Msgs sent/rcvd: 143/144; Downstream Up time: 00:00:55 LDP discovery sources: FastEthernet9/0/0, Src IP addr: 10.13.5.22 Addresses bound to peer LDP Ident: 10.13.1.52 10.13.5.18 200.37.52.5 200.6.52.13 10.13.0.52 10.13.5.22

These are the interface IP addresses of the LDP peer 10.13.1.52


MPLS Training - Basic 85

LDP Session Detail IOS Command


router#show mpls ldp neighbor detail Peer LDP Ident: 10.13.1.52:0; Local LDP Ident 10.13.1.59:0 TCP connection: 10.13.1.52.646 - 10.13.1.59.12331 State: Oper; Msgs sent/rcvd: 150/153; Downstream; Last TIB rev sent 1138 Up time: 00:07:49; UID: 74; Peer Id 0; Hello holdtime, Hello Interval LDP discovery sources: FastEthernet9/0/0; Src IP addr: 10.13.5.22 holdtime: 15000 ms, hello interval: 5000 ms Addresses bound to peer LDP Ident: 10.13.1.52 10.13.5.18 200.37.52.5 200.6.52.13 10.13.0.52 10.13.5.22 Peer holdtime: 180000 ms; KA interval: 60000 ms; Peer state: estab

LDP TCP session holdtime, keepalive interval

MPLS Training - Basic

86

Label Distribution, Control and Retention

2003 Cisco Systems, Inc. All rights reserved.

87

Label Distribution Methods


Router Control Retention Advertisement Control Retention Advertisement Independent Liberal Unsolicited Downstream IP+ATM Ordered Conservative On-demand

Whether labels are distributed regardless if there an outgoing label is available for the prefix Whether received labels are kept on local router Whether labels are distributed if requested

The modes shown here are generally how Router and ATM switches are configured for MPLS
MPLS Training - Basic 88

Label Distribution: Unsolicited Downstream

B X

Label for a prex is allocated and advertised to all neighbor LSRs, regardless of whether the neighbors are upstream or downstream LSRs for the destination.!
MPLS Training - Basic 89

Label Distribution: Unsolicited Downstream


LIB on Router B"
Network X LSR Local Label 25

X = 25!
A B

X = 25!
C E X D

Label for a prex is allocated and advertised to all neighbor LSRs, regardless of whether the neighbours are upstream or downstream LSRs for the destination.!
MPLS Training - Basic 90

Label Distribution: Downstream on Demand


Routing Table B"
Network X Next-Hop C

Routing Table C"


Network X Next-Hop D

Routing Table D"


Network X Next-Hop E

Routing Table E"


Network X Next-Hop Conn

RQ X!
B C D E X

A LSR can always assign a label for a prex, even if it has no downstream label. ! Independent control can only be used for LSRs with layer-3 capabilities.!
MPLS Training - Basic 91

LSP Control: Independent Control


Routing Table B"
Network X Next-Hop C

Routing Table C"


Network X Next-Hop D

Routing Table D"


Network X Next-Hop E

Routing Table E"


Network X Next-Hop Conn

RQ X!
B C D E X

LFIB on Router C
Destination X IN Label X = 25! OUT Label Next Hop 37 Router E

A LSR can always assign a label for a prex, even if it has no downstream label. ! Independent control can only be used for LSRs with layer-3 capabilities.!
MPLS Training - Basic 92

LSP Control: Independent Control


Routing Table B"
Network X Next-Hop C

Routing Table C"


Network X Next-Hop D

Routing Table D"


Network X Next-Hop E

Routing Table E"


Network X Next-Hop Conn

RQ X!
B C D E X

X=37!
LFIB on Router C
Destination X IN Label X = 25! OUT Label Next Hop 37 Router E

A LSR can always assign a label for a prex, even if it has no downstream label. ! Independent control can only be used for LSRs with layer-3 capabilities.!
MPLS Training - Basic 93

LSP Control: Ordered Control


Network X Next-Hop C Network X Next-Hop D Network X Next-Hop E Network X Next-Hop Conn

RQ X!
B C

RQ X!
D

RQ X!
E

X=37!
Destination X

X=17!
LFIB on Router C

X=82!

IN Label X = 25! OUT Label Next Hop 37 17 Router E

A LSR can only assign a label if it has already received a label from the next-hop LSR; otherwise it must request a label from the next-hop LSR. Used in IP+ATM switches!
MPLS Training - Basic 94

Label Retention: Liberal Retention Mode


LIB on Router A"
Network X LSR B Label 25 -

LIB on Router C"


Network X LSR B Label 25 -

X = 25!
A B

X = 25!
C E X D

LIB on Router D"


Network X LSR B Label 25 -

Every LSR stores the received label in its LIB, even when the label is not received from a next-hop LSR.! Liberal retention mode improves convergence speed.!
MPLS Training - Basic 95

Label Retention: Conservative Retention Mode


LIB on Router A"
Network X LSR B Label 25 -

LIB on Router C"


Network X LSR Label -

X = 25!
A B

X = 25!
C E X D

LIB on Router D"


Network X LSR Label -

LSR stores only the labels received from next-hop LSRs; all other labels are ignored.! Downstream-on-demand distribution is required during the convergence phase.!
MPLS Training - Basic 96

Some IOS commands

2003 Cisco Systems, Inc. All rights reserved.

97

IOS Show commands


router#sh mpls ldp neig | inc TCP TCP connection: 10.7.0.1.646 - 10.7.0.3.11011 TCP connection: 10.7.0.5.11026 - 10.7.0.3.646 TCP connection: 10.7.0.6.11024 - 10.7.0.3.646 TCP connection: 10.7.0.9.11034 - 10.7.0.3.646 router#show mpls ldp bind 10.5.0.8 255.255.255.252 tib entry: 10.5.0.8/30, rev 46 local binding: tag: 33 LIB structure remote binding: tsr: 10.7.0.5:0, tag: 17 remote binding: tsr: 10.7.0.1:0, tag: 29 remote binding: tsr: 10.7.0.6:0, tag: 19 This one chosen remote binding: tsr: 10.7.0.9:0, tag: 20 router#show tag for 10.5.0.8 Local Outgoing Prefix Bytes tag Outgoing Next Hop tag tag or VC or Tunnel Id switched interface 33 20 10.5.0.8/30 0 Et3/0 10.5.0.17 LFIB structure
MPLS Training - Basic 98

IOS Show commands


router#show ip route 10.5.0.8 Routing entry for 10.5.0.8/30 Known via "ospf 1", distance 110, metric 30, type intra area Last update from 10.5.0.17 on Ethernet3/0, 1w0d ago Routing Descriptor Blocks: * 10.5.0.17, from 10.7.0.2, 1w0d ago, via Ethernet3/0 Route metric is 30, traffic share count is 1 router#show mpls ldp neig 10.7.0.9 Peer LDP Ident: 10.7.0.9:0; Local LDP Ident 10.7.0.3:0 TCP connection: 10.7.0.9.11034 - 10.7.0.3.646 State: Oper; Msgs sent/rcvd: 12932/12965; Downstream Up time: 1w0d LDP discovery sources: Ethernet3/0, Src IP addr: 10.5.0.17 Addresses bound to peer LDP Ident: 10.5.0.17 10.7.0.9 10.5.0.38 10.5.0.46 10.6.3.1 10.5.0.57 10.6.3.5 10.5.0.21
99

MPLS Training - Basic

VPN Concepts

2003 Cisco Systems, Inc. All rights reserved.

100

What is an MPLS-VPN?
An IP network infrastructure delivering private network services over a public infrastructure
Use a layer 3 backbone Scalability, easy provisioning Global as well as non-unique private address space QoS Controlled access Easy configuration for customers

MPLS Training - Basic

101

VPN Models
There are two basic types of design models that deliver VPN functionality Overlay Model Peer Model

MPLS Training - Basic

102

The Overlay model


Private trunks over a TELCO/SP shared infrastructure
Leased/Dialup lines FR/ATM circuits IP (GRE) tunnelling

Transparency between provider and customer networks Optimal routing requires full mesh over over backbone

MPLS Training - Basic

103

The Peer model


Both provider and customer network use same network protocol and control plane CE and PE routers have routing adjacency at each site All provider routers hold the full routing information about all customer networks Private addresses are not allowed May use the virtual router capability
Multiple routing and forwarding tables based on Customer Networks

MPLS Training - Basic

104

MPLS-VPN = True Peer model


MPLS-VPN is similar in operation to peer model Provider Edge routers receive and hold routing information only about VPNs directly connected Reduces the amount of routing information a PE router will store Routing information is proportional to the number of VPNs a router is attached to MPLS is used within the backbone to switch packets (no need of full routing)

MPLS Training - Basic

105

MPLS VPN Connection Model

2003 Cisco Systems, Inc. All rights reserved.

106

MPLS VPN Connection Model


A VPN is a collection of sites sharing a common routing information (routing table) A site can be part of different VPNs A VPN has to be seen as a community of interest (or Closed User Group) Multiple Routing/Forwarding instances (VRF) on PE

MPLS Training - Basic

107

MPLS VPN Connection Model


Site-4! Site-1!

VPN-C!

VPN-A!
Site-2! Site-3!

VPN-B!

A site belonging to different VPNs may or MAY NOT be used as a transit point between VPNs If two or more VPNs have a common site, address space must be unique among these VPNs
MPLS Training - Basic 108

MPLS VPN Connection Model


The VPN backbone is composed by MPLS LSRs
PE routers (edge LSRs) P routers (core LSRs)

The customer router connecting to the VPN backbone is called the Customer Edge (CE) PE routers are faced to CE routers and distribute VPN information through MP-BGP to other PE routers
VPN-IPv4 addresses, Extended Community, Label

P routers do not run MP-BGP and do not have any VPN knowledge
MPLS Training - Basic 109

MPLS VPN Components


CE PE LSR P LSR PE CE

ELSR

ELSR

ELSR

ELSR

LSR

LSR

C Network (Customer Control)

P Network (Provider Control)

C Network (Customer Control)

MPLS Training - Basic

110

PECE Routing

2003 Cisco Systems, Inc. All rights reserved.

111

PE-CE Routing
CE1 PE CE2
PE-CE routing!

PE and CE routers exchange routing information through eBGP, Static, OSPF, ISIS, RIP, EIGRP The CE router runs standard routing software, not aware it is connected to a VPN network
MPLS Training - Basic 112

PE-CE routing protocols


Static/BGP are the most scalable
Single PE router can support 100s or 1000s of CE routers

BGP is the most flexible


Particularly for multi-homing but not popular with Enterprise Very useful if Enterprise requires Internet routes

Use the others to meet customer requirements


OSPF popular with Enterprises but sucks up processes EIGRP not popular with Service Providers (Cisco proprietary) IS-IS less prevalent in Enterprise environments RIPv2 provides very simple functionality
MPLS Training - Basic 113

Routing Protocol Contexts

Routing processes

BGP

RIP

Static
Routing processes run within specific routing contexts

Routing contexts

BGP 1

BGP 2

BGP 3

RIP 1

RIP 2

Populate specific VPN routing table and FIBs (VRF) Interfaces are assigned to VRFs"

VRF Routing tables VRF Forwarding tables


VRF Site A VRF Site B VRF Site C

MPLS Training - Basic

114

OSPF and Single Routing Instances

Routing processes

OSPF

OSPF

OSPF
With OSPF there is a single process per VRF Same for IS-IS No routing contexts Prior to 12.0(27)S and 12.3(4)T maximum of 28 processes allowed

Routing contexts

VRF Routing tables VRF Forwarding tables

VRF Site A

VRF Site B

VRF Site C

MPLS Training - Basic

115

Routing Tables

2003 Cisco Systems, Inc. All rights reserved.

116

Routing Tables
CE1
VRF!

PE CE2
PE-CE routing! VPN Backbone IGP (OSPF, ISIS)!

Global Routing Table!

PE routers maintain separate routing tables Global Routing Table


All the PE and P routes populated by the VPN backbone IGP (ISIS or OSPF)

VPN Routing and Forwarding Tables (VRF)


Routing and Forwarding table associated with one or more directly connected sites (CEs) VRF are associated to (sub/virtual/tunnel) interfaces Interfaces may share the same VRF if the connected sites may share the same routing information
MPLS Training - Basic 117

IGP and label distribution in the backbone


CE1 CE2 LFIB for PE-1
Dest PE2 P2 P1 Next Hop P1 P1 S0/0 IN 17 18 19 OUT 50 65 POP

PE1

P1

P2

PE2

CE3 CE4

LFIB for P1
Dest PE2 P2 PE1 Next Hop P2 E0/2 S3/0 IN 50 65 67 OUT 34 POP POP

LFIB for P2
Dest PE2 P1 PE1 Next Hop P1 E0/1 P1 IN 34 38 39 OUT POP POP 67

LFIB for PE2


Dest P1 P2 PE1 Next Hop P2 P2 P2 IN 44 36 18 OUT 38 65 39

All routers (P and PE) run an IGP and label distribution protocol Each P and PE router has routes for the backbone nodes and a label is associated to each route MPLS forwarding is used within the core
MPLS Training - Basic 118

VPN Routing and Forwarding Table


CE1 CE2
MP-iBGP session!

PE1

P1

P2

PE2

CE3 CE4

Multiple routing tables (VRFs) are used on PEs Each VRF contain customer routes Customer addresses can overlap VPNs are isolated Multi-Protocol BGP (MP-BGP) is used to propagate these addresses + labels between PE routers only
MPLS Training - Basic 119

MPLS VPN Requirements


CE1 CE2
MP-iBGP session!

PE1

P1

P2

PE2

CE3 CE4

VPN services allow


Customers to use the overlapping address space Isolate customer VPNs Intranets Join VPNs - Extranets

MPLS-VPN backbone MUST


Distinguish between customer addresses Forward packets to the correct destination
MPLS Training - Basic 120

VPN Address Overlap


CE1 CE2
MP-iBGP session!

PE1

P1

P2

PE2

CE3 CE4

BGP propagates ONE route per destination


Standard path selection rules are used

What if two customers use the same address? BGP will propagate only one route - PROBLEM !!! Therefore MP-BGP must DISTINGUISH between customer addresses
MPLS Training - Basic 121

VPN Address Overlap


CE1 CE2
MP-iBGP session!

PE1

P1

P2

PE2

CE3 CE4

When PE router receives VPN routes from MP-BGP how do we know what VRF to place route in? How do we distinguish overlapping addresses between two VPNs

MPLS Training - Basic

122

Route-Target and Route-Distinguisher


update X ! update X !

x x

CE1 CE2

PE1

P1

P2

PE2

CE3 CE4

MP-iBGP session! update X ! update X ! VPN-IPv4 updates are translated into IPv4 address and inserted into the VRF corresponding to the RT value

VPN-IPv4 update: RD1:X, Next-hop=PE1 RT=RED, Label=10!

VPN-IPv4 update: RD2:X, Next-hop=PE1 RT=ORANGE, Label=12!

MP-BGP prepends an Route Distinguisher (RD) to each VPN route in order to make it unique MP-BGP assign a Route-Target (RT) to each VPN route to identify VPN it belongs to (or CUG)
Route-Target is the colour of the route
MPLS Training - Basic 123

Route Propagation through MP-BGP


update X ! update X !

x x

CE1 CE2

PE1

P1

P2

PE2

CE3 CE4

MP-iBGP session! update X ! update X ! VPN-IPv4 updates are translated into IPv4 address and inserted into the VRF corresponding to the RT value

VPN-IPv4 update: RD1:X, Next-hop=PE1 RT=RED, Label=10!

VPN-IPv4 update: RD2:X, Next-hop=PE1 RT=ORANGE, Label=12!

When a PE router receives an MP-BGP VPN route:


It checks the route-target value to VRF route-targets If match then route is inserted into appropriate VRF The label associated with the VPN route is stored and used to send packets towards the destination
MPLS Training - Basic 124

Multi-Protocol BGP
Propagates VPN routing information
Customer routes held in VPN Routing and Forwarding tables (VRFs)

Only runs on Provider Edge


P routers are not aware of VPNs only labels

PEs are fully meshed


Using Route Reflectors or direct peerings between PE routers

MPLS Training - Basic

125

Forwarding Example

2003 Cisco Systems, Inc. All rights reserved.

126

MPLS VPN Protocols


OSPF/IS-IS
Used as IGP provides reachability between all Label Switch Routers (PE <-> P <-> PE)

TDP/LDP
Distributes label information for IP destinations in core

MP-BGP4
Used to distribute VPN routing information between PEs

RIPv2/BGP/OSPF/eiGRP/ISIS/Static
Can be used to route between PE and CE

MPLS Training - Basic

127

VPN Components
VRF Tables
Hold customer routes at PE

Route-Distinguisher
Allows MP-BGP to distinguish between identical customer routes that are in different VPNs

Route-Targets
Used to import and export routes between different VRF tables (creates Intranets and Extranets)

Route-maps
Allows finer granularity and control of importing exporting routes between VRFs instead of just using route-target
MPLS Training - Basic 128

MPLS VPN Operation


CE = RT? PE RD + RD + VPN labels, RTs P
RR

= RT? PE

CE

PE CE RD +

RR

PE CE

RD + RD + VPN labels, RTs

Import routes into VRF if route-targets match (export = import) Customer routes placed into separate VRF tables at each PE IGP (OSPF,ISIS) used to establish reachability to destination networks. Label Distribution Protocol establishes mappings to IGP addresses CE-PE dynamic routing (or static) populate the VRF routing tables MP-BGP between PE router to distribute routes between VPNs
MPLS Training - Basic

129

MPLS VPN Label Stack


There are at least two labels when using MPLS-VPN The first label is distributed by TDP/LDP
Derived from an IGP route Corresponds to a PE address (VPN egress point) PE addresses are MP-BGP next-hops of VPN routes

The second label is distributed MP-BGP


Corresponds to the actual VPN route Identifies the PE outgoing interface or routing table

L2 Header

Label 1

Label 2

L3 Header

Data

Frame, e.g. HDLC, PPP, Ethernet


MPLS Training - Basic 130

MPLS VPN Forwarding Example

CE PE P CE PE
Swap IGP Label (From LFIB) Push VPN Label (Red Route) Push IGP Label (Green PE Router)
MPLS Training - Basic

CE PE P CE PE
POP IGP Label (Pentultimate Hop)

Pop VPN Label (Red Route)


131

VPN Topologies

2003 Cisco Systems, Inc. All rights reserved.

132

Basic Intranet Full Mesh


Finance Site 3 VLAN 205

F FF FF F
Finance Site 1 MPLS Core

F FF FF F

F FF FF F

Finance Site 2

VRF

Each site has of all other sites (same VPN)


CE can be router or switch

MP-BGP VPNv4 updates propagated between PEs Routing is optimal in the backbone
No site is used as central point for connectivity
MPLS Training - Basic 133

Basic Extranet Partial Mesh


Engineering Site B (EB) DA DA EB DA E EB E EA E EB EA E E MPLS Core

Engineering Site A (EA)

E Design Site A (DA)

E E E

D E

D D D

Design Site B (DB) D

VRF EB EB D EB D D D D

Basic Extranet Routes can be imported directly into corresponding VRF NAT may be necessary if Enterprise have overlapping addressing Import granularity can be very fine
Single host address can be imported as Extranet route
MPLS Training - Basic 134

Branch to HQ Hub and Spoke


Bank Branch 3
S3

VRF
S3 S1h X S2h

BGP/OSPF/ RIProuting Spoke OUT S2h S1h S3h

Optional Firewall NAT to X

MPLS Core Bank Branch 2


S2 S2 S1h X S3h

S1 S2 S3 X Hub IN BGP/OSPF/RIP routing

VRF Bank Branch 1

S1 S2h X S3h

Central HQ

VRF
S1

Forces all branches through the Central HQ Spokes cannot communicate directly Appropriate security screening can be applied Firewalls can be used with NAT to ensure correct return path
MPLS Training - Basic 135

Per Group Internet Access


Legal VRF
L L L L D3 L Gateway 3 S S S S D1

D3

Internet

Legal Only

Sales MPLS Core


L

D2

S M Gateway 2

Internet

Legal/Sales & Marketing Backup

Marketing
M M M D1 S MI 1

Internet
Gateway 1

Sales and Marketing

Choose appropriate Internet Gateway per group requirements Use other gateways as backup in case of failure Gateways can provide different service attributes/levels
Speed of access Type of Content accessed Address translation if required
MPLS Training - Basic 136

VPN with Internet


This example uses default route only to access Internet If customer addresses are RFC1983 then NAT must be done
Can be done at Internet Gateway or at customer edge

Another model could use default route pointing to gateway in the global table
This assumes that customer uses registered address space

MPLS Training - Basic

137

Enterprise Disaster Recovery


Backup Data Centre (LOCALPREF=50) C CC C CC Primary Data Centre (LOCALPREF=100)

S1 C S2 C C S3 VRF S1 C CC S2 C CC

S1 C S2 C C S3 Site 3

Site 1

MPLS Core

S3 C CC

Site 2

Disaster recovery can be provided to each site in the Enterprise If Primary site fails, Backup site takes over with no intervention Virtualisation/Mirroring takes place between Primary/Secondary
MPLS Training - Basic 138

MPLS VPN Mechanisms

2003 Cisco Systems, Inc. All rights reserved.

139

Virtual Routing and Forwarding Table


A VRF is the routing and forwarding instance for a set of sites with identical connectivity requirements. Data structures associated with a VRF:
IP routing table Cisco Express Forwarding (CEF) forwarding table Set of rules and routing protocol parameters (routing protocol contexts) List of interfaces that use the VRF

Other information associated with a VRF:


Route Distinguisher (RD) Set of import and export route targets
MPLS Training - Basic 140

Need for Routing Protocol Contexts


VPN A!
10.1.1.0/24!

There are two backbones with overlapping addresses.!


MPLS VPN Backbone!

CE-VPN-A

VPN B!

PE Router!

CE-VPN-B

10.1.1.0/24!

Routing Information Protocol (RIP) is running in both VPNs.! RIP in VPN A has to be different from RIP in VPN B, but Cisco IOS software supports only one RIP process per router.!
141

MPLS Training - Basic

VPN-Aware Routing Protocols


Routing context = routing protocol run in one VRF
Supported by VPN-aware routing protocols: External BGP (EBGP), OSPF, RIP version 2 (RIPv2), EIGRP, IS-IS, Dtatic routes Implemented as several instances of a single routing process (EBGP, RIPv2) or as several routing processes (OSPF) Independent per-instance router variables for each instance

MPLS Training - Basic

142

VRF Routing Table


Contains routes that should be available to a particular set of sites Analogous to standard Cisco IOS software routing table; supports same set of mechanisms VPN interfaces (physical interface, subinterfaces, logical interfaces) assigned to VRFs
Many interfaces per VRF Each interface assignable to only one VRF

MPLS Training - Basic

143

Routing Contexts, VRF, and MP-BGP Interaction: 1/9


RIP Routing Process!
CE-RIP-A

VRF-A Routing Table!

BGP Routing Process! Backbone! Multiprotocol ! BGP!

Instance for VRF-A! Instance for VRF-B! VRF-B Routing Table!

CE-RIP-B

Instance for VRF-A!


CE-BGP-A

!
Instance for VRF-B!

CE-BGP-B

Two VPNs attached to the same PE router! Each VPN represented by a VRF (VRF-A and VRF-B)! RIP and BGP running between PE and CE routers!
MPLS Training - Basic 144

Routing Contexts, VRF, and MP-BGP Interaction: 2/9


RIP Routing Process!
CE-RIP-A

VRF-A Routing Table!

BGP Routing Process! Backbone! Multiprotocol ! BGP!

Instance for VRF-A! Instance for VRF-B! VRF-B Routing Table!

CE-RIP-B

Instance for VRF-A!


CE-BGP-A

!
Instance for VRF-B!

CE-BGP-B

RIP-speaking CE routers announce their prexes to the PE router via RIP.! Instance of RIP process associated with the VRF into which the PE-CE interface belongs collects the routes and inserts them into VRF routing table.!
MPLS Training - Basic 145

Routing Contexts, VRF, and MP-BGP Interaction: 3/9


RIP Routing Process!
CE-RIP-A

VRF-A Routing Table!

BGP Routing Process! Backbone! Multiprotocol ! BGP!

Instance for VRF-A! Instance for VRF-B! VRF-B Routing Table!

CE-RIP-B

Instance for VRF-A!


CE-BGP-A

!
Instance for VRF-B!

CE-BGP-B

BGP-speaking CE routers announce their prexes to the PE router via BGP.! Instance of BGP process associated with the VRF into which the PE-CE interface belongs collects the routes and inserts them into VRF routing table.!
MPLS Training - Basic 146

Routing Contexts, VRF, and MP-BGP Interaction: 4/9


RIP Routing Process!
CE-RIP-A

VRF-A Routing Table!

BGP Routing Process! Backbone! Multiprotocol ! BGP!

Instance for VRF-A! Instance for VRF-B! VRF-B Routing Table!

CE-RIP-B

Instance for VRF-A!


CE-BGP-A

!
Instance for VRF-B!

CE-BGP-B

RIP routes entered in the VRF routing table are redistributed into BGP for further propagation into the MPLS VPN backbone.! Redistribution between RIP and BGP has to be congured for proper MPLS VPN operation.!
MPLS Training - Basic 147

Routing Contexts, VRF, and MP-BGP Interaction: 5/9


RIP Routing Process!
CE-RIP-A

VRF-A Routing Table!

BGP Routing Process! Multiprotocol ! BGP!

Instance for VRF-A!

CE-RIP-B

Instance for VRF-B!

VRF-B Routing Table!

Instance for VRF-A!


CE-BGP-A

!
Instance for VRF-B!

CE-BGP-B

Route distinguisher is prepended during route export to the BGP routes from VRF instance of BGP process to convert them into VPNv4 prexes. Route targets are attached to these prexes.! VPNv4 prexes are propagated to other PE routers.!
MPLS Training - Basic 148

Routing Contexts, VRF, and MP-BGP Interaction: 6/9


RIP Routing Process!
CE-RIP-A

VRF-A Routing Table!

BGP Routing Process! Multiprotocol ! BGP!

Instance for VRF-A!

CE-RIP-B

Instance for VRF-B!

VRF-B Routing Table!

Instance for VRF-A!


CE-BGP-A

!
Instance for VRF-B!

CE-BGP-B

VPNv4 prexes are received from other PE routers.! The VPNv4 prexes are inserted into proper VRF routing tables based on their route targets and import route targets congured in VRFs.! Route distinguisher is removed during this process.!
MPLS Training - Basic 149

Routing Contexts, VRF, and MP-BGP Interaction: 7/9


RIP Routing Process!
CE-RIP-A

VRF-A Routing Table!

BGP Routing Process! Backbone! Multiprotocol ! BGP!

Instance for VRF-A!

CE-RIP-B

Instance for VRF-B!

VRF-B Routing Table!

Instance for VRF-A!


CE-BGP-A

!
Instance for VRF-B!

CE-BGP-B

Routes received from backbone MP-BGP and imported into a VRF are forwarded as IPv4 routes to EBGP CE neighbors attached to that VRF.!
MPLS Training - Basic 150

Routing Contexts, VRF, and MP-BGP Interaction: 8/9


RIP Routing Process!
CE-RIP-A

VRF-A Routing Table!

BGP Routing Process! Multiprotocol ! BGP!

Instance for VRF-A!

CE-RIP-B

Instance for VRF-B!

VRF-B Routing Table!

Instance for VRF-A!


CE-BGP-A

!
Instance for VRF-B!

CE-BGP-B

MP-IBGP routes imported into a VRF are redistributed into the instance of RIP congured for that VRF.! Redistribution between BGP and RIP has to be congured for end- to-end RIP routing between CE routers.!
MPLS Training - Basic 151

Routing Contexts, VRF, and MP-BGP Interaction: 9/9


RIP Routing Process!
CE-RIP-A

VRF-A Routing Table!

BGP Routing Process! Backbone! Multiprotocol ! BGP!

Instance for VRF-A!

CE-RIP-B

Instance for VRF-B!

VRF-B Routing Table!

Instance for VRF-A!


CE-BGP-A

!
Instance for VRF-B!

CE-BGP-B

Routes redistributed from BGP into a VRF instance of RIP are sent to RIP-speaking CE routers.!

MPLS Training - Basic

152

Configuring VRF tables

2003 Cisco Systems, Inc. All rights reserved.

153

Configuring VRF Tables


VRF configuration tasks:
Create a VRF table Assign RD to the VRF Specify export and import route targets Assign interfaces to VRFs

MPLS Training - Basic

154

Creating VRF Tables and Assigning RDs


router(cong)#"

ip vrf name

!!

Creates a new VRF or enters conguration of an existing VRF.! VRF names are case-sensitive.! VRF is not operational unless you congure RD.! VRF names have only local signicance.!
router(cong-vrf)#"

rd route-distinguisher!

Assigns a route distinguisher to a VRF.! You can use ASN:xx or A.B.C.D:xx format for RD.! Each VRF in a PE router has to have a unique RD.!
MPLS Training - Basic 155

Specify Export and Import RTs


router(cong-vrf)#"

route-target export RT

!!

Species an RT to be attached to every route exported from this VRF to MP-BGP! Allows specication to many export RTsall to be attached to every exported route!
router(cong-vrf)#"

route-target import RT!

Species an RT to be used as an import lteronly routes matching the RT are imported into the VRF! Allows specication of many import RTsany route where at least one RT attached to the route matches any import RT is imported into the VRF!

MPLS Training - Basic

156

Specify Export and Import RTs


router(cong-vrf)#"

route-target both RT!

In cases where the export RT matches the import RT, use this form of route-target command.!

Sample router conguration for simple customer VPN:!


ip vrf Customer_ABC rd 12703:15 route-target export 12703:15 route-target import 12703:15
MPLS Training - Basic 157

Assigning an Interface to VRF Table


router(cong-if)#"

ip vrf forwarding vrf-name

!!

Associates an interface with the specied VRF! Existing IP address removed from the interface when interface is put into VRFIP address must be recongured! CEF switching must be enabled on interface! Sample router conguration:!
ip cef ! interface serial 0/0 ip vrf forwarding Customer_ABC ip address 10.0.0.1 255.255.255.252
MPLS Training - Basic 158

Sample VPN Network


MPLS VPN Backbone!
CE-RIP-A1! CE-RIP-A2!

CE-BGP-A1!

CE-BGP-A2!

PE-Site-X!
CE-RIP-B1!

PE-Site-Y!
CE-RIP-B2!

The network supports two VPN customers. Customer A runs RIP and BGP with the service provider; customer B uses only RIP. Both customers use network 10.0.0.0.

MPLS Training - Basic

159

Sample VPN Network VRF Configuration


MPLS VPN Backbone! ip vrf Customer_A
CE-RIP-A1!

rd 115:43 route-target both 115:43

CE-RIP-A2!

CE-BGP-A1!

PE-Site-X!
CE-RIP-B1!

! ip vrf Customer_B CE-BGP-A2! rd 115:47 route-target both 115:47 PE-Site-Y! ! interface serial 1/0/1 CE-RIP-B2! ip forwarding vrf Customer_A ip address 10.1.0.1 255.255.255.252 ! interface serial 1/0/2 ip vrf forwarding Customer_A ip address 10.1.0.5 255.255.255.252 ! interface serial 1/1/3 ip vrf forwarding Customer_B ip address 10.2.0.1 255.255.255.252
160

MPLS Training - Basic

Configuring MP-BGP

2003 Cisco Systems, Inc. All rights reserved.

161

BGP Address Families


The BGP process in an MPLS VPN-enabled router performs three separate tasks:
Global BGP routes (Internet routing) are exchanged as in traditional BGP setup VPNv4 prefixes are exchanged through MP-BGP VPN routes are exchanged with CE routers through perVRF EBGP sessions.

Address families (routing contexts) are used to configure these three tasks in the same BGP process.

MPLS Training - Basic

162

Selecting the BGP Address Family


router(cong)#"

router bgp as-number

!!

Selects global BGP routing process!


router(cong-router)#"

address-family vpnv4

!!

Selects conguration of VPNv4 prex exchanges under MP-BGP sessions!


router(cong-router)#"

address-family ipv4 vrf vrf-name

!!

Selects conguration of per-VRF PE-CE EBGP parameters!


MPLS Training - Basic 163

BGP Neighbors
MP-BGP neighbors are configured under the BGP routing process.
These neighbors need to be activated for each global address family they support. Per-address-family parameters can be configured for these neighbors.

VRF-specific EBGP neighbors are configured under corresponding address families.

MPLS Training - Basic

164

Configuring MP-BGP
MPLS VPN MP-BGP configuration steps:
Configure MP-BGP neighbor under BGP routing process Configure BGP address family VPNv4 Activate configured BGP neighbor for VPNv4 route exchange Specify additional parameters for VPNv4 route exchange (filters, next hops, and so forth)

MPLS Training - Basic

165

Configuring MP-IBGP
router(cong)#"

router bgp AS-number! neighbor IP-address remote-as AS-number! neighbor IP-address update-source loopback-interface

!!

All MP-BGP neighbors have to be congured under global BGP routing conguration.! MP-IBGP sessions have to run between loopback interfaces.!
router(cong-router)#"

address-family vpnv4!

Starts conguration of MP-BGP routing for VPNv4 route exchange.! Parameters that apply only to MP-BGP exchange of VPNv4 routes between already congured IBGP neighbors are congured under this address family.!
MPLS Training - Basic 166

Configuring MP-IBGP
router(cong-router-af)#"

neighbor IP-address activate

!!

The BGP neighbor dened under BGP router conguration has to be activated for VPNv4 route exchange.!
router(cong-router-af)#"

neighbor IP-address next-hop-self!

The next-hop-self command must be congured on the MP-IBGP session for proper MPLS VPN conguration if EBGP is being run with a CE neighbor.!
MPLS Training - Basic 167

BGP Community Propagation


router(cong-router-af)#"

neighbor IP-address send-community [extended | both]

!!

This command congures propagation of standard and extended BGP communities attached to VPNv4 prexes.! Default value: only extended communities are sent.! Extended BGP communities attached to VPNv4 prexes must be exchanged between MP-BGP neighbors for proper MPLS VPN operation.! To propagate standard BGP communities between MP-BGP neighbors, use the both option.!

MPLS Training - Basic

168

Sample MP-IBGP Configuration


MPLS VPN Backbone!
CE-RIP-A1! CE-RIP-A2!

CE-BGP-A1!

CE-BGP-A2!

PE-Site-X!
CE-RIP-B1!

PE-Site-Y!

interface loopback 0 ip address 172.16.1.1 255.255.255.255 CE-RIP-B2! ! router bgp 115 neighbor 172.16.1.2 remote-as 115 neighbor 172.16.1.2 update-source loopback 0 ! address-family vpnv4 neighbor 172.16.1.2 activate neighbor 172.16.1.2 next-hop-self neighbor 172.16.1.2 send-community both
169

MPLS Training - Basic

Disabling IPv4 Route Exchange


router(cong-router)#"

no bgp default ipv4 unicast

!!

Exchange of IPv4 routes between BGP neighbors is enabled by defaultevery congured neighbor will also receive IPv4 routes! This command disables default exchange of IPv4 routesneighbors that need to receive IPv4 routes have to be activated for IPv4 route exchange! Use this command when the same router carries Internet and VPNv4 routes and you dont want to propagate Internet routes to some PE neighbors.!

MPLS Training - Basic

170

Sample Router Configuration


Neighbor 172.16.32.14 receives only Internet routes. Neighbor 172.16.32.15 receives only VPNv4 routes. Neighbor 172.16.32.27 receives Internet and VPNv4 routes.

router bgp 12703 no bgp default ipv4 unicast neighbor 172.16.32.14 remote-as 12703 neighbor 172.16.32.15 remote-as 12703 neighbor 172.16.32.27 remote-as 12703 ! Activate IPv4 route exchange neighbor 172.16.32.14 activate neighbor 172.16.32.27 activate ! Step#2 VPNv4 route exchange address-family vpnv4 neighbor 172.16.32.15 activate neighbor 172.16.32.27 activate
MPLS Training - Basic 171

Configuring PE-CE Routing

2003 Cisco Systems, Inc. All rights reserved.

172

Configuring PE-CE Routing Protocols


PE-CE routing protocols are configured for individual VRFs. Per-VRF routing protocols can be configured in two ways:
There is only one BGP or RIP process per router, per-VRF parameters are specified in routing contexts, which are selected with the address family command. A separate OSPF process has to be started for each VRF.

Overall number of routing processes per router is limited to 32


Will be lifted in 12.0(27)S

MPLS Training - Basic

173

VRF Routing Context for BGP and RIP


router(cong)#"

router bgp AS-number! address-family ipv4 vrf vrf-name! ... Per-VRF BGP denitions ...

!!

Per-VRF BGP context is selected with the address-family command.! CE EBGP neighbors are congured in VRF context, not in the global BGP conguration.!
router(cong)#"

router rip! address-family ipv4 vrf vrf-name! ... Per-VRF RIP denitions ...! Similar to BGP, select per-VRF RIP context with the address-family command.! Congure all per-VRF RIP parameters therestarting with network numbers.!
MPLS Training - Basic 174

Configuring per-VRF BGP Routing


CE neighbors have to be specified within the perVRF context, not in global BGP. CE neighbors have to be activated with the neighbor activate command. All non-BGP per-VRF routes have to be redistributed into per-VRF BGP context to be propagated by MP-BGP to other PE routers. Per-VRF BGP context has auto-summarization and synchronization disabled by default.

MPLS Training - Basic

175

Sample PE-CE BGP Configuration


MPLS
CE-RIP-A1!

router bgp 65001 VPN Backbone! neighbor 10.200.1.2 remote-as 115 CE-RIP-A2 network 10.1.0.0 mask 255.255.0.0 !

CE-BGP-A1!

CE-BGP-A2!

PE-Site-X!
CE-RIP-B1!

PE-Site-Y!
CE-RIP-B2!

router bgp 115 ! address-family ipv4 vrf Customer_A neighbor 10.200.1.1 remote-as 65001 neighbor 10.200.1.1 activate
MPLS Training - Basic 176

Configuring RIP PE-CE Routing


A routing context is configured for each VRF running RIP RIP parameters have to be specified in the VRF Some parameters configured in the RIP process are propagated to routing contexts (for example, RIP version) Only RIPv2 is supported

MPLS Training - Basic

177

RIP Metric Propagation


router(cong)#"

router rip! address-family ipv4 vrf vrf-name! redistribute bgp metric transparent

!!

BGP routes have to be redistributed back into RIP if you want to have end-to-end RIP routing in the customer network.! The RIP hop count is copied into BGP multi-exit discriminator attribute (default BGP behavior).! The RIP hop count has to be manually set for routes redistributed into RIP.! With metric transparent option, BGP MED is copied into the RIP hop count, resulting in a consistent end-to-end RIP hop count.!
MPLS Training - Basic 178

Sample RIP Configuration


MPLS VPN Backbone!
CE-RIP-A1! CE-RIP-A2!

CE-BGP-A1!

CE-BGP-A2!

PE-Site-X!
CE-RIP-B1!

PE-Site-Y!

router rip CE-RIP-B2! version 2 address-family ipv4 vrf Customer_ABC network 10.0.0.0 redistribute bgp 12703 metric transparent ! router bgp 12703 address-family ipv4 vrf Customer_ABC redistribute rip
179

MPLS Training - Basic

Configuring OSPF PE-CE Routing


A separate OSPF routing process is configured for each VRF running OSPF. OSPF route attributes are attached as extended BGP communities to OSPF routes redistributed into MP-BGP. Routes redistributed from MP-BGP into OSPF get proper OSPF attributes.
No additional configuration is needed.

MPLS Training - Basic

180

Configuring PE-CE OSPF Routing


router(cong)#"

router ospf process-id vrf name! ... Standard OSPF parameters ...!

This command congures the per-VRF OSPF routing process.! Sample router conguration:!
router ospf 123 vrf Customer_ABC network 0.0.0.0 255.255.255.255 area 0 redistribute bgp 12703 ! router bgp 12703 address-family ipv4 vrf Customer_ABC redistribute ospf 123
MPLS Training - Basic 181

Configuring Per-VRF Static Routes


router(cong)#"

ip route vrf name static route parameters

!!

This command congures per-VRF static routes. ! The route is entered in the VRF table.! On Ethernet Interfaces, you must specify the the next hop as well as the outgoing interface! Sample router conguration:!
ip route vrf Customer_ABC 10.0.0.0 255.0.0.0 10.250.0.2 ethernet 0/0 ! router bgp 12703 address-family ipv4 vrf Customer_ABC redistribute static
MPLS Training - Basic 182

Monitoring MPLS VPN Operation

2003 Cisco Systems, Inc. All rights reserved.

183

Monitoring VRF
router#"

show ip vrf

!!

Displays the list of all VRFs congured in the router!


router#"

show ip vrf detail

!!

Displays detailed VRF conguration!


router#"

show ip vrf interfaces

!!

Displays interfaces associated with VRFs!


MPLS Training - Basic 184

show ip vrf
Router#show ip vrf Name SiteA2 SiteB SiteX Router# Default RD 103:30 103:11 103:20 Interfaces Serial1/0.20 Serial1/0.100 Ethernet0/0

MPLS Training - Basic

185

show ip vrf detail


Router#show ip vrf detail VRF SiteA2; default RD 103:30 Interfaces: Serial1/0.20 Connected addresses are not in global routing table No Export VPN route-target communities Import VPN route-target communities RT:103:10 No import route-map Export route-map: A2 VRF SiteB; default RD 103:11 Interfaces: Serial1/0.100 Connected addresses are not in global routing table Export VPN route-target communities RT:103:11 Import VPN route-target communities RT:103:11 RT:103:20 No import route-map No export route-map
MPLS Training - Basic 186

show ip vrf interfaces


Router#show ip vrf interfaces Interface IP-Address Serial1/0.20 150.1.31.37 Serial1/0.100 150.1.32.33 Ethernet0/0 192.168.22.3

VRF SiteA2 SiteB SiteX

Protocol up up up

MPLS Training - Basic

187

Monitoring VRF Routing


router#"

show ip protocols vrf name

!!

Displays the routing protocols congured in a VRF!


router#"

show ip route vrf name

!!

Displays the VRF routing table!


router#"

show ip bgp vpnv4 vrf name

!!

Displays per-VRF BGP parameters (PE-CE neighbors )!


MPLS Training - Basic 188

show ip protocol vrf


Router#show ip protocol vrf SiteX Routing Protocol is "rip" Sending updates every 30 seconds, next due in 10 seconds Invalid after 180 seconds, hold down 180, flushed after 240 Outgoing update filter list for all interfaces is Incoming update filter list for all interfaces is Redistributing: rip, bgp 3 Default version control: send version 2, receive version 2 Interface Send Recv Triggered RIP Key-chain Ethernet0/0 2 2 Routing for Networks: 192.168.22.0 Routing Information Sources: Gateway Distance Last Update Distance: (default is 120)

MPLS Training - Basic

189

show ip route vrf


Router#show ip route vrf SiteA2 Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, * - candidate default, U - per-user static route, o - ODR P - periodic downloaded static route Gateway of last resort is not set O O B B B B 203.1.20.0/24 [110/782] via 150.1.31.38, 02:52:13, Serial1/0.20 203.1.2.0/32 is subnetted, 1 subnets 203.1.2.1 [110/782] via 150.1.31.38, 02:52:13, Serial1/0.20 203.1.1.0/32 is subnetted, 1 subnets 203.1.1.1 [200/1] via 192.168.3.103, 01:14:32 203.1.135.0/24 [200/782] via 192.168.3.101, 02:05:38 203.1.134.0/24 [200/1] via 192.168.3.101, 02:05:38 203.1.10.0/24 [200/1] via 192.168.3.103, 01:14:32

rest deleted

MPLS Training - Basic

190

show ip bgp vpnv4 vrf <x> neighbor


Router#show ip bgp vpnv4 vrf SiteB neighbors BGP neighbor is 150.1.32.34, vrf SiteB, remote AS 65032, external link BGP version 4, remote router ID 203.2.10.1 BGP state = Established, up for 02:01:41 Last read 00:00:56, hold time is 180, keepalive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast: advertised and received Received 549 messages, 0 notifications, 0 in queue Sent 646 messages, 0 notifications, 0 in queue Route refresh request: received 0, sent 0 Minimum time between advertisement runs is 30 seconds For address family: VPNv4 Unicast Translates address family IPv4 Unicast for VRF SiteB BGP table version 416, neighbor version 416 Index 4, Offset 0, Mask 0x10 Community attribute sent to this neighbor 2 accepted prefixes consume 120 bytes Prefix advertised 107, suppressed 0, withdrawn 63 rest deleted

MPLS Training - Basic

191

show ip bgp vpnv4 all summary


Router#show ip bgp vpnv4 all summary BGP router identifier 10.7.0.5, local AS number 100 BGP table version is 35, main routing table version 35 20 network entries and 40 paths using 4980 bytes of memory 5 BGP path attribute entries using 300 bytes of memory 6 BGP rrinfo entries using 144 bytes of memory 4 BGP extended community entries using 96 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP activity 21/43 prefixes, 41/1 paths, scan interval 15 secs Neighbor V PfxRcd 10.7.0.17 4 10.7.0.18 4 rest deleted AS MsgRcvd MsgSent 100 100 13041 13041 13037 13037 TblVer 35 35 InQ OutQ Up/Down 0 0 0 1w2d 0 1w2d State/ 13 13

MPLS Training - Basic

192

Monitoring MP-BGP Sessions

router#"

show ip bgp neighbor

!!

Displays global BGP neighbors and the protocols negotiated with these neighbors!

MPLS Training - Basic

193

show ip bgp neighbor (1/2)


Router#show ip bgp neighbor 192.168.3.101 BGP neighbor is 192.168.3.101, remote AS 3, internal link BGP version 4, remote router ID 192.168.3.101 BGP state = Established, up for 02:15:33 Last read 00:00:33, hold time is 180, keepalive interval is 60 seconds Neighbor capabilities: Route refresh: advertised and received Address family IPv4 Unicast: advertised and received Address family VPNv4 Unicast: advertised and received Received 1417 messages, 0 notifications, 0 in queue Sent 1729 messages, 2 notifications, 0 in queue Route refresh request: received 9, sent 29 Minimum time between advertisement runs is 5 seconds For address family: IPv4 Unicast BGP table version 188, neighbor version 188 Index 2, Offset 0, Mask 0x4 1 accepted prefixes consume 36 bytes Prefix advertised 322, suppressed 0, withdrawn 230 ... Continued
MPLS Training - Basic 194

show ip bgp neighbor (2/2)


Router#show ip bgp neighbor 192.168.3.101 ... Continued For address family: VPNv4 Unicast BGP table version 416, neighbor version 416 Index 2, Offset 0, Mask 0x4 NEXT_HOP is always this router Community attribute sent to this neighbor 6 accepted prefixes consume 360 bytes Prefix advertised 431, suppressed 0, withdrawn 113 Connections established 7; dropped 6 Last reset 02:18:33, due to Peer closed the session ... Rest deleted

MPLS Training - Basic

195

Monitoring an MP-BGP VPNv4 Table


router#!

show ip bgp vpnv4 all

!!

Displays whole VPNv4 table!


router#!

show ip bgp vpnv4 vrf name!

Displays only BGP parameters (routes or neighbors) associated with specied VRF! Any BGP show command can be used with these parameters!
router#!

show ip bgp vpnv4 rd value!

Displays only BGP parameters (routes or neighbors) associated with specied RD!
MPLS Training - Basic 196

show ip bgp vpnv4 vrf


Router#show ip bgp vpnv4 vrf SiteA2 BGP table version is 416, local router ID is 192.168.3.102 Status codes: s suppressed, d damped, h history, * valid, > best, i internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 103:30 (default for vrf SiteA2) *> 150.1.31.36/30 0.0.0.0 0 32768 ? *>i150.1.31.128/30 192.168.3.101 0 100 0 ? *>i150.1.31.132/30 192.168.3.101 0 100 0 ? *>i203.1.1.1/32 192.168.3.103 1 100 0 65031 i *> 203.1.2.1/32 150.1.31.38 782 32768 ? *>i203.1.10.0 192.168.3.103 1 100 0 65031 i *> 203.1.20.0 150.1.31.38 782 32768 ? *>i203.1.127.3/32 192.168.3.101 1 100 0 ? *>i203.1.127.4/32 192.168.3.101 782 100 0 ? *>i203.1.134.0 192.168.3.101 1 100 0 ? *>i203.1.135.0 192.168.3.101 782 100 0 ?

MPLS Training - Basic

197

show ip bgp vpnv4 rd


Router#show ip bgp vpnv4 rd 103:30 203.1.127.3 BGP routing table entry for 103:30:203.1.127.3/32, version 164 Paths: (1 available, best #1, table SiteA2) Not advertised to any peer Local, imported path from 103:10:203.1.127.3/32 192.168.3.101 (metric 10) from 192.168.3.101 (192.168.3.101) Origin incomplete, metric 1, localpref 100, valid, internal, best Extended Community: RT:103:10

MPLS Training - Basic

198

Monitoring per-VRF CEF and LFIB structures


router#!

show ip cef vrf name

!!

Displays per-VRF CEF table!


router#!

show ip cef vrf name prex detail

!!

Displays details of an individual CEF entry, including label stack!


router#!

show tag-switching forwarding vrf name

!!

Displays labels allocated by MPLS VPN for routes in specied VRF!


MPLS Training - Basic 199

show ip cef vrf


Router#show ip cef vrf SiteA2 203.1.1.1 255.255.255.255 detail 203.1.1.1/32, version 57, cached adjacency to Serial1/0.2 0 packets, 0 bytes tag information set local tag: VPN-route-head fast tag rewrite with Se1/0.2, point2point, tags imposed: {26 39} via 192.168.3.103, 0 dependencies, recursive next hop 192.168.3.10, Serial1/0.2 via 192.168.3.103/32 valid cached adjacency tag rewrite with Se1/0.2, point2point, tags imposed: {26 39}

The show ip cef command can also display the label stack associated with the MP-IBGP route.
MPLS Training - Basic 200

show tag-switching forwarding vrf


Router#show tag-switching forwarding Local Outgoing Prefix tag tag or VC or Tunnel Id 26 Aggregate 150.1.31.36/30[V] 37 Untagged 203.1.2.1/32[V] 38 Untagged 203.1.20.0/24[V] vrf SiteA2 Bytes tag switched 0 0 0

Outgoing interface Se1/0.20 Se1/0.20

Next Hop point2point point2point

Router#show tag-switching forwarding vrf SiteA2 Local Outgoing Prefix Bytes tag tag tag or VC or Tunnel Id switched 37 Untagged 203.1.2.1/32[V] 0 MAC/Encaps=0/0, MTU=1504, Tag Stack{} VPN route: SiteA2 Per-packet load-sharing

tags 37 detail Outgoing Next Hop interface Se1/0.20 point2point

MPLS Training - Basic

201

Monitoring Labels on VPNv4 Routes


router#!

show ip bgp vpnv4 [ all | rd value | vrf name ] tags

!!

Displays labels associated with VPNv4 routes!


Router#show ip bgp vpnv4 all tags Network Next Hop In tag/Out tag Route Distinguisher: 100:1 (vrf1) 2.0.0.0 10.20.0.60 34/notag 10.0.0.0 10.20.0.60 35/notag 12.0.0.0 10.20.0.60 26/notag 10.20.0.60 26/notag 13.0.0.0 10.15.0.15 notag/26

MPLS Training - Basic

202

MPLS Troubleshooting

2003 Cisco Systems, Inc. All rights reserved.

203

MPLS Troubleshooting Agenda

Troubleshooting falls under two categories CONTROL Plane


Involves LDP, LIB, etc.

FORWARDING Plane
Involves FIB, LFIB, etc.

MPLS Training - Basic

204

MPLS Control Plane


LDP is one of the primary ways, but not the only one, to enable MPLS on an interface; other ways are
TDP BGP+Label RSVP

Each of these protocols can distribute a label for IPv4 prefixes Enabling MPLS meansthe ability to send/receive MPLS packets on an interface

MPLS Training - Basic

205

MPLS Control Plane


This Section Is All About LDP (and Its Related Components)
LDP vs. TDP LDP (Discovery, Session Setup, Label Xchange) RIB/FIB/LIB/LFIB Relationship Troubleshooting Tips Troubleshooting Case Studies

MPLS Training - Basic

206

MPLS Control Plane: LDP vs. TDP


LDP is quite similar to TDP LDP is standardized by IETF LDP has more features such as abort, MD5 authentication, notification, backoff logic, etc. LDP is now the default on Cisco routers

MPLS Training - Basic

207

MPLS Control Plane


Control Plane
LDP vs. TDP LDP (Discovery, Session Setup, Label Xchange) RIB/FIB/LIB/LFIB Relationship Troubleshooting Tips Troubleshooting Case Studies

Forwarding Plane

MPLS Training - Basic

208

MPLS Control Plane: LDP


LDP/TDP operates in three steps
Neighbor Discovery Session establishment Label Distribution/exchange

Once labels are exchanged, LIB is built LIB and FIB together helps to build LFIB

MPLS Training - Basic

209

MPLS Control Plane: TDP (i)


TDP Neighbors are discovered via TDP Hellos (like most of the routing protocols) TDP Hellos are sent to 255.255.255.255 TDP hellos are sent to UDP port = 711 TDP hellos are sent only after mpls ip is configured on an interface

Tx Hello (PE1:0)

PE1!
MPLS Training - Basic

Rx Hello (PE2:0)

PE2!
210

MPLS Control Plane: LDP (i)


LDP Neighbors are discovered via LDP Hellos (like most of the routing protocols) LDP Hellos are sent to 224.0.0.2 LDP hellos are sent to UDP port = 646 LDP hellos are sent only after both mpls ip and mpls label protocol ldp are configured on an interface **

Tx Hello (PE1:0)

PE1!
MPLS Training - Basic

Rx Hello (PE2:0)

PE2!
211

** If LDP is the global default, then interface-level LDP is not needed.

MPLS Control Plane: LDP (i)


LDP_ID should be hardcoded via
mpls ldp router-ID <interface>

The above wont do any good unless


<interface> is UP when LDP gets started Existing LDP_ID (usually an interface) is shut/unshut

Following avoids both shortcomings


mpls ldp router-ID <interface> force

MPLS Training - Basic

212

MPLS Control Plane: LDP (i)


Use the same Loopback0 as the router-ID for LDP, IGP, BGP, etc. Assign an IP address to the Loopback0 from the separate IP address subnet (or space) Avoid the IGP summarization of prefixes that correspond to the router-ids

MPLS Training - Basic

213

MPLS Control Plane: LDP (i)


sh mpls ldp discovery [detail]
Must show xmit/recv on LDP enabled interface
PE1#sh mpls ldp discovery Local LDP_ID Local LDP Identifier: Xmit & Received 10.13.1.61:0 Hellos Discovery Sources: Interfaces: Ethernet0/0 (ldp): xmit/recv Discovered E0/0 is configured LDP Id: 10.13.1.101:0 Neighbours with LDP LPD_ID Ethernet1/0 (ldp): xmit/recv LDP Id: 10.13.1.101:0

debug mpls ldp transport connections


Should give information regarding whether the HELLOS are advertised/received
MPLS Training - Basic 214

MPLS Control Plane: LDP (i)


sh mpls interface [detail]
Lists whether MPLS is enabled and the application that enabled MPLS on the interface
PE2#sh mpls interface Interface Serial2/0 PE2#

Serial2/0
IP Yes (ldp) Tunnel No Operational Yes

PE2!

P1!
! interface Serial2/0 description To P1 ser2/0 ip address 10.13.2.6/30 mpls label protocol ldp tag-switching ip tag-switching mtu 1508 !

MPLS Enabled
PE2#sh mpls interface ser2/0 detail Interface Serial2/0: IP labeling enabled (ldp) LSP Tunnel labeling not enabled BGP tagging not enabled Tagging operational Fast Switching Vectors: IP to MPLS Fast Switching Vector MPLS Turbo Vector MTU = 1508 PE2#

LDP Enabled

MPLS MTU

MPLS Training - Basic

215

MPLS Control Plane: LDP (i)


This slide is to show that BGPipv4+label (or MPeBGP) is another application that can enable MPLS; WHATS DIFFERENT HERE
RSP-PE-SOUTH-6#sh mpls int Interface IP Fddi1/0/0 Yes (ldp) ATM1/1/0.108 No RSP-PE-SOUTH-6# Tunnel No No Operational Yes Yes

MPLS is Operational. LDP not enabled LDP not enabled BGP+Label Enabled

RSP-PE-SOUTH-6#sh mpls int ATM1/1/0.108 de Interface ATM1/1/0.108: IP labeling not enabled LSP Tunnel labeling not enabled BGP tagging enabled Tagging operational Optimum Switching Vectors: IP to MPLS Feature Vector MPLS Feature Vector Fast Switching Vectors: IP to MPLS Fast Feature Switching Vector MPLS Feature Vector MTU = 4470 RSP-PE-SOUTH-6#

MPLS MTU

MPLS Training - Basic

216

MPLS Control Plane: LDP (ii)


After discovering each other, they want to get cozy and establish the session.
(Even routers have the dating concept)

LDP INITIALIZATION, KEEPALIVE and ADDRESS messages are exchanged to establish LDP session LSR_ID (Transport address) MUST be IP reachable
LDP Session Hello

PE1!
10.13.1.61/32

Hello

P1!
10.13.1.101/32

MPLS Training - Basic

217

MPLS Control Plane: LDP (ii)


LDP_ID =>
LSR_ID
W! !! X Y Z !! !! n! LSR ID! Label Space ID!

The LSR_ID is a four byte number that identifies a specific LSR. It is derived from an interface on the LSR. By default, it is the highest IP address, or highest IP address of a loopback if its available.

Label_Space_Id
A two byte number that identifies a specific label space on the LSR. 0x00 is reserved for the platform label space (i.e. frame-mode MPLS). Non-zero refers to the interface label space (i.e. cell-mode MPLS).

MPLS Training - Basic

218

MPLS Control Plane: LDP (ii)


LDP session is a TCP session (port = 646) Multiple links between two routers still mean single LDP session
LDP_ID PE1#sh mpls ldp neighbor Peer LDP Ident: 10.13.1.101:0; Local LDP Ident 10.13.1.61:0 TCP connection: 10.13.1.101.11031 - 10.13.1.61.646 Unsolicited Label State: Oper; Msgs sent/rcvd: 58/60; Downstream Distribution Up time: 00:39:27 LDP discovery sources: Interfaces on Ethernet0/0, Src IP addr: 10.13.1.5 which peers Ethernet1/0, Src IP addr: 10.13.1.9 identified Addresses bound to peer LDP Ident: 10.13.1.9 10.13.1.5 10.13.2.5 10.13.1.101 Peers connected interfaces PE1#sh tcp brief| i 646 43ABB020 10.13.1.101.11031
MPLS Training - Basic

10.13.1.61.646

ESTAB
219

MPLS Control Plane: LDP (ii)


Relevant LDP Session Commands/Debugs:
sh mpls ldp neighbor [neighbor]
Shows LDP neighbor and relevant info

sh mpls ldp neighbor [interface]


LDP neighbors discovered over this interface

Debug mpls ldp session io|state


Useful when the session doesnt come up

Debug mpls ldp messages sent|receive


Shows all the LDP messages sent or received

MPLS Training - Basic

220

MPLS Control Plane: LDP (iii)


Now, the LDP session is established, LDP neighbors start exchanging label bindings via LABEL MAPPING message (after the Keepalive gets exchanged) Label binding => prefix + Label Label bindings are stored in the LIB
LIB => Label Information Base
Label exchange 10.13.1.61/32

PE1!

P1!
10.13.1.101/32

MPLS Training - Basic

221

MPLS Control Plane: LDP (iii)


LIB entry can be verified with the following
PE1#sh mpls ip bindings 10.13.1.62 32 10.13.1.62/32 in label: 20 out label: 2001 lsr: 10.13.1.101:0 PE1#
Ok. I hear you 10.13.1.101:0. I have the binding from you in my LIB now But whether I use your binding or not will be dictated by RIB entry This is 10.13.1.101:0. Use label 2001 to reach 10.13.1.62/32

Local binding Remote binding

10.13.1.61/32
Oh ok. Per RIB, 10.13.1.101 is the next-hop for 10.13.1.62/32. I have to use label 2001 in LFIB.

PE1!

E0/0 E0/1

P1!
10.13.1.101/32

10.13.1.62/32

PE1#sh Local tag 20 PE1#

mpls forwarding 10.13.1.62 Outgoing Prefix tag or VC or Tunnel Id 2001 10.13.1.62/32 2001 10.13.1.62/32

Bytes tag switched 0 0

Outgoing interface Et0/0 Et1/0

Next Hop 10.13.1.5 10.13.1.9

MPLS Training - Basic

222

MPLS Control Plane: LDP (iii)


sh mpls ip binding detail
Lists all prefixes with labels and LDP neighbors

sh mpls ip binding <prefix> <mask> det


Lists ACLs (if any), prefix bindings, and LDP neighbors Notice Advertised to: field

sh mpls ip binding advertisement-acls


Lists LDP filter, if there is any, on the first line. Prefixes followed by Advert acl(s): are advertised via LDP, others are not

MPLS Training - Basic

223

MPLS Control Plane


Control Plane
LDP vs. TDP LDP (Discovery, Session Setup, Label Xchange) RIB/FIB/LIB/LFIB Relationship Troubleshooting Tips Troubleshooting Case Studies

Forwarding Plane

MPLS Training - Basic

224

RIB/FIB/LIB/LFIB
RIB is the Routing Information Base that is analogous to the ip routing table FIB aka CEF is Forwarding information base that is derived from the ip routing table LIB is Label Information Base that contains all the label bindings learned via LDP LFIB is Label Forwarding Information Base that is derived from FIB entries and corresponding LIB entries Lets go through the pictorial view

MPLS Training - Basic

225

MPLS Control Plane: RIB/FIB/LIB/LFIB


Control plane
Routing Protocols Database
Routing Updates from other routers

Label Bindings Learned Via LDP from Other Routers

Forwarding plane
Incoming IP Packet

IP forwarding table (FIB) Label forwarding table (LFIB)

Managed by CEF

Incoming MPLS Packet

Outgoing MPLS/IP Packet

Population of RIB/FIB/LIB/LFIB in a LSR


MPLS Training - Basic 226

MPLS Control Plane: Debugs


Be Careful on the Production Routers
debug mpls ldp advertisements
Useful to see label bindings that are advertised

debug mpls ldp binding


Useful to see label bindings that are received

debug mpls ldp message sent|received


Useful for the protocol understanding purposes

MPLS Training - Basic

227

MPLS Control Plane


Control Plane
LDP vs. TDP LDP (Discovery, Session Setup, Label Xchange) RIB/FIB/LIB/LFIB relationship Troubleshooting Tips Troubleshooting Case Studies

Forwarding Plane

MPLS Training - Basic

228

MPLS Control Plane: Troubleshooting Tips


1. Check for same label protocol to be configured on both sides of the interface
Sh mpls ldp discovery | inc ldp|tdp

2. Check whether correct local LSR_ID is used on both LSRs (sh mpls ldp disc)
sh mpls ldp discovery2nd line in output

PE1#sh mpls ldp disc | i ldp|tdp Ethernet0/0 (ldp): xmit/recv PE1#

3. Dont assume that the neighbor discovery means everything is good

PE1#sh mpls ldp disco Local LDP Identifier: 10.13.1.61:0

MPLS Training - Basic

229

MPLS Control Plane: Troubleshooting Tips


4. Check IP reachability to remote LSR_ID on both LSRs
ping <lsr_id>
PE1#ping 10.13.1.101 source 10.13.1.61 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.13.1.101, timeout is 2 seconds: Packet sent with a source address of 10.13.1.61 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 32/49/72 ms PE1#

5. Check for ACL or ICMP unreachable blockages

6. Untagged outgoing label for /32 routes i.e. PEs loopbacks is almost always alarming
sh mpls ldp bind <prefix> <mask>

7. Check the label binding for a prefix on both LSRs

PE1#sh mpls ldp bind 10.13.1.62 32 tib entry: 10.13.1.62/32, rev 16 local binding: tag: 17 remote binding: tsr: 10.13.1.101:0, tag: 2001 PE1#
MPLS Training - Basic 230

MPLS Control Plane: Troubleshooting Tips


8. Good practice is to configure the Loopback0 as the router-ID for LDP
mpls ldp router-id loopback0 force

MPLS Training - Basic

231

MPLS Control Plane


Control Plane
LDP vs. TDP LDP (Discovery, Session Setup, Label Xchange) RIB/FIB/LIB/LFIB Relationship Troubleshooting Tips Troubleshooting Case Studies

Forwarding Plane

MPLS Training - Basic

232

MPLS Control Plane: LDP Troubles


Lets do some REAL Troubleshooting now

MPLS Training - Basic

233

MPLS Control Plane: LDP Problems


Prob #1Session Establishment (Protocol Mismatch)
Atm1/1/0.108 10.13.1.61/32

PE1!

P1!
10.13.1.101/32

PE1#sh mpls ldp discovery Local LDP Identifier: 10.13.1.61:0 Discovery Sources: Interfaces: ATM1/1/0.108 (tdp): xmit PE1#

Why no recv?

P1#sh mpls ldp discovery Local LDP Identifier: 10.13.1.101:0 LDP Discovery Sources: Interfaces: ATM2/0.108(ldp): xmit P1#

Why no recv?

TDP

LDP

TIPCheck for the Protocol Mismatch and Fix It


PE1(config)#int atm1/1/0.108 PE1(config-if)#mpls label protocol ldp
MPLS Training - Basic 234

MPLS Control Plane: LDP Troubles


Prob #2Session Establishment (No Route to Peer)
Atm1/1/0.108 10.13.1.61/32
PE1#sh mpls ldp discovery Local LDP Identifier: 10.13.1.61:0 Discovery Sources: Interfaces: ATM1/1/0.108 (ldp): xmit/recv LDP Id: 10.13.1.101:0 PE1# PE1#sh mpls ldp neigh 10.13.1.101 PE1#

PE1!

P1!
10.13.1.101/32
P1#sh mpls ldp discovery Local LDP Identifier: 10.13.1.101:0 LDP Discovery Sources: Interfaces: ATM2/0.108: xmit/recv LDP Id: 10.13.1.61:0; no route P1# P1#sh ip route 10.13.1.61 % Network not in table P1#

Looks Good But No relationship

This is the problem

TIPCheck for IP reachability to LDP_ID; Fix It by Letting PE1 Advertise 10.13.1.61/32 via IGP to P1
MPLS Training - Basic 235

MPLS Control Plane: LDP Troubles


Prob #3Session Establishment (No Specific Route)
Gig8/0/0.44 10.13.1.41/32
oops Ok. PE1#sh mpls ldp neighbor 10.13.1.48 PE1# PE1#sh mpls ldp discovery Local LDP Identifier: 10.13.1.41:0 Gi8/0/0.44 (ldp): xmit/recv LDP Id: 10.13.1.48:0 PE1# PE1#sh ip route 10.13.1.48 Routing entry for 10.13.1.48/32 Known via "isis", distance 115, metric 10, type level-1 Redistributing via isis Last update from 10.13.4.9 on Gig8/0/0.44, 20:22:14 ago Routing Descriptor Blocks: * 10.13.4.9, from 10.13.1.48, via Gigt8/0/0.44 Route metric is 10, traffic share count is 1 PE1#

PE1!

P1!
10.13.1.48/32
P1#sh mpls ldp neighbor 10.13.1.41 oops P1# P1#sh mpls ldp discovery Local LDP Identifier: Ok. 10.13.1.48:0 Gi3/0/0.44 (ldp): xmit/recv LDP Id: 10.13.1.41:0 P1# P1#sh ip route 10.13.1.41 Routing entry for 10.13.0.0/22 Ouchhh Known via "bgp 30000", distance 200, metric 0 Tag 1, type internal Last update from 10.13.1.251 20:10:38 ago Routing Descriptor Blocks: * 10.13.1.251, from 10.13.1.40, 20:10:38 ago Route metric is 0, traffic share count is 1 AS Hops 5 P1#

Ok.

MPLS Training - Basic

P1 doesnt have a specific route to PE1.

236

MPLS Control Plane: LDP Troubles


Prob #3Session Establishment (Cont.)
Gig8/0/0.44 10.13.1.41/32

PE1!

P1!
10.13.1.48/32

PE1#ping 10.13.1.48 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.13.1.48, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms PE1#

P1#ping 10.13.1.41 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.13.1.41, timeout is 2 seconds: ..... Success rate is 0 percent (0/5) P1#

Eeeekks !! It is an IP problem.

TIPCheck for IP connectivity first. Unless Layer3 is up, Layer4 (TCP session for LDP) wont come up.
MPLS Training - Basic 237

MPLS Control Plane: LDP Troubles


Prob #4Untagged Problem
PE1#sh tag for 11.10.128.138 Local Outgoing Prefix Bytes tag Outgoing Next Hop tag tag or VC or Tunnel Id switched interface 16 Untagged 11.10.128.138/32 0 PO4/1/0 point2point PE1# PE1#sh mpls ldp bind 11.10.128.138 32 tib entry: 11.10.128.138/32, rev 14 local binding: tag: 16 PE1# P1#sh mpls ldp bind 11.10.128.138 32 tib entry: 11.10.128.138/32, rev 4849(no route) local binding: tag: 630 remote binding: tsr: 10.13.1.54:0, tag: 16 remote binding: tsr: 11.10.65.12:0, tag: 48 P1# P1#sh ip route 11.10.128.138 Routing entry for 11.10.0.0/16 Known via "isis", distance 115, metric 44, type level-2 Redistributing via isis Last update from 11.10.65.13 on POS0/0, 1d00h ago Routing Descriptor Blocks: * 11.10.65.13, from 11.10.128.31, via POS0/0 Route metric is 44, traffic share count is 1 P1# MPLS Training - Basic

PE1!
Pos4/1/0

Untagged ? No remote binding. Huh No route

P1!
But there is a RIB entry. Lets check FIB entry -

11.10.128.138

238

MPLS Control Plane: LDP Troubles


Prob #4No Route Problem (Cont.)
P1#sh ip cef 11.10.128.138 11.10.0.0/16, version 142, cached adjacency to POS0/0 0 packets, 0 bytes tag information set local tag: 307 fast tag rewrite with PO0/0, point2point, tags imposed {48} via 11.10.65.13, POS0/0, 0 dependencies next hop 11.10.65.13, POS0/0 unresolved <<<<<<<<<<<<<<<<<<<<<<<<<< valid cached adjacency tag rewrite with PO0/0, point2point, tags imposed {48} P1# P1#clear ip route 11.10.128.138 P1#sh mpls ldp bind 11.10.128.138 32 tib entry: 11.10.128.138/32, rev 4849 local binding: tag: 307 remote binding: tsr: 10.13.1.54:0, tag: 16 remote binding: tsr: 11.10.65.20:0,tag:48 P1#

FIBs local label is different from that of LIB Unresolved ?

PE1!
Pos4/1/0

P1!
Pos0/0 11.10.128.138

TIPIf Local Label for a Prefix Is Not Same in FIB and LIB, Then Issue clear ip route <prefix> to fix
MPLS Training - Basic 239

MPLS Control Plane: LDP Troubles


Prob #5LFIB Entry Disappears No LFIB entry This might occur if the RIB owner for an IPv4 routes changes from IGP to BGP LDP doesnt allocate labels for the BGP learned IPv4 routes Notice the absence of local binding in LIB for that route

MPLS Training - Basic

240

MPLS Control Plane: LDP Troubles


7206-PE-SOUTH-1#sh mpls ldp bind 4.4.0.0 24 tib entry: 4.4.0.0/24, rev 152 remote binding: tsr: 10.13.1.69:0, tag: 213 remote binding: tsr: 10.13.1.68:0, tag: 212 7206-PE-SOUTH-1# 7206-PE-SOUTH-1#sh ip route 4.4.0.0 Routing entry for 4.4.0.0/24 Known via "bgp 30000", distance 200, metric 0 Tag 1, type internal Redistributing via isis, ospf 1 Last update from 10.13.1.251 5d17h ago Routing Descriptor Blocks: * 10.13.1.251, from 10.13.1.40, 5d17h ago Route metric is 0, traffic share count is 1 AS Hops 5 Route tag 1 7206-PE-SOUTH-1#

No Local Binding

Because it is a BGP learned prefix

LDP doesnt allocate labels for the BGP learned IPv4 routes.

MPLS Training - Basic

241

MPLS Forwarding Plane


With MPLS, the idea is to de-couple the forwarding from the IP The forwarding decision is based on the MPLS header, not the IP header The above is true once the packet is inside the MPLS network Forwarding is still based on the IP header at the edge where the packet first enters the MPLS network

MPLS Training - Basic

242

MPLS Forwarding Plane


Control Plane Forwarding Plane
CEFs Role New Ethertype, What Is a Label, Types of Labels Forwarding Explained Loadsharing Fragmentation and MTU Troubleshooting Tips Troubleshooting Case Studies LSP Ping/Traceroute
MPLS Training - Basic 243

MPLS Forwarding Plane: CEFs Role


CEF must be configured on all the routers in a MPLS network CEF takes care of the crucial recursion and resolution operations MPLS relies on CEF CEF is must for the MPLS

MPLS Training - Basic

244

MPLS Forwarding Plane


Control Plane Forwarding Plane
CEFs Role New Ethertype, What Is a Label, Types of Labels Forwarding Explained Loadsharing Fragmentation and MTU Troubleshooting Tips Troubleshooting Case Studies LSP Ping/Traceroute
MPLS Training - Basic 245

MPLS Forwarding Plane: Ethertype


Ethertype 0x0800 refers to IP Ethertype 0x8847 refers to MPLS Based on the Ethertype, the packet is handed over to the appropriate processing engine in the router

MPLS Training - Basic

246

MPLS Fwd Plane: What Is a Label


A LABEL is 4 bytes identifier, which is carried by the packet and used to identify a prefix

Label

EXP S

TTL

0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

Label EXP/QoS S TTL

= Actual Label, = Experimental bits, = End of Stack, = Time to Live,

= 20 bits = 3 bits = 1 bit = 8 bits

MPLS Training - Basic

247

MPLS Fwd Plane: Where Is Label?


Typically resides between L2 and L3 header
Ethernet Packet Frame-relay Packet PPP Packet ATM Cell Header
GFC MAC Header FR Header PPP Header VPI Label VCI Label Label Label PTI CLP HEC Layer 3 Layer 3 Layer 3 Layer 3 Data

Routers always makes forwarding-decision based on the topmost label i.e. label1 belowMAC Label1 Label2 Label3 Layer 3

MPLS Training - Basic

Label stack

248

MPLS Forwarding Plane: Outgoing Labels


RSP-PE-SOUTH-5#sh mpls forwarding 10.13.1.11 Local Outgoing Prefix Bytes tag tag tag or VC or Tunnel Id switched 59 46 10.13.1.11/32 0 RSP-PE-SOUTH-5# Outgoing interface Se10/0/0 Next Hop point2point

Outgoing label also conveys what treatment the packet is going to get; it could also be
Pop Untagged Aggregate 0 Pops the topmost label Untag the incoming MPLS packet Untag and then do a FIB lookup Nullify the top label (first 20bits)

Label values 0-15 are reserved


MPLS Training - Basic 249

MPLS Forwarding Plane: Outgoing Labels


PE1#sh Local tag 16 17 18 19 20 21 22 23 24 26 PE1# mpls forwarding-table Outgoing Prefix tag or VC or Tunnel Id 2002 10.13.1.22/32 2002 10.13.1.22/32 2001 10.13.1.62/32 2001 10.13.1.62/32 Pop tag 10.13.1.101/32 Pop tag 10.13.1.101/32 Pop tag 10.13.2.4/30 Pop tag 10.13.2.4/30 Untagged 5.5.5.5/32[V] Pop tag 10.13.21.4/30 Pop tag 10.13.21.4/30 Pop tag 10.13.22.4/30 Pop tag 10.13.22.4/30 Aggregate 0.0.0.0/0[V] Aggregate 200.1.61.4/30[V] Untagged 30.30.30.1/32[V] Bytes tag switched 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Outgoing interface Et0/0 Et1/0 Et0/0 Et1/0 Et1/0 Et0/0 Et1/0 Et0/0 Se2/0 Et1/0 Et0/0 Et1/0 Et0/0 Se2/0 Next Hop 10.13.1.5 10.13.1.9 10.13.1.5 10.13.1.9 10.13.1.9 10.13.1.5 10.13.1.9 10.13.1.5 point2point 10.13.1.9 10.13.1.5 10.13.1.9 10.13.1.5 point2point

V means it is a VPN prefix


MPLS Training - Basic 250

MPLS Forwarding Plane: Outgoing Labels


Untagged
Convert the incoming MPLS packet to an IP packet and forward it

Pop
Pop the top label from the label stack present in an incoming MPLS packet and forward it as an MPLS packet.; if there was only one label in the stack, then forward it as an IP packet; SAME as imp-null label

Aggregate
Convert the incoming MPLS packet to an IP packet and then do a FIB lookup for it to find out the outgoing interface

0 (zero)
Same as exp-null label; simplify fills 0 in the first 20 bits of label; helps to preserve the EXP value of the top label

MPLS Training - Basic

251

MPLS Forwarding Plane


Control Plane Forwarding Plane
CEFs Role New Ethertype, What Is a Label, Types of Labels Forwarding Explained Loadsharing Fragmentation and MTU Troubleshooting Tips Troubleshooting Case Studies LSP Ping/Traceroute
MPLS Training - Basic 252

MPLS Forwarding Plane


Three cases in the MPLS forwarding
1. Label ImpositionIP to MPLS conversion 2. Label swappingMPLS to MPLS 3. Label dispositionMPLS to IP conversion

So, depending upon the case, we need to check


1. FIBFor IP packets that get forwarded as MPLS 2. LFIBFor MPLS packets that get fwded as MPLS 3. LFIBFor MPLS packets that get fwded as IP

MPLS Training - Basic

253

MPLS Forwarding Plane


Case 1: IP Packets Get Forwarded as MPLS
PE1 PE2 P1 1.1.1.0/30

2001 IP Packet IP Packet

PE1 does a FIB lookup for the incoming IP packet It imposes the label (if there is one) For troubleshooting, look at the FIB (not LFIB)

PE1#sh ip cef 1.1.1.0 1.1.1.0/30, version 25, epoch 0, cached adjacency 10.13.1.5 0 packets, 0 bytes tag information set local tag: 20 fast tag rewrite with Et0/0, 10.13.1.5, tags imposed: {2001} via 10.13.1.5, Ethernet0/0, 0 dependencies next hop 10.13.1.5, Ethernet0/0 valid cached adjacency tag rewrite with Et0/0, 10.13.1.5, tags imposed: {2001} PE1#

MPLS Training - Basic

254

MPLS Forwarding Plane


Case 2: MPLS Packets Get Forwarded as MPLS
PE1 PE2 P1 1.1.1.0/30

2001 IP Packet

20

IP Packet

P1 does the LFIB lookup for incoming MPLS packets P1 could swap (or dispose) the label For troubleshooting, look at the LFIB (not FIB)

P1#sh mpls for 1.1.1.0 Local Outgoing Prefix tag tag or VC or Tunnel Id 2001 20 1.1.1.1.0/30 0 P1#

Bytes tag Outgoing Next Hop switched interface Se2/0 point2point

MPLS Training - Basic

255

MPLS Forwarding Plane


Case 3: MPLS Packets Get Forwarded as IP
PE1 PE2 P1 1.1.1.0/30

20

IP Packet

IP Packet

Typically happen at the edge Could also happen at the PHP router For troubleshooting, look at the LFIB (not FIB)

PE2#sh mpls for 1.1.1.0 Local Outgoing Prefix tag tag or VC or Tunnel Id 20 Untagged 1.1.1.1.0/30 PE2#

Bytes tag Outgoing Next Hop switched interface 0 Se2/0 point2point

MPLS Training - Basic

256

MPLS Forwarding Plane


Control Plane Forwarding Plane
CEFs Role New Ethertype, What Is a Label, Types of Labels Forwarding Explained Loadsharing in MPLS Fragmentation and MTU Troubleshooting Tips Troubleshooting Case Studies LSP Ping/Traceroute
MPLS Training - Basic 257

MPLS Forwarding Plane: Loadsharing


Loadsharing (due to multiple paths to a prefix) in MPLS is no different from that of IP Hashing-algorithm is still the typical FIB based i.e. per-dest loadsharing by default ** So the below show command is still relevant
Sh ip cef exact-route <source> <dest> etc

But the dest must be known in the FIB table, otherwise the command wont work
Wont work on P routers for the VPN prefixes

MPLS Training - Basic

258

MPLS Forwarding Plane


Control Plane Forwarding Plane
CEFs Role New Ethertype, What Is a Label, Types of Labels Forwarding Explained Loadsharing Fragmentation and MTU Troubleshooting Tips Troubleshooting Case Studies LSP Ping/Traceroute
MPLS Training - Basic 259

MPLS Fwd Plane: Fragmentation


After the Layer 2 header is added to the IP packet, the resulting packet size shouldnt exceed the max packet size (IP MTU size) applicable; otherwise, packet will be fragmented MTU size needs to be tuned to avoid fragmentation in MPLS network MTU could be increased only for MPLS packets => MPLS MTU

MPLS Training - Basic

260

MPLS Fwd Plane: Fragmentation MTU Setting in MPLS


Two things to remember
1. DF bit of an incoming packet 2. MTU size of an outgoing interface

Label imposition(s) increases the packet size by 4 bytes/label, hence the outgoing packet size may exceed interface MTU size, hence the need to tune MTU Q is: which MTU to tune in MPLS network?

MPLS Training - Basic

261

Fragmentation MTU Setting in MPLS


Most of the interfaces (depending upon the hardware) support transmitting packets bigger than the interface MTU size mpls mtu <bytes> can be applied to an interface to change the MPLS MTU size on the interface MPLS MTU size is checked by the router
While converting an IP packet into a labeled packet or transmitting a labeled packet

MPLS Training - Basic

262

Fragmentation MTU Setting in MPLS


Remember That:
mpls mtu <bytes> command has no effect on interface or IP MTU size By default, MPLS MTU = interface MTU MPLS MTU setting doesnt affect MTU handling for IP-to-IP packet switching

MPLS Training - Basic

263

Fragmentation MTU Setting in MPLS


If the label imposition makes the packet bigger than the MPLS MTU size of an outgoing interface, then
If the DF bit set, then discard the packet and send ICMP reply back (with code=4) If the DF bit is not set, then fragment the IP packet (say, into 2 packets), and then impose the same label(s) on both the packets, and then transmit MPLS packets

Fragmentation should be done at the edge itself

MPLS Training - Basic

264

MTU Setting in MPLS Configuring the MPLS MTU

RSP-PE-WEST-4(config)#int fa1/1/0 RSP-PE-WEST-4(config-if)#mpls mtu 1508 RSP-PE-WEST-4(config-if)#^Z RSP-PE-WEST-4#

MPLS Training - Basic

265

MTU Setting in MPLS Before setting the MPLS MTU


Interface MTU is 1500 bytes (no change):
RSP-PE-WEST-4#sh int fa1/1/0 FastEthernet1/1/0 is up, line protocol is up Hardware is cyBus FastEthernet Interface, address is 0004.4e75.4828 MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec, RSP-PE-WEST-4#

MPLS MTU is 1508 bytes (changed):


RSP-PE-WEST-4#sh mpls interface fa1/1/0 de Interface FastEthernet1/1/0: IP tagging enabled TSP Tunnel tagging not enabled Tagging operational .. MTU = 1508 RSP-PE-WEST-4#
MPLS Training - Basic 266

MPLS Fwd Plane: Show Commands


sh mpls forwarding
Shows all LFIB entries (vpn, non-vpn, TE, etc.)

sh mpls forwarding <prefix>


LFIB lookup based on a prefix

sh mpls forwarding label <label>


LFIB lookup based on an incoming label

sh mpls forwarding <prefix> detail


Shows detailed info such as L2 encap, etc.

MPLS Training - Basic

267

MPLS Fwd Plane: Show Command


RSP-PE-WEST-4#sh mpls for 10.13.1.11 detail Local Outgoing Prefix Bytes tag Outgoing tag tag or VC or Tunnel Id switched interface 45 51 10.13.1.11/32 0 Fa1/1/1 MAC/Encaps=14/18, MRU=1500, Tag Stack{51} 0003FD1C828100044E7548298847 00033000 No output feature configured Per-packet load-sharing RSP-PE-WEST-4#

Detail Is Optional
Next Hop 10.13.7.33

Only One Outgoing Label in the Label Stack

MRU Max Receivable Unit. The received packet will be transmitted unfragmented on Fa1/1/1, if its size is not more than 1500B.

MAC header =0003FD1C828100044E754829 MPLS Ethtype= 0x8847 Label = 0x00033000

14/18 means that the L2 header is of 14 bytes, but L2+label header is 18 bytes (one label is 4 bytes)

MPLS Training - Basic

268

MPLS Fwd Plane: Debugs


Be Careful on the Production Routers
Debug mpls lfib cef
Useful for seeing FIB and LFIB interaction when a label is missing for a prefix

debug mpls lfib struct


Shows changes in the LFIB structures when label is allocated/deallocated

MPLS Training - Basic

269

MPLS Forwarding Plane


Control Plane Forwarding Plane
CEFs Role New Ethertype, What Is a Label, Types of Labels Forwarding Explained Loadsharing Fragmentation and MTU Troubleshooting Tips Troubleshooting Case Studies LSP Ping/Traceroute
MPLS Training - Basic 270

MPLS Fwd Plane: Troubleshooting Tips


1. Label imposition is always done using FIB 2. Label swapping and disposition is always done using LFIB 3. Increase the MPLS MTU to accommodate the largest packet payload size 4. Make sure that baby giant/jumbo is enabled on the Ethernet switches

MPLS Training - Basic

271

MPLS Fwd Plane: Troubleshooting Tips


5. Check that MPLS enabled interface has TAG adjacency via
sh adjacency <interface>

6. Check that the LFIBs outgoing label is same as the incoming label in neighbors LFIB 7. Check the LSP via traceroute that shows labels used by each router in the path **
traceroute <prefix>

MPLS Training - Basic

272

MPLS Forwarding Plane: TAG adj


8. Make sure that the interface has the tag adjacency along with IP adj, otherwise MPLS packets will not get switched on that interface

PE1#sh adjacency e0/0 de Protocol Interface TAG Ethernet0/0

IP

Ethernet0/0

Address 10.13.1.5(6) 0 packets, 0 bytes AABBCC006500AABBCC0001008847 mpls adj never Epoch: 0 10.13.1.5(35) 0 packets, 0 bytes AABBCC006500AABBCC0001000800 ARP 03:46:13 Epoch: 0

L2 header for MPLS

L2 header for IP

PE1#

MPLS Training - Basic

273

MPLS Forwarding Plane


Control Plane Forwarding Plane
CEFs Role New Ethertype, What Is a Label, Types of Labels Forwarding Explained Loadsharing Fragmentation and MTU Troubleshooting Tips Troubleshooting Case Studies

MPLS Training - Basic

274

MPLS Fwd Plane: Troubles


Lets do some real trouble(shooting)

MPLS Training - Basic

275

MPLS Fwd Plane: Troubles and Shooting


Prob #1No Entries in LFIB
P1#sh mpls forwarding-table 10.13.1.61 Tag switching is not operational. CEF or tag switching has not been enabled. Local Outgoing Prefix Bytes tag tag tag or VC or Tunnel Id switched P1# P1#sh mpls ip binding 10.13.1.61/32 out label: out label: 10.13.1.62/32 out label: out label: 10.13.1.101/32 out label: out label: 10.13.2.4/30 out label: out label: P1# P1#sh ip cef %CEF not running Prefix P1# Outgoing interface Next Hop

imp-null 21 imp-null 17 19 18 imp-null 19

lsr: 10.13.1.61:0 lsr: 10.13.1.62:0 lsr: 10.13.1.62:0 lsr: 10.13.1.61:0 lsr: 10.13.1.62:0 lsr: 10.13.1.61:0 lsr: 10.13.1.62:0 lsr: 10.13.1.61:0

Next Hop

Interface

TIPEnable CEF; It Is Must for MPLS


MPLS Training - Basic 276

MPLS Fwd Plane: Troubles and Shooting


Prob #2 Recursive Rewrite Problem If you ever see Recursive rewrite via in the sh ip cef .. output, then it might indicate a problem.

2611-CE-30#sh ip cef 10.13.1.74 10.13.1.74/32, version 43, epoch 0, cached adjacency 5.5.5.14 0 packets, 0 bytes tag information set local tag: BGP route head fast tag rewrite with Recursive rewrite via 217.60.217.2/32, tags imposed {23} via 217.60.217.2, 0 dependencies, recursive next hop 5.5.5.14, Ethernet0/0.2 via 217.60.217.2/32 valid cached adjacency tag rewrite with Recursive rewrite via 217.60.217.2/32, tags imposed {23} 2611-CE-30#

Problem with the 217.60.217.2. Check its label binding in FIB/ LIB.

MPLS Training - Basic

277

MPLS Fwd Plane: Troubles and Shooting (Cont.)


Recursive rewrite usually means that
(a) Either the label to the next-hop is not available (b) Or there is an internal problem with the CEF recursion resolution process

(a) usually turns out to be a LDP problem, and should be fixed by investigating into LDP (b) could be fixed by clear ip route <prefix> or clear ip bgp *

MPLS Training - Basic

278

MPLS Fwd Plane: Troubles and Shooting (Cont.)


In order to troubleshoot (a), check the label availability for the next-hop (in LIB). If it is missing, then fix LDP
2611-CE-30#sh mpls for 217.60.217.2 Local Outgoing Prefix Bytes tag Outgoing Next Hop tag tag or VC or Tunnel Id switched interface 17 Untagged 217.60.217.2/32 0 Et0/0.2 5.5.5.14 2611-CE-30# 2611-CE-30#sh mpls ldp bind 217.60.217.2 32 tib entry: 217.60.217.2/32, rev 14 local binding: tag: 17 2611-CE-30# 2611-CE-30#sh mpls ldp dis Local LDP Identifier: 217.60.217.3:0 Discovery Sources: Interfaces: Ethernet0/0.2 (ldp): xmit 2611-CE-30#co
MPLS Training - Basic

Untagged outgoing label

No remote label binding in LIB

Because there is no LDP neighbor.

279

MPLS Fwd Plane: Troubles and Shooting (Cont.)


LDP session needs to be established first It is an LDP (control plane) problem Troubleshoot for the LDP (as shown in the control plane section)

MPLS Training - Basic

280

MPLS VPN Troubleshooting

2003 Cisco Systems, Inc. All rights reserved.

281

MPLS VPN Troubleshooting - Problems


VPN traffic is not getting through VPN routes are missing from the VRF table VPN Routes are present, but labels missing Labels are in BGP VPNv4 table but not in LFIB

MPLS Training - Basic

282

MPLS VPN - Troubleshooting Tips


Symptom 1
VPNv4 Prefix Is Not Received at the Remote (Receiving) PE

Tip

Cisco IOS Command

Make Sure that export RT Sh ip vrf detail <X> at the Advertising PE <vrf> | inc Export | Matches with import RT Import | RT <X> at the Receiving PE Validate the Match/Set sh ip vrf de <vrf> | Clause within the Export- inc route-map; Map or Import-Map (if Any) sh route-map <map> If BGP Is Not the Chosen PE-CE Protocol, then Validate BGP->IGP Redistribution Check whether the Remote PE Is Configured as the rr-client within VPNv4 af at the RouteReflectors sh run | b router <igp>

VPNv4 Prefix Is Not Received at the Remote (Receiving) PE

VPNv4 Prefix Is Not Received at the Remote (Receiving) PE

VPNv4 Prefix Is Not Received at the Remote (Receiving) PE

Sh run | b address-family vpnv4

MPLS Training - Basic

283

MPLS VPN - Troubleshooting Tips (Cont.)


Symptom
VPNv4 Prefix Is Not Received at the Remote (Receiving) PE

Tip
Make Sure that the RouteReflectors and PEs Are Configured to Send ExtCommunity towards the iBGP Peers within the VPNv4 af Check the Label Information in BGP and LFIB at the Advertising PE Router Check the Label Information in BGP and FIB at the Receiving PE Router

Cisco IOS Command


sh run | b addressfamily vpnv4

5
VPNv4 Traffic Is Not Getting Forwarded End-to-End

sh ip bgp vpn vrf <vrf> label | inc <prefix> sh mpls for vrf <vrf> | inc <prefix>

6
VPNv4 Traffic Is Not Getting Forwarded End-to-End

sh ip bgp vpn vrf <vrf> label | inc <prefix> sh ip cef vrf <vrf> <prefix>

MPLS Training - Basic

284

MPLS VPN - Label Stack


Outer (or IGP) label in the label stack provides a LSP from ingress PE to egress PE via MPLS cloud Inner (or BGP) label refers to the VPNv4 prefix at the egress PE

tag rewrite with Se2/0, point2point, tags imposed: {2003 20}

MPLS Training - Basic

285

Agenda
Control Plane
Control Plane Troubleshooting Tips Real-life Examples Summary of Helpful Cisco IOS Commands

Forwarding Plane
Dissecting LFIB Load sharing in MPLS VPN Networks Forwarding Plane Troubleshooting Tips Real-life Examples Summary of Helpful Cisco IOS Commands

Conclusion
MPLS Training - Basic 286

MPLS VPN Ctrl PlaneTrouble #1


#1: VPN prefix doesnt have any label in the LFIB on the local PE
PE1#sh mpls forwarding vrf v1 | i 200.1.61.4 PE1# PE1#sh ip bgp vpn vrf v1 label | i 200.1.61.4 PE1# PE1#sh ip bgp vpn vrf v1 200.1.61.4 %Network not in the table PE1#

PE1
Ser2/0

CE1

Loop0:10.13.1.61/32

AS#1 MPLS Backbone

TIP: Label allocation is done by BGP. So make sure the prefix is in the BGP VRF table. Hintredistribute connected

MPLS Training - Basic

287

MPLS VPN Ctrl PlaneTrouble #1 (Cont.)


PE1(conf)#router bgp 1 PE1(conf-router)#address-family ipv4 vrf v1 PE1(conf-router-af)#redistribute connected PE1(conf-router-af)#end

PE1
Ser2/0

CE1

Loop0:10.13.1.61/32

AS#1 MPLS Backbone

PE1#sh ip bgp vpn vrf v1 label | i 200.1.61.4 200.1.61.4/30 0.0.0.0 30/nolabel PE1# PE1#sh mpls forwarding vrf v1 | i 200.1.61.4 30 Aggregate 200.1.61.4/30[V] 0 PE1#

As soon as BGP gets the VPN prefix, it allocates the local label, and installs the prefix+label in both BGP and LFIB

MPLS Training - Basic

288

MPLS VPN Ctrl PlaneTrouble #2


#2: LFIB doesnt have any label for the VPNv4 prefix at the local PE, though BGP now does. TIP: clear ip route vrf <vrf> <prefix>
If the above doesnt fix, then (soft) reset the BGP session

MPLS Training - Basic

289

MPLS VPN Ctrl PlaneTrouble #3


#3: Remote PE (PE2) doesnt get the VPNv4 prefix from PE1 ip vrf v1
PE1
Ser2/0 Loop0:10.13.1.61/32

rd 1:1 route-target import 1:1

RR1

CE1

PE2#sh ip bgp vpn vrf v1 200.1.61.4 % Network not in the table PE2# PE2#sh ip vrf de v1 | beg Import No Import VPN route-target communities No import route-map No export route-map PE2#

AS#1 MPLS Backbone

PE2
Loop0:10.13.1.62/32

CE-2

TIP: Validate route-target import config at PE2. If not present, then configure it; Check for import-map as well

MPLS Training - Basic

290

MPLS VPN Ctrl PlaneTrouble #4


#4: Remote PE (PE2) still doesnt get the VPNv4 prefix from PE1
RR1
! ip vrf v1 rd 1:1 route-target import 1:1

PE1
Ser2/0 Loop0:10.13.1.61/32

CE1

PE2#sh ip bgp vpn vrf v1 200.1.61.4 % Network not in the table PE2#

AS#1 MPLS Backbone

PE2
Loop0:10.13.1.62/32

CE-2

We already fixed PE2; so lets go to PE1 Validate Route-target export in the VRF at the PE1

MPLS Training - Basic

291

MPLS VPN Ctrl PlaneTrouble #4 (Cont.)


PE1(conf)#ip vrf v1 PE1(conf-vrf)#route-target export 1:1 PE1#sh ip bgp vpnv4 vpn vrf v1 200.1.61.4 BGP routing table entry for 1:1:200.1.61.4/30, version 10 PE1(conf-vrf)#end
Paths: (2 available, best #2, table v1) Advertised to non peer-group peers: 10.13.1.21 200.1.61.6 Local 0.0.0.0 from 0.0.0.0 (10.13.1.61) Origin incomplete, metric 0, localpref 100, weight RR1 32768, valid, sourced, best PE2 PE1#

Ooops..RT Is Missing
Ser2/0

PE1

Loop0:10.13.1.61/32

CE1

AS#1 MPLS Backbone

Loop0:10.13.1.62/32

CE-2

TIP: Configure Route-target export in the VRF on the local PE i.e. PE1 Lets make sure that RT is getting tagged to the VPNv4 prefix

MPLS Training - Basic

292

MPLS VPN Ctrl PlaneTrouble #4 (Cont.) AS#1 MPLS CE1 Backbone RT is getting tagged
RR1 PE1#sh ip bgp vpnv4 vpn vrf v1 200.1.61.4 PE2 BGP routing table entry for 1:1:200.1.61.4/30, version 10 Paths: (2 available, best #2, table v1) Ser2/0 Advertised to non peer-group peers: 10.13.1.21 200.1.61.6 Loop0:10.13.1.61/32 Local 0.0.0.0 from 0.0.0.0 (10.13.1.61) Origin incomplete, metric 0, localpref 100, weight 32768, valid, sourced, best Extended Community: RT:1:1 PE1# PE1

Extra-TIP If export or import map are also configured, then check the RT in set clause, along with the match clause

MPLS Training - Basic

293

MPLS VPN Ctrl PlaneTrouble#5


#5: Remote PE (PE2) STILL doesnt get the VPNv4 prefix from PE1
RR1 PE1
RR1#sh ip bgp vpnv4 rd 1:1 200.1.61.4 BGP routing table entry for 1:1:200.1.61.4/30, version 14 Paths: (1 available, best #1, no table) PE2 Advertised to non peer-group peers: 10.13.1.62 CE-2 Local, (Received from a RR-client) Loop0:10.13.1.62/32 10.13.1.61 (metric 75) from 10.13.1.61 (10.13.1.61) Origin incomplete, metric 0, localpref 100, valid, internal, best Extended Community: RT:1:1 RR1#

Loop0:10.13.1.61/32

CE1

AS#1 MPLS Backbone

RR1 is indeed receiving the prefix from PE1 Make sure that RR is configured with neighbor <PE2> send-community extended under vpnv4 address-family

MPLS Training - Basic

294

MPLS VPN Ctrl PlaneTrouble #5 (Cont.)


Ooops. PE2 i.e 10.13.1.62 Is Missing
RR1(conf)#router bgp 1 RR1(conf-router)#address-family vpnv4 RR1(conf-router-af)#neighbor 10.13.1.62 sendcommunity extended RR1(conf-router-af)#neighbor 10.13.1.62 routereflector-client PE1 RR1(conf-router-af)#end
Ser2/0 Loop0:10.13.1.61/32

RR1#sh run | inc send-community ext neighbor 10.13.1.61 send-community extended PE1# RR1#sh run | inc send-community ext neighbor 10.13.1.61 send-community extended neighbor 10.13.1.62 send-community extended PE1# RR1

CE1

AS#1 MPLS Backbone

PE2

Loop0:10.13.1.62/32

CE-2

TIP: All the MP-BGP peers must be configured with send-community extended|both Also make sure that PE1 and PE2 are configured as route-reflector-client under vpnv4 af at the RR1

MPLS Training - Basic

295

MPLS VPN Control PlaneTrouble #6 #6: Remote PE (PE2) STILL doesnt get the VPNv4 prefix from PE1 AS#1 MPLS Backbone

PE2#sh ip vrf detail v1 | i Import Import route-map: raj-import PE2# PE2#sh route-map raj-import RR1 route-map raj-import, permit, sequence 10 PE1 PE2 Match clauses: extcommunity (extcommunity-list filter):1 Ser2/0 Set clauses: Loop0:10.13.1.62/32 Policy 200.1.61.4/30matches: 0 packets, 0 bytes routing Loop0:10.13.1.61/32 PE2# PE2#sh ip extcommunity-list 1 CE1 Extended community standard list 1 PE2#sh ip bgp vpn vrf v1 200.1.61.4 deny RT:1:1 % Network not in the table deny RT:2:2 PE2# PE2#

CE-2

Thats ok. Lets Remove RT 1:1 from the Filter.

Hmm we have already verified PE1 and RR1; something must be missing on PE2 then Lets check for any import-map at PE2 again

MPLS Training - Basic

296

MPLS VPN Control PlaneTrouble #6 (Cont.)


PE2(conf)#no ip extcommunity-list 1 deny rt 1:1 RR1 PE2(conf)#end

PE1

Ser2/0 Loop0:10.13.1.61/32

CE1

PE#clear ip bgp * vpnv4 unicast in PE2#sh ip bgp vpnv4 vrf v1 200.1.61.4 BGP routing table entry for 1:1:200.1.61.4/30, version 180 Paths: (1 available, best #1, table v1) Advertised to non peer-group peers: 200.1.62.6 Local 10.13.1.61 (metric 75) from 10.13.1.21 (10.13.1.21) Origin incomplete, metric 0, localpref 100, valid, internal, best Extended Community: RT:1:1 Originator: 10.13.1.61, Cluster list: 10.13.1.21 PE2#

AS#1 MPLS Backbone

PE2
Loop0:10.13.1.62/32

CE-2

TIP: If import-map is configured within the VRF, then import route-target <rt> must be configured within the VRF for the relevant RT

MPLS Training - Basic

297

MPLS VPN Control PlaneTrouble #7 #7: Label mismatch between BGP and FIB
PE2#sh ip bgp vpnv4 vrf v1 labels | i 200.1.61.4 200.1.61.4/30 10.13.1.61 nolabel/25 RR1 PE2# PE1 PE2#sh ip cef vrf v1 200.1.61.4 200.1.61.4/30, version 64, epoch 0, cached adjacency to Serial2/0 Ser2/0 0 packets, 0 bytes tag information set local tag: VPN-route-head Loop0:10.13.1.61/32 fast tag rewrite with Se2/0, point2point, tags imposed: {2003 20} via 10.13.1.61, 0 dependencies, recursive CE1 next hop 10.13.2.5, Serial2/0 via 10.13.1.61/32 valid cached adjacency tag rewrite with Se2/0, point2point, tags imposed: {2003 20} PE2#

AS#1 MPLS Backbone

PE2
Loop0:10.13.1.62/32

CE-2

Fix: clear ip route vrf <vrf> <prefix>. If the mismatch doesnt go away, then debug ip bgp vpn and debug mpls lfib cef to dig in.

MPLS Training - Basic

298

MPLS VPN Control PlaneTrouble #8 #8: Remote PE receives the route, but remote CE doesnt
PE1
router bgp 1 ! address-family ipv4 vrf v1 neighbor 200.1.62.6 as-override exit-address-family !

AS#65000
CE1

Ser2/0

Loop0:10.13.1.61/32

Loop0:5.5.5.5/32

AS#1 MPLS Backbone

PE2

CE-2 Loop0:10.13.1.62/32

AS#65000

TIP: If eBGP on PE-CE and VPN sites use the same ASN, then configure as-override on the BGP VRF af on both PEs If IGP on PE-CE, then validate BGP->IGP redistribution (within IGP VRF) on the PE

MPLS Training - Basic

299

MPLS VPN Control Plane Show Commands on PE


1. sh ip bgp vpn all summary
Analogous to sh ip bgp summary; Lists all the MPBGP and CE peers

2. 3. 4. 5.

sh ip bgp vpn all


Lists all the VPN prefixes advertised/rcvd by the router

sh ip bgp vpn vrf <vrf> summary


Similar to the first one, but for a specific VRF

sh ip bgp vpn vrf <vrf>


Lists all the VPN prefixes received in a specific VRF

sh ip bgp vpn vrf <vrf> labels


List labels for the VPN prefixes in a VRF

MPLS Training - Basic

300

MPLS VPN Control Plane Show Commands on PE


If OSPF on PE-CE sh ip ospf neighbors
Lists both VPN(s) and non-VPN(s) OSPF neighbors

sh ip ospf <process-id>
Select the VRF associated process-id to see relevant OSPF info (a lot of info)

sh ip ospf <process-id> database


Select the VRF associated process-id to see the OSPF database for that VRF

clear ip ospf <process-id>


Clear OSPF neighbors in the VRF if VRF associated process-id is chosen

MPLS Training - Basic

301

MPLS VPN Control Plane Show Commands on PE


If EIGRP on PE-CE sh ip eigrp vrf <vrf> topology
Lists VRF specific EIGRP topology

sh ip eigrp vrf <vrf> neighbor|interface


Lists EIGRP neighbors or interfaces in the VRF

sh ip eigrp vrf <vrf> events


Shows VRF specific EIGRP events

clear ip eigrp vrf <vrf> neighbors


Clears VRF specific EIGRP neighbors

MPLS Training - Basic

302

MPLS VPN Control Plane Clear Commands on PE

Relevant towards RR (or remote PE) peers: clear ip bgp * vpnv4 unicast in
Route-refresh request is sent to all the MP-BGP peers

clear ip bgp <MP-BGP peer> vpnv4 unicast in


Route-refresh request is sent to a specific MP-BGP peer

MPLS Training - Basic

303

MPLS VPN Control Plane Clear Commands on PE


Relevant towards CEs:
clear ip bgp * vrf < vrf >
Clear all PE-CE eBGP sessions in that vrf

clear ip bgp * vrf <vrf> in


Route-refresh message is sent to all the CEs in that vrf

clear ip bgp * vrf < vrf > out


Send respective VPN routes to all the CEs in that vrf

clear ip bgp <CE> vrf < vrf > soft in|out


soft reset of BGP session

MPLS Training - Basic

304

MPLS VPN Control Plane Show Commands on RR


Route-reflector know nothing about VRF
Following commands come quite handy (especially on RR)

1. sh ip bgp vpn all 2. sh ip bgp vpn rd <RD>


Lists all VPNv4 prefixes that have RD in them

3. sh ip bgp vpn rd <RD> label


Lists labels for VPNv4 prefixes that have RD

MPLS Training - Basic

305

MPLS VPN Control Plane Debugs on PE


Be Careful on the Production Routers
1. debug ip bgp vpnv4
Useful while troubleshooting label related problems in BGP (could spit a lot of output)

2. 3.

debug mpls lfib cef [acl]


Useful troubleshooting label mismatch in FIB/LFIB

debug ip bgp vpnv4 import


Useful when VPN prefixes dont get imported in the VRF table (could spit a lot of output)

4.

debug ip routing vrf <vrf> [acl]


Useful when VPN prefixes dont get installed in the VRF routing table

MPLS Training - Basic

306

Agenda
Control Plane
Control Plane Troubleshooting Tips Real-life Examples Summary of Helpful Cisco IOS Commands

Forwarding Plane
Dissecting LFIB Loadsharing in MPLS VPN Networks Forwarding Plane Troubleshooting Tips Real-life Examples Summary of Helpful Cisco IOS Commands

Conclusion
MPLS Training - Basic 307

MPLS VPN Forwarding Plane Dissecting LFIB: show mpls forward


IP (or IGP) Prefix in the LFIB
RSP-PE-WEST-4#sh mpls forward 10.13.1.11 detail Local Outgoing Prefix Bytes tag Outgoing tag tag or VC or Tunnel Id switched interface 45 51 10.13.1.11/32 0 Fa1/1/1 MAC/Encaps=14/18, MRU=1500, Tag Stack{51} 0003FD1C828100044E7548298847 00033000 No output feature configured Per-packet load-sharing RSP-PE-WEST-4#

Next Hop 10.13.7.33

Only One Outgoing Label in the Label Stack

MRUMax Receivable Unit; The Received Packet Will Be Transmitted Unfragmented on Fa1/1/1, If Received Packets Size Is Not More Than 1500B

MAC header = 0003FD1C828100044E754829 MPLS Ethertype = 0x8847 Label = 0x00033000 = 51 0x00033000 = EXP+S 0x00033000 = MPLS TTL

Although MAC Header Is of 14 Bytes, Actual Encapsulation I.E. MAC+MPLS Header Is of 18 Bytes (One Label Is 4 Bytes)

MPLS Training - Basic

308

MPLS VPN Forwarding Plane Dissecting LFIB: show mpls forward (Cont.)
VPN Prefix in the LFIB
PE1#sh Local tag 27 mpls for vrf v1 5.5.5.5 detail Outgoing Prefix Bytes tag tag or VC or Tunnel Id switched Untagged 5.5.5.5/32[V] 0 MAC/Encaps=0/0, MRU=1504, Tag Stack{} VPN route: v1 No output feature configured Per-packet load-sharing Outgoing interface Se2/0
PE1 P1 PE2 CE1 5.5.5.5/32

Next Hop point2point

Se2/0 Is a PE-CE Interface which Is under VRF v1

PE1#

Only 1504 Byte Size Packet Can Be Received because 15044 (for One Label 27) = 1500 Is the MTU Size of Se2/0 MAC/Encaps Field Corresponds to the tag adj, and because the VRF Interface Doesnt Typically Have MPLS Enabled, tag adj Is 0; hence, 0/0 Output

MPLS Training - Basic

309

Agenda
Control Plane
Control Plane Troubleshooting Tips Real-life Examples Summary of Helpful Cisco IOS Commands

Forwarding Plane
Dissecting LFIB Load sharing in MPLS VPN Networks Forwarding Plane Troubleshooting Tips Real-life Examples Summary of Helpful Cisco IOS Commands

Conclusion
MPLS Training - Basic 310

MPLS VPN Fwd PlaneLoadsharing


Loadsharing in MPLS VPN network is same as that of the IP network
i.e. FIB per-destination loadsharing

IP src and dest addresses inside the MPLS packet are hashed to find the right LSP

Lets Go through PE-P and P-P Loadsharing


MPLS Training - Basic 311

MPLS VPN Fwd PlaneLoadsharing (I)


PE-P Loadsharing (Cont.)
PE1#sh ip cef vrf v1 200.1.62.4 200.1.62.4/30, version 13, epoch 0, per-destination sharing 0 packets, 0 bytes tag information set local tag: VPN-route-head fast tag rewrite with Recursive rewrite via 10.13.1.62/32, tags imposed {25} via 10.13.1.62, 0 dependencies, recursive next hop 10.13.1.9, Ethernet1/0 via 10.13.1.62/32 valid adjacency tag rewrite with Recursive rewrite via 10.13.1.62/32, tags imposed {25} Recursive load sharing using 10.13.1.62/32. PE1# E0/0 PE1

E1/0

P1 Se2/0 Loop0:10.13.1.62/32 PE2

Because There Are Loadshared Paths to the Egress PE i.e. 10.13.1.62/32

Only VPN Label Is Shown

Dont panicIGP label is chosen during the forwarding (depending on the hash-bucket)
MPLS Training - Basic 312

MPLS VPN Fwd PlaneLoadsharing (I)


PE-P Loadsharing (Cont.)
PE1#sh ip cef 10.13.1.62 10.13.1.62/32, version 30, epoch 0, per-destination sharing 0 packets, 0 bytes tag information set, shared local tag: 18 via 10.13.1.5, Ethernet0/0, 1 dependency traffic share 1 next hop 10.13.1.5, Ethernet0/0 valid adjacency tag rewrite with Et0/0, 10.13.1.5, tags imposed: {2001} via 10.13.1.9, Ethernet1/0, 1 dependency traffic share 1 next hop 10.13.1.9, Ethernet1/0 valid adjacency tag rewrite with Et1/0, 10.13.1.9, tags imposed: {2001} 0 packets, 0 bytes switched through the prefix tmstats: external 0 packets, 0 bytes internal 0 packets, 0 bytes PE1# E0/0 CE1 PE1

E1/0

P1 Se2/0 Loop0:10.13.1.62/32 CE2 PE2

IGP Label Is Right Here

IGP Label and the outgoing interface are derived after the hash-bucket is decided
MPLS Training - Basic 313

MPLS VPN Fwd PlaneLoadsharing (I)


PE-P Loadsharing (cont.)
PE1#sh ip cef vrf v1 exact-route 30.1.61.4 200.1.62.4 internal 30.1.61.4 -> 200.1.62.4 : Ethernet1/0 (next hop 10.13.1.9) Bucket 7 from 16, total 2 paths E0/0 CE1 PE1

E1/0

P1 Se2/0 Loop0:10.13.1.62/32 CE2 PE2

In summary, the show-output in load-sharing case gets bit tricky; but the fundamental is the same

MPLS Training - Basic

314

MPLS Fwd PlaneLoadsharing (II)


P-P Loadsharing
P1#sh mpls for 10.13.1.62 Local Outgoing Prefix tag tag or VC or Tunnel Id 52 21 10.13.1.62/32 27 10.13.1.62/32 P1# 0 0 Bytes tag switched Outgoing interface Eth0/0 Eth1/0 Next Hop point2point point2point E0/0 P2 PE1

P1 E1/0 P3

For VPN traffic, P router hashes the IP src+dest to apply the packet to the correct hash bucket sh ip cef exact-route command cant be used on the P router since it doesnt know the VPN addresses

Se2/0 Loop0:10.13.1.62/32 PE2

Hence, rely on (LFIB) counters to make sure the traffic is getting loadshared

MPLS Training - Basic

315

MPLS VPN Fwd PlaneMPLS TTL

Just like TTL in the IP header, MPLS header also has a 1-byte TTL field When an IP packet is first labelled at the ingress, the (IP TTL -1) is copied to the MPLS TTL Later, when the label is popped/disposed, the MPLS TTL value of the removed label is copied to the either MPLS TTL of inner label or IP TTL field (if no inner label), provided MPLS TTL < IP TTL.

Otherwise IP TTL used and decremented

MPLS Training - Basic

316

Agenda MPLSVPN Troubleshooting


Control Plane
Control Plane Troubleshooting Tips Real-life Examples Summary of Helpful Cisco IOS Commands

Forwarding Plane
Dissecting LFIB Loadsharing in MPLS VPN Networks Forwarding Plane Troubleshooting Tips Real-life Examples Summary of Helpful Cisco IOS Commands

Conclusion
MPLS Training - Basic 317

MPLS VPN Forwarding Plane Troubleshooting Tips


Symptom Tip Cisco IOS Command
PE#ping vrf <vrf> <remote prefix>

CE CE Traffic Fails

Verify That the PE PE VPN Traffic Can Pass Using vrf Pings (Assuming the Control Plane Information Has Already Been Verified*)

PE PE MPLS Traffic Fails PE PE IP Traffic Passes, but MPLS Traffic Fails Incoming MPLS Traffic Is Dropped at the Egress PE

Validate the PE->PE IP Connectivity; and then Check for the LSP Find out where Exactly the LSP Is Broken

PE#ping <remotePE>

PE#ping mpls ipv4 <remotePE> **

Check the LFIB Entries on Both RP, LC (and Relevant HW Engines, if Present)

PE#sh mpls for | i <prefix>

MPLS Training - Basic

318

MPLS VPN Forwarding Plane Troubleshooting Tips (Cont.)


Symptom Tip Cisco IOS Command
Router#sh mpls int de | in MTU

CE CE Traffic Fails for Certain MTU Sizes

Check the MPLS MTU Size of the MPLS Enabled Interfaces and Make Sure It Is More than the Reported Failed MTU Size

CE CE Traffic Fails for Certain MTU Sizes

Verify that the Ethernet Switch Ports inside the MPLS Core Is Enabled with Baby Giant Support

Switch#sh port jumbo

*Please See the Troubleshooting Control Plane TechTalk; **12.0(26)S Onwards


MPLS Training - Basic 319

MPLS VPN Fwd Plane - Troubleshooting


Problem: A VPN Site Cant Reach Other Sites Cause: The CE CE Traffic Is Getting Dropped Somewhere
PE1
200.1.61.4/30

P1
E0/0 E1/0 Ser2/0

PE2
200.1.62.4/30

Loop0:10.13.1.61/32

CE1

MPLS Loop0:10.13.1.62/32 Backbone

CE2

Tip 1: Check the control plane information first

FI B

PE1#sh ip cef vrf v1 200.1.62.4; PE2#sh ip cef vrf v1 200.1.61.4;

LF PE1#sh mpls for vrf v1 | inc 200.1.61.4 IB PE2#sh mpls for vrf v1 | inc 200.1.62.4

Make sure that the label information is correct

Turn on deb ip icmp on both PEs Step 1: Issue ping vrf v1 <remote_PE-CE_address> on both PEs Step 2: If they pass, then we have verified that the problem is not in the MPLS core

MPLS Training - Basic

320

MPLS VPN Fwd Plane - Troubleshooting


200.1.61.4/30 Ser2/0

PE1

E0/0 E1/0

P1
Ser2/0

PE2

200.1.62.4/30

CE1
5.5.5.5/32

Loop0:10.13.1.61/32

MPLS Loop0:10.13.1.62/32 CE-2 6.6.6.6/32 Backbone 2


PE2#sh mpls for vrf v1 | inc 200.1.62.4 25 Aggregate 200.1.62.4/30[V] 0 PE2# PE1#sh ip cef 10.13.1.62 10.13.1.62/32, version 56, epoch 0, per-destination sharing 0 packets, 0 bytes tag information set local tag: 18 via 10.13.1.5, Ethernet0/0, 1 dependency traffic share 1 next hop 10.13.1.5, Ethernet0/0 valid adjacency tag rewrite with Et0/0, 10.13.1.5, tags imposed: {2001} via 10.13.1.9, Ethernet1/0, 2 dependencies traffic share 1 next hop 10.13.1.9, Ethernet1/0 valid adjacency tag rewrite with Et1/0, 10.13.1.9, tags imposed: {2001} 0 packets, 0 bytes switched through the prefix PE1#
321

PE1#sh ip cef vrf v1 200.1.62.4 200.1.62.4/30, version 10, epoch 0, per-destination sharing 0 packets, 0 bytes tag information set local tag: VPN-route-head fast tag rewrite with Recursive rewrite via 10.13.1.62/32, tags imposed {25} via 10.13.1.62, 0 dependencies, recursive next hop 10.13.1.9, Ethernet1/0 via 10.13.1.62/32 valid adjacency tag rewrite with Recursive rewrite via 10.13.1.62/32, tags imposed {25} Recursive load sharing using 10.13.1.62/32. PE1#

Validated the Labels in PE1->PE2 Direction


MPLS Training - Basic

MPLS VPN Fwd Plane - Troubleshooting


200.1.61.4/30 Ser2/0

PE1

E0/0 E1/0

P1
Ser2/0

PE2

200.1.62.4/30

CE1
5.5.5.5/32

Loop0:10.13.1.61/32

MPLS Loop0:10.13.1.62/32 CE-2 6.6.6.6/32 Backbone 1


PE2#sh ip cef vrf v1 200.1.61.4 200.1.61.4/30, version 73, epoch 0, cached adjacency to Serial2/0 0 packets, 0 bytes tag information set local tag: VPN-route-head fast tag rewrite with Se2/0, point2point, tags imposed: {2003 28} via 10.13.1.61, 0 dependencies, recursive next hop 10.13.2.5, Serial2/0 via 10.13.1.61/32 valid cached adjacency tag rewrite with Se2/0, point2point, tags imposed: {2003 28} PE2#

PE1#sh mpls for vrf v1 | i 200.1.61.4 28 Aggregate 200.1.61.4/30[V] 0 PE1#

Validated the Labels in PE2 PE1 Direction


MPLS Training - Basic 322

MPLS VPN Fwd Plane - Troubleshooting


200.1.61.4/30 Ser2/0

PE1

E0/0 E1/0

P1
Ser2/0

PE2

200.1.62.4/30

CE1
5.5.5.5/32

Loop0:10.13.1.61/32

MPLS Loop0:10.13.1.62/32 CE-2 6.6.6.6/32 Backbone 1 2


PE2#deb ip icmp ICMP packet debugging is on PE2# PE2# *May 11 00:42:16.353: ICMP: echo reply sent, src 200.1.62.5, dst 200.1.61.5 *May 11 00:42:16.473: ICMP: echo reply sent, src 200.1.62.5, dst 200.1.61.5 *May 11 00:42:16.581: ICMP: echo reply sent, src 200.1.62.5, dst 200.1.61.5 *May 11 00:42:16.701: ICMP: echo reply sent, src 200.1.62.5, dst 200.1.61.5 *May 11 00:42:16.813: ICMP: echo reply sent, src 200.1.62.5, dst 200.1.61.5 PE2#

PE1#deb ip icmp ICMP packet debugging is on PE1# Step 1 PE1#ping vrf v1 200.1.62.5 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 200.1.61.5, timeout is 2 seconds: ..... Success rate is 0 percent (0/5) PE1#

Step 3: Okalthough the vrf pings failed at PE1, ICMP debugs at PE2 confirms that PE1->PE2 LSP is error free Lets ping in the opposite direction to check the PE2 PE1 LSP
MPLS Training - Basic 323

MPLS VPN Fwd Plane - Troubleshooting


200.1.61.4/30 Ser2/0

PE1

E0/0 E1/0

P1
Ser2/0

PE2

200.1.62.4/30

CE1
5.5.5.5/32

Loop0:10.13.1.61/32

MPLS Loop0:10.13.1.62/32 CE-2 6.6.6.6/32 Backbone 1 2


PE2#deb ip icmp ICMP packet debugging is on PE2# PE2#ping vrf v1 200.1.61.5 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 200.1.61.5, timeout is 2 seconds: ..... Success rate is 0 percent (0/5) PE2#

PE1#deb ip icmp ICMP packet debugging is on PE1# PE1# PE1#

Since PE1 didnt get/show any ICMP echos for the vrf pings
a) Either PE2 PE1 LSP is broken

b) PE1 is dropping the received MPLS packets for some reason

Okso lets troubleshoot for (a) first

MPLS Training - Basic

324

MPLS VPN Fwd Plane - Troubleshooting


200.1.61.4/30 Ser2/0

PE1

E0/0 E1/0

P1
Ser2/0

PE2

200.1.62.4/30

CE1
5.5.5.5/32

Loop0:10.13.1.61/32

MPLS Loop0:10.13.1.62/32 CE-2 6.6.6.6/32 Backbone 3 2


PE2#ping 10.13.1.61 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.13.1.61, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 28/52/72 ms PE2#

Step 4 PE1#ping 10.13.1.62 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.13.1.62, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 40/57/92 ms PE1#

P1#sh mpls forward Local Outgoing tag tag or VC 2003 Untagged Untagged P1#

10.13.1.61 Prefix or Tunnel Id 10.13.1.61/32 10.13.1.61/32

Bytes tag switched 0 0

Outgoing interface Et0/0 Et1/0

Next Hop 10.13.1.6 10.13.1.10

Step 5: IP reachability is confirmed between PE1 and PE2 (steps 1 and 2); GOOD; but that doesnt validate the LSP in both directions Step 6: Per P1s LFIB, it doesnt have the right label to reach PE1 (untagged vs. Pop).
MPLS Training - Basic 325

MPLS VPN Fwd Plane - Troubleshooting


Rememberuntagged outgoing label means that get rid of the label stack; hence, the VPN label would be lost at P1 untagged label for /32 routes inside the MPLS core is almost always bad To fix this untagged problem in LFIB,
Check whether LIB and LFIB are in-sync about this entry. If not, then clear ip route 10.13.1.61 on P1 If yes, then flap the LDP neighbor by clear mpls ldp neighbor 10.13.1.61 on P1 to relearn the correct binding
If you love to dig further, then deb mpls lfib cef, deb mpls ldp bind ** etc.
** See more Debugs at the Show commands section
MPLS Training - Basic 326

MPLS VPN Fwd Plane - Troubleshooting


LSP pings* can DETECT the broken LSPs LSP traceroute* can PIN-POINT the culprit router where the LSP could be broken
But we will still have to fix the LSP

*12.0(26)S Onwards
MPLS Training - Basic 327

MPLS VPN Forwarding Plane -Trouble #1 (Cont.)


PE1#ping mpls ipv4 10.13.1.62/32 Sending 5, 100-byte MPLS Echos to 10.13.1.62/32, timeout is 2 seconds, send interval is 0 msec: Codes: '!' - success, 'Q' - request not transmitted, '.' - timeout, 'U' - unreachable, 'R' - downstream router but not target

LSP Ping Failed

Type escape sequence to abort. RRRRR Success rate is 0 percent (0/5) PE1#

PE1#ping mpls ipv4 10.13.1.62/32 Sending 5, 100-byte MPLS Echos to 10.13.1.62/32, timeout is 2 seconds, send interval is 0 msec: Codes: '!' - success, 'Q' - request not transmitted, '.' - timeout, 'U' - unreachable, 'R' - downstream router but not target

LSP Ping Succeeded

Type escape sequence to abort. !!!!! Success rate is 0 percent (0/5) PE1#

MPLS Training - Basic

328

MPLS VPN Forwarding Plane -Trouble #1 (Cont.)


LSP traceroute* is capable of differentiating between untagged and pop/null
PE1#trace mpls ipv4 10.13.1.62/32 Tracing MPLS Label Switched Path to 10.13.1.62/32, timeout is 2 seconds Codes: '!' - success, 'Q' - request not transmitted, '.' - timeout, 'U' - unreachable, 'R' - downstream router but not target Type escape sequence to abort. 0 10.13.1.10 MRU 1500 [Labels: 2002 Exp: 0] R 1 10.13.2.14 MRU 1204 [No Label] 52 ms ! 2 10.13.2.14 52 ms PE1# PE1#trace mpls ipv4 10.13.1.62/32 Tracing MPLS Label Switched Path to 10.13.1.62/32, timeout is 2 seconds Codes: '!' - success, 'Q' - request not transmitted, '.' - timeout, 'U' - unreachable, 'R' - downstream router but not target Type escape sequence to abort. 0 10.13.1.10 MRU 1500 [Labels: 2002 Exp: 0] R 1 10.13.2.13 MRU 1512 [implicit-null] 40 ms ! 2 10.13.2.14 68 ms PE1#

*12.0(26)S Onwards
MPLS Training - Basic 329

Agenda MPLSVPN Troubleshooting


Control Plane
Control Plane Troubleshooting Tips Real-life Examples Summary of Helpful Cisco IOS Commands

Forwarding Plane
Dissecting LFIB Loadsharing in MPLS VPN Networks Forwarding Plane Troubleshooting Tips Real-life Examples Summary of Helpful Cisco IOS Commands

Conclusion
MPLS Training - Basic 330

MPLS VPN Fwd PlaneShow Commands


sh mpls forwarding
Shows all LFIB entries (vpn, non-vpn, TE etc.)

sh mpls forwarding | inc <prefix>


Whether the prefix is present in the LFIB or not

sh mpls forwarding vrf <vrf> <prefix>


LFIB lookup based on a VPN prefix

sh mpls forwarding label <label>


LFIB lookup based on an incoming label

MPLS Training - Basic

331

MPLS VPN Fwd PlaneShow Commands


sh ip arp vrf <vrf>
Lists ARP entries relevant to the <vrf> only

sh ip cef vrf <vrf > <prefix>


Displays the label stack, outgoing interface etc

sh mpls forwarding vrf <vrf>


Lists labels for the VPN prefixes learned from the CE(s)

MPLS Training - Basic

332

MPLS VPN Fwd PlaneDebugs


Be Careful on the Production Routers
debug arp
Useful for VPN prefixes as well

debug mpls lfib cef [acl]


Useful when VPN prefixes have label mismatch among BGP, FIB and LFIB.

MPLS Training - Basic

333

Conclusion
MPLS seems cryptic, but it is not Whether to look at FIB or LFIB? Whether it is a BGP or MPLS problem? Whether the problem is within the core or outside the core? Ongoing MPLS OAM work .

MPLS Training - Basic

334

MPLS Training - Basic

335

335