BGP Troubleshooting Guide V1

Redback Networks
Customer Engineering
BGP Troubleshooting Guide
Issue Date: 2009-Feb-03 8:58 PM
Version: 5
Document ID: CE-TG-0009
Created by: Jimmy Wong on: 1/14/2009

Updated by: etxjiwo on: 2/3/2009 08:58:00 PM
Proprietary and Confidential
 Copyright 2009 Redback Networks Inc.

All rights reserved. No part of this publication may be used or
reproduced in any form by any means without prior written
permission from Redback Networks Inc.
CE-TG-0009
Document Information
Document Name BGP Trouble-shooting Guide
Redback Document ID CE-TG-0009
Version 1.0
Issue Date 2009-Feb-03 8:58 PM
Document Acceptance
I have reviewed this document and it is approved for use effective from the signing date below.
Name Position Signature Date
Revision History
Issue Author(s) Date Description

1.0 Jimmy Wong 1/14/2009 Initial Draft
Created by: Jimmy Wong on: 1/14/2009 Page 2 of 12

Updated by: jimmyw on: 2/3/2009 08:58:00 PM /conversion/tmp/activity_task_scratch/647234912.doc
Copyright 2009 Redback Networks Inc. All rights reserved.
Proprietary and Confidential.
No part of this publication may be used or reproduced in any form by any means without prior written permission from Redback Networks Inc.
CE-TG-0009
Table of Contents
1 Introduction............................................................................................................4
2 Topology...................................................................................................................4
3 Local Configuration problem............................................................................4
3.1 Peer Establishment................................................................................................................................ 4
3.1.1 Common problem.......................................................................................................................................................... 4
3.1.2 Peer establishment – iBGP......................................................................................................................................... 5
3.1.3 Peer establishment – eBGP........................................................................................................................................ 7
3.1.4 Peer Establishment - Password................................................................................................................................ 9
3.2 Missing route......................................................................................................................................... 11
3.2.1 Routing Origination.................................................................................................................................................... 11

CE-TG-0009
1 Introduction
This document is intended to provide for troubleshooting steps to be taken for debugging BGP related
issues. This document is supposed to be generic troubleshooting guide, for specific feature related
information other resources should be considered.
2 Topology
3 Local Configuration problem

 Peer Establishment
 Missing routes
3.1 Peer Establishment
 Router establish a TCP session

- Port 179 – Permit in interface filters
- IP connectivity (route from IGP)
 OPEN messages are exchanged
- Peering addresses must match the TCP session
- Local AS configuration parameters
3.1.1 Common problem
 Session are not established

- No IP reachability

CE-TG-0009
- Incorrect configuration
 Peers are flapping
- Layer 2 problem
3.1.2 Peer establishment – iBGP

[local]R3#show config bgp
router-id 11.0.0.3
context local
!
router bgp 100
!
neighbor 11.0.0.2 internal
address-family ipv4 unicast
!
neighbor 11.0.0.6 external
remote-as 200
end
Symptoms
[local]R3#show tcp
Active Internet connections
PCB Recv-Q Send-Q Local Address Foreign Address State
f9abed8 0 0 11.2.3.3.62071 11.0.0.2.179 SYN_SENT
[local]R3#show bgp summary

Address Family: ipv4 unicast
BGP router identifier: 11.0.0.3, local AS number: 100
BGP route table version: 0, RIB table version: 0, deleted vers: 0
Neighbors Configured: 2, Established: 0
Sourced paths: redistributed: 0, networked: 0, aggregated: 0
Dampening: Disabled
Flap-statistics: Disabled
Entry Type Count Memory

Network 0 20
Path 0 0
Neighbor AS MsgRcvd MsgSent TblVer InQ OutQ Rst Up/Down PfxRcvd/Sent

11.0.0.2 100 0 0 0 0 0 0 3d03h Connect
11.0.0.6 200 0 0 0 0 0 0 3d03h Idle
[local]R3#
First check the route is on the routing table and then tries to ping it.
[local]R3#show ip route
Codes: C - connected, S - static, S dv - dvsr, R - RIP, e B - EBGP, i B - IBGP
O - OSPF, O3 - OSPFv3, IA - OSPF(v3) inter-area,
N1 - OSPF(v3) NSSA external type 1, N2 - OSPF(v3) NSSA external type 2
E1 - OSPF(v3) external type 1, E2 - OSPF(v3) external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, N - NAT
IPH - IP Host, SUB A - Subscriber address, SUB S - Subscriber static
M F - Mobile Sub Foreign Agent, M H - Mobile Sub Home Agent
A - Derived Default, MeH - Media Nexthop
> - Active Route, * - LSP
Type Network Next Hop Dist Metric UpTime Interface

> O 11.0.0.2/32 11.2.3.2 110 2 05:06:36 to-R2
> C 11.0.0.3/32 0 0 21:50:38 loopback0
> S 11.0.0.6/32 11.3.6.6 1 0 00:00:02 to-R6-1
> 11.33.66.66 to-R6-2
> C 11.2.3.0/24 0 0 05:11:25 to-R2
> C 11.3.6.0/24 0 0 21:22:47 to-R6-1
> C 11.33.66.0/24 0 0 21:22:47 to-R6-2
CE-TG-0009
[local]R3#ping 11.0.0.2
PING 11.0.0.2 (11.0.0.2): source 11.2.3.3, 36 data bytes,
timeout is 1 second
!!!!!
[local]R3#debug bgp session-state

Jan 23 23:06:07: [0001]: %BGP-7-SESSION: 11.0.0.2 active open: start
Jan 23 23:06:07: [0001]: %BGP-7-SESSION: 11.0.0.2 active open: local addr 11.2.3.3, if_grid 0x0
Jan 23 23:06:07: [0001]: %BGP-7-SESSION: 11.0.0.2 state changed from Idle to Connect
Jan 23 23:06:07: [0001]: %BGP-7-SESSION: 11.0.0.2 active open: try socket connect
Jan 23 23:06:07: [0001]: %BGP-7-SESSION: 11.0.0.2 active open: set pending fd 13 for select
 Source address is the outgoing interface towards the destination but peering in this case is using
loopback interfaces!
 Force both routers to source from the correct interface
 Use “update-source” to specify the loopback when loopback peering
The debug command show the BGP session is using the interface address to establish the peering. Let’s
change the config to this.
[local]R3#show config bgp

router-id 11.0.0.3
context local
!
router bgp 100
!
neighbor 11.0.0.2 internal
update-source loopback0
!
remote-as 200
end
[local]R3# debug bgp session-state

Jan 30 00:10:41: [0001]: %BGP-7-SESSION: 11.0.0.2 active open: local addr 11.0.0.3, if_grid 0x0
Jan 30 00:10:41: [0001]: %BGP-7-SESSION: 11.0.0.2 state changed from Idle to Connect
Jan 30 00:10:41: [0001]: %BGP-7-SESSION: 11.0.0.2 active open: try socket connect
Jan 30 00:10:41: [0001]: %BGP-7-SESSION: 11.0.0.2 active open: set pending fd 13 for select
Jan 30 00:10:41: [0001]: %BGP-7-SESSION: 11.0.0.2 active connect fd 13 selected, state Connect
Jan 30 00:10:41: [0001]: %BGP-7-SESSION: 11.0.0.2 post socket connect, fd 13, state Connect
Jan 30 00:10:41: [0001]: %BGP-7-SESSION: 11.0.0.2 post connect, moving actopen fd 13 to sockfd2
(sockfd -1)
Jan 30 00:10:41: [0001]: %BGP-7-SESSION: 11.0.0.2 state changed from Connect to OpenSent
Jan 30 00:10:41: [0001]: %BGP-7-SESSION: 11.0.0.2 state changed from OpenSent to OpenConfirm
Jan 30 00:10:41: [0001]: %BGP-7-SESSION: 11.0.0.2 state changed from OpenConfirm to Established
Jan 30 00:10:41: [0001]: %BGP-6-INFO: 11.0.0.2 UP
Here the IBGP is established.

[local]R3#sh bgp summary
BGP route table version: 0, RIB table version: 0, deleted vers: 0
Neighbors Configured: 2, Established: 1
Sourced paths: redistributed: 0, networked: 0, aggregated: 0
Dampening: Disabled
Flap-statistics: Disabled

CE-TG-0009
Entry Type Count Memory
Network 0 20
Path 0 0

11.0.0.2 100 22 22 0 0 0 2 00:16:23 0 0
11.0.0.6 200 0 0 0 0 0 0 05:31:04 Idle
Summary
 Assume that IP connectivity has been checked Including IGP reachability between peers
 Check TCP to find out what connections we are accepting
o Check the ports and source/destination addresses
o Do they match the configuration?
Common problem:
iBGP is run between loopback interfaces on router (for stability), but the configuration is missing from the
router ⇒ iBGP fails to establish
Remember that source address is the IP address of the outgoing interface unless otherwise specified.
3.1.3 Peer establishment – eBGP
 R2 is established now
 eBPG still having problem
 Trying to load-balance over multiple links to the eBGP peer

 Verify IP connectivity
o Check the routing table
o Use ping/trace to verify two way reachability

CE-TG-0009
[local]R3#ping 11.0.0.6
timeout is 1 second
!!!!!
----11.0.0.6 PING Statistics----
5 packets transmitted, 5 packets received, 0.0% packet loss
[local]R3#ping
Target IP address: 11.0.0.6
Extended commands [n]: y
Source IP address: 11.0.0.3
timeout is 1 second
.....
----11.0.0.6 PING Statistics----

5 packets transmitted, 0 packets received, 100.0% packet loss
Routing to R6 loopback is reachable but when using extended ping to test loopback to loopback
connectivity. It seems R3 does not have a route to loopback at R6.
[local]R3#show bgp neighbor 11.0.0.6

BGP neighbor: 11.0.0.6, remote AS: 200, external link
Version: 4, router identifier: 0.0.0.0
State: Idle for 23:34:53
Last read 23:34:53, last send 23:34:53
Hold time: configured 180, negotiated 0
Keepalive time: configured 60, negotiated 0
Local restart timer 120 sec, stale route retain timer 180 sec
Received restart timer 0 sec, flag 0x0
Number of hops external BGP neighbor may be away: 1
Minimum time between advertisement runs: 30 secs
Source IP address used from interface: loopback
Source (local) IP address: 0.0.0.0
Received messages: 0 (0 bytes), notifications: 0, in queue: 0
Sent messages: 0 (0 bytes), notifications: 0, out queue: 0
Last active open: 00:00:05, reason: no active or connected route
Address family: ipv4 unicast

BGP table version: 0, neighbor version: 0
Routes: rcvd 0, imported 0, active 0, history 0, dampend 0, sent 0
[local]R3#debug bgp neighbor 11.0.0.6 session-state

Jan 30 18:21:30: [0001]: %BGP-7-SESSION: 11.0.0.6 active open: no connected route
 eBGP peers are normally directly connected and the TTL is set to 1 for eBGP peers. If not directly
connected, specify ebgp-multihop
 At this point the when we add the ebgp-multihop at the config the session should come up

CE-TG-0009
[local]R3#
router bgp 100
remote-as 200
ebgp-multihop 2

11.0.0.6 200 0 0 0 0 0 1 00:02:42 Idle
Still not working but connectivity has already been check.

[local]R3#debug bgp event
Jan 30 18:47:17: [0001]: %BGP-6-INFO: 11.0.0.6 rcv NOTIFICATION: 2/2 (open: bad peer AS) with 4
byte data
Jan 30 18:47:17: [0001]: %BGP-7-EVENT: 11.0.0.6 completed reset
Summary
 If an error is detected, a notification is sent and the session is closed
 R6 is configured incorrectly – change from “remote 10” to “remote 100”
 After R6 makes this correction the session should come up
 Remember to allow TCP/179 through edge filters

11.0.0.6 200 3 3 0 0 0 3 00:00:36 0 0
Few things to remember for multihop eBGP

 Always check IP connectivity both local and remote routing tables
 Remember to source updates from loopback
 Watch for filters anywhere in the path
 TTL must be at least 2 for ebgp-multihop between directly connected neighbors
3.1.4 Peer Establishment - Password

When using passwords on iBGP and eBGP session. If you have been through all the previous
troubleshooting steps but the link still won’t come up.
11.0.0.6 200 0 0 0 0 0 4 00:00:25 Connect
Configuration on R3 looks ok. Check the log message.

[local]R3#
remote-as 200
password encrypted C029D7961F0801F0
ebgp-multihop 2
The log messages

Jan 30 21:02:45: %SYSLOG-6-INFO: /netbsd: TCP from 11.0.0.6(179) to 11.0.0.3(63180): no MD5
signature
Jan 30 21:02:45: %SYSLOG-6-INFO: /netbsd: TCP from 11.0.0.6(179) to 11.0.0.3(63180): bad MD5
digest
Jan 30 21:02:48: %SYSLOG-6-INFO: /netbsd: TCP from 11.0.0.6(179) to 11.0.0.3(63180): no MD5
signature

CE-TG-0009
digest
It seems the password is missing in R6. Let’s fix R6 configuration then the peering should come up.
[local]R6#
remote-as 100
password encrypted C029D7961F0801F0
ebgp-multihop 2
But it do not.
[local]R3#
digest
We are getting invalid MD5 digest messages – password mismatch!
We must have typo’ed the password on one of the peering routers. Fix the password – best to re-enter
password on both routers and eBGP session now comes up.
[local]R3#
Jan 30 21:19:08: [0001]: %BGP-6-INFO: 11.0.0.6 UP
Summary
 Common problems:
o Missing password – needs to be on both ends
o Cut and paste errors
o Typographical & transcription errors
o Capitalisation, extra characters, white space…
 Common solutions:
o Check for symptoms/messages in the logs
o Re-enter passwords using keyboard, from scratch – don’t cut&paste

CE-TG-0009
3.2 Missing route

 Once the session has been established, UPDATEs are exchanged
 All the locally known routes
 Only the bestpath is advertised
 Incremental UPDATE messages are exchanged afterwards
 Bestpath received from eBGP peer which will advertise to all peers
 Bestpath received from iBGP peer which will advertise only to eBGP peers
o A full iBGP mesh must exist (Unless we are using Route Reflectors)
3.2.1 Routing Origination

 Common problem occurs when putting prefixes into the BGP table
 BGP table is NOT the RIB
o (RIB = Routing Information Base – a.k.a the Routing Table)
o BGP table, as with OSPF table, ISIS table, static routes, etc, is used to feed the RIB, and
hence the FIB
o Each routing protocol has a different priority or “distance”
 To get a prefix into BGP, it must exist in another routing process too, typically:
o Static route pointing to customer (for customer routes into your iBGP)
o Static route pointing to Null (for aggregates you want to put into your eBGP)
Example 1
Network Statement
[local]R3#
router bgp 100
network 100.100.0.0/16
BGP is not originating the route???

[local]R3#sh bgp route | include 100.100.0.0
[local]R3#
Do we have the exact route?

[local]R3#sh ip route 100.100.0.0
[local]R3#
Nail down routes you want to originate

ip route 100.100.0.0/16 null0
Check the RIB

[local]R3#sh ip route
> S 100.100.0.0/16 1 0 00:41:17 null0
BGP originates the route!!

[local]R3#sh bgp route
BGP table version is 1, local router ID is 11.0.0.3
Status codes: d damped, h history, > best, i internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path

> 100.100.0.0/16 0.0.0.0 0 100 32768 i
Example 2
CE-TG-0009
 Trying to originate an aggregate route

o aggregate-address 9.9.0.0/16 summary-only
 The RIB has a component but BGP does not create the aggregate??
[local]R3#sh ip rou 9.9.0.0/16 longer-prefixes

Type Network Next Hop Dist Metric UpTime Interface
> C 9.9.9.9/32 0 0 00:00:19 loopback1
[local]R3#sh bgp route | include 9.9.0.0

[local]R3#
Remember, to have a BGP aggregate you need a BGP component, not a RIB component
[local]R3#sh bgp route 9.9.0.0 /16 longer-prefixes
[local]R3#
 Once BGP has a component route we originate the aggregate

[local]R3#
router bgp 100
aggregate-address 9.9.0.0/16 summary-only
network 9.9.9.9/32
[local]R3#sh bgp route 9.9.0.0/16 longer-prefixes

BGP table version is 3, local router ID is 11.0.0.3
Status codes: d damped, h history, > best, i internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path

> 9.9.0.0/16 0.0.0.0 0 100 32768 i
> 9.9.9.9/32 0.0.0.0 0 100 32768 i
[local]R3#
Troubleshooting Tips
• BGP Network statement rules - Always need an exact route (RIB)
• Aggregate-address looks in the BGP table, not the RIB
• “show ip route x.x.x.x y.y.y.y longer” - Great for finding RIB component routes
• “show ip bgp x.x.x.x y.y.y.y longer” - Great for finding BGP component routes


BGP Troubleshooting Guide V1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

BGP Troubleshooting Guide V1

Uploaded by

Copyright:

Available Formats

Redback Networks

BGP Troubleshooting Guide

Issue Date: 2009-Feb-03 8:58 PM

Document ID: CE-TG-0009

Created by: Jimmy Wong on: 1/14/2009

Proprietary and Confidential

 Copyright 2009 Redback Networks Inc.

Document Name BGP Trouble-shooting Guide

Redback Document ID CE-TG-0009

Issue Date 2009-Feb-03 8:58 PM

Name Position Signature Date

Issue Author(s) Date Description

Created by: Jimmy Wong on: 1/14/2009 Page 2 of 12

Created by: Jimmy Wong on: 1/14/2009 Page 3 of 12

3 Local Configuration problem

3.1 Peer Establishment

 Router establish a TCP session

3.1.1 Common problem

 Session are not established

Created by: Jimmy Wong on: 1/14/2009 Page 4 of 12

3.1.2 Peer establishment – iBGP

[local]R3#show bgp summary

Entry Type Count Memory

Neighbor AS MsgRcvd MsgSent TblVer InQ OutQ Rst Up/Down PfxRcvd/Sent

Type Network Next Hop Dist Metric UpTime Interface

[local]R3#debug bgp session-state

[local]R3#show config bgp

[local]R3# debug bgp session-state

Here the IBGP is established.

Created by: Jimmy Wong on: 1/14/2009 Page 6 of 12

Neighbor AS MsgRcvd MsgSent TblVer InQ OutQ Rst Up/Down PfxRcvd/Sent

3.1.3 Peer establishment – eBGP

 Trying to load-balance over multiple links to the eBGP peer

Created by: Jimmy Wong on: 1/14/2009 Page 7 of 12

----11.0.0.6 PING Statistics----

[local]R3#show bgp neighbor 11.0.0.6

Address family: ipv4 unicast

[local]R3#debug bgp neighbor 11.0.0.6 session-state

Created by: Jimmy Wong on: 1/14/2009 Page 8 of 12

[local]R3#sh bgp summary

Still not working but connectivity has already been check.

[local]R3#sh bgp summary

Few things to remember for multihop eBGP

3.1.4 Peer Establishment - Password

Configuration on R3 looks ok. Check the log message.

The log messages

Created by: Jimmy Wong on: 1/14/2009 Page 9 of 12

We are getting invalid MD5 digest messages – password mismatch!

Created by: Jimmy Wong on: 1/14/2009 Page 10 of 12

3.2 Missing route

3.2.1 Routing Origination

BGP is not originating the route???

Do we have the exact route?

Nail down routes you want to originate

Check the RIB

BGP originates the route!!

Network Next Hop Metric LocPrf Weight Path

 Trying to originate an aggregate route

[local]R3#sh ip rou 9.9.0.0/16 longer-prefixes

[local]R3#sh bgp route | include 9.9.0.0

 Once BGP has a component route we originate the aggregate

[local]R3#sh bgp route 9.9.0.0/16 longer-prefixes

Network Next Hop Metric LocPrf Weight Path