Inter VRF prv2

Fabric
1
Spine1 Spine 2

1/2 1/4 1/2 1/4
97 98 98
97
Leaf 2 Leaf 4
60 60
vmping1 pr-‐inter-‐vrf
33.1.1.10 14.14.14.2
0050.5699.f4b5 0050.5699.a78a
pray?ck: 2 EPs, inter-‐vrf, same tenant, different leafs

1
Tenant pr
VRF pr VRF pr2

BD pr-‐rh-‐1
BD inter-‐vrf
EPG rh1 1.1.1.1/24
BD -‐ Private 14.14.14.1/24
EPG private/shared
ap: p r-‐ppv
BD pr-‐vmping3 EPG_pr-‐inter-‐vrf
3.1.1.1/24 ap: pr-‐inter-‐vrf

EPG vmping3
BD -‐ Private
BD pr-‐vmping1
EPG vmping1 33.1.1.1/24

EPG – Private/shared
BD pr-‐vmping2
ap: pr-‐ipv4-‐ping
EPG vmping2 3.3.3.1/24

BD – Private
Contract: ipv4-‐ping
Scope: Tenant
consumer EP: 1.1.1.35 – leaf 2
EPG rh1
ap: pr-‐ppv
VRF pr
EPG vmping3
consumer
provider
EP: 3.1.1.50 – leaf 2
provider
EPG_pr-‐inter-‐vrf EP: 14.14.14.2
EP: 33.1.1.10 EPG vmping1 Leaf 4
Leaf 2
ap: pr-‐ipv4-‐ping VRF pr ap: pr-‐inter-‐vrf
VRF pr2
EP: 3.3.3.10 EPG vmping2
Leaf 4
consumer
Leaf 2
fab1-leaf2# show endpoint ip 33.1.1.10

Legend:
33.1.1.10
O - peer-attached H - vtep a - locally-aged S - static Learned local on
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce leaf 2
+---------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+---------------+---------------+-----------------+--------------+-------------+
30 vlan-3425 0050.5699.f4b5 L eth1/60
pr:pr vlan-3425 33.1.1.10 L
Leaf 4
fab1-leaf4# show endpoint ip 14.14.14.2
Legend:
O - peer-attached H - vtep a - locally-aged S - static
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce
+---------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+---------------+---------------+-----------------+--------------+-------------+
31 vlan-3428 0050.5699.a78a L eth1/60
pr:pr2 vlan-3428 14.14.14.2 L
14.14.14.2
Learned local on
leaf 4
4
fab1-leaf2# show ip route vrf pr:pr fab1-leaf4# show ip route vrf pr:pr2
IP Route Table for VRF "pr:pr" IP Route Table for VRF "pr:pr2"
'*' denotes best ucast next-hop '*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'[x/y]' denotes [preference/metric] '%<string>' in via output denotes VRF <string>
'%<string>' in via output denotes VRF <string>
14.14.14.0/24, ubest/mbest: 1/0, attached, direct,
[snip] pervasive
*via 10.0.24.65%overlay-1, [1/0], 19:20:11, static
14.14.14.0/24, ubest/mbest: 1/0, attached, direct, 14.14.14.1/32, ubest/mbest: 1/0, attached
pervasive *via 14.14.14.1, vlan38, [1/0], 19:21:28, local,
*via 10.0.24.65%overlay-1, [1/0], 09:13:15, static local
pervasive
[snip] *via 10.0.24.65%overlay-1, [1/0], 19:20:11, static

pervasive
*via 10.0.24.65%overlay-1, [1/0], 09:13:15, static
33.1.1.1/32, ubest/mbest: 1/0, attached
*via 33.1.1.1, vlan35, [1/0], 09:14:20, local,
local
Local 14.14.14.0
[snip] subnet
14.14.14.0 Route
leaked into the
“pr” VRF 33.1.1.0 Route
Local 33.1.1.0 leaked into the
subnet “pr2” VRF
10.0.24.65 is the Proxy Anycast address for this fabric (show isis dteps vrf overlay-‐1) 5
Check both spines EP Database (COOP) if proxy used for forwarding
6
VRF VNID Important for
next CLI command
EP
learned
Is it
learned
off
correct
leaf?
Is it
learned
off
correct
leaf?
EP
learned
7
Spine 1
fab1-‐spine1# show coop internal info ip-‐db key 3112960 14.14.14.2

IP address : 14.14.14.2
Vrf : 3112960
Flags : 0
EP bd vnid : 15826917 VRF VNID
EP mac : 00:50:56:99:A7:8A
Publisher Id : 10.0.104.64
(Segment ID)
Record ?mestamp : 08 06 2015 15:16:20
URIB Tunnel Info
Num tunnels : 1
Tunnel address : 10.0.104.64
Tunnel ref count : 1

fab1-‐spine1# show coop internal info ip-‐db key 3047425 33.1.1.10

IP address : 33.1.1.10
Vrf : 3047425
Flags : 0
EP bd vnid : 15990738 VRF VNID
EP mac : 00:50:56:99:F4:B5
Publisher Id : 10.0.168.90
(Segment ID)
Record ?mestamp : 08 06 2015 15:22:31
URIB Tunnel Info
Num tunnels : 1
Tunnel address : 10.0.168.90
Tunnel ref count : 1
8
Leaf 2
fab1-leaf2# show vlan extended
VLAN Name Status Ports

---- -------------------------------- --------- -------------------------------
9 infra:default active Eth1/1, Eth1/47, Eth1/48,
Eth1/79, Eth1/80, Eth1/82, Po1,
Po2, Po3
[snip]
35 pr:pr-vmping1 active Eth1/60
[snip]
BD VLAN
53 pr:pr-ipv4-ping:vmping1 active Eth1/60
VLAN Type Vlan-mode Encap

---- ----- ---------- -------------------------------
9 enet CE vxlan-16777209, vlan-4093
35 enet CE vxlan-15990738 EPG VLAN
[snip]
53 enet CE vlan-3425
ENCAP VLAN
9
Leaf 4
fab1-leaf4# show vlan extended
VLAN Name Status Ports

---- -------------------------------- --------- -------------------------------
9 infra:default active Eth1/1, Eth1/25, Eth1/47,
Eth1/48, Eth1/80, Eth1/82,
Eth1/95, Po1, Po2
[snip]
38 pr:inter-vrf active Eth1/60

39 pr:pr-inter-vrf:pr-inter-vrf active Eth1/60
[snip] BD VLAN
38 enet CE vxlan-15826917
39 enet CE vlan-3428
EPG VLAN
ENCAP VLAN
10
Leaf 2
module-1# show system internal eltmc info vlan brief
VLAN-Info
VlanId HW_VlanId Type Access_enc Access_enc Fabric_enc Fabric_enc BDVlan
Type Type
==================================================================================
9 1 BD_CTRL_VLAN 802.1q 4093 VXLAN 16777209 0
[snip] BD
35 11 BD_VLAN Unknown 0 VXLAN 15990738 35
37 34 FD_VLAN 802.1q 3424 VXLAN 10196 36
38 35 FD_VLAN 802.1q 52 VXLAN 9694 15
40 36 FD_VLAN 802.1q 256 VXLAN 9795 39
44 38 FD_VLAN 802.1q 238 VXLAN 9300 43
46 39 EPG FD_VLAN 802.1q 3428 VXLAN 10200 45
47 40 FD_VLAN 802.1q 51 VXLAN 9693 23
48
49
17
41
BD_VLAN
FD_VLAN
Unknown
802.1q
0
230
VXLAN
VXLAN
15728623
9292
48
48
ENCAP
50 10 FD_VLAN 802.1q 601 VXLAN 8693 27 VLAN
52 31 FD_VLAN 802.1q 600 VXLAN 8692 51
53 54 FD_VLAN 802.1q 3425 VXLAN 10197 35
55 37 FD_VLAN 802.1q 234 VXLAN 9296 54
56 55 FD_VLAN 802.1q 254 VXLAN 9793 54
Broadcomm
Hardware VLAN ID
11
Leaf 4
module-1# show system internal eltmc info vlan brief
VLAN-Info
VlanId HW_VlanId Type Access_enc Access_enc Fabric_enc Fabric_enc BDVlan
Type Type
==================================================================================
9 1 BD_CTRL_VLAN 802.1q 4093 VXLAN 16777209 0
11 16 FD_VLAN 802.1q 53 VXLAN 9695 10
13 17 FD_VLAN 802.1q 55 VXLAN 9697 12
[snip] BD ENCAP
VLAN
39 32 FD_VLAN 802.1q 3428 VXLAN 10200 38
41 33 FD_VLAN 802.1q 374 VXLAN 8810 40
44 9 FD_VLAN 802.1q 601 VXLAN 8693 43
46 27 FD_VLAN 802.1q 600 VXLAN 8692 45
Broadcomm
Hardware VLAN ID
EPG
12
Only Leaf 2 displayed (check encap entries on leaf 4 as well)
module-1# bcm-shell
NUM ARGS: 0
bcm shell path: /lc/isan/bin/bcm-shell-hw
Interface in correct
unit is 0
Available Unit Numbers: 0
hardware VLAN
bcm-shell.0> vlan show 54
vlan 54 ports cpu,xe59,hg (0x00000000000000000000000000000000000001fe000000001000000000000001), untagged none
(0x0000000000000000000000000000000000000000000000000000000000000000) MCAST_FLOOD_UNKNOWN
bcm-shell.0> search vlan_xlate old_vlan_id=0xd61 (3435 in hex = 0xd61)

Correct encap to hardware
Searching VLAN_XLATE table indexes 0x0 through 0x3fff... vlan translate entry
VLAN_XLATE.ipipe0[2400]:
<XLATE:VSAN_ID=0,XLATE:VLAN_ID=0xd61,XLATE:VLAN_ACTION_VALID=1,XLATE:TGID=0x3ff,XLATE:TAG_ACTION_PROFILE_PTR=1,XLATE:T=1,XL
ATE:SVP_VALID=0,XLATE:SOURCE_VP=0,XLATE:SOURCE_FIELD=0xffff,XLATE:RSVD_SOURCE_VP=0,XLATE:RSVD_FLEX_CTR_POOL_NUMBER=0,XLATE:
RSVD_FLEX_CTR_BASE_COUNTER_IDX=0,XLATE:RESERVED_DUMMY=0,XLATE:RESERVED_0=0,XLATE:PORT_NUM=0x7f,XLATE:OVID=0xd61,XLATE:OTAG=
0xd61,XLATE:OPRI=0,XLATE:OLD_VLAN_ID=0xd61,XLATE:NEW_VLAN_ID=0x36,XLATE:NEW_OVID=0x36,XLATE:NEW_OPRI=0,XLATE:NEW_OCFI=0,XLA
TE:NEW_IVID=0,XLATE:NEW_IPRI=0,XLATE:NEW_ICFI=0,XLATE:MPLS_ACTION=0,XLATE:MODULE_ID=0xff,XLATE:L3_IIF_VALID=0,XLATE:L3_IIF=
0,XLATE:KEY_ZERO_1=0,XLATE:KEY=0x6b0ffff84,XLATE:IVID=0,XLATE:ITAG=0xd61,XLATE:INCOMING_VIDS=0xd61,XLATE:HASH_LSB=0x561,XLA
TE:GLP=0xffff,XLATE:FLEX_CTR_POOL_NUMBER=0,XLATE:FLEX_CTR_OFFSET_MODE=0,XLATE:FLEX_CTR_BASE_COUNTER_IDX=0,XLATE:FCOE_VSAN_P
RI=0,XLATE:FCOE_VSAN_ID=0,XLATE:DUMMY=0,XLATE:DISABLE_VLAN_CHECKS=0,XLATE:DATA=0x40000000100000000d8,VSAN_ID=0,VLAN_ID=0xd6
1,VLAN_ACTION_VALID=1,VALID=1,TGID=0x3ff,TAG_ACTION_PROFILE_PTR=1,T=1,SVP_VALID=0,SOURCE_VP=0,SOURCE_TYPE=0,SOURCE_FIELD=0x
ffff,RSVD_SOURCE_VP=0,RSVD_FLEX_CTR_POOL_NUMBER=0,RSVD_FLEX_CTR_BASE_COUNTER_IDX=0,RESERVED_DUMMY=0,RESERVED_0=0,PORT_NUM=0
x7f,OVID=0xd61,OTAG=0xd61,OPRI=0,OLD_VLAN_ID=0xd61,NEW_VLAN_ID=0x36,NEW_OVID=0x36,NEW_OPRI=0,NEW_OCFI=0,NEW_IVID=0,NEW_IPRI
=0,NEW_ICFI=0,MPLS_ACTION=0,MODULE_ID=0xff,L3_IIF_VALID=0,L3_IIF=0,KEY_ZERO_1=0,KEY_TYPE=4,KEY=0x6b0ffff84,IVID=0,ITAG=0xd6
1,INCOMING_VIDS=0xd61,HASH_LSB=0x561,GLP=0xffff,FLEX_CTR_POOL_NUMBER=0,FLEX_CTR_OFFSET_MODE=0,FLEX_CTR_BASE_COUNTER_IDX=0,F
COE_VSAN_PRI=0,FCOE_VSAN_ID=0,EVEN_PARITY=0,DUMMY=0,DISABLE_VLAN_CHECKS=0,DATA=0x40000000100000000d8>
VLAN_XLATE.ipipe0[9232]: Two HW vlan entries for this encap vlan (ppv)
<XLATE:VSAN_ID=0,XLATE:VLAN_ID=0xd61,XLATE:VLAN_ACTION_VALID=1,XLATE:TGID=9,XLATE:TAG_ACTION_PROFILE_PTR=1,XLATE:T=0,XLATE:
SVP_VALID=0,XLATE:SOURCE_VP=0,XLATE:SOURCE_FIELD=9,XLATE:RSVD_SOURCE_VP=0,XLATE:RSVD_FLEX_CTR_POOL_NUMBER=0,XLATE:RSVD_FLEX
_CTR_BASE_COUNTER_IDX=0,XLATE:RESERVED_DUMMY=0,XLATE:RESERVED_0=0,XLATE:PORT_NUM=9,XLATE:OVID=0xd61,XLATE:OTAG=0xd61,XLATE:
OPRI=0,XLATE:OLD_VLAN_ID=0xd61,XLATE:NEW_VLAN_ID=0x19,XLATE:NEW_OVID=0x19,XLATE:NEW_OPRI=0,XLATE:NEW_OCFI=0,XLATE:NEW_IVID=
0,XLATE:NEW_IPRI=0,XLATE:NEW_ICFI=0,XLATE:MPLS_ACTION=0,XLATE:MODULE_ID=0,XLATE:L3_IIF_VALID=0,XLATE:L3_IIF=0,XLATE:KEY_ZER
O_1=0,XLATE:KEY=0x6b08004a4,XLATE:IVID=0,XLATE:ITAG=0xd61,XLATE:INCOMING_VIDS=0xd61,XLATE:HASH_LSB=0x561,XLATE:GLP=9,XLATE:
FLEX_CTR_POOL_NUMBER=0,XLATE:FLEX_CTR_OFFSET_MODE=0,XLATE:FLEX_CTR_BASE_COUNTER_IDX=0,XLATE:FCOE_VSAN_PRI=0,XLATE:FCOE_VSAN
_ID=0,XLATE:DUMMY=0,XLATE:DISABLE_VLAN_CHECKS=0,XLATE:DATA=0x4000000010000000064,VSAN_ID=0,VLAN_ID=0xd61,VLAN_ACTION_VALID=
1,VALID=1,TGID=9,TAG_ACTION_PROFILE_PTR=1,T=0,SVP_VALID=0,SOURCE_VP=0,SOURCE_TYPE=1,SOURCE_FIELD=9,RSVD_SOURCE_VP=0,RSVD_FL
EX_CTR_POOL_NUMBER=0,RSVD_FLEX_CTR_BASE_COUNTER_IDX=0,RESERVED_DUMMY=0,RESERVED_0=0,PORT_NUM=9,OVID=0xd61,OTAG=0xd61,OPRI=0
,OLD_VLAN_ID=0xd61,NEW_VLAN_ID=0x19,NEW_OVID=0x19,NEW_OPRI=0,NEW_OCFI=0,NEW_IVID=0,NEW_IPRI=0,NEW_ICFI=0,MPLS_ACTION=0,MODU
LE_ID=0,L3_IIF_VALID=0,L3_IIF=0,KEY_ZERO_1=0,KEY_TYPE=4,KEY=0x6b08004a4,IVID=0,ITAG=0xd61,INCOMING_VIDS=0xd61,HASH_LSB=0x56
1,GLP=9,FLEX_CTR_POOL_NUMBER=0,FLEX_CTR_OFFSET_MODE=0,FLEX_CTR_BASE_COUNTER_IDX=0,FCOE_VSAN_PRI=0,FCOE_VSAN_ID=0,EVEN_PARIT
Y=0,DUMMY=0,DISABLE_VLAN_CHECKS=0,DATA=0x4000000010000000064> 13
bcm-shell.0> l2 show
[snip]
mac=00:22:bd:f8:19:ff vlan=11 GPORT=0x7f modid=2 port=127 Static

mac=00:22:bd:f8:19:ff vlan=14 GPORT=0x7f modid=2 port=127 Static Hit
mac=00:50:56:89:16:65 vlan=20 GPORT=0x0 Trunk=3 Hit
mac=00:22:bd:f8:19:ff vlan=8 GPORT=0x7f modid=2 port=127 Static BD SVI MAC
mac=00:22:bd:f8:19:ff
mac=00:22:bd:f8:19:ff
vlan=13 GPORT=0x7f modid=2 port=127
Static Hit
Static Hit
programmed
mac=00:22:bd:f8:19:ff vlan=17 GPORT=0x7f modid=2 port=127 Static on correct
mac=00:50:56:bb:12:cb vlan=24 GPORT=0x23 modid=0 port=35/xe34 Hit
mac=00:22:bd:f8:19:ff vlan=12 GPORT=0x7f modid=2 port=127 Static HW VLAN
mac=00:22:bd:f8:19:ff vlan=9 GPORT=0x7f modid=2 port=127 Static Host on
mac=00:22:bd:f8:19:ff
mac=00:15:5d:fe:c9:1a
vlan=36 GPORT=0x0 Trunk=2 Hit
Static correct HW
mac=00:50:56:89:f8:be vlan=29 GPORT=0x0 Trunk=4 Hit VLAN
mac=00:50:56:89:e7:c8 vlan=22 GPORT=0x0 Trunk=4 Hit
mac=00:50:56:89:4e:15 vlan=35 GPORT=0x0 Trunk=3 Hit
mac=00:50:56:89:2a:5b vlan=26 GPORT=0x0 Trunk=4 Hit
mac=00:15:5d:fe:c9:1c vlan=58 GPORT=0x0 Trunk=2 Packets seen
mac=00:15:5d:fe:c9:18 vlan=1 GPORT=0x0 Trunk=2 Hit from host
mac=00:50:56:99:f4:b5 vlan=54 GPORT=0x3c modid=0 port=60/xe59 Hit
mac=00:22:bd:f8:19:ff vlan=57 GPORT=0x7f modid=2 port=127 Static Hit
mac=00:50:56:89:2c:55 vlan=30 GPORT=0x0 Trunk=3 Hit
Only Leaf 2 displayed
14
bcm-shell.0> show c rpkt
RPKT.xe0 : 1,454 +1,454
RPKT.xe8 : 56,744 +56,744 2/s
[snip]
RPKT.xe59 : 183,848 +183,848 7/s
RPKT.xe78 : 99,266 +99,266 3/s
RPKT.xe79 : 14,365 +14,365
RPKT.xe80 : 28,885 +28,885 1/s
RPKT.xe81 : 17,245 +17,245 Rx
RPKT.xe94
RPKT.hg0
:
:
9,688
301,713
+9,688
+301,713 6/s
Packets
RPKT.hg1 : 273,315 +273,315 5/s
RPKT.hg2 : 922,688 +922,688 20/s
RPKT.hg3 : 610,531 +610,531 10/s
RPKT.hg4 : 530,983 +530,983 9/s
RPKT.hg5 : 357,086 +357,086 8/s
RPKT.hg6 : 342,574 +342,574 10/s
RPKT.hg7 : 419,767 +419,767 17/s
bcm-shell.0> show c tpkt
TPKT.xe0 : 1,827 +1,827
[snip]
TPKT.xe59 : 154,625 +154,625 7/s
TPKT.xe60 : 1,473 +1,473
TPKT.xe64 : 1,474 +1,474
TPKT.xe78 : 101,359 +101,359 4/s
TPKT.xe79 : 9,454 +9,454 Tx
TPKT.xe80 : 7,352 +7,352
TPKT.xe81 : 19,546 +19,546 1/s Packets
TPKT.xe94 : 5,848 +5,848
TPKT.hg0 : 408,491 +408,491 15/s
TPKT.hg1 : 278,363 +278,363 14/s
TPKT.hg2 : 608,771 +608,771 13/s
TPKT.hg3 : 489,486 +489,486 10/s

15
bcm-shell.0> show c rpkt
RPKT.xe0 : 1,454 +1,454
RPKT.xe8 : 56,744 +56,744 2/s
[snip]
RPKT.xe59 : 183,848 +183,848 7/s
RPKT.xe78 : 99,266 +99,266 3/s
RPKT.xe79 : 14,365 +14,365
RPKT.xe80 : 28,885 +28,885 1/s
RPKT.xe81 : 17,245 +17,245 Rx
RPKT.xe94
RPKT.hg0
:
:
9,688
301,713
+9,688
+301,713 6/s
Packets
RPKT.hg1 : 273,315 +273,315 5/s
RPKT.hg2 : 922,688 +922,688 20/s
RPKT.hg3 : 610,531 +610,531 10/s
RPKT.hg4 : 530,983 +530,983 9/s
RPKT.hg5 : 357,086 +357,086 8/s
RPKT.hg6 : 342,574 +342,574 10/s
RPKT.hg7 : 419,767 +419,767 17/s
bcm-shell.0> show c tpkt
TPKT.xe0 : 1,827 +1,827
[snip]
TPKT.xe59 : 154,625 +154,625 7/s
TPKT.xe60 : 1,473 +1,473
TPKT.xe64 : 1,474 +1,474
TPKT.xe78 : 101,359 +101,359 4/s
TPKT.xe79 : 9,454 +9,454 Tx
TPKT.xe80 : 7,352 +7,352
TPKT.xe81 : 19,546 +19,546 1/s Packets
TPKT.xe94 : 5,848 +5,848
TPKT.hg0 : 408,491 +408,491 15/s
TPKT.hg1 : 278,363 +278,363 14/s
TPKT.hg2 : 608,771 +608,771 13/s
TPKT.hg3 : 489,486 +489,486 10/s

16
Leaf 2
bcm-shell.0> show c xe59
RUC.xe59 : 162,668 +162,668 7/s
RDBGC0.xe59 : 32,182 +32,182
RDBGC1.xe59 : Drop Counters 320 +320
ING_NIV_RX_FR.xe59 : 195,170 +195,170 7/s
TDBGC4.xe59 : 160,872 +160,872 7/s
TDBGC6.xe59 : 160,872 +160,872 7/s
R127.xe59 : 194,307 +194,307 7/s
R255.xe59 : 1,574 +1,574
R511.xe59 : 50 +50
RPKT.xe59 : 195,931 +12,083 7/s
RUCA.xe59 : 162,668 +162,668 7/s
RMCA.xe59 Received
: unicast 1,664 +1,664
RBCA.xe59 : 31,599 +31,599
RPRM.xe59 : 162,668 +162,668 7/s
RVLN.xe59 : 195,170 +195,170 7/s
RPOK.xe59 Received
: mul?cast 195,931 +195,931 7/s
RBYT.xe59 : 16,838,428 +16,838,428 585/s
T127.xe59 : 163,492 +163,492 8/s
T255.xe59 : 1,625 +1,625
T511.xe59 : 1,634 +1,634
TPOK.xe59 Received
: broadcast 166,751 +166,751 8/s
TPKT.xe59 : 166,751 +12,126 8/s
TUCA.xe59 : 162,665 +162,665 7/s
TMCA.xe59 : 3,504 +3,504 1/s
TBCA.xe59 : 582 +582
TVLN.xe59 Received
:tagged 165,217 +165,217 8/s
TBYT.xe59 : 15,363,499 +15,363,499 724/s
PERQ_PKT(0).xe59 : 2,549 +2,549 2/s
PERQ_PKT(3).xe59 : 3 +3
PERQ_BYTE(0).xe59 : 379,567 +379,567 139/s
PERQ_BYTE(3).xe59 : 204 +204
UC_PERQ_PKT(0).xe59 : 160,872 +160,872 7/s
UC_PERQ_PKT(9).xe59 : 3,327 +3,327
UC_PERQ_BYTE(0).xe59 : 14,286,624 +14,286,624 585/s
UC_PERQ_BYTE(9).xe59 : 697,104 +697,104
module-1# show system internal epmc endpoint ip 33.1.1.10
MAC : 0050.5699.f4b5 ::: Num IPs : 1
IP# 0 : 33.1.1.10 ::: IP# 0 flags :
Vlan id : 53 ::: Vlan vnid : 10197 ::: BD vnid : 15990738
Encap vlan : 802.1Q/3425
VRF name : pr:pr ::: VRF vnid : 3047425 Local interface
phy if : 0x1a03b000 ::: tunnel if : 0 ::: Interface : Ethernet1/60
Flags : 0x80004c04 correct
Ref count : 5 ::: sclass : 10932
Timestamp : 01/01/1970 12:17:44.517680
last mv timestamp 01/01/1970 00:00:00.000000 ::: ep move count : 0
last loop_detection_ts 01/01/1970 00:00:00.000000
previous if : 0 ::: loop detection count : 0
EP Flags : local,IP,MAC,class-set,timer,
Aging:Timer-type : Host-tracker timeout ::: Timeout-left : 238 ::: Hit-bit : Yes ::: Timer-reset count : 18
PD handles:
Bcm l2 hit-bit : Yes
[L2]: Asic : NS ::: ADJ : 0xc ::: LST SA : 0xaf0 ::: LST DA : 0xaf0 ::: GST ING : 0x13bc ::: BCM : Yes
[L3-0]: Asic : NS ::: ADJ : 0xc ::: LST SA : 0xf3a ::: LST DA : 0xf3a ::: GST ING : 0x1c31 ::: BCM : Yes
<detail> SDB Data:
mod 0 ::: port 60
is_ns_learn_port_valid : YES ::: ns_learn_port 60
is_bcm_vrf_id_valid : YES ::: bcm_vrf_id 0x6 Tunnel to leaf 4. Resolu?on
is_bcm_l3_if_valid : YES ::: bcm_l3_if 0x40
is_rmac_idx_valid : YES ::: rmac_idx 0x1 immediate, so remote endpoint
VLAN programmed and learned.
module-1# show system internal epmc endpoint ip 14.14.14.2
MAC : 0050.5699.a78a ::: Num IPs : 1 But this is inter-‐vrf, so is proxy
IP# 0 : 14.14.14.2 ::: IP# 0 flags :
Vlan id : 46 ::: Vlan vnid : 10200 ::: BD vnid : 15826917
used, or is the packet sent to
VRF name : pr:pr2 ::: VRF vnid : 3112960 leaf DTEP directly?
phy if : 0x1a03b000 ::: tunnel if : 0x18010007 ::: Interface : Tunnel7
Flags : 0x80000c80
Ref count : 5 ::: sclass : 16387
Timestamp : 01/01/1970 00:03:14.547000
last mv timestamp 01/01/1970 00:00:00.000000 ::: ep move count : 0
last loop_detection_ts 01/01/1970 00:00:00.000000
previous if : 0 ::: loop detection count : 0
EP Flags : on-peer,IP,MAC,class-set,
PD handles: 0xa006840 =
Bcm l2 hit-bit : No
[L2]: Asic : NS ::: ADJ : 0x2c ::: GST ING : 0x191e ::: GST EGR : 0x191e ::: BCM : No
10.0.104.64 (TEP for
[L3-0]: Asic : NS ::: ADJ : 0x2c ::: GST ING : 0x11eb ::: GST EGR : 0x11eb ::: BCM : No leaf 4)
<detail> SDB Data:
mod 4 ::: port 6
is_tep_ip_valid : YES ::: tep_ip 0xa006840
18
module-1# show system internal eltmc info interface tunnel brief
Name State M/Port Dst IP Encap Ept STepIdx DEnc Leaf 2
================================================================================
Tunnel1 up 4 /1 10.0.168.93 ivxlan 1 452 12276
Tunnel2 up 4 /2 10.0.168.94 ivxlan 1 366 12277
Tunnel3 up 4 /3 10.0.208.64 ivxlan 1 33 12278
Tunnel4 up 4 /4 10.0.232.95 ivxlan 1 166 12279
Tunnel5 up 4 /5 10.0.168.92 ivxlan 1 117 12280
Tunnel7 up 4 /6 10.0.104.64 ivxlan 1 275 12281
Tunnel8 up 4 /7 10.0.24.66 ivxlan 11 497 12282
Tunnel9 up 4 /8 10.0.24.65 ivxlan 5 347 12283
Tunnel10 up 4 /9 10.0.24.64 ivxlan 9 234 12284
Tunnel11 up 4 /10 10.0.0.1 ivxlan 1 409 12285
Tunnel12 up 4 /11 10.0.0.2 ivxlan 1 307 12286
Tunnel13 up 4 /12 10.0.0.3 ivxlan 1 130 12287
Tunnel15 up 4 /13 10.0.88.65 vxlan 2 290 12288
fab1-leaf2# acidiag fnvread

ID Name Serial Number IP Address Role State LastUpdMsgId
-------------------------------------------------------------------------------------------------
101 fab1-leaf1 SAL1819SAN6 10.0.232.95/32 leaf active 0
102 fab1-leaf2 SAL172682S0 10.0.168.90/32 leaf active 0
103 fab1-leaf3 SAL1802KLJF 10.0.168.92/32 leaf active 0
104 fab1-leaf4 SAL1802KATA 10.0.104.64/32 leaf active 0
201 fab1-spine1 FGE173400H2 10.0.168.94/32 spine active 0
202 fab1-spine2 FGE173400H7 10.0.168.93/32 spine active 0
fab1-leaf2# show isis dteps vrf overlay-1

IS-IS Dynamic Tunnel End Point (DTEP) database:
DTEP-Address Role Encapsulation Type
10.0.168.94 SPINE N/A PHYSICAL
10.0.24.65 SPINE N/A PHYSICAL,PROXY-ACAST-V4
10.0.24.66 SPINE N/A PHYSICAL,PROXY-ACAST-MAC
10.0.24.64 SPINE N/A PHYSICAL,PROXY-ACAST-V6
10.0.168.93 SPINE N/A PHYSICAL
10.0.208.64 LEAF N/A PHYSICAL
19
module-1# show platform internal counters tep | grep -i Tunnel7 Leaf 2
[103:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 0
[104:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 0
[101:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 0
[102:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 0
[99:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 0
[100:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 0
[97:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 18 27
[98:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 17 26
module-1# show platform internal counters tep | grep -i Tunnel7
[103:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 Leaf 4 tunnel
0
[104:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 0
[101:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 0
[102:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 0
[99:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 0
[100:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 0 0
[97:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 26 41
[98:11] Tunnel7 Y 10. 0.104. 64 Packets :- 0 0 29 42

[103:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 0
[104:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 0
[101:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 0
[102:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 0
[99:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 0
[100:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 0
[97:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 118
[98:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 112
[103:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 0
[104:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 Spine P0roxy
[101:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 0
[102:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 0
[99:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 0
[100:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 0
[97:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 261
[98:5 ] Tunnel9 Y 10. 0. 24. 65 Packets :- 0 0 0 255
Need to verify if spine proxy is used or if the path is leaf to leaf DTEP. I was told spine proxy will be used all the
?me for interf-‐vrf. Based on tes?ng proxy is also used when the encap VLAN of the remote endpoint is not
programmed on the local leaf, and therefore endpoint is not learned. This can happen when on-‐demand -‐vs-‐ 20
immediate resolu?on is used.
module-1# show platform internal ns forwarding lst-l3
================================================================================================================================
TABLE INSTANCE : 0
================================================================================================================================
Legend:
POS: Entry Position O: Overlay Instance
V: Valid Bit MD/PT: Mod/Port
PT: Pointer Type(A=Adj, E=ECMP, D=DstEncap N=Invalid) Leaf 2
PTR: ECMP/Adj/DstEncap/MET pointer
ML: MET Last
ST: Static PTH: Num Paths
BN: Bounce CP: Copy To CPU
PA: Policy Applied PI: Policy Incomplete
DL: Dst Local SP: Spine Proxy
--------------------------------------------------------------------------------------------------------------------------------
----------------------
MO SRC P M S B C P P D S
POS O VNID Address V DE MD/PT CLSS T PTR L T PTH N P A I L P
--------------------------------------------------------------------------------------------------------------------------------
----------------------
106 0 2e8001 1.1.1.35 1 0 00/0a c001 A 0 0 0 1 0 0 0 0 0 0
267 0 2f8000 14.14.14.1 1 0 00/00 1 A 0 0 1 1 0 0 0 1 0 0
[snip]
3898 0 2e8001 33.1.1.10 1 0 00/3c 2ab4 A 0 0 0 1 0 0 0 0 0 0

3978 0 278000 82.1.1.200 1 0 00/2f 8006 A 0 0 0 1 0 0 0 0 0 0
================================================================================================================================
TABLE INSTANCE : 1
================================================================================================================================
Legend:
PT: Pointer Type(A=Adj, E=ECMP, D=DstEncap N=Invalid)
Adjacency Pointer
ML: MET Last (used in next
command)
--------------------------------------------------------------------------------------------------------------------------------
----------------------
--------------------------------------------------------------------------------------------------------------------------------
----------------------
[snip]
3898 0 2e8001 33.1.1.10 1 0 00/3c 2ab4 A c 0 0 1 0 0 0 0 1 0

3978 0 278000 82.1.1.200 1 0 00/00 8006 A 27 0 0 1 0 0 0 0 1 0 21
module-1# show platform internal ns forwarding adj 0xc
===========================================================================================================
TABLE INSTANCE : 0
===========================================================================================================
Legend
TD: TTL Dec Disable UP: USE PCID Encap Pointer
(HEX)
DM: Dst Mac Rewrite SM: Src Mac Rewrite
RM IDX: Router Mac IDX SR: Seg-ID Rewrite
----------------------------------------------------------------------------------------------------------------
-----------------
ENCP T U USE D S RM S SRC Leaf 1
POS SEG-ID PTR D P PCI M DST-MAC M IDX R SEG-ID CLSS
-----------------------------------------------------------------------------------------------------------
12 36 66 0 0 0 1 00:0b:0b:0b:0b:0b 1 1 1 36 0
===========================================================================================================
TABLE INSTANCE : 1
===========================================================================================================Legen
d
TD: TTL Dec Disable UP: USE PCID
----------------------------------------------------------------------------------------------------------------
-----------------
ENCP T U USE D S RM S SRC
-----------------------------------------------------------------------------------------------------------
12 36 66 0 0 0 1 00:0b:0b:0b:0b:0b 1 1 1 36 0
Encap
Pointer
Decimal
module-1# show system internal eltmc info interface all | grep "dest_encap_idx\|interface" | grep -B 1 102
interface: Ethernet1/60 ::: ifindex: 436449280
dest_encap_idx: 102
22
module-1# show system internal eltmc info interface ethernet 1/60
IfInfo:
interface: Ethernet1/60 ::: ifindex: 436449280
iod: 105 ::: state: up
Mod: 0 ::: Port: 60
port_layer: L2 ::: fabric_port: 0
port_mode: trunk ::: native_vlan_id: 0
switchingSt: enabled ::: speed: 1000
Storm Ctrl Info:

Type: Percent
Stm_rate: 100.000000 ::: Stm_burst: 100.000000
Stm_rate(Mbps): 1000.000000 ::: Stm_burst(Mbps): 1000.000000
Stm_rate(toks): 625 ::: Stm_burst(toks): 65535
Stm_Pol_Apply: 0
xlate_l2_classid_unset: 0
vlan_bmp:25-26,35-37,45-46,53
vlan_bmp_count: 8
acc_vlan_bmp:3420,3424-3425,3428
acc_vlan_bmp_count: 4
scope(0:G, 1:L): 0 ::: class_id:: 0
NorthStar Info:
pc_mbr_idx: 59 ::: dest_learn_port: 60
dest_encap_idx: 102
Leaf 2
Encap
Pointer
Decimal
23
module-1# show platform internal ns forwarding gst-l3 ingress
=======================================================================================================================
======================
TABLE INSTANCE : 0
=======================================================================================================================
======================
Legend:
POS: Entry Position O: Overlay Instance Remote entry for DA, but this is
PT: Pointer Type(A=Adj, E=ECMP, D=DstEncap N=Invalid) proxy? Where does this point? We
ML: MET Last will dump this adjacency (2c) in
ST: Static
BN: Bounce
PTH: Num Paths
CP: Copy To CPU
page 26-‐27
-----------------------------------------------------------------------------------------------------------------------
----------------------
-----------------------------------------------------------------------------------------------------------------------
----------------------
4372 0 240000 81.1.1.200 1 0 00/00 16 A 2b 0 0 1 0 0 0 0 1 0
4587 0 2f8000 14.14.14.2 1 1 00/00 4003 A 2c 0 0 1 0 0 0 0 0 0
4852 0 240000 99.1.1.137 1 0 00/00 1 A 8 0 0 1 0 0 0 0 1 0
4990 0 230000 192.168.1.5 1 0 00/00 c003 A e 0 0 1 0 0 0 0 1 0
5970 0 230000 192.168.1.10 1 0 00/00 c003 A e 0 0 1 0 0 0 0 1 0
6597 0 2a0002 51.0.0.1 1 0 00/00 1 A 21 0 0 1 0 0 0 0 1 0
6755 0 200001 83.1.1.200 1 0 00/00 c004 A 2a 0 0 1 0 0 0 0 1 0
6831 0 240000 99.1.1.143 1 0 00/00 1 A 2 0 0 1 0 0 0 0 1 0
7217 0 2e8001 33.1.1.10 1 0 00/00 2ab4 A c 0 0 1 0 0 0 0 1 0
7451 0 2e8001 181.181.181.181 1 0 00/00 1 A 23 0 0 1 0 0 0 0 1 0
7472 0 240000 4.4.4.4 1 0 00/00 1 A 22 0 0 1 0 0 0 0 1 0
7487 0 258000 11.11.11.25 1 0 00/00 4003 A d 0 0 1 0 0 0 0 1 0
7535 0 240000 1.1.1.1 1 0 00/00 1 A 21 0 0 1 0 0 0 0 1 0
7941 0 278000 82.1.1.200 1 0 00/00 8006 A 27 0 0 1 0 0 0 0 1 0
7953 0 278000 84.1.1.100 1 0 00/00 8007 A 26 0 0 1 0 0 0 0 1 0
8046 0 258000 93.93.93.93 1 0 00/00 1 A 21 0 0 1 0 0 0 0 1 0
8075 0 250000 66.66.1.1 1 0 00/00 1 A 21 0 0 1 0 0 0 0 1 0
Leaf 2
24
module-1# show platform internal ns forwarding gst-l3 egress
================================================================================================================
================
======================
TABLE INSTANCE : 1
================================================================================================================
================
======================
Legend:
PT: Pointer Type(A=Adj, E=ECMP, D=DstEncap N=Invalid)
ML: MET Last
----------------------------------------------------------------------------------------------------------------
----------------
----------------------
----------------------------------------------------------------------------------------------------------------
----------------
----------------------
3600 0 2e8001 3.3.3.10 1 1 04/06 8004 E 0 0 0 40 0 0 0 0 0 0
4587 0 2f8000 14.14.14.2 1 1 04/06 4003 E 0 0 0 40 0 0 0 0 0 0
Leaf 2
25
module-1# show platform internal ns forwarding adj 0x2c
======================
TABLE INSTANCE : 0
======================
Legend
TD: TTL Dec Disable UP: USE PCID Encap Pointer
(HEX)
----------------------
POS SEG-ID PTR D P PCI M DST-MAC M IDX R SEG-ID CLSS Leaf 1
----------------------------------------------------------------------------------------------------------------
----------------
----------------------
44 0 2ff9 0 0 0 1 00:0c:0c:0c:0c:0c 0 0 0 0 0
======================
TABLE INSTANCE : 1
======================
Legend
TD: TTL Dec Disable UP: USE PCID
----------------------
----------------------------------------------------------------------------------------------------------------
----------------
----------------------
44 0 2ff9 0 0 0 1 00:0c:0c:0c:0c:0c 0 0 0 0 0
26
module-1# show platform internal ns forwarding encap 0x2ff9
Leaf 2
======================
TABLE INSTANCE : 0
======================
Legend
MD: Mode (LUX & RWX) LB: Loopback
LE: Loopback ECMP LB-PT: Loopback Port
ML: MET Last TD: TTL Dec Disable Points to
DV: Dst Valid DT-PT: Dest Port
DT-NP: Dest Port Not-PC ET: Encap Type DTEP Leaf 4
OP: Override PIF Pinning HR: Higig DstMod RW
HG-MD: Higig DstMode KV: Keep VNTAG
----------------------
M PORT L L LB MET M T D DT DT E TST O H HG K M E
POS D FTAG B E PT PTR L D V PT NP T IDX P R MD V D T Dst MAC DIP
----------------------
12281 0 c00 0 1 0 0 0 0 0 0 0 3 b 0 0 0 0 0 3 00:00:00:00:00:00 10.0.104.64
======================
TABLE INSTANCE : 1
======================
Legend
ML: MET Last TD: TTL Dec Disable
DT-NP: Dest Port Not-PC ET: Encap Type
----------------------
----------------------
12281 0 fff 0 1 0 0 0 0 0 0 0 3 b 0 0 0 0 0 3 00:00:00:00:00:00 10.0.104.64
This is inter-‐vrf, but gst entry exists for des?na?on endpoint. So is proxy or direct
forwarding used? What does ELAM show?
27
fab1-leaf2# vsh_lc
Leaf 2
module-1# debug platform internal ns elam asic 0
module-1(NS-elam)# trigger init ingress in-select 3 out-select 0
module-1(NS-elam-insel3)# set outer ipv4 src_ip 33.1.1.10 dst_ip 14.14.14.2
module-1(NS-elam-insel3)# start
module-1(NS-elam-insel3)# status
Status: Triggered
Src and Dst IP

look correct
GST showed
0x2ff9, ELAM
shows 0x2i
28
module-1# show platform internal ns forwarding encap 0x2ffb
Leaf 2
======================
TABLE INSTANCE : 0
====================== ELAM encap is the
Legend
MD: Mode (LUX & RWX) LB: Loopback spine proxy anycast
LE: Loopback ECMP
ML: MET Last
LB-PT: Loopback Port
TD: TTL Dec Disable
for IPv4
----------------------
----------------------
12283 0 c00 0 1 0 0 0 0 0 0 0 3 5 0 0 0 0 0 3 00:00:00:00:00:00 10.0.24.65
======================
TABLE INSTANCE : 1
======================
Legend
ML: MET Last TD: TTL Dec Disable
----------------------
----------------------
12283 0 fff 0 1 0 0 0 0 0 0 0 3 5 0 0 0 0 0 3 00:00:00:00:00:00 10.0.24.65
So based on ELAM proxy is being used despite the endpoint being learned. So verifying
COOP as shown in slides 6-‐8 is important. Remember to check all spines.
29

Inter VRF prv2

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Inter VRF prv2

Uploaded by

Copyright:

Available Formats

Fabric

pray?ck: 2 EPs, inter-­‐vrf, same tenant, diﬀerent leafs

VRF pr VRF pr2

3.1.1.1/24 ap: pr-­‐inter-­‐vrf

EPG vmping1 33.1.1.1/24

EPG vmping2 3.3.3.1/24

consumer EP: 1.1.1.35 – leaf 2

ap: pr-­‐ipv4-­‐ping VRF pr ap: pr-­‐inter-­‐vrf

fab1-leaf2# show endpoint ip 33.1.1.10

33.1.1.0/24, ubest/mbest: 1/0, attached, direct,

VLAN Name Status Ports

35 pr:pr-vmping1 active Eth1/60

VLAN Type Vlan-mode Encap

VLAN Name Status Ports

38 pr:inter-vrf active Eth1/60

bcm-shell.0> search vlan_xlate old_vlan_id=0xd61 (3435 in hex = 0xd61)

mac=00:22:bd:f8:19:ff vlan=11 GPORT=0x7f modid=2 port=127 Static

Only Leaf 2 displayed

Only Leaf 2 displayed

Only Leaf 2 displayed

fab1-leaf2# acidiag fnvread

fab1-leaf2# show isis dteps vrf overlay-1

module-1# show platform internal counters tep | grep -i Tunnel9

3898 0 2e8001 33.1.1.10 1 0 00/3c 2ab4 A 0 0 0 1 0 0 0 0 0 0

3898 0 2e8001 33.1.1.10 1 0 00/3c 2ab4 A c 0 0 1 0 0 0 0 1 0

Storm Ctrl Info:

module-1(NS-elam)# trigger init ingress in-select 3 out-select 0

module-1(NS-elam-insel3)# set outer ipv4 src_ip 33.1.1.10 dst_ip 14.14.14.2

Src and Dst IP

You might also like

pray?ck: 2 EPs, inter-‐vrf, same tenant, diﬀerent leafs

3.1.1.1/24 ap: pr-‐inter-‐vrf

ap: pr-‐ipv4-‐ping VRF pr ap: pr-‐inter-‐vrf