You are on page 1of 58

Nexus 7000 virtual Port-Channel

Best Practices & Design Guidelines

Roberto Mari Technical Marketing Engineer Data Center Business Unit


November 2009 version 1.1

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC and Services vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Feature Overview & Terminology


Intelligent L2 Domains POD Evolution
Inter-POD Connectivity across L3 (Failure Boundary Preservation)
IP Cloud

Failure Boundary Core Aggregation

L3 L3 L2

vPC

L2MP

Access
L2 vPC vPC

Servers

STP+
STP Enhancements Bridge Assurance

vPC
NIC Teaming Simplified loop-free trees 2x Multi-pathing

Cisco L2MP
16x ECMP Low Latency / Lossless MAC Scaling Operational Flexibility
3

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Feature Overview & Terminology


vPC Definition
 Allow a single device to use a port channel across two upstream switches  Eliminate STP blocked ports  Uses all available uplink bandwidth  Dual-homed server operate in active-active mode  Provide fast convergence upon link/device failure  Reduce CAPEX and OPEX  Available on current and future hardware for M1 and D1 generation cards.
Logical Topology with vPC
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Logical Topology without vPC

Feature Overview & Terminology


vPC Terminology
vPC peer-keepalive link vPC peer-link

 vPC peer a vPC switch, one of a pair  vPC member port one of a set of ports (port channels) that form a vPC  vPC the combined port channel between the vPC peers and the downstream device  vPC peer-link Link used to synchronize state between vPC peer devices, must be 10GbE  vPC peer-keepalive link the keepalive link between vPC peer devices, i.e., backup to the vPC peer-link  vPC VLAN one of the VLANs carried over the peer-link and used to communicate via vPC with a peer device.

CFS protocol

vPC peer vPC

vPC vPC member member port port

vPC non-vPC device

 non-vPC VLAN One of the STP VLANs not carried over the peer-link  CFS Cisco Fabric Services protocol, used for state synchronization and configuration validation between vPC peer devices

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Building a vPC Domain


Configuration Steps
Following steps are needed to build a vPC (Order does Matter!)
1. Configure globally a vPC domain on both vPC devices 2. Configure a Peer-keepalive link on both vPC peer switches (make sure is operational) NOTE: When a vPC domain is configured the keepalive must be operational to allow a vPC domain to successfully form. 3. Configure (or reuse) an interconnecting port-channel between the vPC peer switches 4. Configure the inter-switch channel as Peer-link on both vPC devices (make sure is operational) 5. Configure (or reuse) Port-channels to dual-attached devices 6. Configure a unique logical vPC and join port-channels across different vPC peers
vPC peerkeepalive link vPC peer-link

vPC peer

Standalone Port-channel
2009 Cisco Systems, Inc. All rights reserved.

vPC
Cisco Confidential

vPC member port


6

Building a vPC Domain


Peer Link
 Definition:
Standard 802.1Q Trunk Can Carry vPC and non vPC VLANs* Carries Cisco Fabric Services messages (tagged as CoS=4 for reliable communication) Carries flooded traffic from a vPC peer Carries STP BPDUs, HSRP Hellos, IGMP updates, etc.
vPC peer-link

 Requirements:
Member ports must be 10GE interfaces one of the N7KM132XP-12 modules Peer-link are point-to-point. No other device should be inserted between the vPC peers.

 Recommendations (strong ones!)


Minimum 2x 10GbE ports on separate cards for best resiliency. Dedicated 10GbE ports (not shared mode ports)
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

*It is Best Practice to split vPC and non-vPC


7

VLANs on different Inter-switch Port-Channels.

Building a vPC Domain


Peer Link with Single 10G Module  Common Nexus 7000 configuration: 1x 10G, 7x 1G cards  vPC recommendation is 2 10G cards  Potential problem occurs if Nexus 7000 is L3 boundary with single 10G card  Use Object Tracking Feature available in 4.2  More information from CCO:
http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nxos/interfaces/configuration/guide/if_vPC.html#wp1529488

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

Building a vPC Domain


Peer Link with Single 10G Module Object Tracking
Scenario:  vPC deployments with a single N7KM132XP-12 card, where core and peerlink interfaces are localized on the same card.  This scenario is vulnerable to accesslayer isolation if the 10GE card fails on the primary vPC. vPC Object Tracking Solution:  Leverages object tracking capability in vPC (new CLI commands are added).  Peer-link and Core interfaces are tracked as a list of boolean objects.  vPC object tracking suspends vPCs on the impaired device, so traffic can get diverted over the remaining vPC peer.
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

e1/ e1/

e1/ e1/ e1/ e1/ vPC PKL vPC PL e1/ e1/ e2/

L3 L2
vPC Primary

e2/

vPC Secondary

rhs-7k-1(config-vpc-domain)# track <object>


9

Building a vPC Domain


Cisco Fabric Services (CFS)
 Definition/Uses:
Configuration validation/comparison MAC member port synchronization vPC member port status STP Management HSRP and IGMP snooping synchronization vPC status
CFS Messaging

 Characteristics:
Transparently enabled with vPC features CFS messages encapsulated in standard Ethernet frames delivered between peers exclusively on the peer-link Cisco Fabric Services messages are tagged as CoS=4 for reliable communication. Based on CFS from MDS product development Many years in service, robust protocol
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

10

Building a vPC Domain


Peer-Keepalive (1 of 2)
 Definition:  Heartbeat between vPC peers  Active/Active (no Peer-Link) detection  Messages sent on 2 second interval  3 second hold timeout on peer-link loss  Fault Tolerant terminology is specific to VSS and deprecated in vPC.  Packet Structure:  UDP message on port 3200, 96 bytes long (32 byte payload), includes version, time stamp, local and remote IPs, and domain ID.  Keepalive messages can be captured and displayed using the onboard Wireshark Toolkit.
vPC peerkeepalive link

 Recommendations:
 Should be a dedicated link (1Gb is adequate)  Should NOT be routed over the Peer-Link  Can optionally use the mgmt0 interface (along with management traffic)  As last resort, can be routed over L3 infrastructure
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

11

Building a vPC Domain


Peer-Keepalive (2 of 2)
Cautions/Additional Recommendations:  When using supervisor management interfaces to carry the vPC peerkeepalive, do not connect them back to back between the two switches.  Only one management port will be active a given point in time and a supervisor switchover may break keep-alive connectivity  Use the management interface only if you have an out-of-band management network (management switch in between).
Management Switch vPC_PK

Management Network
vPC_PK

Standby Management Interface Active Management Interface

vPC_PL

vPC1

vPC2

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

12

Building a vPC Domain


vPC Member Port
 Definition: Port-channel member of a vPC peer.  Requirements: Configuration needs to match other vPC peers member port config. In case of inconsistency a VLAN or the entire port-channel may suspend (i.e. MTU mismatch). Number of member ports on both vPC peers is not required to match. Up to 8 active ports between both vPC peers (16-way port-channel can be build with multi-layer vPC)
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

vPC member port

13

Building a vPC Domain


VDC Interaction
 vPC works seamlessly in any VDC based environment.  One vPC domain per VDC is supported, up to the maximum number of VDCs supported in the system.  It is still necessary to have a separate vPC peer-link and vPC PeerKeepalive Link infrastructure for each VDC deployed. Can vPC run between VDCs on the same switch?  This scenario should technically work, but it is NOT officially supported and has not been extensively tested by our QA team.  Could be useful for Demo or hands on, but It is NOT recommended for production environments. Will consolidate redundant points on the same box with VDCs (e.g. whole aggregation layer on a box) and introduce a single point of failure.  ISSU will NOT work in this configuration, because the vPC devices can NOT be independently upgraded.
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

14

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC and Services vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

15

Attaching to a vPC domain


The One and Only Rule

ALWAYS dual attach devices to a vPC Domain!!!


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

16

Attaching to a vPC Domain


IEEE 802.3ad and LACP
 Definition:
Port-channel for devices for devices dual-attached to the vPC pair. Provides local load balancing for port-channel members STANDARD 802.3ad port channel

 Access Device Requirements


STANDARD 802.3ad capability LACP Optional

 Recommendations:
 Use LACP when available for better failover and misconfiguration protection

vPC vPC Regular member Portport channel port

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

17

Attaching to a vPC Domain


My device cant be dual attached!
Recommendations (in order of preference):
1. ALWAYS try to dual attach devices using vPC (not applicable for routed links). PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dualactive scenarios. Ensures full redundant active/active paths through vPC. CONS: None 2. If (1) is not an option connect the device via a vPC attached access switch (could use VDC to create a virtual access switch). PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dualactive scenarios. Availability limited by the access switch failure. CONS: Need for an additional access switch or need to use one of the available VDCs. Additional administrative burden to configure/manage the physical/Virtual Device 3. If (2) is not an option connect device directly to (primary) vPC peer in a non-vPC VLAN* and provide for a separate interconnecting port-channel between the two vPC peers. PROS: Traffic diverted on a secondary path in case of peer-link failover CONS: Need to configure and manage additional ports (i.e. port-channel) between the Nexus 7000 devices. 4. If (3) is not an option connect device directly to (primary) vPC peer in a vPC VLAN PROS: Easy deployment CONS: VERY BAD. Bound to vPC roles (no role preemption in vPC) , Full Isolation on peer-link failure when attached vPC toggles to a secondary vPC role.

* VLAN that is NOT part of any vPC and not present on vPC peer-link
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

18

Attaching to a vPC Domain


vPC and non-vPC VLANs (i.e. single attached .. )
P

1. Dual Attached

2. Attached via VDC/Secondary Switch


Orphan Ports
P S

P S

Primary vPC Secondary vPC

3. Secondary ISL Port-Channel


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

4. Single Attached to vPC Device


19

Attaching to a vPC Domain


My device only does STP!
Recommendations (in order of preference): 1. ALWAYS try dual attach devices using vPC PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dual-active scenarios. Ensures full redundant active/active paths through vPC. CONS: None 2. If (1) is not an option connect the device via two independent links using STP. Use nonvPC VLANs ONLY on the STP switch.* PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dual-active scenarios. Ensures full redundant Active/Active paths on vPC VLANs. CONS: Requires an additional STP port-channel between the vPC devices. Operational burden in provisioning and configuring separate STP and vPC VLAN domains. Only Active/Standby paths on STP VLANs. 3. If (2) is not an option connect the device via two independent links using STP. (Use vPC VLANs on this switch) PROS: Simplify VLAN provisioning and does not require allocation of an additional 10GE port-channel. CONS: STP and vPC devices may not be able to communicate each other in certain failure scenarios (i.e. when STP Root and vPC primary device do not overlap). All VLANs carried over the peer-link may suspend until the two adjacency forms and vPC is fully synchronized". * Run the same STP mode as the vPC domain. Enable portfast/port type edge on host facing ports
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

20

vPC Design principles


Attaching to a vPC Domain - vPC and non-vPC VLANs (STP/vPC Hybrid)
Non vPC port-channel
P

S P

SR

PR S

1. All devices Dual Attached via vPC


SR P

2. Separate vPC and STP VLANs


PR S P S PR SR Primary vPC Secondary vPC Primary STP Root Secondary STP Root

3. Overlapping vPC and STP VLANs


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

21

Attaching to a vPC Domain


16-way Port-Channel (1 of 2)
 Multi-Layer vPC can join 8 active ports port-channels in a unique 16way port-channel*  vPC peer side load-balancing is LOCAL to the peer  Each vPC peer has only 8 active links, but the pair has 16 active load balanced links
Nexus 7000 16-way port channel Nexus 5000

vPC/MCEC and 8-way active port-channels


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

* Possible with any device supporting

22

Attaching to a vPC Domain


16-way Port-Channel (2 of 2)
 16 active ports between 8 active port-channel devices and 16 active port-channel devices?  vPC peer side load-balancing is LOCAL to the peer  Each vPC peer has only 8 active links, but the pair has 16 active load balanced links to the downstream device supporting 16 active ports  D-series N7000 line cards will also support 16 way active port-channel load balancing, providing for a potential 32 way vPC port channel!
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Nexus 7000 16-port port-channel Nexus 5000

Nexus 5000 16-port port-channel support introduced in 4.1(3)N1(1a) release


23

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC and Services vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

24

Layer 3 and vPC


Recommendations
 Use separate L3 links to hook up routers to a vPC domain is still standing.  Dont use L2 port channel to attach routers to a vPC domain unless you can statically route to HSRP address  If both, routed and bridged traffic is required, use individual L3 links for routed traffic and L2 port-channel for bridged traffic
Switch Po2 Po2

Switch

7k1 Po1

7k2

L3 ECMP

Router
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Router
25

Layer 3 and vPC


What can happen (1 of 3)

vPC view

Layer 2 topology

Layer 3 topology

7k vPC 7k1 7k2 7k1 7k2

R R R

R could be any router, L3 switch or VSS building a port-channel

Port-channel looks like a single L2 pipe. Hashing will decide which link to chose
Cisco Confidential

Layer 3 will use ECMP for northbound traffic

2009 Cisco Systems, Inc. All rights reserved.

26

Layer 3 and vPC


What can happen (2 of 3)
1) Packet arrives at R 2) R does lookup in routing table and sees 2 equal paths going north (to 7k1 & 7k2) 3) Assume it chooses 7k1 (ECMP decision) 4) R now has rewrite information to which router it needs to go (router MAC 7k1 or 7k2) 5) L2 lookup happens and outgoing interface is port-channel 1 6) Hashing determines which port-channel member is chosen (say to 7k2) 7) Packet is sent to 7k2 8) 7k2 sees that it needs to send it over the peer-link to 7k1 based on MAC address
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

S Po2

7k1 Po1

7k2

27

Layer 3 and vPC


What can happen (3 of 3)
9) 7k1 performs lookup and sees that it needs to send to S 10) 7k1 performs check if the frame came over peer link & is going out on a vPC. 11) Frame will only be forwarded if outgoing interface is NOT a vPC or if outgoing vPC doesnt have active interface on other vPC peer (in our example 7k2)
7k1 Po1 7k2

S Po2

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

28

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC and Services vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

29

Spanning Tree Recommendations


Overview STP Interoperability
 STP Uses:
Loop detection (failsafe to vPC) Non-vPC attached device Loop management on vPC addition/removal

 Requirements:
Needs to remain enabled, but doesnt dictate vPC member port state Logical ports still count, need to be aware of number of VLANs/port-channels deployed!

 Best Practices:
Not recommended to enable Bridge Assurance feature on vPC channels (i.e. no STP network port type). Tracked by CSCsz76892. vPC vPC STP is running to manage Make sure all switches in you layer 2 domain are running loops outside of vPCs with Rapid-PVST or MST (IOS default is non-rapid PVST+), direct domain, or before to avoid slow STP convergence (30+ secs) initial vPC configuration Remember to configure portfast (edge port-type) on host facing interfaces to avoid slow STP convergence (30+ secs)

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

30

Spanning Tree Recommendations


Port Configuration Overview
Data Center Core
N E B R L

Network port Edge or portfast port type Normal port type BPDUguard Rootguard Loopguard

Primary vPC
HSRP ACTIVE Primary Root
R R R N

Secondary vPC vPC Domain


N

Aggregation

HSRP STANDBY Secondary Root


R R R

Layer 3

R R

Layer 2 (STP + Rootguard)

Access
L

E B

E B

E B

E B

E B

Layer 2 (STP + BPDUguard)

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

31

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC and Services vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

32

Data Center Interconnect


Multi-layer vPC for Agg and DCI
DC 1
CORE
vPC domain 11 vPC domain 21

N E B F R

Network port Edge or portfast port type Normal port type BPDUguard BPDUfilter Rootguard

Long Distance

DC 2
CORE

N R R N -

AGGR

R N

- R

AGGR

vPC domain 10

vPC domain 20

R R

Key Recommendations
ACCESS ACCESS
E B

 vPC Domain id for facing vPC layers should be different  No Bridge Assurance on interconnecting vPCs  BPDU Filter on the edge devices to avoid BPDU propagation  No L3 peering between DCs (i.e. L3 over vPC)

E B

Server Cluster
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Server Cluster
33

Data Center Interconnect


Encrypted Interconnect
DC-1
Nexus 7010

DC-2
Nexus 7010

vPC

vPC

CTS Manual Mode (802.1AE 10GE line-rate encryption) No ACS is required

Nexus 7010

Nexus 7010

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

34

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC and Services vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

35

HSRP with vPC


FHRP Active/Active
 Support for all FHRP protocols in Active/Active mode with vPC  No additional configuration required
HSRP/VRRP Active: Active for shared L3 MAC HSRP/VRRP Standby: Active for shared L3 MAC

 Standby device communicates with vPC manager produces to determine if vPC peer is Active HSRP/VRRP peer  General HSRP best practices still applies.  When running active/active aggressive timers can be relaxed (i.e. 2-router vPC case)
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

L3 L2

36

HSRP with vPC


Do NOT use Object Tracking
Cautions:  Not recommended using HSRP link tracking in a vPC configuration  Reason: vPC will not forward a packet back on a vPC once it has crossed the peer-link, except in the case of a remote member port failure
L3 CORE
ACTIVE HSRP GW GW
VLAN 100, 200

STANDBY HSRP GW

L2/L3 Aggregation

VLAN 100

VLAN 200
37

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

HSRP with vPC


L3 Backup Routing
 Use an OSPF point-to-point adjacency (or equivalent L3 protocol) between the vPC peers to establish a L3 backup path to the Core through in case of uplinks failure  A single point-to-point VLAN/SVI will suffice to establish a L3 neighborship.
OSPF OSPF
VLAN 99

L3 L2
Primary vPC

OSPF

Secondary vPC

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

38

HSRP with vPC


Dual L2/L3 Pod Interconnect
Scenario:  Provide L2/L3 interconnect between L2 Pods, or between L2 attached Datacenters (i.e. sharing the same HSRP group).  A vPC domain without an active HSRP instance in a group would not able to forward traffic. Active Multi-layer vPC with single HSRP:  L3 on the N7K supports Active/Active on one pair, and still allows normal HSRP behavior on other pair (all in one HSRP group)  L3 traffic will run across Intra-pod link for non Active/Active L3 pair

Standby

Listen

Listen

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

39

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC and Services vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

40

vPC and Services


Catalyst 6500 Services Chassis w. Services VDC Sandwich
Two Nexus 7000 Virtual Device Contexts used to sandwich services between virtual switching layers Layer-2 switching in Services Chassis with transparent services Services Chassis provides Etherchannel capabilities for interaction with vPC vPC running in both VDC pairs to provide Etherchannel for both inside and outside interfaces to Services Chassis

Design considerations:
Access switches requiring services are connected to subaggregation VDC Access switches not requiring services may be connected to aggregation VDC May be extended to support multiple virtualized service contexts by using multiple VRF instances in the subaggregation VDC

Design Cautions:
Be aware of the Layer 3 over vPC design caveat. If Peering at Layer 3 is required across the two vPC layers an alternative solution should be explored (i.e. using STP rather than vPC to attach service chassis)
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

41

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC and Services vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

42

vPC Latest Enhancements


Summary
Several enhancements to vPC:

 vPC Object Tracking  vPC Peer-Gateway  vPC Delay Restore  Multi-layer vPC with single HSRP group  vPC unicast ARP handling  vPC Exclude Interface-VLAN  vPC single attached device Listing  vPC Convergence and Scalability
For more details:  4.2 Release Notes
http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nx-os/release/notes/42_nxos_release_note.html#wp218085

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

43

vPC Latest Enhancements


vPC Peer-Gateway for NAS interoperability
Scenario:  Interoperability with non RFC compliant features of some NAS devices (i.e. NETAPP Fast-Path or EMC IPReflect)  NAS device may reply to traffic using the MAC address of the sender device rather than the HSRP gateway.  Packet reaching vPC for the non local Router MAC address are sent across the peer-link and can be dropped if the final destination is behind another vPC. vPC Peer-Gateway Solution:  Allows a vPC switch to act as the active gateway for packets addressed to the peer router MAC (CLI command added in the vPC global config) N7k(config-vpc-domain)# peer-gateway
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

Local Routing for peer router mac Traffic

vPC PL

vPC PKL

L3 L2

44

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC and Services vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

45

In-Service Software Upgrade (ISSU)


vPC System Upgrade/Downgrade
 ISSU is still the recommended system upgrade in a multi-device vPC environment  vPC system can be independently upgraded with no disruption to traffic.  Upgrade is serialized and must be run one at the time (i.e. config lock will prevent synchronous upgrades)  Configuration is locked on other vPC peer during ISSU.
4.2(1) 4.1(3) 4.2(1) 4.1(3)

4.2(1) 4.1(3)

Begin 4.1(x) 4.2(x)

End 4.2(x) 4.1(x)

Caveats None None

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

46

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

47

4.2(1) vPC Enhancements


Convergence Topology
20 flows @1000 pps

OSPF

L3 Core Nexus 7000

N7K-1 OSPF Po10

N7K-2

L2/L3 Aggregation Nexus 7000 vPC

16-way port-channel
Po160 Po20

4-way port-channel L2 Access Nexus 5000

vPC Peer Link LACP Channel (2x10 GigE) vPC Peer-Keepalive (GigE)
2009 Cisco Systems, Inc. All rights reserved.

20 flows @1000 pps


Cisco Confidential

20 flows @1000 pps


48

vPC on Nexus 7000


Convergence Numbers
Failover case
Failure of secondary vPC peer*

Failure Topology Failure


4.1(4) P S North-Bound: ~700 ms South-Bound: ~2.5 sec 4.2(1) North-Bound: ~50 ms. South-Bound: ~100 ms

Convergence Time Restoration


4.1(4) North-Bound: ~3 sec South-Bound: ~3.4 sec 4.2(1) North-Bound: 100 900 ms South-Bound: 1.2 -2 s 4.1(4) North-Bound:~4.5 secs South-Bound: ~5 secs 4.2(1) North-Bound: ~400 ms-1.5 s South-Bound: ~1.5 s 4.1(4) North-Bound: ~900 ms South-Bound: up to 10+ s (CSCsz88998) 4.2(1) North-Bound: 150 - 900 ms South-Bound: ~ 900 ms1.5 s

Failure of a primary vPC peer*

4.1(4) P S North-Bound: ~150 ms South-Bound: ~3 sec 4.2(1) North-Bound: ~50 ms South-Bound: ~100 ms

Failover of the vPC Peer Link P S

4.1(4) North-Bound: ~1.3 s South-Bound: ~1.8 s 4.2(1) North-Bound: 100-300 ms South-Bound: 50-500 ms

NOTE: Convergence numbers may vary depending on the specific configuration (i.e. scaled number of VLANs/SVIs or HSRP groups) and traffic patterns (i.e. L2 vs L3 flows).
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

49

vPC on Nexus 7000


Scalability Number Improvements
Release 4.1(5) Supported Scalability 192 vPCs (2-port) with the following,
200 VLANs 200 HSRP Groups 40K MACs & 40K ARPs 10K (S,G) w. 66 OIFs (L3 sources) 3K (S,G) w. 34 OIFs (L2 sources)

Latest Ankara 4.2(1)

256 vPCs (4-port) with the following,


260 VLANs 200 SVI/HSRP Groups 40k MACs & 40K ARPs 10K (S,G) w. 66 OIFs (L3 sources) 3K (S,G) w. 64 OIFs (L2 sources)

NOTE: Supported numbers of VLANs/vPCs are NOT related to an hardware or software limit but reflect what has been currently validated by our QA. The N7k BU is planning to continuously increase these numbers as soon as new data-points become available.

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

50

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC and Services vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

51

vPC Hands-on Lab Information


On Demand vPC Lab Overview
N7K-Aggr N7K-Aggr

Pod 1

Pod 2

N7K-1 POD 1-2 VPC

N7K-2 POD 1-2 VPC

Pod 1

Pod 2

 Instructor-led hands-on lab introducing the vPC (virtual Portchannel) feature for the Nexus 7000.  Participants exposed to the configuration of vPC with NX-OS.  Lab needs to be manually booked through Nexus 7000 TMEs.
2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

N7K-3 POD 3-4 VPC

N7K-4 POD 3-4 VPC

Pod 3

Pod 4

N7K-7 POD 5-6 VPC

N7K-8 POD 5-6 VPC

Pod 5

Pod 6

52

vPC Hands-on Lab Information


vPC Lab Logistics and Timing
 The vPC Laboratory consists of 6 independent PODs.  A group of 2 students is assigned to each Pod.  Each student will configure a vPC peer device.  PODs are logically independent. Two adjacent PODs are physically bound to the same Nexus. Virtual Device Contexts (VDCs) are used to define logically independent devices on the same Nexus 7010 box.  The vPC Lab session is expected to be completed in around two hours.

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

53

Agenda
 Feature Overview & Terminology  vPC Design Guidance & Best Practices
Building a vPC domain Attaching to a vPC domain Layer 3 and vPC Spanning Tree Recommendations Data Center Interconnect (& Encryption) HSRP with vPC vPC and Services vPC latest enhancements ISSU

 Convergence and Scalability  vPC Hands-on Lab Information  Reference Material


2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential

54

Reference Material
Physical

vPC/VSS Interop Test Details


Logical L3 Core

N7K-1

N7K-2

L2/L3 Aggregation Nexus 7000 vPC

Po10
E1/26 E1/25

Po100
Te1/2/1 Te2/2/1

Po100

6K-1

6K-2

L2 Access 6500 VSS

vPC Peer Link LACP Channel (2x10 GigE) vPC PeerKeepalive (GigE) VSS VSL Channel (2x10 GigE)

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

55

Reference Material

vPC/VSS Interop Test Details  The following scenarios were tested:


VSS and vPC member failover and convergence Dual active scenarios and behavior Best practice guidelines for STP, L3 (NSF), Multicast

 Catalyst 6500/Nexus 7000 interoperability:


Multiple ports per chassis act as one larger ether-channel

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

56

Reference Material

Other Solution Tests and Recent vPC Documentation


 Enterprise Solutions Engineering:
http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DC_3_0/DC-3_0_IPInfra.html

 Implementing Nexus 7000 in the Data Center Aggregation Layer with Services:
https://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/nx_7000_dc.html

 Configuration Guide for Object Tracking Feature:


http://www.cisco.com/en/US/partner/docs/switches/datacenter/sw/4_2/nxos/interfaces/configuration/guide/if_vPC.html#wp1530133

 vPC white Paper:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/white_paper_c11516396.html

2009 Cisco Systems, Inc. All rights reserved.

Cisco Confidential

57