You are on page 1of 112

BCEFP in a Nutshell

Study Guide for Exam


150-810
Brocade University
Revision 0513
Corporate Headquarters - San Jose, CA USA
T: (408) 333-8000
info@brocade.com
European Headquarters - Geneva, Switzerland
T: +41 22 799 56 40
emea-info@brocade.com
Asia Pacific Headquarters - Singapore
T: +65-6538-4700
apac-info@brocade.com
2013 Brocade Communications Systems, Inc. All Rights Reserved.
ADX, AnyIO, Brocade, Brocade Assurance, the B-wing symbol, DCX, Fabric OS, ICX, MLX,
MyBrocade, OpenScript, VCS, VDX, and Vyatta are registered trademarks, and HyperEdge, The
Effortless Network, and The On-Demand Data Center are trademarks of Brocade
Communications Systems, Inc., in the United States and/or in other countries. Other brands,
products, or service names mentioned may be trademarks of their respective owners.
Notice: This document is for informational purposes only and does not set forth any warranty,
expressed or implied, concerning any equipment, equipment feature, or service offered or to
be offered by Brocade. Brocade reserves the right to make changes to this document at any
time, without notice, and assumes no responsibility for its use. This informational document
describes features that may not be currently available. Contact a Brocade sales office for
information on feature and product availability. Export of technical data contained in this
document may require an export license from the United States government
Revision 0513
2013 Brocade Communications Systems, Inc. All Rights Reserved. iii
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Introduction to BCEFP in a Nutshell First Edition
Objective: The BCEFP Nutshell guide is designed to help you prepare for the BCEFP Certification, exam
number 150-810.
Audience: The BCEFP Nutshell self-study guide is intended for those who have successfully completed the
CEF 200 Certified Ethernet Fabric Training and the CEFP 300-WBT Advanced Ethernet Fabric Administration
course, and who wish to undertake self-study or review activities before taking the actual BCEFP exam. The
BCEFP guide is not intended as a substitute for classroom training or hands-on time with Brocade products.
How to make the most of the BCEFP guide: The BCEFP guide summarizes the key topics on the BCEFP exam
for you in an easy to use format. It is organized closely around the exam objectives. We suggest this guide be
used in conjunction with our free online knowledge assessment test. To benefit from the BCEFP guide, we
strongly recommend you have successfully completed the CEF 200 Certified Ethernet Fabric Training and the
CEFP 300-WBT Advanced Ethernet Fabric Administration course.
We hope you find this useful in your journey towards BCEFP Certification, and we welcome your feedback by
sending an email to jcannata@brocade.com.
Joe Cannata
Certification Manager
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
iv 2013 Brocade Communications Systems, Inc. All Rights Reserved.
2013 Brocade Communications Systems, Inc. All Rights Reserved. v
Brocade Certified Ethernet Fabric Professional in a Nutshell Second Edition
Table of Contents
Brocade VCS Fabric Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Brocade VCS Distributed Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Distributed Fabric Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Masterless control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Shared port profile information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Automatic distribution of zoning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
VCS Fabric: Use Case Aggregation Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
VCS Fabric: Use Case Combined Access/Aggregation Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Hardware Products and Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Brocade VDX Product Comparisons and Positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Brocade VDX Switch Protocol and Fabric Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Brocade VDX 6720 Data Center Switches Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Brocade VDX 6720-60 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Brocade VDX 6710-54 Data Center Switch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Brocade VDX 6730-32 Data Center Switch Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Brocade VDX 6730-32 Data Flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Brocade VDX 6730-76 Data Center Switch Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Brocade VDX 8770-8 and VDX 8770-4 Data Center Switch Overview . . . . . . . . . . . . . . . . . 12
VDX 8770 Port Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
VDX 8770 Interface Nomenclature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
VDX 8770 Switch Fabric Module (SFM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
VDX 8770 Line Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Slot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
VDX8770 CID Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
VDX8770 Management Module (MM) See Figure14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Long Distance ISL Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Brocade VDX Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
VCS Fabric License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
FCoE License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Ports on Demand (POD) License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
NOS v3.0.0 Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
VCS Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
TRILL Defined. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
TRILL Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Hop Count. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
VCS Fabric Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Automatic Layer 2 Adjacency Formation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
RBridge ID Conflicts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
VCS Edge Port Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
L2 Forwarding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
vi 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Brocade Certified Ethernet Fabric Professional in a Nutshell Second Edition
FSPF Protocol for ECMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
VCS Layer 2 ECMP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
VCS Fabric L2 Multi-Destination Tree. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Multi-cast Tree Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Traffic Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
VCS Known Unicast Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
VCS Multicast Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
IGMP Snooping in a VCS Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
VCS Fabric Edge Loop Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
How Edge Loop Detection Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
BUM Storm Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
L3: Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
VCS Fabric - Layer 3 Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
VCS Fabric L3 Routing Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
How OSPF works in VCS Fabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Assigning OSPF areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Configuring a Summary-Address for the NSSA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Using Designated Routers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Assigning virtual links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Using track ports and track priority with VRRP and VRRP-E. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
VRRP-E differences for basic configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Short Path Forwarding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Enabling preemption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
VCS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Verifying RBridge ID and VCS ID Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
VCS ISL Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Fabric ISL Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Verifying VCS Fabric ISL Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Verifying VCS Fabric ISL Formation (cont.) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
NOS Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
vLAG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
vLAG Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
vLAG Example Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
vLAG Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
LAG Type Brocade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Configuring the vLAG Ignore Split. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
The minimum-links Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Support for Static MAC Pre-Provisioning on vLAG. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
LACP troubleshooting tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
xSTP reconvergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
QOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2013 Brocade Communications Systems, Inc. All Rights Reserved. vii
Brocade Certified Ethernet Fabric Professional in a Nutshell Second Edition
Random Early Discard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Brocade VCS Fabric QoS Restrictions for Layer 3 features in VCS mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Configuring QoS profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Configuring Brocade VCS Fabric QoS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
CEE Map CoS Value. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
FCoE Queuing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Deficit weighted round robin scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Port-Based Policer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Data Center Bridging (DCB) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
DCB Enhancements to Ethernet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
DCBX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
ETS Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
ETS and Class of Service. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Priority Flow Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Strict Priority Scheduling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Data Center Bridging Map Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Defining a priority group table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Defining a Priority-Table Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
FCoE and iSCSI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
FCoE Co-Existence with LAG/vLAG. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Configuring FCoE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Configuring FCoE and LLDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Enable and Disable FCoE Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
FCoE Show Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
iSCSI TLV Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Assigning an FCoE map onto a LAG member . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Zone Merging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Zone merging scenarios: Default access mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
The show name-server detail command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
FC Bridging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
LSAN naming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Brocade VCS Fabric and Brocade Fibre Channel SAN Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Principal Routing Bridge Availability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
FC-FC Routing to Bridge FCoE and FC Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Managing domain IDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
FC to FC Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Brocade VDX Switch TRILL Termination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Edge to BB Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
AMPP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
VM Mobility Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Solution Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Port Profile Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
vCenter Integration Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
vCenter Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
viii 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Brocade Certified Ethernet Fabric Professional in a Nutshell Second Edition
NOS vCenter Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Life of a Port-Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Configuring FCoE Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Management and Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
sFlow Global Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
SNMP community strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Collecting Support Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
Login Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
TACACS+ Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
RBAC Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
DH-CHAP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Configuring Security Profiles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
ACL Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
ACL Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Firmware Download. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Automatic firmware synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Firmware Upgrade Rules for vLAGs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Firmware Download Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Troubleshooting Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
FRU Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Display the SFP information for a Specific Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Verifying LACPDUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Ping Failure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
BNA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Brocade Network Advisor v11.3 Enhancements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
VDX Additional Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Virtual IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
Selecting the MTU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
Registered State Change Notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Fabric Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
show firmwaredownloadhistory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
show port-channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
oscmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Taking the Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
2013 Brocade Communications Systems, Inc. All Rights Reserved. ix
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
List of Figures
VCS Fabric Use Case: Aggregation Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
VCS Fabric Use Case: Combined Access/Aggregation Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Brocades VDX 6720 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Brocade VDX 6720-24 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Brocade VDX 6720-60 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Brocade VDX 6710-54 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Brocade VDX 6710-54 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Brocade VDX 6730-32 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Brocade VDX 6730-32 Ports Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Brocade VDX 6730-76 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
VDX 8770-8 Switch Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
VDX 8770-4 Switch Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
VDX 8770 Port Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15
VDX 8770 Management Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
TRILL Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Layer 2 Adjacency Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22
FSPF for ECMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24
ECMP Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
VCS Fabric L2 Multi-Destination Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
Unicast Ethernet Frame Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
Unicast Ethernet Frame with TRILL Header Added . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
TRILL Ethernet Frame: Link Transport Header Added . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
TRILL Ethernet Frame: Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Ethernet Frame: End of Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
VCS Multicast Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32
IGMP Snooping in a VCS Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
VCS Fabric Edge Loop Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
VCS Fabric L3 Routing Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38
The virtual servers are dynamically moved between Host Server 1 and Host Server 2. . . . . . . . . . . . . . . .43
show VCS command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44
show vcs detail command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44
show fabric all command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47
show fabric islports command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
vLAGs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50
vLAGs with MCTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52
vLAG Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53
MAC Pre-Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56
Wrr schedule - two queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60
ETS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64
ETS and Class of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64
FCoE Co-Existence with LAG/vLAG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .68
show fcoe interface brief command example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70
Default Access mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73
Brocade VCS Fabric to Brocade FC SAN Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75
FC to FC Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77
Brocade VDX Switch Trill Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .78
Edge to BB Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .78
VM Migration Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .79
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
x 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Port Profiles: Distributed Network Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80
Port Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .81
NOS and vCenter Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83
2013 Brocade Communications Systems, Inc. All Rights Reserved. xi
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
List of Tables
Brocade VDX Product Comparisons and Positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Brocade VDX Switch Protocol and Fabric Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
FSPF Key to Figure17 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
xii 2013 Brocade Communications Systems, Inc. All Rights Reserved.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 1
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Brocade VCS

Fabric Technology
Brocade VCS Distributed Intelligence
Distributed Fabric Services
Self-forming fabric (with minimum configuration)
Information shared across all fabric members
Fabric aware of all connected devices
Masterless control
Switch or link failure does not require full fabric reconvergence
Shared port profile information
Automatic Migration of Port Profiles (AMPP)
Enables seamless virtual server migration
Automatic distribution of zoning
Zoning changes are automatically distributed through out the fabric
Brocade VCS

Fabric technology supports a fully distributed control plane. That means every switch is aware
of the entire network topology. You can configure it in a ring, a mesh, or a tree. You can configure enough links
to make it entirely non-blocking or configure the topology to be over subscribed at whatever level you choose.
Every switch knows about every attached device and Virtual Machine (VM).
When a VM is attached to the network, the network configuration is automatically bound to the VM and then
distributed throughout the Brocade VCS Fabric cloud. If the VM moves, the destination port already knows
what to do and the VM is automatically reconnected to all of its old resources. The network configuration
automatically migrates with the VM.
VCS Fabric: Use Case Aggregation Layer
Low cost, highly flexible logical chassis at aggregation layer. See Figure1.
Building block scalability
Per port price of a ToR switch
Availability, reliability, manageability of a chassis
Flexible subscription ratios
Ideal aggregator for 1 Gbps ToR switches
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
2 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Optimized multi-path network
No single point failure
No passive connections
STP not necessary
FIGURE 1 VCS Fabric Use Case: Aggregation Layer
In this use case, Brocade VCS Fabric technology is used as an aggregation technology for 1 GbE server
connections. This design gives the architect a building-block approach to network scalability, leveraging lower-
cost switches at the aggregation layer. This design approach provides the ability to scale by adding another
Brocade VDX

switch into the fabric. Stackable 1 GbE switches are used at ToR, and those switches are
connected into an Ethernet fabric. The Ethernet fabric is very flexible and can be scaled like building blocks.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 3
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Subscription ratios can be anything, including 1:1 through the fabric. This use case example shows a three-
switch stack configuration at the ToR and two 10 GbE links per switch for a total of 6 links to the Brocade VCS
Fabric technology aggregation layer through a single LAG.
This building block design enables you to pay as you grow. To increase port count, simply add a Brocade VDX
switch non-disruptively into the fabric. Since the Brocade VCS fabric looks and acts like a single logical entity,
minimal management is required moving forward. This is an advantage of using Brocade VCS Fabric in the
Access/Aggregation Layers.
VCS Fabric: Use Case Combined Access/Aggregation Layers
Flatter, simpler network design. See Figure2.
- Logical two-tier architecture
- VCS fabrics at the edge
Greater Layer 2 scalability/flexibility
- Increased sphere of VM mobility
- Seamless network expansion
Optimized multi-path network
- All paths are active
- No single point failure
- STP not necessary
Convergence ready
- End-to-end enhanced Ethernet (DCB)
- Multi-hop FCoE support
- iSCSI DCB support

Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
4 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 2 VCS Fabric Use Case: Combined Access/Aggregation Layers
Storage is now introduced to the Ethernet fabric. This use case illustrates end-to-end native multi-hop FCoE
and iSCSI DCB connectivity.
This use case illustrates two ways the fabric can be configured. In this diagram, a ToR Mesh architecture is
used. The benefit is that a true flat network edge, where the switches are connected to its peers. The
challenge of this design is to maintain common subscription ratios between each server (between racks). This
topology is a single stage design.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 5
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Hardware Products and Features
Brocade VDX Product Comparisons and Positioning
Brocade VDX Switch Protocol and Fabric Support
The Brocade VDX 6710, 6730, and 8770 switches support convergence through iSCSI, FCoE, and FC protocol
TABLE 1 Brocade VDX Product Comparisons and Positioning
Brocade VDX 6710 Brocade VDX 6720 Brocade VDX 6730
Brocade VDX 8770-4
Brocade VDX 8770-8
Physical Description
1U
6 x 10 GbE SFP+
and 48 x 1 GbE
GE copper
2U
24 x 10 GbE SFP+
ports
60 x 10 GbE SFP+
ports
2U
24 x 10 GbE
SFP+ and 8 x 8
Gbps FC ports
60 x 10 GbE
SFP+ ports and
16 x 8 Gbps FC
ports
8770-4: 8U
8770-8: 15U
48 x 1 GbE line
card provides up to
48 SFP/SFP-cop-
per ports
48 x 10 GbE line
card provides up to
48 SFP+ ports
12 x 40 GbE line
card provides up to
12 40 GbE QSFP
ports
Licenses Brocade VCS Fabric
License
POD License
Brocade VCS Fabric
License
FCoE License
POD License
Brocade VCS
Fabric License
FCoE License
Brocade VCS
Fabric License
FCoE License
Layer 3
Advanced Services
Positioning High Performance,
low latency, enabled
1 Gbps switch provid-
ing cost- effective
connectivity to 1G
servers
High performance, low
latency, enabled 10 GbE
switch targeted at
virtual data centers.
Enables end- to-end
LAN/SAN convergence
solutions with VCS fabric
High Performance, low
latency enabled
convergence switch
with FC connectivity
High Performance, low
latency enabled
modular convergence
switch with FC connec-
tivity
TABLE 2 Brocade VDX Switch Protocol and Fabric Support
Product iSCSI Ethernet FCoE FC VCS Fabric
Brocade VDX 6710
Yes
1
Yes
No
2
No
Yes
3
Brocade VDX 6720 Yes Yes Yes No Yes
Brocade VDX 6730 Yes Yes Yes
Yes
4
Yes
Brocade VDX 8770 Yes Yes Yes
No
Yes
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
6 2013 Brocade Communications Systems, Inc. All Rights Reserved.
support and also are part of a VCS fabric. The VDX 6710 is a Ethernet switch that can be part of a VCS fabric.
Note 1: The 1GbE ports do not support DCB so the iSCSI TLV is not supported however iSCSI traffic can still go
across these ports. iSCSI TLV is supported on the 10 GbE ports.
Note 2: Cannot attach an FCoE device to a port on this switch however FCoE traffic can flow through the 10 G
(Trill) ports as long as the destination is not on that switch. As the switch would look at the destination RB and
modify the outer header with next hop information so the switch would never look at the type of traffic in that
frame.
Note 3: The 1 GbE ports can not be used as ISL connections to other switches in the fabric.
Note 4: The FC ports can only be used as E_Port (Brocade VDX switch) to EX_Port (SAN router) connection.
Cannot directly attach a FC end device to any of the FC ports on the Brocade VDX 6730.
Brocade VDX 6720 Data Center Switches Overview
Built for the Virtual Data Center. See Figure3.
- Uses Brocade fabric switching eAnvil2 ASIC
- Supports Brocade Network Operating System (NOS) including VCS technology
Performance and Density
- 24- and 60-port models (VDX 6720-24 and VDX 6720-60)
- Ports on Demand (POD) enables 24 to 60 port configurations
- 600ns latency for 24 port and 1.8usecs for 60 port (when spanning port groups)
Configuration Flexibility
- 1 Gbps or 10 Gbps supported on every port
- Front-to-back or back-to-front airflow
Enables Network Convergence
- Complete FCoE support including multi-hop (license required)
- iSCSI Data Center Bridging support (DCB)
2013 Brocade Communications Systems, Inc. All Rights Reserved. 7
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 3 Brocades VDX 6720
VDX 6720-24 platform contains one eAnvil2 ASIC. Each eAnvil2 ASIC supports a Routing Engine (RTE) and
each RTE supports 24 ports, 1 or 10 GbE. Every packet coming in through the external SFP+ interfaces
destined to external ports are switched in a single stage within the eAnvil2 ASIC.
Port groups are shown in Figure4.
FIGURE 4 Brocade VDX 6720-24 Port Groups
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
8 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Brocade VDX 6720-60 Data Flow
The Brocade VDX 6720-60 platform contains nine eAnvil2 ASICs:
Six front end eAnvil2 ASICs
Each provides 10 external SFP+ interfaces
Switching bandwidth from switch backplane (data rate at full duplex): 6720-60: 1200 Gbps
Forwarding bandwidth from ports (data rate at full duplex):6720-60: 896 Mpps
Port groups are shown in Figure5. .
FIGURE 5 Brocade VDX 6720-60 Port Groups
Brocade VDX 6710-54 Data Center Switch
The Brocade VDX 6710-54 is a low-cost Ethernet switch that is VCS capable. Uses Brocade fabric switching
eAnvil2 ASIC and supports Brocade Network Operating System (NOS) including Brocade VCS Fabric
technology. See Figure6.
54 total ports
6 x 10 GbE DCB capable optical ports.
The 10 GbE ports are designed to be used as uplink ports to the VCS fabric however do support the direct
attachment of 10 GbE devices. FCoE direct attached devices are not supported on this switch.
48 x 1 GbE copper ports
2013 Brocade Communications Systems, Inc. All Rights Reserved. 9
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 6 Brocade VDX 6710-54
Port groups are shown in Figure7.
Over subscription: There is no over subscription. From the Main CPU there is a connection to each of the three
ASICs. Shown is only the connection to one of the eAnvil2 ASICs.
FIGURE 7 Brocade VDX 6710-54 Port Groups
Brocade VDX 6730-32 Data Center Switch Overview
Uses Brocade fabric switching eAnvil2 ASIC. See Figure8.
Supports Brocade Network Operating System (NOS) including Brocade VCS Fabric technology
Configuration flexibility: Ports on Demand (POD) enables ports to be added. Default is 16 ports enabled with a
single 8 POD license available to support a total of 32 ports
32 total ports
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
10 2013 Brocade Communications Systems, Inc. All Rights Reserved.
8 x 8 Gbps FC ports
24 x 10 GbE DCB capable ports. Twinax and SFP optical connectivity options
Front-to-back or back-to-front airflow
Supports Network Convergence: Complete FCoE support including multi-hop (license required) and iSCSI Data
Center Bridging support (DCB).
FIGURE 8 Brocade VDX 6730-32
Brocade VDX 6730-32 Data Flow
As shown above, the Brocade VDX 6730-32 platform contains one eAnvil2 and one Condor2 ASIC. The eAnvil2
ASIC support 1 RTE (Routing Engine) and each RTE supports 24, 10/1 Gbps Ethernet Ports. So every packet
coming in through the external SFP+ interfaces destined to external ports are switched in a single stage within
the eAnvil2 ASIC.
Port groups are shown in Figure9.
There is no over subscription for network traffic. FC Traffic over subscription worst case would be 240:64
(15:4)
240: 24 x 10 GbE FCoE devices
64: 8 x 8 Gbps FC traffic between the eAnvil2 ASIC and the Condor 2
2013 Brocade Communications Systems, Inc. All Rights Reserved. 11
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 9 Brocade VDX 6730-32 Ports Groups
Brocade VDX 6730-76 Data Center Switch Overview
Uses Brocade fabric switching eAnvil2 ASIC.
Supports Brocade Network Operating System (NOS) including Brocade VCS Fabric technology
Configuration flexibility: Ports on Demand (POD) enables ports to be added. Default is 40 ports enabled with
two 10 POD licenses available to support a total of 60 GE ports
76 total ports
16 x 8 Gbps FC ports
60 x 10 GbE DCB capable ports: Twinax and SFP optical connectivity options
Supports Network Convergence: Complete FCoE support including multi-hop (license required) and iSCSI Data
Center Bridging support (DCB)
Port groups are shown in Figure10.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
12 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 10 Brocade VDX 6730-76 Port Groups
Brocade VDX 8770-8 and VDX 8770-4 Data Center Switch Overview
Two models are available: 4 and 8 line card (LC) switch models
LCs are available in 1, 10 and 40 Gbps models
In 8-slot switch, up to: See Figure11
- 384 x 10 GbE or 1 GbE user ports
- 96 x 40 GbE user ports
2013 Brocade Communications Systems, Inc. All Rights Reserved. 13
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 11 VDX 8770-8 Switch Design
In 4-slot switch, up to: See Figure12
92 x 10 GbE or 1 GbE user ports
48 x 40 GbE user ports
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
14 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 12 VDX 8770-4 Switch Design
In 4-slot switch, up to:
192 x 10 GbE or 1 GbE user ports
48 x 40 GbE user ports
- Fully redundant Management Modules (MM), Switch Fabric Modules (SFM) and PSUs
- 4 microsecond (s) latency from any port to any port
- 100 GbE/SDN-ready and 4 Tbps-capable backplane to support future releases
- VCS mode support only (no Standalone mode)
VDX 8770 Port Numbering
Brocade VDX 8770-8
Ports are numbered from 1 through 48, from top to bottom, with the odd-numbered ports on the right row and
the even-numbered ports on the left row when installed in the switch.
Brocade VDX 8770-4:
Ports are numbered from 1 through 48, from left to right, with the odd-numbered ports on the upper row and
the even-numbered ports on the lower row when installed in the switch. See Figure13
2013 Brocade Communications Systems, Inc. All Rights Reserved. 15
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 13 VDX 8770 Port Numbering
For both the VDX 8770-4 and VDX 8770-8:
ISL Trunking is supported only on the 48x10GbE line card.
The port groups can consist of up to 8 ports.
The octet groups are ports 1-8, 9-16, 17-24, 25-32, 33-40, and 41-48.
VDX 8770 Interface Nomenclature
The 10GE SFP+ interfaces in LC48x10G line card are named TenGigabitEthernet or in short TE
The 40GE QSFP interfaces in the LC12x40G line card are named as FortyGigabitEthernet or in short FO
The 1GE SFP+ interfaces in LC48x1G line card are named GigabitEthernet or in short GE
VDX 8770 Switch Fabric Module (SFM)
SFMs contain 3 x Condor3 ASICs for switching packets between LCs
Hot swappable SFM modules provides resiliency
- VDX 8770-8 is N+1 with loss of one SFM
- VDX 8770-4 is not N+1; if one SFM is lost
There are message link ports between the line card Hawks and the SFMs:
- For VDX 8770-8, S3 and S4 have to be populated with at least one SFM
- For VDX 8770-4, S1 and S2 have to be populated with at least one SFM
The SFM blade has an early-removal indicator that enables the software to re-route the data traffic flowing
through the SFM to the other SFMs in the system thereby providing a zero-frame-drop performance during
SFM hot-plug-out. Based on the available bandwidth, the user may or may not experience a small percentage
loss.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
16 2013 Brocade Communications Systems, Inc. All Rights Reserved.
VDX 8770 Line Cards
SFP+ and QSFP media interfaces
3 Hawk ASICs provide the traffic manager functionality
6 Wolverine ASICs provide packet processor functionality
Local CPU to offload MM and provide distributed processing
No local switching, all traffic is sent to SFMs for switching by Condor3 ASICs Brocade VDX 8770-8
Slot Configuration
Interface modules are registered with the system by type, and the slot must be configured with the correct
type before you can install an interface module in that slot. When you install a new interface module, the
system checks whether or not a previous configuration is associated with the slot. The following rules apply
when you install or replace an interface module: When you install an interface module and boot it up to an
online state in a slot that was never occupied or configured, the module type information is automatically
detected and saved to the database. No special configuration is required. If you install an interface module in
a slot that was previously occupied by an interface module of the same type and the slot is configured for that
same type, you can hot-swap the modules without powering off the interface modules. No slot configuration
changes are required.
If the slot was previously configured for a different type of interface module, the installation fails and the
module is faulted with a "Type mismatch" error. A RASlog error message is generated. You must power off the
interface module and clear the slot configuration with the no linecard command before you can configure the
slot for a new interface module.
VDX8770 CID Card
2 Cards - (reads are from CID 1 unless problems occur, writes are to both)
Located behind fan 2 on VDX8770-4, fan 1 on VDX8770-8
Critical data sets (cannot be written by NOS)
A header with the CID part number, serial number, and other data about the CID card
Chassis PN/SN
License ID/WWN
Non-critical data sets (can be written by NOS)
FRU history log (not verified)
Table with management IPs, switch name
Power off list for line cards
DCE MAC data containing switch MAC addresses
2013 Brocade Communications Systems, Inc. All Rights Reserved. 17
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
VDX8770 Management Module (MM) See Figure14
Handles management and control plane functions for switch fabric modules, line cards and all other FRUs
MMs are redundant, with one active and one standby MM
NOS 3.0 requires cold reboot of standby MM during failover and firmware upgrades
Connections:
10 Gbps Ethernet connection between the two MMs
1 Gbps Ethernet connection between MM and LCs
PCIe connection between MM, proprietary buses and switch fabric modules
FIGURE 14 VDX 8770 Management Module
Long Distance ISL Support
With NOS v2.1.1 or higher, the maximum ISL distance is 10 km
Supported on the Brocade VDX 6720-60 and 6730-76 switches
All other Brocade switches support a maximum distance of 200 m
Use the long-distance-isl command to specify one of these distances:
200 m (default)
2000 m
5000 m
10000 m
Here is an example:
VDX11(conf-if-te-11/0/14)# long-distance-isl ?
Possible completions:
2000 2000 meter distance link (Warning: It May disable other ISLs in the port
group)
5000 5000 meter distance link (Warning: It May disable other ISLs in the
port group)
10000 10,000 meter distance link (Warning: It May disable other ISLs in the
port group)
Additional notes:
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
18 2013 Brocade Communications Systems, Inc. All Rights Reserved.
For a 10 km ISL link, no other ISL links are allowed on the same eAnvil2 ASIC.
For 2 km and 5 km ISL links, another short distance ISL link can be configured.
A maximum of 3 PFCs (per priority flow control) can be supported on a long distance ISL link. Use the no
long-distance-isl command to revert to the default value which is 200 m. For more information refer to
the Brocade Network OS Administrators Guide Supporting Network OS v2.1.1, 53-1002491.
Brocade VDX Licenses
VCS Fabric License
If the Brocade VCS Fabric does not exceed two Brocade VDX switches, no VCS Fabric license is required.
(There is one exception if the other node has an Advanced Service bundle, see below for more details)
A two switch Brocade VCS Fabric license is built into NOS
If there are more than two nodes in the fabric, each switch requires a VCS Fabric license
FCoE License
Required to support FCoE on Brocade VDX 6720, 6730 and 8770 switches.
Required to activate FC ports on Brocade VDX 6730-32 and 6730-76 switches.
An FCoE license enables FCoE on the Brocade VDX platforms. Since FCoE requires a VCS fabric, a VCS Fabric
license is a prerequisite for enabling FCoE in a fabric with more than two switches. In addition, a separate
FCoE license is required to enable FCoE on a VCS edge port. Without an FCoE license, FCoE logins are not
permitted, and all FCoE commands with the exception of a few commands return an error of No FCoE
license present when executed. See release notes for more information on FC port activation and the
FCoE license.
Ports on Demand (POD) License
Brocade VDX 6720-24 and 6730-32: One 8-port POD license
Base switch has 16 ports enabled
Brocade VDX 6720-60 and 6730-76: Two 10-port POD licenses
Base switch has 40 ports enabled
NOS v3.0.0 Licensing
Supports all NOS 2.0/2.1/2.1.1-supported VDX 67XX platforms using same license rules for legacy
features
Supports new chassis-based platforms (VDX 8770)
2013 Brocade Communications Systems, Inc. All Rights Reserved. 19
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Legacy licensable features:
- FCoE
- Ports on Demand (POD1 and POD2) - not on VDX 6710 or VDX 8770
- VCS Fabric
New licensable features in NOS v3.0.0
- Layer 3 - only supported in VCS mode, no license required on VDX 67XX platforms
- Advanced Services - (FCoE + VCS Fabric + Layer 3 bundle) - only on VDX 8770
Adding the Advanced Services bundle, which includes the VCS Fabric license, requires the other
node in a 2-node fabric to have a VCS license
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
20 2013 Brocade Communications Systems, Inc. All Rights Reserved.
VCS Fabric
Brocade VCS Fabric technology leverages proven Fibre Channel fabric protocols to build a TRILL-based fabric.
TRILL Defined
Transparent Interconnection of Lots of Links (TRILL) provides a solution for shortest path frame routing for
multihop Layer 2 Ethernet that:
Supports arbitrary topologies (ring, mesh, star)
Uses link-state routing protocols
- The link-state protocol is performed by every switching node in the network. The basic concept of link-
state routing is that every node constructs a map of the connectivity to the network, in the form of a
graph, showing which nodes are connected to which other nodes. Each node then independently
calculates the next best logical path from it to every possible destination in the network. The collection
of best paths will then form the node's routing table.
- Brocade uses Fabric Shortest Path First (FSPF) as the routing protocol
- TRILL is currently a proposed Internet Engineering Task Force (IETF) standard and is an Equal-Cost
Multi-Path (ECMP)-capable protocol. Equal-cost multi-path routing (ECMP) is a routing strategy where
next-hop packet forwarding to a single destination can occur over multiple "best paths" which tie for
top place in routing metric calculations. Multipath routing can be used in conjunction with most
routing protocols, since it is a per-hop decision that is limited to a single router. It potentially offers
substantial increases in bandwidth by load-balancing traffic over multiple paths
TRILL uses a link state-based control plane to form loop-free optimized paths between a source and
destination. TRILL provides the following features:
Minimal configuration required
Load balancing among multiple paths
Forwarding loop mitigation without the need for STP
Support of multiple points of attachment to the TRILL network. In STP, a single node with multiple
attachments to a single spanning tree segment always receives and sends traffic over only one of the
those attachment points. TRILL must manage all traffic, including multicast and broadcast traffic, so as
not to create traffic loops involving Ethernet segments with multiple TRILL attachment points. This
includes multiple attachments to a single TRILL node and attachments to multiple TRILL nodes. Support
for multiple attachments can improve support for forms of mobility that induce topology changes
Support for broadcast and multicast
Behaves as a normal bridge to devices outside the TRILL network
TRILL Frames
TRILL encapsulates an Ethernet frame to specify information to route the frame through the network
(Standard frames as well as 2.5 k Mini Jumbo frames are also supported). See Figure15.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 21
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Link transport header: A header that is added and removed at each hop and is used to specify the next
hop RBridge and the transmitting RBridge This is also known as the outer MAC header
TRILL header: Specifies nickname values of the egress and ingress RBridges, unless the frame is multi-
destined, in which case the egress nick.name specifies the root of the distribution tree on which the
frame is being sent. The egress nickname is Equivalent to the RBridge ID.
FIGURE 15 TRILL Frame
Hop Count
The Hop Count field is a 6-bit unsigned integer. An RBridge drops frames received with a hop count of zero,
otherwise it decrements the hop count. (This behavior is different from IPv4 and IPv6 in order to support the
later addition of a trace route-like facility that would be able to get a hop count exceeded from an egress
RBridge.)
For known unicast frames, the ingress RBridge should set the hop count in excess of the number of RBridge
hops it expects to the egress RBridge to allow for alternate routing later in the path. For multi-destination
frames, the hop count should be set by the ingress RBridge to at least the expected number of hops to the
most distant RBridge.
To accomplish this, RBridge RBn calculates, for each branch from RBn of the specified distribution tree rooted
at RBi, the maximum number of hops in that branch. Multi-destination frames are of particular danger
because a loop involving one or more distribution tree forks could result in the rapid generation of multiple
copies of the frame, even with the normal TTL mechanism. It is for this reason that multi-destination frames
are subject to a stringent Reverse Path Forwarding Check and other checks. As an optional additional traffic
control measure, when forwarding a multi-destination frame onto a distribution tree branch, transit RBridge
RBn may decrease the hop count by more than 1, unless decreasing the hop count by more than 1 would
result in a hop count insufficient to reach all destinations in that branch of the tree rooted at RBi. Using a hop
count close or equal to the minimum needed on multi-destination frames provides additional protection
against problems with temporary loops when forwarding. Although the RBridge MAY decrease the hop count
of multi-destination frames by more than 1, under the circumstances described above, the RBridge
forwarding a frame must decrease the hop count by at least 1, and discards the frame if it cannot do so
because the hop count is 0. The option to decrease the hop count by more than 1 under the circumstances
described above applies only to multi-destination frames, not to known unicast frames.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
22 2013 Brocade Communications Systems, Inc. All Rights Reserved.
VCS Fabric Formation
Brocade VCS Fabric technology leverages proven Fibre Channel fabric protocols to build a TRILL-based fabric.
The main functions of the fabric formation protocols are:
Confirming that each switch in the VCS fabric is assigned the same VCS ID
Confirming that each switch in the VCS fabric is assigned a unique RBridge ID
Create a network topology database using a link state routing protocol (FSPF)
Compute a broadcast tree to distribute fabric broadcast and multicast traffic
Before connecting VDX switches, the following should be configured:
Set the VCS Fabric ID
Set the RBridge ID
Enable VCS Fabric mode
Automatic Layer 2 Adjacency Formation
Brocade VCS Fabric forms adjacencies with its directly connected Brocade VCS Fabric-enabled neighboring
switches. See Figure16.
These adjacencies are called Inter Switch Links (ISLs)
The ISLs can be separate links or Brocade proprietary VCS Fabric trunks
FIGURE 16 Layer 2 Adjacency Formation
2013 Brocade Communications Systems, Inc. All Rights Reserved. 23
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
RBridge ID Conflicts
If there is a conflict for an RBridge ID, one of the offending switches must have their RBridge ID changed
Valid RBridge IDs are 1-239
An RBridge ID change requires a reboot of the switch.The switch needs to be rebooted because an
RBridge ID change requires a reset to all configuration databases to accommodate the new RBridge ID
VCS Edge Port Configuration
VCS fabric edge ports are switch ports used to connect to external devices including end-stations or non-VCS
Fabric mode switches or routers.
Standard Layer 2 configuration options are available (i.e. VLANs and LAGs)
STP is not configurable on edge ports. STP is not configurable on edge ports, but VCS will pass BPDU
traffic through the fabric. STP sees the fabric as a wire.
When an edge port converts to an ISL port, the edge port configuration is ignored. Similarly, when an ISL
port converts to an edge port, all the user provisioned edge port configuration for that port is activated.
LACP can be configured on edge ports only with either Brocade or Standard types. If Brocade is chosen, so
that Link Reset (LR) primitives are exchanged properly, make sure the edge peering device is a Brocade
Converged Network Adapter (CNA), a standalone Brocade VDX switch, or a Brocade 8000 switch.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
24 2013 Brocade Communications Systems, Inc. All Rights Reserved.
L2 Forwarding
FSPF Protocol for ECMP
Brocade VCS Fabrics uses the FSPF routing protocol to distribute link-state information of all ISLs. FSPF is a
Link State Path Selection protocol, similar to OSPF, which is an Interior Gateway Protocol (IGP) widely used in
IP networks. FSPF keeps track of the state of the links on all switches in the Fabric. It also associates a cost
with each link. The protocol computes paths from a switch to all the other switches in the fabric, by adding the
cost of all the links traversed by the path, and choosing the path that minimizes the cost. See Figure17.
FSPF is similar to Layer 3 routing protocols like OSPF. Although it has roots from OSPF, FSPF only defines and
implements point-to-point links. In other words, there is no concept of a designated router (DR) and a backup
designated router (BDR), areas or summarization, or anything similar like that being managed in FSPF.
FSPF forms a single adjacency per fabric trunk.
FIGURE 17 FSPF for ECMP
TABLE 3 FSPF Key to Figure17
From RB3
Destination RB Outgoing Interface
RB1 te 3/0/1, te 3/0/3
RB2 te 3/0/1, te 3/0/3
RB4 te 3/0/1, te 3/0/3
RB5 te 3/0/1
RB6 te 3/0/3
2013 Brocade Communications Systems, Inc. All Rights Reserved. 25
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
VCS Layer 2 ECMP
With NOS v2.0.0a, up to 8 ECMP paths are supported per switch. When multiple ECMP paths exist, the traffic
is load balanced across all available equal cost paths based on a hash from fields in the frames. The hash
algorithm can be changed using the fabric ecmp load-balance command. For more information see
the.Network OS Administrators Guide Supporting Network OS v2.1.1 manual. See Figure18.
IP: MAC DA, MAC SA, VLAN, IP DA, IP SA, TCP/UDP ports
FCoE: Input port, MAC DA, MAC SA, VLAN, D_ID, S_ID, OX_ID
Other: MAC DA, MAC SA, VLAN
FIGURE 18 ECMP Paths
VCS Fabric L2 Multi-Destination Tree
VCS Fabric technology uses FSPF to calculate a loop-free multi-destination tree root switch. The multi-
destination tree is calculated after the unicast routes are computed. The following rules determine the multi-
destination root RBridge:
RBridge ID with the highest priority. The default multicast RBridge priority is zero. The default can be
changed using the fabric route mcast RBridge ID <RB-ID> priority <priority> command. The priority range
is 0 through 255.
Lowest RBridge ID
When multiple links are available between two nodes in the multicast tree The one with the highest BW at that
instant is selected. Like if there is a ISL with 40G and a ISL with 20G then the 40G is chosen. If there are more
than one equal bandwidth ISLs, then the port on the lowest switch WWN and lowest port will be chosen. The
following rules determine which down stream links will be selected.
Highest bandwidth
Lowest WWN - lowest port on that switch
In Figure19, RB5 is not the lowest RBridge number but it has been assigned the highest priority
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
26 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 19 VCS Fabric L2 Multi-Destination Tree
Multi-cast Tree Changes
The following scenarios cause changes to multi-destination tree:
Root failure or removal from the VCS fabric
All possible primary links to the neighbor RB closest to the multi-destination root are down
Increasing the priority of an existing RBridge to be the highest priority will cause the tree to change/
rebuild
Data Path
Traffic Types
The data path through the fabric changes depending on the type of traffic. Traffic is classified into two distinct
types:
Known unicast (destination MAC is known)
Broadcast, unknown unicast, or multicast
VCS Known Unicast Data Path
For a known unicast frame, no flooding occurs when the one of the following occurs:
MAC has been previously learned
RBridges know the RB that is hosting the MAC
FSPF knows the shortest path to get to the destination RBridge because the routing topology has already
been created
2013 Brocade Communications Systems, Inc. All Rights Reserved. 27
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Learning of a source MAC to an ingress RBridge is done through the data plane. The ingress RBridge
encapsulates the native Ethernet frame with TRILL headers until the frame reaches the destination RBridge.
The outer MAC header is changed at every hop. The frame enters the fabric through the ingress RB (RB1). The
source MAC (ES_A) is learned by RB1. See Figure20.
FIGURE 20 Unicast Ethernet Frame Example
The ES_B is a known MAC and RB1 knows that MAC ES_B is hosted by RB2
RB1 adds a TRILL header with a egress RB nickname of RB2. See Figure21.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
28 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 21 Unicast Ethernet Frame with TRILL Header Added
RB1 uses the FSPF routing table to determine the path to RB2. See Figure22.
RB1 adds the link transport header which specifies:
- The next hop RB for the path (RB3)
- Itself as the Outer MAC SA
2013 Brocade Communications Systems, Inc. All Rights Reserved. 29
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 22 TRILL Ethernet Frame: Link Transport Header Added
As a transit switch, RB3 removes the previous link transport header and replaces it with one that specifies
the next hop RB. See Figure23.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
30 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 23 TRILL Ethernet Frame: Data Path
The frame arrives at the egress RB, RB2
RB2 removes the TRILL header and delivers the frame to the end station. See Figure23.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 31
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 24 Ethernet Frame: End of Data Path
VCS Multicast Data Path
Unknown MAC, broadcast, or multicast traffic enters the fabric
Traffic is flooded and sent to multicast root. See Figure25.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
32 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 25 VCS Multicast Data Path
IGMP Snooping in a VCS Fabric
IGMP snooping is the process of listening to Internet Group Management Protocol (IGMP) network traffic.
IGMP snooping, as implied by the name, is a feature that allows a Layer 2 switch to listen in on the IGMP
conversation between hosts and routers. By listening to these conversations the switch maintains a map of
links that need specific IP multicast streams. Multicasts may then be sent only to the links that need them.All
IGMP snooped traffic rides on the fabric multicast tree. See Figure26.
All multicast traffic known and unknown is sent to all RBridges through the multicast tree
All unknown multicast groups are flooded devices attached to the fabric
If traffic for a known multicast group does not have any receivers on that RBridge then the traffic is
dropped
2013 Brocade Communications Systems, Inc. All Rights Reserved. 33
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 26 IGMP Snooping in a VCS Fabric
VCS Fabric Edge Loop Detection
Edge-loop detection (ELD) detects and disables loops that would cause broadcast storms. These loops are
generally caused by misconfigurations. See Figure27.
ELD is configured and enabled on Brocade VCS Fabrics
ELD is not supported on switches in standalone mode
ELD requires configuration at the global and interface levels
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
34 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 27 VCS Fabric Edge Loop Detection
How Edge Loop Detection Works
ELD works by multicasting Protocol Data Unit (PDU) packets on edge ports. If a port is blocked by STP or
another L2 protocol, ELD pdus are not processed or flooded on that port. A RBridge recognizes a loop when it
receives a PDU that it initiated. If an ELD pdu is received on a port/VLAN that is not configured with ELD, the
pdu will be dropped. Once an RBridge recognizes that a loop exists, it shuts down a port and breaks the loop.
In the case of LAG ports, ELD pdu are sent out on the active primary port of a lag group. ELD protocol will
shut down all the member ports of the lag if an ELD pdu is received on any member port.
In case of vLAG ports, ELD pdu are sent out on the active primary port of the vLAG. ELD protocol will shut
down all the member ports of the vLAG on the RBridge receiving the pdu. If there are other member ports
exist on any other RBridge, they will still be up. This way loop will be broken and connectivity will still be
maintained.
VLAN Configuration Guidelines and Restrictions
Follow these guidelines and restrictions when configuring VLANs:
2013 Brocade Communications Systems, Inc. All Rights Reserved. 35
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
In an active topology, MAC addresses can be learned, per VLAN, using Independent VLAN Learning (IVL) only.
A MAC address ACL always overrides a static MAC address entry. In this case, the MAC address is the
forwarding address and the forwarding entry can be overwritten by the ACL. The Brocade DCB switch supports
Ethernet DIX frames and 802.2 LLC SNAP encapsulated frames only. You must configure the same native
VLAN on both ends of an 802.1q trunk link. Failure to do so can cause bridging loops and VLAN leaks. All
switches in a Brocade VCS Fabric cluster must be configured with the same VLAN number.
The default VLAN id is 1.
VCS Fabric and STP
In VCS mode the VDX switch operates in transparent mode with respect to spanning tree.
STP BPDUs are tunneled across the multicast tree
All edge switches see each others BPDU
Meaning the VCS Fabric will just forward the BPDU on edge ports
Enabling Port Fast (STP)
From the DCB interface, use this command to enable port fast on an interface to allow the interface to quickly
transition to the forwarding state. Port fast immediately puts the interface into the forwarding state without
having to wait for the standard forward time.
If you enable the portfast bpdu-guard option on an interface and the interface receives a BPDU,
thesoftware disables the interface and puts the interface in the ERR_DISABLE state.
BUM Storm Control
A traffic storm occurs when packets flood the LAN, creating excessive traffic and degrading network
performance. Broadcast, unicast and unknown multicast (BUM) storm control can prevent disruptions on
Layer 2 physical ports. BUM storm control allows you to limit the amount of broadcast, unknown unicast, and
multicast ingress traffic on a specified interface or on the entire system. All traffic received in excess of the
configured rate gets discarded. You also have the option to specify whether to shutdown an interface if the
maximum defined rate is exceeded within a five-second sampling period. When a port is shutdown, you
receive a log message. You must then manually re-enable the interface using the no shut command.
Considerations
BUM storm control is supported in VCS or standalone mode.
BUM storm control replaces the multicast rate-limit feature for VDX 8770-4 and 8770-8, and later
platforms. This command is not supported on VDX 6XXX modules, such as the VDX 6710, 6720, and
6730.
BUM storm control must be configured on one of the following physical interface:
- tengigabitethernet
- gigabitethernet
- fortygigabitethernet
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
36 2013 Brocade Communications Systems, Inc. All Rights Reserved.
L3: Routing
VCS Fabric - Layer 3 Routing
L3 functionality in a VCS Fabric operates on an individual switch basis as a separate router
L3 commands need to be issued on a per switch basis
Routing is enabled on an a per RBridge basis
From a routing perspective VCS provides a flat L2 topology
- Meaning that all routers enabled on a given VLAN appear as if they are on the same subnet
VCS Fabric L3 Routing Example
In the example, See Figure28 we have a 3 node VCS Fabric. Only RB1 has L3 routing enabled. RB2 and RB3
are acting as L2 switches. There are two VLANs active. VLAN 10 (red) and VLAN 20 (blue). The router (RB1)
has Ves for VLAN 1 and 2 configured.
We assume that ARP has already been resolved and H1 knows how to reach H2 via RB1. The router RB1 has
proxy-arp turned on so that it can respond to ARP requests for either VLAN 10 or 20 on any interface that has
routing enabled.
H1 now wants to talk to H2.
Stage 1:
Before the packet enters the VCS Fabric notice:
The destination MAC address for the packet as M3 (RB1s Ve MAC address)
The IP Dst Addr is that of H2.
As in a regular stand alone switch all Ves within a single RBridge share the same MAC address. This can
be noted by observing that both VLAN 10 and 20 on RB1 have the same MAC address
The packet has an IP TTL of 63
The packet is tagged on VLAN 10 (red)
Stage 2:
The packet first hits RB2 which is a pure L2 switch.
RB2 looks at its L2 forwarding tables and determines that the frame needs to be forwarded to destined
MAC address M3 on VLAN 10 (red)
RB2 will encapsulate the frame in a TRILL frame and forward it along to its next hop (RB1).
The outer header has the destination MAC address of Mc (Te 1/0/1 on RB1)
The outer header has a source MAC address of Ma (Te 2/0/1 on RB2)
The TRILL header has a Egress of RB1 and a Ingress of RB2
The inner header is untouched at this point
Stage 3:
2013 Brocade Communications Systems, Inc. All Rights Reserved. 37
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
At this point the packet has reached RB1
RB1 determines the packet is destined to itself looking at the Egress RBridge address RB1 and the outer
MAC address of Mc.
It decapsulates the packet and looks at the inner VLAN tag of VLAN 10 and inner MAC address M3 and
realizes that M3 is its own (Ve) MAC address as well as the fact that routing is turned on VLAN 10
It looks at the inner packet and performs an IP lookup
Based on the IP lookup it determines the next hop of the packet
It changes the inner destination MAC address to M2 (the address of H2)
It changes the inner source MAC address to M3 (Its own Ve MAC address)
It decrements the TTL by 1
The router also realizes that the destination is in the VCS Fabric
It will now encapsulate this packet into a TRILL header
The TRILL encapsulation will include the egress of RB3 and ingress RB1 (itself)
The outer header has the destination MAC address of Mb which is the MAC address of Te 3/0/1 (on RB3)
The outer header has the source MAC address of Md which is the MAC address for Te 1/0/2 (on RB1)
Stage 4:
At RB3, the switch looks at the outer MAC address Mb and egress RB3 realizes the packet is destined for
itself
It decapsulates the packet and looks at the inner VLAN tag of 20 (blue) and inner destination mac
address M2 and realizes that it needs to do a L2 lookup
Assuming the L2 table is already populated it will know that it needs to forward the packet to host H2
It will forward the decapsulated packet towards H2
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
38 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 28 VCS Fabric L3 Routing Example
How OSPF works in VCS Fabric
Each RBridge running OSPF in VCS Fabric exchanges routes with neighbors
OSPF neighbor can be inside or outside VCS
An exchange between switches within the fabric encapsulates the frame inside a TRILL frame. Within VCS
Fabric, OSPF must be enabled on Ve ports.
OSPF neighbor outside VCS can be connected over:
Ve
Router (edge) port
Port Channel
To create the VLAN used for OSPF the VLAN must be previously created and match the configured Ve
number.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 39
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Assigning OSPF areas
Once OSPF is enabled on the system, you can assign areas. Assign an IP address or number as the area ID for
each area. The area ID is representative of all IP addresses (subnets) on a router port. Each port on a router
can support one area.
An area can be normal, a stub, or a Not-So-Stubby Area (NSSA):
Normal - OSPF routers within a normal area can send and receive External Link State Advertisements
(LSAs).
Stub - OSPF routers within a stub area cannot send or receive External LSAs. In addition, OSPF routers in a
stub area must use a default route to the area's Area Border Router (ABR) or Autonomous System
Boundary Router (ASBR) to send traffic out of the area.
NSSA - The ASBR of an NSSA can import external route information into the area. ASBRs redistribute
(import) external routes into the NSSA as type 7 LSAs. Type-7 External LSAs are a special type of LSA
generated only by ASBRs within an NSSA, and are flooded to all the routers within only that NSSA.
ABRs translate type 7 LSAs into type 5 External LSAs, which can then be flooded throughout the AS. You
can configure summary-addresses on the ABR of an NSSA so that the ABR converts multiple type-7
External LSAs received from the NSSA into a single type-5 External LSA.
When an NSSA contains more than one ABR, OSPF elects one of the ABRs to perform the LSA translation for
NSSA. OSPF elects the ABR with the highest router ID. If the elected ABR becomes unavailable, OSPF
automatically elects the ABR with the next highest router ID to take over translation of LSAs for the NSSA. The
election process for NSSA ABRs is automatic.
Configuring a Summary-Address for the NSSA
If you want the ABR that connects the NSSA to other areas to summarize the routes in the NSSA before
translating them into Type-5 LSAs and flooding them into the other areas, configure a summary-address. The
ABR creates an aggregate value based on the summary-address. The aggregate value becomes the address
that the ABR advertises instead of advertising the individual addresses represented by the aggregate. You can
configure up to 32 ranges in an OSPF area.
To configure a summary-address in NSSA 2.2.2.2 (This example assumes that you have already configured
NSSA 2.2.2.2.):
1. In privileged EXEC mode, issue the configure command to enter global configuration mode.
2. Enter the rbridge-id command followed by the RBridge ID to enter Rbridge sub-configuration mode.
3. Enter the router ospf command to enable OSPF on the router and to enter router configuration mode.
4. Enter the area operand followed by the area ID, then enter the nssa operand followed by the nssa ID.
5. Enter the summary-address command followed by the IP address and mask for the summary route.
switch# configure
switch(config)# rbridge-id 101
switch(config-rbridge-id-101)# router ospf
switch(conf-ospf-router)# area 2.2.2.2 nssa 10
switch(conf-ospf-router)# summary-address 209.157.1.1 255.255.0.0
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
40 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Using Designated Routers
In an OSPF broadcast network, OSPF elects one router to serve as the designated router (DR) and another
router on the segment to act as the backup designated router (BDR). This minimizes the amount of repetitive
information that is forwarded on the network. OSPF forwards all messages to the designated router. Backup
designated routers forward updates throughout the network.
On broadcast networks such as LAN links, all routers on the LAN other than the DR and BDR form full
adjacencies with the DR and BDR and pass LSAs only to them. The DR forwards updates received from one
neighbor on the LAN to all other neighbors on that same LAN. One of the main functions of a DR is to ensure
that all the routers on the same LAN have identical LSDBs. Therefore, on broadcast networks, an LSDB is
synchronized between a DROther (a router that is not a DR or a BDR) and its DR and BDR.
If the DR goes off-line, the BDR automatically becomes the DR. The router with the next highest priority
becomes the new BDR.
If two neighbors share the same priority, the router with the highest router ID is designated as the DR. The
router with the next highest router ID is designated as the BDR.
Assigning virtual links
All ABRs (area border routers) must have either a direct or indirect link to the OSPF backbone area (0.0.0.0 or
0). If an ABR does not have a physical link to the area backbone, the ABR can configure a virtual link to
another router within the same area, which has a physical connection to the area backbone. The path for a
virtual link is through an area shared by the neighbor ABR (router with a physical backbone connection), and
the ABR requires a logical connection to the backbone. Two parameters fields must be defined for all virtual
links-transit area ID and neighbor router:
The transit area ID represents the shared area of the two ABRs and serves as the connection point
between the two routers. This number should match the area ID value.
The neighbor router field is the router ID (IP address) of the router that is physically connected to the
backbone, when assigned from the router interface requiring a logical connection. When assigning the
parameters from the router with the physical connection, be aware that the router ID is the IP address of
the router requiring a logical connection to the backbone.
Overview of Virtual Routers
Virtual router is a collection of physical routers that can use the Virtual Router Redundancy Protocol (VRRP) to
provide redundancy to routers within a LAN. Two or more VRRP-configured routers can create a virtual router.
Each VRRP router can participate in as many as 255 virtual routers per LAN interface.
VRRP eliminates a single point of failure in a static, default-route environment by dynamically assigning virtual
IP routers to participating hosts. The interfaces of all routers in a virtual routermust belong to the same IP
subnet. There is no restriction against reusing a virtual router ID (VRID) with a different address mapping on
different LANs.
The following platforms support VRRP and VRRP-E:
Brocade VDX 6710-54
Brocade VDX 6720
2013 Brocade Communications Systems, Inc. All Rights Reserved. 41
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Brocade VDX 6730
VDX8770-4
VDX8770-8
Supported ports:
For VRRP-FortyGigabitEthernet, TenGigabitEthernet, Gigabitethernet, and ve.
For VRRP-E-ve ports only.
Only IPv4 support is provided. IPv6 and VRRPv3 are not supported
Brocade Supports Two VRRP Protocols:
Standard VRRP-The standard router redundancy protocol, VRRP v2 supports the IPv4
environment. Also, the Brocade version of standard VRRP is compliant with RFC 3768.
VRRP-E (Extended)-A Brocade proprietary protocol similar to standard VRRP that is not
standard compliant and cannot inter-operate with VRRP
Using track ports and track priority with VRRP and VRRP-E
A track port allows you to monitor the state of the interfaces on the other end of a the route path. A
track-port also allows the virtual router to lower its priority if the exit path interface goes down,
allowing another virtual router in the same VRRP (or VRRP-E) group to take over.
Rules
Track priorities must be lower than VRRP/VRRP-E priorities.
The dynamic change of router priority can trigger mastership switchover if preemption is enabled.
However, if the router is an owner (applicable only for VRRP), the mastership switchover will not occur.
Maximum number of interfaces that can be tracked for a virtual router is 16.
Port tracking is allowed only for physical interfaces and port channels
Used to track up-link ports, if they go down the RBridge priority will be reduce to by the amount of the tracking
priority which should result in a new RBridge becoming the master1
Valid priority range is 1-254
VRRP-E differences for basic configuration
The command protocol vrrp enables VRRP-E as well as VRRP. There is no command called protocol-
vrrp-extended.
The group command for VRRP-E is vrrp-extended-group <group-id>.
VRRP-E virtual routers can be configured on ve interfaces only.
Virtual Router Address- The address you are backing up:
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
42 2013 Brocade Communications Systems, Inc. All Rights Reserved.
For VRRP: The virtual router IP address must belong to the same subnet as a real IP address configured
on the VRRP interface, and can be the same as a real IP addresses configured on the VRRP interface. The
virtual router whose virtual IP address is the same as a real IP address is the IP address owner and the
default master.
For VRRP-E: The virtual router IP address must belong to the same subnet as a real IP address configured
on the VRRP-E interface, but cannot be the same as a real IP address configured on the VRRP-E interface
Short Path Forwarding
VRRP-E is enhanced with the VRRP-E extension for Server Virtualization feature so that Brocade devices
attempt to bypass the VRRP-E master router and directly forward packets to their destination through
interfaces on the backup router. This is called short-path forwarding. A backup router participates in a VRRP-E
session only when short-path-forwarding is enabled. VRRP-E active-active load-balancing is achieved with
ingress RBridge, by hashing either the L2-7 header information (VDX8770) or the destination MAC address
(VDX67xx) to determine the path. All nodes in the VCS are aware of all VRRP-E sessions and the participating
RBridges in each session.
If short-path forwarding is enabled, traffic travels through the short-path forwarding path to reach the client.
Any packets coming from the local subnet of the virtual IP address are routed to the VRRP-E master router.
See Figure29
VRRPe feature on the VDX8770 only
When enabled it allows VDX devices to use the master as well as the backup router for forwarding of
packets
Results in active-active load-balancing
2013 Brocade Communications Systems, Inc. All Rights Reserved. 43
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 29 The virtual servers are dynamically moved between Host Server 1 and Host Server 2.
Enabling preemption
You can allow a backup router that is acting as the master to be preempted by another backup
router with a higher priority value.
Default: Preemption is enabled for VRRP; disabled for VRRP-E.
If preemption is disabled for VRRP, the owner router is not affected because the owner router always
preempts the active master. To enable preemption for a virtual router, run the preempt-mode command in
virtual-router-group configuration mode, as shown in the following example:
switch(config-vrrp-group-5)# preempt-mode
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
44 2013 Brocade Communications Systems, Inc. All Rights Reserved.
VCS Configuration
Verifying RBridge ID and VCS ID Assignment
After assigning an RBridge ID, a VCS ID, and rebooting, you can verify the VCS Fabric configuration using the
show vcs command. The coordinator is also known as the principal switch. This output in Figure30 shows
the VCS_ID, total number of switches in the VCS fabric, their RBridge IDs, WWNs, Management port IP
address, status and Hostname (if assigned, the default is sw0). See next slide on how to set hostname.
FIGURE 30 show VCS command output
The command show vcs detail provides additional details as shown in Figure31.
FIGURE 31 show vcs detail command output
2013 Brocade Communications Systems, Inc. All Rights Reserved. 45
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Principal Switch Selection
Every Brocade VCS Fabric-enabled switch, upon boot-up and after the Fabric port formation, declares itself to
be a principal switch and advertises this intent on all fabric ports. The intent includes a priority and its switch
WWN. If all switches boot up at the same time, the default priority is the same and all switches will compare
their mutual intents. The switch with the lowest Switch WWN becomes the principal switch. The WWN is an
industry-standard burnt-in switch identifier, similar to the Bridge-MAC except it is 8 bytes. The role of the
principal switch is to decide whether a new RBridge joining the fabric conflicts with any of the RBridge IDs
already present in the fabric. At the end of the principal switch selection process, all the switches in the
cluster have formed a tree with the principal switch at the root.
VCS ISL Introduction
Brocade trunks is the only aggregation method that works using ISLs.
Brocade ISL trunks are formed automatically with other switches using Line Reset (LR) primitives signaling
with the peer switch.
All ISL ports connected to the same neighbor Brocade switch attempt to form a trunk. For a successful trunk
formation, all ports on the local switch must be part of the same port group and must be configured at the
same speed. The number of ports allowed per trunk group is release-dependent. The trunk is turned on by
default. Brocade trunks are not supported over 1G links.
Fabric ISL Configuration
No configuration needed for normal ISL operation (default configuration is already configured)
interface TenGigabitEthernet 11/0/1
fabric isl enable
fabric trunk enable
no shutdown
ISLs can be shutdown and have ISL and trunk functionality turned on or off
VDX11(config-if-te-11/0/1)# [no] shutdown
VDX11(config-if-te-11/0/1)# [no] fabric isl enable
VDX11(config-if-te-11/0/1)# [no] fabric trunk enable
No edge port configuration is allowed on an active ISL
VDX11(config-if-te-11/0/1)# switchport
% Interface Port Role is ISL, nothing to be done
This command controls whether an ISL is formed between two cluster members. With the default setting of
ISL discovery mode set to auto, and ISL administrative mode set to enable, an ISL automatically forms
between two fabric switches.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
46 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Performing the fabric isl enable command on an operational ISL has no effect. However, performing
the no fabric isl enable command on an interface toggles its link status, and subsequently disables
ISL formation. In addition, the no fabric isl enable command triggers the switch to inform its
neighbor that the local interface is ISL disabled. Upon receiving such information, a neighbor switch stops its
ISL formation activity regardless of its current interface state.
Note that a shutdown on an operating ISL interface not only brings down the physical link but also its FSPF
adjacency. The main difference between a shutdown and the no fabric isl enable command is that
the link stays up after the no fabric isl enable command has been issued, whereas the link stays
down after a shutdown. It is recommended that users use the no fabric isl enable command to
expedite ISL state transition as its link state stays up.
The no fabric isl enable command has no effect on an interface when no fabric isl mode
auto is configured. In addition, the no fabric isl enable command on a member link disassociates
the link with its trunk group.
Verifying VCS Fabric ISL Formation
The show fabric all command is equivalent to the FOS fabricshow command as shown in Figure32.
Use the show fabric all command to display information about the fabric. If the switch is initializing or is
disabled, the message Local Switch disabled or fabric is re-building is displayed. If the fabric is re-
configuring, some or all switches may not be displayed; otherwise, the following fields are displayed. See
Figure32.
VCS Id: VCS id of the switch
Config Mode: VCS mode of the switch. For fabric cluster mode, "Local-Only" is displayed.
RBridge-id: The RBridge-id of the switch.
WWN: The switch World Wide Name.
IP Address: The switch Ethernet IP address.
Name: The switch symbolic name. An arrow (>) indicates the coordinator or principal switch. An asterisk
(*) indicates the switch on which the command is entered.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 47
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 32 show fabric all command output
Verifying VCS Fabric ISL Formation (cont.)
Use the show fabric islports command to display information for all ISL ports in the switch as show
in Figure33. Trunk primary is equivalent to trunk master. This is similar to the FOS switchshow command.
The command output includes the following information:
Name: Switch name.
Type Switch: model and revision number.
State: Switch state. The valid values are Online, Offline, Testing, or Faulty.
Role: Switch role. The valid values are Principal, Subordinate, or Disabled.
VCS Id: The range of valid values is from 1 through 8192
Config Mode: VCS mode. The valid values are Standalone/Local-Only/Distributed.
RBridge-id: RBridge-id of the switch. The range of valid values is from 1 through 239.
WWN: Switch world wide name (WWN).
FCF MAC: Mac address
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
48 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Index: Port Index is a number between 0 and the maximum number of supported ports on the platform.
The port index identifies the port number relative to the switch.
Interface: Interface of the local RBridge in the format "local-rbridge-id/slot/port".
State: Port state information:
- Up If the ISL is connected and the link is up
- Down No ISL is connected
Operational State: Displays the operational state of the ISL
FIGURE 33 show fabric islports command output
2013 Brocade Communications Systems, Inc. All Rights Reserved. 49
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
NOS Configuration
NOS configuration management uses three types of configuration files:
The default configuration files are provided as part of the NOS firmware package. Default configuration
files: The default configuration is used, if no startup configuration is available. There are different default
configuration files for standalone and VCS Fabric mode. The default configuration files are provided as
part of the Network OS 2.1.0 or higher firmware package. The default configuration is applied, if no
customized configuration is available. There are different default configuration files for standalone and
VCS mode: defaultconfig.novcs and defaultconfig.vcs
Start-up configuration file: Committed user-defined configuration that becomes effective after reboot
Running configuration: Current configuration active on the switch. Whenever the configuration is changed,
it is written to the running configuration The running configuration does not persist across reboots, unless
it is copied to the start-up configuration
To copy the running configuration to the startup configuration run the following command:
VDX# copy running-config startup-config
This operation will modify your startup configuration. Do you want to
continue? [y/n]:y
At the next switch reboot, the contents of the startup-config become the running-config
This command can not be shortcutted however you can use tab to complete the command option.
Example you can not type in: copy run start
However, you can type in: copy run[tab] start[tab] and the tab key will fill out the command to be:
copy running-config startup-config
Best practice: backup copy of the running configuration
To create a backup copy of the running configuration, use one of these methods:
Copy the running-config or startup-config files to the switches flash directory
VDX# copy running-config flash://filename
Upload the startup-config or running-config files to a remote location using FTP or SCP, or locally
using USB or flash
VDX# copy running-config ftp://login:password@IPaddress/path/filename
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
50 2013 Brocade Communications Systems, Inc. All Rights Reserved.
vLAG
A vLAG is a fabric service that allows LAGs to originate from multiple Brocade VDX switches acting as a single
logical switch to an external switch or server. It acts the same way as a standard LAG using the Link
Aggregation Control Protocol (LACP), a method to control the bundling of several physical ports together to
form a single logical link or trunk. See Figure34.
FIGURE 34 vLAGs
vLAG Features
Provisioning and management is consistent with a standard LAG implementation
Interoperable with servers and third-party switches
- Standard LACP (IEEE 802.3ad) based interoperable solution
Supports vLAG links across multiple Brocade VDX switches
- NOS v2.1.x and higher supports four Brocade VDX switches in a vLAG2
- They do not need to be directly connected
- NOS v2.0.x only supports two Brocade VDX switches in a vLAG.
From a user perspective, features running on top of the vLAG are configured and operate the same as
features running over a standard LAG (i.e. ACL, QoS)
NOS v2.0.x also supports a Brocade-proprietary aggregation in standalone mode only. NOS v2.1.x and
higher supports the Brocade-proprietary aggregation in standalone and Brocade VCS mode.
Only ports with the same speed are aggregated.
Brocade proprietary LAGs are not available for vLAGs.
LACP automatically negotiates and forms the vLAG.
A port-channel interface is created on all the vLAG members.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 51
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
The Brocade VCS Fabric relies on you to consistently configure all nodes in the vLAG.
Similar to static LAGs, vLAGs are not able to detect configuration errors.
A zero port vLAG is allowed.
IGMP snooping fits into the primary link of a vLAG to carry multicast traffic.
The current DCE implementation of vLAGs and LAGs has a concept of a so-called primary port. One of the
member ports of the vLAG and LAG is anointed the primary port and all multicast traffic egressing from
the LAG or vLAG is sent on the primary port.
vLAG Example Topology
Connection between the core Multi-Chassis Trunking (MCT) chassis and Ethernet fabric forms a single vLAG.
Figure35 displays the following:
The two MCT chassis form a single logical device
Brocade VDX switches form a single logical fabric
Eliminates need to run STP between tiers (from access to core)
Active-active redundant paths between the network tiers
Server connectivity into the fabric gain active-active link utilization. Brocade ISLs or trunks are formed within
the Ethernet fabric.
Multi-Chassis Trunking (MCT), currently available in Brocade MLX routers, is a Brocade technology that allows
multiple switches to act as a single logical switch connected to another switch using a standard LAG. MCT at
the core with vLAG in the edge fabric enables a single LAG between the two logical elements which results
in a fully active-active network.
Virtual LAG (vLAG): MCT is an industry accepted solution to avoid spanning tree in Layer 2 topologies. LAG-
based MCT is a special case of LAG covered in IEEE 802.3ad in which the LAG ends terminate on two separate
chassis. Virtual LAG (vLAG), a Brocade innovation, further extends the concept of LAG by allowing its formation
across two or more physical switches.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
52 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 35 vLAGs with MCTs
vLAG Provisioning
Similar to LAG provisioning. Once the VCS fabric detects that the LAG configuration spans multiple switches,
the LAG automatically becomes a vLAG. The standard Admin Key (Channel #) provisioned needs to be same
for ports that belong to the same vLAG. Only ports matching the port-channel speed will be aggregated. See
Figure36.
vLAG formation:
LACP will be used to automatically negotiate and form a vLAG with a server/switch.
LACP on VCS fabric will emulate a single logical switch by
Sending same LACP SYSTEM ID
Sending same admin/oper key
2013 Brocade Communications Systems, Inc. All Rights Reserved. 53
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
A port-channel interface is created on all the vLAG participating members.
LACP based vLAGs will detect partner mis-wiring scenarios (Remote SID will not match)
For static vLAGs, admin is responsible to resolve partner mis-wiring
All vLAGs rely on admin to configure consistent configuration (viz., switchport, vlan membership etc.) on
all nodes in the vLAG.
Switchport interfaces-Interfaces configured as "switchport" interfaces cannot be aggregated into a LAG.
However, a LAG can be configured as a switchport
The default configuration is to treat FCoE traffic as non-VLAG traffic. This command must be performed on
every switch in the vLAG.
Mode specifies the mode of Link Aggregation:
Active enables the initiation of LACP negotiation on an interface.
Passive enables LACP in passive mode, meaning the port will respond to received LACP frames but not
initiate the LACP exchange
On enables static link aggregation on an interface.
FIGURE 36 vLAG Provisioning
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
54 2013 Brocade Communications Systems, Inc. All Rights Reserved.
LAG Type Brocade
A Brocade-proprietary aggregation1 is similar to standards-based link aggregation but differs in how the
traffic is distributed. Utilizes frame-based load balancing to distribute traffic across the member link.
For a LAG type of Brocade: All member links must be part of the same port-group
Maximum of 8 ports in one LAG
Maximum of 6 LAGs in one port group (6 x 2 port LAGs)
Port Groups and the Brocade LAG:
The Brocade VDX 8770-4 and 8770-8 has six port groups; ports 1-8, 9-16, 17-24, 25-32, 33-40, and 41-
48
The Brocade VDX 6720-24 and VDX 6730-32 has two port groups; ports 1-12 and 13-24.
The Brocade VDX 6720-60 and VDX 6730-76 has six port groups; ports 1-10, 11-20, 21-30, 31-40, 41-50
and 51-60.
The Brocade VDX 6710-54 port groups are: 1GE: 1-14, 15-27, 28-48 and 10GE 49-54
In NOS v2.1.x and higher Brocade LAGs are supported in VCS Fabric mode; NOS v2.0.x and above supports
standalone mode.
This type of LAG could be used when connecting to a:
Brocade 8000 switch or FCoE 10-24 blade
Between two Brocade VDX switches where both are in standalone mode
Between two Brocade VDX switches where one is in standalone mode and the other in VCS Fabric mode
Between two Brocade VDX switches from different VCS fabrics
To create a LAG with type Brocade:
sw0(conf-if-te-32/0/12)# channel-group 20 mode active type brocade
To view avialable LAG types:
sw0(conf-if-te-32/0/12)# channel-group 20 mode active type ?
Possible completions:
brocade Brocade LAG
standard Standards based LAG
Configuring the vLAG Ignore Split
The vlag ignore-split command is for LACP-based vLAGs. The scope of this configuration is per port-channel. In
scenarios where the vLAG spans more than 1 node, it minimizes the extent of packet loss in the event of one
of the nodes in the vLAG going down. In a case where connectivity between nodes is lost due to a fabric split
(as opposed to one of members going down), there will be duplication of multicast/broadcast packets.
Brocade recommends that you build redundancy in the fabric so that individual links aren't single points of
failure.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 55
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
The minimum-links Command
Sets the minimum bandwidth.
Synopsis minimum-links num-of-links
no minimum-links
Operands num-of-links Number of links. The range of valid values is from 1 through 16.
Defaults The default number of links is 1.
Command Modes Port channel interface configuration mode
Description: Use this command to allow a port-channel to operate at a certain minimum bandwidth all the
time. If the bandwidth of the port-channel drops below that minimum number, then the port-channel is
eclared operationally DOWN even though it has operationally UP members. In order to provide link and node
level redundancy, the Brocade VCS Fabric supports static vLAGs. A Brocade VCS Fabric vLAG functions with
servers that do not implement LACP because it supports static vLAGs as well.
Support for Static MAC Pre-Provisioning on vLAG
The state of the static MAC depends on the state of vLAG on the switch in which the static MAC was created
The configuration will be stored in its database, but is not activated if either:
Forwarding interface is not online
VLAN does not exist
Forwarding port does not belong to the vlan
Once the port is part of the VLAN, the static MAC is activated and downloaded to the ASIC
For example, see Figure37, a static MAC is created on Rbridge A however since the interface on RBridge A is
not part of the vLAG, the Static MAC is not active
This is a different behavior from NOS v2.1
NOS v3.0.0 adds a new state REMOTE to clearly identify the MACs exists on remote switch
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
56 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 37 MAC Pre-Provisioning
LACP troubleshooting tips
To troubleshoot problems with your LACP configuration, use the following troubleshooting tips.
If a standard IEEE 802.3ad-based dynamic trunk is configured on a link and the link is not able to join the
LAG:
Make sure that both ends of the link are configured as standard for the trunk type.
Make sure that both ends of the link are not configured for passive mode. They must be configured as
active/active, active/passive, or passive/active.
Make sure that the port-channel interface is in the administrative "up" state by ensuring that the no
shutdown command was entered on the interface on both ends of the link.
Make sure the speed parameter is configured to 1000 if the port-channel is using the gigabit interface.
Make sure that the system ID of the switches connected by the link is unique. This can be verified by
entering the show lacp sys-id command on both switches.
Make sure that LACPDUs are being received and transmitted on both ends of the link and there are no
error PDUs. This can be verified by entering the show lacp counters number command and looking at the
rx and tx statistics. The statistics should be incrementing and should not be at zero or a fixed value. If the
PDU rx count is not incrementing, check the interface for possible CRC errors by entering the show
interface link-name command on the neighboring switch.
Make sure that the fiber length of the link has a deskew value of 7 microseconds. If it does not, the link
will not be able to join the LAG and the following RASLOG message is generated: Deskew calculation failed
for link <link-name>. When a link has this problem, the show port-channel command displays the
following message: Mux machine state : Deskew not OK
2013 Brocade Communications Systems, Inc. All Rights Reserved. 57
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
xSTP reconvergence
For topologies that have redundant LAGs between the intermediate-switch and the VCS (same or different
FCFs), one of the LAGs will be in a xSTP-Blocked state. If one LAG fails for some reason, xSTP unblocks the
other LAG to restore Layer 2 connectivity.
After a LAG failure, all Enodes are expected to logout and login back again if they discover an alternate path to
the FCF. The availability of an alternate path depends on whether the second LAG is configured for FCoE or
not. Also, this determines the time-taken for the logout of the Enodes during LAG failures. If the alternate LAG
is not configured for FCoE, the system responds as if the LAG does not exist. If the alternate LAG is configured
for FCoE, then the Unsolicited-advertisements continue to be sent by the FCF and the Enode keep-alives
continue to reach the FCF. However, the FCF does not have any login sessions associated with the Enode. So a
CVL is sent to the Enode as soon as the first keep-alive is received by the FCF. This clears the login session in
the Enode and forces a re-login. For the default configuration, this happens within one FKA interval, about
eight seconds. It may be slightly higher for faster configurations, as some packets (both RX and TX) are lost
until STP unblocks the port
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
58 2013 Brocade Communications Systems, Inc. All Rights Reserved.
QOS
Random Early Discard
Traditionally, Random Early Discard (RED) is used for TCP traffic streams, which are generally more
aggressive, as well as reactive, to network drops. If RED is not configured, queues build up at the switch and
become full, resulting in tail drop. Tail drop situations can cause head-of-line blocking issues at the switch,
which is not desirable. By configuring RED, you set a probability for dropping packets before traffic in the
queue reaches a specific threshold. This allows congestion to ease more gradually, avoids retransmit
synchronization, resolves "bursty" TCP connections during congestion conditions, and controls packet latency.
Configure RED using the following parameters:
RED profile identification (0-384)
Minimum threshold of a queue (0-100%)
Maximum threshold of a queue (0-100%)
Drop probability (0-100%)
You can also map a specific CoS priority value (0 through 7) to a specific RED profile.
Brocade VCS Fabric QoS Restrictions for Layer 3 features in VCS mode
When the switch is in VCS mode, the lossless priority for carrying FCoE traffic and the fabric priority for
carrying fabric traffic must be isolated from any Layer 3 QoS markings and classification. Therefore, specific
restrictions apply to some Layer 3 DSCP QoS features when the switch is working in VCS mode.
The following are restrictions for using applicable Layer 3 DSCP-Traffic-Class map, DSCP-CoS map, and DSCP
Trust features in VCS mode. Note that DSCP mutation maps and the egress RED feature are not affected in
VCS mode.
DSCP trust will be disabled in VCS mode like it is for CoS trust.
There will be no default DSCP maps while in VCS mode. Default maps occur when DSCP trust is enabled
in standalone mode.
A non-default DSCP-Traffic-Class map has the following restrictions:
- A DSCP value cannot be classified to Traffic Class 7.
- A DSCP value cannot be classified to a queue that carries lossless traffic (by default Traffic Class 3).
A non-default DSCP-CoS map has the following restrictions:
- A DSCP value cannot be marked to CoS 7.
- A DSCP value cannot be marked to lossless priority (by default CoS 3).
Lossless priorities will be identified through the CEE map.
To enable DSCP based marking or classification, a non-default DSCP-Traffic-Class map and a DSCP-CoS
map have to be applied on the interface.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 59
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
To apply a DSCP-Traffic-Class or DSCP-CoS map to an interface, the CoS and Traffic Class values have to
be re-marked for lossless priorities. For example, when DSCP-Traffic-Class map "abcd" is created, it will
have the default contents. When applied to an interface, an error will display that the fabric and lossless
priorities are used in the map and it cannot be applied on the interface.
When a valid DSCP-Traffic-Class map and DSCP-CoS map are applied on the interface, then DSCP trust is
enabled with the configured maps
Configuring QoS profiles
QoS profiles define the following values:
Incoming 802.1p priority is set to internal queue priority. If the port is in QoS untrusted mode, all incoming
priorities will be mapped to default best effort priority.
Incoming priority is set to outgoing priority.
Mapping of incoming priorities is set to strict or WRR traffic classes.
Enabling of flow control on a strict or a WRR traffic class.
The QoS profile has two flavors: CEE QoS and Ethernet QoS. The QoS profile may contain either CEE QoS
or Ethernet QoS. Server side ports typically are carrying converged traffic
Configuring Brocade VCS Fabric QoS
To configure the remapping priorities for the Brocade VCS Fabric, perform the following steps from global
configuration mode.
1. Use the cee-map command to enter CEE map configuration mode.
switch(config)#cee-map default
2. Use the remap lossless priority command to set the lossless priority for Brocade VCS Fabric QoS. The
default lossless remap priority is set to 0.
switch(config-cee-map-default)# remap lossless-priority priority 2
CEE Map CoS Value
In this example looking at the default CEE map indicates FCoE CoS 3 is a value of 1:
cee-map default
priority-group-table 1 weight 40 pfc on
priority-group-table 2 weight 60 pfc off
priority-table 2 2 2 1 2 2 2 15.0
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
60 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FCoE Queuing
The QoS configuration controls the FCoE traffic distribution. Note that changing these settings requires
changes on both the Brocade VDX hardware and the Converged Network Adapter (CNA); therefore, the link
must be taken offline and put back online after a change is made. Traffic scheduler configuration changes
affect FCoE traffic distribution as follows:
Changing the priority group for a port causes the FCoE traffic distribution to be updated. The priority group
and bandwidth are updated.
Changing the priority table for a port causes the FCoE traffic distribution to be updated. The CoS-to-
priority group mapping is updated.
Changing the class map for a port causes the FCoE traffic distribution to be updated.
Changing the policy map for a port causes FCoE traffic distribution to be updated.
Changing the DCB map for a port causes the FCoE traffic distribution to be updated. The FCMAP-to-VLAN
mapping determines the FCoE VLAN allowed for the FCoE session
Deficit weighted round robin scheduling
Weighted Round Robin (WRR) scheduling is used to facilitate controlled sharing of the network bandwidth.
WRR assigns a weight to each queue; that value is then used to determine the amount of bandwidth allocated
to the queue. The round robin aspect of the scheduling allows each queue to be serviced in a set ordering,
sending a limited amount of data before moving onto the next queue and cycling back to the highest priority
queue after the lowest priority is serviced.
Figure38 describes the frame scheduling order for a WRR scheduler servicing two WRR queues. The higher
numbered queue is considered higher priority (WRR2) and the weights indicate the network bandwidth should
be allocated in a 2:1 ratio between the two queues. In Figure38, WRR2 should receive 66 percent of
bandwidth and WRR1 receives 33 percent. The WRR scheduler tracks the extra bandwidth used and
subtracts it from the bandwidth allocation for the next cycle through the queues. In this way, the bandwidth
utilization statistically matches the queue weights over longer time periods.
FIGURE 38 Wrr schedule - two queues
2013 Brocade Communications Systems, Inc. All Rights Reserved. 61
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Port-Based Policer
The port-based Policer feature controls the amount of bandwidth consumed by an individual flow or aggregate
of flows by limiting the inbound and outbound traffic rate on an individual port according to criteria defined by
the user. The Policer provides rate control by prioritizing or dropping ingress and egress packets classified
according to a two-rate, three-color marking scheme defined by RFC 4115. This feature is only supported on
VDX 8770-4, VDX 8770-8, and later models.
The Policer supports the following features:
Color-based priority mapping scheme for limiting traffic rate:
- One rate - two color policing with conform color options. Violate color traffic will be dropped.
- Two rate - three color policing with conform and exceed color options. Violate color traffic will be
dropped.
Policing option that allows packet headers to be modified for IP precedence.
Policing options that allows packet headers to be modified for Class of Service (COS).
Policing options that allows packet headers to be modified for Differentiated Services Code Point (DSCP).
Policing options that allow packets to be assigned to a traffic class (0-7).
Data Center Bridging (DCB)
Traditional Ethernet is not suitable for protocols that require or could benefit from a lossless, low latency and
low congestion medium such as Fibre Channel or iSCSI. DCB is an umbrella term for an Ethernet technology
enhanced by additional standards to meet these requirements. Also referred to as Converged Enhanced
Ethernet (CEE).
Key features of DCB include:
A lossless, full-duplex Ethernet environment which provides in-order delivery and supports a minimum of
2.5 KB mini-jumbo frames
Convergence of multiple protocols with different requirements over the same Ethernet network
High speed transport of Ethernet traffic
Enhanced Ethernetalso called Converged Enhanced Ethernet (CEE), Data Center Ethernet or Data Center
Bridging (DCB)eliminates Layer 3 TCP/IP protocols in favor of native Layer 2 Ethernet. Traditional Ethernet
commonly experiences network congestion, latency and dropped frames, which renders it unreliable for Fibre
Channel traffic. However, 10 GbE Enhanced Ethernet changes this by dispensing with TCP/IP in favor of a
lossless Ethernet fabric. The lossless environments basic requirements are Priority Flow Control (priority
pause), ETS (scheduler), and the discovery protocol. (Congestion management is attractive but optional.)
These capabilities allow the Fibre Channel frames to run directly over 10Gbps Ethernet segments with no
performance degradation.
In order to make Ethernet as reliable as Fibre Channel at the lower layers, it is necessary to add several
enhancements to Ethernet to mimic the behavior at the lower layers of Fibre Channel.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
62 2013 Brocade Communications Systems, Inc. All Rights Reserved.
For example, the PAUSE command is necessary to avoid congestion and frame drop, as is the case with a
best-effort network such as Ethernet. There have to be logical channels on the Ethernet link, just as virtual
channels exist in our Fibre Channel technology. This allows storage traffic to move reliably through the
network, while IP traffic is handled on a best-effort basis.
Fibre Channel FSPF allows multiple paths to be used between switches if their path cost is the same, but
Ethernet with its Spanning Tree (and variations) routing model does not.
Other enhancements allow for the detection and communication of congestion in the network, to avoid hot
spots and throttle back inbound traffic.
DCB Enhancements to Ethernet
Data Center Bridging eXchange (DCBX) is based on IEEE 802.1Qaz and leverages functionality provided by
IEEE 802.1AB (Link Layer Discovery Protocol - LLDP). It is used for conveying capabilities and configuration
features between single-hop neighbors.
Priority Flow Control (PFC) based on IEEE 802.1Qbb provide the following:
Link level flow control mechanism
Controlled independently for each Class of Service (CoS). The Ethernet CoS is different from the Fibre
Channel CoS. It is defined as the priority level in DCB.
Goal of this mechanism is to ensure zero loss under congestion in DCB networks
Enhanced Transmission Selection (ETS) is based on IEEE 802.1Qaz and enables bandwidth management by
assigning bandwidth segments to different traffic flows.
DCBX
The Data Center Bridging Capability Exchange Protocol (DCBX) is used to exchange DCB-related parameters
with neighbors including bandwidth allocation via ETS and flow control configuration via PFC. It is based on
IEEE 802.1AB which defines a set of DCBX specific TLVs. DCBX capabilities include:
DCB peer discovery
Mismatched configuration detection
DCB-link peer configuration
DCBX uses LLDP to exchange parameters between two link peers. The DCBX TLVs are added to the LLDP
frame. There are two types of LLDP TLVs, as specified in the IEEE 802.3AB standard:
2013 Brocade Communications Systems, Inc. All Rights Reserved. 63
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Basic management TLVs consist of both optional general system information TLVs as well as mandatory
TLVs. Mandatory TLVs cannot be manually configured. They are always the first three TLVs in the LLDPDU,
and are part of the packet header. General system information TLVs are optional in LLDP implementations
and are defined by the Network Administrator. Common Basic Management TLVs include:
- Chassis ID (mandatory)
- Port ID (mandatory)
- Time to Live (mandatory)
- DCBX
- DCBX Control Sub-TLV
- Priority group sub-TLV
- Priority flow control sub-TLV
- Logical link down feature sub-TLV
- Port description
- System name
- System description
- System capabilities
- Management address
- End of LLDPDU
ETS Overview
Enhanced Transmission Selection (ETS) shown in Figure39 allocates bandwidth between different traffic
classes such as:
LAN
FCoE
iSCSI
Inter-process communication (IPC) is normally server-to-server control traffic, and therefore has a very
high priority.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
64 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 39 ETS
ETS and Class of Service
ETS uses Class of Service (CoS) to distinguish between traffic classes and allocate bandwidth to each traffic
class. Allows bandwidth-allocated traffic classes to coexist with strict priorities traffic classes through the use
of a hybrid scheduler. Only enforced when congestion is detected. See Figure40.
FIGURE 40 ETS and Class of Service
2013 Brocade Communications Systems, Inc. All Rights Reserved. 65
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Priority Flow Control
With Priority Flow Control (PFC), it is important to provide lossless frame delivery for certain traffic classes
while maintaining existing LAN behavior for other traffic classes on the converged link. This differs from the
traditional 802.3 PAUSE type of flow control where the pause affects all traffic on an interface. PFC is defined
by a one-byte bitmap. Each bit position stands for a user priority. If a bit is set, the flow control is enabled in
both directions (Rx and Tx).
Strict Priority Scheduling
A strict priority scheduler handles only very high priority traffic such as IPC traffic. Inter-process
communication (IPC) is normally server-to-server control traffic, and therefore has a very high priority.
Strict priority uses priority group ID 15. There are 8 levels within group 15, 15.0 15.7, with 15.0 being the
highest priority and 15.7 being the lowest priority. Strict priority scheduling does not use any type of round
robin servicing of queues and there is no bandwidth allocation. If there is more than one level of group 15
traffic, all of the highest priority traffic goes first, followed by the next highest priority
For example, if there is traffic set for strict priority 15.1, and 15.2, all 15.1 traffic would be handled first, and
then all of 15.2 traffic
Data Center Bridging Map Configuration
The DCB QoS covers frame classification, priority and traffic class (queue) mapping, congestion control, and
scheduling. Under the DCB Provisioning model all of these features are configured utilizing two configuration
tables, Priority Group Table and Priority Table. DCB Priority Group Table defines each Priority Group ID (PGID)
and its scheduling policy (Strict Priority versus DWRR, DWRR weight, relative priority), and partially defines the
congestion control (PFC) configuration. There are 16 rows in the DCB Priority Group Table.
Note only a single CoS can be mapped to a PFC-enabled priority queue..The switch automatically maps the
CoS number to the same TC number when PFC is enabled. The PGID can be anything from 0-7. If your
configuration violates this restriction an error message displays and the Priority Group Table is set back to the
default values. When the DCB map is applied, and the interface is connected to the CNA, only one strict
priority PGID (PGID 15.0 to PGID 15.7) is allowed.
Defining a priority group table
To define a priority group table map, perform the following steps from privileged EXEC mode.
1. Enter global configuration mode.
switch# configure terminal
2. Specify the name of the DCB map to define using the cee-map command. The only map name allowed is
"default."
switch(config)# cee-map default
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
66 2013 Brocade Communications Systems, Inc. All Rights Reserved.
3. Define the DCB map for PGID 0.
switch(config-cee-map-default)# priority-group-table 0 weight 50 pfc on
4. Define the DCB map for PGID 1.
switch(config-cee-map-default)# priority-group-table 1 weight 50 pfc off
5. Return to privileged EXEC mode.
switch(config-cee-map-default)# end
6. Enter the copy command to save the running-config file to the startup-config file.
switch#copy running-config startup-config
Defining a Priority-Table Map
To define a priority-table map, perform the following steps from privileged EXEC mode.
1. Enter global configuration mode.
switch# configure terminal
2. Specify the name of the DCB map to define using the cee-map command. In this example 'default' is used.
switch(config)# cee-map default
3. Define the map.
switch(config-cee-map)# priority-table 1 1 1 0 1 1 1 15.0
4. Return to privileged EXEC mode.
switch(config-cee-map)# end
2013 Brocade Communications Systems, Inc. All Rights Reserved. 67
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FCoE and iSCSI
Physical port mapping:
<Switch_RBID>/<Slot_Number>/<Physical_Port>
For Brocade VDX departmental switches the slot number is always 0
Example: 10/0/24 would be switch RBID 10, slot 0 and port 24
FCoE port format
<Mapped-VN-Number>1/<Switch_RBID>/<Logical_Port>
The VN number is a virtual network number which is mapped to a VLAN (used for FCoE traffic). In this release,
the VN number is fixed to 1 and the default VLAN used for FCoE traffic is 1002. The default VLAN number,
1002, can be changed.
<Mapped-VN-Number> is fixed to 1
<Switch_RBID> is the switch ID of the Brocade VDX switch
<Logical_Port> is the logical port number
Example: Interface FCoE 1/10/24 would be VN 1, switch RBID 10 and port 24
1:1 Mapping of physical port to FCoE logical port
Example: TenGigabitEthernet 10/0/24 = FCoE 1/10/24 where port 24 matches
An FCoE license is required on any Brocade VDX switch that an FSB is attached to:
The FCoE license enables the following:
The FCF services
The FCoE configuration on the switch
FCoE license is required on any Brocade VDX switch that an FSB is attached to.
The FCoE license:
Enables the FCF services
Enables FCoE configuration on the switch
If a switch in the fabric is not:
Directly attached to FCoE devices
Connected to FSB
Connected to Fibre Channel fabric (VDX 6730)
It does not require a license to forward the FCoE traffic through the fabric
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
68 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FCoE Co-Existence with LAG/vLAG
FCoE provisioning is allowed on LAG members (v3.0). One or more members of a LAG can be provisioned for
FCoE. FCoE traffic is not allowed outside the fabric. See Figure41. Note: The LAG hash algorithms will be
used for FCoE load balancing and host MPIO drivers will be used to load balance traffic if 2 or more physical
ports in the LAG are enabled for FCoE.
FIGURE 41 FCoE Co-Existence with LAG/vLAG
Configuring FCoE
Configures and enables FCoE on the physical port. Use the FCoE map named default which contains:
CEE map configuration
Fabric map configuration
If no changes to any of the defaults are required then the only command needed is: fcoeport default
which enables FCoE and applies the CEE and fabric maps
RB1# configure terminal
RB1(config)# interface TenGigabitEthernet 10/0/24
RB1(config-TenGigabitEthernet-10/0/24)# fcoeport default
The At this point the attached FCoE device should login
The CEE map defines the traffic type to a CoS and defines the allocated bandwidth for each traffic type.
The Fabric map defines the: (defaults) which are:
- FCoE VLAN (1002)
- FCoE CoS (3)
- FCMAP used for FPMA addressing (0E:FC:00)
- Keep-alive timeout value (enabled)
- Advertisement interval (8 seconds)
2013 Brocade Communications Systems, Inc. All Rights Reserved. 69
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Configuring FCoE and LLDP
Verify the LLDP advertisements in the running-config:
RB1# show running-config protocol lldp
protocol lldp
advertise dcbx-fcoe-app-tlv
advertise dcbx-fcoe-logical-link-tlv
advertise dcbx-tlv
Enable LLDP FCoE TLV1 advertisement
- DCBX FCoE application TLV which advertises the DCBX2 FCoE application TLV
- DCBX FCoE logical link TLV which advertises the DCBX FCoE logical link TLV3
- The DCBX-TLV is advertised by default which means the switch is DCB4 capable
Notes:
TLV is Time Length Value.
The TLV provides the status of the FCoE link. The FCoE link must be up before the attached FCoE device
can send a FIP frame.
After all changes are made to enable the physical port for FCoE, you use:
RB1# configure terminal
RB1(config)# interface tengigabitethernet 10/0/24
RB1(config-TenGigabitEthernet-10/0/24)# fcoeport default
At this point the FCoE device should login. The command fcoeport default provides the FCoE which
contains the CEE and fabric maps
Enable and Disable FCoE Ports
FCoE logical ports are enabled (no shutdown) by default. The configuration commands are:
Shutdown (disable)
No shutdown (enable)
Example where 1/10/24 is the FCoE logical port:
RB1# configure
RB1(config)# interface fcoe 1/10/24
RB1(config-Fcoe-1/10/24)# shutdown
RB1(config-Fcoe-1/10/24)# no shutdown
This is a good way to bounce the FCoE logical link but keep the non-FCoE traffic flowing
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
70 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FCoE Show Commands
Display interface information in brief. See Figure42. Shows the status of the protocol: Is the switch and
attached device exchanging frames and displays the number of VN ports for each port. Here are details:
FCOE IF is the FCoE logical port.
Mode Config/Current: In this release, this always show VF (Virtual Fabrics). Virtual Fabrics is not
supported in this release, the switch is always in the default fabric. This is setup for future enhancements.
Status Config is the status (enabled or disabled) of the logical FCoE port.
Status Proto displays the status of the protocol. Are the two devices talking to each other?
- This does not mean that all is OK, the configuration could still have problems, but is still sending
frames back and fourth. If the Protocol is up but the VN Port count is 0, check the configuration of the
physical port.
Binding displays the physical port where the FCoE logical port is mapped to.
Num VN Ports displays the number of devices that have successfully logged into the fabric.
Note: If the Config status shows as Admin-Down that mean the FCoE logical port is shutdown.
FIGURE 42 show fcoe interface brief command example
2013 Brocade Communications Systems, Inc. All Rights Reserved. 71
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
iSCSI TLV Support
DCB support for the iSCSI TLV1 provides deterministic delivery of iSCSI traffic. Where DCB support is required
for FCoE, iSCSI does not require DCB capabilities. That said there are benefits to using it. Such as the ability to
pause traffic and the allocation of bandwidth.
Intent is to minimize TCP retransmissions by eliminating packet loss do to congestion.
Supports DCBX, PFC and ETS.
The DCB iSCSI TLV is supported in both standalone and VCS Fabric modes. NOS version 2.0.0a supported the
TLV in standalone mode only.
Supported on all the Brocade VDX 10 GbE ports
Brocade VDX 6710: 1 GbE ports do support iSCSI but since the ports are not DCB capable they do not
support the iSCSI TLV
Assigning an FCoE map onto a LAG member
The fcoeport default is a command under interface configuration mode used to provision a port to be
an FCoE port. This puts the port in Layer 2 mode, but only for FCoE VLANs. Starting from Network OS v3.0.0a,
the fcoeport default command is supported for LAG member ports where FCoE provisioning is applied to
individual tengigabit Ethernet ports.
You must apply the fcoeport default command on each LAG member interface. Once this command is applied,
and if the member port of the LAG is CEE-capable, it carries FCoE-traffic only.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
72 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Zoning
FCoE only zoning requires Brocade VDX switches to be running NOS v2.1.0 or higher
For FCoE to FC Routing requires:
Brocade VDX 6730 switches to be NOS v2.1.1 or higher
Brocade SAN backbone fabric to be running FOS v7.0.1
Zone members are WWN only (port or node). Port-based zones, i.e., "Domain,Index" format as a zone
member, is not supported
For LSAN zones, only port WWNs can be used
VCS Fabric mode must be enabled
Zone enforcement is done by Name Server
Meaning it is soft zoning
For Fibre Channel zoning, there are two types of zoning:
Session-based hardware enforcement
Name Server restricts PLOGIs
Frame-based hardware enforcement
Source device is denied access to destination device if they are not defined in the same zone
Available through ASIC hardware logic checking at the destination port
More secure than session enforcement
Zone Merging
When a new switch is added to the fabric, it automatically takes on the zone configuration information from
the fabric.
If a switch is being added that is already configured for zoning, clear the zone configuration on that switch
before connecting it to the zoned fabric.
If settings do not match the new switch will segmented
Zone merging scenarios: Default access mode
See Figure43
2013 Brocade Communications Systems, Inc. All Rights Reserved. 73
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 43 Default Access mode
The show name-server detail command
Its output displays both the port WWN and node WWN; the port WWN must be used for LSANs.
switch# show name-server detail
PID: 012100
Port Name: 10:00:00:00:c9:2b:c9:2c
Node Name: 20:00:00:00:c9:2b:c9:2c
SCR: 3
FC4s: FCP
PortSymb: [27] "Brocade-1020|2.3.0.0|localhost.localdomain|Red Hat
Enterprise Linux Server release 5.5"
NodeSymb: NULL
Fabric Port Name: 20:21:00:05:1E:CD:79:7A
Permanent Port Name: 10:00:00:00:c9:2b:c9:2c
Device type: Physical Initiator
Interface: Fcoe 1/1/9
Physical Interface: Te 1/0/9
Share Area: No
Redirect: No
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
74 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FC Bridging
LSAN naming
Zones that contain hosts and targets that are shared between the two fabrics need to be explicitly
coordinated. To share devices between any two fabrics, you must create an LSAN zone in both fabrics
containing the WWNs of the devices to be shared. Although an LSAN zone is managed using the same tools
as any other zone on the edge fabric, two behaviors distinguish an LSAN zone from a conventional zone:
A required naming convention. The name of an LSAN zone begins with the prefix "LSAN_". The LSAN name
is case-insensitive; for example, lsan_ is equivalent to LSAN_, Lsan_, and so on.
LSAN zone members in all fabrics must be identified by their WWN. You cannot use the port IDs that are
supported only in Fabric OS fabrics.
To enable device sharing across multiple fabrics, you must create LSAN zones on the edge fabrics (and
optionally on the backbone fabric as well), using normal zoning operations to create zones with names that
begin with the special prefix "LSAN_", and adding host and target port WWNs from both local and remote
fabrics to each local zone as desired. Zones on the backbone and on multiple edge fabrics that share a
common set of devices will be recognized as constituting a single multi-fabric LSAN zone, and the devices that
they have in common will be able to communicate with each other across fabric boundaries.
Brocade VCS Fabric and Brocade Fibre Channel SAN Connectivity
Enabled through FC-FC Routing. See Figure44.
Connectivity is provided through an E_Port on a Brocade VDX 6730 which connects to an EX_Port on a FC
Router
2013 Brocade Communications Systems, Inc. All Rights Reserved. 75
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 44 Brocade VCS Fabric to Brocade FC SAN Connectivity
FC Fabric Routing Configuration:
Standard FCR and EX_Port configuration
Except selecting interopmode 5 when configuring the EX_Port
Fabric OS v 7.0.1 adds the IM mode 5 option to the portcfgexport CLI command
FCR:admin> portcfgexport
Usage: portcfgexport [SlotNumber/]PortNumber
[-a 1-enable 2-disable] [-f fid(1..128)]
[-r r_a_tov] [-e e_d_tov] [-d domain]
[-p 0-native 1-core 2-extended edge]
[-m 0-Brocade 1-Open 2-McDATA Fabric 3-McDATA Fabric Legacy
5-Brocade NOS fabric]
[-t 1-Enable 2-Disable]
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
76 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Principal Routing Bridge Availability
If a new principal routing bridge is introduced into a working VCS Fabric cluster, or if the principalrouting
bridge is lost and a new switch must be elected, the fabric is rebuilt from the control-plane viewpoint, whereas
the data plane continues to forward traffic without disruption. The primary responsibilities of the principal
routing bridge in a VCS Fabric are:
Routing bridge ID allocation
Ownership of virtual management IP address
Keeping the configuration database synchronized
FC-FC Routing to Bridge FCoE and FC Traffic
Fabric OS provides Layer 3 Fibre Channel-to-Fibre Channel routing (FC-FC Routing) between fabrics. Allows
device access between two or more fabrics without merging the fabrics. Brocade VCS Fabrics use the FC-FC
routing technology to bridge traffic between Brocade VCS Fabric FCoE devices and Brocade FOS FC targets.
Managing domain IDs
FCoE connectivity across the Fibre Channel link between Brocade VCS Fabric clusters and FC routers uses
domain IDs to identify switches. Within a Brocade VCS Fabric cluster, a domain ID is the same as a routing
bridge ID. When you connect to a Fibre Channel router, the FC fabric FC router service emulates virtual
phantom FC domains in the FCoE fabric. Each FCR enabled switch emulates a single "front" phantom domain
and each FC fabric is represented by a translate phantom domain.
It is important to ensure that front domain IDs and translate domain IDs presented by the FC router do not
overlap routing bridge IDs in the FCoE fabric, otherwise the connectivity will fail and the Network OS switch
with the overlapping routing bridge ID becomes isolated from the fabric. To prevent potential overlap, use the
portCfgExport -d Fabric OS command on the FC router to apply a unique front domain ID-one that will
not be used in the FCoE fabric
In an FCoE fabric that spans Network OS switches and Fabric OS switches, a Network OS switch with a routing
bridge ID that matches a front phantom domain ID or translate phantom domain ID of a connecting Fibre
Channel router can become isolated. FCoE connectivity across the Fibre Channel link between VCS Fabric
clusters and Fibre Channel routers uses domain IDs to identify switches. Within a VCS Fabric cluster, a domain
ID is the same as a routing bridge ID. When you connect to a Fibre Channel router, the Fibre Channel router
service in the Fibre Channel fabric emulates virtual phantom Fibre Channel domains in the FCoE fabric. Each
Fibre Channel router enabled switch emulates a single front phantom domain and each FC fabric is
represented by a translate phantom domain
With NOS v3.0.0a the Maximum # of 8770 switches in VCS mode in a Fabric is 24.
FC to FC Routing
Physical connectivity is accomplished through the use of a Fibre Channel Router (FCR).Includes implementing
and configuring the underlying physical connectivity between the fabrics that will share devices using
EX_Ports/Inter Fabric Links (IFL). See Figure45.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 77
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FC-FC routing was introduced in Fabric OS v5.1 on the Brocade 7500 and Brocade FR4-18i blade. FC-FC
routing is also known as, FCRS (Fibre Channel Routing Service), FC-to-FC routing, FCR, FC routing and routed
SANs.
The FC router in effect enforces an implied DENY_ALL, and the administrator must configure the PERMIT
entries (ACLs) via LSAN zoning.
Logical connectivity is accomplished through the use of Logical Storage Area Networks (LSANs), by creating
uniquely named zones called LSAN zones.
FIGURE 45 FC to FC Routing
Brocade VDX Switch TRILL Termination
FD and XD domains are represented as RBridges in the NOS fabric. The front domain and xlate domains count
against the 24 RBridge fabric limit. See Figure46.
Brocade VDX 6730 switch provides TRILL path termination for FC domains
Proxy RBridge ID
Proxy FCF MAC
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
78 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 46 Brocade VDX Switch Trill Termination
Edge to BB Configuration
From the FCR, configure an EX_Port to communicate with the Brocade VCS fabric. See Figure47.
Disable the port that you wish to connect to the Brocade VDX switch FCR:admin> portdisable x
To configure and activate the EX_Port on port 0 of the FCR, with the Interopmode set to 5 (Brocade VCS
Fabric), issue command: FCR:admin> portcfgexport 0 a 1 m 5
FIGURE 47 Edge to BB Configuration
2013 Brocade Communications Systems, Inc. All Rights Reserved. 79
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
AMPP
VM Mobility Challenges
Virtual machine mobility (migration) challenge is when server managers, like VMwares vCenter, control the
migration of server side profiles to ensure the server-side consistency. See Figure48.
In traditional networks, post-migration tasks often require manual configuration changes. VM migration
across physical server and switches can result in non-symmetrical network policies. Port setting information
must be identical at destination switch/port.Brocade AMPP technology enhances network-side Virtual
Machine migration by allowing VM migration across physical switches, switch ports, and collision domains.
FIGURE 48 VM Migration Challenges
Solution Overview
Deploy and distribute port profiles within the Ethernet fabric. VMs can be migrated without a need for network
ports to be manually configured on the destination switch. Brocade VCS Fabrics support automatically moving
the port profile in synchronization with a VM moving to a different physical server on the same or different
access layer switches. Allows for rapid migration of applications in the data center. SeeFigure49.
AMPP belongs under the IEEE 802.1 work group and the emerging standard called Edge Virtual Bridging (EVB)
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
80 2013 Brocade Communications Systems, Inc. All Rights Reserved.
FIGURE 49 Port Profiles: Distributed Network Settings
Port Profile Overview
A port profile contains the entire configuration needed for a VM to gain access to the LAN and FCoE
The contents of port profiles can be any of the following:
Only LAN configuration
Only FCoE configuration
Both FCoE and LAN configurations
A port profile does not contain some of the interface level configurations such as LLDP, SPAN, and LAG. A port
profile is capable of operating as a self-contained configuration container. In other words, if a port profile is
applied on a new switch without any configuration, a port profile should be sufficient enough to start carrying
traffic.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 81
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 50 Port Profiles
For normal AMPP operation, port profiles need to be pre-created manually across fabric. Manual configuration
of ports/LAG/vLAG in port-profile mode across the fabric.
Beginning with NOS v2.1.0 VMware vCenter integration provides the ability to:
Automatically create AMPP port-profiles from VM port groups
Automatically create VLANs
Automatically create association of VMs to port groups
Automatically configure port-profile modes on ports
NOS v2.1.0 supports vCenter version 4.0 or greater
NOS v2.1.1 is required to support vCenter 5.0
Different security (ACLs) port profiles simultaneously active a port
Support on the Brocade VDX 8770 platforms only
The Brocade VDX 67xx platforms only support a single security port profile on a port
vCenter Integration Implementation
Brocade VDX switch is configured with vCenter access information and credentials (URL, user name and
password). Brocade VDX switch performs discovery of virtual infrastructure assets.
Based on discovered assets, the VCS fabric will automatically configure corresponding objects:
Port-profiles and VLAN creation
MAC address association to port-profiles
Port, LAGs, vLAGs are put into profile mode automatically based on ESX host connectivity
The VCS fabric is ready for Virtual Machine movements
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
82 2013 Brocade Communications Systems, Inc. All Rights Reserved.
vCenter Discovery
The Brocade VDX switch connected to VMware ESX/ESXi hosts and virtual machines must be aware of
network policies in order to allow or disallow traffic, which requires a discovery process by the VDX switch.
During VDX switch configuration, relevant vCenters that exist in its environment and the discovery of virtual
assets from the vCenter occurs in the following circumstances:
When a switch boots up
When a new vCenter is configured on the VDX switch and activated (activation turns on the
timer processing, set to 180-second intervals)
When the discovery is explictly initiated with the CLI
The following assets are discovered from the vCenter:
Hosts associated with the vCenter
Virtual machines (VMs) that have been created on the hosts
Standard port groups
Standard virtual switches
Distributed virtual switches
Distributed virtual port groups
NOS vCenter Integration
Each RBridge in the fabric listens for CDP1 (Cisco Discovery Packet) packets from ESX hosts on switch ports.
Automatically ports/LAGs/vLAGs are put in profile mode when the connected ESX host transmits CDP.
Removes the port from profile mode in the case that CDP is timed out on that port. vCenter uses CDP so
Brocade VDX switches have to support this. The VMware vSwitch or dvSwitch must be configured for CDP. CDP
is configured through vCenter using the Properties dialog of the logical switch. See Figure51.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 83
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
FIGURE 51 NOS and vCenter Integration
Life of a Port-Profile
A port-profile during creation will go through multiple states. The states of a port-profile are:
Created-This state specifies that a port-profile is created or modified, but may not be complete.
Activated-This state specifies that a port-profile is activated and is available for MAC-> port-profile
association. If the port-profile created is not complete then the activation fails; you must resolve any
conflicts or dependencies and reactivate the port-profile.
Associated-This state specifies that one or more MAC addresses have been associated to this port-profile
within the fabric.
Applied-This state indicates that the port-profile is applied on the profiled port where the
associated MAC address appeared. In the absence of any signaling protocol, the system snoops the
packet to detect if the associated MAC address has appeared on the profiled port. Configuration of two
different port-profiles can co-exist on a profiled port, but if there is a conflict then the application of the
later port-profile fails.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
84 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Configuring FCoE Profiles
Only the FCoE profile of the default profile can be modified. This is allowed only when there are no other active
profiles on the switch.
The FCoE profile can only be part of the default profile. When it is part of the default profile, FCoE is enabled
globally and all the profiled ports automatically become FCoE ports.
In the absence of the FCoE profile in the default AMPP profile, you can configure FCoE on a per-interface
basis, based on the profiled ports
2013 Brocade Communications Systems, Inc. All Rights Reserved. 85
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Management and Troubleshooting
sFlow Global Configuration
Enable sFlow
RB1(config)# sflow enable
Configure sample rate. sFlow sample Rate: Default is 32768, range is from 2 1677721. A flow sample is
based on random packets being forwarded to the sFlow collector at defined numeric intervals, either for the
entire Brocade switch or for a single port interface. For example, every 4,096th packet is forwarded to the
sFlow collector for analysis and storage.
RB1(config)# sflow sample-rate 512
Configure polling interval (In seconds). sFlow polling interval: Default is 20, range is from 1 40.A polling
interval defines how often the sFlow octet and packet counter for a specific interface are sent to the collector,
but the sFlow agent is free to schedule the polling in order to maximize internal efficiency. If the regular
schedule is chosen, each counter start time will be chosen randomly to eliminate bottlenecks in network
performance.
RB1(config)# sflow polling-interval 20
SNMP community strings
SNMP versions 1 and 2c use community strings to restrict SNMP access. There are six defaultcommunity
strings configured for the user: three read-write strings and three read-only strings.
NOTE:
You can specify one of the six default community strings when the system first comes up. If you create a new
community string, you must delete one of the six default strings to make space for the new one.
The following community strings are read-write:
Secret Code
OrigEquipMfr
private
The following community strings are read-only:
public
common
ConvergedNetwork
Collecting Support Data
If escalation is required capture one of the following:
From the CLI run the copy support command for every switch in the fabric
From Brocade Network Advisor capture a supportsave from every switch in the fabric
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
86 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Security
Login Authentication
Authentication mode is defined as the order of authentication sources to be used for user authentication (the
login process). Two sources of authentication are supported: primary and secondary. The secondary source of
authentication is used in the event of primary source failover and is optional for configuration. There are three
possible sources:
Local (the default source)
RADIUS
TACACS+
TACACS+ Overview
The Terminal Access Controller Access-Control System Plus (TACACS+) is a protocol used in AAA server
environments. It is used only for authentication. A maximum of five TACACS+ servers can be configured per
switch. A role should be assigned to a user configured on the TACACS+ server and configured on the switch.
The user role is assigned by default when the following occur:
If the switch fails to get the users role from the TACACS+ server after successful authentication
If the role does not match any of the roles present on the switch
RBAC Overview
Role-based access control (RBAC) is used as an authorization mechanism. Roles can be created dynamically.
Roles are associated with rules to define permissions (i.e. read-only, read-write). User accounts must be
associated with only one role. Permissions cannot be assigned directly to the user accounts. Permissions can
only be acquired through the associated role. RBAC is the function of specifying access rights to resources for
roles. When a user executes a command, privileges are evaluated to determine access to the command
based on the role of the user.
Rules can be created for the specified operational commands. By default, every role can display all the
operational commands but cannot execute them. The show commands can be accessed by all the roles.
The following rules govern operational commands:
If a role has a rule with a read-write operation and the accept action for an operational command, the user
associated with this role can execute the command.
If a role has a rule with a read-only operation and the accept action for an operational command, the user
associated with this role can access but cannot execute the command.
If a role has a rule with a read-write operation and the reject action for an operational command, the user
associated with this role can neither access nor execute the command.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 87
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
DH-CHAP
Network OS use the Diffie Hellman - Challenge Handshake Authentication Protocol (DH-CHAP) to control
access between devices. DH-CHAP is a password-based, key exchange authentication protocol that negotiates
hash algorithms and Diffie Hellman (DH) groups before performing authentication. It supports both MD5 and
SHA-1 hash algorithm-based authentication.
The Fibre Channel Security Protocol (FC-SP) defines the DH groups supported in the DH-CHAP protocol.
Following current FC-SP standards, Network OS supports the following DH groups:
00 - DH Null option
01 - 1024 bit key
02 - 1280 bit key
03 - 1536 bit key
04 - 2048 bit key
To configure DH-CHAP authentication between Network OS switches (E_Ports) and FC routers (EX_Ports) you
must apply a matching configuration to both sides of the connection. Each device must be configured locally.
Configuring DH-CHAP shared secrets
To configure the DH-CHAP shared secrets, execute the fcsp auth-secret command in privileged EXEC mode.
Provide the following information as shown in the example:
The world wide name (WWN) of the peer.
The secret of the peer that authenticates the peer to the local switch.
The local secret that authenticates the local switch to the peer.
Configuring Security Profiles
A security profile defines all the security rules needed for the server port. A typical security profile contains
attributes for MAC-based standard and extended ACLs. Security profiles are applied to the ACLs based on the
profile or PolicyID. Therefore, multiple security profiles can be applied to the same profiled port
ACL Overview
ACLs filter traffic for the Brocade VDX hardware and permit or deny frames on ingress interfaces that have the
ACLs applied to them. You can apply ACLs on the three kinds of Layer 2 interfaces that Brocade Network OS v
2.1.1 supports; physical (both tengigabitethernet and gigabitethernet), VLAN, and port-channel (both static
and dynamic LAG).
Each ACL is a unique collection of permit and deny statements (rules) that apply to frames. When a frame is
received on an interface, the switch compares the fields in the frame against any ACLs applied to the
interface to verify that the frame has the required permissions to be forwarded. The switch compares the
frame, sequentially, against each rule in the ACL and either forwards the frame or drops the frame.
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
88 2013 Brocade Communications Systems, Inc. All Rights Reserved.
A security profile defines all the security rules needed for the server port. A typical security profile contains
attributes for MAC-based standard and extended ACLs.
MAC ACLs are supported on the following interface types and do not take effect until applied to a Layer 2
interface:
Physical interfaces
Logical interfaces (LAGs)
VLANs
ACL Logging
Logging-enabled ACLs provides insight into traffic as it traverses the network or is dropped by network
devices
The ACL logging feature is enabled whenever an ACL rule has "log" as one of its actions
Any packet that matches the rule is trapped to CPU and logged
CPU intensive
Firmware Download
During firmware upgrade/downgrade within a VCS Fabric, when mixed NOS v2.1.x and v3.0.0 firmware
versions exist
Fabric Distribution Services (FDS) are not supported
Commands such as show ip interface brief and show fabric islports show ISLs functioning
correctly, however show vcs will only show a single switch fabric until all switches in the fabric have been
upgraded to NOS v3.0.0 (Even if directly connected to another switch running NOS v3.0.0). All distributed
services in VCS Fabric mode are not supported, including: Zoning, Virtual IP and vCenter during the download
process.
Automatic firmware synchronization
When you replace a management module or insert a second management module into a chassis, the active
management module automatically synchronizes the hot-plugged standby management module with the
same firmware version. The standby management module reboots with the upgraded firmware. The
automatic firmware synchronization takes place only if all of the following conditions are met:
The standby management module is inserted while the chassis is up (hot-plugged insert).
There was no firmware download process running when the standby management module was inserted.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 89
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Firmware Upgrade Rules for vLAGs
vLAG VSID selection:
NOS v2.0.x: The VSID is derived from a virtual MAC address
NOS v2.1.x: The VSID is derived from the actual MAC address which is one of the participating RBridges
vLAGs cannot be shared between RBridges with different firmware releases (NOS v2.0.x and v2.1.0 or higher)
Best practice: vLAGs shared with another RBridge should be disabled before upgrading to v2.1.0 or higher
vLAGs are allowed between RBridges running the same code level
Two v2.0.x RBridges can share a vLAG and v2.1.0 or higher RBridges can share a different vLAG
Firmware Download Error Handling
If the firmware download command fails while a management module is upgraded, the process aborts and
the previous firmware is repaired on all modules and partitions. If the firmware download command fails while
an interface module is upgraded, the interface module is faulted, but the firmware download continues on the
remaining interface modules. If an interface module (line card) is faulted during a firmware download, issue
the power-off linecard command followed by the power-on linecard command to power cycle the module.
When the interface module comes back up, auto-leveling begins to upgrade the firmware
Troubleshooting Commands
FRU Status
SW1# show running-config system-monitor
system-monitor fan threshold marginal-threshold 1 down-threshold 2
system-monitor fan alert state removed action raslog
system-monitor power threshold marginal-threshold 3 down-threshold 4
system-monitor power alert state removed action raslog
system-monitor temp threshold marginal-threshold 1 down-threshold 2
system-monitor cid-card threshold marginal-threshold 1 down-threshold 0
system-monitor cid-card alert state removed action raslog
system-monitor sfp alert state none action none
system-monitor compact-flash threshold marginal-threshold 1 down-threshold 0
system-monitor MM threshold marginal-threshold 1 down-threshold 0
system-monitor LineCard threshold marginal-threshold 1 down-threshold 0
system-monitor LineCard alert state removed action raslog
system-monitor SFM threshold marginal-threshold 1 down-threshold 0
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
90 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Display the SFP information for a Specific Interface
show media interface [tengigabitethernet rbridge-id/slot/port | gigabitethernet rbridge-id/slot/port |
fibrechannel rbridge-id/slot/port]
Verifying LACPDUs
LACPDUs should be transmitted and received on both ends of the vLAG. This procedure verifies whether that
is happening, and also checks for PDU errors.
On both switches, enter the show lacp counter command to verify that LACPDUs are
transmitted and received, and there are no error PDUs.
switch# show lacp counter 10
% Traffic statistics
Port LACPDUs Marker Pckt err
Sent Recv Sent Recv Sent Recv
% Aggregator Po 10 1000000
Te 0/1 65 0 0 0 0 0
Te 0/2 64 0 0 0 0 0
Te 0/3 64 0 0 0 0 0
Te 0/4 0 0 0 0 0 0
In this case, LACPDUs are being transmitted by the switch, but none are being received
Error Messages
Message Excessive LSU length = <LSU length>.
Message Type LOG | FFDC
Severity ERROR
Probable Cause Indicates that the link state update (LSU) size exceeds the value that the system can support.
Recommended Action
Reduce the number of switches in the fabric or reduce the number of redundant inter-switch links (ISLs)
between two switches
Message Critical fabric size (<current RBridges>) exceeds supported configuration (<supported RBridges>).
Message Type FFDC | LOG
Severity CRITICAL
Probable Cause Indicates that this switch is a value-line switch and has exceeded the configured fabric size:
that is, a specified limit to the number of RBridges. This limit is defined by your specific value-line license key.
The fabric size has exceeded this specified limit and the grace period counter has started.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 91
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Recommended Action:
Bring the fabric size within the licensed limits. Either a full fabric license must be added or the size of the
fabric must be changed to within the licensed limit.
Ping Failure
If pings do not successfully traverse the switch, try the following operations.
1. Trace the packet flow and check whether ARP or ICMP packets are getting dropped.
2. Trace which direction is failing using interface statistics.
3. Locate the device that is dropping the packets.
4. Look for any error counters incrementing on that device.
5. Check the MAC address table to determine whether the MAC addresses are learnt on the correct port or
port-channel
BNA
NOS v3.0.0 supports these configuration backup types:
Discovery
Re-sync
Event based
Manual
Scheduled
Ethernet Fabric Traceroute
Source, destination1 and
VLAN information is required
RBridge ID can be manually entered or auto-detected from the source MAC address
Additionally user can provide:
Source and destination IP addresses
Layer 4 attributes like:
Protocol type (TCP/UDP)
Port numbers
Test count (1-10)
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
92 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Brocade Network Advisor v11.3 Enhancements
VDX Platform
8770-4 and 8770-8
NOS v3.0.0 support
Zoning alias
Address Finder
Fabric Traceroute
FastIron/NetIron
6650
FastIron v7.5
NetIron v5.4
Key Features
MCT& Y1731
Adapters
v3.1 release
Network Advisor
Licensing changes
Feature Usage Analysis
ESXi support
2013 Brocade Communications Systems, Inc. All Rights Reserved. 93
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
VDX Additional Functionality
Brocade NOS supports two modes of operation for Brocade VDX platforms:
Brocade VCS Fabric technology mode
- Ethernet Fabric
- FCoE
- DCB capable with iSCSI support
- VM-aware networking
Standalone mode
- Functions as a standard 802.3 switch
- Uses Spanning Tree to prevent loops (no Ethernet Fabric support)
- DCB capable with iSCSI support
- No FCoE support. End to End FCoE is only supported in Brocade VCS Fabric mode.
Here are the features available in both standalone and Brocade VCS fabric technology mode:
Layer 2 data forwarding
MAC learning and aging
BPDU Drop
PING and Trace Route
STP, Rapid Spanning Tree Protocol (RSTP), Multiple Spanning Tree Protocol (MSTP)
Per VLAN Spanning Tree Plus (PVST+), Per VLAN Rapid Spanning Tree Plus (PVRST+) [
LACP, Brocade ISL Trunking
Link-Level Discovery Protocol (LLDP) and Data Center Bridging Exchange (DCBX)
IEEE 802.1x
sFlow
Switched Port Analyzer (SPAN)
Layer 2 access control lists (ACLs)
Management port IP ACL (standard and extended)
SNMP
NETCONF support (RFC 4741)
LDAP v3 (RFC 4510)
Enhanced Transmission Selection (802.1Qaz)
Priority-based flow control (802.1Qbb)
Internet Group Management Protocol (IGMP) snooping
Automatic Migration of Port Profiles (AMPP)
In-band management
TACACS+
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
94 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Internet Small Computer Systems Interface (iSCSI) DCBX support
Here are features available only in Brocade VCS fabric technology mode:
Transparent LAN service
Virtual link aggregation groups (vLAGs)
Distributed configuration management
End-to-end FCoE
Fibre Channel fabric connectivity
VM-aware networking
Virtual IP Address
Allows administrator to access a VCS fabric using one virtual IP address.
Not available in Standalone Mode: Standalone Mode is one VCS Fabric disabled switch. One VCS Fabric
enabled switch still constitutes a VCS Fabric.
The Brocade VCS Fabric can be assigned an IP address, known as a Virtual IP (VIP) address. When the VIP is
used to connect to the fabric, successful authentication will log in the user into the Coordinator (principle
switch) of the fabric.
By default, a virtual IP address (VIP) is not set, once set, it is always bound to the Coordinator. When the
Coordinator fails over, it will be bound to the new Coordinator (Principal Switch).
When two fabrics are merged, the new virtual IP address is the one that was set for the new Coordinator (new
Principal Switch). When a fabric is segmented, both fabrics retain the virtual IP address.
When accessing the coordinator using the VIP any changes to the running configuration will automatically get
copied to the startup configuration.
The VIP can be used to manage vCenter Integration, BNA Discovery, Zoning and management of the
coordinator switch.
Selecting the MTU
Always set the switch MTU to the maximum host MTU plus 100 bytes. This method is recommended because
the definition of MTU sometimes varies among different vendors. If the switch MTU is set to the same as the
connected host MTU, packets could be dropped.
2013 Brocade Communications Systems, Inc. All Rights Reserved. 95
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Registered State Change Notification
The Brocade VDX hardware Registered State Change Notification (RSCN) function operates as
follows:
RSCN events generated in the FC fabric are forwarded to the ENodes. RSCN events generated on the
FCoE side are forwarded to the FC devices. DCB is not aware of RSCN events.
Device RSCN-An RSCN is generated to all registered and affected members when an ENode either logs in
or logs out of an FCF through any means. An RSCN is generated when an FC N_port device either logs in
or logs out of the FC fabric.
When transmitting an RSCN, zoning rules still apply for FCoE devices as the devices are treated as regular
FC N_ports.
VF_port RSCN-An RSCN is generated to all registered members when a VF_port goes online or offline,
causing ENode or FC devices to be added or removed.
Domain RSCN-An RSCN is generated to all registered and affected members when an FC switch port goes
online or offline, causing ENode or FC devices to be added or removed. An RSCN is generated when two
FC switches merge or segment, causing ENode or FC devices to be added or removed. When FC switches
merge or segment, an RSCN is propagated to ENodes.
Zoning RSCN-An RSCN is generated to all registered and affected members when a zoning exchange
occurs in the FC fabric
Fabric Commands
show firmwaredownloadhistory
Displays the history of firmware downloads.
Synopsis show firmwaredownloadhistory [rbridge-id {rbridge-id | all}]
show port-channel
Displays the Link Aggregation Group (LAG) information for a port-channel.
Synopsis show port-channel [channel-group-number | detail | load-balance | summary]
Operands channel-group-number Specifies a LAG port channel-group number to display. The range of valid
values is from 1 through 63 for standalone mode. The range of valid values is from 1 through 6144 for
Brocade VCS Fabric mode.
detail Displays detailed LAG information for a port-channel.
load-balance Displays the load-balance or frame-distribution scheme among ports in the port-channel.
summary Displays the summary information per channel-group.
Defaults None
Command Modes Privileged EXEC mode
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
96 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Description Displays the LAGs present on the system with details about the LACP counters on their member
links. LAG interfaces are called port-channels.
Usage Guidelines If you do not specify a port-channel, all port-channels are displayed.
switch# show port-channel 38
LACP Aggregator: Po 38
Aggregator type: Standard
Admin Key: 0038 - Oper Key 0038
Partner System ID - 0x8000,01-e0-52-00-20-00
Partner Oper Key 0038
Member ports:
Link: Te 0/13 (0x180D0102) sync: 1
Link: Te 0/14 (0x180E0103) sync: 1
oscmd
Provides a command shell for selected Linux commands.
Synopsis oscmd Linux command
Operands Linux command The following Linux commands are supported with oscmd:
arp [-a] Displays the Address Resolution Protocol (ARP) tables.
cat Concatenates files and displays to standard output.
cp Copies files and directories in a file system.
ftp Transfers files to and from a remote server.
ifconfig [netmask] [up]
Configures the active network interface.
ls [-al] [path] Lists files and directories on the switch.
mkdir dir Creates a directory.
mv [i] file1 file2 Renames a file or directory.
rm [-rf] file Removes a file or directory.
rmdir Removes a directory.
tcpdump Analyzes network traffic. The following options are supported with the Network OS implementation.
Refer to the Linux documentation for more information on how to use this command.
-B buffer_size
-c count
-C file_size
2013 Brocade Communications Systems, Inc. All Rights Reserved. 97
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
-G rotate_seconds
-F file
-i interface
-m module
-M secret
-r file
-s snaplen
-T type
-w file
-W filecount
-E spi@ipaddr
-y datalinktype
-z postrotate-command
-Z user [expression]
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
98 2013 Brocade Communications Systems, Inc. All Rights Reserved.
Taking the Test
After the Introduction Screen, once you click on Next, you will see the following non-disclosure agreement:
IMPORTANT: PLEASE READ THE FOLLOWING BROCADE NON-DISCLOSURE CONFIDENTIALITY AGREEMENT
CAREFULLY BEFORE TAKING THIS EXAM.
The following Non-Disclosure Confidentiality Agreement (the Agreement) sets forth the terms and
conditions of your use of the exam materials as defined below.
The Disclosure to you of this Exam and any questions, answers, worksheets, computations, drawings,
diagrams, or any communications, including verbal communication by any party, regarding or related to
the Exam and such Exam Materials and any derivatives thereof is subject to the Terms and Conditions of
this Agreement.
You understand, acknowledge and agree:
- That the questions and answers of the Exam are the exclusive and confidential property of Brocade
and are protected by Brocade intellectual property rights;
- That you may not disclose the Exam questions or answers or discuss any of the content of the Exam
Materials with any person, without prior approval from Brocade;
- Not to copy or attempt to make copies (written, photocopied, or otherwise) of any Exam Material,
including, without limitation, any Exam questions or answers;
- Not to sell, license, distribute, or give away the Exam Materials, questions, or answers;
- You have not purchased, solicited or used unauthorized (non-Brocade sanctioned) Exam Materials,
questions, or answers in preparation for this exam;
- That your obligations under this Agreement shall continue in effect after the Exam and, if applicable,
after termination of your credential, regardless of the reason or reasons for terminations, and whether
such termination is voluntary or involuntary.
Brocade reserves the right to take all appropriate actions to remedy or prevent disclosure or misuse,
including, without limitation, obtaining an immediate injunction. Brocade reserves the right to validate all
results and take any appropriate actions as needed. Brocade also reserves the right to use any
technologies and methods for verifying the identity of candidates. Such technology may include, without
limitation, personally identifiable information, challenge questions, identification numbers, photographic
information, and other measures to protect against fraud and abuse.
Neither this Agreement nor any right granted hereunder shall be assignable or otherwise transferable by
you.
By clicking on the "A" button (YES, I AGREE), you are consenting to be bound by the terms and
conditions of this agreement and state that you have read this agreement carefully and you understand
and accept the obligations which it imposes without reservation. You further state that no promises or
representations have been made to induce agreement and that you accept this agreement voluntarily and
freely.
A. YES, I AGREE
B. NO, I DO NOT AGREE
2013 Brocade Communications Systems, Inc. All Rights Reserved. 99
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition
100 2013 Brocade Communications Systems, Inc. All Rights Reserved.

You might also like