BCFD in A Nutshell Study Guide For Exam 143-260 8gbit

BCFD in a Nutshell
Study Guide for

Exam 143-260
Exam Preparation Materials
Revision November 2008

Corporate Headquarters - San Jose, CA USA
T: (408) 333-8000
info@brocade.com
European Headquarters - Geneva, Switzerland

T: +41 22 799 56 40
emea-info@brocade.com
Asia Pacific Headquarters - Singapore

T: +65-6538-4700
apac-info@brocade.com
© 2008 Brocade Communications Systems, Inc. All Rights Reserved.
Brocade, the Brocade B-weave logo, Fabric OS, File Lifecycle Manager, MyView, Secure Fabric OS,
SilkWorm, and StorageX are registered trademarks and the Brocade B-wing symbol and Tapestry are
trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries.
FICON is a registered trademark of IBM Corporation in the U.S. and other countries. All other brands,
products, or service names are or may be trademarks or service marks of, and are used to identify,
products or services of their respective owners.
Notice: This document is for informational purposes only and does not set forth any warranty,
expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered
by Brocade. Brocade reserves the right to make changes to this document at any time, without notice,
and assumes no responsibility for its use. This informational document describes features that may not
be currently available. Contact a Brocade sales office for information on feature and product availability.
Export of technical data contained in this document may require an export license from the United
States government.
Revision: November, 2008

BCFD in a Nutshell 8 Gbit/sec Edition
Objective: The BCFD Nutshell guide is designed to help you prepare for the BCFD Certification, exam number
143-260.
Audience: The BCFD Nutshell self-study guide is intended for those who have successfully completed the
CFD 200 Brocade 8 Gb Certified Fabric Designer course, and who wish to undertake self-study or review
activities before taking the actual BCFD exam. The BCFD guide is not intended as a substitute for classroom
training or hands-on time with Brocade products.
How to make the most of the BCFD guide: The BCFD guide summarizes the key topics on the BCFD exam for
you in an easy to use format. It is organized closely around the exam objectives. We suggest this guide be
used in conjunction with our free online knowledge assessment test - CFD 201 BCFD Knowledge Assessment.
To benefit from the BCFD guide, we strongly recommend you have successfully completed the CFD 200
Brocade 8 Gb Certified Fabric Designer course.
We hope you find this useful in your journey towards BCFD Certification, and we welcome your feedback by
sending an email to jcannata@brocade.com.
Helen Lautenschlager Joe Cannata

Director of Education Solutions Certification Manager
©2008 Brocade Communications 1

2 ©2008 Brocade Communications

Table of Contents
1 - Data Collection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
How to Assess Current and Target SAN Infrastructure Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Tools to Assess the Current SAN Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
SAN Health . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Top Talker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Fabric Watch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Current SAN Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Target SAN Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
SAN Design Installation Plan Documentation Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Gathering Business Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Key Items to Collect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 - Design Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Designing a Solution To Meet Customer Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Hardware Products - Switches, Directors and Backbone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
B-Series Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Hardware Products - Fabric Extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Availability Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Design Assessment Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Reference Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Data to Gather. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3 - Management and Monitoring Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Tools for Gathering Monitoring Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Applying Management Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Repositories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
FCIP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4 - Hardware/Software Products and Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Fabric OS and M-EOS Interop. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Interop Modes Introduced in FOS v6.x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
FC Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Integrated Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Access Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Access Gateway Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Access Gateway Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
F_Port Trunking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Brocade DCX Backbone ICLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Software Products. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Adaptive Networking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Port Fencing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Cascaded FICON Requirements and Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5 - Distance Solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
FCIP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
FCIP Performance Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
The IPPerf Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Supported FCIP Platform Connectivity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Supported FCIP Tunnel Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
FCIP FastWrite and Tape Pipelining. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
FCIP FastWrite. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Tape Pipelining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Fibre Channel Long Distance Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Native FC Over Dark Fiber . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Wave Division Multiplexing (WDM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Time Division Multiplexing (TDM). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
SONET/SDH. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Extended Distance Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
6 - Performance Tuning and Optimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Quantifying Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Proximity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
ISL Trunking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
M-EOS Open Trunking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
ISL Oversubscription. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Leveraging 8 Gbit/sec Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Choosing the Right Routing Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Fabric Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Exchange-based . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Port-based . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Design Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Preferred Pathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
7 - Migration and Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Migrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Migration Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Migration Plan Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Migration Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
M-Series to B-Series Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Using a Router. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
M-Series to B-Series Migration Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Design Element Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Integrated/Consolidated Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Integrating 8 Gbit/sec Into 4 Gbit/sec Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Why Do It? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Inserting a DCX at the Core When 48000s are Deployed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
8 - Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Access Control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Password Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Role Based Access Control (RBAC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Authentication, Authorization, Accounting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Access Control Lists (ACLs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Access Control Lists – Device and Switch Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Access Control Lists – Administrative Access. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Securing DCF Management Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Taking the Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

List of Tables
Switch Attribute Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
FC Router Platforms Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

List of Figures
Sample SAN Health Report ....................................................................................................................................................................................................... 8
B-Series Switches ................................................................................................................................................................................................................... 13
Ports on Demand Increments ................................................................................................................................................................................................ 14
Buffer Credits .......................................................................................................................................................................................................................... 14
Backbone and Directors ......................................................................................................................................................................................................... 15
Single Switch Topology ........................................................................................................................................................................................................... 16
Cascade and Ring Topologies ................................................................................................................................................................................................ 17
Full Mesh Topology ................................................................................................................................................................................................................. 18
Core/Edge Topology................................................................................................................................................................................................................ 19
Non-resilient vs. Resilient....................................................................................................................................................................................................... 20
Fibre Channel Routed Fabric ................................................................................................................................................................................................. 25
Brocade 7500E and FR4-18i Routers................................................................................................................................................................................... 26
Access Gateway ...................................................................................................................................................................................................................... 29
Access Gateway Use Case...................................................................................................................................................................................................... 30
Scalability Using Access Gateway .......................................................................................................................................................................................... 30
F_Port Trunking ....................................................................................................................................................................................................................... 31
Two DCX Bacbones With ICLs ................................................................................................................................................................................................ 32
Port Fencing Concept.............................................................................................................................................................................................................. 34
FCIP Implementation Example............................................................................................................................................................................................... 35
FCIP Tunnels on a GbE Port ................................................................................................................................................................................................... 36
Brocade 7500 & FR4-18i ....................................................................................................................................................................................................... 36
IPPerf Example ........................................................................................................................................................................................................................ 38
Single Fabric Using VE_Port-to-VE_Port Connection............................................................................................................................................................. 39
Two Routed Fabrics Using VEX_Port-to-VE_Port Connection ............................................................................................................................................... 39
FCIP FastWrite......................................................................................................................................................................................................................... 40
Tape Pipelining........................................................................................................................................................................................................................ 41
FastWrite and Pipeling Problem Installation......................................................................................................................................................................... 42
Local Proximity ........................................................................................................................................................................................................................ 45
Remote Proximity .................................................................................................................................................................................................................... 46
Benefits of Trunking................................................................................................................................................................................................................ 47
Fabric Without Open Trunking ............................................................................................................................................................................................... 49
Fabric With Open Trunking ..................................................................................................................................................................................................... 49
Oversubscription Ratio ........................................................................................................................................................................................................... 50
Migration Process Phases ...................................................................................................................................................................................................... 54
M-Series Migration to B-Series Using Routers ...................................................................................................................................................................... 55
Merging Fabric OS and M-EOS Fabrics................................................................................................................................................................................. 56
Inserting DCXs at the Core ..................................................................................................................................................................................................... 57
RADIUS Configuration ............................................................................................................................................................................................................. 59
ACL Policies ............................................................................................................................................................................................................................. 61
Sample NDA ............................................................................................................................................................................................................................ 62
Sample Question..................................................................................................................................................................................................................... 63
Example Exam Score .............................................................................................................................................................................................................. 64

1 - Data Collection
How to Assess Current and Target SAN Infrastructure Requirements
Tools to Assess the Current SAN Infrastructure

• supportshow
• SAN Health
• Top Talkers
• Fabric Watch
supportshow is a simple way of gathering the state of each switch in the SAN. It can be a time-consuming
process. Also related to this would be uploading the switch configuration files.
SAN Health
• A tool that automates the documentation of a SAN
- For SAN design purposes
• SAN Health is a free utility that helps you create:
- Comprehensive documentation
- Historical performance graphs
- Detailed topology diagrams
- Best practice recommendations
Top Talker
• The Top Talker (TT) is an enhancement to Advanced Performance Monitor end-to-end monitors
- First available in Fabric OS v6.0
- When enabled, these monitors determine which SID-DID pairs are the major users of switch F_Port
bandwidth
- Can be enabled on specific switch E_Ports or F_Ports in the fabric
Fabric Watch
• A licensed product that monitors switch thresholds

At A Glance – Comprehensive
SAN Visibility
Error Checking and Analysis
Figure 1: Sample SAN Health Report
Current SAN Requirements

• Any SAN design must take into consideration the existing SAN infrastructure
• Before creating a Data Center Fabric (DCF) design, collect information on the following requirements:
- SAN-Enabled Applications
- SAN Infrastructure
o Is server or storage consolidation an option?
o What is the current storage utilization?
o What are the current backup needs?
o Is there a disaster tolerance plan in effect?
o Are there additional security needs?
o Is the current SAN scalable?
o Is server or storage virtualization an option?
- Servers
o Name, manufacturer and model
o OS type, revision level
o Ethernet requirements with I/O profile
o Number of HBAs

o HBA manufacturer, model and driver revision

o Disk storage source, amount, growth rate
o Physical location
- Storage (Disk and Tape)
o Name, manufacturer and model
o Connection count
o I/O profile
o Software applications
o Usable storage capacity and capacity being used
o Is remote mirroring being used?
o Physical location
- Availability (Data and Application)
- Performance
- Security
- SAN FC-to-FC Routing Infrastructure
o Number of front, translate and physical domains per edge fabric
o Number of local and remote devices per edge fabric
o Number imported/exported devices per edge fabric
o Number of FC routers per routed fabric
- LAN Infrastructure
- Multiprotocol Infrastructure
- Facilities
Target SAN Requirements

• For the target SAN environment, collect the same information as for the current SAN environment:
- Facilities
- SAN Infrastructure
- Servers and Storage (Disk and Tape)
- Availability (Data and Application)
- Performance
- Security
- LAN and Multiprotocol Infrastructure
• Focus on new initiatives, planning, and growth
- Growth is driven by business requirements, solving problems, and addressing scalability
- Growth is often measured by rate, port or device count and storage capacity
- Capture how often changes will be made
- Determine how much rack space is dedicated to future cabling

- Obtain customer’s target time for pilot and first implementation; assess overall time line
- Document all switch vendor and SAN partner information
- Document all support providers
SAN Design Installation Plan Documentation Components

• Data Center Functional Deployment Plan
- Translate SAN design and assessment data into a functional deployment plan
- Components include
o SAN installation and production documentation
o A verification plan
o A growth plan
• Logical SAN Layout Diagram
- A simple picture of the logical design
• Datacenter Interface Plan
- Floor space, power and cooling layout
- Fiber cable requirements of devices
- Rack layouts
- Ethernet and serial access requirements
Gathering Business Requirements
Key Items to Collect

• What are the business challenges that need to be addressed by the SAN solution
• Look for both positive and negative statements of need - help the customer define their requirements
- Consolidation
o Too many slow servers, we are moving to fewer, faster servers
o Too many small, under used storage systems
- Availability
o Need a more rigorous disaster recovery solution
o Need an always-on infrastructure, designed around a redundant architecture that includes
high MTBF, low MTTR components.
- Competitive Advantage
o Backups are still running in the morning when we begin entering new orders
o Data mining activities take too long
• Once the problem is identified, rephrase it as a quantitative technical requirement against which success
or failure can be measured
- Server and Storage Consolidation
- High Availability
- Backup Needs

- Disaster Tolerance and Business Continuance

- Performance
- Manageability
- Scalability
- Security Needs
- Return on Investment
- Uptime
- Reduced Total Cost of Ownership
- Storage Utilization
- Maintenance Windows

2 - Design Assessment
Designing a Solution To Meet Customer Requirements
Hardware Products - Switches, Directors and Backbones
Before designing a SAN, it is important to identify the resources that are available to use as “building blocks”
in your infrastructure design.
Before discussing design concepts, here are the Brocade products and services that you may use in a design:
• Hardware solutions:
- Switches
- Directors and Backbones
- Fabric routing and extension products
- Server connectivity options
- Access Gateway
• Software solutions
- FC Routing
- FCIP
- Trunking
- Adaptive Networking
- Zoning
- FICON

B-Series Switches
Figure 2: B-Series Switches
Table 1: Switch Attribute Summary
Brocade Brocade Brocade Brocade Brocade Brocade

Attribute 200E 4900 5000 300 5100 5300
Port Density 8, 12, 16 32, 48, 64 16, 24, 32 8, 16, 24 24, 32, 40 48, 60, 64
Port Speed 4 Gbit/sec 4 Gbit/sec 4 Gbit/sec 8 Gbit/sec 8 Gbit/sec 8 Gbit/sec
Form Factor 1U 2U 1U 1U 1U 2U
Watts per Gbit/sec .94 .68 .55 .30 .28 .43
USB Device Support N N N N Y Y
FICON N Y Y N Y Y
Integrated Routing or FCR N N N N Y Y
HA - Dual Power/Cooling N Y Y N Y Y
Adaptive Networking N N N Y Y Y

Figure 3: Ports on Demand Increments
Figure 4: Buffer Credits

Figure 5: Backbone and Directors
Hardware Products - Fabric Extension

• Fabric Extension and Routing
- 7500, 7500E, FR4-18i and USD-X
- Integrated Routing on 5100, 5300 and DCX

Topologies
Single Switch
• The simplest DCF solution has only a single switch
o For higher availability, start with two unconnected switches
• Pro: Locality between server and storage; well-suited to higher-port count switches
• Con: Single fault domain
Figure 6: Single Switch Topology

Cascade and Ring

• The cascade topology is the simplest for small DCF solutions
o Connect a series of switches in a simple queue
• The ring topology is a variation on the cascade
o Connect the switches to “close the ring”
• Pro: Lowest port count for ISLs
o Straightforward way to simplify switch management
o DCF starts small, stays small, and has high levels of locality
• Con: Neither topology scales well
Cascade
Ring
Figure 7: Cascade and Ring Topologies

Full Mesh
• The full mesh topology connects each switch to every other switch in the DCF
• Pro: Provides more paths than Cascade or Ring; can add new servers or storage to any open port
(maximum of one hop to any device); can leverage ISL Trunking; can increase locality
• Con: Does not scale well; only a single, lowest-cost path, so cannot leverage DPS
Full Mesh
Figure 8: Full Mesh Topology
Core/Edge
• For better scalability, the Core/Edge Topology specializes the role of the switch:
- Connect servers and storage to switches at the Edge
- Connect switches at the Core to the Edge switches
• The result is a resilient topology with two Core switches
• The Core/Edge topology improves scalability, availability, and performance
- Adding a new Edge switch requires connections only to the Core switches, which makes this a
highly scalable topology
- There are always two equal-cost paths between any two Edge switches which leverages DPS and
ISL Trunking
- Having two Core switches makes this is a highly resilient topology

Figure 9: Core/Edge Topology
The Core/Edge topology shown above modifies the “connect-to-all-switches” methodology of the Full Mesh to
achieve better scalability. The switches to which servers and storage are attached are moved to the outside –
or “Edge” – of the topology. The Edge switches are not connected directly to each other; instead, they are
connected into a fabric through two or more connection-only switches at the “Core” of the fabric.
In the example above, the two Brocade DCX Backbones are acting as the Core switches, while four Brocade
5300 switches are placed as the Edge switches.

Availability
• A redundant SAN solution is based on at least two resilient fabrics
• Design for long term maintenance, which will allow you to test new solutions on one fabric while
maintaining data flow through the other fabric
• Make each fabric resilient enough to survive hardware, software, user and site failures
• When assessing the availability of the fabric, a well-connected SAN should avoid single points of failure
(SPOFs): any component that, if failed, results in devices and/or switch(es) becoming disconnected from
the fabric
- Can include: ISLs, ports, switches, HBAs, storage ports, fabric itself
• Fabrics with no SPOFs are considered resilient fabrics
- Each server has at least 2 HBAs, multi-path software and is connected to both fabrics
- Each storage system has at least 2 storage ports; with connections into different fabrics
- Using Directors or Backbones greatly improves availability
Availability Levels
• There are four basic levels of availability in a Core/Edge-based design:
• Level 1: Single fabric, single Core: Not Highly Available
- Fabric and Core are single points of failure (SPOFs)
• Level 2: Single fabric, multi-Core: Not Highly Available
- Core is no longer an SPOF; fabric is an SPOF
• Level 3: Two fabrics, single Core: Highly Available (better if a Director is the Core)
- No SPOFs; most popular deployment – but not the highest HA level
• Level 4: Two fabrics, multi-Core: Most Highly Available
- No SPOFs; more faults caught in the fabrics, rather than at the HBA/storage level
Figure 10: Non-resilient vs. Resilient

Design Assessment Documentation
Reference Sources
There are several different types of documentation required for a design assessment. They range from
technical manuals to tools or methods to collect certain specific pieces of information. Technical manuals
include:
• Admin Guides
• EFCM or Fabric Manager User Manuals
• SAN Health Documentation
• Advanced Performance Monitor Manual
• Hardware Reference Manuals
Data to Gather
Miscellaneous information to be gathered:

• Site information
- Power usage
- Cooling
- Free rack space
• Total Cost of Ownership (TCO) and Return on Investment (ROI) expectations
• Expected growth rates and changes
- Pull data using a management tool
• Project implementation time lines

3 - Management and Monitoring Tools
Tools for Gathering Monitoring Information

• Fabric Watch
• Top Talkers
• EFCM Enterprise
• EFCM Basic
• Fabric Manager
• portperfshow
• SAN Health
• Web Tools performance views
Applying Management Tools
Security
Management of switch accounts can be achieved using:

• LDAP
• RADIUS
• Working switch by switch
• Password policies
• SSH public key authentication
Repositories
Information could be centrally stored for many reasons:

• Support
• Repository for firmware
• Configuration file uploads
Zoning
Zoning information could be gathered using:

• Fabric Manager for B-Series
• EFCM for M-Series or B-Series
FCIP
Managing of FCIP can be achieved with any of the GUI products designed for management.

4 - Hardware/Software Products and Features
Interoperability
Fabric OS and M-EOS Interop

• Fabric OS v6.1.x interoperability options:
- Interopmode 2 - for McDATA Fabric mode, which supports M-EOS switches v9.6.2 and later running
in McDATA Fabric mode
- Interopmode 3 - for McDATA Open Fabric mode, which supports M-EOS switches v9.6.2 and higher
running in Open Fabric mode
- Interopmode 0 - for Brocade Native mode, which supports all stand-alone Brocade fabrics, but no
interoperability support
• You can no longer configure switches for interopmode 1; interopmode 3 replaces it
Interop Modes Introduced in FOS v6.x

• Interopmode 2 - McDATA Fabric Mode
- Allows management of zoning from B-Series or M-Series switches
- Supports most Fabric OS features
- Ideal option for seamlessly adding Fabric OS products to M-EOS fabrics in McDATA Fabric Mode
- Cannot co-exist in fabrics with Fabric OS 5.x Interopmode 1
- Supports larger fabrics
- M-EOS v9.6.2 minimum (required)
- Should never be used in a pure Fabric OS fabric
• Interopmode 3 - Open Fabric Mode
- Zoning done via M-EOS switches when in Interopmode 3
- Some FOS features not qualified in Interopmode 3 (refer to Release Notes for latest information)
- Ideal option for adding Fabric OS products to M-EOS fabrics in Open Fabric Mode
- Cannot co-exist in fabrics with Fabric OS 5.x Interopmode 1
- Limited to smaller fabrics (15 domains/800 devices)
- M-EOS v9.6.2 minimum (required)
- Should never be used in a pure Fabric OS fabric
- Only supports M-Series products – no other vendors

FC Routing
Overview
• Many of today’s IT organizations manage multiple Storage Area Network (SAN) islands to support specific
applications, projects, and sites throughout their enterprise Data Centers
- Increasingly, administrators are having to share resources between fabrics
• Organizations can interconnect their SAN islands for greater resource utilization, scalability, and long-
distance extension
• FC Routing:
- Introduces an advanced level of device connectivity without the associated risk and complexity of
physically merging SAN islands into a single large fabric
- Supports strategic business initiatives such as disaster recovery, data migration, new data center
infrastructures, ongoing technology upgrades, and data center infrastructure consolidation
- Simplifies interconnection and support for heterogeneous SAN environments in different interoper-
ability modes
• Fibre Channel Routing can be implemented between distinct fabrics to facilitate physical and logical
connectivity
• Introducing FC Routing into your existing SAN environment requires:
- Physical Connectivity between FC fabrics
• Connect autonomous fabrics (SAN Islands) to an FC Router platform via EX_Ports
- Logical Connectivity between FC devices
• Configure and enable LSAN zones in each autonomous fabric to specify which devices will be shared
• FCR enables device sharing access across fabrics with minimal changes to the existing fabrics
• The Fibre Channel Router (FCR):
- Implements fabric-to-fabric routing (Layer 3)
- Fabrics can be B-Series, M-Series, or a mixture of B- and M-Series
- Continues to provide basic Fibre Channel switch functionality (Layer 2)
- Uses an EX_Port to prevent the fabrics from merging; an IFL forms
• Design implementation considerations related to EX_Port Physical Connectivity
- Identify a meaningful domain ID numbering convention and assign domain IDs
o A unique range of IDs for physical switches
o A unique range of IDs for translate (xlate) domains (XDs)
o A unique range of IDs for front domains (FDs)
- Identify a meaningful fabric Identification convention and assign FIDs
o A unique range of FIDs for the backbone (BB) fabrics
o A unique range of FIDs for the Edge fabrics
- Connect inter-fabric links (IFLs) to take advantage of trunking between the FCR and each edge fab-
ric
- Continue using single initiator zoning when creating LSAN zones
- Make LSAN zone names the same in each fabric sharing the devices, for easy identification

Figure 11: Fibre Channel Routed Fabric

Platforms
• The Brocade FR4-18i is a bladed version of the Brocade 7500 SAN Router and both provide FC Routing
service
- As of Fabric OS v5.3, they both support FC FastWrite
o Improves SCSI write operations over long-distance ISLs
o accomplished by the creation of a Proxy Target (PT) local to the initiator and a Proxy Initiator
(PI) local to the target storage device
o
• 16 FC ports for devices, switching, and FC routing (1/2/4 Gbit/sec; F, FL, E, and EX_Ports)
• 2 GbE ports for FCIP and FC Routing
• The FR4-18i can only be installed in the Brocade 48000 Director and the Brocade DCX Backbone
• No FC Routing license required on the FR4-18i or 7500
• The Brocade 7500E SAN Router is an entry level 1U platform that supports advanced FC Routing
functionality
- Limited to 2 FC ports for devices, switching, and FC routing (1/2/4 Gbit/sec; F, FL, E, and EX_Ports)
- Redundant hot-pluggable power supplies, fans, SFPs
- Supports non-disruptive firmware activation
- License upgradeable to full Brocade 7500 capabilities
- No FC Routing license required
FC and
FC Routing
Figure 12: Brocade 7500E and FR4-18i Routers

Integrated Routing
Platforms
• All ports on Brocade 5100 and 5300 8 Gbit/sec switches, as well as all ports on any FC8-xx port blade
installed in the DCX, are available for configuration as EX_Ports
- Integrated Routing is a “per-chassis” licensed feature introduced in Fabric OS v6.1
- Ports are individually configurable
- Licensing is checked upon configuration and enablement of the EX_Port
• Integrated Routing (IR) Restrictions
- Not supported on:
o Core (CR8) blades in the DCX
o Brocade 300 switches
o 8 Gbit/sec blades in the 48000
o 4 Gbit/sec switches or blades
o Embedded switches
• Integrated Routing:
- Can use any port on any 8 Gbit/sec blade in the DCX for FC routing
- Can use any port on the 5100 or 5300 for FC routing
- At 8 Gbit/sec, IR offers twice the bandwidth of the 4 Gbit/sec FCR hardware platforms
Limitations
• No M-Series switches are allowed in the Backbone fabric
• The DCX must be disabled to change the backbone fabric ID, if using FC8 blades
• Firmware Upgrade/Downgrade
- IR licensed EX_Ports must be unconfigured if downgrading below Fabric OS v6.1
• Maximum 128 EX_Ports and 48 Front Domains per DCX

Table 2: FC Router Platforms Comparison
Brocade 7500E
Brocade Upgrade
Capability 7500 Brocade 7500E License
Number of Fibre Channel Ports 16 2 14 additional
Redundant power supplies and fans   n/a
Fibre Channel switch qualified  Only 2 FC ports - no switching 
FCR across FCIP links (isolation)   n/a
FCR between edge fabrics  Only 2 FC ports - no routing 
Tunnels per Gigabit Ethernet port 8 1 7 additional/port
Committed rate per FCIP tunnel Up to 1G Up to 50 Mbps per GbE port Increased to 1G
FCIP-FW FastWrite over FCIP   n/a
FC-FW FastWrite over native Fibre Channel  - 
Open Systems Tape Pipelining (OSTP)  - 
FICON emulation (XRC and tape)  - 
Call Home  - 
Hardware-based encryption  - 
Hardware-based compression   n/a
Storage-Optimized TCP (SO-TCP)   n/a
Managed by DCFM   n/a

Access Gateway
Access Gateway Features

• The Brocade Access Gateway option aggregates Fibre Channel (FC) traffic from all the blade servers in a
chassis without consuming domain IDs (NPIV support required)
• Fabric OS feature that allows the configuration of an Enterprise fabric to focus on additional N_Ports
instead of domains
• It greatly improves the scalability of blade server environments by eliminating a large number of low-port-
count switches
• It improves manageability by cleanly separating server administration from SAN administration
• Allows server administrators to configure the Access Gateway features with no adverse impact on the
fabric
• F_Port Trunking, high bandwidth utilization, and automatic link failover simplify management and reduce
costs
• With NPIV support, storage management is extended past the physical server HBA to the virtual machine
• Can be managed by either server team or SAN team
• Design considerations:
- No license required
- No increase in domain count which improves the scalability of the fabric
- Focus is connectivity, not bandwidth
- HBAs are mapped via NPIV
- Can connect to Brocade, Cisco, and M-Series fabrics
- Supported on Brocade 200E, 300 and embedded switches
Figure 13: Access Gateway

Access Gateway Use Cases

• Large server count environments where management of multiple domains is increasingly complex and
limiting
• Mixed-vendor SAN configurations that utilize full capabilities, without the restrictions of interoperability
mode
Figure 14: Access Gateway Use Case
Figure 15: Scalability Using Access Gateway

F_Port Trunking
• F_Port Trunking aggregates the bandwidth of the ports connecting an Access Gateway to the edge switch in
the fabric
• Design considerations:
- Trunking license required
- Port group to port group
- Same speed
- Ports enabled for trunking on switch
- Device PIDs do not change upon failed link in the trunk
Figure 16: F_Port Trunking

Brocade DCX Backbone ICLs

• Inter Chassis Links (ICLs) are ISL connections between two DCX switches on the CR8 (Core) blades
• Creates a dual DCX core fabric: Doubles the port count to 768 user ports at the core of the fabric without
the use of any user ports
• Two domains
• ICLs provide an aggregate bandwidth of 1 Tbit/sec
• For FICON purposes, the ICLs are considered a “hop of no concern”
• Each cable provides 16 x 8 Gbit/sec bandwidth
• Licensed feature
• No user ports are required for ICL connections
• ICLs are oversubscribed
- 4:1 at 256 ports @ 8 Gbit/sec
- 6:1 at 384 ports @ 8 Gbit/sec
Figure 17: Two DCX Backbones With ICLs

Software Products
Adaptive Networking
• Adaptive Networking is a suite of tools and capabilities that enable optimized behavior in a SAN
• Traffic Isolation Zones
- Provides the ability to dedicate ISLs to specific hosts and targets; no license required
- Uses zoning to implement
- Available on all 4 & 8 Gbit/sec switches running Fabric OS 6.x
- Does not supersede FSPF rules
- Can dedicate an ISL to high priority traffic
- Can force high volume, low priority traffic onto an ISL to limit its effect on the fabric
• Ingress Rate Limiting
- Restricts speed of traffic from a particular device to the switch port
- ASIC delays the return of BB credits to the external device
- Design considerations:
o Adaptive Networking license required
o Available only on an 8 Gbit/sec capable F_Port or FL_Port
o Can proactively avoid congestion
o Can reduce existing congestion
o Can offer flexible bandwidth limit services based on requirements
o Rate limit in effect for port whether congestion occurs or not
- Ingress Rate Limiting is designed to help alleviate choke points in the fabric caused by slow drain
devices, congested ISLs, etc.
• QoS SID/DID Traffic Prioritization
- Allows prioritization of the traffic flow between a given host and target as having low, medium or
high priority
- By default, all flows are assigned medium priority
- Uses zoning to implement low and high priorities
- Hosts and targets must be connected to ports on Condor2 or GoldenEye2 ASICs
- Adaptive Networking license required
• Top Talkers
- Determines the largest users of F_Port bandwidth by monitoring all flows (SID-DID pairs) through
one or more switch F_Ports on any switch in the fabric
- Top Talker (TT) is an enhancement to APM end-to-end monitors
- APM license required
- When enabled, these monitors determine which SID-DID pairs are the major users of switch F_Port
bandwidth
- Can be enabled on specific switch E_Ports or F_Ports in the fabric
- Only supported on Condor, Condor2 and GoldenEye2 based switches

Port Fencing
• Fabric OS v6.1 adds the Port Fencing feature, which disables a port operating outside the bounds of
normal operation
- Based on Fabric Watch event monitoring
- Requires a Fabric Watch license to configure and use
• Configurable only through CLI
• Once a port is disabled, user intervention is necessary for the port to be enabled
• The following Fabric Watch classes support Port Fencing:
- Port class
- E_Port class
- F/FL Port class
• The following areas under each class support Port Fencing:
- Link loss
- Sync loss
- Protocol error
- Invalid words
- Invalid CRCs
• Only the Fabric Watch “above” event will be monitored for Port Fencing
Figure 18: Port Fencing Concept
Cascaded FICON Requirements and Attributes

• The process starts with the FICON Configuration Utility
• Turns on FICON Management Server (FMS) mode on all switches. If some switches already have FMS
mode enabled, it is re-enabled.
• An Insistent Domain ID is required
• The Switch Connection Control policy is created in strict mode
• Port-based routing is used
• IOD is set; DLS is turned off

5 - Distance Solutions
FCIP
• Applications for FCIP include:
- Backup, consolidation, mirroring, business continuity solutions
- Longer-than-FC-distance connectivity
- Existing IP-based networks connect two sites, but not dark fiber
- FICON connectivity over long distance
• Fibre Channel continues to be the dominant choice for Storage Area Networks (SANs)
• Fibre Channel provides a reliable, high-speed, and low-latency transport for SCSI Initiators and Targets
• Brocade does not recommend FCIP for use in every distance extension scenario
• The Fibre Channel-over-IP (FCIP) protocol connects Fibre Channel switches over a GbE-based network
- IP packets generated by an FCIP-compliant port are IPv4 compliant, so that they can navigate any
IP network to reach the destination end point
- Implementation uses standards-based TCP, which interoperates with regular network equipment
Figure 19: FCIP Implementation Example
• FCIP is a tunneling protocol that allows a transparent interconnection of geographically distributed SAN
islands through an IP-based network
- Allows remote disk access, tape backup, and live mirroring
- From the fabric view, an FCIP link is an ISL, transporting all needed FC control and data frames
between switches - the IP network is invisible to the fabric
- The FC fabric and protocols are invisible to the TCP/IP network
• An FCIP tunnel is represented in a Brocade fabric as a Virtual E_Port (VE_Port)
- No different from E_Ports, except underlying transport is IP
• Since the VE_Port emulates an E_Port on either end of the FCIP tunnel:
- Standard E_Port link initialization occurs
- The FCIP platforms at both ends of the link merge to form a single fabric
• VE_Ports do not support ISL Trunking, but support exchange-based routing (Dynamic Path Selection)
• Some FCIP platforms support FC routing over an FCIP tunnel, creating a Virtual EX_Port (VEX_Port)
- Allows long-distance FCIP connections with fabric-to-fabric isolation
- VEX_Ports are no different from EX_Ports, except underlying transport is IP

Figure 20: FCIP Tunnels on a GbE Port
• On FCIP platforms that support FC routing, there are these additional rules:
- A GbE port can have VEX_Ports only or VE_Ports only, but not a mix of both
- A VEX_Port connects only to a VE_Port – it may not connect to another VEX_Port
• There can be multiple VEX-to-VE port connections to each Edge fabric, but do not have EX-to-E and VEX-to-
VE connections to the same Edge fabric
Figure 21: Brocade 7500 & FR4-18i
• The Brocade 7500 SAN Router and the Brocade FR4-18i Director Blade both support FCIP
- 2 GbE ports with up to 8 tunnels/VE_Port or VEX_Port port
- Must be all VE_Ports or all VEX_Ports per GbE port
- SACK, hardware compression, traffic shaping, and jumbo frames can be enabled on a per-tunnel
basis
• Supported platform-to-platform connections are:
- Brocade 7500(E)/FR4-18i VE_Port <-> Brocade 7500(E)/FR4-18i VE_Port
- Brocade 7500(E)/FR4-18i VEX_Port <-> Brocade 7500(E)/FR4-18i VE_Port

FCIP Performance Considerations

• In FCIP-based connections, there are several performance variables that need to be considered:
- Effective Bandwidth
- Delay
- Packet Loss
• Use the ipperf utility to identify these values for your links
- Uses a different TCP port than the one used for user data
- Can be affected by tunnel committed bandwidth
• Brocade’s FCIP implementation provides configurable features that increase effective bandwidth
utilization:
- Selective Acknowledgement (SACK) improves loss detection, retransmission techniques, enables
faster recovery
- Hardware Compression
- Jumbo Frames allow full-size FC frames to be encapsulated in Ethernet frames without any frag-
mentation
o Requires end-to-end jumbo frame support across the network
- Committed bandwidth per tunnel allocates a portion of the GbE bandwidth on a per tunnel basis
o Traffic is routed across the tunnel with the highest bandwidth
Selective Acknowledgement (SACK) improves loss detection, retransmission techniques, and enables faster
recovery. SACK is an extension to a protocol which allows the acknowledge reception of specific packets or
messages. The SACK option allows the receiver to acknowledge multiple lost packets in a single ACK, enabling
faster recovery.
Hardware compression maximizes throughput over WAN links without performance degradation. The
compression feature is a per-tunnel feature (disabled by default) that allows the FC data frames (but not FC
command frames) to be compressed before they are sent over the tunnel as FCIP frames. The compression is
done by hardware on a frame-by-frame basis. Compression is performed on FC data frames in the ingress
direction before they are encapsulated as FCIP frames. The latency introduced by the compression depends
on the size of frame being compressed; for a 2184 byte FC frame, the latency introduced due to compression
is about 14 microseconds.
The default Maximum Transfer Unit (MTU) size of an Ethernet packet is typically 1518 bytes. This is smaller
than the FC frame maximum of 2148 bytes, so a FC frame would be fragmented into two Ethernet frames. The
receiving end would need to reassemble the frames. This all requires processing overhead.
Different applications use protocols with different block sizes to transfer data. Block access protocols access
“blocks” of data in portions that are a multiple of the OS system block. Consider using the following guidelines
to determine block sizes: Transaction data (4-8k block size); Office automation (16-32k); Data warehousing
(64 - 256k); CAD/Design (64-128k); Multimedia (512k - 4M). Small block sizes of contiguous data mean more
I/O especially if the data is spread across the disk; large block sizes that don't use all the space read the
whole block just to get a small piece of data. Applications can be configured to allow for multiple outstanding
I/Os to occur before requiring an acknowledgement. The # of outstanding I/Os is typically 1 to 16.

The IPPerf Utility

• Requirements for the IPPerf WAN analysis tools
- Brocade 7500/FR4-18i at both ends of the FCIP link
- Fabric OS v5.2 and an FCIP license installed on both switches
• Only a single IPPerf session can be active on a FCIP port at any time (sending or receiving test data)
• An IPPerf session can operate between a pair of ports while an FCIP tunnel is online
- Reason: IPPerf uses a different TCP port than the FCIP tunnels
- Result: Can test the FCIP tunnel without stopping the flow of user data
- Caveat: IPPerf session data would then compete for bandwidth with user data – particularly in a
committed-rate tunnel
• An IPPerf session can run simultaneously with other user traffic through an FCIP tunnel
• Validate the available parameters of a long-distance FCIP connection without bringing the tunnel down
- IPPerf sessions use TCP port 3227; the tunnels use 3225 and 3226
• Adding a committed-rate IPPerf session reduces the total uncommitted bandwidth shared by all the
uncommitted bandwidth FCIP tunnels
• Adding an uncommitted bandwidth IPPerf session adds another flow competing for the shared
uncommitted bandwidth
• To capture end-to-end IP performance data over a GbE port, use a Fabric OS command on both ends
- portcmd -–ipperf [slot]/port
- Always specify local GbE port, the source IP (-s) and destination IP (-d), and whether this port is
the sender (-S) or receiver (-R)
- Start the IPPerf receiver first, then start the IPPerf sender
- If no time interval is specified, type Ctrl-C on the sender to stop
Figure 22: IPPerf Example

Supported FCIP Platform Connectivity

• The Brocade 7500(E)/FR4-18i support:
- VE_Port <-> VE_Port
- VEX_Port <-> VE_Port
• Why select VE or VEX configurations?
- VE -> VE will merge both ends of the link into a single fabric
- VEX -> VE will isolate fabrics across the link
Figure 23: Single Fabric Using VE_Port-to-VE_Port Connection
Figure 24: Two Routed Fabrics Using VEX_Port-to-VE_Port Connection
Supported FCIP Tunnel Connectivity

• Some Brocade FCIP platforms allow for up to 8 tunnels per GbE port (7500E is limited to 1)
- A GbE port can have VEX_Port tunnels only or VE_Port tunnel only, but not a mix of both types
- Cannot “isolate” and “merge” in the same tunnel group
- A VEX_Port connects only to a VE_Port – it may not connect to another VEX_Port
• There can be multiple VEX ->VE port connections to each Edge fabric from the same of different backbone
fabrics
- Do not configure EX->E and VEX->VE connections to the same Edge fabric

FCIP FastWrite and Tape Pipelining
FCIP FastWrite
• The FastWrite feature improves synchronous SCSI writes over FCIP
• When the initiator sends a SCSI write, the local gateway (Brocade 7500/FR4-18i) sends an XFR_RDY
immediately
- The local gateway buffers the data
- The XFR_RDY sent by the target is discarded by the remote gateway
• The final status (FCP_RSP) is always sent by the SCSI target, so if the link fails, the Write command also
fails
• The result is that only one round trip is needed, reducing overall latency
• The FCIP FastWrite feature improves the efficiency of communications by leveraging the two gateways
(Brocade 7500/FR4-18is) on either side of the link to reduce the time needed for handshaking:
- The local FCIP gateway receives all SCSI commands and data, and sends XFR_RDY messages
immediately, reducing the time needed for the first round trip (and for any XFR_RDYs sent by long
I/O transfers).
- The remote FCIP gateway receives all XFR_RDY messages from the physical target and discards
them, as the XFR_RDY has already been sent to the SCSI initiator by the local gateway.
- The remote target always generates the FCP_RSP status message, not the local gateway. This is
appropriate, as we do not want to receive a good status until the data has been safely written to
storage.
- The result: each SCSI write requires only one round trip. As the size of the write operation
increases, the efficiency of this approach increases.
• The GbE port on the local gateway buffers up to 512 Mbytes of data, sending it over the FCIP link as the
WAN bandwidth allows. This allows the initiator to keep more exchanges open, and allows the GbE port to
keep the pipe more full, improving efficiency.
Figure 25: FCIP FastWrite

Tape Pipelining
• The Tape Pipelining feature improves sequential SCSI writes over FCIP to tape devices
• The local gateway now sends two messages to the local SCSI initiator:
- Auto-responds to each SCSI write request with an XFR_RDY (as in FastWrite)
- Also auto-responds to the last data frame in each SCSI write request with a FCP_RSP message
(new)
• Results:
- FastWrite-based reductions in RTTs
- Numerous sequential I/Os are queued at the same port, resulting in a full, steady data flow through
the FCIP tunnel
• The tape pipelining process (shown above) builds on the fastwrite feature to optimize sequential I/Os to a
remote device:
- First, the FastWrite feature causes the local gateway to send a XFR_RDY message (XFR_RDY_A,
etc.) after each SCSI write command is received. This results in only one round-trip per write, a
major performance improvement.
- After the data is sent by the initiator to the local gateway (FCP_DATA_A), the local FCIP port immedi-
ately responds with an FCP_RSP message (FCP_RSP_A, etc.). The initiator will interpret this as the
completion of this write, and the next write begins.
- Because FastWrite is enabled, the local FCIP port is buffering data, allowing it to keep the pipe full
and maintain a steady flow of data to the remote device. This should help reduce shoe-shining at
the remote tape device, improving the life expectancy of the tape media.
Figure 26: Tape Pipelining

• In the prior graphic, three sequential writes (A, B, C) are shown. Note the following:
- To the SCSI initiator, the response time of the remote tape device seems almost instantaneous.
- To the remote tape device, the initiator seems to be sending a very steady stream of data, and
responds almost instantly to all XFR_RDYs.
- The longest delay in the entire process is at the end, when the WRITE_MARK message (end of I/Os)
message is sent, as only the remote tape device responds to this message.
• Since I/Os are completed/acknowledged at the local initiator, any error returned from the remote tape at
the remote FCIP port is handled at the remote FCIP Port appropriately:
- BUSY, QUEUE_FULL and other retry-able errors are retried by the remote gateway.
- For any media errors and hardware errors, the WRITE_MARK message is sent to the remote tape,
which will fail the command and send the appropriate status, which will be forwarded to the host
as-is. The initiator will then handle such hard errors appropriately.
- The Status response is intercepted by the remote gateway to allow for cleanup of any state and
resources held for the relevant SCSI Write.
• Fastwrite and tape pipelining are supported over both VE-to-VE_Port and VE-to-VEX_Port links
• When fastwrite and/or tape pipelining is enabled, you may have only one FCIP tunnel (data path) between
switches or front domains
- The same FCIP tunnel must be used to send and receive data
- If multiple FCIP tunnels are configured, traffic may return over a different path/FCIP tunnel
Figure 27: FastWrite and Pipelining Problem Installation
In the example above, we see two FCIP tunnels (0 and 1) connecting two sites. On both tunnels, fastwrite is
enabled (F = 1), as is tape pipelining (T = 1). In this configuration, data could be sent from initiator H1 to tape
T0 via tunnel 0, with traffic returning via tunnel 1. This is a problem, as the same tunnel must be used to send
and receive data when fastwrite and/or tape pipelining is enabled. For this reason, the configuration shown
above is not supported.
• Use FastWrite and Tape Pipelining to connect multiple sites to a central facility
- Remember: only one FCIP tunnel between the central facility and each remote site

Fibre Channel Long Distance Connectivity

• There are a number of methods in which FC SANs can be extended over long-distance optical networks
- Native FC over dark fiber
- Wave Division Multiplexing
- Transponder-based solutions
- SFP-based solutions
- Time Division Multiplexing
- SONET/SDH
- Extended distance solutions
Native FC Over Dark Fiber

• The term “dark fiber” typically refers to fiber optic cabling that has been deployed, but remains unlit or
unused
• The simplest is to connect FC switches directly to the dark fiber using:
- Long-wavelength SFPs (LWLs)
- Extended long-wavelength SFPs (ELWLs)
- FC Extenders with LWL SFP transceivers
• Native Fibre Channel has a limit of 10 km; longer distances require some augmentation
• Use an Extended Fabric license to allocate sufficient buffers to the long distance E_Ports
- Configure Brocade Extended Fabrics for the appropriate distance
Wave Division Multiplexing (WDM)

• Wave Division Multiplexing (WDM) describes the concept of combining several streams of data onto the
same physical fiber-optic cabling
- Light of different wavelengths does not interfere
• Dense Wavelength Division Multiplexing is optimized for high-speed, high-capacity networks and longer
distances
- Designed for longer distances, and typically can go 100 km or greater
• Coarse Wavelength Division Multiplexing (CWDM) provides the same optical transport and features of
DWDM, but at a lower capacity, which allows for lower cost
- Designed for shorter distances, and typically can go 50-80 km
• There are two basic types of Wavelength Division Multiplexing (WDM) solutions:
- Transponder-based
o Using 850, 1310 nm downstream or 1510 nm upstream, it converts signals using Optical-
to-Electrical-to-Optical (O/E/O) conversion WDM frequencies for transport across a single
fiber
- SFP-based
o These eliminate the need for transponders by requiring switch equipment to utilize special
WDM transceivers (also know as colored optics), reducing overall cost

Time Division Multiplexing (TDM)

• Time Division Multiplexing takes multiple client-side data channels, such as FC, and maps them onto a
single higher-bit-rate channel for transmission on a single wavelength
• TDM is used in conjunction with a WDM solution to provide additional scalability and bandwidth utilization.
• Because TDM sometimes relies on certain FC primitives to maintain synchronization, it may require special
configuration on B-Series switches when Extended Fabrics is enabled
- By default, Extended Fabrics E_Ports use ARB primitives (specific to Virtual Channels) as fill words
between frames
- Most TDM devices require idles as fill words.
- Configuring a B-Series switch to use R_RDY flow control
SONET/SDH
• Synchronous Optical Network (SONET) and Synchronous Digital Hierarchy (SDH) are standards for
transmission of digital information over optical networks and are often the underlying transport protocols
that carry enterprise voice, video, data and storage traffic across metropolitan and area networks
• FC-SONET/SDH is the protocol that provides the means for transporting FC frames over SONET/SDH
networks
• FC frames are commonly mapped onto a SONET or SDH payload using an International
Telecommunications Union (ITU) standard called Generic Framing Procedure (GFP)
• Like TDM, FC-SONET devices typically require enabling R_RDY flow control on FOS based switches
Extended Distance Solutions

• Some FC distance extension equipment (WDM, FC-SONET, FC-IP, and so on) can participate in FC buffer-to-
buffer flow control to increase the distance to be greater than what is possible with an Extended Fabrics
license
• Such devices typically participate in E_Port link initialization with an FC switch or snoop the receive-buffer
field in the Exchange Link Parameters (ELP) payload
• These devices are not aware of Brocade VC_RDY flow control, R_RDY flow control must be enabled on a B-
Series switch
- These devices return R_RDY credit to the switch in order to maintain performance over hundreds or
even thousands of kilometers
- Flow control and error correction between distance extension nodes are performed independent of
the switch and are usually dependent on the long haul network protocols
• Additional considerations must be taken to configure these types of networks
- Effective Bandwidth, Delay, Packet Loss (FC-IP), MTU sizes, etc.
• With an Extended Fabrics license, longer distances can be achieved based on speed, switch model and
QoS setting. Typical numbers are:
- 500 km @ 1 Gbit/sec

6 - Performance Tuning and Optimization
Quantifying Performance
• Performance in fabrics and SANs is discussed and quantified through several methods:
- Proximity of servers and storage in a fabric
- Identifying potential network contention issues
- Traffic flow between switches in a fabric
- Performance measurement
Proximity
• The proximity of servers and storage in a fabric is either local or remote
• Local (No hops) – A server is connected to the same switch as the storage that it accesses
- Can be achieved with careful planning
- Not realistic to maintain over time
- Focus on servers that need highest-levels of performance
- Improves availability in a non-redundant SAN
Figure 28: Local Proximity

When a server and the storage that it accesses are local to the same switch (sometimes spoken as “they have
achieved locality”), they communicate directly, without using an ISL. In the preceding example, servers SERV1,
SERV2, and SERV3 are local if they access STOR1, but not if they access STOR2. Locality provides the fastest
access to the storage from the servers.
When creating the initial SAN design, it may be straightforward to keep servers and storage local by mapping
each server to a small group of storage systems. Over time, locality is increasingly difficult to maintain, as
storage usage patterns change with the addition of new servers and storage systems. As a result, switches
may not enough have enough ports to accommodate the demands of locality. We often limit locality only to
those servers that demand the best possible performance.
• Remote (One or more hops) – A server is connected to a different switch than the storage that it accesses
- Simplifies current and future planning
- Need techniques to monitor ISL contention over time
Figure 29: Remote Proximity
When a server and the storage that it accesses are connected to different switches (sometimes spoken as
“they are remote”), they communicate by sending data between the different switches through ISLs in the
fabric. In the above example, servers SERV1, SERV2, and SERV3 are remote to storage system STOR2. If we
limit the use of locality when creating the initial SAN design, most servers and storage will be remote. This
simplifies long-term maintenance as well, as new servers or storage systems can be attached to the nearest
switch without considering locality. To ensure that the ISLs are not over-burdened, we need techniques for
monitoring network contention.

ISL Trunking
• Two terms define traffic flow between directly-connected switches in a fabric:
• Load sharing – If there are two or more equal-cost paths between Brocade switches, each server is
assigned one of the equal-cost paths
- Done at the connection level, not the frame level
- Can result in overloaded or under-loaded paths in a fabric
- Available on all Brocade switches
• Trunking - If there are two or more equal-cost paths between switches, each frame is routed to the first
available equal-cost path
- Requires 4 or 8 Gbit/sec Brocade switches, special licenses, specific port selections
- 4 Gbit/sec switches: Up to eight ISLs per trunk – up to 32 Gbit/sec per trunk
- 8 Gbit/sec switches: Up to eight ISLs per trunk - up to 64 Gbit/sec per trunk
Figure 30: Benefits of Trunking

Trunking ensures that no equal-cost paths are ever overloaded by selecting the first available path when the
frame arrives. This is a much more flexible approach than load sharing that results in the complete use of the
full bandwidth of all paths in the route. There are other requirements for trunking, including judicious port
selection and selected switch products. The preceding diagram illustrates how trunking transforms an
unbalanced load sharing environment into a balanced solution.
• ISL trunks are created when the ports are all part of the same port hardware and the same port group
- Trunking license and port state change required
- On Goldeneye ASIC-based platforms (Brocade 200E), port groups have 4 ports
- On Condor/ Condor2/Goldeneye2-based platforms, port groups have 8 ports
• Trunks may be monitored by Fabric Watch
M-EOS Open Trunking

• The Brocade M-Series Open Trunking feature automatically balances performance throughout a storage
network, while minimizing storage administrator involvement
• Brocade Open Trunking could be characterized as “load balancing” in that it detects conditions where the
fabric is experiencing congestion on a single link and checks to see if there are other uncongested links
available
• If it can relieve the congestion, it permanently shifts some of the traffic, essentially "balancing" the loads
across all available links over time
• The term "dynamic load-balancing" is often used to reflect the fact that it continuously monitors for
cascaded link congestion and can automatically rebalance the flows across these fabric links at any time,
adjusting as traffic patterns change and continuously balancing loads as conditions dictate
• Open Trunking is much more flexible, because it sets no limit to the number of ports that can be used for
an “open trunk” group
• No benefit is derived from frame interleaving over links that are not suffering from congestion
• unless you experience link congestion, you do not want to "balance" an ISL
• Open Trunking is transparent to all applications and requires no user interaction
• FSPF can and should do your initial link routing automatically, and then Open Trunking automatically solves
link congestion problems that occur, when they occur and without your involvement

Figure 31: Fabric Without Open Trunking
Figure 32: Fabric With Open Trunking

ISL Oversubscription
• Oversubscription – When more nodes could potentially contend for a resource than the resource can
simultaneously support
- Example: 4 servers share 1 ISL to reach their storage
• Congestion – In real-time, more actual traffic than available bandwidth; oversubscription that is realized
• The Brocade method for quantifying the potential for network contention is the ISL oversubscription ratio
- For each switch, it is the ratio of the bandwidth needed by HBAs to available bandwidth on the ISLs
- With all 4,8 Gbit/sec devices and switches: # servers/# ISLs
• Other methods for quantifying network contention:
- Fan-out – # of HBAs per storage port (“storage port oversubscription”)
- Fan-in – # of storage ports per HBA (“HBA oversubscription”)
Figure 33: Oversubscription Ratio

Performance Measures
• There are several measures of SAN performance, and each is appropriate for different SAN applications:
- Bandwidth – Maximum data rate through a device
o Best-suited for measuring large-I/O application performance (portperfshow)
- SAN Latency – Time it takes to move data through a SAN
o Includes switch latencies (<= 2-6 µs) and cable latencies (5 ns/meter)
o Does not include latencies in servers or storage systems
o Best suited for measuring long-distance SAN solution performance
- Response Time – Total time to fetch data to a user
o Equals SAN latency plus latencies in server hardware, storage system, operating system
and application
o Best suited for measuring small-I/O application performance
Leveraging 8 Gbit/sec Technology

• There are several techniques that can be used to leverage 8 Gbit/sec technology to maximize
performance:
- Deploying 8 Gbit/sec Directors at the core of a new SAN
- Attaching 8 Gbit/sec Directors to an existing 4 Gbit/sec fabric
- Adding 8 Gbit/sec blades to an existing 48000 at the core of an existing SAN
- Upgrading servers to 8 Gbit/sec HBAs
• 8 Gbit/sec provides advantages whenever performance is key, ISL budget is scarce, or physical cables are
limited
• Advantages to upgrading the core to 8 Gbit/sec:
- Upgrading the core makes it possible for ISLs to run faster
- Better suited for high-performance devices
• 8 Gbit/sec should be used for long-distance connections if possible
- Increases utilization of expensive dark fiber or xWDM resources

Choosing the Right Routing Policy
Fabric Routing
• Data moves through a fabric from switch to switch and from storage to server along one or more paths that
make up a route
• FSPF calculates minimum cost paths from switch to switch
• Routing policies determine the route for each data flow. In fabrics with multiple equal-cost paths between 4
or 8 Gbit/sec switches and Directors, Dynamic Path Selection (DPS) further enhances performance
Exchange-based
• Based on the Source ID (SID), Destination ID (DID), and Fibre Channel originator exchange ID (OXID)
• Optimizes path utilization for the best performance
• Every exchange can take a different path through the fabric
• Exchange-based routing gives you additional options:
- I/O traffic is balanced across the available lowest-cost paths on an exchange-by-exchange basis
- Designed to work well with ISL trunking
- Particularly effective in core/edge SAN architectures
Port-based
• Input port from the source is assigned to an output port toward the destination switch (a route)
• Routes are allocated via round-robin assignment
• Chosen routes are used until one of the devices in the fabric goes offline or the fabric changes
Design Considerations
• 4 and 8 Gbit/sec switches use exchange-based routing (Brocade default) but can be changed to port-
based routing
• 1 and 2 Gbit/sec switches only support port-based routing
• A fabric can have switches with different routing policies
• For most configurations, the default routing policy is optimal and provides the best performance
• Change the routing policy only if there is a performance issue that is of concern, or if a particular fabric
configuration requires it
Preferred Pathing
Preferred pathing allows the administrator to configure which ISL a source port will use when communicating
to a specific domain The Preferred Path optional feature enables you to influence the route of data traffic
when traversing multiple switches or Directors in a fabric. If more than one ISL connects switches in your SAN,
this feature will be useful when you want to indicate an ISL preference for a particular flow. The data path
consists of the source port of the switch or Director being configured, the exit port of that switch or Director,
and the domain ID of the destination switch or director. Each switch or Director must be configured for its part
of the desired path for optimal performance.

7 - Migration and Integration
Migrations
Migration Assessment
• Requires a detailed understanding of:
- Connection intent: Which hosts need to talk with which storage?
- Application dependency: Which applications must be moved together?
- Maintenance windows: Most physical work must be scheduled and performed during maintenance
windows or planned outages
- Complexity:
o There are many IT groups that must be involved (storage, SAN, OS administration, applica-
tions
o Compatibility matrix is large (host OS and maintenance level, multipathing driver, HBA
model, HBA driver, HBA firmware, FC switch OS, storage firmware)
- Risk: Must be well planned and executed
Migration Plan Development

• Using information gathered in the Assessment phase, build a migration plan based on these items:
- Current and future requirements for:
o Port count
o High Availability
o Power, cooling, space constraints
- Current and future technologies
o Higher bandwidth and throughput infrastructure
o Server and storage virtualization
o Power and cooling efficiencies of the newer devices
- Data Center availability requirements
o Is there any acceptable downtime?
• Compatibility Issues and Considerations
- Mi10K & M6140
o Extended fabrics and hot code load work with Fabric OS on the DCX
o Open Trunking to the DCX will function
o Domain ID offsets
o M-EOS levels
o McDATA Fabric vs. Open Mode

Figure 34: Migration Process Phases
Migration Strategies
• There are several main migration strategies that most projects follow:
- Simple replacement of existing switches
- Integration or consolidation of fabrics
o Using this method, the existing fabrics are supplemented with new switches and Directors
o Host and storage connections are migrated to the new switching elements within the same
fabrics over time
o This may require the old switches/Directors to first be upgraded (firmware) and/or modified
(e.g. interoperability mode and other switch settings) prior to the new switches/Directors
being added
- Dual/parallel fabrics (rip and replace)
o Using this method, new fabrics are built using new switches and Directors
- Dual/parallel fabrics with FC Routing
o If the parallel fabric method is used, the old and new fabrics may be interconnected via
Fibre Channel routing technology
o Leverages the advantages of the parallel fabric method (separate fabrics) with those of the
integrated fabric method

M-Series to B-Series Migration
Using a Router
• Just as a normal migration, to perform a non-disruptive migration, there must be redundant fabrics, so that
one fabric can be migrated while the other stays intact
• The Fabric OS SAN should also have redundant fabrics
• When the migration is complete the Fabric OS SAN will have an identical topology to the M-EOS SAN
Figure 35: M-Series Migration to B-Series Using Routers
M-Series to B-Series Migration Steps

1. Create two new Fabric OS fabrics
2. Identify the ports on each of the two routers and configure them
3. Attach the two routers to both M-EOS and Fabric OS fabrics
4. Verify the connectivity
5. Create LSANs in the migrating M-EOS and Fabric OS fabrics and verify the LSAN implementation
6. Physically move cables from M-EOS to Fabric OS fabrics
7. Verify the connectivity
8. Repeat steps 2 through 7 for the second pair of fabrics
9. Remove the routers
Design Element Considerations

• Interoperability
- Modes and supported configurations
- M-EOS/Fabric OS code revision supported levels
• Bandwidth

Integrated/Consolidated Fabrics
• New switches are integrated into the existing fabrics and fabrics merge
• Hosts and storage can be moved independently
• Fabric OS + Fabric OS
• Fabric OS + M-EOS (using interoperability modes)
Figure 36: Merging Fabric OS and M-EOS Fabrics
You can incorporate new or existing fabric components into an existing SAN by merging new switches into the
existing SAN. With a redundant fabric configuration, this would typically be done one fabric at a time, with
primary data path first followed by the secondary data path.
This option would best leverage existing assets in the new architecture. However, this option would also be the
most disruptive to the existing storage environment. While many of the key tasks could be performed in a
phased approach, much of the front-end work would impact entire fabrics at once. When merging SAN fabric
components, storage administrators must consider the intricacies of characteristics such as uniqueness of
Domain IDs, PID compatibility, and FOS compatibility. In many cases, resolving these issues would require that
host and fabric components be modified prior to the introduction of a new switch into the fabric. These
modifications would affect the entire SAN fabric and thus impact all hosts with a data path through the fabric.
As a result, this method would require considerable planning and carry greater risk.
There is much planning and preparation that must occur at the front end to facilitate this type of migration. In
fact, the effort required for a successful merge grows exponentially with the number of SANs involved.
Merging many small SANs from multiple vendors into a core-to-edge architecture would be less feasible than if
it were only one or two SANs.

Integrating 8 Gbit/sec Into 4 Gbit/sec Environments
Why Do It?
• Next-generation ASIC family, the Condor2 and GoldenEye2 ASICs, introduces new features and also
extends current features found in 4 Gbit/sec Fabric OS switches, which include more:
- Buffer credits
- Virtual channels
- Ports per ASIC
• Higher-performance ISLs facilitate fabric architectures that minimize bottlenecks and enable optimum use
of fabric resources
• Selecting an architecture that accommodates both moderate and high performance requirements
provides the ability to implement a variety of applications without having to micro-manage the
infrastructure
• For some applications, 2 Gbit/sec or 4 Gbit/sec may be adequate
Inserting a DCX at the Core When 48000s are Deployed

• Place a DCX at the core of the fabric and propagate the 48000 Director toward the edge
• Provides a higher speed and a denser fabric core, with a denser concentration of 4 Gbit/sec at the edge
• Increases chassis locality at the edge
Figure 37: Inserting DCXs at the Core

8 - Security
Access Control
Password Rules
• Password rules are enforced only when defining new passwords
• Set password strength policy by specifying the minimum number of:
- Lowercase letters
- Uppercase letters
- Digits (0-9)
- Punctuation characters
- Minimum length
• Limit password re-use by setting the password history policy
- Passwords kept in history
• Avoid stale passwords by setting a password expiration policy
- Minimum age
- Maximum age
- Expiration warning (days)
• Set the account lockout policy
- Password failures allowed
- Set lockout duration (minutes)
Role Based Access Control (RBAC)

• RBAC is a role-based approach to restricting system access to authorized users
- RBAC introduced the distinction between a user account and the role assigned to the account
- Each account has an administratively defined role
- This distinction allows the switch to track login activity
• There are three main parts to RBAC based accounts:
- Account name - Login name and password
- Role - Grants or denies access to switch commands
- Description - Optional parameter to add additional detail about the account
• Fabric OS implements two classes of accounts:
- Default accounts:
o root
o factory
o admin
o user

• User-defined accounts:
- Unique role with a set of pre-defined permissions
- Expand ability to track account access
- Audit administrative activities
Authentication, Authorization, Accounting

• RADIUS centrally controls user logins
• LDAP maps Active Directory server roles to Fabric OS RBAC roles
• SSH Public Key Authentication provides a mechanism to authenticate user logins without a password
• SCP is a protocol that provides full data encryption for switch upload and download tasks
Figure 38: RADIUS Configuration

RADIUS: The Remote Authentication Dial-In User Service (or RADIUS) is a protocol for carrying authentication,
authorization, and accounting (aaa) information about remote user access between a Network Access Server
(which desires to authenticate its links) and a shared Authentication Server. RADIUS is an open standard
(IETF RFC 2865 and RFC 2866).
Client/server: The RADIUS client must pass user information to designated RADIUS servers, and act on the
returned response. The RADIUS server receives user connection requests, authenticates the users, and then
returns all configuration information needed for the RADIUS client to deliver service. In this case, a Brocade
switch is configured as a Network Access Server that acts as a RADIUS client.
Network Security: To ensure that user names and passwords remain private, all client/server communication
is encrypted, and authenticated with a shared secret key. RADIUS is focused on authenticating, authorizing,
and accounting remote user access – in particular, logins and logouts. RADIUS does not perform these roles
for devices or switches entering a fabric – these roles continue to be handled by existing Fibre Channel
protocols.
In a fabric with switches running a mix of Fabric OS versions, the way a switch authenticates users depends
on whether a RADIUS server is set up for that switch. For a switch with RADIUS support and configuration
enabled, authentication bypasses the local password database. On a RADIUS-enabled switch, logins through
the serial port are not authenticated with the RADIUS server, but through the local switch database. For a
switch with RADIUS support or configuration disabled, authentication uses switch local account names and
passwords.
Access Control Lists (ACLs)

• Access Control List policies restrict switch, device, and administrative access within a fabric
- ACLs can be deployed on a switch-by-switch basis
- Each switch can be managed independently or through a trusted switch (FCS)
• ACLs are part of the base Fabric OS
- No license required
- Pre-Fabric OS v5.2 switches do not recognize ACLs, and do not participate in security policy
enforcement
The Access Control List (ACL) feature provides the strong device access features created in Fabric OS with a
minimum of management overhead. ACLs can be deployed flexibly, so that device access security can be
implemented on only those switches that need it. Because pre-Fabric OS v5.2 switches can co-exist in the
same fabric as ACL-secured switches, ACLs can be used in any mix of Brocade switches.
The switch password database can be distributed manually throughout a fabric. This could create uniform
login credentials in environments not using RADIUS for user authentication.

Access Control Lists – Device and Switch Access

• There are three ACL policies that restrict switch and device access to a fabric:
- Switch Connection Control (SCC) policy restricts which switches may join a fabric
- Device Connection Control (DCC) policy restricts which FC devices can connect to which FC switch
ports
- Fabric Element Authentication (AUTH) policy requires shared secrets and digital certificates to
authenticate devices or switches
Figure 39: ACL Policies
Access Control Lists – Administrative Access

• There are three ACL policies that restrict administrative access to a fabric:
- The Fabric Configuration Server (FCS) policy restricts which switches may change the configuration
of the fabric (zoning, policy distribution)
- The IP Filter (IPFILTER) policy restricts incoming administrative traffic on the IP management inter-
faces
- The Password (PWD) policy distributes the user account and password database to other switches
in the fabric
Securing DCF Management Traffic

• There are a few ways to secure traffic in the Data Center Fabric
- SCP
- SSH
- SNMPv3
- SSL
- IPSec

Taking the Test

After the Introduction Screen, once you click on Next, you will see the non-disclosure agreement:
Figure 40: Sample NDA

Once you agree to the non-disclosure terms, the timed exam will begin. This is a sample of how the questions
will look. In this example, you see a multiple-choice question.
Figure 41: Sample Question

This is a sample of the score sheet you will see at the end of the exam. You also see the breakdown of how
many questions there are in each section of the exam. A hard copy of this will be printed at the testing center.
It is vital that you obtain and save this hard copy as proof and validation.
Figure 42: Example Exam Score

BCFD in A Nutshell Study Guide For Exam 143-260 8gbit

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

BCFD in A Nutshell Study Guide For Exam 143-260 8gbit

Uploaded by

Copyright:

Available Formats

BCFD in a Nutshell

Study Guide for

Revision November 2008

European Headquarters - Geneva, Switzerland

Asia Pacific Headquarters - Singapore

© 2008 Brocade Communications Systems, Inc. All Rights Reserved.

Revision: November, 2008

BCFD in a Nutshell 8 Gbit/sec Edition

Helen Lautenschlager Joe Cannata

©2008 Brocade Communications 1

2 ©2008 Brocade Communications

©2008 Brocade Communications 3

Fibre Channel Long Distance Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

4 ©2008 Brocade Communications

©2008 Brocade Communications 5

©2008 Brocade Communications 6

How to Assess Current and Target SAN Infrastructure Requirements

Tools to Assess the Current SAN Infrastructure

©2008 Brocade Communications 7

Error Checking and Analysis

Figure 1: Sample SAN Health Report

Current SAN Requirements

8 ©2008 Brocade Communications

o HBA manufacturer, model and driver revision

Target SAN Requirements

©2008 Brocade Communications 9

SAN Design Installation Plan Documentation Components

Gathering Business Requirements

Key Items to Collect

10 ©2008 Brocade Communications

- Disaster Tolerance and Business Continuance

©2008 Brocade Communications 11

Designing a Solution To Meet Customer Requirements

Hardware Products - Switches, Directors and Backbones

12 ©2008 Brocade Communications

Figure 2: B-Series Switches

Table 1: Switch Attribute Summary

Brocade Brocade Brocade Brocade Brocade Brocade

©2008 Brocade Communications 13

Figure 3: Ports on Demand Increments

Figure 4: Buffer Credits

14 ©2008 Brocade Communications

Figure 5: Backbone and Directors

Hardware Products - Fabric Extension

©2008 Brocade Communications 15

Figure 6: Single Switch Topology

16 ©2008 Brocade Communications

Cascade and Ring

©2008 Brocade Communications 17

18 ©2008 Brocade Communications

Figure 9: Core/Edge Topology

©2008 Brocade Communications 19

Figure 10: Non-resilient vs. Resilient

20 ©2008 Brocade Communications

Design Assessment Documentation

Miscellaneous information to be gathered:

©2008 Brocade Communications 21

3 - Management and Monitoring Tools

Tools for Gathering Monitoring Information

Applying Management Tools

Management of switch accounts can be achieved using:

Information could be centrally stored for many reasons:

Zoning information could be gathered using: