Module 3

BROCADE
Silk Worm
 Generations of Brocade

Gen-5
8Gbps
Gen-4
4Gbps
Gen-3
2Gbps
Gen-2
1Gbps
Gen-1 Bandwidth increased generation
1Gbps by generation(1Gbps – 8 Gbps)

Outdated
in 1998
 Hardware components of Switch

In normal systems the components are: :

RAM - CPU
- RAM
- Hard-Disk
In embedded systems :
- CPU
ASIC - RAM
FP-ROM - FP-ROM (In place of HDD)

In Brocade switches
- HDD is replaced by FP-ROM.
- The processor is called ASIC
- RAM is SD-RAM
 ASIC (Application Specific Integrated Circuit)

• ASIC (Application Specific Integrated Circuit): is a processor, specially

designed for Brocade switches and has the entire functional logics of switch
embedded in it, and more powerful than generic switch.
• There are five generations in ASIC:
ASSIC

Generation -1 - Stitch
Generation -2 - Loom
Generation -3 - Bloom, Bloom-2
Generation -4 - Golden Eye
Generation -5 - Contour
 GENERATIONS OF ASIC

Gen-2 : Loom ASIC Gen-4 : Golden-Eye (2005 – 2007)

- Supports 4 ports - Supports 16 ports
- Speed 1Gbps - Speed 4 Gbps
- Frame Processing > 1.5µ secs.
- Auto negotiable up to 1 Gbps
GEN-3:Bloom ASIC - Frame Processing < 1µ sec
- Supports 8 ports
- Speed 2 Gbps Gen-4 : Golden-Eye ( 2007 to -----)
- Auto negotiable - Supports 16 ports
- Frame Processing < 1.5µ secs.
- Speed 8 Gbps
GEN-3:Bloom II ASIC(2001 – 2006)
- Auto negotiable up to 2 Gbps
- Supports 8 ports
- Speed 2 Gbps - Frame Processing < 1µ sec
- Auto negotiable
- Frame Processing < 1.5µ secs.
 Capacities of RAM and FP-ROM
• Depending on the speed of the switch the Capacities of RAM changes,
basically SD-RAM is used: The capacities are as follows:

32mb,64mb,128 mb,256mb ……..

• The capacity of FP-ROM is initially very low as the cost of flash chips are
expensive those days. Now the prices of Flash chips are slashed down and
the flash chips with huge capacities are placed in the switch depending on
the functional requirements.

32mb,64mb,128 mb,256mb …………n

 FP-ROM
• In embedded system FP-ROM plays major role as hard disk does in the
normal computer.

• FP-ROM plays major role in switch, it is the heart of a switch.

• To operate an embedded system an application and Real time Operating

system to organize the application are required.

FOS.x Application
VXWORKS
& RTOS

LINUX
 RTOS (Real Time Operating System)
• RTOS is more powerful than normal Operating System, It consists of kernel, drivers
and modules.
• In Brocade switches up to Bloom ASIC the RTOS is VXWORKS, later from Bloom 2
Linux RTOS is introduced.

VXWORKS:
- It is a powerful RTOS that never crashes
- It is very thin in size
- It has almost all kinds of device drivers , maintained in binary file format

Prior to Bloom II the cost of FPROM was high, later the prices slashed down the size of
chip is not a cost factor anymore.

Linux RTOS:
- It is a powerful RTOS with more advanced features
- It is thick in size
- It has almost all kinds of device drivers in RPM format
 Fabric OS (FOS)
FOS is an application that runs on top of RTOS.

FOS changes according to the FOS contains Licenses to support

generations various functional aspects of switch
Generation -1 FOS 1.x
Generation -2 FOS 2.x Licenses:
Generation -3 Web tools
Bloom FOS 3.x Fabric
Bloom –II FOS 4.x Trunking
Generation -4 FOS 5.x Fabric Watch
Generation -5 FOS 6.x Advanced Performance Monitoring
Extended Fabric
Remote switch
 Licenses of Switch
Licenses are required to support various functional aspects of switch.

LICENSES
WEB TOOLS To manage switch using browser

Default To join switch in a fabric

Fabric
licenses
Zoning To create zones in the fabric

To join multiple ISLs and aggregate

Trunking bandwidth
To set traps using SNMP to monitor
Fabric watch switch

Advanced Performance To monitor the performance of switch.

monitoring

Extended Fabric To extend switch more than 250 meters.

Remote Switch To extend switch beyond 5 kms

 Cont’d
• ISL( Inter switch Link) :ISL is a cable used to join two switches and extend
the scope of fabric.

• Trunking: To aggregate the bandwidth between the two switches, to reduce

the latency, two switches are joined using multiple ISL’s between quads of
switches.

• SNMP (Simple Network Monitoring Protocol): is used in network

management systems to monitor network-attached devices for conditions
that warrant administrative attention. SNMP is a component of the Internet
Protocol Suite as defined by the Internet Engineering Task Force (IETF). It
consists of a set of standards for network management, including an
application layer protocol, a database schema, and a set of data objects.
 Classification of Switches

LOW Middle HIGH

Switch Model: 3200
Ports: 8
FOS 3.x/VXworks
ASIC: Bloom

Switch Model: 3800

Ports: 16
FOS 3.x/VXworks
ASIC: Bloom

Switch Model: 3250 Switch Model: 3900 Switch Model: 12000

Ports: 8 Ports: 32 Ports: 64 *2 = 128
FOS 4.x/Linux FOS 4.x/Linux FOS 4.x/Linux
ASIC: Bloom-II ASIC: Bloom-II ASIC: Bloom-II

Switch Model: 3850

Ports: 15
FOS 4.x/Linux
ASIC: Bloom -II
 Classification of Switches

LOW Middle HIGH

Switch Model: 200E Switch Model: 24000
Ports: 16 Ports: 128 * 2 = 256
FOS 4.x/Linux FOS 4.x/Linux

Switch Model: 4100 Switch Model: 4900 Switch Model: 48000

Ports: 32 Ports: 32 Ports: 256 * 2 = 512
FOS 5.x/Linux FOS 5.x/Linux FOS 5.x/Linux

High end switches are hot pluggable and highly available, they are called core
switches or director switches
 Brocade MPR
MPR (Multi – Protocol – Router): MPR AP 7420
Router supports all protocols like- FOS XPath
- FCP RTOS Netbsd
- IFCP
FOS XPath cannot be upgraded to
- FCIP
- ISCSI FOS 5.X, but Netbsd can be replaced
With Linux

MPR AP 7500 FOS

5.x RTOS
Linux Both FOS
4.x and XPath are consolidated and
FOS 5.x is released, with RTOS
Linux
 LEDS of Switch
LEDs on Front Panel:
- Front panel of the switch has ports arranged in Quads.
- Up on every port, there are two LEDs
- First LED is status LED, and the next LED speaks of speed.
- When a cable is connected to switch the status LED glows
- If a LED glows in green, Flogi is successful
- If a LED glows in amber, Flogi is unsuccessful
- If port is running in maximum speed the speed led glows else,it is stopped as it was
initially.

Port STATUS LED

RJ -45
Port Speed LED DB-9

…..

Port Front View of Switch

 LEDS of Switch
LEDS on Rear panel:
A single LED is seen at the center of the back panel of the switch.
- When the switch is first turned on the Colour of the LED is “Amber”.

Colour of LED
- If the switch boots - Green
Else - Amber
- If the fan of Switch works properly - Green
Else - Amber
- If the power supply is normal - Green
Else - Amber

LED
Power FAN Power
Button

Rear Panel
 DIFFERENT WAYS TO ACCESS SWITCH
Switch can be accessed through various tools and services.

- Telnet
- Secure Telnet
- SSH
- Web Tools
- SNMP
- SMI - S
- Fabric Manager( GUI tool of brocade)
- Brocade API (Freely downloadable)
- Hyper Terminal
 ACCESS LEVELS OF SWITCH

There are four access levels in switch, they are:

1. User
2. Admin
3. Factory
4. Root

1. User : To monitor the switch.

2. Admin : To perform administrative tasks, e.g. firm ware upgrades, config

upload download etc.

3. Factory : To set factory defaults.
4. Root : Set the switch to factory defaults and also all tasks can be
executed in this level.
 INSTALL AND SANITIZE
• Every brocade switch have serial/console port and Ethernet (RJ-45) port ,apart
from the other ports.

• Newly shipped brocade switch is provided with Ethernet cable, power cable
and serial cable.

• Serial cable connects host to the DB-9 port (management/serial port)

• Ethernet cable connects host to the RJ-45 port (management/serial port)

• By default the IP address of brocade switch is 10.77.77.77

• Default user name and password

- User name: admin
- Password: password
• If no default user name, password or IP address are provided need to contact
the vendor.
 CONT’D
• Connect the cable to HyperTerminal or Tipper 9600 ,(default setting)
• Default port configuration
- login to the switch
User name : Admin
Password : Password

- Set the IP address

>ipaddrset (follow the instructions and set IP address)
>ipaddrshow (displays the IP address)
>licenceshow (displays the licenses assigned to the switch)

- check the environmental conditions

>tempshow (displays temperatures at power supply, fans, CPU,
RAM. Temperatures vary from component to
component but ambiant temperature shoud be
93F to 95F
 CONT’D
- Switch level ventilation
>fanshow ( displays RPM’s of each fan)
The difference between the RPM’s of fans should be narrow, if the difference is
more the fan with less RPM’s should be replaced.

- Power supply checking

>psshow
displays the rate of power consumption

- Switch parameters
>timeout 5 (current time out value of telnet)
generally set 5minutes
>quietmode 1 (to stop alert infos)
>switchshow (displays the summary of the switch)
•
CONT’D
Configure switch (initial configuration of switch is disruptive, disable the switch)
>switchdisable
>configure (change the required parameters like domain ID, PID etc)
:d (save and quit)
:c (quit without save)
>switchenable (To enable the switch)
> config show (Displays the parameters of the switch)
> license show (To show the licenses of switch)
>license add (To add the license)
>switch beacon 1 (To turn on the LEDs)
>switch beacon 0(To turn off the LEDs)

Switch port level commands:

>portcfgshow (Switch ports and their summary - global)
>portshow <port number>(To show the information of an individual port)
>portenable <port number>(To enable a required port)
>portdisable <port number>(To disable a required port)
 Cont’d
- To configure ports:
Configure a port as loop port
>portcfgLport <port number> <true 1/false 0>

Configure a port as expansion port

>portcfgEport <port number> <true 1/false 0>

Configure a port as trunk port

>portcfgtrunkport <port number> <true 1/false 0>

Configure a port as fabric/default port

>portcfgdefault <port number>

Configure the speed of a port

>portcfgspeed <port number>,<0> Auto negotiation
<1>(1Gbps)
<2>(2Gbps)
<4>(4Gbps)
 Backup & Restore Switch configuration
• To upgrade the firmware of switch, the configuration of the switch should
be taken back-up.
• Upgrade of firm –ware is to upgrade the version of FOS.
• Once the upgrade is completed restore the configuration if necessary.
• Back-up the configuration of switch is non – disruptive
• Restoring the switch is disruptive
• Taking the back-up of switch is to upload the configuration file to the
FTP server ,as the back-up cannot be performed in the local switch

>configupload Config.txt Config.txt Config.txt

(Not disruptive)
FTP
Server >switchdisable
>configdownload
(disruptive)

Switch
 Firmware upgrade
FOS 3. x / VXworks
• FOS is a single binary file of size
2mb.
• Once the firmware is upgraded
the previous version is over-
written by new version and
cannot be reverted
FOS 4. x / Linux
• FOS is a zip file with a bundle of
rpms of size 40 mb.
• Once firmware is upgraded, it will
not over-write the previous older
one. It divides FP-ROM into two
partitions, primary and
secondary. If the newer version
is not supportive, we can revert
the changes and retain to the
older firmware. If the latest
version is comprehensive ,the
upgrade can be saved, and
reboot is required for the
changes to take effect.
 Cont’d
• Download the upgrades from the website to the FTP server and later down
load to the switch for upgrade.
• The version difference to upgrade/downgrade the firmware should be two.

Firmware
upgrade
FOS 4.0 FOS 4.2 FOS 4.4
Firmware
Down grade
1. Download firmware from website to FTP server
2. Back-up/upload switch configuration to FTP server. http://
http://
3. Download firmware to switch
4. Restore / Download switch configuration to switch 1
from FTP server

Firmware

Firmware

Config.txt
Config.txt

3
2
4
Config.txt
Switch
Firmware
 Security
Zoning : To create the virtual bus in a physical san to limit the scope of
initiator and enhance security is called zoning.

SAN ensures any – to- any connectivity, which has advantages as well as
disadvantages. Any host can access any Lun,that results in threat to the
security.

Security can be implemented at various levels across the fabric:

- HBA - Selective Lun Presentation
- Storage sub system - Lun Security / Lun
masking
- Switch - Zoning
 Selective Lun presentation
• Initially the security is conducted by HBA driver, which enables “selective lun presentation”
• HBA driver detects the number of drives, that a host should be able to Access.e.g lpfc driver.

According to our illustrated figure it is

clearly visible that 2 hosts are connected
to Jbod via hub.
Now both the hosts can access all the
drives in the Jbod, as initiator (HBA)
probes and identifies the discs.
Hosts accessing all the drives may arise
complications like corruptions hangouts
Hub etc., to restrict the hosts from accessing
all the discs and confine their
authorization to access a particular set of
Jbod with
discs, “Selective lun presentation“ was
20 discs introduced.
Selective lun presentation process
1. In this illustrated figure how the
discs are presented to the host is
disclosed.
sd.conf 2. Host identifies number of
presented discs through sd.conf.
3. In this example, 10 discs out of
Lpfc.conf 20 in Jbod are presented to the
host.

HBA 4. The information pertaining to the

presented discs is mapped by
host from sd.conf.
5. But HBA probes identifies and
maps the entire discs in Jbod.
6. It is the responsibility of lpfc to
suppress the access to unwanted
discs by entering them entering
their wwn’s and disabling them in
lpfc.conf.
 Disadvantages of selective lun presentation

• Selective lun presentation process is vast and those days lpfc is not
dynamic and any new additions of discs should be entered manually in
lpfc.conf files in entire host in the network which is practically impossible in
the environments where there are more than two hundreds of hosts.
• The next layer of HBA was introduced with dynamic lpfc drivers which are
used by switches to be the member of zone to provide security to Luns, but
this concept is confined to Jbod level.
- lp8000 is static
-lp9000 is dynamic.
• At switch level discs of Jbod are visible but not the luns of storage sub
system. To provide lun security, the following techniques are developed.
- Lun mapping
- Lun security
- Lun masking
 Zoning
• In a fabric there can be many number of storage sub systems as well as hosts with
heterogeneous operating platforms and applications and conducted via devices like
switches ,hubs etc.
• In this vast infrastructure ,if the access between hosts and storage sub systems is
not aligned and restricted properly, this turns to be major threat to data.
- By any chance two hosts within the fabric with two different operating
environments access same lun,its catastrophic.
- To a fabric up to 16000 devices can be connected, if a host reboots and
initiator starts probing the devices, It takes the management host days together to
boot.
Considering the above adverse factors ,the concept of zoning has evolved, to
restrict the access between hosts and storage subsystems and assign a few
particular hosts to gain access to the storage sub systems

• Zoning : To create the virtual bus in a physical san to limit the scope of initiator and
enhance security and performance is called zoning.
 Zoning
3
2 Z1 –Zone 1
4
Z2 –Zone2

1 Zone1:
5
Allows hosts 1 and 2 to access
storage subsystem A. Remaining
SAN
SAN hosts are forbidden.
Z1
Z2 Zone2:
Allows hosts 4 and 5 to access
storage subsystem B Remaining
hosts are forbidden.

Host 3 belongs to none of the

both zones, it has no access to
A any of the storage sub systems.
B
 TYPES OF ZONING

• In brocade point of view we have two

types of zones
- Hardware enforced
- Software enforced
 HARDWARE ZONING

Static zone
Works with domain ID and port number
Frames are filtered by ASIC
Secured but not flexible
Lack of flexibility is the main disadvantage, for every change at switch level
should be configured which may happen quiet often
- Device zoned to a particular port cannot be a member of zone if it is
connected to another port in a fabric
- If a domain ID of a switch changes, zones will be disabled
- If any switch component is replaced, the zones should be re-configured, but
this is an exception for HBA
- If HBA is replaced at host level need not re-configure
 Hardware Zoning
zone

Host HOS
T
Storage

0 5 0 0 0 0 0 5 0 1 0 0 0 5 0 2 0 0
 Software Zoning

zone

Host HOS
Storage T

5 0 0 1 0 0 0 5 0 1 0 0

1 0 0 1 b 4 0 5 0 1 2 0

wwpn 1 0 0 1 0 f 0 e 0 1 4 3
 SOFTWARE ZONING
In brocade switches software zoning traverses through two phases, one phase
Is up to loom and another phase starts from bloom

Software zoning at loom level Software zoning from bloom

level
-Dynamic zoning
-Dynamic zoning
-Uses wwpn and wwnn of
devices -Uses wwpn and wwnn of
devices
-Session level verification by
name server -Session level verification by
name server
-flexible
-Frame filtration by ASIC
-flexible and secured

Reconfiguration of zone required only when HBA is replaced

 MIXED ZONING

• Zoning that is performed using WWPN’S as well as domain and port id’s is
mixed zoning.

• ASIC level frame filtration is not performed

• Session level verification is done

• It is less secured when compared to hardware zoning as well as software

zoning and used in very small scale environments
 ZONE ARCHITECHTURE
OBJECTS OF ZONE

Members
Members
WWN numbers of Hba's , discs as well as
storage sub systems are given alias names
called members

Zones Zones
Combination of multiple members is called
Zone

Configs Configs
Combination of zones or a zone is
called config. There may be any
number of configs, but only one
config is enabled at a time
 Associating objects of Zone

3 Members
2
4
1 2

1 A
5

SAN
SAN Zone 1
Z1 (Z1)

1 2
A

Config cfg1
Zone 1
(Z1)
A
B 1 2
A
 Associating objects of Zone

3 Members Members
2
4
1 2 4 5
1 A
5 B

SAN
SAN Zone 1 Zone 2
(Z2)
Z1 (Z1)
Z2
1 2 4 5
A B

Config cfg2
Zone 2 (Z2)
Zone 1 (Z1)

A
B 1 2 4 5
A B
 Features of Zone

• Zone allows grouping of SAN devices, which communication each

other.

• Zone or a group of zones are bound to the configuration.

• Zone should include atleast one initiator member and one target
member

• Zone is always fabric centric, but not switch centric.

 Configuration Management
• Configuration plays important role in the fabric

• Configurations are stored in FP ROM

• ASIC looks in to FP ROM and finds the configuration which is loaded in

RAM

• This way switch internal hardware, CPU, RAM and FP ROM manage zones.

• Any time only one configuration is effective across the fabric.

• If another configuration is required ,never disable effective configuration,

always enable required configuration upon the effective configuration.

• Upon enabling the new configuration on effective configuration, the current

configuration will become dormant
 Types of Configuration

• Defined configuration
• Effective configuration
• Saved configuration
• Disabled configuration

Defined Configuration : configurations defined by the user to implement in a

fabric is called defined configuration.

Effective configuration : configuration which is enforced in a fabric

Saved configuration : defined configuration + Effective configuration

Disabled configuration : remove effective configuration from RAM

 Types of configuration
RAM
Defined config If the configuration is
Effective
removed from RAM,
(cfg1 + cfg2) configuration
it is called disabled.
cfg1

>cfgdisable –Disable the configuration

> cfgclear-Remove info from RAM &
Saved FPROM.
Configuration > cfgsave -Save the configuration
(cfg1 & cfg2) ASIC
+ (To sanitize switch)
cfg1 > cfgenable - Enable configuration
> cfgshow – To see the all configurations
FP ROM
 Addition a switch to the fabric

• While adding a switch to the fabric many factors should be considered.

• The factors differ if the switch added is :
1) New switch to fabric
2) Existing switch with its own configuration
New switch to fabric

ISL

RAM RAM
Defined config Effective Defined config Effective
config config
(cfg1 Z1) (cfg1 Z1)
Cfg1 Z1 Cfg1 Z1

Saved Saved
Configuration Configuration
(cfg1) ASIC (cfg1) ASIC
+ +
cfg1 cfg1

FP ROM FP ROM

NEW SWITCH
OLD SWITCH
 New switch to fabric
• When a new switch is attached to the fabric, it has no configurations stored
either in RAM or FP-ROM.

• Once the new switch is joined to the switch in fabric, the following
propagations takes place:
1) Defined configuration in the inactive portion of RAM in old
switch is propagated to inactive portion of the RAM in New switch.
2) Effective configuration in the active portion of RAM in old switch
is propagated to active portion of the RAM in New switch
3) Later the configurations are propagated to FPROM

Thus the new switch added to the fabric participates in enforcing existing
effective configuration
Addition of switch with
own configurations

Before Connecting
switches

RAM RAM
Defined config Effective Defined config Effective
config config
(cfg1 Z1) (cfg2 Z2)
Cfg1 Z1 Cfg2 Z2

Saved Saved
Configuration Configuration
(cfg1) ASIC (cfg2) ASIC
+ +
cfg1 cfg2

FP ROM FP ROM

Existing Switch with

OLD SWITCH
own configuration
 After Connecting
switches

RAM RAM
Defined config Effective config Defined Effective config
(cfg1 Z1) config(cfg1 Z1)
Cfg1 Z1 Cfg1 Z1
(cfg2 Z2) (cfg2 Z2)

Saved Saved
Configuration Configuration
(cfg1)(cfg2) ASIC (cfg2)(CFG1) ASIC
+ +
cfg1 Cfg1

FP ROM FP ROM

Existing Switch with

OLD SWITCH
own configuration
 After Connecting
switches

RAM RAM
Defined config Effective config Defined config Effective config
(cfg1 Z1)(cfg2 (cfg1 Z1)(cfg2
Z2) Cfgall(z1,z2)
Z2)
(cfgall(z1,z2))
(cfgall(z1,z2) (cfgall(z1,z2))

Saved Saved
Configuration Configuration
(cfg1)(cfg2)(cfgall) ASIC (cfg2)(CFG1)(cfgall) ASIC
+ +
cfgall Cfg1

FP ROM FP ROM

Existing Switch with

OLD SWITCH
own configuration
 Addition of switch with own configurations
• While joining a switch with own configuration to the new switch following
conflicts may arise:
- Zone content conflicts
- Zone type conflicts
- Zone name conflicts
• The above conflicts have to be resolved by identifying them, by using the
third party software tools specially designed to diagnose the conflicts
mentioned above.
• Once the conflicts are resolved join the switch, the connection may not be
established as the effective configurations conflict, to resolve the mismatch.

1) Disable the E-port of new switch to disable connectivity between switches

>portdisable <port number>
2) Disable the configuration of new switch to erase the configurations in
RAM.
>cfgdisable
 Addition of switch with own configurations
3) Enable the E-port to open communication between switches
>port enable <port number>

• Once the port is enabled the communication between the two switches open
up and the configuration of new switch is propagated to the existing switch
and loaded and saved in FPROM.

• Immediately new defined configuration is loaded to the inactive partition of

RAM.

• The defined as well as effective configurations are propagated to their

respective partitions of RAM on the new switch.
 Inter Switch Link (ISL)
ISL (Inter Switch Link):
- The cable that merges two switches are called ISL.
- Used for routing and load balancing in fabric.
- Allows exchange of switch parameters between two switches.
- ISL is divided into 8 channels, through which data is transferred.
- Control packets and data packets are sent through ISL for data integrity.

The 8 channels of ISL are (VC-0 – VC-7)

- The priorities are set to channels to be identified by ASIC .
- Based on such priorities buffers are assigned.
- ISL has 27 buffers allocated to channels depending upon the requirement
and data flow.
 Virtual Channels of ISL

VC0 VC0
S VC1 VC1 S
w w
VC2 VC2
I I
VC3 VC3
t t
VC4 VC4
C C
VC5 VC5 h
h VC6 VC6
1 2
VC7 VC7

VC0 Vc0 –priority -1 Transfers Link control frames

S VC1 Vc1 –priority -2 Link control and link control
w VC2 Vc2 –priority -3 Transfers data frames
I VC3 Vc3 –priority -3 Transfers data frames
t VC4 Vc4 –priority -3 Transfers data frames
C VC5 Vc5 –priority -3 Transfers data frames
h VC6 Vc6 –priority -4 Multicast traffic
1
VC7 Vc7 –priority -4 Broad-cast traffic
 Priorities of Buffer
ISL has 27 buffers that has been shared among the Virtual Channels based
on the priority and requirement level.

Priority 1 level - One buffer 1*1=1

Priority 2 level - Zero buffer 1*0=0
Priority 3 level - Six buffers 6 * 4 = 24
Priority 4 level - One buffer 1*2=2

ISL Subscription Ratio

- As suggested by the brocade for every three hosts ISL should be used for
better performance.
- 3:1
- Ratio = Sum of all devices bandwidth : Sum of all ISL’s bandwidth
(Number of device ports : Number of E ports e.g. 3:1,7:1,15:1)

- Maximum number of hops suggested between devices using ISL is “ 7”

- In a fabric core – edge - topology is followed.

 CORE EDGE
TOPOLOGY

Switch Switch Switch

Core switch

Switch Switch Switch

Storage
 Principal Switch/Sub-ordinate switch
• In a fabric 239 switches can be connected

• Only one switch can be a principal switch in a fabric ,that maintains domain
integrity.

• Remaining switches in the fabric are called sub-ordinate switches

• principal switch maintains information of all the sub-ordinate switches.

• principal switch updates its time from NTP server and pushes the same to
the sub-ordinate switches.

• During reconfiguration there is a chance for down time of atleast 10

seconds.
• From FOS 4.x static configuration of switch was introduced to avoid down
time.
 Reconfiguration of Fabric
• Reconfiguration of fabric takes place when a new switch is joined to the
fabric.

• If ISL is connected between the switches they immediately merge, and

messages will be generated.

• To establish communication between two switches the 24 bit dynamic

address is required

• The first octet consists of domain ID which should be unique for both the
switches, independently

• It takes 10 seconds for principal switch to ensure uniqueness of Domain ID

• Any switch at the time of booting is a sub –ordinate switch

 Merge two switches with same domain ID

Sub-ordinate
principal Switch
switch
Domain ID 1
Domain ID 1

- Once the sub-ordinate switch is powered on, principal switch send Domain
ID list to sub –ordinate switch.

- Now domain ID is conflicting

- Sub-ordinate switch updates its domain ID which ever is available

- Sends the confirmation of updated domain ID to the principal switch

 Merge two principal switches
principal
principal Switch
switch
Domain ID 1
Domain ID 1

- Both are principal and both have same domain IDs

- Now there is a race condition, as exchanging domain Id is conflicting
- So the connection is segmented as domain overlapped.
- To resolve this issue
- Login to switch
- Disable the switch
- Set the configuration
 Merge two principal switches

principal
principal Switch
switch
Domain ID 1
Domain ID 2

- Both the switches are up and are running

- Domain IDs are unique and different

- But both are principal.

- Connect the ISL cable

- The switch with least wwn is considered as principal switch

 Exchange Parameters
• When two switches are connected using ISL cable the following process
takes place
1. Ports on both the switches to which the ISL is connected are re-
configured to expansion ports E-port
2. Once configured switches generate ELP (Exchange Link Parameters)
- if the parameters match switches proceed to next process
- if parameters mismatch a message is generated stating ELP- rejected
- find out the parameter that conflicts and rectify the parameter
3. Exchange fabric parameters such as static & dynamic addresses
4. Exchange WWN
5. principal switch sends DIA ( Domain Identifier Address)
6. Up on receiving DIA sub-ordinate switch sends confirmation RDIA
(Received Domain Identifier Address)
• Once the above parameters are exchanged the switch becomes the
member of fabric
 Exchange Parameters
Principal
Switch
E-Port

Exchange Link Parameter

Exchange Fabric Parameter
ISL Exchange WWN
Domain Identifier Address
Received Domain Identifier
Address

E-Port

Sub-ordinate
Switch
 Fabric Shortest Path First
Fabric shortest path first:
- It is first proposed by Brocade
- Now it is available with almost all SAN switches
- It is basically routing protocol

What is Path?
- Distance between two points
- Reference point is required (hops) A b c D
- Involves source and destination

What is route?
- It is a part of a path
- A map between in port to out port.
b c
- Finally reaches the destination
 FSPF
- FSPF down loads routing table to ASIC.
- The cumulative cost of all ISL’s between source and target is called ISL cost
- Cost is calculated based on the speed

2Gpbs 4Gpbs 1Gpbs

- The total ISL cost = 250+500+750
500 =250
1500 750
- ASIC routes switch to switch frames using lowest cumulative costs of all available
traversal links
- Trunks with aggregated higher band-width are favored over 1 Gbps
- FSPF implemented using FOS
- FSPF does load balance at path level
 ISL Cost
• Each ISL has a metric cost based up on band width
• Cumulative cost based up on sum of all costs of all traversed ISL
• Path selected is the one with the minimum cost
• Once the path is selected the path is used until it fails
ROUTE :
- A map used to reach the next hop between import and out port with in a
path.
- Only 7 hops are allowed between source and destination switch.
500
0

50
75

0
500 750 500 250
0
25
25

0 500
25
0

500
 LINK COST METRIC

• Link cost may be dynamic or static

• By default it is set dynamic
• Link cost set on out port
• Link cost is not bi-directional
• The default metrics are

1000 at 1Gbps
500 at 2 Gbps
250 at 4 Gbps

• Link cost and bandwidth are inversely related .

 DYNAMIC LOAD SHARING
• Allows load sharing across multiple equal cost paths

• Provide better utilization of ISL and increase performance

• Its selects the path in round robin way using algorithm developed by
brocade

• The sharing is switch centric

• If DLS is set and any host is removed the other host will shift to next ISL

a
Round robin b
sharing
c
 DYNAMIC LOAD SHARING

1 2 3 1 2 3 1 2 3
 TRUNKING

• Combines multiple ISL’s in to a single ISL

• Minimum two ISL’s are required

• Maximum limit is four

• ISL’s in one trunk group should be from the same quad

• Trunk does frame level load balance

• Supports multiple trunk between switches

 TRUNKING

Frames
ISL
 Cont’d
• Preserves in-order delivery

• Trunking license key is required on both the sides of ISL

• Ports must be operating at the speed of 2Gbps

• Between switches different lengths of cables are used

• Difference between longest ISL and shortest ISL is called diskew value
 IOD (In order delivery)
• Frames are delivered in order in spite of being traversed across multiple-
path.
• If any changes occurs in the topology, the principal ISL goes down. the
frames may go out-of –order, during a brief time during the RCF.
• The default behavior is to automatically enable out-of-order frames during
fabric topology changes.
• To reroute frames after a fabric topology change and ensure in – order
delivery.

- iodreset : If an E-Port route goes down ,frames are transmitted across a

new route or re-uses previous route.
- iodset : Once the Para meter is set ,the frame are delivery in order
- iodshow : To show if the IOD is set or not