Exploring Brocade ISL Trunking

SOLUTIONware ----Rev1 Tech Note
Last modified: January 2002

This document provides an in-depth examination of trunking and how trunking impacts the design, implementation, and maintenance of a SAN.

Category: SOLUTIONware Technical Note Sub-category: SAN Design Key words: ISL trunking, SAN Design

1

53-0000263-01

Table of Contents

Table of Contents..........................................................................................................................................................2 Introduction ..................................................................................................................................................................3 Trunking Behavior .......................................................................................................................................................3 Virtual Channels......................................................................................................................................................6 High Availability and Dynamic Bandwidth Allocation ...............................................................................................7 Adding An ISL .........................................................................................................................................................7 An ISL Failure or Removal ....................................................................................................................................8 Performance................................................................................................................................................................12 Monitoring Performance.......................................................................................................................................12 ISL Performance....................................................................................................................................................12 Trunking Maximizes Performance and Minimizes The SAN Management Effort .........................................13 How Does Trunking Impact SAN Design..................................................................................................................15 Tip: Place Trunking-Capable Switches Adjacent To Each other .....................................................................15 Tip: Place Trunking-Capable Switches In The Core .........................................................................................15 Tip: Leave Open Ports On The Quad..................................................................................................................16 Tip: If Two Trunkable Switches Are Connected By More than One ISL, Use Trunking...............................16 Trunking Requirements..............................................................................................................................................17 ISL Trunking Administration ....................................................................................................................................19 Summary .....................................................................................................................................................................21 Appendix A: Terms and Definitions..........................................................................................................................22 Appendix B: Helpful Trunking Commands..............................................................................................................24 Copyright.....................................................................................................................................................................25 Evaluation Form.........................................................................................................................................................26

Page 2 of 26

Out of order and in order delivery is shown in Figure 1. 1 is considered out of order. As long as at least one ISL link remains. three. 2. 3. implementation. 3 and any other order.without impacting I/O -. 1. It is also possible to dynamically increase bandwidth by adding ISLs to a trunk -. The in order delivery for the storage to receive this sequence is 0. Trunking Behavior Due to the implementation of some Fibre Channel protocol devices. Currently the SilkWorm 3800 and 12000 switches support trunking. Additionally. A trunk group logically joins two. Use of trunking can minimize or eliminate congestion in the SAN because trunking optimizes ISL utilization. For reference. Figure 1 In order and Out of Order Delivery In Order Delivery 3210 SAN 3210 2301 Out Of Order Delivery Page 3 of 26 . and maintenance of a SAN.albeit at a lower bandwidth.53-0000263-01 Introduction This document provides an in-depth examination of trunking and how trunking impacts the design. Trunking can also increase availability. or four ISLs into one logical ISL. a convenient list of trunking definitions and terms is provided at the end of this document in Appendix A. 3.to enable up to 8Gb/s of bandwidth over a single logical link. I/O continues if an ISL failure occurs -. the use of trunking minimizes the effort of managing a SAN since ISLs are now managed as a group instead of individually. frame traffic between a source device and a destination device must be delivered in-order within an exchange. 2. Trunking is a feature that enables traffic to be optimally shared across available inter-switch links (ISLs) while preserving in-order delivery. 0. The host is sending a SCSI sequence in the order 0. 1. such as 2.

In Figure 2. The two ISLs provide an aggregate bandwidth of 400 MB/s. It is possible to deterministically setup the routes of a fabric with static routes by using the uRouteConfig command and prevent the scenario depicted in Figure 2 from happening. as there is sufficient bandwidth to support the 10 MB/s requirement. This situation is depicted in Figure 2. With trunking it is not necessary to setup static routes or manage individual ISLs to achieve optimal performance. As a consequence. it is still possible to encounter congestion due to load variations and fluctuations. For example. which require 150 MB/s each of bandwidth will share the left-hand ISL. as only 200 MB/s is available from the utilized ISL and the bandwidth requirement is 300 MB/s. However. even if the route assignments were different. Hosts A & B will experience congestion. certain traffic patterns may become unevenly distributed. however. Host C will experience no congestion.53-0000263-01 This restriction forces the current Brocade frame routing method to fix a routing path within a fabric. hosts A & B. a tape backup might create enough of a spike in demand to cause congestion on an ISL that would otherwise be avoided if trunking were utilized. If the routing initialization happened in a different order. while host C will utilize the right-hand ISL. doing so require some effort and doing so for a large SAN with numerous ISLs can become a very complex task. Note that the aggregate bandwidth required is 310 MB/s. Page 4 of 26 . Fabric Shortest Path First (FSPF) establishes routes in a round robin fashion while preserving in-order delivery. Hosts A and B require an aggregate of 300 MB/s of bandwidth while host C requires 10 MB/s of bandwidth (Note: The SilkWorm 3800 switch supports 2 Gb/s technology). leaving some paths congested and other paths underutilized. there would not be any congestion. say host A and host C on one ISL and host B on the other ISL.

h a n d IS L ) is 2 0 0 M B /s . A v a ila b le b a n d w id th ( le ft. Notice that all variables are the same. This means that we have gone from an unbalanced load situation with a deficit of 100 MB/s to a surplus bandwidth of 90MB/s. the trunk provides sufficient bandwidth for all traffic streams.h a n d IS L ) is 2 0 0 M B /s B a n d w id th S u r p lu s = 1 9 0 M B /s Figure 3 depicts a similar scenario as shown in the Figure 2 configuration. except that trunking is utilized. Page 5 of 26 . The trunk is a logical aggregation of the two 200 MB/s ISLs into one logical 400 MB/s trunk. A v a ila b le b a n d w id th ( r ig h t. with two 2-ISL trunks (4 Gb/s each) – each trunk connected to a different switch. It is also possible to divide the four ISLs in a quad between two trunk groups.53-0000263-01 Figure 2 Uneven distribution of traffic without trunking host A 1 5 0 M B /s host B 1 5 0 M B /s host C 1 0 M B /s A g g r e g a te r e q u ir e d b a n d w id th o f h o s t A a n d h o s t B is 3 0 0 M B /s . As shown in Figure 3. Trunking is dynamic and responds in real-time to performance issues. Note that trunking allows the grouping of up to four 2 Gb/s ISLs into a maximum of one logical 8 Gb/s trunk. B a n d w id th D e fe c it = 1 0 0 M B /s host A host B host C H o s t C o n ly r e q u ir e s 1 0 M B /s o f b a n d w id th .

A v a ila b le b a n d w id th is 4 0 0 M B /s . and reducing the total cost of SAN ownership. simplifying capacity planning and fabric design.53-0000263-01 Figure 3 Performance and management benefits of trunking A g g re g a te re q u ire d b a n d w id th o f h o s t A . Critical traffic for management communication and link control receive dedicated channels and the highest priorities. & h o s t C is 3 1 0 M B /s . B a n d w id th S u r p lu s = 9 0 M B /s host A 1 5 0 M B /s host B 1 5 0 M B /s host C 1 0 M B /s host A host B host C host A host B host C 2 IS L T ru n k = 4 0 0 M B /s Virtual Channels Brocade Virtual Channels is a unique technology that logically partitions physical bandwidth and prioritizes traffic within inter-switch links (ISLs). Data traffic can be sent across multiple channels with different priority levels. It is essential for enabling large fabrics by insuring the reliable operation of the network. Page 6 of 26 . improving the efficiency and performance of switch-to-switch communication. Brocade Virtual Channels has individual circuits for eight different channels with four different priority levels. h o s t B .

The virtual channel technology logically partitions bandwidth within a single ISL and ISL Trunking logically aggregates bandwidth across up to four different ISLs. which consists of two 2 Gb/s ISLs. The underlying technology uses an antistarvation algorithm that insures that all Brocade switches are non-blocking and it is used as the foundation for Brocade Extended Fabrics to communicate over long distances.2 drivers Clariion 4500 w/ 2 controllers 4 x JMR JBODs w/ (8) Seagate ST336605 FC disks 3800 3800 Page 7 of 26 .53-0000263-01 Brocade Virtual Channels is integral to Brocade ISL Trunking and is built into the switch ASIC.1 FabricOS 6 x Sun Hosts w/ Solaris 5. With trunking. the hosts and storage communicate via a 4 Gb/s trunk. there is no disruption of I/O and I/O is immediately spread across the existing ISLs and the new ISL. High Availability and Dynamic Bandwidth Allocation When adding ISLs to an existing trunk. Adding An ISL Figure 4 depicts the configuration utilized to quantify the impact of an ISL addition to a trunk Essentially.0. Figure 4 Configuration Utilized To Quantify Dynamic Bandwidth Allocation Sun Hosts CLARiiON 4500 JMR JBODs I/O 3800 3800 4 x SilkWorm 3800 w/ v3. the deletion or failure of an ISL to an existing trunk can impact the I/O traversing that trunk.8 Emulex PCI LP8000 (1 Gb/s) & LP9000 (2 Gb/s) w/ 4. which is a part of every Brocade switch product. they provide maximum network efficiency and they enable Brocade switches to scale cost effectively into large high performance reliable storage area networks. Together.

53-0000263-01 In Figure 5. the I/O consumption expands from 400 MB/s to 420 MB/s to utilize the additional bandwidth enabled by the ISL addition to the trunk. the behavior and impact to I/O is dependent upon the host. Furthermore. Figure 5 I/O Immediately Spreads To The New ISL When An ISL Is Added To A trunk 2 ISL Trunk 3rd ISL 7 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 -------------------------------------------------------------------------------0 0 112 0 0 0 0 0 0 0 0 0 0 0 0 0 0 112 0 0 0 0 0 0 0 0 0 0 0 0 201m 201m 198m 199m 201m 201m 200m 201m 0 0 0 0 0 0 0 0 0 0 0 0 55m 55m 55m 55m 55m 55m 55m 55m 91m 91m 91m 91m 91m 91m 91m 92m 93m 92m 92m 91m 94m 93m 93m 92m 55m 55m 55m 55m 55m 55m 55m 55m 22m 22m 22m 21m 30m 31m 30m 31m 22m 21m 22m 22m 28m 31m 30m 30m 30m 28m 31m 31m 30m 30m 30m 30m 30m 29m 31m 31m 31m 30m 30m 31m 2.2k 2. there is minimal or no disruption of I/O and the I/O will be distributed over the remaining ISLs that form the trunk. Once port 6 was enabled. and HBA utilized. storage. Notice how the aggregate bandwidth utilized. The test involved the use of the UNIX dd utility issuing a maximal I/O load from the six Sun hosts to the Clariion and JMR storage. application.0k 154m 154m 109m 140m 140m 140m 139m 139m 139m 140m 139m 139m Point where 3rd ISL is added An ISL Failure or Removal When an ISL that is not a trunk master is removed or fails. now distributes over the additional ISL. then the time for a fabric reconfiguration also needs to be included. Port 6 was then enabled. If the master link happens to be an upstream or downstream port (see Appendix A: Terms and Definitions for more detail). It is possible that upon failure or removal of the ISL that some frames were in flight and that these frames will be lost. No interruption to I/O was noticeable. When an ISL that is a trunk master is removed or fails. the portPerfShow command is executed while an I/O load is issued between hosts and storage. it immediately joined the trunk and I/O was immediately distributed. Page 8 of 26 . The recovery period for lost frames needs to be accounted for before I/O can be expected to fully recover. which is approximately 400 MB/s. When this happens. Ports 4 and 5 are the existing 4 Gb/s trunk. there will be a pause in I/O associated with that trunk as the I/O is re-routed and distributed over the remaining ISLs that form the trunk.

For this test.0. The same test was executed and instead of disabling a trunking slave. the timer stops. The I/O rate decreases slightly and after approximately three seconds. The test involves performing I/O between the host and the storage with the dd UNIX utility.2 drivers Clariion 4500 w/ 2 controllers 4 x JMR JBODs w/ (8) Seagate ST336605 FC disks 3800 3800 Page 9 of 26 . Once I/O resumes. the I/O rate returns to steady state. the timer starts. at which point. a trunking master was disabled. Specifically the portPerfShow command is monitored for a pause or decrease in I/O. Figure 6 Configuration Utilized To Quantify The Impact Of An ISL Failure Within A Trunk Upon I/O Sun Hosts CLARiiON 4500 JMR JBODs I/O 3800 3800 4 x SilkWorm 3800 w/ v3. Once the I/O begins. there was no pause in I/O. the average pause in I/O was twenty seconds.53-0000263-01 Figure 6 depicts the configuration utilized to quantify the impact of an ISL failure within a trunk. a trunk port is disabled (portDisable command) and the period until I/O resumes is measured.8 Emulex PCI LP8000 (1 Gb/s) & LP9000 (2 Gb/s) w/ 4. When a trunking slave is disabled.1 FabricOS 6 x Sun Hosts w/ Solaris 5.

The flow of data essentially continues with minimal impact. Note that I/O stops for port 6.53-0000263-01 Figure 7 displays a capture of portPerfShow data from a switch while I/O was running and a trunk slave ISL was disabled (portDisable). which was the port that was disabled.0m Slave failure Page 10 of 26 . however. Figure 7 PortPerfShow Data During A Trunk Slave Failure Trunk 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 36 0 3 0 0 0 0 0 0 0 0 0 76 0 4 0 0 0 0 0 0 0 0 0 0 0 5 6 7 8 30m 30m 30m 29m 26m 29m 27m 28m 29m 28m 25m 9 91m 91m 91m 89m 88m 87m 87m 87m 89m 91m 91m 10 93m 93m 91m 91m 90m 90m 89m 87m 91m 92m 93m 11 93m 93m 93m 91m 90m 89m 89m 90m 91m 92m 12 27m 25m 27m 15m 0 0 21m 19m 21m 21m 13 27m 25m 27m 27m 26m 27m 25m 19m 20m 21m 20m 14 35m 34m 35m 19m 0 0 14m 34m 35m 35m 34m 15 35m 35m 35m 40m 48m 48m 42m 36m 35m 35m 35m -------------------------------------------------------------------------------145m 144m 145m 143m 142m 143m 144m 143m 144m 162m 185m 186m 193m 202m 207m 209m 207m 0 0 0 0 0 0 0 0 162m 185m 186m 192m 202m 206m 209m 207m 89m 8. there is a pause while ports 12 and 14 get rerouted across the remaining ISLs that form the trunk.

6k 191m 190m 206m 206m 206m 206m 205m 205m Master failure I/O Resumes Page 11 of 26 .4m 1. Once the trunk master (port 5) fails by disabling the trunk master. Figure 8 PortPerfShow Data During A Trunking Master Failure Trunk 0 0 0 0 0 680 0 1 0 0 0 0 40 0 2 0 0 0 0 936 0 3 0 0 0 0 220 0 4 0 0 0 0 0 0 0 5 6 7 8 30m 31m 30m 20m 0 0 0 9 88m 89m 89m 61m 0 0 10 90m 92m 92m 63m 0 0 11 91m 90m 91m 61m 0 0 12 27m 27m 25m 18m 0 0 0 13 27m 27m 25m 18m 0 0 0 14 35m 35m 34m 21m 0 84 0 15 35m 35m 35m 21m 0 0 0 -------------------------------------------------------------------------------142m 141m 141m 143m 143m 143m 142m 141m 141m 0 0 0 0 94m 824 0 0 95m 1.6k 1.53-0000263-01 The portPerfShow data shown in Figure 8 is taken during the failure of a trunk master ISL.6k 6.6k 1. I/O pauses for approximately twenty seconds before it resumes.8m 6.1k 0 0 133k 133k 275k 259k 267k 267k 267k data deleted to preserve space – essentially a 20 second pause when no I/O is active 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1.6k 1.4m 2.6k 1.1k 832 25m 35m 25m 35m 51m 51m 50m 29m 30m 30m 13k 18m 91m 90m 88m 88m 13k 0 62m 91m 91m 90m 0 85m 91m 91m 90m 0 19m 19m 20m 20m 0 17m 19m 20m 20m 832 0 26m 34m 35m 35m 832 0 28m 36m 35m 35m 13k 1.6k 6.

it is only necessary to have an I/O load that is capable of saturating a single ISL. The level of uniformity depends on the payload variations at the frame level. To realize the value of trunking. As discussed earlier in this document and as is detailed later in this section. ISL Performance In Figure 9. trunking eliminates this problem. it is possible to starve other devices – even if additional ISLs are available between the communicating devices. While optimal bandwidth utilization is the goal of the Brocade ISL Trunking feature. if Host A writes to Storage 1 at 200 MB/s and Host B reads from Storage 2 at 200 MB/s it is possible to generate 400 MB/s of I/O on a single ISL. Monitoring Performance Port statistics may not be the same across all participating ISLs within a trunk. Once this saturation occurs. This performance capability equates to 1600 MB/s (16Gb/s) of potential bandwidth for a four ISL trunk. Figure 9 2 Gb/s Duplex Performance Host A 200 MB/s (write) 3800 3800 Storage 1 200 MB/s (read) Host B Storage 2 all device connections and ISL are 2 Gb/s Page 12 of 26 . With duplex 2 Gb/s ISLs. To do so requires perfectly balanced I/O and devices capable of generating sufficient levels of I/O.53-0000263-01 Performance The combination of 2 Gb/s technology and the capability of ISL trunking enable SilkWorm switches to deliver impressive performance. the potential bandwidth is really 400 MB/s – one stream achieving 200 MB/s in one direction and another stream achieving 200 MB/s in the opposite direction. traffic might not be distributed exactly the same across all ISL links within a trunk. Achieving such performance in a real life situation is unlikely.

the ISL on port 5 is operating at maximum bandwidth (208 MB/s) in the test where no trunking is utilized. with a small percentage of writes. It is also possible that the devices using port 5 were experiencing congestion. The load is run with and without trunking. that device would encounter congestion and it is likely that intervention would be necessary to manage the performance conflict.2 drivers Clariion 4500 w/ 2 controllers 4 x JMR JBODs w/ (8) Seagate ST336605 FC disks Trunking Maximizes Performance and Minimizes The SAN Management Effort In Figure 11. Should another device attach to the switch and be assigned a route over port 5. and 6 are utilized for ISLs. In addition.53-0000263-01 In a lab setting it is possible to approach maximal ISL bandwidth. The load is primarily read oriented. notice how the loads in the test run without trunking enabled are unbalanced.1 FabricOS 6 x Sun Hosts w/ Solaris 5. Given the configuration shown in Figure 10 and using the dd UNIX utility. Ports 4.8 Emulex PCI LP8000 (1 Gb/s) & LP9000 (2 Gb/s) w/ 4. an identical read oriented I/O load is generated in the configuration with the dd UNIX utility. it was possible to generate over 350 MB/s on a single ISL. Page 13 of 26 . 5. The limiting variable in this test was that there was not enough storage to generate a 400 MB/s I/O load.0. Due to the way in which FSPF established routes. Figure 10 Configuration Used To Achieve Maximal ISL Bandwidth ISL portPerfShow 1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -------------------------------------------------------------------------------0 0 0 0 355m 0 0 0 31m 43m 93m 94m 22m 22m 23m 23m 0 0 0 0 354m 0 0 0 31m 43m 94m 93m 22m 22m 23m 22m 0 0 0 0 349m 0 0 0 31m 43m 93m 93m 22m 22m 20m 20m Sun Hosts CLRiiON 4500 JMR JBODs I/O 3800 3800 2 x SilkWorm 3800 w/ v3. while ports 8 through 15 are used for storage.

the load is evenly distributed and there exists surplus bandwidth within the trunk to accommodate additional devices. It is not necessary to intervene to manage the addition of devices since the resources are optimally utilized.8 Emulex PCI LP8000 (1 Gb/s) & LP9000 (2 Gb/s) w/ 4.53-0000263-01 In the test run with trunking enabled.2 drivers Clariion 4500 w/ 2 controllers 4 x JMR JBODs w/ (8) Seagate ST336605 FC disks Page 14 of 26 .0. Figure 11 Fabric Performance With and Without Trunking Without Trunking 3 ISLs portPerfShow 1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -------------------------------------------------------------------------------0 0 0 0 101m 208m 89m 0 55m 76m 76m 90m 28m 27m 23m 23m 112 0 112 0 103m 208m 94m 0 55m 76m 76m 94m 28m 28m 23m 23m 0 0 0 0 100m 208m 93m 0 55m 76m 76m 93m 26m 28m 22m 22m With Trunking 6 Gb/s trunk portPerfShow 1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -------------------------------------------------------------------------------0 0 0 0 132m 132m 132m 0 55m 90m 93m 55m 27m 27m 23m 23m 0 0 0 0 132m 132m 131m 0 55m 90m 92m 55m 27m 27m 23m 23m 0 0 0 0 131m 132m 131m 0 55m 91m 93m 55m 26m 27m 22m 22m Sun Hosts CLARiiON 4500 JMR JBODs I/O 3800 3800 2 x SilkWorm 3800 w/ v3.1 FabricOS 6 x Sun Hosts w/ Solaris 5.

it will allow the added switches to connect with trunking-capable switches. The high performance capabilities of a trunking-capable switch also makes these types of switches ideal for placement in the Core of a Core/Edge fabric (refer to the Brocade SAN Design Guide. Cascade. Page 15 of 26 . including Core/Edge. Migrating existing SilkWorm switches not capable of trunking from the core to the edge and inserting trunking-capable switches into the core is an effective strategy. and Ring topologies.53-0000263-01 How Does Trunking Impact SAN Design Trunking optimizes the utilization of ISLs and reduces the SAN administration effort. these switches can harness the power of trunking. Doing so enables the ability to trunk. it is less likely that congestion will be experienced and it is possible that fewer ISL are required in the fabric topology. Tip: Place Trunking-Capable Switches Adjacent To Each other When designing a SAN with trunking-capable switches or introducing trunkable switches to an existing SAN. Mesh. publication number 53-0000231-xx for a detailed discussion regarding SAN Design). yielding additional ports for attaching SAN devices. the SAN that is easier to manage has the advantage. As new trunking-capable switches are added to a Core/Edge topology. a single trunk is now monitored. Tip: Place Trunking-Capable Switches In The Core When working with a Core/Edge topology. These benefits enhance the utility of ISLs and so enhance the utility of designs that make use of ISLs. as shown in Figure 12. Then as new trunking switches are added. Given two SANs of equivalent functionality. Because the trunk is efficient. it is important to place these switches adjacent to each other when working with any topology. This strategy also preserves the investment in switches that are not capable of trunking. it is recommended to place the trunkable switches in the Core. Instead of monitoring multiple ISLs.

53-0000263-01 Figure 12 Place trunking-capable switches in the core and redeploy the former core to the edge Tip: Leave Open Ports On The Quad When connecting two switches in a fabric with one. consider leaving open the other ports on the quad for future trunk growth and fill in the remaining open ports with SAN devices first. two or three ISLs. Tip: If Two Trunkable Switches Are Connected By More than One ISL. Page 16 of 26 . Use Trunking When connecting two switches with two or more ISLs consider utilizing trunking to achieve optimal performance and minimize the management effort.

Note that trunking is enabled by default in v3. 2. Page 17 of 26 . Note. 8-11. The difference in cable length between all ports in a trunking group must be less be less than 400 meters. Current switches that support trunking are the SilkWorm 3800. 3. 12-15. Trunking is not supported on the SilkWorm 2000 series or SilkWorm 1000 series (Fabric OS 1.53-0000263-01 Trunking Requirements For trunking to be enabled several requirements must first be met. that the SilkWorm 3800 and 12000 quads are color coded to delineate a quad. 4. To check whether or not a switch is licensed for trunking.x) switches. supported. so nothing is required to enable 2 Gb/s connectivity as the default is to connect at 2 Gb/s if the other device is 2 Gb/s capable. and 12000. as shown in Figure 14. and should be kept to 30 meters or less within a trunk to ensure optimal performance and bandwidth utilization.x and v4.x Fabric OS if there exists a trunking license. The two switches that are being trunked together must be licensed for trunking. The default state of 2 Gb/s switch ports is to auto-negotiate.x or 2. If a switch is licensed and meets these requirements. Also the default state for SilkWorm 2000. use the command portCfgShow to verify the port configuration. It is possible. If there is some question as to a port’s suitability for trunking. The ports on both switches being trunked must be configured as E_Ports and these ports must run at 2Gb/s. For example: 0-3. It is supported to have two trunk groups per quad. and 12000 series switch ports is to automatically configure as an E-port when attached to another Brocade switch. The switch platform must support trunking. 4-7. use the licenseShow command. and encouraged to connect switches that are not capable of trunking to switches that are capable of trunking. it is only necessary to connect up the ISLs to form a trunk. The following requirements are necessary to trunk ISLs: 1. 3000. The ports that form a trunk must reside in the same contiguous four-port groups. meaning that two 2-ISL trunks (4 Gb/s each) are resident on the same quad and each trunk then connects to a different switch.

grouped into four quads Figure 14 Port groupings that form a Quad For SilkWorm 3800 Quads 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Page 18 of 26 .53-0000263-01 Figure 13 Port groupings that form a Quad For SilkWorm 12000 Quads Each SilkWorm 12000 switch assembly blade consists of 16-ports.

both methods provide excellent ease of use and an appropriate level of control. All ports have ISL Trunking enabled by default.53-0000263-01 ISL Trunking Administration Once an ISL Trunking license has been purchased and installed ISL Trunking can be administered either through telnet commands or through Brocade Web Tools. Figure 16 shows the Port Setting Tab that displays which ports in the switch have ISL Trunking enabled. Figure 15 shows the Trunk Information Tab that lists the trunk groups in the switch. Combined together. Brocade Web Tools provides an intuitive graphical interface to enable and review ISL Trunking settings. The ISL Trunking screens are accessed through a web browser using the Admin View of Web Tools. the Master Ports for the trunk groups. a list of helpful trunking commands has been included as a reference in Appendix B of this document. For convenience. and the Member Ports for the trunk groups. Figure 15 Web Tools Trunk Information Tab Page 19 of 26 . ISL Trunking can be enabled or disabled by simply clicking the check box.

53-0000263-01 Figure 16 Web Tools Port Settings Tab Shows Which Ports Have Trunking Enabled Page 20 of 26 .

A four ISL trunk is potentially capable of 1600 MB/s when operating full duplex. however. Virtual Channels are integral to Brocade ISL Trunking and are essential for enabling large fabrics by insuring the reliable operation of the network. Brocade 2 Gb/s switches also have the ability to create up to two trunk groups per quad.53-0000263-01 Summary A trunk logically joins two. as long as one ISL remains communication between the switches will continue. consider the use of trunking to achieve optimal performance and minimize than SAN management effort. When designing and implementing SANs. place trunking capable switches adjacent to each other and in the core of a core/edge fabric. or four ISLs into one logical ISL with a capacity of up to 8 Gb/s while preserving in-order delivery. An ISL is capable of supporting up to 400 MB/s when operating full duplex. To trunk. it is necessary to use switches that are licensed for trunking and support trunking. Use of trunking can minimize or eliminate congestion in the SAN because trunking optimizes ISL utilization. In supported products. the use of trunking minimizes the effort of managing a SAN. such as the SilkWorm 3800 and 12000. If two switches are connected by more than one ISL. It is possible to dynamically increase bandwidth in a trunk and not impact ongoing I/O by simply adding an ISL to the trunk. three. Ports that form a trunk must reside in a quad. Page 21 of 26 . and improving the efficiency and performance of switch-to-switch communication. Fill up the open ports of a quad last so as to leave open these ports for trunk expansion. Trunking also increases high availability. run at 2 Gb/s. Since ISLs are now managed as a group instead of individually. The failure of an ISL within a trunk will result in some impact to I/O. and be cabled together with cables that ideally differ in length by less than 30 meters.

Trunking Master: Only one of the trunking ports is used to set up all routing paths for the entire trunking group. FSPF: Fabric Shortest Path First protocol. software functionalities. Trunking Ports: Ports that belong to a trunking group are called trunking ports (see Figure 17). Such a set of links is called trunking group (see Figure 17). If using the same cable lengths to connect two switches. The higher the deskew counter value. and definitions relative to trunking and ISLs are defined and explained below. • • • • • • • Page 22 of 26 . Upstream/Downstream Designator: The terms upstream and downstream designate that the particular position of a switch in reference to the principal switch in the fabric. with each trunk connected to a different switch. Trunking Group: Trunking uses a simple algorithm to optimally distribute frames across a set of available paths that link two adjacent switches. Trunking Links: The components that comprise a trunk are termed trunking links. The failure of any slave port has minimal impact on the trunk or the traffic the traverses this trunk. The FSPF protocol was developed by Brocade and subsequently adopted by the Fibre Channel standards community for allowing switches to discover the fabric topology and route frames correctly. the longer the cable lengths. This means having two 2-isl trunks. A trunking link is analogous to an ISL. the remaining slave links will also go offline and regroup with a new master if there are at least two remaining active trunking ports remaining. The principal switch has no upstream ports. It is the industry standard routing protocol for Fibre Channel networks. It is possible to have two trunk groups per quad. Trunking Slave: Non-master trunking ports from a trunking group are termed trunking slaves. the path to the principal switch is upstream through port 0. The failure of any master port impacts the traffic that traverses this trunk since a regrouping of the trunks slaves must occur and because some in-flight data may be lost. A link consists of the two trunking ports and the connecting components: SFP and cable. Note that if a master trunking link goes offline or fails. • Deskew Counter: This value is a metric used to identify the differences in cabling distances between trunking links within a trunking group. This port is called trunking master port (see Figure 17). expect to see the deskew counters for each ISL in the trunk to have very similar values. In Figure 17. For switch core1. This metric is used to ensure in-order delivery across trunking links within the same trunking group. These paths are used in the process for assigning switch domain IDs.53-0000263-01 Appendix A: Terms and Definitions Several terms. notice that switch “edge1” is the principal switch and all “stream” designators are downstream of switch edge1.

53-0000263-01 Figure 17 Trunking and ISL Elements Principal edge1 edge2 edge3 edge4 edge5 edge6 edge7 core1 core2 1 2 3 4 3800 2800 ISL ISL Trunk ISL Trunk ISL Trunk edge1:admin> switchshow switchName: edge1 switchType: 2.4 switchState: Online switchRole: Principal switchDomain: 2 switchId: fffc02 switchW wn: 10:00:00:60:69:10:8d:fd switchBeacon: OFF port 0: sw Online E-Port 10:00:00:60:69:50:03:de "core1" (downstream) port 1: sw Online E-Port 10:00:00:60:69:50:03:a2 "core2" (downstream) port 2: sw No_Light port 3: cu No_Light port 4: sw No_Light port 5: sw No_Light port 6: -.No_Module port 7: -.1 switchState: Online switchRole: Subordinate switchDomain: 8 switchId: fffc08 switchW wn: 10:00:00:60:69:50:03:de switchBeacon: OFF port 0: id 1G Online E-Port 10:00:00:60:69:10:8d:fd "edge1" (upstream) port 1: id 1G Online E-Port 10:00:00:60:69:11:f9:f7 "edge2" port 2: id 1G Online E-Port 10:00:00:60:69:10:9b:52 "edge3" port 3: id 1G Online E-Port 10:00:00:60:69:12:f9:8c "edge4" port 4: id 1G Online E-Port 10:00:00:60:69:12:f9:8c "edge4" port 5: id 1G Online E-Port 10:00:00:60:69:10:9b:5b "edge5" port 6: id 2G No_Light port 7: id 2G No_Light port 8: id 2G Online E-Port 10:00:00:60:69:50:03:b3 "edge7" (Trunk master) port 9: id 2G Online E-Port (Trunk port.No_Module port 12: id No_Light port 13: sw No_Light port 14: -.No_Module port 15: sw No_Light core1:admin> switchshow switchName: core1 switchType: 9. master is port #12) Trunk Master } Trunking Group } Trunking Group Page 23 of 26 . master is port #12) port 14: id 2G Online E-Port (Trunk port.No_Module port 10: id No_Light port 11: -. master is port #12) port 15: id 2G Online E-Port (Trunk port.No_Module port 8: id No_Light port 9: -. master is port #8) port 11: id 2G No_Light port 12: id 2G Online E-Port 10:00:00:60:69:50:02:d0 "edge6" (Trunk master) port 13: id 2G Online E-Port (Trunk port. master is port #8) port 10: id 2G Online E-Port (Trunk port.

53-0000263-01 Appendix B: Helpful Trunking Commands An annotated list of trunking related commands follow to provide a quick reference. Helpful for identifying deskew constant. Page 24 of 26 .0 Fabric OS.0 Fabric OS. This command is available in all versions of Fabric OS. please reference the manuals or “help” page. This command is new in V3. which would prevent that port from joining into the trunk group. This command is new in V3. Trunking is enabled by default in the V3. a port could be configured as a nonE_Port. This command is new in V3. trunkDebug: Use this command to assist with troubleshooting a trunk. switchShow: Use this command to check status of switch ports and trunking.0 Fabric OS. Identifies aggregate trunk bandwidth. portCfgTrunkport: Use this command to individually configure a port for trunking. switchCfgTrunk: Use this command to enable/disable an entire switch for trunking. This command is new in V3.x and V4. trunkShow: Use this to display detailed trunking information.x Fabric OS releases. portCfgShow: Use this command to view the configuration status of all ports. Supply two local port numbers. An improperly configured port cannot join a trunk group.0 Fabric OS. This command is new in V3. For example. For detail on a particular command.0 Fabric OS. islShow: Use this command to display ISL status. This command is new in V3.0 Fabric OS.

2002. WITHOUT NOTICE. BROCADE RESERVES THE RIGHT TO MAKE CHANGES TO THIS DOCUMENT AT ANY TIME. CA 95110 Page 25 of 26 . nor does it imply product availability. SilkWorm Express. Brocade Communications Systems. Brocade Communications Systems. CONCERNING ANY EQUIPMENT. and are used to identify. Brocade does not warrant or guarantee that anyone will be able to recreate or achieve the results described in this document. products or services of their respective owners. All other brands. Incorporated 1745 Technology Drive San Jose. It is intended solely as an aid for installing and configuring Storage Area Networks constructed with Brocade switches. and the Brocade logo are trademarks or registered trademarks of Brocade Communications Systems. in the United States and/or in other countries. equipment. or service. The installation and configuration described in this document made use of third party software and hardware. SilkWorm. Brocade does not make any warranties or guarantees concerning such third party software and hardware. products. AND ASSUMES NO RESPONSIBILITY FOR ITS USE. Brocade is not responsible for the use of this document and does not guarantee the results of its use. Incorporated. NOTICE: THIS DOCUMENT IS FOR INFORMATIONAL PURPOSES ONLY AND DOES NOT SET FORTH ANY WARRANTY. CONTACT A BROCADE SALES OFFICE FOR INFORMATION ON FEATURE AND PRODUCT AVAILABILITY. This document does not provide a warranty to any Brocade software. EXPRESS OR IMPLIED. Inc. THIS INFORMATIONAL DOCUMENT DESCRIBES FEATURES THAT MAY NOT BE CURRENTLY AVAILABLE. OR SERVICE OFFERED OR TO BE OFFERED BY BROCADE. ALL RIGHTS RESERVED. Part number: 53-0000263-01 Brocade. EQUIPMENT FEATURE. or service names are or may be trademarks or service marks of. Export of technical data contained in this document may require an export license from the United States Government..53-0000263-01 Copyright IMPORTANT NOTICE This document is the property of Brocade.

Please FAX this form back to: FAX (408) 392-5200. In particular. problem. we are interested in understanding how the SOLUTIONware report helped make a difference in a SAN task. SOLUTIONware Title: Exploring Brocade ISL Trunking Document number: 53-0000263-01 Review: What other subjects would you like to see SOLUTIONware reports on? Please rate your overall satisfaction: ( )very satisfied ( )satisfied ( )not satisfied Please identify yourself as belonging to one of these groups: ( ) End-User Customer ( ) Fabric Integrator ( ) Master Reseller ( ) Partner ( ) Brocade Employee ( ) Brocade OEM ( )Other Page 26 of 26 . or decision.53-0000263-01 Evaluation Form Your feedback is valued by Brocade.