This action might not be possible to undo. Are you sure you want to continue?
Fibre Channel Theory Fundamentals Introduction
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
This FREE Web-based Fibre Channel Theory Fundamentals course was designed to act as a level set for new SAN adopter's and serve as a prerequisite for Brocade courseware. It is presented as a FREE service offering for all.
After completing this course, attendees should be able to:
Describe key reasons, benefits and components related to Fibre Channel (FC) Storage Area Networks (SANs) Identify FC protocol layers and key related tasks & components List key FC topologies, terminology, and addresses Describe FC services and expected behaviors Summarize and state relevance of FC theory fundamentals learned and know where to find additional FC information
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
Introduction Why Fibre Channel? Fibre Channel layers and related components Fibre Channel topologies, terminology and addresses Expected Fibre Channel behaviors Summary
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
Objectives summary: •Introduction objectives include setting expectations, sharing best practices and providing links to additional resources. •Why Fibre Channel? objectives include reasons for existence of the FC protocol; FC markets and components related to FC SANs. •FC layers and related component objectives include discussions of the FC layers and components/ knowledge related to each layer. For example, FC-0 includes feeds and speeds therefore distance, cable, GBIC/SFP information is incorporated. FC-1 incorporates ordered sets and link control information and FC-2 incorporates packaging: exchanges, sequences & frames; COS and flow control. •FC topologies, terminology and address objectives include a discussion of FC topologies, terminology, and FC addressing: WWN’s; Fibre Channel addresses called port identifiers (PIDs) and Well-Known addresses. •Expected FC behavior objectives include FC communication methodology, services and expected interactive behaviors. •Summary objectives include a review of FC theory information presented during this course.
1_FC101_2003 4 . Incorporated.Best Practices See “Course Navigation Instructions” to learn how to move around presentations – please also notice and implement “best sound” recommendations Download and print *. download and/or print additional presentation resources as time and interest permit Set your own start and stop times but be consistent until course material is completed .pdfs of PowerPoint presentations – use these to take notes as you listen Download and print course resources such as the FC Recommended Reading List and SAN Glossary files Optionally view. Revision0.please schedule regular breaks while taking this online course ☺ Most lectures are recorded in small time blocks – you should be able to complete each in less than one hour © 2003 Brocade Communications Systems.
The 10% discount applies to the listed price only and may not be used in conjunction with other discounts at non-Brocade facilities •Industry Recognition as a Brocade Certified Fabric Professional.Brocade Certifications Brocade Certified Fabric Professional (BCFP) 5 This level of certification indicates you have mastered all of the basics of the Brocade SilkWorm switch and are knowledgeable in Fibre Channel theory. or a Brocade Certified SAN Manager •A skill set that could translate into a better position in your organization •An additional differentiator that can be used by potential employers •Access to the certification logo and a set of usage guidelines •A certificate of completion of the program and achievement award Employer Benefits •A direct return on the investment in your training •A watermark for determining future training needs •Improved customer support and satisfaction •A greater credibility with the customer base .brocade. Revision0. Benefits •A 10% discount off the listed price on all future Brocade training classes delivered at a Brocade facility (excludes non-Brocade facilities). Go to www.com and follow education links to certification for more information. Brocade Certified SAN Designer (BCSD) This level of certification indicates that you have mastered the concepts and intricacies of building a SAN from the basic components through the integration of industry applications and state-of-the-art storage components. Brocade Certified SAN Manager (BCSM) This level of certification indicates that you have a detailed understanding of administering Brocade SilkWorm switches and managing aspects of a SAN.1_FC101_2003 The BSCD exam can be passed with SAN experience and extensive knowledge of Brocade sponsored books and white papers. © 2003 Brocade Communications Systems. Incorporated.
It also includes this free WBT FC Theory fundamentals course. . “Baseline Knowledge” refers to previous experience with SANs and/or equivalent work experience associated with SCSI storage and Local Area Networks (LAN). or Maintenance Services OR Choose the BCSM path if you are part of Post-Sales Support.com Brocade Connect tab . Additional information. Maintenance. Brocade Fabric Watch and SNMP. It combines both SAN 320 and SFO 200 information needed to pass BSCM certification test. Additional course information can be found at www. a solid understanding of the Brocade Advanced Fabric Services software that are part of the Fabric OS that runs on the SilkWorm 12000. AFS 160 is an ONLINE course designed to enable students to identify and understand the following Brocade SilkWorm 12000 switch information: the key hardware and software components.brocade. CSM 260 is an accelerated class designed to help students pass the BSCM exam. AFS 156 is an ONLINE course that enables familiarity with Brocade Advanced Performance Monitor.Brocade Connect registration requires a Brocade switch serial number Brief course descriptions: AFS 200 provides the student with a thorough understanding of the Brocade SilkWorm family of Fibre Channel switches. Revision0. AFS 300 focuses on techniques needed by the second level and higher support engineer and lead Storage Area Network (SAN) administrator to troubleshoot Brocade SANs. Incorporated. In the slide above. Repair. or a SAN Manager BCFP is considered a prerequisite OR AFS200 CFP260 AFS300 SAN320 AFS160 (Web) AFS212 AFS156 (Web) CSM260 SFO200 SFO101 (Web) © 2003 Brocade Communications Systems. AFS 212 provides students with knowledge and experience with the Brocade SilkWorm 12000 SFO 200 provides students with the technical skills required to plan and implement security in a Brocade 1gb and/or 2gb Fabrics.brocade. basic deployment tasks. SFO 101 is an ONLINE web course that provides students with an overview of Brocade Secure Fabric OS.1_FC101_2003 including white papers helpful to pass BCSD exam. SAN 320 provides students with knowledge and experience about various ways to manage Brocade SANs including Brocade Fabric Manager.com education.Brocade Certification Roadmap Baseline Knowledge including FC 101 6 Choose the BCFP path if you are part of Post-Sales Support. CFP 260 is an accelerated class that combines both AFS 200 and AFS 300 certification specific information. can be found at www.
1_FC101_2003 The exam cost is $150 US for each attempt. . All examinees are required to accept a non-disclosure agreement. They operate over 2800 testing centers worldwide. No student may take the exam more than 2 times in a two week period. Incorporated.com/brocade Call 866-361-5817 toll free in North America Visit http://www.com/contact/brocade_numbers. This agreement means the examinee will not discuss or disclose any of the questions or exam contents. To register for an exam or locate a testing center nearest to you: Visit http://www. Failure to comply with agreement may result in forfeiture of certification status and benefits. VUE accepts many of the major world currencies.Registering For Brocade Certification Exams 7 Virtual University Enterprises (VUE) is our chosen test vendor.html for other contact numbers worldwide (some locations may not have toll free numbers) © 2003 Brocade Communications Systems.vue.vue. Revision0.
com resources for this introductory module include: Course Catalog follow education link to Course Catalog for course descriptions and schedules Certification follow education link to Certification for additional certification and test information Brocade Connect Additional information. can be found from Brocade Connect tab Brocade Connect registration requires a Brocade switch serial number © 2003 Brocade Communications Systems.brocade. Revision0.1_FC101_2003 8 Brocade Connect will ask you to provide a Brocade switch serial number when registering.Additional Information General course resources for all presentation can be found under syllabus in “Fibre Channel Theory Fundamentals Resource & Reference Material” section – general course resources include: Fibre Channel Recommended Reading & Resource List SAN Glossary Most presentations in this course will have an “Additional Information” slide at the end with a list of related resources Additional www. including white papers helpful to pass BCSD exam. . Incorporated.
. Incorporated.Brocade Product Training Why Fibre Channel? 1 © 2003 Brocade Communications Systems.1_FC101_2003 In this presentation we will investigate and exemplify Fibre Channel’s purpose and role in today's IT infrastructure. Revision0.
attendees should be able to: Identify reasons the FC protocol exists Identify benefits of FC SANs Identify some components related to FC SANs 2 © 2003 Brocade Communications Systems.Objectives After completing this module. Revision0.1_FC101_2003 . Incorporated.
Revision0.Topics Identify reasons and markets for FC protocol Storage access history and SAN introduction What is Fibre Channel? Why FC SANs? What is a FC SAN? 3 Identify some components related to FC SANs Embedded throughout this presentation we will discuss FC standards and reasons for the success of FC SANs © 2003 Brocade Communications Systems.1_FC101_2003 . Incorporated.
high levels of reliability could be ensured during data transfers between servers and storage. server-storage connections traditionally avoided networks as providing insufficient levels of confidence. Storage-server connections must have high levels of reliability – if there are any glitches in a server-storage transfer. It is an open standard that has been enhanced over the years to support increases in device speed and functionality.1_FC101_2003 The primary external interface in the early days of external storage was the Small Computer System Interface (SCSI). For this reason. By providing a dedicated physical channel.Storage Access – History Attaching storage to non-mainframe servers during the 1970s and 1980s was straightforward: Storage was directly attached to the server Networks were avoided. . a bus architecture with dedicated parallel cabling between servers and storage devices. Incorporated. Revision0. a parallel interface with a limited number of devices was used Result: A high-speed channel from server to storage 4 © 2003 Brocade Communications Systems. to ensure best performance and reliability To enhance performance. valuable data is compromised permanently.
Storage Access – Storage Area Networks (SANs) Over time. • The various SCSI committees tried to keep up with the exploding storage market. the number of servers and storage devices that intercommunicate has risen rapidly. The SCSI driver is usually implemented to be more efficient in interacting with an operating system than the IP stack. OCR.1_FC101_2003 We can now see that several factors contributed to the rise of Storage Area Networks: • For a variety of reasons (business mergers. Revision0.) have seen more rapid improvements in performance than SCSI-style parallel buses. Incorporated. explosive data growth). storage-server connection requirements have changed Network-level flexibility with channel-like performance and reliability Leverage SCSI command set over emerging serial interfaces Extend over a much larger geographic area These are the origins of the Storage Area Network 5 Storage Area Network © 2003 Brocade Communications Systems. introducing new technologies. • The flexibility required for server-storage access has reached network-like levels – but with a need for channel-like reliability and performance. . etc. and had success in maintaining a rich set of device commands. • Newer serial-based technologies (Ethernet. and makes it more well-suited to handling block data transfers.
our focus . with 10 Gbit/sec (1 GBytes/sec) coming.SAN Technology – Fibre Channel The dominant technology used in modern Storage Area Networks is Fibre Channel An open standard (IEEE T11 committee) Designed to emulate channels large block transfer behavior while extending distance and allowing many-to-many connectivity Connectivity: Thousands of devices per fabric (network) Performance: Current speeds: 1 and 2 Gbit/sec (100 and 200 MBytes/sec).1_FC101_2003 The above is a brief overview of the Fibre Channel protocol. Revision0. Incorporated.2 Gbit/sec Initiator arbitrates for access before transmitting (ensures channel-like access to target) All SCSI commands and user data is sent over 2112 byte Fibre Channel payload frames 6 © 2003 Brocade Communications Systems. . More details about Fibre Channel are available in Chapter 2 of Building SANs with Brocade Fabric Switches.
Point-to-Point.1_FC101_2003 Fibre Channel Highlights: A standard: AN ANSI standard providing flexible serial data transport at long distances for Storage Area and System Area Networks . 10 Gb/s speeds Low latency: Less than 2 micro second latency input port to output port of FC switch Long distance: Up to 10KM distance (longer with extenders). 4. Now an ISO/IEC Standard High performance and speed: Hardware based transport mechanism for high performance. Fibre Channel allows a theoretical 16M devices to be connected to one Fabric -Support for multiple physical media types . 1. IP.What is Fibre Channel? A standard High performance and speed Low latency Long distance Robust data integrity Large connectivity 7 © 2003 Brocade Communications Systems. Revision0. VIA. can be extended non-natively over ATMs up to 3000 km Robust data integrity: Uses IBM’s 8B/10B encoding scheme for robust integrity plus FC has a bit error rate (BER) of 10-12 . one bit was in error Large connectivity: -Per the standard. Switched. Ficon.ratified as ANSI standard in 1994. Loop and mixed topologies -Heterogeneous interconnect scheme for computing and peripheral devices .000 bits transmitted.Copper. and mixed protocols -Support for multiple topologies . Optical Fibre (multimode and Single mode) and Mixed media -Support for multiple protocols .000. etc. out of 10.SCSI. Incorporated.a transmission might have a BER of 10-12 means that. 2.
e. SCSI). Ficon. Fibre Channel allows for a heterogeneous set of devices to participate. such as SCSI.Fibre Channel . IP. Manageability of the SAN can be done by “typical” networking management applications. I. BB. HP Openview.Hybrid Transport System - 8 © 2003 Brocade Communications Systems. This means that hosts and applications see the disk devices as locally attached storage.g. And. . Revision0. It also incorporates the best of the networking world as Fibre Channel allows multiple protocol support. Incorporated.1_FC101_2003 Fibre Channel combines the best of both worlds: It is a “channel” transport that shares many of the characteristics of an I/O bus (e. and others.
Finally. long distance large network) Here is a formal definition for SAN from the Storage Network Industry Association (SNIA): “A network whose primary purpose is the transfer of data between computer systems and storage elements and among storage elements. user workstations and storage systems use different network protocols. A SAN consists of a communication infrastructure. For one thing. it would be asking too much to also provide ongoing access to storage systems. the storage units can be secured separately from the servers and totally apart from the user network. which provides physical connections. LAN hardware and operating systems are geared toward user traffic—they are designed for a fast user response to messaging requests. different tools are physically insecure at the desktop and potentially vulnerable at the server seldom have spare capacity for storage networking are tuned to favor short. With a SAN. continuous data transfers While Local Area Networks (LANs) may do a good job of supporting user access to servers. performance. they are less than ideal for providing servers with access to storage systems. The intermittent flurries of user broadcasts can be disruptive to bulk data transfers. (Note: WAN = Wide Area Network. it should be noted that user networks frequently employ broadcasts to coordinate access activities. “bursty” user transmissions versus large.Fibre Channel = Foundation of SAN Fabric A Networking Model for Storage Clients WAN 9 LAN Servers Fabric “Fabric” is a well-designed NETWORK of highly intelligent Fibre Channel switches which provides enterprise-class scalability. 2000 . Revision0. user networks have to go to where the users are and often this means that the servers may also be located all over the enterprise. manageability and availability. Better to move all the storage traffic to the SAN and give the LAN a room to breath. they are needlessly included in such broadcasts.”--SNIA Technical Dictionary.1_FC101_2003 Why give storage its own network—Fabric? Answer: A good LAN does not make a good SAN! • • • • LANs LANs LANs LANs use different protocols. Incorporated. By definition. copyright Storage Networking Industry Association. If storage devices are attached to the main network. Storage Area Network Storage Subsystems © 2003 Brocade Communications Systems. For them. storage elements. Most enterprises are in an ongoing struggle to maintain adequate LAN performance in the face of the rapid increase in user utilization rates. Abbreviated SAN. which organizes the connections. and a management layer. and computer systems so that data transfer is secure and robust.
you need to throw away what you know and learn a new protocol/language again!” . Up to 256 Upper Layer Protocols (ULP) support In this book. NT. moving to 10Gb/sec. Note that the Ethernet 10Gb group and the Fibre Channel 10Gb group have a joint working group and both technologies will be released simultaneously. Some Fibre Channel vendors will skip the 4Gbps speed generation and will go directly to 10Gbps.Benefits of Fibre Channel A new multi-purpose network infrastructure for connecting open system storages. video and clustering protocols as well. SCSI. network. AIX. Today. Incorporated. Networking technologies are synergistic. IP protocols.1_FC101_2003 10 The high-speed. ATM (Asynchronous Transfer Mode) is moving towards 10 Gbps speed paradise. Novell. Please note that the Fibre Channel is not a SAN only technology. etc. Solaris. Long Distance Fibre Channel is 10 kilometers by the standard specification.000 kilometers using ATM as a WAN transport) without breaking the Fibre Channel standard. networks. LINUX. The last thing an IT professional want is —“sorry. etc. Furthermore. as they do not need to learn a new storage or network command set. videos and cluster servers Provides a general hardware transport vehicle for Upper Level Protocols (e. low-delay connections offered by Fibre Channel makes it ideal for a variety of dataintensive applications. full duplex dedicated connection. moving to 10Gb/sec and beyond Congestion free transmission Up to 10 kilometers (plus extensions for supporting thousand kilometers distances. Highlight in some of the Fibre Channel features: High Speed Currently at 2Gb/sec. excellent deployment for Disaster Recovery) Advanced follow control system to guarantee in-order delivery Heterogeneous systems support (e. Revision0. The Fibre Channel technology has been used for networking in the movie and TV companies for the postproduction of moving the video imaging between servers and editing stations. we will focus on SCSI and TCP/IP protocols support only. But do aware that Fibre Channel has the capability to support many other storage. not competing against each other.) Adopts legacy environments and applications © 2003 Brocade Communications Systems.) Not Yet-Another-New-Protocol High Speed: 2 Gigabit/sec rate today (200MB). This makes Fibre Channel easier for IT professionals to understand and support. some Fibre Channel vendors have found solutions to implement long distance SAN (Up to 3.g.g.
1_FC101_2003 A Storage Area Network (SAN) is an enabling Infrastructure that provides network class of benefits for IT data centers Data can become a unified. “virtual”.What is a FC SAN? Open Systems Model for Networked Storage Enhanced Storage Management Flexibility to add or reconfigure storage as needed without downtime Independent Scaling of CPU and Storage capacity De-couples servers and storage so that either can be scaled separately Easy Migration Current applications run without software changes Incremental deployment allows flexible adoption 11 © 2003 Brocade Communications Systems. Revision0. resource Legacy systems can be seamlessly integrated SANs provide the flexibility for deploying various enterprise IT applications using a single infrastructure: SAN Backup Storage Consolidation Remote Data Replication High Availability Fast Server Failover . Incorporated.
What’s inside a FC SAN? FC SAN Node N_Port F_Port F_Port N_Port 12 Node Node Port (N_Port) has no knowledge of the path. FC devices use FC protocols to connect and communicate. This relieves the N_Port of having local routing tables. In this picture we see connecting E_Ports or Expansion ports. The Node ports absorb only the Fabric service information they need to build device lists and communicate across the Fabric. they just know what they are connected to across the Fabric. called Fabric logins or FLOGIs and query methodologies represent Fabric services that are essentially hidden from end ports . They do not know or care how connections are routed. This translates to easy Network Operations (Fabric Services) • Fabric hierarchical structure • Logical management • Internal switching function • Switch communication • Routing path selection E_Port Fabric Port (F_Port) is an N_Ports connection into a FC SAN to connect and manage E_Port Switch Internal structure and operations are not visible to the N_Ports © 2003 Brocade Communications Systems. that occurs when a device connects to a Fabric port. Revision0. When switches connect to each other they exchange link parameters (ELP) letting them know what is attached at the other end. Incorporated. From the Fabric perspective an F_Port implies a N_Port is at the other end of the cable.the node ports do not need to keep track of all Fabric service information. This implementation means we can now offload many requirements from the N_Port CPUs. Fabric access. thus making things simpler. as well as its fairly rich set of network capabilities and services offering to the attached nodes. We also see an N_Port (Node Port) attached to a F_Port (Fabric Port). as the name implies. we’ll examine the internal operations of a switched Fabric. for example. The port that the device attaches will “become” the the type of communication portal attached devices needs. There is an initialization process. In this course. FC has a switch-switch protocol called Inter link services (ILS) with a rich set of commands that allow switches to exchange information. they reside in the Fabric.1_FC101_2003 Fibre Channel provides an interesting network scenario where network clients have very little idea about what is going on inside the network. These protocols often represent Fabric services and. . both in terms of its interaction with a various elements of the Fabric. Node ports need Fabric ports to communicate.
Fabric is a term used to describe a generic switching environment. It can consists one or more interconnected switches (domains) One Fibre Channel Switch = One Fabric Domain Maximum of 239 domains in a single Fabric Fabric communication is based on 24-bits address space partitioning Special Agent: • Principal Switch Special Delivery: • Class F Service
24 Bit Address Space Domain ID 8 Bits Area ID 8 Bits ALPA ID 8 Bits
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
In Fibre Channel, the Fabric is normally an entity that distributes address identifiers to the N_Ports. In general, the N_Ports need to be aware of how the Fabric manages address identifier allocation. A Domain is the highest logical construct in the hierarchy of Port Identifiers. Areas are the intermediate level logical construct and ALPAs are the lowest level logical construct in the hierarchy. To facilitate Fabric communication address management, a partitioning scheme has been developed. The 24-bit address is divided into three 8-bit fields
• The upper 8-bits are the Domain • The middle 8-bits are the Area • The lower 8-bits are the ALPA (Arbitrated Loop Physical Address)
The domain is used to identify a Fibre Channel Switch. When a frame is received, it is routed to the correct domain (Switch). Once the frame reaches the correct domain, it is routed to the correct area and finally, the frame is routed to the correct port. Fibre Channel has the concept of a “Principal Switch.” The function of the Principal Switch is to simplify the problem of determining precedence between Fibre Channel Switches without adding a separate external Fabric management software component, The Principal Switch facilitates the bring up of the Fabric, acts as controller of domains (ensures each Switch coming into a Fabric has a unique domain) and handles time services when available. Class F is the communication class of service used between switches. It is a special internal communication service within a multi-switch Fabric. The primary purpose for the Class F is Fabric management and operation.
FC SAN Components
Host Bus Adapter in Server
Note: Cables & GBIC (SFP) will be discussed in next section
FC SANs can also include other interconnecting devices like HUBs and Bridges
Fibre Channel Switch Storage Subsystem
FC SANs usually include some SAN management applications
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
A SAN is a mass storage infrastructure that frees up the LAN or WAN, while it operates faster and does FC SAN Fabrics are high-performance networks based on Fibre Channel, and dedicated to storage. They provide any-to-any connectivity for the resources in the SAN. Any server can potentially talk to any storage device, and the SAN Fabric also enables a communication between storage and SAN devices (switches, hubs, routers, bridges). SANs employ fiber optic and copper connections to create dedicated networks for servers and their storage systems. More on these later. •Servers/HBAs (Host bus Adapters) - HBA’s are similar to the Network Interface Card (NIC) that is used to connect devices to a LAN UNIX Windows Linux •Storage - Disks (RAID/JBOD) •RAID stands for Redundant Array of Independent Disks. These Arrays look like a a single disk volume to the server and they are fault-tolerant either through Mirroring or Parity-checking. They also typically have their own management software just for that RAID array. JBOD - Just a Bunch of Disks that usually plug into an enclosure that has the connection to the SAN. These disks have no protection against failure. •Tape •Interconnecting Devices Hubs/Switches Bridges/FC Extenders •Software - SAN Management Applications Telnet Front Panels/Serial Connections - depends on the device being managed WEB Browser Fabric Manager (FM) SNMP - i.e. HP Openview, CMNS, CA Unicenter, Adventnet … Application Programming Interface (API) 3rd party applications that use SNMP and/or API
Interconnecting Devices - Fibre Channel Switch
Core of the Network Intelligent
Multi-port connectivity (8, 16, 64, 128 or more ports) Support Fabric and legacy loop devices Fabric port types: F, E, FL 1 or 2 Gbit/sec speed (auto-sensing) FC header/protocol support Full duplex performance with cutthrough routing Embedded services Login
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
Fibre Channel Switches (named Switch Element in Fibre Channel terminology) are intelligent devices able to interconnect individual nodes, devices, and even other switch elements. At the physical layers, switch intelligence means that a switch is very much plug-and-play, it can detect whatever type of device is plugged in and, provided they have the proper GBIC/ SFP installed. The reason for small switch versus bigger switch approach is to enable pay-asyou-grow implementation model. Most IT organizations can start deploying a small SAN island using two 8-ports switches, once they felt comfortable with the new technology, they can buy bigger switches (for example, 16-ports). The 64-ports and 128-ports core switches are mainly used for connecting many SAN islands across an organization.
Fabric port types: F_Port: For direct connection E_Port: For switch connection FL_Port: For loop/hub connection FC header/protocol support: Hardware based cut through frame routing to keep latency small Link level flow control to prevent loss of frames Link level error detection/recovery for high application performance Small to large frame sizes to meet different application throughputs/latencies Embedded services: Name Service; Alias Service; Management Service and more. Login: Establishment of operating characteristics Automatic address assignment (24 bit wide)
Some leading Switches in today’s market offer the following features:
• • •
Integrated SNMP and MIB-compliant management for remote management (as well as Telnet) Configuration management tools and utilization monitoring (web-based graphical user interface)
Automated port isolation and device fail-over for fault-tolerance, along with N+1 hot-swappable components
port designator for direct fabric attached devices NL_Port -. storage) . Incorporated.expansion port.a Fabric port to which an N-port attaches FL_Port -. host.1_FC101_2003 Switch Ports: E_port -. Revision0.A Fabric Loop port to which a Loop attaches Device Ports: N_port -.device that is attached to the loop (ie.Interconnect Devices – Fibre Channel Switch Port Interfaces NL_Port FL_Port N_Port F_Port N F 1 node to 1 port N_Port NL FL Up to 126 AL_PA nodes to 1 port © 2003 Brocade Communications Systems. connects two switches to make a fabric F_port -.
and event tracking. SNMP. Incorporated. First is hardware additions to the hub. The Fibre Channel Hub is what connects devices together to create an Arbitrated Loop. Managed Hubs . They will generally have bypassing technology as long as the signaling thresholds are met. thus sharing the backplanes throughput. and/or port density. Telnet. There are two levels to this functionality. With this in mind. Hubs posses the ability to auto bypass ports to allow ease of connectivity. Managed Hubs also provide the ability to recognize ordered sets.. port density. As members of the Fabric/SAN. port type. Ports must be able to recover valid clocked info at the rate of 1. Revision0.These hubs introduce another level of intelligence for manageability.e. Basic functionality can now be managed via TCP/IP I.Hubs Hub 17 © 2003 Brocade Communications Systems. Every device on the loop must share that bandwidth and access.These hubs are usually used in small environments since they are simple. signaling processing. second is the software to run the new hardware. As long as vendors do not supercede Fibre Channel standards they will create their Hubs with different Management functionality. invalid transmission words. CRC error detection. Loops support a maximum of 126 devices and has a maximum bandwidth of 100MB per second on the whole loop. Only two devices can be communicating on the loop at any point in time. FC-AL-2 Fibre Channel standards. Web. The hub will tie circuits through each port. most active AL_Pas. Hubs are a way for the devices that support FC Loop ONLY to become part of the Fabric. They will usually provide simplistic LED functionality. loops are multiple independent networks with limited connectivity Unmanaged Hubs . joining the last ports Tx circuit to the first ports Rx circuit.1_FC101_2003 Hubs utilize arbitrated loop topology. topology mappings. loop status.0625 gigabaud. . low cost and posses an entry level interconnection scheme. Hubs follow FC-AL.Interconnect Devices . There is no addressing scheme between the Hub and the connected device. link errors.
a bridge can also be used to connect servers or workstations to the SAN in “initiator” mode. Removing the tape device from an application server and attaching it to the bridge allows for faster. FC-based tape libraries are not yet prevalent so bridges are used to bring SCSI-based tape libraries into Fibre Channel SANs.Interconnect devices .Bridges FC-SCSI Router/Bridge Maps SCSI devices to units of a single Arbitrated Loop Physical Address (AL_PA) Bridge Configurable mapping table SNMP management SCSI Only 18 © 2003 Brocade Communications Systems. . It also allows the tape device to be shared by other servers across the SAN versus it being a dedicated resource. more accurate backups. or to connect Fibre Channel links to devices without Fibre Channel ports. Revision0.1_FC101_2003 Fibre Channel-to-SCSI Bridges (also known as routers) allow the connection of non-Fibre Channel devices to the SAN. Bridges interface Fibre Channel and SCSI. Typically used to connect SCSI tape devices to the SAN. Incorporated. The need for these systems is declining with the development of SAN capable devices.
but these capabilities are available with host-based software. Depending on how we configure the array. JBOD stands for Just a Bunch of Disks and is a counterpart of RAID. They have little or no “buffering” (or cache memory) or an intelligent controller that enables advanced features. or Redundant Array of Inexpensive Disks. These can be used in combination to deliver the balance of performance and reliability that the user requires. JBOD represents the simplest and least expensive "raw storage" option. they are good candidates for sharing across a SAN. Incorporated. Generally used to store large amounts of data for backup purposes. The individual disks are arranged in a simple cabinet and are available to its servers as a group of independently accessible disks. Although we can certainly have a SAN without RAID. Because of the high capacity (and cost) of RAID storage systems. or parity facilities that RAID systems do.1_FC101_2003 Two terms often heard in discussions of SAN storage subsystem are RAID and JBOD. Tape storage would be a common method of saving information in the event of a drive failure. or parity protected (extra data written to identify errors). . parity checking or data striping. striped (interleaved across several drives). Tape helps you to store data sequentially on a magnetic tape cartridge. we can have the data mirrored (duplicate copies on separate drives). Since they have no inherent intelligence. there is no protection in the event of a drive failure. is a disk clustering technology that has been available on larger systems for many years. Revision0.Heterogeneous Attachment Storage RAID – Redundant Array of Independent Disks JBOD – Just a Bunch of Disks Tape – Primary use for Backup and Recovery 19 © 2003 Brocade Communications Systems. RAID. JBOD has a limited growth capacity and it usually scales to less than 1 terabyte (TB) per cabinet. these two technologies are often used hand in hand. but don’t include the mirroring. It is a collection of disks that share a common connection to the server. striping.
1_FC101_2003 Storage Devices Raid Controllers act as the interface between the actual disks and the fabric. port initialization and communication management. They handle all tasks necessary to present the disks in the RAID array to the Fabric. including LUN masking. Incorporated. .Heterogeneous Attachment Cont. Revision0. cache. Storage Intelligence in the controllers 20 © 2003 Brocade Communications Systems.
1_FC101_2003 Every device connected to a SAN requires a Fibre Channel interface or adapter board.g. Incorporated. Fibre Channel Host Bus Adapters (HBAs) are available for a variety of bus types including PCI and Sbus. Revision0.Fibre Channel Host Bus Adapters (HBAs) Provides an Interface between the Servers or Workstations Internal Bus (e. Some leading adapters on the market today provide the following features: • Plug-and-play flexibility and copper/optical connector support • SNMP (Simple Network Management Protocol) and MIB (Management Information Base) support • Support for both Arbitrated Loop and Switched Fabric topologies . PCI or SBUS) and Fibre Channel network HBA software driver provides the storage information required by the Operating System 21 Handles I/O and Control requests Copper/Optical media support (may be dual port cards) © 2003 Brocade Communications Systems.
Heterogeneous Attachment Cont. . .1_FC101_2003 Major responsibilities of a Host Bus Adapter (HBA): • • • • • • • • • • Framing packets Provide physical addressing Link level error checking Sequence level error checking Manage flow control Provide linking of I/O request and packet Handle chaining of multiple requests Manage many concurrent I/O Analyze and manage data Run Storage virtualization Note: When compared with normal Ethernet network card. Revision0. Incorporated. Ethernet card only handles the first three operations and the rest of the functionality require server CPU handling.Intelligence in the Host – HBAs 22 HBA card = Node Connectors = Ports © 2003 Brocade Communications Systems.
. Incorporated. is the interface between the physical card and entry point into the Kernel OS SCSI drivers. A SCSI adaptor will take SCSI commands and package them into the parallel format needed on the SCSI data bus where the disk and tape devices are sitting. Note: SCSI Command Descriptor Blocks (CDBs) are a specific unit of work to be acted upon by a SCSI Initiator or Target.HBAs Yesterday SCSI Adapter and Driver Netscape Payroll Inventory 23 Application Layer System Call Interface Layer File Subsystem Layer Kernel Operating System NTFS FAT UFS Disk Driver Tape DriverCD-ROMDriver You can load this driver SCSI Driver SCSI Adapter SCSI Adapter Driver Driver INTERNAL I/O BUS INTERNAL I/O BUS SCSI SCSI Adapter SCSI Adapter SCSI CARD CARD © 2003 Brocade Communications Systems. Revision0. tape or CD ROM device drivers are responsible for handling specific request (i. •The Kernel OS System call interface and file subsystem layers process application layer communication. Opens/Reads/Writes/Closes) from one or more applications – these drivers put the necessary device specific handlers on the communication going down the chain. •When we look at SCSI communication in this picture. Payroll. . •The SCSI adaptor drivers that you load or let the server operating system load for you. •The disk. and inventory applications.1_FC101_2003 In this slide we can see the different layers that go into servicing a particular SCSI command from the Application layer down to the SCSI Adaptor card. How does this work? •The application layer depicts Netscape. the Kernel OS SCSI driver is responsible for accepting generic I/O requests from upper layers in the Operating Systems Kernel and converting them to the appropriate Device Specific SCSI Command Descriptor Blocks.e. This view is an illustration of a parallel SCSI system.
A FC HBA communication process today is very similar: •The application layer still handles Netscape. tape or CD ROM device drivers are responsible for handling specific request (i. the Kernel OS SCSI driver is still responsible for accepting generic I/O requests from upper layers in the Operating Systems Kernel and converting them to the appropriate Device Specific SCSI Command Descriptor Blocks. By the way. These adapters will take the SCSI commands and package them into sequences of frames.1_FC101_2003 Here is a view of the different layers that go into servicing a SCSI command. register statistical data. Note: SCSI Command Descriptor Blocks (CDBs) are a specific unit of work to be acted upon by a SCSI Initiator or Target. this level also includes backup. All steps necessary to send these frames out the link through the Fabric to their destination ports. called HBA drivers. . represent the interface between the physical card and entry point into the Kernel OS SCSI drivers. Payroll. Opens/Reads/Writes/Closes) from one or more applications – these drivers put the necessary device specific handlers on the communication going down the chain. serialize. •When we look at SCSI communication (most prevalent FC protocol). Revision0.HBAs Today SCSI/Fibre Channel Adapter and Driver Netscape Payroll Inventory 24 Application Layer System Call Interface Layer File Subsystem Layer Kernel Operating System NTFS FAT UFS New host SCSI driver to support Fibre Channel Disk Driver Tape DriverCD-ROMDriver You typically load this driver New and Improved SCSI Driver FC Adapter Driver FCAdapter Driver INTERNAL I/O BUS Fibre Channel INTERNAL I/O BUS FC HBA CARD FC HBA CARD Fibre Channel © 2003 Brocade Communications Systems. •The disk. •The FC adaptor or HBA drivers that you load or let the server operating system load for you. multipathing. The difference between the previous slide and this one is the fact that the device drivers are using Serial SCSI-3 commands that enhance error recovery and device sharing. encode & decode FC frames.. •The Kernel OS System call interface and file subsystem layers still process application layer communication. append FC addressing information and flow control parameters. These new and improved SCSI drivers also understand how to communicate with FC Adaptors. Incorporated. HBA components also assign counters.e. clustering and volume management applications. They handle what we call FC Layer 2 processes. plus the block level device or device Kernel OS is communicating with does not have to be on the same physical parallel bus for the host to access it. and inventory applications.
hubs. Revision0. storage (storage controllers) and hosts (HBA’s) 25 © 2003 Brocade Communications Systems. Incorporated. congestion free communication between heterogeneous devices with legacy support SAN components include interconnect devices (switches.Summary FC combines the best of Channel and Network protocols FC SANs are a new multi-protocol network that provide high speed. bridges).1_FC101_2003 .
find SAN ED 101 link “SAN Fabric Foundation”.Additional Information 26 Use the resources page Internet Links. go to chapter 1 (Introduction to SAN Concepts and Benefits) for additional information related to this presentation © 2003 Brocade Communications Systems. Revision0. Incorporated.1_FC101_2003 .
Revision0.1_FC101_2003 . Incorporated.Brocade Product Training Fibre Channel Layers and Related Components 1 © 2003 Brocade Communications Systems.
attendees should be able to: ! Identify ! Link ! Link 2 FC protocol layers components to FC layers functions to FC layers © 2003 Brocade Communications Systems.Course Objectives After completing this module. Revision0.1_FC101_2003 . Incorporated.
! Ethernet ! FC
" " " " "
and FC layer relationships
FC 0 – Speeds and Feeds FC 1 – 8b/10b Encoding FC 2 – Data Delivery FC 3 – Common Services FC 4 – Upper Level Protocols (ULPs)
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
Ethernet and FC Layer Comparison
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
A look at network protocol layers Ethernet is the most widely used LAN technology today. Originally conceived and developed by Xerox Corporation, it is specified in the IEEE 802.3 standard. Since its inception, initial speeds of 10 Mbit/sec have evolved into Fast Ethernet (100BASET), providing transmission speeds up to 100 Mbit/sec. Gigabit Ethernet provides an even higher level of backbone support at 1000 Mbit/sec primarily on fiber optic cable. In the diagram, we illustrate how Ethernet utilizes the OSI model. Although it looks like Ethernet provides solutions for layers one through four of the OSI model, in fact, Ethernet by itself only covers the two bottom layers. Ethernet is, in general, accessed through TCP/IP (or UDP/IP) protocol stack, where the SAN is accessed through a simple SCSI protocol stack with fewer overheads on the server processor. Most Ethernet adapters work at the packet level. All higher-level segmentation and reassembly into IP datagrams, or TCP-level sockets are software driven and require server CPU intervention. Be aware that TCP/IP often drops packets of data when the network becomes congested. When this happens, the packet must then be retransmitted using more bandwidth. Footnote 1: FC-2 has elements of OSI layer four, so it is not a perfect one-to-one correlation between the OSI and the Fibre Channel layers. Overall, it is a modular architecture, 5 layers (FC-0 to FC-4) and each level does not define physical or programming interfaces between the levels.
On Common Fibre Channel Transport
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
FC - 0 and 1 layers specify physical and data link functions needed to physically send from one port to another. FC – 0 specifications include information about media connections and cables, it is sometimes referred to at the feeds and speeds layer. 10 Mbaud speed, although not depicted, is also in development. FC – 1 layer contains specifications for 8b/10b encoding, ordered set and link control communication functions. FC – 2 specify content and structure of information along with how to control and manage information delivery. This layer contains basic rules needed for sending data across network. This includes: (1) how to divide the data into smaller frames, (2) how much data should be sent at one time before sending more (flow control), and (3) where the frame should go. It also includes Classes of Services, which define different implementations that can be selected depending on the application. FC – 3 defines advanced features such as striping (to transmit one data unit across multiple links) and multicast (to transmit a single transmission to multiple destinations) and hunt group (mapping multiple ports to a single node). So, while FC-2 level concerns itself with the definition of functions with a single port. The FC-3 level deals with functions that span multiple ports. There are parallel link service functions that many think reside at this layer because they logically fit. These link service functions utilize all FC-2 and lower services like an ULP but are not ULPs neither but neither do they reside at FC-3. I think of them as performing FC-4 like functions in a category parallel to FC-3. FC – 4 provides mapping of Fibre Channel capabilities to pre-existing protocols, such as IP or SCSI, or ATM, etc.
One big purpose of Fibre Channel is to have the selected protocol operate over various physical media and data rates. Single Mode optical fiber is used between buildings or sites because of its long distance data transmission capabilities. allowing existing cable plants and a number of different technologies to be used to meet a wide variety of system requirements. Multi Mode Fiber (MMF) is used within a building between nodes and switches or between floors. specifies the physical link.FC-0 Layer: Media Connectors and Cables ! 6 ! Serial interface (separate one-bit transmit and receive lines) Media types: " Cables # Fiber Optic Cables • Single Mode • Multi Mode " Connectors Fiber and Electrical Cable specifications ! FC-0 and FC-1 combined are referred to as FC signaling interface © 2003 Brocade Communications Systems. GBIC (Gigabit Interface Connector) and MIA (Media Interface Adapter). This approach ensures maximum flexibility. Common cables are either copper or optics. Revision0. Serial data received from FC-1 is converted to a signal type associated with the transmission media and sent out transmission port. These connectors include SFP (Small Form Pluggable). Optical connectors are used for interconnection between devices such as nodes. Incorporated. fibre channel switches and hubs. Transmission continues as long as link remains in OFC Active state also called Open-Fiber State.1_FC101_2003 FC-0: Physical Interface The lowest level. FC-0. .
Revision0. 50 micron or 62. The optical fiber is very thin and could easily become damaged. a human hair is about 75 micron thick. and fiber coating wrapped around the cladding. one for transmitting information. For comparison. surrounded by cladding (to keep the light in the core). Using a switched fabric a nodes TX is attached to the Brocade ports RX and the Brocade TX is connected to the nodes RX. Each side TX connects to the other sides RX consisting of a single point-to-point connection. the other for fiber for receiving information. Incorporated. . The fiber core can be either 9 micron.5 micron.FC-0 Fiber Optic Cables Glass Core Glass Cladding Coating Glass Core Glass Cladding Coating 7 © 2003 Brocade Communications Systems. A ferrule Optical FC links consists of two fibers.1_FC101_2003 Fibre Channel links are driven optically or electronically. The optical and electrical links can be combined in a single system when there is a Fabric or other media-type converter available. The active part of the optical cable is constructed out of the core (the optical conduit).
intermediate links. Multi-mode propagation refers to the fact that the pulses of light can travel down the optical wave guide taking different reflective paths. Following are the prefix and measurement for the small numbers: • Deci-meter (dm) = 10-1 of a meter • Centi-meter (cm) = 10-2 of a meter • Milli-meter (mm) = 10-3 (One thousandth) of a meter • Micro-meter (µm) = 10-6 (One millionth) of a meter (µ . long distance links.5 micron cores. By eliminating the chromatic dispersion (by utilizing monochromatic laser light sources).5 µm) 9 micron Single Mode Fiber 1300 nm Longwave Laser Cladding (125 µm) Core (9 µm) © 2003 Brocade Communications Systems. The light travels along the same path since the diameter of the core is reduced (9 um) to such a degree it constrains the light. The distance under this propagation method is significantly reduced. Single-mode is used for high speed. Single-mode fiber (SMF) is used for long wavelength. and a cladding of 125 microns.1_FC101_2003 Fiber types are available in two types: single mode or multimode. Incorporated. Multimode offers a core of 50 or 62.5 micron Multi Mode Fiber 8 780 nm Shortwave Lasers 850 nm Shortwave Lasers Cladding (125 µm) Core (50/62.Greek word) • Nano-meter (nm) = 10-9 (One billionth) of a meter • Pico-meter (pm) = 10-12 (One trillionth) of a meter • Femto-meter (fm) = 10-15 of a meter • Atto-meter (am) = 10-18 of a meter . The smaller the glass core the greater the distance. the idea is to reduce the core size until the possibility of modal dispersion is reduced to such a degree that only one mode of photon flight is exhibited. The core is much larger (50/62.5 um) allowing for multiple modes and paths the light can follow. In Single-mode fiber. It comes in 50 & 62. Revision0.FC-0 Optical Cable Links Supported 50/62. This propagation method is referred as Modal Dispersion. with a cladding of 125 microns. while Multimode is utilized in lower cost. 9 micron goes 2m – 10km distances. It comes with a 9 micron core.5 micron in shorter distances and have a smaller cost base. (MMF) is used for short wavelength. great distance and bandwidths become possible in single-mode fiber.
and ease connectivity. The 1Gbit/sec switches typically use an SC connector.5 m – 500 m . .5 micron Multi-Mode Fibre (Short Distance) Electrical (Copper) © 2003 Brocade Communications Systems. providing for a screw-less. Each optical patch in a fiber is called a mode. The 2Gbit/sec switches use an LC connector. The HSSDC is a new connector designed for Fibre Channel. Both cables have white tips.5 m – 175 m . easy to plug/unplug connection. that stick into the SFP (GBIC).5 m – 90 m 0 m – 33 m Distance (m=meter) $ 10 km 9 micron Single-Mode Fibre (Long Distance) 50 micron Multi-Mode Fibre (Short Distance) 62. Multi-mode is by far the most common today. They are available as Coaxial cable and Twisted Pair with connectors in DB-9 or High Speed Serial Data Connectors (HSSDCs). Fiber optic cables are more expensive but they give reliability. Incorporated. distance (up to 10 kilometers). Each has its own advantages and disadvantages. typically providing 500 meters distance at 1Gbit/sec. 2Gbit/sec switches typically use the LC connectors. called ferrules.FC-0 Cables and Connector Distances ! ! 9 Fibre Channel does not necessarily mean just fiber cable Optical and Copper cables are the most common Media Type Speed 100 MB/s 200 MB/s 100 MB/s 200 MB/s 100 MB/s 200 MB/s 100 MB/s . Copper cables are less expensive but suffer from more reliability problems and limited distance.5 m – 300 m . Revision0.1_FC101_2003 Fibre Channel can be implemented using either fiber optic or copper cabling. Fiber pairs come with easyto-use push-pull SC (Sieman Connector) or LC (Lucent Connector) connectors at the cable ends. They come in two kinds: single-mode (yielding greater distance) and multi-mode.
2Gbit/sec cables use LC connectors to SFPs while 1Gbit/sec fiber cables use SC connectors to GBICs © 2003 Brocade Communications Systems. features a ceramic ferrule and looks like a mini SC connector. This GBIC module supports 9 micron single-mode fiber optic cables up to 10 kilometers in length with a maximum of five splices.1_FC101_2003 The SC connector features a molded body and a push-pull locking system and was designed as a low cost alternative to the ST connector. a small-form factor connector.5 micron. Fiber optic cables 62. . Incorporated. are based on short-wavelength 850 nm lasers supporting 1 or 2 Gbit/sec link speeds. It is utilized in both Multimode and Single mode hi-bandwidth applications. with an SC or LC connector (respectively). multimode fiber optic cables 50/125 microns will go up to 500 meters in length @ 1 Gbit/sec and 300 meters in length @ 2 Gbit/sec. LWL Fiber Optic GBIC/ SFP Modules The LWL fiber optic GBIC/SFP modules. with SC or LC connector are based on long-wavelength 1300nm lasers supporting 1 or 2 Gbit/sec link speeds. The LC connector.5/125 microns will reach 175m @ 1 Gbit/sec and 90m @ 2 Gbit/sec.FC-0 Fiber Optical Cable Connections The two most common optical connections used in FC involve LC $ SFP’s and SC $ GBIC pairs LC (Lucent Connectors) plug into SFPs (Small Form Factor Pluggable) receptors 10 LC SC (Siemens Connectors) plug into GBICs (Gigabit Interface Connector) receptors SFP SC GBIC Typically . SWL Fiber Optic GBIC/SFP Modules The SWL fiber optic GBIC/SFP modules. Revision0. This GBIC/SFP modules support 50/62.
with SC or LC connector are based on long-wavelength 1300nm lasers supporting 1 or 2 Gbit/sec link speeds. Incorporated. Revision0. Brocade supported GBIC’s / current vendors include: Agilent. thereby supporting ANSI X3. SWL Fiber Optic GBIC/SFP Modules The SWL fiber optic GBIC/SFP modules.230 FC-PH intracabinet requirements.Up to 100K © 2003 Brocade Communications Systems. The GBIC provides a female HSSDC connector. Finisar and IBM.5 micron. . Passive/Active Copper GBIC Module The Copper (Cu) GBIC module is based on the High-Speed Serial Data Connection (HSSDC) interface standards. Fiber optic cables 62. multimode fiber optic cables 50/125 microns will go up to 500 meters in length @ 1 Gbit/sec and 300 meters in length @ 2 Gbit/sec. Passive Copper cables up to 13 meters and active up to 33 meters have currently been qualified. Standard cables with HSSDC-to-DB9 male connectors are also available. with an SC or LC connector (respectively).5/125 microns will reach 175m @ 1 Gbit/sec and 90m @ 2 Gbit/sec. This GBIC module supports 9 micron single-mode fiber optic cables up to 10 kilometers in length with a maximum of five splices.Up to 13M (1Gbit/sec) Active Copper (Cu) . are based on short-wavelength 850 nm lasers supporting 1 or 2 Gbit/sec link speeds.FC-0 GBICs and SPF’s Copper GBIC Fiber SFP Fiber GBIC 11 Cable connection: ! ! ! ! ! LC Connector SC Connector Passive Copper (Cu) .1_FC101_2003 Check OEM Compatibility matrices for a list of supported GBICs and SFPs. LWL Fiber Optic GBIC/ SFP Modules The LWL fiber optic GBIC/SFP modules.Up to 33M (1Gbit/sec) Short wavelength (SWL) GBIC / SFP Long wavelength (LWL) GBIC / SFP Extended Fabrics . This GBIC/SFP modules support 50/62.
Revision0.5 are required MIA Media Interface Adapters (MIAs) are used to convert electrical signals to optical signals (DB9 to SC) SC Coupler LC Coupler © 2003 Brocade Communications Systems.3.Cables and Related Components ! 12 ! ! Types of Optical Cable(s) " LC to LC – SFP to SFP " LC to SC – SFP to GBIC " SC to SC – GBIC to GBIC Ethernet Port: RJ45 Serial Port: Straight 9-pin female-female D-sub cable " Pins 2.1_FC101_2003 . Incorporated.
8-bit/10-bit encoding scheme is selected for its superior transmission characteristics. If a loss of sync occurs a Loss-of-Sync procedure takes place (5 invalid transmission words $ receiver attempts to re-sync). FC1 layer encodes it into a ten bit code group (20% overhead). Syncing up summary --$ Serial data comes into a receiver port with 7 bit comma patterns at the front of each ordered set character. the coding provides a mechanism for detection of the transmission and reception errors. Note: 8-bit/10-bit scheme is used in IBM ESCON as well. Revision0.1_FC101_2003 FC-1: Byte Encoding This layer describes the 8-bit/10-bit transmission code that is used to provide balance of the transmitted bit stream. They also count to 40 to word sync. This will guarantee certain characteristics about the information that gets sent across the serial link. FC-1 level defines the method to encode data prior to transmission. Now frames can flow. Each octet is given a code group name according to the bit arrangement. Incorporated. This well-balanced code allows for low-cost component design and provides good transition density for easier clock recovery. As data is passed down in octet form. These ordered set characters act as frame delimiters.FC-1 Layer: 13 ! ! ! 8b/10B encode/ decode Link Control Ordered Sets © 2003 Brocade Communications Systems. In addition. . Receiver ports first bit sync (using these commas) and then count to 10 thus character syncing. primitive signals and primitive sequences.
VC_RDY.Loop Initialization LPB . Fill Word (Idle.5) is called a “special character” to indicate its usage as an ordered set. Primitive Signals are used to indicate events or actions and are normally transmitted once.Dxx.y.signal the events Undefined but valid ordered sets are treated as fill words (no explicit action in necessary. At least 3 primitive sequences must be received before the appropriate response can be generated. Data Words occur within the FC Frame content.Dxx.Dxx.y) Frame Delimiter SOF. LIP) and (LPB. ARB(F0). Dxx. CLS. LRR. EOF) – to delimit.1_FC101_2003 Transmission words can be classified into two categories: 1. or identify start and end of frames Primitive Signals– to indicate events at the sending port.End of Frame Primitive Signal Fill Word IDLE ARB Non-Fill Word R_RDY VC_RDY CLS OPN Primitive Sequence NOS . Primitive Sequences are used to indicate states or conditions and are normally transmitted continuously until something causes the current state to change.Offline LR . ARB(FF)– transmitted on a link whenever a port is operational and has no other specific information to send Non-Fill Word (R_RDY.y.y. LIP . Any character that starts with a “K” (ex: K28. MRK. ARB(X). Each transmission character in both the Ordered Sets and Data Words is 10 bits.Link Reset Resp. Revision0. SYNx.y. LPE) – to control an optional port bypass circuit (for loop). 2. Primitive Sequences (NOS. Incorporated. LR.Dxx. .y.Dxx. so transmission words are 40-bit words. OPN.y.y) Data Word (Dxx.y.Not Operational OLS . Words that begin with an encoded data character are called data words. OLS.Loop Port Enable © 2003 Brocade Communications Systems.Link Reset LRR .Start of Frame EOF.5.y. Ordered Sets (K28. Words that begin with a special character are called ordered sets.z).FC-1 Ordered Sets Fibre Channel uses transmission word-ordered sets (four transmission characters) to perform control and signaling functions: Transmission Word 14 Ordered Set (K28. Dxx.Dxx.y) include: Frame Delimiters (SOF.5.Loop Port Bypass LPE . DHD. primitive sequences and signals. Dxx. Ordered sets occur outside of the FC frame content and include frame delimiters.
Certain conditions encountered within the port may cause the port to exit the active state and perform one of the following primitive sequence protocols: Link Reset. reset or failure: " 15 Port State Machine (PSM) provides control signal specifications for Fabric connections Loop Port State Machine (LPSM) provides control signal specifications for arbitrated loop connections " Rx Rx Nx_Port A Tx Nx_Port B Tx © 2003 Brocade Communications Systems. or Link Failure. Revision0. Offline state. Incorporated. Active State (AC) 2. Link Failure State (LF) Active state (AC) – in the Active state.1_FC101_2003 There are four states of the Port State Machine (PSM) used for Fabric (G_Port) connections: 1. Link Reset state consists of: • LR1 (Link Reset Transmit) • LR2 (Link Reset Receive) • LR3 (Link Reset Response) Offline state consists of : • OL1 (Offline Transmit) • OL2 (Offline Receive) • OL3 (Wait for Offline) Link Failure state consists of: • LF1 (No Operational Receive) • LF2 (No Operational Transmit) . Link Reset State (LR) 3.FC-1 . Offline State (OLS) 4.Link Control Protocols Link control protocols use ordered sets for low-level link initialization and control $ link or arbitrated loop (AL) initialization. the port is able to transmit and receive frames and primitive signals.
Sequence numbers associated with the exchange will be used at the recipient to re-order the sequence segments. However. In other protocols. to the serializer (sfp/gbic) and transmitted to the receiver port where it is deserialized. Incorporated. When the receiving port moves the buffer to the next port. Exchange management is the mechanism that two fibre channel ports use to identify and assign an exchange ID number for a set of related information units. it is sent thru the encoder (8b/10b).1_FC101_2003 FC-2 defines the structure and organization of the information being delivered and how that delivery is controlled and managed. The receiving port sends to the transmitting port a credit to send another frame and decrements a credit from the credit value established during the login session (buffer to buffer credit). Within the exchange ID sequence management is used to number the sequence segments in the stream of data.FC-2 Layer: Framing Protocol / Flow Control 16 ! ! ! ! Exchange and Sequence Management Frame Structure Class of Service Flow Control " Buffer to Buffer " End to End © 2003 Brocade Communications Systems. this is commonly known as fragmentation and re-assembly. the debit is restored. When a N_port and a destination N_port communicate. . Buffer credits regulate the flow of frames into and out of the fabric. When a frame is ready for transmission. to re-assemble as a contiguous stream of data. Flow control is the process to deliver a frame. an end-to-end credit is established. decoded and stored in a receive buffer. When the entire stream of data will fit in a single frame (2112 bytes) a single exchange id is created and a sequence number is assigned. End-to-end credit is established between the pairs and is used to manage the flow of frames between a specific pair of N_ports and allows the receiving port to control which source N_ports are allowed to send frames to the receiver. when a stream of data will not fit into a single frame (2112 bytes). Frame Structure has a start-of-frame delimiter ordered set and ends with an end-of-frame delimiter set. Revision0. data is put into sequences of frames.
Incorporated. related sequences for a single higher-level operation. Sequence . discard a single sequence. A sequence is also the recovery boundary in Fibre Channel. Fibre Channel identifies the sequence in error and allows that sequence to be retransmitted. an operation may consist of several phases: a command to read some data." A frame is not a unit of transfer but sequence is and it always maps with the Upper Layer Protocol (ULP) command instruction. only one sequence direction can be active at a time. followed by the data. flowing in the same direction on the link (unidirectional from one N_Port to another N_Port). all the commands coming from a ULP are mapped into logical constructs called "information units. such as those required in an I/O operation. The sequence and exchange structures are general enough and contain tunable options concerning flow control and error recovery policy. such as video data). . and status is a separate sequence. then followed by the completion status of the operation. Sequences going in one direction can be streamed.1_FC101_2003 With Fibre Channel. Related information units. data. frame sizes are transparent to the software because of a logical construct called a "sequence. The N_Port that transmits a sequence is referred to as the "sequence initiator.A sequence is a set of one or more related data frames transmitted for a single operation.An exchange is composed of one or more non-concurrent. This just means that a 2nd sequence can be sent in that direction before waiting for delivery confirmation. Sequence. but they can form a single exchange. are mapped as a single exchange. For example." An Individual information unit is generally mapped to a sequence.0 & 1 2 3 4 3 Port 2 1 © 2003 Brocade Communications Systems. there are almost no limits on the size of transfers between applications.FC – 2 Exchange. Note: The four Exchange error policies are: (1) Abort. Whereas with Ethernet the software is sensitive to a maximum packet size that can be transmitted (1518 bytes). (2) Abort. Frame Management Command 1 ULPs FC-4 Mapping Single Exchange Multiple Sequences Frames CMD (Sequence) 17 Command 2 DATA IN (Sequence) Command 3 STATUS (Sequence) Information Unit Information Unit Information Unit CMD DATA IN STATUS CMD DATA IN STATUS 1 FC. Sequence initiative is handed over before sequences flow in the other direction." and the N_Port that receives the sequence is referred to as the "sequence recipient". Each phase of the command. Let’s see how it works: First. (3) Process with infinite buffering (this policy is a special design for specific transfers. With Fibre Channel. (Fibre Channel multiplexing support). although sequences for different Exchanges may be concurrently active. discard multiple sequences. Exchange . Within the single exchange. When an error is detected. (4) Discard multiple sequences with immediate retransmission. Revision0.
There are many different payload formats.24) tells which format to use. etc.3. •Type – identifies the protocol of the frame content for Data Frames (i. It could be a a wellknown address like the Name Server FFFFFC. The TYPE field (Word 2. . bit 31. •Sequence ID (SEQ_ID) –used to identify and track all of the frames within a sequence between a source and destination port pair. •Class specific Control Field (CS_CTL) – The control necessary for different classes of service. This field is always zero for classes 2 and 3 per the standards. If CS_CTL is something other than zero in a Brocade port log (a running log extracted from portions of the FC frame displayed with the portLogDump command). The header contains control and addressing information associated with the frame. Classes 1 and 4 use it. and F. Incorporated.Port identifier (PID) or 24 bit address of the source.1_FC101_2003 A frame has a header and may have a payload. •Data Field Control (DF_CTL) – this field indicates if any optional headers are present at the beginning of the data field of the frame. The maximum size is 2112 bytes. last frame of the Exchange. first frame of the Exchange. • Data Field/Payload – The standards limit the size. It could also be a a well-known address like the Name Server FFFFFC … more on this later.Port Identifier (PID) or 24 bit address of the recipient. optionally assigned by the responder to the Exchange. They define the type of frame and its content or function. Brocade switches currently only use Classes 2.e FC_CT. Field Definitions •Routing Control bits (R_CTL) are the first 8 bits of the header. •Destination_ID (D_ID).FC – 2 Fibre Channel Frame Format 18 FRAME S O F 4 HEADER PAYLOAD C E R O C F 4 4 24 Up to 2112 2148 Bytes © 2003 Brocade Communications Systems. •Sequence Count (SEQ_CNT) – used to indicate the sequential order of frame transmission within a sequence or multiple consecutive sequences within the same exchange. Optional headers are used for information that may be required by some applications or protocol mappings. The first 4 bits (Bit 31-28) identifies the frame type. The routing control INFO bit (bit 27-24) determines how to interpret the payload. The payload contains the information being transported by the frame on behalf of the higher level service or FC-4 upper level protocol. Revision0. based on the protocol. The 2nd four bits “INFO bit” (Bit 27-24) defines the contents of the frame or identify the function of the frame. This is a counter that increments as sequences of frames are transmitted. FCP. • Originator_ID (OX_ID) – Exchange ID assigned by the originator port • Responder_ID (RX_ID) . then it is a Brocade internal code called “IU_Status Values”. IPFC) •Frame Control (F_CTL) -This field contains miscellaneous control information regarding the frame such as who owns initiative.Responder_ID. •Source_ID (S_ID) .
deterministic latency. Bandwidth and latency handled by a Quality of Service parameter that is IOD guaranteed Class 2* – Class 3* – Class 4 – Class 6 – behaves like Class 1 with one $ many capabilities Class F* – connectionless: switch-to-switch. in-order delivery (IOD) A circuit-switching COS One $ One connectionless: end-to-end(EE_Credit)/ ack and buffer to buffer / receiver ready (BB_Credit/r_rdy)flow control connectionless: no ack. Additional Notes: Class 1 – Circuit-switching means that when Fabric receives Start of Frame connect Class One (SOFc1) signal it starts establishing a connecting to destination port. Intermix (allows class 2 and 3 frame transmission when class 1 not in use) . 19 There are six classes of Fibre Channel service: Class 1 – connection oriented: port-to-port. or IOD guarantees. Note 1 . Intermix Bandwidth Recovery (allows Fabric to hold class 1 frame one frame cycle and make up that frame cycle in idle cycles). 4 and 6 such as ack. with ack and BB_Credit ready © 2003 Brocade Communications Systems. 3 and F have no bandwidth.There are other link control signals associated with Classes 1. There are variations of COS 1: Exclusive connection. 2. . BB_Credit/r_rdy flow control only Errors are handled at higher level connection oriented with ack: Creates Virtual Circuits with dedicated fractional bandwidths. lcr (link credit reset). f_bsy (fabric busy). Note 2 .FC-2 Class of Service (COS) Associates a set of attributes with sequences and the frames within the sequence to provide different delivery characteristics. nty (destination port engaged in a class-1 connection). Incorporated. Class F – Based on a simplified Class 2 model – It is used to communicate Fabric-related traffic on inter-switch links (ISLs). guaranteed bandwidth. and other control signals.1_FC101_2003 The ability to differentiate Classes of Service characteristics will be needed when discussing Link and Flow control of frames The Classes of service with an asterisk (*) are the ones that Brocade switches currently support. p_bsy (port busy). Other Fabric services can provide this capability. with ack (acknowledge).Classes of service 2. f_rjt (frame reject). Revision0.
Revision0. Incorporated. frames might be delivered out of order #Confirmation of delivered frames #Login with both N_Port and Fabric required #Connectionless service with no turnaround delay to establish connection #N_Port to N_Port flow control #End-to-End (EE) delivery confirmation with ACK #Buffer-to-Buffer (BB) Link level flow control #Notification of frame delivery failure Class 3 Service #Multiplexes frames at the frame boundary #Adaptive routing %Each frame routed separately by Fabric %If multiple routes supported.Connectionless with ACK Class 3 .Connectionless without ACK Bidirectional multiplexed frame streams without dedicated connections © 2003 Brocade Communications Systems. . frames might be delivered out of order #Unconfirmed delivery of frames %Datagram service #Login with both N_Port and Fabric required #Connectionless service with no turnaround delay to establish connection #Buffer-to-Buffer (BB) Link level flow control #ULP recovery from frame delivery failures Brocade supports Class 2 and 3.FC-2 Common Classes of Service ! ! 20 Class 2/F .1_FC101_2003 Class 2 Service #Multiplexes frames at the frame boundary #Adaptive routing %Each frame routed separately by Fabric %If multiple routes supported.
Fabric busy conditions are an unusual event. The default time used by Brocade is 10 seconds. each R_RDY received increments BB_Credit value COS 2 and F use R_RDY and ACK (end to end or EE_Credit) flow control. The default time used by Brocade is 2 seconds Fabric_Reject (F_RJT) Used to signal that a frame could not be delivered. 3 and F " COS 3 uses R_RDY (buffer to buffer or BB_Credit) flow control.Flow Control ! 21 Flow Control mechanisms " " ACK Acknowledgement R_RDY Receiver_Ready ! ! Flow control is related to COS Brocade switches support COS 2. F and not Class 3. each ACK received increments EE_Credit value " ! Brocade switches use cut through routing – read S_ID / D_ID pairs and then shoot out the wire © 2003 Brocade Communications Systems. The time for frame delivery is determined by the Error Error Detection Time Out Value (E_D_TOV). Typical reason is for an ACK . the receiving N_PORT will send a P_BSY to the sender of the frame. Revision0.1_FC101_2003 Flow Control also includes mechanisms to communication “busyness”: F_BSY Fabric Busy F_RJT Fabric_Reject P_BSY N_PORT_BUSY P_RJT N_PORT_REJECT Flow control definitions: Acknowledgement (ACK) is used to confirm successful frame delivery and used by class 2. ACK times are established with the RA_TOV value. N_PORT_REJECT (P_RJT) Used to signal that a frame could not be delivered. Typical reason is for an invalid S_ID N_Port_Busy (P_BSY) When a N_PORT is unable to accept a valid frame due to a busy condition. Fabric Busy (F_BSY) occurs when the fabric is unable to deliver a frame. Incorporated. Receiver Ready (R_RDY) is used by the receiving port to signal it is ready to receive a new frame.
Revision0.1_FC101_2003 .FC-2 Buffer-to-Buffer (BB) Flow Control ! 22 Frames are moved from one buffer to another using Receiver Ready (R_Rdy) primitive signals Frame flow is always from the source buffer to the destination buffer Multiple intermediate buffers may be involved Tx Rx Tx Rx ! ! data data N_Port A Rx Tx Fabric Rx N_Port B Tx © 2003 Brocade Communications Systems. Incorporated.
point topology (two devices.FC-2 BB and End to End (EE) Flow Control ! 23 Frame flow is controlled by the receiver as a back-pressure mechanism Flow control is dependent on class of service (COS).1_FC101_2003 . no switches) ! End to end (EE_Credit) flow control is a originator device port to destination device port flow control mechanism " " EE_Credits are establish during node port login ACKs increments EE_Credit value © 2003 Brocade Communications Systems. but most use BB_Credits " " ! BB_Credit and length established during Fabric login BB_Credits are also exchanged during port login but ignored unless in a point-to. Revision0. Incorporated.
more data buffers are required to maintain a high throughput .At high speeds more data buffers are required to achieve the same link utilization for a given distance.Over extended distances.1_FC101_2003 Bandwidth Considerations . ! 24 Flow control is a mechanism to establish the maximum amount of data that can be sent at any one time A throttle-back mechanism to handle congestion without the need to discard frames Flow control considerations include: " " " ! ! Bandwidth Internal Host Buses Extended Distance © 2003 Brocade Communications Systems. Revision0. Incorporated. Internal Host Buses .FC-2 Flow Control Cont.Internal host Buses (like PCI) can become congested and thus throttle back receiving data from the Fabric Extended Distance .
a 2KB frame at 50km needs 25 buffer credits At high speeds more data buffers are required to achieve the same link utilization for a given distance A 2KB frame propagation delay at 200MB/sec is 40µsec and would need at least 10 E_Port buffers (“credits”) Proportionally. Increasing E_Port BB_Credits (points 3 and 4) allows full bandwidth utilization of extended Fabrics at specified distance 25 Switch 1 ISL Point 2 (F_Port) ! Switch 2 Points 3&4 (E_Ports) BB_credits are exchanged between points 1 and 2 during Fabric login and are used as throttle-back mechanisms BB_credits are also exchanged across E_Ports (points 3 and 4) and are used as both throttle-back and bandwidth utilization (performance) mechanisms © 2003 Brocade Communications Systems. a 2KB frame at 50km needs 50 buffer credits .1_FC101_2003 Point 1 (N_Port) ! Increase E_Port buffer credits to optimize performance at required distance The latency of a laser beam going through a fiber is 5nsec/meter (light wave propagation) or 5µsec/km 10km link round trip latency is 100µsec A 2KB frame propagation delay at 100MB/sec is 20µsec. Incorporated. Revision0. the switch would need at least 5 E_Port buffers (“credits”) Proportionally.FC-2 Flow Control Cont. So to keep a 10km “pipe” full.
Revision0. Incorporated.FC-2 Flow Control Summary FC flow control mechanisms associated with the Classes of Service (COS) that Brocade switches support: Feature Flow control based on Busy/ reject response Seq_id assignment Source sequence complete Destination sequence complete 26 Class 2/ F BB and EE (AWK) credit Yes. they will not be discussed in relation to flow control .1_FC101_2003 Note – Due to limited space and non-use of COS 1 and 4 in Brocade SANs at this time. frame discarded 0 to 255 Acc to ABTS. by Fabric or destination 0 to 255 Final ACK received w/ EOFt All data frames received and all ACKs sent Class 3 BB credit No. or ULP All data frames received and EOFt in last frame © 2003 Brocade Communications Systems.
. one of them is often a wellknown address like the fabric F_Port. Common ELS commands include FLOGI (Fabric login). Basic link service examples include: Abort sequence (ABTS) used to abort a sequence or a frame.FC-3 Layer Common Services 27 ! ! ! Largely unused: mostly theoretic Functions span multiple ports Common service advanced features: " Striping " Hunt groups © 2003 Brocade Communications Systems. Revision0. Incorporated. There is also a remove connection (RMC) class of service (COS) 1 frame that is used to request a COS 1 connections removal. PLOGI (port login). Another basic link frame is called No operation (NOP) which is used to initiate or terminate connections and sequences. Fibre Channel provides internal protocols called Basic Link Services and Extended Link Services. Extended link services (ELS) perform upper level protocol like functions between two fabric ports. PRLI (Process login) and more. Basic link services include low-level functions that are transported as a single data frame within a sequence. BA_ACC or BA_RJT are both ABTS responses and respectively mean Basic link service accept or reject. while FC-2 level concerns itself with the definition of functions with a single port. The FC-3 level deals with functions that span multiple ports.1_FC101_2003 FC-3: Common Services This layer defines advanced features such as striping (to transmit one data unit across multiple links) and multicast (to transmit a single transmission to multiple destinations) and hunt group (mapping multiple ports to a single node). fabric controller or name/directory server. These services would logically reside at FC-3 but are NOT part of the FC-3 standards. So. SCR (State Change Registration). RSCN (Registered State Change Notification).
Mapping for digital TV standard and MPEG (For example) over FC "VIA (FC-VI).Virtual Interface Architecture mapping.1_FC101_2003 "SCSI (SCSI-FCP or FCSI-201) . is used to denote the SCSI over Fibre Channel.Mapping FICON over FC.Transport of SCSI commands and data over the Fibre Channel protocol hierarchy is such a major part of the Fibre Channel usage that a particular acronym. (i. it is like the native protocol for Fibre Channel. Revision0. "IP(IETF Draft FC_IP or FCSI 202).defines mapping for ATM or LAN over FC "Audio Visual (FC-AV).e.FC-4 Upper Level Protocol (ULP) Mapping 28 ! ! ! ! SCSI IP WAN tunneling FICON © 2003 Brocade Communications Systems.Maps the IP protocol over Fibre Channel "WAN Tunneling (FC-BB). so we called it Fibre Channel Protocol. . The standard is called FC-SB-2 for Fibre Channel Single Byte Command 2. clustering) "FICON (FC-SB-2) . Incorporated. “FCP”.
FC-PH2. or ULPs.VI (Virtual Interface) FC-4.1_FC101_2003 A large part of the work done in developing the Fibre Channel architecture has been concentrated in assuring that the architecture could efficiently and naturally operate as a transport or delivery mechanism for a wide variety of well-established Upper Layer Protocols.Small Computer System Interface (SCSI) command set . the ULP mappings.LE FC .IEEE 802.AL FC . This layer maps the ULP onto the Fibre Channel transport layers.AL -2 8b/10b Encoding Copper.IP (Internet Protocol) . .2 . Revision0.3 Command Set Mapping (IPI-3 STD) 29 SCSI-3 IP ATM FC-4 SCSI . Fibre Channel is equally adept at transporting both network and channel information and allows both protocol types to be concurrently transported over the same physical interface. The following protocols are currently support: Storage protocols . the incremental cost in transferring systems over to a Fibre Channel data communication level decreases if the interfaces can be made as similar to previously-existing interfaces as possible. Incorporated.Fibre Channel Protocol Mappings Upper Level Protocols (ULPs) • Support existing protocols • Transparent to OS • No changes + New capabilities IPI .High Performance Parallel Interface (HIPPI) . This allows new capabilities to be added with minimal changes to currently-available interface.Intelligent Peripheral Interface (IPI) Network protocols . FC-PH3 ) FC . Optical © 2003 Brocade Communications Systems.3 Command Set Mapping FC Link Encapsulation FC . some of which are listed. Since much of the investment in current Operating Systems is at the device driver level.PH.ATM FC-3 FC-2 FC-1 FC-0 Framing Protocol Encode / Decode Physical Variant Common Services Fibre Channel Physical & Signaling Interface ( FC.ATM Adaptation Layer for Computer Data (AAL5) .Single Byte command code set (SBCCS) .Link Encapsulation (FC-LW) Cluster Server protocols . isolate the ULPs from the underlying Fibre Channel fabric and define interoperability of processes within a ULP.
or ATM. frame structure.0 and 1 layers specify physical and data link functions needed to physically send from one port to another " " FC-0 defines media connections and cables (speeds and feeds) FC-1 defines 8b/10b encoding. Revision0. such as IP or SCSI.1_FC101_2003 . COS and flow control mechanisms ! While FC-2 level concerns itself with the definition of functions with a single port. Incorporated. FC-3 level deals with functions that span multiple ports FC – 4 provides mapping of Fibre Channel capabilities to pre-existing protocols. link control and ordered sets ! FC – 2 specify content and structure of information along with how to control and manage information delivery " FC-2 defines exchange and sequence management. etc.Summary ! 30 FC . ! © 2003 Brocade Communications Systems.
Additional Information 31 From the resources tab – " Use Internet Links to access SAN ED 101 “SAN Fabric Foundation” chapter 2 (Fibre Channel Essentials) Also use the “Fibre Channel Theory Fundamentals Resource and Reference Material” selection under course syllabus “SAN Glossary” and “FC Recommended Reading and Resource List” © 2003 Brocade Communications Systems.1_FC101_2003 . Incorporated. Revision0.
Terminology and Addressing 1 © 2003 Brocade Communications Systems.Brocade Product Training Fibre Channel Topologies. Incorporated. Revision0.1_FC101_2003 .
attendees should be able to: ! Identify ! Discuss ! Discuss 2 FC Topologies and Review key FC Terminology FC Addresses © 2003 Brocade Communications Systems. Revision0. Incorporated.1_FC101_2003 .Course Objectives After completing this module.
Topics ! ! ! 3 FC Topologies FC Terminology FC Addressing " " " World Wide Names (WWNs) Port Identifiers (PIDs). S_ID. Revision0.1_FC101_2003 . also called 24-bit. Incorporated. or D_ID addresses Well-known Addresses © 2003 Brocade Communications Systems.
so on a Brocade switch port there are 125 available AL_PAs. Incorporated. is reserved for the Fabric Loop Port (FL_Port) Switched Fabric – Allows 16 Million theoretical devices to talk " © 2003 Brocade Communications Systems. Arbitrated Loop Physical Address (AL_PA) “00”. An arbitrated loop attached to a switch allows queuing into and out of the port where the loop is attached. The committee reserves ½ million of these addresses for well known addresses and testing purposes. The embedded port will take one AL_PA.Allows 126 devices to talk. Revision0.Fibre Channel Topologies Three kinds of Fibre Channel Topologies " " 4 Point-to-Point (Pt to Pt) . . Switched fabric can theoretically allow 16 million nodes to talk (16^6 – There are 6 Port Identifier (PID) slots with 16 hex choices per slot)). Without a switch only two of these devices can talk at a time. all others are “blocked” until those two are done. Arbitrated Loop is limited to 126 devices in a blocking architecture (plus one for FL_Port).Allows two devices to talk Arbitrated Loop .1_FC101_2003 Point-to-Point is limited to two devices but they can talk at greater distances than SCSI allows.
benefits from a longer reach with fiber optic connections. yet it has proven to be a fast and powerful method for connecting storage devices/arrays directly to the servers. Revision0. in this case a storage system and a server. Incorporated. This topology is very similar with SCSI direct attached except it is faster and supports longer distance. like all SAN topologies.1_FC101_2003 Point-to-point is a simple topology that allows bi-directional communication between two nodes. It is clear that a point-to-point topology has its limitations. Point to Point (Pt to Pt) 5 Two devices connected together © 2003 Brocade Communications Systems. Point-to-point. .Fibre Channel Topology Cont.
the initiator closes the session and releases control of the loop. •Reduced cost path into FC “SCSI Replacement” •Requires FC Hub technology •Easy for vendors to develop •Difficult for customers to deploy •Limited possible nodes (126) plus the Loop Master (FL_Port) •Lower overall throughput .Fibre Channel Topology Cont. The initiating node engages in a Point-to-Point connection with the recipient node. In order to send data from one node to another.nodes on the loop have to arbitrate for control of the loop in order to be able to communicate with a target device on the loop.the transmit of each node is connected to the receive of the next node.100MB maximum bandwidth •Limited any to any connectivity . Currently. Like an IBM Token Ring network. the maximum bandwidth is 100 MB/sec Fibre Channel Arbitrated Loop . Once the device wins arbitration. .1_FC101_2003 6 The arbitrated loop is a ring topology where each node passes data to its adjacent nodes. the transmitter of each node is connected to the receiver of the next node. the SAN hub arbitrates requests for data to make optimum use of the available bandwidth. The initiating device arbitrates for control of the loop. While this communication is happening all other devices are waiting to get their turn. allowing other devices to arbitrate for the loop. Revision0. Only one connection can be established at a time. Incorporated. When the data transfer is completed. In an Arbitrated Loop configuration. it then opens a communication session with the target and sends the data. devices must arbitrate for access to the loop. Fibre Channel Arbitrated Loop (FCAL) Theoretically up to 126 devices on a shared media for small systems at reduced cost and reduced performance level Requires a port to successfully arbitrate prior to establishing a circuit to send and/or receive frames © 2003 Brocade Communications Systems.
Incorporated. 2. Devices on the loop. the switch will probe them and get them into the Fabric Name Server if possible (private host devices do not accept probes). (The Fabricattached host could write data to the storage on the loop.FC Arbitrated Loop Topology An FCAL attached to a switch 7 Switch F F FL NL NL NL N N Hub NL NL NL © 2003 Brocade Communications Systems. A device on the loop and a device attached to the switch.) 3. A device attached to the switch and a device on the loop. Revision0. The FL_Port that the private loop device is attached to will also provide translation of Fabric 24-bit addresses and FCAL 8-bit addresses.) Devices on the loop can either be public (capable of doing a Fabric Login called a FLOGI) or private (not capable of doing a Fabric Login called a FLOGI) If the devices are private. (A host on the loop could access data from the Fabric-attached storage. .1_FC101_2003 NL This diagram shows an example of a FCAL loop attached to a switch Communication can take place between: 1.
the device with the highest priority succeeds. In arbitrating for control of the loop.Arbitrated Loop Physical Address AL_PA ! 8 Arbitrated Loop Physical Address (AL_PA) needed to communicate " An 8-bit address assigned to each device on loop Maximum of 126 AL_PAs attached to FL_Port " AL_PA “00” reserved for FL_Port These 127 AL_PAs are a unique set out of the possible 256 bit patterns The lower the AL_PA. Revision0. Arbitrated Loop devices receive an AL_PA during the loop initialization process.1_FC101_2003 Arbitrated Loop uses 8 bits to identify each of the devices on a loop. the higher the priority ! ! ! © 2003 Brocade Communications Systems. and are described in later slides. Using certain bit combinations can create disparity errors so the 126 AL_PAs available for the NL_Ports are a fixed set. Not all AL_PAs are created equal. This is the Physical Address for the device and is known as the AL_PA. Incorporated. so 126 unique AL_PAs need to exist for the NL Nodes AL_PA 00 reserved for the Switch FL_Port. The protocol allows for 127 devices. The lower the AL_PA assigned. The next slide shows the valid AL_PA table. . the higher the priority for the device in the loop.
). 05. etc.1_FC101_2003 The 8-bit addresses not shown in this table will never be used as an AL_PA for a device on the loop (03. The AL_PA for the FL_Port on a public loop will always be ’00’. Revision0.Valid AL_PAs 00 01 02 04 25 26 27 29 2A 2B 2C 2D 2E 10 31 32 33 34 35 36 51 52 53 54 55 56 71 72 73 74 75 76 80 81 82 84 A5 A6 A7 A9 AA AB AC AD AE 90 B1 B2 B3 B4 B5 B6 D1 D2 D3 D4 D5 D6 E0 E1 E2 E4 9 Highest priority 23 43 45 46 47 49 4A 4B 4C 4D 4E 63 65 66 67 69 6A 6B 6C 6D 6E A3 C3 C5 C6 C7 C9 CA CB CC CD CE 08 17 18 88 39 3A 3C 59 5A 5C 79 7A 97 98 E8 B9 BA BC D9 DA DC 1B 1D 1E 1F 9B 7C 9D 9E 9F 0F 8F EF Lowest priority © 2003 Brocade Communications Systems. 06. . Incorporated.
Incorporated. The port may use loop initialization to clear this condition. Enter/Leave Participating Mode A port in nonparticipating mode may. If the port is successful in obtaining an AL_PA.Loop Initialization What can cause it? ! 10 Power On /Power On Reset Entering/Leaving a participating mode Loop failure Arbitration Wait timeout Selective Reset LIP ! ! ! ! © 2003 Brocade Communications Systems. this is equivalent to a power-on reset. Arbitration Wait Timeout Excessive unfairness or a hung port may cause a port to not win arbitration. Revision0.1_FC101_2003 Power On / Power On Reset Loop Initialization occurs when a port is powered on or was given an equivalent reset. It relinquishes the AL_PA it was assigned and makes it available for other ports to acquire. being powered off. attempt to become a participating port. . Selective reset LIP Causes the ports on the loop to do a vendor-unique reset. or a physical connection in the loop is broken. A port already in participating mode can change to nonparticipating mode. Usually. after a port dependent timeout. it can participate in loop operations after initialization completes. If the port is unsuccessful it remains in nonparticipating mode. Loop Failure This may have occurred due to a port on the loop failing.
Incorporated. Once all the ports are in the Open-Init state. assign an AL_PA to each device on the loop and report the position of each device (optional). The port enters the Open-Init state and continues to retransmit the LIPs to the next port on the loop. a series of frames are passed around the loop to determine a loop master. .1_FC101_2003 The Loop Initialization Primitive (LIP) is used to begin the process and suspend any activities if the loop is currently active.Loop Initialization What happens? ! ! ! ! ! ! ! ! 11 Loop initialization begins by a port transmitting LIPs All loop activity is suspended All ports enter the Open-Initializing state One port is selected as Master Ports are assigned an AL_PA Positional AL_PA map of loop is built (if supported) All ports return to Monitoring state Normal loop operations resume © 2003 Brocade Communications Systems. Revision0. Receiving ports recognize the loop initialization process when at least three consecutive LIPs are received. The loop returns to the monitoring state and normal loop operations can resume.
LIHA.AL_PS) .loop port in initialization state does not have an AL_PA LIP(F7. LIFA.used to perform a vendor specific reset at loop port AL_PD. This port controls the rest of the loop initialization process. LIRP and LILP are also frames that are passed around the loop but are used to allow the reporting of the position of the device on the loop. Incorporated. A LIP is a Primitive Sequence Ordered Set.F7) . AL_PS port originated the request . in the initializing state. Ordered Sets are special four-character combinations that have special meaning in Fibre Channel.loop port. F7) . This is an optional step in the loop initialization process. Primitive Signals are used to indicate events or actions and are normally transmitted once.1_FC101_2003 LIP and CLS are Ordered Sets used to indicate states or events.AL_PS) .loop port identified by AL_PS detects loop failure LIP(AL_PD. It will determine the device that becomes the loop master. requests loop initialization due to loop failure LIP(F8. CLS is a Primitive Signal Ordered Set. LISA are frames passed around the loop for devices to have their AL_PA assigned. LISM is a frame that each device enters on the loop.AL_PS) . It allows any device to learn not only the AL_PA of all the devices but the order in which they occur on the loop.loop port identified by AL_PS requests loop initialization LIP(F8. without a valid AL_PA (thus the F7). LIPA.Loop Initialization Sequence of Events LIPs LISM LIFA LIPA LIHA LISA LIRP LILP ( ( ( ( ( ( ( (Loop Initialization Primitive Sequence) … Select Master) … Fabric Assigned) … Previous Assigned) … Hardware Assigned) … Software Assigned) … Report Position (if supported)) … Loop Position (if supported)) 12 FC Frames CLS (Close … Primitive Signal ) Initialization is complete # ! Public loop devices can log into the Fabric! © 2003 Brocade Communications Systems. There are different types of LIP sequences: LIP(F7. Primitive Sequences are used to indicate states or conditions and are normally transmitted continuously until something causes the current state to change. Revision0.
ops 1 parameters 13 © 2003 Brocade Communications Systems.dataFieldSize: 2112 <Output truncated> fabric.ops parameters contain configurable parameters that need to be the same on all Fabric switches. Incorporated. The result is a Fibre Channel “Fabric”.ops.ops. Switched Fabric Switch Fabric Topology: ! Highest performance level ! High scalability ! Good fault isolation ! Embedded management and services ! Up to 239 unique domains (switches) with: " Unique switch names " Unique IP addresses " Same fabric. Switched SAN Fabrics • Fullest FC Network topology • Require FC Switch technology • Difficult for vendors to develop • Easy for customers to deploy • Maximum possible nodes (16 Million or 224 theoretical) • Higher overall aggregate throughput . Footnote 1: fabric. Switches can be cascaded and combined with loops to create highly interwoven networks known as fabrics.1_FC101_2003 Switched fabric . Fortunately. these complex solutions can be kept under control by software that takes advantage of SAN management capabilities built directly into the fabric.each connection to the switch is 100 or 100 MB per second.Any device on the switch/Fabric can communicate with any other device on the Fabric • Scaling is easy as switches can be connected together in various topologies. Revision0. examples include: fabric.Fibre Channel Topology Cont.An extensive storage network in which large numbers of servers and storage systems are connected using Fibre Channel switches. • Enterprise any to any connectivity .mode.pidFormat: <Output truncated> 1 .
and name server. Incorporated. . or to a set of global software components such as the routing tables. Revision0.1_FC101_2003 The term Fabric can also refer to the physical switches.Fibre Channel Terminology Review Terminology will be defined throughout the course but some basics would be helpful: " Fabric – A Fabric is a connection of Fibre Channel switches and/or devices capable of routing frames using only a destination identifier (D_ID) – A Fabric is commonly pictured as a cloud 14 © 2003 Brocade Communications Systems. zoning configuration.
Source Identifier (S_ID) when it’s used as a source in FC communications and Destination Identifier (D_ID) when it’s used as a destination address in FC communications.1_FC101_2003 Each Node has a unique 64-bit address called “Node World Wide Name”. . Each Port also has a unique 64-bit address called “Port World Wide Name”. The PID is assigned to the port when it logs into the fabric (FLOGI). N_Ports are node ports that can either attach to other N_Ports or to Fabric Ports (F_Port). ! 15 Nodes – Transmit and Receive information via one or more ports which provide the physical connection(s) for the nodes Ports – Separate transmit (tx) and receive (rx) functions " ! Tx encodes and transforms data to serial format Rx recovers clock from serial data received. Revision0. Nx_Ports could either be N_Ports (x not used) or NL_Ports (Node Loop Ports) used in the Arbitrated Loop topology. Each Nx_Port also has a 24-bit address also referred to as: port identifier (PID).Fibre Channel Terminology Cont. The format of this 64-bit identifier along with the format for the port on this nodes 64-bit identifier are specified by IEEE. decodes and deserializes the data " Node I Tx Tx Node II Nx_Port A Rx Nx_Port B Rx © 2003 Brocade Communications Systems. Incorporated.
also referred to as " " " ! Port identifiers (PIDs) 24-bit addresses Source ids (S_ID) or destination ids (D_ID) ! Well-Known Addresses – The Fabric addresses used for accessing Fabric services Fabric Services – Intelligent “services” provided by a Fabric. Incorporated. Revision0.1_FC101_2003 .Deciphering FC Addresses More FC Terms ! 16 Node and port names – Fixed 64 bit addresses used to uniquely identify Fabric devices also referred to as node and/or port world wide name (WWN) Fabric Address – Required address. necessary for Fabric operation (more on these in next module) ! © 2003 Brocade Communications Systems. needed for devices and services to communicate.
FC Addresses - FC layers
Compare to OSI layers:
PID, 24 bit, S_ID / D_ID
Node / Port WWNs
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
Node WWN Name Format
Assigned by IEEE
Single hexadecimal Name Assignment Authority (NAA) digit (Brocade uses a 1) FC Standard reserved
Assigned by the vendor
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
Based on the IEEE Standard format, a typical SilkWorm Node WWN is: 10:00:00:60:69:xx:xx:xx Where: The first 2 bytes are always 10:00 (format 1 addressing); The next 3 bytes are vendor specific. Brocade was assigned 00:60:69; The last 3 bytes are derived from the Brocade SilkWorm main board; The 3 byte company ID found in the 64 bit IEEE Standard format WWN can be searched at: http://standards.ieee.org/regauth/oui/index.html The 1st 4 bits of FC 64-bit addresses identify the authority responsible for administration of that address or the Name Assignment Authority (NAA). A subset of NAA address authority denotes the naming convention used. FC-PH Rev 4.3 Fibre Channel standards table 41 define the NAA identifiers. Brocade uses a HEX 1 in the first 4 bits – this translates to a binary 0001 and tells you that the Brocade node address represents an IEEE format 1 name which is based directly on the 48-bit MAC address in the middle 3 bytes of 64-bit address (Brocades = 00:60:69). See the notes on the next slide for a list of ommon NAA identifiers.
Port WWN Name Format
Assigned by IEEE
Single hexadecimal Name Assignment Authority (NAA) digit (Brocade uses a 2) Three hex digits usually set by the vendor to uniquely identify a port on a device or switch
Assigned by the vendor
© 2003 Brocade Communications Systems, Incorporated. Revision0.1_FC101_2003
Fabric Port Name 2p:pp:00:60:69:xx:xx:xx The next 3 nibbles (p:pp) are used by Brocade to show the switch port number. 20:04:00:60:69:1f:25:e6 The “0:04” which means this is port 4 on the switch Common NAA identifiers include: HEX
Address based on IEEE 48-bit address (middle 3 bytes of 8-byte (64-bit) address (WWN) - referred to as address format 1. Brocades = 00:60:69
A format 2 address based on the same IEEE address described in NAA HEX 1 identifier but used to define ports associated with a node using IEEE address format 1. Format 5 IEEE registered addressing was added in FC-PH3 standards to extend the number of vendor addresses beyond NAA = Ox1. Format 5 allows vendors to uniquely use the whole address space (all bits) as a Vendor-Specific IDentifier (VSID).
Incorporated.1_FC101_2003 . Revision0.FC Device Node / Port WWNs Switch Main Board Node WWN: 10:00:00:60:69:50:60:02 Switch Port 3 WWN: 20:03:00:60:69:50:60:02 20 ! Each port on the switch will have a unique port WWN !A !A node WWN is often referred to as node name port WWN is often referred to as port name © 2003 Brocade Communications Systems.
Switch Main Board Node WWN: 10:00:00:60:69:50:60:02 Switch Port 3 WWN: 20:03:00:60:69:50:60:02 HBA 21 HBA Node WWN: 10:00:12:34:56:00:10:0E HBA Port 1 WWN: 11:00:12:34:56:00:10:0E HBA Port 2 WWN: 12:00:12:34:56:00:10:0E © 2003 Brocade Communications Systems. Incorporated.FC Device Node / Port WWNs Cont.1_FC101_2003 Not all Fabric devices assign port WWNs the same. Revision0. . The fictitious HBA vendor pictured above used the 2nd nibble of the first byte to designate port numbers while Brocade switch port WWNs use the 2nd nibble of the 2nd byte to designate ports in WWN addressing.
Revision0.FC Addresses Analogy “Telephone Numbers” for FC Devices ! 22 Telephone Service: " " " Telephone number to call Your telephone number Telephone service (accessed with your telephone number) ! Fabric Service: " " " Destination ID (the Fabric address of whom you want to communicate with) Source ID (your Fabric address) Fabric service © 2003 Brocade Communications Systems.1_FC101_2003 . Incorporated.
the new format XXYYZZ has been adopted. .x based products. The 24-bit port address format consists of three bytes defining the Domain identifier.x could originally only support a maximum of 16 ports in one switch. The Domain ID field byte provides domain addressing 1-239. the largest switch has 16 ports. where Y was a hexadecimal number that specified a particular port on a switch and “1” was constant. where “YY” represents a port area designator. The three byte fields of the old PID format were defined as XX1YZZ. When Brocade developed the ASIC for the SilkWorm 2000 series.1_FC101_2003 Native Addressing Mode: The PID format on switches running Fabric OS v2.x. Core PID Addressing Mode: To support the increased port count on the higher port count products based upon Brocade Fabric OS v4.x based products and Fabric OS v2. so only half of the second byte in the Area field of the PID was required to specify ports. a setting was created to enable the PID format to be set to use either the new format or the old format.x and v3. To ensure inter-operability between Fabric OS v4. while maintaining compatibility with older firmware versions. Area address and AL_PA fields respectively. Incorporated.FC Addresses ! 23 Fabric addresses are 24-bits (3 bytes long) A device’s Fabric address indicates: " The switch and port number to which the device is connected " The FC-type of device (Fabric or loop) Fabric addresses are represented in hexadecimal format (0x) which often appear before the address Fabric addresses come in two modes: Native and Core PID address modes ! ! ! © 2003 Brocade Communications Systems. Revision0. Each field can provide 00-FF addressing. Core PID addressing mode is the default address mode on all Brocade switches with greater than 16 ports.x and v3. Using the entire middle byte for the port area designator allows Brocade switches to scale up to the Fibre Channel standard maximum of 256 ports per switch. This is commonly known as the Core Switch PID format setting.
S_ID or D_ID) Address are three bytes long: " " " " ! Byte Zero: Byte One: Byte One: Byte Two: BYTE Zero Domain Number 1-239 1-239 Domain Number Area Number* Native Mode/Port Node Address BYTE One Area Number 0-255 1 0-F 0x01 – 0xEF 0x0-0x255 0x0-0xF 0x00 or > 0x00 BYTE Two Node Address 00 or ALPA 00 or ALPA *Core PID Enabled on 16 port switches © 2003 Brocade Communications Systems. Incorporated. Revision0.1_FC101_2003 . ! 24 Each switch (Domain) is responsible for assigning unique 24 bit Fabric address (also referred to as PID.FC Addresses Cont.
The value of “LL LL” is the same for all Public Loop devices attached to the same FL_Port and has the same meaning as NN NN “Fabric” addressing. Core PID address mode uses an AREA designation to indicate port numbers 0 # 256. so core pid addressing was developed and the 1 offset (2nd byte. This Fabric assigned address 1 byte represents the domain of the switch. and “PP” = the local loop address (AL_PA) These devices 1st LIP to get 8 bit AL_PA and then FLOGI and are assigned the other 16 bits (LLLL) PP is always a non zero value © 2003 Brocade Communications Systems. Private devices use an address format of “00 00 PP”. Address Assignment Dependency – 3 Address Classifications: Fabric Address Public Loop Address Private Loop Address 25 NN NN 00 “NN NN 00” is the generic address of any Fabric device that has logged into the fabric (FLOGI). and “PP” = the local loop address (AL_PA). This type of address is simply a Fabric assigned address for a device attached to an FL_Port (24 bits). the 2nd byte 4th nibble is the port. A Private Loop device has a 1-byte. there are 15 possibilities (0-F). the last byte (2 nibbles) is 00 indicating a Fabric device. called the arbitrated loop physical address (AL_PA). Device FLOGI response assigns 24 bit Fabric address Native Mode has 2nd byte 1st nibble with a 1 Core PID mode uses entire 2nd byte # AREA LL LL PP Where “LL LL” is assigned by the Fabric at login (FLOGI). where “LL LL” is assigned by the Fabric at login. 8-bit address. Incorporated. The 2nd byte or 3rd nibble is 1 (native mode) for a 2000 series.1_FC101_2003 00 00 PP Device “LIPs” # “PP” = the local loop address (AL_PA) Private address only use the last byte (8 bits) of the 24-bit Fabric address PP is always a non zero value Fabric attached devices use an address format of “NN NN 00”. 3rd nibble) was done away with. Port counts greater than 15 required a change in addressing modes. Public Loop attached devices use an address format of “LL LL PP”. This type of address is all that a Private device is capable of receiving or sending (8 bits). where “PP” = the local loop address. Revision0. the Private devices may only communicate with the devices it can see on the local loop. Therefore.FC Addresses Cont. . where “NN NN 00” is the address of any Fabric-attached device that has logged into the fabric.
If non-zero. then it is the AL_PA on the FL_Port. .1_FC101_2003 A sample Fabric address:021500 XX1YZZ XX=02 1 Y=5 ZZ=00 of the device Domain_ID of the switch Native mode Port # If 00. port 4 © 2003 Brocade Communications Systems. Incorporated. Revision0.Fabric Address Example 1 Native Address Mode ! 26 Native address mode can be used when a 3900 or 12000 is not present in the fabric: XX 1Y ZZ XX is a value between 0x1 to 0xEF inclusive The “1” means Native Mode (think of it as a 1 offset) Y is the port number (0-15) ZZ is the AL_PA for a loop device or 00 for an F_Port ! A Brocade “Fabric” device will have low byte = 00 02 14 00 Port Number = 4 Domain (Switch) ID = 2 A “1” can be used in the upper nibble of byte #2 if port count ≤16 AL_PA = 00 (Non-Loop Fabric device) Note: This is a Fabric device connected to switch domain 2. then it is an F_Port.
Using the entire middle byte for the port area designator allows Brocade switches to scale up to the Fibre Channel standard maximum of 256 ports per switch.0+ and OS v3. a setting was created to enable the PID format to be set to use either the new format or the old format. The three byte fields of the old PID format were defined as XX1YZZ. while maintaining compatibility with older firmware versions..x based products and Fabric OS v2. To support the increased port count on the higher port count products based upon Brocade Fabric OS v4. the new format XXYYZZ has been adopted.x and v3. non configurable address mode on switches with greater than 16 ports AREA = 21 HEX so this is port 33 0a 21 00 AL_PA = 00 (Non-Loop Fabric device) AREA Number = 21 Domain (Switch) ID = 0a = 10 ! Set all configurable switches to CORE PID if possible © 2003 Brocade Communications Systems. To ensure inter-operability between Fabric OS v4.1)  ! ! This format allows interoperability for switches with port > 16 Core PID address mode is default.x and v3.x could originally only support a maximum of 16 ports in one switch.x based products. The 24-bit port address format consists of three bytes defining the Domain identifier. Area address and AL_PA fields respectively.x Configurable option under the configure command Core Switch PID Format: (0. where Y was a hexadecimal number that specified a particular port on a switch and “1” was constant. so only half of the second byte in the Area field of the PID was required to specify ports. This is commonly known as the Core Switch PID format setting.02+ " " Enables attachment to higher port count switches that use OS v4.Fabric Address Example 2 Core PID Address Mode ! 27 Configuration required on OS v2. . The Domain ID field byte provides domain addressing 1-239. Incorporated. the largest switch has 16 ports. When Brocade developed the ASIC for the SilkWorm 2000 series.1_FC101_2003 The PID format on switches running Fabric OS v2. where “YY” represents a port area designator. Revision0.x.6. Each field can provide 00-FF addressing.
Public Loop Address Mode ! 28 A “loop” device will have a non-zero address in low byte. Incorporated. AREA = 10 HEX so this is port 16 0a 10 e8 Area Number = 10 Domain (Switch) ID = 0a = 10 ! Loop Address = e8 (loop device) This loop device is connected to switch 10 on port 16 and has a loop address of ”e8” To determine address mode on a switch with 16 or less ports.Fabric Address Example 3 Core PID. check the core PID address mode using configshow or configure commands © 2003 Brocade Communications Systems.1_FC101_2003 ! . Revision0.
The services residing at these addresses provide a service to either nodes or management applications in the fabric. consistent format and behavior when a service provider is accessed for registration and query purposes. Time Server: The time server sends to the member switches in the fabric the time on either the principal switch or the Primary FCS switch. . Fabric Login: Before a fabric node can communicate with services on the switch or other nodes in the fabric an address is assigned by the fabric login server. Alias Server: The Alias server keeps a group of nodes registered as one name to handle for multicast groups Broadcast Server: This service is optional and when frames are transmitted to this address are broadcasted to all operational N and NL ports. Directory Server: The directory server/name server is where fabric/public nodes register themselves and query to discover other devices in the fabric. is used. Incorporated. When registration and query frames are sent to a Well Known Address a different protocol service.1_FC101_2003 Every switch has reserved three byte addresses known as Well Known Addresses. Fabric addresses assigned to nodes are three bytes long and are a combination of the domain id plus the port area number of the port the node is attached to. Management Server: The Management server provides a single point for managing the fabric. This protocol provides a simple. Revision0. Fibre Channel Common Transport (FC-CT).Well-Known Addresses of Fabric Services 29 Well-Known Addresses Fabric Login FFFFFE Directory Server FFFFFC Fabric Controller FFFFFD Time Server FFFFFB Mgmt Server FFFFFA Alias Server FFFFF8 Broadcast Server FFFFFF Brocade Fabric Operating System © 2003 Brocade Communications Systems. Fabric Controller: The fabric controller provides state change notifications to registered nodes when a change in the fabric topology occurs.
1_FC101_2003 An exchange can be bi-directional Sequences and frames are uni-directional . Revision0. Incorporated. Review Fibre Channel uses Exchanges to send Sequences of Frames " 30 " " " The Exchange manages the transaction – it contains a set of related sequences Sequences within the Exchange hold sets of related Fibre Channel frames A Frame contains a header and payload and is up to 2148 bytes An example Small Computer Serial Interface (SCSI) Read Command: Initiator CMD (Se quence) Target One Exchange equence) DATA IN (S STATUS (Seq uence) Sequences of Frames © 2003 Brocade Communications Systems.Fibre Channel Terminology Cont.
Revision0.Fibre Channel Terminology Cont.6 and F). RX_ID is frequently optional " Exchange controls found in F_Ctl frame header field are: Seq_Init (initiating sequence). and exchanges can all be multiplexed © 2003 Brocade Communications Systems. Seq_ID (sequence identifier) Info_CAT. Fibre Channel Information Transfer Sequence 3 Sequence 2 “Information Units” (Large Blocks) Sequence 1 Exchange (Transactions) 31 Sequence Buffer S O F Buffer Data C R C E O F (server/storage/Work Station (WS)) N_Port A FC Frame (Max. sequences. R_CTL (routing control to indicate data.2. One or more frames will be sent and those frames can reside in one or more sequences. Last_Seq (indicates last sequence of exchange). payload 2112 Bytes) Frames. Exc_Contxt (indicates whether originator or responder in Exchange) Header (server/storage/WS) N_Port B Sequence " Composed of 1-n Frames " Unidirectional set of frames for an operation " Each Sequence is identified by initiator: Sequence Identifier (SEQ_ID) " Each frame within a Sequence is numbered: Sequential Count (SEQ_CNT) " Other sequence controls: SOFiX (start of frame for class x used to indicate class of service this sequence is using.or bi-directional flow of sequences for an operation " Exchanges normally uses the same ULP " Exchange may be Identified by each end: Originator / Responder Exchange IDs (OX_ID. The sequences reside in an Exchange. SEQ_CNT (SEQ_CNT is incremented by 1 for each data frame sent). From the other point of view: Exchanges consist of sequences of frames. RX_ID). Exchange " Composed of 1-n non-concurrent sequences " Unit. End_Seq (set to 1 for last sequence). . SOFnX is used for subsequent frames). The flow control (throttling of data from one port to another) depends upon the class of service (COS) being used as specified during PLOGI when common service parameters where exchanged. First_Seq (indicates first sequence of exchange). EOF (last frame of sequence will be indicated by a EOFt) Frame: " Frame is smallest unit of transfer and is discussed in more detail in the next slide. ACK (for COS 1.Unsolicited Command (information category).4. Incorporated.1_FC101_2003 This picture shows the flow of data in the Fibre Channel environment for a point-to-point connection or a Fibre Channel connection through a Fabric.
Indicates the sequential order of the frame in the sequence OX_ID . passing initiative SEQ_ID . and Acknowledge Frames are common..sequence Count . class.Frame Control.Fibre Channel Terminology Cont.Routing Control bits communicate the type of frame we are looking at: Extended Link Service Frame. sequence count) Addressing (Source and Destination) Word 0 Word 1 Word 2 Word 3 Word 4 Word 5 Word 6 … Byte 0 Byte 1 Byte 2 Byte 3 R_CTL Destination ID (D_ID) CS_CTL Source ID (S_ID) Type Frame CTL SEQ_ID DF_CTL SEQ_CNT OX_ID RX_ID Parameter Payload Important bytes: •R_CTL = Routing Control • Destination Fabric Address • Source Fabric Address • Protocol Type:SCSI.Frame Header FC header contains vital frame information: ! ! 32 Control information (routing.Exchange ID assigned by the originator RX_ID . This field indicates if there are any optional headers SEQ_CNT .The exchange ID assigned by the responder to the Exchange Data Field/Payload . 01 = Extended Link Services 05 = ISO/IEC 8802-2 LLC/SNAP (IPFC) 08 = SCSI – FCP 20 = Fibre Channel Services F_CTL . Data Frame.Class specific Control Field.1_FC101_2003 R_CTL . This field contains information related to the frame contents.Sequence ID DF_CTL .Data Structure Type that describes what the data is: i. Revision0. D_ID .Destination ID (Native port address or well-known address) CS_CTL . Note: S_ID and D_Ids are also referred to as PIDs (port identifies) or 24 bit addresses.Responder ID . IP … • Payload word that defines what is being said called command code • Payloads often contain node and port names… © 2003 Brocade Communications Systems. . Incorporated.Originator ID . FC-PH Generic Frame .Data Field Control. This field is always zero for Classes 2 and 3 per the standards but may change in the future S_ID .Source ID (Native port address or well-known address) Type . Example: First/last sequence.This is the payload of the frame and can be from 0 to 2112 bytes in length The R_CTL byte is one of the first items to check.e.
and a review of Exchanges.1_FC101_2003 . Revision0. Incorporated. S_ID. Fabric and Public Loop FC addresses are called Port Identifiers (PIDs). or D_ID addresses Well-known Addresses are used to communicate with Fabric services The FC Frame is 2148 Bytes and includes a SOF. Ports. Header. Nodes.Summary ! 33 FC Topologies include Point-to-Point. Payload. 24-bit addresses. Sequences and Frames FC devices have node and port WWNs FC device address classifications: Private. and S_ID. Arbitrated Loop and Switched Fabric FC Terminology discussed includes Fabric. CRC and EOF " ! ! ! ! ! ! The header contains D_ID. OX_ID and RX_ID identifiers © 2003 Brocade Communications Systems.
Find SAN ED 101 link “SAN Fabric Foundation”. look at chapter 2 (Fibre Channel Essentials) for additional information about material presented in this module Use resources page Reference section .1_FC101_2003 . Revision0.Additional Information 34 Use resources page Internet Link section . Incorporated.FC_AL Initialization presentation for detailed FC_AL initialization information © 2003 Brocade Communications Systems.
Brocade Product Training Expected Fibre Channel Behaviors 1 © 2003 Brocade Communications Systems. Revision0. Incorporated.1_FC101_2003 .
Incorporated.1_FC101_2003 . Revision0.Course Objectives After completing this module. attendees should be able to: ! Identify 2 FC Communication Methods ! Introduce FC services and review associated WellKnown addresses expected FC behaviors ! Identify © 2003 Brocade Communications Systems.
Topics ! 3 FC Communication Methods " " " " " Switch Initialization FC Communication Terminology – Connection Naming Device Initialization Link Communication Well-Known address fabric communication services ! Expected Fabric Communication Behaviors " " " What happens when a Fabric device connects to a Fabric? What happens when a public loop device connects to a Fabric? How do “private” devices communicate in a Fabric? © 2003 Brocade Communications Systems.1_FC101_2003 . Incorporated. Revision0.
. NL nodes have two classifications. Revision0. 2. hence the word private. Fabric Nodes (N_Ports) can communicate with any Fabric Node and can communicate with private or public NL nodes on a loop.1_FC101_2003 Nodes that attach to the fabric can either be a N (Node) or NL (Node Loop). Communication can take place between: 1. Public NL nodes can communicate with any member of the same hub of FL port and have the ability to send a frame to the fabric.FC Communication – Device Naming Review Port Types and Node Topologies Switch attached FC_AL NL_Port (Node Loop Port) FL_Port (Fabric Loop Port) 4 Switch F_Port (Fabric Port) F NL NL N N_Port (Node Port) Hub NL Private NL Note: All NL_Ports are public unless otherwise noted NL Private NL © 2003 Brocade Communications Systems. (The fabric attached host could write data to the storage on the loop. This diagram shows an example of loop that contains public and private devices. Devices on the loop. private or public. Private NL nodes can only communicate other nodes that are attached to the same hub or FL port. Brocade FL_Ports allow communication to occur between: • A device attached to the switch and a device on the loop. Incorporated.
Switch Initialization ! 5 At power on. Revision0. Incorporated. .cable or plug (self loop) ERRSTAT and ERRSTATS generally indicate that the port is good enough to initialize. the boot PROM diagnostics: " " Verify CPU DRAM memory Initialize base Fabric Operating System (FOS) ! The initialized FOS does the following: " Execute Power-On-Self-Test (POST) tests " Initialize ASICs and front panel " Initialize link for all ports (put online) " Explore the fabric and determine the principal switch " Assign addresses to ports " Build unicast routing tables " Enable N_Port operations © 2003 Brocade Communications Systems. but not good enough to sustain traffic # Usually signal integrity. PORTDIED and TIMEOUT errors indicate that frame data issues caused the low level driver or hardware to discard a frame or take a port offline.1_FC101_2003 Port Initialization Error Conditions NO_SYNC and NO_SEGMENT errors indicate that the port has a problem initializing # Usually Media or loopback device .
U_Port starts mode detection process by transmitting at least 12 LIP(F7) Primitive Sequences Transition 2 a. c. U_Port will attempt the OLD_PORT initialization (Link Initialization Protocol for point to point) by taking the link down and then transmitting NOS Primitive Sequence if LIP timeout or any of the loop initialization phases timeout or only one non-zero AL_PA is claimed in Loop Inititalization Sequences (LOOP_EMPTY=false) or no nonzero AL_PAs is claimed (LOOP_EMPTY=true) 5. b. If self loopback detected after ELP exchanges and LOOP_EMPTY = false port exits G_Port and reinitializes as FL_Port State 2 Note – The firmware will automatically attempt to reinitialize a faulty port every two seconds.1_FC101_2003 1. If all these tries fail.Port I’m waiting for someone to talk to me… (State 3) Are you a switch or a Fabric point-to-point device? (Transition 3) F . The normal E_Port or F_Port mode detection procedure follows Transition 2. the port will operate in the G_Port mode State 3. Incorporated. 3. If LIP Primitive Sequences are not received U_Port attempts OLD_PORT initialization by taking the link down then transmitting NOS primitives Transition 2. To ensure N_Port. If ELP succeeds. Revision0. If the ACTIVE state is reached. If Link Initialization Protocol fails after 1 retry or LIP received after 1 second go to Transition 2 (FC-AL) initialization c. the port will be marked as faulty. A switch port is a Universal Port (U_Port) that operates in either E/F_Port (G_Port) mode or FL_Port mode State 1 2.Port (State 2) yes y/n no Do you want to talk loop? (Transition 2) G . If a valid FLOGI is received. Is something connected to the port? If yes Transition 1 continue…. a. the U_Port operates in the E_Port mode State 4. reinitialize the port and the switch port will cut the laser forcing a loss of signal state for at lease 20 µs then the switch port will bring back the laser and issue NOSs Transition 2 4.Port (State 4) © 2003 Brocade Communications Systems.Port What do I want to be when I grow up? (State 1) Is something plugged into the port? (Transition 1) yes 6 no y/n FL . When operating in the FL_Port mode. If at least 3 consecutive LIP primitive sequences LIP(F7) received then port enters OPEN_INIT state and attempts FC-AL loop initialization State 2 b. . a U_Port will try the loop initialization procedure three times.Fabric Port Initialization Process Device Initialization into the Fabric from the switch’s perspective U .Port (State 5) Fabric pt-to-pt switch E . the U_Port becomes an F_Port State 5.
Link Initialization Device Initialization into Fabric from Device Perspective If speed negotiation available device and switch will negotiate to the highest common speed # Speed negotiation process on next slide 7 Switch F N © 2003 Brocade Communications Systems. Revision0. primitive signals and primitive sequences . Incorporated.1_FC101_2003 Ordered set characters act as frame delimiters.
Loss of Sync) is detected the Transmitter (TX) and Receiver (RX) will be set to the highest possible speed for that port in preparation for the Auto-Negotiation process. Speed List is used by the TX and RX to try eligible speed settings. •Auto-Neg Timer (1600ms): This timer controls the amount of time allowed for Speed Auto-Negotiation. it has 16ms to sync up before going to the next lower speed. There are three primary timers associated with setting the TX/RX speeds: •TX Sync Timer (154ms): This timer actually controls the amount of time a Receiver can spend synchronizing on a specific speed from the Speed List. At this time. Speed negotiation can be viewed as a two phase process. the next speed in the Speed List is selected for RX and the RX Sync process is attempted again. 3. . Since the TX Sync Timer is set to 154ms.The port will allow the Receiver 16ms to sync. If a successful sync is accomplished the RX speed is compared to the TX speed to ensure consistency. If the Auto-Negotiation timer has been exceeded. Revision0. If a successful sync was not possible within the TX Sync Timer (154ms) window. Once set. If equivalent. Lower speeds are tried only if higher speeds fail. Auto-Negotiation process: In general. 2. the primary objective of the Auto-Negotiation algorithm is to start at the highest speed possible and step down until both devices agree on a speed. whereas the outer box is used to set the TX speed. the Bloom ASIC will have 2 speeds in the speed list (1Gbps and 2Gbps). Setting the TX Speed: 1.1_FC101_2003 Switch Port.The Speed List exists inside the switch and contains the acceptable values to which the port can AutoNegotiate. When signal (Power ON.Speed Negotiation Speed List 2Gbps 1Gbps (START) 8 Power ON Loss of Signal Loss of Sync Set TX/RX toMAX speed in Speed List Wait for Light From the Switch Port’s Point of View RX = Incoming Speed YES IF RX = TX YES Speed Set (16ms) Set RX Speed NO NO (STOP) Set TX To Next Speed In Speed List Set RX To Next Speed In Speed List Set TX Speed NO AutoYES Negotiate Time Exceeded (1600ms) © 2003 Brocade Communications Systems. Once the lowest speed has been tested by either TX or RX. Loss of Signal. The inner box is used to set the RX Speed. then the RX Speed process is exited and the TX Speed is performed (See Setting TX Speed Below). If the TX Sync Timer has not been exceeded. This would be entered when TX Sync Timer (154ms) has been exceeded. If this timer is exceeded. If the Auto-Negotiation timer has not been exceeded the TX is set to the next Speed in the Speed List and control is passed to the Set RX Speed process described above. 3. Incorporated. this will allow multiple speeds to be tested before exiting to the Set TX Speed Process. •RX Sync Timer (16ms): This timer controls the amount of time allowed for a receiver to sync up on a specific speed. Setting the RX Speed: 1. RX and TX have separate Speed List pointers. Note . the Auto-Negotiation process has completed. When the Receiver selects a speed. 2. that specific pointer will be reset back to the top of the list and the highest speed will be tested again. the process starts all over again. the process stops and control is passed back to the Wait for Light condition. the port will wait for light and then proceed with the Speed Negotiation process.
3. Link Reset state consists of: • LR1 (Link Reset Transmit) • LR2 (Link Reset Receive) • LR3 (Link Reset Response) 3. 9 Active State (AC) Link Reset State (LR) Offline State (OLS) Link Failure State (LF) © 2003 Brocade Communications Systems. Offline state consists of : • OL1 (Offline Transmit) • OL2 (Offline Receive) • OL3 (Wait for Offline) 4. Incorporated. Certain conditions encountered within the port may cause the port to exit the active state and perform one of the following primitive sequence protocols: Link Reset. Revision0. 4. Link Failure state consists of: • LF1 (No Operational Receive) • LF2 (No Operational Transmit) . 2.1_FC101_2003 1. 2.Port State Machine There are four states of the Port State Machine (PSM) used for Fabric (G_Port) connections: 1. Active state (AC) – in the Active state. or Link Failure.Port State Machine Processes Link Control Protocols . the port is able to transmit and receive frames and primitive signals. Offline state.
OLS (Off Line) 2. OLS (Off Line) 3. Incorporated. NOS (Non Operational) 2. LR (Link Reset) 3. IDLE The proper flow # switch portDisable to portEnable: 1. AC (Active State) © 2003 Brocade Communications Systems.1_FC101_2003 10 . IDLE (There are at least 2 idles between any 2 ordered sets) 7. LR (Link Reset) 4. LRR(Link Reset Response) 4. The proper chain of events /flow # Nx_Port starting from power on: 1. AC (Active State) 6. Revision0.Cont.Port State Machine Processes Link Control Protocols .Port State Machine . LRR (Link Reset Response) 5.
and LOGO The Fibre Channel Common Transport (FC_CT) protocol is used for registration and query services to the name server (FFFFFC) 11 Buffer I D L E I D L E R R D Y I D L E EOF t SOF n3 Buffer I D L E I D L E I D L E I D L E I D L E F R A M E I D L E I D L E (switch port) I D L E F R A M E I D L E N_Port A (server/storage/Work Station) © 2003 Brocade Communications Systems. the class of service associated with the frame (designated by the x in SOFix and SOFnx). PLOGI.3 and F) frame delimiters delimit. EOF) – in a Brocade Fabric (classes of service 2. non initiating frame within a class 3 sequence The EOF delimiter is used to indicate: if the frame is terminating the sequence (EOFt) or is a normal frame within the sequence (EOFn). and if the frame is the first sequence of a frame (SOFix) or a normal/ subsequent frame (SOFnx) SOFi3 indicates a start-of-frame. first sequence for a class of service 3 connection SOFn3 indicates a normal. Incorporated. and if Fabric between communicating devices detected an error (EOFni) EOFn indicates the end of a normal. Revision0. RSCN.FC Communication Overview After Speed Negotiation and PSM ACTIVE The Extended Link Service (ELS) protocol is used for the following Fabric services: FLOGI. primitive signals and primitive sequences There are at least two idles (primitive signal) between primitive signals at the source Data Words occur within the FC Frame content There are normally at least six idles (primitive signal) between any two FC Frames at the source Frame delimiters (SOF. non terminating frame within a sequence EOFt indicates the end of the last frame within a sequence EOFa indicates the sender of this frame aborted it EOFni indicates the Fabric between sender and receiver detected an error . SCR. if the sender of the frame aborts transmission of the frame (EOFa). or identify the start and end of frames The SOF delimiter is used to indicate: a start-of-frame.1_FC101_2003 F_Port EOF n SOF i3 Exchange Sequences contain two types of transmission words: Ordered Sets occur outside the FC Frame content Ordered set characters act as frame delimiters (see block arrows below).
. Revision0.1_FC101_2003 Extended Link Services provide a set of command instructions that are used to perform a unique task. Incorporated.Communication Protocols Established Fabric Communication Processes Overview ! 12 Extended Link Services include " " " " " " " FLOGI ACC PLOGI PRLI RSCN SCR LOGO Fabric Login Accept N_PORT Login Process Login Registered State Change Notification State Change Registration Logout © 2003 Brocade Communications Systems.
LIRP 7. LISA and in some cases LIRP and LILP ! Loop Devices " " " © 2003 Brocade Communications Systems.LIPA 4.LISM 2.LIFA 3.1_FC101_2003 Embedded Port (Domain Controller) is responsible for communication to/ from all Well-Known addresses The Fabric embedded port (domain controller) needs a Fabric address Brocade uses FFFCxx where xx represents one of 239 possible domains FFFCxx is used to PLOGI. Incorporated. LIPA. Revision0.PRLI and probe attached devices to retrieve information to put into name server data base – this is how private storage devices “appear” like Fabric devices in Brocade Fabrics Embedded port probing (Fabric probing) is enabled by default thus allowing private targets that accept PRLI into Fabric This Fabric probing can be disabled using the Brocade configure command The Loop Initialization Process that Arbitrated Loop devices would have already gone through include: LIPs 1. LIHA.LILP CLS (Loop Initialization Primitive Sequence) ( ( ( ( ( ( ( (Close) … Select Master) … Fabric Assigned) … Previous Assigned) … Hardware Assigned) … Software Assigned) … Report Position (if supported)) … Loop Position (if supported)) Initialization is complete – Now devices on the loop can log into the fabric! . LISM.LIHA 5. Established Fabric Communication Processes Overview ! 13 Fabric Devices typically " FLOGI # PLOGI to Name Server # SCR to Fabric Controller # Register & Query [using Fibre Channel Common Transport (FC_CT) Protocol] # LOGO Private NL: LIP (FFFCxx PLOGI and PRLI will enable private storage devices that accept PRLI and thus “appear” Fabric capable) Public NL: LIP -> FLOGI -> PLOGI -> SCR -> Register & Query -> LOGO and then PLOGI -> PRLI and communicate with other end nodes in the Fabric LIP process include: LIP. LIFA.Communication Protocols Cont.LISA 6.
Revision0.1_FC101_2003 The three most-common Well-Known Addresses and associated Fabric services are: FFFFFE is the address for Fabric F_Port Service often called “Fabric login” FFFFFD is the address for Fabric Controller Service FFFFFC is the address for Name Server Service .Fabric Service Communication Well-Known Addresses used in FC protocols are associated with Fabric Services ! Login 14 Services F_Port Service ! Fabric ! State Change Notification Services "A Fabric Controller Service ! The Fabric Name Server provides a data base of end nodes (Fabric devices and private targets that accept Switch PRLI) Well-Known address has associated Fabric service(s) ! Each © 2003 Brocade Communications Systems. Incorporated.
Login Services Three different levels of login: ! 15 Fabric Login (FLOGI) is used by an N_Port or NL_Port (Nx_Ports) to establish service parameters with the Fabric " The following information is implicitly captured and put into the Name Server during this process: type. The image pair service parameters are exchanged in ExLnk 0005 depicted below. it can be used to establish the environment between related SCSI processes on an origination Nx_Port and a responding Nx_Port © 2003 Brocade Communications Systems.1_FC101_2003 The Login services are used to exchange Service Parameters between N_Ports and N_Ports and between N_Ports and F_Ports. Incorporated. COS. PID. . port name (port WWN) . Revision0. A PRLI is used to establish ULP "image pairs" and service parameters for a pair or all processes that are to occur between the PRLI initiator and PRLI responder during this PRLI session. and node name (node WWN) ! N_Port Login (PLOGI) is used by one Nx_Ports to establish service parameters with another N_Port or NL_Port Process Login (PRLI) is used by an upper-level process in one port to establish image pairs and service parameters with the corresponding upper-level process in the other port " ! For example. A PRLI requests "image pair" establishment (EIP) and when bit 13 of ExLnk 0002 is set (as reported in capture depicted) both image pair establishment and service parameters between the image pairs are exchanged.
Revision0. Incorporated.1_FC101_2003 .Fabric F_Port Service Common Fabric Services Fabric “F_Port” Service is also referred to as Fabric Login " 16 You need the Fabric F_Port to establish Fabric communication capability FC devices must Fabric Login (FLOGI) to the Fabric F_Port to be part of the Fabric This FLOGI is sent to Well-Known Address FFFFFE do devices normally communicate with the Fabric F_Port? Devices will FLOGI upon initial connection to the Fabric Upon re-connection (possibly due to a link loss) !The Fabric F_Port accepts devices into the Fabric " " !When " " © 2003 Brocade Communications Systems.
then their address will be 0000pp. where pp = ALPA) FLOGI is required before any frame can be sent thru the fabric FLOGI is sent to well-known address FFFFFE (Fabric F_Port) © 2003 Brocade Communications Systems. . Revision0.Fabric Login (FLOGI) 17 FLOGI FFFFFE FLOGI FLOGI Accept ! ! ! When devices 1st connect their address is 000000 (unless they are loop devices. Incorporated.1_FC101_2003 Fabric Login is used by a Nx_PORT to establish a session with the fabric and is necessary before any frames can be sent thru the fabric.
1_FC101_2003 After the FLOGI has completed the HBA will send a session request to FFFFFC asking for permission to login and send a storage query. Revision0. .Port Login (PLOGI) 18 PLOGI FFFFFC SCSI Device SCSI Device SCSI Device Accept The server is asking permission to have a conversation with the Name Server (NS) in the switch © 2003 Brocade Communications Systems. Incorporated.
1_FC101_2003 Brocade provides a configurable feature that allows users to determine RSCN behavior of capable attached devices: Switch:admin> configure Configure. y. Incorporated. Revision0.State Change Notification Services ! 19 State Change Notification (SCN). look for an RSCN table).. no. n): [no] Zoning Operation parameters (yes..issued by the Fabric or an Nx_Port to devices that registered (issued an SCR requesting this notification) " " Not all devices handle RSCNs gracefully Brocade provides methods to help devices deal with RSCNs © 2003 Brocade Communications Systems. n): [no] RSCN Transmission Mode (yes. 2 = Fabric RSCN): (0. 1 = RSCN with multiple PIDs.State Change Notifications (SCN) are used for internal state change notifications. y. Fabric parameters (yes.. no. no. y. . n): [no] y End-device RSCN Transmission Mode (0 = RSCN with single PID. n): [no] Virtual Channel parameters (yes. not external " " This is the switch logging that the port is online or is an Fx_port This is not sent from the switch to the Nx_ports! ! State Change Register (SCR) – Nx_Port request to receive notification when something in the Fabric changes " FC Devices that choose to receive RSCNs must register for this service Devices send a State Change Registration (SCR) to FFFFFD Registration indicates that the device wants to be notified of changes Devices normally register immediately after a PLOGI to Name Server (NS) but could do so at any time after NS PLOGI " ! Registered State Change Notification (RSCN).2)  Brocade zoning also keeps RSCNs within the zone in most FOS versions (see notes on later page in the module. no. y.
okay? Automatic Network Notification To eliminate unnecessary polling traffic. Revision0. device removed. . call me when you have new hard disk. Whenever a change occurs in the Fabric (e. A device registers for a state change notification by transmitting a State Change Registration frame (SCR) to the well-known address of the Fabric Controller (FF FFFD). It is up to the device to query the Name Server (FF FFFC) to assess the state of the Fabric. etc…) RSCN will sent out to those devices that have registered. Who is using this notification services? Devices that use this service are generally servers that want to keep track of a number of storage targets.1_FC101_2003 RSCN is a mechanism defined in Fibre Channel to allow a Fabric to transmit an unsolicited notification about a “state change” to an N_Port that has registered (SCR) to receive it. This is a feature provided by the Fabric Controller to make the network more proactive and easy to manage. new device.g. Sample list of State Change Notification events: • Fabric Detected • An N_Port logs in or re-logs in with the Fabric • An N_Port implicitly logs out (goes offline) • A Fabric reconfiguration has occurred • N_Port detected • An RSCN is received from an N_Port The RSCN frame is simply a notification to the device that there has been a change in the network. link failure. Smart Fabrics use a notification mechanism to notify registered nodes of any changes in the Fabric © 2003 Brocade Communications Systems. the Fabric Controller transmits a Registered State Change Notification (RSCN) frame to the device. Incorporated.State Change Registration (SCR) ! 20 Devices register to receive change notifications using a SCR Fabric Controller provides a unique notification service to its registered nodes called a Registered State Change Notification ! Hi Fabric. And when there is a change in fabric topology. new switch.
Incorporated. 21 SCR FFFFFD Accept The server is registering to receive notification when something in the Fabric changes © 2003 Brocade Communications Systems. Revision0. .State Change Registration (SCR) Cont.1_FC101_2003 After the FLOGI has completed the HBA will send a session request to FFFFFC asking for permission to login and send a storage query.
1_FC101_2003 Fabric Controller (FFFFFD) service is a required logical entity within a Fabric that controls the general operation of the Fabric. Since Fabric Controller is such an important service. • Setup and tear down of Dedicated Connections • Perform general frame routing • Parse and routes of frames directed to well-known addresses • Generation of F_BSY (Fabric Busy) and F_RJT (Fabric Rejected) link responses The F_BSY indicates that the Frame can’t be delivered. because either the Fabric or the destination N_Port is temporarily busy. no transmission errors) but the Frame can not be received for some protocol-related reasons. therefore. It is the Fabric owner as well as the traffic controller. frame routing management. such as non-support of a service or inconsistent Frame header fields. Revision0. Functions include Fabric initialization. On receipt of an F_BSY in response to a Frame transmitted. Fibre Channel deploys a fully distributed environment for this service. The F_RJT response to a Frame indicates that delivery of that Frame is being “rejected”. there is no single point of failure. Major Fabric management responsibilities: • Execution of the Fabric initialization procedure • Advertise RSCN (Registered State Change Notification) Major traffic management responsibilities: F_Ports are interconnected by a routing function which managed by Fabric Controller and allowing frames to flow from one F_Port to another N_Port connect to a F_Port in the Fabric. Incorporated.But wait! There is more… Fabric Controller (FFFFFD) Mandatory service within Fabric ! 22 ! A Fabric shall contain one or more Fabric controllers Responsible for managing the operation of the fabric " Fabric Initialization " Fabric Configuration " Generate links response " Start & stop connections " Frame Routing Management Registered State Change Notification (RSCN) © 2003 Brocade Communications Systems. generation of link responses. and setup and tear down of dedicated connections. Recovery after retry is exhausted is dependent on the FC-4 ULP and the Exchange Error Policy. The Fabric Controller exists in every single Switch in a Fabric. up to some number of retries. the source N_Port is expected to attempt Frame retransmission.e. Rejection indicates that the Frame contents are intact (i. .
RSCNs are FOS dependent: . Revision0. Footnote 1 . topology changes and the SCR/SCN/RSCN processes. Incorporated.Fabric Controller Services The Fabric Controller (FFFFFD) Service alerts devices that changes have occurred in the Fabric by sending a Registered State Change Notification (RSCN) if 1: " " " " " " 23 Device registered to receive RSCN using a SCR A new device has been added An existing device has been removed A zone has been changed A switch name or IP address changed The Fabric reconfigured ! Registration is optional " " SCSI initiators normally register SCSI targets do not register © 2003 Brocade Communications Systems.1_FC101_2003 The Fabric Controller is responsible for routing changes.
Hunt Groups. Try not to confuse these server services with the services offered by FC-3 Common Service Layer.1_FC101_2003 The Fibre Channel Name Server is responsible for directory information about Fabricconnected devices. FC-3 layer focuses on multiple N_Ports services like Multicasting. All associated Name Service information is deleted from the directory automatically when a switch or device goes down (deregisters). Quick note: All the built-in network server services are part of the Fibre Channel Generic Services (FC-GS) specification. . it discovers other devices with a single Name Server query) • Up-to-date device information. Fibre Channel Name Server Characteristics: • No single point of failure • Each Distributed Name Server maintains and “owns” local information and retrieves remote information from other Distributed Name Servers using Server-to-server protocol (based on FC-CT) • Server-to-server communication is transparent to the external Name Service client • A distributed Name Server may cache remote information for a period of time (900 seconds) • Fast. Revision0.The Fabric Name Service Fibre Channel Name Server ! 24 ! ! ! ! Name Service is part of the FC_GS specification Responsible for directory information about Fabric connected devices Every device must registered to the Name Server when it is in network Name Server uses FC-CT (Common Transport) protocol to distribute information throughout the Fabric (dynamic scalability) Fibre Channel device can query any Name Server for network resources information (simple request and response model) Name Server is like TCP/IP Domain Name Server (DNS) service © 2003 Brocade Communications Systems. simple yet effective. maintains name and address information about Fabric-connected devices in a complex. Name Server. diverse environment. Upon registration. Incorporated. the Name Server has essential device information that is available immediately. etc. efficient device discovery (rather than a server device manually polling for other devices sequentially.
The Name Server maintains this information. Since every device registers itself with the Name Server during the login procedures. http://www. Devices are identified by names. • The Name Server is distributed throughout the Fabric. The Name Server has no responsibility for the process of routing data among devices. as well as deregistration. It is the job of a DNS to translate this English name to a native IP address which is the real address destination of that website. When a query requires remote information. Incorporated. the Distributed Name Server will communicate with other Name Servers within the Fabric on behalf of its local device. The Name Server will translate the target device’s World Wide Name (64-bits) and return its actual dynamic native Fibre Channel address (24-bits).Fabric Name Server versus TCP/IP DNS Functionality Distributed Name Server (FFFFFC) TCP/IP DNS Server (Socket 53 UDP) 25 Automatic Server location targeting Dynamic member registration Support detail member characteristics description Support full or restricted queries Automatic replication database Multi-mastering (full read and write update) in the entire network Automatic unify resources view Yes (Well-Known) Yes (via DHCP) Yes (Dynamic DNS) No (IP and Name only) No (partial query only) No (manual zone transfer) No (Primary & Secondary) No (manual work) Yes Yes Yes Yes Yes Yes © 2003 Brocade Communications Systems. • Maintain Critical Member Information. Meaning if a server sends a query to the Name Server. and other attributes. other devices can use the Name Server as a central query point. • Provide automatic registration of essential device information on behalf of the devices.com. • Each Fibre Channel switch contains a Distributed Name Server that maintains local information. addresses.1_FC101_2003 What is a Domain Name Server (DNS) in TCP/IP? When we are surfing the Internet.brocade. Revision0. The DNS helps us locating the actual target address. There can be a multiple of switches and whichever switch is local to the host will serve as the Name Server for the fabric for that host (both read and write). acting like a telephone directory. we want to go to an destination website such as. Like DNS Server. This behind-the-scene communication is transparent to the local device. one of the key functions of the Fibre Channel Name Server is to help device finding the actual address of the target device. that local Name Server will query the rest of the switches and then respond to the host with all of the fabrics information that corresponds to the query that came to it. The Name Server provides local devices with an access to the Name Server. .
FC devices may use Name Server (FFFFFC) to “discover” other devices in the Fabric " " " 26 Devices always login to the Fabric. FLOGI to FFFFFE Devices then login (PLOGI) to FFFFFC After PLOGI. Some legacy devices may have a list of devices (PIDs or WWNs) they communicate with. devices typically register and send queries to NS at FFFFFC. after FLOGI After receiving an RSCN from the Fabric Controller © 2003 Brocade Communications Systems. Query request information returned from the Name Service is used to build a device table data base. devices can also deregister ! When " " do devices communicate with FFFFFC? Upon initial connection. Revision0.1_FC101_2003 SCSI initiators typically communicate with the name server to register information about them selves and to determine who they can communicate with in the Fabric. Devices that attempt to communicate with other Fabric devices not in queried Name Server data base are called bad citizens.The Fabric Name Service Cont. Attributes include: • Type (N or NL) • COS (Class of Service device supports) • PID (Port identifier also called 24-bit address) • Port WWN also called port name • Node WWN also called node name • FC-4 types this device is capable of communicating [08=SCSI (FCP) and 05 = IP over FC] • Symbolic device manufacturer information that device registers . Incorporated. The Name Server keeps a list of attached devices and their attributes.
Revision0.Registration and Query Processes 27 Register Accept FFFFFC Register Accept SCSI Device SCSI Device SCSI Device Register Accept Query * Accept includes List of SCSI devices ! A PLOGI to the Name Server is required before devices can register or query " " Typically. .1_FC101_2003 Fabric Login is used by a Nx_PORT to establish a session with the fabric and is necessary before any frames can be sent thru the fabric. Incorporated. SCSI initiators usually query for a list of allowed communication devices so they can build or rebuild device tables (see Query * above) ! Recall that registration and query processes use the Fibre Channel Common Transport (FC_CT) protocol © 2003 Brocade Communications Systems. both SCSI initiators and targets register Additionally.
When the “SCSI probe/inquiry” reply is received the HBA will assign network address a TARGET/LUN number and present this to the operating system. . Incorporated. The Name Server at FFFFFC will match the host WWN to the storage targets currently in the fabric and will reply back the network address of the match targets. The HBA must perform a FLOGI to the well known address FFFFFE. When the address is acquired the HBA will start a session with the well known address FFFFFC followed by a query to FFFFFC.SAN Attached Initiators and Target Initiator Communication Process Review ! 28 Fabric Initiators are typically server HBAs that will: " " " " FLOGI to FFFFFE and obtain 24-bit Fabric address PLOGI to FFFFFC SCR to FFFFFD Register and query FFFFFC for database of Storage/Tape targets $ FFFFFC matches HBA to zones (Fabric subsets containing devices you“allow” to communicate) and returns 24-bit addresses of targets that HBA requested within authorized zones PLOGI and then sends SCSI probes to fabric destination addresses of targets returned from Name Server $ " Assign target ids (with server OS) to each node probed to build a device table that applications can use to store and retrieve data © 2003 Brocade Communications Systems. Revision0.1_FC101_2003 With Storage Area Networks pure SCSI initiators were replaced with a HBA. The HBA will create a new frame to each destination address it received and encapsulate a PLOGI followed by a “SCSI probe/inquiry” to the network address of the target. With networked storage the HBA must connect to the fabric before storage can be probed.
. Revision0. Incorporated.Initiators PLOGI to Target(s) 29 PLOGI PLOGI Accept Accept The host is asking to have a conversation (PLOGI) with one of the SCSI target devices in preparation probe for information © 2003 Brocade Communications Systems.1_FC101_2003 The HBA will logout of FFFFFC and send a PLOGI to each network address returned from FFFFFC.
Revision0.Initiator PRLI to SCSI probe/inquiry 30 PRLI PRLI Once a host has established a conversation (PLOGI).1_FC101_2003 During the Process PRLI process the HBA will send the SCSI probe/inquiry to learn information on the LUN. it can now PRLI and then probe for information (An example of a host probing for information would be asking a target SCSI device if there are any LUNs to report) © 2003 Brocade Communications Systems. . Incorporated.
Incorporated.SAN Attached Initiators and Target Cont. Loop JBODs (just a bunch of disks) and Tape Non-loop Fabric capable Targets typically: " " " ! FLOGI to FFFFFE and obtain full 24-bit Fabric address PLOGI to FFFFFC Register information with FFFFFC $ Targets typically register symbolic node information that allows easy identification using Name Server commands (nsshow) ! Fabric capable loop targets typically: " Go through LIP initialization to obtain AL_PA and then FLOGI to obtain the other 16-bits of Fabric address PLOGI and register with FFFFFC © 2003 Brocade Communications Systems.1_FC101_2003 " . Target and Loop Communication Processes Review ! 31 SAN Targets are typically storage devices: Fabric RAID controllers. Revision0.
Incorporated.PRLI and probe attached devices to retrieve information to put into name server data base – this is how private storage devices “appear” like Fabric devices in Brocade Fabrics Embedded port probing (Fabric probing) is enabled by default thus allowing private targets that accept PRLI into Fabric • " This Fabric probing can be disabled using the Brocade configure command © 2003 Brocade Communications Systems.Embedded Port Communication Well-Known Addresses are associated with Fabric Services Embedded Port (Domain Controller) is responsible for communication to/ from all Well-Known addresses ! ! 32 The Fabric embedded port (domain controller) needs a Fabric address Brocade uses FFFCxx where xx represents one of 239 possible domains " FFFCxx is used to PLOGI.1_FC101_2003 . Revision0.
Revision0.1_FC101_2003 After the FLOGI has completed the HBA will send a session request to FFFFFC asking for permission to login and send a storage query. . Incorporated.Embedded Port Login (PLOGI) 33 PLOGI Accept PLOGI FFFCxx Accept PLOGI Accept The embedded port is asking permission to have a conversation with each device in the Fabric © 2003 Brocade Communications Systems.
. Revision0. Embedded Port goes on to query for NS information © 2003 Brocade Communications Systems.Embedded Port Process Login (PRLI) PRLI establishes ULP handshaking so that Embedded Port can probe to get information to put into the the Name Server 34 PRLI & NS query NS Info PRLI FFFCxx PRLI Accept Reject NS query NS Info SCSI initiators commonly reject PRLI SCSI targets typically accept PRLI If PRLI accepted. Incorporated.1_FC101_2003 The HBA will send a name server query to FFFFFC inquiring for the network addresses of targets it can communicate with.
Established ELS and FC_CT Processes Allow data to flow 35 R_RDY R_RDY Data Data Each R_RDY allows ULP data frames to flow Even private storage devices that accepts embedded port PRLI are allowed in! The Brocade embedded port probing enables automatic FL_Port translation of private (8-bit) communication to/from Fabric (24-bit) communication 1 © 2003 Brocade Communications Systems. . It is automatically enabled but it can be disabled. Footnote 1: This automatic translation capability is a configurable capability called “translative mode”. Revision0. Incorporated.1_FC101_2003 The HBA will logout of FFFFFC and send a PLOGI to each network address returned from FFFFFC.
Incorporated.1_FC101_2003 .Summary ! 36 FC communication includes switch. Revision0. PRLI and probe processes ! ! ! © 2003 Brocade Communications Systems. device and FC protocol initialization and communication processes Common Fabric services include Login services. and register with FFFFFC Private storage devices that accept PRLIs are put into the Fabric Name Server using Brocade embedded port PLOGI. PLOGI to FFFFFC. SCR to FFFFFD. Fabric controller services to FFFFFD and Fabric Name Services to FFFFFC When Fabric capable loop devices (public loop) connect to a Fabric they will LIP and get an AL_PA and then FLOGI to FFFFFE. Fabric F_Port service to FFFFFE.
Find SAN ED 101 link “SAN Fabric Foundation”. It is a comprehensive FC offering. The FC course that they teach spends about a day on FC-2 layer communication processes alone. Revision0. look at chapter 2 (Fibre Channel Essentials) for additional information about material presented in this module Use resources page Internet Link section – Find Solution Technology link. Incorporated. look for Fibre Channel courses Use resources page Reference section . .Additional Information 37 Use resources page Internet Link section .FC Glossary and additional reading / WEB site locations list © 2003 Brocade Communications Systems.1_FC101_2003 Solution Technology provides detailed Fibre Channel training related to materials presented in this module.
Brocade Product Training Fibre Channel Theory Fundamentals Summary 1 © 2003 Brocade Communications Systems. Incorporated. Revision0.1_FC101_2003 .
Revision0.1_FC101_2003 . and addresses Describe FC services and expected behaviors Summarize and state relevance of FC theory fundamentals learned ! ! ! ! © 2003 Brocade Communications Systems.Course Objectives Check This course was designed to provide you with the ability to: ! 2 Describe key reasons. Incorporated. terminology. benefits and components related to Fibre Channel (FC) Identify FC protocol layers and key related tasks & components List key FC topologies.
Benefits & Components ! 3 FC is the dominant SAN protocol. Incorporated. it provides " " " " " High speed Low latency Long distance Robust data integrity Large connectivity ! FC value adds: " Server & storage consolidation " Centralized management " Increased utilization " Unmatched availability " Performance and scalability FC SAN components " ! " Hardware • Interconnecting Devices • Storage • Servers/HBAs Software • SAN Management Applications © 2003 Brocade Communications Systems. Revision0.1_FC101_2003 .Key FC SAN Reasons.
10 Mbaud speed. FC – 1 layer contains specifications for 8b/10b encoding. although not depicted. FC – 0 specifications include information about feeds and speeds. Revision0. while FC-2 level concerns itself with the definition of functions with a single port. (2) how much data should be sent at one time before sending more (flow control). . This includes: (1) how to divide the data into smaller frames. ordered set and link control communication functions. FC – 3 defines advanced features such as striping (to transmit one data unit across multiple links) and multicast (to transmit a single transmission to multiple destinations) and hunt group (mapping multiple ports to a single node). The FC-3 level deals with functions that span multiple ports. Incorporated. So. which define different implementations that can be selected depending on the application. It also includes Classes of Services.1_FC101_2003 FC . or ATM. etc. such as IP or SCSI. FC – 2 specify content and structure of information along with how to control and manage information delivery.0 and 1 layers specify physical and data link functions needed to physically send from one port to another.FC Protocol Layers Key related tasks & components ! 4 ! ! ! ! FC-4 ULP mapping FC-3 Advanced features FC-2 Framing and flow control FC-1 Encoding & link control FC-0 Speeds and Feeds (SFPs & cable specs) © 2003 Brocade Communications Systems. FC – 4 provides mapping of Fibre Channel capabilities to pre-existing protocols. This layer contains basic rules needed for sending data across network. and (3) where the frame should go. is also in development.
Using the entire middle byte for the port area designator allows Brocade switches to scale up to the Fibre Channel standard maximum of 256 ports per switch. Switched fabric can theoretically allow 16 million nodes to talk (16^6 – There are 6 Port Identifier (PID) slots with 16 hex choices per slot)). Without a switch only two of these devices can talk at a time.x and v3. The embedded port will take one AL_PA. To ensure inter-operability between Fabric OS v4. When Brocade developed the ASIC for the SilkWorm 2000 series. Area address and AL_PA fields respectively. Well-Known addresses and FC services Fabric addresses (PIDs) are 24-bit domain. so only half of the second byte in the Area field of the PID was required to specify ports. Each field can provide 00-FF addressing. The 24-bit port address format consists of three bytes defining the Domain identifier. where Y was a hexadecimal number that specified a particular port on a switch and “1” was constant. Nodes. This is commonly known as the Core Switch PID format setting. port names (Port WWN). The three byte fields of the old PID format were defined as XX1YZZ. so on a Brocade switch port there are 125 available AL_PAs. a setting was created to enable the PID format to be set to use either the new format or the old format. The Domain ID field byte provides domain addressing 1-239.FC Topologies. Terms and Addresses ! 5 Three kinds of Fibre Channel Topologies: ! FC terms include Fabric. Ports. while maintaining compatibility with older firmware versions. To support the increased port count on the higher port count products based upon Brocade Fabric OS v4. Arbitrated Loop is limited to 126 devices in a blocking architecture (plus one for FL_Port). Revision0.1_FC101_2003 ! Point-to-Point is limited to two devices but they can talk at greater distances than SCSI allows. Incorporated.x and v3. area and ALPA designators than enable cut-through routing © 2003 Brocade Communications Systems. Fabric Addressing The PID format on switches running Fabric OS v2. the new format XXYYZZ has been adopted.x based products.x could originally only support a maximum of 16 ports in one switch. where “YY” represents a port area designator. An arbitrated loop attached to a switch allows queuing into and out of the port where the loop is attached. all others are “blocked” until those two are done. The committee reserves ½ million of these addresses for well known addresses and testing purposes. node names (Node WWN). .x based products and Fabric OS v2.x. the largest switch has 16 ports.
Revision0. Services & Expected Behavior ! FC 6 communication methods include the following terminology.1_FC101_2003 . Incorporated. WWN and Fabric Addresses) Device Initialization and Link Communication Well-Known address Fabric communication services " " ! Expected " " " Fabric communication behavior review: What happens when a Fabric device connects to a Fabric? What happens when a public loop device connects to a Fabric? How do “private” devices communicate in a Fabric? © 2003 Brocade Communications Systems. processes and services: " " Switch Initialization FC Communication Terminology – Connection Naming (Port type.FC Terms.
Incorporated. . Revision0.Conclusion: Fibre Channel and SAN Fibre Channel Super SCSI 7 = SAN Advanced Layers Architecture Comprehensive Upper-level Protocol Mapping Smart Low-level Operation Control (hardware-based) © 2003 Brocade Communications Systems.1_FC101_2003 In this course we have gone through a series of key Fibre Channel technologies and we have demonstrated many features that make Fibre Channel the prefect solution for today SAN implementation.
This action might not be possible to undo. Are you sure you want to continue?