Fibre Channel Classes of Service

for Data Transport

Kumar Malavalli
kumar@brocadecomm.com
www.brocadecomm.com

Abstract
The Fibre Channel ANSI Standard defines multiple classes of service to provide different
modes of data transport within a Fibre Channel system. These Classes are independent of
topology. In this document, only Fibre Channel Class-1,-2, and -3 service are discussed. Each
service meets different data transfer and delivery requirements and suits various user
application environments. The choice of a particular class of service depends on user traffic
patterns and loads, the transport protocol used by the application, and performance and
configuration requirements. Brocade Communications Systems, Inc. (Brocade), designed
the SilkWorm switch based on Fibre Channel Class-2 and -3 service to build Fibre Channel
Fabrics. Influential factors in selecting these classes are the requirements for applications and
the data transport protocols used in todays server-storage network.

 1997 Brocade Communications Systems, Inc.

ALL RIGHTS RESERVED

Order number: 0015-WP02-1

The Brocade logotype design is registered with the U.S. Patent and Trademark Office as a trademark of Brocade
Communications Systems, Incorporated. Brocade and SilkWorm are registered with the U.S. Patent and
Trademark Office as trademarks of Brocade Communications Systems, Incorporated.
All other brands, products, or service names are or may be trademarks or service marks of, and are used to
identify, products or services of their respective owners.
Notice: This document is for informational purposes only, and does not set forth any warranty, express or implied,
concerning any equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the
right to make changes to this document at any time, without notice, and assumes no responsibility for its use. This
informational document describes features that may not be currently available. Contact a Brocade sales office for
information on feature and product availability.
Export of technical data contained in this document may require an export license from the United States
Government.

Brocade Communications Systems, Incorporated

1901 Guadalupe Parkway
San Jose, CA 95131

T 408.487.8000
F 408.487.8101

info@brocadecomm.com

http://www.brocadecomm.com

Fibre Channel Service Classes

In Fibre Channel the information interchange is structured hierarchically and common to all classes of
service. This hierarchical constructs are Frames, Sequences and Exchanges. A frame is the atomic unit
of data transmission between the communicating devices (N_Ports). A Sequence is a set of one or more
related data frames transmitted unidirectionally from one device to another device within an Exchange.
An Exchange is the basic construct consisting of a set of one or more related Sequences for
coordinating the information interchange between the communicating devices during upper layer
protocol such as SCSI and TCP/IP.
The Fibre Channel FC-2 construct, which serves as the data transport mechanism, defines the following
key mechanisms to achieve efficient information transfer, in different classes of service:
A means to provide confirmation of delivery of valid frames at the destination device through
Acknowledgment frames. Depending on the application, one of the following methods of
Acknowledgment can be deployed:
1. ACK_1: Acknowledgment of every frame received
2. ACK_N: Acknowledgment of a group of nconsecutive frames received
3. ACK_0: Acknowledgment of every sequence received
A flow control handle to pace the flow of frames (i) end-to-end between devices (N_Ports) and (ii)
buffer-to-buffer (link level), between a device (N_Port) and the intervening Fabric (F_Port) to prevent
data overrun at the receiver. A credit based flow control is deployed. The credit is the number of
receive buffers allocated to a transmitting port (N_Port or F_Port). The following are the frame
messages to implement flow control for different classes of service
1. End-to-End:
ACK_1: For every frame delivered
ACK_N: For a group of n consecutive frames delivered
2. Buffer-to-Buffer: R_RDY (Receiver Ready)
An error handling means both between the device and the Fabric and end-to-end between the
communicating devices. This includes frame messages to indicate whether a frame has been busied or
rejected, with reason, either by the Fabric or the destination device as follows:
1. Link level:
F_BSY: Busied by the Fabric
F_RJT: Rejected by the Fabric with reason
2. End-to-End:
P_BSY: Busied by the destination device
P_RJT: Rejected by the destination device

Brocade Communications Systems, Inc.

Fibre Channel Classes of Service for Data Transport

Class-1 Service
Fibre Channel Class-1 service is connection-oriented and based on a circuit-switched mode of
operation, where switching is performed between connections. Class-1 provides a dedicated connection
with the full bandwidth guaranteed between two communicating devices (Figure 1).

A B C

D E

G H

Device 1

Device 3
D E

A B C

G H

Device 2

Device 4

Bidirectional frame streams on dedicated

connections in a Class-1 Fabric
Figure 1. A Class-1 Fibre Channel system
Before communication can begin, a bidirectional connection between the source and the destination
devices must be established by the intervening switch. Each frame or group of frames is acknowledged
for delivery or nondelivery to the destination device (Figure 2a). This delivery acknowledgment is also
used to manage end-to-end flow control from the source device to the destination device. In addition,
there is support for messaging to indicate whether a frame has been busied or rejected, with reason, by
the destination device once the connection is established
(Figure 2b). However, no flow control, error checking, or busy indication exist at link level between the
device and the switch.

DATA

DATA

FABRIC

FABRIC

Acknowledge
Source

Destination

Source

Reject or
Busy

Acknowledge: Flow
control between the devices

Figure 2b. Class-1 Error Handling

Figure 2a. Normal Class-1Transaction

In Class-1 service, connection management within the switch must prevent a deadlock that might be
caused by a third device requesting connection with one of the two devices already in a connection. A
three-way unidirectional connection among three devices is thereby created. This is achieved by
ensuring that the connection is not confirmed until the originating device that requested the connection
receives an acknowledgement from the destination device on the return path, signifying connection
acceptance. Thus, every time a Class-1 connection is established, there is a penalty of round-trip
connect latency.
Brocade Communications Systems, Inc.

Fibre Channel Classes of Service for Data Transport

When a device sends a connection request to the Fabric, there is no guarantee that the connection will
be granted if the destination device is busy communicating with another device. The connection request
may be stacked (queued) within the Fabric until the destination device has terminated the connection
with the previous device and is free for a new connection. There may be other devices waiting in the
queue ahead of this device. Once a connection is established between two devices, no other device can
communicate with either of them.
In Class-1 there is a possibility of one device monopolizing a connection, thereby reducing the
bandwidth available to other devices. Hence, this class of service is not suitable if multiple devices
need to communicate with multiple other devices, such as in client-server configurations or when
storage is shared by multiple servers.
Class-1 builds a bidirectional circuit between two ports. Data frames and acknowledgement frames can
flow in both directions simultaneously. The ability to transmit multiple frames is controlled by the endto-end flow control rules. The higher the end-to-end credit, the more work can be done and better
utilization of the dedicated bandwidth. However with credit of one, normally each frame must be
acknowledged before sending the next frame.
The main problem with the Class-1 service is that a source device, with no Class-1 data ready wastes
the bandwidth on the dedicated connection with the destination device. To overcome this deficiency,
Class-1 service needs the assistance of the Intermix option. Intermix is a special feature of Class-1
service that allows interleaving of possible Class-2 or -3 data frames or high priority Class-2 and -3
control frames on an established Class-1 connection, one frame at a time, when the connection is idle.
To take advantage of this function, both source and destination devices engaged in the dedicated
connection, and the intervening Fabric must support the Intermix feature. However, implementation of
Intermix requires extra switch resources. In a Fabric of cascaded switches, tracking and managing the
Intermix frames at different switch stages is very difficult.

Brocade Communications Systems, Inc.

Fibre Channel Classes of Service for Data Transport

Class-2 Service
This service is connectionless and based on a frame-switched mode of operation where switching is
performed for each frame. In Class-2, frames can be multiplexed between a source device and one or
more destination devices (Figure 3).

A B

1 2 3

4B

2
4

Device 3

3
A

Device 1

# 2

A 5

FABRIC

4 5

B 4

1 3
Device 4

Device 2
Bidirectional multiplexed frame streams with no dedicated
connection in Class-2 and -3 Fabric
Figure 3. A Class-2 and -3 Fibre Channel system

The Fabric examines the destination address of each frame before forwarding the frame on a path
toward the destination device. Flow-control support is provided for both end-to-end, between the
communicating devices, and at link level between the Fabric and communicating devices. Individual
frames or group of frames are acknowledged for successful delivery by the destination devices (Figure
4a). Also, to facilitate error handling, there is support for messaging to indicate whether a frame has
been busied or rejected, with reason, either by the Fabric or the destination device (Figure 4b). This
low-level flow-control and error-handling tools of Class-2 service offer high data reliability and
integrity, which is mandatory in business-critical applications.

DATA
Ready

DATA
FABRIC

Ready

Ready

Acknowledge Destination

Source

Ready

Ready

Reject or
Busy

Ready

Ready: Flow control between

the Fabric and the device
Acknowledge: Flow control
between devices

Figure 4a. Normal Class-2 transaction

Brocade Communications Systems, Inc.

Source

FABRIC

Ready

Reject or
Busy
Ready

Figure 4b. Class-2 error handling

Fibre Channel Classes of Service for Data Transport

Multiplexing data to several destinations allows very efficient use of the link and Fabric bandwidth. In
server-storage network environments, the existing applications require multiplexing large amounts of
data with many small control (handshake) frames. The frame-switched communication that is offered
by Class-2 service facilitates this multiplexing very efficiently. The ability to multiplex a large number
of frames between the devices through the Fabric yields high bandwidth utilization. As a result, large
systems need fewer components, such as servers, to run the user applications leading to lower system
cost.
In a connectionless Fabric with a mesh topology of cascaded switches, each frame may be forwarded
via a different path. Hence, frame delivery to the destination device may not be in the same order as
transmitted by the source device (Figure 5a). The Brocade Fabric, on the other hand, guarantees inorder delivery of frames by assigning a single path between the communicating devices during the pathselection phase of the Fabric initialization (Figure 5b). In many large systems, the benefits of
multiplexing in realizing increased bandwidth utilization can occur only if in-order delivery by the
Fabric is guaranteed. This is because many large systems use virtual memory and a scatter-and-gather
mechanism to find out where the data payload can be placed in main storage. Without in-order delivery,
the scatter-and-gather list must be scanned for every frame, which is detrimental to performance.

FABRIC

1 2

Switch

FABRIC

Switch

1 2 3

Switch

Switch

1
2

1
3

3
Switch

Sitch

1
3

Figure 5a. Out-of-order frames in a generic Class-2 and -3 Fabric

Figure 5b. In-order frames in a Brocade Class-2 and -3 Fabric

Another advantage of the connectionless communication mode is the support for speed matching
between devices with different transmission speeds. By utilizing switch buffers, communication is
possible between todays 1-gigabit-per-sec (Gb/s) device ports and future 2-Gb/s and 4-Gb/s device
ports. This feature protects todays investment by establishing a good migration path.

Brocade Communications Systems, Inc.

Fibre Channel Classes of Service for Data Transport

Class-3 Service
Class-3 service is also a connectionless service based on frame switching. Unlike Class-2, it is a
datagram service, which does not provide confirmation of delivery or nondelivery of the frames at the
destination devices. The confirmation of delivery is delegated to the upper-level protocol, such as TCP
(Transmission Control Protocol), which manages its own end-to-end flow control and data integrity.
Only link-level flow control exists between the Fabric and each connected device (Figure 6). There is
no acknowledgment of a frame or notification that a frame has been rejected or busied by the Fabric or
by the destination device.

DATA
FABRIC
Source

Ready

Ready

Destination

Ready: Flow control

between the Fabric
and the device

Figure 6. Class-3 transaction

During heavy congestion in Class 3, the Fabric may occasionally discard some frames. However, the
lost frames can be recovered by retransmission of the sequence in which one or more frames have been
lost. The upper-level protocol (such as TCP) is responsible for this type of recovery. The prime
advantage of Class 3 for Multicast Service is precisely the absence of frame-delivery
acknowledgements, rejects, and busy indications from the destination devices. When the source device
sends frames to the members of a multicast group during a multicast transmission, confusion would
result if all the members were to send back acknowledgment frames. In this scenario, the bandwidth
available for transmission of user data would also decrease, resulting in reduction of the Fabrics
aggregate throughput. The absence of acknowledgement frames in Class-3 also eliminates round-trip
delays during information transfers between devices.

Brocade Communications Systems, Inc.

Fibre Channel Classes of Service for Data Transport

Fabric Delays In Different Classes of Service

In Class-1, the connection set-up delay consists of a connection set-up time in each switch in the endto-end data path between the communicating devices and the time it takes for the return confirmation
message to indicate that the connection has been accepted by the destination device (Figure 7). For
transmission of small-to-medium sized data blocks, this delay constitutes a significant proportion of the
overhead. The delays associated with the queuing of Connection Request in the switches are not
considered.
Class-1 service is ideal for transferring very large amounts of data between two devices for a sustained
period of time. Class-1 is also suitable when a constant transfer latency is required, and when the
connection-establishment and tear-down latencies are significantly small in relation to the data transfer
time. For transfers of data blocks among multiple devices, as in storage and transaction applications, a
single device cannot afford connection latency. This leads to very poor link- and Fabric-bandwidth
utilization.

Source
Device

CR

Connection
set-up time within
the switch

Information Block

Source
Switch
CR

FC

CC

Intermediate
Switch

Destination
Device

CR: Connection
Requests
CC: Connection
Confirmation (Ack)
CA: Connection
Acceptance

CR
Destination
Switch

Frames

FC: Final
Confirmation (Ack)
CA
TIME

Connection Set Up Delay

Transmission Delay
Total Delay

Figure 7. Class-1 Fabric delays

In the absence of connection set up in Class-2 and -3, each frame is examined within each switch in the
data path between the communicating devices, for the address information for routing between the inbound port and the out-bound port of each switch. (Figure 8). The latency incurred due to this routing
decision time can be small if the cut-through routing method is employed, such as in the Brocade
Fabric. Cut-through routing allows routing of a frame to occur from the in-bound port of a switch to
the out-bound port of a switch as soon as the frame header with the address information becomes
available. The absence of connections in Class-2 and -3 Fabrics will eliminate the extra overhead for
connection set-up time thus increasing the bandwidth utilization and decreasing the total delay.

Brocade Communications Systems, Inc.

Fibre Channel Classes of Service for Data Transport

Source
Device

Information Block

Source
Switch

2 3

Frames

Intermediate
Switch

2 3

4
FC

FC: Final
Confirmation (ACK)

Destination
Switch

Cut-Through
Frame Routing

Destination
Device

2 3

4
TIME

Total Delay (Class 3)

Total Delay (Class 2)
Figure 8. Class-2 and-3 Fabric delays

The Effect of Classes of Service on Cascading Switches in a Mesh Fabric

A Class-1 switch is not suitable for a mesh Fabric with cascaded switches. For example, Figure 9
illustrates the bottleneck created by an Inter-Switch Link (ISL) between two switches, engaged in a
connection between Devices 1 and 3. In this set-up, device 2 on one switch is blocked from
communicating with Device 4 on the other switch, even though both devices are free to communicate.

Ded
icat

Connectio
nless

Figure 9. Class-1 blocking in

cascaded Fabric

Figure 10. Class-2 or -3

non-blocking in

When a Class-2 or -3 switch is used with a mesh Fabric of cascaded switches, the ISLs are not a part of
any dedicated connection between the devices. Although the ISLs may experience some congestion, as
shown in Figure 10, there is no blockage of information flow to the free devices. Information from
Devices 1 and 2 can be multiplexed on the ISLs for delivery to Devices 3 and respectively 4, and vice
versa.

Brocade Communications Systems, Inc.

Fibre Channel Classes of Service for Data Transport

Brocade further reduces congestion on the ISLs by deploying a technique where the frames can be
segregated into separate frame flows or virtual channels (VCs) with a priority scheme. On a single ISL,
there are eight virtual channels available, with four priority levels and individual queuing mechanisms
for each VC. These virtual channels can be used to tune the Fabric for different applications by
prioritizing frame traffic and reducing congestion on the ISLs. Virtual Channels can separate data flow
between the following types of frames:

Data or interswitch control and signaling frames

Class-2 or -3 frames

Unicast or multicast frames

The frames on the ISLs are subject to independent flow control between the adjacent switches per VC.
ISL
VC 1
VC 2
VC 3
VC 4

Brocade

VC 5
Brocade Switch

VC 6

Brocade

Brocade Switch

VC 7
VC 8
8 Virtual Channels (VCs) with 4 priority levels

Figure 11. Congestion management between Brocade Switches

Brocade Communications Systems, Inc.

Fibre Channel Classes of Service for Data Transport

Fibre Channel Service Classes Summary

Table 1 summarizes the characteristics of Fibre Channel Class-1, 2, and 3 services. Table 2 summarizes
the advantages and disadvantages of each of these services for server-storage network applications.
Table 1. Characteristics of Classes of Services 1, 2 and 3
Service Characteristics

Class-1 Service

Class-2 Service

Class 3 Service

Communication mode

Connection-oriented,
circuit-switched

Connectionless, frameswitched (reliable)

Connectionless, frameswitched (datagram)

Flow control between the

communicating devices (end-toend)

Acknowledge:
ACK-1 each frame

Acknowledge:
ACK-1 each frame

Not provided

Flow control between the Fabric

and the device

Not provided (except for

the connection request)

Receiver Ready:

Confirmation of delivery from the

destination device to the source
device

ACK_1: Each frame

ACK_1: Each frame

ACK-N each nth frame

th

ACK-N each nth frame

Receiver Ready:

R_RDY

R_RDY

Not provided

th

ACK_N: Each n frame

ACK_N: Each n frame

ACK_0: Each sequence

ACK_0: Each sequence

Confirmation of nondelivery from

the destination device to the source
device

Frame Rejected: P_RJT

Frame Rejected: F_RJT

Frame Busied: P_BSY

Frame Busied: F_BSY

Confirmation of nondelivery from

the Fabric to the device

Not provided (except for

the connection request)

Frame Rejected: P_RJT

Broadcast and multicast support

Not supported

Not supported

Supported

In-order delivery of frames

Guaranteed

Fabric-implementation
dependent

Fabric-implementation
dependent

Brocade Communications Systems, Inc.

10

Not provided

Not provided

Frame Busied: P_BSY

Fibre Channel Classes of Service for Data Transport

Table 2. Advantages and Disadvantages of Class-1 Service

Class of Service
Class-1

Advantage

Disadvantage

Users and applications on a dedicated connection

can interact and engage without others interfering.

Connection between the devices must be

established before information transfer can begin,
thus increasing the latency.

The connection is not terminated until both parties

are satisfied that all information has been correctly
transmitted.

The path for end-to-end connection, consisting of

individual links and switches, must be available at
full bandwidth for the full duration of the entire
transfer.

The data transport is ideal for information that

needs to flow at a constant rate for extended
periods of time.

During connections, bandwidth is wasted when

the links are idle while devices are performing
other tasks.

Guaranteed in-order delivery of frames in any

The time to set up and terminate a connection may

take much longer than the amount of time required
to transfer an information block, which is
inappropriate for some applications (e.g, storage
using a SCSI protocol). In circuit switching, the
connection set-up time is a greater proportion of
the overhead for short- to medium-length
information transfers.

Fabric configuration

The time to set up a connection through the

network increases as network traffic increases,
thus increasing the possibility of bandwidth
starvation to the devices wanting to communicate.
Since the applications are based on an information
transfer rate, with a high peak to average ratio,
circuit switching is inefficient for the traffic bursts
that are prevalent in todays data communication
applications.

Brocade Communications Systems, Inc.

11

Fibre Channel Classes of Service for Data Transport

Table 3. Advantages and Disadvantages of Class-2 and Class 3 Services

Class of Service

Class-2 and 3

Advantage

Disadvantage

The transmission resources (such as

bandwidth and buffers) can be shared among
many users on a link (between a switch and
device) and on an ISL (Inter-Switch Link)
between the switches.

Due to the lack of a dedicated connection, the

transfer of very large blocks of raw
information (as in visualization and number
crunching applications) can cause long fabric
delays in a congested Fabric

Due to frame switching, the links are

occupied only for the duration of frame
transmission, making the path available for
frames generated by other sources and
increasing both the link and Fabric bandwidth
utilization.

No error control at the link level or end-toend in Class 3 results in dropping frames
without notification when the Fabric becomes
highly congested. An upper-level protocol
must get involved to handle retransmission,
thus resulting in degradation of device and
Fabric throughput.

Frames in the same information block can be

in transit simultaneously on different paths in
the Fabric, thus fully utilizing expensive link
and Fabric resources.

In transactions between many devices and

one device, improper buffer allocation in the
devices may result in long queuing delays
within the Fabric.

Although Fabric delay can vary based on

traffic, frame switching has a lower Fabric
delay (as compared to circuit-switching) for
short- and medium-size information transfers
in server-to-server and server-to-storage
communications.

Out-of-order frame delivery in the generic

Fabric case. (The Brocade Fabric guarantees
in-order delivery of frames)

The frame switching capability enables the

forwarding of data frames on available paths
in a Fabric of mesh-connected switches,
which optimizes the utilization of system
bandwidth and provides alternate routes when
link or switch failures occur.
Error handling in Class-2 is performed both
at the link level and on an end-to-end basis,
thereby providing the highest level of data
reliability and integrity.
Due to no acknowledgement frames from the
multicast group members, Class 3 is the ideal
service to achieve high throughput for
multicast mode operation.
In a controlled and robust environment,
Class-3 can be deployed to reduce round-trip
delays and to increase the Fabric throughput.
Data transfer between the devices takes place
without having to wait for the end-to-end
acknowledgement frames.

Brocade Communications Systems, Inc.

12

Fibre Channel Classes of Service for Data Transport