You are on page 1of 6

Brocade 8 gb how to tal k when i dl e..

portCfgFi l l Word
Posted on February 14, 2013
Are you having weird lun loss, lun connectivity issues, is your hba connecting but not logging in to the fabric?
Your Brocade FC 8 gb switch may be having issues figuring out what to do when idle..
What am I talking about?? There are some known (but not overly discussed) issues with Brocade 8 gb switches.
For some reason there was a change in how hbas and switches started dealing with idle time when they started
speeding everything up to 8 gb. Before I go to much further let me point out how to change the setting.
From the CLI you use
portCfgFillWord (portCfgFillWord <Port#> <Mode>)
Mode Link Init/Fill Word
Mode 0 IDLE/IDLE
Mode 1 ARBF/ARBF
Mode 2 IDLE/ARBF
Mode 3 If ARBF/ARBF fails, use IDLE/ARBF
Modes 2 and 3 are compliant with FC-FS-3 specifications (standards specify the IDLE/ARBF behavior of
Mode 2, which is used by Mode 3 if ARBF/ARBF fails after 3 attempts). For most environments, Brocade
recommends using Mode 3, as it provides more flexibility and compatibility with a wide range of devices. In
the event that the default setting or Mode 3 does not work with a particular device, contact your switch
vendor for further assistance.
Examples:
portCfgFillWord 0 3 this changes the fill word of port 0 to mode 3
portCfgFillWord 1/19 3 this changes the fill word of slot 1 port 19 to mode 3
How can I figure out if this applies to me? Well here are some questions to ask:
Do I have a Brocade 8 Gb switch? If yes more than likely you are affected see below how to identify what ports..
Are 16 Gb switches affected? Apparently not, the Condor3 ASIC auto detects the proper fill word.
l oopbackconnector
Brocade 8 gb how to talk when idle.. portCfgFillWord | loopbackconnector http://loopbackconnector.com/2013/02/14/brocade-8-gb-how-to-talk-whe...
1 of 6 4/5/2014 6:19 PM
All of my devices connected to my switch are only 4 gb am I affected? Nope, if that is a true statement..
Remember you may have 4gb HBAs in your blades but the link aggregator is 8gb so please check.
Ok I have an 8 Gb switch and I have 8 Gb devices can I blindly change all of my ports to mode 3, I would not.
Not all devices, and 4Gb support ARBF/ARBF fails, use IDLE/ARBF so you can look at each port and see if
there are issue. Here is a quote from Brocade However, some 8G devices are not capable of properly
establishing links with Brocade 8G Fibre Channel switches when ARB/ARB or IDLE/ARB primitives are used.
These 8G devices require the legacy IDLE/IDLE sequence to achieve successful link initialization.
I look at all of my errors and I dont see any errors are you sure that I am having issues? Well Brocade likes to
spread the info across a few places.. be sure to check portstatsshow if you see something like this :
>portstatsshow
. truncated
er_toolong 0 Frames longer than maximum
er_bad_eof 0 Frames with bad end-of-frame
er_enc_out 655692 Encoding error outside of frames
er_bad_os 1102115214 Invalid ordered set
er_rx_c3_timeout 0 Class 3 receive frames discarded due to timeout
er_tx_c3_timeout 0 Class 3 transmit frames discarded due to timeout
er_c3_dest_unreach 0 Class 3 frames discarded due to destination unreachable
er_other_discard 0 Other discards
. truncated
You probably have issues and you just havent noticed. . or you are having issues any you found this post because
you are having issues.
The er_bad_os should be 0, ok it can increment a few hundred and you are fine.. these errors happen if you
have a few hundred er_enc_out (to a few thousand) due to a few reboots, cable unplugs etc.
But if you are having issues er_bad_os issues it will be actively incrementing and in the 10s of thousands+..
Ok what are all of my ports currently set to? You can run
>portcfgshow
Ports of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Brocade 8 gb how to talk when idle.. portCfgFillWord | loopbackconnector http://loopbackconnector.com/2013/02/14/brocade-8-gb-how-to-talk-whe...
2 of 6 4/5/2014 6:19 PM
-++++++++++++++++
Speed AN AN 8G 8G 8G 8G 8G 8G AN AN 8G AN AN AN 8G AN
Fill Word(On Active) 0 0 3 3 3 3 3 3 0 0 3 0 0 0 3 0
Fill Word(Current) 0 0 3 3 3 3 3 3 0 0 3 0 0 0 3 0
AL_PA Offset 13 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Trunk Port ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON
Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
VC Link Init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
. truncated-
Ok I have found a port and made the change how can I see easily if the errors have stopped. Well run statsclear
and check the counts again..
Can I make the changes without interruption? Sorry the portCfgFillWord command does cause a brief
interruption in the flow of traffic.
Now there are some specific issues known out there.
IBM storage arrays: SVC, Storwize V7000 and Storwize V7000 Unified.
http://www-01.ibm.com/support/docview.wss?uid=ssg1S1003699
HP Virtual Connect
When VC 8Gb 20-port FC Module and VC FlexFabric 10Gb/24-port Module Fibre Channel uplink ports are
configured to operate at 8Gb speed and connect to HP B-series (Brocade) Fibre Channel SAN switches, the
minimum supported version of the Brocade Fabric OS (FOS) is v6.4.x. In addition, the Fill Word on those
switch ports must be configured with option Mode 3 to prevent connectivity issues at 8Gb speed.
On HP B-series (Brocade) FC switches, use the
portCfgFillWord (portCfgFillWord <Port#> <Mode>)
command to configure this setting.
Mode Link Init/Fill Word
Mode 0 IDLE/IDLE
Brocade 8 gb how to talk when idle.. portCfgFillWord | loopbackconnector http://loopbackconnector.com/2013/02/14/brocade-8-gb-how-to-talk-whe...
3 of 6 4/5/2014 6:19 PM
Mode 1 ARBF/ARBF
Mode 2 IDLE/ARBF
Mode 3 If ARBF/ARBF fails, use IDLE/ARBF
Modes 2 and 3 are compliant with FC-FS-3 specifications (standards specify the IDLE/ARBF behavior of
Mode 2, which is used by Mode 3 if ARBF/ARBF fails after 3 attempts). For most environments, Brocade
recommends using Mode 3, as it provides more flexibility and compatibility with a wide range of devices. In
the event that the default setting or Mode 3 does not work with a particular device, contact your switch
vendor for further assistance.
http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c03359139/c03359139.pdf
EMC 8 GB link does not initialize to FA adapter when using ARB(FF) as fill word on Brocade switches
EMC Primus emc223491
Here is what an official Brocade Document states in regards to 8gb best practices:
BROCADE FABRIC OS:
LINK INITIALIZATION AND FILL WORD PRIMITIVE CONFIGURATION ON 8G FC PLATFORMS FC
PLATFORMS
Revision 1.0
July 26, 2010
Best practice guideline for 8G FC platforms
For Brocade 8G platforms running FOS v6.3.1 or higher, configuring the fill word to mode 3 typically enables the
port to use the right combination of Fibre Channel primitives to fully interoperate with the attached end device.
Once set to mode 3, user intervention is no longer needed to configure the right primitives to enable the link
between the Brocade 8G switch port and the end device.
If a device is known to require mode 2 (IDLE/ARB) then configuring mode 2 on the 8G switch port will activate
the link sooner. If mode 3 is used in this environment, then the port will first attempt mode 1 then eventually
settle to mode 2 which will cause a delay in initializing the port.
If a device requires IDLE/IDLE combination for link initialization then one needs to set the needed port to mode
0 using portcfgfillword CLI.
If you are changing the fill word primitives on FC long distance ports using portcfglongdistance CLI, please make
sure that there is no conflict between the portcfglongdistance setting and portcfgfillword mode setting.
Brocade 8 gb how to talk when idle.. portCfgFillWord | loopbackconnector http://loopbackconnector.com/2013/02/14/brocade-8-gb-how-to-talk-whe...
4 of 6 4/5/2014 6:19 PM
Please refer to the Procedure to Change Long Distance Fill Word Mode to Change Long Distance Fill Word Mode
Settings section of this document for additional details.
http://h30499.www3.hp.com/hpeb/attachments/hpeb/bladescategory04/859/1
/FOS%208G%20Link%20Init%20Fillword%20Behavior%20v1.pdf
Root Cause Details
The T11 fiber committee changed the FC-PI-3 standard for 8 Gb attachment on February 4, 2008. This change
uses the ARB(FF) primitive signal in place of IDLE as a fill word to maintain link synchronization. During link
initialization, 6 IDLE frames are sent before sending ARB(FF) at port state LR3. Slower link speeds only use the
IDLE fill word and do not have this issue.
Here are some commands that can help:
General commands that are helpful in this process:
Portcfgshow show what the current fillword is set to right now
Portstatsshow look for high er_bad_os counts
portCfgFillWord change the fill word to be used
statsclear set the counters back to 0 and see if you are still having issues
##A command that can show all of the ports specified and there er_bad_os count
>portstatsshow -i 0-23 | grep port:\|\er_bad_os
##Script to disruptively change fillword to mode 3 for all ports on card. Repeat command for each port card slot
in switch. *note I have not tried this and copied it from the EMC primus listed above.
>for ((i=0;i<32;i++)); do (echo <slot>/$i;portcfgfillword <slot>/$i 3); done
or
>or ((slot=1;slot<13;slot++)); do (for ((port=0;port<48;port++)); do (echo $slot/$port; portcfgfillword
$slot/$port 3 passive; portcfggport $slot/$port 1; portcfgeport $slot/$port 0); done); done
Note: there has been issues reporting that running a script like this and the port is not disabled first that it could
cause a panic and a reload of the ports.. DEFECT000316106 see https://community.emc.com/message/640245
for this thread.
This is one of my longer posts with lots of info from different locations. While I believe all of the info to be true
please if you find errors please let me know so I can fix. I hope people find this useful.
This entry was posted in Brocade, FIber Switches, Storage Arrays. Bookmark the permalink.
Brocade 8 gb how to talk when idle.. portCfgFillWord | loopbackconnector http://loopbackconnector.com/2013/02/14/brocade-8-gb-how-to-talk-whe...
5 of 6 4/5/2014 6:19 PM
2 Responses to Brocade 8 gb how to talk when idle.. portCfgFillWord
loopbackconnector
Dave says:
J uly 8, 2013 at 3:57 pm
We had some bizarre issues with our Brocades for a while and they had us try this as a fix. In our case, the fill words didnt
help. We ultimately had to replace a batch of SFPs in 2 switches to correct the bizarre behavior described. It only affected
8Gb ports too, but this is a great write up of a not-so-common problem. thanks
Reply
James says:
J anuary 13, 2014 at 7:13 am
Thanks for the article. We had some issues with our FC tape drive randomly dropping off from a HP Blade. Changing the
fillword to 3 for the attached Blades seems to have fixed the problem
Reply
The Twenty Ten Theme Blog at WordPress.com.
Brocade 8 gb how to talk when idle.. portCfgFillWord | loopbackconnector http://loopbackconnector.com/2013/02/14/brocade-8-gb-how-to-talk-whe...
6 of 6 4/5/2014 6:19 PM