Issue

EMC Symmetrix disk array configuration in Common Serial Number Mode [C-bit setting]

Solution

Dynamic Multi-Pathing (DMP) works with the EMC Symmetrix disk array only if the disk array is
configured in the Common Serial Number Mode [C-bit set]. Without this mode, DMP cannot
provide multipathing functionality.

Until EMC Symmetrix started supporting devices above FFF, if digits 6 to 8 of the EMC
Symmetrix disk serial number are zeroed out then the C-bit is set and the array is configured in
the Common Serial Number Mode as required by DMP. But after EMC Symmetrix started
supporting devices above FFF, they used the last bit to set it to 8 to represent a device above
0xFFF. This "8" is added with a logical "OR" operation so that you device would end with 378
(370 or 008) on port 7dA and 3A8 (3A0 or 008) on port 10dA.

With the "C" bit, the serial number must be identical (common) on all the ports. The "8" remains
to represent a device above 0xFFF, the the rest is set to 0. 000 or 008 = 008.

The disk serial number can be obtained with various commands including the VERITAS
vxdmpinq, vxdmpdebug command or the vxexplorer output.

/etc/vx/diag.d/vxdmpinq /dev/rdsk/c3t12d15s2
Vendor id : EMC
Product id : SYMMETRIX
Revision : 5266
Serial Number : 1004E000a

or

/etc/vx/diag.d/vxdmpinq /dev/rdsk/c3t5006048C52A820D6d766s2
Inquiry for /dev/rdsk/c3t5006048C52A820D6d766s2, evpd 0x0, page code 0x0
Vendor id : EMC
Product id : SYMMETRIX
Revision : 5772
Serial Number : 273E2008a

The serial number above shows that digits 6-8 are zeroed out. The array is setup in Common
Serial Number Mode as required by DMP.

DMP needs to see identical device serial numbers from each path.
If the C-bit is not set then DMP sees multiple disks of the same device through different paths.
If the C-bit is set then DMP sees one unique disk device through multiple paths.

The layout for the EMC Symmetrix disk serial number is:
1004E000a (or) 273E2008a
SSVVVDDPx

SS = Last 2 digits of the Symmetrix serial number

VVV = Symmetrix volume number in hex. 000-FFF
DD = Director number [host adapter] 1-32
P = Port number on director card
x = lower case letter which does not belong to serial number according to EMC.

Note: If the C-bit is not set, data on an existing disk can easily be wiped out by disk operations
to the other path which appears like a different disk all together. Setting the C-bit nulls out digits
6 through 8 of the serial number.

Consult the Storage Foundation Hardware Notes for a complete list of DMP pre-requisites
regarding EMC Symmetrix arrays.

Legacy ID

325308

Article URL http://www.symantec.com/docs/TECH70715

 Common Serial Number (C)

 Host SCSI Compliance 2007 (OS2007)
 SCSI-3 SPC-2 Compliance (SPC2)
 SCSI-3 compliance (SC3)
 Common serial number (C)
 Auto negotiation (EAN) enabled
 Fibrepath enabled on this port (ACLX)
 SCSI 3 (SC3) (Optional)
 Unique Worldwide Name (UWN)
 SPC 2 (Decal) (SPC2) SPC 2 flag set
 Common serial number (C)
 Auto negotiation (EAN) enabled
 Fibrepath enabled on this port (VCM)
 SCSI 3 (SC3) (Optional)
  PTOP for FC-SW OR HDAD for FC-AL
 VCM if using Volume Logix (optional)
 UWN for Unique WWN
 SC3 for heterogeneous port sharing (optional)
 C or common serial number bit
 Common Serial Number (C)
 Host SCSI Compliance 2007 (OS2007)
 SCSI-3 SPC-2 Compliance (SPC2)
 SCSI-3 compliance (SC3)
 For FC Switch Base Topology (FC−SW), Enable Auto Negotiation (EAN), Point−to−Point (PP) , Unique
WWN (UWN).
 For FC Loop Base Topology (FC−AL), Enable Auto Negotiation (EAN), Unique World Wide Name (UWN)

1. 1. What are the "port flags" settings for 5875 Enginuity and what do they mean?

Port flags in Enginuity are used to set behavior that enables a connected server/operating system
combination to work with the Symmetrix port (seen as a disk drive to the server) and to optimize
the operation of that server and “disk” combination. Symmetrix VMAX with 5875 is delivered
with the correct default port flag settings for most users so that few or no changes will be needed
during installation of the new system.
EMC strives to make setup and operation of our products as simple and easy as possible. We
believe that Enginuity 5875 for VMAX is delivered to you with correct default port flag settings
for most users and that few or no changes will be needed during installation of your new system.
Here is a summary of the default flag settings for the most popular operating systems. and
changes required by some of those operating systems.

2. What Servers can connect to 5875 Enginuity without changing Port Flags?
Operating Environment
EMC Celerra
IBM AIX
INM i
LINUX
Hewlett-Packard OpenVMS
Hewlett-Packard True64
Hewlett-Packard HP-UX 11i V1
&V2
Hewlett-Packard HP-UX 11i V3
Microsoft Windows Server 2008
Microsoft Windows Server 2003
Port Settings
Enable 'ARB' and 'D' flags
5875 Defaults, Unchanged
Enable 'AS4' flag
Enable 'D' flag
Enable 'OVMS' flag
Enable 'OVMS' flag, Disable 'SC3 flag
Enable 'V' flag, Disable 'SC3' and 'OS07'
flag
Enable 'V' flag, Disable 'SC3'
5875 Defaults, Unchanged
5875 Defaults, Unchanged
 Oracle Solaris 5875 Defaults, Unchanged
VMware ESX 5875 Defaults, Unchanged

3. What are some additional considerations?

 Directly connecting servers to the array, not using switches or other SAN components, you are
using Fibre-Channel Arbitrated Loop. This requires that the PP flag to be DISABLED and a
Loop ID assigned to a particular number ranging from 0-126. FC-AL is used in certain markets
and with some specialty servers.

 Using Fujitsu (formerly Siemens), Novell Netware or Teradata systems, please look at the
support matrix at EMC.com for specific recommendations.

 Use of Clusters can impact port flag settings. Please look at ELab's Host Connectivity Guide of
the particular operating system for information for details of a cluster's required port flag
settings.

4. Port Flag usage notes:

 These new default values are in some cases different from the 5874 settings. Connection of a
server to both 5874 and 5875 Enginuity may require changes to the 5875 defaults or the user to
set the flags at the Initiator to match the previously installed array with its existing connection to
the server. Note in general that changes of the flags will not become effective until the host is
rebooted or FC connectivity is bounced.

 Script behavior can change based upon these new 5875 default Port Flag settings. Existing
scripts should be tested to ensure port flag settings and scripts are compatible.

 Your VMAX is set to operate with the ACLX flag ENABLED, which enables masking of
specific devices to specific ports on the array. Note that user must still provision ACLX volumes
to enable hosts management via Solution Enabler.

 The OS07 flag, ENABLED by default, can generate warnings (I/O retry) on Solaris, OpenVMS,
and some other hosts. Some high performance or sensitive applications may be degraded by the
additional retry. EMC believes we support the SCSI standard correctly, and retries should not
occur, but that said, it can happen on some systems. Such environments should DISABLE this
flag. Although the OS07 flag is on by default, specifically for VMWARE ESX this setting may
need to be disabled.

Note: that if you DISABLE the OS07 flag and wish to re-ENABLE it, you must be sure that
theSPC2 flag is ENABLED first. Also note that the SPC2 flag value should not be changed from
DISABLED to ENABLED on installed servers as it changes the disk name utilized by the server
and the disks will no longer be visible to that server.

 If SCSI-3 persistent device reservation bit (“PER”) is used (note that this flag is assigned to
devices and not to ports and so is not covered by this document), then the SC3 bit should be set.
Please refer to the Elab Host Connectivity Guide for documentation on the platforms that can
utilize the SCSI-3 reservation bit.
For additional information regarding to port flags, specific flags for a given server, impacts of
changing flags and any other known issues, please go to EMC.com and search PRIMUS for
additional port flag information.

5. How did 5875 Enginuity Port Flag Defaults Change from earlier releases?

The following is the table of default settings for the commonly set flags for 5875 Enginuity
versus other recent Enginuity releases:

Flag Enginuity 5773 Enginuity 5874 Enginuity 5875

PP Enabled Enabled Enabled
UWN Enabled Enabled Enabled
EAN Enabled Enabled Enabled
C Enabled Enabled Enabled
SPC2 Enabled Enabled Enabled
OS07 Disabled Disabled Enabled
ACLX Disabled Disabled Enabled
SC3 Disabled Disabled Enabled

6. What are the port flag descriptions?

Flags come in two types. The first type are flags set only on the array’s storage port. These flags
affect the connectivity behavior of all servers connected to that port. The second type of flags
can be set per storage port or initiator group, a group defined on the array as one or more
initiators (HBAs) for a given server. With this type, the flags settings for the initiator group
override, for the group, the settings on the port itself. In this way, a port can be shared by servers
with different flag requirements.

The following table list where the flags should be set:

Flags that can be set
Flags set only at
at the Storage Portor
FlagFlag Description the StoragePort Initiator Groups
When enabled, specifies a point-to-point
(direct or switched)
topology in the initialization sequence. When
disabled (default),
PP it is initialized as an arbitrated loop. YES N/A
EAN When enabled, allows two fibre ports to YES N/A
 handshake
and settle on an optimal speed for data
transfer.
When this flag is set, it enables the Initiator
ACLX Groups/Access Logix1 function on the port. YES N/A
When enabled, ensures unique World Wide
Names (WWNs)
within the fibre environment by
incorporating Symmetrix serial
numbers and port numbers for each WWN.
Enabled by default
for all environment configuration changes
and new environments.
When disabled, WWNs do not have to be
UWN changed. YES N/A
When enabled along with the Hard
Addressing flag, the
Fibre Channel director only uses hard-
assigned addressing
when it initializes the loop, otherwise, soft-
assigned
addressing is used during loop initialization
NP (the default). YES N/A
When enabled, this flag causes the serial
number
of a specific volume to be the same
regardless of
C the port on which it is visible. N/A YES
When enabled along with the Disk_Array
flag
for HP-UX hosts, the volume setaddressing
mode
is selected. VSA mode allows octal
V addressing. N/A YES
Typical behavior on a SCSI bus reset is to
propagate the reset to
all channels that share devices mapped to the
port that received
the reset. When this bit is set, the reset only
occurs to the port on
ARB which it is received, and is not broadcasted. N/A YES
When this flag is enabled, a Unit Attention
(UA) that is propagated
from another director does not flush the
D queue for this device on this N/A YES
 director. Used for hosts that do not expect the
queue to be flushed on
a SCSI 0629 sense (only on a Hard Reset).
When this flag is set, it alters the Inquiry data
returned by any device
on the port to report that Symmetrix supports
SCSI -3. If this flag is not
set, it reports that SCSI-2 is the supported
protocol. This flag also enable
the support for Auto Contingence Allegiance
SC3 which is required by IBM AIX. N/A YES
The effect of the OS07 setting can be
summarized as follows:
1. Automated device discovery of newly
configured devices and/or device paths.
2. Automated creation of DSFs (device
special files) for newly configured devices.
3. Automated detection and adaptation of
device attribute changes including LUN
OS07 4. Capacity change event. N/A YES
This flag enables the support for SPC-2 SCSI
standard that
are recommended for FC, FCoE, and iSCSI
SPC2 connectivity N/A YES
This flag should be enabled for any AS/400
(IBM i)
AS4 hosts connecting to the port. N/A YES
Enabled for an OpenVMS Fibre Channel
OVMS connection. N/A YES

EMC VMAX – Notes On Setting FA Port Flags For VMware

June 4, 2014 David Ring VMAX, VMwareEAN, EMC, esx, FA Port, FLAGS, OS2007, SCSI-3,SPC-
2, VMAX, VMWARE

When configuring a VMAX for ‘VMware ESX’ connectivity, I will always firstly
consult the ‘EMC Simple Support Matrix – EMC Symmetrix VMAX
Series Director Bit Settings’ found here for the latest listing of port
settings required. Below I will list the relevant VMware flags to be enabled on
the Director FA ports which have ESX Host’s zoned:

♦ ACLX – Allows Initiator Group/Access Logix function on the port.

♦ Common Serial Number (C) – When enabled, this flag causes the serial
number of a specific volume to be the same regardless of the port on which it
is visible and also allows multi-path configurations.
♦ Unique WWN (UWN) – When enabled, ensures unique World Wide
Names (WWNs) within the fibre environment by incorporating Symmetrix
serial numbers and port numbers for each WWN. Enabled by default for all
environment configuration changes and new environments.
♦ Init Point to Point (PP) – Specifies a point-to-point (direct or switched)
topology.
♦ Enable Auto Negotiate (EAN) – With EAN enabled this allows the FA
port to Auto Negotiate the port speed with the switch port it is connected to.
♦ SCSI-3 Interface (SC3) – SC3 provides SCSI-3 (Full range of SCSI-3
commands) support for the ESX hosts. Also note that SC3 is required for
Linux VM’s with RDM’s attached.
♦ SCSI Support1 (OS2007) – OS2007 enables automatic discovery of
newly created LUNs/Paths. When Enabled, this flag provides a stricter
compliance with SCSI standards for managing device identifiers, multi-port
targets, unit attention reports, and the absence of a device at LUN 0. If using
VPLEX 5.2 and higher, the OS2007 flag MUST be enabled.
♦ SCSI Primary Commands (SPC2) – This Flag is required in order for
the FA port to conform to industry standards resulting in an NAA identifier
being used for the LUNs presented to the ESX hosts. To change the status the
Port must be offline.

Note: These Flags may already be enabled by default for the associated
Director Ports and no additional configuration change is required. Consulting
the ESM for default settings will verify this.

FA Port Settings from Factory Bin File:

Green = Enabled
Yellow = Disabled

Setting director port attributes via Unisphere:

1. Select the Symmetrix system.
2. Select System > Dashboard.
3. From the Hardware panel, click Front End Directors to open the Front End
Directors list view.
4. Select the director, and click Set Port Attributes to open the Set Port
Attributes dialog box.
5. Optional:Select a port whose flag settings you want to copy.
VIA SYMCLI to enable flags on FA Port 1F:1, create a text file as per the
following example:
symconfigure -sid xxxx -file c:\file.txt
where file.txt has the following contents:
set port 1F:1 Common_Serial_Number=enable, SCSI_3=enable,
SPC2_Protocol_Version=enable, SCSI_Support1=enable;
View 1F:1 Port configuration include -v in order to view the Flag settings:
symcfg -sid xxxx list -fa 1F -p 1 -v
List all Ports:
symcfg -sid xxxx list -v -dir all
While all the port flags described above can be set at a Director Port level, in
some cases you may not want the enabled flags to affect different host types
connected to the Port such as Linux or Windows, in this case it may be a better
option to set the Flags at an Initiator Group level. By enabling the relevant
host type Flag settings at the Initiator Group level you are only affecting the
ESX Hosts included in the Initiator Group.
The Initiator Group VMware Specific flags are : C, SC3, OS2007, SPC2
In this way the flag settings for the initiator group override flag settings on the
port itself, thus allowing for the port to be shared by servers with differing flag
requirements.
To set the required flags at an IG level use the following symaccess cmd:
symaccess -sid xxxx -type initiator -name IGName set ig_flags on
C,OS2007,SC3,SPC2

There is also an option to set the Flags at the host initiator (WWN) level. For
example setting the flags on a host initiator connected to port 1F:1 then use the
following symmask commands:
symmask -sid xxxx -wwn xxxxxxxxxxxxx -dir 1F -p 1 set hba_flags on
C,OS2007,SC3,SPC2 -enable
symmask refresh
To display what director flags have been configured per host initiator:
symmaskdb -sid xxxx list database -v

Note: If there is a requirement to enable SPC-2 in a live VMware environment

then please reference EMC Support material; as doing so may result in a LUN
UUID change, depending on the version of VMware. One such solution to
overcome this problem was to create a new path and datastore then perform
an svmotion. Also changing the OS2007, SPC2, SC3 FA flags then the hosts
affected by these changes will require a reboot as the SCSI protocols in use will
have changed at the FA level and the HBA needs to reinitialize communication
with the FA.
How to check bits / flags

$ symmaskdb -sid <####> list database -v -wwn <HBA port pWWN> (e.g 10000000c951c57c)

$ symcfg list -sid 0426 -v -dir all | grep -E "[a-zA-Z1_ ]+\(.*\)" | grep abled | sort | uniq

Auto_Negotiate(EAN) : Disabled
Auto_Negotiate(EAN) : Enabled
Avoid_Reset_Broadcast(ARB) : Disabled
Common_Serial_Number(C) : Enabled
Disable_Q_Reset_on_UA(D) : Enabled
Environ_Set(E) : Disabled
HP3000_Mode(B) : Disabled
Init_Point_to_Point(PP) : Enabled
Negotiate_Reset(N) : Disabled
Non_Participating(NP) : Disabled
OpenVMS(OVMS) : Disabled
SCSI_Support1(OS2007) : Disabled
SPC2_Protocol_Version(SPC2) : Disabled
Sequent(SEQ) : Disabled
Siemens(S) : Disabled
Soft_Reset(S) : Disabled
Sunapee(SCL) : Disabled
Unique_WWN(UWN) : Enabled
VCM_State(VCM) : Enabled
Volume_Set_Addressing(V) : Disabled
Volume_Set_Addressing(V) : Enabled
Device Masking (VCM) Config State : Enable