KB-000071

Oct 8, 2013

Solaris MPxIO and Configuration (External)

Summary
This procedure details how to properly configure the Solaris Operating System to access a Nimble
Storage Array via iSCSI.

Introduction
The default Solaris iSCSI configuration timeout settings do not allow for non-disruptive failovers of the
Nimble Storage Array. In addition to these timeout settings there are also configuration details in
regards to Asymmetric and Symmetric LUN access that should be addressed for the possibility of future
Nimble Storage enhancements.
(Note: This document does not cover the configuration or tuning of the Solaris ZFS volume management.
ZFS specific information is covered in KB-000078)

Page 1 of 9

KB-000071
Oct 8, 2013

Solaris MPxIO and Configuration (External)

Host Configuration
Ensuring iSCSI Initiator is running:
In Solaris 10 and above the iSCSI initiator is typically enabled from the moment of install. If this is an
existing install you may want to verify that iSCSI is actively running using the following command:
svcs a |grep i iscsi
online
Mar_15

svc:/network/iscsi/initiator:default

If that command reports disabled you can enable the iSCSI initiator by running the following command:
svcadm enable svc:/network/iscsi/initiator

Verify the SCSI timeout settings:

The default Solaris SCSI timeout value is 60 seconds, which is the value that Nimble recommends for
access to our storage arrays. If this is a new system the setting should be correct but its likely that an
existing system will have this modified. To check the global timeout setting run the following command:
echo "sd_io_time::print" | mdb k
0x3c

0x3c is the hexadecimal value for 60. If the response is something else we recommend that you modify
the value to 0x3c. To globally modify the setting edit the /etc/system file to either remove any existing
setting or change it to the following:
set sd:sd_io_time = 0x3c

(NOTE: Any modification of the /etc/system file will require a reboot. There are other modifications to
take into account that also require a reboot so please verify your settings against the other
recommended settings in this doc to limit the amount of times you may need to reboot)

(NOTE: Depending on the architecture of the server (x86 or SPARC) you may have to modify which driver
you are using. For x86 both SCSI and FCP/iSCSI block devices are handled by the sd driver. In SPARC, SCSI
is handled by the sd driver but FCP/iSCSI is configured with the ssd driver. The above examples are
proper for an x86 host. On a SPARC system it would be:
echo ssd_io_time::print |mdb k and set ssd:ssd_io_time = 0x3c

Page 2 of 9

KB-000071
Oct 8, 2013

Solaris MPxIO and Configuration (External)

Symmetric Multipathing settings:

The nimble array supports Asymmetric Lun Unit Access (ALUA) by default. For iSCSI implementations it
is recommended to use Symmetric Multipathing instead. To accomplish this you must modify
/kernel/drv/scsi_vhci.conf to have the following entry:
For Solaris 10 installs:
device-type-scsi-options-list =
"Nimble Server", "symmetric-option";
symmetric-option = 0x1000000;

For Solaris 11 isntalls:

scsi-vhci-failover-override =
"Nimble Server",

"f_sym";

(for both versions make sure there are TWO spaces between Nimble and Server)
Solaris Sun Traffic Manager Service (MPxIO):
If you intend to have multiple paths to your iscsi targets you can configure the system ahead of device
discovery to employ the native multipathing I/O solution for Solaris (MPxIO). To accomplish this you can
simply run the following command:
stmsboot e

This will configure the system to use the Native Stack MPxIO and reboot the system two times in a row
to complete the configuration. These reboots will also enable all the previous changes that you have
made to the kernel options that have been discussed in this document.
If you choose not to use MPxIO for your configuration this point of the configuration would be the point
where the system should be rebooted for the previous configuration changes to take effect.

Page 3 of 9

KB-000071
Oct 8, 2013

Solaris MPxIO and Configuration (External)

iSCSI Initiator Configuration:

Review the current iSCSI discovery method settings and enable Send Targets if not already enabled:
iscsiadm list discovery
Discovery:
Static: disabled
Send Targets: enabled
iSNS: disabled

To enable Send Targets use the following command:

iscsiadm modify discovery t enable

You can re-run the list discovery command to verify.

In order to use the Nimble Storage iSCSI targets you must add the arrays Discovery Address to the
Solaris host. This can be accomplished with the following command:
iscsiadm add discovery-address <target discovery ip of your array>:3260

To verify the setting you can run this command:

iscsiadm list discovery-address
Discovery Address: <IP>:3260

Page 4 of 9

KB-000071
Oct 8, 2013

Solaris MPxIO and Configuration (External)

To provide the storage array with an initiator for creation (or addition to) an Initiator Group you can
determine the Solaris initiator name with the following command:
iscsiadm list initiator-node
Initiator node name: iqn.1986-03.com.sun:01:eaefccf7ffff.513f6c0e
Initiator node alias: unknown
Login Parameters (Default/Configured):
Header Digest: NONE/Data Digest: NONE/Authentication Type: NONE
RADIUS Server: NONE
RADIUS access: unknown
Tunable Parameters (Default/Configured):
Session Login Response Time: 60/Maximum Connection Retry Time: 180/Login Retry Time Interval: 60/Configured Sessions: 1

The output of the initiator node name (the iqn number) is what will be used in the Storage Arrays
Initiator group setting.

At this stage of the configuration it would be prudent to direct the Operating System to discover any
devices that it may not have automatically detected up until now. For the iSCSI stack this can be
accomplished with the following command:
devfsadm i iscsi

Page 5 of 9

KB-000071
Oct 8, 2013

Solaris MPxIO and Configuration (External)

By default the Solaris initiator is only going to be configured/see one session to the array. If you have
multiple data ports on your array then there is the capability of multiple sessions and Solaris MPxIO
would be available for path failover management on the Solaris side. If you know how many data ports
you have configured that the Solaris host can access you can set the iscsi initiator accordingly. To
discover what the host can see you can use the following set of command:

mpathadm list lu
/dev/rdsk/c2t27D274187B95AF356C9CE900F4AC0733d0s2
Total Path Count: 1
Operational Path Count: 1
/dev/rdsk/c2tC9D2DC20D8CD40F16C9CE900F4AC0733d0s2
Total Path Count: 1
Operational Path Count: 1
/dev/rdsk/c2t56593DF098D982536C9CE900F4AC0733d0s2
Total Path Count: 1
Operational Path Count: 1

Page 6 of 9

KB-000071
Oct 8, 2013

Solaris MPxIO and Configuration (External)

Choose one of the paths to your storage array that you want to investigate:
mpathadm show lu /dev/rdsk/c2t56593DF098D982536C9CE900F4AC0733d0s2
Logical Unit: /dev/rdsk/c2t56593DF098D982536C9CE900F4AC0733d0s2
mpath-support: libmpscsi_vhci.so
Vendor: Nimble
Product: Server
Revision: 1.0
Name Type: unknown type
Name: 56593df098d982536c9ce900f4ac0733
Asymmetric: no
Current Load Balance: round-robin
Logical Unit Group ID: NA
Auto Failback: on
Auto Probing: NA
Paths:
Initiator Port Name: iqn.198603.com.sun:01:000000000500.5153c4d6,4000002a00ff
Target Port Name: 4000002a0000,iqn.200711.com.nimblestorage:x862-v67d5633b54d2cfcf.00000013.3307acf4,2460
Override Path: NA
Path State: OK
Disabled: no
Target Ports:
Name: 4000002a0001,iqn.2007-11.com.nimblestorage:x862v67d5633b54d2cfcf.00000013.3307acf4,2460
Relative ID: 0
Name: 4000002a0002,iqn.2007-11.com.nimblestorage:x862v67d5633b54d2cfcf.00000013.3307acf4,2460
Relative ID: 0
Name: 4000002a0000,iqn.2007-11.com.nimblestorage:x862v67d5633b54d2cfcf.00000013.3307acf4,2460
Relative ID: 0
Name: 4000002a0003,iqn.2007-11.com.nimblestorage:x862v67d5633b54d2cfcf.00000013.3307acf4,2460
Relative ID: 0

Page 7 of 9

KB-000071
Oct 8, 2013

Solaris MPxIO and Configuration (External)

In the previous example we see there are 4 possible target paths available to this host. To correctly
configure the host to access all of these sessions the following command can be employed:
iscsiadm modify initiator-node -c 4

Re-Running the before mentioned mpathadm show lu command will now display that you have 4
active paths instead of one.

Using the devices:

At this phase of the configuration the devices should be available to the OS for manipulation via the
format command. Depending on if Solaris SPARC or x86 is being used will depend on how you move
forward with the configuration. Before adding the devices to any filesystems or volume managers it
would be highly desirable to ensure the OS views the new devices as Nimble Devices so that previous
configurations suggested in this document can have the most effect.
In some instance the initial output of format will look like this:
AVAILABLE DISK SELECTIONS:
0. c1t0d0 <DEFAULT cyl 1302 alt 2 hd 255 sec 63>
/pci@0,0/pci15ad,1976@10/sd@0,0
1. c2t6BD344293506D2AC6C9CE900F4AC0733d0 <DEFAULT cyl 53467 alt 2 hd
255 sec 315>
/scsi_vhci/disk@g6bd344293506d2ac6c9ce900f4ac0733

In this example the issue is that the iscsi disk is showing up as DEFAULT and not as a Nimble Storage
Device. To correct this do the following:
Choose the device in the format menu (in this case option 1). From the next menu type in the command
type like this:
format> type
AVAILABLE DRIVE TYPES:
0. Auto configure
1. DEFAULT
2. DEFAULT
3. other
Specify disk type (enter its number)[2]: 0

Page 8 of 9

KB-000071
Oct 8, 2013

Solaris MPxIO and Configuration (External)

At the Specify disk type prompt enter the option 0 for Auto Configure. This should result in
output similar to this:
c2t6BD344293506D2AC6C9CE900F4AC0733d0: configured with capacity of
2047.89GB
<Nimble-Server-1.0 cyl 53467 alt 2 hd 255 sec 315>
selecting c2t6BD344293506D2AC6C9CE900F4AC0733d0

Once that is complete you want to label the disk. From the format prompt enter the following:
format> label
Ready to label disk, continue? yes

(if this is an x86 configuration you may be required to use the fdisk command first)
Once these steps have been completed you can use the key sequence Control+D to exit the format
utility or simply enter the letter q at the format> prompt. To verify that the changes took place you
can run the format command again. This time you should see:

format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c1t0d0 <DEFAULT cyl 1302 alt 2 hd 255 sec 63>
/pci@0,0/pci15ad,1976@10/sd@0,0
1. c2t6BD344293506D2AC6C9CE900F4AC0733d0 <Nimble-Server-1.0 cyl
53466 alt 2 hd 255 sec 315>
/scsi_vhci/disk@g6bd344293506d2ac6c9ce900f4ac0733

Your Nimble Storage Array iSCSI devices should be ready to use with your OS.

(end of document)

Page 9 of 9