Storage Center

Multipath IO Manager 1.0

for IBM AIX Servers

User Guide

Multipath IO 1.0 User Guide for IBM AIX Servers

Document Number 680-033-001 Revision Date
A B C D E F G 6/13/08 7/29/08 11/06/08 July 2009 May 2009 June 2011 October 2011

Description
Initial release. Updated installation requirements. Updated installation instructions and Queue Depth section. Updated installation requirements to include support for IBM AIX 6.1. Updated Dynamic Tracking section to include new guidance about the 15-second failover limit on volumes. Clarified Dynamic Tracking; applied Dell branding. Updated installation requirements to indicate that Storage Center 4.1 and above is supported.

Disclaimers
Information in this document is subject to change without notice.

Trademarks and Copyright

Trademarks used in this text: Dell, the DELL logo, and Compellent are trademarks of Dell Inc. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their products. Dell Inc. disclaims any proprietary interest in trademarks and trade names other than its own.

Contacting Dell Support Services

Please contact Dell Technical Support Services from the following address if you need support assistance: http://support.dell.com/compellent

2011 Dell Inc. All rights reserved.

Contents

Installation and Configuration

Multiple Path IO Overview 2 Before You Begin 3 System Requirements 3 Options for Successful Controller Failover with IBM AIX Servers Installing MPIO Manager for IBM AIX Servers 5 Notes for a Virtual IO Environment 5 Accessing Volume and Device Information 7 Uninstalling the Dell Compellent MPIO Manager 8

Common IBM AIX MPIO Parameters

Enabling Dynamic Tracking in the IBM AIX Fibre Channel Interface Scheduling Algorithms 11 Round Robin Algorithm 11 Failover Algorithm 11 Changing the Algorithm 11 Queue Depth 12 Storage Center Version-Specific Instructions 12 10

Dell Compellent

iii

Contents

iv

Multipath IO 1.0 User Guide for IBM AIX Servers

Installation and Configuration

Multiple Path IO Overview 2 Before You Begin 3 Installing MPIO Manager for IBM AIX Servers 5 Accessing Volume and Device Information 7 Uninstalling the Dell Compellent MPIO Manager 8

Dell Compellent

Installation and Configuration

Multiple Path IO Overview

Installing Multipath IO (MPIO) Manager for IBM AIX Servers enables IBM AIX servers to use multiple paths to Dell Compellent volumes for failover and load balancing. This installp compliant fileset consists of a path-control module (PCM) to provide path management and attributes to update device configuration in the IBM AIX Object Data Manager (ODM). This guide describes how to install and configure the MPIO Manager for IBM AIX Servers. Once installed, the MPIO Manager for IBM AIX Servers enables and/or sets the following on the IBM AIX server: Sets the queue depth to 32 for all volumes mapped from Storage Center Increases the rw_timeout value to 60 seconds Notifies the IBM AIX server that Compellent volumes are bootable under the Compellent name Utilizes the built-in IBM AIX multipathing software called MPIO

Multipath IO 1.0 User Guide for IBM AIX Servers

Before You Begin

Before You Begin

Before installing and configuring the Dell Compellent Storage Center Multipath IO (MPIO) Manager, review the following system requirements and failover considerations.

System Requirements
The following table lists the requirements for using Storage Center MPIO Manager for IBM AIX servers. Requirement
Operating System

Description
IBM AIX 5.2 ML1 SP0 (5200-01-00), IBM 6228 HBA IBM AIX 5.3 ML8 SP2 (5300-08-02), IBM 5759 HBA IBM AIX 6.1 TL 6100-02-01, IBM 5759 4-Gb dual-port fibre channel PCI-X 2.0 DDR Adapter 4.1 and above 4.1.2 or later for queue depth settings

Storage Center

Disks

Fibre channel -- multipathing across iSCSI interfaces is not supported. Note: Queue depth settings require Storage Center 4.1.2 or later. For iSCSI disks the Object Data Manager (ODM) path control module must be installed to enable a queue depth greater than one.

Device Drivers

Fibre channel -- devices.fcp.disk.rte version 5.2.0.30 or above

Options for Successful Controller Failover with IBM AIX Servers

For Storage Center Front End port connectivity, Dell Compellent strongly recommends using a fibre channel switch that supports persistent FCid (for example, Cisco) or persistent PID (for example, Brocade). This allows IBM AIX to use the configured read write timeout value (for example, 60 seconds) on each hdisk. To prevent IBM AIX servers from being disconnected during a Storage Center controller failover, follow these general guidelines: If the IBM AIX switch
Supports persistent FC ID (for example, a Cisco Fibre Channel)

Then
Before mapping the server to the Storage Center:

1 Install the Dell Compellent MPIO Manager for IBM AIX Servers. See Installing MPIO Manager for IBM AIX Servers on page 5 for instructions. 2 Reboot the IBM AIX server. 3 Map the server to the Storage Center.

Dell Compellent

Installation and Configuration

If the IBM AIX switch

Supports persistent PID (for example, a Brocade fabric)

Then
Before mapping the server to the Storage Center:

1 Install the Dell Compellent MPIO Manager for IBM AIX Servers. See Installing MPIO Manager for IBM AIX Servers on page 5 for instructions. 2 Reboot the IBM AIX server. 3 Map the server to the Storage Center. Note: Brocade Fabric OS (FOS) version 6.3.0a has persistent PID. However, known fabric login issues have been identified in this version. FOS version 6.4 or later is recommended.

Does not support persistent FC ID or PID

Before mapping the server to the Storage Center:

1 Enable Dynamic Tracking in the IBM AIX server fibre channel interface. See Enabling Dynamic Tracking in the IBM AIX Fibre Channel Interface on page 9 for instructions. 2 Install the Dell Compellent MPIO Manager for IBM AIX Servers. See Installing MPIO Manager for IBM AIX Servers on page 5 for instructions. 3 Reboot the IBM AIX server. 4 Map the server to the Storage Center. Caution: Because dynamic tracking has a 15-second hold IO limitation, which in some cases is not long enough to complete a Storage Center controller failover, enabling dynamic tracking works only when the controller failover completes within 15 seconds.

Multipath IO 1.0 User Guide for IBM AIX Servers

Installing MPIO Manager for IBM AIX Servers

Installing MPIO Manager for IBM AIX Servers

Install MPIO Manager for IBM AIX Servers before the server is exposed to the Storage Center for the first time. When planning your installation, be aware the procedure below requires a server reboot.

Notes for a Virtual IO Environment

In a virtual IO (VIO) server/client environment, determine if Virtual Fibre Channels (VFCs) are being used on the client. This will affect where to install the MPIO Manager. If VFCs are not being used, install MPIO Manager for IBM AIX only in the VIO server partition. It is not necessary to install the MPIO Manager for IBM AIX in the VIO client because the client partitions depend on the VIO server for all connections to actual devices. If VFCs are being used, install MPIO Manager for IBM AIX on the server partition and each client using VFC.

Invoke the $ oem_setup_env command to enter the superuser shell of the VIO server (ioscli). Within the VIO server super shell, use the # installp command to install the MPIO Manager as described below.

To install Storage Center MPIO for IBM AIX servers

1 Log in to the Knowledge Center via your Dell Compellent Customer Support Account: http://support.dell.com/compellent. 2 Download the following: Dell Compellent AIX Best Practices document [Resources > Best Practices area] AIX MPIO 1.0.0 Software [Downloads > Software area]; extract the files.

3 Copy Compellent_SC.1.0.0.n.bff, where n is the build number, to the IBM AIX machine. 4 Install the package using the installp command. For example:
# installp ac d Compellent_SC.1.0.0.5.bff all

The following files are installed: /usr/lib/methods/aixcomp-rtl /usr/lib/methods/comp_disk.cat /usr/lib/methods/comp_pcm.cat /usr/lib/drivers/aixcomp-ke

/usr/lib/boot/protoext/disk.proto.ext.fcp.disk.compellent_sc.rte In addition, the following entries are added to the IBM AIX Object Data Manager (ODM). These entries have the uniquetype values of: disk/fcp/compellent_sc disk/iscsi/compellent_sc

PCM/friend/compellent_sc The installp program will invoke bosboot. This will push ODM records and the protoext file into the IBM AIX boot record.

Dell Compellent

Installation and Configuration

5 Reboot the IBM AIX machine. This is necessary for a Dell Compellent volume to be used in a boot from SAN application. 6 Verify ODM changes by displaying disk details. Instead of a generic label such as Other FC SCSI Disk Drive, the drive should now have a Compellent-specific label as shown below. This state is shown in the output of the lsdev Cc disk command on non- virtualized IBM AIX servers or the #lsdev type disk command on VIOS.
$> lsdev -Cc disk hdisk0 Available 01-08-00-1,0 SCSI Disk Drive hdisk1 Available 01-10-01 Compellent FC SCSI Disk Drive

7 Review the current Dell Compellent AIX Best Practices document you downloaded in Step 2.

Multipath IO 1.0 User Guide for IBM AIX Servers

Accessing Volume and Device Information

Accessing Volume and Device Information

After installation, map new volumes in the Storage Center to the IBM AIX server. Then, scan for those volumes on the IBM AIX server using the cfgmgr command. Use the lsdev command to list the Dell Compellent devices. The Dell Compellent devices are labeled Compellent FC SCSI Disk Drive. For example:
# lsdev -c disk hdisk0 Available 10-60-00-4,0 16 Bit SCSI Disk Drive hdisk1 Available 10-60-00-5,0 16 Bit SCSI Disk Drive hdisk2 Available 20-58-01 hdisk3 Available 20-58-01 hdisk4 Available 20-58-01 Compellent FC SCSI Disk Drive Compellent FC SCSI Disk Drive Compellent FC SCSI Disk Drive

To see the paths available to the devices, use the lspath command. For example:
# lspath Enabled hdisk0 scsi0 Enabled hdisk1 scsi0 Enabled hdisk2 fscsi0 Enabled hdisk3 fscsi0 Enabled hdisk2 fscsi1 Enabled hdisk3 fscsi1 Enabled hdisk4 fscsi0 Enabled hdisk4 fscsi1

Note: If a volume has already been mapped to the IBM AIX server before you install the Dell Compellent MPIO Manager, you must remove the device (see Step 1b in the procedure below) and then use the cfgmgr command to scan for the device again.

Dell Compellent

Installation and Configuration

Uninstalling the Dell Compellent MPIO Manager

To uninstall MPIO Manager, you must first remove all disks that make use of the Dell Compellent Storage Center definitions. 1 Remove the hdiskn devices that refer to Compellent Disk Drives. a Find the Dell Compellent devices by issuing the following command: # lsdev c disk This command displays the disk information. For example:
hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 Available Available Available Available Available 10-60-00-4,0 10-60-00-5,0 20-58-01 20-58-01 20-58-01 16 Bit SCSI Disk Drive 16 Bit SCSI Disk Drive Compellent FC SCSI Disk Drive Compellent FC SCSI Disk Drive Compellent FC SCSI Disk Drive

b Remove all devices listed as Compellent FC SCSI Disk Drive. This may involve any of the following procedures: Unmounting any file systems using the disks. Using varyoffvg for any volume groups referencing the disks. Using exportvg for any volume groups referencing the disks. Using rmdev -d -l hdiskn for the hdisk devices.

2 Once all the hdiskn devices have been removed, use # installp to uninstall the package. For example:
# installp -u Compellent_SC

Multipath IO 1.0 User Guide for IBM AIX Servers

Common IBM AIX MPIO Parameters

MPIO Manager allows users to adjust common AIX MPIO parameters. This includes enabling dynamic tracking, changing the scheduling algorithm, and adjusting the queue depth. This section explains each parameter and how it can be adjusted. Enabling Dynamic Tracking in the IBM AIX Fibre Channel Interface 10 Scheduling Algorithms 11 Queue Depth 12

Dell Compellent

Common IBM AIX MPIO Parameters

Enabling Dynamic Tracking in the IBM AIX Fibre Channel Interface

To enable dynamic tracking on the IBM AIX server, change the setting for the dyntrk attribute in the IBM fibre channel interface. Note: Do NOT enable dynamic tracking if your switch supports persistent FCid (for example, Cisco) or persistent PID (for example, Brocade). To find the current setting, issue the following command:
# lsattr -HE -l fscsi0

This returns a result such as the following example:

attribute attach dyntrk scsi_id sw_fc_class value switch no 0x670a00 3 description How this adapter is CONNECTED Dynamic Tracking of FC Devices Adapter SCSI ID FC Class for Fabric user_settable False True False True

fc_err_recov delayed_fail

FC Fabric Event Error RECOVERY Policy True

The example above shows that dynamic tracking is disabled (dyntrk has the value no). To enable it, use the following command:
# chdev -l fscsi0 -a dyntrk=yes

If an error is returned that the device is unavailable because the specified device is busy, change the setting of the device in the database. The changes are applied to the device when the system is rebooted. To change the setting of the device in the database, use the -P flag with the chdev command:
# chdev -l fscsi0 -a dyntrk=yes -P

The lsattr command returns a result that reflects the change after the next reboot of the IBM AIX server. The following shows an example where dynamic tracking is enabled (dyntrk has the value yes):
# lsattr -HE -l fscsi0 attribute attach dyntrk fc_err_recov scsi_id sw_fc_class value switch yes delayed_fail 0x670b00 3 description How this adapter is CONNECTED Dynamic Tracking of FC Devices Adapter SCSI ID FC Class for Fabric user_settable False True False True

FC Fabric Event Error RECOVERY Policy True

10

Multipath IO 1.0 User Guide for IBM AIX Servers

Scheduling Algorithms

Scheduling Algorithms
By default, all paths to Dell Compellent devices are used with a round robin scheduling algorithm for SCSI requests. This section explains round robin and failover algorithms as well as how to change which is used.

Round Robin Algorithm

The round robin scheduling algorithm works by sending the first SCSI request down the first path, the second request down the second path, etc., until all paths have been used. The next request is then sent down the first path again, and the process repeats. Each path carries roughly the same number of requests. There is no analysis of which path has greater queue depth or more outstanding read or write requests. The round robin algorithm generally offers the best performance, but in some configurations it is not desirable.

Failover Algorithm
In those situations, the algorithm can be changed to a failover algorithm. In the failover algorithm all requests are sent down one path until it fails. At that time an alternative path is chosen and all requests are sent down that path until it fails.

Changing the Algorithm

The Dell Compellent Storage Center treats all requests the same regardless of their path. Therefore, choosing one algorithm over another depends on the network between the IBM AIX server and the Dell Compellent Storage Center. To view the current algorithm setting, issue the following command:
# lsattr -HE -l hdiskn

To change the algorithm to failover, use the following command:

# chdev -l hdiskn -a algorithm=fail_over

To change the algorithm to round robin, use the following command:

# chdev -l hdiskn -a algorithm=round_robin

If an error is returned that the device is unavailable because the specified device is busy, change the setting of the device in the database. The changes are applied to the device when the system is rebooted. To change the setting of the device in the database, use the -P flag with the chdev command:
# chdev -l hdiskn -a algorithm=round_robin -P

The change is reflected in the output from the lsattr command after the next reboot of the AIX server.

Dell Compellent

11

Common IBM AIX MPIO Parameters

Queue Depth
By default, the queue depth is set to 32. This means that up to 32 SCSI requests may be queued via each available path. If two paths are available while the round robin algorithm is being used, then up to 64 requests may be queued at one time. The queue depth can be set to any value from 1 to 256. The best value depends on the Dell Compellent Storage Center configuration, the network between the Dell Compellent Storage Center and the IBM AIX server, and the type of application using the volume. To view the current queue depth setting, issue the following command:
# lsattr -HE -l hdisk2

To change the queue depth to 16, issue the following command:

# chdev -l hdiskn -a queue_depth=16

If an error is returned that the device is unavailable because the specified device is busy, change the setting of the device in the database. The changes are applied to the device when the system is rebooted. To change the setting of the device in the database, use the -P flag with the chdev command:
# chdev -l hdiskn -a queue_depth=16 -P

The change is reflected in the output from the lsattr command after the next reboot of the AIX server.

Storage Center Version-Specific Instructions

Storage Center 4.1.1 or Earlier
If the Dell Compellent Storage Center software is 4.1.1 or earlier, then lsattr may show a queue depth greater than 1, but the effective queue depth remains fixed at 1.

Storage Center 4.1.3 or Above

If the Dell Compellent Storage Center software is version 4.1.3 or above, the queue depth setting is effective for volumes that were created after the upgrade to version 4.1.3 or above. If the Dell Compellent Storage Center software is version 4.1.3 or above, but the volume was created with version 4.1.2 or below, choose one of the following procedures: Procedure 1 (no reboot required) 1 Copy/Migrate the volume to a newly created volume. 2 After the Copy/Migrate has finished, stop all IO on the server. 3 Unmount any file systems using the volumes 4 Use varyoffvg to deactivate any volume groups referencing the disks. 5 Use exportvg to remove the definition of any volume groups referencing the disks. 6 Use rmdev -d -l hdiskn to remove the hdisk devices. 7 Use cfgmgr to rescan the SCSI bus and rediscover the hdisk devices.

12

Multipath IO 1.0 User Guide for IBM AIX Servers

Storage Center Version-Specific Instructions

8 Use importvg to import a new volume group definition for each volume group referencing the disks. 9 Use varyonvg to activate each volume group referencing the disks. The volume group is now online for use. 10 Mount any file systems using the volumes. Procedure 2 (reboot required) 1 Copy/Migrate the volume to a newly created volume. 2 When the Copy/Migrate has finished, reboot the AIX server.

Dell Compellent

13

Common IBM AIX MPIO Parameters

14

Multipath IO 1.0 User Guide for IBM AIX Servers

680-033-001 G