EMC2 EMC CLARiiON

CLAR-PSP-091
Support
Procedure FCO F0626031 Instructions

Revision History
7/15/2003 Draft sent for Engineering review
7/21/2003 Released in support of the FCO

Introduction This document describes the procedure for the implementation of FCO
F0626031.

It will always involve updating CX-Series arrays based software using the Non-
Disruptive Upgrade (NDU) capability of the array to version 02.04.x.xx.5.007,
and will likely involve updating other array based Software and some EMC
software pieces on attached servers. See table 1 for the minimum revisions for
this FCO and table 2 for all previous revisions.

Please read all the rules and comments about updating CX-Series array based
software and any host specific information.

During this procedure, each storage processor will reboot independently.

Therefore, it is imperative that all servers have access to both storage
processors in the array and have properly configured/working failover
software or they will lose access to the array.

It is important to ensure that I/O is at a very low level when this update is
performed. Times of heavy I/O MUST be avoided. A method to determine the
I/O level to the database drives is available in this document in Appendix A

Since every array and every environment is unique, table 1 and table 2 in this
document will define what pieces of array based software and host software
must be updated when this FCO is implemented by updating the array FLARE
software

The update procedure differs if the revision of software, which the array is
running, is earlier than Release 11 (see definition on next page). If updating
from a release prior to Release 11 you must load the Utility Partition. The Utility
Partition allows for recovery of a corrupted boot image for SPA or SPB.

If the CLARiiON CX600 to be updated is part of a Celerra configuration see the

Notes and Warnings section of this document and reference ETA Technical
Advisory emc70418; A CLARiiON FLARE upgrade to Release 11 patch 007
requires that the attached Celerra upgrade to EMCNAS 5.1.18.3 before the
FLARE Upgrade.

Read this entire document before beginning the FCO.

FCO F0626031 Instructions Page 1 of 13

CLARiiON FCO F0626031 Instructions

Definitions Target for this FCO is FLARE Software 02.04.x.XX.5.007. All prior revisions can be
found in table 2.

Minimum Release 11 SW Revisions to complete this FCO

Table 1
CX-Series FLARE/Access Logix 02.04.x.20.5.007 for CX200 &
CX200LC
02.04.x.40.5.007 for CX400
02.04.x.60.5.007 for CX600
array based Navisphere Agent & 6.4.0
Mgt. Server and providers
All Navisphere UIs 6.4.1
SnapView 2.10.06
Host AdmSnap 2.1.0.0.xx (2.1.1.0.xx for Linux 2.4)
MirrorView 1.70.02
SanCopy 1.00.02
CX200 HA enabler 01.01.01
CX200 30-Drive Enabler 01.01.01
Host Navisphere Agents 6.2
Host Navisphere CLI 6.4
Power path 3.x
Claralert 6.4

FLARE Software available prior to Release 11

o CX200 – Core SW 02.03.x.20.5.xxx 6.2 Navisphere array
components (Release 10 CX200 only)
o CX400 – Core SW 02.02.x.40.5.xxx 6.2 Navisphere array
components (Release 8)
o CX600 – Core SW 02.02.x.60.5.xxx 6.2 Navisphere array
components (Release 8)
o CX600 – Core SW 02.01.x.60.5.xxx 6.1 Navisphere array
components (Release 7)

NDU Software Update Types (visible in the NDU dialogue box during the process.)
• Disruptive – Updating an array with one functioning storage processor. This is
standard for a CX200LC but is only possible if in engineering mode for all other
array types via the GUI. Navisphere CLI will prompt you for a confirmation.
• Non-Disruptive (This FCO) – Updating an array that has 2 functioning storage
processors. Each storage processor updates individually, always leaving one
working on the array. If all servers have valid fail-over software, this typical
update will not disrupt server access to data on the array.
• Rebootless – This type of software update will reboot neither storage processor.
A server will not be affected at all during this procedure.
• Utility Partition – Software loaded to previously unused but reserved space on
the first 4 drives of a CX-Series array that provides menus/utilities to perform
recovery and conversion.
• Repository – A previously reserved but un-used space on drive 0_4 (5th drive) in
the boot chassis that holds Recovery Images that are NDU’d to the array or
added via the FTP method.

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 2 of 13

CLARiiON FCO F0626031 Instructions

Utility Partition FAQs

Q What is the Utility Partition
A It is a new section added to each of the Fibre Channel boot drives. It is loaded
to previously unused but reserved space. User space is not affected between
earlier releases and Release 11.

Q When is it loaded
A All arrays shipped since March 10th 2003 have the Utility Partition preloaded
to each of the first 4 drives. In addition, there is a recovery image preloaded at
the factory. This Image is loaded to private space on drive 4 called the
“repository”.

Q How do you get to the Utility Partition

A To get to the Utility Partition, you interrupt the power up of one of the SPs
during its POST test. When the correct password is entered, a menu is
presented. One of the options is to boot the Utility Partition. This tells the SP
to boot the alternate code rather than the standard NT image it would typically
boot.

Q Why would I want to Boot the Utility Partition

A The Utility Partition allows you to use a recovery image to re-image a
corrupted boot image on two mirrored boot pairs for either SP, or to convert
the array from one CX type array to another.

Q Where exactly is the Utility Partition

A It is on the first 4 drives (0-3). It is in previously unused but reserved space.
The Utility Partition area is on the alternate boot drives. SPA boot drives are
drives 0 & 2, but its Utility boot area is on drives 1 & 3. SPB primary boot
drives are drives 1 & 3 with its Utility boot Partition on drives 0 & 2.

Q What is the image repository

A This is an area on drive 4 that holds a recovery image or a conversion image
if loaded via NDU.

Q Where is it documented about the use of the Utility Partition

A CLAR-PSP-078 documents and illustrates the use of the Utility Partition to
recover a corrupted boot image of an SP.

Q Why do some files have a .NDU extension and some have .MIF
A The Utility Partition is available only as an .NDU file and must be loaded via
an NDU operation. The Recovery Image is available as a .NDU file and a
.MIF. The .MIF is the version that is FTP’d directly from your laptop to the SP.
See CLAR-PSP-078.

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 3 of 13

CLARiiON FCO F0626031 Instructions

Rules for NDU

When performing software updates on a CX-Series array there are some
important RULES that always apply.
9 RULE - There should be no faults on the array. A green flashing SPS LED
on either or both SPS indicating a charging SPS is acceptable.

9 RULE - You cannot NDU a new revision of code to the array if the previous
revision requires a commit. Perform a commit on the “active” version if the
Array Software Tab shows that a “commit” is required.
9 RULE - Do Not Start an NDU operation if there are any LUN rebuilds, disk
equalizations, or Raid Group/LUN expansions in progress.

9 RULE - View the Event Log to confirm that the array has not been
experiencing recent problems which should be investigated and understood
prior to this NDU.

9 RULE – Load a Utility Partition via NDU If updating from a release prior to
Release 11. This is done AFTER the array is running Release 11 minimum.
9 RULE All connected servers must have working and supported “path-
failover” software in order to perform this procedure. If they do not, stop and
prevent all I/O from that server to the array
9 RULE - Confirm that there is at least one working path from each
connected server to each storage processor of the array
9 RULE - All array based software within a “release” must be updated at the
same time unless stated otherwise. See Software revision table 1 in this
document.
9 RULE - CX600, CX400 and CX200 arrays should have less than 100 I/Os
per second to each of the first 4 drives in the array (drives 0_0 through
0_3). This low I/O activity to the boot drives avoids timeout problems during
the NDU process. See appendix A for instructions on measuring them I/O
rate to the first 4 drives. This procedure may not be necessary if there is
little or no I/O to the array.
9 RULE - Following an NDU where Navisphere Management Server or the
Manager UI is revised, you MUST close your browser and re-open it in
order to use the new content that may be in the new revisions of
Navisphere Management server or UI.
9 RULE - Navisphere agents and CLI on Servers must be no more than one
revision behind the SP based Navisphere agent.
9 RULE – Navisphere CLI (being used to manage) must be equal to or no
more than one revision greater than any Navisphere Host or SP Agent it will
manage
9 RULE - Host based AdmSnap software must be at the same revision as
SnapView running in the array
9 RULE - If updating FROM a version prior to Release 11, add the Utility
Partition and the Recovery Image via a second NDU AFTER the array has
been update to Release 11 or higher.
9 RULE - Stop all SAN Copy sessions before starting an NDU
9 RULE - Stop all Snap Sessions prior to NDU

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 4 of 13

CLARiiON FCO F0626031 Instructions

9 RULE - Move Snap Cache LUNs out of the Snap Cache pool prior to NDU.
9 RULE – CLARalert software must be at the same release level as the SP
Agent
9 RULE – The Navisphere Agent used by the CLARalert monitoring (dial-
home) workstation must be at the same level as the SP Agent
9 RULE - The Software Installation Wizard is for the CX200 and CX200LC
only.

Notes &
Warnings about
NDU
• When updating an array, all array based software to be updated MUST BE
General comments updated via the SAME NDU software load. Perform ONE NDU operation with
all the software you intend to update. A dependency error will be generated if
one of the “already installed packages” is missing from the NDU package set.
• There are several Primus cases available to help diagnose NDU issues. Search
Primus using “NDU” as a keyword if needed.
• Loading the Utility Partition and the Recovery image is not required if
updating from any version of Release 11 to this FCO level. It should already be
loaded.
• CX400 and CX600 Updates should use the standard Software update window
which has been moved to engineering mode in Navisphere 6.4.x
• Using a Cross-over LAN cable connected to one SP will result in NDU failure
if updating to SnapView 2.10.06 if the solution procedure in Primus solution
emc69371 is not followed. When SnapView is being updated to Version 2.10.06
the 2 storage processors MUST be able to communicate with each other on the
same LAN in order for the NDU to complete if updating SnapView that is already
loaded. Primus solution emc69371 is a workaround solution to perform an
NDU updated to SnapView 2.10.06 from a LAN crossover cable to one SP.
• If this array is part of a Celerra configuration - A CLARiiON FLARE upgrade
to Release 11 patch 007 requires that the attached Celerra be upgraded to
EMCNAS 5.1.18.3 or above before the FLARE upgrade.

The upgrade of CLARiiON FLARE with an attached Celerra CFS-14 can cause
the Control Station(s) to reboot.
With the customer's permission it is recommended to halt the Control Station(s)
and the Data Movers before the start of the CLARiiON Firmware Upgrade.
When the CLARiiON firmware has completed its upgrade and is fully
operational, the Control Station(s) need to be rebooted and verify proper
operation. If the upgrade of CLARiiON FLARE with an attached Celerra CFS-
14 must be an online event then ask Celerra Level II to examine the Dynamic
Binding Table and verify that the table contains valid WWNs. See ETA
Technical Advisory emc70418

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 5 of 13

CLARiiON FCO F0626031 Instructions

Notes for specific

attached host
environments
AIX Servers
• Stop all I/O from any connected AIX server during the NDU process if
it is running the native HBA driver. See primus emc67186

Red-hat Linux Servers

• Stop all I/O from any connected Red-hat Linux host during the NDU
process.

Solaris Servers
• Ensure that the no-device-delay setting is configured to '0' (zero) in
lpfc.conf. See PRIMUS emc66969
• NDU Delay - The Software Installation window of the Navisphere
Manager GUI has a check-box for NDU delay. When checked, the
default for time will be 360 seconds. When there is a Server zoned to
the array that has DMP fail-over software, use the default of 360
seconds.

Window 2000 Servers

• If the array that has a W2K server zoned to it and that W2K server is
using Dynamic Disks you must use an “NDU delay” of 360 seconds
when a Software NDU is performed. The NDU Wizard automatically
invokes a 360 second delay. It must be manually entered when not
using the Software Install Wizard.

Notes on NDU with

layered Applications
SnapView
• SnapView 1.30 and higher allocate Cache LUNs completely differently
than earlier revisions of SnapView, See clar-psp-073
• Stop all SnapView sessions
• Move Snap Cache LUNs out of the Snap Cache pool prior to NDU.

• Using a Cross-over LAN cable connected to one SP for NDU will

result in failure if updating to SnapView 2.10.06 if the procedure in
Primus solution emc69371 is not followed. When SnapView is being
updated to Version 2.10.06 the 2 storage processors MUST be able to
communicate with each other on the same LAN in order for the NDU to
complete if updating SnapView that is already loaded. Primus solution
emc69371 is a workaround solution to perform an NDU updated to
SnapView 2.10.06 from a LAN crossover cable to one SP.

MirrorView
• Do not “deactivate” a mirror prior to an NDU. In fact, deactivating the
mirror will cause all server access to the primary LUN to be lost until the
Mirror is reactivated.

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 6 of 13

CLARiiON FCO F0626031 Instructions

• Multiple arrays involved in a MirrorView Configuration should not be

targeted during the same software NDU operation you should perform
the NDU on one array at a time.
• Provided the software on the arrays is within 2 revisions, no special
preparation of the mirrors is required prior to an NDU. The array
software ensures that mirrors continue to operate correctly during an
NDU, without requiring a full re-synchronization of any secondary
image. System fractures are a normal part of the NDU operation, but
these result in partial re-synchronization only (which will automatically
start if the mirror recovery policy is set to “Automatic”)

If the destination revision is more than two revisions higher than

the current revision, you have 2 options
• Stop all I/O to the Mirrors before proceeding otherwise they will
become more than two revisions apart after the update of one
of the arrays.
• Update all the array based components of each array in
increments until the final destination, to ensure they are never
more than 2 revisions apart.

Clones
• Do not start an NDU operation while there are more than 4 reverse
synchronization operations active on the array.

San Copy
• All San Copy sessions should be stopped prior to NDU.

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 7 of 13

CLARiiON FCO F0626031 Instructions

Procedure 1. Repair any Faults on the array - Confirm there are no faults on the array
before you begin
2. Confirm valid working failover software and access to both SPs for
each Server with access to the array.
3. Ensure you have new versions of all array based software currently
running.
4. Ensure there are no LUN Rebuilds, Equalizations or RG/LUN
Expansions in progress. If there are, wait for them to complete.
5. Download the appropriate Recovery Image and Utility Partition (if
required) before starting the update.
6. Using Navisphere Manager, view the software tab of “array
properties” to view the “active” array based files that are loaded to the
array.
7. Put all the newer versions of the same files (see Table 1) to be
updated, into the same directory on your Service laptop or server you will
be using to run Navisphere Manager. See the software compatibility table
in this document. As a general rule, all array-based software must be
updated at the same time via the same NDU.
8. Before the software update measure I/O to the first 4 drives of the
array (CX200, CX400 and CX600) and ask the customer to reduce the I/O
if possible. There should be no more than 100 I/Os per sec to the first 4
drives.
9. Stop all snap sessions
10. Move all Snap Cache LUNs out of the Cache LUN Pool. Make note of
which SP is assigned to each LUN.
11. Connect to the array using Navisphere Manager.
12. Enter Engineering mode (note: the Software Installation Wizard can be
used only for the CX200 and CX200LC)
13. Perform the Software update
• Right-click the array icon
• Select the array properties and the array Software tab of the
properties dialogue.
• Choose the update option and browse to the files to be updated
• Load the selected files.
14. Monitor status via the status window which automatically opens. When
the status “operation has completed successfully” appears, the update is
complete.
15. Confirm that the new software is listed as ACTIVE via the Software tab of
the array properties dialogue box.
16. COMMIT the new Base (FLARE or Access Logix) code.
17. Refresh the VIEW of the Browser. This step is necessary to begin using
the new Navisphere UI loaded to the array.

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 8 of 13

CLARiiON FCO F0626031 Instructions

18. Add the Utility Partition? YES if not already @ Release 11

• If updating FROM a version of Core Software prior to Release 11

(see definitions section on page 2), perform a second NDU
Software update to the array to load the Utility Partition.

Note: Loading the Recovery image.ndu file at this point has had
several problems due to its size (approx. 140mb) and frequently the
NDU will fail. Only the Utility Partition needs to be added at this time.
If updating FROM release 11 to this version of Release 11, the Utility
Partition is already there.

The use of the Utility Partition and Recovery Image is something that
hopefully is never needed, but if it is, the Recovery Image (.mif
version) is available on the Global Services WEB page and the
EMC/CLARiiON Partners Page in the Software Download sections.
The procedure to use this .mif file is documented in clar-psp-78.
19. If Power path is running and LUNs are not listed under their default
owner, perform a powermt restore.

20. Ensure that all servers have access to their data

21. Return Snap Cache LUNs to the Cache pool of the appropriate SP If
previously removed. Restart snap sessions as needed.
22. Update CLARalert If installed and test call home

23. Confirm that there are no array faults (the SPS units may still be charging
based upon the SP reboots, this is normal).

24. Confirm that Cache Settings are set properly. If new layered applications
are being added, Cache setting may not be able to be set as they were
previously.

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 9 of 13

CLARiiON FCO F0626031 Instructions

CX-Series Software Compatibility

The Minimum revisions to satisfy this FCO are in shaded cells in the following table. See notes
following this table.
Table 2
Release... Rel. 7 Rel. 8 Rel. 10 Rel. 11 Rel. 11 patched Rel 11
(05/08/03)
Date of .007 patch
release (08/02) (10/02) (12/02) (03/03)

CX600 Flare 02.01.0.60.5.x 02.02.0.60.xxx Not revised 02.04.0.60.5.002 02.04.0.60.5.00 02.04.0.60.5.007

xx 5
CX600 02.01.1.60.xxx 02.02.1.60.xxx Not revised 02.04.1.60.5.002 02.04.1.60.5.00 02.04.1.60.5.007
5
AccessLogix
CX400 Flare Not released 02.02.0.40.5.xxx Not revised 02.04.0.40.5.002 02.04.0.40.5.00 02.04.0.40.5.007
5
CX400 Not released 02.02.1.40.5.xxx Not revised 02.04.1.40.5.002 02.04.1.40.5.00 02.04.1.40.5.007
5
AccessLogix
CX200 Flare Not released Not released CX200 introduced 02.04.0.20.5.002 02.04.0.20.5.00 02.04.0.20.5.007
02.03.0.20.5.xxx 5

CX200 Not released Not released CX200 introduced 02.04.1.20.5.002 02.04.1.20.5.00 02.04.1.20.5.007
5
AccessLogix 02.03.1.20.5.xxx

SnapView 1.5 2.0 Not released 2.10.00 2.10.04 2.10.06

Host 1.5 2.0 Not released 2.1.0.0 Not revised Not revised
ADMsnap 2.1.1 (Linux 2.4)
MirrorView 1.5 1.6 Not released 1.7 1.70.02 Not revised

SanCopy Not released Not released Not released 1.00.0 1.00.02 Not revised

HA Enabler Not released Not released 01.00.02 01.01.01 Not revised Not revised

for CX200
Thirty Drive Not released Not released 01.00.02
Enabler for 01.01.01 Not revised Not revised
CX200
array based 6.1.0 6.2.0 6..2.0
Navisphere (The
6.4.0 & 6.41 UIs Not revised Not revised
Components Management
Svr/Agent went to
6.2.1)
Host Agents 6.1 6.2 6.2 6.4 Not revised Not revised
6.2 Not revised
Navi CLI 6.1 6.2 6.4 Not revised
Not revised

NOTES
• Power Path does not have to be updated if bringing array software to this FCO level.
• If updating from one release to the next, update all the below “array-based” software via the same
software update.
• Arrays shipped from the factory with Release 11 code on them had the Utility Partition and recovery Image
pre-loaded. Reloading the Utility Partition and Recovery Image is NOT required if updating from Release 11
to Release 11 .007 Core.
• There was not a Release 9.
• Host Navisphere Agents follow the array based agent revision (one rev back on the host agent is
acceptable).
• Navisphere UIs revised to 6.4.1 after Release 11 GA and before 11+ GA
• All Release 11 components are compatible, but the latest versions should always be run together.

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 10 of 13

CLARiiON FCO F0626031 Instructions

Appendix A - Measuring I/O to the Boot Drives of a CX array.

To ensure the success of NDU operations on CX200, CX400 and CX600 systems, the I/O level on first
four disk drives (FRUs 0, 1, 2, 3) should be measured prior to executing an NDU operation.

• If the I/O level to any of these disk drives is higher than 100 IOs per second, the NDU should be
postponed or the IO load to the LUs based on these CRUs should be reduced until the I/O load
falls below these thresholds.
• The NDU may Time-out and fail if any of the first 4 drives are performing greater than 100 IOs per
second.

Using Navisphere Manager

1. Enable Performance Statistics

If the gathering of performance statistics is not currently enabled, it must be turned
on. To do this, right click on the CX400/CX600/CX200 array name, then select
"Properties". Select (check) both the "SPA Statistics Logging" and "SP B Statistics
Logging" checkboxes then click the "Update" button.

2. Gather Performance Statistics

Select (expand) the "Physical" attributes Icon in the tree, followed by selecting
(expanding) the "Bus 0 Enclosure 0" attribute selector. The first four FRUs (disk
drives) will be listed as "Bus 0 Enclosure 0 Disk 0", "Bus 0 Enclosure 0 Disk 1", "Bus
0 Enclosure 0 Disk 2", and "Bus 0 Enclosure 0 Disk 3".

For each of these disk drive selectors, right click on the selector, then select
"Properties". Select the "Statistics" tab in the new window. Add together the "Read
Throughput (IO/Second)" and "Write Throughput (IO/Second)" statistics to produce
the current number of I/O requests per second for the indicated CRU (disk drive). If
the values are displayed as "unavailable", go back to step one above to enable
gathering performance statistics.

The I/O requests per second load for the first four disk drives should each average
under 100 I/O requests per second on a CX200, CX400 or CX600.

3. Return Statistics logging to its setting

When complete, ensure that the “check boxes” in the Array Properties dialogue box
which “enable statistics logging” for each SP are returned to the way they were prior
to this procedure.

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 11 of 13

CLARiiON FCO F0626031 Instructions

Using the Navisphere CLI Interface

Enable Performance Statistics

If the gathering of performance statistics is not currently enabled, they must be turned-on
on both SPs. To do this, issue the following commands:

navicli -h <SP-A address> setstats -on

navicli -h <SP-B address> setstats -on

Gather Performance Statistics

The overall procedure for gathering performance statistics, using the first disk drive as an
example:

Example for Drive 0

1. Execute the Navi CLI command from a server with LAN or PPP access to the array.
navicli -h <SP-x address> getdisk 0_0_0

This gives Read and Writes since boot of the array. Note the results

2. Wait period (30 seconds recommended)

3. Request current performance numbers again

navicli -h <SP-x address> getdisk 0_0_0

4. Subtract the original cumulative read and write values returned in step one from the
values returned in step three and then add the two differences together to produce
the total number of I/Os performance for the wait period.

5. Divide the total number of I/Os by the wait period (in seconds) to produce the number
of IOs per second.

Example: First execution = 100,000 IOs (read and write combined)

Second execution = 100,995 (read and write combined)
Subtract 100,000 from 100,095 = 995
995/30 (30 “wait seconds”) = 33.2 I/Os per second for this drive.

The five steps above must be executed twice - once for SPA and once for SP-B for a
given disk and the number of IOs per second added together to produce the total number
of IOs per second for the disk drive.

The five steps above must also be executed on both SPs for all of the first four disk drives
(0_0_0, 0_0_1, 0_0_2, and 0_0_3).

The I/O requests per second load for the first four disk drives should each average under 100 I/O
requests per second on a CX200, CX400 or CX600.

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 12 of 13

CLARiiON FCO F0626031 Instructions

Reducing Array I/O Load

If the I/O load exceeds the recommended limits listed above, it must be reduced for the LUs (disk
Logical Units) based on these disk drives. The Navisphere Manager windows and Navisphere
CLI commands used above to gather the statistics data display the LUNs based on these disk
drives.
When Complete, return the Statistics logging capability to off

navicli -h <SP-A address> setstats -off

navicli -h <SP-B address> setstats -off

EMC2 Global Services, Coslin Drive, Southboro, MA 01772 Page 13 of 13