LG 30apr09

Course ID: 16912
Rev. 9.21
HP Restricted
Use of this material to deliver training without prior written permission from HP is prohibited.
 Copyright 2008, 2009 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice. The only warranties for HP
products and services are set forth in the express warranty statements accompanying such products
and services. Nothing herein should be construed as constituting an additional warranty. HP shall
not be liable for technical or editorial errors or omissions contained herein.
This is an HP copyrighted work that may not be reproduced without the written permission of HP.
You may not use these materials to deliver training to any person outside of your organization
without the written permission of HP.
Intel® and Itanium® are trademarks and registered trademarks of Intel Corporation or its
subsidiaries in the United States and other countries.
Linux® is a U.S. registered trademark of Linus Torvalds.
Microsoft®, Windows®, and Windows NT® are U.S. registered trademarks of Microsoft
Corporation.
UNIX® is a registered trademark of The Open Group.
Printed in USA
HP StorageWorks Enterprise Virtual Array Advanced Troubleshooting
Lab guide
April 2009
HP Restricted — Contact HP Education for customer training materials.
Module 1 Lab 1 — Using the M5x14x Drive Enclosure EMU Serial Port
Objectives......................................................................................................1
Requirements ..................................................................................................1
Introduction ....................................................................................................2
Connecting the special RS232 cable..................................................................3
Starting a HyperTerminal session .......................................................................4
Using the EMU console commands....................................................................5
Module 1 Lab 2 — Using the M6412A Drive Enclosure Serial Port

Objectives......................................................................................................1
Requirements ..................................................................................................1
Introduction ....................................................................................................2
Connecting the serial port cable........................................................................3
Starting a HyperTerminal session .......................................................................4
Using common I/O module serial commands .....................................................5
Reading and analyzing enclosure event logs ......................................................7
Module 6 Lab 1 — Using HSV Controller Diagnostics

Objectives......................................................................................................1
Requirements ..................................................................................................1
Introduction ....................................................................................................2
Using HSV controller diagnostic interfaces ..........................................................3
Using the HSV controller OCP ....................................................................3
Using the HSV controller console port and boot menu....................................4
Using the Field Service Options page ..........................................................7
Additional exercise..........................................................................................9
Module 6 Lab 2 — Analyzing FCS and SCS Outputs

Objectives......................................................................................................1
Requirements ..................................................................................................1
Introduction ....................................................................................................2
Generating the FCS and SCS diagnostic outputs .................................................3
Analyzing SCS_SHOW_CONFIG outputs ..........................................................6
Analyzing FCS_SHOW_CONFIG outputs ........................................................25
Analyzing FCS_LINK_ERRORS outputs ............................................................. 32
Rev. 9.21 HP Restricted 1

Analyzing FCS_DELTA_LINKS outputs ..............................................................39

Lab exercise 1 ........................................................................................39
Answers ................................................................................................ 57
Lab exercise 2 ........................................................................................60
Answers ................................................................................................68
Additional exercise........................................................................................ 71
Module 8 Lab — Using HP Navigator

Objectives......................................................................................................1
Requirements ..................................................................................................2
Introduction ....................................................................................................2
Using the Navigator GUI .................................................................................3
Exploring the user interface........................................................................3
Reviewing the menu options .......................................................................5
Reviewing the tab options ..........................................................................8
Getting real-time configuration data using Navigator ......................................... 10
Accessing configuration data ................................................................... 10
Using the configuration display................................................................. 11
Using different views......................................................................... 11
Using other features.......................................................................... 12
Saving current configurations.................................................................... 12
Module 9 Lab 1 — Generating and Correlating Events

Objectives......................................................................................................1
Requirements ..................................................................................................1
Introduction ....................................................................................................1
How to use this lab..........................................................................................1
Generating and then correlating Controller Event Log entries.................................2
Lab preparation and setup .........................................................................3
Actions to generate and then correlate.........................................................4
Capturing and correlating the event log entries .................................................25
Accidental actions performed while completing this lab...................................... 27
Module 9 Lab 2 — Analyzing Case Studies

Objectives......................................................................................................1
Requirements ..................................................................................................1
How to use this lab..........................................................................................1
Case study 1 ..................................................................................................3
Customer scenario ....................................................................................3
Configuration information ..........................................................................3
Questions ................................................................................................4
2 HP Restricted Rev. 9.21

Contents
Case study 2 ..................................................................................................6

Customer scenario ....................................................................................6
Configuration information ..........................................................................6
Questions ................................................................................................7
Case study 3 ................................................................................................ 11
Customer scenario .................................................................................. 11
Configuration information ........................................................................ 11
Questions .............................................................................................. 12
Case study 4 ................................................................................................ 15
Questions .............................................................................................. 16
Case study 5 ................................................................................................20
Customer scenario ..................................................................................20
Configuration information ........................................................................20
Questions .............................................................................................. 21
Case study 6 ................................................................................................ 24
Questions ..............................................................................................25
Case study 7 ................................................................................................29
Questions ..............................................................................................30
Case study 8 ................................................................................................35
Case study 9 ................................................................................................36
Case study 10 .............................................................................................. 37
Case study 11 ...............................................................................................38
Case study 12...............................................................................................39
Case study 13...............................................................................................40

Case study 14............................................................................................... 41

Module 10 Lab 1 — Using EVAPerf

Objectives......................................................................................................1
Requirements ..................................................................................................1
Introduction ....................................................................................................1
Performance analysis considerations ..................................................................2
Reviewing EVAPerf components.........................................................................4
Files, components, and locations.................................................................4
Friendly names .........................................................................................5
Updating friendly names............................................................................6
Creating the friendly names host file .....................................................6
Generating and viewing friendly names ................................................6
Entering friendly names manually .........................................................7
Using short names ....................................................................................8
Creating and viewing short names........................................................9
Using EVAPerf with Windows Perfmon ............................................................. 10
Objects and counters .............................................................................. 13
Logging performance metrics with Perfmon................................................. 15
Using EVAPerf with a command line ................................................................ 19
Additional CLI lab exercises .....................................................................20
Answers ................................................................................................ 21
Additional lab exercises (optional) ............................................................ 21
Appendix A: EVAPerf displays ........................................................................ 23
HP EVA DR tunnels.................................................................................. 23
HP EVA host connection........................................................................... 24
HP EVA host port statistics ........................................................................25
HP EVA physical disk group ..................................................................... 26
HP EVA storage array.............................................................................. 27
HP EVA storage controller ........................................................................ 27
HP EVA virtual disk .................................................................................28
Appendix B: EVAPerf CLI commands................................................................30
Command options ..................................................................................30
Output formatting options ........................................................................ 31
Module 10 Lab 2 — Using TLViz Formatter Output Files

Objectives......................................................................................................1
Requirements ..................................................................................................1
Introduction ....................................................................................................2

Contents
Viewing an EVAPerf output file in Excel ..............................................................3

Generating individual TLViz output files ..............................................................4
Formatting and graphing TLViz output files..........................................................7
Building the Access database ...........................................................................9
Running database queries .............................................................................. 10
Lab solutions................................................................................................. 12
Viewing an EVAPerf output file in Excel ...................................................... 12
Generating individual TLViz output files...................................................... 12
Formatting and graphing TLViz output files ................................................. 14
Running database queries........................................................................ 14
Module 10 Lab 3 — Formatting and Analyzing TLViz Data

Objectives......................................................................................................1
Requirements ..................................................................................................2
Introduction ....................................................................................................2
Installing and starting the TLViz Viewer...............................................................3
Viewing and using array status information.........................................................4
Viewing and using controller status information ...................................................6
Viewing and using host port statistics information ................................................8
Viewing and using host connections information................................................ 11
Viewing and using port status information ........................................................ 13
Viewing and using physical disk group information............................................ 14
Viewing and using physical disk information..................................................... 17
Viewing and using tunnel statistics...................................................................20
Viewing and using virtual disk group statistics ................................................... 21
Viewing and using virtual disk group statistics ...................................................25
Lab solutions................................................................................................. 27
Viewing and using array status information ................................................ 27
Viewing and using controller status information...........................................28
Viewing and using host port statistics information ........................................29
Viewing and using host connections information .........................................30
Viewing and using port status information .................................................. 31
Viewing and using physical disk group information ..................................... 31
Viewing and using physical disk information ..............................................33
Viewing and using tunnel statistics ............................................................34
Viewing and using virtual disk group statistics.............................................34
Viewing and using virtual disk group statistics.............................................36
Module 10 Lab 4 — Using Additional Performance Tools

Objectives......................................................................................................1
Requirements ..................................................................................................1
Introduction ....................................................................................................2

Using PerfMonkey ...........................................................................................3

Installing and starting PerfMonkey...............................................................3
Navigating the PerfMonkey display.............................................................3
Using the counter functions ........................................................................4
Using the charts........................................................................................4
Using EVApnggen ...........................................................................................6
Installing EVApnggen ................................................................................6
Generating charts.....................................................................................6
Opening and reviewing the charts ..............................................................7

Module 1— Lab 1
Note
If doing this lab remotely, the instructor will ensure that the required equipment is
available and that connections are made for you.
After completing this lab, you should be able to:

 Connect your laptop or PC to the EMU serial port.
 Access and use EMU console commands accessible through the various menus.
The following are the requirements for performing this lab:
Note
Your lab will have some combination of the storage arrays listed below.
 An EVA 3000/5000 series storage system with a current version of VCS

firmware
 An EVA 4000/6000/8000 or EVA 4100/6100/8100 series storage system
with a current version of XCS firmware
 An EMU console port serial cable
 Approximately 45 minutes to perform all of the lab exercises
Rev. 9.21 HP Restricted L1.1 – 1

This lab allows you to become familiar with the most important EMU console
commands used for troubleshooting. You will be able to connect to the console port
and then explore all of the EMU menu and sub-menus available.
As you go through the lab, remember to:
 Read and perform all of the lab steps that you can in the allowable time.
 Concentrate on those tasks that might help you to diagnose and troubleshoot the
array.
L1.1 – 2 HP Restricted Rev. 9.21

Using the M5x14x Drive Enclosure EMU Serial Port
Note
Skip this topic if you are doing the lab remotely.
To connect the EMU console port cable:

1. Locate your RS232 EMU serial cable, part number 17-04875-04.
Note
Your instructor should make this cable available.
2. Plug the RJ45 male connector end of this cable into any EMU RS232 ONLY slot.
Note
The RS232 ONLY slot is the topmost connector on the EMU.
3. Using the diagram below as a reference, plug the J3 Console Commands DB9
female connector into the COM1 port on one of your lab PCs. Optionally, you
could use a personal laptop for this exercise.
Note
Some cables are actually stamped with J2 and J3 on the different DB9 ends and
others are not.
EMU serial cable

On your PC or laptop, do the following:

1. Start a HyperTerminal session.
2. When prompted to choose a name, select any name, such as EMU — Boot
Terminal.
3. When a Connect To dialog box appears, for Connect using, select COM1.
4. A COM1 Properties, Port Settings dialog box appears. Change the settings to
match the settings in this table.
Connection property Setting

Baud rate 115200
Data bits 8
Parity None
Stop bits 1
Flow control None
5. Click OK.

To use the EMU console commands:

1. Click Enter to display the EMU main menu.
Note
When using the EMU commands, you will sometimes see command time-outs.
This does not indicate a device failure. If this happens, retry the command.
--- Main Menu ---

c - CRU Menu
C - Cabinet Bus Menu
e - ESI Menu
E - Error Menu
p - Primary EMU Menu
s – SES Page Menu
S - System Menu
? - Code Version Information
 ?
2. Select ? – Code Version Information to determine the current EMU firmware

version. Note the version here.
................................................................................................................
3. Where did this version of the EMU firmware come from?
................................................................................................................
4. Enter e to select the ESI menu.
5. From the ESI menu, enter m and choose a populated disk slot.
Do the listed AL_PAs match those of your storage system configuration?
................................................................................................................
6. Select d – Summary Of All Drive Info.
Note
You will not be able to do this step if you are performing the lab remotely.
a. Physically remove a drive from the drive enclosure you are monitoring.
b. Select d once again, then explain the difference between this output and
the one captured before the disk was removed.
.........................................................................................................
.........................................................................................................

c. Reinsert the drive you just removed. Immediately enter and continue entering
d to continuously output the results of the Summary Of All Drive Info
command.
Answer the following questions:
1) When did the WWN of the drive become known?
...................................................................................................
2) When the drive was inserted, which device removed the bypass first?
...................................................................................................
...................................................................................................
7. Enter ESC and then E to enter the Error menu.
8. Select a. Record the latest error below.
................................................................................................................
Note
The duration value indicates the length in which the error was occurring.
9. Enter ESC, then S to enter the System menu.

10. From the System menu, select t - Type/Revision Data. Use the output of the
display to answer the following questions:
a. What is the serial number of the EMU?
.........................................................................................................
b. What is the firmware build number of the EMU?
.........................................................................................................
c. What is the WWN of the drive enclosure shelf?
.........................................................................................................
d. What is the base Loop ID of this drive enclosure shelf?
.........................................................................................................
e. Which internal revision numbers are the two installed I/O modules?
.........................................................................................................
.........................................................................................................
Note
The internal revision numbers are not the spare part numbers printed on the I/O
modules themselves.

11. Select N – Display I2C NVRAM Resources from the System menu. View the
displayed output. This is another method to determine particular drive enclosure
information.
12. Return to the EMU main menu.
13. Select e – ESI Menu, then d – Summary of All Drive Info. Using the information
displayed, answer the following questions:
a. Which drive bays are populated?
.........................................................................................................
b. Which drive firmware version is being used?
.........................................................................................................
c. Which AL_PAs are being used on the shelf?
.........................................................................................................
.........................................................................................................
14. Observe this output and answer the following questions.
## ….WWN…. Pos-A/B AL-PA Cp Rate FRev Min/Temp/Max EDV Byp-A/B

01 (no drive) 000/00 FF/FF 00 00/0 -20/-20C/- Off -EDB/-EDB
0 0 20
0 0 20
0 0 20
04 20000004CF393FFA 012/02 66/6 C 06/0 3BE8 22/ 23C/ On ----/----
0 6 0 6 26
05 20000004CF39A0D 013/02 65/6 C 06/0 3BE8 22/ 23C/ On ----/----
E 1 5 0 6 26
06 20000004CF399FF1 014/02 63/6 C 06/0 3BE8 23/ 24C/ On ----/----
2 3 0 6 26
07 20000004CF399E57 015/02 5C/5 C 06/0 3BE8 23/ 24C/ On ----/----
3 C 0 6 26
08 20000004CF395379 016/02 5A/5 C 06/0 3BE8 18/ 24C/ On ----/----
4 A 0 6 26
09 20000004CF39A0E1 017/02 59/5 C 16/0 3BE8 24/ 25C/ On --DB/----
5 9 0 6 29
0 0 20
0 0 20
0 0 20
0 0 20
0 0 20

a. Explain the output of the bypass data for the disk in slot 9.
.........................................................................................................
.........................................................................................................
b. Is the slot bypassed by the EMU or by the disk itself?
.........................................................................................................
15. Return to the EMU main menu and select c - CRU Menu.
16. Select d – Drive Menu. The output should be similar to the following display.
--- Drive CRU Menu ---
b - Bypass reasons menu
c – Clear the drive present trace
i - Perform an INQUIRY command via EiESI
l - Set LED Behavior
p – Print drive present here
t – Print drive present tracing (On)
ESC - previous menu
-->
17. Return to the EMU main menu and select E - Error Menu. Output should be
similar to the following display.
--- Error Menu ---
a - Recent Alarm Log Listing ('A' for all)
d - Display Current Alarm Queue
f - Recent Fatal System Error Listing
l - Lookup/decode an error message
m - Mute alarm (Off)
r - Toggle the REMIND bit (Off)
t - Test errors and alarms
ESC - previous menu
-->
18. If you select a – Recent Alarm Log Listing (‘A’ for all), the last 10 events logged
by this EMU will display. If you select A, the last 63 events logged by this EMU
will display. Select a.
The output should be similar to the following display.

Current time stamp is: 0088838585 (054B91B9h)

Alarms from the EEProm FIFO (most recent last):
Nn - Age (sec) - Snnnnnnnnnn-TTNNEE: Description
-- ---------- ----------------------------------
01 - 104455 - C 88734130-0F0405:Transceiver 4 invalid character; check module
Duration: 8 seconds
02 - 89710 - C 88748875-0F0402:Transceiver 4 lost signal; check cabling
Duration: 7 seconds
Duration: 26 seconds
06 - 88649 - N 88749936-0F0404:Transceiver 4 removed; install transceiver.
Duration: 8 seconds
07 - 88296 - N 88750289-870213:I/O module 2 removed; install module.
09 - 81443 - C 88757142-870202:I/O module 2 communication error; reseat
or replace I/O module.
Duration: 6 seconds
10 - 81437 - N 88757148-870213:I/O module 2 removed; install module.
19. From the Error menu, select the l – Lookup/decode an error message option to
look up any of the previously listed TTNNEE numbers, for example, 0F0405 in
01 in the list above.
--- Error Menu ---
a - Recent Alarm Log Listing ('A' for all)
d - Display Current Alarm Queue
f - Recent Fatal System Error Listing
l - Lookup/decode an error message
m - Mute alarm (Off)
r - Toggle the REMIND bit (Off)
t - Test errors and alarms
ESC - previous menu
--> l

You might see a display in the following format for your error code:
Enter an error code in the form 'tteenn', letter codes (c, h) or
empty line to exit. --> 0F0405 Searching for: 0F.04.05…
Helptext Msg:
--> Transceiver 4 invalid character; check module.
LCD Msg:
--> 0F0405:I/O Xcvr 4 invalid character; check module.
Enter an error code in the form 'tteenn', letter codes (c, h)
or empty line to exit. -->
You have completed this lab.

Module 1 — Lab 2
Note

 Connect your laptop or PC to the M6412x drive enclosure I/O module serial
port.
 Access and use the command line interface to enter commands.
 Download and analyze drive enclosure event logs.
Note
 An EVA 4400/6400/8400 storage system with a current version of XCS

firmware
 An M6412x drive enclosure I/O module serial cable
 HP Navigator
 Approximately 1 hour to perform all of the lab exercises

This lab allows you to become familiar with the most important M6412x serial
commands used for troubleshooting. You will be able to connect to the serial port on
an M6412x drive enclosure I/O module, explore the most important CLI commands
available, and download and analyze logs.
array.

Using the M6412x Drive Enclosure Serial Port
Note
Skip this topic if you are doing the lab remotely.
To connect the M6412x I/O module serial port cable:

1. Locate your serial cable, probably labeled part number 259992-001, but
separately orderable as 316131-001.
Note
Your instructor should make this cable available. This cable is also used to
connect to the controller.
2. Plug the RJ45 male connector end of this cable into the I/O module of a
selected drive enclosure.
Note
The Mfg slot is in the right-most position on the I/O module.
3. Plug the DB9 female connector into the COM1 port on one of your lab PCs.
Optionally, you could use a personal laptop for this exercise.

On your PC or laptop, do the following:

1. On your PC or laptop, start a HyperTerminal session.
2. When prompted to choose a name, select any name, such as I/O Module —
Boot Terminal.
3. When a Connect To dialog box appears, for Connect using, select COM1.
4. A COM1 Properties, Port Settings dialog box appears. Change the settings to
match the settings in this table.

Baud rate 115200
Data bits 8
Parity None
Stop bits 1
Flow control None
5. Click OK.

To use the most common I/O module serial commands:

1. Click Enter to display the initial prompt.
2. What does the prompt mean, for example, if the prompt is 01a4:0001>, what
do each of the elements signify?
................................................................................................................
3. Enter menu to display the list of commands. Your display should look similar to
the following:

4. Enter info to display shelf information. Note what you see here.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
5. Enter stat to display the dynamic status of the enclosure elements. Note what
you see here.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
6. Enter dbs 1 to display page 1 of the database. Note what you see here.
................................................................................................................
7. Enter wellRd 0 to read from the wellness log for the enclosure link monitor. Note
what you see here.
................................................................................................................
8. Enter ctsErrCnt a to display Cut-Through Switch error count absolutes in hex
format. Note what you see here.
................................................................................................................
9. Enter eeRead 0 to read from the EEPROM for the enclosure link monitor. Note
what you see here.
................................................................................................................
10. Enter fanTach to display the fan tachometer settings in 1-second intervals. Note
what you see here.
................................................................................................................
Enter fanTach again to disable it.

Read the enclosure event log by doing the following:

1. Using HyperTerminal, select Transfer  Capture Text and enter a file name to
store the captured text, for example, c:\logread1.txt.
2. Enter logRead 1 to display the contents of the enclosure event log. What are the
most common entity codes and event codes?
................................................................................................................
................................................................................................................
................................................................................................................
3. Using HyperTerminal, select Transfer  Capture Text  Stop.
4. Use WordPad to open the captured text file. This is the file you will import into
the HP Navigator translation tool.
5. Enter logRead 2 to display the contents of the temperature log. What are the
most common entity codes and event codes?
................................................................................................................
................................................................................................................
................................................................................................................
6. Start Navigator by using the Start menu or double-clicking the icon or shortcut
on the desktop.
Note
For now, you will use Navigator only to import and view an M6412x shelf log.
You will examine all of the features of Navigator in a later lab.
7. Select File  Open  Workspace to open a new workspace (.nw). An empty

workspace is displayed.
8. Select File  Import  Storage Shelf Log to import the shelf log file that you
saved through the HyperTerminal session.

The shelf log data files are used to examine specific hardware issues at the M6412x
shelf level. Each log contains information specific to one module on one loop in one
shelf. Each log file can contain data from multiple modules.
To analyze the shelf data in Navigator:
1. Double-click any of the one or more entries in the Workspace tab.
2. View the log data in the Reports tab.
3. Filter your log for each of the following entity types (click below the column
heading and use the down arrow):
 Event Log
 Power Supply
 Fan
 Temp Sensor
 ELMo (I/O module)
 Display
 XCVR
 Array Device
 Console
 Enclosure
4. Display the entire log again.

5. Filter your log for entity type ELMo events and note the types of events and their
meanings.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
6. Filter your log for entity type Array Device events and note the types of events
and their meanings.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
7. Filter your log for entity type Fan events and note the types of events and their
meanings.
................................................................................................................
................................................................................................................

8. Filter your log for event codes of type Reset and note their meanings.
a. Note why the resets were issued.
.........................................................................................................
b. Note if there were any unexpected resets.
.........................................................................................................
9. Determine if shelf IDs were set from the console or from the midplane.
................................................................................................................

Module 6 — Lab 1
Note

 Use the HSV controller OCP to set debug or print flags.
 Use the HSV console port and boot menu to set debug or print flags.
 Use the Command View EVA Field Service Options page to set debug or print
flags, and to set debug commands.
Note

firmware
 An EVA 4400/6400/8400 series storage system with a current version of XCS
firmware
 Controller serial port cables for access to the different arrays
Note
The HSV300 and HSV4x0 controllers use the same serial port cable as the
M6412x drive enclosure. The HSV1x0 and HSV2x0 controllers use a different
serial port cable from the EMU console port.

This lab is allows you to become familiar with the most important HSV controller
diagnostics used for troubleshooting. You will be able to view and set debug flags
and print flags, as well as input debug commands. You will also explore all of the
interfaces available for viewing and using HSV controller diagnostics.
array.
Note
Every lab configuration may be different, therefore keep in mind that the lab
exercises are not written for a specific configuration or one having a specific rack
infrastructure.

Using HSV Controller Diagnostics
Setting flags and using debug commands to capture diagnostic outputs should rarely
be necessary when troubleshooting EVA storage systems. Typically, the controller
event log provides you with all of the information you need to determine the cause of
an EVA storage system failure.
There will be times when you will need to access and use debug flags, print flags,
and debug commands. For example, the PROMPT_FOR_GO debug flag may be set
at the factory and you may not have access to the OCP.
The subtopics describe how to access and use flags and commands.
Using the HSV controller OCP

Note
Skip this topic if you are doing the lab remotely. The instructor will do this for
you.
The EVA 3000/5000, EVA 4000/6000/8000, EVA 4100/6100/8100, and EVA

6400/8400 series arrays have a controller OCP that allows access to debug and
print flags.
Follow these steps to access the debug flags:
Note
The steps below are written for the EVA 4000/6000/8000 and EVA
4100/6100/8100 series arrays, but are similar for the EVA 3000/5000 and
EVA 6400/8400 series. The process for print flags is similar.
1. On the master controller, press any OCP pushbutton while the default display is
in view.
Note
Determine the master controller by navigating through the System Information
menu.
2. Select System Information and press the right arrow.

3. Press the down arrow to sequence down through the System Information menu
options until Debug flags is displayed.
4. Press the right arrow.

5. Turn on the debug flag for PROMPT_FOR_GO (00000001) and

PRINTF_TO_CONSOLE (00000008) by doing the following.
a. Press the down arrow to sequence down through all of the debug flags.
b. Press the right arrow to toggle the flag off or on.
c. Press ENTER to set the flag.
6. Press ENTER to invoke the changes and return to the main OCP display.
Note
These changes do not go into effect until you restart the controller (see below).
7. On the other controller, navigate to the print flags.

8. View but do not change any of the print flag values.
Using the HSV controller console port and boot menu

You should also be able to use the controller console port and boot menu to access
debug and print flags. This is especially important when the PROMPT_FOR_GO
debug flag is set at the factory and you have no access to the OCP.
More importantly, you must use the HSV controller console port to display and
capture output from debug commands that you enter through the Command View
EVA Field Service Options page, which you will use later.
Follow these steps to reset the debug flags that you set through the OCP:
Note
The steps below are written for the EVA 3000/5000, EVA 4000/6000/8000,
and EVA 4100/6100/8100 series arrays, but are similar for the EVA
3000/5000 and EVA 6400/8400 series.
1. Start a HyperTerminal session to the console output of the same HSV controller
as you set the debug flags for PROMPT_FOR_GO and PRINTF_TO_CONSOLE.
Use the port settings indicated in the following table.

Baud rate 19200
Data bits 8
Parity None
Stop bits 1
Flow control None
You are now ready to use the boot menu.

2. Press CTRL/r to restart the controller.

After the controller reboots, note what is displayed on the OCP (it may show
“Waiting for user input”).
Note
If using the lab remotely, you will not be able to note the OCP display.
This denotes that you have set the PROMPT_FOR_GO flag to 1. You must use
the console port and boot menu to change the value back to 0.
3. The controller should have stopped during the boot sequence and displayed the
boot menu, which includes the current settings of the debug and print flags. The
debug flags value should be 9, which you set through the OCP.
Note
The debug flag value is displayed as the sum of the bitmap values of the
individual flags. For example, a value of 02000009 is the combined setting of
VERBOSE_MODE (02000000), PRINTF_TO_CONSOLE (00000008), and
PROMPT_FOR_GO (00000001).
4. Enter f for debug flags, then enter g to bring up the prompt for debug flags.
Note
Though not shown here, you would perform similar steps for print flags.

5. At the prompt, enter a value of 0 to turn off the debug flags.
6. Press Ctrl/j to terminate the flag string.

The controller boots and executes any flags set through the boot menu.
Note
You will no longer be given any prompts because the PROMPT_FOR_GO flag is
now 0. Any time you want to return to the boot menu, press CTRL/p (halts the
controller) and CTRL/r (to restart the controller) on the PC keyboard.
Important
! You should normally return the debug flags to a value of 0 before leaving the
console port and boot menu.

Using the Field Service Options page

You can also use the Command View EVA Field Service Options page to view and
set debug and print flags, but more importantly, you can issue debug commands and
view and capture the output on the HSV controller console port.
Follow these steps:
Note
The steps below are written for the EVA 3000/5000, EVA 4000/6000/8000,
EVA 4100/6100/8100, and EVA 6400/8400 series storage systems.
1. Start a Command View EVA session to the storage system and go to the Field
Service Options page.
Note
You may have to stop and start the Command View EVA service to see both
controllers.
2. Select a storage system to work with, and then select the Open command line
interface option.
3. Check the debug flags by clicking Get Debug Flags, then clicking Execute.
Determine if the debug flags are zeroed.
4. Whether or not the debug flags are not zeroed, use the Set Debug Flags
command to zero them.
5. You are now ready to enter a sample debug command.
Enter the value 05 in the “Enter hex equivalent of command:” field and click
Execute.
Note
Value 05 is to generate the SCS_SHOW_CONFIG output. You will examine the
output of this and other debug commands in the next lab.
Note
In VCS 2.003 and later, the PRINTF_TO_CONSOLE flag is automatically set and
cleared by the four primary diagnostic commands: SCS_SHOW_CONFIG,
FCS_SHOW_CONFIG, FCS_LINK_ERRORS, and FCS_DELTA_LINKS. However, if
a controller crashes or resyncs during one of these commands, the
PRINTF_TO_CONSOLE flag may be left set.

SCS_SHOW_CONFIG request
The GUI should indicate that the operation succeeded.

6. Check your HyperTerminal session to ensure that you received output of the
debug command. Click OK.

If you have access to an HSV300 controller on an EVA4400, use the console port
and boot menu and the Field Service Options page to set and view debug and print
flags. Note that the web-based OCP (WOCP) on the EVA4400 does not allow
access to flags.
For the new HyperTerminal session, use a 115200 baud rate.
Note what you observe here.
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................


Module 6 — Lab 2
Note

 Generate SCS_SHOW_CONFIG, FCS_SHOW_CONFIG, FCS_LINK_ERRORS and
FCS_DELTA_LINKS outputs.
 Analyze SCS_SHOW_CONFIG, FCS_SHOW_CONFIG, FCS_LINK_ERRORS and
FCS_DELTA_LINKS outputs.
Note

firmware
 An EVA 4400/6400/8400 series storage system with a current version of XCS
firmware
 Controller serial port cables for access to the different arrays
Note
The HSV300 controller on the EVA4400 uses the same serial port cable as the
M6412 drive enclosure. The HSV1x0 and HSV2x0 controllers use a different
serial port cable from the EMU console port.
 Approximately 2 hours to perform all of the lab exercises

Being able to generate and analyze the outputs of various diagnostic tools is
essential for effective troubleshooting. This lab leads the student through the
generation and analysis of the outputs for the SCS_SHOW_CONFIG,
FCS_SHOW_CONFIG, FCS_LINK_ERRORS, and FCS_DELTA_LINKS commands.
Note
Remember that SCS_SHOW_CONFIG (05) and FCS_SHOW_CONFIG (29) data
can also be displayed in Navigator for easier reading and analysis.

Analyzing FCS and SCS Outputs
Using debug commands to capture diagnostic outputs should rarely be necessary

when troubleshooting EVA storage systems. Typically, the controller event log
provides you with all of the information you need to determine the cause of an EVA
storage system failure.
Caution
Although this lab will step you through the generation and analysis of various
diagnostic outputs, debug commands should be utilized only as a final effort
when troubleshooting.
Use the following process to generate FCS and SCS diagnostic outputs:
1. Start a HyperTerminal session to the console output of the primary HSV
controller.
2. Select Transfer  Capture Text and use a file name such as debug_outputs.txt
as the destination of the captured controller output.
3. Start a Command View EVA session to the storage system and go to the Field
Service Options page.
4. Select a storage system to work with, and then select the Open command line
interface option.
Note
In VCS 2.003 and later, the PRINTF_TO_CONSOLE flag is automatically set and
cleared by the four primary diagnostic commands: SCS_SHOW_CONFIG,
FCS_SHOW_CONFIG, FCS_LINK_ERRORS, and FCS_DELTA_LINKS. However,
if a controller crashes or resyncs during one of these commands, the
PRINTF_TO_CONSOLE flag may remain set.
5. Check the debug flags by clicking Get Debug Flags, then clicking Execute. If the
debug flags are not zeroed, use the Set Debug Flags command to zero them.

6. Enter the value 05 in the “Enter hex equivalent of command:” field and click
Execute.
Note
Value 05 is to generate the SCS_SHOW_CONFIG output.
SCS_SHOW_CONFIG request
7. The GUI should indicate Operation Succeeded and the HyperTerminal session
should show the output of the debug command. Click OK.
8. After waiting at least 1 minute, enter the value 29 into the “Enter hex equivalent
of command:” field and click Execute.
Note
Value 29 generates the FCS_SHOW_CONFIG output.
Caution
Do not issue debug commands in quick succession. Doing so may cause the
storage system to crash.

Note
Value 34 generates the FCS_LINK_ERRORS output.
Note
Value 35 generates the FCS_CLEAR_LINKS output.
Note
Value 36 generates the FCS_DELTA_LINKS output.
16. You are finished using the Command Line Interface area of the Field Service
Options page. Before exiting the page, always check to ensure that you have
not accidentally left any debug flags set.
a. Under the “Select command from list:” drop-down menu, select Get Debug
Flag. This should show a hex equivalent of command value of 10.
b. Click Execute. The output should display as 0x00000000.
c. If the displayed value is anything other than 0x00000000, you must
execute the Set Debug Flag option with an Argument 1 value of 0.
17. Exit the Field Service Options page.
18. Using HyperTerminal, select Transfer  Capture Text  Stop.
19. Use WordPad to open the c:\debug_outputs.txt file.

The following is a full-length example of an SCS_SHOW_CONFIG output. Read

through the example and the associated notes, and try to understand all of the fields
associated with this output.
As you work through this lab section, you may want to analyze your own storage
system SCS_SHOW_CONFIG output generated in the first lab section.
Note
Instructional comments in the form of Notes, similar to the one you are reading,
are included throughout the SCS_SHOW_CONFIG output. In addition, the more
important data fields of the output are bolded and edited with comments. Tables
have been added to describe various column headings.
SCell: Active MFC via Port 4  The storage system state is active, and the controller
communications are active via the Mirror port.
6-00508b-4000142c2x 0008c-000003d-[0008]x  This is the storage system UUID
SCell Master - Quorum POIDs:801x 80ax 806x 807x 800x  These are the quorum disk
POIDs (Physical Object IDs) associated with this storage system.
NSC: UUID = 5-00508b-40001464cx 00000-0000000-0000x  This is the UUID of the primary
controller at the time this output was generated.
MEMBER_TAG = 6-00508b-4000142c2x 0008c-000004e-[0021]x  This is the UUID of the
secondary controller at the time this output was generated. In some instances, the
MEMBER_TAG will show the UUID of the primary controller.
NOID type Range of values

LDADs 100-1FF
RSS 200-23F
Volumes 400-7FF
PStores 800-FFF
SCVDs 1000-17FF
Ldisks 2000-3FFF
Clients 8000-87FF
Punits 8800-8FFF
Dunits A800-AFFF
NOID table

Column heading Description Possible values

Lp Loop port  0 = Loop 1A
 1 = Loop 1B
 2 = Loop 2A
 3 = Loop 2B
ALPA Arbitrated Loop address ALPA of the disk on the indicated loop.
Type Type of device  Disk — When it is a disk
 NSC — When it is a storage controller
Node WWN Node WWN of the disk
FNB Do not use. Ignore all do not use fields.
Temp NOID Temporary NOID  0 — If the device is allocated to a disk group
 900+ — If the device is ungrouped
ID State/ID Usablty Current state and usability  Valid/Usable — If the disk is in a disk group
of the indicated NOID  Invalid/Usable — If the disk is not grouped,
but is available
 Unknown/Usable — If the disk is not grouped
and is not available.
FC Nodes: Temp ID ID
Lp ALPA Lp ALPA Type Node WWN FNB Noid State Usablty
0. 9ex 1. 9ex disk 2-000-0004cf-399f00x 33603c 900 Invalid Usable
2. adx 3. adx disk 2-000-0004cf-395002x 33b010 0 Valid Usable
0. 63x 1. 63x disk 2-000-0004cf-29e205x 3380fc 902 Invalid Usable
0. b1x 1. b1x disk 2-000-0004cf-bf3d08x 33580c 0 Valid Usable
2. b2x 3. b2x disk 2-000-0004cf-398608x 33a9ec 904 Invalid Usable
2. 9bx 3. 9bx disk 2-000-0004cf-29e110x 33b840 0 Valid Usable
0. 9bx 1. 9bx disk 2-000-0004cf-399e14x 336454 0 Valid Usable
0. 31x 1. 31x disk 2-000-0004cf-394819x 339b98 907 Invalid Usable
2. 31x 3. 31x disk 2-000-0004cf-2cab1bx 33eb6c 908 Invalid Usable
2. 73x 3. 73x disk 2-000-0004cf-29e31dx 33c8a0 0 Valid Usable
2. 75x 3. 75x disk 2-000-0004cf-39531ex 33c488 90a Invalid Usable
2. 72x 3. 72x disk 2-000-0004cf-39a029x 33caac 90b Invalid Usable
0. 59x 1. 59x disk 2-000-0004cf-241c2ax 338720 0 Valid Usable
2. 79x 3. 79x disk 2-000-0004cf-399f34x 33c070 90d Invalid Usable
2. b3x 3. b3x disk 2-000-0004cf-39a034x 33a7e0 90e Invalid Usable
0. 66x 1. 66x disk 2-000-0004cf-1fe247x 337ad8 90f Invalid Usable
0. 2ex 1. 2ex disk 2-000-0004cf-39604cx 339da4 0 Valid Usable
2. 5cx 3. 5cx disk 2-000-0004cf-399e57x 33d2dc 0 Valid Usable
2. 32x 3. 32x disk 2-000-0004cf-395359x 33e960 912 Invalid Usable
0. 5ax 1. 5ax disk 2-000-0004cf-395e6ax 338514 0 Valid Usable
2. aex 3. aex disk 2-000-0004cf-395b6fx 33ae04 0 Valid Usable

0. b2x 1. b2x disk 2-000-0004cf-39866fx 335600 915 Invalid Usable

2. 5ax 3. 5ax disk 2-000-0004cf-395379x 33d4e8 0 Valid Usable
2. 2dx 3. 2dx disk 2-000-0004cf-2cae84x 33ef84 0 Valid Usable
2. b1x 3. b1x disk 2-000-0004cf-1fed8bx 33abf8 0 Valid Usable
0. 2cx 1. 2cx disk 2-000-0004cf-393d90x 33a1bc 0 Valid Usable
0. 45x 1. 45x disk 2-000-0004cf-940c92x 339574 0 Valid Usable
0. 32x 1. 32x disk 2-000-0004cf-29d493x 33998c 91b Invalid Usable
0. 2dx 1. 2dx disk 2-000-0004cf-393694x 339fb0 0 Valid Usable
0. 98x 1. 98x disk 2-000-0004cf-395e94x 336660 0 Valid Usable
2. 49x 3. 49x disk 2-000-0004cf-393697x 33df24 0 Valid Usable
2. 4ax 3. 4ax disk 2-000-0004cf-395498x 33dd18 91f Invalid Usable
2. 2ex 3. 2ex disk 2-000-0004cf-241f9ax 33ed78 0 Valid Usable
0. 65x 1. 65x disk 2-000-0004cf-39379bx 337ce4 921 Invalid Usable
0. 2bx 1. 2bx disk 2-000-0004cf-393c9bx 33a3c8 0 Valid Usable
0. aex 1. aex disk 2-000-0004cf-39869bx 335a18 0 Valid Usable
2. 4bx 3. 4bx disk 2-000-0004cf-393ea1x 33db0c 924 Invalid Usable
2. 98x 3. 98x disk 2-000-0004cf-394aa1x 33ba4c 0 Valid Usable
0. 8fx 1. 8fx disk 2-000-0004cf-393da2x 336c84 0 Valid Usable
2. 47x 3. 47x disk 2-000-0004cf-3935a4x 33e130 0 Valid Usable
0. 47x 1. 47x disk 2-000-0004cf-393ea4x 33915c 0 Valid Usable
0. acx 1. acx disk 2-000-0004cf-394aadx 335e30 929 Invalid Usable
2. 90x 3. 90x disk 2-000-0004cf-3986adx 33be64 0 Valid Usable
0. 49x 1. 49x disk 2-000-0004cf-3964afx 338f50 0 Valid Usable
0. 4ax 1. 4ax disk 2-000-0004cf-29e3b3x 338d44 92c Invalid Usable
0. 74x 1. 74x disk 2-000-0004cf-3937b4x 3374b4 0 Valid Usable
2. 76x 3. 76x disk 2-000-0004cf-3937b7x 33c27c 92e Invalid Usable
2. 2cx 3. 2cx disk 2-000-0004cf-3949b8x 33f190 0 Valid Usable
0. 33x 1. 33x disk 2-000-0004cf-3960bcx 339780 931 Invalid Usable
0. 75x 1. 75x disk 2-000-0004cf-1fecbex 33709c 0 Valid Usable
0. 90x 1. 90x disk 2-000-0004cf-3949bfx 336a78 0 Valid Usable
2. 97x 3. 97x disk 2-000-0004cf-39a0c3x 33bc58 0 Valid Usable
0. 76x 1. 76x disk 2-000-0004cf-bf40c9x 3372a8 935 Invalid Usable
0. 4bx 1. 4bx disk 2-000-0004cf-3949cfx 33892c 936 Invalid Usable
2. acx 3. acx disk 2-000-0004cf-39a0cfx 33b21c 0 Valid Usable
0. b3x 1. b3x disk 2-000-0004cf-1fe1cfx 3353f4 938 Invalid Usable
0. 46x 1. 46x disk 2-000-0004cf-399fd1x 339368 0 Valid Usable
0. 4cx 1. 4cx disk 2-000-0004cf-1fe0d1x 338b38 93a Invalid Usable
2. 4cx 3. 4cx disk 2-000-0004cf-2cacd5x 33d900 93b Invalid Usable

0. adx 1. adx disk 2-000-0004cf-3964d6x 335c24 0 Valid Usable

0. 5cx 1. 5cx disk 2-000-0004cf-3940dbx 338308 0 Valid Usable
0. 72x 1. 72x disk 2-000-0004cf-3947ddx 3378cc 0 Valid Usable
0. 73x 1. 73x disk 2-000-0004cf-399fddx 3376c0 0 Valid Usable
2. 65x 3. 65x disk 2-000-0004cf-39a0dex 33cec4 940 Invalid Usable
2. 9dx 3. 9dx disk 2-000-0004cf-3943e1x 33b428 941 Invalid Usable
2. 59x 3. 59x disk 2-000-0004cf-39a0e1x 33d6f4 0 Valid Usable
2. 46x 3. 46x disk 2-000-0004cf-399ee5x 33e33c 0 Valid Usable
0. 97x 1. 97x disk 2-000-0004cf-29e2e6x 33686c 0 Valid Usable
2. 9ex 3. 9ex disk 2-000-0004cf-399ee9x 33b634 945 Invalid Usable
2. 74x 3. 74x disk 2-000-0004cf-241febx 33c694 0 Valid Usable
0. 9dx 1. 9dx disk 2-000-0004cf-399fefx 336248 947 Invalid Usable
0. 79x 1. 79x disk 2-000-0004cf-3986f1x 336e90 948 Invalid Usable
2. 63x 3. 63x disk 2-000-0004cf-399ff1x 33d0d0 949 Invalid Usable
2. 33x 3. 33x disk 2-000-0004cf-241cf2x 33e754 94a Invalid Usable
2. 66x 3. 66x disk 2-000-0004cf-393ffax 33ccb8 94b Invalid Usable
0. 2x 1. 2x --
2. 2x 3. 2x NSC 5-00508b-40001464cx 337ef0 0  This line and the line above

it indicate that the 464c controller is active on all four device-side loops with an ALPA of 2.
0. 1x 1. 1x --
2. 1x 3. 1x --
4. 1x NSC 5-00508b-4000142c2x 33a5d4 0  Loop 4 here indicates the HSV

controller mirror port.

NSCNoid HSV Controller NOID 20x
21x
WWN UUID of controller UUID of indicated controller
NUB Do not use.
Gate Do not use.
InUse nub list:

NSCNoid WWN
Member Tag NUB Gate
21x 5-00508b-40001464cx
6-00508b-4000142c2x 0008c-000004e-[0021]x 3a50d4 1x
20x 5-00508b-4000142c2x
6-00508b-4000142c2x 0008c-000004c-[0020]x 3a5150 0x

Column heading Description Values

Lp Loop port the indicated 0 = Loop 1A
POID is online to 1 = Loop 1B
2 = Loop 2A
3 = Loop 2B
ALPA Arbitrated Loop Physical ALPA of the disk
Address
Poid Physical Object ID POID of the indicated disk appended with ‘Q’ if
the disk is a quorum disk.
Volnoid Volume NCS Object ID NOID of the volume associated with the POID
MAIRFA Indicates the disk status M = Missing
A = Abnormal
I = Inoperative
R = Rmap Valid
F = Failure predicted
A = Attempt Replace
CSMM Do not know. IDLE
WWN World Wide Name of the
associated disk
DUB Do not use.
Gate The gate is how you stop The gate column indicates if the gate is open or
work to a drive so you closed, and if closed, why it is closed.
can change
configurations, discover
new drives or perform
other similar loop
operations.
rss id The RSS ID this disk is a Notice the disk devices with the even RSS IDs are
member of online to the even loops (0 & 2) and the odd RSS
IDs are online to the odd loops (1 & 3).
It is not uncommon to have the RSS IDs not run in
consecutive numerical order.
rss ndx The RSS index of the Note: The index values do not have to run in
associated disk consecutive numerical order.
InUse dub list: rss rss

Lp ALPA Poid Volnoid MAIRFA CSMM WWN DUB Gate id ndx
0. 2bx 801xQ 401x IDLE 2-000-0004cf-393c9bx 1add520x 0x 006 09
0. 59x 80axQ 40ax IDLE 2-000-0004cf-241c2ax 1add860x 0x 002 01
2. acx 806xQ 406x IDLE 2-000-0004cf-39a0cfx 1addba0x 0x 006 06
2. 90x 807xQ 407x IDLE 2-000-0004cf-3986adx 1addee0x 0x 006 07
0. 8fx 800xQ 400x IDLE 2-000-0004cf-393da2x 1ade220x 0x 006 01
2. adx 812x 412x IDLE 2-000-0004cf-395002x 1ade560x 0x 002 06
1. b1x 828x 428x IDLE 2-000-0004cf-bf3d08x 1ade8a0x 0x 003 07
2. 9bx 82bx 42bx IDLE 2-000-0004cf-29e110x 1adebe0x 0x 004 02
1. 9bx 827x 427x IDLE 2-000-0004cf-399e14x 1adef20x 0x 003 05

3. 73x 815x 415x IDLE 2-000-0004cf-29e31dx 1adf260x 0x 003 03

1. 2ex 823x 423x IDLE 2-000-0004cf-39604cx 1adf5a0x 0x 003 06
2. 5cx 825x 425x IDLE 2-000-0004cf-399e57x 1adf8e0x 0x 004 05
1. 5ax 816x 416x IDLE 2-000-0004cf-395e6ax 1adfc20x 0x 005 01
3. aex 81ex 41ex IDLE 2-000-0004cf-395b6fx 1adff60x 0x 005 06
3. 5ax 819x 419x IDLE 2-000-0004cf-395379x 1ae02a0x 0x 005 08
3. 2dx 814x 414x IDLE 2-000-0004cf-2cae84x 1ae05e0x 0x 003 02
2. b1x 82ax 42ax IDLE 2-000-0004cf-1fed8bx 1ae0920x 0x 004 03
0. 2cx 80bx 40bx IDLE 2-000-0004cf-393d90x 1ae0c60x 0x 002 03
0. 45x 802x 402x IDLE 2-000-0004cf-940c92x 1ae0fa0x 0x 006 04
1. 2dx 817x 417x IDLE 2-000-0004cf-393694x 1ae12e0x 0x 005 03
1. 98x 81bx 41bx IDLE 2-000-0004cf-395e94x 1ae1620x 0x 005 00
2. 49x 826x 426x IDLE 2-000-0004cf-393697x 1ae1960x 0x 004 04
2. 2ex 820x 420x IDLE 2-000-0004cf-241f9ax 1ae1ca0x 0x 004 07
1. aex 81cx 41cx IDLE 2-000-0004cf-39869bx 1ae1fe0x 0x 005 02
3. 98x 81fx 41fx IDLE 2-000-0004cf-394aa1x 1ae2320x 0x 005 07
3. 47x 81ax 41ax IDLE 2-000-0004cf-3935a4x 1ae2660x 0x 005 09
1. 47x 818x 418x IDLE 2-000-0004cf-393ea4x 1ae29a0x 0x 005 04
0. 49x 824x 424x IDLE 2-000-0004cf-3964afx 1ae2ce0x 0x 004 01
1. 74x 81dx 41dx IDLE 2-000-0004cf-3937b4x 1ae3020x 0x 005 05
2. 2cx 808x 408x IDLE 2-000-0004cf-3949b8x 1ae3360x 0x 006 08
0. 75x 829x 429x IDLE 2-000-0004cf-1fecbex 1ae36a0x 0x 004 00
0. 90x 803x 403x IDLE 2-000-0004cf-3949bfx 1ae39e0x 0x 006 00
2. 97x 813x 413x IDLE 2-000-0004cf-39a0c3x 1ae3d20x 0x 002 07
0. 46x 80cx 40cx IDLE 2-000-0004cf-399fd1x 1ae4060x 0x 002 04
0. adx 810x 410x IDLE 2-000-0004cf-3964d6x 1ae43a0x 0x 002 02
1. 5cx 822x 422x IDLE 2-000-0004cf-3940dbx 1ae46e0x 0x 003 04
0. 72x 805x 405x IDLE 2-000-0004cf-3947ddx 1ae4a20x 0x 006 05
0. 73x 811x 411x IDLE 2-000-0004cf-399fddx 1ae4d60x 0x 002 05
3. 59x 80dx 40dx IDLE 2-000-0004cf-39a0e1x 1ae50a0x 0x 003 00
3. 46x 80ex 40ex IDLE 2-000-0004cf-399ee5x 1ae53e0x 0x 003 01
0. 97x 80fx 40fx IDLE 2-000-0004cf-29e2e6x 1ae5720x 0x 002 00
2. 74x 821x 421x IDLE 2-000-0004cf-241febx 1ae5a60x 0x 004 06

Column heading Description Values

RSS Noid NOID of the RSS 0x200 – 0x23F
LDAD Noid The NOID of the disk group this 0x100 – 0x1FF
RSS NOID belongs to
Free PSEGs Free Physical Segments (PSEGs) This number seems to always indicate 0
regardless of how full a disk group is.
Members How many members are in the
RSS
Abnrml Bitmask representing the Example: ffe0x is hex for the following:
abnormal state of each RSS 1111111111100000. The five zeros indicate the RSS
member (member 0 in the LSB has five member volumes in indices 0, 1, 2, 3,
on the right ) and 4.
Example: fc0cx is hex for the following:
1111110000001100. The eight zeros indicate the
RSS has eight members in indices 0, 1, 4, 5, 6, 7,
8, and 9.
Missing Bitmask representing the missing Same description as the Abnrml field
state of each RSS member
(member 0 in the LSB on the
right )
MgrFlgs Migration flags. Bits are defined as follows:
 The source is the RSS from RSS_MIGRATION_TARGET 1
which members are
RSS_MIGRATION_SOURCE 2
migrating.
 The target is the RSS to which RSS_MIGRATION_MERGE 4
members are migrating. RSS_MIGRATION_SPLIT 8
 Merge combines two RSSs With the following possible field values:
into one. 5 — RSS is Merge Target
 Split indicates one RSS 6 — RSS is Merge Source
splitting into two. 9 — RSS is Split Target
a — RSS is Split Source
MbrMgrt Bitmask that indicates the The LSB on the right indicates member index 0.
members in this RSS that are Examples:
migrating (in or out) Merge Source 3ex is hex for the following:
00111110 – indicates the source RSS indices which
will be merged with the target RSS group
Merge Target 7c0x is hex for the following:
011111000000 — Indicates the indices where the
RSS merge source volumes will end up in the
target RSS group.
Split Source 80x is hex for the following:
10000000 — Indicates the drive in the RSS index
indicated by the 1 is the source of a split
operation.
Split Target 2x is hex for the following:
10 — Indicates the drive in RSS index location
indicated by the 1 is the target of a split
operation. The volume NOID of this and the
source will be the same.
ScrRIdx Source RSS indication For example, 5x indicates the source RSS NOID is
0x205.

TrgRIdx Target RSS indication For example, 11x indicates the target RSS NOID
is 0x211
MgrAr[0] A nibble array for the first For example, a merge source of 9876a0x
8 positions in the RSS indicates the following:
group. index 0 — Not present
index 1 — Volume to position a in dest RSS
index 2 — Volume to position 6 in dest RSS
index5 — Volume to position 9 in dest RSS
MgrAr[1] A nibble array for the last For example, a merge target of 154x indicates the
8 positions in the RSS following:
group. index 8 — Comes from source RSS index 4
index 9 — Comes from source RSS index 5
index a — Comes from source RSS index 1
Member Volnoid The NOID of the volume 0x400 – 0x7FF
used in the RSS.
Capacity The capacity of the
member NOID given in
BLOCKs.
All RSSs:
RSS LDAD Free Members
Noid Noid PSEGS
Abnrml Missing MgrFlgs MbrMgrt SrcRIdx TrgRIdx MgrAr[0] MgrAr[1]
Member Blk
Volnoid Capacity
200x 0x 0. 5.
ffe0x ffe0x 0x 0x 0x 0x 0x 0x
401x 142264000.
40ax 142264000.
406x 142264000.
407x 142264000.
400x 142264000.
Null_DUB
Null_DUB
Note
This first listed RSS with NOID 200x is the quorum disk RSS. Notice it has five
members in it and that these are the exact five members that were initially listed
at the start of this SCS_SHOW_CONFIG output.
The constituent volumes of the quorum RSS are the only volumes that will ever be
listed in more than one RSS.

Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
201x Invalid RSS
ffffx ffffx 0x 0x 0x 0x 0x 0x
Note
It is not uncommon to have unused RSS NOID numbers (like 201x) strewn
throughout used RSS NOID numbers.
202x 101x 0. 8.
ff00x ff00x 0x 0x 0x 0x 0x 0x
40fx 142264000.
40ax 142264000.
410x 142264000.
40bx 142264000.
40cx 142264000.
411x 142264000.
412x 142264000.
413x 142264000.
Note
The previous output indicates that RSS 202x has eight members in it, all 72GB
capacity.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB

203x 101x 0. 8.
40dx 142264000.
40ex 142264000.
414x 142264000.
415x 142264000.
422x 142264000.
427x 142264000.
423x 142264000.
428x 142264000.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
204x 101x 0. 8.
429x 142264000.
424x 142264000.
42bx 142264000.
42ax 142264000.
426x 142264000.
425x 142264000.
421x 142264000.
420x 142264000.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB

205x 101x 0. 10.

fc00x fc00x 0x 0x 0x 0x 0x 0x
41bx 142264000.
416x 142264000.
41cx 142264000.
417x 142264000.
418x 142264000.
41dx 142264000.
41ex 142264000.
41fx 142264000.
419x 142264000.
41ax 142264000.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
206x 100x 0. 8.
fc0cx fc0cx 0x 0x 0x 0x 0x 0x
403x 142264000.
400x 142264000.
Null_DUB
Null_DUB
402x 142264000.
405x 142264000.
Note
This RSS illustrates that RSS index IDs do not have to run in consecutive numerical
order. In this case, RSS index values 0 and 1 are followed by 4, 5, 6, 7, 8, and
9, skipping 2 and 3.
406x 142264000.
407x 142264000.
408x 142264000.
401x 142264000.

Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
207x Invalid RSS

208x Invalid RSS
Note
All possible RSS NOIDs are listed, even when most of them are not being used.
209x Invalid RSS

20ax Invalid RSS
20bx Invalid RSS
20cx Invalid RSS
20dx Invalid RSS
20ex Invalid RSS
20fx Invalid RSS
210x Invalid RSS
211x Invalid RSS
212x Invalid RSS
213x Invalid RSS

214x Invalid RSS

215x Invalid RSS
216x Invalid RSS
217x Invalid RSS
218x Invalid RSS
219x Invalid RSS
21ax Invalid RSS
21bx Invalid RSS
21cx Invalid RSS
21dx Invalid RSS
21ex Invalid RSS
21fx Invalid RSS
220x Invalid RSS
221x Invalid RSS
222x Invalid RSS
223x Invalid RSS
224x Invalid RSS
225x Invalid RSS
226x Invalid RSS

227x Invalid RSS

228x Invalid RSS
229x Invalid RSS
22ax Invalid RSS
22bx Invalid RSS
22cx Invalid RSS
22dx Invalid RSS
22ex Invalid RSS
22fx Invalid RSS
230x Invalid RSS
231x Invalid RSS
232x Invalid RSS
233x Invalid RSS
234x Invalid RSS
235x Invalid RSS
236x Invalid RSS
237x Invalid RSS
238x Invalid RSS
239x Invalid RSS

23ax Invalid RSS

23bx Invalid RSS
23cx Invalid RSS
23dx Invalid RSS
23ex Invalid RSS
23fx Invalid RSS

LDAD Noid NOID associated with the 0x100-0x1FF
disk group
RSS noid NOID associated with the 0x200-0x23F
RSS ID comprising the
associated disk group.
LDADs:
LDAD RSS
Noid noid
100x
206x
Note
This previous output indicates that the disk group with NOID 0x100 is comprised
of a single RSS.
101x
202x
203x
204x
205x
Note
This output indicates that the disk group with NOID 0x101 is comprised of a four
RSS groups.


disk group
Member Volnoid NOID of the volume in the 0x400-0x7FF
associated disk group.
LDAD Member
Noid Volnoid
100x
400x
401x
402x
403x
405x
406x
407x
408x
Note
The previous disk group NOID (0x100) output indicates it is comprised of only
eight volumes.
101x
40ax
40bx
40cx
40dx
40ex
40fx
410x
411x
412x
413x
414x
415x
416x
417x
418x

419x
41ax
41bx
41cx
41dx
41ex
41fx
420x
421x
422x
423x
424x
425x
426x
427x
428x
429x
42ax
42bx


LD Noid NOID associated with the 0x2000-0x3FFF
logical disk
disk group the logical disk
is created from
Max LDA Maximum logical disk Hex value indicating the maximum addressable
address logical block number for this logical disk
L2MAP addr L2Map address for the The L2MAP address will be valid only when the
listed logical disk disk is online to this (the master) controller. The
field will always indicate 0 otherwise.
Redun Logical disk redundancy  Parity — VRAID5
type  Mirror — VRAID1
 None — VRAID0
Cache State Indicates which disk the  OnlThis — Online to this controller (primary)
logical disk is online to  OnlOther — Online to the other controller
(secondary)
LDSB Flags Do not use.
Pres Count Presentation count Number of hosts (or clients) this logical disk is
currently presented to.
Host Qscs Host quiesces This indicates when you stop work to a logical
disk for administrative purposes.
Pres Othr Indicates if the logical  1 = Indicates the disk is online to the other
disk is online to the other controller
(secondary) controller  0 = Indicates the disk is online to this controller
(the primary controller)
Rlzd SCS Realized SCS - Indicates if  1 = Indicates the disk is online to the primary
the logical disk is online controller
to the primary controller.  0 = Indicates the disk is online to the
secondary controller
LDs
LD LDAD Max L2MAP Cache LDSB Pres Host Pres Rlzd
Noid Noid LDA addr Redun State Flags Count Qscs Othr SCS
2000x 100x 13fffffx add8c00x Parity OnlThis 00100030x 1. 0. 0. 1.
2001x 101x 31ffffffx 0x Parity OnlOthr 00000010x 1. 0. 1. 0.


SCVD Noid NOID associated with the 0x1000-0x17FF
storage system virtual disk
LDAD Noid NOID of the disk group 0x100-0x1FF
LD Noid NOID associated with the 0x2000-0x3FFF
logical disk
LDSB Do not use.
SCVDs
SCVD LDAD LD
Noid Noid Noid LDSB
1000x 100x 2000x d7da5c0x
1001x 101x 2001x d7dac60x

The following is a full-length example of an FCS_SHOW_CONFIG output. Read

through the example along with the associated notes in an effort to understand all of
the fields associated with the output.
system FCS_SHOW_CONFIG output generated in the first lab section.
Note
Instructional comments in the form of Notes, similar to the one you are reading,
are included throughout the FCS_SHOW_CONFIG output. In addition, some of the
more important output data fields are bolded and edited with comments. Tables
are added to describe various column headings.

Master Master or slave indication Master
Slave
Send_mir Master sent disk map to
slave controller.
Need_send_mir Master needs to send the
disk map to slave
controller.
Too_many Too many enclosures have 0 — Everything is OK.
been detected on a loop. 1 — More than 10 enclosures exist on a single
loop.
Dd_run Device discovery run 0 — Everything is OK
1 — The EMU process stopped while device
discovery was running.
Swapped Device-side loop cabling 0 — Everything is OK.
is swapped. 1 — A device-side loop contains both the A side
and B side of disk enclosures
gen Map generation If the high bit is set, CV EVA has not read the
latest device map.
Master send_mir 0 need_send_mir 0 too_many 0 dd_run 0 swapped 0 gen 3
EMU device status codes:

UNKNOWN "K" CRITICAL "c" NONCRITICAL "n" UNRECOVERABLE "U"
UNSUPPORTED "X" NOT_AVAILABLE "?" NOT_INSTALLED "o" OK "."
Device bypass codes: Reason(A Side/B Side/Both Sides
Controller(A/B/C) EMU (a/b/c) Drive (R/L/E) Common (r/l/e)


DP-1A/B: (loop 0) Loop pair 1 indication Loop 0 — Loop pair 1
Loop 1 — Loop pair 2
X shelves Number of shelves (x)
present on the indicated
loop
HA mask Hard ALPA addressing 0 — Normal
state If either of the top 2 bits are set, the drives have
not adopted the expected hard ALPAs.
# Internal enclosure number The number indicates the internal shelf number
assigned to this enclosure.
Master Do not use.
EMU EMU firmware build
version
Cab Rack number Should always be 0
Shelf # Enclosure number as
displayed by the EMU
ID Loop ID for hard The Li value as viewed from the EMU
addressing
WWN WWN of the enclosure
123456789abcde Device status of the bays o = empty
in the enclosure . = occupied
K = UNKNOWN
C = CRITICAL
N = NONCRITICAL
U = UNRECOVERABLE
X = UNSUPPORTED
? = NOT AVAILABLE
123456789abcde Bypass status of the bays . = The drive is not bypassed in any way
in the enclosure A = Controller bypassed drive on the A side
B = Controller bypassed drive on the B side
C = Controller bypassed drive on both sides
a = EMU bypassed drive on the A side
b = EMU bypassed drive on the B side
c = EMU bypassed drive on both sides
R = Drive bypassed itself on the A side
L = Drive bypassed itself on the B side
E = Drive bypassed itself on both sides
r = Two devices bypassed drive on the A side
l = Two devices bypassed drive on the B side
e = Two devices bypassed drive on both sides
DP-1A/B: (loop 0) 6 shelves - HA mask 0 Device Status Bypass Status

# Master EMU Cab Shelf# ID WWN 123456789abcde 123456789abcde
1 33603C 0071 0 5 2A 50001FE1 0014DF50 ooo.......oooo ccc.......cccc
2 337AD8 0071 0 3 46 50001FE1 0014DF60 ooo......ooooo ccc......ccccc
3 3353F4 0071 0 6 1C 50001FE1 0014DF40 ooo......ooooo ccc......ccccc
4 339780 0071 0 1 62 50001FE1 00159C90 ooo.......oooo ccc.......cccc
5 338B38 0071 0 2 54 50001FE1 00159CB0 ooo.......oooo ccc.......cccc
6 336E90 0071 0 4 38 50001FE1 0014D340 ooo......ooooo ccc......ccccc

DP-2A/B: (loop 1) 6 shelves - HA mask 0 Device Status Bypass Status

# Master EMU Cab Shelf# ID WWN 123456789abcde 123456789abcde
12 33A7E0 0071 0 13 1C 50001FE1 00159AE0 ooo......ooooo ccc......ccccc
13 33B634 0071 0 12 2A 50001FE1 00159C70 ooo......ooooo ccc......ccccc
14 33E754 0071 0 8 62 50001FE1 00159D00 ooo......ooooo ccc......ccccc
15 33C070 0071 0 11 38 50001FE1 00159CA0 ooo......ooooo ccc......ccccc
16 33CCB8 0071 0 10 46 50001FE1 00159A70 ooo......ooooo CcC......ccccc
17 33D900 0071 0 9 54 50001FE1 00159900 ooo......ooooo ccc......ccccc
Note
The following OB and IB Task information is not of any known use in
troubleshooting the EVA.
DP-1A/B (loop 0)
IB Task 0: 00000000
IB Task 1: 00000000 !
IB Task 2: 00000000 "
IB Task 3: 00000000 #
IB Task 4: 00000000 $
IB Task 5: 00000000 %
IB Task 6: 00000000 &
IB Task 7: 00000000 '
IB Task 8: 00000000 (
IB Task 9: 00000000 )
IB Task 10: 00000000 *
IB Task 11: 00000000 +
DP-2A/B (loop 1)
IB Task 0: 00000000
IB Task 1: 00000000 !
IB Task 2: 00000000 "
IB Task 3: 00000000 #
IB Task 4: 00000000 $
IB Task 5: 00000000 %
IB Task 6: 00000000 &
IB Task 7: 00000000 '
IB Task 8: 00000000 (
IB Task 9: 00000000 )
IB Task 10: 00000000 *
IB Task 11: 00000000 +
DP-1A/B (loop 0)
OB Task 0: 00000000
OB Task 1: 00000000
OB Task 2: 00000000
OB Task 3: 00000000
OB Task 4: 00000000
OB Task 5: 00000000
OB Task 6: 00000000
DP-2A/B (loop 1)
OB Task 0: 00000000
OB Task 1: 00000000
OB Task 2: 00000000
OB Task 3: 00000000
OB Task 4: 00000000
OB Task 5: 00000000
OB Task 6: 00000000


Loop Loop ID for which DP1A, DP1B, DP2A, DP2B
information is displayed
Pcb Do not use. Internal structure address
Loop map Number of disks and
controllers on this loop
Reporting group Reporting group number The Rg value as viewed from the EMU
Link Link status Up
Down
Failed
Unknown
ALPA ALPA of the associated ALPAs are displayed in their current ALPA order
device around the loop.
POID Physical Object ID of the 0x800- 0x8FF
disk
GUB/FNB Do not use.
Product ID Product Identification [Not a disk] – indicates a controller
XXXXXXXXXX  Model # of the disk drive
WWN WWN of the indicated Least significant 32 bits of the device WWN
device
Serial # Serial number of the disk
REV Firmware version of the disk
CAP Capacity of the disk The number is shown in GBs.
SHELF Enclosure number for the  99 means unknown
drive  The number before the colon indicates the
internal shelf number as displayed previously
in this FCS_SHOW_CONFIG output.
BAY Drive bay 0 for controllers
1-14 for disk devices
OL Old LED pattern Do not use.
NL New LED pattern Do not use.
UM Unmapped Do not use.
Drive failed status field An F in this column indicates the disk drive has
failed.
Note there is no discernable heading for this
column.
DP-1A pcb 330a04 loop map: 41 devices, reporting group 4052 Link Up
ALPA POID GUB/FNB PRODUCT ID WWN SERIAL # REV CAP SHELF BAY OL NL UM
1 20 3A5150 [Not a disk] : 7 0
33 931 339780# BD07254498 CF3960BC 3EK0PJVW 3BE8 72 4: 1 4 0 0
32 91B 33998C# BD07254498 CF29D493 3EK0LVMZ 3BE8 72 4: 1 5 0 0
31 907 339B98# BD07254498 CF394819 3EK0P66E 3BE8 72 4: 1 6 0 0
2E 823 1ADF5A0 BD07254498 CF39604C 3EK0PCYP 3BE8 72 4: 1 7 0 0
2D 817 1AE12E0 BD07254498 CF393694 3EK0P6S2 3BE8 72 4: 1 8 0 0
2C 80B 1AE0C60 BD07254498 CF393D90 3EK0P7LA 3BE8 72 4: 1 9 0 0
2B 801 1ADD520 BD07254498 CF393C9B 3EK0P7DZ 3BE8 72 4: 1 10 0 0
4C 93A 338B38# BD07254498 CF1FE0D1 3EK0HV62 3BE8 72 5: 2 4 0 0
4B 936 33892C# BD07254498 CF3949CF 3EK0PDWB 3BE8 72 5: 2 5 0 0

4A 92C 338D44# BD07254498 CF29E3B3 3EK0LSL7 3BE8 72 5: 2 6 0 0

49 824 1AE2CE0 BD07254498 CF3964AF 3EK0PG7N 3BE8 72 5: 2 7 0 0
47 818 1AE29A0 BD07254498 CF393EA4 3EK0P68F 3BE8 72 5: 2 8 0 0
46 80C 1AE4060 BD07254498 CF399FD1 3EK0Q634 3BE8 72 5: 2 9 0 0
45 802 1AE0FA0 BD07254498 CF940C92 3EK1J25A 3BE8 72 5: 2 10 0 0
66 90F 337AD8# BD07254498 CF1FE247 3EKYNH2R 3BE8 72 2: 3 4 0 0
65 921 337CE4# BD07254498 CF39379B 3EK0P7BK 3BE8 72 2: 3 5 0 0
63 902 3380FC# BD07254498 CF29E205 3EK0LYT4 3BE8 72 2: 3 6 0 0
5C 822 1AE46E0 BD07254498 CF3940DB 3EK0P7M0 3BE8 72 2: 3 7 0 0
5A 816 1ADFC20 BD07254498 CF395E6A 3EK0MFZS 3BE8 72 2: 3 8 0 0
59 80A 1ADD860 BD07254498 CF241C2A 3EK0JA10 3BE8 72 2: 3 9 0 0
79 948 336E90# BD07254498 CF3986F1 3EK0PXE4 3BE8 72 6: 4 4 0 0
76 935 3372A8# BD07254498 CFBF40C9 3EK2FTBV 3BE8 72 6: 4 5 0 0
75 829 1AE36A0 BD07254498 CF1FECBE 3EK0HYY4 3BE8 72 6: 4 6 0 0
74 81D 1AE3020 BD07254498 CF3937B4 3EK0P77F 3BE8 72 6: 4 7 0 0
73 811 1AE4D60 BD07254498 CF399FDD 3EK0PXCN 3BE8 72 6: 4 8 0 0
72 805 1AE4A20 BD07254498 CF3947DD 3EK0PECK 3BE8 72 6: 4 9 0 0
9E 900 33603C# BD07254498 CF399F00 3EK0Q2GC 3BE8 72 1: 5 4 0 0
9D 947 336248# BD07254498 CF399FEF 3EK0Q618 3BE8 72 1: 5 5 0 0
9B 827 1ADEF20 BD07254498 CF399E14 3EK0Q20K 3BE8 72 1: 5 6 0 0
98 81B 1AE1620 BD07254498 CF395E94 3EK0PJM2 3BE8 72 1: 5 7 0 0
97 80F 1AE5720 BD07254498 CF29E2E6 3EK0LN1P 3BE8 72 1: 5 8 0 0
90 803 1AE39E0 BD07254498 CF3949BF 3EK0PF0N 3BE8 72 1: 5 9 0 0
8F 800 1ADE220 BD07254498 CF393DA2 3EK0P88L 3BE8 72 1: 5 10 0 0
B3 938 3353F4# BD07254498 CF1FE1CF 3EKYNDZH 3BE8 72 3: 6 4 0 0
B2 915 335600# BD07254498 CF39866F 3EK0PXYD 3BE8 72 3: 6 5 0 0
B1 828 1ADE8A0 BD07254498 CFBF3D08 3EK2FS3L 3BE8 72 3: 6 6 0 0
AE 81C 1AE1FE0 BD07254498 CF39869B 3EK0PM5F 3BE8 72 3: 6 7 0 0
AD 810 1AE43A0 BD07254498 CF3964D6 3EK0MPTB 3BE8 72 3: 6 8 0 0
AC 929 335E30# BD07254498 CF394AAD 3EK0PER0 3BE8 72 3: 6 9 0 0
2 21 3A50D4 [Not a disk] : 7 0
DP-1B pcb 3313d4 loop map: 41 devices, reporting group 4052 Link Up
1 20 3A5150 [Not a disk] : 7 0
33 931 339780# BD07254498 CF3960BC 3EK0PJVW 3BE8 72 4: 1 4 0 0
32 91B 33998C# BD07254498 CF29D493 3EK0LVMZ 3BE8 72 4: 1 5 0 0
31 907 339B98# BD07254498 CF394819 3EK0P66E 3BE8 72 4: 1 6 0 0
2E 823 1ADF5A0 BD07254498 CF39604C 3EK0PCYP 3BE8 72 4: 1 7 0 0
2D 817 1AE12E0 BD07254498 CF393694 3EK0P6S2 3BE8 72 4: 1 8 0 0
2C 80B 1AE0C60 BD07254498 CF393D90 3EK0P7LA 3BE8 72 4: 1 9 0 0
2B 801 1ADD520 BD07254498 CF393C9B 3EK0P7DZ 3BE8 72 4: 1 10 0 0
4C 93A 338B38# BD07254498 CF1FE0D1 3EK0HV62 3BE8 72 5: 2 4 0 0
4B 936 33892C# BD07254498 CF3949CF 3EK0PDWB 3BE8 72 5: 2 5 0 0
4A 92C 338D44# BD07254498 CF29E3B3 3EK0LSL7 3BE8 72 5: 2 6 0 0
49 824 1AE2CE0 BD07254498 CF3964AF 3EK0PG7N 3BE8 72 5: 2 7 0 0
47 818 1AE29A0 BD07254498 CF393EA4 3EK0P68F 3BE8 72 5: 2 8 0 0
46 80C 1AE4060 BD07254498 CF399FD1 3EK0Q634 3BE8 72 5: 2 9 0 0
45 802 1AE0FA0 BD07254498 CF940C92 3EK1J25A 3BE8 72 5: 2 10 0 0
66 90F 337AD8# BD07254498 CF1FE247 3EKYNH2R 3BE8 72 2: 3 4 0 0
65 921 337CE4# BD07254498 CF39379B 3EK0P7BK 3BE8 72 2: 3 5 0 0
63 902 3380FC# BD07254498 CF29E205 3EK0LYT4 3BE8 72 2: 3 6 0 0
5C 822 1AE46E0 BD07254498 CF3940DB 3EK0P7M0 3BE8 72 2: 3 7 0 0
5A 816 1ADFC20 BD07254498 CF395E6A 3EK0MFZS 3BE8 72 2: 3 8 0 0
59 80A 1ADD860 BD07254498 CF241C2A 3EK0JA10 3BE8 72 2: 3 9 0 0
79 948 336E90# BD07254498 CF3986F1 3EK0PXE4 3BE8 72 6: 4 4 0 0
76 935 3372A8# BD07254498 CFBF40C9 3EK2FTBV 3BE8 72 6: 4 5 0 0
75 829 1AE36A0 BD07254498 CF1FECBE 3EK0HYY4 3BE8 72 6: 4 6 0 0
74 81D 1AE3020 BD07254498 CF3937B4 3EK0P77F 3BE8 72 6: 4 7 0 0
73 811 1AE4D60 BD07254498 CF399FDD 3EK0PXCN 3BE8 72 6: 4 8 0 0
72 805 1AE4A20 BD07254498 CF3947DD 3EK0PECK 3BE8 72 6: 4 9 0 0
9E 900 33603C# BD07254498 CF399F00 3EK0Q2GC 3BE8 72 1: 5 4 0 0

9D 947 336248# BD07254498 CF399FEF 3EK0Q618 3BE8 72 1: 5 5 0 0

9B 827 1ADEF20 BD07254498 CF399E14 3EK0Q20K 3BE8 72 1: 5 6 0 0
98 81B 1AE1620 BD07254498 CF395E94 3EK0PJM2 3BE8 72 1: 5 7 0 0
97 80F 1AE5720 BD07254498 CF29E2E6 3EK0LN1P 3BE8 72 1: 5 8 0 0
90 803 1AE39E0 BD07254498 CF3949BF 3EK0PF0N 3BE8 72 1: 5 9 0 0
8F 800 1ADE220 BD07254498 CF393DA2 3EK0P88L 3BE8 72 1: 5 10 0 0
B3 938 3353F4# BD07254498 CF1FE1CF 3EKYNDZH 3BE8 72 3: 6 4 0 0
B2 915 335600# BD07254498 CF39866F 3EK0PXYD 3BE8 72 3: 6 5 0 0
B1 828 1ADE8A0 BD07254498 CFBF3D08 3EK2FS3L 3BE8 72 3: 6 6 0 0
AE 81C 1AE1FE0 BD07254498 CF39869B 3EK0PM5F 3BE8 72 3: 6 7 0 0
AD 810 1AE43A0 BD07254498 CF3964D6 3EK0MPTB 3BE8 72 3: 6 8 0 0
AC 929 335E30# BD07254498 CF394AAD 3EK0PER0 3BE8 72 3: 6 9 0 0
2 21 3A50D4 [Not a disk] : 7 0
DP-2A pcb 331da4 loop map: 38 devices, reporting group 4077 Link Up
1 20 3A5150 [Not a disk] : 7 0
33 94A 33E754# BD07254498 CF241CF2 3EK0J9FH 3BE8 72 14: 8 4 77 77
32 912 33E960# BD07254498 CF395359 3EK0PG93 3BE8 72 14: 8 5 77 77
31 908 33EB6C# BD07254498 CF2CAB1B 3EK0NBX8 3BE8 72 14: 8 6 77 77
2E 820 1AE1CA0 BD07254498 CF241F9A 3EK0J8E6 3BE8 72 14: 8 7 77 77
2D 814 1AE05E0 BD07254498 CF2CAE84 3EK0N7WN 3BE8 72 14: 8 8 77 77
2C 808 1AE3360 BD07254498 CF3949B8 3EK0PEW2 3BE8 72 14: 8 9 77 77
4C 93B 33D900# BD07254498 CF2CACD5 3EK0NAVW 3BE8 72 17: 9 4 77 77
4B 924 33DB0C# BD07254498 CF393EA1 3EK0P7VQ 3BE8 72 17: 9 5 77 77
4A 91F 33DD18# BD07254498 CF395498 3EK0PG9M 3BE8 72 17: 9 6 77 77
49 826 1AE1960 BD07254498 CF393697 3EK0P6N8 3BE8 72 17: 9 7 77 77
47 81A 1AE2660 BD07254498 CF3935A4 3EK0P6VK 3BE8 72 17: 9 8 77 77
46 80E 1AE53E0 BD07254498 CF399EE5 3EK0Q3BP 3BE8 72 17: 9 9 77 77
66 94B 33CCB8# BD07254498 CF393FFA 3EK0PCZF 3BE8 72 16:10 4 77 77
65 940 33CEC4# BD07254498 CF39A0DE 3EK0Q642 3BE8 72 16:10 5 77 77
63 949 33D0D0# BD07254498 CF399FF1 3EK0Q423 3BE8 72 16:10 6 77 77
5C 825 1ADF8E0 BD07254498 CF399E57 3EK0Q4KB 3BE8 72 16:10 7 77 77
5A 819 1AE02A0 BD07254498 CF395379 3EK0PG75 3BE8 72 16:10 8 77 77
59 80D 1AE50A0 BD07254498 CF39A0E1 3EK0Q637 3BE8 72 16:10 9 77 77
79 90D 33C070# BD07254498 CF399F34 3EK0Q4LH 3BE8 72 15:11 4 77 77
76 92E 33C27C# BD07254498 CF3937B7 3EK0P71V 3BE8 72 15:11 5 77 77
75 90A 33C488# BD07254498 CF39531E 3EK0PG3N 3BE8 72 15:11 6 77 77
74 821 1AE5A60 BD07254498 CF241FEB 3EK0JAMA 3BE8 72 15:11 7 77 77
73 815 1ADF260 BD07254498 CF29E31D 3EK0LMYZ 3BE8 72 15:11 8 77 77
72 90B 33CAAC# BD07254498 CF39A029 3EK0Q3NV 3BE8 72 15:11 9 77 77
9E 945 33B634# BD07254498 CF399EE9 3EK0Q5H5 3BE8 72 13:12 4 77 77
9D 941 33B428# BD07254498 CF3943E1 3EK0NP18 3BE8 72 13:12 5 77 77
9B 82B 1ADEBE0 BD07254498 CF29E110 3EK0LXYK 3BE8 72 13:12 6 77 77
98 81F 1AE2320 BD07254498 CF394AA1 3EK0PFA0 3BE8 72 13:12 7 77 77
97 813 1AE3D20 BD07254498 CF39A0C3 3EK0PKH6 3BE8 72 13:12 8 77 77
90 807 1ADDEE0 BD07254498 CF3986AD 3EK0PAN5 3BE8 72 13:12 9 77 77
2 21 3A50D4 [Not a disk] : 7 0
B3 90E 33A7E0# BD07254498 CF39A034 3EK0Q2WP 3BE8 72 12:13 4 77 77
B2 904 33A9EC# BD07254498 CF398608 3EK0PXH8 3BE8 72 12:13 5 77 77
B1 82A 1AE0920 BD07254498 CF1FED8B 3EK0J1MZ 3BE8 72 12:13 6 77 77
AE 81E 1ADFF60 BD07254498 CF395B6F 3EK0PCQ2 3BE8 72 12:13 7 77 77
AD 812 1ADE560 BD07254498 CF395002 3EK0NGAQ 3BE8 72 12:13 8 77 77
AC 806 1ADDBA0 BD07254498 CF39A0CF 3EK0Q5HC 3BE8 72 12:13 9 77 77
DP-2B pcb 332774 loop map: 38 devices, reporting group 4077 Link Up
1 20 3A5150 [Not a disk] : 7 0
B3 90E 33A7E0# BD07254498 CF39A034 3EK0Q2WP 3BE8 72 12:13 4 77 77
B2 904 33A9EC# BD07254498 CF398608 3EK0PXH8 3BE8 72 12:13 5 77 77
B1 82A 1AE0920 BD07254498 CF1FED8B 3EK0J1MZ 3BE8 72 12:13 6 77 77
AE 81E 1ADFF60 BD07254498 CF395B6F 3EK0PCQ2 3BE8 72 12:13 7 77 77

AD 812 1ADE560 BD07254498 CF395002 3EK0NGAQ 3BE8 72 12:13 8 77 77

AC 806 1ADDBA0 BD07254498 CF39A0CF 3EK0Q5HC 3BE8 72 12:13 9 77 77
9E 945 33B634# BD07254498 CF399EE9 3EK0Q5H5 3BE8 72 13:12 4 77 77
9D 941 33B428# BD07254498 CF3943E1 3EK0NP18 3BE8 72 13:12 5 77 77
9B 82B 1ADEBE0 BD07254498 CF29E110 3EK0LXYK 3BE8 72 13:12 6 77 77
98 81F 1AE2320 BD07254498 CF394AA1 3EK0PFA0 3BE8 72 13:12 7 77 77
97 813 1AE3D20 BD07254498 CF39A0C3 3EK0PKH6 3BE8 72 13:12 8 77 77
90 807 1ADDEE0 BD07254498 CF3986AD 3EK0PAN5 3BE8 72 13:12 9 77 77
2 21 3A50D4 [Not a disk] : 7 0
33 94A 33E754# BD07254498 CF241CF2 3EK0J9FH 3BE8 72 14: 8 4 77 77
32 912 33E960# BD07254498 CF395359 3EK0PG93 3BE8 72 14: 8 5 77 77
31 908 33EB6C# BD07254498 CF2CAB1B 3EK0NBX8 3BE8 72 14: 8 6 77 77
2E 820 1AE1CA0 BD07254498 CF241F9A 3EK0J8E6 3BE8 72 14: 8 7 77 77
2D 814 1AE05E0 BD07254498 CF2CAE84 3EK0N7WN 3BE8 72 14: 8 8 77 77
2C 808 1AE3360 BD07254498 CF3949B8 3EK0PEW2 3BE8 72 14: 8 9 77 77
4C 93B 33D900# BD07254498 CF2CACD5 3EK0NAVW 3BE8 72 17: 9 4 77 77
4B 924 33DB0C# BD07254498 CF393EA1 3EK0P7VQ 3BE8 72 17: 9 5 77 77
4A 91F 33DD18# BD07254498 CF395498 3EK0PG9M 3BE8 72 17: 9 6 77 77
49 826 1AE1960 BD07254498 CF393697 3EK0P6N8 3BE8 72 17: 9 7 77 77
47 81A 1AE2660 BD07254498 CF3935A4 3EK0P6VK 3BE8 72 17: 9 8 77 77
46 80E 1AE53E0 BD07254498 CF399EE5 3EK0Q3BP 3BE8 72 17: 9 9 77 77
66 94B 33CCB8# BD07254498 CF393FFA 3EK0PCZF 3BE8 72 16:10 4 77 77
65 940 33CEC4# BD07254498 CF39A0DE 3EK0Q642 3BE8 72 16:10 5 77 77
63 949 33D0D0# BD07254498 CF399FF1 3EK0Q423 3BE8 72 16:10 6 77 77
5C 825 1ADF8E0 BD07254498 CF399E57 3EK0Q4KB 3BE8 72 16:10 7 77 77
5A 819 1AE02A0 BD07254498 CF395379 3EK0PG75 3BE8 72 16:10 8 77 77
59 80D 1AE50A0 BD07254498 CF39A0E1 3EK0Q637 3BE8 72 16:10 9 77 77
79 90D 33C070# BD07254498 CF399F34 3EK0Q4LH 3BE8 72 15:11 4 77 77
76 92E 33C27C# BD07254498 CF3937B7 3EK0P71V 3BE8 72 15:11 5 77 77
75 90A 33C488# BD07254498 CF39531E 3EK0PG3N 3BE8 72 15:11 6 77 77
74 821 1AE5A60 BD07254498 CF241FEB 3EK0JAMA 3BE8 72 15:11 7 77 77
73 815 1ADF260 BD07254498 CF29E31D 3EK0LMYZ 3BE8 72 15:11 8 77 77
72 90B 33CAAC# BD07254498 CF39A029 3EK0Q3NV 3BE8 72 15:11 9 77 77

The following is a full-length example of an FCS_LINK_ERRORS output. Read

through the example and associated notes in an effort to understand all of the fields
associated with the output.
system FCS_LINK_ERRORS output generated in the first lab section.
Note
Instructional comments, in the form of Notes like the one you are reading, are
included throughout the FCS_LINK_ERRORS output. In addition, tables are
added to describe various column headings.
Column Heading Description Possible values

Loop, Pcb, loop map, reporting group, and link are all identical to what was displayed in the FCS_SHOW_CONFIG
output.
ALPA ALPA of the device
POID Physical object ID of the disk or
controller
WWN WWN of the indicated device Least significant 32 bits of the device WWN
Serial # Serial number of the disk
ENC Enclosure number for the drive or Numbers above 100 are 100+ the last known
controller value.
BAY Drive bay 0 for controllers
1-14 for disk devices
Numbers above 100 are 100+ the last known
value.
BAD BAD indication None — Device is good
X – The controller determined that the device is
bad on this port.
LINK_FL Link failure counter The number of times the receive port of the drive
had a link failure
LS_SYNC Loss of Sync counter The number of times sync was lost on the receive
port of this disk drive
BAD_WORD Bad received word counter The number of bad 10-bit words received by this
disk drive due to either bad running disparity or
due to an 8-bit to 10-bit FC encoding error
BAD_CRC Bad frame CRC counter The number of bad CRCs detected by this drive
DP-1A pcb 330a04 loop map: 41 devices, reporting group 4052 Link Up
ALPA POID WWN SERIAL # ENC BAY BAD LINK_FL LS_SYNC BAD_WORD BAD_CRC
1 20 [Controller] 7 0##
33 931 CF3960BC 3EK0PJVW 1 4 16C 3F45 9AFC 0
Note
It is not uncommon for the first populated disk bay to have higher error counter
values than the rest of the disk devices in the drive enclosure.

32 91B CF29D493 3EK0LVMZ 1 5 1 92 9CF3 0

31 907 CF394819 3EK0P66E 1 6 1 73 9CCB 0
2E 823 CF39604C 3EK0PCYP 1 7 1 83 9D3D 0
2D 817 CF393694 3EK0P6S2 1 8 1 78 9D6E 0
2C 80B CF393D90 3EK0P7LA 1 9 1 71 9D56 0
2B 801 CF393C9B 3EK0P7DZ 1 10 1 89 9CFC 0
4C 93A CF1FE0D1 3EK0HV62 2 4 94 12F4 95FE 0
4B 936 CF3949CF 3EK0PDWB 2 5 1 77 9F54 0
4A 92C CF29E3B3 3EK0LSL7 2 6 1 70 9F34 0
49 824 CF3964AF 3EK0PG7N 2 7 1 94 9F71 0
47 818 CF393EA4 3EK0P68F 2 8 1 64 9F7B 0
46 80C CF399FD1 3EK0Q634 2 9 1 7A 9FC6 0
45 802 CF940C92 3EK1J25A 2 10 1 73 9F74 0
66 90F CF1FE247 3EKYNH2R 3 4 14 FFE 971C 0
65 921 CF39379B 3EK0P7BK 3 5 1 74 A0DF 0
63 902 CF29E205 3EK0LYT4 3 6 1 67 A039 0
5C 822 CF3940DB 3EK0P7M0 3 7 1 6F A0B1 0
5A 816 CF395E6A 3EK0MFZS 3 8 1 7C A057 0
59 80A CF241C2A 3EK0JA10 3 9 1 6A A083 0
79 948 CF3986F1 3EK0PXE4 4 4 2 FA3 969C 0
76 935 CFBF40C9 3EK2FTBV 4 5 1 6C A0F0 0
75 829 CF1FECBE 3EK0HYY4 4 6 1 6B A06D 0
74 81D CF3937B4 3EK0P77F 4 7 1 7C A129 0
73 811 CF399FDD 3EK0PXCN 4 8 1 71 A134 0
72 805 CF3947DD 3EK0PECK 4 9 1 68 A03B 0
9E 900 CF399F00 3EK0Q2GC 5 4 5 FA0 9780 0
9D 947 CF399FEF 3EK0Q618 5 5 1 60 A13D 0
9B 827 CF399E14 3EK0Q20K 5 6 1 6A A076 0
98 81B CF395E94 3EK0PJM2 5 7 1 71 A08A 0
97 80F CF29E2E6 3EK0LN1P 5 8 1 62 A128 0
90 803 CF3949BF 3EK0PF0N 5 9 0 2C 17A 0
8F 800 CF393DA2 3EK0P88L 5 10 0 2C 14C 0
B3 938 CF1FE1CF 3EKYNDZH 6 4 8A 2B2C 17797 0
B2 915 CF39866F 3EK0PXYD 6 5 1 F2 A0B0 0
B1 828 CFBF3D08 3EK2FS3L 6 6 1 A6 A0DF 0
AE 81C CF39869B 3EK0PM5F 6 7 1 A0 A143 0

AD 810 CF3964D6 3EK0MPTB 6 8 0 2C 16B 0

AC 929 CF394AAD 3EK0PER0 6 9 0 2C 79 0
2 21 [Controller] 7 0 0 0 1FE 0
Note
The statistics for all four device-side loops are displayed separately. What follows
are the statistics for loop 1B.
DP-1B pcb 3313d4 loop map: 41 devices, reporting group 4052 Link Up
1 20 [Controller] 7 0
33 931 CF3960BC 3EK0PJVW 1 4 1 BF 4C6 0
32 91B CF29D493 3EK0LVMZ 1 5 1 9A 616 0
31 907 CF394819 3EK0P66E 1 6 1 98 607 0
2E 823 CF39604C 3EK0PCYP 1 7 1 8B 5FA 0
2D 817 CF393694 3EK0P6S2 1 8 1 8C 5EB 0
2C 80B CF393D90 3EK0P7LA 1 9 1 8E 5F6 0
2B 801 CF393C9B 3EK0P7DZ 1 10 1 93 5CE 0
4C 93A CF1FE0D1 3EK0HV62 2 4 1 A4 5EA 0
4B 936 CF3949CF 3EK0PDWB 2 5 1 92 5EB 0
4A 92C CF29E3B3 3EK0LSL7 2 6 1 8E 5EB 0
49 824 CF3964AF 3EK0PG7N 2 7 1 A1 5EB 0
47 818 CF393EA4 3EK0P68F 2 8 1 8E 5F7 0
46 80C CF399FD1 3EK0Q634 2 9 1 97 5FA 0
45 802 CF940C92 3EK1J25A 2 10 1 99 5ED 0
66 90F CF1FE247 3EKYNH2R 3 4 1 9D 5DE 0
65 921 CF39379B 3EK0P7BK 3 5 1 8D 5EE 0
63 902 CF29E205 3EK0LYT4 3 6 1 7F 5DF 0
5C 822 CF3940DB 3EK0P7M0 3 7 1 88 5DF 0
5A 816 CF395E6A 3EK0MFZS 3 8 1 8A 5DF 0
59 80A CF241C2A 3EK0JA10 3 9 1 8D 5DF 0
79 948 CF3986F1 3EK0PXE4 4 4 1 9A 5E6 0
76 935 CFBF40C9 3EK2FTBV 4 5 1 94 5EF 0
75 829 CF1FECBE 3EK0HYY4 4 6 1 8C 5ED 0
74 81D CF3937B4 3EK0P77F 4 7 1 8F 5DE 0
73 811 CF399FDD 3EK0PXCN 4 8 1 90 5ED 0
72 805 CF3947DD 3EK0PECK 4 9 1 92 5ED 0
9E 900 CF399F00 3EK0Q2GC 5 4 1 9A 5ED 0
9D 947 CF399FEF 3EK0Q618 5 5 1 84 5ED 0

9B 827 CF399E14 3EK0Q20K 5 6 1 94 5DE 0

98 81B CF395E94 3EK0PJM2 5 7 1 9E 5DE 0
97 80F CF29E2E6 3EK0LN1P 5 8 1 90 5FC 0
90 803 CF3949BF 3EK0PF0N 5 9 0 4F 24F 0
8F 800 CF393DA2 3EK0P88L 5 10 0 4F 26C 0
B3 938 CF1FE1CF 3EKYNDZH 6 4 1 CE 636 0
B2 915 CF39866F 3EK0PXYD 6 5 1 BE 5ED 0
B1 828 CFBF3D08 3EK2FS3L 6 6 1 A0 5FC 0
AE 81C CF39869B 3EK0PM5F 6 7 1 A1 5EE 0
AD 810 CF3964D6 3EK0MPTB 6 8 0 51 25E 0
AC 929 CF394AAD 3EK0PER0 6 9 0 51 25B 0
2 21 [Controller] 7 0 0 1 19B 0
Note
You will sometimes see an error indication that states *****Port 2 Loop Order
Invalid—Using ordering from port 3*****. This is indicated when a device-side
loop fails and a loop map for it is not available. When this happens, the
controller uses the loop map from the other loop.
DP-2A pcb 331da4 loop map: 38 devices, reporting group 4077 Link Up
33 94A CF241CF2 3EK0J9FH 8 4 26 37B 5546 0
32 912 CF395359 3EK0PG93 8 5 1 93 28C3 0
31 908 CF2CAB1B 3EK0NBX8 8 6 1 75 28A5 0
2E 820 CF241F9A 3EK0J8E6 8 7 0 29 121 0
2D 814 CF2CAE84 3EK0N7WN 8 8 0 1D 110 0
2C 808 CF3949B8 3EK0PEW2 8 9 1 78 2883 0
4C 93B CF2CACD5 3EK0NAVW 9 4 3 2E7 2693 0
4B 924 CF393EA1 3EK0P7VQ 9 5 1 75 28F5 0
4A 91F CF395498 3EK0PG9M 9 6 1 6B 28B6 0
49 826 CF393697 3EK0P6N8 9 7 1 66 289C 0
47 81A CF3935A4 3EK0P6VK 9 8 4F 169A7 1DD9 0
Note
The previous disk device has orders of magnitude higher signal errors than the
rest of the devices on this loop. It is likely that this device will be sending a large
number of LIP F8s in an effort to obtain a cleaner signal. Assuming there are
errors being logged on the 2A loop, candidates for replacement include the disk
just before this one (9, 7), this disk (9, 8) or the A-side I/O module. In the rare
event none of those replacements fixes the problem, you must consider replacing
the disk enclosure.

Note
Typically, HP recommends that you replace the disk to the left of the disk that is
reporting the errors. In this case, because the disk is in Bay 8, consider replacing
the A-side I/O module first. There is extra signal regeneration logic in the I/O
modules between Bays 7 and 8.
46 80E CF399EE5 3EK0Q3BP 9 9 1 5F 28B1 0

66 94B CF393FFA 3EK0PCZF 10 4 2 29F 2681 0
65 940 CF39A0DE 3EK0Q642 10 5 1 57 28F3 0
63 949 CF399FF1 3EK0Q423 10 6 1 5F 2870 0
5C 825 CF399E57 3EK0Q4KB 10 7 1 5E 28C7 0
5A 819 CF395379 3EK0PG75 10 8 0 61 28FF 0
59 80D CF39A0E1 3EK0Q637 10 9 0 5C 2909 0
79 90D CF399F34 3EK0Q4LH 11 4 63 17DAE 2776 0
76 92E CF3937B7 3EK0P71V 11 5 B 9C2A 3A55 0
75 90A CF39531E 3EK0PG3N 11 6 1 62E 35B1 0
74 821 CF241FEB 3EK0JAMA 11 7 1 B88 3AC0 0
73 815 CF29E31D 3EK0LMYZ 11 8 51 2767C 21D3 0
Note
Disk 11, 8 is also reporting fairly high signal errors as compared to the rest of the
loop. The analysis of the 2A loop from the controller event log indicates that this
A-side I/O module should be replaced as well as the A-side I/O module in shelf
number nine.
72 90B CF39A029 3EK0Q3NV 11 9 0 18 110 0

9E 945 CF399EE9 3EK0Q5H5 12 4 43 5C5 2670 0
9D 941 CF3943E1 3EK0NP18 12 5 1 73 27C6 0
9B 82B CF29E110 3EK0LXYK 12 6 1 5E 27FE 0
98 81F CF394AA1 3EK0PFA0 12 7 1 54 2737 0
97 813 CF39A0C3 3EK0PKH6 12 8 1 5A 279A 0
90 807 CF3986AD 3EK0PAN5 12 9 1 61 2815 0
2 21 [Controller] 7 0 0 0 1FE 0
B3 90E CF39A034 3EK0Q2WP 13 4 18 39B 55B2 0
B2 904 CF398608 3EK0PXH8 13 5 1 90 3152 0
B1 82A CF1FED8B 3EK0J1MZ 13 6 1 86 306B 0
AE 81E CF395B6F 3EK0PCQ2 13 7 1 7C 307A 0
AD 812 CF395002 3EK0NGAQ 13 8 1 73 307B 0
AC 806 CF39A0CF 3EK0Q5HC 13 9 1 69 30A9 0

DP-2B pcb 332774 loop map: 38 devices, reporting group 4077 Link Up
B3 90E CF39A034 3EK0Q2WP 13 4 4 ED 9CC 0
B2 904 CF398608 3EK0PXH8 13 5 1 5D 753 0
B1 82A CF1FED8B 3EK0J1MZ 13 6 1 50 753 0
AE 81E CF395B6F 3EK0PCQ2 13 7 1 50 742 0
AD 812 CF395002 3EK0NGAQ 13 8 1 46 757 0
AC 806 CF39A0CF 3EK0Q5HC 13 9 1 4C 773 0
9E 945 CF399EE9 3EK0Q5H5 12 4 3 76 6DE 0
9D 941 CF3943E1 3EK0NP18 12 5 1 4D 72D 0
9B 82B CF29E110 3EK0LXYK 12 6 1 47 6F8 0
98 81F CF394AA1 3EK0PFA0 12 7 1 42 736 0
97 813 CF39A0C3 3EK0PKH6 12 8 1 4D 764 0
90 807 CF3986AD 3EK0PAN5 12 9 1 45 734 0
2 21 [Controller] 7 0 0 0 201 0
33 94A CF241CF2 3EK0J9FH 8 4 2 EA 6F7 0
32 912 CF395359 3EK0PG93 8 5 1 76 74C 0
31 908 CF2CAB1B 3EK0NBX8 8 6 1 6E 6FF 0
2E 820 CF241F9A 3EK0J8E6 8 7 0 1E 28B 0
2D 814 CF2CAE84 3EK0N7WN 8 8 0 1C 28A 0
2C 808 CF3949B8 3EK0PEW2 8 9 1 5B 71C 0
4C 93B CF2CACD5 3EK0NAVW 9 4 3 83 6B0 0
4B 924 CF393EA1 3EK0P7VQ 9 5 1 59 73B 0
4A 91F CF395498 3EK0PG9M 9 6 1 4D 72C 0
49 826 CF393697 3EK0P6N8 9 7 1 4E 72D 0
47 81A CF3935A4 3EK0P6VK 9 8 1 55 75B 0
46 80E CF399EE5 3EK0Q3BP 9 9 1 49 752 0
66 94B CF393FFA 3EK0PCZF 10 4 3 A7 6F7 0
65 940 CF39A0DE 3EK0Q642 10 5 1 46 755 0
63 949 CF399FF1 3EK0Q423 10 6 1 42 728 0
5C 825 CF399E57 3EK0Q4KB 10 7 1 51 746 0
5A 819 CF395379 3EK0PG75 10 8 0 48 6A5 0
59 80D CF39A0E1 3EK0Q637 10 9 0 49 6B0 0
79 90D CF399F34 3EK0Q4LH 11 4 3 B4 6EF 0
76 92E CF3937B7 3EK0P71V 11 5 1 46 736 0
75 90A CF39531E 3EK0PG3N 11 6 1 41 726 0

74 821 CF241FEB 3EK0JAMA 11 7 1 55 772 0

73 815 CF29E31D 3EK0LMYZ 11 8 1 4B 70D 0
72 90B CF39A029 3EK0Q3NV 11 9 0 1A 28A 0
Note
The following area of the FCS_LINK_ERRORS output shows disk devices that
were once recognized on the loops, but that are no longer present or seen by the
controller.
FNBs not on loops - information is stale - oldest first

SIDE WWN SERIAL # ENC BAY BAD LINK_FL LS_SYNC BAD_WORD BAD_CRC
A CF399EB8 3EK0Q4JA 109 110 0 3 F 0
B CF399EB8 3EK0Q4JA 109 110 0 2 E1 0

Analyzing FCS_DELTA_LINKS outputs is no different than analyzing

FCS_LINK_ERRORS outputs. Once the FCS_CLEAR_LINKS debug command is
entered, a new baseline is set for the FCS_DELTA_LINKS output.
Because the loop Fibre Channel error counters are cumulative over time, sometimes
setting a new baseline using FCS_CLEAR_LINKS is helpful in getting a baseline
starting point for error analysis.
Note
You might want to save an output of the FCS_DELTA_LINKS to file prior to
issuing the FCS_CLEAR_LINKS command.
Lab exercise 1
The following SCS_SHOW_CONFIG output is actual output from a storage system that
was having problems recognizing its own configuration. Analyze the output and
answer the associated questions.
SCell: Active NO MFC Port
6-00508b-400011173x 0000f-0000006-[0008]x
SCell Master - Quorum POIDs:800x 815x 81ex 82cx 832x 84bx 80ax
NSC: UUID = 5-00508b-400011173x 00000-0000000-0000x
MEMBER_TAG = 6-00508b-400011173x 0000f-0000018-[0020]x
FC Nodes: Temp ID ID
Lp ALPA Lp ALPA Type Node WWN FNB Noid State Usablty
2. b6x 3. b6x disk 2-000-000c50-28cd01x 337968 0 Valid Usable
0. 67x 1. 67x disk 2-000-000c50-2b4a02x 332828 901 Valid Usable
2. 27x 3. 27x disk 2-000-000c50-2b4804x 341ff8 0 Valid Usable
0. 71x 1. 71x disk 2-000-000c50-22c105x 3317e8 0 Valid Usable
0. 31x 1. 31x disk 2-000-000c50-28e805x 336518 0 Valid Usable
2. 81x 3. 81x disk 2-000-000c50-28ce07x 33ac30 0 Valid Usable
2. 45x 3. 45x disk 2-000-000c50-28d007x 33fd70 0 Valid Usable
2. abx 3. abx disk 2-000-000c50-28cd09x 338598 0 Valid Usable
2. 66x 3. 66x disk 2-000-000c50-28cd0ex 33d4d0 0 Valid Usable
2. 6ax 3. 6ax disk 2-000-000c50-28ca0fx 33ceb8 0 Valid Usable
0. 34x 1. 34x disk 2-000-000c50-28ce0fx 335f00 0 Valid Usable
2. 55x 3. 55x disk 2-000-000c50-28ca13x 33e308 0 Valid Usable
0. a5x 1. a5x disk 2-000-000c50-22a614x 32e728 0 Valid Usable
0. 84x 1. 84x disk 2-000-000c50-22b915x 32fd80 0 Valid Usable
2. 5ax 3. 5ax disk 2-000-000c50-598016x 33dcf0 0 Valid Usable
0. 4bx 1. 4bx disk 2-000-000c50-28ce1ax 3346a0 0 Valid Usable
0. 4ex 1. 4ex disk 2-000-000c50-28cc1bx 334088 0 Valid Usable
0. 65x 1. 65x disk 2-000-000c50-28cd20x 332c38 0 Valid Usable
0. 66x 1. 66x disk 2-000-000c50-2b4a21x 332a30 912 Invalid Usable
2. 33x 3. 33x disk 2-000-000c50-28ca25x 340db0 0 Valid Usable
0. 98x 1. 98x disk 2-000-000c50-2b4526x 32f560 0 Valid Usable
2. acx 3. acx disk 2-000-000c50-28ca26x 3393d0 0 Valid Usable
0. b4x 1. b4x disk 2-000-000c50-289927x 32d2d8 0 Valid Usable
0. 90x 1. 90x disk 2-000-000c50-28cd27x 32f970 0 Valid Usable
0. 54x 1. 54x disk 2-000-000c50-28ce2ax 333868 0 Valid Usable
0. 82x 1. 82x disk 2-000-000c50-22a62bx 32ff88 0 Valid Usable
0. 4ax 1. 4ax disk 2-000-000c50-22c62bx 3348a8 0 Valid Usable

2. b1x 3. b1x disk 2-000-000c50-2b432cx 3397e0 0 Valid Usable

0. aex 1. aex disk 2-000-000c50-22c52fx 32daf8 0 Valid Usable
2. 39x 3. 39x disk 2-000-000c50-28d02fx 340590 0 Valid Usable
2. b2x 3. b2x disk 2-000-000c50-28c930x 337f80 0 Valid Usable
0. 26x 1. 26x disk 2-000-000c50-28ca31x 337558 0 Valid Usable
0. 9dx 1. 9dx disk 2-000-000c50-22c632x 32f358 0 Valid Usable
2. a7x 3. a7x disk 2-000-000c50-28c933x 3389a8 0 Valid Usable
0. 9ex 1. 9ex disk 2-000-000c50-22b239x 32f150 0 Valid Usable
0. 81x 1. 81x disk 2-000-000c50-22c539x 330190 0 Valid Usable
0. 79x 1. 79x disk 2-000-000c50-28e839x 330bb8 0 Valid Usable
2. 74x 3. 74x disk 2-000-000c50-2b453ax 33be78 0 Valid Usable
2. 6dx 3. 6dx disk 2-000-000c50-28cd3bx 33c8a0 0 Valid Usable
2. 80x 3. 80x disk 2-000-000c50-2b093cx 33ae38 0 Valid Usable
0. 7cx 1. 7cx disk 2-000-000c50-28cc3cx 3309b0 0 Valid Usable
2. b5x 3. b5x disk 2-000-000c50-28ce3ex 337b70 0 Valid Usable
2. 36x 3. 36x disk 2-000-000c50-28ca40x 340798 0 Valid Usable
2. 9ex 3. 9ex disk 2-000-000c50-28c742x 3395d8 0 Valid Usable
0. 6bx 1. 6bx disk 2-000-000c50-28cc44x 332008 0 Valid Usable
0. 2cx 1. 2cx disk 2-000-000c50-598045x 336b30 0 Valid Usable
2. aax 3. aax disk 2-000-000c50-28ca46x 3387a0 0 Valid Usable
0. 46x 1. 46x disk 2-000-000c50-28cb46x 334ec0 0 Valid Usable
2. 46x 3. 46x disk 2-000-000c50-28ca47x 33fb68 0 Valid Usable
2. 3cx 3. 3cx disk 2-000-000c50-28c749x 340180 0 Valid Usable
2. 79x 3. 79x disk 2-000-000c50-28ce4cx 33b860 0 Valid Usable
0. 59x 1. 59x disk 2-000-000c50-28c94ex 333250 933 Invalid Usable
0. 3ax 1. 3ax disk 2-000-000c50-28cf4ex 3356e0 0 Valid Usable
0. 4dx 1. 4dx disk 2-000-000c50-22a650x 334290 0 Valid Usable
2. 4cx 3. 4cx disk 2-000-000c50-28cb51x 33f140 0 Valid Usable
2. 7cx 3. 7cx disk 2-000-000c50-28d051x 33b248 0 Valid Usable
0. 2dx 1. 2dx disk 2-000-000c50-28c957x 336928 0 Valid Usable
2. 8fx 3. 8fx disk 2-000-000c50-28cb57x 33a410 0 Valid Usable
2. 98x 3. 98x disk 2-000-000c50-28cb58x 33a000 0 Valid Usable
2. adx 3. adx disk 2-000-000c50-28cf58x 338390 0 Valid Usable
2. 5cx 3. 5cx disk 2-000-000c50-2b4359x 33dae8 0 Valid Usable
2. 71x 3. 71x disk 2-000-000c50-59095cx 33c490 0 Valid Usable
0. 5ax 1. 5ax disk 2-000-000c50-28c75cx 333048 93e Valid Usable
2. aex 3. aex disk 2-000-000c50-28cd5cx 338188 0 Valid Usable
2. 26x 3. 26x disk 2-000-000c50-28d05ex 342200 0 Valid Usable
2. 2ex 3. 2ex disk 2-000-000c50-28ca5fx 3413c8 0 Valid Usable
0. 52x 1. 52x disk 2-000-000c50-28cf5fx 333c78 0 Valid Usable
0. 27x 1. 27x disk 2-000-000c50-28ce60x 337350 0 Valid Usable
2. 32x 3. 32x disk 2-000-000c50-28cf60x 340fb8 0 Valid Usable
2. 63x 3. 63x disk 2-000-000c50-28d061x 33d8e0 0 Valid Usable
2. b3x 3. b3x disk 2-000-000c50-28cc62x 337d78 0 Valid Usable
2. 6bx 3. 6bx disk 2-000-000c50-28ce62x 33ccb0 0 Valid Usable
2. 54x 3. 54x disk 2-000-000c50-28ca64x 33e510 0 Valid Usable
0. 72x 1. 72x disk 2-000-000c50-22c665x 3315e0 0 Valid Usable
2. a6x 3. a6x disk 2-000-000c50-28ce65x 338bb0 0 Valid Usable
0. aax 1. aax disk 2-000-000c50-597768x 32e110 0 Valid Usable
2. 52x 3. 52x disk 2-000-000c50-28cf68x 33e920 0 Valid Usable
2. 56x 3. 56x disk 2-000-000c50-28cf69x 33e100 0 Valid Usable
2. 82x 3. 82x disk 2-000-000c50-28d069x 33aa28 0 Valid Usable
0. 2ax 1. 2ax disk 2-000-000c50-22a66ax 336f40 0 Valid Usable
2. 2bx 3. 2bx disk 2-000-000c50-59956bx 3419e0 0 Valid Usable
0. 97x 1. 97x disk 2-000-000c50-22c46cx 32f768 0 Valid Usable
2. 4bx 3. 4bx disk 2-000-000c50-28ca6cx 33f348 0 Valid Usable
2. 97x 3. 97x disk 2-000-000c50-28ce71x 33b040 0 Valid Usable
2. 67x 3. 67x disk 2-000-000c50-2b4375x 33d2c8 0 Valid Usable
0. 56x 1. 56x disk 2-000-000c50-599878x 333458 0 Valid Usable
0. 45x 1. 45x disk 2-000-000c50-28d078x 3350c8 0 Valid Usable
2. 4ax 3. 4ax disk 2-000-000c50-28d079x 33f550 0 Valid Usable

2. 34x 3. 34x disk 2-000-000c50-22c07fx 340ba8 0 Valid Usable

2. 76x 3. 76x disk 2-000-000c50-28ca7fx 33ba68 0 Valid Usable
0. 55x 1. 55x disk 2-000-000c50-28cc82x 333660 0 Valid Usable
2. 2ax 3. 2ax disk 2-000-000c50-2b3886x 341be8 0 Valid Usable
0. 75x 1. 75x disk 2-000-000c50-22c488x 330fc8 0 Valid Usable
0. b1x 1. b1x disk 2-000-000c50-289989x 32d8f0 0 Valid Usable
0. 8fx 1. 8fx disk 2-000-000c50-28cc89x 330398 0 Valid Usable
0. abx 1. abx disk 2-000-000c50-2b138bx 32df08 0 Valid Usable
0. a3x 1. a3x disk 2-000-000c50-28cb90x 32eb38 0 Valid Usable
0. 53x 1. 53x disk 2-000-000c50-28e790x 333a70 0 Valid Usable
2. 9dx 3. 9dx disk 2-000-000c50-28ce92x 3399e8 0 Valid Usable
0. b6x 1. b6x disk 2-000-000c50-22c394x 32cec8 0 Valid Usable
2. 47x 3. 47x disk 2-000-000c50-28c794x 33f960 0 Valid Usable
2. 90x 3. 90x disk 2-000-000c50-28c994x 33a208 0 Valid Usable
0. adx 1. adx disk 2-000-000c50-2b1595x 32dd00 0 Valid Usable
2. a9x 3. a9x disk 2-000-000c50-28cb95x 338fc0 0 Valid Usable
0. 6cx 1. 6cx disk 2-000-000c50-22a096x 331e00 0 Valid Usable
0. 33x 1. 33x disk 2-000-000c50-22c698x 336108 0 Valid Usable
2. 53x 3. 53x disk 2-000-000c50-2b3599x 33e718 0 Valid Usable
2. 2cx 3. 2cx disk 2-000-000c50-28cf9bx 3417d8 0 Valid Usable
2. 9fx 3. 9fx disk 2-000-000c50-28ce9dx 3391c8 0 Valid Usable
0. 6dx 1. 6dx disk 2-000-000c50-22c59ex 331bf8 0 Valid Usable
0. 2ex 1. 2ex disk 2-000-000c50-a898a2x 336720 0 Valid Usable
2. 43x 3. 43x disk 2-000-000c50-28c9a3x 33ff78 0 Valid Usable
0. 6ex 1. 6ex disk 2-000-000c50-22c1acx 3319f0 0 Valid Usable
0. a7x 1. a7x disk 2-000-000c50-28cfaex 32e318 0 Valid Usable
0. 5cx 1. 5cx disk 2-000-000c50-28ceb2x 332e40 0 Valid Usable
0. 49x 1. 49x disk 2-000-000c50-22c4b4x 334ab0 0 Valid Usable
2. 59x 3. 59x disk 2-000-000c50-28ccb6x 33def8 0 Valid Usable
0. b5x 1. b5x disk 2-000-000c50-22dbb8x 32d0d0 0 Valid Usable
0. b2x 1. b2x disk 2-000-000c50-2b09bax 32d6e8 0 Valid Usable
2. 84x 3. 84x disk 2-000-000c50-28cbbax 33a820 0 Valid Usable
0. 43x 1. 43x disk 2-000-000c50-28cebax 3352d0 0 Valid Usable
0. 35x 1. 35x disk 2-000-000c50-22c4bbx 335cf8 0 Valid Usable
0. 29x 1. 29x disk 2-000-000c50-2b14bfx 337148 0 Valid Usable
0. 80x 1. 80x disk 2-000-000c50-28ccbfx 3305a0 0 Valid Usable
2. 2dx 3. 2dx disk 2-000-000c50-2b44c0x 3415d0 0 Valid Usable
2. 73x 3. 73x disk 2-000-000c50-28cec1x 33c080 0 Valid Usable
2. 75x 3. 75x disk 2-000-000c50-2b44c6x 33bc70 0 Valid Usable
0. 47x 1. 47x disk 2-000-000c50-28cfc6x 334cb8 0 Valid Usable
0. 76x 1. 76x disk 2-000-000c50-22a7c7x 330dc0 0 Valid Usable
0. 36x 1. 36x disk 2-000-000c50-22aac7x 335af0 0 Valid Usable
2. 9bx 3. 9bx disk 2-000-000c50-2b37c8x 339df8 0 Valid Usable
0. 3cx 1. 3cx disk 2-000-000c50-22abc8x 3354d8 0 Valid Usable
0. acx 1. acx disk 2-000-000c50-22c4c9x 32ed40 0 Valid Usable
2. 3ax 3. 3ax disk 2-000-000c50-28cfcbx 340388 0 Valid Usable
2. 29x 3. 29x disk 2-000-000c50-28cfccx 341df0 0 Valid Usable
2. 72x 3. 72x disk 2-000-000c50-5982cdx 33c288 0 Valid Usable
0. 9bx 1. 9bx disk 2-000-000c50-229ecdx 3307a8 0 Valid Usable
2. 31x 3. 31x disk 2-000-000c50-28cecdx 3411c0 0 Valid Usable
0. 4cx 1. 4cx disk 2-000-000c50-28d0cfx 334498 0 Valid Usable
0. 32x 1. 32x disk 2-000-000c50-2b12d2x 336310 0 Valid Usable
2. 65x 3. 65x disk 2-000-000c50-28ced2x 33d6d8 0 Valid Usable
2. 69x 3. 69x disk 2-000-000c50-28ced4x 33d0c0 0 Valid Usable
2. b4x 3. b4x disk 2-000-000c50-28cad5x 339bf0 0 Valid Usable
0. 88x 1. 88x disk 2-000-000c50-22c5d7x 32fb78 0 Valid Usable
2. 6cx 3. 6cx disk 2-000-000c50-28c8d7x 33caa8 0 Valid Usable
0. b3x 1. b3x disk 2-000-000c50-2898d9x 32d4e0 0 Valid Usable
0. 7ax 1. 7ax disk 2-000-000c50-22bad9x 332418 0 Valid Usable
0. 9fx 1. 9fx disk 2-000-000c50-22c1dex 32ef48 0 Valid Usable
0. 73x 1. 73x disk 2-000-000c50-2299dfx 3313d8 0 Valid Usable

2. 51x 3. 51x disk 2-000-000c50-28cde3x 33eb28 0 Valid Usable

0. 69x 1. 69x disk 2-000-000c50-22c3e6x 332620 0 Valid Usable
2. 35x 3. 35x disk 2-000-000c50-28cfe6x 3409a0 0 Valid Usable
0. a6x 1. a6x disk 2-000-000c50-2b42e7x 32e520 0 Valid Usable
2. 88x 3. 88x disk 2-000-000c50-28c8e9x 33a618 0 Valid Usable
2. a3x 3. a3x disk 2-000-000c50-28c9ebx 33b450 0 Valid Usable
2. 6ex 3. 6ex disk 2-000-000c50-28ccf0x 33c698 0 Valid Usable
2. 49x 3. 49x disk 2-000-000c50-28cef0x 33f758 0 Valid Usable
2. 7ax 3. 7ax disk 2-000-000c50-28cbf1x 33b658 0 Valid Usable
0. 74x 1. 74x disk 2-000-000c50-229ef3x 3311d0 0 Valid Usable
2. a5x 3. a5x disk 2-000-000c50-28c9f3x 338db8 0 Valid Usable
0. 39x 1. 39x disk 2-000-000c50-22b5f5x 3358e8 0 Valid Usable
2. 4ex 3. 4ex disk 2-000-000c50-28cbf9x 33ed30 0 Valid Usable
0. a9x 1. a9x disk 2-000-000c50-28cdf9x 32e930 0 Valid Usable
0. 6ax 1. 6ax disk 2-000-000c50-28e7fax 332210 0 Valid Usable
0. 51x 1. 51x disk 2-000-000c50-28cdfbx 333e80 0 Valid Usable
0. 2bx 1. 2bx disk 2-000-000c50-22c5fdx 336d38 0 Valid Usable
2. 4dx 3. 4dx disk 2-000-000c50-28c9fex 33ef38 0 Valid Usable
0. 1x 1. 1x --
2. 1x 3. 1x NSC 5-00508b-400011173x 337760 0
1. In the previous output, how many disks are not grouped, and how many
controllers currently comprise the storage system?
................................................................................................................
................................................................................................................
InUse nub list:

NSCNoid WWN
Member Tag NUB Gate
20x 5-00508b-400011173x
6-00508b-400011173x 0000f-0000018-[0020]x 39c3b8 0x
InUse dub list: rss rss

Lp ALPA Poid Volnoid MAIRFA CSMM WWN DUB Gate id ndx
1. a5x 800xQ 400x IDLE 2-000-000c50-22a614x 1c12160x 0x 001 02
2. 35x 815xQ 415x IDLE 2-000-000c50-28cfe6x 1c124a0x 0x 004 03
2. 67x 81exQ 41ex IDLE 2-000-000c50-2b4375x 1c127e0x 0x 006 00
2. 79x 82cxQ 42cx IDLE 2-000-000c50-28ce4cx 1c12b20x 0x 008 02
3. b1x 832xQ 432x IDLE 2-000-000c50-2b432cx 1c12e60x 0x 009 02
2. 81x 84bxQ 44bx IDLE 2-000-000c50-28ce07x 1c131a0x 0x 00c 07
3. 51x 80axQ 40ax IDLE 2-000-000c50-28cde3x 1c134e0x 0x 001 04
2. b6x 806x 406x IDLE 2-000-000c50-28cd01x 1c13820x 0x 002 00
3. 27x 856x 456x IDLE 2-000-000c50-2b4804x 1c13b60x 0x 017 03
1. 71x 874x 474x IDLE 2-000-000c50-22c105x 1c13ea0x 0x 013 04
1. 31x 881x 481x IDLE 2-000-000c50-28e805x 1c141e0x 0x 011 06
3. 45x 87cx 47cx IDLE 2-000-000c50-28d007x 1c14520x 0x 013 00
2. abx 879x 479x IDLE 2-000-000c50-28cd09x 1c14860x 0x 012 05
2. 66x 82ax 42ax IDLE 2-000-000c50-28cd0ex 1c14ba0x 0x 008 00
2. 6ax 807x 407x IDLE 2-000-000c50-28ca0fx 1c14ee0x 0x 002 01
1. 34x 819x 419x IDLE 2-000-000c50-28ce0fx 1c15220x 0x 006 0a
2. 55x 872x 472x IDLE 2-000-000c50-28ca13x 1c15560x 0x 014 07
1. 84x 860x 460x IDLE 2-000-000c50-22b915x 1c158a0x 0x 015 04
2. 5ax 8a7x 4a7x IDLE 2-000-000c50-598016x 1c15be0x 0x 00e 07
0. 4bx 83ax 43ax IDLE 2-000-000c50-28ce1ax 1c15f20x 0x 00a 02
1. 4ex 80dx 40dx IDLE 2-000-000c50-28cc1bx 1c16260x 0x 003 01
0. 65x 838x 438x A IDLE 2-000-000c50-28cd20x 1c165a0x 0x 00a 00
2. 33x 82bx 42bx IDLE 2-000-000c50-28ca25x 1c168e0x 0x 008 01
1. 98x 890x 490x IDLE 2-000-000c50-2b4526x 1c16c20x 0x 00f 04

3. acx 885x 485x IDLE 2-000-000c50-28ca26x 1c16f60x 0x 011 00

1. b4x 81bx 41bx IDLE 2-000-000c50-289927x 1c172a0x 0x 006 07
1. 90x 884x 484x IDLE 2-000-000c50-28cd27x 1c175e0x 0x 011 01
0. 54x 85fx 45fx IDLE 2-000-000c50-28ce2ax 1c17920x 0x 016 05
1. 82x 854x 454x IDLE 2-000-000c50-22a62bx 1c17c60x 0x 017 01
0. 4ax 882x 482x IDLE 2-000-000c50-22c62bx 1c17fa0x 0x 012 02
1. aex 897x 497x IDLE 2-000-000c50-22c52fx 1c182e0x 0x 00f 03
3. 39x 84cx 44cx IDLE 2-000-000c50-28d02fx 1c18620x 0x 00d 00
3. b2x 83ex 43ex IDLE 2-000-000c50-28c930x 1c18960x 0x 00b 01
0. 26x 845x 445x IDLE 2-000-000c50-28ca31x 1c18ca0x 0x 00c 01
0. 9dx 83cx 43cx IDLE 2-000-000c50-22c632x 1c18fe0x 0x 00a 05
3. a7x 855x 455x IDLE 2-000-000c50-28c933x 1c19320x 0x 017 00
1. 9ex 828x 428x IDLE 2-000-000c50-22b239x 1c19660x 0x 008 08
0. 81x 848x 448x IDLE 2-000-000c50-22c539x 1c199a0x 0x 00c 04
1. 79x 829x 429x IDLE 2-000-000c50-28e839x 1c19ce0x 0x 008 09
2. 74x 895x 495x IDLE 2-000-000c50-2b453ax 1c1a020x 0x 010 01
2. 6dx 865x 465x IDLE 2-000-000c50-28cd3bx 1c1a360x 0x 016 01
2. 80x 808x 408x IDLE 2-000-000c50-2b093cx 1c1a6a0x 0x 002 02
1. 7cx 80ex 40ex IDLE 2-000-000c50-28cc3cx 1c1a9e0x 0x 003 02
2. b5x 812x 412x IDLE 2-000-000c50-28ce3ex 1c1ad20x 0x 004 00
2. 36x 809x 409x IDLE 2-000-000c50-28ca40x 1c1b060x 0x 002 03
2. 9ex 82ex 42ex IDLE 2-000-000c50-28c742x 1c1b3a0x 0x 008 04
0. 6bx 844x 444x IDLE 2-000-000c50-28cc44x 1c1b6e0x 0x 00c 00
1. 2cx 87ax 47ax IDLE 2-000-000c50-598045x 1c1ba20x 0x 011 04
2. aax 86dx 46dx IDLE 2-000-000c50-28ca46x 1c1bd60x 0x 014 00
1. 46x 8a5x 4a5x IDLE 2-000-000c50-28cb46x 1c1c0a0x 0x 00d 06
2. 46x 888x 488x IDLE 2-000-000c50-28ca47x 1c1c3e0x 0x 011 05
2. 3cx 864x 464x IDLE 2-000-000c50-28c749x 1c1c720x 0x 016 00
0. 3ax 852x 452x IDLE 2-000-000c50-28cf4ex 1c1ca60x 0x 018 00
1. 4dx 81ax 41ax IDLE 2-000-000c50-22a650x 1c1cda0x 0x 006 06
2. 4cx 82fx 42fx IDLE 2-000-000c50-28cb51x 1c1d0e0x 0x 008 05
2. 7cx 814x 414x IDLE 2-000-000c50-28d051x 1c1d420x 0x 004 02
0. 2dx 899x 499x IDLE 2-000-000c50-28c957x 1c1d760x 0x 018 01
2. 8fx 87bx 47bx IDLE 2-000-000c50-28cb57x 1c1daa0x 0x 012 07
3. 98x 893x 493x IDLE 2-000-000c50-28cb58x 1c1dde0x 0x 00f 07
2. adx 89dx 49dx IDLE 2-000-000c50-28cf58x 1c1e120x 0x 00e 00
2. 5cx 896x 496x IDLE 2-000-000c50-2b4359x 1c1e460x 0x 018 07
3. 71x 889x 489x IDLE 2-000-000c50-59095cx 1c1e7a0x 0x 017 06
3. aex 891x 491x IDLE 2-000-000c50-28cd5cx 1c1eae0x 0x 00f 05
2. 26x 84ax 44ax IDLE 2-000-000c50-28d05ex 1c1ee20x 0x 00c 06
3. 2ex 892x 492x IDLE 2-000-000c50-28ca5fx 1c1f160x 0x 00f 06
0. 52x 847x 447x A IDLE 2-000-000c50-28cf5fx 1c1f4a0x 0x 00c 03
0. 27x 851x 451x IDLE 2-000-000c50-28ce60x 1c1f7e0x 0x 018 04
3. 32x 840x 440x IDLE 2-000-000c50-28cf60x 1c1fb20x 0x 00b 03
3. 63x 833x 433x IDLE 2-000-000c50-28d061x 1c1fe60x 0x 009 03
2. b3x 82dx 42dx IDLE 2-000-000c50-28cc62x 1c201a0x 0x 008 03
3. 6bx 84dx 44dx IDLE 2-000-000c50-28ce62x 1c204e0x 0x 00d 01
2. 54x 866x 466x IDLE 2-000-000c50-28ca64x 1c20820x 0x 016 02
1. 72x 8a3x 4a3x IDLE 2-000-000c50-22c665x 1c20b60x 0x 00d 04
2. a6x 849x 449x IDLE 2-000-000c50-28ce65x 1c20ea0x 0x 00c 05
0. aax 8a2x 4a2x IDLE 2-000-000c50-597768x 1c211e0x 0x 00e 06
3. 52x 84ex 44ex IDLE 2-000-000c50-28cf68x 1c21520x 0x 00d 02
3. 56x 87ex 47ex IDLE 2-000-000c50-28cf69x 1c21860x 0x 013 02
3. 82x 857x 457x IDLE 2-000-000c50-28d069x 1c21ba0x 0x 017 02
0. 2ax 869x 469x IDLE 2-000-000c50-22a66ax 1c21ee0x 0x 014 06
2. 2bx 87dx 47dx IDLE 2-000-000c50-59956bx 1c22220x 0x 012 06
0. 97x 89cx 49cx IDLE 2-000-000c50-22c46cx 1c22560x 0x 00e 01
3. 4bx 843x 443x IDLE 2-000-000c50-28ca6cx 1c228a0x 0x 00b 04
2. 97x 89fx 49fx IDLE 2-000-000c50-28ce71x 1c22be0x 0x 00e 02
1. 56x 877x 477x A IDLE 2-000-000c50-599878x 1c22f20x 0x 013 05
1. 45x 876x 476x IDLE 2-000-000c50-28d078x 1c23260x 0x 013 06

3. 4ax 837x 437x IDLE 2-000-000c50-28d079x 1c235a0x 0x 009 07

2. 34x 821x 421x IDLE 2-000-000c50-22c07fx 1c238e0x 0x 006 03
3. 76x 841x 441x IDLE 2-000-000c50-28ca7fx 1c23c20x 0x 00b 02
0. 55x 86bx 46bx ------ IDLE 2-000-000c50-28cc82x 1c23f60x 0x 000 00
2. 2ax 86ex 46ex IDLE 2-000-000c50-2b3886x 1c242a0x 0x 014 03
0. 75x 880x 480x IDLE 2-000-000c50-22c488x 1c245e0x 0x 012 01
1. b1x 831x 431x IDLE 2-000-000c50-289989x 1c24920x 0x 009 01
0. 8fx 878x 478x IDLE 2-000-000c50-28cc89x 1c24c60x 0x 012 04
1. abx 873x 473x IDLE 2-000-000c50-2b138bx 1c24fa0x 0x 013 07
1. a3x 80cx 40cx IDLE 2-000-000c50-28cb90x 1c252e0x 0x 003 00
0. 53x 853x 453x A IDLE 2-000-000c50-28e790x 1c25620x 0x 018 02
3. 9dx 842x 442x IDLE 2-000-000c50-28ce92x 1c25960x 0x 00b 05
1. b6x 801x 401x IDLE 2-000-000c50-22c394x 1c25ca0x 0x 001 03
2. 47x 8a0x 4a0x IDLE 2-000-000c50-28c794x 1c25fe0x 0x 00e 05
3. 90x 887x 487x IDLE 2-000-000c50-28c994x 1c26320x 0x 011 02
0. adx 88bx 48bx IDLE 2-000-000c50-2b1595x 1c26660x 0x 010 07
3. a9x 861x 461x IDLE 2-000-000c50-28cb95x 1c269a0x 0x 015 05
0. 6cx 850x 450x IDLE 2-000-000c50-22a096x 1c26ce0x 0x 018 03
1. 33x 825x 425x IDLE 2-000-000c50-22c698x 1c27020x 0x 008 0a
2. 53x 85ax 45ax IDLE 2-000-000c50-2b3599x 1c27360x 0x 018 05
3. 2cx 886x 486x IDLE 2-000-000c50-28cf9bx 1c276a0x 0x 011 03
2. 9fx 822x 422x IDLE 2-000-000c50-28ce9dx 1c279e0x 0x 006 04
0. 6dx 85cx 45cx IDLE 2-000-000c50-22c59ex 1c27d20x 0x 016 04
0. 2ex 804x 404x IDLE 2-000-000c50-a898a2x 1c28060x 0x 002 0b
2. 43x 870x 470x IDLE 2-000-000c50-28c9a3x 1c283a0x 0x 014 05
0. 6ex 868x 468x IDLE 2-000-000c50-22c1acx 1c286e0x 0x 015 01
0. a7x 85bx 45bx IDLE 2-000-000c50-28cfaex 1c28a20x 0x 016 07
0. 5cx 88fx 48fx A IDLE 2-000-000c50-28ceb2x 1c28d60x 0x 010 05
0. 49x 88ex 48ex IDLE 2-000-000c50-22c4b4x 1c290a0x 0x 010 06
3. 59x 88ax 48ax IDLE 2-000-000c50-28ccb6x 1c293e0x 0x 011 07
1. b5x 80fx 40fx IDLE 2-000-000c50-22dbb8x 1c29720x 0x 003 03
0. b2x 83dx 43dx IDLE 2-000-000c50-2b09bax 1c29a60x 0x 00a 04
3. 84x 863x 463x IDLE 2-000-000c50-28cbbax 1c29da0x 0x 015 07
1. 43x 86ax 46ax IDLE 2-000-000c50-28cebax 1c2a0e0x 0x 015 02
1. 35x 810x 410x IDLE 2-000-000c50-22c4bbx 1c2a420x 0x 003 04
0. 29x 85dx 45dx IDLE 2-000-000c50-2b14bfx 1c2a760x 0x 016 03
1. 80x 802x 402x IDLE 2-000-000c50-28ccbfx 1c2aaa0x 0x 001 00
2. 2dx 89ex 49ex IDLE 2-000-000c50-2b44c0x 1c2ade0x 0x 00e 03
2. 73x 8a1x 4a1x IDLE 2-000-000c50-28cec1x 1c2b120x 0x 00e 04
3. 75x 835x 435x IDLE 2-000-000c50-2b44c6x 1c2b460x 0x 009 05
1. 47x 89ax 49ax IDLE 2-000-000c50-28cfc6x 1c2b7a0x 0x 00f 02
0. 76x 83bx 43bx IDLE 2-000-000c50-22a7c7x 1c2bae0x 0x 00a 03
1. 36x 803x 403x IDLE 2-000-000c50-22aac7x 1c2be20x 0x 001 01
3. 9bx 836x 436x IDLE 2-000-000c50-2b37c8x 1c2c160x 0x 009 06
0. 3cx 85ex 45ex IDLE 2-000-000c50-22abc8x 1c2c4a0x 0x 016 06
0. acx 87fx 47fx IDLE 2-000-000c50-22c4c9x 1c2c7e0x 0x 012 03
3. 3ax 858x 458x IDLE 2-000-000c50-28cfcbx 1c2cb20x 0x 017 05
3. 29x 862x 462x IDLE 2-000-000c50-28cfccx 1c2ce60x 0x 015 06
3. 72x 8a4x 4a4x IDLE 2-000-000c50-5982cdx 1c2d1a0x 0x 017 07
1. 9bx 830x 430x IDLE 2-000-000c50-229ecdx 1c2d4e0x 0x 009 00
3. 31x 834x 434x IDLE 2-000-000c50-28cecdx 1c2d820x 0x 009 04
1. 4cx 826x 426x IDLE 2-000-000c50-28d0cfx 1c2db60x 0x 008 06
0. 32x 839x 439x IDLE 2-000-000c50-2b12d2x 1c2dea0x 0x 00a 01
3. 65x 83fx 43fx IDLE 2-000-000c50-28ced2x 1c2e1e0x 0x 00b 00
2. 69x 813x 413x IDLE 2-000-000c50-28ced4x 1c2e520x 0x 004 01
2. b4x 81fx 41fx IDLE 2-000-000c50-28cad5x 1c2e860x 0x 006 01
0. 88x 86cx 46cx IDLE 2-000-000c50-22c5d7x 1c2eba0x 0x 014 01
3. 6cx 859x 459x IDLE 2-000-000c50-28c8d7x 1c2eee0x 0x 017 04
1. b3x 827x 427x IDLE 2-000-000c50-2898d9x 1c2f220x 0x 008 07
1. 7ax 81dx 41dx IDLE 2-000-000c50-22bad9x 1c2f560x 0x 006 09
1. 9fx 81cx 41cx IDLE 2-000-000c50-22c1dex 1c2f8a0x 0x 006 08

1. 73x 898x 498x IDLE 2-000-000c50-2299dfx 1c2fbe0x 0x 00f 01

1. 69x 811x 411x A IDLE 2-000-000c50-22c3e6x 1c2ff20x 0x 003 05
1. a6x 84fx 44fx IDLE 2-000-000c50-2b42e7x 1c30260x 0x 00d 07
2. 88x 86fx 46fx IDLE 2-000-000c50-28c8e9x 1c305a0x 0x 014 02
2. a3x 817x 417x IDLE 2-000-000c50-28c9ebx 1c308e0x 0x 004 05
2. 6ex 871x 471x IDLE 2-000-000c50-28ccf0x 1c30c20x 0x 014 04
2. 49x 894x 494x IDLE 2-000-000c50-28cef0x 1c30f60x 0x 010 00
2. 7ax 820x 420x IDLE 2-000-000c50-28cbf1x 1c312a0x 0x 006 02
0. 74x 88cx 48cx IDLE 2-000-000c50-229ef3x 1c315e0x 0x 010 04
3. a5x 80bx 40bx IDLE 2-000-000c50-28c9f3x 1c31920x 0x 001 05
0. 39x 846x 446x IDLE 2-000-000c50-22b5f5x 1c31c60x 0x 00c 02
2. 4ex 816x 416x IDLE 2-000-000c50-28cbf9x 1c31fa0x 0x 004 04
1. a9x 867x 467x IDLE 2-000-000c50-28cdf9x 1c322e0x 0x 015 03
0. 6ax 8a8x 4a8x A IDLE 2-000-000c50-28e7fax 1c32620x 0x 018 06
0. 51x 805x 405x IDLE 2-000-000c50-28cdfbx 1c32960x 0x 002 0a
1. 2bx 875x 475x IDLE 2-000-000c50-22c5fdx 1c32ca0x 0x 013 03
2. 4dx 823x 423x IDLE 2-000-000c50-28c9fex 1c32fe0x 0x 006 05
2. Analyze the previous InUse dub listing. List any indications of problems along
with any associated analysis.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
All RSSs:
RSS LDAD Free Members
Noid Noid PSEGS
Abnrml Missing MgrFlgs MbrMgrt SrcRIdx TrgRIdx MgrAr[0] MgrAr[1]
Member Blk
Volnoid Capacity
200x 0x 0. 7.
400x 286749488.
415x 286749488.
41ex 286749488.
42cx 286749488.
432x 286749488.
44bx 286749488.
40ax 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB

3. Based on the quorum RSS membership previously shown, how many total disk
groups should be present in this storage system?
Note
The answer to this question can be confirmed using the data displayed
immediately following question 4 of this lab.
................................................................................................................
201x 100x 0. 6.
ffc0x ffc0x 0x 0x 0x 0x 0x 0x
402x 286749488.
403x 286749488.
400x 286749488.
401x 286749488.
40ax 286749488.
40bx 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
202x 100x 0. 6.
f3f0x f3f0x 0x 0x 0x 0x 0x 0x
406x 286749488.
407x 286749488.
408x 286749488.
409x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
405x 286749488.
404x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
203x 101x 0. 6.
ffe0x ffc0x 0x 0x 0x 0x 0x 0x
40cx 286749488.
40dx 286749488.
40ex 286749488.
40fx 286749488.
410x 286749488.
411x 286749488.
Null_DUB
Null_DUB
Null_DUB

Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
204x 101x 0. 6.
412x 286749488.
413x 286749488.
414x 286749488.
415x 286749488.
416x 286749488.
417x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
205x 102x 0. 5.
ffc1x ffc1x 6x 3ex 5x 6x 9876a0x 0x
Null_DUB
419x 286749488.
41ax 286749488.
41bx 286749488.
41cx 286749488.
41dx 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
206x 102x 0. 11.
f800x f800x 5x 7c0x 5x 6x 32000000x 154x
41ex 286749488.
41fx 286749488.
420x 286749488.
421x 286749488.
422x 286749488.
423x 286749488.
41ax 286749488.
41bx 286749488.
41cx 286749488.
41dx 286749488.
419x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB

207x 103x 0. 5.
ffc1x ffc1x 6x 3ex 7x 8x 9876a0x 0x
Null_DUB
425x 286749488.
426x 286749488.
427x 286749488.
428x 286749488.
429x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
208x 103x 0. 11.
f800x f800x 5x 7c0x 7x 8x 32000000x 154x
42ax 286749488.
42bx 286749488.
42cx 286749488.
42dx 286749488.
42ex 286749488.
42fx 286749488.
426x 286749488.
427x 286749488.
428x 286749488.
429x 286749488.
425x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
209x 104x 0. 8.
430x 286749488.
431x 286749488.
432x 286749488.
433x 286749488.
434x 286749488.
435x 286749488.
436x 286749488.
437x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
20ax 104x 0. 6.
438x 286749488.
439x 286749488.
43ax 286749488.
43bx 286749488.
43dx 286749488.

43cx 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
20bx 104x 0. 6.
43fx 286749488.
43ex 286749488.
441x 286749488.
440x 286749488.
443x 286749488.
442x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
20cx 105x 0. 8.
444x 286749488.
445x 286749488.
446x 286749488.
447x 286749488.
448x 286749488.
449x 286749488.
44ax 286749488.
44bx 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
20dx 105x 0. 7.
ff20x ff20x 9x 8x 11x dx 5000x 0x
44cx 286749488.
44dx 286749488.
44ex 286749488.
488x 286749488.
4a3x 286749488.
Null_DUB
4a5x 286749488.
44fx 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB

Null_DUB
Null_DUB
Null_DUB
Null_DUB
20ex 105x 0. 8.
49dx 286749488.
49cx 286749488.
49fx 286749488.
49ex 286749488.
4a1x 286749488.
4a0x 286749488.
4a2x 286749488.
4a7x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
20fx 105x 0. 8.
ff00x ff00x 9x 1x 15x fx 1x 0x
468x 286749488.
498x 286749488.
49ax 286749488.
497x 286749488.
490x 286749488.
491x 286749488.
492x 286749488.
493x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
210x 105x 0. 6.
ff2cx ff0cx 0x 0x 0x 0x 0x 0x
494x 286749488.
495x 286749488.
Null_DUB
Null_DUB
48cx 286749488.
48fx 286749488.
48ex 286749488.
48bx 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB

211x 105x 0. 8.
ff00x ff00x ax 20x 11x dx 300000x 0x
485x 286749488.
484x 286749488.
487x 286749488.
486x 286749488.
47ax 286749488.
488x 286749488.
481x 286749488.
48ax 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
212x 105x 0. 7.
ff01x ff01x ax 80x 12x 13x 10000000x 0x
Null_DUB
480x 286749488.
482x 286749488.
47fx 286749488.
478x 286749488.
479x 286749488.
47dx 286749488.
47bx 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
213x 105x 0. 8.
ff20x ff00x 9x 2x 12x 13x 70x 0x
47cx 286749488.
47bx 286749488.
47ex 286749488.
475x 286749488.
474x 286749488.
477x 286749488.
476x 286749488.
473x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
214x 105x 0. 8.
46dx 286749488.
46cx 286749488.
46fx 286749488.
46ex 286749488.
471x 286749488.

470x 286749488.
469x 286749488.
472x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
215x 105x 0. 7.
ff01x ff01x ax 2x 15x fx 0x 0x
Null_DUB
468x 286749488.
46ax 286749488.
467x 286749488.
460x 286749488.
461x 286749488.
462x 286749488.
463x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
216x 105x 0. 8.
464x 286749488.
465x 286749488.
466x 286749488.
45dx 286749488.
45cx 286749488.
45fx 286749488.
45ex 286749488.
45bx 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
217x 105x 0. 8.
455x 286749488.
454x 286749488.
457x 286749488.
456x 286749488.
459x 286749488.
458x 286749488.
489x 286749488.
4a4x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB

Null_DUB
Null_DUB
Null_DUB
Null_DUB
218x 105x 0. 8.
452x 286749488.
499x 286749488.
453x 286749488.
450x 286749488.
451x 286749488.
45ax 286749488.
4a8x 286749488.
496x 286749488.
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
Null_DUB
4. Is there anything unusual about the RSS memberships shown for the above-listed
disk groups?
................................................................................................................
................................................................................................................
................................................................................................................
219x Invalid RSS

21ax Invalid RSS
Truncated output
LDADs:
LDAD RSS
Noid noid
100x
201x
202x
101x
203x
204x
102x
205x
206x
103x
207x
208x
104x
209x
20ax
20bx

105x
20cx
20dx
20ex
20fx
210x
211x
212x
213x
214x
215x
216x
217x
218x
LDAD Member
Noid Volnoid
100x
400x
401x
402x
403x
404x
405x
406x
407x
408x
409x
40ax
40bx
101x
40cx
40dx
40ex
40fx
410x
411x
412x
413x
414x
415x
416x
417x
102x
419x
41ax
41bx
41cx
41dx
41ex
41fx
420x
421x
422x
423x
103x
425x
426x
427x
428x

429x
42ax
42bx
42cx
42dx
42ex
42fx
104x
430x
431x
432x
433x
434x
435x
436x
437x
438x
439x
43ax
43bx
43cx
43dx
43ex
43fx
440x
441x
442x
443x
105x
444x
445x
446x
447x
448x
449x
Truncated Output
4a7x
44bx
4a8x
LDs
LD LDAD Max L2MAP Cache LDSB Pres Host Pres Rlzd
Noid Noid LDA addr Redun State Flags Count Qscs Othr SCS
SCVDs
SCVD LDAD LD
Noid Noid Noid LDSB

5. Analyze and comment on the previous listing of storage system LDs and SCVDs.
................................................................................................................
................................................................................................................
................................................................................................................
6. What is your analysis of the previous SCS_SHOW_CONFIG output?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................

Answers
1. In the previous output, how many disks are not grouped, and how many
controllers currently comprise the storage system?
Four disks are not grouped, and the storage system is comprised of a single
controller. Although temp NOIDs 901 and 93E indicate a state of Valid/Usable
(which is unusual and points to a possible metadata corruption), this indicates
the disk is grouped, yet if it were, the disk would not have a temp NOID. Also,
the InUse dub listing that later follows, does not reference the WWNs
associated with NOIDs 901 and 93E, further indicating those two disks are not
grouped.
2. Analyze the previous InUse dub listing. List any indications of problems along
with any associated analysis.
Seven of the disks are marked with an A in the MAIRFA flags area. An A in the
2nd position of MAIRFA indicates that the current disk state is Abnormal.
In addition, volume 46bx has a MAIRFA flags indication of ‘------‘. This disk has
an RSS ID of 0 with an INDEX setting of 0. This is not a valid combination for a
device that is in use. In-use devices have RSS IDs that are non-zero. Further, if
you were to look back into the FC Nodes output, this disk (with WWN ending
in CC82) shows up as having a temp NOID of 0, indicating the drive is
grouped. Therefore, volume 46bx is thought to be a part of a disk group, but it
is not part of any disk group.
3. Based on the quorum RSS membership previously shown, how many total disk
groups should be present in this storage system?
Seven quorum disks is an indication that there should be seven disk groups
present. However, when there are six disk groups present in the system (and the
default disk group has not been deleted), there will be two quorum disks present
in the default disk group. Therefore, in this situation, although there are seven
quorum disks, there are only six total disk groups.
4. Is there anything unusual about the RSS memberships shown for the above-listed
disk groups?
Three of the disk groups (NOIDS 0x102, 0x103, and 0x105) are comprised of
more than one odd-membered RSS group. Under normal circumstances within a
disk group, there should only ever be one RSS group with an odd number of
members in it. In this case, there are three different disk groups undergoing
merge/split operations.

5. Analyze and comment on the previous listing of storage system LDs and SCVDs.
Either no logical disks and storage system virtual disks have been carved out of
the six present disk groups, or the storage system metadata has been corrupted
such that the logical disk information cannot be read.
6. What is your analysis of the previous SCS_SHOW_CONFIG output?
There are seven disks in abnormal states and one that believes it is in a disk
group when in fact it is not.
Three different disk groups are comprised of more than one odd-membered RSS
group.
No LD or SCVD information can be displayed, or no LD of SCVD information is
any longer present on the system.
Analysis: The metadata on this system has been corrupted.
In this situation, it was due to passing a bad pointer to the merge/split recovery
subroutine, which was a bug in VCS 3.000/3.001 that was fixed in VCS 3.014.
This specific problem was a direct result of a bad power-on sequence. The
controllers were booted first, and then the disk enclosures. The controllers went
into a recursive bugcheck state.
In the process of dealing with disks that were ‘missing’, the VCS code ran into a
bug and this was the result.

The following brief output is from the controller console port that the previous outputs
were generated from. Review the bolded area.
SCSCSM - Start Cell Realization

SCSCSM - Cell Realization: quorum valid
SCSCSM - Cell Realization: quorum reconciled
FCS: prt 0 Short status 08020000 al_pa b4 - shelf 6, slot
3+&*
FCS: prt 0 Short status 08020000 al_pa b4 - shelf 6, slot 3*+
SCSCSM - Cell Realization: volumes realized
SCSCVM - RSS 203x member 5. is MERGING
SCSCVM - RSS 20ax member 0. is MERGING
SCSCVM - RSS 20cx member 3. is MERGING
SCSCSM - Cell Realization: rsss ldads realized
SCSCSM - Cell Realization: cvm reconciled
SCSCSM - Cell Realization: meta reconciled
SCSCSM - Cell Realization: csld realized
SCSCSM - Cell Realization: cmaps realized
Note the seven RSS members that indicate that they are in merging state. These are
the same seven disks that appeared as Abnormal previously in the
SCS_SHOW_CONFIG output.

Lab exercise 2
The following FCS_LINK_ERRORS output is actual output from a storage system that
was having loop errors on both loop pairs. Analyze the output and answer the
associated questions.
***** DP-1A Loop Order Invalid -- Using ordering from DP-1B
DP-1A pcb 32cf94 loop map: 42 devices, reporting group 4052 Link Failed Delta
1 0 [Controller] 7 0##
33 0 CF3960BC 3EK0PJVW 1 4 0 D 5B 0
32 0 CF29D493 3EK0LVMZ 1 5 0 D 5C 0
31 0 CF394819 3EK0P66E 1 6 0 E 4C 0
2E 0 CF39604C 3EK0PCYP 1 7 0 F 4C 0
2D 0 CF393694 3EK0P6S2 1 8 0 C 4C 0
2C 0 CF393D90 3EK0P7LA 1 9 0 16 4C 0
2B 0 CF393C9B 3EK0P7DZ 1 10 0 D 4C 0
4C 0 CF1FE0D1 3EK0HV62 2 4 0 9 4D 0
4B 0 CF3949CF 3EK0PDWB 2 5 0 E 4C 0
4A 0 CF29E3B3 3EK0LSL7 2 6 0 E 4C 0
49 0 CF3964AF 3EK0PG7N 2 7 0 13 1004C 0
1. What does the first line of this FCS_LINK_ERRORS output indicate?

................................................................................................................
................................................................................................................
2. Should you be concerned that drive 2, 7 has a much larger number of
BAD_WORDs than the other members of enclosure 2?
................................................................................................................
................................................................................................................
................................................................................................................
47 0 CF393EA4 3EK0P68F 2 8 0 F 4E 0
46 0 CF399FD1 3EK0Q634 2 9 0 E 61 0
45 0 CF940C92 3EK1J25A 2 10 0 A 40 0
B3 0 CF1FE1CF 3EKYNDZH 6 4 0 A A2 0

B2 0 CF39866F 3EK0PXYD 6 5 0 E 4E 0
B1 0 CFBF3D08 3EK2FS3L 6 6 0 E 4D 0
AE 0 CF39869B 3EK0PM5F 6 7 0 13 4C 0
AD 0 CF3964D6 3EK0MPTB 6 8 0 D 4E 0
AC 0 CF394AAD 3EK0PER0 6 9 0 E 4E 0
9E 0 CF399F00 3EK0Q2GC 5 4 0 9 4E 0
9D 0 CF399FEF 3EK0Q618 5 5 0 13 1004C 0

BAD_WORDs than the other members of enclosure 5? Also, did you notice
anything interesting about the actual number of BAD_WORDs detected?
................................................................................................................
................................................................................................................
................................................................................................................
9B 0 CF399E14 3EK0Q20K 5 6 0 E 4C 0
98 0 CF399EA1 3EK0Q50Q 5 7 0 11 4C 0
90 0 CF3949BF 3EK0PF0N 5 9 13A 3CC0E 60100 0
4. How would you analyze the error statistics for drive 5, 9?

................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
8F 0 CF395E94 3EK0PJM2 5 10 0 B 4C 0
79 0 CF3986F1 3EK0PXE4 4 4 0 8 4E 0
76 0 CFBF40C9 3EK2FTBV 4 5 0 A 4C 0
75 0 CF1FECBE 3EK0HYY4 4 6 0 C 1004C 0


................................................................................................................
74 0 CF3937B4 3EK0P77F 4 7 0 9 4C 0
73 0 CF399FDD 3EK0PXCN 4 8 0 A 52 0
72 0 CF3947DD 3EK0PECK 4 9 0 5 4C 0
71 0 CF394AA1 3EK0PFA0 4 10 0 12 4C 0
66 0 CF1FE247 3EKYNH2R 3 4 0 12 4C 0
65 0 CF39379B 3EK0P7BK 3 5 0 10 4C 0
63 0 CF29E205 3EK0LYT4 3 6 0 B 1004C 0

................................................................................................................
................................................................................................................
5C 0 CF3940DB 3EK0P7M0 3 7 0 9 4C 0
5A 0 CF395E6A 3EK0MFZS 3 8 0 11 4C 0
59 0 CF241C2A 3EK0JA10 3 9 0 6 4C 0
56 0 CF393DA2 3EK0P88L 3 10 0 C 4C 0
7. What can you conclude from the analysis of the loop 1A error statistics?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
DP-1B pcb 32cf94 loop map: 42 devices, reporting group 4052 Link Up Delta
1 20 [Controller] 7 0 0 0 0 0
33 826 CF3960BC 3EK0PJVW 1 4 81 DF1 C15CB 0
8. How would you analyze the error statistics for drive 1, 4 on loop 1B?
................................................................................................................
................................................................................................................

32 84A CF29D493 3EK0LVMZ 1 5 0 7E7 150F9D 0

31 83E CF394819 3EK0P66E 1 6 0 7E8 130FC3 0
2E 832 CF39604C 3EK0PCYP 1 7 0 7EA 141020 0
2D 81A CF393694 3EK0P6S2 1 8 1 7C3 120F35 0
2C 80E CF393D90 3EK0P7LA 1 9 1 800 130F74 0
2B 802 CF393C9B 3EK0P7DZ 1 10 1 7D0 140F90 0
4C 827 CF1FE0D1 3EK0HV62 2 4 10 CEF B12F8 0
4B 84B CF3949CF 3EK0PDWB 2 5 0 7DA 150FA6 0
4A 83F CF29E3B3 3EK0LSL7 2 6 0 7B3 150F4F 0
49 833 CF3964AF 3EK0PG7N 2 7 1 7C1 150F2E 0
47 81B CF393EA4 3EK0P68F 2 8 B0 4C948 954809 0

................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
46 80F CF399FD1 3EK0Q634 2 9 3 8C4 F1170 0

45 803 CF940C92 3EK1J25A 2 10 1 87E F1078 0
B3 829 CF1FE1CF 3EKYNDZH 6 4 5A DC1 C15C4 0
................................................................................................................
................................................................................................................
B2 843 CF39866F 3EK0PXYD 6 5 0 840 14107F 0

B1 837 CFBF3D08 3EK2FS3L 6 6 0 80C 14104B 0
AE 81D CF39869B 3EK0PM5F 6 7 0 813 181032 0
AD 811 CF3964D6 3EK0MPTB 6 8 0 80F 1A1019 0
AC 805 CF394AAD 3EK0PER0 6 9 0 7F7 161083 0
9E 824 CF399F00 3EK0Q2GC 5 4 2A DBC F1362 0
9D 830 CF399FEF 3EK0Q618 5 5 0 7FB 17107C 0

9B 841 CF399E14 3EK0Q20K 5 6 0 7FF 170FFC 0

98 835 CF399EA1 3EK0Q50Q 5 7 0 807 141012 0
90 80C CF3949BF 3EK0PF0N 5 9 0 7FD 161083 0
8F 800 CF395E94 3EK0PJM2 5 10 0 807 18106B 0
79 828 CF3986F1 3EK0PXE4 4 4 38 E04 D139B 0
76 84C CFBF40C9 3EK2FTBV 4 5 0 7F6 181016 0
75 840 CF1FECBE 3EK0HYY4 4 6 0 7F6 141035 0
74 834 CF3937B4 3EK0P77F 4 7 0 808 151040 0
73 81C CF399FDD 3EK0PXCN 4 8 0 811 15108F 0
72 810 CF3947DD 3EK0PECK 4 9 0 7FB 181096 0
71 804 CF394AA1 3EK0PFA0 4 10 0 827 171042 0
66 825 CF1FE247 3EKYNH2R 3 4 60 2A4F 1F1D78 0
65 831 CF39379B 3EK0P7BK 3 5 0 80A 14101D 0
63 842 CF29E205 3EK0LYT4 3 6 0 812 16100D 0
5C 836 CF3940DB 3EK0P7M0 3 7 0 806 171063 0
5A 819 CF395E6A 3EK0MFZS 3 8 0 806 181041 0
59 80D CF241C2A 3EK0JA10 3 9 0 81F 170FD9 0
56 801 CF393DA2 3EK0P88L 3 10 0 7FF 160FEA 0
11. What can you conclude from the analysis of the loop 1B error statistics?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
DP-2A pcb 32d964 loop map: 36 devices, reporting group 4077 Link Up Delta
1 20 [Controller] 7 0 2B 1E2 39A 0
33 82C CF241CF2 3EK0J9FH 8 4 0 7 35 0
32 846 CF395359 3EK0PG93 8 5 0 4 F 0
31 83A CF2CAB1B 3EK0NBX8 8 6 0 6 F 0
2E 820 CF241F9A 3EK0J8E6 8 7 0 5 F 0
2D 814 CF2CAE84 3EK0N7WN 8 8 0 2 F 0
2C 808 CF3949B8 3EK0PEW2 8 9 0 4 F 0
4C 82F CF2CACD5 3EK0NAVW 9 4 0 6 2D 0
4B 849 CF393EA1 3EK0P7VQ 9 5 0 3 F 0

4A 83D CF395498 3EK0PG9M 9 6 0 3 F 0

49 823 CF393697 3EK0P6N8 9 7 0 4 F 0
47 817 CF3935A4 3EK0P6VK 9 8 0 6 F 0
46 80B CF399EE5 3EK0Q3BP 9 9 0 2 F 0
66 82E CF393FFA 3EK0PCZF 10 4 0 6 1F 0
65 848 CF39A0DE 3EK0Q642 10 5 0 2 F 0
63 83C CF399FF1 3EK0Q423 10 6 0 3 F 0
5C 822 CF399E57 3EK0Q4KB 10 7 0 4 F 0
5A 816 CF395379 3EK0PG75 10 8 0 5 F 0
59 80A CF39A0E1 3EK0Q637 10 9 0 2 F 0
B3 82A CF39A034 3EK0Q2WP 13 4 0 7 3D 0
B2 844 CF398608 3EK0PXH8 13 5 0 C 27 0
B1 838 CF1FED8B 3EK0J1MZ 13 6 0 3 27 0
AE 81E CF395B6F 3EK0PCQ2 13 7 0 7 27 0
AD 812 CF395002 3EK0NGAQ 13 8 0 4 27 0
AC 806 CF39A0CF 3EK0Q5HC 13 9 0 6 27 0
9E 82B CF399EE9 3EK0Q5H5 12 4 0 6 11 0
9D 845 CF3943E1 3EK0NP18 12 5 0 9 1E 0
9B 839 CF29E110 3EK0LXYK 12 6 0 4 1E 0
90 807 CF3986AD 3EK0PAN5 12 9 0 8 27 0
79 82D CF399F34 3EK0Q4LH 11 4 0 7 11 0
76 847 CF3937B7 3EK0P71V 11 5 0 C 1E 0
75 83B CF39531E 3EK0PG3N 11 6 0 6 1E 0
74 821 CF241FEB 3EK0JAMA 11 7 0 3 1E 0
73 815 CF29E31D 3EK0LMYZ 11 8 0 7 1E 0
72 809 CF39A029 3EK0Q3NV 11 9 0 3 1E 0
................................................................................................................
................................................................................................................
***** DP-2B Loop Order Invalid -- Using ordering from DP-2A

13. What does the first line of the loop 2B output indicate?
................................................................................................................
DP-2B pcb 32d964 loop map: 36 devices, reporting group 4077 Link Failed
Delta
33 0 CF241CF2 3EK0J9FH 8 4 0 2 2D 0
32 0 CF395359 3EK0PG93 8 5 0 3 1E 0
31 0 CF2CAB1B 3EK0NBX8 8 6 0 2 1E 0
2E 0 CF241F9A 3EK0J8E6 8 7 0 3 1F 0
2D 0 CF2CAE84 3EK0N7WN 8 8 0 2 1E 0
2C 0 CF3949B8 3EK0PEW2 8 9 0 5 1E 0
4C 0 CF2CACD5 3EK0NAVW 9 4 0 1 1E 0
4B 0 CF393EA1 3EK0P7VQ 9 5 0 3 1E 0
4A 0 CF395498 3EK0PG9M 9 6 0 1 1E 0
49 0 CF393697 3EK0P6N8 9 7 0 4 1E 0
47 0 CF3935A4 3EK0P6VK 9 8 0 7 16 0
46 0 CF399EE5 3EK0Q3BP 9 9 0 1 1F 0
66 0 CF393FFA 3EK0PCZF 10 4 0 4 21 0
65 0 CF39A0DE 3EK0Q642 10 5 0 2 1E 0
63 0 CF399FF1 3EK0Q423 10 6 0 4 1E 0
5C 0 CF399E57 3EK0Q4KB 10 7 0 1 1E 0
5A 0 CF395379 3EK0PG75 10 8 0 2 21 0
59 0 CF39A0E1 3EK0Q637 10 9 0 1 1E 0
B3 0 CF39A034 3EK0Q2WP 13 4 0 6 22 0
B2 0 CF398608 3EK0PXH8 13 5 0 8 1E 0
B1 0 CF1FED8B 3EK0J1MZ 13 6 0 3 1E 0
AE 0 CF395B6F 3EK0PCQ2 13 7 0 4 1E 0
AD 0 CF395002 3EK0NGAQ 13 8 0 2 1E 0
AC 0 CF39A0CF 3EK0Q5HC 13 9 0 2 1E 0
9E 0 CF399EE9 3EK0Q5H5 12 4 0 4 1E 0
9D 0 CF3943E1 3EK0NP18 12 5 0 6 1E 0
9B 0 CF29E110 3EK0LXYK 12 6 0 2 1E 0
90 0 CF3986AD 3EK0PAN5 12 9 C F2FE 1506F7 0

................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
79 0 CF399F34 3EK0Q4LH 11 4 0 3 1C 0
76 0 CF3937B7 3EK0P71V 11 5 0 7 1E 0
75 0 CF39531E 3EK0PG3N 11 6 0 2 1E 0
74 0 CF241FEB 3EK0JAMA 11 7 0 4 1E 0
73 0 CF29E31D 3EK0LMYZ 11 8 0 7 1E 0
72 0 CF39A029 3EK0Q3NV 11 9 0 3 1E 0
................................................................................................................
................................................................................................................
................................................................................................................
FNBs not on loops - information is stale - oldest first
SIDE WWN SERIAL # ENC BAY BAD LINK_FL LS_SYNC BAD_WORD BAD_CRC
A CF29E2E6 3EK0LN1P 105 108 16 15E6 1 0
B CF29E2E6 3EK0LN1P 105 108 0 1 B 0
A CF399C72 3EK0Q4L6 112 107 0 A 1E 0
B CF399C72 3EK0Q4L6 112 107 0 6 F 0
A CF39A0C3 3EK0PKH6 112 108 0 0 0 0
B CF39A0C3 3EK0PKH6 112 108 0 0 0 0
16. Using the output shown above, determine which three disk drives were removed
from the storage system.
................................................................................................................
................................................................................................................
................................................................................................................

Answers
1. What does the first line of this FCS_LINK_ERRORS output indicate?
Loop 1A was down at the time this output was generated. If Loop 1A is down,
the controllers will obtain the drive error statistics (maintained on each drive for
both loops) for both loops using the 1B loop.
Larger numbers of errors can be generated because there are different drive
types or different drive firmware versions being used. In this specific situation, an
FCS_SHOW_CONFIG output was checked and showed that the drive type
BD07254498 and firmware (3BE3) were the same as every other drive in the
enclosure (so drive type was not the cause).
Because the link synchronization and state are being maintained, outside of
potentially monitoring this drive, no action should be taken.
BAD_WORDs than the other members of enclosure 5? Also, do you notice
anything interesting about the actual number of BAD_WORDs detected?
Again, because the link synchronization and state are being maintained, outside
of potentially monitoring this drive, no action should be taken. What is
interesting is that the number of BAD_WORDS is identical to the number of
BAD_WORDS that were detected by the drive in 2, 7.
Drive 5, 9 is indicating some serious problems. The number of detected link
failures and loss of syncs is much higher than the rest of the drives on the
enclosure and on loop 1A. The cause could be the drive itself, the previous
drive, the A-side I/O module, or in rare cases, the shelf enclosure itself.
Typically, first attempt to replace the drive just before 5, 9, but notice there is no
drive present in location 5, 8. The drive in slot 5, 8 was previously removed in
an attempt to resolve this problem. At this point, the likely candidate for
replacement is the A-side I/O module in shelf number five.
Same answer as the previous two drives indicating a large number of
BAD_WORDs. Again, notice the number of BAD_WORDS detected was
identical to the number of BAD_WORDS detected by the drives in slots 2, 7 and
5, 5.


Same answer as the previous three drives indicating a large number of
BAD_WORDs. Again, notice the number of BAD_WORDS detected was
identical to the number of BAD_WORDS detected by the drives in slots 2, 7, 4,
6 and 5, 5.
It is interesting (though at this time it goes unexplained) that one drive—not the
first or last in the enclosure—in Enclosures 2, 3, 4 and 5 have the exact number
of BAD_WORDs much higher than the other disks on the same loop.
7. What can you conclude, if anything, from the analysis of the loop 1A error
statistics?
The enclosure five A-side I/O module, or disk 5, 9 are logical candidates for
replacement.
Drive 1, 4 does have a higher LINK_FL rate than the other drives in the
enclosure, but it is also the first populated drive in Enclosure 1. It is possible for
the first disk in each enclosure to have higher numbers of LINK_FLs than other
disks in the same enclosure.
Drive 2, 8 is detecting a large number of signal problems. Somewhere between
drive 2, 7 and drive 2, 8 the signal is being corrupted.
Interestingly, the loop that was indicating errors in the logs was loop 1A. Loop
1B should be monitored after loop 1A is fixed. Suspect the B-side I/O module in
enclosure 2, drive 2, 7 or drive 2, 8.
Drive 6, 4 does have a higher LINK_FL rate than the other drives in the
enclosure, but it is also the first populated drive in Enclosure 6. It is not
uncommon for the first disk in each enclosure to have higher numbers of
LINK_FLs than other disks in the same enclosure.
Loop 1B shows all of the first populated disks in each enclosure having higher
LINK_FLs than the other disks in each enclosure – this is not uncommon.
In addition, drive 2, 8 is indicating major signal problems, which will likely not
go away once the loop 1A issues have been resolved. The B-side I/O module in
enclosure 2 should be considered for replacement (were it not a Bay 8 disk
device, the disk in Bay 7 would be the first candidate for replacement).

Loop 2A is functional and is not indicating any problems at this time.
13. What does the first line of the loop 2B output indicate?
Loop 2B was down at the time this output was generated. If Loop 2B is down,
the controllers will obtain the drive error statistics (maintained on each drive for
both loops) for both loops using the 2A loop.
Drive 12, 9 is indicating some major signaling problems. The number of Loss of
Syncs is orders of magnitude higher than the rest of the drives on the Enclosure
and on the loop.
Notice there is no drive present in locations 12, 7 and 12, 8. Both of these
drives were previously removed in an attempt to resolve high LS_SYNC problems
with drive 12, 8. At this point, the B-side I/O module in enclosure 12 should be
replaced.
The B-side I/O module in enclosure 12 should be replaced.
16. Using the previous output, determine which three disk drives were removed from
the storage system.
The drives in the following bays: 5, 8, 12, 7, and 12, 8.

As an additional exercise, generate outputs from different controllers and compare

them. Note what you see here.
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................
.......................................................................................................................


Module 8 Lab

 Use the HP StorageWorks Navigator user interface to:
 Hide and move columns
 Filter data by entering values per column
 Drag and drop columns for grouping
 Sort by column
 Use tool tips to display contents of the column the cursor is over
 Group data by its attributes
 Filter data
 Use the Navigator menus to:
 Open, compress, and close workspaces
 Import and export files
 Save and load GUI layouts
 Locate specific data
 View data by groups, in an inverted format, in multiple lines, or in a
zoomed display
 Load configuration data
 Use the Navigator tabs to view workspace, controller events, controller
termination events, reports, graphed reports, and analysis options
 View configuration information by using the hierarchical, hardware, and
properties displays
Rev. 9.21 HP Restricted L8 – 1


 An EVA storage system
 Access to the latest version of Navigator software
This lab allows you to explore many of the features of Navigator, including viewing
configuration and event data in various formats.
array.
L8 – 2 HP Restricted Rev. 9.21

Using HP Navigator
In this section, you can explore how to use all aspects of the Navigator GUI,
including how to use the general GUI functions, the menus, and the tabs.
Exploring the user interface

To explore functional aspects of the user interface:
1. Start Navigator by using the Start menu or double-clicking the icon or shortcut
on the desktop.
2. Select File  Open  Workspace to open an existing workspace (.nw). The
workspace and its current contents are displayed.
The workspace displays all items in the currently open workspace, including any
available report data, event data, termination data, and configuration data.

3. Note the following components of the display:

a. Title bar showing current workspace, event table, and World Wide ID
b. Menu bar
c. Tabs
d. Column headings
e. Row for filtering data (this is the blank row below the column headings)
f. Row and column entries for the current available objects in the workspace
(for example, reports that are available)
g. Section for displaying text about the current field that has focus (at bottom of
display)
h. Status bar at the bottom
4. Place your cursor over parts of the display to view the tool tips for those items.
5. Size columns and rows by dragging on the gridlines where they are active.
6. Show all columns by right-clicking in the column heading and selecting Unhide
All Columns from the menu.
7. Hide any columns by clicking a column heading, right-clicking in the column
heading, and selecting Hide Columns from the menu.
Note
On your keyboard, use the Ctrl key to select multiple columns that are not
contiguous and use the Shift key to select a range of contiguous columns.
8. Click any column heading and drag it to a new location.
Note
This can be helpful for grouping portions of the display.
9. Click any column heading and note the small sort descriptor (small triangle) as
you toggle between ascending and descending order.
10. In the blank row used for filtering, enter a letter or a number in a column or
multiple columns to display a filtered list. Delete the value to return to the full list.

Using HP Navigator
Reviewing the menu options

When you start Navigator and open a workspace, the GUI automatically comes up
showing all of the contents of your workspace (Workspace tab active). Use the GUI
to review or use all of the following menu options:
 File
 New — Opens a new workspace
 Open — Opens an existing workspace
 Close — Closes the open workspace
 Import — Imports an entire or subset of a controller event log or a
termination event log file. It also allows import of configuration data files
that come directly from the serial port of the controller (the Configuration
05/29 option), from the ScanMaster (HP Configuration Collector) tool, from
the SSSU, or from the EVA 4400/6400/8400 drive enclosure shelf.
 Export — Exports an event log, report, or analysis to a file in Excel format,
comma-separated value (CSV) format, or binary (termination events)
 Merge — Merges or unmerges an event table
 Workspace — Saves a workspace in compressed format (.nwc) or in normal
format with a new name
1) Save your workspace in compressed format.
2) Verify that the file exists.
 Layout — Saves or loads a user-defined (.nlo) or default GUI layout
1) Rearrange your display and hide some columns, then save the layout.
2) Unhide all columns.
3) Open the saved layout.
 Exit — Exits the Navigator application

 Edit
 Delete (only in Workspace view) — Deletes the row where your cursor is
located
 Find — Find where specified data is located
1) Open an event table.
2) Select Edit  Find, and when the dialog opens, enter a search string
and select Find All.
Note
You can restrict your search to a specific column by using the drop-down list on
the right.
3) Select Find Next and Find Prev to scroll through the search list
simultaneously with the event table.
 View
 Normal — Displays the default view of the data
 By Group — Groups data by its attributes (columns)
1) Select View  By Group, and drag a column heading to group by that
column.
Note
A fuller impact of this option is obvious when you open an event log (through the
Events tab, see below) and view the data by group.
2) Click the plus “+” symbol to expand the group.

3) Select View  Normal, to return to an ungrouped display.
 Inverted — Inverts the rows and columns of data
 Multiple Lines — Displays data on multiple lines
 Zoom — Displays data in larger or smaller format
Note
Return to normal view and zoom to your customary level.
 Event Code — Displays data by event code

Using HP Navigator
 Tools
 Configuration Data — Creates a separate window for viewing the
configuration data of any storage array
Note
This option is described later in detail.
 Compare Configuration — Allows you to compare two sets of configuration

data
 Communications Terminal — For EVA6x00 or EVA8x00 storage systems,
allows connection to the Fiber Channel loop switch or EMU
 CV Log Viewer — Allows you to open and view a Command View EVA log,
such as a trace or SCMI log.
 Compact Workspace — Performs an internal defragmentation of the
workspace to reduce its size if possible. This option closes the workspace,
performs its work, and then reopens the workspace.
 Environment Manager — Allows you to share your reports and custom filters
between your workspaces
 Zip — Opens the Navigator Zip window used to access data in ZIP files.
You can open and close ZIP files, and add files to, extract files from, and
delete files from ZIP files.
 Change Parse Files list — Allows you to insert additional parse files into the
environment if Navigator does not provide all you need
 Advisory Editor — Allows you to customize your Navigator environment by
specifying additional information to accompany events with a particular
event code
 Options — Allows you to select Navigator options for default and new
workspaces
 Help
 Navigator Help — Displays full help file for Navigator
 Check for updates — Checks the Internet to see if there are new updates for
Navigator
 About Navigator — Displays version information, gives a pointer to the
Navigator website, provides a pointer to the QuiX-PCT problem reporting
website, and a link to the TechBB website.
 End User License Agreement — Displays the HP license agreement.

Reviewing the tab options

When you start Navigator and open a workspace, the GUI automatically displays
the Workspace tab active. Use the GUI to review or use all of the following tab
options:
 Workspace — Displays all items in the currently open workspace, including
event data, termination data, and configuration data
a. Review the contents of your workspace.
b. Double-click an event file, configuration file, or termination file to load that
file in separate window.
 Events — Shows a listing of events in the file that you have imported, highlighted
by triggering rules provided by a rules engine.
a. Review the contents of your event file.
b. Use the Row options to go to the next row, the end of the list, to the previous
row, or back to the beginning of the list.
c. Filter event code data by highlighting a row, right clicking in the row, and
selecting Display [Event Code] NOT LIKE ‘...’ from the right-click menu.
d. Using the same right-click menu, turn off all filters.
e. Select View  By Group, and drag the Controller column, then the Severity
column to display a list by controller severity.
f. Use the right mouse button to collapse and expand rows.
g. Use the plus “+” and minus “-” icons to roll the events into severity
categories.
h. Select View  By Group, and this time show a grouping of events by
severity level and event code.
i. Collapse the rows to get a quick summary.
 Termination — Displays the selected termination data
a. Download and then import a termination file from your storage system.
b. Click the Termination tab to display the contents of the file.
c. View the listing by severity level and description. Collapse and expand
rows, and roll events into severity categories.
d. Select the Workspace tab and verify that the termination data file is in your
workspace.

Using HP Navigator
 Reports — Displays a list of available reports. The reports use the event data
loaded through the Events tab. User-defined reports and analysis are supported.
a. Open an event table.
b. Select any of the available reports.
Note
For each report, make note of the description of the report at the bottom of the
page.
c. Sort the data for any report by clicking the column headers.
d. Drag the columns of the report to different locations.
 Graphed Reports — Displays a list of available graphed reports based on the
data currently shown through the Reports tab.
a. Create a graphed report for Port Stats.
b. Review and use some of the toolbar options by placing your cursor over a
button, looking at the tooltips, and clicking the button.
c. Right-click anywhere within the graph and choose Data Editor. Navigator
displays a table showing the graph’s data.
1) Position the cursor over data in the table or over the graph to highlight
the related table or graph information.
2) Use the up/down or left/right scrollbar to view more data.
d. Review the data area below the graph by placing your cursor over portions
of the graph.
 Analysis — Allows selection of a type of analysis of the event table.
a. Open an event table.
b. Using the drop-down list, select each type of analysis and review the results.
Important
! The analysis information allows you to locate start and finish sequences in the
event log.
c. Locate a start and finish sequence in the event table.

d. Get an analysis summary.

When you use Navigator, you can view the configuration information from any active
storage system as long as you have some basic information about that storage
system. The subtopics describe how to obtain configuration data using Navigator.
Accessing configuration data

Use the following steps to view configuration information gathered by Navigator:
1. If you are not already running Navigator, start it by using the Start menu or
double-clicking the icon or shortcut on the desktop.
2. Select Tools  Configuration Data.
3. Under the Get Current Config button, provide the following information
 Host Name/IP — System name or TCP/IP address of the managing server
 User Name — User name used to access the managing server (default is
administrator)
 User Password — User password used to access the managing server
Note
Your instructor will provide any missing information.
4. Press the Get Current Config button.
Important
! If you receive an EMClientAPITNG error indicating an invalid username or
password, your management server may have been loaded with SmartStart V7.2
or above.
From a command prompt, navigate to c:\Program Files\Hewlett-
Packard\SANworks\Element Manager for StorageWorks HSV\Bin\, and enter
elmsetup.exe -pA:administrator –f where administrator equals the
password for the HP Command View API Administrator account. Any password
can be substituted for administrator.
Go to the Services window and restart the HP Command View EVA service. The
new password takes effect only after you restart this service.
A dialog box appears that contains a list of storage arrays that the server knows
about. The arrays managed by the server appear with a normal background,
while those arrays managed by other servers appear with a gray background.
5. Choose an array by either double-clicking the entry or clicking the entry and
then pressing Get.

Using HP Navigator
Using the configuration display

When you load a configuration, either by double-clicking an existing configuration
file in your workspace, loading an old configuration, or getting a current
configuration, you will see a display similar to the following.
Using different views

Configuration data is displayed in any of three views. The title bar of the window
shows the date and time when the configuration was obtained. Take time to become
familiar with each of the following views:
 Hierarchy
The Hierarchy view contains a hierarchical list of all the objects in the selected
storage array.
 Hardware
The Hardware view contains a graphical representation of the disks and
controllers on each shelf.
 Properties
The Properties view contains a list of all the properties for whichever object is
currently selected in either the Hierarchy view or the Hardware view. Each
property consists of a property name equated to a property value. The list is
alphabetized.

Work with the different views by doing the following:

1. In the Hierarchy view, drill down into the various folders until you can click a
disk drive or a drive enclosure bay.
2. Note the changes in the Hardware and Properties displays as you click different
items.
3. In the Hardware view, click each choice in the drop-down list (ALPA, Disk
Number, and so on).
4. Click any item in the Hardware view to display its properties.
5. Examine the RSS numbers and determine if their layout is consistent with what
you expect from your configuration.
6. Determine the loop positions for selected drives.
Using other features

The left-most column offers some other uses. Perform the following:
1. Click the Warnings button to see any current warnings.
2. In the drop-down list for Find Disk ALPA, select an ALPA and note the changes
on the display.
3. In the drop-down list for Find Full/Partial Disk WWID, select a partial WWID
and note the changes on the display.
4. In the “Select by” drop-down list, select a disk drive model number and view the
changes on the display.
5. Use the “Select by” drop-down list to locate drives with firmware versions or RSS
index numbers.
Saving current configurations

When you have obtained a configuration you want to save, save it to your
workspace by pressing the Save Current Configuration button.
The current configuration is stored using the World Wide Identifier (WWID) of the
array and the exact time when it was obtained. You can retrieve it at any time by
pressing the Load Old Config button and selecting from the list or by clicking the
configuration displayed under the Workspace tab.

Module 9 — Lab 1
After completing this lab, you should be able to relate Controller Event Log entries to
many common and uncommon EVA storage system events.
 Access to an EVA5000 storage system with two HSV110 controllers

 A minimum of 25 disk drives
 Documents related to this exercise that are stored on the course DVD
 Allow approximately 4 hours to complete this lab
This lab allows you to generate and then correlate Controller Event Log entries. As
you go through the lab, remember to:
Note
If you are doing the lab remotely, you will not be generating event logs.
array.
Carefully read and follow all of the steps in this lab. You will be performing many
actions and be required to note when those actions took place. It is important that
your lab group works as a team and does not rush while performing all of the lab
activities that follow.
Note
If you are doing the lab remotely, you will not be performing the actions to
generate event logs.

Important
! If you are doing this lab remotely, you will not be performing the actions to
generate the event logs. Those logs will be supplied for you on the course DVD.
Read through this section but do not perform any actions. Continue with the
section called Capturing and correlating the event log entries, where you will
later be directed to come back to do the correlation exercise in this section
(under the topic called Actions to generate and then correlate).
This lab steps you through the process of creating, modifying, and deleting many
different object types. In addition, you will emulate a small number of storage system
component failures.
Important
! It is critical that you do not deviate from this lab in any way until all the requested
lab actions have been completed.
The following items are a general outline of how this lab is designed to work:
 Perform each of the actions listed in the section called Actions to generate and
then correlate one at a time and record the exact time you performed the action.
 Once all actions are completed, capture the Controller Event Log, and then
translate it using Navigator.
 Go back through each of the actions listed in the lab section called Actions to
generate and then correlate. This time, using the timestamps you wrote down
when first performing the actions, correlate each of the Controller Event Log
entries with the action they corresponded to.
Effectively, you will be going through the following lab section twice. The first time, to
generate an action and record the exact time that action was performed, and the
second time, to correlate each event log entry with the action taken.
Important
! While completing the lab actions, it is possible that accidental actions will be
performed—actions that the lab did not specifically request you to perform.
If you perform an accidental action, record this action along with the time it was
performed in the section called Accidental actions performed while completing
this lab at the end of this lab.

Generating and Correlating Events
Lab preparation and setup

To prepare for the lab:
1. Uninitialize your storage system.
2. Go to the management server and set its date and time to the current date and
time.
Note
All lab participants should consider synchronizing their watches to the time set
on the management server and make sure that time matches that of the server
you are browsing from.
3. Using the Command View EVA GUI on the uninitialized storage system, select
Use management server date/time and Re-sync controller time with the SAN
management time, then click Save changes.
Note
Having an accurate time setting is critical to this lab. Failure to perform this step
could make the analyzing of your event log entries later more difficult.
Note
It is highly recommended that you always select Re-sync controller time with the
SAN management time. This enables you to correlate the Controller Event Log
entry times with the Management Server Event Log entry times, as well as
generate periodic events to determine if the storage system is still logging events.

Actions to generate and then correlate

1. Initialize your storage system and create the Default Disk Group with 10
members and a disk failure protection level of Single.
Note
Consider recording the Time Action Started or Time Action Completed entries
below in 24-hour time format (for example, 15:21:03 instead of 3:21:03 PM).
This is the time format utilized by the translation tools.
Time action started: ……………Time action completed:……………….…
Note
If this is your first time performing Step 1 in this lab section, go to Step 2. The
space provided below with headings titled Controller Event Code or Event time:
is used when you return to this step to correlate the Controller Event Log event
entries with the actions.
Note
As an alternative to documenting the event information in the format provided
below, you can insert a new column into your translated Excel spreadsheet
outputs and place your comments there.
Controller event code Time Description
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................

2. Create a second disk group with 14 members and a disk failure protection level
of Single.
Time action started: ……………Time action completed:……………….…
Note
Throughout this lab, for multiple events that have the same event number, only list
the event number once. For example, event 0x096c000f will be listed 14 times
for the disk group creation action just performed. Do not write down all 14 times
this event entry was logged.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................
3. Delete the 14-member disk group created in the previous step.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................

4. Create another disk group with 14 members and a disk failure protection level of
Single.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
5. Add another member to the 14-member disk group making it a 15-member disk
group.
Time action performed: ………………
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................

6. Ungroup one member from the 15-member disk group—when prompted, select
the Ungroup and wait option.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
7. Change the Disk failure protection  Requested level: from Single to Double on
the 14-member disk group, then click Save changes. When the action is
complete, use the GUI to ensure the Actual level changed to Double.
................................................................................................................
Notes:
................................................................................................................
................................................................................................................

8. Change the Occupancy  Alarm level for the 14-member disk group to 90%,
then click Save changes.
................................................................................................................
Note
This is the first event that will have no Controller Event Log entries. You will see an
event code of #2052 in the Management Server Event Log for this action.
9. Sync up the controller time with the management server time using System
options  Set time options  Re-sync controller time with the SAN
management time.
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
10. Add a Windows host to the system.
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................

11. Add an HP-UX host to the system.
Note
Ensure you set the correct Host OS type for the operating system you are using.
................................................................................................................
................................................................................................................
12. Delete the Windows host entry from the system.
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
13. Add the Windows host back into the system.
................................................................................................................
................................................................................................................
14. Add a second HBA port to the Windows host.
................................................................................................................
Notes:
................................................................................................................
................................................................................................................

15. Create a 5GB VRAID5 virtual disk from the Default Disk Group, prefer it to the
“A” controller, and do not present it to any hosts.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
“A” controller, and do not present it to any hosts.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................

“B” controller, and do not present it to any hosts.
18. Change the requested capacity for the VRAID0 virtual disk from 1GB to 2GB.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
19. Delete the 5GB VRAID5 virtual disk.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
20. Present the 2GB VRAID0 virtual disk to the Windows host.
................................................................................................................
Notes:
................................................................................................................

21. Unpresent the 2GB VRAID0 virtual disk from the Windows host.
................................................................................................................
Notes:
................................................................................................................
22. Create a Demand-allocated snapshot of the 2GB VRAID0 virtual disk, but do not
present it to any hosts.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................
23. Delete the Demand-allocated snapshot.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................

24. Restart the slave controller using the slave controller OCP: Shutdown Options 
Restart  Yes. Pause for 3 or 4 minutes before proceeding to the next lab step.
Note
It is not necessary to list the entire series of events associated with the controller
reboot. Space is provided below to capture the majority of them.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................

25. Having paused for 3 or 4 minutes since the last lab action, shut down your slave
controller followed by your master controller using the Shut down  Shutdown
Options  Power down GUI option from the “Controller Shutdown” section of
the Shutdown Options page. When both controllers are completely shut down,
switch the On/Off switches on the back of the controllers to the Off position.
Note
It is not necessary to list the entire series of events associated with the system
shutdown. Space is provided below to capture many of them.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................

26. Switch the On/Off switches of both controllers to the On position and take
about a 5-minute break while your controllers boot and re-establish
communication with the management server.
Note
The entire series of events associated with the system startup is much more than
will fit on this page. Space is provided below to capture the events that you find
most interesting.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................

Caution
For the rest of the lab, pay extra attention when performing all actions and when
noting the times that those actions took place.
27. Use the Locate function to locate all of the disks in the Default Disk Group and
observe the disk positions. Cancel the Locate, and then physically remove one of
the Default Disk Group members you just located.
Note
When recording the time for this event, record the time the disk was physically
removed from the system. You are emulating a member failure of the VRAID0
virtual disk.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................

28. Within a small time period after you physically remove the disk in the previous
step, the VCS code pops up a window asking if you would like to Continue with
no changes or Start deletion process. For this lab exercise, select Start deletion
process and record the time when you select this option below.
Note
Because of the selection you just made, the VRAID0 virtual disk information is
irretrievably lost. Never select this option unless all other attempts to resolve your
problem have been attempted. Often, one or more loop and/or disk issues can
be resolved allowing the virtual disks to resume normal operations.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................

29. Physically reinsert the disk that was pulled out previously. Wait 1 or 2 minutes
for the system to bring the disk online, then check the GUI to ensure it is ready
for use.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
30. Regroup the reinserted disk into the Default Disk Group.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
“B” controller, and do not present it to any hosts.
................................................................................................................
Notes:
................................................................................................................
................................................................................................................

32. Physically remove the disk device that was just added and grouped into the
Default Disk Group.
Note
This is the same disk that was removed earlier from the storage system.
Note
When recording the time for this event, record the time the disk was physically
removed from the system. You are once again emulating a member failure of the
VRAID0 virtual disk.
Note
It is not necessary to list the entire series of events associated with the disk
removal. You may want to indicate only those events (if there were any) that were
different from the last time you performed the disk removal.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................

33. Within a short time period after physically removing the disk in the previous
step, the controllers pop up a window asking if you would like to Continue with
no changes or Start deletion process. For this lab exercise, select Continue with
no changes and record the time when this option was selected below.
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................

34. Physically reinsert the disk that was pulled out previously. Wait a minute for the
system to bring the disk online, and check the GUI to ensure it is ready for use.
How many disks are seen in the Default Disk Group?
................................................................................................................
What is the status of the VRAID0 virtual disk?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
What did you learn about the difference between “Continue with no changes” and
“Start deletion process?”
................................................................................................................
................................................................................................................
Did what you learn apply only to VRAID0 virtual disks, or does it also apply to
VRAID1 and VRAID5 virtual disks?
................................................................................................................
................................................................................................................

35. Physically remove one of the EMUs from a drive enclosure shelf and wait 2
minutes.
Note
The drive enclosure shelf should now show up in the Unmappable Hardware
folder in the GUI navigation pane.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................
36. After waiting for 2 minutes, physically reinsert the EMU that was removed in the
previous lab step.
Note
It may take several minutes for the Command View EVA GUI to rediscover the
reinserted EMU.
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................

37. Physically remove a Fibre Channel cable from one of the I/O modules and wait
2 minutes.
Time action performed: ……………… .........Cable removed:..……….…..………
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................

38. After waiting for 2 minutes, reconnect the Fibre Channel cable to the I/O
module.
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Notes:
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................

To capture and correlate the event log entries, perform the following:
1. Capture the Controller Event log from the storage system.
Important
! If you are doing this lab remotely, you do not need to capture these logs, they
will be on the course DVD. There will be two: one named
Controller_Event_Log_Pod30_GetEventFileSC.5767168 and one named
Controller_Event_Log_Pod30_tmpScEventFile000001.idi.
2. Translate the Controller Event Log using your translation tool of choice.
If you are doing the lab remotely, choose one of the following files to translate:
a. Translate the Controller_Event_Log_Pod30_tmpScEventFile000001.idi log
beginning with a date of March 18, 2009, at 16:06:30, and ending at
18:37:59.
b. Translate the Controller_Event_Log_Pod30_GetEventFileSC.5767168 log
beginning with a date of May 1, 2004, at 17:37:18, and ending at
18:22:59.
Note
If you are doing this lab locally, note that although the storage system was
uninitialized when you started this lab, the management server still has older
events stored that are not related to your lab activities. Therefore, when
translating your Controller Event Log, only translate those events starting with the
timestamp associated with your initializing the storage system.
3. Open an Excel spreadsheet output of your events and sort them all by
Date/Time.
Note
Any other additional formats of the Controller Event Log output can be opened
and used as well.

4. On your course DVD, in a directory called Labs and extra related lab materials,
there are a few documents related to this exercise. Locate the following
documents:
 <NAPP or EVE>_Pod30_main.xls — This is an Excel spreadsheet example
output of the above actions performed on a storage system. The entire
output (nearly 500 events) is fully documented with information regarding
the meaning of many of the events. These comments have been added to
the first column of the spreadsheet. Many items of particular interest are in
bold font. Open this spreadsheet and have it available for viewing before
you continue your lab. Do not actually refer to it unless you need to refer to
it during the correlation exercise.
Important
! The use of this commented output is a mandatory part of the lab. Compare your
results to the results of the output from the <NAPP or EVE>_Pod30_main.xls
spreadsheet.
 Controller_Event_Log_Pod30_GetEventFileSC.5767168 — This is the

Controller Event Log that was utilized to generate the example <NAPP or
EVE>_Pod30_main.xls. It is provided so that you can generate other output
formats from it, if desired.
 Controller_Event_Log_Pod30_tmpScEventFile00001.idi — For remote lab
activities, this is the Controller Event Log that was generated on a different
storage system and allows you to correlate the original results with the
results from a different storage system. It is also provided so that you can
generate other output formats from it, if desired.
 Management_Agent_Events_Log_Pod30_GetEventFile.5767168 — This is the
Management Server Event Log associated with the completed example of
this lab.
5. Return to the previous lab section called Actions to generate and then correlate,
and correlate each action performed with the events generated by the storage
system. While using the uncommented Excel spreadsheet to perform the
correlation, use the example <NAPP or EVE>_Pod30_main.xls file as a
reference.
Note
There was an action or two that did not generate any Controller Event Log
entries. For those actions, use the Management Server Event Log to find an
associated event for the action.

Important
! You do not need to complete this section if you are doing the lab remotely.

Accidental action performed: .......................................................................
................................................................................................................
................................................................................................................
................................................................................................................

................................................................................................................
................................................................................................................
................................................................................................................

................................................................................................................
................................................................................................................
................................................................................................................


Module 9 — Lab 2

 Troubleshoot multiple types of EVA storage system problems by analyzing
controller event logs.
 Analyze controller event logs utilizing outputs translated by Navigator and
NAPP.
Allow as much time as possible to thoroughly analyze each presented case study.
A total of 13 case studies are presented in this lab. Your instructor will choose two or
three of these to work through with you as classroom exercises.
For case studies 1, 2, 5, 6, and 7:
 Use Navigator to translate the binary files on your student CD.
 Thoroughly analyze the translated output.
 Answer the questions in your lab guide for each case study. Check the answers
as directed by the instructor. He will provide instructions for finding the answers
to the questions.

For case studies 3 and 4:

 Use any translated output type (Excel spreadsheet or eventcode.txt) to analyze
the case studies.
 Answer the questions in the lab guide.
 Check the answers in Column A of the NAPP_csX_main_commented.xls file
using the procedure provided by the instructor.
For case studies 8 through 14:
 These labs will be free form, that is, there are no questions included as part of
the case study.
 Students should use their personal experience and information gained through
case studies 1–7 to analyze the data provided to determine the cause of the
problems described.

Analyzing Case Studies
Customer scenario
A customer had the following problem:
 Device-side loops were failing.
The problem was first noticed at this time and date:
 As soon as the storage system was booted on 4/16/04 at 19:33:00
The following customer and service actions were taken:
 None
The problem is now:
 Not fixed
Configuration information
The configuration is the following:
 This is a 2C/12D configuration
 No loop switches are installed
 This system was just initialized
 One disk group containing all installed disks was created
 Two virtual disks were created (one preferred to each controller)
 Heavy I/O was being issued from a single Windows server

Questions
1. Sequence number 93 — Why did the leveling of capacity in a Disk Group start?
................................................................................................................
................................................................................................................
2. Sequence number 99 — Is ALPA 0xEF a valid device ALPA for a single rack EVA
configuration? If not, how can you explain an N_Port login failure to it?
................................................................................................................
................................................................................................................
3. Sequence number 103 — Which I/O module transceiver is not detecting any
laser signal?
................................................................................................................
4. Sequence number 119 — How many directed LIPs were issued by the 4252
controller to all devices on loop 1B prior to the 4252 controller loop 1B failure?
................................................................................................................
5. Sequence number 193 — If the 4252 controller loop 1B port is currently in a
Failed state, how is it possible that it is still reporting loop receiver errors?
................................................................................................................
................................................................................................................
................................................................................................................
6. Sequence number 195 — Within the last minute, how many loop receiver errors
were detected by the 4247 controller from ALPA 6A?
................................................................................................................
7. Sequence number 209 — What is the likely cause of this status change for an
enclosure?
................................................................................................................
................................................................................................................
8. Sequence number 217 — What does this event indicate is about to happen?
................................................................................................................
................................................................................................................

9. Sequence number 316 — How long has it been since the indicated transceiver
last lost its laser signal?
................................................................................................................
................................................................................................................
10. Sequence number 409 — Why is a new loop pair 1 device map being
generated?
................................................................................................................
................................................................................................................
11. Sequence number 412 — Is this event in any way related to the loop errors that
are being reported on loop 1B? Also, based on this reported error, should this
disk be replaced?
................................................................................................................
................................................................................................................
12. Sequence number 418 — How much time has passed since the system was
initially booted?
................................................................................................................
13. Sequence number 419 — What is your analysis of the customer loop problems?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................

Customer scenario
A customer reported a storage system with device-side loops that are failing. This is
the same system that was used for case study #1. The following service actions were
taken:
 The disk in 1/1 was ungrouped and removed from the storage system on
4/23/04 at 18:04:00
 The B-side I/O module in Enc 3 was replaced on 4/23/04 at 18:08:00
 A few minutes later, the loop 1B device ports were restarted from the CV EVA
GUI
 The time was set from the CV EVA GUI at 18:26:00
 No other service actions were taken
The problem is now:
 Still not fixed; in fact, things appear to have gotten worse
 No loop switches are installed
 One disk group containing all installed disks was created
 Two virtual disks have been created (one preferred to each controller)
 Heavy I/O was being issued from a single Windows server

Questions
1. Sequence number 10557 — Why was this directed LIP issued to all devices on
loop 1B?
................................................................................................................
................................................................................................................
2. Sequence number 10643 — To which device did the 4252 controller issue the
directed LIP?
................................................................................................................
3. Sequence number 10656 — How much time passes between this event and the
next event?
................................................................................................................
4. Sequence number 10657 — The 4252 controller is detecting loop receiver errors
from which devices?
................................................................................................................
................................................................................................................
5. Sequence number 10674 and 10675 — Which transceivers have detected a loss
of laser condition? Are either of these transceivers ones that were reporting this
condition in case study #1?
................................................................................................................
................................................................................................................
6. Sequence number 10676 — What does this enclosure status change indicate?
................................................................................................................
................................................................................................................
7. Sequence number 10687 — What is the likely cause of this directed LIP to all
loop 1A devices?
................................................................................................................
................................................................................................................
8. Sequence number 10687 — How long has it been since the I/O module in
enclosure 3 was replaced?
................................................................................................................

9. Sequence number 10774 — What is the likely reason this non-data exchange
has timed out?
................................................................................................................
................................................................................................................
................................................................................................................
10. Sequence number 10853 — Which enclosures have all of their disks transition
to the SPOF state? Can you determine which controller (or both) is having
problems communicating on both loops to the indicated SPOF disks?
................................................................................................................
................................................................................................................
................................................................................................................
11. Sequence number 11008 — Why is the 4252 controller issuing a directed LIP to
the disk in enclosure 1 Bay 2?
................................................................................................................
12. Sequence number 11011 — Which sequence of events led up to the volume
transitioning to the MISSING state?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
13. Sequence number 11036 — Are all virtual disks from the indicated disk group
now inoperative?
................................................................................................................
14. Sequence number 11064 — How long has it been since the volume went
missing?
................................................................................................................

15. Sequence number 11107 — Which sequence of events led up to the disk drive
disappearing?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
16. Sequence number 11132 — What does this event indicate?
................................................................................................................
................................................................................................................
17. Sequence number 11134 — Is it normal for a device in a reconstructing state to
transition to a reverting state?
................................................................................................................
................................................................................................................
................................................................................................................
18. Sequence number 11162 — Why is a new device map generated here?
................................................................................................................
................................................................................................................
................................................................................................................
19. Sequence number 11169 — Why does the indicated leveling event take place?
................................................................................................................
................................................................................................................
20. Sequence number 11203 — Which service actions should be taken on the disk
in enclosure 1 Bay 2?
................................................................................................................
21. Sequence number 11714 — Why was there no leveling start event for this
leveling finished event?
................................................................................................................
................................................................................................................
................................................................................................................

22. Sequence number 11904 — How many hours have passed since the last
indication of any problems?
................................................................................................................
23. Sequence number 12210 — How many attempts were made to reconstruct the
data from this volume before the reconstruction was finally successful?
................................................................................................................
24. Sequence number 12212 — What is the likely cause of the indicated split
occurring?
................................................................................................................
................................................................................................................
25. Sequence number 12222 — At this point, which service actions should be
taken?
................................................................................................................
................................................................................................................
................................................................................................................

Customer scenario
 A disk device failed.
The problem was first noticed:
 On 2/9/04
 The failed disk device was removed and replaced with a new disk device on
2/9/04 at approximately 14:36:00.
The problem is now:
 Resolved. However, the customer would like to know what caused the disk to
fail.
Not relevant for this case study.

Questions
................................................................................................................
................................................................................................................
................................................................................................................
2. Sequence number 1311 — Based on this first indicated check condition, should
this drive be replaced?
................................................................................................................
................................................................................................................
................................................................................................................
3. Sequence number 1313 — How many days have passed since the last “retry
count exhausted” event?
................................................................................................................
4. Sequence number 1328 — Should the drive in 3/9 be replaced?
................................................................................................................
................................................................................................................
5. Sequence number 1338 — How many exchange time-outs to the drive in 3/9
took place prior to this event being logged?
................................................................................................................
6. Sequence number 1339 — Which loop remains good for access to the drive in
3/9?
................................................................................................................
7. Sequence number 1341 — How long after the drive in 3/9 went SPOF did it
return to a Normal status?
................................................................................................................
................................................................................................................
8. Sequence number 1334 — Is this the first error logged by the 11d5 controller?
................................................................................................................

9. Sequence number 1345 — How many hours have passed since the last check
condition error? Which loop is this check condition error being reported on?
................................................................................................................
................................................................................................................
10. Sequence number 1347 — Because the error is reported on loop 1B, does this
indicate the problem is likely related to the FC loops or the disk drive itself?
................................................................................................................
11. Sequence number 1351 — Which type of error is logged against the drive in
3/9 and how many of these are allowed before a drive should be replaced?
................................................................................................................
................................................................................................................
12. Sequence number 1397 — The drive is rendered inoperable. When this event
takes place, will the controllers attempt to migrate the data off of the disk?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
14. Sequence number 1411 — Does this event indicate the drive in 3/9 is now
ready for normal data access?
................................................................................................................
................................................................................................................
15. Sequence number 1423 — What was the current state of the indicated volume
prior to it transitioning to the reconstructing state?
................................................................................................................
16. Sequence number 1426 — What is the likely cause of the drive map being
updated?
................................................................................................................
................................................................................................................

17. Sequence number 1427 — Which mistake was just made (indicated by the
action that just took place)?
................................................................................................................
................................................................................................................
18. Sequence numbers 1429/1430 — What do these two events indicate?
................................................................................................................
................................................................................................................
19. Sequence number 1435 — Is this a normal event, or something to be concerned
about?
................................................................................................................
................................................................................................................
20. Sequence number 1436 — How long did the reconstruction of the MISSING
volume take?
................................................................................................................
................................................................................................................
21. Sequence number 1438 — What is the likely cause for the split occurring?
................................................................................................................
................................................................................................................
22. Sequence number 1439 — The leveling event starts as a result of the completion
of which action?
................................................................................................................
................................................................................................................
23. Sequence number 1445 — Why did the leveling event take place?
................................................................................................................
................................................................................................................
24. Sequence number 1447 — Is this an anticipated event given that the old disk in
3/9 was physically removed from the storage system?
................................................................................................................
................................................................................................................
25. Sequence number 1465 — Which events over the last few days are missing?
Why are these events missing?
................................................................................................................
................................................................................................................

Customer scenario
A customer has the following problem:
 They noticed A-side loop problems and other A-side storage system anomalies.
 On 10/08/03
 None
The problem is now:
 Not resolved.
 Loop switches are installed

Questions
1. Sequence number 28 — What is the reason for the controller resync taking
place?
................................................................................................................
................................................................................................................
2. Sequence number 402 —This is the 2nd boot of the 11d5 controller. Why is this
the first time there were any FP1 or FP2 errors reported? That is, why were there
no errors on FP1 or FP2 during the first boot of the controllers?
................................................................................................................
................................................................................................................
................................................................................................................
3. Sequence number 58 — Are any events ever logged to indicate the transition of
fabric ports FP1 and FP2 from the Failed to the Normal state?
................................................................................................................
................................................................................................................
................................................................................................................
Note
In later VCS versions, the transition states to Normal would most likely have been
logged.
4. Sequence number 59 — How many hours have passed since the last event was
logged, and what does this event indicate?
................................................................................................................
................................................................................................................
................................................................................................................
5. Sequence number 67 — How many days have passed since the last event was
logged? How can these periods of no events being reported be avoided?
................................................................................................................
................................................................................................................
6. Sequence number 69 — Are there any indications why this directed LIP was
issued?
................................................................................................................

7. Sequence number 70 — Another directed LIP. Is this directed LIP issued to the
same or a different device-side loop?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
9. Sequence number 406 and 407 — Which controller is issuing these directed
LIPs? What does that indicate about the current problem?
................................................................................................................
................................................................................................................
10. Sequence number 77 — The errors appear to be common between loop 1A
and loop 2A. What are the common elements between these two loops?
................................................................................................................
................................................................................................................
11. Sequence number 78 — Is the indicated power supply physically located nearer
the A-side or B-side device loop?
................................................................................................................
12. Sequence number 95 — How many power supplies have lost their AC power?
Over what timeframe did they lose AC power? What are the possible sources of
this AC power loss?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
13. Sequence number 99 — What does this event most likely indicate?
................................................................................................................
................................................................................................................

14. Sequence number 110 — How many days have passed since the last errors on
loop 1A were reported?
................................................................................................................
15. Sequence number 112 — What does this event mean, and what subsequent
action is taken by the controller to fix the anomaly?
................................................................................................................
................................................................................................................
16. Sequence number 116 — Can either controller communicate via FC to the drive
in enclosure 3 Bay 4?
................................................................................................................
17. Sequence number 117 — What is the cause of the loop map being
regenerated?
................................................................................................................
................................................................................................................
18. Sequence number 118 — How many days have passed since the A-side of the
storage system last lost AC input power?
................................................................................................................
19. Sequence number 139 — What happened to the drive in enclosure 3 Bay 4? Is
the reason for the bypass listed? How could you determine the actual cause for
this drive being bypassed?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
21. Sequence number 190 — This event implies that the AC power was lost to the A-
side of the storage system for how long a period of time?
................................................................................................................

22. Sequence number 197 — Days have passed since the last indication of a power
problem. Which service actions should be taken?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................

Customer scenario
 They noticed numerous check conditions taking place in the controller event log
and wanted an analysis done.
 On 3/17/04
 On 3/23/04 at 10:18:09, the drive in 6/7 was physically removed from the
storage system.
 On 4/13/04 at 16:45:54, the drive in 10/9 was ungrouped from a disk group.
The problem is now:
 ?
Not relevant.

Questions
1. Sequence number 8092 — How many events are logged in association with the
controller syncing up its time with the SMA?
................................................................................................................
2. Sequence number 8121 — Based on the number and type of check conditions
within the last 24 hours, is the drive in 10/9 a candidate for replacement?
................................................................................................................
3. Sequence number 8139 — Is there any previous indication of problems on the
drive in 6/7?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
5. Sequence number 8147 — Does this event indicate that the same drive removed
from 6/7 was just reinserted back into the storage system?
................................................................................................................
................................................................................................................
6. Sequence number 8194 — Is the drive in 5/11 a candidate for replacement?
................................................................................................................
................................................................................................................
7. Sequence number 8223 — How many days have passed since the last check
condition error was reported by the drive in 10/9?
................................................................................................................
8. Sequence number 8282 — How many check condition errors took place on the
drive in 6/8 prior to this event being logged?
................................................................................................................
9. Sequence number 8285 — Should the drive in 6/8 be replaced?
................................................................................................................

10. Sequence number 8385 — What caused the migration of data off of the 10/9
drive to start taking place?
................................................................................................................
................................................................................................................
................................................................................................................
11. Sequence number 8388 — What is the cause for the start of this leveling event?
................................................................................................................
................................................................................................................
12. Sequence number 8428 — What caused the migration of data off of the 6/8
drive to start taking place?
................................................................................................................
................................................................................................................
................................................................................................................
13. Sequence number 8456 — Does this event take place during or after the 6/8
drive data migration?
................................................................................................................
................................................................................................................
14. Sequence number 8484 — Can drives that have been rendered inoperable
have their data migrated off of them?
................................................................................................................
15. Sequence number 8487 — Is this the first time the 0103 disk group transitioned
to a state of “Disk Group with no redundancy is inoperative”? Why?
................................................................................................................
16. Sequence number 8490 — The volume begins to be reconstructed. How long
after the volume went missing did this event take place?
................................................................................................................
17. Sequence number 8495 — Why didn’t the disk group indicate the start of a
leveling event after the successful completion of reconstruction?
................................................................................................................
................................................................................................................
................................................................................................................

18. Sequence number 8495 — What are the possible reasons for the merge event
taking place?
................................................................................................................
................................................................................................................
................................................................................................................
19. Sequence number 8559 — Why are we still getting SMA communication
failures to the drive in 6/8? How long has the data already been reconstructed
off of the 6/8 drive?
................................................................................................................
................................................................................................................
................................................................................................................
20. Sequence number 8559 — Has the merge (which started on sequence number
8495) completed? If not, why not?
................................................................................................................
................................................................................................................
21. Sequence number 8559 — Which service actions should be taken on this
storage system?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................

Customer scenario
 Their storage system went offline and approximately 10 minutes later came back
online.
 On 3/29/04 at approximately 3:30 AM
 None
The problem is now:
 Fixed, but under investigation
Analyze
 The customer would like root cause analysis for the outage.
 2C18D configuration
 Loop switches are present

Questions
1. Sequence number 13749 — Is the directed LIP activity isolated to loop 1B?
................................................................................................................
2. Sequence number 13630 — The 020E controller has issued directed LIPs to both
loops of the loop 1 pair. What is the only device that could be causing both
controllers on both loops to issue directed LIPs to all devices?
................................................................................................................
................................................................................................................
3. Sequence number 13753 — This event indicates the 0219 controller is about to
perform which action (within the next couple of seconds or milliseconds)?
................................................................................................................
................................................................................................................
4. Sequence number 13759 — What is the cause for this controller resync?
................................................................................................................
................................................................................................................
5. Sequence number 13655 — How many loop switches were detected by the two
controllers? How many should have been detected?
................................................................................................................
................................................................................................................
6. Sequence number 13788 — How many directed LIPs were issued to the 1A loop
before the 0219 controller transitioned it to the Failed state?
................................................................................................................
7. Sequence number 13689 — Is this a standard controller startup message?
................................................................................................................
8. Sequence number 13805 — How many times has the 0219 controller marked
the 1A loop as Failed since the controller began booting?
................................................................................................................
9. Sequence number 13814 — Prior to controller 0219 failing the 1B loop for the
3rd time since boot, did it indicate it was going to enable the 1A loop?
................................................................................................................
................................................................................................................

10. Sequence number 13702 — At this point, all four loop pair 1 loops have failed
how many times since they controllers were booted?
................................................................................................................
................................................................................................................
11. Sequence number 13816 — What does the error message “An HSV110
controller has failed” indicate?
................................................................................................................
................................................................................................................
................................................................................................................
12. Sequence number 13704 — Since the errors were first logged against the drive
in 2/4, how much time has elapsed?
................................................................................................................
13. Sequence number 13705 — In a normal storage system startup sequence, which
boot events are typically logged prior to this event?
................................................................................................................
................................................................................................................
14. Sequence number 13718 — This is the first message associated with the LID
recovery code attempting to recover access to the devices on the indicated loop
port. Is this event logged by both controllers or just the master controller?
................................................................................................................
................................................................................................................
15. Sequence number 13721 — How much time passed between the LID recovery
code start and the bypassing of the troublesome drive in 2/4?
................................................................................................................
16. Sequence number 13724 — Why did this controller resync take place?
................................................................................................................
................................................................................................................
17. Sequence number 13736 — With the drive in 2/4 now bypassed, how many
loop switches were detected by the controllers on boot?
................................................................................................................

18. Sequence number 13737 — Which volume just transitioned to the Missing state?
................................................................................................................
................................................................................................................
19. Sequence number 13838 — Which device has just transitioned from the Failed
to the Normal state?
................................................................................................................
................................................................................................................
20. Sequence number 13872 — Which device has just transitioned from the Failed
to the Normal state? Now that both have transitioned to this state, how much
time passes before the storage system transitions to the online state?
................................................................................................................
................................................................................................................
21. Sequence number 13873 — How much time has passed since the controllers
originally resynced due to a VRAID1 inoperative condition?
................................................................................................................
................................................................................................................
22. Sequence number 13874 — Is this event cause for concern?
................................................................................................................
................................................................................................................
................................................................................................................
23. Sequence numbers 13878 through 13881 — Did the two disk groups level at the
same time, or did they level one after the other?
................................................................................................................
................................................................................................................
24. Sequence number 13882 — What are the possible causes for this event being
logged? Based on the rest of the controller event log, should any service actions
be taken on the indicated disk?
................................................................................................................
................................................................................................................
................................................................................................................

25. Sequence number 13889 — At this time, what service actions should be taken
on this storage system?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................

Customer scenario
 Their storage system went offline and approximately 10 minutes later came back
online.
 On 3/29/04 at approximately 3:28 AM
 On 3/29/04 at approximately 13:41 a drive was removed and replaced.
The problem is now:
 Fixed, but under investigation
Analyze
 The customer would like root cause analysis for the outage.
 2C18D configuration
 Loop switches are present

Questions
1. Sequence number 13802 — Which drive is the apparent cause for the directed
LIPs on loop 1A?
................................................................................................................
2. Sequence number 13811 — How many directed LIPs to ALPA FF were done to
loop 1A prior to the 1334 controller telling the Failed loop 1B to enable itself?
................................................................................................................
................................................................................................................
3. Sequence number 13814 — What is the likely component that is causing both
loop 1A and loop 1B to fail?
................................................................................................................
................................................................................................................
4. Sequence number 13817 — Why was the disk on 5/4 rendered inoperable?
................................................................................................................
................................................................................................................
5. Sequence number 13819 — Why is the 1334 controller resyncing? Is there any
event notification yet of the 12F9 controller resyncing?
................................................................................................................
................................................................................................................
................................................................................................................
6. Sequence number 13829 — When the 1334 controller boots, how many loop
switches does it find? What does this indicate?
................................................................................................................
................................................................................................................
................................................................................................................
7. Sequence number 13872 — At this point, for controller 1334, what is the current
state of the two loop pair 1 loops?
................................................................................................................
................................................................................................................
................................................................................................................


................................................................................................................
................................................................................................................
9. Sequence number 13441 — How long after the 1334 controller resynced due to
a VRAID1 inoperative, did the 12F9 controller resync due to a VRAID1
inoperative?
................................................................................................................
................................................................................................................
10. Sequence number 13499 — At this point, for controller 12F9, what is the current
state of the two loop pair 1 loops?
................................................................................................................
................................................................................................................
11. Sequence number 13500 — Why did the map generation number go from a 0
to a 1? The earlier controller reboot did not cause this – explain.
................................................................................................................
................................................................................................................
................................................................................................................
12. Sequence number 13501 — Why did this volume transition to the Missing state?
................................................................................................................
................................................................................................................
................................................................................................................
13. Sequence number 13616 — Why did the controller transition to the Failed state?
................................................................................................................
................................................................................................................
14. Sequence number 13629 — What does this event mean, and why did it take
place?
................................................................................................................
................................................................................................................
................................................................................................................

15. Sequence number 13630 — Which recovery process is this event the start of?
................................................................................................................
................................................................................................................
16. Sequence number 13632 — Which device was bypassed by the LID recovery
code? Would this be the device you would have guessed would have been
bypassed?
................................................................................................................
................................................................................................................
17. Sequence number 13632 — How long did the LID recovery code take to bypass
the LID drive?
................................................................................................................
18. Sequence number 13650 — Is it possible for a volume to transition from the
Missing to the Reverting state without first transitioning to the Reconstructing
state?
................................................................................................................
19. Sequence number 13776 — Why was this event reported?
................................................................................................................
................................................................................................................
20. Sequence number 13901 — How long has it been since the 1334 controller
transitioned to the Failed state?
................................................................................................................
................................................................................................................
21. Sequence numbers 13906, 13907 and 13908 — Why are 3 disks becoming
quorum disks? Can you determine where all system quorum disks will be
physically located after these three events take place?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................

22. Sequence number 13911 — If the customer had VRAID5 virtual disks carved out
of disk group 0102, is all of that data now forever lost?
................................................................................................................
................................................................................................................
23. Sequence number 13922 — How many virtual disks transitioned to the Failed
state?
................................................................................................................
24. Sequence number 14037 — Approximately how long is it taking for the volumes
to complete their reverting processes?
................................................................................................................
................................................................................................................
25. Sequence number 14085 — Approximately how long has it been since the first
controller resynced (the system being brought back online)?
................................................................................................................
................................................................................................................
26. Sequence number 14087 — How many drives are running 3BE7 firmware?
Which caution exists with drives that are running this firmware version?
................................................................................................................
................................................................................................................
................................................................................................................
27. Sequence number 14184 — Which VRAID types will now be online to disk
group 0103?
................................................................................................................
................................................................................................................
28. Sequence number 14185 — Which disk group is the indicated virtual disk likely
to be carved out of?
................................................................................................................
................................................................................................................
29. Sequence number 14232 — Have all of the virtual disks, which were initially
marked as Failed, now transitioned to the Normal state?
................................................................................................................
................................................................................................................

30. Sequence number 14293 — Which volume is being reconstructed? Which disk
group does this volume belong to?
................................................................................................................
................................................................................................................
31. Sequence number 14302 — Have all of the disk groups transitioned to a
Normal state?
................................................................................................................
32. Sequence number 14328 — Why would the EMU be attempting to assign a
hard address to any device at this time?
................................................................................................................
................................................................................................................
................................................................................................................
33. Sequence number 14332 — What is this event a likely result of?
................................................................................................................
................................................................................................................
34. Sequence number 14332 — Are any service actions required on this storage
system at this time?
................................................................................................................

Customer scenario
 OCP of one controller displays “FC Loop Misconfigured: Restart”
 OCP of the other controller displays “STsys has been lost”
 On 30-Mar-2006 at approximately 13:00
 EMUs reseated
The problem is now:
 Resolved, but customer would like to know the cause
Analyze
 EVA6000 2C8D.

Customer scenario
A customer experienced the following problem:
 Single Port on Fibre events logged on Loop 2B
 All devices in shelf 13 showing single path on Loop 2A
 On 11-12 September.
 No actions were taken by the customer or Services.
Analyze.
Configuration information: Not relevant.

Customer scenario
 Multiple disk failures in his system
The problem was noticed:
 On 10 October.
 No actions were taken by the customer or Services.
Analyze.

Customer scenario
 LID recovery with VCS v3.014
 No actions were taken.
Analyze.
Configuration information: Not relevant

Customer scenario
 Loop 1A and 2A logging
 Exchange complete with missing data in the same time frame
 Actions taken: None.
Analyze.

Customer scenario
 End users of mail server reported intermittent hangs of up to 2–3 minutes.
 These hangs were occurring 4–-5 times a day.
The problem was noticed:
 May, 2008
 Checked all GroupWise (mail server) applications, NetWare, and network
configurations and logs
 Checked EVA logs through Navigator, SAN switch commands
 Collected EVAPerf data for 24 hours and analyzed with PerfMonkey
 Replaced both Fibre Channel cables, and HBAs and SFPs on SAN switches
 Upgraded Qlogic driver and upgraded SAN switch firmware
The problem is now:
 Resolved, but needs to be confirmed
Analyze.
 EVA4100 2C1D configuration
 XCS 6.110
 14 146GB 10K rpm disks running in latest firmware
 Two servers attached (one file server and one mail server)

Customer scenario
 Many disks were failing
 May, 2008
 Reseating failed drives did not fix the problem.
 Issued debug commands 34/35/36 to track suspected disks (1B loop would go
down).
 The output from debug command 36 indicated many bad words on enclosure 2
starting at Bay 5.
 Suspecting drives downstream, the drive in enclosure 2, Bay 4 was ungrouped,
which appeared to clear up the problem.
 When the drive was replaced and grouped around noon, the 1B loop would not
stay up and disk errors were logged in the controller event log.
The problem is now:
 Resolved.
Analyze.
 EVA5000, 2C12D configuration
 VCS 3.028


Module 10 — Lab 1

 Use the EVAPerf command line to generate a friendly names host file and a
friendly names file.
 Use the EVAPerf command line to monitor components of EVA system
performance.
 Use Windows Perfmon to add counters and visually monitor performance.

 An EVA 3000/5000, EVA 4000/6000/8000, EVA 4100/6100/8100, or EVA
4400/6400/8400 series storage system, with the latest version of VCS or XCS
firmware
 Access to the latest version of Command View EVA and EVAPerf
This lab allows you to explore many of the features of EVAPerf, including command
line and GUI features. The lab activities concentrate on how to use the tool, not how
to analyze the data that you retrieve.
Note
Examples in this lab pertain to the EVA8000, however, they are as easily
applied to the EVA 4400/6400/8400 series.

 Concentrate on tasks that might help you to diagnose and troubleshoot the
array.

Performance analysis is complex. Often, wrong conclusions are drawn from extracted
data due to an insufficient understanding of the operations of all components
involved in the analysis.
Without baseline performance numbers for a given system’s operation, it can be very
challenging to determine if a system’s performance is at or near optimal.
The performance of the EVA is impacted by many factors, some of which include:
 The number of disks in a disk group
 The total number of disk groups
 The speed (10K or 15K RPM) of the drives used in a disk group
 The number of virtual disks carved out of a disk group
 The total number of LUNs being served by the storage system
 What type of host I/O operations are taking place (random versus sequential
I/O patterns and large versus small I/O data transfer sizes)
 The types of ongoing EVA operations, such as:
 Snapshots being used
 Snapshots or snapclones being created
 Data reconstruction or migration taking place
 Virtual disks or other objects being created

Using EVAPerf
The performance of the system beyond the EVA is impacted by many factors, some of
which include:
 The number of hosts accessing the storage system
 The fabric infrastructure used to access the storage system
 Host computer properties, such as:
 Application types and the number of applications being used
 Operating system type being used
 Number and speed of processors
 Amount and types of cache
 Amount and types of RAM
 Number and types of HBAs being used
 Number of and bus type on the host computer
Note
See the performance analysis white paper on the course CD for more
information about EVA performance analysis.

EVAPerf enables you to monitor and display EVA performance metrics for:
 Replication data
 Host connection data
 Host port statistics
 Physical disk data
 Port status
 Storage array data
 Storage controller data
 Virtual disk data
You can display performance metrics graphically in the Windows Perfmon utility or
you can display metrics in tabular form in a command prompt window (using
EVAPerf from the command line). You can also output the metrics in CSV (comma-
separated value) or TSV (tab-separated value) format for use with external
applications, such as Microsoft Excel.
Files, components, and locations

EVAPerf files are installed when you install Command View EVA. The EVAPerf files
are installed in c:\Program Files\Hewlett-Packard\EVA Performance Monitor.
The following EVAPerf components are installed:
 evaperf. Use to view performance metrics in a command line interface (CLI). You
must run evaperf.exe from the directory where it was installed or it will not find
the necessary configuration files. The application uses data generated by the
EVA Data Collection Service.
 Extension DLL for Perfmon (evapmext.dll). Use to view performance metrics in a
GUI.
If you use Windows Perfmon for background logging, HP recommends that you
set the EVA Data Collection service to start automatically. If you execute logging
before starting the service, the startup time for the service might exceed the time
that Windows Perfmon waits for the first data samples.

Using EVAPerf
 EVA Data Collection Service (evapdcs.exe). Use to gather data from EVAs that
are visible to the host and store it in memory cache. You can then use either
evaperf.exe or evapmext.dll to retrieve and view the information.
The service is set to manual when you install EVAPerf. When you run
evaperf.exe, the service starts and remains running until you reboot the host.
You can also start and stop the service using Windows Service Manager.
 EVAPerf TLViz Formatter. Three files are installed to support the TLViz Formatter:
 EVAPerf-TLViz-Format.exe — A file that formats the EVAPerf all command
output so you can view it with the TLViz tool.
 EVADATA.MDB — A Microsoft Access database template used to view the
all command output in a database. The data from the all command
resides in individual tables.
 MSADODC.OCX — A file necessary to operate the TLViz Formatter
interface. This should be located in c:\windows\system32.
Friendly names
EVAPerf queries the management server for storage system information from all EVAs
it is managing. The storage system information queried includes:
 Storage system WWNs
 Virtual disk WWNs
 Host connection identifiers
 Disk group identifiers
This information is stored in a text file named fnames.conf.
Note
The friendly names file must reside in the same directory as the EVAPerf tools.
The fnames.conf file is used to associate the storage system WWN, virtual disk
WWN, host connection, and disk group name information, with human-readable
names. These friendly names improve the readability of performance reports and
data.

Updating friendly names

You can create friendly names manually or you can have them generated. When
they are created, update the friendly names file when you change monitored
configurations.
Creating the friendly names host file

Before you generate friendly name information for an array, create the friendly name
host file (fnamehosts.conf).
To do this, you need the account user name and password created during installation
of Command View EVA.
1. Use the following syntax:
evaperf fnh [host] [username]
Replace host with the hostname or IP address of the management server

running Command View EVA. Replace username with the account user name
created during Command View EVA installation.
Example:
evaperf fnh evagen2 administrator
2. When prompted for the password, enter the account password that was created
during Command View EVA installation.
EVAPerf verifies that it can access the Command View EVA before adding the
information to the fnamehosts.conf file.
If you enter evaperf fnh without arguments, it displays a list of known
management servers running Command View EVA.
3. Locate the fnamehosts.conf file and open it with a text editor.
Note
At this point, you should only have a file named fnamehosts.conf.
Generating and viewing friendly names

To view the friendly names for your system, do the following:
1. Use the evaperf fn command to generate and populate your friendly names
file (fnames.conf).
2. Locate the fnames.conf file and open it with a text editor.
3. Using Command View EVA, add two more virtual disks to your storage system.

Using EVAPerf
4. Use the fn command to update your friendly names file.
Note
Every time you update the friendly names file, a backup (fnames_conf.bak) is
made of the previous version.
5. Locate and open the friendly names file, then verify that it is updated with the
new virtual disks.
A sample output is given below.
Note
If you reinstall EVAPerf, the fnames.conf file is removed. Therefore, save a copy
of the file in a different directory before uninstalling EVAPerf. After the newer
version of EVAPerf is installed, copy the saved fnames.conf back to the
installation directory.
Entering friendly names manually

You can manually create and maintain the fnames.conf file using a standard text
editor. Each line in the file contains a WWN that uniquely identifies an object,
followed by the friendly name of the object. When reading this file, EVAPerf ignores
blank lines and lines that begin with a # character.

Using short names

If the friendly names you created for objects in Command View EVA are lengthy, it
can be difficult to view those names using EVAPerf. In the EVAPerf command line
interface, you can substitute contractions for full names.
When you install EVAPerf, the fnames_sample.dict file is also installed. You can do
either of the following:
 Rename this file to fnames.dict and modify its contents to include the short
names.
 Create a separate file name fnames.dict using a standard text editor.
Within the fnames.dict file, you can enter a short name for each long friendly name
as follows:
# <long name> <short name>
Consider the following when creating names:

 If the long or short name contains spaces, enclose the name in quotes (“).
 If a backslash character occurs in a string enclosed in quotes, add an additional
backslash.
 Enter one long and short name combination per line.
The following is a sample short names file.
To use the short name you entered in this file, add the –cn modifier to a command
you enter in the EVAPerf CLI. The short name is substituted when a long name is
encountered.

Using EVAPerf
Creating and viewing short names

To create and view the friendly names for your system, do the following:
1. Navigate to the EVAPerf directory.
This should be c:\Program Files\Hewlett-Packard\EVA Performance Monitor.
2. Locate and open the fnames_sample.dict file.
3. Enter a short name for each long friendly name using the following format:
# <long name> <short name>
4. Save the file with a name of fnames.dict.
Note
To use the short names you entered in this file, add the –cn modifier to a
command you enter in the EVAPerf CLI. The short name is substituted when a
long name is encountered.

You can use Windows Perfmon to monitor and view EVA performance data.
The following example shows one of several ways to access and configure Perfmon
to display EVA performance metrics.
Note
It is assumed that you are somewhat familiar with Perfmon.
1. Click Start  Run and type perfmon, then click OK.

The Perfmon window appears, similar to the following.

Using EVAPerf
2. Right-click in the graph area and a menu appears.

3. Select Add Counters and click OK.
The Add Counters dialog box appears.
4. In the Select counters from computer drop-down list box, select the computer
where the EVA performance tools are running.
5. In the Performance object drop-down list box, select an HP EVA object to
monitor (for example HP EVA Storage Array).
6. Click All counters to select every listed counter, or select only those counters you
are interested in.
Note
If you click a counter in the list, you can click the Explain button for a brief
explanation of the meaning of the selected performance counter.
7. Click All instances to select every listed performance object type, or select those
instances you are interested in.
8. Click the Add button to add the counters to the window.
9. Click Close to close the Add Counters dialog box.

10. Using IOmeter or Winthrax, generate as much I/O activity as possible from your
host to a storage system virtual disk.
Note
Open IOmeter or Winthrax from a shortcut on your desktop or course CD.
Perfmon immediately begins to display the performance of the selected EVA

objects, similar to the following.
Note
To add other objects, repeat the steps or use the plus “+” icon. To remove
metrics, select the metric from the list and click the remove icon (to the right of the
+ icon) in the toolbar.

Using EVAPerf
Objects and counters

The following table is a complete listing of all EVA objects and their associated
performance counters along with the EVAPerf generated explanation of each counter.
Most of the counters have been defined in Appendix A of this lab.
EVA objects Performance Description

counters
HP EVA DR tunnels
Round trip delay (ms) Round trip delay time in milliseconds
Copy retries Number of copy retries
Write retries Number of write retries
Copy in MB/s Incoming copy rate in MB/s
Copy out MB/s Outgoing copy rate in MB/s
Write in MB/s Incoming write rate in MB/s
Write out MB/s Outgoing write rate in MB/s
HP EVA host connections
Port Identifies the port
Queue Depth Connection queue depth
Busies Busy responses sent to host
HP EVA host port statistics
Read Req/s Number of read requests per second
Read MB/s Read rate in MB/s
Read Latency Read latency in microseconds
Write Req/s Number of write requests per second
Write KB/s Write rate in KB/s
Write Latency Write latency in microseconds
Av Queue Depth Average queue depth
HP EVA physical disk group
Average Drive Queue Depth of the drive queue
Depth
Average Drive Latency Drive latency in microseconds
Average Read Req/s Read requests per second
Average Read MB/s Read in MB/s
Average Read Latency Average read latency in microseconds
Average Write Req/s Write requests per second
Average Write MB/s Writes in MB/s
Average Write Latency Average write latency in microseconds
Number of Disks Number of disks in the disk group.

EVA objects Performance Description

counters
HP EVA storage array
Total Host Req/s The total number of host requests per
second
Total Host MB/s The total number of host requests in
MB/s
HP EVA storage controller
% Processor Time Percentage CPU time
% Data Transfer Time Percentage CPU time used to perform
data transfer operations
HP EVA Virtual Disk

Read Hit Req/s Read hits in requests per second
Read Hit MB/s Read hits in MB/s
Read Hit Latency Read hit latency time in microseconds
Read Miss Req/s Read misses in requests per second
Read Miss Data Rate Rate data is read from physical disks
because data was not present in the
array cache memory
Read Miss Latency Read misses latency time in
microseconds
Write Req/s Write requests per second
Write data rate Rate at data is written to virtual disks
Write Latency Write latency time in microseconds
Flush Data Rate Rate at which data is written to a
physical disk
Mirror Data Rate Rate at which data travels across the
mirror port
Prefetch Data Rate Rate at which data is read from the
physical disk to cache

Using EVAPerf
Logging performance metrics with Perfmon

The following example shows how to configure Perfmon to log EVA performance metrics.
1. Click Start  Run and enter perfmon, then click OK.
The Perfmon window appears, similar to the following.
2. Double-click Performance Logs and Alerts.

3. Double-click Counter Logs.
4. From the menu, click Action  New Log Settings, enter the name of the log file,
and then click OK.

5. When the next dialog appears, select the Add Counters button.
6. Select the EVA object that you want to log.
7. Select the counters and instances for the selected object, then click Add.
8. Repeat the above steps for each object you want to log.
9. Click the Close button.
10. Select the Log Files tab to specify the log file type (select a type of text file), then
click the Configure button.

Using EVAPerf
11. Specify the log file location and name, and then click OK.
12. Select the Schedule tab to schedule running the Perfmon job, then click OK.
13. After running the job for a while, view the log from the Perfmon system monitor
by selecting the View Log Data button (Ctrl/L).

14. Select Log files as the data source and then click Add; then, when prompted for
the file, locate the file you created and click OK.
15. To view the counters, select the Add button (+).

16. Add the objects and counters you want to look at.
17. Modify the output display as needed.
18. Open the log file to view its contents.

Using EVAPerf
You can use a command prompt window to display EVA performance data in a
tabular format. You can also output the tabular data in CSV (comma-separated
value) or TSV (tab-separated variable) format.
The following example assumes that you are familiar with command prompt use. The
example shows running EVAPerf to display storage cell tabular data.
1. Click Start  Run, enter cmd, and then click OK.
A command prompt window appears.
2. Change to the directory where the EVA Performance Tools are installed (for
example, c:\Program Files\Hewlett-Packard\EVA Performance Monitor).
3. To display only a summary of storage system information, enter evaperf ls.
The window displays a summary of current storage system data. This window is
not refreshed.
Important
! See Appendix B for a list of supported qualifiers when running the command
prompt option.
4. Enter evaperf ls -cont to continuously display refreshed storage system data.

The window refreshes every second.
Note
To stop the continuous display, use Ctrl-c.
5. Enter evaperf ls -cont 10 to continuously display refreshed storage system

data.
The window refreshes every 10 seconds.
6. To display help, enter evaperf to display commands and formatting.
You can also use arguments or a combination of arguments to control the
display of data.
7. Enter evaperf ls -nh -csv and describe the results below.
................................................................................................................
................................................................................................................

Additional CLI lab exercises

Perform the following through the command line and enter the command in the
space provided (answers are on the next page):
1. Show the summary of the array status.
................................................................................................................
2. Show the array status in comma-separated format.
................................................................................................................
3. Show the controller status continuously every 5 seconds for a duration of 10
seconds.
................................................................................................................
4. Show the controller status continuously every 5 seconds, for a duration of 20
seconds, in TSV format, and output to a file. Check the file with a text editor.
................................................................................................................
5. Show the host connections.
................................................................................................................
6. Show the host port statistics in tab-separated format.
................................................................................................................
7. Show the LUNs visible to the system, but with no headings.
................................................................................................................
8. Show the average disk group metrics.
................................................................................................................
9. Show the average disk group metrics every 5 seconds, for a duration of 20
seconds, using a KB display, and output to a file. Check the file with a text
editor.
................................................................................................................
10. Show the port status in tab-separated format with a timestamp.
................................................................................................................
11. Show the virtual disks continuously every 5 seconds and in tab-separated
format, and for only two specific virtual disks.
................................................................................................................
12. Show the EVA storage systems in CSV format and redirect the output to a text
file. Check the file with a text editor and then use Microsoft Excel to open it.
................................................................................................................

Using EVAPerf
Answers
1. evaperf all
2. evaperf as –csv
3. evaperf cs –cont 5 –dur 10
4. evaperf cs –cont 5 –dur 20 –tsv -fo name.txt
5. evaperf hc
6. evaperf hps –tsv
7. evaperf luns -nh
8. evaperf pdg
9. evaperf pdg –cont 5 –dur 20 –KB –fo name.txt
10. evaperf ps –tsv –ts1
11. evaperf vd –cont 5 –tsv –fvd vdisk1 vdisk2
12. evaperf ls –csv -fo name.txt
Additional lab exercises (optional)

This lab attempts to show a performance difference between two differently
configured storage systems. Perform the following exercises to determine a
performance difference:
1. Create a disk group with only eight members in it and a disk failure protection
level of Single.
2. Create a single 5GB VRAID5 virtual disk from the disk group and present it to a
Windows host running EVAPerf.
3. Using Winthrax or any other I/O generation utility, generate as much I/O
activity as possible to the 5GB virtual disk.
4. Using EVAPerf, determine how many MB/s you are able to transfer on average
between the Windows server and the EVA storage system. Write the achieved
number in MB/s below.
................................................................................................................
Note
Allow the test to run for a couple of minutes to get a more accurate determination
of average MB/s performance.

5. Create a disk group with as many disks as possible with a disk failure protection
level of Single.
6. Create one 5GB VRAID5 virtual disk from the disk group and set the Preferred
path/mode as Path B-Failover only.
7. Create one 10GB VRAID1 virtual disk from the disk group and set the Preferred
path/mode as Path A-Failover only.
8. Present both created virtual disks to the Windows host.
9. Using Winthrax or any other I/O generation utility, generate as much I/O
activity as possible to the two virtual disks.
10. Using EVAPerf, determine how many MB/s you are able to transfer on average
between the Windows server and the EVA storage system. Write the achieved
number in MB/s below.
................................................................................................................
11. Calculate the difference in MB/s between the two tests and write the result
below.
................................................................................................................
Note
When this same test was performed on a first generation EVA lab system with 75
10K RPM disks, the difference between the two tests was 38MB/s.

Using EVAPerf
The following sections give a description of each category of performance object

and a list of counters for that object.
HP EVA DR tunnels
The HP EVA DR tunnels object reports the intensity and behavior of the link traffic
between source and destination arrays. The counters for this object display
information only if there is at least one active DR group on the array. Otherwise, only
the header appears. You can display metrics in either MBs or KBs.
Although some arrays allow up to four open tunnels on a host port, only one tunnel is
active for a single DR group. Multiple DR groups can share the same tunnel. Statistics
for each tunnel are reported by both the source and destination arrays, but the
directional counters are complementary.
The counters are:
 Round Trip Delay — The average time, in milliseconds, for a signal (ping) to
travel from the source to the destination and back. In replication traffic, the
signal is queued behind data transmissions, which increases the round trip
delay. If the destination controller is busy, the value also increases. Round trip
delay is reported for all active tunnels.
 Copy Retries — The number of copies from the source EVA that were
retransmitted due to a failed copy transmission. Each retry creates a 128KB
copy. Retries are reported by both the source and destination arrays.
 Write Retries — The number of writes from the source EVA that were
retransmitted due to a failed write to the destination EVA. Each retry creates an
8KB copy. If the write contains multiple 8KB segments, only the failed segments
are retransmitted. Retries are reported by both the source and destination
arrays.

 Copy In MB/s — The rate at which data is copied to an array to populate the
members of a DR group with data when an initial copy or full copy is
requested.
 Copy Out MB/s — The rate at which data is copied from an array to populate
the members of a DR group with data when an initial copy or full copy is
requested.
 Write In MB/s — The rate at which data is written to an array because of write
activity to the members of the source array. The write activity includes host
writes, merges, and replication retries. A merge is an action initiated by the
source array to write new host data that has been received and logged while a
replication write to the destination array was interrupted, and now has been
restored.
 Write Out MB/s — The rate at which data is written from an array because of
write activity to the members of the source array. The write activity includes host
writes, merges, and replication retries.
HP EVA host connection

The HP EVA host connection object provides information for each host bus adapter
that has a connection to an array.
The counters are:
 Port — The port number the array controller uses internally to identify the port
(HSV100 controller series only).
 Queue Depth — The average number of outstanding requests from each host
adapter.
 Busies — The number of busy responses sent to a specific host. A busy response
is a request from the array to the host to cease I/O traffic until an internal job
queue is reduced.

Using EVAPerf
HP EVA host port statistics

The EVA host port statistics object provides information about performance and data
flow of host-initiated activity on each array host port. Data replication traffic is not
included in these counters. Depending on the array model, there are either two ports
per controller (four per controller pair) or four host ports per controller (eight per
controller pair).
The counters are:
 Read Req/s — The number of read requests (per second) completed from each
host port.
 Read MB/s — The rate at which data is read from each host port.
 Read Latency — The amount of time it takes to complete a read request (from
initiation to information receipt) through a host port. The time is an average of
the read request latency for all virtual disks accessed through this port, and
includes cache hits and misses.
 Write Req/s — The number of write requests (per second) completed from each
host port.
 Write MB/s — The rate at which data is written from each host port.
 Write Latency — The amount of time it takes to complete a write request (from
initiation to information receipt) through a host port. The time is an average of
the write request latency for all virtual disks accessed through this port.
 Average Queue Depth — The average number of outstanding host requests
against all virtual disks accessed through this host port. This number includes all
host-initiated commands, including non-data transfer commands.

HP EVA physical disk group

The HP EVA physical disk group object provides information about physical disk
activity per disk group. For each disk group, metrics are reported that represent the
averages of various counters across all the disks in the disk group. The counters
record all activity to the disks including traffic for host data transfers and internal
system support. This activity includes metadata updates, cache flushes, prefetch,
sparing, leveling, snapclone and snapshot support, and redundancy traffic such as
parity reads and writes or mirror copy writes. Each controller’s activity is reported
separately, so the total activity to each disk group is the sum of activity from both
controllers.
Note
For each counter, the results are an average of all disks in the disk group.
The counters are:

 Average Drive Queue Depth — The average number of all active requests to
each disk in the disk group, over all the disks in the disk group.
 Average Drive Latency — The average time between when a data transfer
command is sent to a disk and when command completion is returned from the
disk. The time is not separated into read and write latencies. Completion of a
disk command does not necessarily imply host request completion because the
request to a specific physical disk might be only a part of a larger request
operation to a virtual disk.
Note
On the HSV100 series of controllers, only average latency—the average of read
and write latencies—is reported. On the HSV200 series of controllers, separate
metrics are provided for read and write latency.
 Average Read Req/s — The number of read requests (per second) sent to
physical disks.
 Average Read MB/s — The rate at which data is read (per second) from
physical disk.
 Average Read Latency — The average time it takes for a disk to complete a
read request. This average is weighted by requests per second. (HSV200
controller series only).

Using EVAPerf
 Average Write Req/s — The number of write requests (per second) sent to
physical disks.
 Average Write MB/s — The amount of data written (per second) to physical
disks.
 Average Write Latency — The average time it takes for a disk to complete a
write request. This average is weighted by requests per second. (HSV200
controller series only)
 Number of Disks — The number of disks in the disk group.
HP EVA storage array

The HP EVA storage array object provides information about the total workload on
the array.
The counters are:
 Total Host Req/s — The number of all host-initiated requests (per second) sent to
each controller.
 Total Host MB/s — The rate at which data is read from or written to disk (per
second) per controller.
HP EVA storage controller

The HP EVA storage controller object provides information about controller
processor and host data transfer utilizations.
The counters are:
 % Processor Time — The percentage of time that the central processing unit on
the controller is active. A completely idle controller shows 0%. A controller
saturated with activity shows 100%.
 % Data Transfer Time — Similar to % Processor Time except that it does not
include time for internal processes not related to host-initiated data transfers. For
example, it does not include time for sparing, leveling, snapclones, snapshots,
replication traffic, virtual disk management, or communication with other
applications. The value is always equal to or less than the % Processor Time
counter and the difference is the amount of processor time engaged in non-data
transfer activity.

HP EVA virtual disk

The Virtual Disk object provides information about workload and performance for
each virtual disk on the array. Activity is reported separately for each controller
accessing a virtual disk. The total activity for each virtual disk is the sum of the
reported activity for each controller. A virtual disk may also be a snapshot,
snapclone, or a DR group member. In the output, logical unit number (LUN) is used
interchangeably with virtual disk.
Virtual disks must be presented to a host to be seen by HP Command View EVAPerf.
However, replication volumes on the replication system are visible without being
presented.
If the array controllers are active/standby, all activity to a virtual disk is through the
active controller. If the array controllers are active/active, one controller is preferred
(the owning controller) but requests can still be processed by the other controller (the
proxy controller). In active/active controllers, all host requests are logged by the
receiving controller only, whether owning or proxy. Therefore, all request rate and
data rate activity for a virtual disk is the sum of both controllers.
The counters are:
 Read Hit Req/s — The number of read requests per second completed from the
array cache memory. Data may reside in the cache memory due to a previous
cache miss or because of a prefetch operation generated by a sequential read
data stream.
 Read Hit MB/s — The rate at which data is read from the array cache memory
because of read hit requests.
 Read Hit Latency — The average time it takes to complete a read request (from
initiation to information receipt) from the array cache memory.
 Read Miss Req/s — The number of read requests (per second) that failed to
complete from the array cache memory and were completed from physical
disks instead.
 Read Miss Data Rate — The rate at which data is read from physical disks
because the data was not present in the array cache memory.
 Read Miss Latency — The average time it takes to complete a read request
(from initiation to information receipt) from the physical disks.

Using EVAPerf
 Write Req/s — The number of write requests per second completed to a virtual
disk that were received from all hosts. Write requests may include transfers from
a source array to this array for data replication and host data written to
snapshot or snapclone volumes.
 Write Data Rate — The rate at which data is written to the virtual disk by all
hosts and includes transfers from the source array to the destination array.
 Write Latency — The average time it takes to complete a write request (from
initiation to receipt of write completion).
 Flush Data Rate — The rate at which data is written to a physical disk for the
associated virtual disk. The sum of flush counters for all virtual disks on both
controllers is the rate at which data is written to the physical drives and is equal
to the total host write data. Data written to the destination array is included.
Host writes to snapshots and snapclones are included in the flush statistics, but
data flow for internal snapshot and snapclone normalization and copy-before-
write activity are not included.
 Mirror Data Rate — The rate at which data travels across the mirror port to
complete read and write requests to a virtual disk. This data is not related to the
physical disk mirroring for VRAID1 redundancy. Write data is always copied
through the mirror port when cache mirroring is enabled for redundancy. In
active/active controllers, this counter includes read data from the owning
controller that must be returned to the requesting host through the proxy
controller. Reported mirror traffic is always outbound from the referenced
controller to the other controller.
 Prefetch Data Rate — The rate at which data is read from the physical disk to
cache in anticipation of subsequent reads when a sequential stream is detected.
Note that a sequential data stream may be created by host I/O and other I/O
activity that occurs because of a DR initial copy or DR full copy.

To get help on the internal commands, go to the c:\Program Files\Hewlett-

Packard\EVA Performance Monitor directory and execute the command evaperf –
help.
Note
The following commands are from EVAPerf version 9.0.
Command options
Command options are included in the following table.
Command Description
all (or nall) Displays a summary of the array status by running the following commands
together: ls, as, cs, vd, vdg, hc, ps, hps, pd, pdg, drg, and drt
as Displays array status
cs Displays controller status
cvconfig Configures Command View EVA login parameters to be used for state data
collection
dginfo Disk group configuration
drg Displays data replication groups.
drt Displays data replication tunnel statistics
dpw wwn Deletes the password for the specified array from the host’s Windows
registry. The password is not deleted from the array.
fnh Displays a list of known friendly-name Command View EVA hosts and adds
a friendly name host to the friendly name host list
fn Reloads friendly-names from known Command View EVA hosts to the
fnames.conf file
h, help, or evaperf Displays help for EVAPerf
hc Displays host connections. The Port column in the output does not display
data for the HSV2x0 series of controllers (-a appears in the Port column).
hist Displays historical information and state data
hps Displays host port statistics
ls Displays visible EVA storage systems visible to the host
luns Displays LUNS visible to this host
mof Displays output for the ls, as, cs, vd, vdg, hc, ps, hps, pd, and pdg
commands and saves the output for each command in a separate file. The
-csv and -od modifiers are required.
pd Displays physical disk data
pda Displays statistics for physical disk activity
pdg Displays the total physical disk data by disk group
pfa Sets the array filter list in Windows Perfmon
pfd Deletes the filter configurations for Windows Perfmon
pfh Displays help for the Windows Perfmon filter commands
pfs Displays the filter configuration for Windows Perfmon
pfvd Sets the virtual disk filter list in Windows Perfmon
ps Displays port status

Using EVAPerf
Command Description
rc Resets the error counters reported by the ps command
Server host port Configures EVAPerf RPC server for remote client.
[username]
sfn Show friendly name map
showcvconfig Displays Command View EVA login parameters used for state data
collection
spw wwn password Sets the password for an array. This password must match the password
entered on the controller OCP of the array.
vd Displays virtual disk statistics for only those virtual disks that have been
presented to a host
vdg Displays total LUN activity by disk group
vdts Displays virtual transfer size histograms. This command is only available
on the HSV2x0 series of controllers.
vdtsg [lunwwn] Graphs virtual disk transfer size histograms for all LUNs or a given
WWN. This command is only available on the HSV2x0 series of
controllers.
vdrl Displays virtual disk read latency histograms (HSV2x0 only)
vdrlg [lunwwn] Graphs virtual disk read latency histograms for all LUNs or a specific
controllers.
vdwl Displays virtual disk write latency histograms. This command is only
available on the HSV2x0 series of controllers.
vdwlg [lunwwn] Graphs virtual disk write latency histograms for all LUNS or a specific
controllers.
verifycvconfig Verifies if Command View EVA is accessible for state data collection
vpw Verifies array passwords for use with EVAPerf
Output formatting options

Output formatting options are included in the following table.
Command Description
-cn Substitutes friendly names from the fnames.dict file.
-cont n Runs an EVAPerf command continuously. You can specify the interval by adding a
number (n). Otherwise, the default interval is one second. Press Ctrl+c to exit from
this mode.
-csv Displays data in CSV (comma-separated value) format and automatically includes
a time stamp. The time stamp format can be modified using the –ts1 or –ts2
modifiers.
–dur n Specifies the duration of a continuous mode session. For example, if you enter
evaperf vd –csv –cont –dur 30, virtual disk data is displayed in CSV
format at one second intervals for a total of 30 seconds.

Command Description
-fd keyword Displays data that contains the specified keywords. You must enter at least one
keyword. To enter multiple keywords, separate each keyword with a space. For
example, if you enter evaperf –fd test prelim good, the data that
displays contains the words test, prelim, and good.
-fnid Display the WWN, group, DRM group, and host name along with corresponding
friendly names.
–fo filename Copies output to a file as well as displaying it in the command prompt.
You can combine this modifier with –cont and –dur for a fixed-time data capture.
For example, if you enter evaperf vd –cont 5 –dur 30 –fo
capture.log, virtual disk data is displayed in CSV at five second intervals for a
total of 30 seconds and is also written to the capture.log file.
-fvd vdisk Limits virtual disk data collection to the specified virtual disk(s). You must enter at
least one virtual disk. You can also combine this modifier with –sz to limit data
collection to the specified array(s). For example, if you enter evaperf vd –fvd
test1 test2 –sz server1, data is collected for virtual disks test1 and test2
on array server1 only. You can use this modifier with the vd, vdrl, vdwl, and vdts
commands.
-KB Displays output data in kilobytes per second (1024). The default is megabytes per
second (1,000,000).
-nfn Specifies that friendly names should not be used.
-nh Specifies that no headings be included in CSV (comma-separated value) output.
-nots Specifies that a time stamp not be included in the CSV output.
-od Specifies the directory in which the output files from the mof command are saved.
-sz Filters the arrays that are interrogated.
-tlc Displays TLC-compliant data for the mof command.
-tsv Displays output in tab-separated variable format with a time stamp.
-ts1 Adds a time stamp to the –csv output in the following format: Fri Jul 23 16:23:05
2004.
-ts2 Adds a time stamp to the –csv output in the following format: 23/Jul/2004
16:23:05 2004. This is the default format.
-us Display times in microseconds (the default is milliseconds). Latencies are
displayed in milliseconds (ms) by default. Time that is less than one millisecond
appears as zero. Use the –us option to show times in microseconds for more
accuracy.

Module 10 — Lab 2

 View the output of the evaperf all command using Microsoft Excel.
 Use the TLViz Formatter to convert an EVAPerf output file into individual files.
 View each of the formatted output files in Excel.
 Use Excel to format the EVAPerf virtual disk output file.
 Use Excel to graph virtual disk latency on EVA controllers.
 Use the TLViz Formatter to build a Microsoft Access database.
 Open an Access database and perform canned queries.
The following are the requirements for performing the lab:

 A laptop or PC with the following:
 TLViz Formatter
 Microsoft Excel
 Microsoft Access and a pre-defined database file
 Required TLViz files or access to files

This lab allows you to familiarize yourself with the most important aspect of EVA
performance assessment data collection, that is, using and manipulating the TLViz-
formatted output files.
You will be able to take a given EVAPerf output file in comma-separated value (CSV)
format, use it as input to TLViz Formatter, and generate individual EVAPerf CSV files.
You can then examine, format, and graph these files in Excel. Another exercise
allows you to use the TLViz Formatter to build the Access database and run queries.
 Concentrate on those tasks that might help you to diagnose and troubleshoot
performance problems with the array.

Using TLViz Formatter Output Files
Before generating individual TLViz output files, it is useful to view the output of the
evaperf all command in a single Excel file.
Perform the following and answer the questions:
1. Locate and unzip the All Files.zip file.
Note
This file is available as part of the course files in the folder called 03-EVAPERF
Formatter.
2. Copy the EVA2.csv file to an empty directory. This is the raw comma-separated
value output from the evaperf all command.
3. Open EVA2.csv with Excel.
a. Which information is stored in this file?
...........................................................................................................
...........................................................................................................
b. How would you track a specific virtual disk or disk group read miss latency
over the entire file capture?
...........................................................................................................
...........................................................................................................
4. Close the file but do not save changes.

Use the following procedure to generate individual TLViz output files:

1. Locate the TLViz Formatter utility and download it to your PC or laptop.
Note
If needed, your instructor will point you to the location of the TLViz Formatter.
2. Start the TLVIZ Formatter utility.

3. Complete the following:
a. For the input file, browse to and select the file EVA2.csv.
b. For the TLVIZ Title, enter EVA2.
4. Click the Start button.
Note the status bar at the bottom and wait until it shows that it is done.
5. Close the TLViz Formatter.
6. Examine the directory that contained the EVA2.csv file and note the other files
that have been created.
7. Open the following files in Excel. Which data do they contain?
a. ControllerStats.csv
...........................................................................................................
...........................................................................................................
b. HostsPort.csv
...........................................................................................................
...........................................................................................................
c. HostStats.csv
...........................................................................................................
...........................................................................................................
d. LoopStats.csv
...........................................................................................................
...........................................................................................................

e. PhyDiskStats.csv
...........................................................................................................
...........................................................................................................
f. PhysicalDisk.csv
...........................................................................................................
...........................................................................................................
g. PresentedEVA.csv
...........................................................................................................
...........................................................................................................
h. PresentedLUNS.csv
...........................................................................................................
...........................................................................................................
i. Tunnel.csv
...........................................................................................................
...........................................................................................................
j. Vdisk.csv
...........................................................................................................
...........................................................................................................
k. Vdiskstats.csv
...........................................................................................................
...........................................................................................................
You should have noted that each file contains an object class like Vdisk, host
ports, physical disks, and so on. Within each file are the specific counters
(MB/s, and so on).
Note
If you have any question on what a specific file metric represents you can look
these up in the White paper called “Performance analysis of the HP
StorageWorks Enterprise Virtual Array storage systems using HP StorageWorks
Command View EVAPerf software tool,” located at
http://h71028.www7.hp.com/ERC/downloads/5983-1674EN.pdf

8. Does this format of data allow you to more simply follow a specific virtual disk
or disk group over time? If not, why?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
9. What other files are created?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
Note
The other files that are created will be discussed later in this lab.

This portion of the lab allows you to format and graph the virtual disk output file in
Excel.
Use the following procedure to format and view the virtual disk file:
1. Open Vdisk.csv in Excel.
2. Format time
a. Select column A (Time).
b. Right-click and select Format Cells.
c. Select the Number tab.
d. Under Category, select Time, and under Type, select a time which displays
seconds.
e. Click OK to apply changes.
3. Format the header row:
a. Select row 1.
b. Right-click and select Format Cells.
c. Select the Alignment tab.
d. Under “Text control”, select Wrap text.
e. Click OK to apply changes.
4. Lock the header row:
a. Select row 2.
b. From the menu, select Window → Freeze Panes.
5. Filter data:
a. Select row 1.
b. From the menu select Data → Filter → AutoFilter.
6. Take some time to see how your changes have impacted the file.

To graph latency for LUN ALF9-Non-Replicated Disk on controller T04K, do the

following:
1. From the drop-down list under LUN, select ALF9-Non-Replicated Disk.
2. Note that the worksheet only shows one LUN but both controllers.
Why is there data from only one controller after a few minutes?
................................................................................................................
3. From the drop-down list, select controller T04K.
4. To graph:
a. Select the Time column.
b. While holding the CTRL key, select the Read Miss Latency column.
Note
At this point, both columns should be highlighted.
c. From the menu, select Insert → Chart to begin the chart wizard.
d. Under “Chart type”, select Line, then Next.
e. Select the defaults in the rest of the wizard, then Finish.
5. Change the controller by selecting controller F03S in the row, and view the
change in the chart.
6. Close Excel.

This portion of the lab allows you to build the Access database through the TLViz
Formatter. Use the following procedure:
1. Copy the file EVADATA.mdb into the directory that contains the formatted files
from the previous section.
2. Start the TLVIZ Formatter.
3. For the input file, browse to and select the same file as before, EVA2.csv.
If all the required input files are in the directory, the Build Access Database
button should be enabled.
4. Click the Build Access Database button and wait until there is a done status at
the lower left.
5. Close the TLVIZ Formatter.

This portion of the lab allows you to run pre-canned queries against the Access
database. Use the following procedure:
1. Start Access and open the Access database.
2. Run the query DiskGroup Percent Read by selecting Open or by double-clicking
the query.
a. How many disk groups are on this EVA?
...........................................................................................................
b. What is the percentage of read for both disk groups?
...........................................................................................................
c. Why is knowing the percentage of read useful in analyzing EVA
performance issues?
...........................................................................................................
3. Run the query Avg-KBperRead-Avg-KBPerWrite.
a. What is the average read and write KB for each disk group?
...........................................................................................................
b. Do these values seem odd?
...........................................................................................................
c. Does EVAPerf use base 2 or base 10 numbers for megabytes?
...........................................................................................................
d. Why is knowing the size of I/O transfers useful?
...........................................................................................................

4. Run the VDISK-Top10TotaRequsts query. What does this query tell us?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
5. Run the VDISK-Top10ReadMissReq query. What does this query tell us?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
6. Run the VDISK-Top10WriteReq query. What does this query tell us?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
7. Close Access.

Lab solutions are provided below for each corresponding topic.
Viewing an EVAPerf output file in Excel

3. Open EVA2.csv with Excel.
a. What information is stored in this file?
The output from an evaperf all command connected May 31, 2006 from an
EVA5000. The file is a CSV file but each object class has its own headers
and format. The file starts with the first sample time and runs through each
object class until the next sampling period.
b. How would you track a specific virtual disk or disk group read miss latency
over the entire file capture?
Because of the different object class formats, it is hard to track a single LUN
or port from one sample period to the next. Collections for LUNs are on
vastly different rows.
Generating individual TLViz output files

7. Open the following files in Excel. What data do they contain?
a. ControllerStats.csv
Time, CPU%, Data%, CTRL, Serial Number, node name
b. HostsPort.csv
Time, name, read req/s, read MB/s, read latency (ms), write req/s, write
MB/s, write latency (ms), ave queue depth, ctrl, node
c. HostStats.csv
Time, host name, port, queue depth, busies, ctrl, node name.
d. LoopStats.csv
Time, name, type, status, loss of signal, bad rx char, loss of sync, link fail,
RX EOFa, discard frames, bad CRC, proto err, ctrl, node.

e. PhyDiskStats.csv
Time, Disk Group, Average Drive Queue Depth, Average Drive Latency
(ms),Average Read Req/s, Average Read MB/s, Average Read Latency
(ms), Average Write Req/s, Average Write MB/s, Average Write Latency
(ms) ,Number of Disks, Ctlr, Node
f. PhysicalDisk.csv
Time, ID, Drive Queue Depth, Drive Latency (ms), Read Req/s, Read MB/s,
Read Latency (ms), Write Req/s, Write MB/s, Write Latency (ms), Enc, Bay
Grp ID, Ctlr, Node
g. PresentedEVA.csv
Time, Total Host Req/s, Total Host MB/s, Node
h. PresentedLUNS.csv
Time, Device, Path ID, Target ID, LUN, Product ID, Product Rev, Ctlr, Serial,
Hardware Ver, Name, Node
i. Tunnel.csv
Time, Ctlr, Tunnel Number, Host Port, Source ID, Dest. ID, Round Trip Delay
(ms), Copy Retries per second, Write Retries per second, Copy In MB/s,
Copy Out MB/s, Write In MB/s, Write Out MB/s, Node
j. Vdisk.csv
Time, ID, Read Hit Req/s, Read Hit MB/s, Read Hit Latency (ms),Read Miss
Req/s, Read Miss MB/s, Read Miss Latency (ms), Write Req/s, Write
MB/s, Write Latency (ms), Flush MB/s, Mirror MB/s, Prefetch MB/s, Group
ID, Online To, Mirr, Wr Mode, Ctlr, LUN, Node
k. Vdiskstats.csv
Time, Disk Group, Total Read Hit Req/s, Total Read Hit MB/s, Average
Read Hit Latency (ms), Total Read Miss Req/s, Total Read Miss MB/s,
Average Read Miss Latency (ms), Total Write Req/s, Total Write MB/s,
Average Write Latency (ms), Total Flush MB/s, Total Mirror MB/s, Total
Prefetch MB/s, Ctlr, Node

8. Does this format of data allow you to more simply follow a specific virtual disk
or disk group over time? If not, why?
Yes, each class of object like Vdisk are all now grouped together, however, each
row contains a specific object. There are two rows for each object like Vdisk.
While better than the evaperf all output, it still is hard to review a single LUN
between samples.
9. What other files are created?
Files that start with TLVIZ-. These are files that can be directly opened with the
TLViz Viewer utility.
The mapping files, one for the virtual disks and one for the physical disks. These
files give a mapping from the TLVIZ display name to the Vdisk name or physical
disk shelf and bay.
Formatting and graphing TLViz output files

2. Note that the worksheet only shows one LUN but both controllers.
Why is there data from only one controller after a few minutes?
Data is actually displayed for both controllers. EVA2.csv displays this array
running 4.004 code (Active-active).
Running database queries

2. Run the query DiskGroup Percent Read by selecting Open or by double-clicking
the query.
a. How many disk groups are on this EVA?
Two, the default disk group and FATA disk group.
b. What is the percentage of read for both disk groups?
84.6% for the default disk group and 75.16% for the FATA group

c. Why is knowing the percentage of read useful in analyzing EVA

performance issues?
The random I/O performance of an EVA is governed by the following
variables:
 Number of drives in a disk group
 Type of drives (10K rpm, 15K rpm)
 Percentage of read
 Percentage of VRAID1 and VRAID5 LUNs.
With these variable, you can predict the total number of IOPs that a
backend disk group configuration can handle, considering the VRAID write
penalties.
3. Run the query Avg-KBperRead-Avg-KBPerWrite.
a. What is the average read and write KB for each disk group?
For default, 46.6 and 9.06. For FATA, 16.5 and 16.8.
b. Do these sizes seem odd?
FATA is reading and writing about equal.
c. Does EVAPerf use base 2 or base 10 numbers for megabytes?
Base 2.
d. Why is knowing the size of I/O transfers useful?
Random versus sequential.
4. Run the VDISK-Top10TotaRequsts query. What does this query tell us?
An EVA disk group configuration defines the response time of (latencies) of
requests. This query shows which Vdisks/LUNs are the 10 most active for total
read miss and write requests. The read miss and write requests are combined
sometimes and labeled “total backend requests.” These requests are ones that
must be serviced by the physical drives.
Sometimes it is important knowing which LUNs are contributing to the total
workload, because many times it is easier to reconfigure the LUN/application
than reconfigure the EVA to support the workload.

5. Run the VDISK-Top10ReadMissReq query. What does this query tell us?
This query reports the EVA, group, LUN, and total read miss requests, along with
the average and maximum read latencies.
Read misses are typically caused by random I/O and can impact the read miss
response times on the server. If read miss latencies are high, the user or
application on the server can experience “pauses” or sluggish performance.
Depending on the nature of the application, read misses can be normal, but
configuring the application and disk group sometimes helps. Many times, with
applications like SQL or Oracle databases, queries with improper indexes can
cause large amounts of read miss activity. Many times, changing the query is
the simplest resolution to high read miss latencies.
6. Run the VDISK-Top10WriteReq query. What does this query tell us?
This query reports the EVA, group, LUN, and total write requests, along with the
average and maximum write latencies.
Writes and the RAID type can double or quadruple physical disk activity.
VRAID1 has two physical disk writes for each host write. VRAID5 has two reads
and two writes for each random write. Knowing which volumes are heavy writes
and the type of VRAID is important to understanding which LUNs are
contributing to the workload of the physicals disks.

Module 10 — Lab 3

 Install and start the TLViz Viewer.
 Review data contained in the TLViz-formatted data files.
 For the TLViz-formatted files, manipulate the display by performing the following
tasks:
 Filter chart data.
 Remove and add items used for review.
 Compare data values using dual graphs.
 Zoom and un-zoom the display.
 Stack and un-stack items.
 Analyze charts for their implications on performance.


 TLViz Viewer
 Microsoft Excel
 Microsoft Access
 EVA-CD
This lab allows you to get started with EVA performance data analysis by using and
manipulating the TLViz Formatter output files. You will be able to take any individual
TLViz file in CSV format, review and analyze its contents, remove or add items to it,
do comparisons of data using graphs, and save charts.

Formatting and Analyzing TLViz Data
You must install the TLViz Viewer on your laptop or PC before using the utility. Follow
these instructions to install and start the TLViz Viewer:
1. Create a temporary directory.
2. Download the TLVIZ_1609_Kit.zip file to the directory.
Note
This file should be in the course files. If needed, your instructor will point you to
its location.
3. Extract the files to the directory.

4. Double-click the setup.exe file and go through the setup instructions.
5. Download any TLViz update zip files and extract the update to the C:\Program
Files\TLViz\ directory.
Note
The TLViz version should minimally be TLViz_V16-14.
6. Start the TLViz Viewer by using the Start menu.

The data from the EVAPerf array status (as) command is useful in giving the customer
a picture of the load on the array. The data represents the total MB/s and IOPs as
seen by the array. This file and output is useful in gauging the host workload on an
array. These counters do not account for multiple disk groups nor do they
differentiate between read or write MB or IOPs. They also do not show cache
hit/miss ratios.
Perform the following exercises and answer the questions:
1. Using TLViz Viewer, open the TLVIZ-Array.csv file.
Note
This file is available as part of the course files in the folder called 04-TLVIZ.
a. Which two items does this chart display?

...........................................................................................................
b. What is the peak MB/s and when did this occur?
...........................................................................................................
c. What is the maximum MB/s throughput for an EVA5000 with 100% read?
Note
Hint: use the Safe IOPs spreadsheet.
...........................................................................................................
d. What is the peak Req/s?
...........................................................................................................
2. Select both items (use the CTRL key) and zoom in to the time between 11:20 and
11:23. To zoom, use your mouse to click and drag a rectangular boundary on
the screen.
Note
If you need to unzoom the display, select the Undo Zoom/Scroll button.

3. From the menu, select Options → Cycle Dual Y or Dual Charts.

What happened to the display? What is on the left Y-axis and what is on the
right Y-axis?
................................................................................................................
What happened to the display?
................................................................................................................
5. Ensure that one item is selected and from the menu, select Options → Toggle
Moving Average.
What happened to the display? What is a moving average and how might this
be helpful in presenting data to a customer?
................................................................................................................
................................................................................................................
6. Close the TLVIZ-Array.csv file.

The data from the EVAPerf controller status (cs) command is useful for determining
the utilization of the EVA Power PC. The data is represented by total percentage of
CPU utilization and percentage of data. Subtracting the percentage of data from the
percentage of CPU yields the overhead of the array. This overhead is typically from
1–2%. Operations that contribute to overhead are leveling, splits, merges, VRAID5
parity calculations, and other background processes.
The counters are useful in a couple of ways:
 First, you would like to see utilization under 40% per controller. This is because if
an EVA loses a controller, the total workload of the array should be capable of
running on just one controller.
 Workload should be evenly distributed between controllers as much as possible.
This can be checked by comparing the percent CPU between controllers and the
percent data between controllers. If one controller has substantially higher
activity, then balancing the Vdisks between controllers can even out the
workload.
 Last, if overhead (percent CPU minus percent data) is greater than 1–2%, this is
typically an indication of “something” else going on within the array.
Configuration and controller logs should be checked along with other EVAPerf
data.
1. Using the TLViz Viewer, open the TLVIZ-CPU.csv file.
a. Which four items are displayed?
...........................................................................................................
b. What is the difference between PercentCPU and PercentData?
...........................................................................................................
2. Select and map PercentCPU and PercentData for controller T04K.
a. Why is there a difference between the PercentCPU and the PercentData?
...........................................................................................................
b. Approximately how much of a difference is there?
...........................................................................................................

3. Select and map PercentCPU and PercentData for controller F03S.

Again examine the difference between PercentCPU and PercentData and
describe here.
................................................................................................................
4. Select PercentCPU from each controller. Which controller is more highly utilized?
................................................................................................................
5. Add a new item for one controller representing the total overhead.
a. From the menu, select Options → Add New Item to List Box.
b. Select PROC F03S – PercentCPU for item #1.
c. For the operator, select Subtract.
d. Select PROC F03S – PercentData for item #2.
e. Enter the new item name, Controller F03S Overhead.
Note
Next you will repeat the above for the second controller.
6. Add a new item for the other controller representing the total overhead.
a. From the menu, select Options → Add New Item to List Box.
b. Select PROC T04K – PercentCPU for item #1.
c. For the operator, select Subtract.
d. Select PROC T04K – PercentData for item #2.
e. Enter the new item name, Controller T04K Overhead.
7. Map the overhead of both controllers.
a. Which controller has more overhead?
.........................................................................................................
b. Why would one controller have a higher percentage of overhead than the
other?
.........................................................................................................
8. Close the TLVIZ-CPU.csv file.

The data from the EVAPerf host port statistics (hps) command is valuable for
reviewing load balancing between controllers and ports. Statistics for IOPs and MB/s
are indications of load. Queues and latencies are useful in gauging the EVA’s ability
to respond to load.
Most people like to start with the host port statistics when reviewing EVAPerf data.
Because EVA performance is based on configuration and load of a disk group, and
an EVA can have multiple disk groups, it is best to start with the virtual disk group
statistics first and look at host port load balancing second.
1. Using the TLViz Viewer, open the TLVIZ-HostPort.csv file.
Which data is represented in this file?
................................................................................................................
2. From the menu, select Modify Item List → Remove Item(s) NOT containing string,
and use the string “ReadMB”.
Note
The search string is not case sensitive.
On the left side, which items are left?

................................................................................................................
3. Select all of the ReadMB.
How well load-balanced is the demand across all four ports during the peak
utilization?
................................................................................................................
4. Stack all of the ReadMB (use the Stack button).
How does the stacked value compare to the TLVIZ-Array.csv MB/s?
................................................................................................................
5. Un-stack the items.

6. From the menu, select Modify Item List → Revert to Previous Item List.
What happens to the display?
................................................................................................................
Note
Next you will repeat Steps 2 through 5 for write MB.
and use the string “WriteMB”.
................................................................................................................
8. Select all of the WriteMB.
utilization?
................................................................................................................
9. Stack all of the WriteMB.
................................................................................................................
10. Un-stack the items.
12. Remove all items not containing the string “Req”.
What is left in the display?
................................................................................................................
13. Stack all of these items.
How does this stacked value correspond to the TLVIZ-Array.csv host requests?
................................................................................................................
14. Un-stack these items.
How well-balanced are the requests between ports?
................................................................................................................

16. Remove all items not containing the string “Latency”.
17. Review the latencies for each port.
How do the latencies for reads and writes compare for each port?
................................................................................................................
18. Close the TLVIZ-HostPorts.csv file.

The data from the EVAPerf host connections (hc) command is valuable for
determining whether EVA write buffers can handle workload. This is quantified by the
number of busies.
1. Using the TLViz Viewer, open the TLVIZ-HostStats.csv file.
a. Which two items are represented in this file?
.........................................................................................................
b. What is the difference between a host queue item and a busy?
.........................................................................................................
c. Under which circumstances would the EVA respond back with a busy?
.........................................................................................................
2. Select busies for host LAB9.
How many entries are there for this host?
................................................................................................................
3. Open the HostStats.csv file in Excel (not TLViz Viewer).
a. Format this file as per the following:
1) Time in seconds.
2) Headers wrap.
3) Data filter is auto filter.
4) Locked header on row 2.
b. Select only host LAB9.
c. Scroll down to a time when LAB9 has 11 or more queued I/Os per port.

4. Compare the Excel values to the TLViz output.

Why don’t you have separate graphed lines for each port captured by EVAPerf?
................................................................................................................
Note
The simple answer is this is a bug with the TLViz Formatter. The TLViz Formatter
builds its items list based on HostName+Port. Since both ports show up with “-“,
the formatter cannot tell which port is which, therefore overwriting the values of
one with another. This will be fixed in a future release of the formatter.
5. From the menu, select Modify Item List → Remove Items with Zero Values.
Are there any busies displayed in the items list?
................................................................................................................
6. Close the HostStats.csv and TLVIZ-HostStats.csv files.

The data from the EVAPerf port status (ps) command is valuable for determining
issues on the bus. There should be no increases in any of the loop counters during a
run. The items collected in the loop statistics are only cleared by a reboot, resync or
evaperf rc command.
Note
The values are counters and typically will increase over time as errors are
logged. Some EVAPerf traces will indicate a value like 65535 throughout the
whole trace. It has not been determined if these values have an upper limit and
will not increment beyond the limit.

1. Using the TLViz Viewer, open the TLVIZ-LoopsStats.csv file.
a. Which items are displayed in this file?
.........................................................................................................
b. Which debug commands in Command View EVA would you use to retrieve
this same information?
.........................................................................................................
a. Which items are left in the display?
.........................................................................................................
b. Do these value change at all during the collection period?
.........................................................................................................
3. Close the TLVIZ-LoopsStats.csv file.

The data from the EVAPerf physical disk group statistics (pdg) command is valuable
for determining if the EVA load on a disk group is within guidelines. Typically, you
want to see the following:
 Average read and write requests combined:
 Under 120 for 10K drives
 Under 170 for 15K drives
 Disk queues close to an average of 1
 Average read and write MB/s combined should not exceed 4MB/s with
average IOPs above these numbers
The physical disk group command can be useful, but due to EVA sampling time
issues, it can also be very misleading. Depending on the number of physical disks,
sampling rate, and version of VCS, XCS, and EVAPerf, sometimes these statistics
show little or no load. As a best practice, it is advisable to check a sample of
physical disk activity to see if you can trust these statistics.
Note
On an EVA3000 or EVA5000 running VCS V3.X, you will find that physical disk
statistics are reported against only one controller. Also, on VCS V3.X, read and
write latencies are not reported. Instead, you will get a drive latency. This is
normal and is a summation of both controllers. On EVAs running active-active
code, you will get statistics from both controllers and no drive latency statistics,
however, VCS V4.X and XCS V5.X report per-drive read and write latencies.
You will see this discrepancy in a later lab.

1. Using the TLViz Viewer, open the TLVIZ-PhyDiskStats.csv file.
How many disk groups are on this array?
................................................................................................................

3. Select the following from controller F03S:

a. Average read MB/s
b. Average read Req/s
c. Average read latency
4. What are the average read requests at around 11:20?
................................................................................................................
5. Given these are 10K drives, are we exceeding the capability of the drives?
................................................................................................................
6. For the default disk group, select average queue depth on controller F03S.
a. During the peak of activity, what is the queue depth?
.........................................................................................................
b. Why is knowing the queue depth helpful in gauging workload?
.........................................................................................................
Note
Now you will repeat the steps above for controller T04K.
7. Select the following from controller T04K:

................................................................................................................
................................................................................................................
10. For the default disk group, select average queue depth on controller T04K.
During the peak of activity, what is the queue depth?
................................................................................................................
Note
Now you will determine the load, latency, and queue depth for both controllers
using the FATA disk group.

11. For the FATA disk group, select the following from controller F03S:
12. For the FATA disk group, select average queue depth on controller F03S.
................................................................................................................
13. For the FATA disk group, select the following from controller T04K:
14. For the FATA disk group, select average queue depth on controller T04K.
................................................................................................................
15. Answer these questions:
a. How is the average workload of the FATA compared to the default disk
group?
.........................................................................................................
b. Is the workload for the FATA drives within guidelines?
.........................................................................................................
16. Close the TLVIZ-PhyDiskStats.csv file.

The data from the EVAPerf physical disk (pd) command presents a lot of data. By
using TLViz and modifying the items you display, you can reduce the data to more
manageable comparisons. Normally, the physical disk output of EVAPerf is one of
the last things you should look at other than checking that the physical disk group
statistics are valid.
Note
On the EVA3000 or EVA5000 running VCS V3.X, you will find that physical disk
statistics are reported against only one controller. Also, on VCS V3.X, read and
write latencies are not reported. Instead, you will get a drive latency. This is
normal and is a summation of both controllers. On EVAs running active-active
code, you will get statistics from both controllers and no drive latency statistics,
however, VCS V4.X and XCS V5.X report per-drive read and write latencies.

1. Using the TLViz Viewer, open the TLVIZ-Physical-Disk.1.csv file.
2. Trim down the display to show only read MB, read latency, and read requests
from controller F03S:
a. From the menu, select Modify Item List → Remove Items with Zero Values.
b. From the menu, select Modify Item List and remove all items not containing
the string “Default”.
c. From the menu, select Modify Item List and remove all items not containing
the string “Read”.
d. From the menu, select Modify Item List and remove all items not containing
the string “F03S”.

3. Scroll down the item list.

a. Which items are left on the display?
.........................................................................................................
b. How do the read requests and read MB/s compare to the physical disk
statistics output from the previous section for the default disk group on
controller F03S?
.........................................................................................................
.........................................................................................................
4. Modify the items list one more time by removing all items not containing string
“ReadReq”.
5. Select all the items for display.
How many of the disks are displayed?
................................................................................................................
7. From the menu, select Modify Item List and remove all items not containing the
string “Latency”.
8. Select all items and zoom the display between 11:20 and 11:22.
9. Remove one disk at a time from the display by checking the box in the items list
at the bottom of the chart.
Reviewing this display, are there any drives that have much higher latencies than
others?
................................................................................................................
10. Open the PDDISK-MAP.txt file.
11. From the TLViz display, find PD-5 and look up PD-5 in the PDDISK-MAP.txt file.
Which shelf and bay is this drive located in?
................................................................................................................

12. Compare the latencies between PD-5 and PD-42 by performing the following
tasks:
a. Select both latencies from the items list.
b. From the menu, select Options → Toggle Moving Average.
c. Zoom in on both disks.
Does PD-5 have higher latency, and, if so, what could possibly cause this?
.........................................................................................................
.........................................................................................................
d. Open EVA1_16_113_12_35.xml in EVA-CD. Find the bay and shelf used by
PD-5.
Which role does this drive have?
.........................................................................................................
13. Close the TLVIZ-Physical-Disk.1.csv and PDDISK-MAP.txt files.

The data from the EVAPerf tunnel statistics (drt) command is valuable when
troubleshooting replication issues.
1. Using the TLViz Viewer, open the TLVIZ-Tunnel.csv file.
a. Are there any statistics recorded from tunnels?
.........................................................................................................
b. What is the difference between CopyInMB and WriteInMB?
.........................................................................................................
c. Is there any activity on these tunnels?
.........................................................................................................
d. If not, why does the RT-Delay increase?
.........................................................................................................
e. Do these increases correspond to the times when the EVA was under heavy
load?
.........................................................................................................
3. Close the TLVIZ-Tunnel.csv file.

The data from the EVAPerf virtual disk group statistics (vdg) command is an excellent
starting point to look at load and responsiveness. The number of drives within a disk
group will dictate the maximum load that a disk group can support and still deliver
acceptable latencies. Typically, a good starting point is to look at just latencies and
see if there are times when the following instances occur:
 Read miss latencies exceeding 20ms
 Write latencies exceeding 8ms
If there are samples that do exceed these, then chances are that the load is
exceeding the design capabilities of the configuration. When this happens, you must
look at load (MB/s, IOPs) along with background work (flushing, prefetch, leveling,
splits, mergers) to see which workload is causing the bottleneck.
Typically, the rule of thumb is that MirrorMB of around 80 on an EVA3000 or
EVA5000 will start to impact write latencies. You can see that you are at or near the
maximum bandwidth for the mirror port due to proxy read activity associated with
active-active configurations. You can see this proxy read in more detail looking at the
individual virtual disk files (TLVIZ-VDisk-#.csv).
1. Using the TLViz Viewer, open the TLVIZ-VDiskStats.csv file.
2. Modify the items list to display only the default disk group.
3. Select total backend requests from both controllers.
4. Stack these values.
How do the maximum values compare to the TLVIZ-Array.csv requests per
second?
................................................................................................................
5. Add read hit requests from both controllers into this value.
a. Now how do these values compare to the TLVIZ-Array.csv total requests?
.........................................................................................................
b. Why are read hit requests missing from total backend requests?
.........................................................................................................

6. Un-stack the display and compare the total backend requests between
controllers.
Are IOPs fairly balanced between controllers?
................................................................................................................
7. Add a new item to the display by using Options → Add New Item to List Box.
a. Select controller F03S total backend requests for item #1.
b. Select Add for the function.
c. Select controller T04K total backend requests for item #2.
d. Give the new item a name of Default Total Backend Requests.
8. Calculate Safe IOPs for the default disk group:
a. Open the EVADATA.mdb file and run the DiskGroup Percent Read query.
Record the percent read for both disk groups here.
.........................................................................................................
b. Open the EVA1_16_113_12_35.xml file in EVA-CD and record the number
and type of drives used in the default disk group.
.........................................................................................................
c. Open the Safe IOPs spreadsheet and input these numbers for number of
drives and percent read. Look at the appropriate disk type VRAID5 for total
IOPs.
What is the Safe IOPs number? Do the total backend requests exceed the
calculation? If so, by how much?
.........................................................................................................
9. Compare the new item (Default Total Backend Requests) to read miss latencies
on both controllers:
a. As requests go up, do latencies also go up?
.........................................................................................................
b. Are these latencies higher than 15ms when the Safe IOPs calculation is
exceeded?
.........................................................................................................

10. Select ReadMissMB from each controller:

How many MB/s are we pushing through this disk group?
................................................................................................................
................................................................................................................
11. From the menu, select Modify Item List → Revert to Previous Item List. The FATA
disk group should reappear.
12. Look at the total mirror MB/s by removing all items not containing the string
“MirrorMB”.
13. Stack the values for mirror MB/s.
What is the maximum throughput of the mirror port during this collection?
................................................................................................................
15. Add a new item to the chart called Array Total MirrorMB.
a. From the menu, select Options → Add New Item to List Box
1) Select controller F03S Default Disk Group MirrorMB for item #1.
2) Select Add for the operator.
3) Select controller F03S FATA Disk Group MirrorMB for item #2.
4) Enter the new item name, MirrorMB F03S.
b. From the menu, select Options → Add New Item to List Box
1) Select controller T04K Default Disk Group MirrorMB for item #1.
3) Select controller T04K FATA Disk Group MirrorMB for item #2.
4) Enter the new item name, MirrorMB T04K.
c. From the menu, select Options → Add New Item to List Box
1) Select MirrorMB F03S for item #1.
3) Select MirrorMB T04K for item #2.
4) Enter the new item name, Array Total MirrorMB.

16. Compare the Array Total MirrorMB with the write latency from each disk group
on each controller.
As MirrorMB goes up, do latencies also go up?
................................................................................................................
17. Compare Array Total MirrorMB to write MB from each disk group on each
controller.
Why is mirror port MB so much higher than write MB?
................................................................................................................
18. Close the TLVIZ-VDiskStats.csv file.

The data from the EVAPerf virtual disk statistics (vd) command contains all the values
of load and latency for each virtual disk within the EVA. The data is useful in
determining which Vdisks or LUNs contribute heavily to a disk group’s load. Many
times, identifying the Vdisk and load pattern can help with performance isolation or
resolution because an application may be contributing to the load. Therefore,
changing the application’s behavior may be more cost-effective than adding more
drives to a disk group.
Note
The TLVIZ-Vdisk-#.csv output is limited to 10MB each so you can more easily
open the files with TLViz. The consequences of this limitation are that it is hard to
profile a virtual disk over a large time period.

1. Using the TLViz Viewer, open the TLVIZ-Vdisk-1.csv file.
2. Remove all items with zero values.
3. Remove all items not containing the string “Backend”.
a. How many Vdisks were active during the sample period?
.........................................................................................................
b. How many Vdisks were being accessed using multipathing?
.........................................................................................................
c. How many Vdisks were being accessed by a single path?
.........................................................................................................
4. Select and map all of the drives.
Which drives have the highest amount of IOPs?
................................................................................................................

5. Revert to the previous Items list.

a. Remove all items not containing the string “Non-Replicated”.
b. Select WriteMB from each controller.
1) Are both controllers doing writes?
...................................................................................................
2) Which controller is doing more write MB/s?
...................................................................................................
3) Which item tells us which controller is the managing controller. Hint:
which controller has flush MB and why would only one controller have
flush MB?
...................................................................................................
...................................................................................................
6. Revert to the previous menu item.
a. Remove all items not containing the string “Req” (for requests).
b. Remove all items containing the string “Total”.
Which items are left in the items list?
.........................................................................................................
c. Select all of these items and stack them.
1) Does the total have the same pattern as the TLVIZ-Array.csv total
requests?
...................................................................................................
2) How many drives in how many disk groups are active?
...................................................................................................
7. Close the TLVIZ-Vdisk-1.csv file and any other files that you may have opened
during the lab.

Lab solutions are provided below for each corresponding topic.
Viewing and using array status information

1. Using the TLViz Viewer, open the TLVIZ-Array.csv file.
a. Which two items does this chart display?
Host MB/s and Total Host req/s
b. What is the peak MB/s and when did this occur?
253.79 @ 11:21:38 (select Options  Show Raw Chart Data)
c. What is the maximum MB/s throughput for an EVA5000 with 100% read?
525MB/s (from the Safe IOPS calculator)
d. What is the peak Req/s?
3980 @ 11:22:12 (select Options  Show Raw Chart Data)
What happened to the display? What is on the left Y-axis and what is on the
right Y-axis?
Left is the MB/s and right is the Total Host req/s.
What happened to the display?
TLViz displays two charts, one above the other.
5. From the menu, select Options → Toggle Moving Average.
What happened to the display? What is a moving average and how might this
be helpful in presenting data to a customer?
The moving average smooths out the data. For a certain period of time,
averages are calculated. This is useful for looking at a lengthy collection with a
small collection interval, however, the smoothing makes it possible to miss spikes
of work or latencies.

Viewing and using controller status information

1. Using the TLViz Viewer, open the TLVIZ-CPU.csv file.
a. Which four items are displayed?
Percent CPU and percent data for both controllers
b. What is the difference between PercentCPU and PercentData?
It is the percent of time the CPU is not idle. PercentData does not include
internal processes.
2. Select and map PercentCPU and PercentData for controller T04K.
a. Why is there a difference between the PercentCPU and the PercentData?
Because of the overhead on the CPU tending to all processes
b. Approximately how much of a difference is there?
About 1%
3. Select and map PercentCPU and PercentData for controller F03S.
Again examine the difference between PercentCPU and PercentData and
describe here.
About 1%
4. Select PercentCPU from each controller. Which controller is more highly utilized?
F03S
7. Map the overhead of both controllers.
a. Which controller has more overhead?
F03S
b. Why would one controller have a higher percentage of overhead than the
other?
Background processes

Viewing and using host port statistics information

1. Using the TLViz Viewer, open the TLVIZ-HostPort.csv file.
Which data is represented in this file?
Queue depth, read/write latency, read/write MB, read/write requests per host
port and per controller
and use the string “ReadMB”.
Read MB per host port per controller
3. Select all of the ReadMB.
utilization?
T04K has less load, but there is good load-balancing
4. Stack all of the ReadMB (use the Stack button).
Tracks very closely
What happens to the display?
Goes back to the previous list
and use the string “WriteMB”.
Write MB per host port per controller
8. Select all of the WriteMB.
utilization?
T04K has less load, but there is good load-balancing
9. Stack all of the WriteMB.
It is just a small portion of the total MB/s

12. Remove all items not containing the string “Req”.

What is left in the display?
Read and write requests per host port per controller
13. Stack all of these items.
How does this stacked value correspond to the TLVIZ-Array.csv host requests?
Tracks very closely
14. Un-stack these items.
How well-balanced are the requests between ports?
Reads and writes across ports are very well load-balanced
17. Review the latencies for each port.
How do the latencies for reads and writes compare for each port?
Read latencies are significantly higher across all ports.
Viewing and using host connections information

1. Using the TLViz Viewer, open the TLVIZ-HostStats.csv file.
a. Which two items are represented in this file?
Queue depth and busies
b. What is the difference between a host queue item and a busy?
Queue depth represents the average number of outstanding host requests
against all virtual disks accessed through the EVA host port. Busies
represent the number of busy responses sent to a host during the sample
interval.
c. Under which circumstances would the EVA respond back with a busy?
If some internal resource has reached a critical limit and the host requests
must be throttled. A high level of busies is a symptom of heavy write load
on the array.
2. Select busies for host LAB9.
How many entries are there for this host?
Two, each with 0 busies.

4. Compare the Excel values to the TLViz output.

Why don’t you have separate graphed lines for each port captured by EVAPerf?
TLViz Formatter bug. As of February 2007, this bug has not been fixed.
Are there any busies displayed in the items list?
No
Viewing and using port status information

1. Using the TLViz Viewer, open the TLVIZ-LoopsStats.csv file.
a. Which items are displayed in this file?
Loop and link counters by controller
b. Which debug commands in Command View EVA would you use to retrieve
this same information?
FCS_LINK_ERRORS (34) or FCS_DELTA_LINKS (36)
a. Which items are left in the display?
Device, host, and mirror status by controller
b. Do these value change at all during the collection period?
No
Viewing and using physical disk group information

How many disk groups are on this array?
Two
36
No

6. For the default disk group, select average queue depth on controller F03S.
a. During the peak of activity, what is the queue depth?
It is 69 at 11:27.
b. Why is knowing the queue depth helpful in gauging workload?
If there is a bottleneck, it helps determine if the workload is sufficient for the
array or if server-side tuning of queue depths is needed.
6
No
10. For the default disk group, select average queue depth on controller T04K.
Zero
12. For the FATA disk group, select average queue depth on controller F03S.
Zero
14. For the FATA disk group, select average queue depth on controller T04K.
Zero
15. Answer these questions:
a. How is the average workload of the FATA compared to the default disk
group?
Much lower
b. Is the workload for the FATA drives within guidelines?
Yes

Viewing and using physical disk information

a. Which items are left on the display?
ReadMB, read requests, and read latencies for all physical disks and
controller F03S.
b. How do the read requests and read MB/s compare to the physical disk
statistics output from the previous section for the default disk group on
controller F03S?
The average of all the physical disks are about the same as the previous
totals.
5. Select all the items for display.
How many of the disks are displayed?
16 of the 24 (16 is a TLViz display maximum)
9. Remove one disk at a time from the display by checking the box in the items list
at the bottom of the chart.
Reviewing this display, are there any drives which have much higher latencies
than others?
PD-4, PD-5 (the highest), PD-38, PD-41, PD-42
11. From the TLViz display, find PD-5 and look up PD-5 in the PDDISK-MAP.txt file.
Which shelf and bay is this drive located in?
Enclosure 8, bay 14
12. Compare the latencies between PD-5 and PD-42 by performing the following
tasks:
c. Zoom in on both disks.
Does PD-5 have higher latency, and, if so, what could possibly cause this?
Yes, possibly a higher request rate.
d. Open EVA1_16_113_12_35.xml in EVA-CD. Find the bay and shelf used by
PD-5.
Which role does this drive have?
Quorum

Viewing and using tunnel statistics

a. Are there any statistics recorded from tunnels?
Yes
b. What is the difference between CopyInMB and WriteInMB?
CopyInMB is the rate at which data is copied to an array to populate the
members of a DR group with data when an initial or full copy is requested.
WriteInMB is the rate at which data is written to an array because of write
activity to the members of the source array. The write activity includes host
writes, merges, and replication retries.
c. Is there any activity on these tunnels?
No
d. If not, why does the RT-Delay increase?
Destination controller was busy.
e. Do these increases correspond to the times when the EVA was under heavy
load?
Yes
Viewing and using virtual disk group statistics

4. Stack these values.
How do the maximum values compare to the TLVIZ-Array.csv requests per
second?
Very nearly equal at the peak.
5. Add read hit requests from both controllers into this value.
a. Now how do these values compare to the TLVIZ-Array.csv total requests?
About the same
b. Why are read hit requests missing from total backend requests?
Because the requests are going to cache, not to disk.
6. Un-stack the display and compare the total backend requests between
controllers.
Are IOPs fairly balanced between controllers?
F03S is a little lower during the peak

8. Calculate Safe IOPs for the default disk group:

a. Open the EVADATA.mdb file and run the DiskGroup Percent Read query.
Record the percent read for both disk groups here.
Default disk group is 84.7%, FATA disk group is 75.2%
b. Open the EVA1_16_113_12_35.xml file in EVA-CD and record the number
and type of drives used in the default disk group.
24, Fibre Channel drives (model # BD14655B2A is a 146GB 10K drive)
c. Open the Safe IOPs spreadsheet and input these numbers for number of
drives and percent read. Look at the appropriate disk type VRAID5 for total
IOPs.
What is the Safe IOPs number? Do the total backend requests exceed the
calculation? If so, by how much?
Safe IOPs is 1,974. Yes, by approximately 1800–2000 IOPs.
9. Compare the new item (Default Total Backend Requests) to read miss latencies
on both controllers:
a. As requests go up, do latencies also go up?
Yes
b. Are these latencies higher than 15ms when the Safe IOPs calculation is
exceeded?
Yes
10. Select ReadMissMB from each controller:
How many MB/s are we pushing through this disk group?
On average, about 33, but up to 113.
13. Stack the values for mirror MB/s.
What is the maximum throughput of the mirror port during this collection?
Approximately 114MB/s

16. Compare the Array Total MirrorMB with the write latency from each disk group
on each controller.
As MirrorMB goes up, do latencies also go up?
Yes
17. Compare Array Total MirrorMB to write MB from each disk group on each
controller.
Why are mirror port MB so much higher than write MB?
By reviewing EVA CD, the customer has V4.004 and is using VRAID5. Proxy
reads and VRAID5 will cause higher mirror port traffic.
Viewing and using virtual disk group statistics

3. Remove all items not containing the string “Backend”.
a. How many Vdisks were active during the sample period?
Six
b. How many Vdisks were being accessed using multipathing?
Four
c. How many Vdisks were being accessed by a single path?
Two
4. Select and map all of the drives.
Which drives have the highest amount of IOPs?
ALF-9 Non-Replicated Disk
5. Revert to the previous Items list and select WriteMB from each controller.
1) Are both controllers doing writes?
Yes
2) Which controller is doing more write MB/s?
F03S
3) Which item tells us which controller is the managing controller. Hint:
which controller has flush MB and why would only one controller have
flush MB?
F03S has FlushMB

6. Revert to the previous menu item.

b. Remove all items containing the string “Total”.
Which items are left in the items list?
Read miss requests, read hit requests, write requests
c. Select all of these items and stack them.
1) Does the total have the same pattern as the TLVIZ-Array.csv total
requests?
Yes
2) How many drives in how many disk groups are active?
Two drives in FATA disk group, four drives in default disk group


Module 10 — Lab 4

 Install and start the following performance analysis tools:
 PerfMonkey
 EVApnggen
 Use PerfMonkey to:
 Navigate the interface
 Load performance data
 Remove and add items used for review
 Use various charts and graphs
 Filter chart data
 Use EVApnggen to:
 Generate charts
 Open and review charts

 PerfMonkey
 EVApnggen

This lab allows you to use some additional analysis tools for EVA performance data
analysis. The current set of additional tools consists of PerfMonkey and EVApnggen.
The first part of the lab allows you install, view, and use the PerfMonkey interface.
The second part of the lab allows you to install and use EVApnggen to generate
performance charts.

Using Additional Performance Analysis Tools
For this portion of the lab, you will be installing PerfMonkey and then viewing and
using the tool.
Installing and starting PerfMonkey

To install and start PerfMonkey:
1. Copy the installation zip file to your PC or laptop.
Note
The latest version of this file is 2.5.14, available through PerfMonkey 2.5.14.zip. If
the instructor makes the setup file available, just copy setup.exe.
2. If required, unzip the file.

3. Double-click setup.exe to begin the installation.
Typically, you need Windows Installer 3.1 and the .NET Framework 2.0 for
installation. PerfMonkey will determine if you have these components. If you
have an active internet connection, PerfMonkey will acquire them from the
Microsoft site.
Note
If these components are not available, ask your instructor how to acquire them.
4. Run PerfMonkey by using the start menu, for example, select All Programs 
PerfMonkey  PerfMonkey.
Navigating the PerfMonkey display

Navigate the PerfMonkey GUI by doing the following:
1. View the three primary panes of the default layout, including Counter Explorer,
Groups, and the document area.
2. In Counter Explorer, check the options with the down arrow, then toggle the Auto
Hide option and note the result.
3. Save a different layout by selecting View  Save Layout as.
4. Manipulate the layout in some way.

5. Load the saved layout by selecting View  Load Layout.

6. Use the menu bar to view all of the menu choices.
7. Use the mouseover function to view all of the options in the toolbar.
Using the counter functions

To use the counter functions within Counter Explorer:
1. Open a sample TLViz-formatted file, for example, the TLVIZ-CPU.csv file from your
earlier lab.
Note
2. Expand the levels of the Counter Explorer hierarchy to view the counters.
3. Practice selecting a single counter, multiple counters, or a group of counters.
4. Double-click any counter to display the data.
5. Create a new calculated counter by doing the following:
a. Select Tools  Add Calculated Counter.
b. Drag the counter PercentCPU for controller F03S to Value 1 and drag the
counter PercentData for the same controller to Value 2.
c. Choose subtraction for the operator and give the name of the new counter,
for example, F03S Overhead.
d. Click the Create button and note the calculated counter in the Counter
Explorer.
Using the charts

To create and use the charting functions:
1. Use any of the following methods to create a line chart:
a. Use the toolbar drop-down menu and select Line Chart.
b. Double-click any single counter in the hierarchy.
c. Drag any single counter or group of counters in the hierarchy to the line
chart.
2. At the bottom of the line chart, select and unselect items to change the chart.
3. Click the Chart Options button above the chart to view chart properties and
settings.

4. Change the color theme to WarmTones, show the 95th percentile, and close the
properties display.
5. Clear the current chart by using the toolbar.
6. Chart the calculated counter you created earlier for F03S Overhead.
7. Right-click in the chart to add an annotation.
8. Save the file as a .png file.
9. Open the .png file you saved, view it, then close the file.
10. Plot counters on different scales by doing the following:
a. Open TLVIZ-VDiskStats.csv.
Note
b. Clear the current chart.

c. Drag the counters for ReadMissLatency and TotBackEndReq to the middle of
the chart and note how the results are displayed.
d. Clear the current chart.
e. Drag the counter for ReadMissLatency to the left axis and drag the counter
for TotBackEndReq to the right axis and note how the results are displayed.
11. Zoom into the display by doing the following:
a. Within the chart, hold down the left mouse button.
b. When the red line appears, drag to select the zoom area.
c. Reset the zoom by selecting the small icon on the left of the scrollbar ( ).
12. Use the timeline filter by doing the following:
a. Zoom in the chart to define the window.
b. Select Filter Time Line from the PerfMonkey toolbar.
c. Mouse over the Filter Time Line button to see what the current status is.
d. Turn off the timeline filter.

For this portion of the lab, you will be installing EVApnggen and then using the tool
to generate charts.
Installing EVApnggen
To install EVApnggen:
1. Create a folder on your PC or laptop to hold EVApnggen installation files.
2. Copy the installation zip file to your folder.
Note
The latest version of this file is EVAPNGGEN1_13.zip. If the instructor makes the
csvpng.exe and evapnggen.bat files available, just copy them to your folder.
3. If required, unzip the file to your folder.
Generating charts
To generate charts:
1. Copy EVApnggn.bat and csvpng.exe to the folder containing your EVAPerf data
you used earlier.
Note
2. From the command line prompt, navigate to the folder and enter EVApnggen.
Note the command line output.
Note
Optionally, within Windows Explorer, double-click EVApnggen.bat. This will start
the command line.
3. In Windows Explorer, navigate to your folders and note the files that were
created.

Opening and reviewing the charts

Now you can open and review the charts you created to determine the most useful
for your analysis. To do so:
1. Double-click any one of the generated files.
2. Review the data in the graph.
3. Based on other tools you have used, which charts would you find the most useful
for analyzing performance?
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................
................................................................................................................


LG 30apr09

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

LG 30apr09

Uploaded by

Copyright:

Available Formats

Course ID: 16912

Module 1 Lab 2 — Using the M6412A Drive Enclosure Serial Port

Module 6 Lab 1 — Using HSV Controller Diagnostics

Module 6 Lab 2 — Analyzing FCS and SCS Outputs

Rev. 9.21 HP Restricted 1

Analyzing FCS_DELTA_LINKS outputs ..............................................................39

Module 8 Lab — Using HP Navigator

Module 9 Lab 1 — Generating and Correlating Events

Module 9 Lab 2 — Analyzing Case Studies

2 HP Restricted Rev. 9.21

Case study 2 ..................................................................................................6

Rev. 9.21 HP Restricted 3

Case study 14............................................................................................... 41

Module 10 Lab 1 — Using EVAPerf

Module 10 Lab 2 — Using TLViz Formatter Output Files

4 HP Restricted Rev. 9.21

Viewing an EVAPerf output file in Excel ..............................................................3

Module 10 Lab 3 — Formatting and Analyzing TLViz Data

Module 10 Lab 4 — Using Additional Performance Tools

Rev. 9.21 HP Restricted 5

Using PerfMonkey ...........................................................................................3

6 HP Restricted Rev. 9.21

After completing this lab, you should be able to:

The following are the requirements for performing this lab:

 An EVA 3000/5000 series storage system with a current version of VCS

Rev. 9.21 HP Restricted L1.1 – 1

L1.1 – 2 HP Restricted Rev. 9.21

To connect the EMU console port cable:

EMU serial cable

Rev. 9.21 HP Restricted L1.1 – 3

On your PC or laptop, do the following:

Connection property Setting

L1.1 – 4 HP Restricted Rev. 9.21

To use the EMU console commands:

--- Main Menu ---

2. Select ? – Code Version Information to determine the current EMU firmware

Rev. 9.21 HP Restricted L1.1 – 5

9. Enter ESC, then S to enter the System menu.

L1.1 – 6 HP Restricted Rev. 9.21

## ….WWN…. Pos-A/B AL-PA Cp Rate FRev Min/Temp/Max EDV Byp-A/B

Rev. 9.21 HP Restricted L1.1 – 7

L1.1 – 8 HP Restricted Rev. 9.21

Current time stamp is: 0088838585 (054B91B9h)

Rev. 9.21 HP Restricted L1.1 – 9

You have completed this lab.

L1.1 – 10 HP Restricted Rev. 9.21

After completing this lab, you should be able to:

The following are the requirements for performing this lab:

 An EVA 4400/6400/8400 storage system with a current version of XCS

Rev. 9.21 HP Restricted L1.2 – 1

L1.2 – 2 HP Restricted Rev. 9.21

To connect the M6412x I/O module serial port cable:

Rev. 9.21 HP Restricted L1.2 – 3

On your PC or laptop, do the following:

Connection property Setting

L1.2 – 4 HP Restricted Rev. 9.21

To use the most common I/O module serial commands:

Rev. 9.21 HP Restricted L1.2 – 5

L1.2 – 6 HP Restricted Rev. 9.21

Read the enclosure event log by doing the following:

7. Select File  Open  Workspace to open a new workspace (.nw). An empty

Rev. 9.21 HP Restricted L1.2 – 7

L1.2 – 8 HP Restricted Rev. 9.21