You are on page 1of 56

HP 10500 Switch Series (Comware V5)

Troubleshooting Guide

Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained


herein is subject to change without notice. The only warranties for HP products and services are
set forth in the express warranty statements accompanying such products and services. Nothing
herein should be construed as constituting an additional warranty. HP shall not be liable for
technical or editorial errors or omissions contained herein.

Part number: 5889-4907

Contents
General troubleshooting procedures 1
Obtaining diagnostic information 1
Obtaining other information 2
Checklist for deployment 2
Troubleshooting hardware 7
MPU reboot failure 7
Symptom 7
Troubleshooting flowchart 7
Solution 8
Operating power supply failure 8
Symptom 9
Solution 9
Newly-installed power supply failure 9
Symptom 9
Solution 9
Fan tray failure 10
Symptom 10
Solution 10
Related commands 11
Troubleshooting system management 11
Temperature alarming 11
Symptom 11
Troubleshooting flowchart 12
Solution 12
Related commands 13
Troubleshooting ports 13
10/100/1000Base-T copper port fails to go up 13
Symptom 13
Troubleshooting flowchart 14
Solution 14
100/1000-Mbps SFP fiber port fails to go up 15
Symptom 15
Troubleshooting flowchart 16
Solution 16
10-Gigabit XFP fiber port fails to go up 18
Symptom 18
Troubleshooting flowchart 19
Solution 19
10-Gigabit SFP+ fiber port fails to go up 21
Symptom 21
Troubleshooting flowchart 22
Solution 23
40-GE QSFP+ fiber port fails to go up 24
Symptom 24
Troubleshooting flowchart 25
Solution 25
40-GE CFP fiber port fails to go up 26
i

Symptom 26
Troubleshooting flowchart 27
Solution 28
Error frames (for example, CRC errors) on a port 28
Symptom 28
Troubleshooting flowchart 29
Solution 29
Failure to receive packets 31
Symptom 31
Troubleshooting flowchart 31
Solution 31
Failure to send packets 33
Symptom 33
Troubleshooting flowchart 33
Solution 33
Related commands 34

Troubleshooting IRF 35
IRF fabric establishment failure 35
Symptom 35
Troubleshooting flowchart 36
Solution 37
IRF split 38
Symptom 38
Troubleshooting flowchart 39
Solution 39
BFD MAD failure 40
Symptom 40
Troubleshooting flowchart 41
Solution 41
LACP MAD failure 43
Symptom 43
Troubleshooting flowchart 44
Solution 44
Related commands 45
Troubleshooting QoS and ACL 46
ACL application failure for unsupported ACL rules 46
Symptom 46
Troubleshooting flowchart 47
Solution 47
ACL application failure for insufficient resources 47
Symptom 47
Troubleshooting flowchart 48
Solution 48
ACL application failure without an error message 49
Symptom 49
Troubleshooting flowchart 49
Solution 49
Packet loss or forwarding failure 50
Symptom 50
Troubleshooting flowchart 51
Solution 51
Related commands 53

ii

General troubleshooting procedures


This chapter describes how to obtain information for troubleshooting problems. This document is not
restricted to specific software or hardware versions.

Obtaining diagnostic information


Diagnostic information is important for locating and solving problems. You can execute the display
diagnostic-information command to obtain diagnostic information.
HP recommends that you execute the display diagnostic-information command over Telnet instead of the
console port because information collection through the console port (9600 bps) takes a long time.
Information collection on an IRF fabric takes more time than on a single device.
To obtain diagnostic information:
<Sysname> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:y
Please input the file name(*.diag)[flash:/default.diag]:aa.diag
Diagnostic information is outputting to flash:/aa.diag.
Please wait.................
Save succeeded.

The following describes the output information above:


save or display diagnostic information (Y=save, N=display)? [Y/N]:

This prompt asks you whether to save diagnostic information into a .diag file in the flash memory.

If the free space of the flash memory is larger than 10 MB, HP recommends that you enter y to save
diagnostic information into a .diag file. After you enter y, the following prompt appears:
Please input the file name(*.diag)[flash:/default.daig]:

Enter a name for the .diag file at the prompt. If you press Enter, the default name default.diag is
used. You can enter a name to differentiate between different .diag files.
If the name you entered already exists, the following prompt appears:
The file already exists, overwrite it? [Y/N]

If you enter y, the new file overwrites the old file. If you enter n, the system returns to the view in
which you executed the display diagnostic-information command. You need to re-execute the
display diagnostic-information command and enter a new name for the .diag file.
Please wait.................

The system keeps printing dots during saving the diagnostic information. If a fault occurs, the
interval between dots will increase, but it will not exceed 2 minutes.
After the system saves the diagnostic information, you can execute the dir command in user view
to verify whether the diagnostic file exists in the flash memory.
<Sysname>dir
Directory of flash:/
0

-rw-

6797

Jun 14 2012 10:37:42

startup.cfg

drw-

Sep 03 2010 11:03:06

patch6613

-rw-

3395

May 13 2013 13:07:13

system.xml

-rw-

429516

Aug 19 2010 15:59:51

mpu.btw

-rw-

13090

May 13 2013 13:07:03

config.cwmp

drw-

Apr 09 2010 13:47:34

seclog

-rw-

33339502

May 13 2013 12:59:04

10500-cmw520-r1208.bin

-rw-

21288

Aug 09 2012 12:36:27

10500-cmw520-r1208.bin

-rw-

468380

Jan 14 2013 09:44:36

lsqsrp2xb05500.btw

724660

May 03 2013 14:05:15

default.diag

drw-

If the diagnostic file is larger than 10000 bytes and the saving time is correct, the saving operation
is successful. If the diagnostic file has a size of 0 (-1) or the saving time is incorrect, the saving
operation failed, and you need to configure the system to print the diagnostic information on the
console. If the flash memory has no more space during the saving process, the system prints the
exceeding diagnostic information on the console.
You can use FTP to download the diagnostic file to a PC and send the file to HP Support.

If you enter n at the prompt "save or display diagnostic information (Y=save, N=display)? [Y/N]:",
the system prints all diagnostic information on the console. Before you select n, enable information
capture on the HyperTerminal or Telnet window.
If you are using the console port, the information collection takes about 20 minutes. If you are
using Telnet, the information collection takes about 3 minutes.
After you obtain the diagnostic information, send the file to HP Support.

Obtaining other information


In addition to diagnostic information, the following information is also helpful for troubleshooting:

LED state information, such as operation and alarm LEDs, power LEDs, fan LEDs, and port LEDs.

Operation information about other relevant devices, configuration information, and log files.

Your opinions on this problem.

Measures taken on site and the results.

Solution to the problem if the problem has been solved.

Checklist for deployment


The following checklist serves as a deployment guide to help you eliminate potential configuration errors
at different deployment sites. Select items according to your deployment environment.
Table 1 Checklist
Item

Environment
and card
hardware
status

Sub-item

Environment

Command or
method
display environment

Result

Remarks

OK

The temperature of all MPUs


and interface cards must be
lower than 70C (158F).

NOK
Not related
OK

Fan

display fan

NOK
Not related

The fans must be operating


correctly.

Item

Sub-item

Command or
method

Result
OK

Power supply

display power

NOK
Not related

LED

Card operating
status

Active and
standby MPUs

CPU usage

Memory
usage

Are the software


versions of the
active and standby
MPUs consistent?

Examine the status of


the RUN LEDs and
alarm LEDs on all
cards.

NOK
Not related
OK

display device

NOK
Not related
OK

display boot-loader

NOK
Not related
OK

Does the standby


MPU have a
configuration file?

dir slot#

Does the CPU


usage keep
changing between
10% and 60% or
stay at a high level
(the CPU usage of
the MPU or service
cards exceeds
60%)?

Execute the display


cpu command
repeatedly.

Does the memory


usage of the MPU
and service cards
exceed 60%?

OK

NOK
Not related

OK
NOK
Not related

OK
display memory slot

NOK
Not related

Is the port
operating in
half-duplex mode?

display interface
brief

Is flow control
enabled on the
port when not
necessary?

Check the
configuration to see
if flow control is
enabled.

Port

OK
NOK
Not related
OK
NOK
Not related

Remarks
The power supplies must be
operating correctly. (Evaluate
power backup by referring to
the power and system power
consumption.)
The RUN LED flashes slowly,
and the alarm LED is off.
All service cards must be
operating correctly, and the
MPUs are in active and
standby states.
Make sure the software
versions of the active and
standby MPUs are consistent.
If no configuration file exists,
execute the save command to
save the configuration file.

Execute the debug ip packet


command to view the packets
delivered to the CPU and
analyze the cause.

If the memory usage exceeds


60%, execute the display
memory command to view
memory usage and
troubleshoot the module that
occupies the most memory.
For example, if a port is
operating in half-duplex
mode, verify that the
configurations on the two
ends are consistent.
Use the undo flow control
command to disable flow
control on the port.

Item

Sub-item

Command or
method

Are large numbers


of error packets
generated in the
outbound/inbound
direction of the
port?

Use the display


interface command
to verify that a large
number of errors are
generated and that
the number keeps
increasing.

Result

Remarks
Do the following:

OK

1.

Examine link quality and


the optical-electrical
converter.

2.

Verify that the


configurations on the two
ends are consistent.

NOK
Not related

Do the following:

Does the port


change to
up/down
frequently?

Are the
configurations on
the local and peer
fiber ports
consistent?
Fiber port

Trunk port
configurations

Does CRC error


occur on the fiber
port? Is the number
of CRC errors
increasing?

OK
display logbuffer

1.

Examine the link and the


optical-electrical
converter.

2.

Verify that the optical


power of the GE port has
reached the threshold
value.

3.

Verify that the


configurations on the two
ends are consistent.

NOK
Not related

display current
interface

OK
NOK
Not related

OK
display interface

NOK
Not related

Is the undo port


trunk permit vlan 1
command
configured on the
trunk port?

display current
interface

Is the PVID of the


trunk port
consistent with the
peer port?

display current
interface

Are the VLANs to


which the port is
assigned
consistent with the
peer port?

display current
interface

OK
NOK
Not related

OK
NOK
Not related
OK
NOK
Not related

To connect an HP device to a
device of another vendor, HP
recommends that you set
consistent fiber port rate and
duplex mode on the two ends.
Verify that the optical power
has reached the threshold
value. To resolve this issue,
replace the transceiver
module or pigtail fiber, or
clean the connector of the
transceiver module.
If GVRP is configured for the
system and the undo port
trunk permit vlan 1 command
is configured for the trunk
port, configure the PVID of the
trunk port as the ID of a VLAN
permitted by the trunk port.
The VLANs permitted by the
trunk ports and the PVIDs of
the trunk ports connecting the
two devices are consistent.
The VLANs permitted by the
trunk ports connecting the two
devices are consistent.

Item

Sub-item

Command or
method

Are the ports


connecting two
devices configured
as a trunk port and
an access port?

display current
interface

Does a loop occur


in VLAN 1?

Use the display


interface command
to verify that the trunk
ports of all devices
are assigned to
VLAN 1.

Timeout factor

Is the port on the


device connected
to the PC
configured as an
edge port?

STP

Can the HP device


running
MSTP/STP/RSTP
communicate with
the Cisco device
running PVST+?

Are there too many


overlapping paths
among the
topologies of
different MSTIs?

display
current-configuration

Use the display


current interface
command to
examine the port
configurations. If an
edge port is
configured, stp
edged-port enable is
displayed.

Verify that STP


calculation on each
device is correct.

Use the display


current interface
command to
examine port
configurations.

Result
OK
NOK
Not related

OK
NOK
Not related

OK
NOK
Not related

OK
NOK
Not related

OK
NOK
Not related

OK
NOK
Not related

Remarks

Configure the same link type


for the ports as needed.

Remove the ports from VLAN


1 as needed.

Verify that the stp timer-factor


command is configured. If
not, configure a timeout factor
in the range of 5 to 7 to
enhance the stability of STP.
HP recommends that you
configure the port on the
device connected to the PC as
an edge port or disabling STP
on the port. To prevent state
change of the ports from
interrupting STP calculation,
disable STP on the ports
connected to the devices that
do not support STP.
To avoid communication
between the HP device
running MSTP/STP/RSTP and
the Cisco device running
PVST+, HP recommends that
you change the connection
mode to Layer 3 connection.
Plan VLANs and
VLAN-to-instance mappings
according to your network
environment in order to do the
following:

Forward traffic of various


VLANs along different
paths.

Avoid too many

overlapping paths among


the topologies of different
MSTIs.

Item

VRRP

Sub-item

Command or
method

Does a TC attack
exist to cause
frequent STP status
changes on the
port?

Use the display stp tc


and display stp
history commands to
examine the statistics
of TC/TCN BPDUs
received and sent on
the port and the
historical port role
calculation
information.

Is the handshake
interval set to 3
seconds? Are the
handshake
intervals of the two
ends consistent?

Are the router IDs


of two devices the
same?

Result

OK
NOK
Not related

OK
display vrrp

NOK
Not related

OK
display ospf peer

NOK
Not related

Are there a lot of


errors from the
output of the
display ospf error
command?

display ospf error

Does severe route


flapping occur?

Use the display ip


routing-table
statistics command to
examine the number
of added and
deleted routes.

OK
NOK
Not related

OSPF

Is the OSPF status


stable?

ARP

Are there a lot of


ARP entry
conflicts?

OK
NOK
Not related

OK
display ospf peer

NOK
Not related
OK

display logbuffer

NOK
Not related

Remarks
Make sure the stp edged-port
enable command is
configured on the port of the
device connected to the PC or
otherwise STP is disabled.
Disable STP on the ports
connected to the devices that
do not support STP.
Change the handshake
interval to 3 seconds if the
number of VRRP groups is less
than five. If five or more VRRP
groups exist, assign three or
five VRRP groups into one
group, and configure the
handshake interval as 3
seconds, 5 seconds, and 7
seconds for each group.
A router ID conflict might
result in a route learning error.
To restart route learning, you
must modify the router ID and
execute the reset ospf process
command.
If a large amount of OSPF
errors exist and the number
continues to increase, execute
the debugging ospf event
command for further analysis.
If route flapping occurs, locate
the flapping route and the
source device to analyze the
cause. You can use the
display ospf lsdb command
multiple times to view the age
of routes and locate the
flapping route.
View the up time of the OSPF
neighbor.
Verify that the IP address of
the host does not conflict with
other IP addresses. If a conflict
occurs, modify the IP address
of the host.

Item

Route

Sub-item

Command or
method

Is the default route


correct? Is there
any routing loop?

Trace the path to a


nonexistent network
(1.1.1.1, for
example) by using
the tracert command
to check routing
loops. Use the debug
ip packet command
to check if packets
with TTL 0 or 1 exist.

Result

OK
NOK
Not related

Remarks
If a routing loop exists, verify
the configurations of involved
devices. Adjust the route to
eliminate the loop. If TTL
exceeded packets are
received, verify that the
corresponding network route
is correct.

Troubleshooting hardware
This section provides troubleshooting information for common hardware problems.
NOTE:
This section describes how to troubleshoot MPUs, LPUs, power supplies, and fan trays. To troubleshoot
transceiver modules, ports, and temperature alarms, see "Troubleshooting system management" and
"Troubleshooting ports."

MPU reboot failure


Symptom
An MPU fails to reboot.

Troubleshooting flowchart

Figure 1 Troubleshooting MPU reboot failure

Solution
To resolve the problem:
1.

Verify that the system software image on the MPU is correct.


a. Log in to the MPU through the console port and restart the switch. If the system reports that a
CRC error occurs or that no system software image is available during the BootWare loading
process, reload the system software image.
b. Verify that the system software image in the flash memory is the same as the one on the server.
If no system software image is available in the flash memory, or if the image is different than
the one on the server, reload the system software image. Then set the reloaded system software
image to the current system software image. The system software image in the flash memory is
automatically set to the current system software image during the BootWare loading process.

2.

Verify that the MPU memory is running correctly.


Reboot the MPU, and immediately press CTRL+T to detect the memory. If a memory fault is detected,
replace the MPU.

3.

Verify that no error is reported during the BootWare loading process.


If there are still errors reported during the BootWare loading process, identify the faulty module
and verify that the MPU is installed securely.

4.

If the MPU is not securely installed, remove and reinstall the MPU.

If the MPU is installed securely, replace the MPU.

If the problem persists, contact HP Support.

Operating power supply failure


8

Symptom
A trap is generated indicating that an operating power supply has failed.

Solution
To resolve the problem:
1.

Execute the display power command to display power supply information.


<HP> display power
Chassis 1:
Power

1 State: Absent

Power

2 State: Normal

Chassis 2:
Power

1 State: Normal

Power

2 State: Absent

If the power supply is in Absent state, go to step 2. If the power supply is in Fault state, go to step 3.
2.

Remove and reinstall the power supply to make sure the power supply is installed correctly. Then,
execute the display power command to verify that the power supply has changed to Normal state.
If the power supply remains in Absent state, replace the power supply.

3.

A power supply might run into Fault state when it operates continuously at high temperatures or
when it is faulty.
a. Verify that the power supply surface is clean. If there is dust accumulated on the power supply,
remove the dust. Then remove and reinstall the power supply. Execute the display power
command to verify that the power supply has changed to Normal state. If the power supply
remains in Fault state, go to step b.
b. Install the power supply into an empty power supply slot. Then execute the display power
command to verify that the power supply has changed to Normal state in the new slot. If the
power supply remains in Fault state, replace the power supply.

4.

If the problem persists, contact HP Support.

Newly-installed power supply failure


Symptom
A newly-installed power supply fails.

Solution
To resolve the problem:
1.

Execute the display power command to display power supply information.


<HP> display power
Chassis 1:
Power

1 State: Absent

Power

2 State: Normal

Chassis 2:

Power

1 State: Normal

Power

2 State: Absent

If the power supply is in Absent state, go to step 2. If the power supply is in Fault state, go to step 3.
2.

A newly-installed power supply might run into Absent state when it is not securely installed.
a. Remove and reinstall the power supply to make sure the power supply is installed securely.
Then execute the display power command to verify that the power supply has changed to
Normal state. If the power supply remains in Absent state, go to step b.
b. Remove and install the power supply into an empty power supply slot. Then execute the display
power command to verify that the power supply has changed to Normal state in the new slot.
If the power supply remains in Absent state, go to step 4.

3.

Remove and install the power supply into an idle power supply slot. Then execute the display
power command to verify that the power supply has changed to Normal state in the new slot. If the
power supply remains in Fault state, go to step 4.

4.

If the problem persists, contact HP Support.

Fan tray failure


Symptom
An operating fan tray or a newly-installed fan tray fails.

Solution
To resolve the problem:
1.

Execute the display fan command to display the operating states of the fan tray.
<HP> display fan
Chassis 1:
Fan

1 State: Normal

Chassis 2:
Fan

1 State: Normal

If the fan tray is in Absent state, go to step 2.

If the fan tray is in Fault state, go to step 3.

2.

Remove and reinstall the fan tray to make sure the fan tray is securely installed. Then execute the
display fan command to verify that the fan tray has changed to Normal state. If the fan tray
remains in Absent state, replace the fan tray.

3.

Execute the display environment command to display temperature information. If the temperature
continues to rise, put your hand at the air outlet to verify that there is air being exhausted from the
air outlet. If there is no air being exhausted from the air outlet, remove and reinstall the fan tray.
Then execute the display fan command to verify that the fan tray has changed to Normal state. If
the fan tray remains in Fault state, replace the fan tray.
{
{

4.

If there is no new fan tray, power off the switch to avoid damage caused by high temperatures.
If there are cooling measures to maintain the switch operating temperature below 60C (140F),
the switch can be used temporarily.

If the problem persists, contact HP Support.


10

Related commands
This section lists the commands that you might use for troubleshooting the hardware.
Command

Description

dir

Displays information about files and directories.

display boot-loader

Displays current configuration files and system software


images.

display environment

Displays temperature information.

display fan

Displays the operating states of the fan tray.

display logbuffer

Displays the state of the log buffer and the log information in
the log buffer.

display power

Displays power supply information.

Troubleshooting system management


This section provides troubleshooting information for common system management problems.

Temperature alarming
Symptom
Temperature alarms occur.

11

Troubleshooting flowchart
Figure 2 Troubleshooting temperature alarming

Solution
To resolve the problem:
1.

Identify whether the device temperature or the ambient temperature is too high.
{

If the ambient temperature is too high, lower the temperature by adding air conditioners or
taking other heat dissipation measures.
If the device temperature is too high, go to step 2.

2.

Use the display fan command to verify that the fan tray is operating correctly. If the system displays
"Fault," see HP 10500 Switch Series Installation Guide to resolve the problem.

3.

Verify that the air filters are clean. If they are not, clean them.

4.

Use the temperature-limit command to set the temperature alarm thresholds.


You can use the display environment command to identify whether the temperature alarm
thresholds are set successfully.
{

If they are not, replace the card.


12

If they are but temperature alarms still occur, contact HP Support.

Related commands
This section lists the commands that you might use for troubleshooting system management.
Command

Remarks

display environment

Displays temperature information, including the current temperature


and the temperature alarm thresholds.

display fan

Displays the operating status of all fans on the switch.

temperature-limit

Sets temperature alarm thresholds.

Troubleshooting ports
This section provides troubleshooting information for common port problems.

10/100/1000Base-T copper port fails to go up


Symptom
A 10/100/1000Base-T copper port cannot go up.

13

Troubleshooting flowchart
Figure 3 Troubleshooting link up failure on a port
A port failed to go up

Speed/duplex
match on local and peer
port?

No

Configure matching
speed/duplex

Yes

Network cable correct?

Replace network
cable

Yes

Yes
Resolved?

No

No

Yes
Replace local port

Yes

Peer port
operating correctly?

Yes

No

No

Local port
operating correctly?

Resolved?

Resolved?

No

No

Yes
Replace peer port

Resolved?

No

Yes

Contact HP Support

End

Solution
To resolve the problem:
1.

Verify that the speed and duplex mode of the local port match the speed and duplex mode of the
peer port:
14

a. Execute the display interface brief command to examine whether the speed and duplex mode
of the port match the speed and duplex mode of the peer port.
b. If they do not match, use the speed command and the duplex command to set the speed and
duplex mode for the port.
2.

Replace the network cable with a new one to verify that the network cable is in good condition.

3.

Replace the local port with a new one (on a card of the same type, if possible) to verify that the
local port is operating correctly.

4.

Replace the peer port with a new one (on a card of the same type, if possible) to verify that the peer
port is operating correctly.

5.

If the problem persists, contact HP Support.

100/1000-Mbps SFP fiber port fails to go up


Symptom
A 100/1000-Mbps SFP fiber port cannot go up.

15

Troubleshooting flowchart
Figure 4 Troubleshooting link up failure on a port
A port failed to go up

Speed/duplex
match on local and peer
port?

No

Configure matching
speed/duplex

Yes
Speed/duplex
match on transceiver
module and port?

No

Configure matching
speed/duplex

Resolved?

No

Replace transceiver
module

Yes
Resolved?
No

No
Fiber correct?

Yes
Replace fiber

Resolved?
No

Yes

No

Yes
Replace local port

Resolved?
No

Yes
No
Peer port
operating correctly?

Yes

No

Yes

Local port
operating correctly?

Yes

No

Yes

Transceiver
module correct?

Resolved?

Yes
Replace peer port

Resolved?
No

Yes

Contact HP Support

End

Solution
To resolve the problem:

16

1.

Verify that the speed and duplex mode of the local port match the speed and duplex mode of the
peer port:
a. Execute the display interface brief command to examine whether the speed and duplex mode
of the port match the speed and duplex mode of the peer port.
b. If they do not match, use the speed command and the duplex command to set the speed and
duplex mode for the port.

2.

Verify that the speed and duplex mode of the port match the speed and duplex mode of the
transceiver module:
a. Execute the display interface brief command to examine whether the speed and duplex mode
of the port match the speed and duplex mode of the transceiver module.
b. If they do not match, use the speed command and the duplex command to set the speed and
duplex mode for the port.

3.

Verify that the transceiver module is operating correctly:


a. Execute the display transceiver alarm interface command to examine the alarms present on the
transceiver module.

The device displays None if no error occurs.

The device displays alarms if the transceiver module fails or the type of the transceiver
module does not match the port type.

b. Use an optical power meter to verify that the Tx power and Rx power of the transceiver module
are stable and are within the correct range.
c. Execute the display transceiver interface command to verify that the wavelength and
transmission distance of the local transceiver module are consistent with the wavelength and
transmission distance of the peer transceiver module.
[HP] display transceiver interface ten-gigabitethernet 1/3/0/15
Ten-GigabitEthernet1/3/0/15 transceiver information:
Transceiver Type

: 1000_BASE_SX_SFP

Connector Type

: LC

Wavelength(nm)

: 850

Transfer Distance(m)

: 550(50um),270(62.5um)

Digital Diagnostic Monitoring : YES


Vendor Name

: HP

Ordering Name

: SFP-GE-SX-MM850-A

d. If the transceiver module is not operating correctly, replace it with a new HP transceiver module
that matches the fiber port.
For more information about transceiver modules, see related installation guides.
4.

Replace the fiber with a new one to verify that the fiber is in good condition.

5.

Replace the local port with a new one (on a card of the same type, if possible) to verify that the
local port is operating correctly.

6.

Replace the peer port with a new one (on a card of the same type, if possible) to verify that the peer
port is operating correctly.

7.

If the problem persists, contact HP Support.

17

10-Gigabit XFP fiber port fails to go up


Symptom
A 10-Gigabit XFP fiber port cannot go up.

18

Troubleshooting flowchart
Figure 5 Troubleshooting link up failure on a port

Solution
To resolve the problem:
19

1.

Verify that the speed and duplex mode of the local port match the speed and duplex mode of the
peer port:
a. Execute the display interface brief command to examine whether the speed and duplex mode
of the port match the speed and duplex mode of the peer port.
b. If they do not match, use the speed command and the duplex command to set the speed and
duplex mode for the port.

2.

Verify that the speed and duplex mode of the port match the speed and duplex mode of the
transceiver module:
a. Execute the display interface brief command to examine whether the speed and duplex mode
of the port match the speed and duplex mode of the transceiver module.
b. If they do not match, use the speed command and the duplex command to set the speed and
duplex mode for the port.

3.

Verify that the transceiver module is operating correctly:


a. Execute the display transceiver alarm interface command to examine the alarms present on the
transceiver module.

The device displays None if no error occurs.

The device displays alarms if the transceiver module fails or the type of the transceiver
module does not match the port type.

b. Use an optical power meter to verify that the Tx power and Rx power of the transceiver module
are stable and are within the correct range.
c. Execute the display transceiver interface command to verify that the wavelength and
transmission distance of the local transceiver module are consistent with the wavelength and
transmission distance of the peer transceiver module.
[HP] display transceiver interface ten-gigabitethernet 1/3/0/15
Ten-GigabitEthernet1/3/0/15 transceiver information:
Transceiver Type

: 10G_BASE_LR

Connector Type

: LC

Wavelength(nm)

: 1310

Transfer Distance(km)

: 10(SMF)

Digital Diagnostic Monitoring : YES


Vendor Name

: FINISAR CORP.

Ordering Name

: XFP-Lx10-SM1310

d. If the transceiver module is not operating correctly, replace it with a new HP transceiver module
that matches the fiber port.
For more information about transceiver modules, see related installation guides.
4.

Verify that the fiber matches the transceiver module. If they do not match, replace the fiber with a
new one that matches the transceiver module.

5.

Replace the local port with a new one (on a card of the same type, if possible) to verify that the
local port is operating correctly.

6.

Replace the peer port with a new one (on a card of the same type, if possible) to verify that the peer
port is operating correctly.

7.

If the problem persists, execute the display diagnostic-information command to save the
diagnostic information, and contact HP Support.
<HP> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

20

10-Gigabit SFP+ fiber port fails to go up


Symptom
A 10-Gigabit SFP+ fiber port cannot go up.

21

Troubleshooting flowchart
Figure 6 Troubleshooting link up failure on a port

22

Solution
To resolve the problem:
1.

Verify that the speed and duplex mode of the local port match the speed and duplex mode of the
peer port:
a. Execute the display interface brief command to examine whether the speed and duplex mode
of the port match the speed and duplex mode of the peer port.
b. If they do not match, use the speed command and the duplex command to set the speed and
duplex mode for the port.

2.

Verify that the speed and duplex mode of the port match the speed and duplex mode of the
transceiver module:
a. Execute the display interface brief command to examine whether the speed and duplex mode
of the port match the speed and duplex mode of the transceiver module.
b. If they do not match, use the speed command and the duplex command to set the speed and
duplex mode for the port.

3.

Verify that the local and peer ports are operating correctly:
a. Use a 10-Gigabit SFP+ cable to directly connect the local port and another 10-Gigabit SFP+
fiber port of the same card. An SFP+ cable is suitable for short connections.
b. If the local port can go up, replace the peer port with a new one (on a card of the same type,
if possible) to verify that the peer port is operating correctly.
c. If the local port cannot go up, replace the local port with a new one (on a card of the same type,
if possible) to verify that the local port is operating correctly.

4.

Verify that the transceiver module is operating correctly:


a. Execute the display transceiver alarm interface command to examine the alarms present on the
transceiver module.

The device displays None if no error occurs.

The device displays alarms if the transceiver module fails or the type of the transceiver
module does not match the port type.

b. Use an optical power meter to verify that the Tx power and Rx power of the transceiver module
are stable and are within the correct range.
c. Execute the display transceiver interface command to verify that the wavelength and
transmission distance of the local transceiver module are consistent with the wavelength and
transmission distance of the peer transceiver module.
[HP] display transceiver interface Ten-GigabitEthernet 1/3/0/15
Ten-GigabitEthernet1/3/0/15 transceiver information:
Transceiver Type

: 1000_BASE_SX_SFP

Connector Type

: LC

Wavelength(nm)

: 850

Transfer Distance(m)

: 550(50um),270(62.5um)

Digital Diagnostic Monitoring : YES


Vendor Name

: HP

Ordering Name

: SFP-GE-SX-MM850-A

d. If the transceiver module is not operating correctly, replace it with a new HP transceiver module
that matches the fiber port.
For more information about transceiver modules, see related installation guides.
23

5.

Verify that the fiber matches the transceiver module. If they do not match, replace the fiber with a
new one that matches the transceiver module.

6.

If the problem persists, execute the display diagnostic-information command to save the
diagnostic information, and contact HP Support.
<HP> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

40-GE QSFP+ fiber port fails to go up


Symptom
A 40-Gigabit QSFP+ fiber port cannot go up.

24

Troubleshooting flowchart
Figure 7 Troubleshooting link up failure on a port
A port failed to go up

No
Local/peer port
operating correctly?

Yes
Replace local/peer
port

Resolved?

Yes

Transceiver
module/cable
correct?

No

No

Yes
Replace transceiver
module/cable

Resolved?

No

Yes

No
Fiber correct?

Yes
Replace fiber

Resolved?

No

Yes

Contact HP Support

End

Solution
To resolve the problem:
1.

Verify that the local and peer ports are operating correctly:
a. Use a QSFP+ cable to directly connect the local port and another 40-GE QSFP+ fiber port of
the same card. A QSFP+ cable is suitable for short connections.
b. If the local port can go up, replace the peer port with a new one (on a card of the same type,
if possible) to verify that the peer port is operating correctly.
c. If the local port cannot go up, replace the local port with a new one (on a card of the same type,
if possible) to verify that the local port is operating correctly.
25

2.

Verify that the transceiver module is operating correctly:


a. Execute the display transceiver alarm interface command to examine the alarms present on the
transceiver module.

The device displays None if no error occurs.

The device displays alarms if the transceiver module fails or the type of the transceiver
module does not match the port type.

b. Use an optical power meter to verify that the Tx power and Rx power of the transceiver module
are stable and are within the correct range.
c. Execute the display transceiver interface command to verify that the wavelength and
transmission distance of the local transceiver module are consistent with the wavelength and
transmission distance of the peer transceiver module.
[HP] display transceiver interface fortygige 1/0/1
FortyGigE1/1/0/4 transceiver information:
Transceiver Type

: 40G_BASE_SR4_QSFP_PLUS

Connector Type

: MPO

Wavelength(nm)

: 850

Transfer Distance(m)

: 100(OM3)

Digital Diagnostic Monitoring : YES


Vendor Name

: HP

d. If the transceiver module is not operating correctly, replace it with a new HP transceiver module
that matches the fiber port.
For more information about transceiver modules, see related installation guides.
IMPORTANT:

If a QSFP+ to SFP+ cable is used, use an optical attenuator to make sure the Tx power of the QSFP+
module stays below the Rx power of the SFP+ module.
3.

Verify that the fiber matches the transceiver module. If they do not match, replace the fiber with a
new one that matches the transceiver module.

4.

If the problem persists, execute the display diagnostic-information command to save the
diagnostic information, and contact HP Support.
<HP> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

40-GE CFP fiber port fails to go up


Symptom
A 40-GE CFP fiber port cannot go up.

26

Troubleshooting flowchart
Figure 8 Troubleshooting link up failure on a port
A port failed to go up

Transceiver
module correct?

No

Replace transceiver
module

Yes
Resolved?

Yes

No

No
Fiber correct?

Yes
Replace fiber

Resolved?

Yes

Local port
operating correctly?

No

No

Yes
Replace local port

Resolved?

Yes

Peer port
operating correctly?

No

No

Yes
Replace peer port

Resolved?

No

Yes

Yes
Contact HP Support

End

27

Solution
To resolve the problem:
1.

Verify that the transceiver module is operating correctly:


a. Execute the display transceiver alarm interface command to examine the alarms present on the
transceiver module.

The device displays None if no error occurs.

The device displays alarms if the transceiver module fails or the type of the transceiver
module does not match the port type.

b. Use an optical power meter to verify that the Tx power and Rx power of the transceiver module
are stable and are within the correct range.
c. Execute the display transceiver interface command to verify that the wavelength and
transmission distance of the local transceiver module are consistent with the wavelength and
transmission distance of the peer transceiver module.
d. If the transceiver module is not operating correctly, replace it with a new HP transceiver module
that matches the fiber port.
For more information about transceiver modules, see related installation guides.
2.

Verify that the fiber matches the transceiver module. If they do not match, replace the fiber with a
new one that matches the transceiver module.

3.

Replace the local port with a new one (on a card of the same type, if possible) to verify that the
local port is operating correctly.

4.

Replace the peer port with a new one (on a card of the same type, if possible) to verify that the peer
port is operating correctly.

5.

If the problem persists, execute the display diagnostic-information command to save the
diagnostic information, and contact HP Support.
<HP> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

Error frames (for example, CRC errors) on a port


Symptom
In the output from the display interface command, error frames exist (for example, CRC error frames).

28

Troubleshooting flowchart
Figure 9 Troubleshooting error frames (for example, CRC errors) on a port
Error frames on the port

Determine the type of error


frames accumulating

No
Optical power
of transceiver module
normal?

Replace it with a
transceiver
module with
normal optical
power

Yes

Speed and
duplex match the
peer port?

No

No

Configure the
speed and duplex
for ports

Yes
Resolved?

No

Yes

Port and link


medium normal?

Yes

Resolved?

No

Use normal port


and link medium

Yes

Yes
Resolved?
No

View the number


of flow control
frames received

Contact HP Support

End

Solution
To resolve the problem:
1.

Examine the error frame statistics and identify the error frame type:
a. (Optional.) Use the reset counter interface command in user view to clear the packet statistics
of the port.
This command resets the values of each statistical field to 0, so that you can view the statistics
changes more clearly.

29

b. Use the display interface command to display the incoming packet statistics and outgoing
packet statistics of the port.
c. Determine the type of error frames that are accumulating.
2.

If the port is a fiber port, verify that the optical power of the transceiver module is operating
correctly:
a. Use the display transceiver diagnosis interface command to view the present measured values
of the digital diagnosis parameters for the transceiver module.
b. If the optical power of the transceiver module is not within the correct range, replace the
transceiver module with a transceiver module of the same model that is operating correctly.
[HP] display transceiver diagnosis interface gigabitethernet1/0/1
GigabitEthernet1/0/1 transceiver diagnostic information:
Current diagnostic parameters:

3.

Temp.(C)

Voltage(V)

40

3.34

Bias(mA)
1.13

RX power(dBM)
-20.43

TX power(dBM)
0.20

Verify that the port configurations are correct:


a. Execute the display interface brief command.
b. Determine whether the speed and duplex mode of the port match the speed and duplex mode
of the peer port.
c. If they do not match, use the speed command and the duplex command to set the speed and
duplex mode for the port.

4.

Verify that the port and the link medium are operating correctly:
a. Plug the link medium into another port that is operating correctly, and determine whether the
same problem occurs.
b. If the problem still occurs, verify that the following items are operating correctly:

Intermediate links and devices, including optical-to-electrical converters, cable terminal


racks, and transmission devices.

Transmission medium, including network cables, optical fibers, and transceiver modules.

c. Examine the port:

5.

If the port is a copper port, directly connect the port to a PC.

If the port is a fiber port, replace the transceiver module plugged into the port.

Determine whether the port has received a large amount of flow control frames:
a. Use the display interface command to view the number of pause frames.
If the number of pause frames is accumulating, you can determine that the port has sent or
received a large amount of flow control frames.
b. Verify that the incoming traffic and outgoing traffic have not exceeded the maximum traffic
processing capability of the local device and the peer device.

6.

If you determine that the configurations, the peer port, and the link are correct but the problem
persists, perform the following tasks:
{

Use the display diagnostic-information command to collect the diagnostic information.

Contact HP Support.

<HP> display diagnostic-information


Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

30

Failure to receive packets


Symptom
A port is up, but it cannot receive packets.

Troubleshooting flowchart
Figure 10 Troubleshooting the failure to receive packets
A port fails to receive
packets

View packet statistics of


the port

Port
configurations affect packet
receiving?

No

Modify port
configurations

Yes

Port and link


medium normal?

Resolved?

Yes

No

No

Use normal port


and link medium

Yes
Resolved?
No

Yes

Contact HP Support

End

Solution
To resolve the problem:
1.

Examine the packet statistics of the port:


a. (Optional.) Use the reset counter interface command in user view to clear the packet statistics
of the port.
This command resets the values of each statistical field to 0, so that you can view the statistics
changes more clearly.
b. Determine whether the ports at both ends are always up, and use the display interface
command to determine whether the number of incoming packets is accumulating. At the same
time, examine the outgoing packet statistics of the peer port.
c. View the values of the error packet fields and determine whether the number of error packets
is accumulating.

2.

Verify that the port configurations do not affect packet receiving:


31

a. Use the display interface brief command to verify that the port configurations are correct.
The port configurations include the duplex mode, speed, port type, and VLAN configurations
of the ports at both ends of the link.
b. If configuration errors exist, modify the port configurations. If the number of incoming packets
in the output from the display interface command is accumulating, you can determine that the
port can receive packets.
c. If the port fails to receive packets, use the shutdown command and then the undo shutdown
command to re-enable the port. If the number of incoming packets in the output from the
display interface command is accumulating, you can determine that the port can receive
packets.
d. If the port is configured with STP, use the display stp brief command to verify that the port is not
in the discarding state.
e. If a port is set to the discarding state by STP, examine and modify the STP-related
configurations to resolve the problem.
HP recommends that you configure the port as an edge port or disable STP on the port if it is
directly connected to a terminal.
f. If the port is assigned to an aggregation group, use the display link-aggregation summary
command to verify that the status of the port is Selected.
g. If the status of the port is Unselected, the port cannot send or receive data packets.
h. Determine the reasons why the port becomes Unselected, for example, the attribute
configurations of the port are different from the reference port. Modify the attribute
configurations of the port to make the port become Selected.
3.

Verify that the port and the link medium are operating correctly:
a. Plug the link medium into another port that is operating correctly, and determine whether the
same problem occurs.
b. If the problem still occurs, verify that the following items are operating correctly:

Intermediate links and devices, including optical-to-electrical converters, cable terminal


racks, and transmission devices.

Transmission medium, including network cables, optical fibers, and transceiver modules.

c. Examine the port.

4.

If the port is a copper port, directly connect the port to a PC.

If the port is a fiber port, replace the transceiver module plugged into the port.

If you determine that the configurations, the peer port, and the link are correct but the problem
persists, perform the following tasks:
{

Use the display diagnostic-information command to collect the diagnostic information.

Contact HP Support.

<HP> display diagnostic-information


Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

32

Failure to send packets


Symptom
A port is up, but it cannot send packets.

Troubleshooting flowchart
Figure 11 Troubleshooting the failure to send packets

Solution
To resolve the problem:
1.

Examine the packet statistics of the port:


a. (Optional.) Use the reset counter interface command in user view to clear the packet statistics
of the port.
This command resets the values of each statistical field to 0, so that you can view the statistics
changes more clearly.
b. Determine whether the ports at both ends are always up, and use the display interface
command to determine whether the number of outgoing packets is accumulating.
c. Determine whether the number of error packets is accumulating.

2.

Verify that the port configurations do not affect packet sending:


33

a. Use the display interface brief command to verify that the port configurations are correct.
The port configurations include the duplex mode, speed, port type, and VLAN configurations
of the ports at both ends of the link.
b. If configuration errors exist, modify the port configurations. If the number of outgoing packets
in the output from the display interface command is accumulating, you can determine that the
port can send packets.
c. If the port fails to send packets, use the shutdown command and then the undo shutdown
command to re-enable the port. If the number of outgoing packets in the output from the display
interface command is accumulating, you can determine that the port can send packets.
d. If the port is configured with STP, use the display stp brief command to verify that the port is not
in the discarding state.
e. If a port is set to the discarding state by STP, examine and modify the STP-related
configurations to resolve the problem.
HP recommends that you configure the port as an edge port or disable STP on the port if it is
directly connected to a terminal.
f. If the port is assigned to an aggregation group, use the display link-aggregation summary
command to verify that the status of the port is Selected.
g. If the status of the port is Unselected, the port cannot send or receive data packets.
h. Determine the reasons why the port becomes Unselected, for example, the attribute
configurations of the port are different from the reference port. Modify the attribute
configurations of the port to make the port become Selected.
3.

Verify that the port and the link medium are correct:
a. Plug the link medium into another port that is operating correctly, and determine whether the
same problem occurs.

If the port is a copper port, replace the peer device with a PC.

If the port is a fiber port, replace the transceiver module.

b. If the problem still occurs, determine whether intermediate links and devices are operating
correctly (for example, optical converters, cable terminal racks, and transmission devices).
4.

If you determine that the configurations, the peer port, and the link are correct but the problem
persists, perform the following tasks:
{

Use the display diagnostic-information command to collect the diagnostic information.

Contact HP Support.

<HP> display diagnostic-information


Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

Related commands
This section lists the commands that you might use for troubleshooting ports.
Command

Description

display diagnostic-information

Displays or saves running status data for multiple


feature modules.

display interface

Displays Ethernet interface information.

display interface brief

Displays brief interface information.


34

Command

Description

display link-aggregation summary

Displays the summary information for all


aggregation groups.

display logbuffer

Displays the state of the log buffer and the log


information in the log buffer.

display stp brief

Displays brief spanning tree status and statistics.

display transceiver alarm interface

Displays the current transceiver module alarms.

display transceiver diagnosis

Displays the present measured values of the


digital diagnosis parameters for transceiver
modules.

display transceiver interface

Displays the key parameters of transceiver


modules.

Troubleshooting IRF
This section provides troubleshooting information for common IRF problems.

IRF fabric establishment failure


Symptom
A chassis cannot be added to an IRF fabric.

35

Troubleshooting flowchart
Figure 12 Troubleshooting IRF fabric establishment failure
IRF setup
failure

Fewer than 4
chassis
in the fabric?
Yes
Same chassis model?

No

You cannot add another


chassis to the fabric

Replace with a chassis


No that is the same model as
existing IRF member
chassis

Yes

Chassis in IRF mode?

No

Enable IRF mode

No

Assign a unique member


ID to the chassis

No

Modify IRF port bindings


or reconnect physical ports

No

Remove link failure


conditions to bring up the
IRF links

No

Upgrade the MPUs with


the software version
running in the IRF fabric

Resolved?

Yes

Resolved?

Yes

Resolved?

Yes

Resolved?

Yes

No

Yes
Bridge MAC unique?

Yes

No

Yes
Same software version?

Resolved?

No

Yes
IRF links up?

No

No

Yes
Physical IRF
connections correct?

Yes

No

Yes
Member ID unique?

Resolved?

No

Remove the duplicate


bridge MAC condition

Resolved?

Yes

No

Yes
Contact HP
Support

End

36

Solution
To resolve the problem:
1.

Execute the display irf command to verify that the number of member chassis in the IRF fabric does
not exceed the upper limit of four.
If the upper limit is reached, you cannot add new chassis to the IRF fabric.

2.

Verify that the chassis is the same model as the chassis in the IRF fabric.
The 10500 switches must be the same model to form an IRF fabric.

3.

Verify that the chassis is operating in IRF mode:


a. Execute the display interface brief command to verify that the physical port numbers on the
chassis have changed from three segments to four segments. In IRF mode, the member ID is
added to physical port numbers as the first segment (for example, 10-GigabitEthernet
1/4/0/1).
b. Execute the chassis convert mode irf command to change the operating mode to IRF.

4.

Verify that the member ID of the chassis does not conflict with any member chassis in the IRF fabric:
a. Execute the display irf command to view member IDs.
b. If the member ID of the chassis is not unique, use the irf member renumber command to assign
a new member ID to the chassis.

5.

Verify that the physical IRF links are connected correctly:


IMPORTANT:
When you connect two neighboring IRF members, you must connect the physical ports of IRF-port 1
on one member to the physical ports of IRF-port 2 on the other.
a. Execute the display irf configuration command on each member chassis, and check the
IRF-Port1 and IRF-Port2 fields for IRF port bindings.
b. Verify that the physical IRF connections are consistent with the IRF port bindings.
c. If there are inconsistencies, reconfigure the IRF port bindings or reconnect the physical IRF
ports.

6.

Verify that all IRF links are up:


a. Execute the display irf topology command, and then check the Link field.
b. If the Link field for an IRF port displays DOWN, execute the display interface command.
c. Check the current state field for each physical port bound to the IRF port.

If the field displays DOWN ( Administratively ), execute the undo shutdown command to
bring up the port.

If the field displays DOWN, verify that SFP+ or QSFP+ transceiver modules or cables are
used for IRF connection and that they are operating correctly.

d. If the IRF port is down when it contains at least one up link, verify that the MPUs of the chassis
are operating correctly.
7.

Verify that the member chassis is running the same software version as the IRF fabric:
a. Execute the display version command to identify the software version.
b. Upgrade the MPUs in the chassis to use the same software version as the IRF fabric.

37

NOTE:
Typically, the irf auto-update enable command can automatically synchronize a member chassis
with the software version of the master chassis. However, the synchronization might fail when the gap
between the software versions is large.
8.

Verify that the chassis has a unique bridge MAC address:


a. Execute the display interface vlan-interface 1 command, and then check the Hardware
Address field.
[HP] display interface vlan-interface 1
Vlan-interface1 current state: UP
Line protocol current state: UP
Description: Vlan-interface1 Interface
The Maximum Transmit Unit is 1500
Internet protocol processing : disabled
IP Packet Frame Type: PKTFMT_ETHNT_2,

Hardware Address: 0023-8912-3d07

IPv6 Packet Frame Type: PKTFMT_ETHNT_2,

Hardware Address: 0023-8912-3d07

b. If the chassis has the same bridge MAC address as the IRF fabric, remove the bridge MAC
conflict.
9.

If the problem persists, contact HP Support.


Execute the display diagnostic-information command, save the diagnostic information, and
provide it to HP Support.
<HP> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

IRF split
Symptom
An IRF fabric splits.

38

Troubleshooting flowchart
Figure 13 Troubleshooting IRF split
IRF fabric splits

IRF physical ports down?

Yes

Identify the cause and


resolve the problem

Yes

Identify the cause and


resolve the problem

Resolved?

Yes

No

No

Member chassis rebooted?

Yes

No

No

Card rebooted?

Resolved?

Yes

Identify the cause and


resolve the problem

Resolved?

Yes

No

No
Contact HP
Support

End

Solution
To resolve the problem:
1.

Verify that the IRF physical ports are operating correctly:


a. Execute the display trapbuffer command or view system logs to check for physical IRF link
down events that occurred around the split time.
b. If the split followed an IRF link down event, execute the display interface command to check
port statistics for CRC errors.
c. If the port is an SFP+ or QSFP+ port, execute the display transceiver diagnosis command to
verify that the transmit and receive power does not exceed the power specifications of the
fiber-optic module.

2.

Remove hardware problems that might cause recurring IRF split events:
a. Execute the display version command to identify the uptime of MPUs and interface cards that
have IRF links.
b. Compare the uptime of MPUs and interface cards to determine whether a member chassis or
interface card rebooted before the IRF split.
c. If the IRF split is caused by a chassis or card reboot, perform one of the following tasks:

If the split is caused by recurring card reboots, contact HP Support to resolve the problem.
39


3.

If the split is caused by a chassis reboot, use the methods described in "MPU reboot failure"
to resolve the problem.

If the problem persists, contact HP Support.


Execute the display diagnostic-information command, save the diagnostic information, and
provide it to HP Support.
<HP> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

BFD MAD failure


Symptom
BFD MAD fails to detect an IRF split event. Two IRF fabrics are operating with the same Layer 3
configurations, including the same IP address.

40

Troubleshooting flowchart
Figure 14 Troubleshooting the BFD MAD failure
BFD MAD failure

BFD MAD link connections correct?

No

Correct the connections

No

Bring up the ports

Yes
BFD MAD VLAN interface
configured correctly?

No

Correct the settings

No

Correct the settings

No

Correct the BFD MAD IP


address settings

Yes

Resolved?

Yes

Resolved?

Yes

No

No

Assign each IRF fabric a


unique BFD MAD VLAN

Resolved?

Yes

No

Yes
BFD MAD port state changes
were occasional?

Resolved?

No

Yes
BFD MAD VLAN unique in a
multi-IRF fabrics network?

Yes

No

Yes
BFD MAD IP addresses
configured correctly?

Resolved?
No

Yes

BFD MAD VLAN and physical


ports configured correctly?

Yes

No

Yes
All BFD MAD ports up?

Resolved?

No

Identify event cause and


remove the issue

Resolved?

Yes

No

Yes

End

Contact HP Support

Solution
To resolve the problem:
41

1.

Verify that BFD MAD link connections are correct:


{

2.

If you do not use an intermediate device, verify that each pair of member chassis has a
dedicated BFD MAD link.
If you use an intermediate device, verify that each member chassis has a dedicated BFD MAD
link with the intermediate device.

Verify that all physical ports used for BFD MAD are up:
a. Execute the display interface command.
b. Check the current state field in the command output:

3.

If the field displays DOWN ( Administratively ) for a port, execute the undo shutdown
command to bring up the port.

If the field displays DOWN for a port, check the physical link for a link failure.

Verify that the BFD MAD VLAN interface is configured correctly:


a. Execute the display mad verbose command.
b. Check the MAD BFD enabled interface field to verify that VLAN-interface 1 is not used for BFD
MAD.
c. If VLAN-interface 1 is used for BFD MAD, disable BFD MAD on VLAN-interface 1, and then
enable BFD MAD on another VLAN interface.
d. Verify that the BFD MAD-enabled VLAN interface does not have any other features, including
VPN.
e. Execute the display interface command to verify that the interface does not have the following
IP addresses:

4.

IP address assigned by using the ip address command.

VRRP virtual IP address.

Verify that the BFD MAD VLAN and physical ports in the VLAN are configured correctly:
a. Execute the display vlan command to verify the following items:

The BFD VLAN contains all physical ports used for BFD MAD.

The BFD VLAN does not contain physical ports that are not used for BFD MAD.

If an intermediate device is used, this step must also be performed on the intermediate device.
b. Execute the display interface command to verify that the physical ports in the BFD VLAN are not
configured with any features.
For BFD MAD to operate correctly, you must disable all features on the physical ports in the
BFD MAD VLAN, including the spanning tree feature, ARP, and LACP.
5.

Verify that the MAD IP addresses are correct:


a. Execute the display mad verbose command.
b. Check the mad ip address field to verify the following items:

Each member chassis has a MAD IP address.

All MAD IP addresses are on the same subnet.

The MAD IP addresses are not addresses being used on the IRF fabric.

6.

If the network has multiple IRF fabrics, execute the display mad verbose command to verify that
their BFD MAD VLANs are different.

7.

If the physical ports in the BFD MAD VLAN are down continually:

42

a. Execute the display trapbuffer command or use system logs to check for BFD MAD port down
events that occurred around the split time.
b. Identify the cause of the event, and remove the issue.
8.

If the problem persists, contact HP Support.


Execute the display diagnostic-information command, save the diagnostic information, and
provide it to HP Support.
<HP> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

LACP MAD failure


Symptom
LACP MAD fails to detect an IRF split event. Two IRF fabrics are operating with the same Layer 3
configurations, including the same IP address.

43

Troubleshooting flowchart
Figure 15 Troubleshooting LACP MAD failure

Solution
To resolve the problem:
1.

Verify that the intermediate device is an HP device that supports extended LACPDUs for MAD.
If the intermediate device does not support extended LACPDUs for MAD, replace the intermediate
device, or use BFD MAD for split detection.

2.

Verify that each member chassis has a link in the link aggregation with the intermediate device.

3.

Verify that the link aggregation is operating in dynamic mode.


To enable dynamic aggregation mode, use the link-aggregation mode dynamic command.
44

4.

Verify that the aggregate interface and its member ports are up:
a. Execute the display interface command.
b. Check the current state field of the aggregate interface:

If the field displays DOWN ( Administratively ), execute the undo shutdown command to
bring up the interface.

If the field displays DOWN, check the state of all its physical ports.
An aggregate interface goes down only if all its physical ports are down.

c. Check the current state field of each member port:

5.

If the field displays DOWN ( Administratively ), execute the undo shutdown command to
bring up the port.

If the field displays DOWN, check the physical link of the port for a link failure.

If the intermediate device is also an IRF fabric, verify that the IRF domain IDs of the two IRF fabrics
are unique:
CAUTION:
The IRF member devices send extended LACPDUs with TLVs that convey the domain ID and the active
ID of the IRF fabric. For correct split detection, makes sure the IRF fabric has a unique domain ID.
a. Execute the display irf command to identify the domain ID of each IRF fabric.
b. If the IRF fabrics use the same domain ID, execute the irf domain command to change the
domain ID on one IRF fabric.

6.

If the physical ports in the link aggregation are down continually:


a. Execute the display trapbuffer command or use system logs to check for port down events
around the split time.
b. Identify the event cause and remove the issue.

7.

If the problem persists, contact HP Support.


Execute the display diagnostic-information command, save the diagnostic information, and
provide it to HP Support.
<HP> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

Related commands
This section lists the commands that you might use for troubleshooting IRF:
Command

Description

display diagnostic-information

Displays or saves the operating statistics for multiple feature


modules.

display interface

Displays interface information.

display interface brief

Displays brief interface information.

display irf

Displays IRF fabric information, including the member ID, role,


priority, bridge MAC address, and description of each IRF
member.

display irf configuration

Displays the IRF configuration on each member chassis.


45

Command

Description

display irf topology

Displays the IRF topology.

display mad verbose

Displays detailed MAD configuration.

display transceiver diagnosis

Displays the present measured values of the digital diagnosis


parameters for transceiver modules.

display trapbuffer

Displays trap data in the trap buffer.

display version

Displays system version information.

display vlan

Displays VLAN information.

Troubleshooting QoS and ACL


This section provides troubleshooting information for common QoS and ACL problems.

ACL application failure for unsupported ACL rules


Symptom
The system fails to apply a packet filter or an ACL-based classifier in a QoS policy to the hardware. It
displays an error message that the ACL is not supported:
Error: Slot=2 Fail to apply or refresh packet filter policy 3180 rule 5 on interface
Vlan-interface20. Not supported.

46

Troubleshooting flowchart
Figure 16 Troubleshooting an ACL application failure

Solution
To resolve the problem:
1.

Verify that the ACL is configured correctly:


Execute the display acl command to identify whether the ACL is configured correctly. If the ACL is
configured incorrectly, reconfigure it.

2.

Verify that the matching criteria in each ACL rule are of the same length:
a. If an ACL rule includes much more matching criteria than the other ACL rules, split it into multiple
rules.
b. Apply the adjusted ACL.

3.

If the problem persists, contact HP Support.


Execute the display diagnostic-information command, save the diagnostic information, and
provide it to HP Support.
<HP> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

ACL application failure for insufficient resources


Symptom
The system fails to apply a packet filter or an ACL-based classifier in a QoS policy to the hardware. It
displays one of the following error messages that the hardware resources are not sufficient:
47

Error: Slot=2 Fail to apply or refresh packet filter policy 3001 rule 25 on int
erface Vlan-interface6 due to lack of resources.
Warning: Classifier-behavior test in policy test applied on vlan 4079 failed in
slot 2.
Reason: Not enough hardware resource.

Troubleshooting flowchart
Figure 17 Troubleshooting an ACL application failure

Solution
Insufficient hardware resources might be caused by insufficient Counter, Meter, or ACL resources.
To resolve the problem:
1.

Verify that the number of ACL rules to be applied is no more than the number of remaining
resources:
Execute the display acl resource command to display the ACL resource usage. If the ACL is applied
globally (for example, to a VLAN), check remaining ACL resources on all cards.
[HP] display acl resource
Interface:
GE3/0/1 to GE3/0/24
--------------------------------------------------------------------Type

Total

Reserved

Configured

Remaining

Usage

--------------------------------------------------------------------VFP ACL

1024

256

768

25%

IFP ACL

4096

IFP Meter

2048

1024

316

2756

32%

512

1422

1422

30%

IFP Counter
EFP ACL

2048

512

1422

1422

30%

512

512

0%

EFP Meter

256

256

0%

EFP Counter

512

512

0%

IFPResource usage for inbound traffic.


48

EFPResource usage for outbound traffic.

a. If the number of ACL rules to be applied is greater than the number of remaining ACL resources,
delete unnecessary ACL rules.
b. Apply the ACLs.
2.

If the problem persists, contact HP Support.


Execute the display diagnostic-information command, save the diagnostic information, and
provide it to HP Support.
<HP> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

ACL application failure without an error message


Symptom
The system applies a QoS policy that contains ACLs to the hardware. However, the ACL does not take
effect.

Troubleshooting flowchart
Figure 18 Troubleshooting an ACL application failure
Failure to apply an ACL

No

Overlapping
ACLs exist?
Yes

Conflicting behaviors
for packets matching
overlapping ACLs?

Yes

Reconfigure conflicting
behaviors

Yes

No

No

Contact HP Support

End

Solution
To resolve the problem:
1.

Problem resolved?

Verify that overlapping ACL rules that have been applied exist.
49

a. Use one of the following commands to display applied ACLs:

display qos policy user-defined

display traffic classifier user-defined

b. Execute the display acl command to verify that the applied ACL rules overlap.
For example, the output shows that rule 0 in ACL 3100 and rule 0 in ACL 3009 overlap. They
both can match traffic sourced from 2.2.2.1.
ACL number 3100
rule 0 permit ip source 2.2.2.2 255.255.0.0
ACL number 3009
rule 0 permit ip source 2.2.2.2 255.255.255.0

2.

Verify that the behaviors for packets matching the overlapping ACLs do not conflict.
If the behaviors conflict, the device chooses a behavior according to the rules shown in Table 2.
The ineffective behavior and the ACL for the associated traffic class do not take effect.
Table 2 Rules for an effective behavior between conflicting behaviors

3.

Conflicting behaviors

Effective behavior

redirect
filter permit

redirect

redirect
filter deny

filter deny

filter permit
filter deny

The behavior configured first.

If the problem persists, contact HP Support.


Execute the display diagnostic-information command, save the diagnostic information, and
provide it to HP Support.
<HP> display diagnostic-information
Save or display diagnostic information (Y=save, N=display)? [Y/N]:Y

Packet loss or forwarding failure


Symptom
The network experiences packet loss or packet forwarding failure. You can use ACLs to count traffic statistics
to determine the location of the fault.

50

Troubleshooting flowchart
Figure 19 Troubleshooting a forwarding failure

Solution
To resolve the problem:
1.

Identify the characteristics of the packets that are not forwarded correctly.
Use a packet capture tool to identify the characteristics of the traffic, such as the IP address, MAC
address, or VLAN. Make sure the characteristics are unique among all traffic to be forwarded.

2.

Configure a QoS policy and apply it to an interface:


Figure 20 Traffic path

a. Configure a QoS policy according to the traffic characteristics. In this example, use a
destination IP address 2.2.2.2.
<HP> system-view
System View: return to User View with Ctrl+Z.
[HP] time-range t1 from 12:00 11/11/2013 to 12:01 11/11/2013
[HP] acl number 3000 name geliACL
[HP-acl-adv-3000-geliACL] rule 0 permit ip destination 2.2.2.2 0 time-range t1
[HP-acl-adv-3000-geliACL] quit
[HP] traffic classifier count
[HP-classifier-count] if-match acl 3000
[HP-classifier-count] quit
[HP] traffic behavior count

51

[HP-behavior-count] accounting
[HP-behavior-count] quit
[HP] qos policy count
[HP-qospolicy-count] classifier count behavior count

b. Apply the QoS policy to interfaces on different devices on the forwarding path. In this example,
apply the QoS policy to GigabitEthernet 1/0/1 on Device B and GigabitEthernet 2/0/1 on
Device C.
The following shows the application on Device B:
[HP-qospolicy-count] quit
[HP] interface gigabitethernet 1/0/1
[HP-GigabitEthernet1/0/1] qos apply policy count inbound

3.

Execute the display qos policy interface command to display traffic statistics on the interfaces:
{

On GigabitEthernet 1/0/1 on Device B:


[HP-GigabitEthernet1/0/1] display qos policy interface gigabitethernet 1/0/1

Interface: GigabitEthernet1/0/1

Direction: Inbound

Direction: Inbound
Policy: count
Classifier: count
Operator: AND
Rule(s) : If-match ACL 3000
Behavior: count
Accounting Enable:
251 (Packets)

The output shows that a total of 251 packets match ACL 3000.
{

On GigabitEthernet 2/0/1 on Device C:


[HP-GigabitEthernet2/0/1] display qos policy interface gigabitethernet 2/0/1

Interface: GigabitEthernet2/0/1
Direction: Inbound

Direction: Inbound
Policy: count
Classifier: count
Operator: AND
Rule(s) : If-match ACL 3000
Behavior: count
Accounting Enable:
50 (Packets)

The output shows that a total of 50 packets match ACL 3000.


4.

Compare the traffic statistics on the two interfaces. Packet loss or forwarding fault exists between
Device B and Device C.

52

Related commands
This section lists the commands that you might use for troubleshooting QoS and ACLs.
Command

Description

display acl

Displays configuration and match statistics for ACLs.

display acl resource

Displays ACL resource usage.

display diagnostic-information

Displays or saves running status data for ACLs.

display qos policy interface

Displays information about the QoS policies applied


to specified interfaces.

display qos policy user-defined

Displays user-defined QoS policy configuration.

display traffic classifier user-defined

Displays user-defined classes.

53