Professional Documents
Culture Documents
txt
Evaluation Report
Case Number: 02627448
Project: 02627448
Platform: exos
Owner: ssafiee
Fail Items
ISSUE UNABLE TO POLL minor Fail EXOS Version: 30.7.2.1 Problem Description: Unable to poll switch
SWITCH VIA SNMP V2
Condition:
ISSUE FAN OPERATIONAL minor Fail Slot-1 FanTray Fans:[Fan-1 Operational 0,Fan-2 Operational Check Fan status for switch. If the fan is
CHECK 0]
failed or log has fan fail error message, this
rule identifies the problem and logs the useful
information. Please ask customer to run
extended diagnostics to verify whether the
failure is related with hardware or not. Please
refer to article All fans rotating though
output shows failed with zero RPM for all
fans.
SYSTEM LOG ERROR CHECK major Fail 07/13/2022,15:18:08.03, Module:cm, This rule check existence of any error logs in
Comp:sys.LoadApplCfgObjFail, Message(1): "otm"
application failed to load "otmGlobal" configuration object:
the system and shows information. Please
No Action -- Insufficient License check error message from EXOS error
message decoder
07/13/2022,15:18:06.79, Module:HAL, Comp:Port.Error,
Message(1): Failed to set eee config on port 8, rv = -16
1/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
2/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
Pass Items
ISSUE X465 VIM5 1G major Pass This rule is to check whether HAL Crash occurs when port flaps with SRP feature
NEIGHBOR LINK NOT UP
CHECK
happens.
Environments:
Platform: X465
VIM5-4X,VIM5-4XE,VIM5-YE
Symptoms:
Issue1: X465-24MU, -24MU-24W and -24XE models only support 10 Gb operation (no 1Gb) on VIM5-4X
and VIM5-4XE modules.
Issue2: X465-48T,X465-48P,X465-48W with VIM5-4X module for 1GB : 1G link doesn't come up.
Issue3: X465-24W, X465-48T, X465-48P, X465-48W and X465-24S models are limited to 2 x 10/ 25Gb
port s on VIM5-4YE modules.
Remedy Action :
Issue1: Please use 10GE instead of 1G
Issue2: Please configure auto-negotiation as off with 1000Mbps + Full duplex in the neighboring side or
server/switch.
Issue3: only First 2 Ports in the VIM can be used.
ISSUE VRRP GROUP LOAD major Pass When switch with VRRP group configuration reboots or switch is uploaded with
CONFIG LOSS CRASH EXOS 22
30 CHECK
configuration(which includes VRRP group configuration) followed by reboot,
switch can lose VRRP group configurations and also leads into VRRP process
crash.
Symptoms:
Losing VRRP Group configuration for some of VLANs
VRRP Process Crash
Remedy Action :
Upgrade firmware into > 30.3 (recommended version is 30.7.2 or 31.3
ISSUE VPEX SYSTEM major Pass VPEX System (ONIE Platform: 590,690,495) with the below 22.7.1.2-patch1-3
UPGRADE LST IMAGE FAIL
POST 30 2 Version
version can't be upgraded with ONIE LST image into post 30.2 version because
scratch partition size is too small.
Remedy Action:
3/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE VPEX SYSTEM LACP major Pass VPEX System with LACP ports can lead into temporary loop situation when user
TEMPORARY LOOP
ports are enabled with LACP protocol to connect WING AP or Server ( 2 ports are
connected to user ports of BPE switch). Before LACP enabled port is added into the
aggregator, the user traffic is not dropped and loop occurs. When ELRP is enabled,
logs for ELRP loop are generated.
ISSUE VPEX NETLOGIN USER major Pass This rule is to check whether problem xos0078656 is the root cause of customer
PARTIAL CONNECTIVITY
issue or not.
Affected versions:
EXOS 22.1 - 22.6 and 22.7.1
EXOS 30.4
EXOS 30.6.1.11-patch1-2 and EXOS 30.6.1.11-patch1-4
Symptom : Partial connectivity issue (Clients within the same L2 domain can reach
some hosts but not others)
Remedy Action : Please upgrade VPEX topology with EXOS 22.7.2 or 30.7 or
higher.
ISSUE VLAN AGGREGATION major Pass This rule is to check whether memory leaking issue related in the VLAN aggregation
BOOTPRELAY MEMORY LEAK
CHECK
(Super/Subvlan) with DHCP relay information (EXOS-27853, EXOS-28306) is the
root cause of customer issue or not.
Environments:
Super/Sub VLAN (VLAN Aggregation)
DHCP Relay is configured in the Super VLAN.
Symptoms:
Memory leaking occurs in the kernel device driver for user application, leading into memory shortage
Affected versions:
EXOS 30.6, 30.7.1.1.1, 30.7.1.1.1-patch1-23, 30.7.1.1-patch1-54, 30.7.1.1-patch1-86, 30.7.2
EXOS 31.1, 31.2
ISSUE VLANMGR CRASH BY major Pass This rule is to check whether problem xos0075383 is the root cause of customer
SNMP POLLING
issue or not.
Affected versions:
EXOS 16.2.1 - 16.2.4
EXOS 16.2.5.4-patch1-3 and EXOS 16.2.5.4-patch1-25
EXOS 22.1 - 22.6 and 22.7.1
EXOS 30.1 - 30.6
Symptom : VLANmgr crash with signal 11 seen with SNMP polling sometimes on a
L2 switch
Remedy Action : Please upgrade the firmware version with EXOS 16.2.5.4-patch1-
29, EXOS 22.7.2 or EXOS 30.7 or higher.
ISSUE SYSTEM PROCESS major Pass In normal environment, messages, exchanged among EXOS processes, must
MESSAGE QUEUE DROP
CHECK
not be dropped. If the dropped count for the process is observed, there would be
possibility that the process may be experiencing delay in processing messages.
Please check whether there is a loop condition in the network environment
and escalate the case.
4/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE STP MEMORY LEAK major Pass Check whether STP memory leak issue is exposed to the backup node in the
WITH STACKING MLAG
stacking. Even if STP is disabled, this leaking issues were observed.
Cause: Backup node doesn't need to create MLAG checkpoint message. However,
the issue causes Backup node to create checkpoint message for MLAG peer
without being freed.
Refer to JIRA-27385
ISSUE STACK X465 POE NON major Pass This rule is to check whether unsupported 3rd party stacking cable or optics are
MASTER NODE NOT DELIVER
POWER
affecting POE issues on the devices connected to non-master node in x465
stacking.
Condition:
x465 Stacking
In output of "show power detail", PSU PartInfo and Revision data for non-master PSUs show as 'ÿ's or is
otherwise garbled
Symptom: Only devices on the master node are powered but other nodes
Please refer to article x465 PoE Stack Only Powers Devices on Master Node.
Remedy Action : It seems like unsupported 3rd party stacking cables or optics are
used. This may trigger incorrect POE operation. Please use only Officially
Supported Optics, especially on stack ports to allow for proper hardware bus
communication.
ISSUE STACK TEMPORARY major Pass This rule is to check whether system is exposed to temporary loop issue when
LOOP MASTER FAILOVER LAG
EXOS 30.7.1.1 CHECK
master slot reset with link aggregation across slots.
Environments:
EXOS Version: EXOS 30.7.1.1, EXOS 30.7.1.1-patch1-23, EXOS 30.7.1.1-patch1-54,EXOS 30.7.1.1-
patch1-86, EXOS 31.1.1.3
Link Aggregation across slots
Platforms: Stacking
Master Slot Reset with Link aggregation Ports
Symptoms:
Temporary Loop occurs over LAG ports
if ELRP is enabled, temporary loop can be detected by ELRP and logs are generated.
Remedy Action :
Upgrade firmware to ≥ 30.7.1.1-patch1-103, 30.7.2.1, ≥ 31.2.X
ISSUE STACKING STACKPORT major Pass In the EXOS Stacking, Stack Port should be Operational State. If Port is
WRONG STATE CHECK
Operational State, it should have Control Path Active Status. Except Link Down
and Operational State, The stacking port may be stuck into No Neighbor(it doesn't
find any neighbor which was caused by no stacking BPDUs from neighbor) or
Inhibited State (duplicate slot from stack merging). In the loop condition,
because of CPU congestion, Stack BPDU may be dropped and led into no neighbor.
ISSUE STACKING NODE major Pass In the EXOS Stacking, if the slot is not matched with configured type, the
MISMATCH STATE CHECK
corresponding slot failed state. In the show stacking, it is displayed with <none>'s
role. In the show slot detail, its state is shown as Mismatch. Please refer to article
Summit Stacking "show stacking" shows slot role as none. In most of cases, this
problem happens when one of slots is replaced with new switch (different type).
5/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE SILENT REBOOT 590 major Pass This problem is related with X590/X690/X870 switch just reboots without any crash or real
690 870 power loss. In the logs you only see:
02/13/2021 08:21:05.99 <Warn:EPM.UnexpctRebootDtect> Booting after System
Failure.
Platforms:
Cause
Article: KB 94856
Workaround: No, Please refer to the above knowledge article for the further
actions.
ISSUE PORT NOT UP BD major Pass This rule is to check whether problem EXOS-27627 is the root cause of customer
10G24XC MODULE
issue or not.
Affected versions:
EXOS 16.2.5.4-patch1-24
Condition:
Switch type: BD8810 or BD8806
Module type: 8900-10G24X-c
Remedy Action : Please upgrade the firmware version with EXOS 16.2.5.4-patch1-
29.
ISSUE POLICY DYNAMIC VLAN major Pass This rule checks whether switch is exposed to issue that dynamic VLAN is not
PRUNE FAIL STATIC VLAN
NOUPLINK EXOS 30.6 CHECK
deleted when there is no authenticated port in the dynamic VLAN and uplink port
where FA server exist is not added into the static VLAN when port is authenticated
over static VLAN.
Symptoms:
Static VLAN: uplink port is not added in the static VLAN when the port is authenticated
over static VLAN at first
Dynamic VLAN: when the last authenticated port is removed from the dynamic VLAN,
dynamic VLAN is not deleted (prune).
6/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE NODEALIAS CRASH major Pass This rule is to check whether nodealias process crash is caused by problem (EXOS-
VPEX STACK EXOS 22 30 7
CHECK
27469, EXOS-28076, EXOS-28320).
Conditions:
VPEX with BPE switches or STACK
nodealias process crash even if nodealias is not enabled
SNMP GET for nodealias MIB can trigger the problem
When BPE s/w is added or stack node is added
Affected versions:
EXOS 22.1 - 22.7
EXOS 30.1-30.6,30.7.1.1, 30.7.1.1.1-patch1-23, 30.7.1.1-patch1-54
EXOS 31.1
Remedy Action :
Please upgrade the firmware version with ≥ 30.7.1.1-patch1-86, ≥30.7.2.x or ≥31.2 or
remove nodealias mibview by command “configure snmpv3 add mib-view defaultUserView subtree
1.3.6.1.4.1.52.4.1.3.7.1.1.8 type excluded”
ISSUE MVRP PORT ADD VLAN major Pass This rule is to check whether problem EXOS-16998 is the root cause of customer
FAIL VPIF INSTANCE MAX
issue or not.
Affected versions:
EXOS 22.1 - 22.6
EXOS 30.1 - 30.3
Condition:
MVRP is enabled
"KERNEL_EXVLAN_ERROR: failed to find VPIF" error is seen in logs
Remedy Action : Please upgrade the firmware version with EXOS 30.4 or later.
ISSUE MRP CRASH MVRP LAG major Pass This rule is to check whether system is exposed to MRP process crash problem
PORTS EXOS 30.2 CHECK
when MVRP is enabled over port and the port is configured with link
aggregation(LAG).
Symptoms:
MRP Process crash when executing the show mvrp tag command
Remedy Action :
Upgrade firmware into ≥ EXOS 30.7.2.1
ISSUE MLAG VPEX ISCPORT major Pass This rule checks whether switch is exposed to issue that ISC Port is removed from
REMOVAL FA CLIENT
AGEDOUT EXOS 30 CHECK
all the dynamic VLANs when FA-Client is aged out.
Symptoms:
ISC Ports are removed from all Dynamic VLANs even if the other active FA client ports exist. it affects all
clients in the dynamic VLANs.
Remedy Action :
Upgrade firmware into ≥ EXOS 30.7.1.1-patch1-103
7/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE MLAG PEER major Not Relevant In normal status, peer connection failure count of MLAG should be zero. If this
CONNECTION FAILCOUNT
CHECK
count increases continuously, it can introduce wrong behaviors (VPEX with MLAG
for ring formation or MLAG only features). Please check ISC VLAN status (any port
in the ISC VLAN should be up status without any CRC values) and also CPU loads.
ISSUE MLAG ISC VLAN major Pass When MLAG is used, ISC VLAN must be disabled for routing (disabled
ROUTING CHECK
IPForwarding). MLAG peers communicate with each other via ISC VLAN. If routing
is enabled, the reachability can be achieved via shortest path (Inter-switch link) and
also via routing path(routed via routing protocol). This can lead into the situation
that peer can't detect losing neighbor via routing path even if the direct path is
broken or disconnected.
Remedy Action: disable routing or ipforwarding for ISC vlan via "disable
ipforwarding vlan <ISC vlanname>" command.
ISSUE MLAG ISC PORT major Pass In EXOS, ISC VLAN is used to exchange information between MLAG peers (please refer to the
RATELIMIT CHECK EXOS MLAG requirements documents. Broadcast, Flooding, Multicast Rate limit must not
be configured for Ports in the ISC VLAN
ISSUE MLAG ISC PORT ELRP major Not Relevant In EXOS, ISC VLAN is used to exchange information between MLAG peers (please refer to the
EXCLUDE PORT CHECK EXOS MLAG requirements documents). ISC Ports (in the ISC VLAN) should be defined in
the excluded list of ELRP disabled ports. Please refer to limitation section of ELRP excluded
port list documentation.
ISSUE LOG HAL PROCESS major Not Relevant This rule checks whether problem EXOS-23830 affects customer switch or not.
CRASH EEPROM CHECK
The affected version: EXOS 22.1 - 22.6
Remedy Action: Please upgrade switch with EXOS version ≥ 22.7.1 version.
ISSUE HAL CRASH SRP PORT major Pass This rule is to check whether HAL Crash occurs when port flaps with SRP feature
FLAP XOS 22.5 30.7 CHECK
happens.
Environments:
SRP(Software Redundant Port) Feature is configured
SRP Port flaps (up/downs) happens
HAL Process crash with signal 6
Symptoms:
HAL Process with signal 6 crash.
Affected versions:
8/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE FDB CRASH UPGRADE major Pass This rule is to check whether problem EXOS-27082 is the root cause of customer
TO 30.X PLATFORM X435
issue or not.
Affected versions:
EXOS 30.1 - 30.5
EXOS 30.7.1.1
EXOS 30.7.1.1-patch1-23
Symptom : Fdb process crash when upgrading X435-8p to EXOS 30.7.x versions
prior to 30.7.1.1-patch1-54
ISSUE FABRIC ATTACH STATIC major Pass This rule is to check whether problem xos0077879 is the root cause of customer
NSI BINDING PENDING CHECK
issue or not.
Affected versions:
EXOS 30.1 - 30.4
EXOS 30.6.1.11-patch1-2 and EXOS 30.6.1.11-patch1-4
Symptom : Static NSI bindings go into the pending state when dynamic binding with
identical NSI is removed.
Remedy Action : Please upgrade the firmware version with 30.7 or higher.
ISSUE EXSSHD PROCESS major Pass This rule checks whether problem EXOS-27541 affects customer switch or not.
CRASH WITH EXOS 30.7.1.1
The affected version:
EXOS 30.7.1.1
EXOS 30.7.1.1-patch1-24
9/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE EXSSHD CRASH EXOS major Pass This rule is to check whether system is exposed to EXSSHD(ssh daemon) Process
30.7.1.1 CHECK
crash problem.
Environments:
EXOS Version: EXOS 30.7.1.1, EXOS 30.7.1.1-patch1-23
SSH Feature : Enabled
Platforms: All Platforms
Symptoms:
EXSSHD Process Crash happens and then system resets with service interruption
Remedy Action :
Upgrade firmware to ≥ 30.7.1.1-patch1-54
ISSUE ELRP PROCESS CRASH major Pass When invalid packets with ELRP mac address is received in the switch, it can lead
MALICIOUS TRAFFIC TCP
into ELRP process crash because there was no validation in the for ELRP header.
In case of VPEX environment, even if ELRP is disabled, this problem can happen
because BPE switch is programmed with ELRP mac address of CB switch.
Remedy Action: Please apply ACL filter into user ports of BPE switch. In the below
example, change ethernet-destination-address as CB's ELRP MAC address in the
VPEX environment or as switch's ELRP address in the non-VPEX environment.
The fix is in 22.7.3, 30.7.1.1-patch1-86, 30.7.2, 31.2. ≥31.3.
entry elrp_mf_BB1 {
if match all {
protocol tcp ;
ethernet-destination-address 01:04:96:ce:7a:84 ;
then {
count elrp_mf_bb1_denied ;
deny ;
ISSUE DIRECT BROADCAST major Pass This rule checks whether switch is exposed to issue that flooding & loop occurs by
MLAG FLOOD LOOP 30.7
CHECK
“enable ipforwarding broadcast vlan <vlanname>” configurations in the MLAG
environment.
Symptoms:
Continuous Broadcast traffic transmission over ISC ports and normal ports (even if one directed
broadcast packet is received)
CPU utilization increases with CPU congestion and losing control packets
Remedy Action :
Upgrade firmware into EXOS 30.7.2.1-patch1-20 or 31.4
disable ipforwarding broadcast vlan <vlanname> for vlans
ACL to block traffic to CPU
10/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
REC TEMPERATURE CHECK warning Pass Check temperature of each slot in the stack and chassis product and standalone
switch. If the temperature is beyond upper 95% and under lower 95% of normal
temperature range, there will be possibility that temperature will go cross the
normal temperature range. This rule checks whether the current system is
under the right condition for temperature. Please ask customer or partner to
monitor the temperature closely (also check whether the right facility is set up to
keep temperature under control).
REC SYSTEM FREE MEMORY warning Pass Check whether Free Memory is below 10% of total memory. Please check memory status regularly by "show
CHECK memory detail" command. If free memory continues to decrease, there is possibility of memory leaking from
system or user processes.
REC MLAG PORT NETLOGIN warning Not Relevant Netlogin must not be enabled for MLAG Ports (MLAG Port doesn't support Netlogin).
CHECK Please refer to the limitation and requirement section of MLAG in the documentation.
REC MLAG ISC PORT SHARING warning Pass In EXOS, ISC VLAN is used to exchange information between MLAG peers (please refer to the
CHECK EXOS MLAG requirements documents. To provide higher availability from link failure, port
sharing (link aggregation) using multiple ports are strongly recommended.
Remedy Action: Even if one port with ISC VLAN doesn't introduce problem,
recommend using multiple ports with link aggregation(port sharing).
REC MLAG ISC EAPS warning Not Relevant In EXOS, Port in the ISC VLAN must not be configured as EAPS Ring Secondary Port(which will
SECPORT CHECK be blocked). Please refer to documentation MLAG Limitations and Requirements. it's not
supported configuration.
REC MLAG EAPS RINGPORT warning Not Relevant In EXOS, MLAG Port must not be configured as EAPS Ring Port (Primary Port or Secondary
CHECK Port). Please refer to documentation MLAG Limitations and Requirements. it's not supported
configuration.
REC IPMCFWD PIM CONFIG warning Pass Check whether IPMC forwarding is disabled but PIM is configured. To make PIM work correctly, IP Multicast
CHECK Forwarding(enable ipmcforwarding) should be configured.
REC IPFWD OSPF CONFIG warning Pass Check whether IP Forwarding(routing) is disabled but OSPF is configured. To make OSPF work correctly over
CHECK the VLAN, IP Routing(enable ipforwarding) should be configured.
REC IPARP ENTRIES MAX warning Pass This rule checks whether the current number of IPARP entries is more than 90% of
CHECK
maximum IPARP entries. If true, recommend increasing maximum number of ARP
entries.
REC HAL L3 HWTABLE COUNT warning Pass Check whether the number of used L3 Entries is the same across slots(stack and chassis products). If the
CHECK numbers are different with higher margin in the stable environment, there would be possibility that ARP entries
may not be synchronized. In this case, recommend "clear iparp" to synchronize table across slots (Be careful at
service impact when "clear iparp" command is executed)
REC HAL IPV4ROUTE warning Pass Check whether utilization of hardware IPV4 Route Table is full or not (show iproute reserved-entries statistics). If
HWTABLE FULL CHECK it's full, default route is removed and the traffic without programmed route entry will be sent to CPU for software
forwarding. it will impact service. Please check out whether routes from static or dynamic routing protocol can be
reduced by summarization.
REC HAL IPV4MC HWTABLE warning Pass Check whether the number of the used IPV4MC Entries is the same across slots(stack and chassis products). If
COUNT CHECK the numbers are different with higher margin in the stable environment, there would be possibility that IPV4
multicast caches are not synchronized.
REC EAPS RINGPORT EDP warning Not Relevant In EXOS, EAPS RING Ports (configured as primary or secondary port) must be enabled
ENABLE CHECK for EDP protocol. This rule checks whether EAPS Ring Ports are enabled for EDP (Extreme
Discovery Protocol).
ISSUE TRIAL LICENSE CHECK warning Pass In EXOS, trial license can be enabled for POC or internal uses. The valid period
for using trial license is fixed. Please communicate with customer or partner to use
the purchased licenses or system default license.
11/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE STACK SYSTEM TFTP warning Pass Stacking with the following versions can't be upgraded with post 30.2 version
UPGRADE TO POST 30 2
VERSION
because TFTP block size was incorrectly set.
Affected versions:
EXOS 30.1
EXOS 22.x versions prior to EXOS 22.5.1.7-patch1-7 or 22.6.1.4-patch1-8
Refer to Article Upgrading to 30.x fails with tftp: write error on SummitStack.
Remedy Action:
If using EXOS 30.1, then upgrade to 30.2.1.8 GA before attempting 30.x install
If using EXOS 22.x versions, upgrade to EXOS 22.5.1.7-patch1-7 or EXOS 22.6.1.4-patch1-8 before
attempting 30.x install
ISSUE XIQ REACH XIQCLOUD minor Pass This rule is to check whether system tries to reach XIQ Cloud even if iqagent feature
DISABLE IQAGENT EXOS 30
CHECK
is disabled and auto-provision is disabled.
Symptoms:
Switch still tries to reach XIQ Cloud. In the firewall, packets from switch to cloud xiq are
found.
No service impact
entry block_cloud_connectivity {
if match all {
then {
deny ;
ISSUE XIQ FIRST CONFIG minor Pass This rule is to check whether system is exposed to port flap issue for initial
PUSH PORT FLAP DEFAULT VR
EXOS 30 CHECK
configuration push from XIQ when Default-VR is used for connection into XIQ.
When the initial configuration is pushed by XIQ, it sends command “auto on” into
the switch where auto-negotiation is already applied into the port.
Symptoms:
Updates to EXOS (configuration push) fail after a timeout at 30% completion and then are
restored because of port flap.
Remedy Action :
Upgrade firmware into EXOS 30.7.2.1, ≥EXOS 31.3
12/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE VPEX IPMC BPE REPL minor Pass If IPMC replication is configured in the BPE and CB or BPE slot reboots, IPMC
ERROR CB SLOT RESET 30.X
31.1
Errors with pibIPmcEcidUpdateAsyncQueue is logged and traffic duplication and traffic
loss are observed. This rule checks whether IPMC Error with pibIPmcEcidUpdateAsyncQueue
exists and VPEX mode is running with Ring Topology + One Armed MLAG.
Refer to EXOS-26582 Multicast traffic loss observed in enterprise bed VPEX ring +
MLAG setup
ISSUE VPEX IPMC BPE REPL minor Pass If IPMC replication is configured in the BPE, and software version is 30.6, IPMC
ERROR
Errors with pibAsicUpdOneDot1brPortEgressReplication is logged and traffic
duplication and traffic loss are observed. This rule checks whether IPMC Error with
pibAsicUpdOneDot1brPortEgressReplication exists and VPEX mode is running and the software
version is running 30.6.x version.
Refer to Traffic loss and duplication occurs in VPEX environment when using BPE
IPMC replication mode. Article.
Remedy Action: configure forwarding vpex ipmc replication controlling-bridge or upgrade the
image into ≥ 30.7.1.1 patch release
ISSUE VPEX CB IMAGE CHECK minor Pass Check CB(Control Bridge) runs the right EXOS image to allow BPE(bridge port
extender) to be attached into CB . The CB image should have onie- prefix for image
name. Please download image for VPEX and install image into CB.
Refer to V400 is not passing traffic after enabling vpex in the controlling bridge.
Remedy Action: Upgrading the Controlling bridge with .xmod image or .lst image will push correct image
to Bridge port extender
ISSUE STP CRASH MSTP minor Pass This rule is to check whether system is exposed to STP process crash problem
AUTOEDGE CHANGE EXOS
22.6 30.7 CHECK
when multiple MST instances are configured and auto-edge configuration is
changed (off→ on, on→ off).
Symptoms:
STP Process crash if auto-edge configuration is changed
Remedy Action :
Upgrade firmware into ≥ EXOS 22.7.3.5-patch1-4, ≥ EXOS 30.7.2.1
ISSUE STOPPED PROCESS minor Pass This rule checks process' status and notifies process with not ready status (except
CHECK
not start from license). In case of tftpd process, it can be stopped by issuing
command “stop process” command. If tftpd process is stopped, I/O card in the
chassis or Stacking Slot can't download image from master or MSM/MM during boot
or installation of image which leads into failure of slot.
13/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE STACK POE CRASH minor Pass This rule is to check whether system is exposed to POE process crash problem in
RUN TECHSUPPORT EXOS
22.6 30.4 CHECK
the stacking environment with POE switches.
Environments:
EXOS Version: EXOS 22.6, EXOS 22.7.1 EXOS 30.4 - 30.6
Platform: Stacking with POE switches
Symptoms:
POE Process Crash while running tech-support
running CLI command “show inline-power fast”
Remedy Action :
Upgrade firmware to ≥ 22.7.2.4-patch1-3, ≥ 30.7.1.1
ISSUE STACK DOT1X RANDOM minor Not Relevant This rule checks whether problem xos0076201 affects customer switch or not. The
FAIL
affected version: EXOS 30.1 - 30.3.
Remedy Action: Please upgrade switch with EXOS version ≥ 30.4 version.
ISSUE STACKING STACKMAC minor Not Relevant In the EXOS Stacking, Recommend that all nodes should use the same Stack MAC
CHECK
addresses to reduce service interruption when failover happens to the node and the
node has different Stack MAC address from the previous Stack MAC address.
Remedy Action: Please use command “configure stacking slot [slot-number] stack-
mac-address” for the slot which uses different stack mac address.
ISSUE STACKING NODE minor Not Relevant In the EXOS Stacking, All nodes must be active status. This rule checks whether
ACTIVE STATUS CHECK
any node in the stacking is not active status. If “show tech” file is captured
during stack initialization, all nodes in the stacking may not be synchronized. In the
normal running status, all stack nodes should be active status.
ISSUE SRP REDPORT LINK ON minor Pass This rule is to check whether SRP Redundant Port with Link on feature doesn't work
NOACTIVE X440G2 X450G2
X460G2 CHECK
with 1G Link over 10GE ports of 440G2/X450G2/X460G2.
Environments:
SRP Redundant Port over 10GE ports
SRP Redundant Port Link on feature is configured
10GE port is worked as 1G speed with 1G optic
X440G2/X450G2/X460GE with 10GE ports (works as 1G speed with optic)
Symptoms:
Redundant Port doesn't come up even if link on feature is enabled and transceiver info shows receiving
power.
Affected versions:
EXOS 30.7.1.1.1, 30.7.1.1.1-patch1-23, 30.7.1.1-patch1-54, 30.7.1.1-patch1-86, 30.7.1.1-patch1-103
EXOS 31.1, 31.2
Remedy Action : Please upgrade the firmware version with ≥ 30.7.2, ≥31.3 or use
10GE optic instead of 1G
14/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE SHOW FABRIC ATTACH minor Pass This rule is to check whether problem EXOS-27557 is the root cause of customer
ELEMENTS MISSING CHECK
issue or not.
Symptoms:
The customer is unable to see fabric attached elements, but elements are able to communicate without
any issues.
Switches with less than 20 fabric attached elements can be seen via <show fabric attach elements> but
switches with more than ~25 elements can't be seen.
All attached elements are seen via command <show fabric attach statistics>
Affected versions:
30.6.1.11-patch1-11 and < EXOS 30.7.1.1-patch1-86
ISSUE PORT SUPPORTED minor Pass Check whether 100G QSFP28 Cable with 10 meter is recognized as not supported
MEDIA CHECK Q28+OC10m
X695 X870
media type even if the cable is purchased from Extreme (Part Number: 10436).
ISSUE PORT SUPPORTED minor Pass Check whether EXTREME supported GBIC is used or not. If 3rd party GBIC is used,
MEDIA CHECK
it can increase possibility of introducing port's errors or I2C bus issues.
! : unsupported optic. it can be used for trial period (90 days). After 90 days'
trial, it become restricted mode with performance restriction. To use optic
without any restriction, please purchase 3rdpartyoptic license and install it.
ISSUE PORT RESTRICTED minor Pass Check whether port is working with restricted mode or not. If 3rd party optic is used
MEDIA CHECK
without license for more than 90 days, the port will be imposed with performance
restriction.
$ : restricted optic. it had been used for trial period (90 days) and set to
restricted mode. To use optic without any restriction, please purchase
3rdpartyOptic license and install it.
ISSUE PORT PARTITION 1X10G minor Pass This rule is to check whether system is exposed to CLI configuration error of port
PORT 49 50 101 102 ERROR
X770 EXOS 22.6 CHECK
partition 1X10G for port 49,50,101,102.
Environments:
EXOS Version: EXOS 22.6, EXOS 22.7.1 - 22.7.4
Platform: X770
Ports: 1X10G partitioned for any ports among 51,52,103,104
LAG is configured in the above ports.
Symptoms:
Partition command (configure ports [port-number] partition 1X10G for 49,50,101,102 port fails
Remedy Action :
Disable sharing over the ports (51,52,103,104) at first(Traffic interruption occurs) and then configure port
partition on the ports (49,50,101,102).
15/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE ONIE PLATFORM 40G minor Pass This log messege will appear for any 40G optic inserted in ports 57 and 65 (which
PORT RECOGNITION ERROR
are 40G compatible for X690, X870 platform. After Reboot for the ports 65 and 57
which are 40G speed compatible by default are throwing log message as those
'ports not recognized as 100Gb
compatible.port should be configured to operate in appropriate mode.'
Refer to Article Switch incorrectly logs message that a port partitioned for 40G is not
recognized as 100Gb compatible. .
Remedy Action: upgrade the image into ≥ 22.7.1.2 (preferred) or 22.5.1.7-patch1-9 release
ISSUE NETLOGIN FAILS STATIC minor Pass This rule checks whether switch is exposed to issue that Netlogin Client fails
VLAN ISID RADIUS ISID
MAPPING EXOS 30.7 CHECK
authentication when switch is configured with static VLAN + ISID mapping and the
radius provides VLAN-ISID mapping as result of authentication (FA-VLAN-
ISID=VLANID:ISID)
Symptoms:
ISSUE LLDP PKT DROP WHEN minor Pass This rule is to check whether problem xos0078454 is the root cause of customer
NETLOGIN AUTH ONEPOLICY
issue or not.
Affected versions:
EXOS 22.1 - 22.6 and 22.7.1
EXOS 30.1 - 30.5
Condition : Netlogin, Policy and LLDP are enabled in one of affected versions
above.
Symptom :
LLDP packets are not sent/rcvd after successful authentication
some information (i.g. VLAN-ID) is not delivered in LLDP packet
Remedy Action : Please upgrade the firmware version with EXOS 22.7.2 or 30.6 or
higher.
ISSUE KERNEL OOPS IP minor Pass This rule is to check whether problem IP defragmentation Kernel oops issue(EXOS-
DEFRAG EXOS 30 CHECK
27710,EXOS-28417) is the root cause of customer issue or not.
Symptoms:
Kernel oops happened with service impact
Affected versions:
EXOS 30.6, 30.7.1.1, 30.7.1.1.1-patch1-23, 30.7.1.1-patch1-54, 30.7.1.1-patch1-86
EXOS 31.1, 31.2
16/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE HAL IPARP L3 HWTABLE minor Pass Check whether IPARP entries are programmed into hardware correctly by
SYNC CHECK
comparing the number of IPARP entries with hardware resource occupancy for IPAR
entries (HWL3ROUTE with /32 prefix, HWL3TABLE with hash). The good condition
is IPARP - Bias < HWL3ROUTE + HWL3TABLE < IPARP + Bias. if the above condition
fails, we can consider there would be something wrong IPARP table between in the
S/W Table and HW Table. Bias Value is used to consider gap between learning bulk
of IPARP entries and programming into hardware.
Additionally, if you look at higher IPMC entries in the system, please reserve more
resource for IPARP(/32) into HWL3ROUTE Table.
ISSUE FDB ENTRY DROP minor Pass In show FDB command, FDB dropped should be zero in normal situation. If the count is not zero, there is
CHECK possibility that FDB hash collision can occur because of hash collision in the FDB (the number of FDB entries
mapped into the same hash bucket is more than size of hash bucket. Please check the number of occupied FDB
entries in the hash bucket.
ISSUE ELRP PORT DOWN minor Pass Problem Description: Port goes into a disabled state
Platform: EXOS
Condition: Look for the following log in show log “LOOP DETECTED : xxx
transmitted, xxx received, ingress slot:port (x) egress slot:port (x)”
Affected Version: NA
ISSUE ELRP NOT DETECTING minor Not Relevant Rule Name:- ISSUE_ELRP_NOT_DETECTING_LOOP_ON_NETLOGIN_PORTS.
LOOP ON NETLOGIN PORTS
Problem description:- ELRP doesn’t detect a loop when it is enabled on the
netLogin ports. Connect three switches in a point to point connection. Now, form a
physical loop by connecting a link between the two netLogin ports, such that these
three switches would be in Ring topology. In the ELRP configuration, exclude the
uplink port to the Core switch and disable the port in the egress direction.
Platform:- EXOS
Cause:- As the ELRP source MAC has gone unauthenticated on the peer switch
and since it is dropped in the midway. We could see that the ELRP packets were not
received back by the sending switch, resulting in ELRP not detecting a loop, even
though when a loop existed.
Condition:- Look for the following log in show log “Authentication failed for Network
Login MAC user 0EXXXXXXXXXX Mac 0E:XX:XX:XX:XX:XX port x".
Note: If the MAC address of the switch sending the ELRP packet is
"00:04:96:9A:8B:1B”. Then the ELRP source mac-address would be represented as
0E: Switch-mac (0E:04:96:9A:8B:1B).
17/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
ISSUE BOOTP RELAY WHEN minor Pass This rule is to check whether switch is exposed for the problem that it forwards the
YIADDR 0.0.0.0
BOOTP response packets into wrong interface via false routing lookup (must not do)
when BOOTP Relay packet's yiaddress (Your IP address) is 0.0.0.0.
Affected versions:
Symptom :
BOOTP client (PXE) can't receive BOOTP response packets from BOOTP server
via switch's BOOTP relay.
ISSUE 440G2 PORT LICENSE minor Pass In 440G2 Product, Activating 10GE ports without Port License becomes 1G Ports.
CHECK
To use 10G speed, it requires dual port or quad port license. This rule checks
whether switch has “<Warn:HAL.Port.Chk10GPortLcnsFail> Slot-1: 10G license not
present for port 1:51. Speed will be reduced to 1Gbps with auto-negotiation on” log.
In case of stacking, it needs at least Dual Port license.
ISSUE VRRP ADVERTISE VRID minor Pass Problem description: The customer reported that they are noticing slowness in
IGNORED VIP LIST MISMATCH
their network, when logged into the switch, VRRP related error logs are seen in the
switch.
Platform: EXOS
Cause: The VRRP VIP mismatch logs would be observed when there is a physical
layer 2 loop in the network or if there is any VRRP misconfiguration.
REC VRRP ADVERTISE info Pass VRRP default Advertisement Interval is 1 seconds. If it configures below 1 seconds, it can lead into high CPU
INTERVAL CHECK utilization and cause dual master situation to happen to impact service.
REC SYSTEM SERVICE info Pass Check whether telnet is enabled or not in the system. In case of security compliance(GDPR,PCI,HIPPA),
TELNET ENABLE CHECK recommend disabling telnet.
REC SYSTEM SERVICE info Pass Check whether SNMPv1/v2 is enabled or not in the system. In case of security compliance(GDPR,PCI,HIPPA),
SNMPV1V2 ENABLE CHECK recommend disabling SNMPv1/v2 and enable SNMPv3
REC SYSTEM SERVICE NTP info Pass NTP is used to synchronize system clock with high precision time information from
CHECK
NTP server. Without time synchronization, system clock is drifting over time and
introduce issues for the protocol which needs high precision timing. In the Default
configuration, NTP is disabled. However, NTP should be enabled so that system
logs are correctly aligned with the correct time and the other time-based protocols
works correctly.
REC SFLOW PACKET DROP info Not Relevant This checks how many SFLOW packets are dropped in the switch. SFLOW drops
CHECK
can happen from higher number of SFLOW sampled packets and high loads of CPU
from busy tasks. If continuous drops happen, consider increasing sample rate for
SFLOW and reducing the number of ports for sampling.
18/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
REC PORT DUPLEX CHECK info Pass Port Duplex must be running as full duplex. If the port's duplex is not full, it can lead into CRC errors and
Collision, and lead into dropping throughput.
REC OSPF GRACEFUL info Not Relevant OSPF Graceful Restart is recommended configuration for OSPF when control
RESTART CHECK
module failover happens. Default is none (disabling graceful restart feature).
Recommendation is both (unplanned + planned).
REC IPARP ADDRESSCHECK info Pass IPARP Address Check is enabled in the default (show iparp command). If it's disabled, IPARP entries would be
ENABLE CHECK learned even if the ARP doesn't belong to IP address block assigned to VLAN.
REC IGMP SNOOPING CHECK info Pass In the default configuration, IGMP Snooping is enabled. If it's disabled, multicast
traffic will be flooded into active ports in the same VLAN which leads into higher
utilization of B/W even if end-station or client doesn't need to receive multicast.
Unless there is a reason to disable snooping, IGMP snooping should be enabled
REC IGMP CHECK info Pass IGMP must be enabled for the right operation of multicast traffic handling. In the
default configuration, it is enabled as default. If it's disabled, switch can't join in
IGMP querier election.
REC HAL SWITCHMODE info Pass Default Switch mode is store-and-forward (show forwarding configuration). if it's configured as cut-through, it can
CHECK lead into packets drop (traffic forwarding among different speed of ports).
REC HAL L2 info Pass Default L2 Fast Convergence is on (show forwarding configuration). If it's off mode, when topology changes
FASTCONVERGENCE CHECK occurs, Unknown multicast traffic doesn't be flooded. it can lead into longer convergence of multicast traffic.
REC HAL IPMC COMPRESSION info Pass Default IPMC Compression is on (show forwarding configuration). If it's off mode, each multicast group
CHECK consumes dedicate IPMC resources without sharing and system spends more IP multicast hardware resources.
In case of IPTV/IP multicast service environment, IPMC compression must be enabled.
REC FDB AGINGTIME CHECK info Pass Default FDB aging time is 300 seconds. This rule check whether FDB aging time is set as default or not. If the
value is higher, inactive client's MAC stays in the FDB longer and leads into higher number of FDB entries. If this
value is too small, more traffic from inactive clients will be flooded from early aging.
REC EAPS CONTROLVLAN info Not Relevant EAPS Control VLAN has two RING Ports (configured as primary or secondary port). If any
PORTNUMBER CHECK user ports (not ring port) must no be assigned into EAPS control VLAN, used for EAPS control
messages.
REC CUSTOMER SPECIFIC info Pass In EXOS, customer specific patch is released to cover specific issue or issues for specific
PATCH VERSION CHECK customer and the patch must be used for the specific customer. If the case is open from product
with customer specific patch release, please check whether new GA or formal patch release
covers the customer specific issues (covered by customer private patch). if it's covered,
please recommend upgrading the new available GA or patch release.
REC ACL REFRESH info Pass ACL Refresh Blackhole allows traffic during refreshing ACL to be dropped without
BLACKHOLE CHECK
leaking the traffic. Default state is in enable state.
19/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
Platform: EXOS
Remedy Action: - Collect a packet capture from one of the ports where the invalid
LLDPDU is being received by mirroring the port in the ingress direction. This would
help us to narrow down, which device is sending the invalid packets and what is
incorrect in the PDUs.
SYSTEM LOG CRITICAL critical Pass This rule check existence of any critical logs in the system and shows information.
CHECK
Please check error message from EXOS error message decoder
SYSTEM CRASH INFO CHECK major Pass This rule checks whether any system core dump information exists or not. if found, please
check the dump occurred in the same version of switch or old version. if it happens in the same
version, and compare the time when dump created with boot time of system and running period
to see any relevance , please follow the action described in the How to analyze an EXOS
core dump.
20/21
7/23/22, 2:06 AM 02627448_02627448_exos_02627448.txt
21/21