Nexus Network Management Best Practices Summary

TM
Advanced Services
Nexus Network Management Best Practices
Corporate Headquarters
Cisco
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 526-4100
Nexus Network Management Best Practices 1

Contents
Contents .................................................................................................................................................................... 2
Tables......................................................................................................................................................................... 4
About This Document .......................................................................................................................................... 5
1. Introduction ................................................................................................................................................... 6
1.1 Document Purpose .................................................................................................................................. 6
1.2 Intended Audience .................................................................................................................................. 6
1.3 Scope ............................................................................................................................................................. 6
2. Executive Summary ..................................................................................................................................... 7
2.1 Network Management............................................................................................................................ 7
2.2 Detecting required Actions .................................................................................................................. 7
3. Network Monitoring ................................................................................................................................... 9
3.1 Overview ..................................................................................................................................................... 9
3.2 System Resource Monitoring .............................................................................................................. 9
3.2.1 CPU ............................................................................................................................................................ 9
3.2.2 Memory ................................................................................................................................................... 9
3.2.3 Network Interfaces .......................................................................................................................... 10
3.2.4 Statistics and Connections details .............................................................................................. 10
4. Nexus 7000 .................................................................................................................................................. 11
4.1 Target Equipment ................................................................................................................................. 11
4.2 CPU Utilization ....................................................................................................................................... 11
4.3 Memory Availability............................................................................................................................. 13
4.3.1 Line card memory monitoring .................................................................................................... 16
4.4 Network Interfaces............................................................................................................................... 16
4.4.1 vPC and Peer link .............................................................................................................................. 18
4.5 Environmental Parameters ............................................................................................................... 19
4.6 Module Status ......................................................................................................................................... 23
4.7 HSRP........................................................................................................................................................... 25
4.8 COPP ........................................................................................................................................................... 26
4.9 Hardware rate-limiters....................................................................................................................... 27
4.10 Summary .................................................................................................................................................. 28
4.11 Other useful CLI commands .............................................................................................................. 38

Contents
4.12 References ............................................................................................................................................... 41

5. Nexus 5000 .................................................................................................................................................. 43
5.1 Target Equipment ................................................................................................................................. 43
5.2 CPU Utilization ....................................................................................................................................... 43
5.3 Memory Availability............................................................................................................................. 44
5.4 Network Interfaces............................................................................................................................... 45
5.4.1 vPC and Peer link .............................................................................................................................. 46
5.5 Environmental Parameters ............................................................................................................... 47
5.6 Module Status ......................................................................................................................................... 50
5.7 Summary .................................................................................................................................................. 52
6. Nexus 2000 .................................................................................................................................................. 60
6.1 CLI commands ........................................................................................................................................ 60
7. Summary....................................................................................................................................................... 65
Document Acceptance ...................................................................................................................................... 66

Tables
Table 1 SNMP Objects for CPU Monitoring ........................................................................................... 11

Table 2 SNMP Objects for Environmental Monitoring ....................................................................... 19
Table 4 SNMP objects for Module monitoring ...................................................................................... 23
Table 5 SNMP Notifications for Module Monitoring .......................................................................... 24
Table 6 SNMP OID’s for HSRP .................................................................................................................... 25
Table 7 SNMP Notifications for HSRP ...................................................................................................... 26
Table 8 Recommended SNMP OIDs for Nexus 7000 ........................................................................... 28
Table 9 Recommended SNMP Traps for Nexus 7000 .......................................................................... 35
Table 11 SNMP Objects for Module Monitoring ..................................................................................... 50
Table 12 SNMP Notifications for Module Monitoring .......................................................................... 50
Table 13 Recommended SNMP OIDs for Nexus 5000 ........................................................................... 52

About This Document
Author : Vinay Suvarna and Azeem Suleman

Authority : Cisco Advanced Services and Oracle
Reference Number :
History
Revision History
Version No. Issue Date Status Reason for Change

1.0 02/15/2012 Draft Initial
1.1 02/24/2012 Published Updated
Review
Revision Review
Reviewers Details Version No. Date

Balaji SenthamilSelvan 1.1 02/27/2012
Azeem Suleman 1.1 02/29/2012
Chip Nielsen 1.2 03/01/2012

1. Introduction
1.1 Document Purpose
The objective of this document is to provide a detailed view of the metrics that should
be monitored on Cisco Nexus 7000, Nexus 5000 and Nexus 2000 series switches for
network performance, capacity planning and fault management.
1.2 Intended Audience
The intended audience for the document is the Advanced Services Data Center
Networking Team. This document is intended for use by network engineers who are
responsible in providing Network Monitoring/Management and Capacity planning
services in an enterprise.
1.3 Scope
The scope of this document will focus on the key management features that should be
tracked / monitored for network performance, capacity planning and fault
management. This document is to provide the recommended information on SNMP
(preferred) or CLI command for better network management. The idea is that the
customer can build monitoring tools based on the information provided in this
document.
The following platforms with the NX-OS code running are targeted in this document:
 Nexus 7000 with Software release 5.2(3)a

 Nexus 5000 with Software release 5.1.3N1 with Nexus 2000
The methodology should however be applicable and reusable for validation against
future software releases.

2. Executive Summary
2.1 Network Management
Network Management is an integral part of today’s networks to improve the

operational efficiency, ensure superior performance, high availability and reduce
downtime. The two important functional areas of management we will be focusing in
this document are
Fault Management
The goal of Fault Management is to detect, log, notify users of, and (to the extent
possible) automatically fix network problems to keep the network running effectively.
Because faults can cause downtime or unacceptable network degradation, fault
management is perhaps the most widely implemented of the ISO network
management elements.
Performance Management
The goal of Performance Management is to monitor and measure various aspects of

performance so that overall performance can be maintained at an acceptable level.
At the device level, performance metrics can include CPU utilization, memory
allocation and Environmental Information. Various interface statistics can be
collected from network devices to measure the performance level.
An evaluation of the data gathered can be used for proactive capacity planning for
meeting future requirements.
While SNMP is primarily used to gather the data required, Syslog messages and CLI
commands also offer a wealth of information for Fault and Performance management.
2.2 Detecting required Actions
The purpose of monitoring can be seen around two main objectives characterized by
their short- or long-term focus.
On short-term, it is important to be able to detect unusual situations, events or

incidents that could potentially affect business operations. The idea is to be on the
preventive side as much as possible so that incidents are mitigated before they have any
impact.

Executive Summary
On long-term, the goal is to gather data that will help in the planning of network
upgrades, for example to achieve expected performance levels and preserve security
margins.
The goal of this document is to identify the key metrics that should be considered for
evaluating the Network performance. The data collected can then be used to establish
baseline values and threshold for the Network parameters.

3. Network Monitoring
3.1 Overview
In this section, we will describe the various aspects to be considered when talking about
monitoring. Monitoring can be requested for several reasons, which are typically
independent and provide best results when implemented using a global approach as the
tools and methods are similar.
Performance / capacity management and fault detection are applicable to all network
devices. Further down in the document, we will develop technical details and
procedures to evaluate the Network performance/planning.
It is important to note that, although recommendations can be made regarding the

desired value of some metrics, it is critical to establish a baseline profile based on the
actual values measured in the production network under “normal conditions”. Those
baseline values are then used as criteria to detect deviations.
3.2 System Resource Monitoring

3.2.1 CPU
The CPU utilization should be periodically monitored in order to both detect unusual
behavior as well as control long term resource management (track evolution over a long
period of time). Several metrics are available, each one being an average over a given
time period.
Although spikes will occur during normal operations, we recommend setting the
warning threshold around 80% on average to guarantee a correct mode of operation.
3.2.2 Memory
When considering the memory, several items deserve a careful monitoring.
First, any Cisco device has a total and finite amount of memory that can be used by the
various processes. It is mostly important to monitor the amount of available (free)
memory. Although it is difficult to define a threshold value as it would depend on many
factors, it is recommended to monitor the evolution of the available memory pool. A
continuous decrease of available memory could be the symptom of:

Network Monitoring
 Constantly increasing device utilization (this would be observed on the long

term).
 Memory leak caused by a defect in a process/code
3.2.3 Network Interfaces
As for other network equipment, it is recommended to monitor the various statistics

available for network interfaces. Those include volume of data transmitted (and
received) as well as errors and operational status.
3.2.4 Statistics and Connections details
On the Nexus 7000/5000 switches, statistics can be retrieved directly by a user issuing
show commands at the command line interface (CLI). Most of the statistics can also be
retrieved from a remote station through an SNMP request, the relevant MIBs and OIDs
are provided later in this document.
For automated connection and traffic details analysis, it is recommended to use

NetFlow. NetFlow is outside the scope of this document.

4. Nexus 7000
4.1 Target Equipment
The following device is targeted in this section:
 Hardware: Nexus 7000 series switches

 Software release 5.2(3)a
4.2 CPU Utilization

The following MIBs are used to monitor the overall information about the CPU load.
MIB: CISCOPROCESSMIB
Object: cpmCPUTotalEntry
OID: 1.3.6.1.4.1.9.9.109.1.1.1.1
Possible Return Values:

3: cpmCPUTotal5sec
4: cpmCPUTotal1min
5: cpmCPUTotal5min
Descriptions:
‘cpmCPUTotal5sec’: Average CPU utilization over the last 5 seconds
‘cpmCPUTotal1min’: Average CPU utilization over the last 60 seconds.
‘cpmCPUTotal5min’: Average CPU utilization over the last 5 minutes
Table 1 SNMP Objects for CPU Monitoring
MIB Object Name OID Description

CISCO-PROCESS-MIB cpmCPUTotal5secRev 1.3.6.1.4.1.9.9.109.1.1.1.1. The overall CPU busy
6 percentage in the last 5
second
period. This object
deprecates the object
cpmCPUTotal5sec
and increases the value
range to (0..100)

Nexus 7000
CISCO-PROCESS-MIB cpmCPUTotal1minRev 1.3.6.1.4.1.9.9.109.1.1.1.1. The overall CPU busy

minute
period. This object
cpmCPUTotal1min
range to (0..100).
CISCO-PROCESS-MIB cpmCPUTotal5minRev 1.3.6.1.4.1.9.9.109.1.1.1.1. The overall CPU busy
minute
period. This object
cpmCPUTotal5min
range to (0..100).
CISCO-SYSTEM-EXT- cseSysCPUUtilization 1.3.6.1.4.1.9.9.305.1.1.1 The average utilization of
MIB CPU on the active
supervisor
SNMP Traps for CPU Monitoring
The Nexus 7000 switch running 5.x does not currently support CPU-related traps defined
in the CISCO-PROCESS-MIB: cpmCPURisingThreshold and cpmCPUFallingThreshold.
Therefore, asynchronous notifications of violations of user-defined thresholds are not
possible.
SYSLOG Messages for CPU Monitoring
There is no Syslog event message defined for chassis-level CPU monitoring on the Nexus
7000 switch.
CLI Command
The show system resources command displays the high level CPU utilization for the
supervisor module. The show process cpu command with the sort option lists all of the
processes sorted by the highest CPU utilization per process. The show process cpu
history command displays the CPU history in three increments: 60 seconds, 60 minutes,
72 hours. Viewing the CPU history is valuable when correlating a network event with the
past CPU utilization.
It should be noted that Cisco NX-OS takes advantage of preemptive CPU multitasking, so
processes can take advantage of an Idle CPU to complete tasks faster. Therefore, the
history option may report CPU spikes that do not necessarily mean there is an issue.
Additional investigation should take place if the average CPU remains close to 100%.

Nexus 7000
n7000# show system resources

Load average: 1 minute: 0.06 5 minutes: 0.04 15 minutes: 0.00
Processes : 310 total, 1 running
CPU states : 0.0% user, 0.5% kernel, 99.5% idle
Memory usage: 4135780K total, 1180900K used, 2954880K free
0K buffers, 759580K cache
4.3 Memory Availability
The state of the memory status can be monitored with the following MIB.
MIB : CISCOSYSTEMEXTMIB
Object: ciscoSysInfoGroup
OID : 1.3.6.1.4.1.9.9.305.1.1

1: cseSysCPUUtilization
2: cseSysMemoryUtilization
Description:
‘cseSysCPUUtilization’: The average utilization of CPU on the active supervisor.
Thresholds for RMON probes should be set based on baselining the CPU
utilization within a production environment
‘cseSysMemoryUtilization’ : The average utilization of memory on the active

supervisor. Thresholds for RMON probes should be set based on baselining the
memory utilization within a production environment
MIB : CISCOENTITYEXTMIB
Object: ceExtPhysicalProcessorEntry
OID : 1.3.6.1.4.1.9.9.195.1.1.1

2:ceExtNVRAMSize
3:ceExtNVRAMUsed
Description:
‘ceExtNVRAMSize’: Total number of bytes of NVRAM in the entity. A value of 0
for this object means the entity does not support NVRAM or NVRAM information
is not available.
‘ceExtNVRAMUsed’: Number of bytes of NVRAM in use. This object is irrelevant if

ceExtNVRAMSize is 0

Nexus 7000
SNMP Traps for Memory Monitoring
Nexus 7000 does not support the notifications defined in the CISCO-ENHANCED-
MEMPOOL-MIB or CISCO-MEMORY-POOL-MIB. Thus, asynchronous alerts of system- or
customer-defined threshold violations are not possible. Use EEM scripts to create these
notifications if desired.
SYSLOG Messages for Memory Monitoring
If a memory threshold has been passed (OK -> MINOR, MINOR -> SEVERE, SEVERE ->
CRITICAL), the Cisco NX-OS platform manager will capture a snapshot of memory
utilization and log an alert to SYSLOG. This snapshot is useful in determining why
memory utilization is high. The log is generated and this log is very useful for
determining if memory utilization is high due to the memory that was consumed by the
page cache, kernel, or Cisco NX-OS user processes.
Syslog Message %PLATFORM-2-MEMORY_ALERT: Memory Status Alert: MINOR. Usage

85% of Available Memory
The show system internal memory-alerts-log command displays the memory alerts log.
CLI Command
To assess the overall level of memory utilization, use the basic CLI commands
show system resources and show processes memory.
The show system resources command displays platform memory statistics
N7K# show system resources

0K buffers, 1739356K cache
The show process memory command displays the memory allocation per process for
the current VDC. While this output is more detailed, it is only useful for verifying
process-level memory allocation within a specific VDC.
N7K# show processes memory

PID MemAlloc MemLimit MemUsed StackBase/Ptr Process
----- -------- ---------- ---------- ----------------- ----------
4662 52756480 562929945 150167552 bfffdf00/bfffd970 netstack

Nexus 7000
The show system internal kernel command or the show system internal memory-
alerts-log command for a more detailed representation of memory utilization in Cisco
NX-OS.
N7K# show system internal kernel meminfo

MemTotal: 4135780 kB
MemFree: 578032 kB
Buffers: 5312 kB
Cached: 1926296 kB
RAMCached: 1803020 kB
Allowed: 1033945 Pages
Free: 144508 Pages
Available: 177993 Pages
SwapCached: 0 kB
Active: 1739400 kB
Inactive: 1637756 kB
HighTotal: 3287760 kB
HighFree: 640 kB
LowTotal: 848020 kB
LowFree: 577392 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
Mapped: 1903768 kB
Slab: 85392 kB
CommitLimit: 2067888 kB
Committed_AS: 3479912 kB
PageTables: 20860 kB
VmallocTotal: 131064 kB
VmallocUsed: 128216 kB
VmallocChunk: 2772 kB
In the output above, the most important fields are as follows:
MemTotal (kB)- Total amount of memory in the system (4 GB in the Cisco Nexus
7000 Series Sup1)
Cached (kB) - Amount of memory used by the page cache (includes files in tmpfs
mounts and data cached from persistent storage /bootflash)
RamCached (kB) - Amount of memory used by the page cache that cannot be
released (data not backed by persistent storage)
Available (Pages) - Amount of free memory in pages (includes the space that
could be made available in the page cache and free lists)
Mapped (Pages) - Memory mapped into page tables (data being used by
nonkernel processes)
Slab (Pages) - Rough indication of kernel memory consumption
Memory Thresholds
Prior to Release 4.2(4), the default memory alert thresholds were as follows:

Nexus 7000
 70% MINOR
 80% SEVERE
 90% CRITICAL
From Release 4.2(4) and later releases, the memory alert thresholds were changed to
the following:
 85% MINOR
 90% SEVERE
 95% CRITICAL
This change was introduced in part due to baseline memory requirements when many
features/VDCs are deployed.
The thresholds are configurable, using the following command:
system memory-thresholds minor percentage severe percentage critical percentage
4.3.1 Line card memory monitoring
In NX-OS supports the MIB CISCO-MEMORY-POOL-MIB and two memory counters are
implemented in this MIB for each module present in the switch:
 Used memory (cempMemPoolUsed, .1.3.6.1.4.1.9.9.221.1.1.1.1.7)

 Free memory (cempMemPoolFree, .1.3.6.1.4.1.9.9.221.1.1.1.1.8)
An example of line card Memory polling in Nexus 7000 with release 5.0(2) is provided
below:
CISCO-MEMORY-POOL-MIB::cempMemPoolUsed.22.1 = Gauge32: 422817792

CISCO-MEMORY-POOL-MIB::cempMemPoolFree.22.1 = Gauge32: 637808640
CISCO-SYSTEM-EXT-MIB::cseSysMemoryUtilization.0 = Gauge32: 47
Cross-referencing with entPhysicalName and/or entPhysicalDescr to determine the

module name/type:
ENTITY-MIB::entPhysicalName.22 = STRING: "10/100/1000 Mbps Ethernet Module"

ENTITY-MIB::entPhysicalName.23 = STRING: "10 Gbps Ethernet Module"
ENTITY-MIB::entPhysicalName.26 = STRING: "Supervisor module-1X"
ENTITY-MIB::entPhysicalName.27 = STRING: "Supervisor module-1X"
4.4 Network Interfaces

Nexus 7000
The following MIBs contain management information applicable to a particular

interface:
MIB : IFMIB
Object: ifEntry
OID : 1.3.6.1.2.1.2.2.1

5: ifSpeed
7: fAdminStatus
8: ifOperStatus
9: ifLastChange
13: ifInDiscards
14: ifInErrors
15: ifInUnknownProtos
19: ifOutDiscards
20: ifOutErrors
Descriptions:
‘ifOperStatus’: Returns the operational status of the interfaces.
‘ifInDiscards‘: Indicates ingress packets that have been discarded on interfaces
due to no buffer availability. Typically indicates that the device is overloaded.
‘ifInErrors’: Indicates input errors on interfaces.
‘ifOutDiscards’: Indicates egress packets that have been discarded on interfaces
due to no buffer availability. Possible cause is low memory.
‘ifOutErrors’: Indicates output error on interfaces.
The following MIBs contain additional management information applicable to a

particular interface.
MIB: IFMIB
Object: ifXEntry
OID: 1.3.6.1.2.1.31.1.1.1

7: ifHCInUcastPkts
8: ifHCInMulticastPkts
9: ifHCInBroadcastPkts
10:ifHCOutOctets
11:ifHCOutUcastPkts
12:ifHCOutMulticastPkts
13:ifHCOutBroadcastPkts
17:ifConnectorPresent

Nexus 7000
Description:
Each possible return value is used to monitor the ingress/egress unicast/
multicast/ broadcast packets
SNMP Traps for Interface Monitoring
Traps can be generated for the status of the interfaces using the IF-MIB. The linkup and
linkdown is used if the operational status of the interfaces changes.
 snmp-server enable traps link linkDown

 snmp-server enable traps link Up
SYSLOG Messages for Interface Monitoring
Syslog messages are generated for many scenarios impacting the interface operational
status. An example for a Syslog message is as below
Syslog Message PORT-2-IF_DOWN_LINK_FAILURE_CRIT: Interface [chars] is

down (Link failure)
Explanation: The physical link has gone down
CLI Command
Use the below commands to verify information for any given interface
show interface brief displays the interface configuration information, including the
mode.
show interface capabilities displays information on the capabilities of the interfaces.
show interface counters [module module] displays input and output octets unicast
packets, multicast packets, and broadcast packets
show interface counters errors [module module] Displays information on the number
of error packets..
4.4.1 vPC and Peer link
The vPC link and the peer-links are critical to maintaining the vPC infrastructure
deployed. Any change on these two links must be acted upon immediately.
The following “facility” strings can identify events related to the vPC and peer-link:
 VPC
 ETH_PORT_CHANNEL
 ETHPORT

Nexus 7000
 ETH_PM
The below CLI commands can also be used to verify any issues with vPC.
show vpc command to check the status of vPC.

show vpc peer-keepalive command to check the status of the vPC peer-keepalive link.
4.5 Environmental Parameters
The Nexus 7000 switch implements the standard ENTITY-MIB, the CISCO-ENTITY-
SENSOR-MIB, the CISCO-ENTITY-EXT-MIB, and the CISCO-ENTITY-FRU-CONTROL-MIB.
These MIBs contain information about environmental status in the switch, such as
temperature sensors, fan tray status, power supply status, etc
Table 2 SNMP Objects for Environmental Monitoring

CISCO-ENTITY- entSensorType 1.3.6.1.4.1.9.9.91.1.1.1.1.1 This variable
SENSOR-MIB indicates the type of
data reported by the
entSensorValue.eg :
3:voltsAC
4:voltsDC
5:amperes
6:watts
CISCO-ENTITY- entSensorScale 1.3.6.1.4.1.9.9.91.1.1.1.1.2 This variable

SENSOR-MIB indicates the
exponent to apply
to sensor values
reported by
entSensorValue.eg :
6:nano
7:micro
8:milli
9:units
10:kilo
11:mega
CISCO-ENTITY- entSensorValue 1.3.6.1.4.1.9.9.91.1.1.1.1.4 This variable reports

SENSOR-MIB the most recent
measurement seen
by the sensor

Nexus 7000
CISCO-ENTITY- entSensorStatus 1.3.6.1.4.1.9.9.91.1.1.1.1.5 This variable

SENSOR-MIB indicates the present
operational status
of the sensor.
1:ok
2:unavailable
3:nonoperational
CISCO-ENTITY- cefcFanTrayOperStatus 1.3.6.1.4.1.9.9.117.1.4.1.1. The operational state

FRU-CONTROL- 1 of the fan or fan tray.
MIB unknown(1) -
unknown.
up(2) - powered on.
down(3) - powered
down.
warning(4) - partial
failure, needs
replacement
as soon as possible
CISCO-ENTITY- cefcTotalAvailableCurrent 1.3.6.1.4.1.9.9.117.1.1.1.1. Total current
FRU-CONTROL- 3 available for FRU
MIB usage
CISCO-ENTITY- cefcTotalDrawnCurrent 1.3.6.1.4.1.9.9.117.1.1.1.1. Total current drawn
FRU-CONTROL- 4 by powered-on FRUs.
MIB
CISCO-ENTITY- cefcPowerRedundancyMode 1.3.6.1.4.1.9.9.117.1.1.1.1. The administratively
FRU-CONTROL- 1 desired power supply
MIB redundancy
mode.
1:notsupported
2:redundant
3:combined
4:nonRedundant
5:psRedundant
6:inPwrSrcRedundant
CISCO-ENTITY- cefcPowerRedundancyOperMod 1.3.6.1.4.1.9.9.117.1.1.1.1. The power supply

FRU-CONTROL- e 5 redundancy
MIB operational mode.
1:notsupported
2:redundant
3:combined
4:nonRedundant
5:psRedundant
6:inPwrSrcRedundant

Nexus 7000
CISCO-ENTITY- ceExtEntityLEDColor 1.3.6.1.4.1.9.9.195.1.3.1.2 The color of the LED

EXT-MIB on the module
1 : off
2 : green
3 : amber
4 : red
Examples of Environmental status polling on Nexus 7000 are provided below.
Fan Tray Status
The physical index for the Fan modules can be determined from the ENTITY-MIB
description or name
# Physical index for Fan Modules

ENTITY-MIB::entPhysicalDescr.534 = STRING: "Fan Module-1"
The Fan module status can then be obtained from the CISCO-ENTITY-FRU-CONTROL-
MIB, and the LED color status from the CISCO-ENTITY-EXT-MIB:
# Fan module status

CISCO-ENTITY-FRU-CONTROL-MIB::cefcFanTrayOperStatus.534 = INTEGER: 2
# Fan module LED color
CISCO-ENTITY-EXT-MIB::ceExtEmtityLEDColor.534 = INTEGER: 2
Power Supply Status
The physical index for the Power Supply modules can be determined from the ENTITY-
MIB description or name:

Nexus 7000
# Physical index for Power Supplies

ENTITY-MIB::entPhysicalDescr.470 = STRING: "6000.00W 220v AC"
ENTITY-MIB::entPhysicalDescr.471 = STRING: "6000.00W 220v AC"
ENTITY-MIB::entPhysicalName.470 = STRING: "PowerSupply-1"
ENTITY-MIB::entPhysicalName.471 = STRING: "PowerSupply-2"
The Power Supply status can then be obtained from the CISCO-ENTITY-FRU-CONTROL-
MIB, and the LED color status from the CISCO-ENTITY-EXT-MIB:
# Power Supply information

CISCO-ENTITY-FRU-CONTROL-MIB::cefcPowerRedundancyMode.470 = INTEGER: 5
CISCO-ENTITY-FRU-CONTROL-MIB::cefcPowerRedundancyMode.471 = INTEGER: 5
CISCO-ENTITY-FRU-CONTROL-MIB::cefcTotalAvailableCurrent.470 = INTEGER: 6000
CISCO-ENTITY-FRU-CONTROL-MIB::cefcTotalAvailableCurrent.471 = INTEGER: 6000
CISCO-ENTITY-FRU-CONTROL-MIB::cefcTotalDrawnCurrent.470 = INTEGER: 3550
CISCO-ENTITY-FRU-CONTROL-MIB::cefcTotalDrawnCurrent.471 = INTEGER: 3550
# Fan module LED color
CISCO-ENTITY-EXT-MIB::ceExtEmtityLEDColor.470.1 = INTEGER: 2
Temperature
Temperature sensors can be polled using the CISCO-ENTITY-SENSOR-MIB, as in the

example below where the sensor entity table is polled for the Supervisor intake
snmpwalk –v 2c –ccommunity Nexus7000 .1.3.6.1.4.1.9.9.91.1.1.1.1 | grep 21848

CISCO-ENTITY-SENSOR-MIB::entSensorType.21848 = INTEGER: 8
CISCO-ENTITY-SENSOR-MIB::entSensorScale.21848 = INTEGER: 9
CISCO-ENTITY-SENSOR-MIB::entSensorPrecision.21848 = INTEGER: 0
CISCO-ENTITY-SENSOR-MIB::entSensorValue.21848 = INTEGER: 23
CISCO-ENTITY-SENSOR-MIB::entSensorStatus.21848 = INTEGER: 1
CISCO-ENTITY-SENSOR-MIB::entSensorValueTimestamp.21848 = Timeticks: (584032603) 67
days, 14:18:46.03
CISCO-ENTITY-SENSOR-MIB::entSensorValueUpdateRate.21848 = INTEGER: 60
SNMP Traps for Environment Monitoring
The Nexus 7000 switch supports the environmental SNMP trap notifications listed in the
table below.

MIB Notification Object Name OID Description
CISCO-ENTITY- entSensorThresholdNotification 1.3.6.1.4.1.9.9.91.2.0.1 This notification is
SENSOR-MIB generated once
each time the
sensor value
crosses the
threshold
SYSLOG Messages for Environmental Monitoring

Nexus 7000
Syslog messages are generated for various Environmental conditions. An example for a
Syslog message is as below
Syslog Message: PLATFORM-0-MOD_TEMPMAJALRM: [chars] reported Major

temperature alarm. Sensor=[dec] Temperature=[dec] MajThreshold=[dec]
Explanation: Module in slot [decimal digit] exceeded major temperature
threshold
CLI Command
Use the Show environment command to display the environment statistics.
4.6 Module Status
The Nexus 7000 switch implements the standard ENTITY-MIB and the CISCO-ENTITY-
FRU-CONTROL-MIB. These MIBs contain information about field replaceable units (FRU)
in the switch, such as supervisor cards, linecards, fans, power supplies, etc.
The module-related MIB objects that should be monitored are listed in the table below
Table 4 SNMP objects for Module monitoring

CISCO-ENTITY-FRU- cefcModuleOperStatus 1.3.6.1.4.1.9.9.117.1.2.1.1. This object shows the
CONTROL-MIB 2 module's operational
state.
Module status should
be ok (2).
CISCO-ENTITY-FRU- cefcModuleResetReason 1.3.6.1.4.1.9.9.117.1.2.1.1. This object identifies
CONTROL-MIB 3 the reason for the last
reset performed on the
module.
CISCO-ENTITY-FRU- cefcModuleUpTime 1.3.6.1.4.1.9.9.117.1.2.1.1. This object provides
CONTROL-MIB 8 the up time for the
module since it was
last re-initialized.
CISCO-ENTITY-FRU- cefcFRUPowerOperStatu 1.3.6.1.4.1.9.9.117.1.1.2.1. Operational FRU
CONTROL-MIB s 2 power state.
ENTITY-MIB entPhysicalDescr 1.3.6.1.2.1.47.1.1.1.1.2 A textual description of
physical entity. Used
for cross-reference of
physical index.

Nexus 7000
ENTITY-MIB entPhysicalName 1.3.6.1.2.1.47.1.1.1.1.7 The textual name of

the physical entity.
Used for cross-
reference of physical
index.
An example of Module status polling on Nexus 7000 is provided below
CISCO-ENTITY-FRU-CONTROL-MIB::cefcModuleOperStatus.24 = INTEGER: ok(2)

Cross-referencing with entPhysicalName and/or entPhysicalDescr using the same

entPhysicalIndex to determine the module name/type:
ENTITY-MIB::entPhysicalName.24 = STRING: “10 Gbps Ethernet Module”

ENTITY-MIB::entPhysicalName.25 = STRING: “10/100/1000 Mbps Ethernet Module”
ENTITY-MIB::entPhysicalName.26 = STRING: “Supervisor module-1X”
ENTITY-MIB::entPhysicalName.32 = STRING: “Fabric card module”
ENTITY-MIB::entPhysicalDescr.24 = STRING: "10 Gbps Ethernet Module"
ENTITY-MIB::entPhysicalDescr.25 = STRING: "10/100/1000 Mbps Ethernet Module"
ENTITY-MIB::entPhysicalDescr.26 = STRING: "Supervisor module-1X"
ENTITY-MIB::entPhysicalDescr.32 = STRING: "Fabric card module"
SNMP Traps for Module Monitoring
The Nexus 7000 switch supports the module-related SNMP trap notifications listed
Table 5 SNMP Notifications for Module Monitoring

CISCO-ENTITY- cefcModuleStatusChange 1.3.6.1.4.1.9.9.117.2.0. This notification is
FRU-CONTROL-MIB 1 generated when the value
of
cefcModuleOperStatus
changes
CISCO-ENTITY- cefcPowerStatusChange 1.3.6.1.4.1.9.9.117.2.0. The
FRU-CONTROL-MIB 2 cefcFRUPowerStatusChang
e notification indicates
that
the power status of a FRU
has changed.
CISCO-ENTITY- cefcFRUInserted 1.3.6.1.4.1.9.9.117.2.0. The cecfFRUInserted
FRU-CONTROL-MIB 3 notification indicates that
a FRU was inserted.

Nexus 7000
CISCO-ENTITY- cefcFRURemoved 1.3.6.1.4.1.9.9.117.2.0. The cefcFRURemoved

FRU-CONTROL-MIB 4 notification indicates that
a FRU was removed
CISCO-ENTITY- cefcFanTrayStatusChange 1.3.6.1.4.1.9.9.117.2.0. This notification is

FRU-CONTROL-MIB 6 generated when the value
of
cefcFanTrayOperStatus
changes
CISCO-ENTITY- cefcPowerSupplyOutputChang 1.3.6.1.4.1.9.9.117.2.0. The notification indicates
FRU-CONTROL-MIB e 7 that the power supply's
output capacity has
changed
SYSLOG Messages for Module Monitoring
Syslog event messages can be generated for various events related to modules in the
Nexus 7000 switch.
The following “facility” strings can identify module-related events:
 PLATFORM
 BOOTUP_TEST
 BOOTVAR
 CMPPROXY
 DIAGMGR
 MODULE
CLI Command
Use the show module command to determine the status of each module
Use the show module xbar command to determine the status of each fabric module
Use the “show environment fan ” command to verify temperature threshold’s per
module. Sylogs and Traps are generated if a threshold is met.
4.7 HSRP
For HSRP we want to monitor SNMP Traps and syslog event messages, and do SNMP
polling for Neighbor Status and number of neighbors.
The primary MIB for HSRP management is the CISCO-HSRP-MIB.For Nexus 7010 running
NX-OS the following MIB objects are supported and recommended
Table 6 SNMP OID’s for HSRP

Nexus 7000

CISCO-HSRP-MIB cHsrpGrpNumber 1.3.6.1.4.1.9.9.106.1.2.1.1.1 This object along with the
ifIndex of a particular
interface
uniquely identifies an HSRP
group
CISCO-HSRP-MIB cHsrpGrpVirtualIpAddr 1.3.6.1.4.1.9.9.106.1.2.1.1.1 This is the primary virtual IP
1 address used by this
group.
CISCO-HSRP-MIB cHsrpGrpActiveRouter 1.3.6.1.4.1.9.9.106.1.2.1.1.1 Ip Address of the currently

3 active router for this group.
CISCO-HSRP-MIB cHsrpGrpStandbyState 1.3.6.1.4.1.9.9.106.1.2.1.1.1 The current HSRP state of this

5 group on this interface.
Table 7 SNMP Notifications for HSRP
MIB Notification Object OID Description

Name
CISCO-HSRP-MIB cHsrpStateChange 1.3.6.1.4.1.9.9.106.2.0. A cHsrpStateChange
1 notification is sent when
a
cHsrpGrpStandbyState
transitions to either
active or
standby state, or leaves
active or standby state
SYSLOG Messages for HSRP Monitoring
The string HSRP_ENGINE can be used to identify any events related to system generated
HSRP messages.
CLI Commands
Use the show standby command to verify the HSRP status on the switches.
4.8 COPP

Nexus 7000
It is recommended to perform periodic monitoring of Control Plane Policing statistics in

the Nexus 7000 switch, for two reasons:
 To customize the CoPP policies on an on-going basis, adapting them to the

specific features and applications present in the DC network. The Nexus 7000
switch comes with default built-in CoPP policies, which have been created based
on industry/research averages. It is recommended to adapt the policies for the
specific environment where the switch is running.
 To help with the detection and identification of issues related to control-plane
protocols. Examples: ARP storm, Spanning-Tree instabilities, routing protocol
instabilities, etc can be identified via CoPP statistics. Similarly, CoPP could be the
cause for certain types of issues, i.e., a control packet that has been dropped by
the existing CoPP policies could have an impact on the network operation (failed
ARP, failed ICMP, etc).
Ideally, a baseline of CoPP statistics is performed regularly, in order to monitor and

adapt the policies to the evolution of the network. Additionally, “violate” counters
should be monitored regularly, as they can identify an increase of control-plane traffic
related to an abnormal situation.
Finally, counters in the “class-default” should also be monitored. If increasing, an in-

band traffic capture can be performed to analyze the control traffic that is classified in
the default class. If possible, CoPP classes can be modified to match also on control
traffic going to the default class
The MIB OID for CoPP information will be enabled, in the future release of NX-OS
software.
Use the CLI command show policy-map interface control-plane to display the control
plane statistics.
4.9 Hardware rate-limiters
Hardware Rate Limits can be optionally monitored in addition to CoPP statistics, in order
to have further insight on the type of packets that are reaching the Supervisor. For
instance, dropped counters in hardware rate limiters can provide explanations for
issues.
Hardware Rate Limiters can complement CoPP functionality, and the corresponding
counters may be added to the monitoring of the full CoPP policies.
However, monitoring of Hardware Rate Limiters is only possible via CLI currently.

Nexus 7000
The MIB OID for Hardware Rate Limiters information will be enabled in a future release
of NX-OS software.
Use the CLI command show hardware rate-limit to display the rate limit statistics.
4.10 Summary
The Summary of the recommended SNMP OIDs for Nexus 7000 has been listed in the
below table.
Table 8 Recommended SNMP OIDs for Nexus 7000
MIB Object OID Range Description Suggested Polling

Strategy
CISCO- ceExtNVRAM 1.3.6.1.4.1.9.9.1 Best practice is at least once per day
ENTITY-EXT- Size 95.1.1.1.2 to ensure
MIB ceExtNVRAMUs
ceExtNVRAM 1.3.6.1.4.1.9.9.1 ed does not
Used 95.1.1.1.3 exceeds more
than 80% of
ceExtNVRAMSiz
e
CISCO- cpmCPUTota 1.3.6.1.4.1.9.9.1 0..100 The overall CPU once every 5 minutes
PROCESS- l5minRev 09.1.1.1.1.8 busy percentage
MIB in the last 5
minute period.
Value should
remain below
80% normal
conditions.
CISCO- cseSysCPUUt 1.3.6.1.4.1.9.9.3 0..100 The average once every 5-15 minutes
SYSTEM- ilization 05.1.1.1 utilization of
EXT-MIB CPU on the
active
supervisor.
Thresholds for
RMON should
probes should
be set based on
baselining the
CPU utilization
within a
production
environment

Nexus 7000
CISCO- cseSysMemo 1.3.6.1.4.1.9.9.3 0..100 The average once every 5-15 minutes
SYSTEM- ryUtilization 05.1.1.2 utilization of
EXT-MIB memory on the
active
supervisor.
Thresholds for
RMON should
probes should
be set based on
baselining the
memory
utilization
within a
production
environment
CISCO- callHomeAle 1.3.6.1.4.1.9.9.3 The number once every hour
CALLHOME- rts 00.1.2.2 of Call Home
MIB alerts sent.
If this has
incremented
then investigate
the root cause
of what caused
the increment.
CISCO- ceExtSysBoo 1.3.6.1.4.1.9.9.1 These provide a once per day
ENTITY-EXT- tImageList 95.3.2.5 record of the
MIB kickstart/syste
ceExtKicksta 1.3.6.1.4.1.9.9.1 m image that
rtImageList 95.1.2.1.4 the system is
running.
CISCO- cefcPowerRe 1.3.6.1.4.1.9.9.1 Should be in at least once every hour
ENTITY- dundancyMo 17.1.1.1.1.1 redundant(2)
FRU- de
CONTROL-
MIB
CISCO- cefcTotalAva 1.3.6.1.4.1.9.9.1 cefcTotalDrawn at least once every hour
ENTITY- ilableCurrent 17.1.1.1.1.3 Current should
FRU- remain within
CONTROL- cefcTotalDra 1.3.6.1.4.1.9.9.1 cefcTotalAvaila
MIB wnCurrent 17.1.1.1.1.4 bleCurrent

Nexus 7000
CISCO- cefcModuleU 1.3.6.1.4.1.9.9.1 Module uptime at least once every hour

ENTITY- pTime 17.1.2.1.1.8 should be
FRU- incrementing. If
CONTROL- cefcModuleSt 1.3.6.1.4.1.9.9.1 it resets to a
MIB atusLastCha 17.1.2.1.1.4 lower number,
ngeTime then
1.3.6.1.4.1.9.9.1 cefcModuleStat
cefcModuleR 17.1.2.1.1.6 usLastChangeTi
esetReasonD me &
escription cefcModuleRese
tReasonDescrip
tion should be
polled to see the
reason why the
module was
reset.
CISCO- cefcFanTray 1.3.6.1.4.1.9.9.1 All fan trays at least once every hour
ENTITY- OperStatus 17.1.4.1.1.1 should be in the
FRU- up(2) state
CONTROL-
MIB
CISCO- entSensorVal 1.3.6.1.4.1.9.9.9 Provides a table at least once every hour
ENTITY- ueTable 1.1.1.1 of temperature
SENSOR-MIB and power
sensors within
the system.
Best practice
would be to
gather trend
data here to
ensure that
facilities/coolin
g/power are
being
maintained.
CISCO- ciscoFlashPa 1.3.6.1.4.1.9.9.1 Provides at least once every hour
FLASH-MIB rtitionSize 0.1.1.4.1.1.13 information on
flash
ciscoFlashPa 1.3.6.1.4.1.9.9.1 filesystems
rtitionFreeSp 0.1.1.4.1.1.14 within the
ace system.
Best practice is
to ensure
ciscoFlashPartit
ionFreeSpace
does not drop
below 10% of
ciscoFlashPartit
ionSize

Nexus 7000
CISCO- cudldpInterf 1.3.6.1.4.1.9.9.1 shutdow Should be once every 5-15 minutes

UDLDP-MIB aceOperStat 18.1.2.1.1.2 n(1), polled for any
us interfaces in
indeterm shutdown(1)
inant(2) state indicating
that a Uni-
biDirecti Direction link
onal(3) has been
detected and
notAppli the port has
cable(4) been disabled
IF-MIB ifDescr 1.3.6.1.2.1.2.2.1. A textual string Poll IF-MIB values every 5
2 containing the minutes for historical
interface data.
IF-MIB ifAlias 1.3.6.1.2.1.31.1. This object is an
1.1.18 'alias' name for
the interface as Suggest making use of
specified by a RMON probes for specific
network interfaces/counters to
manager, and trigger alerts based on
provides a non- error counters
volatile 'handle' incrementing beyond a
for the certain threshold
interface. (determine via baseline),
IF-MIB ifSpeed 1.3.6.1.2.1.2.2.1. as well as interfaces with
5 sustained performance
IF-MIB ifAdminStatu 1.3.6.1.2.1.2.2.1. up(1) The desired above a high water mark.
s 7 state of the
down(2) interface.
testing(3
)
IF-MIB ifOperStatus 1.3.6.1.2.1.2.2.1. up(1) The current

8 operational
down(2) state of the
interface
testing(3
)
unknow
n(4)
dormant
(5)
notPrese
nt(6)
lowerLay
erDown(
7)
IF-MIB ifLastChange 1.3.6.1.2.1.2.2.1. The value of
9 sysUpTime at
the time the

Nexus 7000
interface
entered its
current
operational
state
IF-MIB ifHCInOctets 1.3.6.1.2.1.31.1. The total
1.1.6 number of
octets received
on the interface,
including
framing
characters.
IF-MIB ifHCOutOctet 1.3.6.1.2.1.31.1. The total
s 1.1.10 number of
octets
transmitted out
of the interface,
including
framing
characters
IF-MIB ifHCInUcastP 1.3.6.1.2.1.31.1. The number of
kts 1.1.7 packets,
delivered by
this sub-layer to
a higher (sub-
)layer, which
were not
addressed to a
multicast or
broadcast
address at this
sub-layer.
IF-MIB ifHCOutUcast 1.3.6.1.2.1.31.1. The total
Pkts 1.1.11 number of
packets that
higher-level
protocols
requested be
transmitted,
and which were
not addressed
to a multicast or
broadcast
address at this
sub-layer,
including those
that were
discarded or not
sent.

Nexus 7000
IF-MIB ifHCInMultic 1.3.6.1.2.1.31.1. The number of

astPkts 1.1.8 packets,
delivered by
this sub-layer to
a higher (sub-
)layer, which
were addressed
to a multicast
address at this
sub-layer.
IF-MIB ifHCOutMulti 1.3.6.1.2.1.31.1. The total
castPkts 1.1.12 number of
packets that
higher-level
protocols
requested be
transmitted,
and which were
addressed to a
multicast
address at this
sub-layer,
including those
that were
discarded or not
sent.
IF-MIB ifHCInBroad 1.3.6.1.2.1.31.1. The number of
castPkts 1.1.9 packets,
delivered by
this sub-layer to
a higher (sub-
)layer, which
were addressed
to a broadcast
address at this
sub-layer.
IF-MIB ifHCOutBroa 1.3.6.1.2.1.31.1. The total
dcastPkts 1.1.13 number of
packets that
higher-level
protocols
requested be
transmitted,
and which were
addressed to a
broadcast
address at this
sub-layer,
including those
that were
discarded or not
sent.
IF-MIB ifInDiscards 1.3.6.1.2.1.2.2.1. The number of
13 inbound
packets which

Nexus 7000
were chosen to
be discarded
even though no
errors had been
detected to
prevent their
being
deliverable to a
higher-layer
protocol. One
possible reason
for discarding
such a packet
could be to free
up buffer space
IF-MIB ifInErrors 1.3.6.1.2.1.2.2.1. The number of
14 inbound
packets that
contained
errors
preventing
them from
being
deliverable to a
higher-layer
protocol..
IF-MIB ifInUnknown 1.3.6.1.2.1.2.2.1. The number of
Protos 15 packets
received via the
interfaces
which were
discarded
because of an
unknown or
unsupported
protocol.
IF-MIB ifOutDiscard 1.3.6.1.2.1.2.2.1. The number of
s 19 outbound
packets which
were chosen to
be discarded
even though no
errors had been
detected to
prevent their
being
transmitted. On
e possible
reason for
discarding such
a packet could
be to free up
buffer space.

Nexus 7000
IF-MIB ifOutError 1.3.6.1.2.1.2.2.1. The number of

20 outbound
packets that
could not be
transmitted
because of
errors.
IF-MIB ifConnectorP 1.3.6.1.2.1.31.1. This object has
resent 1.1.17 the value
'true(1)' if the
interface
sublayer has a
physical
connector and
the value
'false(2)'
otherwise.
The Summary of the recommended SNMP Traps for Nexus 7000 has been listed in
the below table.
Table 9 Recommended SNMP Traps for Nexus 7000
MIB Trap Object Description

Chassis /
Environmentals / System

Nexus 7000
ENTITY-MIB entConfigChange An entConfigChange notification is generated

when the value of entLastChangeTime
changes. It can be utilized by an NMS to trigger
logical/physical entity table maintenance
polls. An agent should not generate more than
one entConfigChange 'notification-event' in a
given time interval (five seconds is the suggested
default). A 'notification-event' is the
transmission of a single trap or inform PDU to a
list of notification destinations. If additional
configuration changes occur within the throttling
period, then notification-events for these
changes should be suppressed by the agent until
the current throttling period expires. At the end
of a throttling period, one notification-event
should be generated if any configuration changes
occurred since the start of the throttling
period. In such a case, another throttling period
is started right away. An NMS should periodically
check the value of entLastChangeTime to detect
any missed entConfigChange notification-events,
e.g., due to throttling or transmission loss.
CISCO-ENTITY-FRU- cefcModuleStatusChange This notification is generated when the value of
CONTROL-MIB cefcModuleOperStatus changes. It can be utilized
by an NMS to update the status of the module it
is managing.
CISCO-ENTITY-FRU- cefcPowerStatusChange The cefcFRUPowerStatusChange notification
CONTROL-MIB indicates that the power status of a FRU has
changed. The varbind for this notification
indicates the entPhysicalIndex of the FRU, and
the new operational-status of the FRU.
CISCO-ENTITY-FRU- cefcFRUInserted The cecfFRUInserted notification indicates that a
CONTROL-MIB FRU was inserted. The varbind for this
notification indicates the entPhysicalIndex of the
inserted FRU, and the entPhysicalIndex of the
FRU's container.
CISCO-ENTITY-FRU- cefcFRURemoved The cefcFRURemoved notification indicates that
CONTROL-MIB a FRU was removed. The varbind for this
notification indicates the entPhysicalIndex of the
removed FRU, and the entPhysicalIndex of the
FRU's container.
CISCO-ENTITY-FRU- cefcFanTrayStatusChange This notification is generated when the value of
CONTROL-MIB cefcFanTrayOperStatus changes.
CISCO-ENTITY-FRU- cefcPowrSupplyOutputCh The notification indicates that the power
CONTROL-MIB ange supply's output capacity has changed.
CISCO-ENTITY-SENSOR- entSensorThresholdNotifi The sensor value crossed the threshold listed in
MIB cation entSensorThresholdTable. This notification is
generated once each time the sensor value
crosses the threshold.
SNMPv2-MIB coldStart A coldStart trap signifies that the SNMP entity
supporting a notification originator application,
is reinitializing itself and that its configuration
may have been altered.

Nexus 7000
SNMPv2-MIB warmStart A warmStart trap signifies that the SNMP entity

supporting a notification originator application
is reinitializing itself such that its configuration
is unaltered
Physical Port related
CISCO-PORT-SECURITY- cpsIfVlanSecureMacAddrV The address violation notification is generated
MIB iolation when port security address violation is detected
on a multi-vlan interface and the
cpsIfViolationAction is set to 'dropNotify'
CISCO-PORT-SECURITY- cpsSecureMacAddrViolati The address violation notification is generated
MIB on when port security address violation is detected
on a secure non-trunk, access interface (that
carries a single vlan) and the
cpsIfViolationAction is set to 'dropNotify'.
IF-MIB linkDown A linkDown trap signifies that the SNMP entity,
acting in an agent role, has detected that the
ifOperStatus object for one of its communication
links is about to enter the down state from some
other state (but not from the notPresent
state). This other state is indicated by the
included value of ifOperStatus
IF-MIB linkUp A linkUp trap signifies that the SNMP entity,
acting in an agent role, has detected that the
ifOperStatus object for one of its communication
links left the down state and transitioned into
some other state (but not into the notPresent
state). This other state is indicated by the
included value of ifOperStatus.
Layer 2 SNMP Traps
BRIDGE-MIB newRoot The newRoot trap indicates that the sending
agent has become the new root of the Spanning
Tree; the trap is sent by a bridge soon after its
election as the new root, e.g., upon expiration of
the Topology Change Timer, immediately
subsequent to its election.
BRIDGE-MIB topologyChange A topologyChange trap is sent by a bridge when
any of its configured ports transitions from the
Learning state to the Forwarding state, or from
the Forwarding state to the Blocking state. The
trap is not sent if a newRoot trap is sent for the
same transition.
CISCO-STP-EXTENSION- stpxInconsistencyUpdate
MIB
CISCO-STP-EXTENSION- stpxLoopInconsistencyUp
MIB date
CISCO-STP-EXTENSION- stpxRootInconsistencyUp
MIB date
Configuration related
CISCO-CONFIG-MAN-MIB ccmCLIRunningConfigCha This notification indicates that the running
nged configuration of the managed system has
changed.
CISCO-LICENSE-MGR-MIB clmLicenseExpiryNotify clmLicenseCountExpiryNotify is sent, when a
feature license expires.

Nexus 7000
CISCO-LICENSE-MGR-MIB clmNoLicenseForFeature clmNoLicenseForFeatureNotify is sent, when an

Notify license is issued for a feature which does not
have license. The license will expire after a grace
period, which is mentioned in the
clmLicenseGracePeriod' object of this
notification. This notification is also sent once
every 24 hours from the start of the grace
period. In the last 7 days before grace period
expiry this notification is sent once every hour.
CISCO-LICENSE-MGR-MIB clmLicenseFileMissingNot clmLicenseFileMissingNotify is generated when
ify thesystem detects that one or more installed
license files are missing. To correct this fault,
any files which might be missing should be
reinstalled, e.g., if it's unknown which ones are
missing, then all files can be re-installed,
ignoring any 'duplicate license' messages
CISCO-LICENSE-MGR-MIB clmLicenseExpiryWarning clmLicenseExpiryWarningNotify is sent, when an
Notify license issued for a feature is going to expire,
and the feature is use. This notification is sent
once in a day starting 30 days before license
expiry. In the last 7 days before expiry this
notification is sent once every hour. The license
will expire on the date and time, mentioned in
the 'clmLicenseExpiryDate' object of this
notification
4.11 Other useful CLI commands
The following CLI commands do not have any SNMP OIDs to monitor and these
commands will be useful to monitor the network performance and the future growth.
MAC Address TCAM Utilization
The Cisco Nexus 7000 Series uses a distributed forwarding architecture in which each
module has a forwarding engine responsible for forwarding packets. A forwarding
engine on an M series module is capable of storing 128,000 MAC Address entries. MAC
address tables are synchronized between Ethernet M series modules that have ports
configured in the same Virtual Device Context (VDC). The following command is useful
for verifying the MAC address table capacity for all modules in a chassis .
n7000# show hardware capacity forwarding | begin L2

L2 Forwarding Resources
-----------------------
L2 entries: Module total used mcast ucast lines
lines_full
1 131072 6 1 5 8192
0
2 131072 6 1 5 8192
0
<Text Omitted>

Nexus 7000
Unicast or Multicast TCAM Tables
Ethernet M series module has a forwarding engine responsible for forwarding packets. A
forwarding engine on an M series module is capable of storing 128,000 IPv4/IPv6
routing entries or 1,000,000 entries if it is an XL module with a Scalable-Feature license
installed. IPv4/IPv6 unicast/multicast tables are synchronized between Ethernet M
series modules that have ports configured in the same Virtual Device Context (VDC). The
following example displays the default TCAM allocation for a non-XL module. Cisco NX-
OS supports dynamic TCAM allocation. This allows for better resource utilization in the
event and address family (i.e. IPv6 unicast) requires additional entries.
n7000# show hardware capacity forwarding | begin TCAM

Key: Log/Phys = Logical entries / Physical entries
Note: IPv4 Multicast/IPv6 Unicast entries share one FIB TCAM entry
pool
Module 1 usage:
Route Type Used %Used Free %Free Total
(Log/Phys) (Log/Phys)
(Log/Phys)
---------------------------------------------------------------------
----
IPv4 Unicast: 19/19 0 57325/57325 99
57344/57344
IPv4 Multicast: 4/8 0 16380/32760 99
16384/32768
IPv6 Unicast: 9/18 0 16375/32750 99
16384/32768 IPv6 Multicast: 5/20 0 2043/8172
99 2048/8192
ACL or QoS TCAM Tables
module has a forwarding engine responsible for forwarding packets. A forwarding
engine on an M series module is capable of storing 64,000 (non-XL) or 128,000 ACL QoS
entries if it is an XL module with the Scalable Feature license installed .
n7000# show hardware capacity | begin ACL

ACL Hardware Resource Utilization (Module 1)
--------------------------------------------
Used Free Percent
Utilization
-----------------------------------------------------
Tcam 0, Bank 0 1 16383 0.00
Tcam 0, Bank 1 2 16382 0.01
Tcam 1, Bank 0 1 16383 0.00
Tcam 1, Bank 1 2 16382 0.01
LOU 0 104 0.00
Both LOU Operands 0

Nexus 7000
Single LOU Operands 0

LOU L4 src port: 0
LOU L4 dst port: 0
LOU L3 packet len: 0
LOU IP tos: 0
LOU IP dscp: 0
LOU ip precedence: 0
TCP Flags 0 16 0.00
Protocol CAM 0 7 0.00
Mac Etype/Proto CAM 0 14 0.00
Non L4op labels, Tcam 0 0 6143 0.00
Non L4op labels, Tcam 1 0 6143 0.00
L4 op labels, Tcam 0 0 2047 0.00
L4 op labels, Tcam 1 0 2047 0.00
Fabric Utilization
The fabric utilization can be monitored to verify the ingress and egress bandwidth
utilization. The show hardware fabric-utilization commands are useful for verifying the
high-level and detailed utilization. The show hardware capacity fabric-utilization is
useful for verifying the peak utilization history.
n7000# show hardware fabric-utilization

------------------------------------------------
Slot Total Fabric Utilization
Bandwidth Ingress % Egress %
------------------------------------------------
1 138 Gbps 0.0 0.0
2 138 Gbps 0.0 0.0
4 138 Gbps 0.0 0.0
5 69 Gbps 0.0 0.0
7 138 Gbps 0.0 0.0
8 138 Gbps 0.0 0.0
9 138 Gbps 0.0 0.0
10 138 Gbps 0.0 0.0
n7000# show hardware capacity fabric-utilization

----------------------------------------------------------------------
--
Fabric Planes:
A -- Unicast fabric interface
B -- Multicast/Multidestination fabric interface
-------------------------PEAK FABRIC UTILIZATION----------------------
--
I/O |-----FABRIC----| Ingress | Egress
Slot |Mod Inst Plane| Util Time | Util Time
----------------------------------------------------------------------
--
1 1 1 A 0% 11-01@23:09:42 0% 11-
01@23:09:42
1 1 1 B 0% 11-01@23:09:42 0% 11-
01@23:09:42
1 1 1 A 0% 11-01@23:09:42 0% 11-
01@23:09:42

Nexus 7000
1 1 1 B 0% 11-01@23:09:42 0% 11-
01@23:09:42
1 2 1 A 0% 11-01@23:09:42 0% 11-
01@23:09:42
1 2 1 B 0% 11-01@23:09:42 0% 11-
01@23:09:42
1 2 1 A 0% 11-01@23:09:42 0% 11-
01@23:09:42
1 2 1 B 0% 11-01@23:09:42 0% 11-
01@23:09:42
1 3 1 A 0% 11-01@23:09:42 0% 11-
01@23:09:42
VDC Resource Utilization
Global VDC resources can be verified with the show vdc resource command. This is
useful to know, since VDCs can contend for common resources such as memory, SPAN
sessions, etc.).
n7000# show vdc resource

vlan 16 used 48 unused 16368 free 16320 avail 16384
total
monitor-session 0 used 0 unused 2 free 2 avail 2
total
monitor-session-erspan-dst 0 used 0 unused 23 free 23 avail
23 total
vrf 8 used 0 unused 992 free 992 avail 1000
total
port-channel 0 used 0 unused 768 free 768 avail 768
total
u4route-mem 120 used 0 unused 396 free 396 avail 516
total
u6route-mem 36 used 0 unused 172 free 172 avail 208
total
m4route-mem 82 used 0 unused 118 free 118 avail 200
total
4.12 References
SNMP MIBs and Notification details

ftp://ftp.cisco.com/pub/mibs/supportlists/nexus7000/Nexus7000MIBSupportList.
html
System Message details for Nexus 7000

http://www.cisco.com/en/US/docs/switches/datacenter/sw/system_messages/ref
erence/sl_7K_MDS.html
SNMP configuration guide for Nexus 7000

http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-
os/system_management/configuration/guide/sm_9snmp.html

Nexus 7000

5. Nexus 5000
5.1 Target Equipment
The following device is targeted:

 Hardware: Nexus 5000 series switches
 Software release 5.1.3N1
5.2 CPU Utilization
The following MIBs used to monitor the Overall information about the CPU load.
MIB : CISCOPROCESSMIB
Object: cpmCPUTotalEntry
OID : 1.3.6.1.4.1.9.9.109.1.1.1.1

3: cpmCPUTotal5sec
4: cpmCPUTotal1min
5: cpmCPUTotal5min
Description:
‘cpmCPUTotal5sec’: Average CPU utilization over the last 5 seconds
‘cpmCPUTotal1min’: Average CPU utilization over the last 60 seconds.
‘cpmCPUTotal5min’: Average CPU utilization over the last 5 minutes
CLI Command
The show system resources command displays the high level CPU utilization for the
supervisor module. The show process cpu command with the sort option lists all of the
processes sorted by the highest CPU utilization per process. The show process cpu
history command displays the CPU history in three increments: 60 seconds, 60 minutes,
72 hours. Viewing the CPU history is valuable when correlating a network event with the
past CPU utilization.
N5K# sh system resources


Nexus 5000
5.3 Memory Availability

The state of the memory status can be monitored with the following MIB.
MIB: CISCOSYSTEMEXTMIB
Object: ciscoSysInfoGroup
OID: 1.3.6.1.4.1.9.9.305.1.1

1: cseSysCPUUtilization
2: cseSysMemoryUtilization
Description:
‘cseSysCPUUtilization’: The average utilization of CPU on the active
supervisor. Thresholds for RMON should probes should be set based on
baselining the CPU utilization within a production environment.
‘cseSysMemoryUtilization’: The average utilization of memory on the active

supervisor. Thresholds for RMON should probes should be set based on
baselining the memory utilization within a production environment
MIB: CISCOENTITYEXTMIB
Object: ceExtPhysicalProcessorEntry
OID: 1.3.6.1.4.1.9.9.195.1.1.1

2: ceExtNVRAMSize
3: ceExtNVRAMUsed
Description:
‘ceExtNVRAMSize’: Total number of bytes of NVRAM in the entity.A value of 0
for this object means the entity does not support NVRAM or NVRAM
information is not available.
‘ceExtNVRAMUsed’: Number of bytes of NVRAM in use. This object is

irrelevant if ceExtNVRAMSize is 0
CLI Command
To assess the overall level of memory utilization, use the basic CLI commands
show system resources and show processes memory.
The show system resources command displays platform memory statistics

Nexus 5000
N5K# show system resources

The show process memory command displays the memory allocation per process and it
is only useful for verifying process-level memory allocation.
5.4 Network Interfaces

The following MIBs contain management information applicable to a particular
interface:
MIB : IFMIB
Object: ifEntry
OID : 1.3.6.1.2.1.2.2.1

5: ifSpeed
7: fAdminStatus
8: ifOperStatus
9: ifLastChange
13: ifInDiscards
14: ifInErrors
15: ifInUnknownProtos
19: ifOutDiscards
20: ifOutErrors
Descriptions:
‘ifOperStatus’: Returns the operational status of the interfaces.
‘ifInDiscards‘: Indicates ingress packets that have been discarded on interfaces
due to no buffer availability. Typically indicates that the device is overloaded.
‘ifInErrors’: Indicates input errors on interfaces.
‘ifOutDiscards’: Indicates egress packets that have been discarded on interfaces
due to no buffer availability. Possible cause is low memory.
‘ifOutErrors’: Indicates output error on interfaces.
The following MIBs contain additional management information applicable to a

particular interface.
MIB: IFMIB
Object: ifXEntry
OID: 1.3.6.1.2.1.31.1.1.1

Nexus 5000

7: ifHCInUcastPkts
8: ifHCInMulticastPkts
9: ifHCInBroadcastPkts
10:ifHCOutOctets
11:ifHCOutUcastPkts
12:ifHCOutMulticastPkts
13:ifHCOutBroadcastPkts
17:ifConnectorPresent
Description:
Each possible return value is used to monitor the ingress/egress unicast/
multicast/ broadcast packets
SNMP Traps for Interface Monitoring
Traps can be generated for the status of the Interfaces using the IF-MIB. The linkup and
linkdown is used if the operational status of the Interfaces changes.
 snmp-server enable traps link linkDown

 snmp-server enable traps link Up
CLI Command
Use the below commands to verify information for any given interface
show interface brief displays the interface configuration information, including the
mode.
show interface capabilities displays information on the capabilities of the interfaces.
show interface counters [module module] displays input and output octets unicast
packets, multicast packets, and broadcast packets
show interface counters errors [module module] Displays information on the number
of error packets.
5.4.1 vPC and Peer link

The vPC link and the peer-links are critical to maintaining the vPC infrastructure
deployed. Any change on these two links must be acted upon immediately.
The following “facility” strings can identify events related to the vPC and peer-link:
 VPC
 ETH_PORT_CHANNEL
 ETHPORT

Nexus 5000
 ETH_PM
The below CLI commands can also be used to verify any issues with vPC.
show vpc command to check the status of vPC.

show vpc peer-keepalive command to check the status of the vPC peer-keepalive link.
5.5 Environmental Parameters

The Nexus 5000 switch implements the standard ENTITY-MIB, the CISCO-ENTITY-
SENSOR-MIB, the CISCO-ENTITY-EXT-MIB, and the CISCO-ENTITY-FRU-CONTROL-MIB.
These MIB’s contain information about environmental status in the switch, such as
temperature sensors, fan tray status, power supply status, etc. The recommended MIBs
are listed below
MIB: CISCOENTITYFRUCONTROLMIB
Object: cefcMIBObjects
OID: 1.3.6.1.4.1.9.9.117.1

1:cefcFRUPower
2:cefcModule
4:cefcFRUFan
Description:
‘cefcFRUPower’: will have possible return value ‘cefcPowerRedundancyMode’,
‘cefcTotalAvailableCurrent’ & ‘cefcTotalDrawnCurrent’ for power related
information.
‘cefcModule’: will have possible return vlaue ‘cefcModuleUpTime’,

‘cefcModuleStatusLastChangeTime’ & ‘cefcModuleResetReasonDescription’ will
provide module related information.
‘cefcFRUFan’: will have possible return ‘vlauecefcFanTrayOperStatus’ will provide

fan trap information

Nexus 5000

CISCO-ENTITY-SENSOR-MIB entSensorType 1.3.6.1.4.1.9.9.91.1.1.1.1.1 This variable indicates the
type of data reported by
the entSensorValue.eg :
3:voltsAC
4:voltsDC
5:amperes
6:watts
CISCO-ENTITY-SENSOR-MIB entSensorScale 1.3.6.1.4.1.9.9.91.1.1.1.1.2 This variable indicates the

exponent to apply
to sensor values reported
by entSensorValue.eg :
6:nano
7:micro
8:milli
9:units
10:kilo
11:mega
CISCO-ENTITY-SENSOR-MIB entSensorValue 1.3.6.1.4.1.9.9.91.1.1.1.1.4 This variable reports the

most recent measurement
seen
by the sensor
CISCO-ENTITY-SENSOR-MIB entSensorStatus 1.3.6.1.4.1.9.9.91.1.1.1.1.5 This variable indicates the
present operational status
of the sensor.
1:ok
2:unavailable
3:nonoperational
CISCO-ENTITY-FRU- cefcFanTrayOperStatus 1.3.6.1.4.1.9.9.117.1.4.1.1.1 The operational state of

CONTROL-MIB the fan or fan tray.
unknown(1) - unknown.
up(2) - powered on.
down(3) - powered down.
warning(4) - partial
failure, needs
replacement
as soon as possible

Nexus 5000
CISCO-ENTITY-EXT-MIB ceExtEntityLEDColor 1.3.6.1.4.1.9.9.195.1.3.1.2 The color of the LED on

the module
1 : off
2 : green
3 : amber
4 : red
CLI Command
Use the Show environment command to display the environment statistics. An example
of displaying the details of Power usage is shown below
N5K# sh environment power
Power Supply:
Voltage: 12 Volts
-----------------------------------------------------------
PS Model Input Power Power Status
Type (Watts) (Amp)
-----------------------------------------------------------
1 N55-PAC-750W-B AC 744.00 62.00 ok
2 -- -- -- --
fail/shutdown
Mod Model Power Power Power Power

Status
Requested Requested Allocated
Allocated
(Watts) (Amp) (Watts) (Amp)
--- ---------------------- ------- ---------- --------- -----
----- --------
--
1 N5K-C5548UP-SUP 492.00 41.00 492.00 41.00
powered-
up
3 N55-D160L3 144.00 12.00 144.00 12.00
powered-
up
Power Usage Summary:

--------------------
Power Supply redundancy mode: Redundant
Power Supply redundancy operational mode: Non-redundant
Total Power Capacity 744.00 W
Power reserved for Supervisor(s) 492.00 W

Power currently used by Modules 144.00 W
-------------
Total Power Available 108.00 W

Nexus 5000
5.6 Module Status
The Nexus 7000 switch implements the standard ENTITY-MIB and the CISCO-ENTITY-
FRU-CONTROL-MIB. These MIBs contain information about field replaceable units (FRU)
in the switch, such as supervisor cards, linecards, fans, power supplies, etc.
The module-related MIB objects that should be monitored are listed in the table below
Table 11 SNMP Objects for Module Monitoring

CISCO-ENTITY-FRU- cefcModuleOperStatus 1.3.6.1.4.1.9.9.117.1.2.1.1.2 This object shows the
CONTROL-MIB module's operational
state.
Module status should be
ok (2).
CISCO-ENTITY-FRU- cefcModuleResetReason 1.3.6.1.4.1.9.9.117.1.2.1.1.3 This object identifies the
CONTROL-MIB reason for the last reset
performed on the
module.
CISCO-ENTITY-FRU- CefcModuleUpTime 1.3.6.1.4.1.9.9.117.1.2.1.1.8 This object provides the
CONTROL-MIB up time for the module
since it was last re-
initialized.
CISCO-ENTITY-FRU- cefcFRUPowerOperStatus 1.3.6.1.4.1.9.9.117.1.1.2.1.2 Operational FRU power
CONTROL-MIB state.
ENTITY-MIB EntPhysicalDescr 1.3.6.1.2.1.47.1.1.1.1.2 A textual description of
physical entity. Used for
cross-reference of
physical index.
ENTITY-MIB EntPhysicalName 1.3.6.1.2.1.47.1.1.1.1.7 The textual name of the
physical entity. Used for
cross-reference of
physical index.
SNMP Traps for Module Monitoring
The Nexus 5000 switch supports the module-related SNMP trap notifications listed
Table 12 SNMP Notifications for Module Monitoring

CISCO-ENTITY-FRU- CefcModuleStatusChange 1.3.6.1.4.1.9.9.117.2.0.1 This notification is
CONTROL-MIB generated when the value
of
cefcModuleOperStatus
changes

Nexus 5000
CISCO-ENTITY-FRU- cefcPowerStatusChange 1.3.6.1.4.1.9.9.117.2.0.2 The

CONTROL-MIB cefcFRUPowerStatusChange
notification indicates that
the power status of a FRU
has changed.
CISCO-ENTITY-FRU- cefcFRUInserted 1.3.6.1.4.1.9.9.117.2.0.3 The cecfFRUInserted
CONTROL-MIB notification indicates that a
FRU was inserted.
CISCO-ENTITY-FRU- cefcFRURemoved 1.3.6.1.4.1.9.9.117.2.0.4 The cefcFRURemoved

CONTROL-MIB notification indicates that a
FRU was removed
CISCO-ENTITY-FRU- cefcFanTrayStatusChange 1.3.6.1.4.1.9.9.117.2.0.6 This notification is

CONTROL-MIB generated when the value
of
cefcFanTrayOperStatus
changes
CISCO-ENTITY-FRU- cefcPowerSupplyOutputChange 1.3.6.1.4.1.9.9.117.2.0.7 The notification indicates
CONTROL-MIB that the power supply's
output capacity has
changed
SYSLOG Messages for Module Monitoring
Syslog event messages can be generated for various events related to modules in the
Nexus 5000 switch.
The following “facility” strings can identify module-related events:
 PLATFORM
 BOOTUP_TEST
 BOOTVAR
 CMPPROXY
 DIAGMGR
 MODULE
CLI Command
Use the show module command to determine the status of each module
SJMGMTSW01# sh module
Mod Ports Module-Type Model
Status
--- ----- -------------------------------- ---------------------- -
-----------
1 32 O2 32X10GE/Modular Universal Pla N5K-C5548UP-SUP
active *
3 0 O2 Daughter Card with L3 ASIC N55-D160L3
ok

Nexus 5000
5.7 Summary
The Summary of the recommended SNMP OIDs for Nexus 5000 has been listed in the
below table.
Table 13 Recommended SNMP OIDs for Nexus 5000
MIB Object OID Range Description Suggested Polling Strategy
CISCO-ENTITY- ceExtNVRAMSi 1.3.6.1.4.1.9.9.19 Best practice is to at least once per day

EXT-MIB ze 5.1.1.1.2 ensure
ceExtNVRAMUsed
ceExtNVRAMU 1.3.6.1.4.1.9.9.19 does not exceeds
sed 5.1.1.1.3 more than 80% of
ceExtNVRAMSize
CISCO- cpmCPUTotal5 1.3.6.1.4.1.9.9.10 0..100 The overall CPU once every 5 minutes
PROCESS-MIB minRev 9.1.1.1.1.8 busy percentage
in the last 5
minute period.
Value should
remain below
80% normal
conditions.
CISCO- cseSysCPUUtili 1.3.6.1.4.1.9.9.30 0..100 The average once every 5-15 minutes
SYSTEM-EXT- zation 5.1.1.1 utilization of CPU
MIB on the active
supervisor.
Thresholds for
RMON should
probes should be
set based on
baselining the
CPU utilization
within a
production
environment
CISCO- cseSysMemory 1.3.6.1.4.1.9.9.30 0..100 The average once every 5-15 minutes
SYSTEM-EXT- Utilization 5.1.1.2 utilization of
MIB memory on the
active supervisor.
Thresholds for
RMON should
probes should be
set based on
baselining the
memory
utilization within
a production
environment

Nexus 5000
CISCO-ENTITY- ceExtSysBootI 1.3.6.1.4.1.9.9.19 These provide a once per day

EXT-MIB mageList 5.3.2.5 record of the
kickstart/system
ceExtKickstartI 1.3.6.1.4.1.9.9.19 image that the
mageList 5.1.2.1.4 system is running.
CISCO-ENTITY- cefcPowerRed 1.3.6.1.4.1.9.9.11 Should be in at least once every hour

FRU- undancyMode 7.1.1.1.1.1 redundant(2)
CONTROL-MIB
CISCO-ENTITY- cefcTotalAvail 1.3.6.1.4.1.9.9.11 cefcTotalDrawnCu at least once every hour
FRU- ableCurrent 7.1.1.1.1.3 rrent should
CONTROL-MIB remain within
cefcTotalDraw 1.3.6.1.4.1.9.9.11 cefcTotalAvailable
nCurrent 7.1.1.1.1.4 Current
CISCO-ENTITY- cefcModuleUp 1.3.6.1.4.1.9.9.11 Module uptime at least once every hour
FRU- Time 7.1.2.1.1.8 should be
CONTROL-MIB incrementing. If it
cefcModuleSta 1.3.6.1.4.1.9.9.11 resets to a lower
tusLastChange 7.1.2.1.1.4 number, then
Time cefcModuleStatus
1.3.6.1.4.1.9.9.11 LastChangeTime
cefcModuleRe 7.1.2.1.1.6 &
setReasonDesc cefcModuleReset
ription ReasonDescriptio
n should be
polled to see the
reason why the
module was reset.
CISCO-ENTITY- cefcFanTrayOp 1.3.6.1.4.1.9.9.11 All fan trays at least once every hour
FRU- erStatus 7.1.4.1.1.1 should be in the
CONTROL-MIB up(2) state
CISCO-ENTITY- entSensorValu 1.3.6.1.4.1.9.9.91. Provides a table at least once every hour

SENSOR-MIB eTable 1.1.1 of temperature
and power
sensors within the
system.
Best practice
would be to
gather trend data
here to ensure
that
facilities/cooling/
power are being
maintained.

Nexus 5000
CISCO-FLASH- ciscoFlashParti 1.3.6.1.4.1.9.9.10. Provides at least once every hour

MIB tionSize 1.1.4.1.1.13 information on
flash filesystems
ciscoFlashParti 1.3.6.1.4.1.9.9.10. within the
tionFreeSpace 1.1.4.1.1.14 system.
Best practice is to
ensure
ciscoFlashPartitio
nFreeSpace does
not drop below
10% of
ciscoFlashPartitio
nSize
IF-MIB ifDescr 1.3.6.1.2.1.2.2.1.2 A textual string Poll IF-MIB values every 5
containing the minutes for historical data.
interface
IF-MIB ifAlias 1.3.6.1.2.1.31.1.1. This object is an
1.18 'alias' name for Suggest making use of RMON
the interface as probes for specific
specified by a interfaces/counters to trigger
network alerts based on error
manager, and counters incrementing
provides a non- beyond a certain threshold
volatile 'handle' (determine via baseline), as
for the interface. well as interfaces with
IF-MIB ifSpeed 1.3.6.1.2.1.2.2.1.5 sustained performance
IF-MIB ifAdminStatus 1.3.6.1.2.1.2.2.1.7 up(1) The desired state above a high water mark.
of the interface.
down(2)
testing(3)
IF-MIB ifOperStatus 1.3.6.1.2.1.2.2.1.8 up(1) The current

operational state
down(2) of the interface
testing(3)
unknown(
4)
dormant(
5)
notPresen
t(6)
lowerLaye
rDown(7)
IF-MIB ifLastChange 1.3.6.1.2.1.2.2.1.9 The value of
sysUpTime at the

Nexus 5000
time the interface

entered its
current
operational state
IF-MIB ifHCInUcastPkt 1.3.6.1.2.1.31.1.1. The number of
s 1.7 packets, delivered
by this sub-layer
to a higher (sub-
)layer, which
were not
addressed to a
multicast or
broadcast address
at this sub-layer.
IF-MIB ifHCOutUcastP 1.3.6.1.2.1.31.1.1. The total number
kts 1.11 of packets that
higher-level
protocols
requested be
transmitted, and
which were not
addressed to a
multicast or
broadcast address
at this sub-layer,
including those
that were
discarded or not
sent.
IF-MIB ifHCInMulticas 1.3.6.1.2.1.31.1.1. The number of
tPkts 1.8 packets, delivered
by this sub-layer
to a higher (sub-
)layer, which
were addressed
to a multicast
address at this
sub-layer.
IF-MIB ifHCOutMultic 1.3.6.1.2.1.31.1.1. The total number
astPkts 1.12 of packets that
higher-level
protocols
requested be
transmitted, and
which were
addressed to a
multicast address
at this sub-layer,
including those
that were
discarded or not
sent.

Nexus 5000
IF-MIB ifHCInBroadca 1.3.6.1.2.1.31.1.1. The number of

stPkts 1.9 packets, delivered
by this sub-layer
to a higher (sub-
)layer, which
were addressed
to a broadcast
address at this
sub-layer.
IF-MIB ifHCOutBroadc 1.3.6.1.2.1.31.1.1. The total number
astPkts 1.13 of packets that
higher-level
protocols
requested be
transmitted, and
which were
addressed to a
broadcast address
at this sub-layer,
including those
that were
discarded or not
sent.
IF-MIB ifInDiscards 1.3.6.1.2.1.2.2.1.1 The number of
3 inbound packets
which were
chosen to be
discarded even
though no errors
had been
detected to
prevent their
being deliverable
to a higher-layer
protocol. One
possible reason
for discarding
such a packet
could be to free
up buffer space
IF-MIB ifInErrors 1.3.6.1.2.1.2.2.1.1 The number of
4 inbound packets
that contained
errors preventing
them from being
deliverable to a
higher-layer
protocol..

Nexus 5000
IF-MIB ifInUnknownPr 1.3.6.1.2.1.2.2.1.1 The number of

otos 5 packets received
via the interfaces
which were
discarded
because of an
unknown or
unsupported
protocol.
IF-MIB ifOutDiscards 1.3.6.1.2.1.2.2.1.1 The number of
9 outbound packets
which were
chosen to be
discarded even
though no errors
had been
detected to
prevent their
being
transmitted. One
possible reason
for discarding
such a packet
could be to free
up buffer space.
IF-MIB ifOutError 1.3.6.1.2.1.2.2.1.2 The number of
0 outbound packets
that could not be
transmitted
because of errors.
IF-MIB ifConnectorPr 1.3.6.1.2.1.31.1.1. This object has
esent 1.17 the value 'true(1)'
if the interface
sublayer has a
physical
connector and
the value 'false(2)'
otherwise.
SNMPv2-MIB coldStart 1.3.6.1.6.3.1.1.5.1 A coldStart trap
signifies that the
SNMP entity,
supporting a
notification
originator
application, is
reinitializing itself
and that its
configuration may
have been
altered.

Nexus 5000
SNMPv2-MIB warmStart 1.3.6.1.6.3.1.1.5.2 A warmStart trap

signifies that the
SNMP entity,
supporting a
notification
originator
application, is
reinitializing itself
such that its
configuration is
unaltered
IF-MIB linkDown 1.3.6.1.6.3.1.1.5.3 A linkDown trap
signifies that the
SNMP entity,
acting in an agent
role, has detected
that the
ifOperStatus
object for one of
its
communication
links is about to
enter the down
state from some
other state (but
not from the
notPresent
state). This other
state is indicated
by the included
value of
ifOperStatus
IF-MIB linkup 1.3.6.1.6.3.1.1.5.4 A linkUp trap
signifies that the
SNMP entity,
acting in an agent
role, has detected
that the
ifOperStatus
object for one of
its
communication
links left the
down state and
transitioned into
some other state
(but not into the
notPresent
state). This other
state is indicated
by the included
value of

Nexus 5000
ifOperStatus

6. Nexus 2000
The Cisco Nexus 2000 Series uses the Cisco Fabric Extender architecture to provide a
highly scalable unified server-access platform across a range of 100 Megabit Ethernet,
Gigabit Ethernet, 10 Gigabit Ethernet, unified fabric, copper and fiber connectivity, rack,
and blade server environments.
The Cisco Nexus 2000 Series Fabric Extenders behave as remote line cards for a parent
Cisco Nexus switch. The fabric extenders are essentially extensions of the parent Cisco
Nexus switch fabric, with the fabric extenders and the parent Cisco Nexus switch
together forming a distributed modular system.
The Cisco Nexus 2000 Series is managed through the parent Cisco Nexus Series Switch
using standard SNMP, XML interfaces, and command-line interface (CLI).
The following MIBs are supported on Nexus 2000
• ENTITY-MIB
• IF-MIB
• FABRIC-EXTENDER MIB
• CISCO-ENTITY-EXT-MIB
• CISCO-ENTITY-FRU-CONTROL-MIB
• CISCO-ENTITY-SENSOR-MIB
• CISCO-ETHERNET-FABRIC-EXTENDER-MIB
6.1 CLI commands

The following sections provide some key CLI commands to display the status of the
Fabric extenders
To display a detailed status of the Fabric extender use the show fex detail command
switch# show fex 100 detail

FEX: 100 Description: FEX0100 state: Online
FEX version: 4.0(1a)N2(0.101) [Switch version: 4.0(1a)N2(0.101)]
Extender Model: N5K-C5110T-BF-1GE, Extender Serial: JAF1241BLFN
Part No: 73-12009-02
Card Id: 70, Mac Addr: 00:0d:ec:b1:28:42, Num Macs: 64
Module Sw Gen: 17 [Switch Sw Gen: 17]
pinning-mode: static Max-links: 1
Fabric port for control traffic: Eth1/40
Fabric interface state:
Eth1/40 - Interface Up. State: Active
Fex Port State Fabric Port Primary Fabric

Nexus 5000
Eth100/1/1 Up Eth1/40 Eth1/40

Eth100/1/2 Down Eth1/40 Eth1/40
Eth100/1/15 Up Eth1/40 Eth1/40
Eth100/1/31 Up Eth1/40 Eth1/40
Eth100/1/47 Up Eth1/40 Eth1/40
Logs:
[12/10/2008 00:11:40.698999] Module timed out
[12/10/2008 00:11:44.501221] Module register received
[12/10/2008 00:11:44.502240] Registration response sent
[12/10/2008 00:11:44.587581] Module Online Sequence
[12/10/2008 00:11:49.140170] Module Online
[12/10/2008 00:18:46.662135] Module disconnected
[12/10/2008 00:18:46.663222] Offlining Module
[12/10/2008 00:18:46.663856] Module Offline Sequence
[12/10/2008 00:18:49.317584] Module Offline

Nexus 5000

[12/10/2008 00:19:32.62827] Module Online
[12/10/2008 00:20:37.196648] Module disconnected
[12/10/2008 00:20:37.197782] Offlining Module
[12/10/2008 00:20:37.199299] Module Offline Sequence
[12/10/2008 00:20:39.859971] Module Offline
[12/10/2008 00:21:15.737667] Module Online
To view the module information about all connected Fabric extender units use the show
module fex command
switch# show module fex
FEX Mod Ports Card
Type Model Status.
--- --- ----- ---------------------------------- ------------------ -
----------
100 1 48 Fabric Extender 48x1GE Module N2K-C2148T-
1GE ok
FEX Mod Sw Hw World-Wide-Name(s) (WWN)

--- --- -------------- ------ -------------------------------------
----------
100 1 4.0(1a)N2(1) 0.2 --
FEX Mod MAC-Address(es) Serial-Num

--- --- -------------------------------------- ----------
100 1 000d.ecb1.3f00 to 000d.ecb1.3f2fff JAF1244ATER
To view the environment status for a specific Fabric Extender unit use the show
environment fex command
switch# show environment fex 100
Temperature Fex 100:

-----------------------------------------------------------------
Module Sensor MajorThresh MinorThres CurTemp Status
(Celsius) (Celsius) (Celsius)
-----------------------------------------------------------------
1 Outlet-1 60 50 29 ok
1 Inlet-1 50 40 21 ok
Fan Fex: 100:

------------------------------------------------------
Fan Model Hw Status
------------------------------------------------------
Chassis N2K-C2148-FAN -- ok
PS-1 N5K-PAC-200W -- failure

Nexus 5000
PS-2 N5K-PAC-200W -- ok
Power Supply Fex 100:

---------------------------------------------------------------------
------
Voltage: 12 Volts
-----------------------------------------------------
PS Model Power Power Status
(Watts) (Amp)
-----------------------------------------------------
1 -- -- -- fail/shutdown
2 N5K-PAC-200W 200.04 16.67 ok
Mod
Model Power Power Power Power Stat
us
Requested Requested Allocated Allocated
(Watts) (Amp) (Watts) (Amp)
--- ------------------- ------- ---------- --------- ----------
----------
1 N5K-C5110T-BF-
1GE 96.00 8.00 96.00 8.00 powered-up
Power Usage Summary:

--------------------
Power Supply redundancy mode: redundant
Total Power Capacity 200.04 W
Power reserved for Supervisor(s) 96.00 W

Power currently used by Modules 0.00 W
-------------
Total Power Available 104.04 W
-------------
To view diagnostic test results for a specific Fabric Extender unit use the show
diagnostic result fex command
switch# show diagnostic result fex 100

FEX-100: 48x1GE/Supervisor SerialNo : JAF1241BLFN
Overall Diagnostic Result for FEX-100 : OK
Test results: (. = Pass, F = Fail, U = Untested)

TestPlatform:
0) SPROM: ---------------> .
1) MV88E6095: ---------------> .
2) Fan: ---------------> .
3) PowerSupply: ---------------> F
4) TempSensor: ---------------> .
TestForwardingPorts:

Nexus 5000
Eth 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
22 23 24
Port ----------------------------------------------------------------
--------
. . . . . . . . . . . . . . . . . . . . .
. . .
Eth 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
46 47 48
Port ----------------------------------------------------------------
--------
. . . . . . . . . . . . . . . . . . . . .
. . .
TestFabricPorts:
Fabric 1 2 3 4
Port ------------
. . . .

7. Summary
In the previous sections, we have identified the critical parameters that should be
monitored and managed for ensuring optimal Network performance.
The next steps must be to integrate these in the overall monitoring processes and
define thresholds for alerts or actions.The recommended plan is to
 Integrate metrics into an SNMP tool

 Capture data for a few days, representative of the production
environment
 Analyze data to validate them and identify patterns
 Establish baseline values and acceptable limits
 Configure alert levels.
In order to be able to measure the long-term evolution, it is necessary to configure
the SNMP tool to store and archive data for an extended time-window. The easiest
way is probably to represent them through graphical display. When an alert level is
reached, two situations can occur:
 If the alert is the consequence of an unusual event, appropriate action

must be taken to mitigate the impact.
 If the alert is the result of a normal long-term evolution, the alert level
must probably be revisited, along with an evaluation of the long-term
performance of the overall equipment.
Consider CPU utilization, a possible short-term alert level at 80% would indicate
heavy activity on the device. However for long-term planning, you could also
consider that a device performing always at 60-80% CPU utilization would deserve
attention so that it is replaced on time (i.e. before operations are impacted) by a
more powerful one. There will be new SNMP MIBs introduced for future NX-OS
releases. The link below will provide the details for respective MIB’s. Please refer
this to see what new MIB’s have been added and can be used for future NX-OS
releases.
SNMP MIBs for respective Platform can be found at

http://www.cisco.com/public/sw-center/netmgmt/cmtk/mibs.shtml
Details for a specific OID or object can be found using the SNMP Object Navigator
http://tools.cisco.com/Support/SNMP/do/BrowseOID.do?local=en
If you have any questions or comments regarding this report, please contact your
Cisco Advanced Services Network Consulting Engineer.
Document Acceptance
Name Name
Title Title
Company Company
Signature Signature
Date Date
Name Name
Title Title
Company Company
Signature Signature
Date Date
Name Name
Title Title
Company Company
Signature Signature
Date Date

Nexus Network Management Best Practices Summary

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Nexus Network Management Best Practices Summary

Uploaded by

Copyright:

Available Formats

TM

Nexus Network Management Best Practices

Nexus Network Management Best Practices 1

Nexus Network Management Best Practices 2

4.12 References ............................................................................................................................................... 41

Nexus Network Management Best Practices 3

Table 1 SNMP Objects for CPU Monitoring ........................................................................................... 11

Nexus Network Management Best Practices 4

Author : Vinay Suvarna and Azeem Suleman

Version No. Issue Date Status Reason for Change

Reviewers Details Version No. Date

Nexus Network Management Best Practices 5

1.1 Document Purpose

1.2 Intended Audience

 Nexus 7000 with Software release 5.2(3)a

Nexus Network Management Best Practices 6

2.1 Network Management

Network Management is an integral part of today’s networks to improve the

The goal of Performance Management is to monitor and measure various aspects of

2.2 Detecting required Actions

On short-term, it is important to be able to detect unusual situations, events or

Nexus Network Management Best Practices 7

Nexus Network Management Best Practices 8

It is important to note that, although recommendations can be made regarding the

3.2 System Resource Monitoring

When considering the memory, several items deserve a careful monitoring.

Nexus Network Management Best Practices 9

 Constantly increasing device utilization (this would be observed on the long

3.2.3 Network Interfaces

As for other network equipment, it is recommended to monitor the various statistics

3.2.4 Statistics and Connections details

For automated connection and traffic details analysis, it is recommended to use

Nexus Network Management Best Practices 10

4.1 Target Equipment

The following device is targeted in this section:

 Hardware: Nexus 7000 series switches

4.2 CPU Utilization

Possible Return Values:

Table 1 SNMP Objects for CPU Monitoring

MIB Object Name OID Description

Nexus Network Management Best Practices 11

CISCO-PROCESS-MIB cpmCPUTotal1minRev 1.3.6.1.4.1.9.9.109.1.1.1.1. The overall CPU busy

SNMP Traps for CPU Monitoring

SYSLOG Messages for CPU Monitoring

Nexus Network Management Best Practices 12

n7000# show system resources

4.3 Memory Availability

Possible Return Values:

‘cseSysMemoryUtilization’ : The average utilization of memory on the active

Possible Return Values:

‘ceExtNVRAMUsed’: Number of bytes of NVRAM in use. This object is irrelevant if

Nexus Network Management Best Practices 13

SNMP Traps for Memory Monitoring

SYSLOG Messages for Memory Monitoring

Syslog Message %PLATFORM-2-MEMORY_ALERT: Memory Status Alert: MINOR. Usage

The show system resources command displays platform memory statistics

N7K# show system resources

N7K# show processes memory

Nexus Network Management Best Practices 14

N7K# show system internal kernel meminfo

In the output above, the most important fields are as follows:

Nexus Network Management Best Practices 15